From: Daniel Jordan <daniel.m.jordan@oracle.com>
To: Steffen Klassert <steffen.klassert@secunet.com>
Cc: Daniel Jordan <daniel.m.jordan@oracle.com>,
Andrea Parri <andrea.parri@amarulasolutions.com>,
Boqun Feng <boqun.feng@gmail.com>,
Herbert Xu <herbert@gondor.apana.org.au>,
"Paul E . McKenney" <paulmck@linux.ibm.com>,
Peter Zijlstra <peterz@infradead.org>,
linux-arch@vger.kernel.org, linux-crypto@vger.kernel.org,
linux-kernel@vger.kernel.org
Subject: [PATCH v2] padata: use smp_mb in padata_reorder to avoid orphaned padata jobs
Date: Tue, 16 Jul 2019 12:32:53 -0400 [thread overview]
Message-ID: <20190716163253.24377-1-daniel.m.jordan@oracle.com> (raw)
In-Reply-To: <c1bbbe94-dbdc-da14-e0c3-850c965d8b5d@oracle.com>
Testing padata with the tcrypt module on a 5.2 kernel...
# modprobe tcrypt alg="pcrypt(rfc4106(gcm(aes)))" type=3
# modprobe tcrypt mode=211 sec=1
...produces this splat:
INFO: task modprobe:10075 blocked for more than 120 seconds.
Not tainted 5.2.0-base+ #16
modprobe D 0 10075 10064 0x80004080
Call Trace:
? __schedule+0x4dd/0x610
? ring_buffer_unlock_commit+0x23/0x100
schedule+0x6c/0x90
schedule_timeout+0x3b/0x320
? trace_buffer_unlock_commit_regs+0x4f/0x1f0
wait_for_common+0x160/0x1a0
? wake_up_q+0x80/0x80
{ crypto_wait_req } # entries in braces added by hand
{ do_one_aead_op }
{ test_aead_jiffies }
test_aead_speed.constprop.17+0x681/0xf30 [tcrypt]
do_test+0x4053/0x6a2b [tcrypt]
? 0xffffffffa00f4000
tcrypt_mod_init+0x50/0x1000 [tcrypt]
...
The second modprobe command never finishes because in padata_reorder,
CPU0's load of reorder_objects is executed before the unlocking store in
spin_unlock_bh(pd->lock), causing CPU0 to miss CPU1's increment:
CPU0 CPU1
padata_reorder padata_do_serial
LOAD reorder_objects // 0
INC reorder_objects // 1
padata_reorder
TRYLOCK pd->lock // failed
UNLOCK pd->lock
CPU0 deletes the timer before returning from padata_reorder and since no
other job is submitted to padata, modprobe waits indefinitely.
Add a pair of full barriers to guarantee proper ordering:
CPU0 CPU1
padata_reorder padata_do_serial
UNLOCK pd->lock
smp_mb()
LOAD reorder_objects
INC reorder_objects
smp_mb__after_atomic()
padata_reorder
TRYLOCK pd->lock
smp_mb__after_atomic is needed so the read part of the trylock operation
comes after the INC, as Andrea points out. Thanks also to Andrea for
help with writing a litmus test.
Fixes: 16295bec6398 ("padata: Generic parallelization/serialization interface")
Signed-off-by: Daniel Jordan <daniel.m.jordan@oracle.com>
Cc: Andrea Parri <andrea.parri@amarulasolutions.com>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Paul E. McKenney <paulmck@linux.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steffen Klassert <steffen.klassert@secunet.com>
Cc: linux-arch@vger.kernel.org
Cc: linux-crypto@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
---
kernel/padata.c | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/kernel/padata.c b/kernel/padata.c
index 2d2fddbb7a4c..15a8ad63f4ff 100644
--- a/kernel/padata.c
+++ b/kernel/padata.c
@@ -267,7 +267,12 @@ static void padata_reorder(struct parallel_data *pd)
* The next object that needs serialization might have arrived to
* the reorder queues in the meantime, we will be called again
* from the timer function if no one else cares for it.
+ *
+ * Ensure reorder_objects is read after pd->lock is dropped so we see
+ * an increment from another task in padata_do_serial. Pairs with
+ * smp_mb__after_atomic in padata_do_serial.
*/
+ smp_mb();
if (atomic_read(&pd->reorder_objects)
&& !(pinst->flags & PADATA_RESET))
mod_timer(&pd->timer, jiffies + HZ);
@@ -387,6 +392,13 @@ void padata_do_serial(struct padata_priv *padata)
list_add_tail(&padata->list, &pqueue->reorder.list);
spin_unlock(&pqueue->reorder.lock);
+ /*
+ * Ensure the atomic_inc of reorder_objects above is ordered correctly
+ * with the trylock of pd->lock in padata_reorder. Pairs with smp_mb
+ * in padata_reorder.
+ */
+ smp_mb__after_atomic();
+
put_cpu();
/* If we're running on the wrong CPU, call padata_reorder() via a
base-commit: 0ecfebd2b52404ae0c54a878c872bb93363ada36
--
2.22.0
next prev parent reply other threads:[~2019-07-16 16:33 UTC|newest]
Thread overview: 42+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-07-11 22:12 [PATCH] padata: use smp_mb in padata_reorder to avoid orphaned padata jobs Daniel Jordan
2019-07-12 10:06 ` Herbert Xu
2019-07-12 10:10 ` Steffen Klassert
2019-07-12 16:07 ` Daniel Jordan
2019-07-13 5:03 ` Herbert Xu
2019-07-15 16:10 ` Daniel Jordan
2019-07-16 10:04 ` Herbert Xu
2019-07-16 11:14 ` Steffen Klassert
2019-07-16 12:57 ` [PATCH] padata: Use RCU when fetching pd from do_serial Herbert Xu
2019-07-16 13:09 ` Herbert Xu
2019-07-16 13:23 ` [v2 PATCH] " Herbert Xu
2019-07-17 8:36 ` Steffen Klassert
2019-07-17 8:50 ` Herbert Xu
2019-07-17 8:28 ` [PATCH] " Steffen Klassert
2019-07-17 8:47 ` Herbert Xu
2019-07-17 8:53 ` Steffen Klassert
2019-07-16 13:15 ` Peter Zijlstra
2019-07-16 13:18 ` Herbert Xu
2019-07-16 13:50 ` Peter Zijlstra
2019-07-16 13:52 ` Herbert Xu
2019-07-16 12:53 ` [PATCH] padata: use smp_mb in padata_reorder to avoid orphaned padata jobs Andrea Parri
2019-07-16 13:13 ` Peter Zijlstra
2019-07-16 15:01 ` Herbert Xu
2019-07-16 15:44 ` Daniel Jordan
2019-07-16 16:32 ` Daniel Jordan [this message]
2019-07-17 11:11 ` [PATCH] padata: Replace delayed timer with immediate workqueue in padata_reorder Herbert Xu
2019-07-17 18:32 ` Daniel Jordan
2019-07-18 3:31 ` Herbert Xu
2019-07-18 14:27 ` Daniel Jordan
2019-07-18 14:56 ` Herbert Xu
2019-07-18 15:01 ` [v2 PATCH] " Herbert Xu
2019-07-19 14:37 ` Daniel Jordan
2019-07-19 14:55 ` Herbert Xu
2019-07-19 19:04 ` [PATCH] padata: purge get_cpu and reorder_via_wq from padata_do_serial Daniel Jordan
2019-07-26 12:36 ` Herbert Xu
2019-07-19 14:27 ` [PATCH] padata: Replace delayed timer with immediate workqueue in padata_reorder Daniel Jordan
2019-07-17 23:21 ` Daniel Jordan
2019-07-18 3:30 ` Herbert Xu
2019-07-18 14:25 ` Daniel Jordan
2019-07-18 14:49 ` Herbert Xu
2019-07-19 14:21 ` Daniel Jordan
2019-07-18 5:42 ` [PATCH v2] padata: use smp_mb in padata_reorder to avoid orphaned padata jobs Herbert Xu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190716163253.24377-1-daniel.m.jordan@oracle.com \
--to=daniel.m.jordan@oracle.com \
--cc=andrea.parri@amarulasolutions.com \
--cc=boqun.feng@gmail.com \
--cc=herbert@gondor.apana.org.au \
--cc=linux-arch@vger.kernel.org \
--cc=linux-crypto@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=paulmck@linux.ibm.com \
--cc=peterz@infradead.org \
--cc=steffen.klassert@secunet.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).