* [PATCH for-next] IB/hfi1: Fix abba locking issue with sc_disable()
@ 2021-10-13 14:18 Dennis Dalessandro
2021-10-13 16:33 ` Jason Gunthorpe
0 siblings, 1 reply; 2+ messages in thread
From: Dennis Dalessandro @ 2021-10-13 14:18 UTC (permalink / raw)
To: jgg, dledford; +Cc: linux-rdma, TOTE Robot, Mike Marciniszyn
From: Mike Marciniszyn <mike.marciniszyn@cornelisnetworks.com>
sc_disable() after having disabled the send context wakes up
any waiters by calling hfi1_qp_wakeup() while holding the
waitlock for the sc.
This is contrary to the model for all other calls to hfi1_qp_wakeup()
where the waitlock is dropped and a local is used to drive calls
to hfi1_qp_wakeup().
Fix by moving the sc->piowait into a local list and driving the wakeup
calls from the list.
Signed-off-by: Mike Marciniszyn <mike.marciniszyn@cornelisnetworks.com>
Reported-by: TOTE Robot <oslab@tsinghua.edu.cn>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>
---
drivers/infiniband/hw/hfi1/pio.c | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/drivers/infiniband/hw/hfi1/pio.c b/drivers/infiniband/hw/hfi1/pio.c
index 489b436..3d42bd2 100644
--- a/drivers/infiniband/hw/hfi1/pio.c
+++ b/drivers/infiniband/hw/hfi1/pio.c
@@ -878,6 +878,7 @@ void sc_disable(struct send_context *sc)
{
u64 reg;
struct pio_buf *pbuf;
+ LIST_HEAD(wake_list);
if (!sc)
return;
@@ -912,19 +913,21 @@ void sc_disable(struct send_context *sc)
spin_unlock(&sc->release_lock);
write_seqlock(&sc->waitlock);
- while (!list_empty(&sc->piowait)) {
+ if (!list_empty(&sc->piowait))
+ list_move(&sc->piowait, &wake_list);
+ write_sequnlock(&sc->waitlock);
+ while (!list_empty(&wake_list)) {
struct iowait *wait;
struct rvt_qp *qp;
struct hfi1_qp_priv *priv;
- wait = list_first_entry(&sc->piowait, struct iowait, list);
+ wait = list_first_entry(&wake_list, struct iowait, list);
qp = iowait_to_qp(wait);
priv = qp->priv;
list_del_init(&priv->s_iowait.list);
priv->s_iowait.lock = NULL;
hfi1_qp_wakeup(qp, RVT_S_WAIT_PIO | HFI1_S_WAIT_PIO_DRAIN);
}
- write_sequnlock(&sc->waitlock);
spin_unlock_irq(&sc->alloc_lock);
}
^ permalink raw reply related [flat|nested] 2+ messages in thread
* Re: [PATCH for-next] IB/hfi1: Fix abba locking issue with sc_disable()
2021-10-13 14:18 [PATCH for-next] IB/hfi1: Fix abba locking issue with sc_disable() Dennis Dalessandro
@ 2021-10-13 16:33 ` Jason Gunthorpe
0 siblings, 0 replies; 2+ messages in thread
From: Jason Gunthorpe @ 2021-10-13 16:33 UTC (permalink / raw)
To: Dennis Dalessandro; +Cc: dledford, linux-rdma, TOTE Robot, Mike Marciniszyn
On Wed, Oct 13, 2021 at 10:18:52AM -0400, Dennis Dalessandro wrote:
> From: Mike Marciniszyn <mike.marciniszyn@cornelisnetworks.com>
>
> sc_disable() after having disabled the send context wakes up
> any waiters by calling hfi1_qp_wakeup() while holding the
> waitlock for the sc.
>
> This is contrary to the model for all other calls to hfi1_qp_wakeup()
> where the waitlock is dropped and a local is used to drive calls
> to hfi1_qp_wakeup().
>
> Fix by moving the sc->piowait into a local list and driving the wakeup
> calls from the list.
>
> Signed-off-by: Mike Marciniszyn <mike.marciniszyn@cornelisnetworks.com>
> Reported-by: TOTE Robot <oslab@tsinghua.edu.cn>
> Signed-off-by: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>
> drivers/infiniband/hw/hfi1/pio.c | 9 ++++++---
> 1 file changed, 6 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/infiniband/hw/hfi1/pio.c b/drivers/infiniband/hw/hfi1/pio.c
> index 489b436..3d42bd2 100644
> +++ b/drivers/infiniband/hw/hfi1/pio.c
> @@ -878,6 +878,7 @@ void sc_disable(struct send_context *sc)
> {
> u64 reg;
> struct pio_buf *pbuf;
> + LIST_HEAD(wake_list);
>
> if (!sc)
> return;
> @@ -912,19 +913,21 @@ void sc_disable(struct send_context *sc)
> spin_unlock(&sc->release_lock);
>
> write_seqlock(&sc->waitlock);
> - while (!list_empty(&sc->piowait)) {
> + if (!list_empty(&sc->piowait))
> + list_move(&sc->piowait, &wake_list);
> + write_sequnlock(&sc->waitlock);
> + while (!list_empty(&wake_list)) {
> struct iowait *wait;
> struct rvt_qp *qp;
> struct hfi1_qp_priv *priv;
>
> - wait = list_first_entry(&sc->piowait, struct iowait, list);
> + wait = list_first_entry(&wake_list, struct iowait, list);
> qp = iowait_to_qp(wait);
> priv = qp->priv;
> list_del_init(&priv->s_iowait.list);
> priv->s_iowait.lock = NULL;
> hfi1_qp_wakeup(qp, RVT_S_WAIT_PIO | HFI1_S_WAIT_PIO_DRAIN);
> }
> - write_sequnlock(&sc->waitlock);
clangd tells me there is no read side to this seqlock? Why is it a
seqlock if everything is a write side? Indeed I was wondering because
I don't know of any seq lock safe algorithm for list manipulation -
but it turns out this is just an obfuscated spinlock. Please send a
patch to fix it.
Applied to for-rc
Thanks,
Jason
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2021-10-13 16:33 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-13 14:18 [PATCH for-next] IB/hfi1: Fix abba locking issue with sc_disable() Dennis Dalessandro
2021-10-13 16:33 ` Jason Gunthorpe
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).