Linux-RDMA Archive on lore.kernel.org
 help / color / Atom feed
From: Bart Van Assche <bvanassche@acm.org>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Leon Romanovsky <leonro@mellanox.com>,
	Doug Ledford <dledford@redhat.com>,
	linux-rdma@vger.kernel.org, Bart Van Assche <bvanassche@acm.org>,
	Or Gerlitz <gerlitz.or@gmail.com>,
	Steve Wise <larrystevenwise@gmail.com>,
	Sagi Grimberg <sagi@grimberg.me>,
	Bernard Metzler <BMT@zurich.ibm.com>,
	Krishnamraju Eraparaju <krishna2@chelsio.com>,
	stable@vger.kernel.org
Subject: [PATCH 02/15] RDMA/iwcm: Fix a lock inversion issue
Date: Mon, 30 Sep 2019 16:16:54 -0700
Message-ID: <20190930231707.48259-3-bvanassche@acm.org> (raw)
In-Reply-To: <20190930231707.48259-1-bvanassche@acm.org>

This patch fixes the lock inversion complaint:

============================================
WARNING: possible recursive locking detected
5.3.0-rc7-dbg+ #1 Not tainted
--------------------------------------------
kworker/u16:6/171 is trying to acquire lock:
00000000035c6e6c (&id_priv->handler_mutex){+.+.}, at: rdma_destroy_id+0x78/0x4a0 [rdma_cm]

but task is already holding lock:
00000000bc7c307d (&id_priv->handler_mutex){+.+.}, at: iw_conn_req_handler+0x151/0x680 [rdma_cm]

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock(&id_priv->handler_mutex);
  lock(&id_priv->handler_mutex);

 *** DEADLOCK ***

 May be due to missing lock nesting notation

3 locks held by kworker/u16:6/171:
 #0: 00000000e2eaa773 ((wq_completion)iw_cm_wq){+.+.}, at: process_one_work+0x472/0xac0
 #1: 000000001efd357b ((work_completion)(&work->work)#3){+.+.}, at: process_one_work+0x476/0xac0
 #2: 00000000bc7c307d (&id_priv->handler_mutex){+.+.}, at: iw_conn_req_handler+0x151/0x680 [rdma_cm]

stack backtrace:
CPU: 3 PID: 171 Comm: kworker/u16:6 Not tainted 5.3.0-rc7-dbg+ #1
Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
Workqueue: iw_cm_wq cm_work_handler [iw_cm]
Call Trace:
 dump_stack+0x8a/0xd6
 __lock_acquire.cold+0xe1/0x24d
 lock_acquire+0x106/0x240
 __mutex_lock+0x12e/0xcb0
 mutex_lock_nested+0x1f/0x30
 rdma_destroy_id+0x78/0x4a0 [rdma_cm]
 iw_conn_req_handler+0x5c9/0x680 [rdma_cm]
 cm_work_handler+0xe62/0x1100 [iw_cm]
 process_one_work+0x56d/0xac0
 worker_thread+0x7a/0x5d0
 kthread+0x1bc/0x210
 ret_from_fork+0x24/0x30

Cc: Or Gerlitz <gerlitz.or@gmail.com>
Cc: Steve Wise <larrystevenwise@gmail.com>
Cc: Sagi Grimberg <sagi@grimberg.me>
Cc: Bernard Metzler <BMT@zurich.ibm.com>
Cc: Krishnamraju Eraparaju <krishna2@chelsio.com>
Cc: <stable@vger.kernel.org>
Fixes: de910bd92137 ("RDMA/cma: Simplify locking needed for serialization of callbacks"; v2.6.27).
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
---
 drivers/infiniband/core/cma.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 0e3cf3461999..d78f67623f24 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -2396,9 +2396,10 @@ static int iw_conn_req_handler(struct iw_cm_id *cm_id,
 		conn_id->cm_id.iw = NULL;
 		cma_exch(conn_id, RDMA_CM_DESTROYING);
 		mutex_unlock(&conn_id->handler_mutex);
+		mutex_unlock(&listen_id->handler_mutex);
 		cma_deref_id(conn_id);
 		rdma_destroy_id(&conn_id->id);
-		goto out;
+		return ret;
 	}
 
 	mutex_unlock(&conn_id->handler_mutex);
-- 
2.23.0.444.g18eeb5a265-goog


  parent reply index

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-09-30 23:16 [PATCH 00/15] RDMA patches for kernel v5.5 Bart Van Assche
2019-09-30 23:16 ` [PATCH 01/15] RDMA/ucma: Reduce the number of rdma_destroy_id() calls Bart Van Assche
2019-10-01 15:07   ` Jason Gunthorpe
2019-10-01 17:13     ` Bart Van Assche
2019-09-30 23:16 ` Bart Van Assche [this message]
2019-10-01 15:17   ` [PATCH 02/15] RDMA/iwcm: Fix a lock inversion issue Jason Gunthorpe
2019-09-30 23:16 ` [PATCH 03/15] RDMA/siw: Simplify several debug messages Bart Van Assche
2019-10-01 15:20   ` Jason Gunthorpe
2019-09-30 23:16 ` [PATCH 04/15] RDMA/siw: Fix port number endianness in a debug message Bart Van Assche
2019-10-01 15:20   ` Jason Gunthorpe
2019-09-30 23:16 ` [PATCH 05/15] RDMA/siw: Make node GUIDs valid EUI-64 identifiers Bart Van Assche
2019-09-30 23:16 ` [PATCH 06/15] RDMA/srp: Remove two casts Bart Van Assche
2019-09-30 23:16 ` [PATCH 07/15] RDMA/srp: Honor the max_send_sge device attribute Bart Van Assche
2019-09-30 23:17 ` [PATCH 08/15] RDMA/srp: Make route resolving error messages more informative Bart Van Assche
2019-09-30 23:17 ` [PATCH 09/15] RDMA/srpt: Fix handling of SR-IOV and iWARP ports Bart Van Assche
2019-10-02 14:14   ` Jason Gunthorpe
2019-10-02 15:21     ` Bart Van Assche
2019-10-02 16:51       ` Jason Gunthorpe
2019-10-02 17:24         ` Leon Romanovsky
2019-10-02 17:43           ` Bart Van Assche
2019-10-03  8:33             ` Leon Romanovsky
2019-09-30 23:17 ` [PATCH 10/15] RDMA/srpt: Fix handling of iWARP logins Bart Van Assche
2019-10-02 14:16   ` Jason Gunthorpe
2019-10-02 15:23     ` Bart Van Assche
2019-09-30 23:17 ` [PATCH 11/15] RDMA/srpt: Improve a debug message Bart Van Assche
2019-09-30 23:17 ` [PATCH 12/15] RDMA/srpt: Rework the approach for closing an RDMA channel Bart Van Assche
2019-09-30 23:17 ` [PATCH 13/15] RDMA/srpt: Rework the code that waits until an RDMA port is no longer in use Bart Van Assche
2019-09-30 23:17 ` [PATCH 14/15] RDMA/srpt: Make the code for handling port identities more systematic Bart Van Assche
2019-09-30 23:17 ` [PATCH 15/15] RDMA/srpt: Postpone HCA removal until after configfs directory removal Bart Van Assche
2019-10-01 11:39 ` [PATCH 03/15] RDMA/siw: Simplify several debug messages Bernard Metzler
2019-10-01 11:45 ` [PATCH 04/15] RDMA/siw: Fix port number endianness in a debug message Bernard Metzler
2019-10-04 18:36 ` [PATCH 00/15] RDMA patches for kernel v5.5 Jason Gunthorpe

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190930231707.48259-3-bvanassche@acm.org \
    --to=bvanassche@acm.org \
    --cc=BMT@zurich.ibm.com \
    --cc=dledford@redhat.com \
    --cc=gerlitz.or@gmail.com \
    --cc=jgg@ziepe.ca \
    --cc=krishna2@chelsio.com \
    --cc=larrystevenwise@gmail.com \
    --cc=leonro@mellanox.com \
    --cc=linux-rdma@vger.kernel.org \
    --cc=sagi@grimberg.me \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-RDMA Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-rdma/0 linux-rdma/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-rdma linux-rdma/ https://lore.kernel.org/linux-rdma \
		linux-rdma@vger.kernel.org
	public-inbox-index linux-rdma

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-rdma


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git