All of lore.kernel.org
 help / color / mirror / Atom feed
From: Andrea Righi <righi.andrea@gmail.com>
To: Robert LeBlanc <robert@leblancnet.us>
Cc: Sean Jenkins <sean@betterservers.com>,
	Sagi Grimberg <sagi@grimberg.me>,
	Doug Ledford <dledford@redhat.com>,
	Sean Hefty <sean.hefty@intel.com>,
	Hal Rosenstock <hal.rosenstock@gmail.com>,
	linux-rdma <linux-rdma@vger.kernel.org>,
	target-devel <target-devel@vger.kernel.org>,
	lkml <linux-kernel@vger.kernel.org>
Subject: [PATCH] ib_isert: prevent NULL pointer dereference in isert_login_recv_done() (was: Re: NULL pointer dereference in isert_login_recv_done in 4.9.32)
Date: Fri, 23 Jun 2017 00:37:57 +0200	[thread overview]
Message-ID: <20170622223757.GC28955@Dell> (raw)
In-Reply-To: <CAANLjFoQcmfkjq9N1fehTC9Zi6aOC7ArOEPXz-he3EnOvWeEDw@mail.gmail.com>

On Wed, Jun 21, 2017 at 10:33:45AM -0600, Robert LeBlanc wrote:
> On Wed, Jun 21, 2017 at 9:17 AM, Robert LeBlanc <robert@leblancnet.us> wrote:
> > On Tue, Jun 20, 2017 at 12:54 PM, Robert LeBlanc <robert@leblancnet.us> wrote:
> >> We have hit this four times today. Any ideas?
> >>
> >> [  169.382113] BUG: unable to handle kernel NULL pointer dereference at           (null)
> >> [  169.382152] IP: [<ffffffffa051e968>] isert_login_recv_done+0x28/0x170 [ib_isert]

So, we spent more time to track down this bug.

It seems that at login time an error is happening, not sure exactly what
kind of error, but isert_connect_error() is called and isert_conn->cm_id
is set to NULL.

Later, isert_login_recv_done() is trying to access
isert_conn->cm_id->device and we get the NULL pointer dereference.

Following there's the patch that we have applied to track down this
problem.

And this is what we see in dmesg with this patch applied:

 [  658.633188] isert: isert_connect_error: conn ffff887f2209c000 error
 [  658.633226] isert: isert_login_recv_done: login with broken rdma_cm_id

As we can see isert_connect_error() is called before isert_login_recv_done
and at that point isert_conn->cm_id is NULL.

Obviously simply checking if the pointer is NULL, returning and ignoring
the error in isert_login_recv_done() is not the best fix ever and I'm
not sure if I'm breaking something else doing so (even if with this
patch the kernel doesn't crash and I've not seen any problem so far).

Maybe a better way is to tear down the whole connection when this
particular case is happening? Suggestions?

Thanks,
-Andrea

---
ib_isert: prevent NULL pointer dereference in isert_login_recv_done()

During a login if an error is happening isert_connect_error() is called
and isert_conn->cm_id is set to NULL.

Later, isert_login_recv_done() is executed, trying to access
isert_conn->cm_id->device, causing the following BUG:

 [  169.382113] BUG: unable to handle kernel NULL pointer dereference at (null)
 [  169.382152] IP: [<ffffffffa051e968>] isert_login_recv_done+0x28/0x170 [ib_isert]

Check if isert_con->cm_id is set to NULL in isert_login_recv_done() to
avoid this problem.

Signed-off-by: Andrea Righi <righi.andrea@gmail.com>
Signed-off-by: Robert LeBlanc <robert@leblancnet.us>
---
 drivers/infiniband/ulp/isert/ib_isert.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/ulp/isert/ib_isert.c b/drivers/infiniband/ulp/isert/ib_isert.c
index fcbed35..a8c1143 100644
--- a/drivers/infiniband/ulp/isert/ib_isert.c
+++ b/drivers/infiniband/ulp/isert/ib_isert.c
@@ -741,6 +741,7 @@ isert_connect_error(struct rdma_cm_id *cma_id)
 {
 	struct isert_conn *isert_conn = cma_id->qp->qp_context;
 
+	isert_warn("conn %p error\n", isert_conn);
 	list_del_init(&isert_conn->node);
 	isert_conn->cm_id = NULL;
 	isert_put_conn(isert_conn);
@@ -1452,7 +1453,13 @@ static void
 isert_login_recv_done(struct ib_cq *cq, struct ib_wc *wc)
 {
 	struct isert_conn *isert_conn = wc->qp->qp_context;
-	struct ib_device *ib_dev = isert_conn->cm_id->device;
+struct ib_device *ib_dev;
+
+	if (unlikely(isert_conn->cm_id == NULL)) {
+		isert_warn("login with broken rdma_cm_id");
+		return;
+	}
+	ib_dev = isert_conn->cm_id->device;
 
 	if (unlikely(wc->status != IB_WC_SUCCESS)) {
 		isert_print_wc(wc, "login recv");
-- 
2.7.4

  parent reply	other threads:[~2017-06-22 22:37 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-06-20 18:54 NULL pointer dereference in isert_login_recv_done in 4.9.32 Robert LeBlanc
2017-06-21 15:17 ` Robert LeBlanc
     [not found]   ` <CAANLjFp6Y8wvLvmWrk1ECjm-OoyEppxD1svNYKpg+HDEZevJig-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2017-06-21 16:33     ` Robert LeBlanc
     [not found]       ` <CAANLjFoQcmfkjq9N1fehTC9Zi6aOC7ArOEPXz-he3EnOvWeEDw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2017-06-21 22:48         ` Robert LeBlanc
2017-06-22 22:37       ` Andrea Righi [this message]
2017-06-25 23:58         ` [PATCH] ib_isert: prevent NULL pointer dereference in isert_login_recv_done() (was: Re: NULL pointer dereference in isert_login_recv_done in 4.9.32) Nicholas A. Bellinger
2017-06-27  7:03           ` [PATCH] ib_isert: prevent NULL pointer dereference in isert_login_recv_done() Sagi Grimberg
2017-06-28 17:53           ` [PATCH] ib_isert: prevent NULL pointer dereference in isert_login_recv_done() (was: Re: NULL pointer dereference in isert_login_recv_done in 4.9.32) Andrea Righi
2017-06-29  5:36             ` [PATCH] ib_isert: prevent NULL pointer dereference in isert_login_recv_done() Sagi Grimberg
2017-06-29  5:36               ` Sagi Grimberg
2017-06-29  8:20               ` Andrea Righi
2017-06-29  8:28                 ` Sagi Grimberg
2017-06-29  8:28                   ` Sagi Grimberg
     [not found]                   ` <7f1b1f07-0670-10ee-620d-0ab3b1a6c2bf-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-06-30  5:53                     ` Nicholas A. Bellinger
2017-06-30  5:53                       ` Nicholas A. Bellinger
2017-06-26  8:53         ` [PATCH] ib_isert: prevent NULL pointer dereference in isert_login_recv_done() (was: Re: NULL pointer dereference in isert_login_recv_done in 4.9.32) Leon Romanovsky
2017-06-26  8:53           ` Leon Romanovsky

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170622223757.GC28955@Dell \
    --to=righi.andrea@gmail.com \
    --cc=dledford@redhat.com \
    --cc=hal.rosenstock@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=robert@leblancnet.us \
    --cc=sagi@grimberg.me \
    --cc=sean.hefty@intel.com \
    --cc=sean@betterservers.com \
    --cc=target-devel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.