* [PATCH v3] nvmet-rdma: fix null dereference under heavy load
@ 2019-01-03 17:35 Raju Rangoju
2019-01-03 23:13 ` Sagi Grimberg
2019-01-19 13:34 ` Christoph Hellwig
0 siblings, 2 replies; 8+ messages in thread
From: Raju Rangoju @ 2019-01-03 17:35 UTC (permalink / raw)
To: sagi, linux-nvme; +Cc: maxg, swise, rajur, stable
Under heavy load if we don't have any pre-allocated rsps left, we
dynamically allocate a rsp, but we are not actually allocating memory
for nvme_completion (rsp->req.rsp). In such a case, accessing pointer
fields (req->rsp->status) in nvmet_req_init() will result in crash.
To fix this, allocate the memory for nvme_completion by calling
nvmet_rdma_alloc_rsp()
Fixes: 8407879c("nvmet-rdma:fix possible bogus dereference under heavy load")
Cc: <stable@vger.kernel.org>
Reviewed-by: Max Gurtovoy <maxg@mellanox.com>
Signed-off-by: Raju Rangoju <rajur@chelsio.com>
--
Changes from v2:
- Dropped the unneeded initialization of ret variable
Changes from v1:
- Moved integer to 'if' block
- Used 'unlikely' in datapath flow condition
---
drivers/nvme/target/rdma.c | 15 ++++++++++++++-
1 file changed, 14 insertions(+), 1 deletion(-)
diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c
index a8d23eb80192..a884e3a0e8af 100644
--- a/drivers/nvme/target/rdma.c
+++ b/drivers/nvme/target/rdma.c
@@ -139,6 +139,10 @@ static void nvmet_rdma_recv_done(struct ib_cq *cq, struct ib_wc *wc);
static void nvmet_rdma_read_data_done(struct ib_cq *cq, struct ib_wc *wc);
static void nvmet_rdma_qp_event(struct ib_event *event, void *priv);
static void nvmet_rdma_queue_disconnect(struct nvmet_rdma_queue *queue);
+static void nvmet_rdma_free_rsp(struct nvmet_rdma_device *ndev,
+ struct nvmet_rdma_rsp *r);
+static int nvmet_rdma_alloc_rsp(struct nvmet_rdma_device *ndev,
+ struct nvmet_rdma_rsp *r);
static const struct nvmet_fabrics_ops nvmet_rdma_ops;
@@ -182,9 +186,17 @@ nvmet_rdma_get_rsp(struct nvmet_rdma_queue *queue)
spin_unlock_irqrestore(&queue->rsps_lock, flags);
if (unlikely(!rsp)) {
- rsp = kmalloc(sizeof(*rsp), GFP_KERNEL);
+ int ret;
+
+ rsp = kzalloc(sizeof(*rsp), GFP_KERNEL);
if (unlikely(!rsp))
return NULL;
+ ret = nvmet_rdma_alloc_rsp(queue->dev, rsp);
+ if (unlikely(ret)) {
+ kfree(rsp);
+ return NULL;
+ }
+
rsp->allocated = true;
}
@@ -197,6 +209,7 @@ nvmet_rdma_put_rsp(struct nvmet_rdma_rsp *rsp)
unsigned long flags;
if (unlikely(rsp->allocated)) {
+ nvmet_rdma_free_rsp(rsp->queue->dev, rsp);
kfree(rsp);
return;
}
--
2.12.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH v3] nvmet-rdma: fix null dereference under heavy load
2019-01-03 17:35 [PATCH v3] nvmet-rdma: fix null dereference under heavy load Raju Rangoju
@ 2019-01-03 23:13 ` Sagi Grimberg
2019-01-07 15:59 ` Steve Wise
2019-01-19 13:34 ` Christoph Hellwig
1 sibling, 1 reply; 8+ messages in thread
From: Sagi Grimberg @ 2019-01-03 23:13 UTC (permalink / raw)
To: Raju Rangoju, linux-nvme; +Cc: maxg, swise, stable
Wonder how the original fix got a tested-by...
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v3] nvmet-rdma: fix null dereference under heavy load
2019-01-03 23:13 ` Sagi Grimberg
@ 2019-01-07 15:59 ` Steve Wise
2019-01-18 7:37 ` Raju Rangoju
0 siblings, 1 reply; 8+ messages in thread
From: Steve Wise @ 2019-01-07 15:59 UTC (permalink / raw)
To: Sagi Grimberg, Raju Rangoju, linux-nvme; +Cc: maxg, stable
On 1/3/2019 5:13 PM, Sagi Grimberg wrote:
> Wonder how the original fix got a tested-by...
>
> Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
That was my bad. But the testers said they couldn't reproduce the
issue, so I assumed that meant it was fixed. I guess they weren't
stressing it enough.
Steve.
^ permalink raw reply [flat|nested] 8+ messages in thread
* RE: [PATCH v3] nvmet-rdma: fix null dereference under heavy load
2019-01-07 15:59 ` Steve Wise
@ 2019-01-18 7:37 ` Raju Rangoju
0 siblings, 0 replies; 8+ messages in thread
From: Raju Rangoju @ 2019-01-18 7:37 UTC (permalink / raw)
To: Christoph Hellwig, SWise OGC, Sagi Grimberg, linux-nvme; +Cc: maxg, stable
Hi Christoph,
In case you missed, could you please pull this patch ?
-Raju
-----Original Message-----
From: Steve Wise <swise@opengridcomputing.com>
Sent: 07 January 2019 21:29
To: Sagi Grimberg <sagi@grimberg.me>; Raju Rangoju <rajur@chelsio.com>; linux-nvme@lists.infradead.org
Cc: maxg@mellanox.com; stable@vger.kernel.org
Subject: Re: [PATCH v3] nvmet-rdma: fix null dereference under heavy load
On 1/3/2019 5:13 PM, Sagi Grimberg wrote:
> Wonder how the original fix got a tested-by...
>
> Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
That was my bad. But the testers said they couldn't reproduce the issue, so I assumed that meant it was fixed. I guess they weren't stressing it enough.
Steve.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v3] nvmet-rdma: fix null dereference under heavy load
2019-01-03 17:35 [PATCH v3] nvmet-rdma: fix null dereference under heavy load Raju Rangoju
2019-01-03 23:13 ` Sagi Grimberg
@ 2019-01-19 13:34 ` Christoph Hellwig
2019-01-21 9:10 ` Sagi Grimberg
1 sibling, 1 reply; 8+ messages in thread
From: Christoph Hellwig @ 2019-01-19 13:34 UTC (permalink / raw)
To: Raju Rangoju; +Cc: sagi, linux-nvme, maxg, swise, stable
This looks ok to me:
Reviewed-by: Christoph Hellwig <hch@lst.de>
In the long run I wonder if we should get rid of the ѕtatic
preallocation entirely and just use a free pool to avoid constant
reallocations..
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v3] nvmet-rdma: fix null dereference under heavy load
2019-01-19 13:34 ` Christoph Hellwig
@ 2019-01-21 9:10 ` Sagi Grimberg
2019-01-21 13:07 ` Max Gurtovoy
2019-01-21 15:46 ` Steve Wise
0 siblings, 2 replies; 8+ messages in thread
From: Sagi Grimberg @ 2019-01-21 9:10 UTC (permalink / raw)
To: Christoph Hellwig, Raju Rangoju; +Cc: maxg, stable, swise, linux-nvme
> This looks ok to me:
>
> Reviewed-by: Christoph Hellwig <hch@lst.de>
Applied to nvme-5.0
> In the long run I wonder if we should get rid of the ѕtatic
> preallocation entirely and just use a free pool to avoid constant
> reallocations..
you mean a kmem_cache? very possible, we can also make the command
struct to contain the rsp behind it to avoid the extra allocation...
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v3] nvmet-rdma: fix null dereference under heavy load
2019-01-21 9:10 ` Sagi Grimberg
@ 2019-01-21 13:07 ` Max Gurtovoy
2019-01-21 15:46 ` Steve Wise
1 sibling, 0 replies; 8+ messages in thread
From: Max Gurtovoy @ 2019-01-21 13:07 UTC (permalink / raw)
To: Sagi Grimberg, Christoph Hellwig, Raju Rangoju; +Cc: stable, swise, linux-nvme
On 1/21/2019 11:10 AM, Sagi Grimberg wrote:
>> This looks ok to me:
>>
>> Reviewed-by: Christoph Hellwig <hch@lst.de>
>
> Applied to nvme-5.0
>
>> In the long run I wonder if we should get rid of the ѕtatic
>> preallocation entirely and just use a free pool to avoid constant
>> reallocations..
>
> you mean a kmem_cache? very possible, we can also make the command
> struct to contain the rsp behind it to avoid the extra allocation...
yes, the container idea sounds good..
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v3] nvmet-rdma: fix null dereference under heavy load
2019-01-21 9:10 ` Sagi Grimberg
2019-01-21 13:07 ` Max Gurtovoy
@ 2019-01-21 15:46 ` Steve Wise
1 sibling, 0 replies; 8+ messages in thread
From: Steve Wise @ 2019-01-21 15:46 UTC (permalink / raw)
To: Sagi Grimberg, Christoph Hellwig, Raju Rangoju; +Cc: maxg, stable, linux-nvme
On 1/21/2019 3:10 AM, Sagi Grimberg wrote:
>> This looks ok to me:
>>
>> Reviewed-by: Christoph Hellwig <hch@lst.de>
>
> Applied to nvme-5.0
>
>> In the long run I wonder if we should get rid of the ѕtatic
>> preallocation entirely and just use a free pool to avoid constant
>> reallocations..
>
> you mean a kmem_cache? very possible, we can also make the command
> struct to contain the rsp behind it to avoid the extra allocation...
Hey guys,
If every command struct requires a rsp, then it seems simpler to contain
it in the command struct.
My 2 cents...
Steve.
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2019-01-21 15:46 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-01-03 17:35 [PATCH v3] nvmet-rdma: fix null dereference under heavy load Raju Rangoju
2019-01-03 23:13 ` Sagi Grimberg
2019-01-07 15:59 ` Steve Wise
2019-01-18 7:37 ` Raju Rangoju
2019-01-19 13:34 ` Christoph Hellwig
2019-01-21 9:10 ` Sagi Grimberg
2019-01-21 13:07 ` Max Gurtovoy
2019-01-21 15:46 ` Steve Wise
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).