linux-nvme.lists.infradead.org archive mirror
 help / color / mirror / Atom feed
From: Jason Gunthorpe <jgg@mellanox.com>
To: Max Gurtovoy <maxg@mellanox.com>
Cc: loberman@redhat.com, Bart Van Assche <bvanassche@acm.org>,
	vladimirk@mellanox.com, idanb@mellanox.com,
	linux-rdma@vger.kernel.org, shlomin@mellanox.com,
	linux-nvme@lists.infradead.org, leonro@mellanox.com,
	dledford@redhat.com, oren@mellanox.com, kbusch@kernel.org,
	rgirase@redhat.com, hch@lst.de, sagi@grimberg.me
Subject: Re: [PATCH v2 3/5] nvmet-rdma: use SRQ per completion vector
Date: Thu, 19 Mar 2020 08:56:54 -0300	[thread overview]
Message-ID: <20200319115654.GV13183@mellanox.com> (raw)
In-Reply-To: <5623419a-39e6-6090-4ae2-d4725a8b9740@mellanox.com>

On Thu, Mar 19, 2020 at 11:15:50AM +0200, Max Gurtovoy wrote:
> 
> On 3/19/2020 6:09 AM, Bart Van Assche wrote:
> > On 2020-03-18 08:02, Max Gurtovoy wrote:
> > > In order to save resource allocation and utilize the completion
> >                     ^^^^^^^^^^^^^^^^^^^
> >                     resources?
> 
> thanks.
> 
> 
> > 
> > > +static int nvmet_rdma_srq_size = 1024;
> > > +module_param_cb(srq_size, &srq_size_ops, &nvmet_rdma_srq_size, 0644);
> > > +MODULE_PARM_DESC(srq_size, "set Shared Receive Queue (SRQ) size, should >= 256 (default: 1024)");
> > Is an SRQ overflow fatal? Isn't the SRQ size something that should be
> > computed by the nvmet_rdma driver such that SRQ overflows do not happen?
> 
> I've added the following code to make sure that the size is not greater than
> device capability:
> 
> +ndev->srq_size = min(ndev->device->attrs.max_srq_wr,
> +                            nvmet_rdma_srq_size);
> 
> In case the SRQ doesn't have enough credits it will send rnr to the
> initiator and the initiator will retry later on.

This is a pretty big change, in bad cases we could significantly
overflow the srq space available...

A big part of most verbs protocols to ensure that the RQ does not
overflow.

Are we sure it is OK? With all initiator/targets out there?

Jason

_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

  reply	other threads:[~2020-03-19 11:57 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-18 15:02 [PATCH v2 0/5] nvmet-rdma/srpt: SRQ per completion vector Max Gurtovoy
2020-03-18 15:02 ` [PATCH v2 1/5] IB/core: add a simple SRQ pool per PD Max Gurtovoy
2020-03-20  5:59   ` Sagi Grimberg
2020-03-20 13:21     ` Max Gurtovoy
2020-03-20 14:27     ` Leon Romanovsky
2020-03-18 15:02 ` [PATCH v2 2/5] nvmet-rdma: add srq pointer to rdma_cmd Max Gurtovoy
2020-03-18 23:32   ` Jason Gunthorpe
2020-03-19  8:48     ` Max Gurtovoy
2020-03-19  9:14       ` Leon Romanovsky
2020-03-19 10:55         ` Max Gurtovoy
2020-03-19 11:54       ` Jason Gunthorpe
2020-03-19 14:08         ` Konstantin Ryabitsev
2020-03-19 21:58         ` Konstantin Ryabitsev
2020-03-19  4:05   ` Bart Van Assche
2020-03-18 15:02 ` [PATCH v2 3/5] nvmet-rdma: use SRQ per completion vector Max Gurtovoy
2020-03-19  4:09   ` Bart Van Assche
2020-03-19  9:15     ` Max Gurtovoy
2020-03-19 11:56       ` Jason Gunthorpe [this message]
2020-03-19 12:48         ` Max Gurtovoy
2020-03-19 13:53           ` Jason Gunthorpe
2020-03-19 14:49             ` Bart Van Assche
     [not found]               ` <50dd8f5d-d092-54bc-236d-1e702fb95240@mellanox.com>
     [not found]                 ` <6e3cc1c4-b24e-f607-42b3-5b83dd8c312c@mellanox.com>
2020-03-19 16:27                   ` Max Gurtovoy
2020-03-20  5:47   ` Sagi Grimberg
2020-03-18 15:02 ` [PATCH v2 4/5] RDMA/srpt: use ib_alloc_cq instead of ib_alloc_cq_any Max Gurtovoy
2020-03-19  4:15   ` Bart Van Assche
2020-03-18 15:02 ` [PATCH v2 5/5] RDMA/srpt: use SRQ per completion vector Max Gurtovoy
2020-03-19  4:20   ` Bart Van Assche
2020-03-19  4:02 ` [PATCH v2 0/5] nvmet-rdma/srpt: " Bart Van Assche

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200319115654.GV13183@mellanox.com \
    --to=jgg@mellanox.com \
    --cc=bvanassche@acm.org \
    --cc=dledford@redhat.com \
    --cc=hch@lst.de \
    --cc=idanb@mellanox.com \
    --cc=kbusch@kernel.org \
    --cc=leonro@mellanox.com \
    --cc=linux-nvme@lists.infradead.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=loberman@redhat.com \
    --cc=maxg@mellanox.com \
    --cc=oren@mellanox.com \
    --cc=rgirase@redhat.com \
    --cc=sagi@grimberg.me \
    --cc=shlomin@mellanox.com \
    --cc=vladimirk@mellanox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).