From: Yamin Friedman <yaminf@mellanox.com>
To: Devesh Sharma <devesh.sharma@broadcom.com>
Cc: Jason Gunthorpe <jgg@mellanox.com>,
Sagi Grimberg <sagi@grimberg.me>,
Or Gerlitz <ogerlitz@mellanox.com>,
Leon Romanovsky <leonro@mellanox.com>,
linux-rdma <linux-rdma@vger.kernel.org>
Subject: Re: [PATCH V3 2/4] RDMA/core: Introduce shared CQ pool API
Date: Wed, 20 May 2020 15:01:56 +0300 [thread overview]
Message-ID: <98c26ee4-742c-a6fc-bb1c-5134c3361dfd@mellanox.com> (raw)
In-Reply-To: <CANjDDBhenmz=k21BBhK91LwQ9OjgrdPUZx-Vvu2PvUpj0YvNAw@mail.gmail.com>
On 5/20/2020 1:50 PM, Devesh Sharma wrote:
> On Wed, May 20, 2020 at 2:53 PM Yamin Friedman <yaminf@mellanox.com> wrote:
>>
>> On 5/20/2020 9:19 AM, Devesh Sharma wrote:
>>>> +
>>>> +static int ib_alloc_cqs(struct ib_device *dev, int nr_cqes,
>>>> + enum ib_poll_context poll_ctx)
>>>> +{
>>>> + LIST_HEAD(tmp_list);
>>>> + struct ib_cq *cq;
>>>> + unsigned long flags;
>>>> + int nr_cqs, ret, i;
>>>> +
>>>> + /*
>>>> + * Allocated at least as many CQEs as requested, and otherwise
>>>> + * a reasonable batch size so that we can share CQs between
>>>> + * multiple users instead of allocating a larger number of CQs.
>>>> + */
>>>> + nr_cqes = min(dev->attrs.max_cqe, max(nr_cqes, IB_MAX_SHARED_CQ_SZ));
>>>> + nr_cqs = min_t(int, dev->num_comp_vectors, num_online_cpus());
>>> No WARN() or return with failure as pointed by Leon and me. Has
>>> anything else changes elsewhere?
>> Hey Devesh,
>>
>> I am not sure what you are referring to, could you please clarify?
>>
> I thought on V2 Leon gave a comment "how this will work if
> dev->num_comp_vectors" is 0.
> there I had suggested to fail the pool creation and issue a
> WARN_ONCE() or something.
I understood his comment to be regarding if the comp_vector itself is 0.
There should not be any issue with that case.
As far as I am aware there must be a non-zero amount of comp_vectors for
the ib_dev otherwise we will not be able to get any indication for cqes.
I don't see any reason to add a special check here.
Thanks
>>>> + for (i = 0; i < nr_cqs; i++) {
>>>> + cq = ib_alloc_cq(dev, NULL, nr_cqes, i, poll_ctx);
>>>> + if (IS_ERR(cq)) {
>>>> + ret = PTR_ERR(cq);
>>>> + goto out_free_cqs;
>>>> + }
>>>> + cq->shared = true;
>>>> + list_add_tail(&cq->pool_entry, &tmp_list);
>>>> + }
>>>> +
>>>> + spin_lock_irqsave(&dev->cq_pools_lock, flags);
>>>> + list_splice(&tmp_list, &dev->cq_pools[poll_ctx - 1]);
>>>> + spin_unlock_irqrestore(&dev->cq_pools_lock, flags);
>>>> +
>>>> + return 0;
>>>> +
>>>> +out_free_cqs:
>>>> + list_for_each_entry(cq, &tmp_list, pool_entry) {
>>>> + cq->shared = false;
>>>> + ib_free_cq(cq);
>>>> + }
>>>> + return ret;
>>>> +}
>>>> +
>>>>
next prev parent reply other threads:[~2020-05-20 12:02 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-19 12:43 [PATCH V3 0/4] Introducing RDMA shared CQ pool Yamin Friedman
2020-05-19 12:43 ` [PATCH V3 1/4] RDMA/core: Add protection for shared CQs used by ULPs Yamin Friedman
2020-05-19 12:43 ` [PATCH V3 2/4] RDMA/core: Introduce shared CQ pool API Yamin Friedman
2020-05-20 6:19 ` Devesh Sharma
2020-05-20 9:23 ` Yamin Friedman
2020-05-20 9:32 ` Leon Romanovsky
2020-05-20 10:50 ` Devesh Sharma
2020-05-20 12:01 ` Yamin Friedman [this message]
2020-05-20 13:48 ` Devesh Sharma
2020-05-25 13:06 ` Yamin Friedman
2020-05-26 7:09 ` Yamin Friedman
2020-05-25 15:14 ` Bart Van Assche
2020-05-25 16:45 ` Jason Gunthorpe
2020-05-26 11:43 ` Yamin Friedman
2020-05-25 16:42 ` Jason Gunthorpe
2020-05-25 16:47 ` Leon Romanovsky
2020-05-26 11:39 ` Yamin Friedman
2020-05-26 12:09 ` Jason Gunthorpe
2020-05-19 12:43 ` [PATCH V3 3/4] nvme-rdma: use new shared CQ mechanism Yamin Friedman
2020-05-19 12:43 ` [PATCH V3 4/4] nvmet-rdma: " Yamin Friedman
2020-05-20 7:03 ` [PATCH V3 0/4] Introducing RDMA shared CQ pool Sagi Grimberg
2020-05-20 8:15 ` Yamin Friedman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=98c26ee4-742c-a6fc-bb1c-5134c3361dfd@mellanox.com \
--to=yaminf@mellanox.com \
--cc=devesh.sharma@broadcom.com \
--cc=jgg@mellanox.com \
--cc=leonro@mellanox.com \
--cc=linux-rdma@vger.kernel.org \
--cc=ogerlitz@mellanox.com \
--cc=sagi@grimberg.me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).