From: Sagi Grimberg <sagi-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
To: Chuck Lever <chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
Cc: List Linux RDMA Mailing
<linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
Linux NFS Mailing List
<linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
Subject: Re: [PATCH v1 03/14] svcrdma: Eliminate RPCRDMA_SQ_DEPTH_MULT
Date: Wed, 22 Mar 2017 21:06:49 +0200 [thread overview]
Message-ID: <68e3eda0-90f4-5bca-28be-b2cf494ed172@grimberg.me> (raw)
In-Reply-To: <C9D8A91C-DE08-4A41-A07D-1F4C42DD9B97-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
> Roughly speaking, I think there needs to be an rdma_rw API that
> assists the ULP with setting its CQ and SQ sizes, since rdma_rw
> hides the registration mode (one of which, at least, consumes
> more SQEs than the other).
Hiding the registration mode was the largely the motivation for
this... It buys us simplified implementation and inherently supports
both IB and iWARP (which was annoying and only existing in svc but
still suboptimal).
> I'd like to introduce one new function call that surfaces the
> factor used to compute how many additional SQEs that rdma_rw will
> need. The ULP will invoke it before allocating new Send CQs.
I see your point... We should probably get a sense on how to
size the completion queue. I think that this issue is solved with
the CQ pool API that Christoph sent a while ago but was never
pursued.
The basic idea is that the core would create a pool of long CQs
and then assigns queue-pairs depending on the sq+rq depth.
If we were to pick it up would you consider using it?
> I'll try to provide an RFC in the nfsd-rdma-rw-api topic branch.
Cool, lets see what you had in mind...
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
WARNING: multiple messages have this Message-ID (diff)
From: Sagi Grimberg <sagi@grimberg.me>
To: Chuck Lever <chuck.lever@oracle.com>
Cc: List Linux RDMA Mailing <linux-rdma@vger.kernel.org>,
Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Subject: Re: [PATCH v1 03/14] svcrdma: Eliminate RPCRDMA_SQ_DEPTH_MULT
Date: Wed, 22 Mar 2017 21:06:49 +0200 [thread overview]
Message-ID: <68e3eda0-90f4-5bca-28be-b2cf494ed172@grimberg.me> (raw)
In-Reply-To: <C9D8A91C-DE08-4A41-A07D-1F4C42DD9B97@oracle.com>
> Roughly speaking, I think there needs to be an rdma_rw API that
> assists the ULP with setting its CQ and SQ sizes, since rdma_rw
> hides the registration mode (one of which, at least, consumes
> more SQEs than the other).
Hiding the registration mode was the largely the motivation for
this... It buys us simplified implementation and inherently supports
both IB and iWARP (which was annoying and only existing in svc but
still suboptimal).
> I'd like to introduce one new function call that surfaces the
> factor used to compute how many additional SQEs that rdma_rw will
> need. The ULP will invoke it before allocating new Send CQs.
I see your point... We should probably get a sense on how to
size the completion queue. I think that this issue is solved with
the CQ pool API that Christoph sent a while ago but was never
pursued.
The basic idea is that the core would create a pool of long CQs
and then assigns queue-pairs depending on the sq+rq depth.
If we were to pick it up would you consider using it?
> I'll try to provide an RFC in the nfsd-rdma-rw-api topic branch.
Cool, lets see what you had in mind...
next prev parent reply other threads:[~2017-03-22 19:06 UTC|newest]
Thread overview: 70+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-03-16 15:52 [PATCH v1 00/14] Server-side NFS/RDMA changes for v4.12 Chuck Lever
2017-03-16 15:52 ` Chuck Lever
[not found] ` <20170316154132.4482.56769.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-16 15:52 ` [PATCH v1 01/14] svcrdma: Move send_wr to svc_rdma_op_ctxt Chuck Lever
2017-03-16 15:52 ` Chuck Lever
[not found] ` <20170316155234.4482.94225.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-21 17:49 ` Sagi Grimberg
2017-03-21 17:49 ` Sagi Grimberg
2017-03-16 15:52 ` [PATCH v1 02/14] svcrdma: Add svc_rdma_map_reply_hdr() Chuck Lever
2017-03-16 15:52 ` Chuck Lever
[not found] ` <20170316155242.4482.64809.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-21 17:54 ` Sagi Grimberg
2017-03-21 17:54 ` Sagi Grimberg
[not found] ` <f5000e25-6ca1-fc24-35c0-6089cf50923c-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-03-21 18:40 ` Chuck Lever
2017-03-21 18:40 ` Chuck Lever
[not found] ` <A18F9D5E-09BA-4268-9AA6-3E5866101F76-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2017-03-22 13:07 ` Sagi Grimberg
2017-03-22 13:07 ` Sagi Grimberg
2017-03-16 15:52 ` [PATCH v1 03/14] svcrdma: Eliminate RPCRDMA_SQ_DEPTH_MULT Chuck Lever
2017-03-16 15:52 ` Chuck Lever
[not found] ` <20170316155250.4482.49638.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-21 17:58 ` Sagi Grimberg
2017-03-21 17:58 ` Sagi Grimberg
[not found] ` <46eb6195-a542-b35c-4902-a2bebb38feba-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-03-21 18:44 ` Chuck Lever
2017-03-21 18:44 ` Chuck Lever
[not found] ` <391F0D90-2A46-4B2F-BCF0-B3BE7D48A3EF-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2017-03-22 13:09 ` Sagi Grimberg
2017-03-22 13:09 ` Sagi Grimberg
[not found] ` <ec82feb4-d6b9-7fb4-5b11-b8007e313845-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-03-22 13:36 ` Chuck Lever
2017-03-22 13:36 ` Chuck Lever
[not found] ` <C9D8A91C-DE08-4A41-A07D-1F4C42DD9B97-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2017-03-22 19:06 ` Sagi Grimberg [this message]
2017-03-22 19:06 ` Sagi Grimberg
[not found] ` <68e3eda0-90f4-5bca-28be-b2cf494ed172-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-03-22 19:30 ` Chuck Lever
2017-03-22 19:30 ` Chuck Lever
2017-03-16 15:52 ` [PATCH v1 04/14] svcrdma: Add helper to save pages under I/O Chuck Lever
2017-03-16 15:52 ` Chuck Lever
[not found] ` <20170316155258.4482.69182.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-21 18:01 ` Sagi Grimberg
2017-03-21 18:01 ` Sagi Grimberg
2017-03-16 15:53 ` [PATCH v1 05/14] svcrdma: Introduce local rdma_rw API helpers Chuck Lever
2017-03-16 15:53 ` Chuck Lever
[not found] ` <20170316155306.4482.68041.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-22 14:17 ` Sagi Grimberg
2017-03-22 14:17 ` Sagi Grimberg
[not found] ` <cfa49433-ab26-d2f0-27d4-2a96ff0adaba-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2017-03-22 15:41 ` Chuck Lever
2017-03-22 15:41 ` Chuck Lever
[not found] ` <1CAD2542-A121-47ED-A47C-624E188EB54F-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2017-03-24 22:19 ` Chuck Lever
2017-03-24 22:19 ` Chuck Lever
2017-03-16 15:53 ` [PATCH v1 06/14] svcrdma: Use rdma_rw API in RPC reply path Chuck Lever
2017-03-16 15:53 ` Chuck Lever
2017-03-16 15:53 ` [PATCH v1 07/14] svcrdma: Clean up RDMA_ERROR path Chuck Lever
2017-03-16 15:53 ` Chuck Lever
[not found] ` <20170316155323.4482.8051.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-22 14:18 ` Sagi Grimberg
2017-03-22 14:18 ` Sagi Grimberg
2017-03-16 15:53 ` [PATCH v1 08/14] svcrdma: Report Write/Reply chunk overruns Chuck Lever
2017-03-16 15:53 ` Chuck Lever
[not found] ` <20170316155331.4482.7734.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-22 14:20 ` Sagi Grimberg
2017-03-22 14:20 ` Sagi Grimberg
2017-03-16 15:53 ` [PATCH v1 09/14] svcrdma: Clean up RPC-over-RDMA backchannel reply processing Chuck Lever
2017-03-16 15:53 ` Chuck Lever
2017-03-16 15:53 ` [PATCH v1 10/14] svcrdma: Reduce size of sge array in struct svc_rdma_op_ctxt Chuck Lever
2017-03-16 15:53 ` Chuck Lever
[not found] ` <20170316155347.4482.74652.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-22 14:21 ` Sagi Grimberg
2017-03-22 14:21 ` Sagi Grimberg
2017-03-16 15:53 ` [PATCH v1 11/14] svcrdma: Remove old RDMA Write completion handlers Chuck Lever
2017-03-16 15:53 ` Chuck Lever
[not found] ` <20170316155355.4482.35026.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-22 14:22 ` Sagi Grimberg
2017-03-22 14:22 ` Sagi Grimberg
2017-03-16 15:54 ` [PATCH v1 12/14] svcrdma: Remove the req_map cache Chuck Lever
2017-03-16 15:54 ` Chuck Lever
[not found] ` <20170316155403.4482.2040.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-22 14:22 ` Sagi Grimberg
2017-03-22 14:22 ` Sagi Grimberg
2017-03-16 15:54 ` [PATCH v1 13/14] svcrdma: Clean out old XDR encoders Chuck Lever
2017-03-16 15:54 ` Chuck Lever
[not found] ` <20170316155411.4482.37224.stgit-Hs+gFlyCn65vLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-03-22 14:23 ` Sagi Grimberg
2017-03-22 14:23 ` Sagi Grimberg
2017-03-16 15:54 ` [PATCH v1 14/14] svcrdma: Clean up svc_rdma_post_recv() error handling Chuck Lever
2017-03-16 15:54 ` Chuck Lever
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=68e3eda0-90f4-5bca-28be-b2cf494ed172@grimberg.me \
--to=sagi-nqwnxtmzq1alnmji0ikvqw@public.gmane.org \
--cc=chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org \
--cc=linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.