From: Bart Van Assche <bvanassche@acm.org>
To: Jack Wang <jinpuwang@gmail.com>,
linux-block@vger.kernel.org, linux-rdma@vger.kernel.org
Cc: axboe@kernel.dk, hch@infradead.org, sagi@grimberg.me,
leon@kernel.org, dledford@redhat.com,
danil.kipnis@cloud.ionos.com, jinpu.wang@cloud.ionos.com,
rpenyaev@suse.de
Subject: Re: [PATCH v6 06/25] rtrs: client: main functionality
Date: Mon, 30 Dec 2019 15:53:10 -0800 [thread overview]
Message-ID: <e242c08f-68e0-49b7-82e6-924d0124b792@acm.org> (raw)
In-Reply-To: <20191230102942.18395-7-jinpuwang@gmail.com>
On 2019-12-30 02:29, Jack Wang wrote:
> + * InfiniBand Transport Layer
InfiniBand or RDMA?
> +MODULE_DESCRIPTION("RTRS Client");
Please spell out RTRS in full.
> +static const struct rtrs_ib_dev_pool_ops dev_pool_ops;
Can this forward declaration be avoided?
> +static struct rtrs_ib_dev_pool dev_pool = {
> + .ops = &dev_pool_ops
> +};
Can this structure be declared 'const'?
> +static inline struct rtrs_permit *
> +__rtrs_get_permit(struct rtrs_clt *clt, enum rtrs_clt_con_type con_type)
> +{
> + size_t max_depth = clt->queue_depth;
> + struct rtrs_permit *permit;
> + int cpu, bit;
> +
> + cpu = get_cpu();
> + do {
> + bit = find_first_zero_bit(clt->permits_map, max_depth);
> + if (unlikely(bit >= max_depth)) {
> + put_cpu();
> + return NULL;
> + }
> +
> + } while (unlikely(test_and_set_bit_lock(bit, clt->permits_map)));
> + put_cpu();
Are the get_cpu() and put_cpu() calls around this loop useful? If not,
please remove these calls. Otherwise please add a comment that explains
the purpose of these calls.
An additional question: is it possible to replace the above loop with an
sbitmap_get() call?
> +static void complete_rdma_req(struct rtrs_clt_io_req *req, int errno,
> + bool notify, bool can_wait)
> +{
> + struct rtrs_clt_con *con = req->con;
> + struct rtrs_clt_sess *sess;
> + int err;
> +
> + if (WARN_ON(!req->in_use))
> + return;
> + if (WARN_ON(!req->con))
> + return;
> + sess = to_clt_sess(con->c.sess);
> +
> + if (req->sg_cnt) {
> + if (unlikely(req->dir == DMA_FROM_DEVICE && req->need_inv)) {
> + /*
> + * We are here to invalidate RDMA read requests
> + * ourselves. In normal scenario server should
> + * send INV for all requested RDMA reads, but
> + * we are here, thus two things could happen:
> + *
> + * 1. this is failover, when errno != 0
> + * and can_wait == 1,
> + *
> + * 2. something totally bad happened and
> + * server forgot to send INV, so we
> + * should do that ourselves.
> + */
Please document in the protocol documentation when RDMA reads are used.
What does "server forgot to send INV" mean?
Additionally, if I remember correctly Jason considers it very important
that invalidation happens from the submitting context because otherwise
the RDMA retry mechanism can't work.
> +static void process_io_rsp(struct rtrs_clt_sess *sess, u32 msg_id,
> + s16 errno, bool w_inval)
> +{
> + struct rtrs_clt_io_req *req;
> +
> + if (WARN_ON(msg_id >= sess->queue_depth))
> + return;
> +
> + req = &sess->reqs[msg_id];
> + /* Drop need_inv if server responsed with invalidation */
> + req->need_inv &= !w_inval;
> + complete_rdma_req(req, errno, true, false);
> +}
Please document the meaning of the "w_inval" argument. Please also fix
the spelling of "responsed".
Thanks,
Bart.
next prev parent reply other threads:[~2019-12-30 23:53 UTC|newest]
Thread overview: 89+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-12-30 10:29 [PATCH v6 00/25] RTRS (former IBTRS) rdma transport library and RNBD (former IBNBD) rdma network block device Jack Wang
2019-12-30 10:29 ` [PATCH v6 01/25] sysfs: export sysfs_remove_file_self() Jack Wang
2019-12-30 10:29 ` [PATCH v6 02/25] rtrs: public interface header to establish RDMA connections Jack Wang
2019-12-30 19:25 ` Bart Van Assche
2020-01-02 13:35 ` Jinpu Wang
2020-01-02 16:36 ` Bart Van Assche
2020-01-02 16:47 ` Jinpu Wang
2019-12-30 10:29 ` [PATCH v6 03/25] rtrs: private headers with rtrs protocol structs and helpers Jack Wang
2019-12-30 19:48 ` Bart Van Assche
2020-01-02 15:27 ` Jinpu Wang
2020-01-02 17:00 ` Bart Van Assche
2020-01-02 18:26 ` Jason Gunthorpe
2020-01-03 12:31 ` Jinpu Wang
2020-01-03 12:27 ` Jinpu Wang
2019-12-31 0:07 ` Bart Van Assche
2020-01-03 13:48 ` Jinpu Wang
2019-12-30 10:29 ` [PATCH v6 04/25] rtrs: core: lib functions shared between client and server modules Jack Wang
2019-12-30 22:25 ` Bart Van Assche
2020-01-07 12:22 ` Jinpu Wang
2019-12-30 10:29 ` [PATCH v6 05/25] rtrs: client: private header with client structs and functions Jack Wang
2019-12-30 22:51 ` Bart Van Assche
2020-01-07 12:39 ` Jinpu Wang
2019-12-30 23:03 ` Bart Van Assche
2020-01-07 12:39 ` Jinpu Wang
2019-12-30 10:29 ` [PATCH v6 06/25] rtrs: client: main functionality Jack Wang
2019-12-30 23:53 ` Bart Van Assche [this message]
2020-01-02 18:23 ` Jason Gunthorpe
2020-01-03 14:30 ` Jinpu Wang
2020-01-03 16:12 ` Bart Van Assche
2019-12-30 10:29 ` [PATCH v6 07/25] rtrs: client: statistics functions Jack Wang
2020-01-02 21:07 ` Bart Van Assche
2020-01-03 14:39 ` Jinpu Wang
2019-12-30 10:29 ` [PATCH v6 08/25] rtrs: client: sysfs interface functions Jack Wang
2020-01-02 21:14 ` Bart Van Assche
2020-01-03 14:59 ` Jinpu Wang
2019-12-30 10:29 ` [PATCH v6 09/25] rtrs: server: private header with server structs and functions Jack Wang
2020-01-02 21:24 ` Bart Van Assche
2020-01-08 16:33 ` Jinpu Wang
2019-12-30 10:29 ` [PATCH v6 10/25] rtrs: server: main functionality Jack Wang
2020-01-02 22:03 ` Bart Van Assche
2020-01-07 13:19 ` Jinpu Wang
2020-01-07 18:25 ` Jason Gunthorpe
2020-01-10 17:38 ` Jinpu Wang
2019-12-30 10:29 ` [PATCH v6 11/25] rtrs: server: statistics functions Jack Wang
2020-01-02 22:02 ` Bart Van Assche
2020-01-08 12:55 ` Jinpu Wang
2019-12-30 10:29 ` [PATCH v6 12/25] rtrs: server: sysfs interface functions Jack Wang
2020-01-02 22:06 ` Bart Van Assche
2020-01-07 14:40 ` Jinpu Wang
2019-12-30 10:29 ` [PATCH v6 13/25] rtrs: include client and server modules into kernel compilation Jack Wang
2020-01-02 22:11 ` Bart Van Assche
2020-01-03 16:19 ` Jinpu Wang
2019-12-30 10:29 ` [PATCH v6 14/25] rtrs: a bit of documentation Jack Wang
2019-12-30 23:19 ` Bart Van Assche
2020-01-07 14:48 ` Jinpu Wang
2020-01-02 22:21 ` Bart Van Assche
2020-01-07 15:49 ` Jinpu Wang
2019-12-30 10:29 ` [PATCH v6 15/25] rnbd: private headers with rnbd protocol structs and helpers Jack Wang
2020-01-02 22:34 ` Bart Van Assche
2020-01-07 16:53 ` Jinpu Wang
2019-12-30 10:29 ` [PATCH v6 16/25] rnbd: client: private header with client structs and functions Jack Wang
2020-01-02 22:37 ` Bart Van Assche
2020-01-07 17:09 ` Jinpu Wang
2019-12-30 10:29 ` [PATCH v6 17/25] rnbd: client: main functionality Jack Wang
2020-01-02 23:55 ` Bart Van Assche
2020-01-08 14:22 ` Jinpu Wang
2020-01-10 14:45 ` Jinpu Wang
2020-01-10 15:09 ` Roman Penyaev
2020-01-10 15:29 ` Jinpu Wang
2019-12-30 10:29 ` [PATCH v6 18/25] rnbd: client: sysfs interface functions Jack Wang
2020-01-03 0:03 ` Bart Van Assche
2020-01-08 13:06 ` Jinpu Wang
2020-01-08 16:39 ` Bart Van Assche
2020-01-08 16:51 ` Jinpu Wang
2019-12-30 10:29 ` [PATCH v6 19/25] rnbd: server: private header with server structs and functions Jack Wang
2019-12-30 10:29 ` [PATCH v6 20/25] rnbd: server: main functionality Jack Wang
2019-12-30 10:29 ` [PATCH v6 21/25] rnbd: server: functionality for IO submission to file or block dev Jack Wang
2019-12-30 10:29 ` [PATCH v6 22/25] rnbd: server: sysfs interface functions Jack Wang
2019-12-30 10:29 ` [PATCH v6 23/25] rnbd: include client and server modules into kernel compilation Jack Wang
2019-12-30 10:29 ` [PATCH v6 24/25] rnbd: a bit of documentation Jack Wang
2019-12-30 10:29 ` [PATCH v6 25/25] MAINTAINERS: Add maintainers for RNBD/RTRS modules Jack Wang
2019-12-30 12:22 ` Gal Pressman
2020-01-02 8:41 ` Jinpu Wang
2019-12-31 0:11 ` [PATCH v6 00/25] RTRS (former IBTRS) rdma transport library and RNBD (former IBNBD) rdma network block device Bart Van Assche
2020-01-02 8:48 ` Jinpu Wang
2019-12-31 2:39 ` Bart Van Assche
2020-01-02 9:20 ` Jinpu Wang
2020-01-02 18:28 ` Jason Gunthorpe
2020-01-03 12:34 ` Jinpu Wang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e242c08f-68e0-49b7-82e6-924d0124b792@acm.org \
--to=bvanassche@acm.org \
--cc=axboe@kernel.dk \
--cc=danil.kipnis@cloud.ionos.com \
--cc=dledford@redhat.com \
--cc=hch@infradead.org \
--cc=jinpu.wang@cloud.ionos.com \
--cc=jinpuwang@gmail.com \
--cc=leon@kernel.org \
--cc=linux-block@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=rpenyaev@suse.de \
--cc=sagi@grimberg.me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).