Re: [PATCH v6 03/25] rtrs: private headers with rtrs protocol structs and helpers

[Bart Van Assche <bvanassche@acm.org>]

On 1/2/20 7:27 AM, Jinpu Wang wrote:
> On Mon, Dec 30, 2019 at 8:48 PM Bart Van Assche <bvanassche@acm.org> wrote:
>> On 2019-12-30 02:29, Jack Wang wrote:
>>> +enum {
>>> +     SERVICE_CON_QUEUE_DEPTH = 512,
>>
>> What is a service connection?
> s/SERVICE_CON_QUEUE_DEPTH/CON_QUEUE_DEPTH/g, do you think
> CON_QUEUE_DEPTH is better or just QUEUE_DEPTH?

The name of the constant is fine, but what I meant is the following: has 
it been documented anywhere what the role of a "service connection" is?

>>> +struct rtrs_ib_dev_pool {
>>> +     struct mutex            mutex;
>>> +     struct list_head        list;
>>> +     enum ib_pd_flags        pd_flags;
>>> +     const struct rtrs_ib_dev_pool_ops *ops;
>>> +};
>>
>> What is the purpose of an rtrs_ib_dev_pool and what does it contain?
> The idea was documented in the patchset here:
> https://www.spinics.net/lists/linux-rdma/msg64025.html
> "'
> This is an attempt to make a device pool API out of a common code,
> which caches pair of ib_device and ib_pd pointers. I found 4 places,
> where this common functionality can be replaced by some lib calls:
> nvme, nvmet, iser and isert. Total deduplication gain in loc is not
> quite significant, but eventually new ULP IB code can also require
> the same device/pd pair cache, e.g. in our IBTRS module the same
> code has to be repeated again, which was observed by Sagi and he
> suggested to make a common helper function instead of producing
> another copy.
> '''

The word "pool" suggest ownership. Since struct rtrs_ib_dev_pool owns 
protection domains instead of RDMA devices, how about renaming that data 
structure into rtrs_pd_per_rdma_dev, rtrs_rdma_dev_pd or something 
similar? How about adding a comment like the following above that data 
structure?

/*
  * Data structure used to associate one protection domain (PD) with each
  * RDMA device.
  */

>>> +/**
>>> + * struct rtrs_msg_conn_req - Client connection request to the server
>>> + * @magic:      RTRS magic
>>> + * @version:    RTRS protocol version
>>> + * @cid:        Current connection id
>>> + * @cid_num:    Number of connections per session
>>> + * @recon_cnt:          Reconnections counter
>>> + * @sess_uuid:          UUID of a session (path)
>>> + * @paths_uuid:         UUID of a group of sessions (paths)
>>> + *
>>> + * NOTE: max size 56 bytes, see man rdma_connect().
>>> + */
>>> +struct rtrs_msg_conn_req {
>>> +     u8              __cma_version; /* Is set to 0 by cma.c in case of
>>> +                                     * AF_IB, do not touch that.
>>> +                                     */
>>> +     u8              __ip_version;  /* On sender side that should be
>>> +                                     * set to 0, or cma_save_ip_info()
>>> +                                     * extract garbage and will fail.
>>> +                                     */
>>
>> The above two fields and the comments next to it look suspicious to me.
>> Does RTRS perhaps try to generate CMA-formatted messages without using
>> the CMA to format these messages?
> The problem is in cma_format_hdr over-writes the first byte for AF_IB
> https://www.spinics.net/lists/linux-rdma/msg22397.html
> 
> No one fixes the problem since then.

How about adding that URL to the comment block above struct 
rtrs_msg_conn_req?

>>
>>> +     *errno = -(int)((payload >> 19) & 0x1ff);
>>
>> Is the '(int)' cast useful in the above expression? Can it be left out?
> I think it's necessary, and make it more clear errno is a negative int
> value, isn't it?

According to the C standard operations on unsigned integers "wrap 
around" so removing the cast should be safe. Anyway, this is not 
something I consider important.

Thanks,

Bart.