Re: [PATCHv12 0/3] rdmacg: IB/core: rdma controller support

[Parav Pandit <pandit.parav-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>]

Hi Tejun,


On Thu, Nov 10, 2016 at 11:02 PM, Tejun Heo <tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
> Hello, Parav.
>
> On Thu, Nov 10, 2016 at 10:34:44PM +0530, Parav Pandit wrote:
>> user-kernel interface:
>> ---------------------------
>> (a) rdma.current - Will continue to report resource count.
>> (b) rdma.max - will continue to accept hca_handles, and hca_objects as
>> absolute number.
>>
>> Instead of mr, pd, qp, ah, srq entries of patch_v12, it will have just
>> two entries for each device.
>> (1) hca_handles, (2) hca_objects.
>>
>> rdmacg - IB stack interface:
>> --------------------------------
>> cgroup_rdma.h will have two enum entries.
>>
>> RDMACG_RESOURCE_HCA_HANDLE
>> RDMACG_RESOURCE_OBJECT
>>
>> IB stack will charge either of the object.
>> When HCA handles are allocated/freed IB core will request charge/uncharge.
>> When standard verb resources such as PD, MR, CQ, QP, SRQ are
>> allocated/freed IB core will request for XX_OBJECT charge/uncharge.
>>
>> Currently defined APIs and interfaces just remains same.
>
> That looks great to me from cgroup side.  Do you have plans for
> exposing the maximum numbers available?
>
I thought more on this.
If I have to expose max limits, I need new file interface as rdma.limit.
Because once rdma.max is set, user space cannot get back the old value.
It needs to cache it. user space tools might have been restarted and
so on, so store in other file etc.
So such user space solutions are just ugly.

Getting and setting values in device agnostic way, through cgroup
files is desirable, however its not must. It can fallback on using
verb based API.

So if there is no objection, I prefer to have rdma.limit file as
incremental patch once base version is merged.


> Thanks.
>
> --
> tejun