From mboxrd@z Thu Jan 1 00:00:00 1970 From: Parav Pandit Subject: Re: [PATCHv12 0/3] rdmacg: IB/core: rdma controller support Date: Fri, 4 Nov 2016 10:27:10 +0530 Message-ID: References: <20161018215134.GB2761@htj.duckdns.org> <20161019143345.GA18532@htj.duckdns.org> <20161019192006.GB3044@htj.duckdns.org> <20161019200536.GC3044@htj.duckdns.org> <20161031065441.GY3617@leon.nu> <20161103180006.GL3617@leon.nu> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Return-path: In-Reply-To: Sender: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Liran Liss Cc: Leon Romanovsky , Tejun Heo , "cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , linux-rdma , Li Zefan , Johannes Weiner , Doug Ledford , Christoph Hellwig , "Hefty, Sean" , Jason Gunthorpe , Haggai Eran , "james.l.morris-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org" , Or Gerlitz , Matan Barak List-Id: linux-rdma@vger.kernel.org On Fri, Nov 4, 2016 at 10:22 AM, Liran Liss wrote: >> From: Parav Pandit [mailto:pandit.parav-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org] > >> > >> > A global HCA metric is indeed in the right direction. >> > However, rethinking this, I think that we should specify the metric in terms of >> RDMA objects rather than percentage. >> > Basically, any resource that consumes an IDR is charged. >> > >> If metric definition is based on RDMA objects (count) and not based on >> percentage, how would user specify the metric without really specifying object >> type. >> Current patch defines the metric as absolute numbers and objects as well. >> > > That is the requested change. The absolute number would account for any object allocation. We won't distinguish between types. > Only a single counter (per device). > In that case ucontext deserve a additional count. Because that is handful in range of 256 to 1K. If we give absolute consolidated number as 2000, one container will allocate all the doorbell uctx and no other container can run. Percentage works for this particular case. >> Comment from Leon about his discussion with Matan, Tejun, Christoph says >> opposite of this for user level configuration. >> May be I am missing something. >> >> > The reasons are: >> > - Some HCAs can have a huge amount of resources (millions of objects), of >> which even a small percentage may consume a considerable amount of kernel >> memory. >> > - We follow the same notion as FD limits, which accounts for numerous >> > resource types that consume file objects in the kernel (files, pipes, >> > sockets) >> > - The namespaces for RDMA resources are large (usually 24 bits). So >> > even large resource counts won't come nowhere close in depleting the >> > namespace. (Compare that to the mere 64K socket port space...) >> > - The metric measures the actual application usage of resources, rather than >> proportional to the resources of a given HCA adapter. >> > - We can continue to use the cgroup mechanism for charging (just as in >> > the original proposal) >> > >> > I have discussed this matter with Doug and Matan, and it seems like this is the >> right direction. >> > --Liran >> > -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html