From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751592AbbHECVx (ORCPT ); Tue, 4 Aug 2015 22:21:53 -0400 Received: from mga03.intel.com ([134.134.136.65]:8645 "EHLO mga03.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750811AbbHECVw (ORCPT ); Tue, 4 Aug 2015 22:21:52 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.15,613,1432623600"; d="scan'208";a="776992614" Date: Tue, 4 Aug 2015 19:21:52 -0700 (PDT) From: Vikas Shivappa X-X-Sender: vikas@vshiva-Udesk To: Tejun Heo cc: Vikas Shivappa , Vikas Shivappa , linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, tglx@linutronix.de, mingo@kernel.org, peterz@infradead.org, Matt Fleming , "Auld, Will" , "Williamson, Glenn P" , "Juvva, Kanaka D" Subject: Re: [PATCH 5/9] x86/intel_rdt: Add new cgroup and Class of service management In-Reply-To: <20150804190324.GH17598@mtj.duckdns.org> Message-ID: References: <1435789270-27010-1-git-send-email-vikas.shivappa@linux.intel.com> <1435789270-27010-6-git-send-email-vikas.shivappa@linux.intel.com> <20150730194458.GD3504@mtj.duckdns.org> <20150802163157.GB32599@mtj.duckdns.org> <20150804190324.GH17598@mtj.duckdns.org> User-Agent: Alpine 2.10 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 4 Aug 2015, Tejun Heo wrote: > Hello, Vikas. > > On Tue, Aug 04, 2015 at 11:50:16AM -0700, Vikas Shivappa wrote: >> I will make this more clear in the documentation - We intend this cgroup >> interface to be used by a root or superuser - more like a system >> administrator being able to control the allocation of the threads , the one >> who has the knowledge of the usage and being able to decide. > > I get that this would be an easier "bolt-on" solution but isn't a good > solution by itself in the long term. As I wrote multiple times > before, this is a really bad programmable interface. Unless you're > sure that this doesn't have to be programmable for threads of an > individual applications, Yes, this doesnt have to be a programmable interface for threads. May not be a good idea to let the threads decide the cache allocation by themselves using this direct interface. We are transfering the decision maker responsibility to the system administrator. - This interface like you said can easily bolt-on. basically an easy to use interface without worrying about the architectural details. - But still does the job. root user can allocate exclusive or overlapping cache lines to threads or group of threads. - No major roadblocks for usage as we can make the allocations like mentioned above and still keep the hierarchy etc and use it when needed. - An important factor is that it can co-exist with other interfaces like #2 and #3 for the same easily. So I donot see a reason why we should not use this. This is not meant to be a programmable interface, however it does not prevent co-existence. - If root user has to set affinity of threads that he is allocating cache, he can do so using other cgroups like cpuset or set the masks seperately using taskset. This would let him configure the cache allocation on a socket. this is a pretty bad interface by itself. > >> There is already a lot of such usage among different enterprise users at >> Intel/google/cisco etc who have been testing the patches posted to lkml and >> academically there is plenty of usage as well. > > I mean, that's the tool you gave them. Of course they'd be using it > but I suspect most of them would do fine with a programmable interface > too. Again, please think of cpu affinity. All the methodology to support the feature may need an arbitrator/agent to decide the allocation. 1. Let the root user or system administrator be the one who decides the allocation based on the current usage. We assume this to be one with administrative privileges. He could use the cgroup interface to perform the task. One way to do the cpu affinity is by mounting cpuset and rdt cgroup together. 2. Kernel automatically assigning the cache based on the priority of the apps etc. This is something which could be designed to co-exist with the #1 above much like how the cpusets cgroup co-exist with the kernel assigning cpus to tasks. (the task could be having a cache capacity mask just like the cpu affinity mask) 3. User programmable interface , where say a resource management program x (and hence apps) could link a library which supports cache alloc/monitoring etc and then try to control and monitor the resources. The arbitrator could just be the resource management interface itself or the kernel could decide. If users use this programmable interface, we need to make sure all the apps just cannot allocate resources without some interfacing agent (in which case they could interface with #2 ?). Do you think there are any issues for the user programmable interface to co-exist with the cgroup interface ? Thanks, Vikas > > Thanks. > > -- > tejun >