From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754017AbbG3RsB (ORCPT ); Thu, 30 Jul 2015 13:48:01 -0400 Received: from mga02.intel.com ([134.134.136.20]:44477 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750935AbbG3RsA (ORCPT ); Thu, 30 Jul 2015 13:48:00 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.15,577,1432623600"; d="scan'208";a="774679122" Date: Thu, 30 Jul 2015 10:47:23 -0700 (PDT) From: Vikas Shivappa X-X-Sender: vikas@vshiva-Udesk To: Marcelo Tosatti cc: "Auld, Will" , "Shivappa, Vikas" , Vikas Shivappa , "linux-kernel@vger.kernel.org" , "x86@kernel.org" , "hpa@zytor.com" , "tglx@linutronix.de" , "mingo@kernel.org" , "tj@kernel.org" , "peterz@infradead.org" , "Fleming, Matt" , "Williamson, Glenn P" , "Juvva, Kanaka D" Subject: Re: [PATCH 3/9] x86/intel_rdt: Cache Allocation documentation and cgroup usage guide In-Reply-To: <20150729193208.GC3201@amt.cnet> Message-ID: References: <1435789270-27010-1-git-send-email-vikas.shivappa@linux.intel.com> <1435789270-27010-4-git-send-email-vikas.shivappa@linux.intel.com> <20150728231516.GA16204@amt.cnet> <96EC5A4F3149B74492D2D9B9B1602C27461EB932@ORSMSX105.amr.corp.intel.com> <20150729193208.GC3201@amt.cnet> User-Agent: Alpine 2.10 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Marcello, On Wed, 29 Jul 2015, Marcelo Tosatti wrote: > > How about this: > > desiredclos (closid p1 p2 p3 p4) > 1 1 0 0 0 > 2 0 0 0 1 > 3 0 1 1 0 #1 Currently in the rdt cgroup , the root cgroup always has all the bits set and cant be changed (because the cgroup hierarchy would by default make this to have all bits as all the children need to have a subset of the root's bitmask). So if the user creates a cgroup and not put any task in it , the tasks in the root cgroup could be still using that part of the cache. Thats the reason i say we can have really 'exclusive' masks. Or in other words - there is always a desired clos (0) which has all parts set which acts like a default pool. Also the parts can overlap. Please apply this for all the below comments which will change the way they work. > > p means part. I am assuming p = (a contiguous cache capacity bit mask) > closid 1 is a exclusive cgroup. > closid 2 is a "cache hog" class. > closid 3 is "default closid". > > Desiredclos is what user has specified. > > Transition 1: desiredclos --> effectiveclos > Clean all bits of unused closid's > (that must be updated whenever a > closid1 cgroup goes from empty->nonempty > and vice-versa). > > effectiveclos (closid p1 p2 p3 p4) > 1 0 0 0 0 > 2 0 0 0 1 > 3 0 1 1 0 > > Transition 2: effectiveclos --> expandedclos > expandedclos (closid p1 p2 p3 p4) > 1 0 0 0 0 > 2 0 0 0 1 > 3 1 1 1 0 > Then you have different inplacecos for each > CPU (see pseudo-code below): > > On the following events. > > - task migration to new pCPU: > - task creation: > > id = smp_processor_id(); > for (part = desiredclos.p1; ...; part++) > /* if my cosid is set and any other > cosid is clear, for the part, > synchronize desiredclos --> inplacecos */ > if (part[mycosid] == 1 && > part[any_othercosid] == 0) > wrmsr(part, desiredclos); > Currently the root cgroup would have all the bits set which will act like a default cgroup where all the otherwise unused parts (assuming they are a set of contiguous cache capacity bits) will be used. Otherwise the question is in the expandedclos - who decides to expand the closx parts to include some of the unused parts.. - that could just be a default root always ? Thanks, Vikas >