From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757184AbcG0Q5n (ORCPT ); Wed, 27 Jul 2016 12:57:43 -0400 Received: from mga04.intel.com ([192.55.52.120]:21326 "EHLO mga04.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752446AbcG0Q5l (ORCPT ); Wed, 27 Jul 2016 12:57:41 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.28,430,1464678000"; d="scan'208";a="740543386" Date: Wed, 27 Jul 2016 09:57:33 -0700 From: "Luck, Tony" To: Nilay Vaish Cc: Fenghua Yu , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , Tejun Heo , Borislav Petkov , Stephane Eranian , Peter Zijlstra , Marcelo Tosatti , David Carrillo-Cisneros , Ravi V Shankar , Vikas Shivappa , Sai Prakhya , linux-kernel , x86 Subject: Re: [PATCH 13/32] Documentation, x86: Documentation for Intel resource allocation user interface Message-ID: <20160727165732.GA19873@intel.com> References: <1468371785-53231-1-git-send-email-fenghua.yu@intel.com> <1468371785-53231-14-git-send-email-fenghua.yu@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Jul 27, 2016 at 11:20:31AM -0500, Nilay Vaish wrote: > And over here you have switched to using CLOS ID and you do not > mention Cache ID at all. > As I said above, I think Cache ID and CLOS ID are the same thing. If > that is the case, I think Cache ID should be completely replaced with > CLOS ID. Thanks for the input. We need to clarify things here. cache id = unique number identifying a cache in the system. At current state of the patch we only support L3 (a.k.a LLC) CAT, so the cache id is pretty much the socket id. CLOS ID = number we program into the PQR_ASSOC MSR to define the resources available to the currently running process. This number indexes into the arrays of bitmasks used to constrain the process. Currenlty just the L3_CBM MSRs ... but when more resources are added, we use the same CLOS ID to index all of them. So if we have a schema file that says: L3:0=00fff,1=ff000 it means a bunch of things: 1) This is a two socket system (since we have two L3 caches to control) 2) Processes assigned to this rdtgroup will be allowed to use different amounts of cache when they run on cpus in each of the two sockets (the "low" 60% on cache id 0 (socket 0) and the "high" 40% on cache id 1. We can't tell from this which CLOS IDs the kernel decided to allocate to implement this policy. If we just had the default rdtgroup and this group as the only groups available, then it is likely that the kernel will pick CLOS ID 1 on both sockets, and then assign the L3_CBM[1] MSR on socket 0 with 0xfff, and the L3_CBM[1] MSR on socket 1 with the value 0xff000 If the default rdtgroup still has all cache available L3:0=fffff,1=fffff and we add another rdtgroup and give it: L3:0=fffff,1=ff000 the kernel will notice that we want this new rdtgroup to use the same masks as exiting groups ... so it can use CLOS ID 0 on socket 0 (same as default) and CLOS ID 1 on socket 1 (same as the first example I gave above). We do this sharing because there are a limited number of CLOS ID values (limited by the size of the array of L3_CBM MSRs. Hope that is clearer (and we will make a new version of the Docs for the next version) -Tony