From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S938622AbcJVNUW (ORCPT ); Sat, 22 Oct 2016 09:20:22 -0400 Received: from mga11.intel.com ([192.55.52.93]:27024 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S938446AbcJVNUU (ORCPT ); Sat, 22 Oct 2016 09:20:20 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.31,382,1473145200"; d="scan'208";a="1048452430" From: "Fenghua Yu" To: "Thomas Gleixner" Cc: "H. Peter Anvin" , "Ingo Molnar" , "Tony Luck" , "Peter Zijlstra" , "Stephane Eranian" , "Borislav Petkov" , "Dave Hansen" , "Nilay Vaish" , "Shaohua Li" , "David Carrillo-Cisneros" , "Ravi V Shankar" , "Sai Prakhya" , "Vikas Shivappa" , "linux-kernel" , "x86" , "Fenghua Yu" Subject: [PATCH v5 00/18] Intel Cache Allocation Technology Date: Sat, 22 Oct 2016 06:19:47 -0700 Message-Id: <1477142405-32078-1-git-send-email-fenghua.yu@intel.com> X-Mailer: git-send-email 1.8.0.1 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Fenghua Yu This version should cover all comments from Thomas. The patches are in the same order as V4 and have small commit messages changes in patch #16. 0001-Documentation-ABI-Add-a-document-entry-for-cache-id.patch 0002-cacheinfo-Introduce-cache-id.patch 0003-x86-intel_cacheinfo-Enable-cache-id-in-x86.patch These three define an "id" for each cache ... we need a "name" for a cache so we can say what restrictions to apply to each cache in the system. All you will see at this point is an extra "id" file in each /sys/devices/system/cpu/cpu*/cache/index*/ directory. 0004-x86-intel_rdt-Feature-discovery.patch Look at CPUID for the features related to cache allocation. At this point /proc/cpuinfo shows extra flags for the features found on your system. 0005-Documentation-x86-Documentation-for-Intel-resource-a.patch Documentation patch could be anywhere in this sequence. We put in early so you can read it to see how to use the interface. 0006-x86-intel_rdt-Add-CONFIG-Makefile-and-basic-initiali.patch Add CONFIG_INTEL_RDT (default "n" ... you'll have to set it to have this, and all the following patches do anything). Template driver here just checks for features and spams the console with one line for each. 0007-x86-intel_rdt-Add-Haswell-feature-discovery.patch There are some Haswell systems that support cache allocation, but they were made before the CPUID bits were fully defined. So we check by probing the CBM base MSR to see if CLOSID bits stick. Unless you have one of these Haswells, you won't see any difference here. 0008-x86-intel_rdt-Pick-up-L3-L2-RDT-parameters-from-CPUID.patch This is all new code, not seen in the previous versions of this patch series. L3 and L2 cache allocations are just the first of several resource control features. Define rdt_resource structure that contains all the useful things we need to know about a resource. Pick up the parameters for the resource from CPUID. The console spam strings change format here. 0009-x86-cqm-Move-PQR_ASSOC-management-code-into-generic-.patch The PQR_ASSOC MSR has a field for the CLOSID (which we need define which allocation rules are in effect). But it also contains the RMID (used by CQM and MBM perf monitoring). The perf code got here first, but defined structures that make it easy for the two systems to co-exist without stomping on each other. This patch moves the relevant parts into a common header file and changes the scope from "static" to global so we can access them. No visible change. 0010-x86-intel_rdt-Build-structures-for-each-resource-bas.patch For each enabled resource, we build a list of "rdt_domains" based on hotplug cpu notifications. Since we only have L3 at this point, this is just a list of L3 caches (named by the "id" established in the first three patches). As each cache is found we initialize the array of CBMs (cache bit masks). No visible change here. 0011-x86-intel_rdt-Add-basic-resctrl-filesystem-support.patch Our interface is a kernfs backed file system. Establish the mount point, and provide mount/unmount functionality. At this point "/sys/fs/resctrl" appears. You can mount and unmount the resctrl file system (if your system supports code/data prioritization, you can use the "cdp" mount option). The file system is empty and doesn't allow creation of any files or subdirectories. 0012-x86-intel_rdt-Add-info-files-to-resctrl-file-system.patch Parameters for each resource are buried in CPUID leaf 0x10. This isn't very user friendly for scripts and applications that want to configure resource allocation. Create an "info" directory, with a subdirectory for each resource containing a couple of useful parameters. Visible change: $ ls -l /sys/fs/resctrl/info/L3 total 0 -r--r--r-- 1 root root 0 Oct 7 11:20 cbm_val -r--r--r-- 1 root root 0 Oct 7 11:20 num_closid 0013-x86-intel_rdt-Add-mkdir-to-resctrl-file-system.patch Each resource group is represented by a directory in the resctrl file system. The root directory is the default group. Use "mkdir" to create new groups and "rmdir" to remove them. The maximum number of groups is defined by the effective number of CLOSIDs. Visible change: If you have CDP (and enable with the "cdp" mount option) you will find that you can only create half as many groups as without (e.g. 8 vs. 16 on Broadwell, but the default group uses one ... so actually 7, 15). 0014-x86-intel_rdt-Add-cpus-file.patch One of the control mechanisms for a resource group is the logical CPU. Initially all CPUs are assigned to the default group. They can be reassigned to other groups by writing a cpumask to the "cpus" file. See the documentation for what this means. Visible change: "cpus" file in the root, and automatically in each created subdirectory. You can "echo" masks to these files and watch as CPUs added to one group are removed from whatever group they previously belonged to. Removing a directory will give all CPUs owned by it back to the default (root) group. 0015-x86-intel_rdt-Add-tasks-files.patch Tasks can be assigned to resource groups by writing their PID to a "tasks" file (which removes the task from its previous group). Forked/cloned tasks inherit the group from their parent. You cannot remove a group (directory) that has any tasks assigned. Visible change: "tasks" files appear. E.g. (we see two tasks in the group, our shell, and the "cat" that it spawned). # echo $$ > p0/tasks; cat p0/tasks 268890 268914 0016-x86-intel_rdt-Add-schemata-file.patch The "schemata" file in each group/directory defines what access tasks controlled by this resource are permitted. One line per resource type. Fields for each instance of the resource. You redefine the access by wrting to the file in the same format. Visible change: "schemata" file which starts out with maximum allowed resources. E.g. $ cat schemata L3:0=fffff;1=fffff Now restrict this group to just 20% of L3 on first cache, but allow 50% on the second # echo L3:0=f;1=3ff > schemata 0017-x86-intel_rdt-Add-scheduler-hook.patch When context switching we check if we are changing resource groups for the new process, and update the PQR_ASSOC MSR with the new CLOSID if needed. Visble change: Everything should be working now. Tasks run with the permitted access to L3 cache. 0018-MAINTAINERS-Add-maintainer-for-Intel-RDT-resource-al.patch New files ... need a maintainer. Fenghua has the job. Fenghua Yu (14): cacheinfo: Introduce cache id x86/intel_cacheinfo: Enable cache id in cache info x86/intel_rdt: Feature discovery Documentation, x86: Documentation for Intel resource allocation user interface x86/intel_rdt: Add CONFIG, Makefile, and basic initialization x86/intel_rdt: Add Haswell feature discovery x86/intel_rdt: Pick up L3/L2 RDT parameters from CPUID x86/cqm: Move PQR_ASSOC management code into generic code used by both CQM and CAT x86/intel_rdt: Add basic resctrl filesystem support x86/intel_rdt: Add "info" files to resctrl file system x86/intel_rdt: Add mkdir to resctrl file system x86/intel_rdt: Add tasks files x86/intel_rdt: Add scheduler hook MAINTAINERS: Add maintainer for Intel RDT resource allocation Tony Luck (4): Documentation, ABI: Add a document entry for cache id x86/intel_rdt: Build structures for each resource based on cache topology x86/intel_rdt: Add cpus file x86/intel_rdt: Add schemata file Documentation/ABI/testing/sysfs-devices-system-cpu | 16 + Documentation/x86/intel_rdt_ui.txt | 167 ++++ MAINTAINERS | 8 + arch/x86/Kconfig | 12 + arch/x86/events/intel/cqm.c | 23 +- arch/x86/include/asm/cpufeatures.h | 5 + arch/x86/include/asm/intel_rdt.h | 219 +++++ arch/x86/include/asm/intel_rdt_common.h | 27 + arch/x86/kernel/cpu/Makefile | 2 + arch/x86/kernel/cpu/intel_cacheinfo.c | 20 + arch/x86/kernel/cpu/intel_rdt.c | 382 ++++++++ arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 1013 ++++++++++++++++++++ arch/x86/kernel/cpu/intel_rdt_schemata.c | 245 +++++ arch/x86/kernel/cpu/scattered.c | 3 + arch/x86/kernel/process_32.c | 4 + arch/x86/kernel/process_64.c | 4 + drivers/base/cacheinfo.c | 5 + include/linux/cacheinfo.h | 3 + include/linux/sched.h | 3 + include/uapi/linux/magic.h | 1 + 20 files changed, 2141 insertions(+), 21 deletions(-) create mode 100644 Documentation/x86/intel_rdt_ui.txt create mode 100644 arch/x86/include/asm/intel_rdt.h create mode 100644 arch/x86/include/asm/intel_rdt_common.h create mode 100644 arch/x86/kernel/cpu/intel_rdt.c create mode 100644 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c create mode 100644 arch/x86/kernel/cpu/intel_rdt_schemata.c -- 2.5.0