linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: James Morse <james.morse@arm.com>
To: x86@kernel.org, linux-kernel@vger.kernel.org
Cc: Fenghua Yu <fenghua.yu@intel.com>,
	Reinette Chatre <reinette.chatre@intel.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	H Peter Anvin <hpa@zytor.com>, Babu Moger <Babu.Moger@amd.com>,
	James Morse <james.morse@arm.com>,
	shameerali.kolothum.thodi@huawei.com,
	Jamie Iles <jamie@nuviainc.com>,
	D Scott Phillips OS <scott@os.amperecomputing.com>,
	lcherian@marvell.com
Subject: [PATCH v5 00/24] x86/resctrl: Merge the CDP resources
Date: Thu, 17 Jun 2021 17:57:56 +0000	[thread overview]
Message-ID: <20210617175820.24037-1-james.morse@arm.com> (raw)

Hi folks,

Changes since v4? Padding in the schemata file, typos and variable name
changes. The schemata file padding is to ensure that if CDP is supported
but not enabled, the file looks like this:
| root@resctrl-cdp-v5:~# cat /sys/fs/resctrl/schemata
|    MB:0=100;1=100
|    L3:0=7ff;1=7ff

Changes are noted in each patch.
----

This series re-folds the resctrl code so the CDP resources (L3CODE et al)
behaviour is all contained in the filesystem parts, with a minimum amount
of arch specific code.

Arm have some CPU support for dividing caches into portions, and
applying bandwidth limits at various points in the SoC. The collective term
for these features is MPAM: Memory Partitioning and Monitoring.

MPAM is similar enough to Intel RDT, that it should use the defacto linux
interface: resctrl. This filesystem currently lives under arch/x86, and is
tightly coupled to the architecture.
Ultimately, my plan is to split the existing resctrl code up to have an
arch<->fs abstraction, then move all the bits out to fs/resctrl. From there
MPAM can be wired up.

x86 might have two resources with cache controls, (L2 and L3) but has
extra copies for CDP: L{2,3}{CODE,DATA}, which are marked as enabled
if CDP is enabled for the corresponding cache.

MPAM has an equivalent feature to CDP, but its a property of the CPU,
not the cache. Resctrl needs to have x86's odd/even behaviour, as that
its the ABI, but this isn't how the MPAM hardware works. It is entirely
possible that an in-kernel user of MPAM would not be using CDP, whereas
resctrl is.

Pretending L3CODE and L3DATA are entirely separate resources is a neat
trick, but doing this is specific to x86.
Doing this leaves the arch code in control of various parts of the
filesystem ABI: the resources names, and the way the schemata are parsed.
Allowing this stuff to vary between architectures is bad for user space.

This series collapses the CODE/DATA resources, moving all the user-visible
resctrl ABI into what becomes the filesystem code. CDP becomes the type of
configuration being applied to a cache. This is done by adding a
struct resctrl_schema to the parts of resctrl that will move to fs. This
holds the arch-code resource that is in use for this schema, along with
other properties like the name, and whether the configuration being applied
is CODE/DATA/BOTH.

This lets us fold the extra resources out of the arch code so that they
don't need to be duplicated if the equivalent feature to CDP is missing, or
implemented in a different way.


The first two patches split the resource and domain structs to have an
arch specific 'hw' portion, and the rest that is visible to resctrl.
Future series massage the resctrl code so there are no accesses to 'hw'
structures in the parts of resctrl that will move to fs, providing helpers
where necessary.

This series adds temporary scaffolding, which it removes a few patches
later. This is to allow things like the ctrlval arrays and resources to be
merged separately, which should make is easier to bisect. These things
are marked temporary, and should all be gone by the end of the series.

This series is a little rough around the monitors, would a fake
struct resctrl_schema for the monitors simplify things, or be a source
of bugs?

A side effect of merging these resources, is their names are no longer printed
in the kernel log at boot. e.g:
| resctrl: L3 allocation detected
| resctrl: MB allocation detected
| resctrl: L3 monitoring detected
would previously have had extra entries for 'L3CODE' and 'L3DATA'.
User-space cannot rely on this to discover CDP support, as the kernel log may
be inaccessible, may have been overwritten by newer messages, and because
parsing the kernel log is a bad idea.

This series is based on tip/master's commit eb7f1579243, and can be retrieved from:
git://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/resctrl_merge_cdp/v5

v4: https://lore.kernel.org/lkml/20210614200941.12383-1-james.morse@arm.com/
v3: https://lore.kernel.org/lkml/20210519162424.27654-1-james.morse@arm.com/
v2: https://lore.kernel.org/lkml/20210312175849.8327-1-james.morse@arm.com/
v1: https://lore.kernel.org/lkml/20201030161120.227225-1-james.morse@arm.com/

Parts were previously posted as an RFC here:
https://lore.kernel.org/lkml/20200214182947.39194-1-james.morse@arm.com/

James Morse (24):
  x86/resctrl: Split struct rdt_resource
  x86/resctrl: Split struct rdt_domain
  x86/resctrl: Add a separate schema list for resctrl
  x86/resctrl: Pass the schema in info dir's private pointer
  x86/resctrl: Label the resources with their configuration type
  x86/resctrl: Walk the resctrl schema list instead of an arch list
  x86/resctrl: Store the effective num_closid in the schema
  x86/resctrl: Add resctrl_arch_get_num_closid()
  x86/resctrl: Pass the schema to resctrl filesystem functions
  x86/resctrl: Swizzle rdt_resource and resctrl_schema in
    pseudo_lock_region
  x86/resctrl: Add a helper to read/set the CDP configuration
  x86/resctrl: Move the schemata names into struct resctrl_schema
  x86/resctrl: Group staged configuration into a separate struct
  x86/resctrl: Allow different CODE/DATA configurations to be staged
  x86/resctrl: Rename update_domains() resctrl_arch_update_domains()
  x86/resctrl: Add a helper to read a closid's configuration
  x86/resctrl: Pass configuration type to resctrl_arch_get_config()
  x86/resctrl: Make ctrlval arrays the same size
  x86/resctrl: Apply offset correction when config is staged
  x86/resctrl: Calculate the index from the configuration type
  x86/resctrl: Merge the ctrl_val arrays
  x86/resctrl: Remove rdt_cdp_peer_get()
  x86/resctrl: Expand resctrl_arch_update_domains()'s msr_param range
  x86/resctrl: Merge the CDP resources

 arch/x86/kernel/cpu/resctrl/core.c        | 276 ++++++-------
 arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 164 +++++---
 arch/x86/kernel/cpu/resctrl/internal.h    | 232 ++++-------
 arch/x86/kernel/cpu/resctrl/monitor.c     |  44 ++-
 arch/x86/kernel/cpu/resctrl/pseudo_lock.c |  12 +-
 arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 457 ++++++++++++----------
 include/linux/resctrl.h                   | 185 +++++++++
 7 files changed, 776 insertions(+), 594 deletions(-)

-- 
2.30.2


             reply	other threads:[~2021-06-17 17:58 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-06-17 17:57 James Morse [this message]
2021-06-17 17:57 ` [PATCH v5 01/24] x86/resctrl: Split struct rdt_resource James Morse
2021-06-17 17:57 ` [PATCH v5 02/24] x86/resctrl: Split struct rdt_domain James Morse
2021-06-17 17:57 ` [PATCH v5 03/24] x86/resctrl: Add a separate schema list for resctrl James Morse
2021-06-17 17:58 ` [PATCH v5 04/24] x86/resctrl: Pass the schema in info dir's private pointer James Morse
2021-06-17 17:58 ` [PATCH v5 05/24] x86/resctrl: Label the resources with their configuration type James Morse
2021-06-17 17:58 ` [PATCH v5 06/24] x86/resctrl: Walk the resctrl schema list instead of an arch list James Morse
2021-06-17 17:58 ` [PATCH v5 07/24] x86/resctrl: Store the effective num_closid in the schema James Morse
2021-06-17 17:58 ` [PATCH v5 08/24] x86/resctrl: Add resctrl_arch_get_num_closid() James Morse
2021-06-17 17:58 ` [PATCH v5 09/24] x86/resctrl: Pass the schema to resctrl filesystem functions James Morse
2021-06-17 17:58 ` [PATCH v5 10/24] x86/resctrl: Swizzle rdt_resource and resctrl_schema in pseudo_lock_region James Morse
2021-06-17 17:58 ` [PATCH v5 11/24] x86/resctrl: Add a helper to read/set the CDP configuration James Morse
2021-06-17 17:58 ` [PATCH v5 12/24] x86/resctrl: Move the schemata names into struct resctrl_schema James Morse
2021-06-17 17:58 ` [PATCH v5 13/24] x86/resctrl: Group staged configuration into a separate struct James Morse
2021-06-17 17:58 ` [PATCH v5 14/24] x86/resctrl: Allow different CODE/DATA configurations to be staged James Morse
2021-06-17 17:58 ` [PATCH v5 15/24] x86/resctrl: Rename update_domains() resctrl_arch_update_domains() James Morse
2021-06-17 17:58 ` [PATCH v5 16/24] x86/resctrl: Add a helper to read a closid's configuration James Morse
2021-06-17 17:58 ` [PATCH v5 17/24] x86/resctrl: Pass configuration type to resctrl_arch_get_config() James Morse
2021-06-17 17:58 ` [PATCH v5 18/24] x86/resctrl: Make ctrlval arrays the same size James Morse
2021-06-17 17:58 ` [PATCH v5 19/24] x86/resctrl: Apply offset correction when config is staged James Morse
2021-06-17 17:58 ` [PATCH v5 20/24] x86/resctrl: Calculate the index from the configuration type James Morse
2021-06-17 17:58 ` [PATCH v5 21/24] x86/resctrl: Merge the ctrl_val arrays James Morse
2021-06-17 17:58 ` [PATCH v5 22/24] x86/resctrl: Remove rdt_cdp_peer_get() James Morse
2021-06-17 17:58 ` [PATCH v5 23/24] x86/resctrl: Expand resctrl_arch_update_domains()'s msr_param range James Morse
2021-06-17 17:58 ` [PATCH v5 24/24] x86/resctrl: Merge the CDP resources James Morse
2021-06-21 20:42 ` [PATCH v5 00/24] " Reinette Chatre
2021-06-22 15:47 ` Babu Moger

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210617175820.24037-1-james.morse@arm.com \
    --to=james.morse@arm.com \
    --cc=Babu.Moger@amd.com \
    --cc=bp@alien8.de \
    --cc=fenghua.yu@intel.com \
    --cc=hpa@zytor.com \
    --cc=jamie@nuviainc.com \
    --cc=lcherian@marvell.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=reinette.chatre@intel.com \
    --cc=scott@os.amperecomputing.com \
    --cc=shameerali.kolothum.thodi@huawei.com \
    --cc=tglx@linutronix.de \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).