* [PATCH v6 00/17] sched: Instrument sched domain flags
@ 2020-08-17 11:29 Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 01/17] ARM, sched/topology: Remove SD_SHARE_POWERDOMAIN Valentin Schneider
` (16 more replies)
0 siblings, 17 replies; 34+ messages in thread
From: Valentin Schneider @ 2020-08-17 11:29 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel
Cc: mingo, peterz, vincent.guittot, dietmar.eggemann,
morten.rasmussen, Quentin Perret
Hi,
I've repeatedly stared at an SD flag and asked myself "how should that be
set up in the domain hierarchy anyway?". I figured that if we formalize our
flags zoology a bit, we could also do some runtime assertions on them -
this is what this series is all about.
Patches
=======
The idea is to associate the flags with metaflags that describes how they
should be set in a sched domain hierarchy ("if this SD has it, all its {parents,
children} have it") or how they behave wrt degeneration - details are in the
comments and commit logs.
The good thing is that the debugging bits go away when CONFIG_SCHED_DEBUG isn't
set. The bad thing is that this replaces SD_* flags definitions with some
unsavoury macros. This is mainly because I wanted to avoid having to duplicate
work between declaring the flags and declaring their metaflags. Conceptually
they are pretty close to the macros used for SCHED_FEAT.
o Patches 1-2 remove a derelict flag and align the arm scheduler topology
with arm64's
o Patches 3-8 instrument SD flags with metadata and add assertions
o Patches 9-10 are additional topology cleanups
o Patches 11-16 each add a new flag to SD_DEGENERATE_GROUPS_MASK
o Patch 17 leverage the previous 6 patches to further factorize domain
degeneration
Revisions
=========
v5 -> v6
--------
o Fixed kernelbot warnings
o Tweaked ARM patches changelogs (Ingo)
v4 -> v5
--------
The final git diff between v4 and v5 isn't too big; there is no diff on the
domain degeneration side of things, it has just been split up in more
patches.
o Shuffled the series around to facilitate bisection (Ingo)
I kept the ARM bits at the start because that was a bit simpler, and in
the unlikely case that needs to be reverted it won't be too hard.
o Split the degeneration mask tweak into individual commits per new flag
(Ingo)
o Collected Reviewed-by from Dietmar; since I shuffled the whole lot I
didn't keep your Tested-by, sorry!
o Turned the SD flags into an enum with automagic power of 2 assignment.
I poked the dwarves and they assured me the SD flag values haven't
changed.
o [new patch] Made SD flag debug file output flag names
v3 -> v4
--------
o Reordered the series to have fixes / cleanups first
o Added SD_ASYM_CPUCAPACITY propagation (Quentin)
o Made ARM revert back to the default sched topology (Dietmar)
o Removed SD_SERIALIZE degeneration special case (Peter)
o Made SD_NUMA and SD_SERIALIZE have SDF_NEEDS_GROUPS
As discussed on v3, I thought this wasn't required, but thinking some more
about it there can be cases where that changes the current behaviour. For
instance, in the following wacky triangle:
0\ 30
| \
20 | 2
| /
1/ 30
there are two unique distances thus two NUMA topology levels, however the
first one for node 2 would have the same span as its child domain and thus
should be degenerated. If we don't give SD_NUMA and SD_SERIALIZE
SDF_NEEDS_GROUPS, this domain wouldn't be denegerated since its child
*doesn't* have either SD_NUMA or SD_SERIALIZE (it's the first NUMA domain),
and we'd have this weird NUMA domain lingering with a single group.
v2 -> v3
--------
o Reworded comment for SD_OVERLAP (it's about the groups, not the domains)
o Added more flags to the SD degeneration mask
o Added generation of an SD flag mask for the degeneration functions (Peter)
RFC -> v2
---------
o Rebased on top of tip/sched/core
o Aligned wording of comments between flags
o Rectified some flag descriptions (Morten)
o Added removal of SD_SHARE_POWERDOMAIN (Morten)
Valentin Schneider (17):
ARM, sched/topology: Remove SD_SHARE_POWERDOMAIN
ARM: Revert back to default scheduler topology.
sched/topology: Split out SD_* flags declaration to its own file
sched/topology: Define and assign sched_domain flag metadata
sched/topology: Verify SD_* flags setup when sched_debug is on
sched/debug: Output SD flag names rather than their values
sched/topology: Introduce SD metaflag for flags needing > 1 groups
sched/topology: Use prebuilt SD flag degeneration mask
sched/topology: Remove SD_SERIALIZE degeneration special case
sched/topology: Propagate SD_ASYM_CPUCAPACITY upwards
sched/topology: Mark SD_PREFER_SIBLING as SDF_NEEDS_GROUPS
sched/topology: Mark SD_BALANCE_WAKE as SDF_NEEDS_GROUPS
sched/topology: Mark SD_SERIALIZE as SDF_NEEDS_GROUPS
sched/topology: Mark SD_ASYM_PACKING as SDF_NEEDS_GROUPS
sched/topology: Mark SD_OVERLAP as SDF_NEEDS_GROUPS
sched/topology: Mark SD_NUMA as SDF_NEEDS_GROUPS
sched/topology: Expand use of SD_DEGENERATE_GROUPS_MASK to flags not
needing groups
arch/arm/kernel/topology.c | 26 ------
include/linux/sched/sd_flags.h | 156 +++++++++++++++++++++++++++++++++
include/linux/sched/topology.h | 45 +++++++---
kernel/sched/debug.c | 56 +++++++++++-
kernel/sched/topology.c | 54 ++++++------
5 files changed, 268 insertions(+), 69 deletions(-)
create mode 100644 include/linux/sched/sd_flags.h
--
2.27.0
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH v6 01/17] ARM, sched/topology: Remove SD_SHARE_POWERDOMAIN
2020-08-17 11:29 [PATCH v6 00/17] sched: Instrument sched domain flags Valentin Schneider
@ 2020-08-17 11:29 ` Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 02/17] ARM: Revert back to default scheduler topology Valentin Schneider
` (15 subsequent siblings)
16 siblings, 1 reply; 34+ messages in thread
From: Valentin Schneider @ 2020-08-17 11:29 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel
Cc: Russell King, Morten Rasmussen, Dietmar Eggemann, mingo, peterz,
vincent.guittot, Quentin Perret
This flag was introduced in 2014 by commit
d77b3ed5c9f8 ("sched: Add a new SD_SHARE_POWERDOMAIN for sched_domain")
but AFAIA it was never leveraged by the scheduler. The closest thing I can
think of is EAS caring about frequency domains, and it does that by
leveraging performance domains.
Remove the flag. No change in functionality is expected.
Cc: Russell King <linux@armlinux.org.uk>
Suggested-by: Morten Rasmussen <morten.rasmussen@arm.com>
Reviewed-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
arch/arm/kernel/topology.c | 2 +-
include/linux/sched/topology.h | 13 ++++++-------
kernel/sched/topology.c | 10 +++-------
3 files changed, 10 insertions(+), 15 deletions(-)
diff --git a/arch/arm/kernel/topology.c b/arch/arm/kernel/topology.c
index b5adaf744630..353f3ee660e4 100644
--- a/arch/arm/kernel/topology.c
+++ b/arch/arm/kernel/topology.c
@@ -243,7 +243,7 @@ void store_cpu_topology(unsigned int cpuid)
static inline int cpu_corepower_flags(void)
{
- return SD_SHARE_PKG_RESOURCES | SD_SHARE_POWERDOMAIN;
+ return SD_SHARE_PKG_RESOURCES;
}
static struct sched_domain_topology_level arm_topology[] = {
diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h
index 820511289857..6ec7d7c1d1e3 100644
--- a/include/linux/sched/topology.h
+++ b/include/linux/sched/topology.h
@@ -18,13 +18,12 @@
#define SD_WAKE_AFFINE 0x0010 /* Wake task to waking CPU */
#define SD_ASYM_CPUCAPACITY 0x0020 /* Domain members have different CPU capacities */
#define SD_SHARE_CPUCAPACITY 0x0040 /* Domain members share CPU capacity */
-#define SD_SHARE_POWERDOMAIN 0x0080 /* Domain members share power domain */
-#define SD_SHARE_PKG_RESOURCES 0x0100 /* Domain members share CPU pkg resources */
-#define SD_SERIALIZE 0x0200 /* Only a single load balancing instance */
-#define SD_ASYM_PACKING 0x0400 /* Place busy groups earlier in the domain */
-#define SD_PREFER_SIBLING 0x0800 /* Prefer to place tasks in a sibling domain */
-#define SD_OVERLAP 0x1000 /* sched_domains of this level overlap */
-#define SD_NUMA 0x2000 /* cross-node balancing */
+#define SD_SHARE_PKG_RESOURCES 0x0080 /* Domain members share CPU pkg resources */
+#define SD_SERIALIZE 0x0100 /* Only a single load balancing instance */
+#define SD_ASYM_PACKING 0x0200 /* Place busy groups earlier in the domain */
+#define SD_PREFER_SIBLING 0x0400 /* Prefer to place tasks in a sibling domain */
+#define SD_OVERLAP 0x0800 /* sched_domains of this level overlap */
+#define SD_NUMA 0x1000 /* cross-node balancing */
#ifdef CONFIG_SCHED_SMT
static inline int cpu_smt_flags(void)
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 9079d865a935..865fff3ef20a 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -148,8 +148,7 @@ static int sd_degenerate(struct sched_domain *sd)
SD_BALANCE_EXEC |
SD_SHARE_CPUCAPACITY |
SD_ASYM_CPUCAPACITY |
- SD_SHARE_PKG_RESOURCES |
- SD_SHARE_POWERDOMAIN)) {
+ SD_SHARE_PKG_RESOURCES)) {
if (sd->groups != sd->groups->next)
return 0;
}
@@ -180,8 +179,7 @@ sd_parent_degenerate(struct sched_domain *sd, struct sched_domain *parent)
SD_ASYM_CPUCAPACITY |
SD_SHARE_CPUCAPACITY |
SD_SHARE_PKG_RESOURCES |
- SD_PREFER_SIBLING |
- SD_SHARE_POWERDOMAIN);
+ SD_PREFER_SIBLING);
if (nr_node_ids == 1)
pflags &= ~SD_SERIALIZE;
}
@@ -1292,7 +1290,6 @@ int __read_mostly node_reclaim_distance = RECLAIM_DISTANCE;
* SD_SHARE_CPUCAPACITY - describes SMT topologies
* SD_SHARE_PKG_RESOURCES - describes shared caches
* SD_NUMA - describes NUMA topologies
- * SD_SHARE_POWERDOMAIN - describes shared power domain
*
* Odd one out, which beside describing the topology has a quirk also
* prescribes the desired behaviour that goes along with it:
@@ -1303,8 +1300,7 @@ int __read_mostly node_reclaim_distance = RECLAIM_DISTANCE;
(SD_SHARE_CPUCAPACITY | \
SD_SHARE_PKG_RESOURCES | \
SD_NUMA | \
- SD_ASYM_PACKING | \
- SD_SHARE_POWERDOMAIN)
+ SD_ASYM_PACKING)
static struct sched_domain *
sd_init(struct sched_domain_topology_level *tl,
--
2.27.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 02/17] ARM: Revert back to default scheduler topology.
2020-08-17 11:29 [PATCH v6 00/17] sched: Instrument sched domain flags Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 01/17] ARM, sched/topology: Remove SD_SHARE_POWERDOMAIN Valentin Schneider
@ 2020-08-17 11:29 ` Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] ARM, sched/topology: " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 03/17] sched/topology: Split out SD_* flags declaration to its own file Valentin Schneider
` (14 subsequent siblings)
16 siblings, 1 reply; 34+ messages in thread
From: Valentin Schneider @ 2020-08-17 11:29 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel
Cc: Russell King, Dietmar Eggemann, mingo, peterz, vincent.guittot,
morten.rasmussen, Quentin Perret
The ARM-specific GMC level is meant to be built using the thread sibling
mask, but no devicetree in arch/arm/boot/dts uses the 'thread' cpu-map
binding. With SD_SHARE_POWERDOMAIN gone, this topology level can be
removed, at which point ARM no longer benefits from having a custom defined
topology table.
Delete the GMC topology level by making ARM use the default scheduler
topology table. This essentially reverts commit
fb2aa85564f4 ("sched, ARM: Create a dedicated scheduler topology table")
No change in functionality is expected.
Cc: Russell King <linux@armlinux.org.uk>
Suggested-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Reviewed-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
arch/arm/kernel/topology.c | 26 --------------------------
1 file changed, 26 deletions(-)
diff --git a/arch/arm/kernel/topology.c b/arch/arm/kernel/topology.c
index 353f3ee660e4..ef0058de432b 100644
--- a/arch/arm/kernel/topology.c
+++ b/arch/arm/kernel/topology.c
@@ -177,15 +177,6 @@ static inline void parse_dt_topology(void) {}
static inline void update_cpu_capacity(unsigned int cpuid) {}
#endif
-/*
- * The current assumption is that we can power gate each core independently.
- * This will be superseded by DT binding once available.
- */
-const struct cpumask *cpu_corepower_mask(int cpu)
-{
- return &cpu_topology[cpu].thread_sibling;
-}
-
/*
* store_cpu_topology is called at boot when only one cpu is running
* and with the mutex cpu_hotplug.lock locked, when several cpus have booted,
@@ -241,20 +232,6 @@ void store_cpu_topology(unsigned int cpuid)
update_siblings_masks(cpuid);
}
-static inline int cpu_corepower_flags(void)
-{
- return SD_SHARE_PKG_RESOURCES;
-}
-
-static struct sched_domain_topology_level arm_topology[] = {
-#ifdef CONFIG_SCHED_MC
- { cpu_corepower_mask, cpu_corepower_flags, SD_INIT_NAME(GMC) },
- { cpu_coregroup_mask, cpu_core_flags, SD_INIT_NAME(MC) },
-#endif
- { cpu_cpu_mask, SD_INIT_NAME(DIE) },
- { NULL, },
-};
-
/*
* init_cpu_topology is called at boot when only one cpu is running
* which prevent simultaneous write access to cpu_topology array
@@ -265,7 +242,4 @@ void __init init_cpu_topology(void)
smp_wmb();
parse_dt_topology();
-
- /* Set scheduler topology descriptor */
- set_sched_topology(arm_topology);
}
--
2.27.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 03/17] sched/topology: Split out SD_* flags declaration to its own file
2020-08-17 11:29 [PATCH v6 00/17] sched: Instrument sched domain flags Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 01/17] ARM, sched/topology: Remove SD_SHARE_POWERDOMAIN Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 02/17] ARM: Revert back to default scheduler topology Valentin Schneider
@ 2020-08-17 11:29 ` Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 04/17] sched/topology: Define and assign sched_domain flag metadata Valentin Schneider
` (13 subsequent siblings)
16 siblings, 1 reply; 34+ messages in thread
From: Valentin Schneider @ 2020-08-17 11:29 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel
Cc: Dietmar Eggemann, mingo, peterz, vincent.guittot,
morten.rasmussen, Quentin Perret
To associate the SD flags with some metadata, we need some more structure
in the way they are declared.
Rather than shove that in a free-standing macro list, move the declaration
in a separate file that can be re-imported with different SD_FLAG
definitions. This is inspired by what is done with the syscall
table (see uapi/asm/unistd.h and sys_call_table).
The value assigned to a given SD flag now depends on the order it appears
in sd_flags.h. No change in functionality.
Reviewed-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
include/linux/sched/sd_flags.h | 35 ++++++++++++++++++++++++++++++++++
include/linux/sched/topology.h | 26 ++++++++++++-------------
2 files changed, 48 insertions(+), 13 deletions(-)
create mode 100644 include/linux/sched/sd_flags.h
diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h
new file mode 100644
index 000000000000..5a74751a1a83
--- /dev/null
+++ b/include/linux/sched/sd_flags.h
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * sched-domains (multiprocessor balancing) flag declarations.
+ */
+
+#ifndef SD_FLAG
+#error "Incorrect import of SD flags definitions"
+#endif
+
+/* Balance when about to become idle */
+SD_FLAG(SD_BALANCE_NEWIDLE)
+/* Balance on exec */
+SD_FLAG(SD_BALANCE_EXEC)
+/* Balance on fork, clone */
+SD_FLAG(SD_BALANCE_FORK)
+/* Balance on wakeup */
+SD_FLAG(SD_BALANCE_WAKE)
+/* Wake task to waking CPU */
+SD_FLAG(SD_WAKE_AFFINE)
+/* Domain members have different CPU capacities */
+SD_FLAG(SD_ASYM_CPUCAPACITY)
+/* Domain members share CPU capacity */
+SD_FLAG(SD_SHARE_CPUCAPACITY)
+/* Domain members share CPU pkg resources */
+SD_FLAG(SD_SHARE_PKG_RESOURCES)
+/* Only a single load balancing instance */
+SD_FLAG(SD_SERIALIZE)
+/* Place busy groups earlier in the domain */
+SD_FLAG(SD_ASYM_PACKING)
+/* Prefer to place tasks in a sibling domain */
+SD_FLAG(SD_PREFER_SIBLING)
+/* sched_domains of this level overlap */
+SD_FLAG(SD_OVERLAP)
+/* cross-node balancing */
+SD_FLAG(SD_NUMA)
diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h
index 6ec7d7c1d1e3..3e41c0401b5f 100644
--- a/include/linux/sched/topology.h
+++ b/include/linux/sched/topology.h
@@ -11,19 +11,19 @@
*/
#ifdef CONFIG_SMP
-#define SD_BALANCE_NEWIDLE 0x0001 /* Balance when about to become idle */
-#define SD_BALANCE_EXEC 0x0002 /* Balance on exec */
-#define SD_BALANCE_FORK 0x0004 /* Balance on fork, clone */
-#define SD_BALANCE_WAKE 0x0008 /* Balance on wakeup */
-#define SD_WAKE_AFFINE 0x0010 /* Wake task to waking CPU */
-#define SD_ASYM_CPUCAPACITY 0x0020 /* Domain members have different CPU capacities */
-#define SD_SHARE_CPUCAPACITY 0x0040 /* Domain members share CPU capacity */
-#define SD_SHARE_PKG_RESOURCES 0x0080 /* Domain members share CPU pkg resources */
-#define SD_SERIALIZE 0x0100 /* Only a single load balancing instance */
-#define SD_ASYM_PACKING 0x0200 /* Place busy groups earlier in the domain */
-#define SD_PREFER_SIBLING 0x0400 /* Prefer to place tasks in a sibling domain */
-#define SD_OVERLAP 0x0800 /* sched_domains of this level overlap */
-#define SD_NUMA 0x1000 /* cross-node balancing */
+/* Generate SD flag indexes */
+#define SD_FLAG(name) __##name,
+enum {
+ #include <linux/sched/sd_flags.h>
+ __SD_FLAG_CNT,
+};
+#undef SD_FLAG
+/* Generate SD flag bits */
+#define SD_FLAG(name) name = 1 << __##name,
+enum {
+ #include <linux/sched/sd_flags.h>
+};
+#undef SD_FLAG
#ifdef CONFIG_SCHED_SMT
static inline int cpu_smt_flags(void)
--
2.27.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 04/17] sched/topology: Define and assign sched_domain flag metadata
2020-08-17 11:29 [PATCH v6 00/17] sched: Instrument sched domain flags Valentin Schneider
` (2 preceding siblings ...)
2020-08-17 11:29 ` [PATCH v6 03/17] sched/topology: Split out SD_* flags declaration to its own file Valentin Schneider
@ 2020-08-17 11:29 ` Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 05/17] sched/topology: Verify SD_* flags setup when sched_debug is on Valentin Schneider
` (12 subsequent siblings)
16 siblings, 1 reply; 34+ messages in thread
From: Valentin Schneider @ 2020-08-17 11:29 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel
Cc: Dietmar Eggemann, mingo, peterz, vincent.guittot,
morten.rasmussen, Quentin Perret
There are some expectations regarding how sched domain flags should be laid
out, but none of them are checked or asserted in
sched_domain_debug_one(). After staring at said flags for a while, I've
come to realize there's two repeating patterns:
- Shared with children: those flags are set from the base CPU domain
upwards. Any domain that has it set will have it set in its children. It
hints at "some property holds true / some behaviour is enabled until this
level".
- Shared with parents: those flags are set from the topmost domain
downwards. Any domain that has it set will have it set in its parents. It
hints at "some property isn't visible / some behaviour is disabled until
this level".
There are two outliers that (currently) do not map to either of these:
o SD_PREFER_SIBLING, which is cleared below levels with
SD_ASYM_CPUCAPACITY. The change was introduced by commit
9c63e84db29b ("sched/core: Disable SD_PREFER_SIBLING on asymmetric CPU capacity domains")
as it could break misfit migration on some systems. In light of this, we
might want to change it back to make it fit one of the two categories and
fix the issue another way.
o SD_ASYM_CPUCAPACITY, which gets set on a single level and isn't
propagated up nor down. From a topology description point of view, it
really wants to be SDF_SHARED_PARENT; this will be rectified in a later
patch.
Tweak the sched_domain flag declaration to assign each flag an expected
layout, and include the rationale for each flag "meta type" assignment as a
comment. Consolidate the flag metadata into an array; the index of a flag's
metadata can easily be found with log2(flag), IOW __ffs(flag).
Reviewed-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
include/linux/sched/sd_flags.h | 147 +++++++++++++++++++++++++++------
include/linux/sched/topology.h | 15 +++-
2 files changed, 134 insertions(+), 28 deletions(-)
diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h
index 5a74751a1a83..ea0ec1a33da4 100644
--- a/include/linux/sched/sd_flags.h
+++ b/include/linux/sched/sd_flags.h
@@ -7,29 +7,124 @@
#error "Incorrect import of SD flags definitions"
#endif
-/* Balance when about to become idle */
-SD_FLAG(SD_BALANCE_NEWIDLE)
-/* Balance on exec */
-SD_FLAG(SD_BALANCE_EXEC)
-/* Balance on fork, clone */
-SD_FLAG(SD_BALANCE_FORK)
-/* Balance on wakeup */
-SD_FLAG(SD_BALANCE_WAKE)
-/* Wake task to waking CPU */
-SD_FLAG(SD_WAKE_AFFINE)
-/* Domain members have different CPU capacities */
-SD_FLAG(SD_ASYM_CPUCAPACITY)
-/* Domain members share CPU capacity */
-SD_FLAG(SD_SHARE_CPUCAPACITY)
-/* Domain members share CPU pkg resources */
-SD_FLAG(SD_SHARE_PKG_RESOURCES)
-/* Only a single load balancing instance */
-SD_FLAG(SD_SERIALIZE)
-/* Place busy groups earlier in the domain */
-SD_FLAG(SD_ASYM_PACKING)
-/* Prefer to place tasks in a sibling domain */
-SD_FLAG(SD_PREFER_SIBLING)
-/* sched_domains of this level overlap */
-SD_FLAG(SD_OVERLAP)
-/* cross-node balancing */
-SD_FLAG(SD_NUMA)
+/*
+ * Expected flag uses
+ *
+ * SHARED_CHILD: These flags are meant to be set from the base domain upwards.
+ * If a domain has this flag set, all of its children should have it set. This
+ * is usually because the flag describes some shared resource (all CPUs in that
+ * domain share the same resource), or because they are tied to a scheduling
+ * behaviour that we want to disable at some point in the hierarchy for
+ * scalability reasons.
+ *
+ * In those cases it doesn't make sense to have the flag set for a domain but
+ * not have it in (some of) its children: sched domains ALWAYS span their child
+ * domains, so operations done with parent domains will cover CPUs in the lower
+ * child domains.
+ *
+ *
+ * SHARED_PARENT: These flags are meant to be set from the highest domain
+ * downwards. If a domain has this flag set, all of its parents should have it
+ * set. This is usually for topology properties that start to appear above a
+ * certain level (e.g. domain starts spanning CPUs outside of the base CPU's
+ * socket).
+ */
+#define SDF_SHARED_CHILD 0x1
+#define SDF_SHARED_PARENT 0x2
+
+/*
+ * Balance when about to become idle
+ *
+ * SHARED_CHILD: Set from the base domain up to cpuset.sched_relax_domain_level.
+ */
+SD_FLAG(SD_BALANCE_NEWIDLE, SDF_SHARED_CHILD)
+
+/*
+ * Balance on exec
+ *
+ * SHARED_CHILD: Set from the base domain up to the NUMA reclaim level.
+ */
+SD_FLAG(SD_BALANCE_EXEC, SDF_SHARED_CHILD)
+
+/*
+ * Balance on fork, clone
+ *
+ * SHARED_CHILD: Set from the base domain up to the NUMA reclaim level.
+ */
+SD_FLAG(SD_BALANCE_FORK, SDF_SHARED_CHILD)
+
+/*
+ * Balance on wakeup
+ *
+ * SHARED_CHILD: Set from the base domain up to cpuset.sched_relax_domain_level.
+ */
+SD_FLAG(SD_BALANCE_WAKE, SDF_SHARED_CHILD)
+
+/*
+ * Consider waking task on waking CPU.
+ *
+ * SHARED_CHILD: Set from the base domain up to the NUMA reclaim level.
+ */
+SD_FLAG(SD_WAKE_AFFINE, SDF_SHARED_CHILD)
+
+/*
+ * Domain members have different CPU capacities
+ */
+SD_FLAG(SD_ASYM_CPUCAPACITY, 0)
+
+/*
+ * Domain members share CPU capacity (i.e. SMT)
+ *
+ * SHARED_CHILD: Set from the base domain up until spanned CPUs no longer share
+ * CPU capacity.
+ */
+SD_FLAG(SD_SHARE_CPUCAPACITY, SDF_SHARED_CHILD)
+
+/*
+ * Domain members share CPU package resources (i.e. caches)
+ *
+ * SHARED_CHILD: Set from the base domain up until spanned CPUs no longer share
+ * the same cache(s).
+ */
+SD_FLAG(SD_SHARE_PKG_RESOURCES, SDF_SHARED_CHILD)
+
+/*
+ * Only a single load balancing instance
+ *
+ * SHARED_PARENT: Set for all NUMA levels above NODE. Could be set from a
+ * different level upwards, but it doesn't change that if a domain has this flag
+ * set, then all of its parents need to have it too (otherwise the serialization
+ * doesn't make sense).
+ */
+SD_FLAG(SD_SERIALIZE, SDF_SHARED_PARENT)
+
+/*
+ * Place busy tasks earlier in the domain
+ *
+ * SHARED_CHILD: Usually set on the SMT level. Technically could be set further
+ * up, but currently assumed to be set from the base domain upwards (see
+ * update_top_cache_domain()).
+ */
+SD_FLAG(SD_ASYM_PACKING, SDF_SHARED_CHILD)
+
+/*
+ * Prefer to place tasks in a sibling domain
+ *
+ * Set up until domains start spanning NUMA nodes. Close to being a SHARED_CHILD
+ * flag, but cleared below domains with SD_ASYM_CPUCAPACITY.
+ */
+SD_FLAG(SD_PREFER_SIBLING, 0)
+
+/*
+ * sched_groups of this level overlap
+ *
+ * SHARED_PARENT: Set for all NUMA levels above NODE.
+ */
+SD_FLAG(SD_OVERLAP, SDF_SHARED_PARENT)
+
+/*
+ * Cross-node balancing
+ *
+ * SHARED_PARENT: Set for all NUMA levels above NODE.
+ */
+SD_FLAG(SD_NUMA, SDF_SHARED_PARENT)
diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h
index 3e41c0401b5f..32f602ff37a0 100644
--- a/include/linux/sched/topology.h
+++ b/include/linux/sched/topology.h
@@ -12,19 +12,30 @@
#ifdef CONFIG_SMP
/* Generate SD flag indexes */
-#define SD_FLAG(name) __##name,
+#define SD_FLAG(name, mflags) __##name,
enum {
#include <linux/sched/sd_flags.h>
__SD_FLAG_CNT,
};
#undef SD_FLAG
/* Generate SD flag bits */
-#define SD_FLAG(name) name = 1 << __##name,
+#define SD_FLAG(name, mflags) name = 1 << __##name,
enum {
#include <linux/sched/sd_flags.h>
};
#undef SD_FLAG
+#ifdef CONFIG_SCHED_DEBUG
+#define SD_FLAG(_name, mflags) [__##_name] = { .meta_flags = mflags, .name = #_name },
+static const struct {
+ unsigned int meta_flags;
+ char *name;
+} sd_flag_debug[] = {
+#include <linux/sched/sd_flags.h>
+};
+#undef SD_FLAG
+#endif
+
#ifdef CONFIG_SCHED_SMT
static inline int cpu_smt_flags(void)
{
--
2.27.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 05/17] sched/topology: Verify SD_* flags setup when sched_debug is on
2020-08-17 11:29 [PATCH v6 00/17] sched: Instrument sched domain flags Valentin Schneider
` (3 preceding siblings ...)
2020-08-17 11:29 ` [PATCH v6 04/17] sched/topology: Define and assign sched_domain flag metadata Valentin Schneider
@ 2020-08-17 11:29 ` Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 06/17] sched/debug: Output SD flag names rather than their values Valentin Schneider
` (11 subsequent siblings)
16 siblings, 1 reply; 34+ messages in thread
From: Valentin Schneider @ 2020-08-17 11:29 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel
Cc: Dietmar Eggemann, mingo, peterz, vincent.guittot,
morten.rasmussen, Quentin Perret
Now that we have some description of what we expect the flags layout to
be, we can use that to assert at runtime that the actual layout is sane.
Reviewed-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
kernel/sched/topology.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 865fff3ef20a..f128fcf46a41 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -29,6 +29,8 @@ static int sched_domain_debug_one(struct sched_domain *sd, int cpu, int level,
struct cpumask *groupmask)
{
struct sched_group *group = sd->groups;
+ unsigned long flags = sd->flags;
+ unsigned int idx;
cpumask_clear(groupmask);
@@ -43,6 +45,21 @@ static int sched_domain_debug_one(struct sched_domain *sd, int cpu, int level,
printk(KERN_ERR "ERROR: domain->groups does not contain CPU%d\n", cpu);
}
+ for_each_set_bit(idx, &flags, __SD_FLAG_CNT) {
+ unsigned int flag = BIT(idx);
+ unsigned int meta_flags = sd_flag_debug[idx].meta_flags;
+
+ if ((meta_flags & SDF_SHARED_CHILD) && sd->child &&
+ !(sd->child->flags & flag))
+ printk(KERN_ERR "ERROR: flag %s set here but not in child\n",
+ sd_flag_debug[idx].name);
+
+ if ((meta_flags & SDF_SHARED_PARENT) && sd->parent &&
+ !(sd->parent->flags & flag))
+ printk(KERN_ERR "ERROR: flag %s set here but not in parent\n",
+ sd_flag_debug[idx].name);
+ }
+
printk(KERN_DEBUG "%*s groups:", level + 1, "");
do {
if (!group) {
--
2.27.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 06/17] sched/debug: Output SD flag names rather than their values
2020-08-17 11:29 [PATCH v6 00/17] sched: Instrument sched domain flags Valentin Schneider
` (4 preceding siblings ...)
2020-08-17 11:29 ` [PATCH v6 05/17] sched/topology: Verify SD_* flags setup when sched_debug is on Valentin Schneider
@ 2020-08-17 11:29 ` Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 07/17] sched/topology: Introduce SD metaflag for flags needing > 1 groups Valentin Schneider
` (10 subsequent siblings)
16 siblings, 1 reply; 34+ messages in thread
From: Valentin Schneider @ 2020-08-17 11:29 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel
Cc: mingo, peterz, vincent.guittot, dietmar.eggemann,
morten.rasmussen, Quentin Perret
Decoding the output of /proc/sys/kernel/sched_domain/cpu*/domain*/flags has
always been somewhat annoying, as one needs to go fetch the bit -> name
mapping from the source code itself. This encoding can be saved in a script
somewhere, but that isn't safe from flags being added, removed or even
shuffled around.
What matters for debugging purposes is to get *which* flags are set in a
given domain, their associated value is pretty much meaningless.
Make the sd flags debug file output flag names.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
kernel/sched/debug.c | 56 +++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 55 insertions(+), 1 deletion(-)
diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c
index 36c54265bb2b..0655524700d2 100644
--- a/kernel/sched/debug.c
+++ b/kernel/sched/debug.c
@@ -245,6 +245,60 @@ set_table_entry(struct ctl_table *entry,
entry->proc_handler = proc_handler;
}
+static int sd_ctl_doflags(struct ctl_table *table, int write,
+ void *buffer, size_t *lenp, loff_t *ppos)
+{
+ unsigned long flags = *(unsigned long *)table->data;
+ size_t data_size = 0;
+ size_t len = 0;
+ char *tmp;
+ int idx;
+
+ if (write)
+ return 0;
+
+ for_each_set_bit(idx, &flags, __SD_FLAG_CNT) {
+ char *name = sd_flag_debug[idx].name;
+
+ /* Name plus whitespace */
+ data_size += strlen(name) + 1;
+ }
+
+ if (*ppos > data_size) {
+ *lenp = 0;
+ return 0;
+ }
+
+ tmp = kcalloc(data_size + 1, sizeof(*tmp), GFP_KERNEL);
+ if (!tmp)
+ return -ENOMEM;
+
+ for_each_set_bit(idx, &flags, __SD_FLAG_CNT) {
+ char *name = sd_flag_debug[idx].name;
+
+ len += snprintf(tmp + len, strlen(name) + 2, "%s ", name);
+ }
+
+ tmp += *ppos;
+ len -= *ppos;
+
+ if (len > *lenp)
+ len = *lenp;
+ if (len)
+ memcpy(buffer, tmp, len);
+ if (len < *lenp) {
+ ((char *)buffer)[len] = '\n';
+ len++;
+ }
+
+ *lenp = len;
+ *ppos += len;
+
+ kfree(tmp);
+
+ return 0;
+}
+
static struct ctl_table *
sd_alloc_ctl_domain_table(struct sched_domain *sd)
{
@@ -258,7 +312,7 @@ sd_alloc_ctl_domain_table(struct sched_domain *sd)
set_table_entry(&table[2], "busy_factor", &sd->busy_factor, sizeof(int), 0644, proc_dointvec_minmax);
set_table_entry(&table[3], "imbalance_pct", &sd->imbalance_pct, sizeof(int), 0644, proc_dointvec_minmax);
set_table_entry(&table[4], "cache_nice_tries", &sd->cache_nice_tries, sizeof(int), 0644, proc_dointvec_minmax);
- set_table_entry(&table[5], "flags", &sd->flags, sizeof(int), 0444, proc_dointvec_minmax);
+ set_table_entry(&table[5], "flags", &sd->flags, sizeof(int), 0444, sd_ctl_doflags);
set_table_entry(&table[6], "max_newidle_lb_cost", &sd->max_newidle_lb_cost, sizeof(long), 0644, proc_doulongvec_minmax);
set_table_entry(&table[7], "name", sd->name, CORENAME_MAX_SIZE, 0444, proc_dostring);
/* &table[8] is terminator */
--
2.27.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 07/17] sched/topology: Introduce SD metaflag for flags needing > 1 groups
2020-08-17 11:29 [PATCH v6 00/17] sched: Instrument sched domain flags Valentin Schneider
` (5 preceding siblings ...)
2020-08-17 11:29 ` [PATCH v6 06/17] sched/debug: Output SD flag names rather than their values Valentin Schneider
@ 2020-08-17 11:29 ` Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 08/17] sched/topology: Use prebuilt SD flag degeneration mask Valentin Schneider
` (9 subsequent siblings)
16 siblings, 1 reply; 34+ messages in thread
From: Valentin Schneider @ 2020-08-17 11:29 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel
Cc: Peter Zijlstra, Dietmar Eggemann, mingo, vincent.guittot,
morten.rasmussen, Quentin Perret
In preparation of cleaning up the sd_degenerate*() functions, mark flags
used in sd_degenerate() with the new SDF_NEEDS_GROUPS flag. With this,
build a compile-time mask of those SD flags.
Note that sd_parent_degenerate() uses an extra flag in its mask,
SD_PREFER_SIBLING, which remains singled out for now.
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Reviewed-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
include/linux/sched/sd_flags.h | 39 ++++++++++++++++++++++++----------
include/linux/sched/topology.h | 7 ++++++
2 files changed, 35 insertions(+), 11 deletions(-)
diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h
index ea0ec1a33da4..21a43ad6f26a 100644
--- a/include/linux/sched/sd_flags.h
+++ b/include/linux/sched/sd_flags.h
@@ -8,7 +8,7 @@
#endif
/*
- * Expected flag uses
+ * Hierarchical metaflags
*
* SHARED_CHILD: These flags are meant to be set from the base domain upwards.
* If a domain has this flag set, all of its children should have it set. This
@@ -29,29 +29,42 @@
* certain level (e.g. domain starts spanning CPUs outside of the base CPU's
* socket).
*/
-#define SDF_SHARED_CHILD 0x1
-#define SDF_SHARED_PARENT 0x2
+#define SDF_SHARED_CHILD 0x1
+#define SDF_SHARED_PARENT 0x2
+
+/*
+ * Behavioural metaflags
+ *
+ * NEEDS_GROUPS: These flags are only relevant if the domain they are set on has
+ * more than one group. This is usually for balancing flags (load balancing
+ * involves equalizing a metric between groups), or for flags describing some
+ * shared resource (which would be shared between groups).
+ */
+#define SDF_NEEDS_GROUPS 0x4
/*
* Balance when about to become idle
*
* SHARED_CHILD: Set from the base domain up to cpuset.sched_relax_domain_level.
+ * NEEDS_GROUPS: Load balancing flag.
*/
-SD_FLAG(SD_BALANCE_NEWIDLE, SDF_SHARED_CHILD)
+SD_FLAG(SD_BALANCE_NEWIDLE, SDF_SHARED_CHILD | SDF_NEEDS_GROUPS)
/*
* Balance on exec
*
* SHARED_CHILD: Set from the base domain up to the NUMA reclaim level.
+ * NEEDS_GROUPS: Load balancing flag.
*/
-SD_FLAG(SD_BALANCE_EXEC, SDF_SHARED_CHILD)
+SD_FLAG(SD_BALANCE_EXEC, SDF_SHARED_CHILD | SDF_NEEDS_GROUPS)
/*
* Balance on fork, clone
*
* SHARED_CHILD: Set from the base domain up to the NUMA reclaim level.
+ * NEEDS_GROUPS: Load balancing flag.
*/
-SD_FLAG(SD_BALANCE_FORK, SDF_SHARED_CHILD)
+SD_FLAG(SD_BALANCE_FORK, SDF_SHARED_CHILD | SDF_NEEDS_GROUPS)
/*
* Balance on wakeup
@@ -69,24 +82,28 @@ SD_FLAG(SD_WAKE_AFFINE, SDF_SHARED_CHILD)
/*
* Domain members have different CPU capacities
+ *
+ * NEEDS_GROUPS: Per-CPU capacity is asymmetric between groups.
*/
-SD_FLAG(SD_ASYM_CPUCAPACITY, 0)
+SD_FLAG(SD_ASYM_CPUCAPACITY, SDF_NEEDS_GROUPS)
/*
* Domain members share CPU capacity (i.e. SMT)
*
* SHARED_CHILD: Set from the base domain up until spanned CPUs no longer share
- * CPU capacity.
+ * CPU capacity.
+ * NEEDS_GROUPS: Capacity is shared between groups.
*/
-SD_FLAG(SD_SHARE_CPUCAPACITY, SDF_SHARED_CHILD)
+SD_FLAG(SD_SHARE_CPUCAPACITY, SDF_SHARED_CHILD | SDF_NEEDS_GROUPS)
/*
* Domain members share CPU package resources (i.e. caches)
*
* SHARED_CHILD: Set from the base domain up until spanned CPUs no longer share
- * the same cache(s).
+ * the same cache(s).
+ * NEEDS_GROUPS: Caches are shared between groups.
*/
-SD_FLAG(SD_SHARE_PKG_RESOURCES, SDF_SHARED_CHILD)
+SD_FLAG(SD_SHARE_PKG_RESOURCES, SDF_SHARED_CHILD | SDF_NEEDS_GROUPS)
/*
* Only a single load balancing instance
diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h
index 32f602ff37a0..2d59ca77103e 100644
--- a/include/linux/sched/topology.h
+++ b/include/linux/sched/topology.h
@@ -25,6 +25,13 @@ enum {
};
#undef SD_FLAG
+/* Generate a mask of SD flags with the SDF_NEEDS_GROUPS metaflag */
+#define SD_FLAG(name, mflags) (name * !!((mflags) & SDF_NEEDS_GROUPS)) |
+static const unsigned int SD_DEGENERATE_GROUPS_MASK =
+#include <linux/sched/sd_flags.h>
+0;
+#undef SD_FLAG
+
#ifdef CONFIG_SCHED_DEBUG
#define SD_FLAG(_name, mflags) [__##_name] = { .meta_flags = mflags, .name = #_name },
static const struct {
--
2.27.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 08/17] sched/topology: Use prebuilt SD flag degeneration mask
2020-08-17 11:29 [PATCH v6 00/17] sched: Instrument sched domain flags Valentin Schneider
` (6 preceding siblings ...)
2020-08-17 11:29 ` [PATCH v6 07/17] sched/topology: Introduce SD metaflag for flags needing > 1 groups Valentin Schneider
@ 2020-08-17 11:29 ` Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 09/17] sched/topology: Remove SD_SERIALIZE degeneration special case Valentin Schneider
` (8 subsequent siblings)
16 siblings, 1 reply; 34+ messages in thread
From: Valentin Schneider @ 2020-08-17 11:29 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel
Cc: Peter Zijlstra, mingo, vincent.guittot, dietmar.eggemann,
morten.rasmussen, Quentin Perret
Leverage SD_DEGENERATE_GROUPS_MASK in sd_degenerate() and
sd_parent_degenerate().
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
kernel/sched/topology.c | 20 ++++----------------
1 file changed, 4 insertions(+), 16 deletions(-)
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index f128fcf46a41..5f2bc99ff659 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -160,15 +160,9 @@ static int sd_degenerate(struct sched_domain *sd)
return 1;
/* Following flags need at least 2 groups */
- if (sd->flags & (SD_BALANCE_NEWIDLE |
- SD_BALANCE_FORK |
- SD_BALANCE_EXEC |
- SD_SHARE_CPUCAPACITY |
- SD_ASYM_CPUCAPACITY |
- SD_SHARE_PKG_RESOURCES)) {
- if (sd->groups != sd->groups->next)
- return 0;
- }
+ if ((sd->flags & SD_DEGENERATE_GROUPS_MASK) &&
+ (sd->groups != sd->groups->next))
+ return 0;
/* Following flags don't use groups */
if (sd->flags & (SD_WAKE_AFFINE))
@@ -190,13 +184,7 @@ sd_parent_degenerate(struct sched_domain *sd, struct sched_domain *parent)
/* Flags needing groups don't count if only 1 group in parent */
if (parent->groups == parent->groups->next) {
- pflags &= ~(SD_BALANCE_NEWIDLE |
- SD_BALANCE_FORK |
- SD_BALANCE_EXEC |
- SD_ASYM_CPUCAPACITY |
- SD_SHARE_CPUCAPACITY |
- SD_SHARE_PKG_RESOURCES |
- SD_PREFER_SIBLING);
+ pflags &= ~(SD_DEGENERATE_GROUPS_MASK | SD_PREFER_SIBLING);
if (nr_node_ids == 1)
pflags &= ~SD_SERIALIZE;
}
--
2.27.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 09/17] sched/topology: Remove SD_SERIALIZE degeneration special case
2020-08-17 11:29 [PATCH v6 00/17] sched: Instrument sched domain flags Valentin Schneider
` (7 preceding siblings ...)
2020-08-17 11:29 ` [PATCH v6 08/17] sched/topology: Use prebuilt SD flag degeneration mask Valentin Schneider
@ 2020-08-17 11:29 ` Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 10/17] sched/topology: Propagate SD_ASYM_CPUCAPACITY upwards Valentin Schneider
` (7 subsequent siblings)
16 siblings, 1 reply; 34+ messages in thread
From: Valentin Schneider @ 2020-08-17 11:29 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel
Cc: Peter Zijlstra, Dietmar Eggemann, mingo, vincent.guittot,
morten.rasmussen, Quentin Perret
If there is only a single NUMA node in the system, the only NUMA topology
level that will be generated will be NODE (identity distance), which
doesn't have SD_SERIALIZE.
This means we don't need this special case in sd_parent_degenerate(), as
having the NODE level "naturally" covers it. Thus, remove it.
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Reviewed-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
kernel/sched/topology.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 5f2bc99ff659..00ad7cef2ec1 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -183,11 +183,9 @@ sd_parent_degenerate(struct sched_domain *sd, struct sched_domain *parent)
return 0;
/* Flags needing groups don't count if only 1 group in parent */
- if (parent->groups == parent->groups->next) {
+ if (parent->groups == parent->groups->next)
pflags &= ~(SD_DEGENERATE_GROUPS_MASK | SD_PREFER_SIBLING);
- if (nr_node_ids == 1)
- pflags &= ~SD_SERIALIZE;
- }
+
if (~cflags & pflags)
return 0;
--
2.27.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 10/17] sched/topology: Propagate SD_ASYM_CPUCAPACITY upwards
2020-08-17 11:29 [PATCH v6 00/17] sched: Instrument sched domain flags Valentin Schneider
` (8 preceding siblings ...)
2020-08-17 11:29 ` [PATCH v6 09/17] sched/topology: Remove SD_SERIALIZE degeneration special case Valentin Schneider
@ 2020-08-17 11:29 ` Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 11/17] sched/topology: Mark SD_PREFER_SIBLING as SDF_NEEDS_GROUPS Valentin Schneider
` (6 subsequent siblings)
16 siblings, 1 reply; 34+ messages in thread
From: Valentin Schneider @ 2020-08-17 11:29 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel
Cc: Quentin Perret, Dietmar Eggemann, mingo, peterz, vincent.guittot,
morten.rasmussen
We currently set this flag *only* on domains whose topology level exactly
match the level where we detect asymmetry (as returned by
asym_cpu_capacity_level()). This is rather problematic.
Say there are two clusters in the system, one with a lone big CPU and the
other with a mix of big and LITTLE CPUs (as is allowed by DynamIQ):
DIE [ ]
MC [ ][ ]
0 1 2 3 4
L L B B B
asym_cpu_capacity_level() will figure out that the MC level is the one
where all CPUs can see a CPU of max capacity, and we will thus set
SD_ASYM_CPUCAPACITY at MC level for all CPUs.
That lone big CPU will degenerate its MC domain, since it would be alone in
there, and will end up with just a DIE domain. Since the flag was only set
at MC, this CPU ends up not seeing any SD with the flag set, which is
broken.
Rather than clearing dflags at every topology level, clear it before
entering the topology level loop. This will properly propagate upwards
flags that are set starting from a certain level.
Reviewed-by: Quentin Perret <qperret@google.com>
Reviewed-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
include/linux/sched/sd_flags.h | 4 +++-
kernel/sched/topology.c | 3 +--
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h
index 21a43ad6f26a..4f07b405564e 100644
--- a/include/linux/sched/sd_flags.h
+++ b/include/linux/sched/sd_flags.h
@@ -83,9 +83,11 @@ SD_FLAG(SD_WAKE_AFFINE, SDF_SHARED_CHILD)
/*
* Domain members have different CPU capacities
*
+ * SHARED_PARENT: Set from the topmost domain down to the first domain where
+ * asymmetry is detected.
* NEEDS_GROUPS: Per-CPU capacity is asymmetric between groups.
*/
-SD_FLAG(SD_ASYM_CPUCAPACITY, SDF_NEEDS_GROUPS)
+SD_FLAG(SD_ASYM_CPUCAPACITY, SDF_SHARED_PARENT | SDF_NEEDS_GROUPS)
/*
* Domain members share CPU capacity (i.e. SMT)
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 00ad7cef2ec1..02fd8db747b2 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -1988,11 +1988,10 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att
/* Set up domains for CPUs specified by the cpu_map: */
for_each_cpu(i, cpu_map) {
struct sched_domain_topology_level *tl;
+ int dflags = 0;
sd = NULL;
for_each_sd_topology(tl) {
- int dflags = 0;
-
if (tl == tl_asym) {
dflags |= SD_ASYM_CPUCAPACITY;
has_asym = true;
--
2.27.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 11/17] sched/topology: Mark SD_PREFER_SIBLING as SDF_NEEDS_GROUPS
2020-08-17 11:29 [PATCH v6 00/17] sched: Instrument sched domain flags Valentin Schneider
` (9 preceding siblings ...)
2020-08-17 11:29 ` [PATCH v6 10/17] sched/topology: Propagate SD_ASYM_CPUCAPACITY upwards Valentin Schneider
@ 2020-08-17 11:29 ` Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 12/17] sched/topology: Mark SD_BALANCE_WAKE " Valentin Schneider
` (5 subsequent siblings)
16 siblings, 1 reply; 34+ messages in thread
From: Valentin Schneider @ 2020-08-17 11:29 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel
Cc: mingo, peterz, vincent.guittot, dietmar.eggemann,
morten.rasmussen, Quentin Perret
SD_PREFER_SIBLING is currently considered in sd_parent_degenerate() but not
in sd_degenerate(). It too hinges on load balancing, and thus won't have
any effect when set on a domain with a single group. Add it to
SD_DEGENERATE_GROUPS_MASK.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
include/linux/sched/sd_flags.h | 4 +++-
kernel/sched/topology.c | 2 +-
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h
index 4f07b405564e..75c9749a6e2d 100644
--- a/include/linux/sched/sd_flags.h
+++ b/include/linux/sched/sd_flags.h
@@ -131,8 +131,10 @@ SD_FLAG(SD_ASYM_PACKING, SDF_SHARED_CHILD)
*
* Set up until domains start spanning NUMA nodes. Close to being a SHARED_CHILD
* flag, but cleared below domains with SD_ASYM_CPUCAPACITY.
+ *
+ * NEEDS_GROUPS: Load balancing flag.
*/
-SD_FLAG(SD_PREFER_SIBLING, 0)
+SD_FLAG(SD_PREFER_SIBLING, SDF_NEEDS_GROUPS)
/*
* sched_groups of this level overlap
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 02fd8db747b2..8064f495641b 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -184,7 +184,7 @@ sd_parent_degenerate(struct sched_domain *sd, struct sched_domain *parent)
/* Flags needing groups don't count if only 1 group in parent */
if (parent->groups == parent->groups->next)
- pflags &= ~(SD_DEGENERATE_GROUPS_MASK | SD_PREFER_SIBLING);
+ pflags &= ~SD_DEGENERATE_GROUPS_MASK;
if (~cflags & pflags)
return 0;
--
2.27.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 12/17] sched/topology: Mark SD_BALANCE_WAKE as SDF_NEEDS_GROUPS
2020-08-17 11:29 [PATCH v6 00/17] sched: Instrument sched domain flags Valentin Schneider
` (10 preceding siblings ...)
2020-08-17 11:29 ` [PATCH v6 11/17] sched/topology: Mark SD_PREFER_SIBLING as SDF_NEEDS_GROUPS Valentin Schneider
@ 2020-08-17 11:29 ` Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 13/17] sched/topology: Mark SD_SERIALIZE " Valentin Schneider
` (4 subsequent siblings)
16 siblings, 1 reply; 34+ messages in thread
From: Valentin Schneider @ 2020-08-17 11:29 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel
Cc: mingo, peterz, vincent.guittot, dietmar.eggemann,
morten.rasmussen, Quentin Perret
Even if no mainline topology uses this flag, it is a load balancing flag
just like SD_BALANCE_FORK and requires 2+ groups to have any effect.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
include/linux/sched/sd_flags.h | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h
index 75c9749a6e2d..9043d18ce418 100644
--- a/include/linux/sched/sd_flags.h
+++ b/include/linux/sched/sd_flags.h
@@ -70,8 +70,9 @@ SD_FLAG(SD_BALANCE_FORK, SDF_SHARED_CHILD | SDF_NEEDS_GROUPS)
* Balance on wakeup
*
* SHARED_CHILD: Set from the base domain up to cpuset.sched_relax_domain_level.
+ * NEEDS_GROUPS: Load balancing flag.
*/
-SD_FLAG(SD_BALANCE_WAKE, SDF_SHARED_CHILD)
+SD_FLAG(SD_BALANCE_WAKE, SDF_SHARED_CHILD | SDF_NEEDS_GROUPS)
/*
* Consider waking task on waking CPU.
--
2.27.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 13/17] sched/topology: Mark SD_SERIALIZE as SDF_NEEDS_GROUPS
2020-08-17 11:29 [PATCH v6 00/17] sched: Instrument sched domain flags Valentin Schneider
` (11 preceding siblings ...)
2020-08-17 11:29 ` [PATCH v6 12/17] sched/topology: Mark SD_BALANCE_WAKE " Valentin Schneider
@ 2020-08-17 11:29 ` Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:30 ` [PATCH v6 14/17] sched/topology: Mark SD_ASYM_PACKING " Valentin Schneider
` (3 subsequent siblings)
16 siblings, 1 reply; 34+ messages in thread
From: Valentin Schneider @ 2020-08-17 11:29 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel
Cc: mingo, peterz, vincent.guittot, dietmar.eggemann,
morten.rasmussen, Quentin Perret
There would be no point in preserving a sched_domain with a single group
just because it has this flag set. Add it to SD_DEGENERATE_GROUPS_MASK.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
include/linux/sched/sd_flags.h | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h
index 9043d18ce418..6eb302528659 100644
--- a/include/linux/sched/sd_flags.h
+++ b/include/linux/sched/sd_flags.h
@@ -112,11 +112,12 @@ SD_FLAG(SD_SHARE_PKG_RESOURCES, SDF_SHARED_CHILD | SDF_NEEDS_GROUPS)
* Only a single load balancing instance
*
* SHARED_PARENT: Set for all NUMA levels above NODE. Could be set from a
- * different level upwards, but it doesn't change that if a domain has this flag
- * set, then all of its parents need to have it too (otherwise the serialization
- * doesn't make sense).
+ * different level upwards, but it doesn't change that if a
+ * domain has this flag set, then all of its parents need to have
+ * it too (otherwise the serialization doesn't make sense).
+ * NEEDS_GROUPS: No point in preserving domain if it has a single group.
*/
-SD_FLAG(SD_SERIALIZE, SDF_SHARED_PARENT)
+SD_FLAG(SD_SERIALIZE, SDF_SHARED_PARENT | SDF_NEEDS_GROUPS)
/*
* Place busy tasks earlier in the domain
--
2.27.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 14/17] sched/topology: Mark SD_ASYM_PACKING as SDF_NEEDS_GROUPS
2020-08-17 11:29 [PATCH v6 00/17] sched: Instrument sched domain flags Valentin Schneider
` (12 preceding siblings ...)
2020-08-17 11:29 ` [PATCH v6 13/17] sched/topology: Mark SD_SERIALIZE " Valentin Schneider
@ 2020-08-17 11:30 ` Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:30 ` [PATCH v6 15/17] sched/topology: Mark SD_OVERLAP " Valentin Schneider
` (2 subsequent siblings)
16 siblings, 1 reply; 34+ messages in thread
From: Valentin Schneider @ 2020-08-17 11:30 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel
Cc: mingo, peterz, vincent.guittot, dietmar.eggemann,
morten.rasmussen, Quentin Perret
Being a load-balancing flag, it requires 2+ groups to have any effect.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
include/linux/sched/sd_flags.h | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h
index 6eb302528659..f855f80f0052 100644
--- a/include/linux/sched/sd_flags.h
+++ b/include/linux/sched/sd_flags.h
@@ -123,10 +123,11 @@ SD_FLAG(SD_SERIALIZE, SDF_SHARED_PARENT | SDF_NEEDS_GROUPS)
* Place busy tasks earlier in the domain
*
* SHARED_CHILD: Usually set on the SMT level. Technically could be set further
- * up, but currently assumed to be set from the base domain upwards (see
- * update_top_cache_domain()).
+ * up, but currently assumed to be set from the base domain
+ * upwards (see update_top_cache_domain()).
+ * NEEDS_GROUPS: Load balancing flag.
*/
-SD_FLAG(SD_ASYM_PACKING, SDF_SHARED_CHILD)
+SD_FLAG(SD_ASYM_PACKING, SDF_SHARED_CHILD | SDF_NEEDS_GROUPS)
/*
* Prefer to place tasks in a sibling domain
--
2.27.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 15/17] sched/topology: Mark SD_OVERLAP as SDF_NEEDS_GROUPS
2020-08-17 11:29 [PATCH v6 00/17] sched: Instrument sched domain flags Valentin Schneider
` (13 preceding siblings ...)
2020-08-17 11:30 ` [PATCH v6 14/17] sched/topology: Mark SD_ASYM_PACKING " Valentin Schneider
@ 2020-08-17 11:30 ` Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:30 ` [PATCH v6 16/17] sched/topology: Mark SD_NUMA " Valentin Schneider
2020-08-17 11:30 ` [PATCH v6 17/17] sched/topology: Expand use of SD_DEGENERATE_GROUPS_MASK to flags not needing groups Valentin Schneider
16 siblings, 1 reply; 34+ messages in thread
From: Valentin Schneider @ 2020-08-17 11:30 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel
Cc: mingo, peterz, vincent.guittot, dietmar.eggemann,
morten.rasmussen, Quentin Perret
A sched_domain can only have overlapping sched_groups if it has more than
one group.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
include/linux/sched/sd_flags.h | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h
index f855f80f0052..021b909d3941 100644
--- a/include/linux/sched/sd_flags.h
+++ b/include/linux/sched/sd_flags.h
@@ -143,8 +143,9 @@ SD_FLAG(SD_PREFER_SIBLING, SDF_NEEDS_GROUPS)
* sched_groups of this level overlap
*
* SHARED_PARENT: Set for all NUMA levels above NODE.
+ * NEEDS_GROUPS: Overlaps can only exist with more than one group.
*/
-SD_FLAG(SD_OVERLAP, SDF_SHARED_PARENT)
+SD_FLAG(SD_OVERLAP, SDF_SHARED_PARENT | SDF_NEEDS_GROUPS)
/*
* Cross-node balancing
--
2.27.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 16/17] sched/topology: Mark SD_NUMA as SDF_NEEDS_GROUPS
2020-08-17 11:29 [PATCH v6 00/17] sched: Instrument sched domain flags Valentin Schneider
` (14 preceding siblings ...)
2020-08-17 11:30 ` [PATCH v6 15/17] sched/topology: Mark SD_OVERLAP " Valentin Schneider
@ 2020-08-17 11:30 ` Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:30 ` [PATCH v6 17/17] sched/topology: Expand use of SD_DEGENERATE_GROUPS_MASK to flags not needing groups Valentin Schneider
16 siblings, 1 reply; 34+ messages in thread
From: Valentin Schneider @ 2020-08-17 11:30 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel
Cc: mingo, peterz, vincent.guittot, dietmar.eggemann,
morten.rasmussen, Quentin Perret
There would be no point in preserving a sched_domain with a single group
just because it has this flag set. Add it to SD_DEGENERATE_GROUPS_MASK.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
include/linux/sched/sd_flags.h | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h
index 021b909d3941..71675afa55f0 100644
--- a/include/linux/sched/sd_flags.h
+++ b/include/linux/sched/sd_flags.h
@@ -151,5 +151,6 @@ SD_FLAG(SD_OVERLAP, SDF_SHARED_PARENT | SDF_NEEDS_GROUPS)
* Cross-node balancing
*
* SHARED_PARENT: Set for all NUMA levels above NODE.
+ * NEEDS_GROUPS: No point in preserving domain if it has a single group.
*/
-SD_FLAG(SD_NUMA, SDF_SHARED_PARENT)
+SD_FLAG(SD_NUMA, SDF_SHARED_PARENT | SDF_NEEDS_GROUPS)
--
2.27.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v6 17/17] sched/topology: Expand use of SD_DEGENERATE_GROUPS_MASK to flags not needing groups
2020-08-17 11:29 [PATCH v6 00/17] sched: Instrument sched domain flags Valentin Schneider
` (15 preceding siblings ...)
2020-08-17 11:30 ` [PATCH v6 16/17] sched/topology: Mark SD_NUMA " Valentin Schneider
@ 2020-08-17 11:30 ` Valentin Schneider
16 siblings, 0 replies; 34+ messages in thread
From: Valentin Schneider @ 2020-08-17 11:30 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel
Cc: mingo, peterz, vincent.guittot, dietmar.eggemann,
morten.rasmussen, Quentin Perret
All SD flags requiring 2+ sched_group to have any effect are now decorated
with the SDF_NEEDS_GROUPS metaflag. This means we can now use the bitwise
negation of SD_DEGENERATE_MASK in sd_degenerate() instead of explicitly
using SD_WAKE_AFFINE (IOW the only flag without SDF_NEEDS_GROUPS).
From now on, any flag without SDF_NEEDS_GROUPS will be correctly accounted
as a flag not requiring 2+ sched_groups.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
kernel/sched/topology.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 8064f495641b..3bb145ef5abd 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -165,7 +165,7 @@ static int sd_degenerate(struct sched_domain *sd)
return 0;
/* Following flags don't use groups */
- if (sd->flags & (SD_WAKE_AFFINE))
+ if (sd->flags & ~SD_DEGENERATE_GROUPS_MASK)
return 0;
return 1;
--
2.27.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [tip: sched/core] sched/topology: Mark SD_NUMA as SDF_NEEDS_GROUPS
2020-08-17 11:30 ` [PATCH v6 16/17] sched/topology: Mark SD_NUMA " Valentin Schneider
@ 2020-08-19 14:02 ` tip-bot2 for Valentin Schneider
0 siblings, 0 replies; 34+ messages in thread
From: tip-bot2 for Valentin Schneider @ 2020-08-19 14:02 UTC (permalink / raw)
To: linux-tip-commits
Cc: Valentin Schneider, Ingo Molnar, Peter Zijlstra, x86, LKML
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 5f4a1c4ea44728aa80be21dbf3a0469b5ca81d88
Gitweb: https://git.kernel.org/tip/5f4a1c4ea44728aa80be21dbf3a0469b5ca81d88
Author: Valentin Schneider <valentin.schneider@arm.com>
AuthorDate: Mon, 17 Aug 2020 12:30:02 +01:00
Committer: Ingo Molnar <mingo@kernel.org>
CommitterDate: Wed, 19 Aug 2020 10:49:50 +02:00
sched/topology: Mark SD_NUMA as SDF_NEEDS_GROUPS
There would be no point in preserving a sched_domain with a single group
just because it has this flag set. Add it to SD_DEGENERATE_GROUPS_MASK.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: https://lore.kernel.org/r/20200817113003.20802-17-valentin.schneider@arm.com
---
include/linux/sched/sd_flags.h | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h
index 29af5f0..34b21e9 100644
--- a/include/linux/sched/sd_flags.h
+++ b/include/linux/sched/sd_flags.h
@@ -151,5 +151,6 @@ SD_FLAG(SD_OVERLAP, SDF_SHARED_PARENT | SDF_NEEDS_GROUPS)
* Cross-node balancing
*
* SHARED_PARENT: Set for all NUMA levels above NODE.
+ * NEEDS_GROUPS: No point in preserving domain if it has a single group.
*/
-SD_FLAG(SD_NUMA, SDF_SHARED_PARENT)
+SD_FLAG(SD_NUMA, SDF_SHARED_PARENT | SDF_NEEDS_GROUPS)
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [tip: sched/core] sched/topology: Mark SD_OVERLAP as SDF_NEEDS_GROUPS
2020-08-17 11:30 ` [PATCH v6 15/17] sched/topology: Mark SD_OVERLAP " Valentin Schneider
@ 2020-08-19 14:02 ` tip-bot2 for Valentin Schneider
0 siblings, 0 replies; 34+ messages in thread
From: tip-bot2 for Valentin Schneider @ 2020-08-19 14:02 UTC (permalink / raw)
To: linux-tip-commits
Cc: Valentin Schneider, Ingo Molnar, Peter Zijlstra, x86, LKML
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 3551e954f5d95faf3dbc340d422da7624658c230
Gitweb: https://git.kernel.org/tip/3551e954f5d95faf3dbc340d422da7624658c230
Author: Valentin Schneider <valentin.schneider@arm.com>
AuthorDate: Mon, 17 Aug 2020 12:30:01 +01:00
Committer: Ingo Molnar <mingo@kernel.org>
CommitterDate: Wed, 19 Aug 2020 10:49:50 +02:00
sched/topology: Mark SD_OVERLAP as SDF_NEEDS_GROUPS
A sched_domain can only have overlapping sched_groups if it has more than
one group.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: https://lore.kernel.org/r/20200817113003.20802-16-valentin.schneider@arm.com
---
include/linux/sched/sd_flags.h | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h
index 2998ece..29af5f0 100644
--- a/include/linux/sched/sd_flags.h
+++ b/include/linux/sched/sd_flags.h
@@ -143,8 +143,9 @@ SD_FLAG(SD_PREFER_SIBLING, SDF_NEEDS_GROUPS)
* sched_groups of this level overlap
*
* SHARED_PARENT: Set for all NUMA levels above NODE.
+ * NEEDS_GROUPS: Overlaps can only exist with more than one group.
*/
-SD_FLAG(SD_OVERLAP, SDF_SHARED_PARENT)
+SD_FLAG(SD_OVERLAP, SDF_SHARED_PARENT | SDF_NEEDS_GROUPS)
/*
* Cross-node balancing
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [tip: sched/core] sched/topology: Mark SD_ASYM_PACKING as SDF_NEEDS_GROUPS
2020-08-17 11:30 ` [PATCH v6 14/17] sched/topology: Mark SD_ASYM_PACKING " Valentin Schneider
@ 2020-08-19 14:02 ` tip-bot2 for Valentin Schneider
0 siblings, 0 replies; 34+ messages in thread
From: tip-bot2 for Valentin Schneider @ 2020-08-19 14:02 UTC (permalink / raw)
To: linux-tip-commits
Cc: Valentin Schneider, Ingo Molnar, Peter Zijlstra, x86, LKML
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 33199b0143daf4778d6301f966cb914d75f122eb
Gitweb: https://git.kernel.org/tip/33199b0143daf4778d6301f966cb914d75f122eb
Author: Valentin Schneider <valentin.schneider@arm.com>
AuthorDate: Mon, 17 Aug 2020 12:30:00 +01:00
Committer: Ingo Molnar <mingo@kernel.org>
CommitterDate: Wed, 19 Aug 2020 10:49:49 +02:00
sched/topology: Mark SD_ASYM_PACKING as SDF_NEEDS_GROUPS
Being a load-balancing flag, it requires 2+ groups to have any effect.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: https://lore.kernel.org/r/20200817113003.20802-15-valentin.schneider@arm.com
---
include/linux/sched/sd_flags.h | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h
index b7f4d80..2998ece 100644
--- a/include/linux/sched/sd_flags.h
+++ b/include/linux/sched/sd_flags.h
@@ -123,10 +123,11 @@ SD_FLAG(SD_SERIALIZE, SDF_SHARED_PARENT | SDF_NEEDS_GROUPS)
* Place busy tasks earlier in the domain
*
* SHARED_CHILD: Usually set on the SMT level. Technically could be set further
- * up, but currently assumed to be set from the base domain upwards (see
- * update_top_cache_domain()).
+ * up, but currently assumed to be set from the base domain
+ * upwards (see update_top_cache_domain()).
+ * NEEDS_GROUPS: Load balancing flag.
*/
-SD_FLAG(SD_ASYM_PACKING, SDF_SHARED_CHILD)
+SD_FLAG(SD_ASYM_PACKING, SDF_SHARED_CHILD | SDF_NEEDS_GROUPS)
/*
* Prefer to place tasks in a sibling domain
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [tip: sched/core] sched/topology: Mark SD_SERIALIZE as SDF_NEEDS_GROUPS
2020-08-17 11:29 ` [PATCH v6 13/17] sched/topology: Mark SD_SERIALIZE " Valentin Schneider
@ 2020-08-19 14:02 ` tip-bot2 for Valentin Schneider
0 siblings, 0 replies; 34+ messages in thread
From: tip-bot2 for Valentin Schneider @ 2020-08-19 14:02 UTC (permalink / raw)
To: linux-tip-commits
Cc: Valentin Schneider, Ingo Molnar, Peter Zijlstra, x86, LKML
The following commit has been merged into the sched/core branch of tip:
Commit-ID: bdb7c802cc0a7e21f5223dc3ce41b7ac220c576e
Gitweb: https://git.kernel.org/tip/bdb7c802cc0a7e21f5223dc3ce41b7ac220c576e
Author: Valentin Schneider <valentin.schneider@arm.com>
AuthorDate: Mon, 17 Aug 2020 12:29:59 +01:00
Committer: Ingo Molnar <mingo@kernel.org>
CommitterDate: Wed, 19 Aug 2020 10:49:49 +02:00
sched/topology: Mark SD_SERIALIZE as SDF_NEEDS_GROUPS
There would be no point in preserving a sched_domain with a single group
just because it has this flag set. Add it to SD_DEGENERATE_GROUPS_MASK.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: https://lore.kernel.org/r/20200817113003.20802-14-valentin.schneider@arm.com
---
include/linux/sched/sd_flags.h | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h
index 729510a..b7f4d80 100644
--- a/include/linux/sched/sd_flags.h
+++ b/include/linux/sched/sd_flags.h
@@ -112,11 +112,12 @@ SD_FLAG(SD_SHARE_PKG_RESOURCES, SDF_SHARED_CHILD | SDF_NEEDS_GROUPS)
* Only a single load balancing instance
*
* SHARED_PARENT: Set for all NUMA levels above NODE. Could be set from a
- * different level upwards, but it doesn't change that if a domain has this flag
- * set, then all of its parents need to have it too (otherwise the serialization
- * doesn't make sense).
+ * different level upwards, but it doesn't change that if a
+ * domain has this flag set, then all of its parents need to have
+ * it too (otherwise the serialization doesn't make sense).
+ * NEEDS_GROUPS: No point in preserving domain if it has a single group.
*/
-SD_FLAG(SD_SERIALIZE, SDF_SHARED_PARENT)
+SD_FLAG(SD_SERIALIZE, SDF_SHARED_PARENT | SDF_NEEDS_GROUPS)
/*
* Place busy tasks earlier in the domain
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [tip: sched/core] sched/topology: Mark SD_BALANCE_WAKE as SDF_NEEDS_GROUPS
2020-08-17 11:29 ` [PATCH v6 12/17] sched/topology: Mark SD_BALANCE_WAKE " Valentin Schneider
@ 2020-08-19 14:02 ` tip-bot2 for Valentin Schneider
0 siblings, 0 replies; 34+ messages in thread
From: tip-bot2 for Valentin Schneider @ 2020-08-19 14:02 UTC (permalink / raw)
To: linux-tip-commits
Cc: Valentin Schneider, Ingo Molnar, Peter Zijlstra, x86, LKML
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 94b858fea1f2246a2fb7f7af21840fd14ced028f
Gitweb: https://git.kernel.org/tip/94b858fea1f2246a2fb7f7af21840fd14ced028f
Author: Valentin Schneider <valentin.schneider@arm.com>
AuthorDate: Mon, 17 Aug 2020 12:29:58 +01:00
Committer: Ingo Molnar <mingo@kernel.org>
CommitterDate: Wed, 19 Aug 2020 10:49:49 +02:00
sched/topology: Mark SD_BALANCE_WAKE as SDF_NEEDS_GROUPS
Even if no mainline topology uses this flag, it is a load balancing flag
just like SD_BALANCE_FORK and requires 2+ groups to have any effect.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: https://lore.kernel.org/r/20200817113003.20802-13-valentin.schneider@arm.com
---
include/linux/sched/sd_flags.h | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h
index d28fe67..729510a 100644
--- a/include/linux/sched/sd_flags.h
+++ b/include/linux/sched/sd_flags.h
@@ -70,8 +70,9 @@ SD_FLAG(SD_BALANCE_FORK, SDF_SHARED_CHILD | SDF_NEEDS_GROUPS)
* Balance on wakeup
*
* SHARED_CHILD: Set from the base domain up to cpuset.sched_relax_domain_level.
+ * NEEDS_GROUPS: Load balancing flag.
*/
-SD_FLAG(SD_BALANCE_WAKE, SDF_SHARED_CHILD)
+SD_FLAG(SD_BALANCE_WAKE, SDF_SHARED_CHILD | SDF_NEEDS_GROUPS)
/*
* Consider waking task on waking CPU.
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [tip: sched/core] sched/topology: Propagate SD_ASYM_CPUCAPACITY upwards
2020-08-17 11:29 ` [PATCH v6 10/17] sched/topology: Propagate SD_ASYM_CPUCAPACITY upwards Valentin Schneider
@ 2020-08-19 14:02 ` tip-bot2 for Valentin Schneider
0 siblings, 0 replies; 34+ messages in thread
From: tip-bot2 for Valentin Schneider @ 2020-08-19 14:02 UTC (permalink / raw)
To: linux-tip-commits
Cc: Valentin Schneider, Ingo Molnar, Quentin Perret,
Dietmar Eggemann, Peter Zijlstra, x86, LKML
The following commit has been merged into the sched/core branch of tip:
Commit-ID: c200191d4c2c1aa2ffb62a984b756ac1f02dc55c
Gitweb: https://git.kernel.org/tip/c200191d4c2c1aa2ffb62a984b756ac1f02dc55c
Author: Valentin Schneider <valentin.schneider@arm.com>
AuthorDate: Mon, 17 Aug 2020 12:29:56 +01:00
Committer: Ingo Molnar <mingo@kernel.org>
CommitterDate: Wed, 19 Aug 2020 10:49:49 +02:00
sched/topology: Propagate SD_ASYM_CPUCAPACITY upwards
We currently set this flag *only* on domains whose topology level exactly
match the level where we detect asymmetry (as returned by
asym_cpu_capacity_level()). This is rather problematic.
Say there are two clusters in the system, one with a lone big CPU and the
other with a mix of big and LITTLE CPUs (as is allowed by DynamIQ):
DIE [ ]
MC [ ][ ]
0 1 2 3 4
L L B B B
asym_cpu_capacity_level() will figure out that the MC level is the one
where all CPUs can see a CPU of max capacity, and we will thus set
SD_ASYM_CPUCAPACITY at MC level for all CPUs.
That lone big CPU will degenerate its MC domain, since it would be alone in
there, and will end up with just a DIE domain. Since the flag was only set
at MC, this CPU ends up not seeing any SD with the flag set, which is
broken.
Rather than clearing dflags at every topology level, clear it before
entering the topology level loop. This will properly propagate upwards
flags that are set starting from a certain level.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Quentin Perret <qperret@google.com>
Reviewed-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: https://lore.kernel.org/r/20200817113003.20802-11-valentin.schneider@arm.com
---
include/linux/sched/sd_flags.h | 4 +++-
kernel/sched/topology.c | 3 +--
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h
index ee5cbfc..40ad0d5 100644
--- a/include/linux/sched/sd_flags.h
+++ b/include/linux/sched/sd_flags.h
@@ -83,9 +83,11 @@ SD_FLAG(SD_WAKE_AFFINE, SDF_SHARED_CHILD)
/*
* Domain members have different CPU capacities
*
+ * SHARED_PARENT: Set from the topmost domain down to the first domain where
+ * asymmetry is detected.
* NEEDS_GROUPS: Per-CPU capacity is asymmetric between groups.
*/
-SD_FLAG(SD_ASYM_CPUCAPACITY, SDF_NEEDS_GROUPS)
+SD_FLAG(SD_ASYM_CPUCAPACITY, SDF_SHARED_PARENT | SDF_NEEDS_GROUPS)
/*
* Domain members share CPU capacity (i.e. SMT)
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index c662c53..f36ed96 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -1988,11 +1988,10 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att
/* Set up domains for CPUs specified by the cpu_map: */
for_each_cpu(i, cpu_map) {
struct sched_domain_topology_level *tl;
+ int dflags = 0;
sd = NULL;
for_each_sd_topology(tl) {
- int dflags = 0;
-
if (tl == tl_asym) {
dflags |= SD_ASYM_CPUCAPACITY;
has_asym = true;
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [tip: sched/core] sched/topology: Mark SD_PREFER_SIBLING as SDF_NEEDS_GROUPS
2020-08-17 11:29 ` [PATCH v6 11/17] sched/topology: Mark SD_PREFER_SIBLING as SDF_NEEDS_GROUPS Valentin Schneider
@ 2020-08-19 14:02 ` tip-bot2 for Valentin Schneider
0 siblings, 0 replies; 34+ messages in thread
From: tip-bot2 for Valentin Schneider @ 2020-08-19 14:02 UTC (permalink / raw)
To: linux-tip-commits
Cc: Valentin Schneider, Ingo Molnar, Peter Zijlstra, x86, LKML
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 3a6712c7685352a5d71eaf459a4fddfc3589f018
Gitweb: https://git.kernel.org/tip/3a6712c7685352a5d71eaf459a4fddfc3589f018
Author: Valentin Schneider <valentin.schneider@arm.com>
AuthorDate: Mon, 17 Aug 2020 12:29:57 +01:00
Committer: Ingo Molnar <mingo@kernel.org>
CommitterDate: Wed, 19 Aug 2020 10:49:49 +02:00
sched/topology: Mark SD_PREFER_SIBLING as SDF_NEEDS_GROUPS
SD_PREFER_SIBLING is currently considered in sd_parent_degenerate() but not
in sd_degenerate(). It too hinges on load balancing, and thus won't have
any effect when set on a domain with a single group. Add it to
SD_DEGENERATE_GROUPS_MASK.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: https://lore.kernel.org/r/20200817113003.20802-12-valentin.schneider@arm.com
---
include/linux/sched/sd_flags.h | 4 +++-
kernel/sched/topology.c | 2 +-
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h
index 40ad0d5..d28fe67 100644
--- a/include/linux/sched/sd_flags.h
+++ b/include/linux/sched/sd_flags.h
@@ -131,8 +131,10 @@ SD_FLAG(SD_ASYM_PACKING, SDF_SHARED_CHILD)
*
* Set up until domains start spanning NUMA nodes. Close to being a SHARED_CHILD
* flag, but cleared below domains with SD_ASYM_CPUCAPACITY.
+ *
+ * NEEDS_GROUPS: Load balancing flag.
*/
-SD_FLAG(SD_PREFER_SIBLING, 0)
+SD_FLAG(SD_PREFER_SIBLING, SDF_NEEDS_GROUPS)
/*
* sched_groups of this level overlap
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index f36ed96..c674aaa 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -184,7 +184,7 @@ sd_parent_degenerate(struct sched_domain *sd, struct sched_domain *parent)
/* Flags needing groups don't count if only 1 group in parent */
if (parent->groups == parent->groups->next)
- pflags &= ~(SD_DEGENERATE_GROUPS_MASK | SD_PREFER_SIBLING);
+ pflags &= ~SD_DEGENERATE_GROUPS_MASK;
if (~cflags & pflags)
return 0;
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [tip: sched/core] sched/topology: Remove SD_SERIALIZE degeneration special case
2020-08-17 11:29 ` [PATCH v6 09/17] sched/topology: Remove SD_SERIALIZE degeneration special case Valentin Schneider
@ 2020-08-19 14:02 ` tip-bot2 for Valentin Schneider
0 siblings, 0 replies; 34+ messages in thread
From: tip-bot2 for Valentin Schneider @ 2020-08-19 14:02 UTC (permalink / raw)
To: linux-tip-commits
Cc: Peter Zijlstra, Valentin Schneider, Ingo Molnar,
Dietmar Eggemann, Peter Zijlstra, x86, LKML
The following commit has been merged into the sched/core branch of tip:
Commit-ID: ab65afb094c7b2e954e8d56ffbc7df843211c2c8
Gitweb: https://git.kernel.org/tip/ab65afb094c7b2e954e8d56ffbc7df843211c2c8
Author: Valentin Schneider <valentin.schneider@arm.com>
AuthorDate: Mon, 17 Aug 2020 12:29:55 +01:00
Committer: Ingo Molnar <mingo@kernel.org>
CommitterDate: Wed, 19 Aug 2020 10:49:48 +02:00
sched/topology: Remove SD_SERIALIZE degeneration special case
If there is only a single NUMA node in the system, the only NUMA topology
level that will be generated will be NODE (identity distance), which
doesn't have SD_SERIALIZE.
This means we don't need this special case in sd_parent_degenerate(), as
having the NODE level "naturally" covers it. Thus, remove it.
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: https://lore.kernel.org/r/20200817113003.20802-10-valentin.schneider@arm.com
---
kernel/sched/topology.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 38e6671..c662c53 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -183,11 +183,9 @@ sd_parent_degenerate(struct sched_domain *sd, struct sched_domain *parent)
return 0;
/* Flags needing groups don't count if only 1 group in parent */
- if (parent->groups == parent->groups->next) {
+ if (parent->groups == parent->groups->next)
pflags &= ~(SD_DEGENERATE_GROUPS_MASK | SD_PREFER_SIBLING);
- if (nr_node_ids == 1)
- pflags &= ~SD_SERIALIZE;
- }
+
if (~cflags & pflags)
return 0;
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [tip: sched/core] sched/topology: Use prebuilt SD flag degeneration mask
2020-08-17 11:29 ` [PATCH v6 08/17] sched/topology: Use prebuilt SD flag degeneration mask Valentin Schneider
@ 2020-08-19 14:02 ` tip-bot2 for Valentin Schneider
0 siblings, 0 replies; 34+ messages in thread
From: tip-bot2 for Valentin Schneider @ 2020-08-19 14:02 UTC (permalink / raw)
To: linux-tip-commits
Cc: Peter Zijlstra, Valentin Schneider, Ingo Molnar, Peter Zijlstra,
x86, LKML
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 6f349818621dff364fc000909135d0065e9756c9
Gitweb: https://git.kernel.org/tip/6f349818621dff364fc000909135d0065e9756c9
Author: Valentin Schneider <valentin.schneider@arm.com>
AuthorDate: Mon, 17 Aug 2020 12:29:54 +01:00
Committer: Ingo Molnar <mingo@kernel.org>
CommitterDate: Wed, 19 Aug 2020 10:49:48 +02:00
sched/topology: Use prebuilt SD flag degeneration mask
Leverage SD_DEGENERATE_GROUPS_MASK in sd_degenerate() and
sd_parent_degenerate().
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: https://lore.kernel.org/r/20200817113003.20802-9-valentin.schneider@arm.com
---
kernel/sched/topology.c | 20 ++++----------------
1 file changed, 4 insertions(+), 16 deletions(-)
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index cbdaf08..38e6671 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -160,15 +160,9 @@ static int sd_degenerate(struct sched_domain *sd)
return 1;
/* Following flags need at least 2 groups */
- if (sd->flags & (SD_BALANCE_NEWIDLE |
- SD_BALANCE_FORK |
- SD_BALANCE_EXEC |
- SD_SHARE_CPUCAPACITY |
- SD_ASYM_CPUCAPACITY |
- SD_SHARE_PKG_RESOURCES)) {
- if (sd->groups != sd->groups->next)
- return 0;
- }
+ if ((sd->flags & SD_DEGENERATE_GROUPS_MASK) &&
+ (sd->groups != sd->groups->next))
+ return 0;
/* Following flags don't use groups */
if (sd->flags & (SD_WAKE_AFFINE))
@@ -190,13 +184,7 @@ sd_parent_degenerate(struct sched_domain *sd, struct sched_domain *parent)
/* Flags needing groups don't count if only 1 group in parent */
if (parent->groups == parent->groups->next) {
- pflags &= ~(SD_BALANCE_NEWIDLE |
- SD_BALANCE_FORK |
- SD_BALANCE_EXEC |
- SD_ASYM_CPUCAPACITY |
- SD_SHARE_CPUCAPACITY |
- SD_SHARE_PKG_RESOURCES |
- SD_PREFER_SIBLING);
+ pflags &= ~(SD_DEGENERATE_GROUPS_MASK | SD_PREFER_SIBLING);
if (nr_node_ids == 1)
pflags &= ~SD_SERIALIZE;
}
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [tip: sched/core] sched/topology: Introduce SD metaflag for flags needing > 1 groups
2020-08-17 11:29 ` [PATCH v6 07/17] sched/topology: Introduce SD metaflag for flags needing > 1 groups Valentin Schneider
@ 2020-08-19 14:02 ` tip-bot2 for Valentin Schneider
0 siblings, 0 replies; 34+ messages in thread
From: tip-bot2 for Valentin Schneider @ 2020-08-19 14:02 UTC (permalink / raw)
To: linux-tip-commits
Cc: Peter Zijlstra, Valentin Schneider, Ingo Molnar,
Dietmar Eggemann, Peter Zijlstra, x86, LKML
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 4ee4ea443a5dc3fc4d8ae338199676eae9d8ef02
Gitweb: https://git.kernel.org/tip/4ee4ea443a5dc3fc4d8ae338199676eae9d8ef02
Author: Valentin Schneider <valentin.schneider@arm.com>
AuthorDate: Mon, 17 Aug 2020 12:29:53 +01:00
Committer: Ingo Molnar <mingo@kernel.org>
CommitterDate: Wed, 19 Aug 2020 10:49:48 +02:00
sched/topology: Introduce SD metaflag for flags needing > 1 groups
In preparation of cleaning up the sd_degenerate*() functions, mark flags
used in sd_degenerate() with the new SDF_NEEDS_GROUPS flag. With this,
build a compile-time mask of those SD flags.
Note that sd_parent_degenerate() uses an extra flag in its mask,
SD_PREFER_SIBLING, which remains singled out for now.
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: https://lore.kernel.org/r/20200817113003.20802-8-valentin.schneider@arm.com
---
include/linux/sched/sd_flags.h | 39 +++++++++++++++++++++++----------
include/linux/sched/topology.h | 7 ++++++-
2 files changed, 35 insertions(+), 11 deletions(-)
diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h
index c24a45b..ee5cbfc 100644
--- a/include/linux/sched/sd_flags.h
+++ b/include/linux/sched/sd_flags.h
@@ -8,7 +8,7 @@
#endif
/*
- * Expected flag uses
+ * Hierarchical metaflags
*
* SHARED_CHILD: These flags are meant to be set from the base domain upwards.
* If a domain has this flag set, all of its children should have it set. This
@@ -29,29 +29,42 @@
* certain level (e.g. domain starts spanning CPUs outside of the base CPU's
* socket).
*/
-#define SDF_SHARED_CHILD 0x1
-#define SDF_SHARED_PARENT 0x2
+#define SDF_SHARED_CHILD 0x1
+#define SDF_SHARED_PARENT 0x2
+
+/*
+ * Behavioural metaflags
+ *
+ * NEEDS_GROUPS: These flags are only relevant if the domain they are set on has
+ * more than one group. This is usually for balancing flags (load balancing
+ * involves equalizing a metric between groups), or for flags describing some
+ * shared resource (which would be shared between groups).
+ */
+#define SDF_NEEDS_GROUPS 0x4
/*
* Balance when about to become idle
*
* SHARED_CHILD: Set from the base domain up to cpuset.sched_relax_domain_level.
+ * NEEDS_GROUPS: Load balancing flag.
*/
-SD_FLAG(SD_BALANCE_NEWIDLE, SDF_SHARED_CHILD)
+SD_FLAG(SD_BALANCE_NEWIDLE, SDF_SHARED_CHILD | SDF_NEEDS_GROUPS)
/*
* Balance on exec
*
* SHARED_CHILD: Set from the base domain up to the NUMA reclaim level.
+ * NEEDS_GROUPS: Load balancing flag.
*/
-SD_FLAG(SD_BALANCE_EXEC, SDF_SHARED_CHILD)
+SD_FLAG(SD_BALANCE_EXEC, SDF_SHARED_CHILD | SDF_NEEDS_GROUPS)
/*
* Balance on fork, clone
*
* SHARED_CHILD: Set from the base domain up to the NUMA reclaim level.
+ * NEEDS_GROUPS: Load balancing flag.
*/
-SD_FLAG(SD_BALANCE_FORK, SDF_SHARED_CHILD)
+SD_FLAG(SD_BALANCE_FORK, SDF_SHARED_CHILD | SDF_NEEDS_GROUPS)
/*
* Balance on wakeup
@@ -69,24 +82,28 @@ SD_FLAG(SD_WAKE_AFFINE, SDF_SHARED_CHILD)
/*
* Domain members have different CPU capacities
+ *
+ * NEEDS_GROUPS: Per-CPU capacity is asymmetric between groups.
*/
-SD_FLAG(SD_ASYM_CPUCAPACITY, 0)
+SD_FLAG(SD_ASYM_CPUCAPACITY, SDF_NEEDS_GROUPS)
/*
* Domain members share CPU capacity (i.e. SMT)
*
* SHARED_CHILD: Set from the base domain up until spanned CPUs no longer share
- * CPU capacity.
+ * CPU capacity.
+ * NEEDS_GROUPS: Capacity is shared between groups.
*/
-SD_FLAG(SD_SHARE_CPUCAPACITY, SDF_SHARED_CHILD)
+SD_FLAG(SD_SHARE_CPUCAPACITY, SDF_SHARED_CHILD | SDF_NEEDS_GROUPS)
/*
* Domain members share CPU package resources (i.e. caches)
*
* SHARED_CHILD: Set from the base domain up until spanned CPUs no longer share
- * the same cache(s).
+ * the same cache(s).
+ * NEEDS_GROUPS: Caches are shared between groups.
*/
-SD_FLAG(SD_SHARE_PKG_RESOURCES, SDF_SHARED_CHILD)
+SD_FLAG(SD_SHARE_PKG_RESOURCES, SDF_SHARED_CHILD | SDF_NEEDS_GROUPS)
/*
* Only a single load balancing instance
diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h
index 32f602f..2d59ca7 100644
--- a/include/linux/sched/topology.h
+++ b/include/linux/sched/topology.h
@@ -25,6 +25,13 @@ enum {
};
#undef SD_FLAG
+/* Generate a mask of SD flags with the SDF_NEEDS_GROUPS metaflag */
+#define SD_FLAG(name, mflags) (name * !!((mflags) & SDF_NEEDS_GROUPS)) |
+static const unsigned int SD_DEGENERATE_GROUPS_MASK =
+#include <linux/sched/sd_flags.h>
+0;
+#undef SD_FLAG
+
#ifdef CONFIG_SCHED_DEBUG
#define SD_FLAG(_name, mflags) [__##_name] = { .meta_flags = mflags, .name = #_name },
static const struct {
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [tip: sched/core] sched/topology: Verify SD_* flags setup when sched_debug is on
2020-08-17 11:29 ` [PATCH v6 05/17] sched/topology: Verify SD_* flags setup when sched_debug is on Valentin Schneider
@ 2020-08-19 14:02 ` tip-bot2 for Valentin Schneider
0 siblings, 0 replies; 34+ messages in thread
From: tip-bot2 for Valentin Schneider @ 2020-08-19 14:02 UTC (permalink / raw)
To: linux-tip-commits
Cc: Valentin Schneider, Ingo Molnar, Dietmar Eggemann,
Peter Zijlstra, x86, LKML
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 65c5e253168dbbb52c20026b0c5b7a2f344b9197
Gitweb: https://git.kernel.org/tip/65c5e253168dbbb52c20026b0c5b7a2f344b9197
Author: Valentin Schneider <valentin.schneider@arm.com>
AuthorDate: Mon, 17 Aug 2020 12:29:51 +01:00
Committer: Ingo Molnar <mingo@kernel.org>
CommitterDate: Wed, 19 Aug 2020 10:49:48 +02:00
sched/topology: Verify SD_* flags setup when sched_debug is on
Now that we have some description of what we expect the flags layout to
be, we can use that to assert at runtime that the actual layout is sane.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: https://lore.kernel.org/r/20200817113003.20802-6-valentin.schneider@arm.com
---
kernel/sched/topology.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index fe03969..cbdaf08 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -29,6 +29,8 @@ static int sched_domain_debug_one(struct sched_domain *sd, int cpu, int level,
struct cpumask *groupmask)
{
struct sched_group *group = sd->groups;
+ unsigned long flags = sd->flags;
+ unsigned int idx;
cpumask_clear(groupmask);
@@ -43,6 +45,21 @@ static int sched_domain_debug_one(struct sched_domain *sd, int cpu, int level,
printk(KERN_ERR "ERROR: domain->groups does not contain CPU%d\n", cpu);
}
+ for_each_set_bit(idx, &flags, __SD_FLAG_CNT) {
+ unsigned int flag = BIT(idx);
+ unsigned int meta_flags = sd_flag_debug[idx].meta_flags;
+
+ if ((meta_flags & SDF_SHARED_CHILD) && sd->child &&
+ !(sd->child->flags & flag))
+ printk(KERN_ERR "ERROR: flag %s set here but not in child\n",
+ sd_flag_debug[idx].name);
+
+ if ((meta_flags & SDF_SHARED_PARENT) && sd->parent &&
+ !(sd->parent->flags & flag))
+ printk(KERN_ERR "ERROR: flag %s set here but not in parent\n",
+ sd_flag_debug[idx].name);
+ }
+
printk(KERN_DEBUG "%*s groups:", level + 1, "");
do {
if (!group) {
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [tip: sched/core] sched/debug: Output SD flag names rather than their values
2020-08-17 11:29 ` [PATCH v6 06/17] sched/debug: Output SD flag names rather than their values Valentin Schneider
@ 2020-08-19 14:02 ` tip-bot2 for Valentin Schneider
0 siblings, 0 replies; 34+ messages in thread
From: tip-bot2 for Valentin Schneider @ 2020-08-19 14:02 UTC (permalink / raw)
To: linux-tip-commits
Cc: Valentin Schneider, Ingo Molnar, Peter Zijlstra, x86, LKML
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 5b9f8ff7b320a34af3dbcf04edb40d9b04f22f4a
Gitweb: https://git.kernel.org/tip/5b9f8ff7b320a34af3dbcf04edb40d9b04f22f4a
Author: Valentin Schneider <valentin.schneider@arm.com>
AuthorDate: Mon, 17 Aug 2020 12:29:52 +01:00
Committer: Ingo Molnar <mingo@kernel.org>
CommitterDate: Wed, 19 Aug 2020 10:49:48 +02:00
sched/debug: Output SD flag names rather than their values
Decoding the output of /proc/sys/kernel/sched_domain/cpu*/domain*/flags has
always been somewhat annoying, as one needs to go fetch the bit -> name
mapping from the source code itself. This encoding can be saved in a script
somewhere, but that isn't safe from flags being added, removed or even
shuffled around.
What matters for debugging purposes is to get *which* flags are set in a
given domain, their associated value is pretty much meaningless.
Make the sd flags debug file output flag names.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: https://lore.kernel.org/r/20200817113003.20802-7-valentin.schneider@arm.com
---
kernel/sched/debug.c | 56 ++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 55 insertions(+), 1 deletion(-)
diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c
index 36c5426..0655524 100644
--- a/kernel/sched/debug.c
+++ b/kernel/sched/debug.c
@@ -245,6 +245,60 @@ set_table_entry(struct ctl_table *entry,
entry->proc_handler = proc_handler;
}
+static int sd_ctl_doflags(struct ctl_table *table, int write,
+ void *buffer, size_t *lenp, loff_t *ppos)
+{
+ unsigned long flags = *(unsigned long *)table->data;
+ size_t data_size = 0;
+ size_t len = 0;
+ char *tmp;
+ int idx;
+
+ if (write)
+ return 0;
+
+ for_each_set_bit(idx, &flags, __SD_FLAG_CNT) {
+ char *name = sd_flag_debug[idx].name;
+
+ /* Name plus whitespace */
+ data_size += strlen(name) + 1;
+ }
+
+ if (*ppos > data_size) {
+ *lenp = 0;
+ return 0;
+ }
+
+ tmp = kcalloc(data_size + 1, sizeof(*tmp), GFP_KERNEL);
+ if (!tmp)
+ return -ENOMEM;
+
+ for_each_set_bit(idx, &flags, __SD_FLAG_CNT) {
+ char *name = sd_flag_debug[idx].name;
+
+ len += snprintf(tmp + len, strlen(name) + 2, "%s ", name);
+ }
+
+ tmp += *ppos;
+ len -= *ppos;
+
+ if (len > *lenp)
+ len = *lenp;
+ if (len)
+ memcpy(buffer, tmp, len);
+ if (len < *lenp) {
+ ((char *)buffer)[len] = '\n';
+ len++;
+ }
+
+ *lenp = len;
+ *ppos += len;
+
+ kfree(tmp);
+
+ return 0;
+}
+
static struct ctl_table *
sd_alloc_ctl_domain_table(struct sched_domain *sd)
{
@@ -258,7 +312,7 @@ sd_alloc_ctl_domain_table(struct sched_domain *sd)
set_table_entry(&table[2], "busy_factor", &sd->busy_factor, sizeof(int), 0644, proc_dointvec_minmax);
set_table_entry(&table[3], "imbalance_pct", &sd->imbalance_pct, sizeof(int), 0644, proc_dointvec_minmax);
set_table_entry(&table[4], "cache_nice_tries", &sd->cache_nice_tries, sizeof(int), 0644, proc_dointvec_minmax);
- set_table_entry(&table[5], "flags", &sd->flags, sizeof(int), 0444, proc_dointvec_minmax);
+ set_table_entry(&table[5], "flags", &sd->flags, sizeof(int), 0444, sd_ctl_doflags);
set_table_entry(&table[6], "max_newidle_lb_cost", &sd->max_newidle_lb_cost, sizeof(long), 0644, proc_doulongvec_minmax);
set_table_entry(&table[7], "name", sd->name, CORENAME_MAX_SIZE, 0444, proc_dostring);
/* &table[8] is terminator */
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [tip: sched/core] sched/topology: Define and assign sched_domain flag metadata
2020-08-17 11:29 ` [PATCH v6 04/17] sched/topology: Define and assign sched_domain flag metadata Valentin Schneider
@ 2020-08-19 14:02 ` tip-bot2 for Valentin Schneider
0 siblings, 0 replies; 34+ messages in thread
From: tip-bot2 for Valentin Schneider @ 2020-08-19 14:02 UTC (permalink / raw)
To: linux-tip-commits
Cc: Valentin Schneider, Ingo Molnar, Dietmar Eggemann,
Peter Zijlstra, x86, LKML
The following commit has been merged into the sched/core branch of tip:
Commit-ID: b6e862f386722e0de6c37f85f1cf438a0efa7f93
Gitweb: https://git.kernel.org/tip/b6e862f386722e0de6c37f85f1cf438a0efa7f93
Author: Valentin Schneider <valentin.schneider@arm.com>
AuthorDate: Mon, 17 Aug 2020 12:29:50 +01:00
Committer: Ingo Molnar <mingo@kernel.org>
CommitterDate: Wed, 19 Aug 2020 10:49:48 +02:00
sched/topology: Define and assign sched_domain flag metadata
There are some expectations regarding how sched domain flags should be laid
out, but none of them are checked or asserted in
sched_domain_debug_one(). After staring at said flags for a while, I've
come to realize there's two repeating patterns:
- Shared with children: those flags are set from the base CPU domain
upwards. Any domain that has it set will have it set in its children. It
hints at "some property holds true / some behaviour is enabled until this
level".
- Shared with parents: those flags are set from the topmost domain
downwards. Any domain that has it set will have it set in its parents. It
hints at "some property isn't visible / some behaviour is disabled until
this level".
There are two outliers that (currently) do not map to either of these:
o SD_PREFER_SIBLING, which is cleared below levels with
SD_ASYM_CPUCAPACITY. The change was introduced by commit:
9c63e84db29b ("sched/core: Disable SD_PREFER_SIBLING on asymmetric CPU capacity domains")
as it could break misfit migration on some systems. In light of this, we
might want to change it back to make it fit one of the two categories and
fix the issue another way.
o SD_ASYM_CPUCAPACITY, which gets set on a single level and isn't
propagated up nor down. From a topology description point of view, it
really wants to be SDF_SHARED_PARENT; this will be rectified in a later
patch.
Tweak the sched_domain flag declaration to assign each flag an expected
layout, and include the rationale for each flag "meta type" assignment as a
comment. Consolidate the flag metadata into an array; the index of a flag's
metadata can easily be found with log2(flag), IOW __ffs(flag).
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: https://lore.kernel.org/r/20200817113003.20802-5-valentin.schneider@arm.com
---
include/linux/sched/sd_flags.h | 147 ++++++++++++++++++++++++++------
include/linux/sched/topology.h | 15 ++-
2 files changed, 134 insertions(+), 28 deletions(-)
diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h
index 373dc45..c24a45b 100644
--- a/include/linux/sched/sd_flags.h
+++ b/include/linux/sched/sd_flags.h
@@ -7,29 +7,124 @@
# error "Incorrect import of SD flags definitions"
#endif
-/* Balance when about to become idle */
-SD_FLAG(SD_BALANCE_NEWIDLE)
-/* Balance on exec */
-SD_FLAG(SD_BALANCE_EXEC)
-/* Balance on fork, clone */
-SD_FLAG(SD_BALANCE_FORK)
-/* Balance on wakeup */
-SD_FLAG(SD_BALANCE_WAKE)
-/* Wake task to waking CPU */
-SD_FLAG(SD_WAKE_AFFINE)
-/* Domain members have different CPU capacities */
-SD_FLAG(SD_ASYM_CPUCAPACITY)
-/* Domain members share CPU capacity */
-SD_FLAG(SD_SHARE_CPUCAPACITY)
-/* Domain members share CPU pkg resources */
-SD_FLAG(SD_SHARE_PKG_RESOURCES)
-/* Only a single load balancing instance */
-SD_FLAG(SD_SERIALIZE)
-/* Place busy groups earlier in the domain */
-SD_FLAG(SD_ASYM_PACKING)
-/* Prefer to place tasks in a sibling domain */
-SD_FLAG(SD_PREFER_SIBLING)
-/* sched_domains of this level overlap */
-SD_FLAG(SD_OVERLAP)
-/* cross-node balancing */
-SD_FLAG(SD_NUMA)
+/*
+ * Expected flag uses
+ *
+ * SHARED_CHILD: These flags are meant to be set from the base domain upwards.
+ * If a domain has this flag set, all of its children should have it set. This
+ * is usually because the flag describes some shared resource (all CPUs in that
+ * domain share the same resource), or because they are tied to a scheduling
+ * behaviour that we want to disable at some point in the hierarchy for
+ * scalability reasons.
+ *
+ * In those cases it doesn't make sense to have the flag set for a domain but
+ * not have it in (some of) its children: sched domains ALWAYS span their child
+ * domains, so operations done with parent domains will cover CPUs in the lower
+ * child domains.
+ *
+ *
+ * SHARED_PARENT: These flags are meant to be set from the highest domain
+ * downwards. If a domain has this flag set, all of its parents should have it
+ * set. This is usually for topology properties that start to appear above a
+ * certain level (e.g. domain starts spanning CPUs outside of the base CPU's
+ * socket).
+ */
+#define SDF_SHARED_CHILD 0x1
+#define SDF_SHARED_PARENT 0x2
+
+/*
+ * Balance when about to become idle
+ *
+ * SHARED_CHILD: Set from the base domain up to cpuset.sched_relax_domain_level.
+ */
+SD_FLAG(SD_BALANCE_NEWIDLE, SDF_SHARED_CHILD)
+
+/*
+ * Balance on exec
+ *
+ * SHARED_CHILD: Set from the base domain up to the NUMA reclaim level.
+ */
+SD_FLAG(SD_BALANCE_EXEC, SDF_SHARED_CHILD)
+
+/*
+ * Balance on fork, clone
+ *
+ * SHARED_CHILD: Set from the base domain up to the NUMA reclaim level.
+ */
+SD_FLAG(SD_BALANCE_FORK, SDF_SHARED_CHILD)
+
+/*
+ * Balance on wakeup
+ *
+ * SHARED_CHILD: Set from the base domain up to cpuset.sched_relax_domain_level.
+ */
+SD_FLAG(SD_BALANCE_WAKE, SDF_SHARED_CHILD)
+
+/*
+ * Consider waking task on waking CPU.
+ *
+ * SHARED_CHILD: Set from the base domain up to the NUMA reclaim level.
+ */
+SD_FLAG(SD_WAKE_AFFINE, SDF_SHARED_CHILD)
+
+/*
+ * Domain members have different CPU capacities
+ */
+SD_FLAG(SD_ASYM_CPUCAPACITY, 0)
+
+/*
+ * Domain members share CPU capacity (i.e. SMT)
+ *
+ * SHARED_CHILD: Set from the base domain up until spanned CPUs no longer share
+ * CPU capacity.
+ */
+SD_FLAG(SD_SHARE_CPUCAPACITY, SDF_SHARED_CHILD)
+
+/*
+ * Domain members share CPU package resources (i.e. caches)
+ *
+ * SHARED_CHILD: Set from the base domain up until spanned CPUs no longer share
+ * the same cache(s).
+ */
+SD_FLAG(SD_SHARE_PKG_RESOURCES, SDF_SHARED_CHILD)
+
+/*
+ * Only a single load balancing instance
+ *
+ * SHARED_PARENT: Set for all NUMA levels above NODE. Could be set from a
+ * different level upwards, but it doesn't change that if a domain has this flag
+ * set, then all of its parents need to have it too (otherwise the serialization
+ * doesn't make sense).
+ */
+SD_FLAG(SD_SERIALIZE, SDF_SHARED_PARENT)
+
+/*
+ * Place busy tasks earlier in the domain
+ *
+ * SHARED_CHILD: Usually set on the SMT level. Technically could be set further
+ * up, but currently assumed to be set from the base domain upwards (see
+ * update_top_cache_domain()).
+ */
+SD_FLAG(SD_ASYM_PACKING, SDF_SHARED_CHILD)
+
+/*
+ * Prefer to place tasks in a sibling domain
+ *
+ * Set up until domains start spanning NUMA nodes. Close to being a SHARED_CHILD
+ * flag, but cleared below domains with SD_ASYM_CPUCAPACITY.
+ */
+SD_FLAG(SD_PREFER_SIBLING, 0)
+
+/*
+ * sched_groups of this level overlap
+ *
+ * SHARED_PARENT: Set for all NUMA levels above NODE.
+ */
+SD_FLAG(SD_OVERLAP, SDF_SHARED_PARENT)
+
+/*
+ * Cross-node balancing
+ *
+ * SHARED_PARENT: Set for all NUMA levels above NODE.
+ */
+SD_FLAG(SD_NUMA, SDF_SHARED_PARENT)
diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h
index 3e41c04..32f602f 100644
--- a/include/linux/sched/topology.h
+++ b/include/linux/sched/topology.h
@@ -12,19 +12,30 @@
#ifdef CONFIG_SMP
/* Generate SD flag indexes */
-#define SD_FLAG(name) __##name,
+#define SD_FLAG(name, mflags) __##name,
enum {
#include <linux/sched/sd_flags.h>
__SD_FLAG_CNT,
};
#undef SD_FLAG
/* Generate SD flag bits */
-#define SD_FLAG(name) name = 1 << __##name,
+#define SD_FLAG(name, mflags) name = 1 << __##name,
enum {
#include <linux/sched/sd_flags.h>
};
#undef SD_FLAG
+#ifdef CONFIG_SCHED_DEBUG
+#define SD_FLAG(_name, mflags) [__##_name] = { .meta_flags = mflags, .name = #_name },
+static const struct {
+ unsigned int meta_flags;
+ char *name;
+} sd_flag_debug[] = {
+#include <linux/sched/sd_flags.h>
+};
+#undef SD_FLAG
+#endif
+
#ifdef CONFIG_SCHED_SMT
static inline int cpu_smt_flags(void)
{
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [tip: sched/core] sched/topology: Split out SD_* flags declaration to its own file
2020-08-17 11:29 ` [PATCH v6 03/17] sched/topology: Split out SD_* flags declaration to its own file Valentin Schneider
@ 2020-08-19 14:02 ` tip-bot2 for Valentin Schneider
0 siblings, 0 replies; 34+ messages in thread
From: tip-bot2 for Valentin Schneider @ 2020-08-19 14:02 UTC (permalink / raw)
To: linux-tip-commits
Cc: Valentin Schneider, Ingo Molnar, Dietmar Eggemann,
Peter Zijlstra, x86, LKML
The following commit has been merged into the sched/core branch of tip:
Commit-ID: d54a9658a75633b839af7a2c6c758807678b8064
Gitweb: https://git.kernel.org/tip/d54a9658a75633b839af7a2c6c758807678b8064
Author: Valentin Schneider <valentin.schneider@arm.com>
AuthorDate: Mon, 17 Aug 2020 12:29:49 +01:00
Committer: Ingo Molnar <mingo@kernel.org>
CommitterDate: Wed, 19 Aug 2020 10:49:47 +02:00
sched/topology: Split out SD_* flags declaration to its own file
To associate the SD flags with some metadata, we need some more structure
in the way they are declared.
Rather than shove that in a free-standing macro list, move the declaration
in a separate file that can be re-imported with different SD_FLAG
definitions. This is inspired by what is done with the syscall
table (see uapi/asm/unistd.h and sys_call_table).
The value assigned to a given SD flag now depends on the order it appears
in sd_flags.h. No change in functionality.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: https://lore.kernel.org/r/20200817113003.20802-4-valentin.schneider@arm.com
---
include/linux/sched/sd_flags.h | 35 +++++++++++++++++++++++++++++++++-
include/linux/sched/topology.h | 26 ++++++++++++-------------
2 files changed, 48 insertions(+), 13 deletions(-)
create mode 100644 include/linux/sched/sd_flags.h
diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h
new file mode 100644
index 0000000..373dc45
--- /dev/null
+++ b/include/linux/sched/sd_flags.h
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * sched-domains (multiprocessor balancing) flag declarations.
+ */
+
+#ifndef SD_FLAG
+# error "Incorrect import of SD flags definitions"
+#endif
+
+/* Balance when about to become idle */
+SD_FLAG(SD_BALANCE_NEWIDLE)
+/* Balance on exec */
+SD_FLAG(SD_BALANCE_EXEC)
+/* Balance on fork, clone */
+SD_FLAG(SD_BALANCE_FORK)
+/* Balance on wakeup */
+SD_FLAG(SD_BALANCE_WAKE)
+/* Wake task to waking CPU */
+SD_FLAG(SD_WAKE_AFFINE)
+/* Domain members have different CPU capacities */
+SD_FLAG(SD_ASYM_CPUCAPACITY)
+/* Domain members share CPU capacity */
+SD_FLAG(SD_SHARE_CPUCAPACITY)
+/* Domain members share CPU pkg resources */
+SD_FLAG(SD_SHARE_PKG_RESOURCES)
+/* Only a single load balancing instance */
+SD_FLAG(SD_SERIALIZE)
+/* Place busy groups earlier in the domain */
+SD_FLAG(SD_ASYM_PACKING)
+/* Prefer to place tasks in a sibling domain */
+SD_FLAG(SD_PREFER_SIBLING)
+/* sched_domains of this level overlap */
+SD_FLAG(SD_OVERLAP)
+/* cross-node balancing */
+SD_FLAG(SD_NUMA)
diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h
index 6ec7d7c..3e41c04 100644
--- a/include/linux/sched/topology.h
+++ b/include/linux/sched/topology.h
@@ -11,19 +11,19 @@
*/
#ifdef CONFIG_SMP
-#define SD_BALANCE_NEWIDLE 0x0001 /* Balance when about to become idle */
-#define SD_BALANCE_EXEC 0x0002 /* Balance on exec */
-#define SD_BALANCE_FORK 0x0004 /* Balance on fork, clone */
-#define SD_BALANCE_WAKE 0x0008 /* Balance on wakeup */
-#define SD_WAKE_AFFINE 0x0010 /* Wake task to waking CPU */
-#define SD_ASYM_CPUCAPACITY 0x0020 /* Domain members have different CPU capacities */
-#define SD_SHARE_CPUCAPACITY 0x0040 /* Domain members share CPU capacity */
-#define SD_SHARE_PKG_RESOURCES 0x0080 /* Domain members share CPU pkg resources */
-#define SD_SERIALIZE 0x0100 /* Only a single load balancing instance */
-#define SD_ASYM_PACKING 0x0200 /* Place busy groups earlier in the domain */
-#define SD_PREFER_SIBLING 0x0400 /* Prefer to place tasks in a sibling domain */
-#define SD_OVERLAP 0x0800 /* sched_domains of this level overlap */
-#define SD_NUMA 0x1000 /* cross-node balancing */
+/* Generate SD flag indexes */
+#define SD_FLAG(name) __##name,
+enum {
+ #include <linux/sched/sd_flags.h>
+ __SD_FLAG_CNT,
+};
+#undef SD_FLAG
+/* Generate SD flag bits */
+#define SD_FLAG(name) name = 1 << __##name,
+enum {
+ #include <linux/sched/sd_flags.h>
+};
+#undef SD_FLAG
#ifdef CONFIG_SCHED_SMT
static inline int cpu_smt_flags(void)
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [tip: sched/core] ARM, sched/topology: Revert back to default scheduler topology
2020-08-17 11:29 ` [PATCH v6 02/17] ARM: Revert back to default scheduler topology Valentin Schneider
@ 2020-08-19 14:02 ` tip-bot2 for Valentin Schneider
0 siblings, 0 replies; 34+ messages in thread
From: tip-bot2 for Valentin Schneider @ 2020-08-19 14:02 UTC (permalink / raw)
To: linux-tip-commits
Cc: Dietmar Eggemann, Valentin Schneider, Ingo Molnar,
Peter Zijlstra, x86, LKML
The following commit has been merged into the sched/core branch of tip:
Commit-ID: d23b3bf8e43faa54a6839aafd6fedc15a3c4174f
Gitweb: https://git.kernel.org/tip/d23b3bf8e43faa54a6839aafd6fedc15a3c4174f
Author: Valentin Schneider <valentin.schneider@arm.com>
AuthorDate: Mon, 17 Aug 2020 12:29:48 +01:00
Committer: Ingo Molnar <mingo@kernel.org>
CommitterDate: Wed, 19 Aug 2020 10:49:47 +02:00
ARM, sched/topology: Revert back to default scheduler topology
The ARM-specific GMC level is meant to be built using the thread sibling
mask, but no devicetree in arch/arm/boot/dts uses the 'thread' cpu-map
binding. With SD_SHARE_POWERDOMAIN gone, this topology level can be
removed, at which point ARM no longer benefits from having a custom defined
topology table.
Delete the GMC topology level by making ARM use the default scheduler
topology table. This essentially reverts commit:
fb2aa85564f4 ("sched, ARM: Create a dedicated scheduler topology table")
No change in functionality is expected.
Suggested-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: https://lore.kernel.org/r/20200817113003.20802-3-valentin.schneider@arm.com
---
arch/arm/kernel/topology.c | 26 --------------------------
1 file changed, 26 deletions(-)
diff --git a/arch/arm/kernel/topology.c b/arch/arm/kernel/topology.c
index 353f3ee..ef0058d 100644
--- a/arch/arm/kernel/topology.c
+++ b/arch/arm/kernel/topology.c
@@ -178,15 +178,6 @@ static inline void update_cpu_capacity(unsigned int cpuid) {}
#endif
/*
- * The current assumption is that we can power gate each core independently.
- * This will be superseded by DT binding once available.
- */
-const struct cpumask *cpu_corepower_mask(int cpu)
-{
- return &cpu_topology[cpu].thread_sibling;
-}
-
-/*
* store_cpu_topology is called at boot when only one cpu is running
* and with the mutex cpu_hotplug.lock locked, when several cpus have booted,
* which prevents simultaneous write access to cpu_topology array
@@ -241,20 +232,6 @@ topology_populated:
update_siblings_masks(cpuid);
}
-static inline int cpu_corepower_flags(void)
-{
- return SD_SHARE_PKG_RESOURCES;
-}
-
-static struct sched_domain_topology_level arm_topology[] = {
-#ifdef CONFIG_SCHED_MC
- { cpu_corepower_mask, cpu_corepower_flags, SD_INIT_NAME(GMC) },
- { cpu_coregroup_mask, cpu_core_flags, SD_INIT_NAME(MC) },
-#endif
- { cpu_cpu_mask, SD_INIT_NAME(DIE) },
- { NULL, },
-};
-
/*
* init_cpu_topology is called at boot when only one cpu is running
* which prevent simultaneous write access to cpu_topology array
@@ -265,7 +242,4 @@ void __init init_cpu_topology(void)
smp_wmb();
parse_dt_topology();
-
- /* Set scheduler topology descriptor */
- set_sched_topology(arm_topology);
}
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [tip: sched/core] ARM, sched/topology: Remove SD_SHARE_POWERDOMAIN
2020-08-17 11:29 ` [PATCH v6 01/17] ARM, sched/topology: Remove SD_SHARE_POWERDOMAIN Valentin Schneider
@ 2020-08-19 14:02 ` tip-bot2 for Valentin Schneider
0 siblings, 0 replies; 34+ messages in thread
From: tip-bot2 for Valentin Schneider @ 2020-08-19 14:02 UTC (permalink / raw)
To: linux-tip-commits
Cc: Morten Rasmussen, Valentin Schneider, Ingo Molnar,
Dietmar Eggemann, Peter Zijlstra, x86, LKML
The following commit has been merged into the sched/core branch of tip:
Commit-ID: cfe7ddcbd72dc67ce5749cc6f451a2b0c6aec5b5
Gitweb: https://git.kernel.org/tip/cfe7ddcbd72dc67ce5749cc6f451a2b0c6aec5b5
Author: Valentin Schneider <valentin.schneider@arm.com>
AuthorDate: Mon, 17 Aug 2020 12:29:47 +01:00
Committer: Ingo Molnar <mingo@kernel.org>
CommitterDate: Wed, 19 Aug 2020 10:49:47 +02:00
ARM, sched/topology: Remove SD_SHARE_POWERDOMAIN
This flag was introduced in 2014 by commit:
d77b3ed5c9f8 ("sched: Add a new SD_SHARE_POWERDOMAIN for sched_domain")
but AFAIA it was never leveraged by the scheduler. The closest thing I can
think of is EAS caring about frequency domains, and it does that by
leveraging performance domains.
Remove the flag. No change in functionality is expected.
Suggested-by: Morten Rasmussen <morten.rasmussen@arm.com>
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: https://lore.kernel.org/r/20200817113003.20802-2-valentin.schneider@arm.com
---
arch/arm/kernel/topology.c | 2 +-
include/linux/sched/topology.h | 13 ++++++-------
kernel/sched/topology.c | 10 +++-------
3 files changed, 10 insertions(+), 15 deletions(-)
diff --git a/arch/arm/kernel/topology.c b/arch/arm/kernel/topology.c
index b5adaf7..353f3ee 100644
--- a/arch/arm/kernel/topology.c
+++ b/arch/arm/kernel/topology.c
@@ -243,7 +243,7 @@ topology_populated:
static inline int cpu_corepower_flags(void)
{
- return SD_SHARE_PKG_RESOURCES | SD_SHARE_POWERDOMAIN;
+ return SD_SHARE_PKG_RESOURCES;
}
static struct sched_domain_topology_level arm_topology[] = {
diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h
index 8205112..6ec7d7c 100644
--- a/include/linux/sched/topology.h
+++ b/include/linux/sched/topology.h
@@ -18,13 +18,12 @@
#define SD_WAKE_AFFINE 0x0010 /* Wake task to waking CPU */
#define SD_ASYM_CPUCAPACITY 0x0020 /* Domain members have different CPU capacities */
#define SD_SHARE_CPUCAPACITY 0x0040 /* Domain members share CPU capacity */
-#define SD_SHARE_POWERDOMAIN 0x0080 /* Domain members share power domain */
-#define SD_SHARE_PKG_RESOURCES 0x0100 /* Domain members share CPU pkg resources */
-#define SD_SERIALIZE 0x0200 /* Only a single load balancing instance */
-#define SD_ASYM_PACKING 0x0400 /* Place busy groups earlier in the domain */
-#define SD_PREFER_SIBLING 0x0800 /* Prefer to place tasks in a sibling domain */
-#define SD_OVERLAP 0x1000 /* sched_domains of this level overlap */
-#define SD_NUMA 0x2000 /* cross-node balancing */
+#define SD_SHARE_PKG_RESOURCES 0x0080 /* Domain members share CPU pkg resources */
+#define SD_SERIALIZE 0x0100 /* Only a single load balancing instance */
+#define SD_ASYM_PACKING 0x0200 /* Place busy groups earlier in the domain */
+#define SD_PREFER_SIBLING 0x0400 /* Prefer to place tasks in a sibling domain */
+#define SD_OVERLAP 0x0800 /* sched_domains of this level overlap */
+#define SD_NUMA 0x1000 /* cross-node balancing */
#ifdef CONFIG_SCHED_SMT
static inline int cpu_smt_flags(void)
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 007b0a6..fe03969 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -148,8 +148,7 @@ static int sd_degenerate(struct sched_domain *sd)
SD_BALANCE_EXEC |
SD_SHARE_CPUCAPACITY |
SD_ASYM_CPUCAPACITY |
- SD_SHARE_PKG_RESOURCES |
- SD_SHARE_POWERDOMAIN)) {
+ SD_SHARE_PKG_RESOURCES)) {
if (sd->groups != sd->groups->next)
return 0;
}
@@ -180,8 +179,7 @@ sd_parent_degenerate(struct sched_domain *sd, struct sched_domain *parent)
SD_ASYM_CPUCAPACITY |
SD_SHARE_CPUCAPACITY |
SD_SHARE_PKG_RESOURCES |
- SD_PREFER_SIBLING |
- SD_SHARE_POWERDOMAIN);
+ SD_PREFER_SIBLING);
if (nr_node_ids == 1)
pflags &= ~SD_SERIALIZE;
}
@@ -1292,7 +1290,6 @@ int __read_mostly node_reclaim_distance = RECLAIM_DISTANCE;
* SD_SHARE_CPUCAPACITY - describes SMT topologies
* SD_SHARE_PKG_RESOURCES - describes shared caches
* SD_NUMA - describes NUMA topologies
- * SD_SHARE_POWERDOMAIN - describes shared power domain
*
* Odd one out, which beside describing the topology has a quirk also
* prescribes the desired behaviour that goes along with it:
@@ -1303,8 +1300,7 @@ int __read_mostly node_reclaim_distance = RECLAIM_DISTANCE;
(SD_SHARE_CPUCAPACITY | \
SD_SHARE_PKG_RESOURCES | \
SD_NUMA | \
- SD_ASYM_PACKING | \
- SD_SHARE_POWERDOMAIN)
+ SD_ASYM_PACKING)
static struct sched_domain *
sd_init(struct sched_domain_topology_level *tl,
^ permalink raw reply related [flat|nested] 34+ messages in thread
end of thread, other threads:[~2020-08-19 14:07 UTC | newest]
Thread overview: 34+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-08-17 11:29 [PATCH v6 00/17] sched: Instrument sched domain flags Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 01/17] ARM, sched/topology: Remove SD_SHARE_POWERDOMAIN Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 02/17] ARM: Revert back to default scheduler topology Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] ARM, sched/topology: " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 03/17] sched/topology: Split out SD_* flags declaration to its own file Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 04/17] sched/topology: Define and assign sched_domain flag metadata Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 05/17] sched/topology: Verify SD_* flags setup when sched_debug is on Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 06/17] sched/debug: Output SD flag names rather than their values Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 07/17] sched/topology: Introduce SD metaflag for flags needing > 1 groups Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 08/17] sched/topology: Use prebuilt SD flag degeneration mask Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 09/17] sched/topology: Remove SD_SERIALIZE degeneration special case Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 10/17] sched/topology: Propagate SD_ASYM_CPUCAPACITY upwards Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 11/17] sched/topology: Mark SD_PREFER_SIBLING as SDF_NEEDS_GROUPS Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 12/17] sched/topology: Mark SD_BALANCE_WAKE " Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:29 ` [PATCH v6 13/17] sched/topology: Mark SD_SERIALIZE " Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:30 ` [PATCH v6 14/17] sched/topology: Mark SD_ASYM_PACKING " Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:30 ` [PATCH v6 15/17] sched/topology: Mark SD_OVERLAP " Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:30 ` [PATCH v6 16/17] sched/topology: Mark SD_NUMA " Valentin Schneider
2020-08-19 14:02 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-08-17 11:30 ` [PATCH v6 17/17] sched/topology: Expand use of SD_DEGENERATE_GROUPS_MASK to flags not needing groups Valentin Schneider
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).