* [PATCH 1/4] jump_label: make static_key_enabled() work on static_key_true/false types too
2015-09-16 1:51 [PATCHSET] cgroup: use static_keys for subsystem enabled and on_dfl tests Tejun Heo
@ 2015-09-16 1:51 ` Tejun Heo
2015-09-17 8:59 ` Peter Zijlstra
2015-09-16 1:51 ` [PATCH 2/4] cgroup: implement static_key based cgroup_subsys_enabled() and cgroup_subsys_on_dfl() Tejun Heo
` (4 subsequent siblings)
5 siblings, 1 reply; 10+ messages in thread
From: Tejun Heo @ 2015-09-16 1:51 UTC (permalink / raw)
To: lizefan, hannes
Cc: cgroups, linux-kernel, kernel-team, Tejun Heo, Peter Zijlstra,
Andrew Morton
static_key_enabled() can be used on struct static_key but not on its
wrapper types static_key_true and static_key_false. The function is
useful for debugging and management of static keys. Update it so that
it can be used for the wrapper types too.
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
---
Hello,
If this patch is acceptable, please let me know how it should be
routed.
Thanks.
include/linux/jump_label.h | 18 +++++++++++-------
1 file changed, 11 insertions(+), 7 deletions(-)
diff --git a/include/linux/jump_label.h b/include/linux/jump_label.h
index 7f653e8..c9ca050 100644
--- a/include/linux/jump_label.h
+++ b/include/linux/jump_label.h
@@ -216,11 +216,6 @@ static inline int jump_label_apply_nops(struct module *mod)
#define STATIC_KEY_INIT STATIC_KEY_INIT_FALSE
#define jump_label_enabled static_key_enabled
-static inline bool static_key_enabled(struct static_key *key)
-{
- return static_key_count(key) > 0;
-}
-
static inline void static_key_enable(struct static_key *key)
{
int count = static_key_count(key);
@@ -267,6 +262,17 @@ struct static_key_false {
#define DEFINE_STATIC_KEY_FALSE(name) \
struct static_key_false name = STATIC_KEY_FALSE_INIT
+extern bool ____wrong_branch_error(void);
+
+#define static_key_enabled(x) \
+({ \
+ if (!__builtin_types_compatible_p(typeof(*x), struct static_key) && \
+ !__builtin_types_compatible_p(typeof(*x), struct static_key_true) &&\
+ !__builtin_types_compatible_p(typeof(*x), struct static_key_false)) \
+ ____wrong_branch_error(); \
+ static_key_count((struct static_key *)x) > 0; \
+})
+
#ifdef HAVE_JUMP_LABEL
/*
@@ -325,8 +331,6 @@ struct static_key_false {
* See jump_label_type() / jump_label_init_type().
*/
-extern bool ____wrong_branch_error(void);
-
#define static_branch_likely(x) \
({ \
bool branch; \
--
2.4.3
^ permalink raw reply related [flat|nested] 10+ messages in thread* Re: [PATCH 1/4] jump_label: make static_key_enabled() work on static_key_true/false types too
2015-09-16 1:51 ` [PATCH 1/4] jump_label: make static_key_enabled() work on static_key_true/false types too Tejun Heo
@ 2015-09-17 8:59 ` Peter Zijlstra
2015-09-17 15:27 ` Tejun Heo
0 siblings, 1 reply; 10+ messages in thread
From: Peter Zijlstra @ 2015-09-17 8:59 UTC (permalink / raw)
To: Tejun Heo
Cc: lizefan, hannes, cgroups, linux-kernel, kernel-team, Andrew Morton
On Tue, Sep 15, 2015 at 09:51:22PM -0400, Tejun Heo wrote:
> static_key_enabled() can be used on struct static_key but not on its
> wrapper types static_key_true and static_key_false. The function is
> useful for debugging and management of static keys. Update it so that
> it can be used for the wrapper types too.
>
> Signed-off-by: Tejun Heo <tj@kernel.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> ---
> Hello,
>
> If this patch is acceptable, please let me know how it should be
> routed.
>
Yeah, no problem with this, I'm assuming there's dependencies in the
patches you didn't send me?
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 1/4] jump_label: make static_key_enabled() work on static_key_true/false types too
2015-09-17 8:59 ` Peter Zijlstra
@ 2015-09-17 15:27 ` Tejun Heo
2015-09-17 15:37 ` Peter Zijlstra
0 siblings, 1 reply; 10+ messages in thread
From: Tejun Heo @ 2015-09-17 15:27 UTC (permalink / raw)
To: Peter Zijlstra
Cc: lizefan, hannes, cgroups, linux-kernel, kernel-team, Andrew Morton
On Thu, Sep 17, 2015 at 10:59:57AM +0200, Peter Zijlstra wrote:
> On Tue, Sep 15, 2015 at 09:51:22PM -0400, Tejun Heo wrote:
> > static_key_enabled() can be used on struct static_key but not on its
> > wrapper types static_key_true and static_key_false. The function is
> > useful for debugging and management of static keys. Update it so that
> > it can be used for the wrapper types too.
> >
> > Signed-off-by: Tejun Heo <tj@kernel.org>
> > Cc: Peter Zijlstra <peterz@infradead.org>
> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > ---
> > Hello,
> >
> > If this patch is acceptable, please let me know how it should be
> > routed.
> >
>
> Yeah, no problem with this, I'm assuming there's dependencies in the
> patches you didn't send me?
Yeap, later patches add static_keys for testing whether a controller
is enabled and whether it's on the new or old hierarchy and management
path in cgroup core needs to index the keys and test them.
Thanks.
--
tejun
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 1/4] jump_label: make static_key_enabled() work on static_key_true/false types too
2015-09-17 15:27 ` Tejun Heo
@ 2015-09-17 15:37 ` Peter Zijlstra
0 siblings, 0 replies; 10+ messages in thread
From: Peter Zijlstra @ 2015-09-17 15:37 UTC (permalink / raw)
To: Tejun Heo
Cc: lizefan, hannes, cgroups, linux-kernel, kernel-team, Andrew Morton
On Thu, Sep 17, 2015 at 11:27:52AM -0400, Tejun Heo wrote:
> On Thu, Sep 17, 2015 at 10:59:57AM +0200, Peter Zijlstra wrote:
> > On Tue, Sep 15, 2015 at 09:51:22PM -0400, Tejun Heo wrote:
> > > static_key_enabled() can be used on struct static_key but not on its
> > > wrapper types static_key_true and static_key_false. The function is
> > > useful for debugging and management of static keys. Update it so that
> > > it can be used for the wrapper types too.
> > >
> > > Signed-off-by: Tejun Heo <tj@kernel.org>
> > > Cc: Peter Zijlstra <peterz@infradead.org>
> > > Cc: Andrew Morton <akpm@linux-foundation.org>
> > > ---
> > > Hello,
> > >
> > > If this patch is acceptable, please let me know how it should be
> > > routed.
> > >
> >
> > Yeah, no problem with this, I'm assuming there's dependencies in the
> > patches you didn't send me?
>
> Yeap, later patches add static_keys for testing whether a controller
> is enabled and whether it's on the new or old hierarchy and management
> path in cgroup core needs to index the keys and test them.
Ok, take it through you tree then, its a small enough patch.
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH 2/4] cgroup: implement static_key based cgroup_subsys_enabled() and cgroup_subsys_on_dfl()
2015-09-16 1:51 [PATCHSET] cgroup: use static_keys for subsystem enabled and on_dfl tests Tejun Heo
2015-09-16 1:51 ` [PATCH 1/4] jump_label: make static_key_enabled() work on static_key_true/false types too Tejun Heo
@ 2015-09-16 1:51 ` Tejun Heo
2015-09-16 1:51 ` [PATCH 3/4] cgroup: replace cgroup_subsys->disabled tests with cgroup_subsys_enabled() Tejun Heo
` (3 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Tejun Heo @ 2015-09-16 1:51 UTC (permalink / raw)
To: lizefan, hannes; +Cc: cgroups, linux-kernel, kernel-team, Tejun Heo
Whether a subsys is enabled and attached to the default hierarchy
seldom changes and may be tested in the hot paths. This patch
implements static_key based cgroup_subsys_enabled() and
cgroup_subsys_on_dfl() tests.
The following patches will update the users and remove duplicate
mechanisms.
Signed-off-by: Tejun Heo <tj@kernel.org>
---
include/linux/cgroup.h | 21 +++++++++++++++++++++
kernel/cgroup.c | 27 ++++++++++++++++++++++++++-
2 files changed, 47 insertions(+), 1 deletion(-)
diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h
index eb7ca55..c3a9f1e 100644
--- a/include/linux/cgroup.h
+++ b/include/linux/cgroup.h
@@ -17,6 +17,7 @@
#include <linux/fs.h>
#include <linux/seq_file.h>
#include <linux/kernfs.h>
+#include <linux/jump_label.h>
#include <linux/cgroup-defs.h>
@@ -50,6 +51,26 @@ extern struct css_set init_css_set;
#include <linux/cgroup_subsys.h>
#undef SUBSYS
+#define SUBSYS(_x) \
+ extern struct static_key_true _x ## _cgrp_subsys_enabled_key; \
+ extern struct static_key_true _x ## _cgrp_subsys_on_dfl_key;
+#include <linux/cgroup_subsys.h>
+#undef SUBSYS
+
+/**
+ * cgroup_subsys_enabled - fast test on whether a subsys is enabled
+ * @ss: subsystem in question
+ */
+#define cgroup_subsys_enabled(ss) \
+ static_branch_likely(&ss ## _enabled_key)
+
+/**
+ * cgroup_subsys_on_dfl - fast test on whether a subsys is on default hierarchy
+ * @ss: subsystem in question
+ */
+#define cgroup_subsys_on_dfl(ss) \
+ static_branch_likely(&ss ## _on_dfl_key)
+
bool css_has_online_children(struct cgroup_subsys_state *css);
struct cgroup_subsys_state *css_from_id(int id, struct cgroup_subsys *ss);
struct cgroup_subsys_state *cgroup_get_e_css(struct cgroup *cgroup,
diff --git a/kernel/cgroup.c b/kernel/cgroup.c
index 2cf0f79..3619389 100644
--- a/kernel/cgroup.c
+++ b/kernel/cgroup.c
@@ -139,6 +139,27 @@ static const char *cgroup_subsys_name[] = {
};
#undef SUBSYS
+/* array of static_keys for cgroup_subsys_enabled() and cgroup_subsys_on_dfl() */
+#define SUBSYS(_x) \
+ DEFINE_STATIC_KEY_TRUE(_x ## _cgrp_subsys_enabled_key); \
+ DEFINE_STATIC_KEY_TRUE(_x ## _cgrp_subsys_on_dfl_key); \
+ EXPORT_SYMBOL_GPL(_x ## _cgrp_subsys_enabled_key); \
+ EXPORT_SYMBOL_GPL(_x ## _cgrp_subsys_on_dfl_key);
+#include <linux/cgroup_subsys.h>
+#undef SUBSYS
+
+#define SUBSYS(_x) [_x ## _cgrp_id] = &_x ## _cgrp_subsys_enabled_key,
+static struct static_key_true *cgroup_subsys_enabled_key[] = {
+#include <linux/cgroup_subsys.h>
+};
+#undef SUBSYS
+
+#define SUBSYS(_x) [_x ## _cgrp_id] = &_x ## _cgrp_subsys_on_dfl_key,
+static struct static_key_true *cgroup_subsys_on_dfl_key[] = {
+#include <linux/cgroup_subsys.h>
+};
+#undef SUBSYS
+
/*
* The default hierarchy, reserved for the subsystems that are otherwise
* unattached - it never has more than a single cgroup, and all tasks are
@@ -1319,9 +1340,12 @@ static int rebind_subsystems(struct cgroup_root *dst_root,
/* default hierarchy doesn't enable controllers by default */
dst_root->subsys_mask |= 1 << ssid;
- if (dst_root != &cgrp_dfl_root) {
+ if (dst_root == &cgrp_dfl_root) {
+ static_branch_enable(cgroup_subsys_on_dfl_key[ssid]);
+ } else {
dst_root->cgrp.subtree_control |= 1 << ssid;
cgroup_refresh_child_subsys_mask(&dst_root->cgrp);
+ static_branch_disable(cgroup_subsys_on_dfl_key[ssid]);
}
if (ss->bind)
@@ -5483,6 +5507,7 @@ static int __init cgroup_disable(char *str)
strcmp(token, ss->legacy_name))
continue;
+ static_branch_disable(cgroup_subsys_enabled_key[i]);
ss->disabled = 1;
printk(KERN_INFO "Disabling %s control group subsystem\n",
ss->name);
--
2.4.3
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH 3/4] cgroup: replace cgroup_subsys->disabled tests with cgroup_subsys_enabled()
2015-09-16 1:51 [PATCHSET] cgroup: use static_keys for subsystem enabled and on_dfl tests Tejun Heo
2015-09-16 1:51 ` [PATCH 1/4] jump_label: make static_key_enabled() work on static_key_true/false types too Tejun Heo
2015-09-16 1:51 ` [PATCH 2/4] cgroup: implement static_key based cgroup_subsys_enabled() and cgroup_subsys_on_dfl() Tejun Heo
@ 2015-09-16 1:51 ` Tejun Heo
2015-09-16 1:51 ` [PATCH 4/4] cgroup: replace cgroup_on_dfl() tests in controllers with cgroup_subsys_on_dfl() Tejun Heo
` (2 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Tejun Heo @ 2015-09-16 1:51 UTC (permalink / raw)
To: lizefan, hannes
Cc: cgroups, linux-kernel, kernel-team, Tejun Heo, Michal Hocko
Replace cgroup_subsys->disabled tests in controllers with
cgroup_subsys_enabled(). cgroup_subsys_enabled() requires literal
subsys name as its parameter and thus can't be used for cgroup core
which iterates through controllers. For cgroup core, introduce and
use cgroup_ssid_enabled() which uses slower static_key_enabled() test
and can be indexed by subsys ID.
This leaves cgroup_subsys->disabled unused. Removed.
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Li Zefan <lizefan@huawei.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
---
include/linux/cgroup-defs.h | 1 -
include/linux/hugetlb_cgroup.h | 4 +---
include/linux/memcontrol.h | 4 +---
kernel/cgroup.c | 28 +++++++++++++++++++++-------
4 files changed, 23 insertions(+), 14 deletions(-)
diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h
index 4d8fcf2..c5d41c3 100644
--- a/include/linux/cgroup-defs.h
+++ b/include/linux/cgroup-defs.h
@@ -419,7 +419,6 @@ struct cgroup_subsys {
struct task_struct *task);
void (*bind)(struct cgroup_subsys_state *root_css);
- int disabled;
int early_init;
/*
diff --git a/include/linux/hugetlb_cgroup.h b/include/linux/hugetlb_cgroup.h
index bcc853e..7edd305 100644
--- a/include/linux/hugetlb_cgroup.h
+++ b/include/linux/hugetlb_cgroup.h
@@ -48,9 +48,7 @@ int set_hugetlb_cgroup(struct page *page, struct hugetlb_cgroup *h_cg)
static inline bool hugetlb_cgroup_disabled(void)
{
- if (hugetlb_cgrp_subsys.disabled)
- return true;
- return false;
+ return !cgroup_subsys_enabled(hugetlb_cgrp_subsys);
}
extern int hugetlb_cgroup_charge_cgroup(int idx, unsigned long nr_pages,
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index ad800e6..9aa7820 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -347,9 +347,7 @@ ino_t page_cgroup_ino(struct page *page);
static inline bool mem_cgroup_disabled(void)
{
- if (memory_cgrp_subsys.disabled)
- return true;
- return false;
+ return !cgroup_subsys_enabled(memory_cgrp_subsys);
}
/*
diff --git a/kernel/cgroup.c b/kernel/cgroup.c
index 3619389..5703ba7 100644
--- a/kernel/cgroup.c
+++ b/kernel/cgroup.c
@@ -224,6 +224,19 @@ static void kill_css(struct cgroup_subsys_state *css);
static int cgroup_addrm_files(struct cgroup *cgrp, struct cftype cfts[],
bool is_add);
+/**
+ * cgroup_ssid_enabled - cgroup subsys enabled test by subsys ID
+ * @ssid: subsys ID of interest
+ *
+ * cgroup_subsys_enabled() can only be used with literal subsys names which
+ * is fine for individual subsystems but unsuitable for cgroup core. This
+ * is slower static_key_enabled() based test indexed by @ssid.
+ */
+static bool cgroup_ssid_enabled(int ssid)
+{
+ return static_key_enabled(cgroup_subsys_enabled_key[ssid]);
+}
+
/* IDR wrappers which synchronize using cgroup_idr_lock */
static int cgroup_idr_alloc(struct idr *idr, void *ptr, int start, int end,
gfp_t gfp_mask)
@@ -1482,7 +1495,7 @@ static int parse_cgroupfs_options(char *data, struct cgroup_sb_opts *opts)
for_each_subsys(ss, i) {
if (strcmp(token, ss->legacy_name))
continue;
- if (ss->disabled)
+ if (!cgroup_ssid_enabled(i))
continue;
/* Mutually exclusive option 'all' + subsystem name */
@@ -1513,7 +1526,7 @@ static int parse_cgroupfs_options(char *data, struct cgroup_sb_opts *opts)
*/
if (all_ss || (!one_ss && !opts->none && !opts->name))
for_each_subsys(ss, i)
- if (!ss->disabled)
+ if (cgroup_ssid_enabled(i))
opts->subsys_mask |= (1 << i);
/*
@@ -2762,7 +2775,8 @@ static ssize_t cgroup_subtree_control_write(struct kernfs_open_file *of,
if (tok[0] == '\0')
continue;
for_each_subsys_which(ss, ssid, &tmp_ss_mask) {
- if (ss->disabled || strcmp(tok + 1, ss->name))
+ if (!cgroup_ssid_enabled(ssid) ||
+ strcmp(tok + 1, ss->name))
continue;
if (*tok == '+') {
@@ -3320,7 +3334,7 @@ static int cgroup_add_cftypes(struct cgroup_subsys *ss, struct cftype *cfts)
{
int ret;
- if (ss->disabled)
+ if (!cgroup_ssid_enabled(ss->id))
return 0;
if (!cfts || cfts[0].name[0] == '\0')
@@ -5082,7 +5096,7 @@ int __init cgroup_init(void)
* disabled flag and cftype registration needs kmalloc,
* both of which aren't available during early_init.
*/
- if (ss->disabled)
+ if (!cgroup_ssid_enabled(ssid))
continue;
cgrp_dfl_root.subsys_mask |= 1 << ss->id;
@@ -5217,7 +5231,8 @@ static int proc_cgroupstats_show(struct seq_file *m, void *v)
for_each_subsys(ss, i)
seq_printf(m, "%s\t%d\t%d\t%d\n",
ss->legacy_name, ss->root->hierarchy_id,
- atomic_read(&ss->root->nr_cgrps), !ss->disabled);
+ atomic_read(&ss->root->nr_cgrps),
+ cgroup_ssid_enabled(i));
mutex_unlock(&cgroup_mutex);
return 0;
@@ -5508,7 +5523,6 @@ static int __init cgroup_disable(char *str)
continue;
static_branch_disable(cgroup_subsys_enabled_key[i]);
- ss->disabled = 1;
printk(KERN_INFO "Disabling %s control group subsystem\n",
ss->name);
break;
--
2.4.3
^ permalink raw reply related [flat|nested] 10+ messages in thread* [PATCH 4/4] cgroup: replace cgroup_on_dfl() tests in controllers with cgroup_subsys_on_dfl()
2015-09-16 1:51 [PATCHSET] cgroup: use static_keys for subsystem enabled and on_dfl tests Tejun Heo
` (2 preceding siblings ...)
2015-09-16 1:51 ` [PATCH 3/4] cgroup: replace cgroup_subsys->disabled tests with cgroup_subsys_enabled() Tejun Heo
@ 2015-09-16 1:51 ` Tejun Heo
2015-09-18 9:14 ` [PATCHSET] cgroup: use static_keys for subsystem enabled and on_dfl tests Zefan Li
2015-09-18 15:55 ` Tejun Heo
5 siblings, 0 replies; 10+ messages in thread
From: Tejun Heo @ 2015-09-16 1:51 UTC (permalink / raw)
To: lizefan, hannes
Cc: cgroups, linux-kernel, kernel-team, Tejun Heo, Vivek Goyal,
Jens Axboe, Michal Hocko
cgroup_on_dfl() tests whether the cgroup's root is the default
hierarchy; however, an individual controller is only interested in
whether the controller is attached to the default hierarchy and never
tests a cgroup which doesn't belong to the hierarchy that the
controller is attached to.
This patch replaces cgroup_on_dfl() tests in controllers with faster
static_key based cgroup_subsys_on_dfl(). This leaves cgroup core as
the only user of cgroup_on_dfl() and the function is moved from the
header file to cgroup.c.
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Vivek Goyal <vgoyal@redhat.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Li Zefan <lizefan@huawei.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
---
block/blk-throttle.c | 2 +-
block/cfq-iosched.c | 4 ++--
include/linux/cgroup.h | 58 --------------------------------------------------
kernel/cgroup.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++
kernel/cpuset.c | 23 +++++++++++---------
mm/memcontrol.c | 4 ++--
6 files changed, 76 insertions(+), 73 deletions(-)
diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index c75a263..2149a1d 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -369,7 +369,7 @@ static void throtl_pd_init(struct blkg_policy_data *pd)
* regardless of the position of the group in the hierarchy.
*/
sq->parent_sq = &td->service_queue;
- if (cgroup_on_dfl(blkg->blkcg->css.cgroup) && blkg->parent)
+ if (cgroup_subsys_on_dfl(io_cgrp_subsys) && blkg->parent)
sq->parent_sq = &blkg_to_tg(blkg->parent)->service_queue;
tg->td = td;
}
diff --git a/block/cfq-iosched.c b/block/cfq-iosched.c
index 04de884..1f9093e 100644
--- a/block/cfq-iosched.c
+++ b/block/cfq-iosched.c
@@ -1581,7 +1581,7 @@ static struct blkcg_policy_data *cfq_cpd_alloc(gfp_t gfp)
static void cfq_cpd_init(struct blkcg_policy_data *cpd)
{
struct cfq_group_data *cgd = cpd_to_cfqgd(cpd);
- unsigned int weight = cgroup_on_dfl(blkcg_root.css.cgroup) ?
+ unsigned int weight = cgroup_subsys_on_dfl(io_cgrp_subsys) ?
CGROUP_WEIGHT_DFL : CFQ_WEIGHT_LEGACY_DFL;
if (cpd_to_blkcg(cpd) == &blkcg_root)
@@ -1599,7 +1599,7 @@ static void cfq_cpd_free(struct blkcg_policy_data *cpd)
static void cfq_cpd_bind(struct blkcg_policy_data *cpd)
{
struct blkcg *blkcg = cpd_to_blkcg(cpd);
- bool on_dfl = cgroup_on_dfl(blkcg_root.css.cgroup);
+ bool on_dfl = cgroup_subsys_on_dfl(io_cgrp_subsys);
unsigned int weight = on_dfl ? CGROUP_WEIGHT_DFL : CFQ_WEIGHT_LEGACY_DFL;
if (blkcg == &blkcg_root)
diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h
index c3a9f1e..355bf2e 100644
--- a/include/linux/cgroup.h
+++ b/include/linux/cgroup.h
@@ -433,64 +433,6 @@ static inline struct cgroup *task_cgroup(struct task_struct *task,
return task_css(task, subsys_id)->cgroup;
}
-/**
- * cgroup_on_dfl - test whether a cgroup is on the default hierarchy
- * @cgrp: the cgroup of interest
- *
- * The default hierarchy is the v2 interface of cgroup and this function
- * can be used to test whether a cgroup is on the default hierarchy for
- * cases where a subsystem should behave differnetly depending on the
- * interface version.
- *
- * The set of behaviors which change on the default hierarchy are still
- * being determined and the mount option is prefixed with __DEVEL__.
- *
- * List of changed behaviors:
- *
- * - Mount options "noprefix", "xattr", "clone_children", "release_agent"
- * and "name" are disallowed.
- *
- * - When mounting an existing superblock, mount options should match.
- *
- * - Remount is disallowed.
- *
- * - rename(2) is disallowed.
- *
- * - "tasks" is removed. Everything should be at process granularity. Use
- * "cgroup.procs" instead.
- *
- * - "cgroup.procs" is not sorted. pids will be unique unless they got
- * recycled inbetween reads.
- *
- * - "release_agent" and "notify_on_release" are removed. Replacement
- * notification mechanism will be implemented.
- *
- * - "cgroup.clone_children" is removed.
- *
- * - "cgroup.subtree_populated" is available. Its value is 0 if the cgroup
- * and its descendants contain no task; otherwise, 1. The file also
- * generates kernfs notification which can be monitored through poll and
- * [di]notify when the value of the file changes.
- *
- * - cpuset: tasks will be kept in empty cpusets when hotplug happens and
- * take masks of ancestors with non-empty cpus/mems, instead of being
- * moved to an ancestor.
- *
- * - cpuset: a task can be moved into an empty cpuset, and again it takes
- * masks of ancestors.
- *
- * - memcg: use_hierarchy is on by default and the cgroup file for the flag
- * is not created.
- *
- * - blkcg: blk-throttle becomes properly hierarchical.
- *
- * - debug: disallowed on the default hierarchy.
- */
-static inline bool cgroup_on_dfl(const struct cgroup *cgrp)
-{
- return cgrp->root == &cgrp_dfl_root;
-}
-
/* no synchronization, the result can only be used as a hint */
static inline bool cgroup_has_tasks(struct cgroup *cgrp)
{
diff --git a/kernel/cgroup.c b/kernel/cgroup.c
index 5703ba7..c24f929 100644
--- a/kernel/cgroup.c
+++ b/kernel/cgroup.c
@@ -237,6 +237,64 @@ static bool cgroup_ssid_enabled(int ssid)
return static_key_enabled(cgroup_subsys_enabled_key[ssid]);
}
+/**
+ * cgroup_on_dfl - test whether a cgroup is on the default hierarchy
+ * @cgrp: the cgroup of interest
+ *
+ * The default hierarchy is the v2 interface of cgroup and this function
+ * can be used to test whether a cgroup is on the default hierarchy for
+ * cases where a subsystem should behave differnetly depending on the
+ * interface version.
+ *
+ * The set of behaviors which change on the default hierarchy are still
+ * being determined and the mount option is prefixed with __DEVEL__.
+ *
+ * List of changed behaviors:
+ *
+ * - Mount options "noprefix", "xattr", "clone_children", "release_agent"
+ * and "name" are disallowed.
+ *
+ * - When mounting an existing superblock, mount options should match.
+ *
+ * - Remount is disallowed.
+ *
+ * - rename(2) is disallowed.
+ *
+ * - "tasks" is removed. Everything should be at process granularity. Use
+ * "cgroup.procs" instead.
+ *
+ * - "cgroup.procs" is not sorted. pids will be unique unless they got
+ * recycled inbetween reads.
+ *
+ * - "release_agent" and "notify_on_release" are removed. Replacement
+ * notification mechanism will be implemented.
+ *
+ * - "cgroup.clone_children" is removed.
+ *
+ * - "cgroup.subtree_populated" is available. Its value is 0 if the cgroup
+ * and its descendants contain no task; otherwise, 1. The file also
+ * generates kernfs notification which can be monitored through poll and
+ * [di]notify when the value of the file changes.
+ *
+ * - cpuset: tasks will be kept in empty cpusets when hotplug happens and
+ * take masks of ancestors with non-empty cpus/mems, instead of being
+ * moved to an ancestor.
+ *
+ * - cpuset: a task can be moved into an empty cpuset, and again it takes
+ * masks of ancestors.
+ *
+ * - memcg: use_hierarchy is on by default and the cgroup file for the flag
+ * is not created.
+ *
+ * - blkcg: blk-throttle becomes properly hierarchical.
+ *
+ * - debug: disallowed on the default hierarchy.
+ */
+static bool cgroup_on_dfl(const struct cgroup *cgrp)
+{
+ return cgrp->root == &cgrp_dfl_root;
+}
+
/* IDR wrappers which synchronize using cgroup_idr_lock */
static int cgroup_idr_alloc(struct idr *idr, void *ptr, int start, int end,
gfp_t gfp_mask)
diff --git a/kernel/cpuset.c b/kernel/cpuset.c
index f0acff0..20eedd8 100644
--- a/kernel/cpuset.c
+++ b/kernel/cpuset.c
@@ -473,7 +473,8 @@ static int validate_change(struct cpuset *cur, struct cpuset *trial)
/* On legacy hiearchy, we must be a subset of our parent cpuset. */
ret = -EACCES;
- if (!cgroup_on_dfl(cur->css.cgroup) && !is_cpuset_subset(trial, par))
+ if (!cgroup_subsys_on_dfl(cpuset_cgrp_subsys) &&
+ !is_cpuset_subset(trial, par))
goto out;
/*
@@ -879,7 +880,8 @@ static void update_cpumasks_hier(struct cpuset *cs, struct cpumask *new_cpus)
* If it becomes empty, inherit the effective mask of the
* parent, which is guaranteed to have some CPUs.
*/
- if (cgroup_on_dfl(cp->css.cgroup) && cpumask_empty(new_cpus))
+ if (cgroup_subsys_on_dfl(cpuset_cgrp_subsys) &&
+ cpumask_empty(new_cpus))
cpumask_copy(new_cpus, parent->effective_cpus);
/* Skip the whole subtree if the cpumask remains the same. */
@@ -896,7 +898,7 @@ static void update_cpumasks_hier(struct cpuset *cs, struct cpumask *new_cpus)
cpumask_copy(cp->effective_cpus, new_cpus);
spin_unlock_irq(&callback_lock);
- WARN_ON(!cgroup_on_dfl(cp->css.cgroup) &&
+ WARN_ON(!cgroup_subsys_on_dfl(cpuset_cgrp_subsys) &&
!cpumask_equal(cp->cpus_allowed, cp->effective_cpus));
update_tasks_cpumask(cp);
@@ -1135,7 +1137,8 @@ static void update_nodemasks_hier(struct cpuset *cs, nodemask_t *new_mems)
* If it becomes empty, inherit the effective mask of the
* parent, which is guaranteed to have some MEMs.
*/
- if (cgroup_on_dfl(cp->css.cgroup) && nodes_empty(*new_mems))
+ if (cgroup_subsys_on_dfl(cpuset_cgrp_subsys) &&
+ nodes_empty(*new_mems))
*new_mems = parent->effective_mems;
/* Skip the whole subtree if the nodemask remains the same. */
@@ -1152,7 +1155,7 @@ static void update_nodemasks_hier(struct cpuset *cs, nodemask_t *new_mems)
cp->effective_mems = *new_mems;
spin_unlock_irq(&callback_lock);
- WARN_ON(!cgroup_on_dfl(cp->css.cgroup) &&
+ WARN_ON(!cgroup_subsys_on_dfl(cpuset_cgrp_subsys) &&
!nodes_equal(cp->mems_allowed, cp->effective_mems));
update_tasks_nodemask(cp);
@@ -1440,7 +1443,7 @@ static int cpuset_can_attach(struct cgroup_subsys_state *css,
/* allow moving tasks into an empty cpuset if on default hierarchy */
ret = -ENOSPC;
- if (!cgroup_on_dfl(css->cgroup) &&
+ if (!cgroup_subsys_on_dfl(cpuset_cgrp_subsys) &&
(cpumask_empty(cs->cpus_allowed) || nodes_empty(cs->mems_allowed)))
goto out_unlock;
@@ -1952,7 +1955,7 @@ static int cpuset_css_online(struct cgroup_subsys_state *css)
cpuset_inc();
spin_lock_irq(&callback_lock);
- if (cgroup_on_dfl(cs->css.cgroup)) {
+ if (cgroup_subsys_on_dfl(cpuset_cgrp_subsys)) {
cpumask_copy(cs->effective_cpus, parent->effective_cpus);
cs->effective_mems = parent->effective_mems;
}
@@ -2029,7 +2032,7 @@ static void cpuset_bind(struct cgroup_subsys_state *root_css)
mutex_lock(&cpuset_mutex);
spin_lock_irq(&callback_lock);
- if (cgroup_on_dfl(root_css->cgroup)) {
+ if (cgroup_subsys_on_dfl(cpuset_cgrp_subsys)) {
cpumask_copy(top_cpuset.cpus_allowed, cpu_possible_mask);
top_cpuset.mems_allowed = node_possible_map;
} else {
@@ -2210,7 +2213,7 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs)
cpus_updated = !cpumask_equal(&new_cpus, cs->effective_cpus);
mems_updated = !nodes_equal(new_mems, cs->effective_mems);
- if (cgroup_on_dfl(cs->css.cgroup))
+ if (cgroup_subsys_on_dfl(cpuset_cgrp_subsys))
hotplug_update_tasks(cs, &new_cpus, &new_mems,
cpus_updated, mems_updated);
else
@@ -2241,7 +2244,7 @@ static void cpuset_hotplug_workfn(struct work_struct *work)
static cpumask_t new_cpus;
static nodemask_t new_mems;
bool cpus_updated, mems_updated;
- bool on_dfl = cgroup_on_dfl(top_cpuset.css.cgroup);
+ bool on_dfl = cgroup_subsys_on_dfl(cpuset_cgrp_subsys);
mutex_lock(&cpuset_mutex);
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 6ddaeba..b35c4cc 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -434,7 +434,7 @@ struct cgroup_subsys_state *mem_cgroup_css_from_page(struct page *page)
memcg = page->mem_cgroup;
- if (!memcg || !cgroup_on_dfl(memcg->css.cgroup))
+ if (!memcg || !cgroup_subsys_on_dfl(memory_cgrp_subsys))
memcg = root_mem_cgroup;
rcu_read_unlock();
@@ -5059,7 +5059,7 @@ static void mem_cgroup_bind(struct cgroup_subsys_state *root_css)
* guarantees that @root doesn't have any children, so turning it
* on for the root memcg is enough.
*/
- if (cgroup_on_dfl(root_css->cgroup))
+ if (cgroup_subsys_on_dfl(memory_cgrp_subsys))
root_mem_cgroup->use_hierarchy = true;
else
root_mem_cgroup->use_hierarchy = false;
--
2.4.3
^ permalink raw reply related [flat|nested] 10+ messages in thread* Re: [PATCHSET] cgroup: use static_keys for subsystem enabled and on_dfl tests
2015-09-16 1:51 [PATCHSET] cgroup: use static_keys for subsystem enabled and on_dfl tests Tejun Heo
` (3 preceding siblings ...)
2015-09-16 1:51 ` [PATCH 4/4] cgroup: replace cgroup_on_dfl() tests in controllers with cgroup_subsys_on_dfl() Tejun Heo
@ 2015-09-18 9:14 ` Zefan Li
2015-09-18 15:55 ` Tejun Heo
5 siblings, 0 replies; 10+ messages in thread
From: Zefan Li @ 2015-09-18 9:14 UTC (permalink / raw)
To: Tejun Heo, hannes; +Cc: cgroups, linux-kernel, kernel-team
On 2015/9/16 9:51, Tejun Heo wrote:
> cgroup_subsys->disabled and cgroup_on_dfl() tests are likely to be
> used in hot paths and seldom change. The former is set once during
> boot and the latter only when a controller is migrated between the
> default hierarchy and traditional ones.
>
> This patchset makes these tests static_key based and contains the
> following four patches.
>
> 0001-jump_label-make-static_key_enabled-work-on-static_ke.patch
> 0002-cgroup-implement-static_key-based-cgroup_subsys_enab.patch
> 0003-cgroup-replace-cgroup_subsys-disabled-tests-with-cgr.patch
> 0004-cgroup-replace-cgroup_on_dfl-tests-in-controllers-wi.patch
>
> 0001 is a prep patch in jump_label. 0002 adds the needed static_keys.
> 0003-0004 convert the existing usages and drop the old tests.
>
> This patchset is on top of v4.3-rc1 and is availalbe in the following
> git branch.
>
> git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git review-jump-labels
>
> diffstat follows. Thanks.
>
> block/blk-throttle.c | 2
> block/cfq-iosched.c | 4 -
> include/linux/cgroup-defs.h | 1
> include/linux/cgroup.h | 79 +++++++---------------------
> include/linux/hugetlb_cgroup.h | 4 -
> include/linux/jump_label.h | 18 +++---
> include/linux/memcontrol.h | 4 -
> kernel/cgroup.c | 113 ++++++++++++++++++++++++++++++++++++++---
> kernel/cpuset.c | 23 ++++----
> mm/memcontrol.c | 4 -
> 10 files changed, 157 insertions(+), 95 deletions(-)
>
Acked-by: Zefan Li <lizefan@huawei.com>
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [PATCHSET] cgroup: use static_keys for subsystem enabled and on_dfl tests
2015-09-16 1:51 [PATCHSET] cgroup: use static_keys for subsystem enabled and on_dfl tests Tejun Heo
` (4 preceding siblings ...)
2015-09-18 9:14 ` [PATCHSET] cgroup: use static_keys for subsystem enabled and on_dfl tests Zefan Li
@ 2015-09-18 15:55 ` Tejun Heo
5 siblings, 0 replies; 10+ messages in thread
From: Tejun Heo @ 2015-09-18 15:55 UTC (permalink / raw)
To: lizefan, hannes; +Cc: cgroups, linux-kernel, kernel-team
On Tue, Sep 15, 2015 at 09:51:21PM -0400, Tejun Heo wrote:
> cgroup_subsys->disabled and cgroup_on_dfl() tests are likely to be
> used in hot paths and seldom change. The former is set once during
> boot and the latter only when a controller is migrated between the
> default hierarchy and traditional ones.
Applying to cgroup/for-4.4.
Thanks.
--
tejun
^ permalink raw reply [flat|nested] 10+ messages in thread