* [PATCH v3 0/7] freezer for cgroup v2
@ 2018-11-17 0:38 Roman Gushchin
2018-11-17 0:38 ` [PATCH v3 1/7] cgroup: rename freezer.c into legacy_freezer.c Roman Gushchin
` (6 more replies)
0 siblings, 7 replies; 23+ messages in thread
From: Roman Gushchin @ 2018-11-17 0:38 UTC (permalink / raw)
To: Tejun Heo
Cc: Oleg Nesterov, cgroups, linux-kernel, kernel-team, Roman Gushchin
This patchset implements freezer for cgroup v2.
It provides similar functionality as v1 freezer, but the interface
conforms to the cgroup v2 interface design principles, and it
provides a better user experience: tasks can be killed, ptrace works,
there is no separate controller, which has to be enabled, etc.
Patches (1), (2) and (3) are some preparational work, patch (4) contains
the implementation, patch (5) is a small cgroup kselftest fix,
patch (6) covers freezer adds 6 new kselftests to cover the freezer
functionality. Patch (7) adds corresponding docs.
v3->v2:
- dropped TASK_FROZEN for now, frozen tasks are put into TASK_INTERRUPTIBLE
state; it's probably not the final version, but the API question can be
discussed separately
- don't clear TIF_SIGPENDING before going to sleep, instead add
task->frozen check in signal_pending_state() and recalc_sigpending()
- cgroup-level counter are now synchronized using css_set_lock,
which simplified the whole code (e.g. per-cgroup works were removed)
- the amount of comments increased significantly
- many other improvements incorporating feedback from Tejun and Oleg
v2->v1:
- fixed locking aroung calling cgroup_freezer_leave()
- added docs
Roman Gushchin (7):
cgroup: rename freezer.c into legacy_freezer.c
cgroup: implement __cgroup_task_count() helper
cgroup: protect cgroup->nr_(dying_)descendants by css_set_lock
cgroup: cgroup v2 freezer
kselftests: cgroup: don't fail on cg_kill_all() error in cg_destroy()
kselftests: cgroup: add freezer controller self-tests
cgroup: document cgroup v2 freezer interface
Documentation/admin-guide/cgroup-v2.rst | 26 +
include/linux/cgroup-defs.h | 31 +
include/linux/cgroup.h | 42 ++
include/linux/sched.h | 2 +
include/linux/sched/jobctl.h | 2 +
include/linux/sched/signal.h | 2 +
kernel/cgroup/Makefile | 4 +-
kernel/cgroup/cgroup-internal.h | 1 +
kernel/cgroup/cgroup-v1.c | 16 -
kernel/cgroup/cgroup.c | 165 ++++-
kernel/cgroup/freezer.c | 641 ++++++----------
kernel/cgroup/legacy_freezer.c | 481 ++++++++++++
kernel/ptrace.c | 7 +
kernel/signal.c | 57 +-
tools/testing/selftests/cgroup/.gitignore | 1 +
tools/testing/selftests/cgroup/Makefile | 2 +
tools/testing/selftests/cgroup/cgroup_util.c | 85 ++-
tools/testing/selftests/cgroup/cgroup_util.h | 7 +
tools/testing/selftests/cgroup/test_freezer.c | 685 ++++++++++++++++++
19 files changed, 1804 insertions(+), 453 deletions(-)
create mode 100644 kernel/cgroup/legacy_freezer.c
create mode 100644 tools/testing/selftests/cgroup/test_freezer.c
--
2.17.2
^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH v3 1/7] cgroup: rename freezer.c into legacy_freezer.c
2018-11-17 0:38 [PATCH v3 0/7] freezer for cgroup v2 Roman Gushchin
@ 2018-11-17 0:38 ` Roman Gushchin
2018-11-17 0:38 ` [PATCH v3 2/7] cgroup: implement __cgroup_task_count() helper Roman Gushchin
` (5 subsequent siblings)
6 siblings, 0 replies; 23+ messages in thread
From: Roman Gushchin @ 2018-11-17 0:38 UTC (permalink / raw)
To: Tejun Heo
Cc: Oleg Nesterov, cgroups, linux-kernel, kernel-team, Roman Gushchin
Freezer.c will contain an implementation of cgroup v2 freezer,
so let's rename the v1 freezer to avoid naming conflicts.
Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: kernel-team@fb.com
---
kernel/cgroup/Makefile | 2 +-
kernel/cgroup/{freezer.c => legacy_freezer.c} | 0
2 files changed, 1 insertion(+), 1 deletion(-)
rename kernel/cgroup/{freezer.c => legacy_freezer.c} (100%)
diff --git a/kernel/cgroup/Makefile b/kernel/cgroup/Makefile
index bfcdae896122..8d5689ca94b9 100644
--- a/kernel/cgroup/Makefile
+++ b/kernel/cgroup/Makefile
@@ -1,7 +1,7 @@
# SPDX-License-Identifier: GPL-2.0
obj-y := cgroup.o rstat.o namespace.o cgroup-v1.o
-obj-$(CONFIG_CGROUP_FREEZER) += freezer.o
+obj-$(CONFIG_CGROUP_FREEZER) += legacy_freezer.o
obj-$(CONFIG_CGROUP_PIDS) += pids.o
obj-$(CONFIG_CGROUP_RDMA) += rdma.o
obj-$(CONFIG_CPUSETS) += cpuset.o
diff --git a/kernel/cgroup/freezer.c b/kernel/cgroup/legacy_freezer.c
similarity index 100%
rename from kernel/cgroup/freezer.c
rename to kernel/cgroup/legacy_freezer.c
--
2.17.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v3 2/7] cgroup: implement __cgroup_task_count() helper
2018-11-17 0:38 [PATCH v3 0/7] freezer for cgroup v2 Roman Gushchin
2018-11-17 0:38 ` [PATCH v3 1/7] cgroup: rename freezer.c into legacy_freezer.c Roman Gushchin
@ 2018-11-17 0:38 ` Roman Gushchin
2018-11-17 0:38 ` [PATCH v3 3/7] cgroup: protect cgroup->nr_(dying_)descendants by css_set_lock Roman Gushchin
` (4 subsequent siblings)
6 siblings, 0 replies; 23+ messages in thread
From: Roman Gushchin @ 2018-11-17 0:38 UTC (permalink / raw)
To: Tejun Heo
Cc: Oleg Nesterov, cgroups, linux-kernel, kernel-team, Roman Gushchin
The helper is identical to the existing cgroup_task_count()
except it doesn't take the css_set_lock by itself, assuming
that the caller does.
Also, move cgroup_task_count() implementation into
kernel/cgroup/cgroup.c, as there is nothing specific to cgroup v1.
Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: kernel-team@fb.com
---
kernel/cgroup/cgroup-internal.h | 1 +
kernel/cgroup/cgroup-v1.c | 16 ----------------
kernel/cgroup/cgroup.c | 33 +++++++++++++++++++++++++++++++++
3 files changed, 34 insertions(+), 16 deletions(-)
diff --git a/kernel/cgroup/cgroup-internal.h b/kernel/cgroup/cgroup-internal.h
index 75568fcf2180..fe01a9fa4a8d 100644
--- a/kernel/cgroup/cgroup-internal.h
+++ b/kernel/cgroup/cgroup-internal.h
@@ -224,6 +224,7 @@ int cgroup_rmdir(struct kernfs_node *kn);
int cgroup_show_path(struct seq_file *sf, struct kernfs_node *kf_node,
struct kernfs_root *kf_root);
+int __cgroup_task_count(const struct cgroup *cgrp);
int cgroup_task_count(const struct cgroup *cgrp);
/*
diff --git a/kernel/cgroup/cgroup-v1.c b/kernel/cgroup/cgroup-v1.c
index 51063e7a93c2..6134fef07d57 100644
--- a/kernel/cgroup/cgroup-v1.c
+++ b/kernel/cgroup/cgroup-v1.c
@@ -336,22 +336,6 @@ static struct cgroup_pidlist *cgroup_pidlist_find_create(struct cgroup *cgrp,
return l;
}
-/**
- * cgroup_task_count - count the number of tasks in a cgroup.
- * @cgrp: the cgroup in question
- */
-int cgroup_task_count(const struct cgroup *cgrp)
-{
- int count = 0;
- struct cgrp_cset_link *link;
-
- spin_lock_irq(&css_set_lock);
- list_for_each_entry(link, &cgrp->cset_links, cset_link)
- count += link->cset->nr_tasks;
- spin_unlock_irq(&css_set_lock);
- return count;
-}
-
/*
* Load a cgroup's pidarray with either procs' tgids or tasks' pids
*/
diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index 4a3dae2a8283..ef3442555b32 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -561,6 +561,39 @@ static void cgroup_get_live(struct cgroup *cgrp)
css_get(&cgrp->self);
}
+/**
+ * __cgroup_task_count - count the number of tasks in a cgroup. The caller
+ * is responsible for taking the css_set_lock.
+ * @cgrp: the cgroup in question
+ */
+int __cgroup_task_count(const struct cgroup *cgrp)
+{
+ int count = 0;
+ struct cgrp_cset_link *link;
+
+ lockdep_assert_held(&css_set_lock);
+
+ list_for_each_entry(link, &cgrp->cset_links, cset_link)
+ count += link->cset->nr_tasks;
+
+ return count;
+}
+
+/**
+ * cgroup_task_count - count the number of tasks in a cgroup.
+ * @cgrp: the cgroup in question
+ */
+int cgroup_task_count(const struct cgroup *cgrp)
+{
+ int count;
+
+ spin_lock_irq(&css_set_lock);
+ count = __cgroup_task_count(cgrp);
+ spin_unlock_irq(&css_set_lock);
+
+ return count;
+}
+
struct cgroup_subsys_state *of_css(struct kernfs_open_file *of)
{
struct cgroup *cgrp = of->kn->parent->priv;
--
2.17.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v3 3/7] cgroup: protect cgroup->nr_(dying_)descendants by css_set_lock
2018-11-17 0:38 [PATCH v3 0/7] freezer for cgroup v2 Roman Gushchin
2018-11-17 0:38 ` [PATCH v3 1/7] cgroup: rename freezer.c into legacy_freezer.c Roman Gushchin
2018-11-17 0:38 ` [PATCH v3 2/7] cgroup: implement __cgroup_task_count() helper Roman Gushchin
@ 2018-11-17 0:38 ` Roman Gushchin
2018-11-20 16:18 ` Tejun Heo
2018-11-17 0:38 ` [PATCH v3 4/7] cgroup: cgroup v2 freezer Roman Gushchin
` (3 subsequent siblings)
6 siblings, 1 reply; 23+ messages in thread
From: Roman Gushchin @ 2018-11-17 0:38 UTC (permalink / raw)
To: Tejun Heo
Cc: Oleg Nesterov, cgroups, linux-kernel, kernel-team, Roman Gushchin
Now the number of descendant cgroups and the number of dying
descendant cgroups are synchronized using the cgroup_mutex.
The number of descendant cgroups will be required by the cgroup v2
freezer, which will use it to determine if a cgroup is frozen
(depending on total number of descendants and number of frozen
descendants). It's not always acceptable to grab the cgroup_mutex,
especially from quite hot paths (e.g. exit()).
To avoid this, let's additionally synchronize these counters
using the css_set_lock.
Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: kernel-team@fb.com
---
include/linux/cgroup-defs.h | 3 +++
kernel/cgroup/cgroup.c | 20 ++++++++++++++++----
2 files changed, 19 insertions(+), 4 deletions(-)
diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h
index 22254c1fe1c5..9e77559c7f49 100644
--- a/include/linux/cgroup-defs.h
+++ b/include/linux/cgroup-defs.h
@@ -346,6 +346,9 @@ struct cgroup {
* Dying cgroups are cgroups which were deleted by a user,
* but are still existing because someone else is holding a reference.
* max_descendants is a maximum allowed number of descent cgroups.
+ *
+ * nr_descendants and nr_dying_descendants are protected
+ * by css_set_lock.
*/
int nr_descendants;
int nr_dying_descendants;
diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index ef3442555b32..2241cb1d1238 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -3409,11 +3409,15 @@ static int cgroup_events_show(struct seq_file *seq, void *v)
static int cgroup_stat_show(struct seq_file *seq, void *v)
{
struct cgroup *cgroup = seq_css(seq)->cgroup;
+ int nr_descendants, nr_dying_descendants;
- seq_printf(seq, "nr_descendants %d\n",
- cgroup->nr_descendants);
- seq_printf(seq, "nr_dying_descendants %d\n",
- cgroup->nr_dying_descendants);
+ spin_lock_irq(&css_set_lock);
+ nr_descendants = cgroup->nr_descendants;
+ nr_dying_descendants = cgroup->nr_dying_descendants;
+ spin_unlock_irq(&css_set_lock);
+
+ seq_printf(seq, "nr_descendants %d\n", nr_descendants);
+ seq_printf(seq, "nr_dying_descendants %d\n", nr_dying_descendants);
return 0;
}
@@ -4684,9 +4688,11 @@ static void css_release_work_fn(struct work_struct *work)
if (cgroup_on_dfl(cgrp))
cgroup_rstat_flush(cgrp);
+ spin_lock_irq(&css_set_lock);
for (tcgrp = cgroup_parent(cgrp); tcgrp;
tcgrp = cgroup_parent(tcgrp))
tcgrp->nr_dying_descendants--;
+ spin_unlock_irq(&css_set_lock);
cgroup_idr_remove(&cgrp->root->cgroup_idr, cgrp->id);
cgrp->id = -1;
@@ -4899,12 +4905,14 @@ static struct cgroup *cgroup_create(struct cgroup *parent)
if (ret)
goto out_idr_free;
+ spin_lock_irq(&css_set_lock);
for (tcgrp = cgrp; tcgrp; tcgrp = cgroup_parent(tcgrp)) {
cgrp->ancestor_ids[tcgrp->level] = tcgrp->id;
if (tcgrp != cgrp)
tcgrp->nr_descendants++;
}
+ spin_unlock_irq(&css_set_lock);
if (notify_on_release(parent))
set_bit(CGRP_NOTIFY_ON_RELEASE, &cgrp->flags);
@@ -4956,6 +4964,7 @@ static bool cgroup_check_hierarchy_limits(struct cgroup *parent)
lockdep_assert_held(&cgroup_mutex);
+ spin_lock_irq(&css_set_lock);
for (cgroup = parent; cgroup; cgroup = cgroup_parent(cgroup)) {
if (cgroup->nr_descendants >= cgroup->max_descendants)
goto fail;
@@ -4968,6 +4977,7 @@ static bool cgroup_check_hierarchy_limits(struct cgroup *parent)
ret = true;
fail:
+ spin_unlock_irq(&css_set_lock);
return ret;
}
@@ -5187,10 +5197,12 @@ static int cgroup_destroy_locked(struct cgroup *cgrp)
if (parent && cgroup_is_threaded(cgrp))
parent->nr_threaded_children--;
+ spin_lock_irq(&css_set_lock);
for (tcgrp = cgroup_parent(cgrp); tcgrp; tcgrp = cgroup_parent(tcgrp)) {
tcgrp->nr_descendants--;
tcgrp->nr_dying_descendants++;
}
+ spin_unlock_irq(&css_set_lock);
cgroup1_check_for_release(parent);
--
2.17.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v3 4/7] cgroup: cgroup v2 freezer
2018-11-17 0:38 [PATCH v3 0/7] freezer for cgroup v2 Roman Gushchin
` (2 preceding siblings ...)
2018-11-17 0:38 ` [PATCH v3 3/7] cgroup: protect cgroup->nr_(dying_)descendants by css_set_lock Roman Gushchin
@ 2018-11-17 0:38 ` Roman Gushchin
2018-11-20 16:25 ` Tejun Heo
2018-11-17 0:38 ` guroan
` (2 subsequent siblings)
6 siblings, 1 reply; 23+ messages in thread
From: Roman Gushchin @ 2018-11-17 0:38 UTC (permalink / raw)
To: Tejun Heo
Cc: Oleg Nesterov, cgroups, linux-kernel, kernel-team, Roman Gushchin
Cgroup v1 implements the freezer controller, which provides an ability
to stop the workload in a cgroup and temporarily free up some
resources (cpu, io, network bandwidth and, potentially, memory)
for some other tasks. Cgroup v2 lacks this functionality.
This patch implements freezer for cgroup v2.
Cgroup v2 freezer tries to put tasks into a state similar to jobctl
stop. This means that tasks can be killed, ptraced (using
PTRACE_SEIZE*), and interrupted. It is possible to attach to
a frozen task, get some information (e.g. read registers) and detach.
It's also possible to migrate a frozen tasks to another cgroup.
This differs cgroup v2 freezer from cgroup v1 freezer, which mostly
tried to imitate the system-wide freezer. However uninterruptible
sleep is fine when all tasks are going to be frozen (hibernation case),
it's not the acceptable state for some subset of the system.
Cgroup v2 freezer is not supporting freezing kthreads.
If a non-root cgroup contains kthread, the cgroup still can be frozen,
but the kthread will remain running, the cgroup will be shown
as non-frozen, and the notification will not be delivered.
* PTRACE_ATTACH is not working because non-fatal signal delivery
is blocked in frozen state.
There are some interface differences between cgroup v1 and cgroup v2
freezer too, which are required to conform the cgroup v2 interface
design principles:
1) There is no separate controller, which has to be turned on:
the functionality is always available and is represented by
cgroup.freeze and cgroup.events cgroup control files.
2) The desired state is defined by the cgroup.freeze control file.
Any hierarchical configuration is allowed.
3) The interface is asynchronous. The actual state is available
using cgroup.events control file ("frozen" field). There are no
dedicated transitional states.
4) It's allowed to make any changes with the cgroup hierarchy
(create new cgroups, remove old cgroups, move tasks between cgroups)
no matter if some cgroups are frozen.
Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: kernel-team@fb.com
---
include/linux/cgroup-defs.h | 28 ++++
include/linux/cgroup.h | 42 +++++
include/linux/sched.h | 2 +
include/linux/sched/jobctl.h | 2 +
include/linux/sched/signal.h | 2 +
kernel/cgroup/Makefile | 2 +-
kernel/cgroup/cgroup.c | 112 ++++++++++++-
kernel/cgroup/freezer.c | 302 +++++++++++++++++++++++++++++++++++
kernel/ptrace.c | 7 +
kernel/signal.c | 57 +++++--
10 files changed, 538 insertions(+), 18 deletions(-)
create mode 100644 kernel/cgroup/freezer.c
diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h
index 9e77559c7f49..d1ba02771baf 100644
--- a/include/linux/cgroup-defs.h
+++ b/include/linux/cgroup-defs.h
@@ -63,6 +63,12 @@ enum {
* specified at mount time and thus is implemented here.
*/
CGRP_CPUSET_CLONE_CHILDREN,
+
+ /* Control group has to be frozen. */
+ CGRP_FREEZE,
+
+ /* Cgroup is frozen. */
+ CGRP_FROZEN,
};
/* cgroup_root->flags */
@@ -314,6 +320,25 @@ struct cgroup_rstat_cpu {
struct cgroup *updated_next; /* NULL iff not on the list */
};
+struct cgroup_freezer_state {
+ /* Should the cgroup and its descendants be frozen. */
+ bool freeze;
+
+ /* Should the cgroup actually be frozen? */
+ int e_freeze;
+
+ /* Fields below are protected by css_set_lock */
+
+ /* Number of frozen descendant cgroups */
+ int nr_frozen_descendants;
+
+ /* Number of tasks to freeze */
+ int nr_tasks_to_freeze;
+
+ /* Number of frozen tasks */
+ int nr_frozen_tasks;
+};
+
struct cgroup {
/* self css with NULL ->ss, points back to this cgroup */
struct cgroup_subsys_state self;
@@ -445,6 +470,9 @@ struct cgroup {
/* If there is block congestion on this cgroup. */
atomic_t congestion_count;
+ /* Used to store internal freezer state */
+ struct cgroup_freezer_state freezer;
+
/* ids of the ancestors at each level including self */
int ancestor_ids[];
};
diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h
index 32c553556bbd..a40a54f322b4 100644
--- a/include/linux/cgroup.h
+++ b/include/linux/cgroup.h
@@ -871,4 +871,46 @@ static inline void put_cgroup_ns(struct cgroup_namespace *ns)
free_cgroup_ns(ns);
}
+#ifdef CONFIG_CGROUPS
+
+void cgroup_enter_frozen(void);
+void cgroup_leave_frozen(void);
+void cgroup_freeze(struct cgroup *cgrp, bool freeze);
+void cgroup_update_frozen(struct cgroup *cgrp, bool frozen);
+void cgroup_freezer_migrate_task(struct task_struct *task, struct cgroup *src,
+ struct cgroup *dst);
+static inline bool cgroup_task_freeze(struct task_struct *task)
+{
+ bool ret;
+
+ if (task->flags & PF_KTHREAD)
+ return false;
+
+ rcu_read_lock();
+ ret = test_bit(CGRP_FREEZE, &task_dfl_cgroup(task)->flags);
+ rcu_read_unlock();
+
+ return ret;
+}
+
+static inline bool cgroup_task_frozen(struct task_struct *task)
+{
+ return task->frozen;
+}
+
+#else /* !CONFIG_CGROUPS */
+
+static inline void cgroup_enter_frozen(void) { }
+static inline void cgroup_leave_frozen(void) { }
+static inline bool cgroup_task_freeze(struct task_struct *task)
+{
+ return false;
+}
+static inline bool cgroup_task_frozen(struct task_struct *task)
+{
+ return false;
+}
+
+#endif /* !CONFIG_CGROUPS */
+
#endif /* _LINUX_CGROUP_H */
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 977cb57d7bc9..3457b09ec044 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -733,6 +733,8 @@ struct task_struct {
#ifdef CONFIG_CGROUPS
/* disallow userland-initiated cgroup migration */
unsigned no_cgroup_migration:1;
+ /* task is frozen by the cgroup freezer */
+ unsigned frozen:1;
#endif
#ifdef CONFIG_BLK_CGROUP
/* to be used once the psi infrastructure lands upstream. */
diff --git a/include/linux/sched/jobctl.h b/include/linux/sched/jobctl.h
index 98228bd48aee..fa067de9f1a9 100644
--- a/include/linux/sched/jobctl.h
+++ b/include/linux/sched/jobctl.h
@@ -18,6 +18,7 @@ struct task_struct;
#define JOBCTL_TRAP_NOTIFY_BIT 20 /* trap for NOTIFY */
#define JOBCTL_TRAPPING_BIT 21 /* switching to TRACED */
#define JOBCTL_LISTENING_BIT 22 /* ptracer is listening for events */
+#define JOBCTL_TRAP_FREEZE_BIT 23 /* trap for cgroup freezer */
#define JOBCTL_STOP_DEQUEUED (1UL << JOBCTL_STOP_DEQUEUED_BIT)
#define JOBCTL_STOP_PENDING (1UL << JOBCTL_STOP_PENDING_BIT)
@@ -26,6 +27,7 @@ struct task_struct;
#define JOBCTL_TRAP_NOTIFY (1UL << JOBCTL_TRAP_NOTIFY_BIT)
#define JOBCTL_TRAPPING (1UL << JOBCTL_TRAPPING_BIT)
#define JOBCTL_LISTENING (1UL << JOBCTL_LISTENING_BIT)
+#define JOBCTL_TRAP_FREEZE (1UL << JOBCTL_TRAP_FREEZE_BIT)
#define JOBCTL_TRAP_MASK (JOBCTL_TRAP_STOP | JOBCTL_TRAP_NOTIFY)
#define JOBCTL_PENDING_MASK (JOBCTL_STOP_PENDING | JOBCTL_TRAP_MASK)
diff --git a/include/linux/sched/signal.h b/include/linux/sched/signal.h
index 1be35729c2c5..46e547ab0c25 100644
--- a/include/linux/sched/signal.h
+++ b/include/linux/sched/signal.h
@@ -368,6 +368,8 @@ static inline int signal_pending_state(long state, struct task_struct *p)
return 0;
if (!signal_pending(p))
return 0;
+ if (unlikely(p->frozen))
+ return __fatal_signal_pending(p);
return (state & TASK_INTERRUPTIBLE) || __fatal_signal_pending(p);
}
diff --git a/kernel/cgroup/Makefile b/kernel/cgroup/Makefile
index 8d5689ca94b9..5d7a76bfbbb7 100644
--- a/kernel/cgroup/Makefile
+++ b/kernel/cgroup/Makefile
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: GPL-2.0
-obj-y := cgroup.o rstat.o namespace.o cgroup-v1.o
+obj-y := cgroup.o rstat.o namespace.o cgroup-v1.o freezer.o
obj-$(CONFIG_CGROUP_FREEZER) += legacy_freezer.o
obj-$(CONFIG_CGROUP_PIDS) += pids.o
diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index 2241cb1d1238..a6d58ec0bf56 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -2356,6 +2356,12 @@ static int cgroup_migrate_execute(struct cgroup_mgctx *mgctx)
get_css_set(to_cset);
to_cset->nr_tasks++;
css_set_move_task(task, from_cset, to_cset, true);
+ /*
+ * If the source or destination cgroup is frozen,
+ * the task might require to change its state.
+ */
+ cgroup_freezer_migrate_task(task, from_cset->dfl_cgrp,
+ to_cset->dfl_cgrp);
put_css_set_locked(from_cset);
from_cset->nr_tasks--;
}
@@ -3401,8 +3407,11 @@ static ssize_t cgroup_max_depth_write(struct kernfs_open_file *of,
static int cgroup_events_show(struct seq_file *seq, void *v)
{
- seq_printf(seq, "populated %d\n",
- cgroup_is_populated(seq_css(seq)->cgroup));
+ struct cgroup *cgrp = seq_css(seq)->cgroup;
+
+ seq_printf(seq, "populated %d\n", cgroup_is_populated(cgrp));
+ seq_printf(seq, "frozen %d\n", test_bit(CGRP_FROZEN, &cgrp->flags));
+
return 0;
}
@@ -3453,6 +3462,40 @@ static int cpu_stat_show(struct seq_file *seq, void *v)
return ret;
}
+static int cgroup_freeze_show(struct seq_file *seq, void *v)
+{
+ struct cgroup *cgrp = seq_css(seq)->cgroup;
+
+ seq_printf(seq, "%d\n", cgrp->freezer.freeze);
+
+ return 0;
+}
+
+static ssize_t cgroup_freeze_write(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
+{
+ struct cgroup *cgrp;
+ ssize_t ret;
+ int freeze;
+
+ ret = kstrtoint(strstrip(buf), 0, &freeze);
+ if (ret)
+ return ret;
+
+ if (freeze < 0 || freeze > 1)
+ return -ERANGE;
+
+ cgrp = cgroup_kn_lock_live(of->kn, false);
+ if (!cgrp)
+ return -ENOENT;
+
+ cgroup_freeze(cgrp, freeze);
+
+ cgroup_kn_unlock(of->kn);
+
+ return nbytes;
+}
+
static int cgroup_file_open(struct kernfs_open_file *of)
{
struct cftype *cft = of->kn->priv;
@@ -4578,6 +4621,12 @@ static struct cftype cgroup_base_files[] = {
.name = "cgroup.stat",
.seq_show = cgroup_stat_show,
},
+ {
+ .name = "cgroup.freeze",
+ .flags = CFTYPE_NOT_ON_ROOT,
+ .seq_show = cgroup_freeze_show,
+ .write = cgroup_freeze_write,
+ },
{
.name = "cpu.stat",
.flags = CFTYPE_NOT_ON_ROOT,
@@ -4905,12 +4954,29 @@ static struct cgroup *cgroup_create(struct cgroup *parent)
if (ret)
goto out_idr_free;
+ /*
+ * New cgroup inherits effective freeze counter, and
+ * if the parent has to be frozen, the child has too.
+ */
+ cgrp->freezer.e_freeze = parent->freezer.e_freeze;
+ if (cgrp->freezer.e_freeze)
+ set_bit(CGRP_FROZEN, &cgrp->flags);
+
spin_lock_irq(&css_set_lock);
for (tcgrp = cgrp; tcgrp; tcgrp = cgroup_parent(tcgrp)) {
cgrp->ancestor_ids[tcgrp->level] = tcgrp->id;
- if (tcgrp != cgrp)
+ if (tcgrp != cgrp) {
tcgrp->nr_descendants++;
+
+ /*
+ * If the new cgroup is frozen, all ancestor cgroups
+ * get a new frozen descendant, but their state can't
+ * change because of this.
+ */
+ if (cgrp->freezer.e_freeze)
+ tcgrp->freezer.nr_frozen_descendants++;
+ }
}
spin_unlock_irq(&css_set_lock);
@@ -5201,6 +5267,12 @@ static int cgroup_destroy_locked(struct cgroup *cgrp)
for (tcgrp = cgroup_parent(cgrp); tcgrp; tcgrp = cgroup_parent(tcgrp)) {
tcgrp->nr_descendants--;
tcgrp->nr_dying_descendants++;
+ /*
+ * If the dying cgroup is frozen, decrease frozen descendants
+ * counters of ancestor cgroups.
+ */
+ if (test_bit(CGRP_FROZEN, &cgrp->flags))
+ tcgrp->freezer.nr_frozen_descendants--;
}
spin_unlock_irq(&css_set_lock);
@@ -5654,6 +5726,23 @@ void cgroup_post_fork(struct task_struct *child)
cset->nr_tasks++;
css_set_move_task(child, NULL, cset, false);
}
+
+ /*
+ * If the cgroup has to be frozen, the new task has too.
+ * Let's update cgroup freezer counter and set the
+ * JOBCTL_TRAP_FREEZE jobctl bit to get the tack into
+ * the frozen state.
+ */
+ if (unlikely(cgroup_task_freeze(child))) {
+ struct cgroup *cgrp;
+
+ spin_lock_irq(&child->sighand->siglock);
+ cgrp = cset->dfl_cgrp;
+ cgrp->freezer.nr_tasks_to_freeze++;
+ child->jobctl |= JOBCTL_TRAP_FREEZE;
+ spin_unlock_irq(&child->sighand->siglock);
+ }
+
spin_unlock_irq(&css_set_lock);
}
@@ -5702,6 +5791,23 @@ void cgroup_exit(struct task_struct *tsk)
spin_lock_irq(&css_set_lock);
css_set_move_task(tsk, cset, NULL, false);
cset->nr_tasks--;
+
+ if (unlikely(test_bit(CGRP_FROZEN, &cset->dfl_cgrp->flags))) {
+ struct cgroup *frozen_cgrp = cset->dfl_cgrp;
+
+ /*
+ * Task frozen bit should be cleared at this moment,
+ * as well as nr_frozen_task should be decreased.
+ * Let's update nr_tasks_to_freeze, and check if
+ * the cgroup is actually frozen.
+ */
+ WARN_ON_ONCE(tsk->frozen);
+ frozen_cgrp->freezer.nr_tasks_to_freeze--;
+ WARN_ON_ONCE(frozen_cgrp->freezer.nr_tasks_to_freeze <
+ frozen_cgrp->freezer.nr_frozen_tasks);
+ cgroup_update_frozen(frozen_cgrp, true);
+ }
+
spin_unlock_irq(&css_set_lock);
} else {
get_css_set(cset);
diff --git a/kernel/cgroup/freezer.c b/kernel/cgroup/freezer.c
new file mode 100644
index 000000000000..94d4600d95e0
--- /dev/null
+++ b/kernel/cgroup/freezer.c
@@ -0,0 +1,302 @@
+//SPDX-License-Identifier: GPL-2.0
+#include <linux/cgroup.h>
+#include <linux/sched.h>
+#include <linux/sched/task.h>
+#include <linux/sched/signal.h>
+
+#include "cgroup-internal.h"
+
+/*
+ * Propagate the cgroup frozen state upwards by the cgroup tree.
+ */
+void cgroup_propagate_frozen(struct cgroup *cgrp, bool frozen)
+{
+ int desc = 1;
+
+ /*
+ * If the new state is frozen, some freezing ancestor cgroups may change
+ * their state too, depending on if all their descendants are frozen.
+ *
+ * Otherwise, all ancestor cgroups are forced into the non-frozen state.
+ */
+ while ((cgrp = cgroup_parent(cgrp))) {
+ if (frozen) {
+ cgrp->freezer.nr_frozen_descendants += desc;
+ if (!test_bit(CGRP_FROZEN, &cgrp->flags) &&
+ test_bit(CGRP_FREEZE, &cgrp->flags) &&
+ cgrp->freezer.nr_frozen_descendants ==
+ cgrp->nr_descendants) {
+ set_bit(CGRP_FROZEN, &cgrp->flags);
+ cgroup_file_notify(&cgrp->events_file);
+ desc++;
+ }
+ } else {
+ cgrp->freezer.nr_frozen_descendants -= desc;
+ if (test_bit(CGRP_FROZEN, &cgrp->flags)) {
+ clear_bit(CGRP_FROZEN, &cgrp->flags);
+ cgroup_file_notify(&cgrp->events_file);
+ desc++;
+ }
+ }
+ }
+}
+
+/*
+ * Revisit the cgroup frozen state.
+ * Checks if the cgroup is really frozen if necessary and performs
+ * all state transitions.
+ */
+void cgroup_update_frozen(struct cgroup *cgrp, bool frozen)
+{
+ lockdep_assert_held(&css_set_lock);
+
+ if (frozen) {
+ /*
+ * If some tasks are still running, the cgroup
+ * isn't ready for the state transitioning.
+ */
+ if (cgrp->freezer.nr_frozen_tasks <
+ cgrp->freezer.nr_tasks_to_freeze)
+ return;
+
+ /* Already there? */
+ if (test_bit(CGRP_FROZEN, &cgrp->flags))
+ return;
+
+ set_bit(CGRP_FROZEN, &cgrp->flags);
+ } else {
+ /* Already there? */
+ if (!test_bit(CGRP_FROZEN, &cgrp->flags))
+ return;
+
+ clear_bit(CGRP_FROZEN, &cgrp->flags);
+ }
+ cgroup_file_notify(&cgrp->events_file);
+
+ /* Update the state of ancestor cgroups. */
+ cgroup_propagate_frozen(cgrp, frozen);
+}
+
+/*
+ * Entry path into frozen state.
+ * If the task was not frozen before, counters are updated and the cgroup state
+ * is revisited. Otherwise, the task is put into the TASK_KILLABLE sleep.
+ */
+void cgroup_enter_frozen(void)
+{
+ if (!current->frozen) {
+ struct cgroup *cgrp;
+
+ spin_lock_irq(&css_set_lock);
+ current->frozen = true;
+ cgrp = task_dfl_cgroup(current);
+ cgrp->freezer.nr_frozen_tasks++;
+ WARN_ON_ONCE(cgrp->freezer.nr_frozen_tasks >
+ cgrp->freezer.nr_tasks_to_freeze);
+ cgroup_update_frozen(cgrp, true);
+ spin_unlock_irq(&css_set_lock);
+ }
+
+ __set_current_state(TASK_INTERRUPTIBLE);
+ schedule();
+ __set_current_state(TASK_RUNNING);
+}
+
+/*
+ * Exit path from the frozen state.
+ * Counters are updated, but the cgroup state is not revisited.
+ * If the frozen task is dying or migrating to a running cgroup, the original
+ * cgroup should remain frozen.
+ * Otherwise it already has been unfrozen by a user.
+ */
+void cgroup_leave_frozen(void)
+{
+ struct cgroup *cgrp;
+
+ spin_lock_irq(&css_set_lock);
+ cgrp = task_dfl_cgroup(current);
+ cgrp->freezer.nr_frozen_tasks--;
+ WARN_ON_ONCE(cgrp->freezer.nr_frozen_tasks < 0);
+ current->frozen = false;
+ spin_unlock_irq(&css_set_lock);
+}
+
+/*
+ * Freeze or unfreeze the task by setting or clearing the JOBCTL_TRAP_FREEZE
+ * jobctl bit.
+ */
+static void cgroup_freeze_task(struct task_struct *task, bool freeze)
+{
+ unsigned long flags;
+
+ /* If the task is about to die, don't bother with freezing it. */
+ if (!lock_task_sighand(task, &flags))
+ return;
+
+ if (freeze) {
+ task->jobctl |= JOBCTL_TRAP_FREEZE;
+ signal_wake_up(task, false);
+ } else {
+ task->jobctl &= ~JOBCTL_TRAP_FREEZE;
+ wake_up_process(task);
+ }
+
+ unlock_task_sighand(task, &flags);
+}
+
+/*
+ * Freeze or unfreeze all tasks in the given cgroup.
+ */
+static void cgroup_do_freeze(struct cgroup *cgrp, bool freeze)
+{
+ struct css_task_iter it;
+ struct task_struct *task;
+
+ lockdep_assert_held(&cgroup_mutex);
+
+ spin_lock_irq(&css_set_lock);
+ if (freeze) {
+ cgrp->freezer.nr_tasks_to_freeze = __cgroup_task_count(cgrp);
+ set_bit(CGRP_FREEZE, &cgrp->flags);
+ } else {
+ clear_bit(CGRP_FREEZE, &cgrp->flags);
+ /*
+ * As soon as the CGRP_FREEZE bit has been cleared, the cgroup
+ * can't be considered as frozen anymore (e.g. a task migrating
+ * to it will remain/become running), so let's mark the cgroup
+ * as non-frozen immediately.
+ */
+ cgroup_update_frozen(cgrp, false);
+ }
+ spin_unlock_irq(&css_set_lock);
+
+ css_task_iter_start(&cgrp->self, 0, &it);
+ while ((task = css_task_iter_next(&it))) {
+ /*
+ * Ignore kernel threads here. Freezing cgroups containing
+ * kthreads isn't supported.
+ */
+ if (task->flags & PF_KTHREAD)
+ continue;
+ cgroup_freeze_task(task, freeze);
+ }
+ css_task_iter_end(&it);
+
+ if (!freeze)
+ return;
+
+ spin_lock_irq(&css_set_lock);
+ /*
+ * If all descendants are frozen (if there are any), let's check
+ * if the cgroup can be considered frozen at this moment.
+ * If a leaf cgroup is empty, it's the only moment when it can be
+ * recognized and marked as frozen.
+ */
+ if (cgrp->nr_descendants == cgrp->freezer.nr_frozen_descendants)
+ cgroup_update_frozen(cgrp, true);
+ spin_unlock_irq(&css_set_lock);
+}
+
+void cgroup_freezer_migrate_task(struct task_struct *task,
+ struct cgroup *src, struct cgroup *dst)
+{
+ lockdep_assert_held(&css_set_lock);
+
+ /*
+ * Kernel threads are not supposed to be frozen at all.
+ */
+ if (task->flags & PF_KTHREAD)
+ return;
+
+ /*
+ * Adjust counters of freezing and frozen tasks.
+ */
+ if (test_bit(CGRP_FREEZE, &src->flags)) {
+ src->freezer.nr_tasks_to_freeze--;
+ WARN_ON_ONCE(src->freezer.nr_tasks_to_freeze < 0);
+ }
+
+ /*
+ * If the task is frozen, let's bump nr_tasks_to_freeze even
+ * if the target cgroup isn't frozen: the counter will be decreased
+ * in cgroup_leave_frozen().
+ */
+ if (test_bit(CGRP_FREEZE, &dst->flags) || task->frozen)
+ dst->freezer.nr_tasks_to_freeze++;
+
+ if (task->frozen) {
+ src->freezer.nr_frozen_tasks--;
+ dst->freezer.nr_frozen_tasks++;
+ WARN_ON_ONCE(src->freezer.nr_frozen_tasks < 0);
+ WARN_ON_ONCE(dst->freezer.nr_frozen_tasks >
+ dst->freezer.nr_tasks_to_freeze);
+ }
+
+ /*
+ * If the task isn't in the desired state, force it to it.
+ */
+ if (task->frozen != test_bit(CGRP_FREEZE, &dst->flags))
+ cgroup_freeze_task(task, test_bit(CGRP_FREEZE, &dst->flags));
+}
+
+void cgroup_freeze(struct cgroup *cgrp, bool freeze)
+{
+ struct cgroup_subsys_state *css;
+ struct cgroup *dsct;
+ bool applied = false;
+
+ lockdep_assert_held(&cgroup_mutex);
+
+ /*
+ * Nothing changed? Just exit.
+ */
+ if (cgrp->freezer.freeze == freeze)
+ return;
+
+ cgrp->freezer.freeze = freeze;
+
+ /*
+ * Propagate changes downwards the cgroup tree.
+ */
+ css_for_each_descendant_pre(css, &cgrp->self) {
+ dsct = css->cgroup;
+
+ if (cgroup_is_dead(dsct))
+ continue;
+
+ if (freeze) {
+ dsct->freezer.e_freeze++;
+ /*
+ * Already frozen because of ancestor's settings?
+ */
+ if (dsct->freezer.e_freeze > 1)
+ continue;
+ } else {
+ dsct->freezer.e_freeze--;
+ /*
+ * Still frozen because of ancestor's settings?
+ */
+ if (dsct->freezer.e_freeze > 0)
+ continue;
+
+ WARN_ON_ONCE(dsct->freezer.e_freeze < 0);
+ }
+
+ /*
+ * Do change actual state: freeze or unfreeze.
+ */
+ cgroup_do_freeze(dsct, freeze);
+ applied = true;
+ }
+
+ /*
+ * Even if the actual state hasn't changed, let's notify a user.
+ * The state can be enforced by an ancestor cgroup: the cgroup
+ * can already be in the desired state or it can be locked in the
+ * opposite state, so that the transition will never happen.
+ * In both cases it's better to notify a user, that there is
+ * nothing to wait.
+ */
+ if (!applied)
+ cgroup_file_notify(&cgrp->events_file);
+}
diff --git a/kernel/ptrace.c b/kernel/ptrace.c
index 21fec73d45d4..198546c7dff5 100644
--- a/kernel/ptrace.c
+++ b/kernel/ptrace.c
@@ -400,6 +400,13 @@ static int ptrace_attach(struct task_struct *task, long request,
spin_lock(&task->sighand->siglock);
+ /*
+ * If the process is frozen, let's wake it up to give it a chance
+ * to enter the ptrace trap.
+ */
+ if (task->frozen)
+ wake_up_process(task);
+
/*
* If the task is already STOPPED, set JOBCTL_TRAP_STOP and
* TRAPPING, and kick it so that it transits to TRACED. TRAPPING
diff --git a/kernel/signal.c b/kernel/signal.c
index 5843c541fda9..fdd89ef0cbcc 100644
--- a/kernel/signal.c
+++ b/kernel/signal.c
@@ -168,7 +168,7 @@ void recalc_sigpending_and_wake(struct task_struct *t)
void recalc_sigpending(void)
{
if (!recalc_sigpending_tsk(current) && !freezing(current) &&
- !klp_patch_pending(current))
+ !klp_patch_pending(current) && !current->frozen)
clear_thread_flag(TIF_SIGPENDING);
}
@@ -2252,7 +2252,7 @@ static bool do_signal_stop(int signr)
}
/**
- * do_jobctl_trap - take care of ptrace jobctl traps
+ * do_jobctl_trap - take care of ptrace and cgroup freezer jobctl traps
*
* When PT_SEIZED, it's used for both group stop and explicit
* SEIZE/INTERRUPT traps. Both generate PTRACE_EVENT_STOP trap with
@@ -2262,26 +2262,44 @@ static bool do_signal_stop(int signr)
* When !PT_SEIZED, it's used only for group stop trap with stop signal
* number as exit_code and no siginfo.
*
+ * When used with JOBCTL_TRAP_FREEZE, it puts the task into an interruptible
+ * sleep if only no fatal signals are pending.
+ *
* CONTEXT:
* Must be called with @current->sighand->siglock held, which may be
* released and re-acquired before returning with intervening sleep.
*/
static void do_jobctl_trap(void)
{
+ struct sighand_struct *sighand = current->sighand;
struct signal_struct *signal = current->signal;
int signr = current->jobctl & JOBCTL_STOP_SIGMASK;
- if (current->ptrace & PT_SEIZED) {
- if (!signal->group_stop_count &&
- !(signal->flags & SIGNAL_STOP_STOPPED))
- signr = SIGTRAP;
- WARN_ON_ONCE(!signr);
- ptrace_do_notify(signr, signr | (PTRACE_EVENT_STOP << 8),
- CLD_STOPPED);
- } else {
- WARN_ON_ONCE(!signr);
- ptrace_stop(signr, CLD_STOPPED, 0, NULL);
- current->exit_code = 0;
+ if (current->jobctl & (JOBCTL_TRAP_STOP | JOBCTL_TRAP_NOTIFY)) {
+ if (current->ptrace & PT_SEIZED) {
+ if (!signal->group_stop_count &&
+ !(signal->flags & SIGNAL_STOP_STOPPED))
+ signr = SIGTRAP;
+ WARN_ON_ONCE(!signr);
+ ptrace_do_notify(signr,
+ signr | (PTRACE_EVENT_STOP << 8),
+ CLD_STOPPED);
+ } else {
+ WARN_ON_ONCE(!signr);
+ ptrace_stop(signr, CLD_STOPPED, 0, NULL);
+ current->exit_code = 0;
+ }
+ } else if (current->jobctl & JOBCTL_TRAP_FREEZE) {
+ /*
+ * Enter the frozen state, unless the task is about to exit.
+ */
+ if (fatal_signal_pending(current)) {
+ current->jobctl &= ~JOBCTL_TRAP_FREEZE;
+ } else {
+ spin_unlock_irq(&sighand->siglock);
+ cgroup_enter_frozen();
+ spin_lock_irq(&sighand->siglock);
+ }
}
}
@@ -2397,12 +2415,23 @@ bool get_signal(struct ksignal *ksig)
do_signal_stop(0))
goto relock;
- if (unlikely(current->jobctl & JOBCTL_TRAP_MASK)) {
+ if (unlikely(current->jobctl &
+ (JOBCTL_TRAP_MASK | JOBCTL_TRAP_FREEZE))) {
do_jobctl_trap();
spin_unlock_irq(&sighand->siglock);
goto relock;
}
+ /*
+ * If the task is leaving the frozen state, let's update
+ * cgroup counters and reset the frozen bit.
+ */
+ if (unlikely(cgroup_task_frozen(current))) {
+ spin_unlock_irq(&sighand->siglock);
+ cgroup_leave_frozen();
+ spin_lock_irq(&sighand->siglock);
+ }
+
signr = dequeue_signal(current, ¤t->blocked, &ksig->info);
if (!signr)
--
2.17.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v3 5/7] kselftests: cgroup: don't fail on cg_kill_all() error in cg_destroy()
2018-11-17 0:38 [PATCH v3 0/7] freezer for cgroup v2 Roman Gushchin
2018-11-17 0:38 ` [PATCH v3 1/7] cgroup: rename freezer.c into legacy_freezer.c Roman Gushchin
@ 2018-11-17 0:38 ` guroan
2018-11-17 0:38 ` [PATCH v3 3/7] cgroup: protect cgroup->nr_(dying_)descendants by css_set_lock Roman Gushchin
` (4 subsequent siblings)
6 siblings, 0 replies; 23+ messages in thread
From: Roman Gushchin @ 2018-11-17 0:38 UTC (permalink / raw)
To: Tejun Heo
Cc: Oleg Nesterov, cgroups, linux-kernel, kernel-team,
Roman Gushchin, Shuah Khan, linux-kselftest
If the cgroup destruction races with an exit() of a belonging
process(es), cg_kill_all() may fail. It's not a good reason to make
cg_destroy() fail and leave the cgroup in place, potentially causing
next test runs to fail.
Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Tejun Heo <tj@kernel.org>
Cc: kernel-team@fb.com
Cc: linux-kselftest@vger.kernel.org
---
tools/testing/selftests/cgroup/cgroup_util.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/tools/testing/selftests/cgroup/cgroup_util.c b/tools/testing/selftests/cgroup/cgroup_util.c
index 14c9fe284806..eba06f94433b 100644
--- a/tools/testing/selftests/cgroup/cgroup_util.c
+++ b/tools/testing/selftests/cgroup/cgroup_util.c
@@ -227,9 +227,7 @@ int cg_destroy(const char *cgroup)
retry:
ret = rmdir(cgroup);
if (ret && errno == EBUSY) {
- ret = cg_killall(cgroup);
- if (ret)
- return ret;
+ cg_killall(cgroup);
usleep(100);
goto retry;
}
--
2.17.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v3 5/7] kselftests: cgroup: don't fail on cg_kill_all() error in cg_destroy()
@ 2018-11-17 0:38 ` guroan
0 siblings, 0 replies; 23+ messages in thread
From: guroan @ 2018-11-17 0:38 UTC (permalink / raw)
If the cgroup destruction races with an exit() of a belonging
process(es), cg_kill_all() may fail. It's not a good reason to make
cg_destroy() fail and leave the cgroup in place, potentially causing
next test runs to fail.
Signed-off-by: Roman Gushchin <guro at fb.com>
Cc: Shuah Khan <shuah at kernel.org>
Cc: Tejun Heo <tj at kernel.org>
Cc: kernel-team at fb.com
Cc: linux-kselftest at vger.kernel.org
---
tools/testing/selftests/cgroup/cgroup_util.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/tools/testing/selftests/cgroup/cgroup_util.c b/tools/testing/selftests/cgroup/cgroup_util.c
index 14c9fe284806..eba06f94433b 100644
--- a/tools/testing/selftests/cgroup/cgroup_util.c
+++ b/tools/testing/selftests/cgroup/cgroup_util.c
@@ -227,9 +227,7 @@ int cg_destroy(const char *cgroup)
retry:
ret = rmdir(cgroup);
if (ret && errno == EBUSY) {
- ret = cg_killall(cgroup);
- if (ret)
- return ret;
+ cg_killall(cgroup);
usleep(100);
goto retry;
}
--
2.17.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v3 5/7] kselftests: cgroup: don't fail on cg_kill_all() error in cg_destroy()
@ 2018-11-17 0:38 ` guroan
0 siblings, 0 replies; 23+ messages in thread
From: Roman Gushchin @ 2018-11-17 0:38 UTC (permalink / raw)
If the cgroup destruction races with an exit() of a belonging
process(es), cg_kill_all() may fail. It's not a good reason to make
cg_destroy() fail and leave the cgroup in place, potentially causing
next test runs to fail.
Signed-off-by: Roman Gushchin <guro at fb.com>
Cc: Shuah Khan <shuah at kernel.org>
Cc: Tejun Heo <tj at kernel.org>
Cc: kernel-team at fb.com
Cc: linux-kselftest at vger.kernel.org
---
tools/testing/selftests/cgroup/cgroup_util.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/tools/testing/selftests/cgroup/cgroup_util.c b/tools/testing/selftests/cgroup/cgroup_util.c
index 14c9fe284806..eba06f94433b 100644
--- a/tools/testing/selftests/cgroup/cgroup_util.c
+++ b/tools/testing/selftests/cgroup/cgroup_util.c
@@ -227,9 +227,7 @@ int cg_destroy(const char *cgroup)
retry:
ret = rmdir(cgroup);
if (ret && errno == EBUSY) {
- ret = cg_killall(cgroup);
- if (ret)
- return ret;
+ cg_killall(cgroup);
usleep(100);
goto retry;
}
--
2.17.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v3 6/7] kselftests: cgroup: add freezer controller self-tests
2018-11-17 0:38 [PATCH v3 0/7] freezer for cgroup v2 Roman Gushchin
2018-11-17 0:38 ` [PATCH v3 1/7] cgroup: rename freezer.c into legacy_freezer.c Roman Gushchin
@ 2018-11-17 0:38 ` guroan
2018-11-17 0:38 ` [PATCH v3 3/7] cgroup: protect cgroup->nr_(dying_)descendants by css_set_lock Roman Gushchin
` (4 subsequent siblings)
6 siblings, 0 replies; 23+ messages in thread
From: Roman Gushchin @ 2018-11-17 0:38 UTC (permalink / raw)
To: Tejun Heo
Cc: Oleg Nesterov, cgroups, linux-kernel, kernel-team,
Roman Gushchin, Shuah Khan, linux-kselftest
This patch implements six tests for the freezer controller for
cgroup v2:
1) a simple test, which aims to freeze and unfreeze a cgroup with 100
processes
2) a more complicated tree test, which creates a hierarchy of cgroups,
puts some processes in some cgroups, and tries to freeze and unfreeze
different parts of the subtree
3) a forkbomb test: the test aims to freeze a forkbomb running in a
cgroup, kill all tasks in the cgroup and remove the cgroup without
the unfreezing.
4) rmdir test: the test creates two nested cgroups, freezes the parent
one, checks that the child can be successfully removed, and a new
child can be created
5) migration tests: the test checks migration of a task between
frozen cgroups: from a frozen to a running, from a running to a
frozen, and from a frozen to a frozen.
6) ptrace test: the test checks that it's possible to attach to
a process in a frozen cgroup, get some information and detach, and
the cgroup will remain frozen.
Expected output:
$ ./test_freezer
ok 1 test_cgfreezer_simple
ok 2 test_cgfreezer_tree
ok 3 test_cgfreezer_forkbomb
ok 4 test_cgrreezer_rmdir
ok 5 test_cgfreezer_migrate
ok 6 test_cgfreezer_ptrace
Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Tejun Heo <tj@kernel.org>
Cc: kernel-team@fb.com
Cc: linux-kselftest@vger.kernel.org
---
tools/testing/selftests/cgroup/.gitignore | 1 +
tools/testing/selftests/cgroup/Makefile | 2 +
tools/testing/selftests/cgroup/cgroup_util.c | 81 ++-
tools/testing/selftests/cgroup/cgroup_util.h | 7 +
tools/testing/selftests/cgroup/test_freezer.c | 685 ++++++++++++++++++
5 files changed, 775 insertions(+), 1 deletion(-)
create mode 100644 tools/testing/selftests/cgroup/test_freezer.c
diff --git a/tools/testing/selftests/cgroup/.gitignore b/tools/testing/selftests/cgroup/.gitignore
index adacda50a4b2..7f9835624793 100644
--- a/tools/testing/selftests/cgroup/.gitignore
+++ b/tools/testing/selftests/cgroup/.gitignore
@@ -1,2 +1,3 @@
test_memcontrol
test_core
+test_freezer
diff --git a/tools/testing/selftests/cgroup/Makefile b/tools/testing/selftests/cgroup/Makefile
index 23fbaa4a9630..8d369b6a2069 100644
--- a/tools/testing/selftests/cgroup/Makefile
+++ b/tools/testing/selftests/cgroup/Makefile
@@ -5,8 +5,10 @@ all:
TEST_GEN_PROGS = test_memcontrol
TEST_GEN_PROGS += test_core
+TEST_GEN_PROGS += test_freezer
include ../lib.mk
$(OUTPUT)/test_memcontrol: cgroup_util.c
$(OUTPUT)/test_core: cgroup_util.c
+$(OUTPUT)/test_freezer: cgroup_util.c
diff --git a/tools/testing/selftests/cgroup/cgroup_util.c b/tools/testing/selftests/cgroup/cgroup_util.c
index eba06f94433b..e9cdad673901 100644
--- a/tools/testing/selftests/cgroup/cgroup_util.c
+++ b/tools/testing/selftests/cgroup/cgroup_util.c
@@ -74,6 +74,16 @@ char *cg_name_indexed(const char *root, const char *name, int index)
return ret;
}
+char *cg_control(const char *cgroup, const char *control)
+{
+ size_t len = strlen(cgroup) + strlen(control) + 2;
+ char *ret = malloc(len);
+
+ snprintf(ret, len, "%s/%s", cgroup, control);
+
+ return ret;
+}
+
int cg_read(const char *cgroup, const char *control, char *buf, size_t len)
{
char path[PATH_MAX];
@@ -196,7 +206,59 @@ int cg_create(const char *cgroup)
return mkdir(cgroup, 0644);
}
-static int cg_killall(const char *cgroup)
+int cg_for_all_procs(const char *cgroup, int (*fn)(int pid, void *arg),
+ void *arg)
+{
+ char buf[PAGE_SIZE];
+ char *ptr = buf;
+ int ret;
+
+ if (cg_read(cgroup, "cgroup.procs", buf, sizeof(buf)))
+ return -1;
+
+ while (ptr < buf + sizeof(buf)) {
+ int pid = strtol(ptr, &ptr, 10);
+
+ if (pid == 0)
+ break;
+ if (*ptr)
+ ptr++;
+ else
+ break;
+ ret = fn(pid, arg);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+int cg_wait_for_proc_count(const char *cgroup, int count)
+{
+ char buf[10 * PAGE_SIZE] = {0};
+ int attempts;
+ char *ptr;
+
+ for (attempts = 10; attempts >= 0; attempts--) {
+ int nr = 0;
+
+ if (cg_read(cgroup, "cgroup.procs", buf, sizeof(buf)))
+ break;
+
+ for (ptr = buf; *ptr; ptr++)
+ if (*ptr == '\n')
+ nr++;
+
+ if (nr >= count)
+ return 0;
+
+ usleep(100000);
+ }
+
+ return -1;
+}
+
+int cg_killall(const char *cgroup)
{
char buf[PAGE_SIZE];
char *ptr = buf;
@@ -238,6 +300,14 @@ int cg_destroy(const char *cgroup)
return ret;
}
+int cg_enter(const char *cgroup, int pid)
+{
+ char pidbuf[64];
+
+ snprintf(pidbuf, sizeof(pidbuf), "%d", pid);
+ return cg_write(cgroup, "cgroup.procs", pidbuf);
+}
+
int cg_enter_current(const char *cgroup)
{
char pidbuf[64];
@@ -367,3 +437,12 @@ int set_oom_adj_score(int pid, int score)
close(fd);
return 0;
}
+
+char proc_read_text(int pid, const char *item, char *buf, size_t size)
+{
+ char path[PATH_MAX];
+
+ snprintf(path, sizeof(path), "/proc/%d/%s", pid, item);
+
+ return read_text(path, buf, size);
+}
diff --git a/tools/testing/selftests/cgroup/cgroup_util.h b/tools/testing/selftests/cgroup/cgroup_util.h
index 9ac8b7958f83..8ee63c00a668 100644
--- a/tools/testing/selftests/cgroup/cgroup_util.h
+++ b/tools/testing/selftests/cgroup/cgroup_util.h
@@ -18,6 +18,7 @@ static inline int values_close(long a, long b, int err)
extern int cg_find_unified_root(char *root, size_t len);
extern char *cg_name(const char *root, const char *name);
extern char *cg_name_indexed(const char *root, const char *name, int index);
+extern char *cg_control(const char *cgroup, const char *control);
extern int cg_create(const char *cgroup);
extern int cg_destroy(const char *cgroup);
extern int cg_read(const char *cgroup, const char *control,
@@ -32,6 +33,7 @@ extern int cg_write(const char *cgroup, const char *control, char *buf);
extern int cg_run(const char *cgroup,
int (*fn)(const char *cgroup, void *arg),
void *arg);
+extern int cg_enter(const char *cgroup, int pid);
extern int cg_enter_current(const char *cgroup);
extern int cg_run_nowait(const char *cgroup,
int (*fn)(const char *cgroup, void *arg),
@@ -41,3 +43,8 @@ extern int alloc_pagecache(int fd, size_t size);
extern int alloc_anon(const char *cgroup, void *arg);
extern int is_swap_enabled(void);
extern int set_oom_adj_score(int pid, int score);
+extern int cg_for_all_procs(const char *cgroup, int (*fn)(int pid, void *arg),
+ void *arg);
+extern int cg_wait_for_proc_count(const char *cgroup, int count);
+extern int cg_killall(const char *cgroup);
+extern char proc_read_text(int pid, const char *item, char *buf, size_t size);
diff --git a/tools/testing/selftests/cgroup/test_freezer.c b/tools/testing/selftests/cgroup/test_freezer.c
new file mode 100644
index 000000000000..0c4106658360
--- /dev/null
+++ b/tools/testing/selftests/cgroup/test_freezer.c
@@ -0,0 +1,685 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <stdbool.h>
+#include <linux/limits.h>
+#include <sys/ptrace.h>
+#include <sys/types.h>
+#include <sys/mman.h>
+#include <unistd.h>
+#include <stdio.h>
+#include <errno.h>
+#include <poll.h>
+#include <stdlib.h>
+#include <sys/inotify.h>
+#include <string.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+
+#include "../kselftest.h"
+#include "cgroup_util.h"
+
+#define DEBUG
+#ifdef DEBUG
+#define debug(args...) fprintf(stderr, args)
+#else
+#define debug(args...)
+#endif
+
+/*
+ * Freeze the given cgroup and wait for the inotify signal.
+ * If there is no signal in 10 seconds, treat this as an error.
+ */
+static int cg_freeze_wait(const char *cgroup, bool freeze)
+{
+ int fd, wd;
+ struct pollfd fds;
+ int ret = -1;
+
+ fd = inotify_init1(IN_NONBLOCK);
+ if (fd == -1)
+ return fd;
+
+ wd = inotify_add_watch(fd, cg_control(cgroup, "cgroup.events"),
+ IN_MODIFY);
+ if (wd == -1) {
+ close(fd);
+ return wd;
+ }
+ fds.fd = fd;
+ fds.events = POLLIN;
+
+ ret = cg_write(cgroup, "cgroup.freeze", freeze ? "1" : "0");
+ if (ret) {
+ close(fd);
+ return ret;
+ }
+
+ while (true) {
+ wd = poll(&fds, 1, 10000);
+
+ if (wd == -1 && errno == EINTR)
+ continue;
+
+ if (wd == 1 && fds.revents & POLLIN)
+ ret = 0;
+
+ break;
+ }
+
+ close(fd);
+
+ return ret;
+}
+
+/*
+ * Check if the process is frozen and parked in a proper place.
+ */
+static int proc_check_frozen(int pid, void *arg)
+{
+ char buf[PAGE_SIZE];
+ int len;
+
+ len = proc_read_text(pid, "stat", buf, sizeof(buf));
+ if (len == -1) {
+ debug("Can't get %d stat\n", pid);
+ return -1;
+ }
+
+ if (strstr(buf, "(test_freezer) S ") == NULL) {
+ debug("Process %d in the unexpected state: %s\n", pid, buf);
+ return -1;
+ }
+
+ len = proc_read_text(pid, "stack", buf, sizeof(buf));
+ if (len == -1) {
+ debug("Can't get stack of the process %d\n", pid);
+ return -1;
+ }
+
+ if (strstr(buf, "[<0>] cgroup_enter_frozen") != buf) {
+ debug("Process %d has unexpected stacktrace: %s\n", pid, buf);
+ return -1;
+ }
+
+ return 0;
+}
+
+/*
+ * Check if the cgroup is frozen and all belonging processes are
+ * parked in a proper place.
+ */
+static int cg_check_frozen(const char *cgroup, bool frozen)
+{
+ if (frozen) {
+ /*
+ * Check the cgroup.events::frozen value.
+ */
+ if (cg_read_strstr(cgroup, "cgroup.events", "frozen 1") != 0) {
+ debug("Cgroup %s isn't unexpectedly frozen\n", cgroup);
+ return -1;
+ }
+
+ /*
+ * Check that all processes are parked in the proper place.
+ */
+ if (cg_for_all_procs(cgroup, proc_check_frozen, NULL)) {
+ debug("Some processes of cgroup %s are not frozen\n",
+ cgroup);
+ return -1;
+ }
+ } else {
+ /*
+ * Check the cgroup.events::frozen value.
+ */
+ if (cg_read_strstr(cgroup, "cgroup.events", "frozen 0") != 0) {
+ debug("Cgroup %s is unexpectedly frozen\n", cgroup);
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+/*
+ * A simple process running in a sleep loop until being
+ * re-parented.
+ */
+static int child_fn(const char *cgroup, void *arg)
+{
+ int ppid = getppid();
+
+ while (getppid() == ppid)
+ usleep(1000);
+
+ return getppid() == ppid;
+}
+
+/*
+ * A simple test for the cgroup freezer: populated the cgroup with 100
+ * running processes and freeze it. Then unfreeze it. Then it kills all
+ * processes and destroys the cgroup.
+ */
+static int test_cgfreezer_simple(const char *root)
+{
+ int ret = KSFT_FAIL;
+ char *cgroup = NULL;
+ int i;
+
+ cgroup = cg_name(root, "cg_test");
+ if (!cgroup)
+ goto cleanup;
+
+ if (cg_create(cgroup))
+ goto cleanup;
+
+ for (i = 0; i < 100; i++)
+ cg_run_nowait(cgroup, child_fn, NULL);
+
+ if (cg_wait_for_proc_count(cgroup, 100))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup, false))
+ goto cleanup;
+
+ if (cg_freeze_wait(cgroup, true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup, true))
+ goto cleanup;
+
+ if (cg_freeze_wait(cgroup, false))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup, false))
+ goto cleanup;
+
+ ret = KSFT_PASS;
+
+cleanup:
+ if (cgroup)
+ cg_destroy(cgroup);
+ free(cgroup);
+ return ret;
+}
+
+/*
+ * The test creates the following hierarchy:
+ * A
+ * / / \ \
+ * B E I K
+ * /\ |
+ * C D F
+ * |
+ * G
+ * |
+ * H
+ *
+ * with a process in C, H and 3 processes in K.
+ * Then it tries to freeze and unfreeze the whole tree.
+ */
+static int test_cgfreezer_tree(const char *root)
+{
+ char *cgroup[10] = {0};
+ int ret = KSFT_FAIL;
+ int i;
+
+ cgroup[0] = cg_name(root, "cg_test_A");
+ if (!cgroup[0])
+ goto cleanup;
+
+ cgroup[1] = cg_name(cgroup[0], "cg_test_B");
+ if (!cgroup[1])
+ goto cleanup;
+
+ cgroup[2] = cg_name(cgroup[1], "cg_test_C");
+ if (!cgroup[2])
+ goto cleanup;
+
+ cgroup[3] = cg_name(cgroup[1], "cg_test_D");
+ if (!cgroup[3])
+ goto cleanup;
+
+ cgroup[4] = cg_name(cgroup[0], "cg_test_E");
+ if (!cgroup[4])
+ goto cleanup;
+
+ cgroup[5] = cg_name(cgroup[4], "cg_test_F");
+ if (!cgroup[5])
+ goto cleanup;
+
+ cgroup[6] = cg_name(cgroup[5], "cg_test_G");
+ if (!cgroup[6])
+ goto cleanup;
+
+ cgroup[7] = cg_name(cgroup[6], "cg_test_H");
+ if (!cgroup[7])
+ goto cleanup;
+
+ cgroup[8] = cg_name(cgroup[0], "cg_test_I");
+ if (!cgroup[8])
+ goto cleanup;
+
+ cgroup[9] = cg_name(cgroup[0], "cg_test_K");
+ if (!cgroup[9])
+ goto cleanup;
+
+ for (i = 0; i < 10; i++)
+ if (cg_create(cgroup[i]))
+ goto cleanup;
+
+ cg_run_nowait(cgroup[2], child_fn, NULL);
+ cg_run_nowait(cgroup[7], child_fn, NULL);
+ cg_run_nowait(cgroup[9], child_fn, NULL);
+ cg_run_nowait(cgroup[9], child_fn, NULL);
+ cg_run_nowait(cgroup[9], child_fn, NULL);
+
+ /*
+ * Wait until all child processes will enter
+ * corresponding cgroups.
+ */
+
+ if (cg_wait_for_proc_count(cgroup[2], 1) ||
+ cg_wait_for_proc_count(cgroup[7], 1) ||
+ cg_wait_for_proc_count(cgroup[9], 3))
+ goto cleanup;
+
+ /*
+ * Freeze B.
+ */
+ if (cg_freeze_wait(cgroup[1], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[1], true))
+ goto cleanup;
+
+ /*
+ * Freeze F.
+ */
+ if (cg_freeze_wait(cgroup[5], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[5], true))
+ goto cleanup;
+
+ /*
+ * Freeze G.
+ */
+ if (cg_freeze_wait(cgroup[6], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[6], true))
+ goto cleanup;
+
+ /*
+ * Check that A and E are not frozen.
+ */
+ if (cg_check_frozen(cgroup[0], false))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[4], false))
+ goto cleanup;
+
+ /*
+ * Freeze A. Check that A, B and E are frozen.
+ */
+ if (cg_freeze_wait(cgroup[0], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[0], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[1], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[4], true))
+ goto cleanup;
+
+ /*
+ * Unfreeze B, F and G
+ */
+ if (cg_freeze_wait(cgroup[1], false))
+ goto cleanup;
+
+ if (cg_freeze_wait(cgroup[5], false))
+ goto cleanup;
+
+ if (cg_freeze_wait(cgroup[6], false))
+ goto cleanup;
+
+ /*
+ * Check that C and H are still frozen.
+ */
+ if (cg_check_frozen(cgroup[2], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[7], true))
+ goto cleanup;
+
+ /*
+ * Unfreezing A failed. Check that A, C and K are not frozen.
+ */
+ if (cg_freeze_wait(cgroup[0], false))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[0], false))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[2], false))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[9], false))
+ goto cleanup;
+
+ ret = KSFT_PASS;
+
+cleanup:
+ for (i = 9; i >= 0 && cgroup[i]; i--) {
+ cg_destroy(cgroup[i]);
+ free(cgroup[i]);
+ }
+
+ return ret;
+}
+
+/*
+ * A fork bomb emulator.
+ */
+static int forkbomb_fn(const char *cgroup, void *arg)
+{
+ int ppid;
+
+ fork();
+ fork();
+
+ ppid = getppid();
+
+ while (getppid() == ppid)
+ usleep(1000);
+
+ return getppid() == ppid;
+}
+
+/*
+ * The test runs a fork bomb in a cgroup and tries to freeze it.
+ * Then it kills all processes and checks that cgroup isn't populated
+ * anymore.
+ */
+static int test_cgfreezer_forkbomb(const char *root)
+{
+ int ret = KSFT_FAIL;
+ char *cgroup = NULL;
+
+ cgroup = cg_name(root, "cg_forkbomb_test");
+ if (!cgroup)
+ goto cleanup;
+
+ if (cg_create(cgroup))
+ goto cleanup;
+
+ cg_run_nowait(cgroup, forkbomb_fn, NULL);
+
+ usleep(100000);
+
+ if (cg_freeze_wait(cgroup, true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup, true))
+ goto cleanup;
+
+ if (cg_killall(cgroup))
+ goto cleanup;
+
+ if (cg_wait_for_proc_count(cgroup, 0))
+ goto cleanup;
+
+ ret = KSFT_PASS;
+
+cleanup:
+ if (cgroup)
+ cg_destroy(cgroup);
+ free(cgroup);
+ return ret;
+}
+
+/*
+ * The test creates two nested cgroups, freezes the parent
+ * and removes the child. Then it checks that the parent cgroup
+ * remains frozen and it's possible to create a new child
+ * without unfreezing. The new child is frozen too.
+ */
+static int test_cgfreezer_rmdir(const char *root)
+{
+ int ret = KSFT_FAIL;
+ char *parent, *child = NULL;
+
+ parent = cg_name(root, "cg_test_A");
+ if (!parent)
+ goto cleanup;
+
+ child = cg_name(parent, "cg_test_B");
+ if (!child)
+ goto cleanup;
+
+ if (cg_create(parent))
+ goto cleanup;
+
+ if (cg_create(child))
+ goto cleanup;
+
+ if (cg_freeze_wait(parent, true))
+ goto cleanup;
+
+ if (cg_check_frozen(parent, true))
+ goto cleanup;
+
+ if (cg_destroy(child))
+ goto cleanup;
+
+ if (cg_check_frozen(parent, true))
+ goto cleanup;
+
+ if (cg_create(child))
+ goto cleanup;
+
+ if (cg_check_frozen(child, true))
+ goto cleanup;
+
+ ret = KSFT_PASS;
+
+cleanup:
+ if (child)
+ cg_destroy(child);
+ free(child);
+ if (parent)
+ cg_destroy(parent);
+ free(parent);
+ return ret;
+}
+
+/*
+ * The test creates two cgroups: A and B. The it runs a process in A,
+ * and performs several migrations:
+ * 1) A (running) -> B (frozen)
+ * 2) B (frozen) -> A (running)
+ * 3) A (frozen) -> B (frozen)
+ *
+ * One each step it checks that the actual state of cgroups matches
+ * the expected state.
+ */
+static int test_cgfreezer_migrate(const char *root)
+{
+ int ret = KSFT_FAIL;
+ char *cgroup[2] = {0};
+ int pid;
+
+ cgroup[0] = cg_name(root, "cg_test_A");
+ if (!cgroup[0])
+ goto cleanup;
+
+ cgroup[1] = cg_name(root, "cg_test_B");
+ if (!cgroup[1])
+ goto cleanup;
+
+ if (cg_create(cgroup[0]))
+ goto cleanup;
+
+ if (cg_create(cgroup[1]))
+ goto cleanup;
+
+ pid = cg_run_nowait(cgroup[0], child_fn, NULL);
+ if (pid < 0)
+ goto cleanup;
+
+ if (cg_wait_for_proc_count(cgroup[0], 1))
+ goto cleanup;
+
+ /*
+ * Migrate from A (running) to B (frozen)
+ */
+ if (cg_freeze_wait(cgroup[1], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[1], true))
+ goto cleanup;
+
+ if (cg_enter(cgroup[1], pid))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[0], false))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[1], true))
+ goto cleanup;
+
+ /*
+ * Migrate from B (frozen) to A (running)
+ */
+ if (cg_enter(cgroup[0], pid))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[0], false))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[1], true))
+ goto cleanup;
+
+ /*
+ * Migrate from A (frozen) to B (frozen)
+ */
+ if (cg_freeze_wait(cgroup[0], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[0], true))
+ goto cleanup;
+
+ if (cg_enter(cgroup[1], pid))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[0], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[1], true))
+ goto cleanup;
+
+ ret = KSFT_PASS;
+
+cleanup:
+ if (cgroup[0])
+ cg_destroy(cgroup[0]);
+ free(cgroup[0]);
+ if (cgroup[1])
+ cg_destroy(cgroup[1]);
+ free(cgroup[1]);
+ return ret;
+}
+
+/*
+ * The test checks that ptrace works with a tracing process in a frozen cgroup.
+ */
+static int test_cgfreezer_ptrace(const char *root)
+{
+ int ret = KSFT_FAIL;
+ char *cgroup = NULL;
+ siginfo_t siginfo;
+ int pid;
+
+ cgroup = cg_name(root, "cg_test");
+ if (!cgroup)
+ goto cleanup;
+
+ if (cg_create(cgroup))
+ goto cleanup;
+
+ pid = cg_run_nowait(cgroup, child_fn, NULL);
+ if (pid < 0)
+ goto cleanup;
+
+ if (cg_wait_for_proc_count(cgroup, 1))
+ goto cleanup;
+
+ if (cg_freeze_wait(cgroup, true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup, true))
+ goto cleanup;
+
+ if (ptrace(PTRACE_SEIZE, pid, NULL, NULL))
+ goto cleanup;
+
+ if (ptrace(PTRACE_INTERRUPT, pid, NULL, NULL))
+ goto cleanup;
+
+ waitpid(pid, NULL, 0);
+
+ if (ptrace(PTRACE_GETSIGINFO, pid, NULL, &siginfo))
+ goto cleanup;
+
+ if (ptrace(PTRACE_DETACH, pid, NULL, NULL))
+ goto cleanup;
+
+ ret = KSFT_PASS;
+
+cleanup:
+ if (cgroup)
+ cg_destroy(cgroup);
+ free(cgroup);
+ return ret;
+}
+
+#define T(x) { x, #x }
+struct cgfreezer_test {
+ int (*fn)(const char *root);
+ const char *name;
+} tests[] = {
+ T(test_cgfreezer_simple),
+ T(test_cgfreezer_tree),
+ T(test_cgfreezer_forkbomb),
+ T(test_cgfreezer_rmdir),
+ T(test_cgfreezer_migrate),
+ T(test_cgfreezer_ptrace),
+};
+#undef T
+
+int main(int argc, char *argv[])
+{
+ char root[PATH_MAX];
+ int i, ret = EXIT_SUCCESS;
+
+ if (cg_find_unified_root(root, sizeof(root)))
+ ksft_exit_skip("cgroup v2 isn't mounted\n");
+ for (i = 0; i < ARRAY_SIZE(tests); i++) {
+ switch (tests[i].fn(root)) {
+ case KSFT_PASS:
+ ksft_test_result_pass("%s\n", tests[i].name);
+ break;
+ case KSFT_SKIP:
+ ksft_test_result_skip("%s\n", tests[i].name);
+ break;
+ default:
+ ret = EXIT_FAILURE;
+ ksft_test_result_fail("%s\n", tests[i].name);
+ break;
+ }
+ }
+
+ return ret;
+}
--
2.17.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v3 6/7] kselftests: cgroup: add freezer controller self-tests
@ 2018-11-17 0:38 ` guroan
0 siblings, 0 replies; 23+ messages in thread
From: guroan @ 2018-11-17 0:38 UTC (permalink / raw)
This patch implements six tests for the freezer controller for
cgroup v2:
1) a simple test, which aims to freeze and unfreeze a cgroup with 100
processes
2) a more complicated tree test, which creates a hierarchy of cgroups,
puts some processes in some cgroups, and tries to freeze and unfreeze
different parts of the subtree
3) a forkbomb test: the test aims to freeze a forkbomb running in a
cgroup, kill all tasks in the cgroup and remove the cgroup without
the unfreezing.
4) rmdir test: the test creates two nested cgroups, freezes the parent
one, checks that the child can be successfully removed, and a new
child can be created
5) migration tests: the test checks migration of a task between
frozen cgroups: from a frozen to a running, from a running to a
frozen, and from a frozen to a frozen.
6) ptrace test: the test checks that it's possible to attach to
a process in a frozen cgroup, get some information and detach, and
the cgroup will remain frozen.
Expected output:
$ ./test_freezer
ok 1 test_cgfreezer_simple
ok 2 test_cgfreezer_tree
ok 3 test_cgfreezer_forkbomb
ok 4 test_cgrreezer_rmdir
ok 5 test_cgfreezer_migrate
ok 6 test_cgfreezer_ptrace
Signed-off-by: Roman Gushchin <guro at fb.com>
Cc: Shuah Khan <shuah at kernel.org>
Cc: Tejun Heo <tj at kernel.org>
Cc: kernel-team at fb.com
Cc: linux-kselftest at vger.kernel.org
---
tools/testing/selftests/cgroup/.gitignore | 1 +
tools/testing/selftests/cgroup/Makefile | 2 +
tools/testing/selftests/cgroup/cgroup_util.c | 81 ++-
tools/testing/selftests/cgroup/cgroup_util.h | 7 +
tools/testing/selftests/cgroup/test_freezer.c | 685 ++++++++++++++++++
5 files changed, 775 insertions(+), 1 deletion(-)
create mode 100644 tools/testing/selftests/cgroup/test_freezer.c
diff --git a/tools/testing/selftests/cgroup/.gitignore b/tools/testing/selftests/cgroup/.gitignore
index adacda50a4b2..7f9835624793 100644
--- a/tools/testing/selftests/cgroup/.gitignore
+++ b/tools/testing/selftests/cgroup/.gitignore
@@ -1,2 +1,3 @@
test_memcontrol
test_core
+test_freezer
diff --git a/tools/testing/selftests/cgroup/Makefile b/tools/testing/selftests/cgroup/Makefile
index 23fbaa4a9630..8d369b6a2069 100644
--- a/tools/testing/selftests/cgroup/Makefile
+++ b/tools/testing/selftests/cgroup/Makefile
@@ -5,8 +5,10 @@ all:
TEST_GEN_PROGS = test_memcontrol
TEST_GEN_PROGS += test_core
+TEST_GEN_PROGS += test_freezer
include ../lib.mk
$(OUTPUT)/test_memcontrol: cgroup_util.c
$(OUTPUT)/test_core: cgroup_util.c
+$(OUTPUT)/test_freezer: cgroup_util.c
diff --git a/tools/testing/selftests/cgroup/cgroup_util.c b/tools/testing/selftests/cgroup/cgroup_util.c
index eba06f94433b..e9cdad673901 100644
--- a/tools/testing/selftests/cgroup/cgroup_util.c
+++ b/tools/testing/selftests/cgroup/cgroup_util.c
@@ -74,6 +74,16 @@ char *cg_name_indexed(const char *root, const char *name, int index)
return ret;
}
+char *cg_control(const char *cgroup, const char *control)
+{
+ size_t len = strlen(cgroup) + strlen(control) + 2;
+ char *ret = malloc(len);
+
+ snprintf(ret, len, "%s/%s", cgroup, control);
+
+ return ret;
+}
+
int cg_read(const char *cgroup, const char *control, char *buf, size_t len)
{
char path[PATH_MAX];
@@ -196,7 +206,59 @@ int cg_create(const char *cgroup)
return mkdir(cgroup, 0644);
}
-static int cg_killall(const char *cgroup)
+int cg_for_all_procs(const char *cgroup, int (*fn)(int pid, void *arg),
+ void *arg)
+{
+ char buf[PAGE_SIZE];
+ char *ptr = buf;
+ int ret;
+
+ if (cg_read(cgroup, "cgroup.procs", buf, sizeof(buf)))
+ return -1;
+
+ while (ptr < buf + sizeof(buf)) {
+ int pid = strtol(ptr, &ptr, 10);
+
+ if (pid == 0)
+ break;
+ if (*ptr)
+ ptr++;
+ else
+ break;
+ ret = fn(pid, arg);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+int cg_wait_for_proc_count(const char *cgroup, int count)
+{
+ char buf[10 * PAGE_SIZE] = {0};
+ int attempts;
+ char *ptr;
+
+ for (attempts = 10; attempts >= 0; attempts--) {
+ int nr = 0;
+
+ if (cg_read(cgroup, "cgroup.procs", buf, sizeof(buf)))
+ break;
+
+ for (ptr = buf; *ptr; ptr++)
+ if (*ptr == '\n')
+ nr++;
+
+ if (nr >= count)
+ return 0;
+
+ usleep(100000);
+ }
+
+ return -1;
+}
+
+int cg_killall(const char *cgroup)
{
char buf[PAGE_SIZE];
char *ptr = buf;
@@ -238,6 +300,14 @@ int cg_destroy(const char *cgroup)
return ret;
}
+int cg_enter(const char *cgroup, int pid)
+{
+ char pidbuf[64];
+
+ snprintf(pidbuf, sizeof(pidbuf), "%d", pid);
+ return cg_write(cgroup, "cgroup.procs", pidbuf);
+}
+
int cg_enter_current(const char *cgroup)
{
char pidbuf[64];
@@ -367,3 +437,12 @@ int set_oom_adj_score(int pid, int score)
close(fd);
return 0;
}
+
+char proc_read_text(int pid, const char *item, char *buf, size_t size)
+{
+ char path[PATH_MAX];
+
+ snprintf(path, sizeof(path), "/proc/%d/%s", pid, item);
+
+ return read_text(path, buf, size);
+}
diff --git a/tools/testing/selftests/cgroup/cgroup_util.h b/tools/testing/selftests/cgroup/cgroup_util.h
index 9ac8b7958f83..8ee63c00a668 100644
--- a/tools/testing/selftests/cgroup/cgroup_util.h
+++ b/tools/testing/selftests/cgroup/cgroup_util.h
@@ -18,6 +18,7 @@ static inline int values_close(long a, long b, int err)
extern int cg_find_unified_root(char *root, size_t len);
extern char *cg_name(const char *root, const char *name);
extern char *cg_name_indexed(const char *root, const char *name, int index);
+extern char *cg_control(const char *cgroup, const char *control);
extern int cg_create(const char *cgroup);
extern int cg_destroy(const char *cgroup);
extern int cg_read(const char *cgroup, const char *control,
@@ -32,6 +33,7 @@ extern int cg_write(const char *cgroup, const char *control, char *buf);
extern int cg_run(const char *cgroup,
int (*fn)(const char *cgroup, void *arg),
void *arg);
+extern int cg_enter(const char *cgroup, int pid);
extern int cg_enter_current(const char *cgroup);
extern int cg_run_nowait(const char *cgroup,
int (*fn)(const char *cgroup, void *arg),
@@ -41,3 +43,8 @@ extern int alloc_pagecache(int fd, size_t size);
extern int alloc_anon(const char *cgroup, void *arg);
extern int is_swap_enabled(void);
extern int set_oom_adj_score(int pid, int score);
+extern int cg_for_all_procs(const char *cgroup, int (*fn)(int pid, void *arg),
+ void *arg);
+extern int cg_wait_for_proc_count(const char *cgroup, int count);
+extern int cg_killall(const char *cgroup);
+extern char proc_read_text(int pid, const char *item, char *buf, size_t size);
diff --git a/tools/testing/selftests/cgroup/test_freezer.c b/tools/testing/selftests/cgroup/test_freezer.c
new file mode 100644
index 000000000000..0c4106658360
--- /dev/null
+++ b/tools/testing/selftests/cgroup/test_freezer.c
@@ -0,0 +1,685 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <stdbool.h>
+#include <linux/limits.h>
+#include <sys/ptrace.h>
+#include <sys/types.h>
+#include <sys/mman.h>
+#include <unistd.h>
+#include <stdio.h>
+#include <errno.h>
+#include <poll.h>
+#include <stdlib.h>
+#include <sys/inotify.h>
+#include <string.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+
+#include "../kselftest.h"
+#include "cgroup_util.h"
+
+#define DEBUG
+#ifdef DEBUG
+#define debug(args...) fprintf(stderr, args)
+#else
+#define debug(args...)
+#endif
+
+/*
+ * Freeze the given cgroup and wait for the inotify signal.
+ * If there is no signal in 10 seconds, treat this as an error.
+ */
+static int cg_freeze_wait(const char *cgroup, bool freeze)
+{
+ int fd, wd;
+ struct pollfd fds;
+ int ret = -1;
+
+ fd = inotify_init1(IN_NONBLOCK);
+ if (fd == -1)
+ return fd;
+
+ wd = inotify_add_watch(fd, cg_control(cgroup, "cgroup.events"),
+ IN_MODIFY);
+ if (wd == -1) {
+ close(fd);
+ return wd;
+ }
+ fds.fd = fd;
+ fds.events = POLLIN;
+
+ ret = cg_write(cgroup, "cgroup.freeze", freeze ? "1" : "0");
+ if (ret) {
+ close(fd);
+ return ret;
+ }
+
+ while (true) {
+ wd = poll(&fds, 1, 10000);
+
+ if (wd == -1 && errno == EINTR)
+ continue;
+
+ if (wd == 1 && fds.revents & POLLIN)
+ ret = 0;
+
+ break;
+ }
+
+ close(fd);
+
+ return ret;
+}
+
+/*
+ * Check if the process is frozen and parked in a proper place.
+ */
+static int proc_check_frozen(int pid, void *arg)
+{
+ char buf[PAGE_SIZE];
+ int len;
+
+ len = proc_read_text(pid, "stat", buf, sizeof(buf));
+ if (len == -1) {
+ debug("Can't get %d stat\n", pid);
+ return -1;
+ }
+
+ if (strstr(buf, "(test_freezer) S ") == NULL) {
+ debug("Process %d in the unexpected state: %s\n", pid, buf);
+ return -1;
+ }
+
+ len = proc_read_text(pid, "stack", buf, sizeof(buf));
+ if (len == -1) {
+ debug("Can't get stack of the process %d\n", pid);
+ return -1;
+ }
+
+ if (strstr(buf, "[<0>] cgroup_enter_frozen") != buf) {
+ debug("Process %d has unexpected stacktrace: %s\n", pid, buf);
+ return -1;
+ }
+
+ return 0;
+}
+
+/*
+ * Check if the cgroup is frozen and all belonging processes are
+ * parked in a proper place.
+ */
+static int cg_check_frozen(const char *cgroup, bool frozen)
+{
+ if (frozen) {
+ /*
+ * Check the cgroup.events::frozen value.
+ */
+ if (cg_read_strstr(cgroup, "cgroup.events", "frozen 1") != 0) {
+ debug("Cgroup %s isn't unexpectedly frozen\n", cgroup);
+ return -1;
+ }
+
+ /*
+ * Check that all processes are parked in the proper place.
+ */
+ if (cg_for_all_procs(cgroup, proc_check_frozen, NULL)) {
+ debug("Some processes of cgroup %s are not frozen\n",
+ cgroup);
+ return -1;
+ }
+ } else {
+ /*
+ * Check the cgroup.events::frozen value.
+ */
+ if (cg_read_strstr(cgroup, "cgroup.events", "frozen 0") != 0) {
+ debug("Cgroup %s is unexpectedly frozen\n", cgroup);
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+/*
+ * A simple process running in a sleep loop until being
+ * re-parented.
+ */
+static int child_fn(const char *cgroup, void *arg)
+{
+ int ppid = getppid();
+
+ while (getppid() == ppid)
+ usleep(1000);
+
+ return getppid() == ppid;
+}
+
+/*
+ * A simple test for the cgroup freezer: populated the cgroup with 100
+ * running processes and freeze it. Then unfreeze it. Then it kills all
+ * processes and destroys the cgroup.
+ */
+static int test_cgfreezer_simple(const char *root)
+{
+ int ret = KSFT_FAIL;
+ char *cgroup = NULL;
+ int i;
+
+ cgroup = cg_name(root, "cg_test");
+ if (!cgroup)
+ goto cleanup;
+
+ if (cg_create(cgroup))
+ goto cleanup;
+
+ for (i = 0; i < 100; i++)
+ cg_run_nowait(cgroup, child_fn, NULL);
+
+ if (cg_wait_for_proc_count(cgroup, 100))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup, false))
+ goto cleanup;
+
+ if (cg_freeze_wait(cgroup, true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup, true))
+ goto cleanup;
+
+ if (cg_freeze_wait(cgroup, false))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup, false))
+ goto cleanup;
+
+ ret = KSFT_PASS;
+
+cleanup:
+ if (cgroup)
+ cg_destroy(cgroup);
+ free(cgroup);
+ return ret;
+}
+
+/*
+ * The test creates the following hierarchy:
+ * A
+ * / / \ \
+ * B E I K
+ * /\ |
+ * C D F
+ * |
+ * G
+ * |
+ * H
+ *
+ * with a process in C, H and 3 processes in K.
+ * Then it tries to freeze and unfreeze the whole tree.
+ */
+static int test_cgfreezer_tree(const char *root)
+{
+ char *cgroup[10] = {0};
+ int ret = KSFT_FAIL;
+ int i;
+
+ cgroup[0] = cg_name(root, "cg_test_A");
+ if (!cgroup[0])
+ goto cleanup;
+
+ cgroup[1] = cg_name(cgroup[0], "cg_test_B");
+ if (!cgroup[1])
+ goto cleanup;
+
+ cgroup[2] = cg_name(cgroup[1], "cg_test_C");
+ if (!cgroup[2])
+ goto cleanup;
+
+ cgroup[3] = cg_name(cgroup[1], "cg_test_D");
+ if (!cgroup[3])
+ goto cleanup;
+
+ cgroup[4] = cg_name(cgroup[0], "cg_test_E");
+ if (!cgroup[4])
+ goto cleanup;
+
+ cgroup[5] = cg_name(cgroup[4], "cg_test_F");
+ if (!cgroup[5])
+ goto cleanup;
+
+ cgroup[6] = cg_name(cgroup[5], "cg_test_G");
+ if (!cgroup[6])
+ goto cleanup;
+
+ cgroup[7] = cg_name(cgroup[6], "cg_test_H");
+ if (!cgroup[7])
+ goto cleanup;
+
+ cgroup[8] = cg_name(cgroup[0], "cg_test_I");
+ if (!cgroup[8])
+ goto cleanup;
+
+ cgroup[9] = cg_name(cgroup[0], "cg_test_K");
+ if (!cgroup[9])
+ goto cleanup;
+
+ for (i = 0; i < 10; i++)
+ if (cg_create(cgroup[i]))
+ goto cleanup;
+
+ cg_run_nowait(cgroup[2], child_fn, NULL);
+ cg_run_nowait(cgroup[7], child_fn, NULL);
+ cg_run_nowait(cgroup[9], child_fn, NULL);
+ cg_run_nowait(cgroup[9], child_fn, NULL);
+ cg_run_nowait(cgroup[9], child_fn, NULL);
+
+ /*
+ * Wait until all child processes will enter
+ * corresponding cgroups.
+ */
+
+ if (cg_wait_for_proc_count(cgroup[2], 1) ||
+ cg_wait_for_proc_count(cgroup[7], 1) ||
+ cg_wait_for_proc_count(cgroup[9], 3))
+ goto cleanup;
+
+ /*
+ * Freeze B.
+ */
+ if (cg_freeze_wait(cgroup[1], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[1], true))
+ goto cleanup;
+
+ /*
+ * Freeze F.
+ */
+ if (cg_freeze_wait(cgroup[5], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[5], true))
+ goto cleanup;
+
+ /*
+ * Freeze G.
+ */
+ if (cg_freeze_wait(cgroup[6], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[6], true))
+ goto cleanup;
+
+ /*
+ * Check that A and E are not frozen.
+ */
+ if (cg_check_frozen(cgroup[0], false))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[4], false))
+ goto cleanup;
+
+ /*
+ * Freeze A. Check that A, B and E are frozen.
+ */
+ if (cg_freeze_wait(cgroup[0], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[0], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[1], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[4], true))
+ goto cleanup;
+
+ /*
+ * Unfreeze B, F and G
+ */
+ if (cg_freeze_wait(cgroup[1], false))
+ goto cleanup;
+
+ if (cg_freeze_wait(cgroup[5], false))
+ goto cleanup;
+
+ if (cg_freeze_wait(cgroup[6], false))
+ goto cleanup;
+
+ /*
+ * Check that C and H are still frozen.
+ */
+ if (cg_check_frozen(cgroup[2], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[7], true))
+ goto cleanup;
+
+ /*
+ * Unfreezing A failed. Check that A, C and K are not frozen.
+ */
+ if (cg_freeze_wait(cgroup[0], false))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[0], false))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[2], false))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[9], false))
+ goto cleanup;
+
+ ret = KSFT_PASS;
+
+cleanup:
+ for (i = 9; i >= 0 && cgroup[i]; i--) {
+ cg_destroy(cgroup[i]);
+ free(cgroup[i]);
+ }
+
+ return ret;
+}
+
+/*
+ * A fork bomb emulator.
+ */
+static int forkbomb_fn(const char *cgroup, void *arg)
+{
+ int ppid;
+
+ fork();
+ fork();
+
+ ppid = getppid();
+
+ while (getppid() == ppid)
+ usleep(1000);
+
+ return getppid() == ppid;
+}
+
+/*
+ * The test runs a fork bomb in a cgroup and tries to freeze it.
+ * Then it kills all processes and checks that cgroup isn't populated
+ * anymore.
+ */
+static int test_cgfreezer_forkbomb(const char *root)
+{
+ int ret = KSFT_FAIL;
+ char *cgroup = NULL;
+
+ cgroup = cg_name(root, "cg_forkbomb_test");
+ if (!cgroup)
+ goto cleanup;
+
+ if (cg_create(cgroup))
+ goto cleanup;
+
+ cg_run_nowait(cgroup, forkbomb_fn, NULL);
+
+ usleep(100000);
+
+ if (cg_freeze_wait(cgroup, true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup, true))
+ goto cleanup;
+
+ if (cg_killall(cgroup))
+ goto cleanup;
+
+ if (cg_wait_for_proc_count(cgroup, 0))
+ goto cleanup;
+
+ ret = KSFT_PASS;
+
+cleanup:
+ if (cgroup)
+ cg_destroy(cgroup);
+ free(cgroup);
+ return ret;
+}
+
+/*
+ * The test creates two nested cgroups, freezes the parent
+ * and removes the child. Then it checks that the parent cgroup
+ * remains frozen and it's possible to create a new child
+ * without unfreezing. The new child is frozen too.
+ */
+static int test_cgfreezer_rmdir(const char *root)
+{
+ int ret = KSFT_FAIL;
+ char *parent, *child = NULL;
+
+ parent = cg_name(root, "cg_test_A");
+ if (!parent)
+ goto cleanup;
+
+ child = cg_name(parent, "cg_test_B");
+ if (!child)
+ goto cleanup;
+
+ if (cg_create(parent))
+ goto cleanup;
+
+ if (cg_create(child))
+ goto cleanup;
+
+ if (cg_freeze_wait(parent, true))
+ goto cleanup;
+
+ if (cg_check_frozen(parent, true))
+ goto cleanup;
+
+ if (cg_destroy(child))
+ goto cleanup;
+
+ if (cg_check_frozen(parent, true))
+ goto cleanup;
+
+ if (cg_create(child))
+ goto cleanup;
+
+ if (cg_check_frozen(child, true))
+ goto cleanup;
+
+ ret = KSFT_PASS;
+
+cleanup:
+ if (child)
+ cg_destroy(child);
+ free(child);
+ if (parent)
+ cg_destroy(parent);
+ free(parent);
+ return ret;
+}
+
+/*
+ * The test creates two cgroups: A and B. The it runs a process in A,
+ * and performs several migrations:
+ * 1) A (running) -> B (frozen)
+ * 2) B (frozen) -> A (running)
+ * 3) A (frozen) -> B (frozen)
+ *
+ * One each step it checks that the actual state of cgroups matches
+ * the expected state.
+ */
+static int test_cgfreezer_migrate(const char *root)
+{
+ int ret = KSFT_FAIL;
+ char *cgroup[2] = {0};
+ int pid;
+
+ cgroup[0] = cg_name(root, "cg_test_A");
+ if (!cgroup[0])
+ goto cleanup;
+
+ cgroup[1] = cg_name(root, "cg_test_B");
+ if (!cgroup[1])
+ goto cleanup;
+
+ if (cg_create(cgroup[0]))
+ goto cleanup;
+
+ if (cg_create(cgroup[1]))
+ goto cleanup;
+
+ pid = cg_run_nowait(cgroup[0], child_fn, NULL);
+ if (pid < 0)
+ goto cleanup;
+
+ if (cg_wait_for_proc_count(cgroup[0], 1))
+ goto cleanup;
+
+ /*
+ * Migrate from A (running) to B (frozen)
+ */
+ if (cg_freeze_wait(cgroup[1], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[1], true))
+ goto cleanup;
+
+ if (cg_enter(cgroup[1], pid))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[0], false))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[1], true))
+ goto cleanup;
+
+ /*
+ * Migrate from B (frozen) to A (running)
+ */
+ if (cg_enter(cgroup[0], pid))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[0], false))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[1], true))
+ goto cleanup;
+
+ /*
+ * Migrate from A (frozen) to B (frozen)
+ */
+ if (cg_freeze_wait(cgroup[0], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[0], true))
+ goto cleanup;
+
+ if (cg_enter(cgroup[1], pid))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[0], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[1], true))
+ goto cleanup;
+
+ ret = KSFT_PASS;
+
+cleanup:
+ if (cgroup[0])
+ cg_destroy(cgroup[0]);
+ free(cgroup[0]);
+ if (cgroup[1])
+ cg_destroy(cgroup[1]);
+ free(cgroup[1]);
+ return ret;
+}
+
+/*
+ * The test checks that ptrace works with a tracing process in a frozen cgroup.
+ */
+static int test_cgfreezer_ptrace(const char *root)
+{
+ int ret = KSFT_FAIL;
+ char *cgroup = NULL;
+ siginfo_t siginfo;
+ int pid;
+
+ cgroup = cg_name(root, "cg_test");
+ if (!cgroup)
+ goto cleanup;
+
+ if (cg_create(cgroup))
+ goto cleanup;
+
+ pid = cg_run_nowait(cgroup, child_fn, NULL);
+ if (pid < 0)
+ goto cleanup;
+
+ if (cg_wait_for_proc_count(cgroup, 1))
+ goto cleanup;
+
+ if (cg_freeze_wait(cgroup, true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup, true))
+ goto cleanup;
+
+ if (ptrace(PTRACE_SEIZE, pid, NULL, NULL))
+ goto cleanup;
+
+ if (ptrace(PTRACE_INTERRUPT, pid, NULL, NULL))
+ goto cleanup;
+
+ waitpid(pid, NULL, 0);
+
+ if (ptrace(PTRACE_GETSIGINFO, pid, NULL, &siginfo))
+ goto cleanup;
+
+ if (ptrace(PTRACE_DETACH, pid, NULL, NULL))
+ goto cleanup;
+
+ ret = KSFT_PASS;
+
+cleanup:
+ if (cgroup)
+ cg_destroy(cgroup);
+ free(cgroup);
+ return ret;
+}
+
+#define T(x) { x, #x }
+struct cgfreezer_test {
+ int (*fn)(const char *root);
+ const char *name;
+} tests[] = {
+ T(test_cgfreezer_simple),
+ T(test_cgfreezer_tree),
+ T(test_cgfreezer_forkbomb),
+ T(test_cgfreezer_rmdir),
+ T(test_cgfreezer_migrate),
+ T(test_cgfreezer_ptrace),
+};
+#undef T
+
+int main(int argc, char *argv[])
+{
+ char root[PATH_MAX];
+ int i, ret = EXIT_SUCCESS;
+
+ if (cg_find_unified_root(root, sizeof(root)))
+ ksft_exit_skip("cgroup v2 isn't mounted\n");
+ for (i = 0; i < ARRAY_SIZE(tests); i++) {
+ switch (tests[i].fn(root)) {
+ case KSFT_PASS:
+ ksft_test_result_pass("%s\n", tests[i].name);
+ break;
+ case KSFT_SKIP:
+ ksft_test_result_skip("%s\n", tests[i].name);
+ break;
+ default:
+ ret = EXIT_FAILURE;
+ ksft_test_result_fail("%s\n", tests[i].name);
+ break;
+ }
+ }
+
+ return ret;
+}
--
2.17.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v3 6/7] kselftests: cgroup: add freezer controller self-tests
@ 2018-11-17 0:38 ` guroan
0 siblings, 0 replies; 23+ messages in thread
From: Roman Gushchin @ 2018-11-17 0:38 UTC (permalink / raw)
This patch implements six tests for the freezer controller for
cgroup v2:
1) a simple test, which aims to freeze and unfreeze a cgroup with 100
processes
2) a more complicated tree test, which creates a hierarchy of cgroups,
puts some processes in some cgroups, and tries to freeze and unfreeze
different parts of the subtree
3) a forkbomb test: the test aims to freeze a forkbomb running in a
cgroup, kill all tasks in the cgroup and remove the cgroup without
the unfreezing.
4) rmdir test: the test creates two nested cgroups, freezes the parent
one, checks that the child can be successfully removed, and a new
child can be created
5) migration tests: the test checks migration of a task between
frozen cgroups: from a frozen to a running, from a running to a
frozen, and from a frozen to a frozen.
6) ptrace test: the test checks that it's possible to attach to
a process in a frozen cgroup, get some information and detach, and
the cgroup will remain frozen.
Expected output:
$ ./test_freezer
ok 1 test_cgfreezer_simple
ok 2 test_cgfreezer_tree
ok 3 test_cgfreezer_forkbomb
ok 4 test_cgrreezer_rmdir
ok 5 test_cgfreezer_migrate
ok 6 test_cgfreezer_ptrace
Signed-off-by: Roman Gushchin <guro at fb.com>
Cc: Shuah Khan <shuah at kernel.org>
Cc: Tejun Heo <tj at kernel.org>
Cc: kernel-team at fb.com
Cc: linux-kselftest at vger.kernel.org
---
tools/testing/selftests/cgroup/.gitignore | 1 +
tools/testing/selftests/cgroup/Makefile | 2 +
tools/testing/selftests/cgroup/cgroup_util.c | 81 ++-
tools/testing/selftests/cgroup/cgroup_util.h | 7 +
tools/testing/selftests/cgroup/test_freezer.c | 685 ++++++++++++++++++
5 files changed, 775 insertions(+), 1 deletion(-)
create mode 100644 tools/testing/selftests/cgroup/test_freezer.c
diff --git a/tools/testing/selftests/cgroup/.gitignore b/tools/testing/selftests/cgroup/.gitignore
index adacda50a4b2..7f9835624793 100644
--- a/tools/testing/selftests/cgroup/.gitignore
+++ b/tools/testing/selftests/cgroup/.gitignore
@@ -1,2 +1,3 @@
test_memcontrol
test_core
+test_freezer
diff --git a/tools/testing/selftests/cgroup/Makefile b/tools/testing/selftests/cgroup/Makefile
index 23fbaa4a9630..8d369b6a2069 100644
--- a/tools/testing/selftests/cgroup/Makefile
+++ b/tools/testing/selftests/cgroup/Makefile
@@ -5,8 +5,10 @@ all:
TEST_GEN_PROGS = test_memcontrol
TEST_GEN_PROGS += test_core
+TEST_GEN_PROGS += test_freezer
include ../lib.mk
$(OUTPUT)/test_memcontrol: cgroup_util.c
$(OUTPUT)/test_core: cgroup_util.c
+$(OUTPUT)/test_freezer: cgroup_util.c
diff --git a/tools/testing/selftests/cgroup/cgroup_util.c b/tools/testing/selftests/cgroup/cgroup_util.c
index eba06f94433b..e9cdad673901 100644
--- a/tools/testing/selftests/cgroup/cgroup_util.c
+++ b/tools/testing/selftests/cgroup/cgroup_util.c
@@ -74,6 +74,16 @@ char *cg_name_indexed(const char *root, const char *name, int index)
return ret;
}
+char *cg_control(const char *cgroup, const char *control)
+{
+ size_t len = strlen(cgroup) + strlen(control) + 2;
+ char *ret = malloc(len);
+
+ snprintf(ret, len, "%s/%s", cgroup, control);
+
+ return ret;
+}
+
int cg_read(const char *cgroup, const char *control, char *buf, size_t len)
{
char path[PATH_MAX];
@@ -196,7 +206,59 @@ int cg_create(const char *cgroup)
return mkdir(cgroup, 0644);
}
-static int cg_killall(const char *cgroup)
+int cg_for_all_procs(const char *cgroup, int (*fn)(int pid, void *arg),
+ void *arg)
+{
+ char buf[PAGE_SIZE];
+ char *ptr = buf;
+ int ret;
+
+ if (cg_read(cgroup, "cgroup.procs", buf, sizeof(buf)))
+ return -1;
+
+ while (ptr < buf + sizeof(buf)) {
+ int pid = strtol(ptr, &ptr, 10);
+
+ if (pid == 0)
+ break;
+ if (*ptr)
+ ptr++;
+ else
+ break;
+ ret = fn(pid, arg);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+int cg_wait_for_proc_count(const char *cgroup, int count)
+{
+ char buf[10 * PAGE_SIZE] = {0};
+ int attempts;
+ char *ptr;
+
+ for (attempts = 10; attempts >= 0; attempts--) {
+ int nr = 0;
+
+ if (cg_read(cgroup, "cgroup.procs", buf, sizeof(buf)))
+ break;
+
+ for (ptr = buf; *ptr; ptr++)
+ if (*ptr == '\n')
+ nr++;
+
+ if (nr >= count)
+ return 0;
+
+ usleep(100000);
+ }
+
+ return -1;
+}
+
+int cg_killall(const char *cgroup)
{
char buf[PAGE_SIZE];
char *ptr = buf;
@@ -238,6 +300,14 @@ int cg_destroy(const char *cgroup)
return ret;
}
+int cg_enter(const char *cgroup, int pid)
+{
+ char pidbuf[64];
+
+ snprintf(pidbuf, sizeof(pidbuf), "%d", pid);
+ return cg_write(cgroup, "cgroup.procs", pidbuf);
+}
+
int cg_enter_current(const char *cgroup)
{
char pidbuf[64];
@@ -367,3 +437,12 @@ int set_oom_adj_score(int pid, int score)
close(fd);
return 0;
}
+
+char proc_read_text(int pid, const char *item, char *buf, size_t size)
+{
+ char path[PATH_MAX];
+
+ snprintf(path, sizeof(path), "/proc/%d/%s", pid, item);
+
+ return read_text(path, buf, size);
+}
diff --git a/tools/testing/selftests/cgroup/cgroup_util.h b/tools/testing/selftests/cgroup/cgroup_util.h
index 9ac8b7958f83..8ee63c00a668 100644
--- a/tools/testing/selftests/cgroup/cgroup_util.h
+++ b/tools/testing/selftests/cgroup/cgroup_util.h
@@ -18,6 +18,7 @@ static inline int values_close(long a, long b, int err)
extern int cg_find_unified_root(char *root, size_t len);
extern char *cg_name(const char *root, const char *name);
extern char *cg_name_indexed(const char *root, const char *name, int index);
+extern char *cg_control(const char *cgroup, const char *control);
extern int cg_create(const char *cgroup);
extern int cg_destroy(const char *cgroup);
extern int cg_read(const char *cgroup, const char *control,
@@ -32,6 +33,7 @@ extern int cg_write(const char *cgroup, const char *control, char *buf);
extern int cg_run(const char *cgroup,
int (*fn)(const char *cgroup, void *arg),
void *arg);
+extern int cg_enter(const char *cgroup, int pid);
extern int cg_enter_current(const char *cgroup);
extern int cg_run_nowait(const char *cgroup,
int (*fn)(const char *cgroup, void *arg),
@@ -41,3 +43,8 @@ extern int alloc_pagecache(int fd, size_t size);
extern int alloc_anon(const char *cgroup, void *arg);
extern int is_swap_enabled(void);
extern int set_oom_adj_score(int pid, int score);
+extern int cg_for_all_procs(const char *cgroup, int (*fn)(int pid, void *arg),
+ void *arg);
+extern int cg_wait_for_proc_count(const char *cgroup, int count);
+extern int cg_killall(const char *cgroup);
+extern char proc_read_text(int pid, const char *item, char *buf, size_t size);
diff --git a/tools/testing/selftests/cgroup/test_freezer.c b/tools/testing/selftests/cgroup/test_freezer.c
new file mode 100644
index 000000000000..0c4106658360
--- /dev/null
+++ b/tools/testing/selftests/cgroup/test_freezer.c
@@ -0,0 +1,685 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <stdbool.h>
+#include <linux/limits.h>
+#include <sys/ptrace.h>
+#include <sys/types.h>
+#include <sys/mman.h>
+#include <unistd.h>
+#include <stdio.h>
+#include <errno.h>
+#include <poll.h>
+#include <stdlib.h>
+#include <sys/inotify.h>
+#include <string.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+
+#include "../kselftest.h"
+#include "cgroup_util.h"
+
+#define DEBUG
+#ifdef DEBUG
+#define debug(args...) fprintf(stderr, args)
+#else
+#define debug(args...)
+#endif
+
+/*
+ * Freeze the given cgroup and wait for the inotify signal.
+ * If there is no signal in 10 seconds, treat this as an error.
+ */
+static int cg_freeze_wait(const char *cgroup, bool freeze)
+{
+ int fd, wd;
+ struct pollfd fds;
+ int ret = -1;
+
+ fd = inotify_init1(IN_NONBLOCK);
+ if (fd == -1)
+ return fd;
+
+ wd = inotify_add_watch(fd, cg_control(cgroup, "cgroup.events"),
+ IN_MODIFY);
+ if (wd == -1) {
+ close(fd);
+ return wd;
+ }
+ fds.fd = fd;
+ fds.events = POLLIN;
+
+ ret = cg_write(cgroup, "cgroup.freeze", freeze ? "1" : "0");
+ if (ret) {
+ close(fd);
+ return ret;
+ }
+
+ while (true) {
+ wd = poll(&fds, 1, 10000);
+
+ if (wd == -1 && errno == EINTR)
+ continue;
+
+ if (wd == 1 && fds.revents & POLLIN)
+ ret = 0;
+
+ break;
+ }
+
+ close(fd);
+
+ return ret;
+}
+
+/*
+ * Check if the process is frozen and parked in a proper place.
+ */
+static int proc_check_frozen(int pid, void *arg)
+{
+ char buf[PAGE_SIZE];
+ int len;
+
+ len = proc_read_text(pid, "stat", buf, sizeof(buf));
+ if (len == -1) {
+ debug("Can't get %d stat\n", pid);
+ return -1;
+ }
+
+ if (strstr(buf, "(test_freezer) S ") == NULL) {
+ debug("Process %d in the unexpected state: %s\n", pid, buf);
+ return -1;
+ }
+
+ len = proc_read_text(pid, "stack", buf, sizeof(buf));
+ if (len == -1) {
+ debug("Can't get stack of the process %d\n", pid);
+ return -1;
+ }
+
+ if (strstr(buf, "[<0>] cgroup_enter_frozen") != buf) {
+ debug("Process %d has unexpected stacktrace: %s\n", pid, buf);
+ return -1;
+ }
+
+ return 0;
+}
+
+/*
+ * Check if the cgroup is frozen and all belonging processes are
+ * parked in a proper place.
+ */
+static int cg_check_frozen(const char *cgroup, bool frozen)
+{
+ if (frozen) {
+ /*
+ * Check the cgroup.events::frozen value.
+ */
+ if (cg_read_strstr(cgroup, "cgroup.events", "frozen 1") != 0) {
+ debug("Cgroup %s isn't unexpectedly frozen\n", cgroup);
+ return -1;
+ }
+
+ /*
+ * Check that all processes are parked in the proper place.
+ */
+ if (cg_for_all_procs(cgroup, proc_check_frozen, NULL)) {
+ debug("Some processes of cgroup %s are not frozen\n",
+ cgroup);
+ return -1;
+ }
+ } else {
+ /*
+ * Check the cgroup.events::frozen value.
+ */
+ if (cg_read_strstr(cgroup, "cgroup.events", "frozen 0") != 0) {
+ debug("Cgroup %s is unexpectedly frozen\n", cgroup);
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+/*
+ * A simple process running in a sleep loop until being
+ * re-parented.
+ */
+static int child_fn(const char *cgroup, void *arg)
+{
+ int ppid = getppid();
+
+ while (getppid() == ppid)
+ usleep(1000);
+
+ return getppid() == ppid;
+}
+
+/*
+ * A simple test for the cgroup freezer: populated the cgroup with 100
+ * running processes and freeze it. Then unfreeze it. Then it kills all
+ * processes and destroys the cgroup.
+ */
+static int test_cgfreezer_simple(const char *root)
+{
+ int ret = KSFT_FAIL;
+ char *cgroup = NULL;
+ int i;
+
+ cgroup = cg_name(root, "cg_test");
+ if (!cgroup)
+ goto cleanup;
+
+ if (cg_create(cgroup))
+ goto cleanup;
+
+ for (i = 0; i < 100; i++)
+ cg_run_nowait(cgroup, child_fn, NULL);
+
+ if (cg_wait_for_proc_count(cgroup, 100))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup, false))
+ goto cleanup;
+
+ if (cg_freeze_wait(cgroup, true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup, true))
+ goto cleanup;
+
+ if (cg_freeze_wait(cgroup, false))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup, false))
+ goto cleanup;
+
+ ret = KSFT_PASS;
+
+cleanup:
+ if (cgroup)
+ cg_destroy(cgroup);
+ free(cgroup);
+ return ret;
+}
+
+/*
+ * The test creates the following hierarchy:
+ * A
+ * / / \ \
+ * B E I K
+ * /\ |
+ * C D F
+ * |
+ * G
+ * |
+ * H
+ *
+ * with a process in C, H and 3 processes in K.
+ * Then it tries to freeze and unfreeze the whole tree.
+ */
+static int test_cgfreezer_tree(const char *root)
+{
+ char *cgroup[10] = {0};
+ int ret = KSFT_FAIL;
+ int i;
+
+ cgroup[0] = cg_name(root, "cg_test_A");
+ if (!cgroup[0])
+ goto cleanup;
+
+ cgroup[1] = cg_name(cgroup[0], "cg_test_B");
+ if (!cgroup[1])
+ goto cleanup;
+
+ cgroup[2] = cg_name(cgroup[1], "cg_test_C");
+ if (!cgroup[2])
+ goto cleanup;
+
+ cgroup[3] = cg_name(cgroup[1], "cg_test_D");
+ if (!cgroup[3])
+ goto cleanup;
+
+ cgroup[4] = cg_name(cgroup[0], "cg_test_E");
+ if (!cgroup[4])
+ goto cleanup;
+
+ cgroup[5] = cg_name(cgroup[4], "cg_test_F");
+ if (!cgroup[5])
+ goto cleanup;
+
+ cgroup[6] = cg_name(cgroup[5], "cg_test_G");
+ if (!cgroup[6])
+ goto cleanup;
+
+ cgroup[7] = cg_name(cgroup[6], "cg_test_H");
+ if (!cgroup[7])
+ goto cleanup;
+
+ cgroup[8] = cg_name(cgroup[0], "cg_test_I");
+ if (!cgroup[8])
+ goto cleanup;
+
+ cgroup[9] = cg_name(cgroup[0], "cg_test_K");
+ if (!cgroup[9])
+ goto cleanup;
+
+ for (i = 0; i < 10; i++)
+ if (cg_create(cgroup[i]))
+ goto cleanup;
+
+ cg_run_nowait(cgroup[2], child_fn, NULL);
+ cg_run_nowait(cgroup[7], child_fn, NULL);
+ cg_run_nowait(cgroup[9], child_fn, NULL);
+ cg_run_nowait(cgroup[9], child_fn, NULL);
+ cg_run_nowait(cgroup[9], child_fn, NULL);
+
+ /*
+ * Wait until all child processes will enter
+ * corresponding cgroups.
+ */
+
+ if (cg_wait_for_proc_count(cgroup[2], 1) ||
+ cg_wait_for_proc_count(cgroup[7], 1) ||
+ cg_wait_for_proc_count(cgroup[9], 3))
+ goto cleanup;
+
+ /*
+ * Freeze B.
+ */
+ if (cg_freeze_wait(cgroup[1], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[1], true))
+ goto cleanup;
+
+ /*
+ * Freeze F.
+ */
+ if (cg_freeze_wait(cgroup[5], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[5], true))
+ goto cleanup;
+
+ /*
+ * Freeze G.
+ */
+ if (cg_freeze_wait(cgroup[6], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[6], true))
+ goto cleanup;
+
+ /*
+ * Check that A and E are not frozen.
+ */
+ if (cg_check_frozen(cgroup[0], false))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[4], false))
+ goto cleanup;
+
+ /*
+ * Freeze A. Check that A, B and E are frozen.
+ */
+ if (cg_freeze_wait(cgroup[0], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[0], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[1], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[4], true))
+ goto cleanup;
+
+ /*
+ * Unfreeze B, F and G
+ */
+ if (cg_freeze_wait(cgroup[1], false))
+ goto cleanup;
+
+ if (cg_freeze_wait(cgroup[5], false))
+ goto cleanup;
+
+ if (cg_freeze_wait(cgroup[6], false))
+ goto cleanup;
+
+ /*
+ * Check that C and H are still frozen.
+ */
+ if (cg_check_frozen(cgroup[2], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[7], true))
+ goto cleanup;
+
+ /*
+ * Unfreezing A failed. Check that A, C and K are not frozen.
+ */
+ if (cg_freeze_wait(cgroup[0], false))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[0], false))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[2], false))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[9], false))
+ goto cleanup;
+
+ ret = KSFT_PASS;
+
+cleanup:
+ for (i = 9; i >= 0 && cgroup[i]; i--) {
+ cg_destroy(cgroup[i]);
+ free(cgroup[i]);
+ }
+
+ return ret;
+}
+
+/*
+ * A fork bomb emulator.
+ */
+static int forkbomb_fn(const char *cgroup, void *arg)
+{
+ int ppid;
+
+ fork();
+ fork();
+
+ ppid = getppid();
+
+ while (getppid() == ppid)
+ usleep(1000);
+
+ return getppid() == ppid;
+}
+
+/*
+ * The test runs a fork bomb in a cgroup and tries to freeze it.
+ * Then it kills all processes and checks that cgroup isn't populated
+ * anymore.
+ */
+static int test_cgfreezer_forkbomb(const char *root)
+{
+ int ret = KSFT_FAIL;
+ char *cgroup = NULL;
+
+ cgroup = cg_name(root, "cg_forkbomb_test");
+ if (!cgroup)
+ goto cleanup;
+
+ if (cg_create(cgroup))
+ goto cleanup;
+
+ cg_run_nowait(cgroup, forkbomb_fn, NULL);
+
+ usleep(100000);
+
+ if (cg_freeze_wait(cgroup, true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup, true))
+ goto cleanup;
+
+ if (cg_killall(cgroup))
+ goto cleanup;
+
+ if (cg_wait_for_proc_count(cgroup, 0))
+ goto cleanup;
+
+ ret = KSFT_PASS;
+
+cleanup:
+ if (cgroup)
+ cg_destroy(cgroup);
+ free(cgroup);
+ return ret;
+}
+
+/*
+ * The test creates two nested cgroups, freezes the parent
+ * and removes the child. Then it checks that the parent cgroup
+ * remains frozen and it's possible to create a new child
+ * without unfreezing. The new child is frozen too.
+ */
+static int test_cgfreezer_rmdir(const char *root)
+{
+ int ret = KSFT_FAIL;
+ char *parent, *child = NULL;
+
+ parent = cg_name(root, "cg_test_A");
+ if (!parent)
+ goto cleanup;
+
+ child = cg_name(parent, "cg_test_B");
+ if (!child)
+ goto cleanup;
+
+ if (cg_create(parent))
+ goto cleanup;
+
+ if (cg_create(child))
+ goto cleanup;
+
+ if (cg_freeze_wait(parent, true))
+ goto cleanup;
+
+ if (cg_check_frozen(parent, true))
+ goto cleanup;
+
+ if (cg_destroy(child))
+ goto cleanup;
+
+ if (cg_check_frozen(parent, true))
+ goto cleanup;
+
+ if (cg_create(child))
+ goto cleanup;
+
+ if (cg_check_frozen(child, true))
+ goto cleanup;
+
+ ret = KSFT_PASS;
+
+cleanup:
+ if (child)
+ cg_destroy(child);
+ free(child);
+ if (parent)
+ cg_destroy(parent);
+ free(parent);
+ return ret;
+}
+
+/*
+ * The test creates two cgroups: A and B. The it runs a process in A,
+ * and performs several migrations:
+ * 1) A (running) -> B (frozen)
+ * 2) B (frozen) -> A (running)
+ * 3) A (frozen) -> B (frozen)
+ *
+ * One each step it checks that the actual state of cgroups matches
+ * the expected state.
+ */
+static int test_cgfreezer_migrate(const char *root)
+{
+ int ret = KSFT_FAIL;
+ char *cgroup[2] = {0};
+ int pid;
+
+ cgroup[0] = cg_name(root, "cg_test_A");
+ if (!cgroup[0])
+ goto cleanup;
+
+ cgroup[1] = cg_name(root, "cg_test_B");
+ if (!cgroup[1])
+ goto cleanup;
+
+ if (cg_create(cgroup[0]))
+ goto cleanup;
+
+ if (cg_create(cgroup[1]))
+ goto cleanup;
+
+ pid = cg_run_nowait(cgroup[0], child_fn, NULL);
+ if (pid < 0)
+ goto cleanup;
+
+ if (cg_wait_for_proc_count(cgroup[0], 1))
+ goto cleanup;
+
+ /*
+ * Migrate from A (running) to B (frozen)
+ */
+ if (cg_freeze_wait(cgroup[1], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[1], true))
+ goto cleanup;
+
+ if (cg_enter(cgroup[1], pid))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[0], false))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[1], true))
+ goto cleanup;
+
+ /*
+ * Migrate from B (frozen) to A (running)
+ */
+ if (cg_enter(cgroup[0], pid))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[0], false))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[1], true))
+ goto cleanup;
+
+ /*
+ * Migrate from A (frozen) to B (frozen)
+ */
+ if (cg_freeze_wait(cgroup[0], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[0], true))
+ goto cleanup;
+
+ if (cg_enter(cgroup[1], pid))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[0], true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup[1], true))
+ goto cleanup;
+
+ ret = KSFT_PASS;
+
+cleanup:
+ if (cgroup[0])
+ cg_destroy(cgroup[0]);
+ free(cgroup[0]);
+ if (cgroup[1])
+ cg_destroy(cgroup[1]);
+ free(cgroup[1]);
+ return ret;
+}
+
+/*
+ * The test checks that ptrace works with a tracing process in a frozen cgroup.
+ */
+static int test_cgfreezer_ptrace(const char *root)
+{
+ int ret = KSFT_FAIL;
+ char *cgroup = NULL;
+ siginfo_t siginfo;
+ int pid;
+
+ cgroup = cg_name(root, "cg_test");
+ if (!cgroup)
+ goto cleanup;
+
+ if (cg_create(cgroup))
+ goto cleanup;
+
+ pid = cg_run_nowait(cgroup, child_fn, NULL);
+ if (pid < 0)
+ goto cleanup;
+
+ if (cg_wait_for_proc_count(cgroup, 1))
+ goto cleanup;
+
+ if (cg_freeze_wait(cgroup, true))
+ goto cleanup;
+
+ if (cg_check_frozen(cgroup, true))
+ goto cleanup;
+
+ if (ptrace(PTRACE_SEIZE, pid, NULL, NULL))
+ goto cleanup;
+
+ if (ptrace(PTRACE_INTERRUPT, pid, NULL, NULL))
+ goto cleanup;
+
+ waitpid(pid, NULL, 0);
+
+ if (ptrace(PTRACE_GETSIGINFO, pid, NULL, &siginfo))
+ goto cleanup;
+
+ if (ptrace(PTRACE_DETACH, pid, NULL, NULL))
+ goto cleanup;
+
+ ret = KSFT_PASS;
+
+cleanup:
+ if (cgroup)
+ cg_destroy(cgroup);
+ free(cgroup);
+ return ret;
+}
+
+#define T(x) { x, #x }
+struct cgfreezer_test {
+ int (*fn)(const char *root);
+ const char *name;
+} tests[] = {
+ T(test_cgfreezer_simple),
+ T(test_cgfreezer_tree),
+ T(test_cgfreezer_forkbomb),
+ T(test_cgfreezer_rmdir),
+ T(test_cgfreezer_migrate),
+ T(test_cgfreezer_ptrace),
+};
+#undef T
+
+int main(int argc, char *argv[])
+{
+ char root[PATH_MAX];
+ int i, ret = EXIT_SUCCESS;
+
+ if (cg_find_unified_root(root, sizeof(root)))
+ ksft_exit_skip("cgroup v2 isn't mounted\n");
+ for (i = 0; i < ARRAY_SIZE(tests); i++) {
+ switch (tests[i].fn(root)) {
+ case KSFT_PASS:
+ ksft_test_result_pass("%s\n", tests[i].name);
+ break;
+ case KSFT_SKIP:
+ ksft_test_result_skip("%s\n", tests[i].name);
+ break;
+ default:
+ ret = EXIT_FAILURE;
+ ksft_test_result_fail("%s\n", tests[i].name);
+ break;
+ }
+ }
+
+ return ret;
+}
--
2.17.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v3 7/7] cgroup: document cgroup v2 freezer interface
2018-11-17 0:38 [PATCH v3 0/7] freezer for cgroup v2 Roman Gushchin
` (5 preceding siblings ...)
2018-11-17 0:38 ` guroan
@ 2018-11-17 0:38 ` Roman Gushchin
2018-11-17 8:02 ` Mike Rapoport
6 siblings, 1 reply; 23+ messages in thread
From: Roman Gushchin @ 2018-11-17 0:38 UTC (permalink / raw)
To: Tejun Heo
Cc: Oleg Nesterov, cgroups, linux-kernel, kernel-team,
Roman Gushchin, linux-doc
Describe cgroup v2 freezer interface in the cgroup v2 admin guide.
Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: linux-doc@vger.kernel.org
Cc: kernel-team@fb.com
---
Documentation/admin-guide/cgroup-v2.rst | 26 +++++++++++++++++++++++++
1 file changed, 26 insertions(+)
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
index 184193bcb262..a065c0bed88c 100644
--- a/Documentation/admin-guide/cgroup-v2.rst
+++ b/Documentation/admin-guide/cgroup-v2.rst
@@ -862,6 +862,8 @@ All cgroup core files are prefixed with "cgroup."
populated
1 if the cgroup or its descendants contains any live
processes; otherwise, 0.
+ frozen
+ 1 if the cgroup is frozen; otherwise, 0.
cgroup.max.descendants
A read-write single value files. The default is "max".
@@ -895,6 +897,30 @@ All cgroup core files are prefixed with "cgroup."
A dying cgroup can consume system resources not exceeding
limits, which were active at the moment of cgroup deletion.
+ cgroup.freeze
+ A read-write single value file which exists on non-root cgroups.
+ Allowed values are "0" and "1". The default is "0".
+
+ Writing "1" to the file causes freezing of the cgroup and all
+ descendant cgroups. This means that all belonging processes will
+ be stopped and will not run until the cgroup will be explicitly
+ unfrozen. Freezing of the cgroup may take some time; when the process
+ is complete, the "frozen" value in the cgroup.events control file
+ will be updated and the corresponding notification will be issued.
+
+ Cgroup can be frozen either by its own settings, either by settings
+ of any ancestor cgroups. If any of ancestor cgroups is frozen, the
+ cgroup will remain frozen.
+
+ Processes in the frozen cgroup can be killed by a fatal signal.
+ They also can enter and leave a frozen cgroup: either by an explicit
+ move by a user, either if freezing of the cgroup races with fork().
+ If a cgroup is moved to a frozen cgroup, it stops. If a process is
+ moving out of a frozen cgroup, it becomes running.
+
+ Frozen status of a cgroup doesn't affect any cgroup tree operations:
+ it's possible to delete a frozen (and empty) cgroup, as well as
+ create new sub-cgroups.
Controllers
===========
--
2.17.2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: [PATCH v3 7/7] cgroup: document cgroup v2 freezer interface
2018-11-17 0:38 ` [PATCH v3 7/7] cgroup: document cgroup v2 freezer interface Roman Gushchin
@ 2018-11-17 8:02 ` Mike Rapoport
2018-11-19 17:42 ` Roman Gushchin
0 siblings, 1 reply; 23+ messages in thread
From: Mike Rapoport @ 2018-11-17 8:02 UTC (permalink / raw)
To: Roman Gushchin
Cc: Tejun Heo, Oleg Nesterov, cgroups, linux-kernel, kernel-team,
Roman Gushchin, linux-doc
Hi,
On Fri, Nov 16, 2018 at 04:38:30PM -0800, Roman Gushchin wrote:
> Describe cgroup v2 freezer interface in the cgroup v2 admin guide.
>
> Signed-off-by: Roman Gushchin <guro@fb.com>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: linux-doc@vger.kernel.org
> Cc: kernel-team@fb.com
> ---
> Documentation/admin-guide/cgroup-v2.rst | 26 +++++++++++++++++++++++++
> 1 file changed, 26 insertions(+)
>
> diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
> index 184193bcb262..a065c0bed88c 100644
> --- a/Documentation/admin-guide/cgroup-v2.rst
> +++ b/Documentation/admin-guide/cgroup-v2.rst
> @@ -862,6 +862,8 @@ All cgroup core files are prefixed with "cgroup."
> populated
> 1 if the cgroup or its descendants contains any live
> processes; otherwise, 0.
> + frozen
> + 1 if the cgroup is frozen; otherwise, 0.
>
> cgroup.max.descendants
> A read-write single value files. The default is "max".
> @@ -895,6 +897,30 @@ All cgroup core files are prefixed with "cgroup."
> A dying cgroup can consume system resources not exceeding
> limits, which were active at the moment of cgroup deletion.
>
> + cgroup.freeze
> + A read-write single value file which exists on non-root cgroups.
> + Allowed values are "0" and "1". The default is "0".
> +
> + Writing "1" to the file causes freezing of the cgroup and all
> + descendant cgroups. This means that all belonging processes will
> + be stopped and will not run until the cgroup will be explicitly
> + unfrozen. Freezing of the cgroup may take some time; when the process
"when the process is complete" sounds somewhat ambiguous, it's unclear
whether freezing is complete or the process that's being frozen is
complete.
Maybe "when this action is completed"?
> + is complete, the "frozen" value in the cgroup.events control file
> + will be updated and the corresponding notification will be issued.
Can you please clarify how exactly cgroup.events would be updated?
> + Cgroup can be frozen either by its own settings, either by settings
^ A cgroup ... and maybe there are more "a" and "the" that should be
fixed, it's hard for me to tell.
Also, I believe "either ..., or ..." sounds better than "either ...,
either ..."
> + of any ancestor cgroups. If any of ancestor cgroups is frozen, the
> + cgroup will remain frozen.
> +
> + Processes in the frozen cgroup can be killed by a fatal signal.
> + They also can enter and leave a frozen cgroup: either by an explicit
> + move by a user, either if freezing of the cgroup races with fork().
ditto
> + If a cgroup is moved to a frozen cgroup, it stops. If a process is
^ process?
> + moving out of a frozen cgroup, it becomes running.
^ moved
> + Frozen status of a cgroup doesn't affect any cgroup tree operations:
> + it's possible to delete a frozen (and empty) cgroup, as well as
> + create new sub-cgroups.
Maybe it's also worth adding that freezing a process has no effect on its
memory consumption, at least directly.
> Controllers
> ===========
> --
> 2.17.2
>
--
Sincerely yours,
Mike.
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v3 7/7] cgroup: document cgroup v2 freezer interface
2018-11-17 8:02 ` Mike Rapoport
@ 2018-11-19 17:42 ` Roman Gushchin
2018-11-20 14:06 ` Mike Rapoport
0 siblings, 1 reply; 23+ messages in thread
From: Roman Gushchin @ 2018-11-19 17:42 UTC (permalink / raw)
To: Mike Rapoport
Cc: Roman Gushchin, Tejun Heo, Oleg Nesterov, cgroups, linux-kernel,
Kernel Team, linux-doc
On Sat, Nov 17, 2018 at 12:02:28AM -0800, Mike Rapoport wrote:
> Hi,
>
> On Fri, Nov 16, 2018 at 04:38:30PM -0800, Roman Gushchin wrote:
> > Describe cgroup v2 freezer interface in the cgroup v2 admin guide.
> >
> > Signed-off-by: Roman Gushchin <guro@fb.com>
> > Cc: Tejun Heo <tj@kernel.org>
> > Cc: linux-doc@vger.kernel.org
> > Cc: kernel-team@fb.com
> > ---
> > Documentation/admin-guide/cgroup-v2.rst | 26 +++++++++++++++++++++++++
> > 1 file changed, 26 insertions(+)
> >
> > diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
> > index 184193bcb262..a065c0bed88c 100644
> > --- a/Documentation/admin-guide/cgroup-v2.rst
> > +++ b/Documentation/admin-guide/cgroup-v2.rst
> > @@ -862,6 +862,8 @@ All cgroup core files are prefixed with "cgroup."
> > populated
> > 1 if the cgroup or its descendants contains any live
> > processes; otherwise, 0.
> > + frozen
> > + 1 if the cgroup is frozen; otherwise, 0.
> >
> > cgroup.max.descendants
> > A read-write single value files. The default is "max".
> > @@ -895,6 +897,30 @@ All cgroup core files are prefixed with "cgroup."
> > A dying cgroup can consume system resources not exceeding
> > limits, which were active at the moment of cgroup deletion.
> >
> > + cgroup.freeze
> > + A read-write single value file which exists on non-root cgroups.
> > + Allowed values are "0" and "1". The default is "0".
> > +
> > + Writing "1" to the file causes freezing of the cgroup and all
> > + descendant cgroups. This means that all belonging processes will
> > + be stopped and will not run until the cgroup will be explicitly
> > + unfrozen. Freezing of the cgroup may take some time; when the process
>
> "when the process is complete" sounds somewhat ambiguous, it's unclear
> whether freezing is complete or the process that's being frozen is
> complete.
>
> Maybe "when this action is completed"?
>
> > + is complete, the "frozen" value in the cgroup.events control file
> > + will be updated and the corresponding notification will be issued.
>
> Can you please clarify how exactly cgroup.events would be updated?
>
> > + Cgroup can be frozen either by its own settings, either by settings
>
> ^ A cgroup ... and maybe there are more "a" and "the" that should be
> fixed, it's hard for me to tell.
>
> Also, I believe "either ..., or ..." sounds better than "either ...,
> either ..."
>
> > + of any ancestor cgroups. If any of ancestor cgroups is frozen, the
> > + cgroup will remain frozen.
> > +
> > + Processes in the frozen cgroup can be killed by a fatal signal.
> > + They also can enter and leave a frozen cgroup: either by an explicit
> > + move by a user, either if freezing of the cgroup races with fork().
>
> ditto
>
> > + If a cgroup is moved to a frozen cgroup, it stops. If a process is
>
> ^ process?
>
> > + moving out of a frozen cgroup, it becomes running.
>
> ^ moved
Hello, Mike!
Thanks for the review! I agree with all comments above; fixes queued for v4.
>
> > + Frozen status of a cgroup doesn't affect any cgroup tree operations:
> > + it's possible to delete a frozen (and empty) cgroup, as well as
> > + create new sub-cgroups.
>
> Maybe it's also worth adding that freezing a process has no effect on its
> memory consumption, at least directly.
Hm, isn't it the expected behavior?
In any case, I assume that cgroup.freeze knob description is not the best place
for a such explanations. Maybe it's better to add a standalone paragraph with
the description of the frozen process state, what's expected to work, what's
not, etc. I'd return to this question a bit later, when we'll agree on the user
interface and the implementation.
Thanks!
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v3 7/7] cgroup: document cgroup v2 freezer interface
2018-11-19 17:42 ` Roman Gushchin
@ 2018-11-20 14:06 ` Mike Rapoport
0 siblings, 0 replies; 23+ messages in thread
From: Mike Rapoport @ 2018-11-20 14:06 UTC (permalink / raw)
To: Roman Gushchin
Cc: Roman Gushchin, Tejun Heo, Oleg Nesterov, cgroups, linux-kernel,
Kernel Team, linux-doc
On Mon, Nov 19, 2018 at 05:42:48PM +0000, Roman Gushchin wrote:
> On Sat, Nov 17, 2018 at 12:02:28AM -0800, Mike Rapoport wrote:
> > Hi,
> >
> > On Fri, Nov 16, 2018 at 04:38:30PM -0800, Roman Gushchin wrote:
> > > Describe cgroup v2 freezer interface in the cgroup v2 admin guide.
> > >
> > > Signed-off-by: Roman Gushchin <guro@fb.com>
> > > Cc: Tejun Heo <tj@kernel.org>
> > > Cc: linux-doc@vger.kernel.org
> > > Cc: kernel-team@fb.com
> > > ---
> > > Documentation/admin-guide/cgroup-v2.rst | 26 +++++++++++++++++++++++++
> > > 1 file changed, 26 insertions(+)
> > >
> > > diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
> > > index 184193bcb262..a065c0bed88c 100644
> > > --- a/Documentation/admin-guide/cgroup-v2.rst
> > > +++ b/Documentation/admin-guide/cgroup-v2.rst
> > > @@ -862,6 +862,8 @@ All cgroup core files are prefixed with "cgroup."
> > > populated
> > > 1 if the cgroup or its descendants contains any live
> > > processes; otherwise, 0.
> > > + frozen
> > > + 1 if the cgroup is frozen; otherwise, 0.
> > >
> > > cgroup.max.descendants
> > > A read-write single value files. The default is "max".
> > > @@ -895,6 +897,30 @@ All cgroup core files are prefixed with "cgroup."
> > > A dying cgroup can consume system resources not exceeding
> > > limits, which were active at the moment of cgroup deletion.
> > >
> > > + cgroup.freeze
> > > + A read-write single value file which exists on non-root cgroups.
> > > + Allowed values are "0" and "1". The default is "0".
> > > +
> > > + Writing "1" to the file causes freezing of the cgroup and all
> > > + descendant cgroups. This means that all belonging processes will
> > > + be stopped and will not run until the cgroup will be explicitly
> > > + unfrozen. Freezing of the cgroup may take some time; when the process
> >
> > "when the process is complete" sounds somewhat ambiguous, it's unclear
> > whether freezing is complete or the process that's being frozen is
> > complete.
> >
> > Maybe "when this action is completed"?
> >
> > > + is complete, the "frozen" value in the cgroup.events control file
> > > + will be updated and the corresponding notification will be issued.
> >
> > Can you please clarify how exactly cgroup.events would be updated?
> >
> > > + Cgroup can be frozen either by its own settings, either by settings
> >
> > ^ A cgroup ... and maybe there are more "a" and "the" that should be
> > fixed, it's hard for me to tell.
> >
> > Also, I believe "either ..., or ..." sounds better than "either ...,
> > either ..."
> >
> > > + of any ancestor cgroups. If any of ancestor cgroups is frozen, the
> > > + cgroup will remain frozen.
> > > +
> > > + Processes in the frozen cgroup can be killed by a fatal signal.
> > > + They also can enter and leave a frozen cgroup: either by an explicit
> > > + move by a user, either if freezing of the cgroup races with fork().
> >
> > ditto
> >
> > > + If a cgroup is moved to a frozen cgroup, it stops. If a process is
> >
> > ^ process?
> >
> > > + moving out of a frozen cgroup, it becomes running.
> >
> > ^ moved
>
> Hello, Mike!
>
> Thanks for the review! I agree with all comments above; fixes queued for v4.
>
> >
> > > + Frozen status of a cgroup doesn't affect any cgroup tree operations:
> > > + it's possible to delete a frozen (and empty) cgroup, as well as
> > > + create new sub-cgroups.
> >
> > Maybe it's also worth adding that freezing a process has no effect on its
> > memory consumption, at least directly.
>
> Hm, isn't it the expected behavior?
You'd be surprised ;-)
Just recently I had a couple of questions about the memory consumption of
the frozen processes.
> In any case, I assume that cgroup.freeze knob description is not the best place
> for a such explanations. Maybe it's better to add a standalone paragraph with
> the description of the frozen process state, what's expected to work, what's
> not, etc. I'd return to this question a bit later, when we'll agree on the user
> interface and the implementation.
Sure.
> Thanks!
>
--
Sincerely yours,
Mike.
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v3 3/7] cgroup: protect cgroup->nr_(dying_)descendants by css_set_lock
2018-11-17 0:38 ` [PATCH v3 3/7] cgroup: protect cgroup->nr_(dying_)descendants by css_set_lock Roman Gushchin
@ 2018-11-20 16:18 ` Tejun Heo
0 siblings, 0 replies; 23+ messages in thread
From: Tejun Heo @ 2018-11-20 16:18 UTC (permalink / raw)
To: Roman Gushchin
Cc: Oleg Nesterov, cgroups, linux-kernel, kernel-team, Roman Gushchin
On Fri, Nov 16, 2018 at 04:38:26PM -0800, Roman Gushchin wrote:
> Now the number of descendant cgroups and the number of dying
> descendant cgroups are synchronized using the cgroup_mutex.
>
> The number of descendant cgroups will be required by the cgroup v2
> freezer, which will use it to determine if a cgroup is frozen
> (depending on total number of descendants and number of frozen
> descendants). It's not always acceptable to grab the cgroup_mutex,
> especially from quite hot paths (e.g. exit()).
>
> To avoid this, let's additionally synchronize these counters
> using the css_set_lock.
Can you change it so that writes are cgroup_mutex and css_set_lock
protected and reads can be done under either cgroup_mutex or
css_set_lock?
Thanks.
--
tejun
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v3 4/7] cgroup: cgroup v2 freezer
2018-11-17 0:38 ` [PATCH v3 4/7] cgroup: cgroup v2 freezer Roman Gushchin
@ 2018-11-20 16:25 ` Tejun Heo
2018-11-20 16:33 ` Roman Gushchin
0 siblings, 1 reply; 23+ messages in thread
From: Tejun Heo @ 2018-11-20 16:25 UTC (permalink / raw)
To: Roman Gushchin
Cc: Oleg Nesterov, cgroups, linux-kernel, kernel-team, Roman Gushchin
On Fri, Nov 16, 2018 at 04:38:27PM -0800, Roman Gushchin wrote:
> +void cgroup_freezer_migrate_task(struct task_struct *task,
> + struct cgroup *src, struct cgroup *dst)
> +{
> + lockdep_assert_held(&css_set_lock);
> +
> + /*
> + * Kernel threads are not supposed to be frozen at all.
> + */
> + if (task->flags & PF_KTHREAD)
> + return;
> +
> + /*
> + * Adjust counters of freezing and frozen tasks.
> + */
> + if (test_bit(CGRP_FREEZE, &src->flags)) {
> + src->freezer.nr_tasks_to_freeze--;
> + WARN_ON_ONCE(src->freezer.nr_tasks_to_freeze < 0);
> + }
> +
> + /*
> + * If the task is frozen, let's bump nr_tasks_to_freeze even
> + * if the target cgroup isn't frozen: the counter will be decreased
> + * in cgroup_leave_frozen().
> + */
> + if (test_bit(CGRP_FREEZE, &dst->flags) || task->frozen)
> + dst->freezer.nr_tasks_to_freeze++;
> +
> + if (task->frozen) {
> + src->freezer.nr_frozen_tasks--;
> + dst->freezer.nr_frozen_tasks++;
> + WARN_ON_ONCE(src->freezer.nr_frozen_tasks < 0);
> + WARN_ON_ONCE(dst->freezer.nr_frozen_tasks >
> + dst->freezer.nr_tasks_to_freeze);
> + }
If a non-frozen task is being moved into a frozen cgroup, shouldn't
that also trigger frozen state update?
Thanks.
--
tejun
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v3 4/7] cgroup: cgroup v2 freezer
2018-11-20 16:25 ` Tejun Heo
@ 2018-11-20 16:33 ` Roman Gushchin
2018-11-20 16:36 ` Tejun Heo
0 siblings, 1 reply; 23+ messages in thread
From: Roman Gushchin @ 2018-11-20 16:33 UTC (permalink / raw)
To: Tejun Heo
Cc: Roman Gushchin, Oleg Nesterov, cgroups, linux-kernel, Kernel Team
On Tue, Nov 20, 2018 at 08:25:29AM -0800, Tejun Heo wrote:
> On Fri, Nov 16, 2018 at 04:38:27PM -0800, Roman Gushchin wrote:
> > +void cgroup_freezer_migrate_task(struct task_struct *task,
> > + struct cgroup *src, struct cgroup *dst)
> > +{
> > + lockdep_assert_held(&css_set_lock);
> > +
> > + /*
> > + * Kernel threads are not supposed to be frozen at all.
> > + */
> > + if (task->flags & PF_KTHREAD)
> > + return;
> > +
> > + /*
> > + * Adjust counters of freezing and frozen tasks.
> > + */
> > + if (test_bit(CGRP_FREEZE, &src->flags)) {
> > + src->freezer.nr_tasks_to_freeze--;
> > + WARN_ON_ONCE(src->freezer.nr_tasks_to_freeze < 0);
> > + }
> > +
> > + /*
> > + * If the task is frozen, let's bump nr_tasks_to_freeze even
> > + * if the target cgroup isn't frozen: the counter will be decreased
> > + * in cgroup_leave_frozen().
> > + */
> > + if (test_bit(CGRP_FREEZE, &dst->flags) || task->frozen)
> > + dst->freezer.nr_tasks_to_freeze++;
> > +
> > + if (task->frozen) {
> > + src->freezer.nr_frozen_tasks--;
> > + dst->freezer.nr_frozen_tasks++;
> > + WARN_ON_ONCE(src->freezer.nr_frozen_tasks < 0);
> > + WARN_ON_ONCE(dst->freezer.nr_frozen_tasks >
> > + dst->freezer.nr_tasks_to_freeze);
> > + }
>
> If a non-frozen task is being moved into a frozen cgroup, shouldn't
> that also trigger frozen state update?
It does! Just below these lines:
/*
* If the task isn't in the desired state, force it to it.
*/
if (task->frozen != test_bit(CGRP_FREEZE, &dst->flags))
cgroup_freeze_task(task, test_bit(CGRP_FREEZE, &dst->flags));
It's also covered by the 5th kselftest.
Thanks!
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v3 4/7] cgroup: cgroup v2 freezer
2018-11-20 16:33 ` Roman Gushchin
@ 2018-11-20 16:36 ` Tejun Heo
2018-11-20 16:43 ` Roman Gushchin
0 siblings, 1 reply; 23+ messages in thread
From: Tejun Heo @ 2018-11-20 16:36 UTC (permalink / raw)
To: Roman Gushchin
Cc: Roman Gushchin, Oleg Nesterov, cgroups, linux-kernel, Kernel Team
On Tue, Nov 20, 2018 at 04:33:11PM +0000, Roman Gushchin wrote:
> > If a non-frozen task is being moved into a frozen cgroup, shouldn't
> > that also trigger frozen state update?
>
> It does! Just below these lines:
>
> /*
> * If the task isn't in the desired state, force it to it.
> */
> if (task->frozen != test_bit(CGRP_FREEZE, &dst->flags))
> cgroup_freeze_task(task, test_bit(CGRP_FREEZE, &dst->flags));
But that wouldn't udpate the cgroup's frozen state and generate
notifications, right?
Thanks.
--
tejun
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v3 4/7] cgroup: cgroup v2 freezer
2018-11-20 16:36 ` Tejun Heo
@ 2018-11-20 16:43 ` Roman Gushchin
2018-11-20 16:48 ` Tejun Heo
0 siblings, 1 reply; 23+ messages in thread
From: Roman Gushchin @ 2018-11-20 16:43 UTC (permalink / raw)
To: Tejun Heo
Cc: Roman Gushchin, Oleg Nesterov, cgroups, linux-kernel, Kernel Team
On Tue, Nov 20, 2018 at 08:36:04AM -0800, Tejun Heo wrote:
> On Tue, Nov 20, 2018 at 04:33:11PM +0000, Roman Gushchin wrote:
> > > If a non-frozen task is being moved into a frozen cgroup, shouldn't
> > > that also trigger frozen state update?
> >
> > It does! Just below these lines:
> >
> > /*
> > * If the task isn't in the desired state, force it to it.
> > */
> > if (task->frozen != test_bit(CGRP_FREEZE, &dst->flags))
> > cgroup_freeze_task(task, test_bit(CGRP_FREEZE, &dst->flags));
>
> But that wouldn't udpate the cgroup's frozen state and generate
> notifications, right?
Why? The task will be eventually trapped into cgroup_enter_frozen(),
and from there cgroup_update_frozen() will be called.
You are right, that notification will not be issued, because the cgroup
is not changing its state (frozen->frozen). I'm not sure that it makes
sense to change the cgroup state back and forth in this case. Are there
any reasons I'm missing?
Thanks!
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v3 4/7] cgroup: cgroup v2 freezer
2018-11-20 16:43 ` Roman Gushchin
@ 2018-11-20 16:48 ` Tejun Heo
2018-11-20 17:39 ` Roman Gushchin
0 siblings, 1 reply; 23+ messages in thread
From: Tejun Heo @ 2018-11-20 16:48 UTC (permalink / raw)
To: Roman Gushchin
Cc: Roman Gushchin, Oleg Nesterov, cgroups, linux-kernel, Kernel Team
Hello,
On Tue, Nov 20, 2018 at 04:43:52PM +0000, Roman Gushchin wrote:
> > But that wouldn't udpate the cgroup's frozen state and generate
> > notifications, right?
>
> Why? The task will be eventually trapped into cgroup_enter_frozen(),
> and from there cgroup_update_frozen() will be called.
Because the cgroup is no longer frozen?
> You are right, that notification will not be issued, because the cgroup
> is not changing its state (frozen->frozen). I'm not sure that it makes
> sense to change the cgroup state back and forth in this case. Are there
> any reasons I'm missing?
Imagine the task being trapped in nfs or wherever and not getting into
the freezer for an extended period of time. That'd make the frozen
state reporting observably and obviously wrong when seen from userland
which can lead to other issues.
But, above all, because the cgroup is not frozen - it may have active
running tasks in it at that point.
Thanks.
--
tejun
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v3 4/7] cgroup: cgroup v2 freezer
2018-11-20 16:48 ` Tejun Heo
@ 2018-11-20 17:39 ` Roman Gushchin
2018-11-20 18:05 ` Tejun Heo
0 siblings, 1 reply; 23+ messages in thread
From: Roman Gushchin @ 2018-11-20 17:39 UTC (permalink / raw)
To: Tejun Heo
Cc: Roman Gushchin, Oleg Nesterov, cgroups, linux-kernel, Kernel Team
On Tue, Nov 20, 2018 at 08:48:59AM -0800, Tejun Heo wrote:
> Hello,
>
> On Tue, Nov 20, 2018 at 04:43:52PM +0000, Roman Gushchin wrote:
> > > But that wouldn't udpate the cgroup's frozen state and generate
> > > notifications, right?
> >
> > Why? The task will be eventually trapped into cgroup_enter_frozen(),
> > and from there cgroup_update_frozen() will be called.
>
> Because the cgroup is no longer frozen?
>
> > You are right, that notification will not be issued, because the cgroup
> > is not changing its state (frozen->frozen). I'm not sure that it makes
> > sense to change the cgroup state back and forth in this case. Are there
> > any reasons I'm missing?
>
> Imagine the task being trapped in nfs or wherever and not getting into
> the freezer for an extended period of time.
Yeah, it's a good point. I've thought about it mostly in the fork() context,
where if freezing of a cgroup races with fork(), it makes no sense to
switch the cgroup state back and forth. But that case is different, as
the child will be trapped just on the return path from fork() call.
Thanks!
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v3 4/7] cgroup: cgroup v2 freezer
2018-11-20 17:39 ` Roman Gushchin
@ 2018-11-20 18:05 ` Tejun Heo
0 siblings, 0 replies; 23+ messages in thread
From: Tejun Heo @ 2018-11-20 18:05 UTC (permalink / raw)
To: Roman Gushchin
Cc: Roman Gushchin, Oleg Nesterov, cgroups, linux-kernel, Kernel Team
Hello,
On Tue, Nov 20, 2018 at 05:39:03PM +0000, Roman Gushchin wrote:
> Yeah, it's a good point. I've thought about it mostly in the fork() context,
> where if freezing of a cgroup races with fork(), it makes no sense to
> switch the cgroup state back and forth. But that case is different, as
> the child will be trapped just on the return path from fork() call.
I think it'd be best to tie the task counter transitions to cgroup
state update and propagation so that the state is always reflective of
the task states.
Thanks.
--
tejun
^ permalink raw reply [flat|nested] 23+ messages in thread
end of thread, other threads:[~2018-11-20 18:05 UTC | newest]
Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-11-17 0:38 [PATCH v3 0/7] freezer for cgroup v2 Roman Gushchin
2018-11-17 0:38 ` [PATCH v3 1/7] cgroup: rename freezer.c into legacy_freezer.c Roman Gushchin
2018-11-17 0:38 ` [PATCH v3 2/7] cgroup: implement __cgroup_task_count() helper Roman Gushchin
2018-11-17 0:38 ` [PATCH v3 3/7] cgroup: protect cgroup->nr_(dying_)descendants by css_set_lock Roman Gushchin
2018-11-20 16:18 ` Tejun Heo
2018-11-17 0:38 ` [PATCH v3 4/7] cgroup: cgroup v2 freezer Roman Gushchin
2018-11-20 16:25 ` Tejun Heo
2018-11-20 16:33 ` Roman Gushchin
2018-11-20 16:36 ` Tejun Heo
2018-11-20 16:43 ` Roman Gushchin
2018-11-20 16:48 ` Tejun Heo
2018-11-20 17:39 ` Roman Gushchin
2018-11-20 18:05 ` Tejun Heo
2018-11-17 0:38 ` [PATCH v3 5/7] kselftests: cgroup: don't fail on cg_kill_all() error in cg_destroy() Roman Gushchin
2018-11-17 0:38 ` Roman Gushchin
2018-11-17 0:38 ` guroan
2018-11-17 0:38 ` [PATCH v3 6/7] kselftests: cgroup: add freezer controller self-tests Roman Gushchin
2018-11-17 0:38 ` Roman Gushchin
2018-11-17 0:38 ` guroan
2018-11-17 0:38 ` [PATCH v3 7/7] cgroup: document cgroup v2 freezer interface Roman Gushchin
2018-11-17 8:02 ` Mike Rapoport
2018-11-19 17:42 ` Roman Gushchin
2018-11-20 14:06 ` Mike Rapoport
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.