From: Todd Poynor <toddpoynor@gmail.com> To: Tejun Heo <tj@kernel.org>, Li Zefan <lizefan@huawei.com>, Johannes Weiner <hannes@cmpxchg.org> Cc: cgroups@vger.kernel.org, Ingo Molnar <mingo@redhat.com>, Peter Zijlstra <peterz@infradead.org>, linux-kernel@vger.kernel.org, kernel-team@android.com, Todd Poynor <toddpoynor@google.com> Subject: [PATCH v2] cgroup: move cgroup_subsys_state parent field for cache locality Date: Thu, 6 Apr 2017 18:47:57 -0700 [thread overview] Message-ID: <1491529677-707-1-git-send-email-todd@tjlinux.dyndns.org> (raw) From: Todd Poynor <toddpoynor@google.com> Various structures embed a struct cgroup_subsys_state, typically at the top of the containing structure. It is common for code that accesses the structures to perform operations that iterate over the chain of parent css pointers, also accessing data in each containing structure. In particular, struct cpuacct is used by fairly hot code paths in the scheduler such as cpuacct_charge(). Move the parent css pointer field to the end of the structure to increase the chances of residing in the same cache line as the data from the containing structure. Signed-off-by: Todd Poynor <toddpoynor@google.com> --- include/linux/cgroup-defs.h | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) root_cpuacct fields .cpuusage and. css.parent show up as frequently- accessed memory in separate cache lines (and usually the only thing accessed in those cache lines until eviction) in armv8 simulations. A quick search turned up struct blkcg, struct mem_cgroup, and struct freezer as other examples using a similar struct layout and access code. Instead, could move the parent field to the top of css, and have hot code paths use __cacheline_aligned with hot data prior to css... or open to suggestions, thanks. v2 fixes subject line. diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h index 6a3f850cabab..53c698207ad0 100644 --- a/include/linux/cgroup-defs.h +++ b/include/linux/cgroup-defs.h @@ -106,9 +106,6 @@ struct cgroup_subsys_state { /* reference count - access via css_[try]get() and css_put() */ struct percpu_ref refcnt; - /* PI: the parent css */ - struct cgroup_subsys_state *parent; - /* siblings list anchored at the parent's ->children */ struct list_head sibling; struct list_head children; @@ -138,6 +135,12 @@ struct cgroup_subsys_state { /* percpu_ref killing and RCU release */ struct rcu_head rcu_head; struct work_struct destroy_work; + + /* + * PI: the parent css. Placed here for cache proximity to following + * fields of the containing structure. + */ + struct cgroup_subsys_state *parent; }; /* -- 2.12.2.715.g7642488e1d-goog
WARNING: multiple messages have this Message-ID (diff)
From: Todd Poynor <toddpoynor-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> To: Tejun Heo <tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>, Li Zefan <lizefan-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>, Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org> Cc: cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Ingo Molnar <mingo-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>, Peter Zijlstra <peterz-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, kernel-team-z5hGa2qSFaRBDgjK7y7TUQ@public.gmane.org, Todd Poynor <toddpoynor-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> Subject: [PATCH v2] cgroup: move cgroup_subsys_state parent field for cache locality Date: Thu, 6 Apr 2017 18:47:57 -0700 [thread overview] Message-ID: <1491529677-707-1-git-send-email-todd@tjlinux.dyndns.org> (raw) From: Todd Poynor <toddpoynor-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> Various structures embed a struct cgroup_subsys_state, typically at the top of the containing structure. It is common for code that accesses the structures to perform operations that iterate over the chain of parent css pointers, also accessing data in each containing structure. In particular, struct cpuacct is used by fairly hot code paths in the scheduler such as cpuacct_charge(). Move the parent css pointer field to the end of the structure to increase the chances of residing in the same cache line as the data from the containing structure. Signed-off-by: Todd Poynor <toddpoynor-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> --- include/linux/cgroup-defs.h | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) root_cpuacct fields .cpuusage and. css.parent show up as frequently- accessed memory in separate cache lines (and usually the only thing accessed in those cache lines until eviction) in armv8 simulations. A quick search turned up struct blkcg, struct mem_cgroup, and struct freezer as other examples using a similar struct layout and access code. Instead, could move the parent field to the top of css, and have hot code paths use __cacheline_aligned with hot data prior to css... or open to suggestions, thanks. v2 fixes subject line. diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h index 6a3f850cabab..53c698207ad0 100644 --- a/include/linux/cgroup-defs.h +++ b/include/linux/cgroup-defs.h @@ -106,9 +106,6 @@ struct cgroup_subsys_state { /* reference count - access via css_[try]get() and css_put() */ struct percpu_ref refcnt; - /* PI: the parent css */ - struct cgroup_subsys_state *parent; - /* siblings list anchored at the parent's ->children */ struct list_head sibling; struct list_head children; @@ -138,6 +135,12 @@ struct cgroup_subsys_state { /* percpu_ref killing and RCU release */ struct rcu_head rcu_head; struct work_struct destroy_work; + + /* + * PI: the parent css. Placed here for cache proximity to following + * fields of the containing structure. + */ + struct cgroup_subsys_state *parent; }; /* -- 2.12.2.715.g7642488e1d-goog
next reply other threads:[~2017-04-07 1:48 UTC|newest] Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-04-07 1:47 Todd Poynor [this message] 2017-04-07 1:47 ` [PATCH v2] cgroup: move cgroup_subsys_state parent field for cache locality Todd Poynor 2017-04-11 0:05 ` Tejun Heo 2017-04-11 0:05 ` Tejun Heo
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1491529677-707-1-git-send-email-todd@tjlinux.dyndns.org \ --to=toddpoynor@gmail.com \ --cc=cgroups@vger.kernel.org \ --cc=hannes@cmpxchg.org \ --cc=kernel-team@android.com \ --cc=linux-kernel@vger.kernel.org \ --cc=lizefan@huawei.com \ --cc=mingo@redhat.com \ --cc=peterz@infradead.org \ --cc=tj@kernel.org \ --cc=toddpoynor@google.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.