From: Marcelo Tosatti <mtosatti@redhat.com>
To: Fenghua Yu <fenghua.yu@intel.com>
Cc: H Peter Anvin <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>,
Thomas Gleixner <tglx@linutronix.de>,
Peter Zijlstra <peterz@infradead.org>,
linux-kernel <linux-kernel@vger.kernel.org>, x86 <x86@kernel.org>,
Vikas Shivappa <vikas.shivappa@linux.intel.com>
Subject: Re: [PATCH V15 11/11] x86,cgroup/intel_rdt : Add a cgroup interface to manage Intel cache allocation
Date: Wed, 18 Nov 2015 18:58:36 -0200 [thread overview]
Message-ID: <20151118205836.GA21918@amt.cnet> (raw)
In-Reply-To: <1443766185-61618-12-git-send-email-fenghua.yu@intel.com>
On Thu, Oct 01, 2015 at 11:09:45PM -0700, Fenghua Yu wrote:
> Add a new cgroup 'intel_rdt' to manage cache allocation. Each cgroup
> directory is associated with a class of service id(closid). To map a
> task with closid during scheduling, this patch removes the closid field
> from task_struct and uses the already existing 'cgroups' field in
> task_struct.
>
> The cgroup has a file 'l3_cbm' which represents the L3 cache capacity
> bitmask(CBM). The CBM is global for the whole system currently. The
> capacity bitmask needs to have only contiguous bits set and number of
> bits that can be set is less than the max bits that can be set. The
> tasks belonging to a cgroup get to fill in the L3 cache represented by
> the capacity bitmask of the cgroup. For ex: if the max bits in the CBM
> is 10 and the cache size is 10MB, each bit represents 1MB of cache
> capacity.
>
> Root cgroup always has all the bits set in the l3_cbm. User can create
> more cgroups with mkdir syscall. By default the child cgroups inherit
> the capacity bitmask(CBM) from parent. User can change the CBM specified
> in hex for each cgroup. Each unique bitmask is associated with a class
> of service ID and an -ENOSPC is returned once we run out of
> closids.
>
> Signed-off-by: Vikas Shivappa <vikas.shivappa@linux.intel.com>
> Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
> ---
> arch/x86/include/asm/intel_rdt.h | 37 +++++++-
> arch/x86/kernel/cpu/intel_rdt.c | 194 +++++++++++++++++++++++++++++++++++++--
> include/linux/cgroup_subsys.h | 4 +
> include/linux/sched.h | 3 -
> init/Kconfig | 4 +-
> 5 files changed, 229 insertions(+), 13 deletions(-)
>
> diff --git a/arch/x86/include/asm/intel_rdt.h b/arch/x86/include/asm/intel_rdt.h
> index afb6da3..fbe1e00 100644
> --- a/arch/x86/include/asm/intel_rdt.h
> +++ b/arch/x86/include/asm/intel_rdt.h
> @@ -3,6 +3,7 @@
>
> #ifdef CONFIG_INTEL_RDT
>
> +#include <linux/cgroup.h>
> #include <linux/jump_label.h>
>
> #define MAX_CBM_LENGTH 32
> @@ -12,20 +13,54 @@
> extern struct static_key rdt_enable_key;
> void __intel_rdt_sched_in(void *dummy);
>
> +struct intel_rdt {
> + struct cgroup_subsys_state css;
> + u32 closid;
> +};
> +
> struct clos_cbm_table {
> unsigned long l3_cbm;
> unsigned int clos_refcnt;
> };
>
> /*
> + * Return rdt group corresponding to this container.
> + */
> +static inline struct intel_rdt *css_rdt(struct cgroup_subsys_state *css)
> +{
> + return css ? container_of(css, struct intel_rdt, css) : NULL;
> +}
> +
> +static inline struct intel_rdt *parent_rdt(struct intel_rdt *ir)
> +{
> + return css_rdt(ir->css.parent);
> +}
> +
> +/*
> + * Return rdt group to which this task belongs.
> + */
> +static inline struct intel_rdt *task_rdt(struct task_struct *task)
> +{
> + return css_rdt(task_css(task, intel_rdt_cgrp_id));
> +}
> +
> +/*
> * intel_rdt_sched_in() - Writes the task's CLOSid to IA32_PQR_MSR
> *
> * Following considerations are made so that this has minimal impact
> * on scheduler hot path:
> * - This will stay as no-op unless we are running on an Intel SKU
> * which supports L3 cache allocation.
> + * - When support is present and enabled, does not do any
> + * IA32_PQR_MSR writes until the user starts really using the feature
> + * ie creates a rdt cgroup directory and assigns a cache_mask thats
> + * different from the root cgroup's cache_mask.
> * - Caches the per cpu CLOSid values and does the MSR write only
> - * when a task with a different CLOSid is scheduled in.
> + * when a task with a different CLOSid is scheduled in. That
> + * means the task belongs to a different cgroup.
> + * - Closids are allocated so that different cgroup directories
> + * with same cache_mask gets the same CLOSid. This minimizes CLOSids
> + * used and reduces MSR write frequency.
> */
> static inline void intel_rdt_sched_in(void)
> {
> diff --git a/arch/x86/kernel/cpu/intel_rdt.c b/arch/x86/kernel/cpu/intel_rdt.c
> index ecaf8e6..cb4d2ef 100644
> --- a/arch/x86/kernel/cpu/intel_rdt.c
> +++ b/arch/x86/kernel/cpu/intel_rdt.c
> @@ -53,6 +53,10 @@ static cpumask_t tmp_cpumask;
> static DEFINE_MUTEX(rdt_group_mutex);
> struct static_key __read_mostly rdt_enable_key = STATIC_KEY_INIT_FALSE;
>
> +static struct intel_rdt rdt_root_group;
> +#define rdt_for_each_child(pos_css, parent_ir) \
> + css_for_each_child((pos_css), &(parent_ir)->css)
> +
> struct rdt_remote_data {
> int msr;
> u64 val;
> @@ -108,17 +112,16 @@ static inline bool cache_alloc_supported(struct cpuinfo_x86 *c)
> return false;
> }
>
> -
> void __intel_rdt_sched_in(void *dummy)
> {
> struct intel_pqr_state *state = this_cpu_ptr(&pqr_state);
> - u32 closid = current->closid;
> + struct intel_rdt *ir = task_rdt(current);
>
> - if (closid == state->closid)
> + if (ir->closid == state->closid)
> return;
>
> - wrmsr(MSR_IA32_PQR_ASSOC, state->rmid, closid);
> - state->closid = closid;
> + wrmsr(MSR_IA32_PQR_ASSOC, state->rmid, ir->closid);
What if another CPU runs
intel_cache_alloc_cbm_write()
if (cbm_search(cbmvalue, &closid)) {
ir->closid = closid;
Here? Probably a spinlock is necessary.
> + state->closid = ir->closid;
> }
>
next prev parent reply other threads:[~2015-11-18 21:00 UTC|newest]
Thread overview: 42+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-10-02 6:09 [PATCH V15 00/11] x86: Intel Cache Allocation Technology Support Fenghua Yu
2015-10-02 6:09 ` [PATCH V15 01/11] x86/intel_cqm: Modify hot cpu notification handling Fenghua Yu
2015-10-02 6:09 ` [PATCH V15 02/11] x86/intel_rapl: " Fenghua Yu
2015-10-02 6:09 ` [PATCH V15 03/11] x86/intel_rdt: Cache Allocation documentation Fenghua Yu
2015-10-02 6:09 ` [PATCH V15 04/11] x86/intel_rdt: Add support for Cache Allocation detection Fenghua Yu
2015-11-04 14:51 ` Luiz Capitulino
2015-10-02 6:09 ` [PATCH V15 05/11] x86/intel_rdt: Add Class of service management Fenghua Yu
2015-11-04 14:55 ` Luiz Capitulino
2015-10-02 6:09 ` [PATCH V15 06/11] x86/intel_rdt: Add L3 cache capacity bitmask management Fenghua Yu
2015-10-02 6:09 ` [PATCH V15 07/11] x86/intel_rdt: Implement scheduling support for Intel RDT Fenghua Yu
2015-10-02 6:09 ` [PATCH V15 08/11] x86/intel_rdt: Hot cpu support for Cache Allocation Fenghua Yu
2015-10-02 6:09 ` [PATCH V15 09/11] x86/intel_rdt: Intel haswell Cache Allocation enumeration Fenghua Yu
2015-10-02 6:09 ` [PATCH V15 10/11] x86,cgroup/intel_rdt : Add intel_rdt cgroup documentation Fenghua Yu
2015-10-02 6:09 ` [PATCH V15 11/11] x86,cgroup/intel_rdt : Add a cgroup interface to manage Intel cache allocation Fenghua Yu
2015-11-18 20:58 ` Marcelo Tosatti [this message]
2015-11-18 21:27 ` Marcelo Tosatti
2015-12-16 22:00 ` Yu, Fenghua
2015-11-18 22:15 ` Marcelo Tosatti
2015-12-14 22:58 ` Yu, Fenghua
2015-10-11 19:50 ` [PATCH V15 00/11] x86: Intel Cache Allocation Technology Support Thomas Gleixner
2015-10-12 18:52 ` Yu, Fenghua
2015-10-12 19:58 ` Thomas Gleixner
2015-10-13 22:40 ` Marcelo Tosatti
2015-10-15 11:37 ` Peter Zijlstra
2015-10-16 0:17 ` Marcelo Tosatti
2015-10-16 9:44 ` Peter Zijlstra
2015-10-16 20:24 ` Marcelo Tosatti
2015-10-19 23:49 ` Marcelo Tosatti
2015-10-13 21:31 ` Marcelo Tosatti
2015-10-15 11:36 ` Peter Zijlstra
2015-10-16 2:28 ` Marcelo Tosatti
2015-10-16 9:50 ` Peter Zijlstra
2015-10-26 20:02 ` Marcelo Tosatti
2015-11-02 22:20 ` cat cgroup interface proposal (non hierarchical) was " Marcelo Tosatti
2015-11-04 14:42 ` Luiz Capitulino
2015-11-04 14:57 ` Thomas Gleixner
2015-11-04 15:12 ` Luiz Capitulino
2015-11-04 15:28 ` Thomas Gleixner
2015-11-04 15:35 ` Luiz Capitulino
2015-11-04 15:50 ` Thomas Gleixner
2015-11-05 2:19 ` [PATCH 1/2] x86/intel_rdt,intel_cqm: Remove build dependency of RDT code on CQM code David Carrillo-Cisneros
2015-11-05 2:19 ` [PATCH 2/2] x86/intel_rdt: Fix bug in initialization, locks and write cbm mask David Carrillo-Cisneros
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20151118205836.GA21918@amt.cnet \
--to=mtosatti@redhat.com \
--cc=fenghua.yu@intel.com \
--cc=hpa@zytor.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
--cc=vikas.shivappa@linux.intel.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).