From: haoxin <xhao@linux.alibaba.com>
To: James Morse <james.morse@arm.com>,
x86@kernel.org, linux-kernel@vger.kernel.org
Cc: Fenghua Yu <fenghua.yu@intel.com>,
Reinette Chatre <reinette.chatre@intel.com>,
Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
H Peter Anvin <hpa@zytor.com>, Babu Moger <Babu.Moger@amd.com>,
shameerali.kolothum.thodi@huawei.com,
D Scott Phillips OS <scott@os.amperecomputing.com>,
lcherian@marvell.com, bobo.shaobowang@huawei.com,
tan.shaopeng@fujitsu.com, Jamie Iles <quic_jiles@quicinc.com>,
Cristian Marussi <cristian.marussi@arm.com>,
xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com,
xhao@linux.alibaba.com
Subject: Re: [PATCH v6 05/21] x86/resctrl: Add domain offline callback for resctrl work
Date: Wed, 7 Sep 2022 14:29:38 +0800 [thread overview]
Message-ID: <d8fbb0c6-efe5-13bd-fb35-c6cb24ea7ff2@linux.alibaba.com> (raw)
In-Reply-To: <20220902154829.30399-6-james.morse@arm.com>
在 2022/9/2 下午11:48, James Morse 写道:
> Because domains are exposed to user-space via resctrl, the filesystem
> must update its state when CPU hotplug callbacks are triggered.
>
> Some of this work is common to any architecture that would support
> resctrl, but the work is tied up with the architecture code to
> free the memory.
>
> Move the monitor subdir removal and the cancelling of the mbm/limbo
> works into a new resctrl_offline_domain() call. These bits are not
> specific to the architecture. Grouping them in one function allows
> that code to be moved to /fs/ and re-used by another architecture.
>
> Reviewed-by: Jamie Iles <quic_jiles@quicinc.com>
> Tested-by: Xin Hao <xhao@linux.alibaba.com>
> Reviewed-by: Shaopeng Tan <tan.shaopeng@fujitsu.com>
> Tested-by: Shaopeng Tan <tan.shaopeng@fujitsu.com>
> Tested-by: Cristian Marussi <cristian.marussi@arm.com>
> Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
> Changes since v2:
> * Moved kfree()ing to domain_destroy_mon_state() for later re-use.
>
> Changes since v1:
> * Removed a redundant mon_capable check
> * Capitalisation
> * Removed inline comment
> * Added to the commit message
> ---
> arch/x86/kernel/cpu/resctrl/core.c | 26 ++-------------
> arch/x86/kernel/cpu/resctrl/internal.h | 2 --
> arch/x86/kernel/cpu/resctrl/rdtgroup.c | 45 +++++++++++++++++++++++---
> include/linux/resctrl.h | 1 +
> 4 files changed, 44 insertions(+), 30 deletions(-)
>
> diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
> index e37889f7a1a5..f69182973175 100644
> --- a/arch/x86/kernel/cpu/resctrl/core.c
> +++ b/arch/x86/kernel/cpu/resctrl/core.c
> @@ -523,27 +523,8 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)
>
> cpumask_clear_cpu(cpu, &d->cpu_mask);
> if (cpumask_empty(&d->cpu_mask)) {
> - /*
> - * If resctrl is mounted, remove all the
> - * per domain monitor data directories.
> - */
> - if (static_branch_unlikely(&rdt_mon_enable_key))
> - rmdir_mondata_subdir_allrdtgrp(r, d->id);
> + resctrl_offline_domain(r, d);
> list_del(&d->list);
> - if (r->mon_capable && is_mbm_enabled())
> - cancel_delayed_work(&d->mbm_over);
> - if (is_llc_occupancy_enabled() && has_busy_rmid(r, d)) {
> - /*
> - * When a package is going down, forcefully
> - * decrement rmid->ebusy. There is no way to know
> - * that the L3 was flushed and hence may lead to
> - * incorrect counts in rare scenarios, but leaving
> - * the RMID as busy creates RMID leaks if the
> - * package never comes back.
> - */
> - __check_limbo(d, true);
> - cancel_delayed_work(&d->cqm_limbo);
> - }
>
> /*
> * rdt_domain "d" is going to be freed below, so clear
> @@ -551,11 +532,8 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)
> */
> if (d->plr)
> d->plr->d = NULL;
> -
> - bitmap_free(d->rmid_busy_llc);
> - kfree(d->mbm_total);
> - kfree(d->mbm_local);
> domain_free(hw_dom);
> +
> return;
> }
>
> diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
> index be48a682dbdb..e12b55f815bf 100644
> --- a/arch/x86/kernel/cpu/resctrl/internal.h
> +++ b/arch/x86/kernel/cpu/resctrl/internal.h
> @@ -522,8 +522,6 @@ void free_rmid(u32 rmid);
> int rdt_get_mon_l3_config(struct rdt_resource *r);
> void mon_event_count(void *info);
> int rdtgroup_mondata_show(struct seq_file *m, void *arg);
> -void rmdir_mondata_subdir_allrdtgrp(struct rdt_resource *r,
> - unsigned int dom_id);
> void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
> struct rdt_domain *d, struct rdtgroup *rdtgrp,
> int evtid, int first);
> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> index 030a70326ccc..5830905a92d2 100644
> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -2499,14 +2499,12 @@ static int mon_addfile(struct kernfs_node *parent_kn, const char *name,
> * Remove all subdirectories of mon_data of ctrl_mon groups
> * and monitor groups with given domain id.
> */
> -void rmdir_mondata_subdir_allrdtgrp(struct rdt_resource *r, unsigned int dom_id)
> +static void rmdir_mondata_subdir_allrdtgrp(struct rdt_resource *r,
> + unsigned int dom_id)
> {
> struct rdtgroup *prgrp, *crgrp;
> char name[32];
>
> - if (!r->mon_capable)
> - return;
> -
> list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
> sprintf(name, "mon_%s_%02d", r->name, dom_id);
> kernfs_remove_by_name(prgrp->mon.mon_data_kn, name);
> @@ -3233,6 +3231,45 @@ static int __init rdtgroup_setup_root(void)
> return ret;
> }
>
> +static void domain_destroy_mon_state(struct rdt_domain *d)
add inline ?
> +{
> + bitmap_free(d->rmid_busy_llc);
> + kfree(d->mbm_total);
> + kfree(d->mbm_local);
> +}
> +
> +void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d)
> +{
> + lockdep_assert_held(&rdtgroup_mutex);
> +
> + if (!r->mon_capable)
> + return;
> +
> + /*
> + * If resctrl is mounted, remove all the
> + * per domain monitor data directories.
> + */
> + if (static_branch_unlikely(&rdt_mon_enable_key))
> + rmdir_mondata_subdir_allrdtgrp(r, d->id);
> +
> + if (is_mbm_enabled())
> + cancel_delayed_work(&d->mbm_over);
> + if (is_llc_occupancy_enabled() && has_busy_rmid(r, d)) {
> + /*
> + * When a package is going down, forcefully
> + * decrement rmid->ebusy. There is no way to know
> + * that the L3 was flushed and hence may lead to
> + * incorrect counts in rare scenarios, but leaving
> + * the RMID as busy creates RMID leaks if the
> + * package never comes back.
> + */
> + __check_limbo(d, true);
> + cancel_delayed_work(&d->cqm_limbo);
> + }
> +
> + domain_destroy_mon_state(d);
> +}
> +
> static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d)
> {
> size_t tsize;
> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
> index d512455b4c3a..5d283bdd6162 100644
> --- a/include/linux/resctrl.h
> +++ b/include/linux/resctrl.h
> @@ -193,5 +193,6 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid);
> u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d,
> u32 closid, enum resctrl_conf_type type);
> int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d);
> +void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d);
>
> #endif /* _RESCTRL_H */
next prev parent reply other threads:[~2022-09-07 6:30 UTC|newest]
Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-02 15:48 [PATCH v6 00/21] x86/resctrl: Make resctrl_arch_rmid_read() return values in bytes James Morse
2022-09-02 15:48 ` [PATCH v6 01/21] x86/resctrl: Kill off alloc_enabled James Morse
2022-09-23 18:02 ` [tip: x86/cache] " tip-bot2 for James Morse
2022-09-02 15:48 ` [PATCH v6 02/21] x86/resctrl: Merge mon_capable and mon_enabled James Morse
2022-09-23 18:01 ` [tip: x86/cache] " tip-bot2 for James Morse
2022-09-02 15:48 ` [PATCH v6 03/21] x86/resctrl: Add domain online callback for resctrl work James Morse
2022-09-23 18:01 ` [tip: x86/cache] " tip-bot2 for James Morse
2022-09-02 15:48 ` [PATCH v6 04/21] x86/resctrl: Group struct rdt_hw_domain cleanup James Morse
2022-09-07 6:28 ` haoxin
2022-09-08 17:01 ` James Morse
2022-09-23 18:01 ` [tip: x86/cache] " tip-bot2 for James Morse
2022-09-02 15:48 ` [PATCH v6 05/21] x86/resctrl: Add domain offline callback for resctrl work James Morse
2022-09-07 6:29 ` haoxin [this message]
2022-09-08 17:00 ` James Morse
2022-09-23 18:01 ` [tip: x86/cache] " tip-bot2 for James Morse
2022-09-02 15:48 ` [PATCH v6 06/21] x86/resctrl: Remove set_mba_sc()s control array re-initialisation James Morse
2022-09-23 18:01 ` [tip: x86/cache] " tip-bot2 for James Morse
2022-09-02 15:48 ` [PATCH v6 07/21] x86/resctrl: Abstract and use supports_mba_mbps() James Morse
2022-09-23 18:01 ` [tip: x86/cache] " tip-bot2 for James Morse
2022-09-02 15:48 ` [PATCH v6 08/21] x86/resctrl: Create mba_sc configuration in the rdt_domain James Morse
2022-09-23 18:01 ` [tip: x86/cache] " tip-bot2 for James Morse
2022-09-02 15:48 ` [PATCH v6 09/21] x86/resctrl: Switch over to the resctrl mbps_val list James Morse
2022-09-23 18:01 ` [tip: x86/cache] " tip-bot2 for James Morse
2022-09-02 15:48 ` [PATCH v6 10/21] x86/resctrl: Remove architecture copy of mbps_val James Morse
2022-09-23 18:01 ` [tip: x86/cache] " tip-bot2 for James Morse
2022-09-02 15:48 ` [PATCH v6 11/21] x86/resctrl: Allow update_mba_bw() to update controls directly James Morse
2022-09-23 18:01 ` [tip: x86/cache] " tip-bot2 for James Morse
2022-09-02 15:48 ` [PATCH v6 12/21] x86/resctrl: Calculate bandwidth from the previous __mon_event_count() chunks James Morse
2022-09-07 6:47 ` haoxin
2022-09-08 17:00 ` James Morse
2022-09-08 21:13 ` Reinette Chatre
2022-09-23 18:01 ` [tip: x86/cache] " tip-bot2 for James Morse
2022-09-02 15:48 ` [PATCH v6 13/21] x86/resctrl: Add per-rmid arch private storage for overflow and chunks James Morse
2022-09-23 18:01 ` [tip: x86/cache] " tip-bot2 for James Morse
2022-09-02 15:48 ` [PATCH v6 14/21] x86/resctrl: Allow per-rmid arch private storage to be reset James Morse
2022-09-23 18:01 ` [tip: x86/cache] " tip-bot2 for James Morse
2022-09-02 15:48 ` [PATCH v6 15/21] x86/resctrl: Abstract __rmid_read() James Morse
2022-09-23 18:01 ` [tip: x86/cache] " tip-bot2 for James Morse
2022-09-02 15:48 ` [PATCH v6 16/21] x86/resctrl: Pass the required parameters into resctrl_arch_rmid_read() James Morse
2022-09-23 18:01 ` [tip: x86/cache] " tip-bot2 for James Morse
2022-09-02 15:48 ` [PATCH v6 17/21] x86/resctrl: Move mbm_overflow_count() " James Morse
2022-09-23 18:01 ` [tip: x86/cache] " tip-bot2 for James Morse
2022-09-02 15:48 ` [PATCH v6 18/21] x86/resctrl: Move get_corrected_mbm_count() " James Morse
2022-09-23 18:01 ` [tip: x86/cache] " tip-bot2 for James Morse
2022-09-02 15:48 ` [PATCH v6 19/21] x86/resctrl: Rename and change the units of resctrl_cqm_threshold James Morse
2022-09-23 18:01 ` [tip: x86/cache] " tip-bot2 for James Morse
2022-09-02 15:48 ` [PATCH v6 20/21] x86/resctrl: Add resctrl_rmid_realloc_limit to abstract x86's boot_cpu_data James Morse
2022-09-07 6:26 ` haoxin
2022-09-08 17:01 ` James Morse
2022-09-23 18:01 ` [tip: x86/cache] " tip-bot2 for James Morse
2022-09-02 15:48 ` [PATCH v6 21/21] x86/resctrl: Make resctrl_arch_rmid_read() return values in bytes James Morse
2022-09-23 18:01 ` [tip: x86/cache] " tip-bot2 for James Morse
2022-09-13 1:00 ` [PATCH v6 00/21] " tan.shaopeng
2022-09-22 9:40 ` James Morse
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d8fbb0c6-efe5-13bd-fb35-c6cb24ea7ff2@linux.alibaba.com \
--to=xhao@linux.alibaba.com \
--cc=Babu.Moger@amd.com \
--cc=baolin.wang@linux.alibaba.com \
--cc=bobo.shaobowang@huawei.com \
--cc=bp@alien8.de \
--cc=cristian.marussi@arm.com \
--cc=fenghua.yu@intel.com \
--cc=hpa@zytor.com \
--cc=james.morse@arm.com \
--cc=lcherian@marvell.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=quic_jiles@quicinc.com \
--cc=reinette.chatre@intel.com \
--cc=scott@os.amperecomputing.com \
--cc=shameerali.kolothum.thodi@huawei.com \
--cc=tan.shaopeng@fujitsu.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
--cc=xingxin.hx@openanolis.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).