RE: [PATCH v11 01/13] x86/resctrl: Replace smp_call_function_many() with on_each_cpu_mask()

["Moger, Babu" <Babu.Moger@amd.com>]

[AMD Official Use Only - General]

Hi Ashok,

> -----Original Message-----
> From: Ashok Raj <ashok_raj@linux.intel.com>
> Sent: Monday, January 9, 2023 5:27 PM
> To: Moger, Babu <Babu.Moger@amd.com>
> Cc: corbet@lwn.net; reinette.chatre@intel.com; tglx@linutronix.de;
> mingo@redhat.com; bp@alien8.de; fenghua.yu@intel.com;
> dave.hansen@linux.intel.com; x86@kernel.org; hpa@zytor.com;
> paulmck@kernel.org; akpm@linux-foundation.org; quic_neeraju@quicinc.com;
> rdunlap@infradead.org; damien.lemoal@opensource.wdc.com;
> songmuchun@bytedance.com; peterz@infradead.org; jpoimboe@kernel.org;
> pbonzini@redhat.com; chang.seok.bae@intel.com;
> pawan.kumar.gupta@linux.intel.com; jmattson@google.com;
> daniel.sneddon@linux.intel.com; Das1, Sandipan <Sandipan.Das@amd.com>;
> tony.luck@intel.com; james.morse@arm.com; linux-doc@vger.kernel.org;
> linux-kernel@vger.kernel.org; bagasdotme@gmail.com; eranian@google.com;
> christophe.leroy@csgroup.eu; jarkko@kernel.org; adrian.hunter@intel.com;
> quic_jiles@quicinc.com; peternewman@google.com; Ashok Raj
> <ashok.raj@intel.com>
> Subject: Re: [PATCH v11 01/13] x86/resctrl: Replace smp_call_function_many()
> with on_each_cpu_mask()
> 
> On Mon, Jan 09, 2023 at 10:43:53AM -0600, Babu Moger wrote:
> > on_each_cpu_mask() runs the function on each CPU specified by cpumask,
> > which may include the local processor.
> >
> > Replace smp_call_function_many() with on_each_cpu_mask() to simplify
> > the code.
> >
> > Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
> > Signed-off-by: Babu Moger <babu.moger@amd.com>
> > ---
> >  arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 11 +++------
> >  arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 29 +++++++----------------
> >  2 files changed, 11 insertions(+), 29 deletions(-)
> >
> > diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
> > b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
> > index 1df0e3262bca..7eece3d2d0c3 100644
> > --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
> > +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
> > @@ -310,7 +310,6 @@ int resctrl_arch_update_domains(struct rdt_resource
> *r, u32 closid)
> >  	enum resctrl_conf_type t;
> >  	cpumask_var_t cpu_mask;
> >  	struct rdt_domain *d;
> > -	int cpu;
> >  	u32 idx;
> >
> >  	if (!zalloc_cpumask_var(&cpu_mask, GFP_KERNEL)) @@ -341,13
> +340,9 @@
> > int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)
> >
> >  	if (cpumask_empty(cpu_mask))
> >  		goto done;
> > -	cpu = get_cpu();
> > -	/* Update resource control msr on this CPU if it's in cpu_mask. */
> > -	if (cpumask_test_cpu(cpu, cpu_mask))
> > -		rdt_ctrl_update(&msr_param);
> > -	/* Update resource control msr on other CPUs. */
> > -	smp_call_function_many(cpu_mask, rdt_ctrl_update, &msr_param, 1);
> > -	put_cpu();
> > +
> > +	/* Update resource control msr on all the CPUs. */
> > +	on_each_cpu_mask(cpu_mask, rdt_ctrl_update, &msr_param, 1);
> 
> Do you require these updates to done immediately via an IPI? or can they be
> done bit lazy via schedule_on_each_cpu()?

I have not experimented with lazy schedule.  At least I know the call update_cpu_closid_rmid should be completed immediately. Otherwise, the result might be inconsistent as the tasks(or CPUs)  could be running on two different closed/rmids before it is updated on all CPUs in the domain.
Thanks
Babu