Re: [PATCH v11 01/13] x86/resctrl: Replace smp_call_function_many() with on_each_cpu_mask()

From: Ashok Raj <ashok.raj@intel.com>
To: "Luck, Tony" <tony.luck@intel.com>
Cc: "Moger, Babu" <Babu.Moger@amd.com>,
	Ashok Raj <ashok_raj@linux.intel.com>,
	"corbet@lwn.net" <corbet@lwn.net>,
	"Chatre, Reinette" <reinette.chatre@intel.com>,
	"tglx@linutronix.de" <tglx@linutronix.de>,
	"mingo@redhat.com" <mingo@redhat.com>,
	"bp@alien8.de" <bp@alien8.de>,
	"Yu, Fenghua" <fenghua.yu@intel.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"x86@kernel.org" <x86@kernel.org>,
	"hpa@zytor.com" <hpa@zytor.com>,
	"paulmck@kernel.org" <paulmck@kernel.org>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"quic_neeraju@quicinc.com" <quic_neeraju@quicinc.com>,
	"rdunlap@infradead.org" <rdunlap@infradead.org>,
	"damien.lemoal@opensource.wdc.com"
	<damien.lemoal@opensource.wdc.com>,
	"songmuchun@bytedance.com" <songmuchun@bytedance.com>,
	"peterz@infradead.org" <peterz@infradead.org>,
	"jpoimboe@kernel.org" <jpoimboe@kernel.org>,
	"pbonzini@redhat.com" <pbonzini@redhat.com>,
	"Bae, Chang Seok" <chang.seok.bae@intel.com>,
	"pawan.kumar.gupta@linux.intel.com" 
	<pawan.kumar.gupta@linux.intel.com>,
	"jmattson@google.com" <jmattson@google.com>,
	"daniel.sneddon@linux.intel.com" <daniel.sneddon@linux.intel.com>,
	"Das1, Sandipan" <Sandipan.Das@amd.com>,
	"james.morse@arm.com" <james.morse@arm.com>,
	"linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"bagasdotme@gmail.com" <bagasdotme@gmail.com>,
	"Eranian, Stephane" <eranian@google.com>,
	"christophe.leroy@csgroup.eu" <christophe.leroy@csgroup.eu>,
	"jarkko@kernel.org" <jarkko@kernel.org>,
	"Hunter, Adrian" <adrian.hunter@intel.com>,
	"quic_jiles@quicinc.com" <quic_jiles@quicinc.com>,
	"peternewman@google.com" <peternewman@google.com>,
	Ashok Raj <ashok.raj@intel.com>
Subject: Re: [PATCH v11 01/13] x86/resctrl: Replace smp_call_function_many() with on_each_cpu_mask()
Date: Wed, 11 Jan 2023 07:42:57 -0800	[thread overview]
Message-ID: <Y77ZAfhc48W41FGp@a4bf019067fa.jf.intel.com> (raw)
In-Reply-To: <DS7PR11MB60772F09B9749307CFA0ED0DFCFF9@DS7PR11MB6077.namprd11.prod.outlook.com>

On Tue, Jan 10, 2023 at 12:58:47PM -0800, Tony Luck wrote:
> > > > + /* Update resource control msr on all the CPUs. */
> > > > + on_each_cpu_mask(cpu_mask, rdt_ctrl_update, &msr_param, 1);
> > >
> > > Do you require these updates to done immediately via an IPI? or can they be
> > > done bit lazy via schedule_on_each_cpu()?
> >
> > I have not experimented with lazy schedule.  At least I know the call
> > update_cpu_closid_rmid should be completed immediately. Otherwise, the
> > result might be inconsistent as the tasks(or CPUs)  could be running on
> > two different closed/rmids before it is updated on all CPUs in the domain.
> 
> I think this does need to happen somewhat urgently. Imagine trying to give
> some extra resources to a CPU bound real-time process. That process will
> keep running with the old resource allocation.

If the resctl was setup before spawning other threads then the thread
starts with the right values from the start, probably inheriting from the
parent?

I wasn't sure if the few ms difference is going to make much material
difference for that process. IPI's does shake things up and introduces
other overheads not related to this process.

Instead of victimizing just this process, we hurt everything else.

Does it make sense to do an experiment and see if there is any other
functional failures?

Cheers,
Ashok