Re: [RFD] CAT user space interface revisited

From: Thomas Gleixner <tglx@linutronix.de>
To: Marcelo Tosatti <mtosatti@redhat.com>
Cc: LKML <linux-kernel@vger.kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	x86@kernel.org, Luiz Capitulino <lcapitulino@redhat.com>,
	Vikas Shivappa <vikas.shivappa@intel.com>,
	Tejun Heo <tj@kernel.org>, Yu Fenghua <fenghua.yu@intel.com>
Subject: Re: [RFD] CAT user space interface revisited
Date: Thu, 19 Nov 2015 09:35:34 +0100 (CET)	[thread overview]
Message-ID: <alpine.DEB.2.11.1511190911550.3898@nanos> (raw)
In-Reply-To: <20151119003414.GA28388@amt.cnet>

On Wed, 18 Nov 2015, Marcelo Tosatti wrote:
> On Wed, Nov 18, 2015 at 08:34:07PM -0200, Marcelo Tosatti wrote:
> > On Wed, Nov 18, 2015 at 07:25:03PM +0100, Thomas Gleixner wrote:
> > > Assume that you have isolated a CPU and run your important task on
> > > it. You give that task a slice of cache. Now that task needs kernel
> > > services which run in kernel threads on that CPU. We really don't want
> > > to (and cannot) hunt down random kernel threads (think cpu bound
> > > worker threads, softirq threads ....) and give them another slice of
> > > cache. What we really want is:
> > > 
> > >     	 1 1 1 1 0 0 0 0    <- Default cache
> > > 	 0 0 0 0 1 1 1 0    <- Cache for important task
> > > 	 0 0 0 0 0 0 0 1    <- Cache for CPU of important task
> > > 
> > > It would even be sufficient for particular use cases to just associate
> > > a piece of cache to a given CPU and do not bother with tasks at all.
> 
> Well any work on behalf of the important task, should have its cache
> protected as well (example irq handling threads). 

Right, but that's nothing you can do automatically and certainly not
from a random application.

> But for certain kernel tasks for which L3 cache is not beneficial
> (eg: kernel samepage merging), it might useful to exclude such tasks
> from the "important, do not flush" L3 cache portion.

Sure it might be useful, but this needs to be done on a case by case
basis and there is no way to do this in any automated way.

> > > It's hard. Policies are hard by definition, but this one is harder
> > > than most other policies due to the inherent limitations.
> 
> That is exactly why it should be allowed for software to automatically 
> configure the policies.

There is nothing you can do automatically. If you want to allow
applications to set the policies themself, then you need to assign a
portion of the bitmask space and a portion of the cos id space to that
application and then let it do with that space what it wants.

That's where cgroups come into play. But that does not solve the other
issues of "global" configuration, i.e. CPU defaults etc.

Thanks,

	tglx