Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration

From: Marcelo Tosatti <mtosatti@redhat.com>
To: Christoph Lameter <cl@linux.com>
Cc: Luiz Capitulino <lcapitulino@redhat.com>,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	Rik van Riel <riel@redhat.com>,
	Linux RT Users <linux-rt-users@vger.kernel.org>,
	cmetcalf@mellanox.com
Subject: Re: [patch 2/2] MM: allow per-cpu vmstat_threshold and vmstat_worker configuration
Date: Fri, 12 May 2017 12:40:28 -0300	[thread overview]
Message-ID: <20170512154026.GA3556@amt.cnet> (raw)
In-Reply-To: <alpine.DEB.2.20.1705121002310.22243@east.gentwo.org>

On Fri, May 12, 2017 at 10:11:14AM -0500, Christoph Lameter wrote:
> On Fri, 12 May 2017, Marcelo Tosatti wrote:
> 
> > > A bit confused by this one. The vmstat worker is already disabled if there
> > > are no updates. Also the patches by Chris Metcalf on data plane mode add a
> > > prctl to quiet the vmstat workers.
> > >
> > > Why do we need more than this?
> >
> > If there are vmstat statistic updates on a given CPU, and you don't
> > want intervention from the vmstat worker, you change the behaviour of
> > stat data collection to directly write to the global structures (which
> > disables the performance optimization of collecting data in per-cpu
> > counters).
> 
> Hmmm.... Ok. That is going to be expensive if you do this for each
> individual vmstat update.

In our case, vmstat updates are very rare (CPU is dominated by DPDK).

> > This way you can disable vmstat worker (because it causes undesired
> > latencies), while allowing vmstatistics to function properly.
> 
> Best then to run the vmstat update mechanism when you leave kernel mode to
> get all the updates in one go.

Again, vmstat updates are very rare (CPU is dominated by DPDK).

> > The prctl from Chris Metcalf patchset allows one to disable vmstat
> > worker per CPU? If so, they replace the functionality of the patch
> > "[patch 3/3] MM: allow per-cpu vmstat_worker configuration"
> > of the -v2 series of my patchset, and we can use it instead.
> >
> > Is it integrated already?
> 
> The data plane mode patches disables vmstat processing  by updating the
> vmstats immediately if necessary and switching off the kworker thread.

OK this is what my patch set is doing.

> So the kworker wont be running until the next time statistics are checked
> by the shepherd task from a remote cpu.

We don't want kworker thread to ever run.

>  If the counters have been updated
> then the shepherd task will reenable the kworker. This is already merged
> and has been working for a long time. Data plan mode has not been merged
> yet but the infrastructure in vmstat.c is there because NOHZ needs it too.

OK.

> 
> See linux/vmstat.c:quiet_vmstat()
> 
> It would be easy to add a /proc file that allows the quieting of the
> vmstat workers for a certain cpu. Just make it call the quiet_vmstat() on
> the right cpu.
> 
> This will quiet vmstat down. The shepherd task will check the stats in 2
> second intervals and will then reenable when necessasry.
> 
> Note that we already are updating the global structures directly if the
> differential gets too high. Reducing the differential may get you what you
> want.

Yes, we reduce the differential to 1 (== direct updates to global
structures).

OK, i'll check if the patches from Chris work for us and then add
Tested-by on that.

Thanks.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>