Re: [PATCH v2 08/11] mm/vmstat: switch counter modification to cmpxchg

From: Marcelo Tosatti <mtosatti@redhat.com>
To: Peter Xu <peterx@redhat.com>
Cc: David Hildenbrand <david@redhat.com>,
	Christoph Lameter <cl@linux.com>,
	Aaron Tomlin <atomlin@atomlin.com>,
	Frederic Weisbecker <frederic@kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org
Subject: Re: [PATCH v2 08/11] mm/vmstat: switch counter modification to cmpxchg
Date: Thu, 2 Mar 2023 16:11:32 -0300	[thread overview]
Message-ID: <ZAD05KayfUwZQ4Vh@tpad> (raw)
In-Reply-To: <ZADM4U49g+g4S5Xf@x1n>

On Thu, Mar 02, 2023 at 11:20:49AM -0500, Peter Xu wrote:
> On Thu, Mar 02, 2023 at 11:47:35AM -0300, Marcelo Tosatti wrote:
> > So it will be:
> > 
> > #ifdef CONFIG_HAVE_CMPXCHG_LOCAL
> > mod_zone_page_state
> > inc_zone_page_state
> > dec_zone_page_state
> > mod_node_page_state
> > inc_node_page_state
> > dec_node_page_state
> > __mod_zone_page_state (new function, calls mod_zone_page_state).
> > __mod_node_page_state (new function, calls mod_node_page_state).
> > __inc_zone_page_state
> > __inc_node_page_state
> > __dec_zone_page_state
> > __dec_node_page_state
> > #else
> > __mod_zone_page_state (old, shared function for both CONFIG_HAVE_CMPXCHG_LOCAL and not)
> > __mod_node_page_state
> > __inc_zone_page_state
> > __inc_node_page_state
> > __dec_zone_page_state
> > __dec_node_page_state
> > mod_zone_page_state
> > inc_zone_page_state
> > dec_zone_page_state
> > mod_node_page_state
> > inc_node_page_state
> > dec_node_page_state
> > #endif
> > 
> > Any suggestion on how to split this into multiple patchsets for easier
> > reviewing? (can't think of anything obvious).
> 
> I figured this out before saw this, but it did take me some time to read
> carefully into the code base..  maybe it'll be a good idea to mention
> something like above in the commit message to ease future reviewers (and
> more likelyhood to attract the experts to start chim in)?
> 
> One fundamental (but maybe another naive.. :) question on this code piece
> (so not directly related to the changeset but maybe it is still..):
> 
> AFAICT CONFIG_HAVE_CMPXCHG_LOCAL only means we can do cmpxchg() without
> locking memory bus, 

CONFIG_HAVE_CMPXCHG_LOCAL means cmpxchg_local is implemented (that is
cmpxchg which is atomic with respect to local CPU).

LOCK cmpxchg is necessary for cmpxchg to be atomic on SMP.

> however when !CONFIG_HAVE_CMPXCHG_LOCAL here we're not
> using non-local version but using preempt_disable_nested() to make sure the
> read is atomic.  Does it really worth it?  What happens if we use cmpxchg()
> unconditionally, but just use local (e.g. no "LOCK" prefix) version when
> CONFIG_HAVE_CMPXCHG_LOCAL?

Can't use local version of cmpxchg because the vmstat counters are supposed
to be accessed from different CPUs simultaneously (this is the objective
of the patchset):

CPU-0					CPU-1

vmstat_shepherd				mod_zone_page_state
xchg location				LOCK cmpxchg location

xchg locks memory bus implicitly.

Is this what you are thinking about or did i misunderstood what you
mean?