RE: [PATCH v8 6/7] apei/ghes: Use unrcu_pointer for cmpxchg

From: Justin He <Justin.He@arm.com>
To: Ard Biesheuvel <ardb@kernel.org>, Borislav Petkov <bp@alien8.de>
Cc: Len Brown <lenb@kernel.org>, James Morse <James.Morse@arm.com>,
	Tony Luck <tony.luck@intel.com>,
	Mauro Carvalho Chehab <mchehab@kernel.org>,
	Robert Richter <rric@kernel.org>,
	Robert Moore <robert.moore@intel.com>,
	Qiuxu Zhuo <qiuxu.zhuo@intel.com>,
	Yazen Ghannam <yazen.ghannam@amd.com>,
	Jan Luebbe <jlu@pengutronix.de>,
	Khuong Dinh <khuong@os.amperecomputing.com>,
	Kani Toshi <toshi.kani@hpe.com>,
	"linux-acpi@vger.kernel.org" <linux-acpi@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-edac@vger.kernel.org" <linux-edac@vger.kernel.org>,
	"devel@acpica.org" <devel@acpica.org>,
	"Rafael J . Wysocki" <rafael@kernel.org>,
	Shuai Xue <xueshuai@linux.alibaba.com>,
	Jarkko Sakkinen <jarkko@kernel.org>,
	"linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
	nd <nd@arm.com>, kernel test robot <lkp@intel.com>
Subject: RE: [PATCH v8 6/7] apei/ghes: Use unrcu_pointer for cmpxchg
Date: Fri, 14 Oct 2022 12:00:25 +0000	[thread overview]
Message-ID: <DBBPR08MB453845A7A15596F6FE96DBC9F7249@DBBPR08MB4538.eurprd08.prod.outlook.com> (raw)
In-Reply-To: <CAMj1kXGtTRaKCKJnsJ9XcRus+H16mO3TGsz+TFJLraOyvfciCA@mail.gmail.com>

Hi Ard

> -----Original Message-----
> From: Ard Biesheuvel <ardb@kernel.org>
> Sent: Thursday, October 13, 2022 11:41 PM
> To: Borislav Petkov <bp@alien8.de>
> Cc: Justin He <Justin.He@arm.com>; Len Brown <lenb@kernel.org>; James
> Morse <James.Morse@arm.com>; Tony Luck <tony.luck@intel.com>; Mauro
> Carvalho Chehab <mchehab@kernel.org>; Robert Richter <rric@kernel.org>;
> Robert Moore <robert.moore@intel.com>; Qiuxu Zhuo
> <qiuxu.zhuo@intel.com>; Yazen Ghannam <yazen.ghannam@amd.com>; Jan
> Luebbe <jlu@pengutronix.de>; Khuong Dinh
> <khuong@os.amperecomputing.com>; Kani Toshi <toshi.kani@hpe.com>;
> linux-acpi@vger.kernel.org; linux-kernel@vger.kernel.org;
> linux-edac@vger.kernel.org; devel@acpica.org; Rafael J . Wysocki
> <rafael@kernel.org>; Shuai Xue <xueshuai@linux.alibaba.com>; Jarkko
> Sakkinen <jarkko@kernel.org>; linux-efi@vger.kernel.org; nd <nd@arm.com>;
> kernel test robot <lkp@intel.com>
> Subject: Re: [PATCH v8 6/7] apei/ghes: Use unrcu_pointer for cmpxchg
> 
> On Thu, 13 Oct 2022 at 15:37, Borislav Petkov <bp@alien8.de> wrote:
> >
> > On Wed, Oct 12, 2022 at 12:04:57PM +0000, Justin He wrote:
> > > I have a concern about what if cmpxchg failed? Do we have to still
> > > guarantee the ordering since cmpxchg will not imply a smp_mb if it
> > > failed.
> >
> > Of course it will imply that. At least on x86 it does. smp_wmb() is a
> > compiler barrier there and cmpxchg() already has that barrier
> > semantics by clobbering "memory". I'm pretty sure you should have the
> > same thing on ARM.
> >
> 
> No it definitely does not imply that. A memory clobber is a codegen construct,
> and the hardware could still complete the writes in a way that could result in
> another observer seeing a mix of old and new values that is inconsistent with
> the ordering of the stores as issued by the compiler.
> 
> > says, "new_cache must be put into array after its contents are written".
> >
> > Are we writing anything into the cache if cmpxchg fails?
> >
> 
> The cache fields get updated but the pointer to the struct is never shared
> globally if the cmpxchg() fails so not having the barrier on failure should not be
> an issue here.
> 
> > > Besides, I didn't find the paired smp_mb or smp_rmb for this smp_wmb.
> >
> > Why would there be pairs? I don't understand that statement here.
> >
> 
> Typically, the other observer pairs the write barrier with a read barrier.
> 
> In this case, the other observer appears to be ghes_estatus_cached(), and the
> reads of the cache struct fields must be ordered after the read of the cache
> struct's address. However, there is an implicit ordering there through an address
> dependency (you cannot dereference a struct without knowing its address) so
> the ordering is implied (and
> rcu_dereference() has a READ_ONCE() inside so we are guaranteed to always
> dereference the same struct, even if the array slot gets updated concurrently.)
> 
> If you want to get rid of the barrier, you could drop it and change the cmpxchg()
> to cmpxchg_release().
> 
> Justin: so why are the RCU_INITIALIZER()s needed here?

In my this patch, I add the "__rcu" to the definition of ghes_estatus_caches. Hence
Sparse will still have the warning on X86 with this RCU_INITIALIZER cast.
drivers/acpi/apei/ghes.c:843:27: sparse: warning: incorrect type in initializer (different address spaces)
drivers/acpi/apei/ghes.c:843:27: sparse:    expected struct ghes_estatus_cache [noderef] __rcu *__old
drivers/acpi/apei/ghes.c:843:27: sparse:    got struct ghes_estatus_cache *[assigned] slot_cache
drivers/acpi/apei/ghes.c:843:27: sparse: warning: incorrect type in initializer (different address spaces)
drivers/acpi/apei/ghes.c:843:27: sparse:    expected struct ghes_estatus_cache [noderef] __rcu *__new
drivers/acpi/apei/ghes.c:843:27: sparse:    got struct ghes_estatus_cache *[assigned] new_cache

On Arm, IMO the macro cmpxchg doesn't care about it, that is, sparse will not report warnings with or
without RCU_INITIALIZER cast.

I tend to remain this cast, what do you think of it.

--
Cheers,
Justin (Jia He)