Re: [PATCH V8 03/10] PCI: Create PCI library functions in support of DOE mailboxes.

From: Ira Weiny <ira.weiny@intel.com>
To: Lukas Wunner <lukas@wunner.de>
Cc: Jonathan Cameron <Jonathan.Cameron@huawei.com>,
	Dan Williams <dan.j.williams@intel.com>,
	Bjorn Helgaas <bhelgaas@google.com>,
	Christoph Hellwig <hch@infradead.org>,
	Alison Schofield <alison.schofield@intel.com>,
	Vishal Verma <vishal.l.verma@intel.com>,
	Ben Widawsky <ben.widawsky@intel.com>,
	linux-kernel@vger.kernel.org, linux-cxl@vger.kernel.org,
	linux-pci@vger.kernel.org
Subject: Re: [PATCH V8 03/10] PCI: Create PCI library functions in support of DOE mailboxes.
Date: Wed, 1 Jun 2022 13:17:38 -0700	[thread overview]
Message-ID: <YpfJYujerG2hM4vU@iweiny-desk3> (raw)
In-Reply-To: <20220601175647.GA21509@wunner.de>

On Wed, Jun 01, 2022 at 07:56:47PM +0200, Lukas Wunner wrote:
> On Wed, Jun 01, 2022 at 10:16:15AM -0700, Ira Weiny wrote:
> > On Wed, Jun 01, 2022 at 09:18:08AM +0200, Lukas Wunner wrote:
> > > You only need to re-check the Data Object Ready bit on the last-but-one
> > > dword in case the function was reset concurrently.  Per sec. 6.30.2,
> > > "An FLR to a Function must result in the aborting of any DOE transfer
> > > in progress."
> > 
> > I think I disagree.  Even if we do that and an FLR comes before the last read
> > the last read could be 0's.
> 
> PCIe r6.0, Table 7-316 says:
> 
>   "If there is no additional data object ready for transfer, the
>    DOE instance must clear this bit after the entire data object has been
>    transferred, as indicated by software writing to the DOE Read Data
>    Mailbox Register after reading the final DW of the data object."
> 
> Remember that you *read* a dword from the mailbox and then acknowledge
> reception to the mailbox by *writing* a dword to the mailbox.
> 
> So you check that the Data Object Ready bit is set before acknowledging
> the final dword with a register write.  That's race-free.

Ok.

> 
> (I realize me talking about the "last-but-one dword" above was quite
> unclear, sorry about that.)
> 

Ah yes.  Ok, I'll put in a check before the final write.

Ira