Re: [PATCH v4 3/3] PCI: Add CRS handling to pci_dev_wait()

["Spassov, Stanislav" <stanspas@amazon.de>]

On Mon, 2020-03-09 at 09:19 -0700, Raj, Ashok wrote:
> On Mon, Mar 09, 2020 at 11:55:11AM -0400, Sinan Kaya wrote:
> > On 3/7/2020 12:20 PM, Stanislav Spassov wrote:
> > > +           rc = pci_dev_poll_until_not_equal(dev, PCI_VENDOR_ID, 0xffff,
> > > +                                             0x0001, reset_type, timeout,
> > > +                                             &waited, &id);
> > > +           if (rc)
> > > +                   return rc;
> > > +
> > 
> > If I remember right, this doesn't work for VF sending CRS because VF
> > always returns 0xffff for VENDOR_ID register.
> 
> Is this required by the PCIe spec? i think the only requirement is
> the 1s wait after PF has done the VF enable. See Implementation Note
> right above section 2.3.1.1 in the Base spec 5.0.
> 
> If this behavior is different for maybe a specific SRIOV device we should
> probably quirk the standard behavior?
> 
> The rules are mentioned in so many places, but looking through the
> SRIOV section's doesn't seem to specify special rules for VF's other than
> the wait time after VF enable.

PCI Express Base Specification Revision 5.0 Version 1.0 (May 22, 2019)
on pages 1139 and 1140 within section 9.3.4 PF/VF Configuration Space Header
describes:

"9.3.4.1.1 Vendor ID Register Changes (Offset 00h)
...
This field in all VFs returns FFFFh when read. VI software should return the Vendor ID value from the associated PF as the
Vendor ID value for the VF."

(and similar for the Device ID)

Right after that is an Implemention Note "Legacy PCI Probing Software" that explains:

"Returning FFFFh for Device ID and Vendor ID values allows some legacy software to ignore VFs."

So, whenever a VF is providing an actual response to a vid/did read, it will respond with a Successful Completion and the data
will be 0xFFFF. However, when the VF is not ready yet after a reset, I would expect it to be returning CRS completions just like
any other device (nothing in the spec explicitly confirms or denies this, as far as I can tell). Then, the root port has no idea
if a device that it received a CRS completion from is a PF or VF, so it has to treat them equivalently, and therefore (for CRS SV enabled)
synthesize 0x0001 for the VID.



Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879