Re: [RFC PATCH v3 2/4] PCI/doe: Add Data Object Exchange support

[Jonathan Cameron <Jonathan.Cameron@Huawei.com>]

On Wed, 19 May 2021 12:20:17 -0700
Dan Williams <dan.j.williams@intel.com> wrote:

> On Wed, May 19, 2021 at 10:03 AM Jonathan Cameron
> <Jonathan.Cameron@huawei.com> wrote:
> [..]
> > > > "The DOE Busy bit can be used to indicate that the DOE responder is
> > > >  temporarily unable to accept a data object. It is necessary for a
> > > >  DOE requester to ensure that individual data object transfers are
> > > >  completed, and that a request/response contract is completed, for
> > > >  example using a mutex mechanism to block other conflicting traffic
> > > >  for cases where such conflicts are possible."  
> > >
> > > I read that as the specification mandating my proposal to disallow
> > > multi-initiator access. My only mistake was making the exclusion apply
> > > to reads and not limiting it to the minimum of config write exclusion.  
> >
> > Key thing is even that isn't enough.   The mutex isn't about stopping
> > temporary access, it's about ensuring "request/response contract is completed".
> > So you would need userspace to be able to take a lock to stop the kernel
> > from using the DOE whilst it completes it's request/response pair and
> > userspace to guarantee it doesn't do anything stupid.  
> 
> A userspace lockout of the kernel is not needed if userspace is
> outright forbidden from corrupting the kernel's state machine. I.e.
> kernel enforced full disable of user initiated config-write to DOE
> registers, not the ephemeral pci_cfg_access_lock() proposal.

That would work but I thought was ruled out as an approach.
@Bjorn would this be acceptable?

> 
> > Easiest way to do that is provide proper interfaces that allows the
> > kernel to fully mediate the access + don't support direct userspace access
> > for normal operation. (treat it the same as an other config space write)  
> 
> Again, it's the parenthetical at issue. I struggle to see this as just
> another errant / unwanted config-write when there is legitimate reason
> for userspace to expect that touching the DOE is not destructive to
> device operation as opposed to writes to other critical registers.

True for specific protocols (CDAT). I'm fairly sure, with IDE you can take down
the link encryption to the device, potentially (worst case?) resulting a memory
access failure and a machine reboot or corruption of persistent memory.

> Where the kernel's legitimate-access and userspace's legitimate-access
> to a resource collide, the kernel provides a mediation interface that
> precludes conflicts. Otherwise, I don't understand why the kernel is
> going through the trouble of /dev/mem and pci-mmap restrictions if it
> is not supposed to be concerned about userspace corrupting driver
> state.

The short answer is that lock requirement, in the above note, rules
out safe direct userspace use of the DOE (unless we can tell the kernel
is not going to ever use it).  Mediation must be done. Even if we safely
protect the kernel side via aborts, userspace transactions can be
interrupted in a fashion that is invisible to userspace (beyond maybe
a timeout if the userspace code is hardened against this). So there is no
legitimate use that is not fully mediated by the kernel. So ioctl
or defined per protocol interfaces are the way forwards.

Perhaps that's putting it rather strongly :)

Jonathan