Re: [RFC PATCH v3 2/4] PCI/doe: Add Data Object Exchange support

[Dan Williams <dan.j.williams@intel.com>]

On Mon, May 17, 2021 at 1:42 AM Jonathan Cameron
<Jonathan.Cameron@huawei.com> wrote:
>
> On Fri, 14 May 2021 11:37:12 -0700
> Dan Williams <dan.j.williams@intel.com> wrote:
>
> > On Fri, May 14, 2021 at 1:50 AM Jonathan Cameron
> > <Jonathan.Cameron@huawei.com> wrote:
> > [..]
> > > > If it simplifies the kernel implementation to assume single
> > > > kernel-initiator then I think that's more than enough reason to block
> > > > out userspace, and/or provide userspace a method to get into the
> > > > kernel's queue for service.
> > >
> > > This last suggestion makes sense to me. Let's provide a 'right' way
> > > to access the DOE from user space. I like the idea if it being possible
> > > to run CXL compliance tests from userspace whilst the driver is loaded.
> >
> > Ah, and I like your observation that once the kernel provides a
> > "right" way to access DOE then userspace direct-access of DOE is
> > indeed a "you get to keep the pieces" event like any other unwanted
> > userspace config-write.
> >
> > > Bjorn, given this would be a generic PCI thing, any preference for what
> > > this interface might look like?   /dev/pcidoe[xxxxxx].i with ioctls similar
> > > to those for the BAR based CXL mailboxes?
> >
> > (warning, anti-ioctl bias incoming...)
>
> I feel very similar about ioctls - my immediate thought was to shove this in
> debugfs, but that feels the wrong choice if we are trying to persuade people
> to use it instead of writing code that directly accesses the config space.
>
> >
> > Hmm, DOE has an enumeration capability, could the DOE driver use a
> > scheme to have a sysfs bin_attr per discovered object type? This would
> > make it simliar to the pci-vpd sysfs interface.
>
> We can discover the protocols, but anything beyond that is protocol
> specific.  I don't think there is a enough info available by any standards
> defined method. Also part of the reason to allow a safe userspace interface
> would be to provide a generic interface for vendor protocols and things like
> CXL compliance tests where we will almost certainly never provide a more
> specific kernel interface.
>
> Whilst sysfs would work for CDAT, some protocols are challenge response rather
> than simple read back and that really doesn't fit well for sysfs model.
> If we get other protocols that are simple data read back, then I would
> advocate giving them a simple sysfs interface much like proposed for CDAT
> as it will always be simpler to use + self describing.
>
> On a lesser note it might be helpful to provide sysfs attrs for
> what protocols are supported.  The alternative is to let userspace run
> the discovery protocol. Perhaps we can do this as a later phase.
>
> >
> > Then the kernel could cache objects like CDAT that don't change
> > outside of some invalidation event.
>
> It's been a while since I last saw any conversation on sysfs bin_attrs
> but mostly I thought the feeling was pretty strongly against them for anything
> but a few niche usecases.
>
> Feels to me like it would break most of the usual rules in a way vpd does
> not (IIRC VPD is supposed to be a simple in the sense that if you write a value
> to a writable part, you will read back the same value).
>
> +CC Greg who is a fount of knowledge in this area (and regularly + correctly
> screams at the ways I try to abuse sysfs :)  Note I don't think Dan was
> suggesting implementing response / request directly, but I think that is
> all we could do given DOE protocols can be vendor specific and the standard
> discovery protocol doesn't let us know the fine grained support (what commands
> within a given protocol).

I'm not all that interested in supporting vendor defined DOE
shenanigans. There's more than enough published DOE protocols that the
kernel could limit its support to the known set. This is similar to
how ACPI DSMs are not generically supported, but when they appear in a
published specification the kernel may then grow the support. The
supported protocols could be limited to: CDAT, PCIe IDE, CXL
Compliance, etc...

Vendor specific DOE is in the same class as unfettered /dev/mem
access, first you need to disable the kernel's integrity and
confidentiality protections, and then you can do whatever you want. If
a vendor wants a DOE protocol supported in the "trusted" set they can
simply publish the specification and send the proper support patches.