Re: [RFC PATCH 2/2] cxl/mem: Add CDAT table reading from DOE

From: Dan Williams <dan.j.williams@intel.com>
To: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Cc: linux-cxl@vger.kernel.org, Linux PCI <linux-pci@vger.kernel.org>,
	Ben Widawsky <ben.widawsky@intel.com>,
	Bjorn Helgaas <helgaas@kernel.org>,
	Chris Browy <cbrowy@avery-design.com>,
	Linux ACPI <linux-acpi@vger.kernel.org>,
	"Schofield, Alison" <alison.schofield@intel.com>,
	Vishal L Verma <vishal.l.verma@intel.com>,
	"Weiny, Ira" <ira.weiny@intel.com>,
	Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
	Linuxarm <linuxarm@huawei.com>, Fangjian <f.fangjian@huawei.com>
Subject: Re: [RFC PATCH 2/2] cxl/mem: Add CDAT table reading from DOE
Date: Mon, 15 Mar 2021 15:00:08 -0700	[thread overview]
Message-ID: <CAPcyv4jq-KovQcEqesA=kCdzdDNtQ9y8g2aBCSXqQv7cvmABtg@mail.gmail.com> (raw)
In-Reply-To: <20210310181459.000005c7@huawei.com>

On Wed, Mar 10, 2021 at 10:15 AM Jonathan Cameron
<Jonathan.Cameron@huawei.com> wrote:
>
> On Thu, 11 Mar 2021 02:03:06 +0800
> Jonathan Cameron <Jonathan.Cameron@huawei.com> wrote:
>
> > This patch simply provides some debug print outs of the entries
> > at probe time + a sysfs binary attribute to allow dumping of the
> > whole table.
> >
> > Binary dumping is modelled on /sys/firmware/ACPI/tables/
> >
> > The ability to dump this table will be very useful for emulation of
> > real devices once they become available as QEMU CXL type 3 device
> > emulation will be able to load this file in.
> >
> > Open questions:
> > * No support here for table updates. Worth including these from the
> >   start, or leave that complexity for later?
> > * Worth logging the reported info for debug, or is the binary attribute
> >   sufficient?  Larger open question of whether to expose this info to
> >   userspace or not left for another day!
> > * Where to put the CDAT file?  Is it worth a subdirectory?
> > * What is maximum size of the SSLBIS entry - I haven't quite managed
> >   to figure that out and this is the record with largest size.
> >   We could support dynamic allocation of the record size, but it
> >   would add complexity that seems unnecessary.
> >   It would not be compliant with the specification for a type 3 memory
> >   device to report this record anyway so I'm not that worried about this
> >   for now.  It will become relevant once we have support for reading
> >   CDAT from CXL switches.
> > * cdat.h is formatted in a similar style to pci_regs.h on basis that
> >   it may well be helpful to share this header with userspace tools.
> > * Move the generic parts of this out to driver/cxl/cdat.c or leave that
> >   until we have other CXL drivers wishing to use this?
>
> Naturally I remembered another open question within 10 seconds of sending :(
>
>   * Do we want to add any sort of header to the RAW dump of CDAT to aid
>     tooling?  Whilst it looks a little like an ACPI table it doesn't have
>     a signature.
>
> My gut feeling is no, because the CDAT specification doesn't define one but
> I can see that it might be very convenient to have something that identified
> the data once it was put in a file.

I'm not yet convinced raw dumping is worth it for the same reason that
command payload logging was eliminated from the v5.12-rc1 submission.
There's not much userspace can do with the information besides debug
the kernel behavior. If the kernel assigns a numa node to target a
given CXL memory range with NUMA apis then HMEM_REPORTING should
enumerate the properties. In other words, don't expand the userspace
ABI problem, funnel users to the canonical source for such data.