Re: [RFC PATCH 0/5] Introduce memdev driver

From: Dan Williams <dan.j.williams@intel.com>
To: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Cc: Ben Widawsky <ben.widawsky@intel.com>,
	linux-cxl@vger.kernel.org,
	Alison Schofield <alison.schofield@intel.com>,
	Ira Weiny <ira.weiny@intel.com>,
	Vishal Verma <vishal.l.verma@intel.com>
Subject: Re: [RFC PATCH 0/5] Introduce memdev driver
Date: Wed, 30 Jun 2021 10:49:32 -0700	[thread overview]
Message-ID: <CAPcyv4jP05cCFjwWAeSfpURBWaGoRPxfzN0VBk5mASQgCWf3JQ@mail.gmail.com> (raw)
In-Reply-To: <20210618152721.00006b71@Huawei.com>

On Fri, Jun 18, 2021 at 7:27 AM Jonathan Cameron
<Jonathan.Cameron@huawei.com> wrote:
>
> On Thu, 17 Jun 2021 17:51:55 -0700
> Ben Widawsky <ben.widawsky@intel.com> wrote:
>
> > The concept of the memdev has existed since the initial support for CXL.io
> > landed in 5.12. Here, supported is furthered by adding a driver that is capable
> > of reporting whether or not the device is also CXL.mem capable. With this, the
> > region driver is able to consume these devices for programming interleave (or
> > x1) sets. Unlike the region driver, no explicit sysfs interaction is needed to
> > utilize this driver.
> >
> > The logic encapsulated here checks two things:
> > 1. The device itself is CXL.mem enabled.
>
> Need comments in relevant places to say checking if it is enabled,
> not capable.
>
> > 2. The device's upstream is CXL.mem enabled [1].
> >
> > What's currently missing is for the cxlmem driver to add the device as an
> > upstream port (since it has HDM decoders). I'm still working out those details.
> > HDM decoder programming still remains undone as well, and isn't pertinent to
> > this series perse.
> >
> > The patches are based on top of my region patches [2].
> >
> > The code itself is pretty rough for now, and so I'm mostly looking for feedback
> > as to whether or not the memdev driver is serving its purpose and checking what
> > needs to be checked on bind. If however you come along something glaringly bad,
> > or feel like reviewing not fully tested code (I know it builds), by all means...
>
> :)
>
> >
> > [1]: This series doesn't actually add real support for switches which would also
> > need to make the determination of CXL.mem enabling.
>
> Any plans to do the QEMU stuff needed for a switch?  I guess it's going to get
> messy if you want the hdm decoders to 'work' but should be fairly trivial to make them look
> plausible from an interfaces point of view.

It's already the case that the stack is modeling host bridges as
devtype 'cxl_decoder_switch' objects. So, I was hoping we, linux-cxl@
community, could convince ourselves that the algorithm that works for
the default 'cxl_decoder_switch' in the hierarchy can apply generally
to N layers of those. That said, it still would be nice to exercise
that case, but I don't see QEMU getting there anytime soon. I'm taking
a look at modeling a CXL hierarchy with kernel mocked resources for
this purpose.