Re: [PATCH v5 18/40] cxl/pmem: Introduce a find_cxl_root() helper

[Jonathan Cameron <Jonathan.Cameron@Huawei.com>]

On Mon, 31 Jan 2022 16:22:35 -0800
Dan Williams <dan.j.williams@intel.com> wrote:

> On Mon, Jan 31, 2022 at 8:20 AM Jonathan Cameron
> <Jonathan.Cameron@huawei.com> wrote:
> >
> > On Wed, 26 Jan 2022 15:59:07 -0800
> > Dan Williams <dan.j.williams@intel.com> wrote:
> >  
> > > In preparation for switch port enumeration while also preserving the
> > > potential for multi-domain / multi-root CXL topologies. Introduce a
> > > 'struct device' generic mechanism for retrieving a root CXL port, if one
> > > is registered. Note that the only know multi-domain CXL configurations
> > > are running the cxl_test unit test on a system that also publishes an
> > > ACPI0017 device.
> > >
> > > With this in hand the nvdimm-bridge lookup can be with
> > > device_find_child() instead of bus_find_device() + custom mocked lookup
> > > infrastructure in cxl_test.
> > >
> > > The mechanism looks for a 2nd level port since the root level topology
> > > is platform-firmware specific and the 2nd level down follows standard
> > > PCIe topology expectations. The cxl_acpi 2nd level is associated with a
> > > PCIe Root Port.
> > >
> > > Reported-by: Ben Widawsky <ben.widawsky@intel.com>
> > > Signed-off-by: Dan Williams <dan.j.williams@intel.com>  
> > A question inline.
> >
> > Thanks,
> >
> > Jonathan
> >  
> > > ---
> > > Changes since v4:
> > > - reset @iter each loop otherwise only the first dport can be scanned.
> > >
> > >  drivers/cxl/core/pmem.c       |   14 ++++++++---
> > >  drivers/cxl/core/port.c       |   50 +++++++++++++++++++++++++++++++++++++++++
> > >  drivers/cxl/cxl.h             |    1 +
> > >  tools/testing/cxl/Kbuild      |    2 --
> > >  tools/testing/cxl/mock_pmem.c |   24 --------------------
> > >  5 files changed, 61 insertions(+), 30 deletions(-)
> > >  delete mode 100644 tools/testing/cxl/mock_pmem.c
> > >
> > > diff --git a/drivers/cxl/core/pmem.c b/drivers/cxl/core/pmem.c
> > > index 40b3f5030496..8de240c4d96b 100644
> > > --- a/drivers/cxl/core/pmem.c
> > > +++ b/drivers/cxl/core/pmem.c
> > > @@ -57,24 +57,30 @@ bool is_cxl_nvdimm_bridge(struct device *dev)
> > >  }
> > >  EXPORT_SYMBOL_NS_GPL(is_cxl_nvdimm_bridge, CXL);
> > >
> > > -__mock int match_nvdimm_bridge(struct device *dev, const void *data)
> > > +static int match_nvdimm_bridge(struct device *dev, void *data)
> > >  {
> > >       return is_cxl_nvdimm_bridge(dev);
> > >  }
> > >
> > >  struct cxl_nvdimm_bridge *cxl_find_nvdimm_bridge(struct cxl_nvdimm *cxl_nvd)
> > >  {
> > > +     struct cxl_port *port = find_cxl_root(&cxl_nvd->dev);
> > >       struct device *dev;
> > >
> > > -     dev = bus_find_device(&cxl_bus_type, NULL, cxl_nvd, match_nvdimm_bridge);
> > > +     if (!port)
> > > +             return NULL;
> > > +
> > > +     dev = device_find_child(&port->dev, NULL, match_nvdimm_bridge);
> > > +     put_device(&port->dev);
> > > +
> > >       if (!dev)
> > >               return NULL;
> > > +
> > >       return to_cxl_nvdimm_bridge(dev);
> > >  }
> > >  EXPORT_SYMBOL_NS_GPL(cxl_find_nvdimm_bridge, CXL);
> > >
> > > -static struct cxl_nvdimm_bridge *
> > > -cxl_nvdimm_bridge_alloc(struct cxl_port *port)
> > > +static struct cxl_nvdimm_bridge *cxl_nvdimm_bridge_alloc(struct cxl_port *port)
> > >  {
> > >       struct cxl_nvdimm_bridge *cxl_nvb;
> > >       struct device *dev;
> > > diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c
> > > index 4c921c49f967..6447f12ef71d 100644
> > > --- a/drivers/cxl/core/port.c
> > > +++ b/drivers/cxl/core/port.c
> > > @@ -457,6 +457,56 @@ int devm_cxl_register_pci_bus(struct device *host, struct device *uport,
> > >  }
> > >  EXPORT_SYMBOL_NS_GPL(devm_cxl_register_pci_bus, CXL);
> > >
> > > +/* Find a 2nd level CXL port that has a dport that is an ancestor of @match */
> > > +static int match_cxl_root_child(struct device *dev, const void *match)
> > > +{
> > > +     const struct device *iter = NULL;
> > > +     struct cxl_port *port, *parent;
> > > +     struct cxl_dport *dport;
> > > +
> > > +     if (!is_cxl_port(dev))
> > > +             return 0;
> > > +
> > > +     port = to_cxl_port(dev);
> > > +     if (is_cxl_root(port))
> > > +             return 0;
> > > +
> > > +     parent = to_cxl_port(port->dev.parent);
> > > +     if (!is_cxl_root(parent))
> > > +             return 0;
> > > +
> > > +     cxl_device_lock(&port->dev);
> > > +     list_for_each_entry(dport, &port->dports, list) {
> > > +             iter = match;  
> >
> > This confuses me.  In the call below to bus_find_device()
> > data == NULL, which ends up as match here.  
> 
> I think you misread, @start is NULL @data becomes @match as the
> starting point for the search.

oops. I read the wrong parameter order as you identified.


> 
> >
> > So how does that ever find a match?
> >  
> > > +             while (iter) {
> > > +                     if (iter == dport->dport)
> > > +                             goto out;
> > > +                     iter = iter->parent;
> > > +             }
> > > +     }
> > > +out:
> > > +     cxl_device_unlock(&port->dev);
> > > +
> > > +     return !!iter;  
> >
> > return iter; should be sufficient as docs just say non zero for a match
> > in bus_find_device() match functions.  
> 
> drivers/cxl/core/port.c:488:16: error: returning ‘const struct device
> *’ from a function with return type ‘int’ makes integer from pointer
> without a cast [-Werror=int-conversion]
Ah. Good point.

Jonathan