Re: [PATCH] cxl: Move register block enumeration to core

[Dan Williams <dan.j.williams@intel.com>]

On Tue, Sep 21, 2021 at 9:44 AM Ben Widawsky <ben.widawsky@intel.com> wrote:
>
> On 21-09-21 07:07:13, Dan Williams wrote:
> > On Mon, Sep 20, 2021 at 3:57 PM Ben Widawsky <ben.widawsky@intel.com> wrote:
> > >
> > > CXL drivers or cxl_core itself will require the ability to map component
> > > registers in order to map HDM decoder resources amongst other things.
> > > Much of the register mapping code has already moved into cxl_core. The
> > > code to pull the BAR number and block office remained within cxl_pci
>
> s/office/offset - before anyone else notices...
>
> > > because there was no need to move it. Upcoming work will require this
> > > functionality to be available outside of cxl_pci.
> > >
> > > There are two intentional functional changes:
> > > 1. cxl_pci: If there is more than 1 component, or device register block,
> > >    only the first one (of each) is checked. Previous logic checked all
> > >    duplicate register blocks and additionally attempted to map unused
> > >    register blocks if present.
> > > 2. cxl_pci: No more dev_dbg for unused register blocks
> >
> > Why not break these out into separate patches before moving the code?
> > It makes it easier to review, and it increases the precision of future
> > Fixes: patches if necessary.
> >
>
> I can. Indeed it was my instinct to do so. I went against my instinct...
>
> What are you thinking (something like...)?
> 1. Change register locator identifiers to enum
> 2. refactor cxl_pci to use the new find() function.

In order to ease the coordination pressure perhaps you could define a
__weak copy of this helper in the CXL core with a comment of:

/* TODO: Delete once this same function is available from the PCI core */

...and then separately send the refactor series to all the stakeholders.

> 3. Remove map.type

Inject a patch for:

"No more dev_dbg() for unused register blocks"

...here?

> 4. move required functionality to core
>
> > >
> > > Cc: Ira Weiny <ira.weiny@intel.com>
> > > Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
> > > ---
> > >  drivers/cxl/core/regs.c |  89 ++++++++++++++++++++++++++++++++++
> > >  drivers/cxl/cxl.h       |   3 ++
> > >  drivers/cxl/pci.c       | 104 +++++++---------------------------------
> > >  drivers/cxl/pci.h       |  14 +++---
> > >  4 files changed, 116 insertions(+), 94 deletions(-)
> > >
> > > diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c
> > > index 41de4a136ecd..ef6164ef449f 100644
> > > --- a/drivers/cxl/core/regs.c
> > > +++ b/drivers/cxl/core/regs.c
> > > @@ -5,6 +5,7 @@
> > >  #include <linux/slab.h>
> > >  #include <linux/pci.h>
> > >  #include <cxlmem.h>
> > > +#include <pci.h>
> > >
> > >  /**
> > >   * DOC: cxl registers
> > > @@ -247,3 +248,91 @@ int cxl_map_device_regs(struct pci_dev *pdev,
> > >         return 0;
> > >  }
> > >  EXPORT_SYMBOL_GPL(cxl_map_device_regs);
> > > +
> > > +static void cxl_decode_register_block(u32 reg_lo, u32 reg_hi, u8 *bar,
> > > +                                     u64 *offset, u8 *reg_type)
> > > +{
> > > +       *offset = ((u64)reg_hi << 32) | (reg_lo & CXL_REGLOC_ADDR_MASK);
> > > +       *bar = FIELD_GET(CXL_REGLOC_BIR_MASK, reg_lo);
> > > +       *reg_type = FIELD_GET(CXL_REGLOC_RBI_MASK, reg_lo);
> > > +}
> > > +
> > > +static int cxl_pci_dvsec(struct pci_dev *pdev, int dvsec)
> > > +{
> > > +       int pos;
> > > +
> > > +       pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_DVSEC);
> > > +       if (!pos)
> > > +               return 0;
> > > +
> > > +       while (pos) {
> > > +               u16 vendor, id;
> > > +
> > > +               pci_read_config_word(pdev, pos + PCI_DVSEC_HEADER1, &vendor);
> > > +               pci_read_config_word(pdev, pos + PCI_DVSEC_HEADER2, &id);
> > > +               if (vendor == PCI_DVSEC_VENDOR_ID_CXL && dvsec == id)
> > > +                       return pos;
> > > +
> > > +               pos = pci_find_next_ext_capability(pdev, pos,
> > > +                                                  PCI_EXT_CAP_ID_DVSEC);
> > > +       }
> >
> > We punted on refactoring this for the initial driver submission
> > because it was difficult to coordinate. Now that cxl.git is an
> > established tree, instead of moving this it seems time to address that
> > refactor that Bjorn asked about. Bjorn, would you be willing to carry
> > a non-rebasing branch with such a cleanup that CXL could pull from?
> >
>
> Also here:
> https://lore.kernel.org/linux-pci/20210913190131.xiiszmno46qie7v5@intel.com/

Looks good, although I notice that find_dvsec_from_pos() from
arch/powerpc/platforms/powernv/ocxl.c wants to start searching for the
dvsec starting from an initial offset.

>
> > > +
> > > +       return 0;
> > > +}
> > > +
> > > +/**
> > > + * cxl_get_register_block() - Find the CXL register block by identifier
> > > + * @pdev: The PCI device implementing the registers
> > > + * @type: Type of register block to find
> > > + * @map: (Output) parameters used to map the regiseter block
> >
> > s/regiseter/register/
> >
> > > + *
> > > + * If register block is found, 0 is returned and @map is populated; else returns
> > > + * negative error code.
> > > + */
> > > +int cxl_get_register_block(struct pci_dev *pdev, enum cxl_regloc_type type,
> >
> > Seeing this broken out again it looks like a 'find' operation rather
> > than a 'get', but not a big deal.
> >
>
> I'm okay to rename it. It also makes me realize map.type becomes a useless
> field.

True.