From: Ben Widawsky <ben.widawsky@intel.com>
To: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org,
Ira Weiny <ira.weiny@intel.com>,
Dan Williams <dan.j.williams@intel.com>,
Vishal Verma <vishal.l.verma@intel.com>,
"Kelley, Sean V" <sean.v.kelley@intel.com>,
Rafael Wysocki <rafael.j.wysocki@intel.com>,
Bjorn Helgaas <helgaas@kernel.org>,
Jon Masters <jcm@jonmasters.org>,
Chris Browy <cbrowy@avery-design.com>,
Randy Dunlap <rdunlap@infradead.org>,
Christoph Hellwig <hch@infradead.org>,
daniel.lll@alibaba-inc.com
Subject: Re: [RFC PATCH v3 13/16] cxl/mem: Create concept of enabled commands
Date: Thu, 21 Jan 2021 10:40:41 -0800 [thread overview]
Message-ID: <20210121184041.r7x7tofhu2qqneus@intel.com> (raw)
In-Reply-To: <20210114172531.0000347a@Huawei.com>
On 21-01-14 17:25:31, Jonathan Cameron wrote:
> On Mon, 11 Jan 2021 14:51:18 -0800
> Ben Widawsky <ben.widawsky@intel.com> wrote:
>
> > CXL devices must implement the Device Command Interface (described in
> > 8.2.9 of the CXL 2.0 spec). While the driver already maintains a list of
> > commands it supports, there is still a need to be able to distinguish
> > between commands that the driver knows about from commands that may not
> > be supported by the hardware. No such commands currently are defined in
> > the driver.
> >
> > The implementation leaves the statically defined table of commands and
> > supplements it with a bitmap to determine commands that are enabled.
> >
> > ---
> >
> > There are multiple approaches that can be taken, but this is nice for a
> > few reasons.
> >
> > Here are some of the other solutions:
> >
> > Create a per instance table with only the supported commands.
> > 1. Having a fixed command id -> command mapping is much easier to manage
> > for development and debugging.
> > 2. Dealing with dynamic memory allocation for the table adds unnecessary
> > complexity.
> > 3. Most tables for device types are likely to be quite similar.
> > 4. Makes it difficult to implement helper macros like cxl_for_each_cmd()
> >
> > If the per instance table did preserve ids, #1 above can be addressed.
> > However, as "enable" is currently the only mutable state for the
> > commands, it would yield a lot of overhead for not much gain.
> > Additionally, the other issues remain.
> >
> > If "enable" remains the only mutable state, I believe this to be the
> > best solution. Once the number of mutable elements in a command grows,
> > it probably makes sense to move to per device instance state with a
> > fixed command ID mapping.
> Agreed with the logic.
>
> However, patch wise, should either drop the --- above or move this below the
> --- after your sign off. Otherwise you run the risk of git dropping your
> sign off and resulting complaints from anyone run validation scripts
> of the kernel tree that check for that.
>
> >
> > Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
> > ---
> > drivers/cxl/cxl.h | 4 ++++
> > drivers/cxl/mem.c | 38 +++++++++++++++++++++++++++++++++++++-
> > 2 files changed, 41 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
> > index 537ac4d8e6bd..963ba30cb200 100644
> > --- a/drivers/cxl/cxl.h
> > +++ b/drivers/cxl/cxl.h
> > @@ -17,6 +17,9 @@
> >
> > #define CXL_GET_FIELD(word, field) FIELD_GET(field##_MASK, word)
> >
> > +/* XXX: Arbitrary max */
> > +#define CXL_MAX_COMMANDS 32
>
> If going this way, probably want a build time check that you don't
> go beyond this value for a given command set. I haven't actually
> thought about how to construct that but should be easy enough.
>
I think BUILD_BUG_ON of ARRAY_SIZE mem_commands is sufficient. Let me know if
you want anything more.
> > +
> > /* Device (CXL 2.0 - 8.2.8.3) */
> > #define CXLDEV_CAP_ARRAY_REG 0x0
> > #define CXLDEV_CAP_ARRAY_CAP_ID 0
> > @@ -88,6 +91,7 @@ struct cxl_mem {
> > } ram;
> >
> > char firmware_version[0x10];
> > + DECLARE_BITMAP(enabled_cmds, CXL_MAX_COMMANDS);
> >
> > /* Cap 0001h - CXL_CAP_CAP_ID_DEVICE_STATUS */
> > struct {
> > diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> > index a824cfd4342a..20b26fa2c466 100644
> > --- a/drivers/cxl/mem.c
> > +++ b/drivers/cxl/mem.c
> > @@ -114,6 +114,8 @@ static struct {
> > * would typically be used for deprecated commands.
> > * * %CXL_CMD_FLAG_MANDATORY: Hardware must support this command. This flag is
> > * only used internally by the driver for sanity checking.
> > + * * %CXL_CMD_INTERNAL_FLAG_PSEUDO: This is a pseudo command which doesn't have
> > + * a direct mapping to hardware. They are implicitly always enabled.
> > *
> > * The cxl_mem_command is the driver's internal representation of commands that
> > * are supported by the driver. Some of these commands may not be supported by
> > @@ -129,6 +131,7 @@ struct cxl_mem_command {
> > #define CXL_CMD_INTERNAL_FLAG_NONE 0
> > #define CXL_CMD_INTERNAL_FLAG_HIDDEN BIT(0)
> > #define CXL_CMD_INTERNAL_FLAG_MANDATORY BIT(1)
> > +#define CXL_CMD_INTERNAL_FLAG_PSEUDO BIT(2)
> > };
> >
> > /*
> > @@ -140,7 +143,7 @@ struct cxl_mem_command {
> > static struct cxl_mem_command mem_commands[] = {
> > CXL_CMD(INVALID, KERNEL, 0, 0, HIDDEN),
> > CXL_CMD(IDENTIFY, NONE, 0, 0x43, MANDATORY),
> > - CXL_CMD(RAW, NONE, ~0, ~0, MANDATORY),
> > + CXL_CMD(RAW, NONE, ~0, ~0, PSEUDO),
> > };
> >
> > #define cxl_for_each_cmd(cmd) \
> > @@ -618,6 +621,10 @@ static int cxl_validate_cmd_from_user(struct cxl_mem *cxlm,
> > c = &mem_commands[send_cmd->id];
> > info = &c->info;
> >
> > + /* Check that the command is enabled for hardware */
> > + if (!test_bit(cxl_cmd_index(c), cxlm->enabled_cmds))
> > + return -ENOTTY;
> > +
> > if (info->flags & CXL_MEM_COMMAND_FLAG_KERNEL)
> > return -EPERM;
> >
> > @@ -1029,6 +1036,31 @@ static int cxl_mem_add_memdev(struct cxl_mem *cxlm)
> > return rc;
> > }
> >
> > +/**
> > + * cxl_mem_enumerate_cmds() - Enumerate commands for a device.
> > + * @cxlm: The device.
> > + *
> > + * Returns 0 if enumerate completed successfully.
> > + *
> > + * CXL devices have optional support for certain commands. This function will
> > + * determine the set of supported commands for the hardware and update the
> > + * enabled_cmds bitmap in the @cxlm.
> > + */
> > +static int cxl_mem_enumerate_cmds(struct cxl_mem *cxlm)
> > +{
> > + struct cxl_mem_command *c;
> > +
> > + /* All commands are considered enabled for now (except INVALID). */
> > + cxl_for_each_cmd(c) {
> > + if (c->flags & CXL_CMD_INTERNAL_FLAG_HIDDEN)
> > + continue;
> > +
> > + set_bit(cxl_cmd_index(c), cxlm->enabled_cmds);
> > + }
> > +
> > + return 0;
> > +}
> > +
> > /**
> > * cxl_mem_identify() - Send the IDENTIFY command to the device.
> > * @cxlm: The device to identify.
> > @@ -1147,6 +1179,10 @@ static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> > if (rc)
> > goto err;
> >
> > + rc = cxl_mem_enumerate_cmds(cxlm);
> > + if (rc)
> > + return rc;
> > +
> > rc = cxl_mem_identify(cxlm);
> > if (rc)
> > goto err;
>
next prev parent reply other threads:[~2021-01-21 18:42 UTC|newest]
Thread overview: 61+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-11 22:51 [RFC PATCH v3 00/16] CXL 2.0 Support Ben Widawsky
2021-01-11 22:51 ` [RFC PATCH v3 01/16] docs: cxl: Add basic documentation Ben Widawsky
2021-01-11 22:51 ` [RFC PATCH v3 02/16] cxl/acpi: Add an acpi_cxl module for the CXL interconnect Ben Widawsky
2021-01-12 7:08 ` Randy Dunlap
2021-01-12 18:43 ` Jonathan Cameron
2021-01-12 19:43 ` Dan Williams
2021-01-12 22:06 ` Jonathan Cameron
2021-01-13 17:55 ` Kaneda, Erik
2021-01-20 19:27 ` Dan Williams
2021-01-20 19:18 ` Verma, Vishal L
2021-01-13 12:40 ` Rafael J. Wysocki
2021-01-20 19:21 ` Verma, Vishal L
2021-01-11 22:51 ` [RFC PATCH v3 03/16] cxl/acpi: add OSC support Ben Widawsky
2021-01-12 15:09 ` Rafael J. Wysocki
2021-01-12 18:48 ` Jonathan Cameron
2021-01-11 22:51 ` [RFC PATCH v3 04/16] cxl/mem: Introduce a driver for CXL-2.0-Type-3 endpoints Ben Widawsky
2021-01-12 7:08 ` Randy Dunlap
2021-01-12 19:01 ` Jonathan Cameron
2021-01-12 20:06 ` Dan Williams
2021-01-11 22:51 ` [RFC PATCH v3 05/16] cxl/mem: Map memory device registers Ben Widawsky
2021-01-12 19:13 ` Jonathan Cameron
2021-01-12 19:21 ` Ben Widawsky
2021-01-12 20:40 ` Dan Williams
2021-01-11 22:51 ` [RFC PATCH v3 06/16] cxl/mem: Find device capabilities Ben Widawsky
2021-01-12 19:17 ` Jonathan Cameron
2021-01-12 19:22 ` Ben Widawsky
2021-01-11 22:51 ` [RFC PATCH v3 07/16] cxl/mem: Implement polled mode mailbox Ben Widawsky
2021-01-13 18:26 ` Jonathan Cameron
2021-01-14 17:40 ` Jonathan Cameron
2021-01-14 17:50 ` Ben Widawsky
2021-01-14 18:13 ` Jonathan Cameron
2021-01-11 22:51 ` [RFC PATCH v3 08/16] cxl/mem: Register CXL memX devices Ben Widawsky
2021-01-14 16:28 ` Jonathan Cameron
2021-01-11 22:51 ` [RFC PATCH v3 09/16] cxl/mem: Add basic IOCTL interface Ben Widawsky
2021-01-14 16:19 ` Jonathan Cameron
2021-01-11 22:51 ` [RFC PATCH v3 10/16] cxl/mem: Add send command Ben Widawsky
2021-01-14 17:10 ` Jonathan Cameron
2021-01-21 18:15 ` Ben Widawsky
2021-01-22 11:43 ` Jonathan Cameron
2021-01-22 17:08 ` Ben Widawsky
2021-01-11 22:51 ` [RFC PATCH v3 11/16] taint: add taint for direct hardware access Ben Widawsky
2021-01-11 22:51 ` [RFC PATCH v3 11/16] taint: add taint for unfettered " Ben Widawsky
2021-01-12 3:31 ` Ben Widawsky
2021-01-11 22:51 ` [RFC PATCH v3 12/16] cxl/mem: Add a "RAW" send command Ben Widawsky
2021-01-11 22:51 ` [RFC PATCH v3 13/16] cxl/mem: Create concept of enabled commands Ben Widawsky
2021-01-14 17:25 ` Jonathan Cameron
2021-01-21 18:40 ` Ben Widawsky [this message]
2021-01-22 11:28 ` Jonathan Cameron
2021-01-11 22:51 ` [RFC PATCH v3 14/16] cxl/mem: Use CEL for enabling commands Ben Widawsky
2021-01-14 18:02 ` Jonathan Cameron
2021-01-14 18:13 ` Ben Widawsky
2021-01-14 18:32 ` Jonathan Cameron
2021-01-14 19:04 ` Ben Widawsky
2021-01-14 19:24 ` Jonathan Cameron
2021-01-11 22:51 ` [RFC PATCH v3 15/16] cxl/mem: Add limited Get Log command (0401h) Ben Widawsky
2021-01-14 18:08 ` Jonathan Cameron
2021-01-23 0:14 ` Ben Widawsky
2021-01-11 22:51 ` [RFC PATCH v3 16/16] MAINTAINERS: Add maintainers of the CXL driver Ben Widawsky
2021-01-12 1:12 ` Joe Perches
[not found] ` <0f2a6d62-09d8-416f-e972-3e9869c3e1a6@alibaba-inc.com>
2021-01-12 15:17 ` [RFC PATCH v3 00/16] CXL 2.0 Support Ben Widawsky
2021-01-12 16:19 ` Bjorn Helgaas
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210121184041.r7x7tofhu2qqneus@intel.com \
--to=ben.widawsky@intel.com \
--cc=Jonathan.Cameron@huawei.com \
--cc=cbrowy@avery-design.com \
--cc=dan.j.williams@intel.com \
--cc=daniel.lll@alibaba-inc.com \
--cc=hch@infradead.org \
--cc=helgaas@kernel.org \
--cc=ira.weiny@intel.com \
--cc=jcm@jonmasters.org \
--cc=linux-acpi@vger.kernel.org \
--cc=linux-cxl@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=rafael.j.wysocki@intel.com \
--cc=rdunlap@infradead.org \
--cc=sean.v.kelley@intel.com \
--cc=vishal.l.verma@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).