From: Jonathan Cameron <Jonathan.Cameron@Huawei.com>
To: Dan Williams <dan.j.williams@intel.com>
Cc: <linux-cxl@vger.kernel.org>,
Ben Widawsky <ben.widawsky@intel.com>, <vishal.l.verma@intel.com>,
<nvdimm@lists.linux.dev>, <alison.schofield@intel.com>,
<ira.weiny@intel.com>
Subject: Re: [PATCH v4 09/21] cxl/mbox: Introduce the mbox_send operation
Date: Fri, 10 Sep 2021 09:58:48 +0100 [thread overview]
Message-ID: <20210910095848.0000687b@Huawei.com> (raw)
In-Reply-To: <163116434098.2460985.9004760022659400540.stgit@dwillia2-desk3.amr.corp.intel.com>
On Wed, 8 Sep 2021 22:12:21 -0700
Dan Williams <dan.j.williams@intel.com> wrote:
> In preparation for implementing a unit test backend transport for ioctl
> operations, and making the mailbox available to the cxl/pmem
> infrastructure, move the existing PCI specific portion of mailbox handling
> to an "mbox_send" operation.
>
> With this split all the PCI-specific transport details are comprehended
> by a single operation and the rest of the mailbox infrastructure is
> 'struct cxl_mem' and 'struct cxl_memdev' generic.
>
> Acked-by: Ben Widawsky <ben.widawsky@intel.com>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Looks good. Not sure why I didn't give a provisional tag for this one
in v4 given the trivial nature of my only review comment.
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> ---
> drivers/cxl/cxlmem.h | 42 ++++++++++++++++++++++++++++
> drivers/cxl/pci.c | 76 ++++++++++++++------------------------------------
> 2 files changed, 63 insertions(+), 55 deletions(-)
>
> diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h
> index c6fce966084a..9be5e26c5b48 100644
> --- a/drivers/cxl/cxlmem.h
> +++ b/drivers/cxl/cxlmem.h
> @@ -66,6 +66,45 @@ struct cxl_memdev *
> devm_cxl_add_memdev(struct cxl_mem *cxlm,
> const struct cdevm_file_operations *cdevm_fops);
>
> +/**
> + * struct cxl_mbox_cmd - A command to be submitted to hardware.
> + * @opcode: (input) The command set and command submitted to hardware.
> + * @payload_in: (input) Pointer to the input payload.
> + * @payload_out: (output) Pointer to the output payload. Must be allocated by
> + * the caller.
> + * @size_in: (input) Number of bytes to load from @payload_in.
> + * @size_out: (input) Max number of bytes loaded into @payload_out.
> + * (output) Number of bytes generated by the device. For fixed size
> + * outputs commands this is always expected to be deterministic. For
> + * variable sized output commands, it tells the exact number of bytes
> + * written.
> + * @return_code: (output) Error code returned from hardware.
> + *
> + * This is the primary mechanism used to send commands to the hardware.
> + * All the fields except @payload_* correspond exactly to the fields described in
> + * Command Register section of the CXL 2.0 8.2.8.4.5. @payload_in and
> + * @payload_out are written to, and read from the Command Payload Registers
> + * defined in CXL 2.0 8.2.8.4.8.
> + */
> +struct cxl_mbox_cmd {
> + u16 opcode;
> + void *payload_in;
> + void *payload_out;
> + size_t size_in;
> + size_t size_out;
> + u16 return_code;
> +#define CXL_MBOX_SUCCESS 0
> +};
> +
> +/*
> + * CXL 2.0 - Memory capacity multiplier
> + * See Section 8.2.9.5
> + *
> + * Volatile, Persistent, and Partition capacities are specified to be in
> + * multiples of 256MB - define a multiplier to convert to/from bytes.
> + */
> +#define CXL_CAPACITY_MULTIPLIER SZ_256M
> +
> /**
> * struct cxl_mem - A CXL memory device
> * @dev: The device associated with this CXL device.
> @@ -88,6 +127,7 @@ devm_cxl_add_memdev(struct cxl_mem *cxlm,
> * @active_persistent_bytes: sum of hard + soft persistent
> * @next_volatile_bytes: volatile capacity change pending device reset
> * @next_persistent_bytes: persistent capacity change pending device reset
> + * @mbox_send: @dev specific transport for transmitting mailbox commands
> *
> * See section 8.2.9.5.2 Capacity Configuration and Label Storage for
> * details on capacity parameters.
> @@ -115,5 +155,7 @@ struct cxl_mem {
> u64 active_persistent_bytes;
> u64 next_volatile_bytes;
> u64 next_persistent_bytes;
> +
> + int (*mbox_send)(struct cxl_mem *cxlm, struct cxl_mbox_cmd *cmd);
> };
> #endif /* __CXL_MEM_H__ */
> diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c
> index 8077d907e7d3..e2f27671c6b2 100644
> --- a/drivers/cxl/pci.c
> +++ b/drivers/cxl/pci.c
> @@ -64,45 +64,6 @@ enum opcode {
> CXL_MBOX_OP_MAX = 0x10000
> };
>
> -/*
> - * CXL 2.0 - Memory capacity multiplier
> - * See Section 8.2.9.5
> - *
> - * Volatile, Persistent, and Partition capacities are specified to be in
> - * multiples of 256MB - define a multiplier to convert to/from bytes.
> - */
> -#define CXL_CAPACITY_MULTIPLIER SZ_256M
> -
> -/**
> - * struct mbox_cmd - A command to be submitted to hardware.
> - * @opcode: (input) The command set and command submitted to hardware.
> - * @payload_in: (input) Pointer to the input payload.
> - * @payload_out: (output) Pointer to the output payload. Must be allocated by
> - * the caller.
> - * @size_in: (input) Number of bytes to load from @payload_in.
> - * @size_out: (input) Max number of bytes loaded into @payload_out.
> - * (output) Number of bytes generated by the device. For fixed size
> - * outputs commands this is always expected to be deterministic. For
> - * variable sized output commands, it tells the exact number of bytes
> - * written.
> - * @return_code: (output) Error code returned from hardware.
> - *
> - * This is the primary mechanism used to send commands to the hardware.
> - * All the fields except @payload_* correspond exactly to the fields described in
> - * Command Register section of the CXL 2.0 8.2.8.4.5. @payload_in and
> - * @payload_out are written to, and read from the Command Payload Registers
> - * defined in CXL 2.0 8.2.8.4.8.
> - */
> -struct mbox_cmd {
> - u16 opcode;
> - void *payload_in;
> - void *payload_out;
> - size_t size_in;
> - size_t size_out;
> - u16 return_code;
> -#define CXL_MBOX_SUCCESS 0
> -};
> -
> static DECLARE_RWSEM(cxl_memdev_rwsem);
> static struct dentry *cxl_debugfs;
> static bool cxl_raw_allow_all;
> @@ -266,7 +227,7 @@ static bool cxl_is_security_command(u16 opcode)
> }
>
> static void cxl_mem_mbox_timeout(struct cxl_mem *cxlm,
> - struct mbox_cmd *mbox_cmd)
> + struct cxl_mbox_cmd *mbox_cmd)
> {
> struct device *dev = cxlm->dev;
>
> @@ -297,7 +258,7 @@ static void cxl_mem_mbox_timeout(struct cxl_mem *cxlm,
> * mailbox.
> */
> static int __cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm,
> - struct mbox_cmd *mbox_cmd)
> + struct cxl_mbox_cmd *mbox_cmd)
> {
> void __iomem *payload = cxlm->regs.mbox + CXLDEV_MBOX_PAYLOAD_OFFSET;
> struct device *dev = cxlm->dev;
> @@ -472,6 +433,20 @@ static void cxl_mem_mbox_put(struct cxl_mem *cxlm)
> mutex_unlock(&cxlm->mbox_mutex);
> }
>
> +static int cxl_pci_mbox_send(struct cxl_mem *cxlm, struct cxl_mbox_cmd *cmd)
> +{
> + int rc;
> +
> + rc = cxl_mem_mbox_get(cxlm);
> + if (rc)
> + return rc;
> +
> + rc = __cxl_mem_mbox_send_cmd(cxlm, cmd);
> + cxl_mem_mbox_put(cxlm);
> +
> + return rc;
> +}
> +
> /**
> * handle_mailbox_cmd_from_user() - Dispatch a mailbox command for userspace.
> * @cxlm: The CXL memory device to communicate with.
> @@ -503,7 +478,7 @@ static int handle_mailbox_cmd_from_user(struct cxl_mem *cxlm,
> s32 *size_out, u32 *retval)
> {
> struct device *dev = cxlm->dev;
> - struct mbox_cmd mbox_cmd = {
> + struct cxl_mbox_cmd mbox_cmd = {
> .opcode = cmd->opcode,
> .size_in = cmd->info.size_in,
> .size_out = cmd->info.size_out,
> @@ -525,10 +500,6 @@ static int handle_mailbox_cmd_from_user(struct cxl_mem *cxlm,
> }
> }
>
> - rc = cxl_mem_mbox_get(cxlm);
> - if (rc)
> - goto out;
> -
> dev_dbg(dev,
> "Submitting %s command for user\n"
> "\topcode: %x\n"
> @@ -539,8 +510,7 @@ static int handle_mailbox_cmd_from_user(struct cxl_mem *cxlm,
> dev_WARN_ONCE(dev, cmd->info.id == CXL_MEM_COMMAND_ID_RAW,
> "raw command path used\n");
>
> - rc = __cxl_mem_mbox_send_cmd(cxlm, &mbox_cmd);
> - cxl_mem_mbox_put(cxlm);
> + rc = cxlm->mbox_send(cxlm, &mbox_cmd);
> if (rc)
> goto out;
>
> @@ -874,7 +844,7 @@ static int cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm, u16 opcode,
> void *out, size_t out_size)
> {
> const struct cxl_mem_command *cmd = cxl_mem_find_command(opcode);
> - struct mbox_cmd mbox_cmd = {
> + struct cxl_mbox_cmd mbox_cmd = {
> .opcode = opcode,
> .payload_in = in,
> .size_in = in_size,
> @@ -886,12 +856,7 @@ static int cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm, u16 opcode,
> if (out_size > cxlm->payload_size)
> return -E2BIG;
>
> - rc = cxl_mem_mbox_get(cxlm);
> - if (rc)
> - return rc;
> -
> - rc = __cxl_mem_mbox_send_cmd(cxlm, &mbox_cmd);
> - cxl_mem_mbox_put(cxlm);
> + rc = cxlm->mbox_send(cxlm, &mbox_cmd);
> if (rc)
> return rc;
>
> @@ -913,6 +878,7 @@ static int cxl_mem_setup_mailbox(struct cxl_mem *cxlm)
> {
> const int cap = readl(cxlm->regs.mbox + CXLDEV_MBOX_CAPS_OFFSET);
>
> + cxlm->mbox_send = cxl_pci_mbox_send;
> cxlm->payload_size =
> 1 << FIELD_GET(CXLDEV_MBOX_CAP_PAYLOAD_SIZE_MASK, cap);
>
>
next prev parent reply other threads:[~2021-09-10 8:58 UTC|newest]
Thread overview: 78+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-09 5:11 [PATCH v4 00/21] cxl_test: Enable CXL Topology and UAPI regression tests Dan Williams
2021-09-09 5:11 ` [PATCH v4 01/21] libnvdimm/labels: Add uuid helpers Dan Williams
2021-09-09 5:11 ` [PATCH v4 02/21] libnvdimm/label: Add a helper for nlabel validation Dan Williams
2021-09-09 5:11 ` [PATCH v4 03/21] libnvdimm/labels: Introduce the concept of multi-range namespace labels Dan Williams
2021-09-09 13:09 ` Jonathan Cameron
2021-09-09 15:16 ` Dan Williams
2021-09-09 5:11 ` [PATCH v4 04/21] libnvdimm/labels: Fix kernel-doc for label.h Dan Williams
2021-09-10 8:38 ` Jonathan Cameron
2021-09-09 5:11 ` [PATCH v4 05/21] libnvdimm/label: Define CXL region labels Dan Williams
2021-09-09 15:58 ` Ben Widawsky
2021-09-09 18:38 ` Dan Williams
2021-09-09 5:12 ` [PATCH v4 06/21] libnvdimm/labels: Introduce CXL labels Dan Williams
2021-09-09 5:12 ` [PATCH v4 07/21] cxl/pci: Make 'struct cxl_mem' device type generic Dan Williams
2021-09-09 16:12 ` Ben Widawsky
2021-09-10 8:43 ` Jonathan Cameron
2021-09-09 5:12 ` [PATCH v4 08/21] cxl/pci: Clean up cxl_mem_get_partition_info() Dan Williams
2021-09-09 16:20 ` Ben Widawsky
2021-09-09 18:06 ` Dan Williams
2021-09-09 21:05 ` Ben Widawsky
2021-09-09 21:10 ` Dan Williams
2021-09-10 8:56 ` Jonathan Cameron
2021-09-13 22:19 ` [PATCH v5 " Dan Williams
2021-09-13 22:21 ` Dan Williams
2021-09-13 22:24 ` [PATCH v6 " Dan Williams
2021-09-09 5:12 ` [PATCH v4 09/21] cxl/mbox: Introduce the mbox_send operation Dan Williams
2021-09-09 16:34 ` Ben Widawsky
2021-09-10 8:58 ` Jonathan Cameron [this message]
2021-09-09 5:12 ` [PATCH v4 10/21] cxl/pci: Drop idr.h Dan Williams
2021-09-09 16:34 ` Ben Widawsky
2021-09-10 8:46 ` Jonathan Cameron
2021-09-09 5:12 ` [PATCH v4 11/21] cxl/mbox: Move mailbox and other non-PCI specific infrastructure to the core Dan Williams
2021-09-09 16:41 ` Ben Widawsky
2021-09-09 18:50 ` Dan Williams
2021-09-09 20:35 ` Ben Widawsky
2021-09-09 21:05 ` Dan Williams
2021-09-10 9:13 ` Jonathan Cameron
2021-09-09 5:12 ` [PATCH v4 12/21] cxl/pci: Use module_pci_driver Dan Williams
2021-09-09 5:12 ` [PATCH v4 13/21] cxl/mbox: Convert 'enabled_cmds' to DECLARE_BITMAP Dan Williams
2021-09-09 5:12 ` [PATCH v4 14/21] cxl/mbox: Add exclusive kernel command support Dan Williams
2021-09-09 17:02 ` Ben Widawsky
2021-09-10 9:33 ` Jonathan Cameron
2021-09-13 23:46 ` Dan Williams
2021-09-14 9:01 ` Jonathan Cameron
2021-09-14 12:22 ` Konstantin Ryabitsev
2021-09-14 14:39 ` Dan Williams
2021-09-14 15:51 ` Konstantin Ryabitsev
2021-09-14 19:03 ` [PATCH v5 " Dan Williams
2021-09-09 5:12 ` [PATCH v4 15/21] cxl/pmem: Translate NVDIMM label commands to CXL label commands Dan Williams
2021-09-09 17:22 ` Ben Widawsky
2021-09-09 19:03 ` Dan Williams
2021-09-09 20:32 ` Ben Widawsky
2021-09-10 9:39 ` Jonathan Cameron
2021-09-09 22:08 ` [PATCH v5 " Dan Williams
2021-09-10 9:40 ` Jonathan Cameron
2021-09-14 19:06 ` Dan Williams
2021-09-09 5:12 ` [PATCH v4 16/21] cxl/pmem: Add support for multiple nvdimm-bridge objects Dan Williams
2021-09-09 22:03 ` Dan Williams
2021-09-14 19:08 ` [PATCH v5 " Dan Williams
2021-09-09 5:13 ` [PATCH v4 17/21] tools/testing/cxl: Introduce a mocked-up CXL port hierarchy Dan Williams
2021-09-10 9:53 ` Jonathan Cameron
2021-09-10 18:46 ` Dan Williams
2021-09-14 19:14 ` [PATCH v5 " Dan Williams
2021-09-09 5:13 ` [PATCH v4 18/21] cxl/bus: Populate the target list at decoder create Dan Williams
2021-09-10 9:57 ` Jonathan Cameron
2021-09-09 5:13 ` [PATCH v4 19/21] cxl/mbox: Move command definitions to common location Dan Williams
2021-09-09 5:13 ` [PATCH v4 20/21] tools/testing/cxl: Introduce a mock memory device + driver Dan Williams
2021-09-10 10:09 ` Jonathan Cameron
2021-09-09 5:13 ` [PATCH v4 21/21] cxl/core: Split decoder setup into alloc + add Dan Williams
2021-09-10 10:33 ` Jonathan Cameron
2021-09-10 18:36 ` Dan Williams
2021-09-11 17:15 ` Ben Widawsky
2021-09-11 20:20 ` Dan Williams
2021-09-14 19:31 ` [PATCH v5 " Dan Williams
2021-09-21 14:24 ` Ben Widawsky
2021-09-21 16:18 ` Dan Williams
2021-09-21 19:22 ` [PATCH v6 " Dan Williams
2021-12-10 19:38 ` Nathan Chancellor
2021-12-10 19:41 ` Dan Williams
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210910095848.0000687b@Huawei.com \
--to=jonathan.cameron@huawei.com \
--cc=alison.schofield@intel.com \
--cc=ben.widawsky@intel.com \
--cc=dan.j.williams@intel.com \
--cc=ira.weiny@intel.com \
--cc=linux-cxl@vger.kernel.org \
--cc=nvdimm@lists.linux.dev \
--cc=vishal.l.verma@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).