From: Jonathan Cameron <Jonathan.Cameron@Huawei.com>
To: Ben Widawsky <ben.widawsky@intel.com>
Cc: "David Hildenbrand" <david@redhat.com>,
"Vishal Verma" <vishal.l.verma@intel.com>,
"John Groves (jgroves)" <jgroves@micron.com>,
"Chris Browy" <cbrowy@avery-design.com>,
qemu-devel@nongnu.org, linux-cxl@vger.kernel.org,
"Markus Armbruster" <armbru@redhat.com>,
"Michael S. Tsirkin" <mst@redhat.com>,
"Igor Mammedov" <imammedo@redhat.com>,
"Dan Williams" <dan.j.williams@intel.com>,
"Ira Weiny" <ira.weiny@intel.com>,
"Philippe Mathieu-Daudé" <f4bug@amsat.org>
Subject: Re: [RFC PATCH v3 03/31] hw/cxl/device: Introduce a CXL device (8.2.8)
Date: Tue, 2 Feb 2021 12:03:57 +0000 [thread overview]
Message-ID: <20210202120357.0000206a@Huawei.com> (raw)
In-Reply-To: <20210202005948.241655-4-ben.widawsky@intel.com>
On Mon, 1 Feb 2021 16:59:20 -0800
Ben Widawsky <ben.widawsky@intel.com> wrote:
> A CXL device is a type of CXL component. Conceptually, a CXL device
> would be a leaf node in a CXL topology. From an emulation perspective,
> CXL devices are the most complex and so the actual implementation is
> reserved for discrete commits.
>
> This new device type is specifically catered towards the eventual
> implementation of a Type3 CXL.mem device, 8.2.8.5 in the CXL 2.0
> specification.
>
> Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Really minor comments inline.
In the interests of avoiding giving myself a headache again
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> ---
> include/hw/cxl/cxl.h | 1 +
> include/hw/cxl/cxl_device.h | 155 ++++++++++++++++++++++++++++++++++++
> 2 files changed, 156 insertions(+)
> create mode 100644 include/hw/cxl/cxl_device.h
>
> diff --git a/include/hw/cxl/cxl.h b/include/hw/cxl/cxl.h
> index 55f6cc30a5..23f52c4cf9 100644
> --- a/include/hw/cxl/cxl.h
> +++ b/include/hw/cxl/cxl.h
> @@ -12,6 +12,7 @@
>
> #include "cxl_pci.h"
> #include "cxl_component.h"
> +#include "cxl_device.h"
>
> #endif
>
> diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
> new file mode 100644
> index 0000000000..a85f250503
> --- /dev/null
> +++ b/include/hw/cxl/cxl_device.h
> @@ -0,0 +1,155 @@
> +/*
> + * QEMU CXL Devices
> + *
> + * Copyright (c) 2020 Intel
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2. See the
> + * COPYING file in the top-level directory.
> + */
> +
> +#ifndef CXL_DEVICE_H
> +#define CXL_DEVICE_H
> +
> +#include "hw/register.h"
> +
> +/*
> + * The following is how a CXL device's MMIO space is laid out. The only
> + * requirement from the spec is that the capabilities array and the capability
> + * headers start at offset 0 and are contiguously packed. The headers themselves
> + * provide offsets to the register fields. For this emulation, registers will
> + * start at offset 0x80 (m == 0x80). No secondary mailbox is implemented which
> + * means that n = m + sizeof(mailbox registers) + sizeof(device registers).
> + *
> + * This is roughly described in 8.2.8 Figure 138 of the CXL 2.0 spec.
> + *
> + * n + PAYLOAD_SIZE_MAX +---------------------------------+
> + * | |
> + * ^ | |
> + * | | |
> + * | | |
> + * | | |
> + * | | Command Payload |
> + * | | |
> + * | | |
> + * | | |
> + * | | |
> + * | | |
> + * n +---------------------------------+
> + * ^ | |
> + * | | Device Capability Registers |
> + * | | x, mailbox, y |
> + * | | |
> + * m +---------------------------------+
> + * ^ | Device Capability Header y |
> + * | +---------------------------------+
> + * | | Device Capability Header Mailbox|
> + * | +------------- --------------------
> + * | | Device Capability Header x |
> + * | +---------------------------------+
> + * | | |
> + * | | |
> + * | | Device Cap Array[0..n] |
> + * | | |
> + * | | |
> + * | | |
> + * 0 +---------------------------------+
> + */
> +
> +#define CXL_DEVICE_CAP_HDR1_OFFSET 0x10 /* Figure 138 */
> +#define CXL_DEVICE_CAP_REG_SIZE 0x10 /* 8.2.8.2 */
> +#define CXL_DEVICE_CAPS_MAX 4 /* 8.2.8.2.1 + 8.2.8.5 */
> +
> +#define CXL_DEVICE_REGISTERS_OFFSET 0x80 /* Read comment above */
> +#define CXL_DEVICE_REGISTERS_LENGTH 0x8 /* 8.2.8.3.1 */
> +
> +#define CXL_MAILBOX_REGISTERS_OFFSET \
> + (CXL_DEVICE_REGISTERS_OFFSET + CXL_DEVICE_REGISTERS_LENGTH)
> +#define CXL_MAILBOX_REGISTERS_SIZE 0x20
Perhaps a ref to 8.2.8.4 or Figure 139 here somewhere?
Thanks for all the refs by the way. They make checking this a lot quicker!
> +#define CXL_MAILBOX_PAYLOAD_SHIFT 11
> +#define CXL_MAILBOX_MAX_PAYLOAD_SIZE (1 << CXL_MAILBOX_PAYLOAD_SHIFT)
> +#define CXL_MAILBOX_REGISTERS_LENGTH \
> + (CXL_MAILBOX_REGISTERS_SIZE + CXL_MAILBOX_MAX_PAYLOAD_SIZE)
> +
> +typedef struct cxl_device_state {
> + MemoryRegion device_registers;
> +
> + /* mmio for device capabilities array - 8.2.8.2 */
> + MemoryRegion caps;
> +
> + /* mmio for the device status registers 8.2.8.3 */
> + MemoryRegion device;
> +
> + /* mmio for the mailbox registers 8.2.8.4 */
> + MemoryRegion mailbox;
> +
> + /* memory region for persistent memory, HDM */
> + MemoryRegion *pmem;
> +
> + /* memory region for volatile memory, HDM */
> + MemoryRegion *vmem;
> +} CXLDeviceState;
> +
> +/* Initialize the register block for a device */
> +void cxl_device_register_block_init(Object *obj, CXLDeviceState *dev);
> +
> +/* Set up default values for the register block */
> +void cxl_device_register_init_common(CXLDeviceState *dev);
> +
> +/* CXL 2.0 - 8.2.8.1 */
> +REG32(CXL_DEV_CAP_ARRAY, 0) /* 48b!?!?! */
Also missing a reserved 64 bits to fill in below the device capability headers
which are offset by 0x10
> + FIELD(CXL_DEV_CAP_ARRAY, CAP_ID, 0, 16)
> + FIELD(CXL_DEV_CAP_ARRAY, CAP_VERSION, 16, 8)
> +REG32(CXL_DEV_CAP_ARRAY2, 4) /* We're going to pretend it's 64b */
> + FIELD(CXL_DEV_CAP_ARRAY2, CAP_COUNT, 0, 16)
> +
> +/*
> + * Helper macro to initialize capability headers for CXL devices.
> + *
> + * In the 8.2.8.2, this is listed as a 128b register, but in 8.2.8, it says:
> + * > No registers defined in Section 8.2.8 are larger than 64-bits wide so that
> + * > is the maximum access size allowed for these registers. If this rule is not
> + * > followed, the behavior is undefined
> + *
> + * Here we've chosen to make it 4 dwords. The spec allows any pow2 multiple
> + * access to be used for a register (2 qwords, 8 words, 128 bytes).
> + */
> +#define CXL_DEVICE_CAPABILITY_HEADER_REGISTER(n, offset) \
> + REG32(CXL_DEV_##n##_CAP_HDR0, offset) \
> + FIELD(CXL_DEV_##n##_CAP_HDR0, CAP_ID, 0, 16) \
> + FIELD(CXL_DEV_##n##_CAP_HDR0, CAP_VERSION, 16, 8) \
> + REG32(CXL_DEV_##n##_CAP_HDR1, offset + 4) \
> + FIELD(CXL_DEV_##n##_CAP_HDR1, CAP_OFFSET, 0, 32) \
> + REG32(CXL_DEV_##n##_CAP_HDR2, offset + 8) \
> + FIELD(CXL_DEV_##n##_CAP_HDR2, CAP_LENGTH, 0, 32)
> +
> +CXL_DEVICE_CAPABILITY_HEADER_REGISTER(DEVICE, CXL_DEVICE_CAP_HDR1_OFFSET)
> +CXL_DEVICE_CAPABILITY_HEADER_REGISTER(MAILBOX, CXL_DEVICE_CAP_HDR1_OFFSET + \
> + CXL_DEVICE_CAP_REG_SIZE)
> +
> +REG32(CXL_DEV_MAILBOX_CAP, 0)
> + FIELD(CXL_DEV_MAILBOX_CAP, PAYLOAD_SIZE, 0, 5)
> + FIELD(CXL_DEV_MAILBOX_CAP, INT_CAP, 5, 1)
> + FIELD(CXL_DEV_MAILBOX_CAP, BG_INT_CAP, 6, 1)
> + FIELD(CXL_DEV_MAILBOX_CAP, MSI_N, 7, 4)
> +
> +REG32(CXL_DEV_MAILBOX_CTRL, 4)
> + FIELD(CXL_DEV_MAILBOX_CTRL, DOORBELL, 0, 1)
> + FIELD(CXL_DEV_MAILBOX_CTRL, INT_EN, 1, 1)
> + FIELD(CXL_DEV_MAILBOX_CTRL, BG_INT_EN, 2, 1)
Is it not worth defining the
CXL_DEV_MAILBOX_CMD register for completeness? off set 0x8
> +
> +/* XXX: actually a 64b register */
> +REG32(CXL_DEV_MAILBOX_STS, 0x10)
> + FIELD(CXL_DEV_MAILBOX_STS, BG_OP, 0, 1)
> + FIELD(CXL_DEV_MAILBOX_STS, ERRNO, 32, 16)
> + FIELD(CXL_DEV_MAILBOX_STS, VENDOR_ERRNO, 48, 16)
> +
> +/* XXX: actually a 64b register */
> +REG32(CXL_DEV_BG_CMD_STS, 0x18)
> + FIELD(CXL_DEV_BG_CMD_STS, BG, 0, 16)
> + FIELD(CXL_DEV_BG_CMD_STS, DONE, 16, 7)
> + FIELD(CXL_DEV_BG_CMD_STS, ERRNO, 32, 16)
> + FIELD(CXL_DEV_BG_CMD_STS, VENDOR_ERRNO, 48, 16)
> +
> +REG32(CXL_DEV_CMD_PAYLOAD, 0x20)
Probably want a comment for this one that it might be huge.
> +
> +#endif
next prev parent reply other threads:[~2021-02-02 12:08 UTC|newest]
Thread overview: 65+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-02 0:59 [RFC PATCH v3 00/31] CXL 2.0 Support Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 01/31] hw/pci/cxl: Add a CXL component type (interface) Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 02/31] hw/cxl/component: Introduce CXL components (8.1.x, 8.2.5) Ben Widawsky
2021-02-02 11:48 ` Jonathan Cameron
2021-02-17 18:36 ` Ben Widawsky
2021-02-11 17:08 ` Jonathan Cameron
2021-02-17 16:40 ` Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 03/31] hw/cxl/device: Introduce a CXL device (8.2.8) Ben Widawsky
2021-02-02 12:03 ` Jonathan Cameron [this message]
2021-02-02 0:59 ` [RFC PATCH v3 04/31] hw/cxl/device: Implement the CAP array (8.2.8.1-2) Ben Widawsky
2021-02-02 12:23 ` Jonathan Cameron
2021-02-17 22:15 ` Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 05/31] hw/cxl/device: Implement basic mailbox (8.2.8.4) Ben Widawsky
2021-02-02 14:58 ` Jonathan Cameron
2021-02-11 17:46 ` Jonathan Cameron
2021-02-18 0:55 ` Ben Widawsky
2021-02-18 16:50 ` Jonathan Cameron
2021-02-11 18:09 ` Jonathan Cameron
2021-02-02 0:59 ` [RFC PATCH v3 06/31] hw/cxl/device: Add memory device utilities Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 07/31] hw/cxl/device: Add cheap EVENTS implementation (8.2.9.1) Ben Widawsky
2021-02-02 13:44 ` Jonathan Cameron
2021-02-11 17:59 ` Jonathan Cameron
2021-02-02 0:59 ` [RFC PATCH v3 08/31] hw/cxl/device: Timestamp implementation (8.2.9.3) Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 09/31] hw/cxl/device: Add log commands (8.2.9.4) + CEL Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 10/31] hw/pxb: Use a type for realizing expanders Ben Widawsky
2021-02-02 13:50 ` Jonathan Cameron
2021-02-02 0:59 ` [RFC PATCH v3 11/31] hw/pci/cxl: Create a CXL bus type Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 12/31] hw/pxb: Allow creation of a CXL PXB (host bridge) Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 13/31] qtest: allow DSDT acpi table changes Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 14/31] acpi/pci: Consolidate host bridge setup Ben Widawsky
2021-02-02 13:56 ` Jonathan Cameron
2021-12-02 10:32 ` Jonathan Cameron via
2021-02-02 0:59 ` [RFC PATCH v3 15/31] tests/acpi: remove stale allowed tables Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 16/31] hw/pci: Plumb _UID through host bridges Ben Widawsky
2021-02-02 15:00 ` Jonathan Cameron
2021-02-02 15:24 ` Michael S. Tsirkin
2021-02-02 15:42 ` Ben Widawsky
2021-02-02 15:51 ` Michael S. Tsirkin
2021-02-02 16:20 ` Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 17/31] hw/cxl/component: Implement host bridge MMIO (8.2.5, table 142) Ben Widawsky
2021-02-02 19:21 ` Jonathan Cameron
2021-02-02 19:45 ` Ben Widawsky
2021-02-02 20:43 ` Jonathan Cameron
2021-02-02 21:03 ` Ben Widawsky
2021-02-02 22:06 ` Jonathan Cameron
2021-02-02 0:59 ` [RFC PATCH v3 18/31] acpi/pxb/cxl: Reserve host bridge MMIO Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 19/31] hw/pxb/cxl: Add "windows" for host bridges Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 20/31] hw/cxl/rp: Add a root port Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 21/31] hw/cxl/device: Add a memory device (8.2.8.5) Ben Widawsky
2021-02-02 14:26 ` Eric Blake
2021-02-02 15:06 ` Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 22/31] hw/cxl/device: Implement MMIO HDM decoding (8.2.5.12) Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 23/31] acpi/cxl: Add _OSC implementation (9.14.2) Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 24/31] tests/acpi: allow CEDT table addition Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 25/31] acpi/cxl: Create the CEDT (9.14.1) Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 26/31] tests/acpi: Add new CEDT files Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 27/31] hw/cxl/device: Add some trivial commands Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 28/31] hw/cxl/device: Plumb real LSA sizing Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 29/31] hw/cxl/device: Implement get/set LSA Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 30/31] qtest/cxl: Add very basic sanity tests Ben Widawsky
2021-02-02 0:59 ` [RFC PATCH v3 31/31] WIP: i386/cxl: Initialize a host bridge Ben Widawsky
2021-02-02 1:33 ` [RFC PATCH v3 00/31] CXL 2.0 Support no-reply
2021-02-03 17:42 ` Ben Widawsky
2021-02-11 18:51 ` Jonathan Cameron
2021-03-11 23:27 ` [RFC PATCH] hw/mem/cxl_type3: Go back to subregions Ben Widawsky
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210202120357.0000206a@Huawei.com \
--to=jonathan.cameron@huawei.com \
--cc=armbru@redhat.com \
--cc=ben.widawsky@intel.com \
--cc=cbrowy@avery-design.com \
--cc=dan.j.williams@intel.com \
--cc=david@redhat.com \
--cc=f4bug@amsat.org \
--cc=imammedo@redhat.com \
--cc=ira.weiny@intel.com \
--cc=jgroves@micron.com \
--cc=linux-cxl@vger.kernel.org \
--cc=mst@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=vishal.l.verma@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).