From: Jonathan Cameron <Jonathan.Cameron@huawei.com> To: <linuxarm@huawei.com>, <qemu-devel@nongnu.org>, <alex.bennee@linaro.org>, Marcel Apfelbaum <marcel@redhat.com>, "Michael S . Tsirkin" <mst@redhat.com>, Igor Mammedov <imammedo@redhat.com>, Markus Armbruster <armbru@redhat.com>, Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>, Adam Manzanares <a.manzanares@samsung.com>, Tong Zhang <ztong0001@gmail.com> Cc: <linux-cxl@vger.kernel.org>, Ben Widawsky <ben.widawsky@intel.com>, "Peter Maydell" <peter.maydell@linaro.org>, Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>, <f4bug@amsat.org>, Peter Xu <peterx@redhat.com>, David Hildenbrand <david@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>, Saransh Gupta1 <saransh@ibm.com>, Shreyas Shah <shreyas.shah@elastics.cloud>, Chris Browy <cbrowy@avery-design.com>, "Samarth Saxena" <samarths@cadence.com>, Dan Williams <dan.j.williams@intel.com>, <k.jensen@samsung.com>, <dave@stgolabs.net>, Alison Schofield <alison.schofield@intel.com> Subject: [PATCH v10 17/45] hw/cxl/device: Add a memory device (8.2.8.5) Date: Fri, 29 Apr 2022 15:40:42 +0100 [thread overview] Message-ID: <20220429144110.25167-18-Jonathan.Cameron@huawei.com> (raw) In-Reply-To: <20220429144110.25167-1-Jonathan.Cameron@huawei.com> From: Ben Widawsky <ben.widawsky@intel.com> A CXL memory device (AKA Type 3) is a CXL component that contains some combination of volatile and persistent memory. It also implements the previously defined mailbox interface as well as the memory device firmware interface. Although the memory device is configured like a normal PCIe device, the memory traffic is on an entirely separate bus conceptually (using the same physical wires as PCIe, but different protocol). Once the CXL topology is fully configure and address decoders committed, the guest physical address for the memory device is part of a larger window which is owned by the platform. The creation of these windows is later in this series. The following example will create a 256M device in a 512M window: -object "memory-backend-file,id=cxl-mem1,share,mem-path=cxl-type3,size=512M" -device "cxl-type3,bus=rp0,memdev=cxl-mem1,id=cxl-pmem0" Note: Dropped PCDIMM info interfaces for now. They can be added if appropriate at a later date. Signed-off-by: Ben Widawsky <ben.widawsky@intel.com> Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com> --- hw/cxl/cxl-component-utils.c | 18 ++++ hw/cxl/cxl-mailbox-utils.c | 46 ++++++++++ hw/mem/Kconfig | 5 ++ hw/mem/cxl_type3.c | 162 +++++++++++++++++++++++++++++++++++ hw/mem/meson.build | 1 + include/hw/cxl/cxl_device.h | 15 ++++ include/hw/cxl/cxl_pci.h | 21 +++++ include/hw/pci/pci_ids.h | 1 + 8 files changed, 269 insertions(+) diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c index 9457a1b029..afc97b17c2 100644 --- a/hw/cxl/cxl-component-utils.c +++ b/hw/cxl/cxl-component-utils.c @@ -252,6 +252,24 @@ void cxl_component_create_dvsec(CXLComponentState *cxl, /* Configure write masks */ switch (type) { case PCIE_CXL_DEVICE_DVSEC: + /* Cntrl RW Lock - so needs explicit blocking when lock is set */ + wmask[offset + offsetof(CXLDVSECDevice, ctrl)] = 0xFD; + wmask[offset + offsetof(CXLDVSECDevice, ctrl) + 1] = 0x4F; + /* Status is RW1CS */ + wmask[offset + offsetof(CXLDVSECDevice, ctrl2)] = 0x0F; + /* Lock is RW Once */ + wmask[offset + offsetof(CXLDVSECDevice, lock)] = 0x01; + /* range1/2_base_high/low is RW Lock */ + wmask[offset + offsetof(CXLDVSECDevice, range1_base_hi)] = 0xFF; + wmask[offset + offsetof(CXLDVSECDevice, range1_base_hi) + 1] = 0xFF; + wmask[offset + offsetof(CXLDVSECDevice, range1_base_hi) + 2] = 0xFF; + wmask[offset + offsetof(CXLDVSECDevice, range1_base_hi) + 3] = 0xFF; + wmask[offset + offsetof(CXLDVSECDevice, range1_base_lo) + 3] = 0xF0; + wmask[offset + offsetof(CXLDVSECDevice, range2_base_hi)] = 0xFF; + wmask[offset + offsetof(CXLDVSECDevice, range2_base_hi) + 1] = 0xFF; + wmask[offset + offsetof(CXLDVSECDevice, range2_base_hi) + 2] = 0xFF; + wmask[offset + offsetof(CXLDVSECDevice, range2_base_hi) + 3] = 0xFF; + wmask[offset + offsetof(CXLDVSECDevice, range2_base_lo) + 3] = 0xF0; break; case NON_CXL_FUNCTION_MAP_DVSEC: break; /* Not yet implemented */ diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index db473135c7..4ae0561dfc 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -50,6 +50,8 @@ enum { LOGS = 0x04, #define GET_SUPPORTED 0x0 #define GET_LOG 0x1 + IDENTIFY = 0x40, + #define MEMORY_DEVICE 0x0 }; /* 8.2.8.4.5.1 Command Return Codes */ @@ -214,6 +216,48 @@ static ret_code cmd_logs_get_log(struct cxl_cmd *cmd, return CXL_MBOX_SUCCESS; } +/* 8.2.9.5.1.1 */ +static ret_code cmd_identify_memory_device(struct cxl_cmd *cmd, + CXLDeviceState *cxl_dstate, + uint16_t *len) +{ + struct { + char fw_revision[0x10]; + uint64_t total_capacity; + uint64_t volatile_capacity; + uint64_t persistent_capacity; + uint64_t partition_align; + uint16_t info_event_log_size; + uint16_t warning_event_log_size; + uint16_t failure_event_log_size; + uint16_t fatal_event_log_size; + uint32_t lsa_size; + uint8_t poison_list_max_mer[3]; + uint16_t inject_poison_limit; + uint8_t poison_caps; + uint8_t qos_telemetry_caps; + } QEMU_PACKED *id; + QEMU_BUILD_BUG_ON(sizeof(*id) != 0x43); + + uint64_t size = cxl_dstate->pmem_size; + + if (!QEMU_IS_ALIGNED(size, 256 << 20)) { + return CXL_MBOX_INTERNAL_ERROR; + } + + id = (void *)cmd->payload; + memset(id, 0, sizeof(*id)); + + /* PMEM only */ + snprintf(id->fw_revision, 0x10, "BWFW VERSION %02d", 0); + + id->total_capacity = size / (256 << 20); + id->persistent_capacity = size / (256 << 20); + + *len = sizeof(*id); + return CXL_MBOX_SUCCESS; +} + #define IMMEDIATE_CONFIG_CHANGE (1 << 1) #define IMMEDIATE_POLICY_CHANGE (1 << 3) #define IMMEDIATE_LOG_CHANGE (1 << 4) @@ -231,6 +275,8 @@ static struct cxl_cmd cxl_cmd_set[256][256] = { [TIMESTAMP][SET] = { "TIMESTAMP_SET", cmd_timestamp_set, 8, IMMEDIATE_POLICY_CHANGE }, [LOGS][GET_SUPPORTED] = { "LOGS_GET_SUPPORTED", cmd_logs_get_supported, 0, 0 }, [LOGS][GET_LOG] = { "LOGS_GET_LOG", cmd_logs_get_log, 0x18, 0 }, + [IDENTIFY][MEMORY_DEVICE] = { "IDENTIFY_MEMORY_DEVICE", + cmd_identify_memory_device, 0, 0 }, }; void cxl_process_mailbox(CXLDeviceState *cxl_dstate) diff --git a/hw/mem/Kconfig b/hw/mem/Kconfig index 03dbb3c7df..73c5ae8ad9 100644 --- a/hw/mem/Kconfig +++ b/hw/mem/Kconfig @@ -11,3 +11,8 @@ config NVDIMM config SPARSE_MEM bool + +config CXL_MEM_DEVICE + bool + default y if CXL + select MEM_DEVICE diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c new file mode 100644 index 0000000000..290c088dd6 --- /dev/null +++ b/hw/mem/cxl_type3.c @@ -0,0 +1,162 @@ +#include "qemu/osdep.h" +#include "qemu/units.h" +#include "qemu/error-report.h" +#include "hw/mem/memory-device.h" +#include "hw/mem/pc-dimm.h" +#include "hw/pci/pci.h" +#include "hw/qdev-properties.h" +#include "qapi/error.h" +#include "qemu/log.h" +#include "qemu/module.h" +#include "qemu/range.h" +#include "qemu/rcu.h" +#include "sysemu/hostmem.h" +#include "hw/cxl/cxl.h" + +static void build_dvsecs(CXLType3Dev *ct3d) +{ + CXLComponentState *cxl_cstate = &ct3d->cxl_cstate; + uint8_t *dvsec; + + dvsec = (uint8_t *)&(CXLDVSECDevice){ + .cap = 0x1e, + .ctrl = 0x2, + .status2 = 0x2, + .range1_size_hi = ct3d->hostmem->size >> 32, + .range1_size_lo = (2 << 5) | (2 << 2) | 0x3 | + (ct3d->hostmem->size & 0xF0000000), + .range1_base_hi = 0, + .range1_base_lo = 0, + }; + cxl_component_create_dvsec(cxl_cstate, CXL2_TYPE3_DEVICE, + PCIE_CXL_DEVICE_DVSEC_LENGTH, + PCIE_CXL_DEVICE_DVSEC, + PCIE_CXL2_DEVICE_DVSEC_REVID, dvsec); + + dvsec = (uint8_t *)&(CXLDVSECRegisterLocator){ + .rsvd = 0, + .reg0_base_lo = RBI_COMPONENT_REG | CXL_COMPONENT_REG_BAR_IDX, + .reg0_base_hi = 0, + .reg1_base_lo = RBI_CXL_DEVICE_REG | CXL_DEVICE_REG_BAR_IDX, + .reg1_base_hi = 0, + }; + cxl_component_create_dvsec(cxl_cstate, CXL2_TYPE3_DEVICE, + REG_LOC_DVSEC_LENGTH, REG_LOC_DVSEC, + REG_LOC_DVSEC_REVID, dvsec); + dvsec = (uint8_t *)&(CXLDVSECDeviceGPF){ + .phase2_duration = 0x603, /* 3 seconds */ + .phase2_power = 0x33, /* 0x33 miliwatts */ + }; + cxl_component_create_dvsec(cxl_cstate, CXL2_TYPE3_DEVICE, + GPF_DEVICE_DVSEC_LENGTH, GPF_PORT_DVSEC, + GPF_DEVICE_DVSEC_REVID, dvsec); +} + +static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp) +{ + MemoryRegion *mr; + + if (!ct3d->hostmem) { + error_setg(errp, "memdev property must be set"); + return false; + } + + mr = host_memory_backend_get_memory(ct3d->hostmem); + if (!mr) { + error_setg(errp, "memdev property must be set"); + return false; + } + memory_region_set_nonvolatile(mr, true); + memory_region_set_enabled(mr, true); + host_memory_backend_set_mapped(ct3d->hostmem, true); + ct3d->cxl_dstate.pmem_size = ct3d->hostmem->size; + + return true; +} + +static void ct3_realize(PCIDevice *pci_dev, Error **errp) +{ + CXLType3Dev *ct3d = CXL_TYPE3(pci_dev); + CXLComponentState *cxl_cstate = &ct3d->cxl_cstate; + ComponentRegisters *regs = &cxl_cstate->crb; + MemoryRegion *mr = ®s->component_registers; + uint8_t *pci_conf = pci_dev->config; + + if (!cxl_setup_memory(ct3d, errp)) { + return; + } + + pci_config_set_prog_interface(pci_conf, 0x10); + pci_config_set_class(pci_conf, PCI_CLASS_MEMORY_CXL); + + pcie_endpoint_cap_init(pci_dev, 0x80); + cxl_cstate->dvsec_offset = 0x100; + + ct3d->cxl_cstate.pdev = pci_dev; + build_dvsecs(ct3d); + + cxl_component_register_block_init(OBJECT(pci_dev), cxl_cstate, + TYPE_CXL_TYPE3); + + pci_register_bar( + pci_dev, CXL_COMPONENT_REG_BAR_IDX, + PCI_BASE_ADDRESS_SPACE_MEMORY | PCI_BASE_ADDRESS_MEM_TYPE_64, mr); + + cxl_device_register_block_init(OBJECT(pci_dev), &ct3d->cxl_dstate); + pci_register_bar(pci_dev, CXL_DEVICE_REG_BAR_IDX, + PCI_BASE_ADDRESS_SPACE_MEMORY | + PCI_BASE_ADDRESS_MEM_TYPE_64, + &ct3d->cxl_dstate.device_registers); +} + +static void ct3d_reset(DeviceState *dev) +{ + CXLType3Dev *ct3d = CXL_TYPE3(dev); + uint32_t *reg_state = ct3d->cxl_cstate.crb.cache_mem_registers; + uint32_t *write_msk = ct3d->cxl_cstate.crb.cache_mem_regs_write_mask; + + cxl_component_register_init_common(reg_state, write_msk, CXL2_TYPE3_DEVICE); + cxl_device_register_init_common(&ct3d->cxl_dstate); +} + +static Property ct3_props[] = { + DEFINE_PROP_LINK("memdev", CXLType3Dev, hostmem, TYPE_MEMORY_BACKEND, + HostMemoryBackend *), + DEFINE_PROP_END_OF_LIST(), +}; + +static void ct3_class_init(ObjectClass *oc, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(oc); + PCIDeviceClass *pc = PCI_DEVICE_CLASS(oc); + + pc->realize = ct3_realize; + pc->class_id = PCI_CLASS_STORAGE_EXPRESS; + pc->vendor_id = PCI_VENDOR_ID_INTEL; + pc->device_id = 0xd93; /* LVF for now */ + pc->revision = 1; + + set_bit(DEVICE_CATEGORY_STORAGE, dc->categories); + dc->desc = "CXL PMEM Device (Type 3)"; + dc->reset = ct3d_reset; + device_class_set_props(dc, ct3_props); +} + +static const TypeInfo ct3d_info = { + .name = TYPE_CXL_TYPE3, + .parent = TYPE_PCI_DEVICE, + .class_init = ct3_class_init, + .instance_size = sizeof(CXLType3Dev), + .interfaces = (InterfaceInfo[]) { + { INTERFACE_CXL_DEVICE }, + { INTERFACE_PCIE_DEVICE }, + {} + }, +}; + +static void ct3d_registers(void) +{ + type_register_static(&ct3d_info); +} + +type_init(ct3d_registers); diff --git a/hw/mem/meson.build b/hw/mem/meson.build index 82f86d117e..609b2b36fc 100644 --- a/hw/mem/meson.build +++ b/hw/mem/meson.build @@ -3,6 +3,7 @@ mem_ss.add(files('memory-device.c')) mem_ss.add(when: 'CONFIG_DIMM', if_true: files('pc-dimm.c')) mem_ss.add(when: 'CONFIG_NPCM7XX', if_true: files('npcm7xx_mc.c')) mem_ss.add(when: 'CONFIG_NVDIMM', if_true: files('nvdimm.c')) +mem_ss.add(when: 'CONFIG_CXL_MEM_DEVICE', if_true: files('cxl_type3.c')) softmmu_ss.add_all(when: 'CONFIG_MEM_DEVICE', if_true: mem_ss) diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index 797a22ddb4..d8da2c7b68 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -230,4 +230,19 @@ REG64(CXL_MEM_DEV_STS, 0) FIELD(CXL_MEM_DEV_STS, MBOX_READY, 4, 1) FIELD(CXL_MEM_DEV_STS, RESET_NEEDED, 5, 3) +struct CXLType3Dev { + /* Private */ + PCIDevice parent_obj; + + /* Properties */ + HostMemoryBackend *hostmem; + + /* State */ + CXLComponentState cxl_cstate; + CXLDeviceState cxl_dstate; +}; + +#define TYPE_CXL_TYPE3 "cxl-type3" +OBJECT_DECLARE_SIMPLE_TYPE(CXLType3Dev, CXL_TYPE3) + #endif diff --git a/include/hw/cxl/cxl_pci.h b/include/hw/cxl/cxl_pci.h index 3f7e44daac..01cf002096 100644 --- a/include/hw/cxl/cxl_pci.h +++ b/include/hw/cxl/cxl_pci.h @@ -67,6 +67,27 @@ QEMU_BUILD_BUG_ON(sizeof(DVSECHeader) != 10); * CXL 2.0 Downstream Port: 3, 4, 7, 8 */ +/* CXL 2.0 - 8.1.3 (ID 0001) */ +typedef struct CXLDVSECDevice { + DVSECHeader hdr; + uint16_t cap; + uint16_t ctrl; + uint16_t status; + uint16_t ctrl2; + uint16_t status2; + uint16_t lock; + uint16_t cap2; + uint32_t range1_size_hi; + uint32_t range1_size_lo; + uint32_t range1_base_hi; + uint32_t range1_base_lo; + uint32_t range2_size_hi; + uint32_t range2_size_lo; + uint32_t range2_base_hi; + uint32_t range2_base_lo; +} CXLDVSECDevice; +QEMU_BUILD_BUG_ON(sizeof(CXLDVSECDevice) != 0x38); + /* CXL 2.0 - 8.1.5 (ID 0003) */ typedef struct CXLDVSECPortExtensions { DVSECHeader hdr; diff --git a/include/hw/pci/pci_ids.h b/include/hw/pci/pci_ids.h index 11abe22d46..898083b86f 100644 --- a/include/hw/pci/pci_ids.h +++ b/include/hw/pci/pci_ids.h @@ -53,6 +53,7 @@ #define PCI_BASE_CLASS_MEMORY 0x05 #define PCI_CLASS_MEMORY_RAM 0x0500 #define PCI_CLASS_MEMORY_FLASH 0x0501 +#define PCI_CLASS_MEMORY_CXL 0x0502 #define PCI_CLASS_MEMORY_OTHER 0x0580 #define PCI_BASE_CLASS_BRIDGE 0x06 -- 2.32.0
WARNING: multiple messages have this Message-ID (diff)
From: Jonathan Cameron via <qemu-devel@nongnu.org> To: <linuxarm@huawei.com>, <qemu-devel@nongnu.org>, <alex.bennee@linaro.org>, Marcel Apfelbaum <marcel@redhat.com>, "Michael S . Tsirkin" <mst@redhat.com>, Igor Mammedov <imammedo@redhat.com>, Markus Armbruster <armbru@redhat.com>, Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>, Adam Manzanares <a.manzanares@samsung.com>, Tong Zhang <ztong0001@gmail.com> Cc: <linux-cxl@vger.kernel.org>, Ben Widawsky <ben.widawsky@intel.com>, "Peter Maydell" <peter.maydell@linaro.org>, Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>, <f4bug@amsat.org>, Peter Xu <peterx@redhat.com>, David Hildenbrand <david@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>, Saransh Gupta1 <saransh@ibm.com>, Shreyas Shah <shreyas.shah@elastics.cloud>, Chris Browy <cbrowy@avery-design.com>, "Samarth Saxena" <samarths@cadence.com>, Dan Williams <dan.j.williams@intel.com>, <k.jensen@samsung.com>, <dave@stgolabs.net>, Alison Schofield <alison.schofield@intel.com> Subject: [PATCH v10 17/45] hw/cxl/device: Add a memory device (8.2.8.5) Date: Fri, 29 Apr 2022 15:40:42 +0100 [thread overview] Message-ID: <20220429144110.25167-18-Jonathan.Cameron@huawei.com> (raw) In-Reply-To: <20220429144110.25167-1-Jonathan.Cameron@huawei.com> From: Ben Widawsky <ben.widawsky@intel.com> A CXL memory device (AKA Type 3) is a CXL component that contains some combination of volatile and persistent memory. It also implements the previously defined mailbox interface as well as the memory device firmware interface. Although the memory device is configured like a normal PCIe device, the memory traffic is on an entirely separate bus conceptually (using the same physical wires as PCIe, but different protocol). Once the CXL topology is fully configure and address decoders committed, the guest physical address for the memory device is part of a larger window which is owned by the platform. The creation of these windows is later in this series. The following example will create a 256M device in a 512M window: -object "memory-backend-file,id=cxl-mem1,share,mem-path=cxl-type3,size=512M" -device "cxl-type3,bus=rp0,memdev=cxl-mem1,id=cxl-pmem0" Note: Dropped PCDIMM info interfaces for now. They can be added if appropriate at a later date. Signed-off-by: Ben Widawsky <ben.widawsky@intel.com> Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com> --- hw/cxl/cxl-component-utils.c | 18 ++++ hw/cxl/cxl-mailbox-utils.c | 46 ++++++++++ hw/mem/Kconfig | 5 ++ hw/mem/cxl_type3.c | 162 +++++++++++++++++++++++++++++++++++ hw/mem/meson.build | 1 + include/hw/cxl/cxl_device.h | 15 ++++ include/hw/cxl/cxl_pci.h | 21 +++++ include/hw/pci/pci_ids.h | 1 + 8 files changed, 269 insertions(+) diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c index 9457a1b029..afc97b17c2 100644 --- a/hw/cxl/cxl-component-utils.c +++ b/hw/cxl/cxl-component-utils.c @@ -252,6 +252,24 @@ void cxl_component_create_dvsec(CXLComponentState *cxl, /* Configure write masks */ switch (type) { case PCIE_CXL_DEVICE_DVSEC: + /* Cntrl RW Lock - so needs explicit blocking when lock is set */ + wmask[offset + offsetof(CXLDVSECDevice, ctrl)] = 0xFD; + wmask[offset + offsetof(CXLDVSECDevice, ctrl) + 1] = 0x4F; + /* Status is RW1CS */ + wmask[offset + offsetof(CXLDVSECDevice, ctrl2)] = 0x0F; + /* Lock is RW Once */ + wmask[offset + offsetof(CXLDVSECDevice, lock)] = 0x01; + /* range1/2_base_high/low is RW Lock */ + wmask[offset + offsetof(CXLDVSECDevice, range1_base_hi)] = 0xFF; + wmask[offset + offsetof(CXLDVSECDevice, range1_base_hi) + 1] = 0xFF; + wmask[offset + offsetof(CXLDVSECDevice, range1_base_hi) + 2] = 0xFF; + wmask[offset + offsetof(CXLDVSECDevice, range1_base_hi) + 3] = 0xFF; + wmask[offset + offsetof(CXLDVSECDevice, range1_base_lo) + 3] = 0xF0; + wmask[offset + offsetof(CXLDVSECDevice, range2_base_hi)] = 0xFF; + wmask[offset + offsetof(CXLDVSECDevice, range2_base_hi) + 1] = 0xFF; + wmask[offset + offsetof(CXLDVSECDevice, range2_base_hi) + 2] = 0xFF; + wmask[offset + offsetof(CXLDVSECDevice, range2_base_hi) + 3] = 0xFF; + wmask[offset + offsetof(CXLDVSECDevice, range2_base_lo) + 3] = 0xF0; break; case NON_CXL_FUNCTION_MAP_DVSEC: break; /* Not yet implemented */ diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index db473135c7..4ae0561dfc 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -50,6 +50,8 @@ enum { LOGS = 0x04, #define GET_SUPPORTED 0x0 #define GET_LOG 0x1 + IDENTIFY = 0x40, + #define MEMORY_DEVICE 0x0 }; /* 8.2.8.4.5.1 Command Return Codes */ @@ -214,6 +216,48 @@ static ret_code cmd_logs_get_log(struct cxl_cmd *cmd, return CXL_MBOX_SUCCESS; } +/* 8.2.9.5.1.1 */ +static ret_code cmd_identify_memory_device(struct cxl_cmd *cmd, + CXLDeviceState *cxl_dstate, + uint16_t *len) +{ + struct { + char fw_revision[0x10]; + uint64_t total_capacity; + uint64_t volatile_capacity; + uint64_t persistent_capacity; + uint64_t partition_align; + uint16_t info_event_log_size; + uint16_t warning_event_log_size; + uint16_t failure_event_log_size; + uint16_t fatal_event_log_size; + uint32_t lsa_size; + uint8_t poison_list_max_mer[3]; + uint16_t inject_poison_limit; + uint8_t poison_caps; + uint8_t qos_telemetry_caps; + } QEMU_PACKED *id; + QEMU_BUILD_BUG_ON(sizeof(*id) != 0x43); + + uint64_t size = cxl_dstate->pmem_size; + + if (!QEMU_IS_ALIGNED(size, 256 << 20)) { + return CXL_MBOX_INTERNAL_ERROR; + } + + id = (void *)cmd->payload; + memset(id, 0, sizeof(*id)); + + /* PMEM only */ + snprintf(id->fw_revision, 0x10, "BWFW VERSION %02d", 0); + + id->total_capacity = size / (256 << 20); + id->persistent_capacity = size / (256 << 20); + + *len = sizeof(*id); + return CXL_MBOX_SUCCESS; +} + #define IMMEDIATE_CONFIG_CHANGE (1 << 1) #define IMMEDIATE_POLICY_CHANGE (1 << 3) #define IMMEDIATE_LOG_CHANGE (1 << 4) @@ -231,6 +275,8 @@ static struct cxl_cmd cxl_cmd_set[256][256] = { [TIMESTAMP][SET] = { "TIMESTAMP_SET", cmd_timestamp_set, 8, IMMEDIATE_POLICY_CHANGE }, [LOGS][GET_SUPPORTED] = { "LOGS_GET_SUPPORTED", cmd_logs_get_supported, 0, 0 }, [LOGS][GET_LOG] = { "LOGS_GET_LOG", cmd_logs_get_log, 0x18, 0 }, + [IDENTIFY][MEMORY_DEVICE] = { "IDENTIFY_MEMORY_DEVICE", + cmd_identify_memory_device, 0, 0 }, }; void cxl_process_mailbox(CXLDeviceState *cxl_dstate) diff --git a/hw/mem/Kconfig b/hw/mem/Kconfig index 03dbb3c7df..73c5ae8ad9 100644 --- a/hw/mem/Kconfig +++ b/hw/mem/Kconfig @@ -11,3 +11,8 @@ config NVDIMM config SPARSE_MEM bool + +config CXL_MEM_DEVICE + bool + default y if CXL + select MEM_DEVICE diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c new file mode 100644 index 0000000000..290c088dd6 --- /dev/null +++ b/hw/mem/cxl_type3.c @@ -0,0 +1,162 @@ +#include "qemu/osdep.h" +#include "qemu/units.h" +#include "qemu/error-report.h" +#include "hw/mem/memory-device.h" +#include "hw/mem/pc-dimm.h" +#include "hw/pci/pci.h" +#include "hw/qdev-properties.h" +#include "qapi/error.h" +#include "qemu/log.h" +#include "qemu/module.h" +#include "qemu/range.h" +#include "qemu/rcu.h" +#include "sysemu/hostmem.h" +#include "hw/cxl/cxl.h" + +static void build_dvsecs(CXLType3Dev *ct3d) +{ + CXLComponentState *cxl_cstate = &ct3d->cxl_cstate; + uint8_t *dvsec; + + dvsec = (uint8_t *)&(CXLDVSECDevice){ + .cap = 0x1e, + .ctrl = 0x2, + .status2 = 0x2, + .range1_size_hi = ct3d->hostmem->size >> 32, + .range1_size_lo = (2 << 5) | (2 << 2) | 0x3 | + (ct3d->hostmem->size & 0xF0000000), + .range1_base_hi = 0, + .range1_base_lo = 0, + }; + cxl_component_create_dvsec(cxl_cstate, CXL2_TYPE3_DEVICE, + PCIE_CXL_DEVICE_DVSEC_LENGTH, + PCIE_CXL_DEVICE_DVSEC, + PCIE_CXL2_DEVICE_DVSEC_REVID, dvsec); + + dvsec = (uint8_t *)&(CXLDVSECRegisterLocator){ + .rsvd = 0, + .reg0_base_lo = RBI_COMPONENT_REG | CXL_COMPONENT_REG_BAR_IDX, + .reg0_base_hi = 0, + .reg1_base_lo = RBI_CXL_DEVICE_REG | CXL_DEVICE_REG_BAR_IDX, + .reg1_base_hi = 0, + }; + cxl_component_create_dvsec(cxl_cstate, CXL2_TYPE3_DEVICE, + REG_LOC_DVSEC_LENGTH, REG_LOC_DVSEC, + REG_LOC_DVSEC_REVID, dvsec); + dvsec = (uint8_t *)&(CXLDVSECDeviceGPF){ + .phase2_duration = 0x603, /* 3 seconds */ + .phase2_power = 0x33, /* 0x33 miliwatts */ + }; + cxl_component_create_dvsec(cxl_cstate, CXL2_TYPE3_DEVICE, + GPF_DEVICE_DVSEC_LENGTH, GPF_PORT_DVSEC, + GPF_DEVICE_DVSEC_REVID, dvsec); +} + +static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp) +{ + MemoryRegion *mr; + + if (!ct3d->hostmem) { + error_setg(errp, "memdev property must be set"); + return false; + } + + mr = host_memory_backend_get_memory(ct3d->hostmem); + if (!mr) { + error_setg(errp, "memdev property must be set"); + return false; + } + memory_region_set_nonvolatile(mr, true); + memory_region_set_enabled(mr, true); + host_memory_backend_set_mapped(ct3d->hostmem, true); + ct3d->cxl_dstate.pmem_size = ct3d->hostmem->size; + + return true; +} + +static void ct3_realize(PCIDevice *pci_dev, Error **errp) +{ + CXLType3Dev *ct3d = CXL_TYPE3(pci_dev); + CXLComponentState *cxl_cstate = &ct3d->cxl_cstate; + ComponentRegisters *regs = &cxl_cstate->crb; + MemoryRegion *mr = ®s->component_registers; + uint8_t *pci_conf = pci_dev->config; + + if (!cxl_setup_memory(ct3d, errp)) { + return; + } + + pci_config_set_prog_interface(pci_conf, 0x10); + pci_config_set_class(pci_conf, PCI_CLASS_MEMORY_CXL); + + pcie_endpoint_cap_init(pci_dev, 0x80); + cxl_cstate->dvsec_offset = 0x100; + + ct3d->cxl_cstate.pdev = pci_dev; + build_dvsecs(ct3d); + + cxl_component_register_block_init(OBJECT(pci_dev), cxl_cstate, + TYPE_CXL_TYPE3); + + pci_register_bar( + pci_dev, CXL_COMPONENT_REG_BAR_IDX, + PCI_BASE_ADDRESS_SPACE_MEMORY | PCI_BASE_ADDRESS_MEM_TYPE_64, mr); + + cxl_device_register_block_init(OBJECT(pci_dev), &ct3d->cxl_dstate); + pci_register_bar(pci_dev, CXL_DEVICE_REG_BAR_IDX, + PCI_BASE_ADDRESS_SPACE_MEMORY | + PCI_BASE_ADDRESS_MEM_TYPE_64, + &ct3d->cxl_dstate.device_registers); +} + +static void ct3d_reset(DeviceState *dev) +{ + CXLType3Dev *ct3d = CXL_TYPE3(dev); + uint32_t *reg_state = ct3d->cxl_cstate.crb.cache_mem_registers; + uint32_t *write_msk = ct3d->cxl_cstate.crb.cache_mem_regs_write_mask; + + cxl_component_register_init_common(reg_state, write_msk, CXL2_TYPE3_DEVICE); + cxl_device_register_init_common(&ct3d->cxl_dstate); +} + +static Property ct3_props[] = { + DEFINE_PROP_LINK("memdev", CXLType3Dev, hostmem, TYPE_MEMORY_BACKEND, + HostMemoryBackend *), + DEFINE_PROP_END_OF_LIST(), +}; + +static void ct3_class_init(ObjectClass *oc, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(oc); + PCIDeviceClass *pc = PCI_DEVICE_CLASS(oc); + + pc->realize = ct3_realize; + pc->class_id = PCI_CLASS_STORAGE_EXPRESS; + pc->vendor_id = PCI_VENDOR_ID_INTEL; + pc->device_id = 0xd93; /* LVF for now */ + pc->revision = 1; + + set_bit(DEVICE_CATEGORY_STORAGE, dc->categories); + dc->desc = "CXL PMEM Device (Type 3)"; + dc->reset = ct3d_reset; + device_class_set_props(dc, ct3_props); +} + +static const TypeInfo ct3d_info = { + .name = TYPE_CXL_TYPE3, + .parent = TYPE_PCI_DEVICE, + .class_init = ct3_class_init, + .instance_size = sizeof(CXLType3Dev), + .interfaces = (InterfaceInfo[]) { + { INTERFACE_CXL_DEVICE }, + { INTERFACE_PCIE_DEVICE }, + {} + }, +}; + +static void ct3d_registers(void) +{ + type_register_static(&ct3d_info); +} + +type_init(ct3d_registers); diff --git a/hw/mem/meson.build b/hw/mem/meson.build index 82f86d117e..609b2b36fc 100644 --- a/hw/mem/meson.build +++ b/hw/mem/meson.build @@ -3,6 +3,7 @@ mem_ss.add(files('memory-device.c')) mem_ss.add(when: 'CONFIG_DIMM', if_true: files('pc-dimm.c')) mem_ss.add(when: 'CONFIG_NPCM7XX', if_true: files('npcm7xx_mc.c')) mem_ss.add(when: 'CONFIG_NVDIMM', if_true: files('nvdimm.c')) +mem_ss.add(when: 'CONFIG_CXL_MEM_DEVICE', if_true: files('cxl_type3.c')) softmmu_ss.add_all(when: 'CONFIG_MEM_DEVICE', if_true: mem_ss) diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index 797a22ddb4..d8da2c7b68 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -230,4 +230,19 @@ REG64(CXL_MEM_DEV_STS, 0) FIELD(CXL_MEM_DEV_STS, MBOX_READY, 4, 1) FIELD(CXL_MEM_DEV_STS, RESET_NEEDED, 5, 3) +struct CXLType3Dev { + /* Private */ + PCIDevice parent_obj; + + /* Properties */ + HostMemoryBackend *hostmem; + + /* State */ + CXLComponentState cxl_cstate; + CXLDeviceState cxl_dstate; +}; + +#define TYPE_CXL_TYPE3 "cxl-type3" +OBJECT_DECLARE_SIMPLE_TYPE(CXLType3Dev, CXL_TYPE3) + #endif diff --git a/include/hw/cxl/cxl_pci.h b/include/hw/cxl/cxl_pci.h index 3f7e44daac..01cf002096 100644 --- a/include/hw/cxl/cxl_pci.h +++ b/include/hw/cxl/cxl_pci.h @@ -67,6 +67,27 @@ QEMU_BUILD_BUG_ON(sizeof(DVSECHeader) != 10); * CXL 2.0 Downstream Port: 3, 4, 7, 8 */ +/* CXL 2.0 - 8.1.3 (ID 0001) */ +typedef struct CXLDVSECDevice { + DVSECHeader hdr; + uint16_t cap; + uint16_t ctrl; + uint16_t status; + uint16_t ctrl2; + uint16_t status2; + uint16_t lock; + uint16_t cap2; + uint32_t range1_size_hi; + uint32_t range1_size_lo; + uint32_t range1_base_hi; + uint32_t range1_base_lo; + uint32_t range2_size_hi; + uint32_t range2_size_lo; + uint32_t range2_base_hi; + uint32_t range2_base_lo; +} CXLDVSECDevice; +QEMU_BUILD_BUG_ON(sizeof(CXLDVSECDevice) != 0x38); + /* CXL 2.0 - 8.1.5 (ID 0003) */ typedef struct CXLDVSECPortExtensions { DVSECHeader hdr; diff --git a/include/hw/pci/pci_ids.h b/include/hw/pci/pci_ids.h index 11abe22d46..898083b86f 100644 --- a/include/hw/pci/pci_ids.h +++ b/include/hw/pci/pci_ids.h @@ -53,6 +53,7 @@ #define PCI_BASE_CLASS_MEMORY 0x05 #define PCI_CLASS_MEMORY_RAM 0x0500 #define PCI_CLASS_MEMORY_FLASH 0x0501 +#define PCI_CLASS_MEMORY_CXL 0x0502 #define PCI_CLASS_MEMORY_OTHER 0x0580 #define PCI_BASE_CLASS_BRIDGE 0x06 -- 2.32.0
next prev parent reply other threads:[~2022-04-29 14:49 UTC|newest] Thread overview: 100+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-04-29 14:40 [PATCH v10 00/45] CXl 2.0 emulation Support Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 01/45] hw/pci/cxl: Add a CXL component type (interface) Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 02/45] hw/cxl/component: Introduce CXL components (8.1.x, 8.2.5) Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-05-30 17:50 ` Jonathan Cameron 2022-05-30 17:50 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 03/45] MAINTAINERS: Add entry for Compute Express Link Emulation Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 04/45] hw/cxl/device: Introduce a CXL device (8.2.8) Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 05/45] hw/cxl/device: Implement the CAP array (8.2.8.1-2) Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 06/45] hw/cxl/device: Implement basic mailbox (8.2.8.4) Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 07/45] hw/cxl/device: Add memory device utilities Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 08/45] hw/cxl/device: Add cheap EVENTS implementation (8.2.9.1) Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 09/45] hw/cxl/device: Timestamp implementation (8.2.9.3) Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 10/45] hw/cxl/device: Add log commands (8.2.9.4) + CEL Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 11/45] hw/pxb: Use a type for realizing expanders Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 12/45] hw/pci/cxl: Create a CXL bus type Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 13/45] cxl: Machine level control on whether CXL support is enabled Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-05-21 9:11 ` Paolo Bonzini 2022-04-29 14:40 ` [PATCH v10 14/45] hw/pxb: Allow creation of a CXL PXB (host bridge) Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 15/45] qtest/cxl: Introduce initial test for pxb-cxl only Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 16/45] hw/cxl/rp: Add a root port Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` Jonathan Cameron [this message] 2022-04-29 14:40 ` [PATCH v10 17/45] hw/cxl/device: Add a memory device (8.2.8.5) Jonathan Cameron via 2023-09-13 12:20 ` Philippe Mathieu-Daudé 2022-04-29 14:40 ` [PATCH v10 18/45] hw/cxl/device: Implement MMIO HDM decoding (8.2.5.12) Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 19/45] hw/cxl/device: Add some trivial commands Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 20/45] hw/cxl/device: Plumb real Label Storage Area (LSA) sizing Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 21/45] hw/cxl/device: Implement get/set Label Storage Area (LSA) Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 22/45] qtests/cxl: Add initial root port and CXL type3 tests Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 23/45] hw/cxl/component: Implement host bridge MMIO (8.2.5, table 142) Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 24/45] acpi/cxl: Add _OSC implementation (9.14.2) Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 25/45] acpi/cxl: Create the CEDT (9.14.1) Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 26/45] hw/cxl/component: Add utils for interleave parameter encoding/decoding Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 27/45] hw/cxl/host: Add support for CXL Fixed Memory Windows Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-05-21 9:07 ` Paolo Bonzini 2022-05-23 15:15 ` Jonathan Cameron 2022-05-23 15:15 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 28/45] acpi/cxl: Introduce CFMWS structures in CEDT Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 29/45] hw/pci-host/gpex-acpi: Add support for dsdt construction for pxb-cxl Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 30/45] pci/pcie_port: Add pci_find_port_by_pn() Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 31/45] CXL/cxl_component: Add cxl_get_hb_cstate() Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 32/45] mem/cxl_type3: Add read and write functions for associated hostmem Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 33/45] cxl/cxl-host: Add memops for CFMWS region Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:40 ` [PATCH v10 34/45] hw/cxl/component Add a dumb HDM decoder handler Jonathan Cameron 2022-04-29 14:40 ` Jonathan Cameron via 2022-04-29 14:41 ` [PATCH v10 35/45] i386/pc: Enable CXL fixed memory windows Jonathan Cameron 2022-04-29 14:41 ` Jonathan Cameron via 2022-04-29 14:41 ` [PATCH v10 36/45] tests/acpi: q35: Allow addition of a CXL test Jonathan Cameron 2022-04-29 14:41 ` Jonathan Cameron via 2022-04-29 14:41 ` [PATCH v10 37/45] qtests/bios-tables-test: Add a test for CXL emulation Jonathan Cameron 2022-04-29 14:41 ` Jonathan Cameron via 2022-04-29 14:41 ` [PATCH v10 38/45] tests/acpi: Add tables " Jonathan Cameron 2022-04-29 14:41 ` Jonathan Cameron via 2022-04-29 14:41 ` [PATCH v10 39/45] qtest/cxl: Add more complex test cases with CFMWs Jonathan Cameron 2022-04-29 14:41 ` Jonathan Cameron via 2022-04-29 14:41 ` [PATCH v10 40/45] hw/arm/virt: Basic CXL enablement on pci_expander_bridge instances pxb-cxl Jonathan Cameron 2022-04-29 14:41 ` Jonathan Cameron via 2022-04-29 14:41 ` [PATCH v10 41/45] qtest/cxl: Add aarch64 virt test for CXL Jonathan Cameron 2022-04-29 14:41 ` Jonathan Cameron via 2022-04-29 14:41 ` [PATCH v10 42/45] docs/cxl: Add initial Compute eXpress Link (CXL) documentation Jonathan Cameron 2022-04-29 14:41 ` Jonathan Cameron via 2022-04-29 14:41 ` [PATCH v10 43/45] pci-bridge/cxl_upstream: Add a CXL switch upstream port Jonathan Cameron 2022-04-29 14:41 ` Jonathan Cameron via 2022-04-29 14:41 ` [PATCH v10 44/45] pci-bridge/cxl_downstream: Add a CXL switch downstream port Jonathan Cameron 2022-04-29 14:41 ` Jonathan Cameron via 2022-04-29 14:41 ` [PATCH v10 45/45] docs/cxl: Add switch documentation Jonathan Cameron 2022-04-29 14:41 ` Jonathan Cameron via 2022-05-13 10:16 ` [PATCH v10 00/45] CXl 2.0 emulation Support Michael S. Tsirkin
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20220429144110.25167-18-Jonathan.Cameron@huawei.com \ --to=jonathan.cameron@huawei.com \ --cc=a.manzanares@samsung.com \ --cc=alex.bennee@linaro.org \ --cc=alison.schofield@intel.com \ --cc=armbru@redhat.com \ --cc=ben.widawsky@intel.com \ --cc=cbrowy@avery-design.com \ --cc=dan.j.williams@intel.com \ --cc=dave@stgolabs.net \ --cc=david@redhat.com \ --cc=f4bug@amsat.org \ --cc=imammedo@redhat.com \ --cc=k.jensen@samsung.com \ --cc=linux-cxl@vger.kernel.org \ --cc=linuxarm@huawei.com \ --cc=marcel@redhat.com \ --cc=mark.cave-ayland@ilande.co.uk \ --cc=mst@redhat.com \ --cc=pbonzini@redhat.com \ --cc=peter.maydell@linaro.org \ --cc=peterx@redhat.com \ --cc=qemu-devel@nongnu.org \ --cc=samarths@cadence.com \ --cc=saransh@ibm.com \ --cc=shameerali.kolothum.thodi@huawei.com \ --cc=shreyas.shah@elastics.cloud \ --cc=ztong0001@gmail.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.