qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: Ben Widawsky <ben.widawsky@intel.com>
To: qemu-devel@nongnu.org
Cc: Igor Mammedov <imammedo@redhat.com>,
	Ben Widawsky <ben.widawsky@intel.com>,
	Chris Browy <cbrowy@avery-design.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Jonathan Cameron <Jonathan.Cameron@huawei.com>
Subject: [RFC PATCH] hw/mem/cxl_type3: Go back to subregions
Date: Thu, 11 Mar 2021 15:27:16 -0800	[thread overview]
Message-ID: <20210311232716.3914084-1-ben.widawsky@intel.com> (raw)
In-Reply-To: <20210202005948.241655-1-ben.widawsky@intel.com>

Each device allocates its memory (persistent only for now) out of a
container memory that represents a "window" that would be defined by the
host bridge. For example, a host bridge may claim all traffic from 0x0 -
0x4000; it might then also direct 0x1000-0x1fff to a specific CXL
device. Change the memory region type claiming 0x1000-0x1fff from an
alias, to a subregion.

v1 and v2 of the patch series both used a subregion. There were two
issues with this and so for v3, an alias was chosen mimicking nvdimm.

The switch to alias left an issue in the implementation. There's logic
that check to make sure two memory regions (ie. two devices under the
same host bridge) couldn't collide. While hardware doesn't make this
guarantee, it's nice for driver debug. There is no clean way to do that
with an alias.

More importantly, this change was inspired by implementing support for
both volatile and persistent memory. In that case, you may have multiple
devices which the BIOS is going to assign address ranges to. Since we
are the BIOS in this case, having a way of finding used space in the
memory window so that you can allocate the next chunk is easily
accomplished here.

With this, I have the following output from `info mtree`
    0000004c00000000-0000004c1fffffff (prio 0, ram): cxl-mem1
      0000004c00000000-0000004c0fffffff (prio 1, i/o): cxl_type3-memory

And `info memory-devices`
Memory device [cxl]: "cxl-pmem0"
  addr: 0x4c00000000
  slot: 0
  node: 0
  size: 268435456
  memdev: /objects/cxl-mem1
  hotplugged: false
  hotpluggable: true

This functionality has been tested with a WIP linux patch which amounts
to this:
       reg = readl(cxlm->regs.hdm_decoder + CXL_HDM_DECODER_CAP_OFFSET);

       dev_err(dev, "decoder cap:\n"
                    "\tcount: %lu\n",
               FIELD_GET(CXL_HDM_DECODER_COUNT_MASK, reg));

       writel(0x4c, cxlm->regs.hdm_decoder + CXL_HDM_DECODER0_BASE_HIGH_OFFSET);
       writel(0, cxlm->regs.hdm_decoder + CXL_HDM_DECODER0_BASE_LOW_OFFSET);
       writel(0, cxlm->regs.hdm_decoder + CXL_HDM_DECODER0_SIZE_HIGH_OFFSET);
       writel(256 << 20, cxlm->regs.hdm_decoder + CXL_HDM_DECODER0_SIZE_LOW_OFFSET);
       writel(BIT(9),  cxlm->regs.hdm_decoder + CXL_HDM_DECODER0_CTRL_OFFSET);

       tmp = ioremap_uc(0x4c00000000, 4096);
       writel(0x20, tmp);

Cc: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
---
 hw/mem/cxl_type3.c | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index bf33ddb915..33991079a6 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -46,7 +46,9 @@ static void build_dvsecs(CXLType3Dev *ct3d)
 static void cxl_set_addr(CXLType3Dev *ct3d, hwaddr addr, Error **errp)
 {
     MemoryDeviceClass *mdc = MEMORY_DEVICE_GET_CLASS(ct3d);
-    mdc->set_addr(MEMORY_DEVICE(ct3d), addr, errp);
+    MemoryRegion *mr = host_memory_backend_get_memory(ct3d->hostmem);
+
+    mdc->set_addr(MEMORY_DEVICE(ct3d), addr - mr->addr, errp);
 }
 
 static void hdm_decoder_commit(CXLType3Dev *ct3d, int which)
@@ -180,13 +182,13 @@ static void cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
 
     memory_region_set_nonvolatile(pmem, true);
     memory_region_set_enabled(pmem, false);
-    memory_region_init_alias(pmem, OBJECT(ct3d), "cxl_type3-memory", mr, 0,
-                             ct3d->size);
+    memory_region_init(pmem, OBJECT(ct3d), "cxl_type3-memory", ct3d->size);
+    memory_region_add_subregion_overlap(mr, 0, pmem, 1);
     ct3d->cxl_dstate.pmem = pmem;
 
 #ifdef SET_PMEM_PADDR
     /* This path will initialize the memory device as if BIOS had done it */
-    cxl_set_addr(ct3d, mr->addr + offset, errp);
+    cxl_set_addr(ct3d, offset, errp);
     memory_region_set_enabled(pmem, true);
 #endif
 }
@@ -246,8 +248,11 @@ static uint64_t cxl_md_get_addr(const MemoryDeviceState *md)
     CXLType3Dev *ct3d = CT3(md);
     MemoryRegion *pmem = ct3d->cxl_dstate.pmem;
 
-    assert(pmem->alias);
-    return pmem->alias_offset;
+    if (pmem) {
+        return pmem->addr + pmem->container->addr;
+    }
+
+    return 0;
 }
 
 static void cxl_md_set_addr(MemoryDeviceState *md, uint64_t addr, Error **errp)
@@ -255,8 +260,6 @@ static void cxl_md_set_addr(MemoryDeviceState *md, uint64_t addr, Error **errp)
     CXLType3Dev *ct3d = CT3(md);
     MemoryRegion *pmem = ct3d->cxl_dstate.pmem;
 
-    assert(pmem->alias);
-    memory_region_set_alias_offset(pmem, addr);
     memory_region_set_address(pmem, addr);
 }
 
-- 
2.30.2



      parent reply	other threads:[~2021-03-11 23:28 UTC|newest]

Thread overview: 65+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-02-02  0:59 [RFC PATCH v3 00/31] CXL 2.0 Support Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 01/31] hw/pci/cxl: Add a CXL component type (interface) Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 02/31] hw/cxl/component: Introduce CXL components (8.1.x, 8.2.5) Ben Widawsky
2021-02-02 11:48   ` Jonathan Cameron
2021-02-17 18:36     ` Ben Widawsky
2021-02-11 17:08   ` Jonathan Cameron
2021-02-17 16:40     ` Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 03/31] hw/cxl/device: Introduce a CXL device (8.2.8) Ben Widawsky
2021-02-02 12:03   ` Jonathan Cameron
2021-02-02  0:59 ` [RFC PATCH v3 04/31] hw/cxl/device: Implement the CAP array (8.2.8.1-2) Ben Widawsky
2021-02-02 12:23   ` Jonathan Cameron
2021-02-17 22:15     ` Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 05/31] hw/cxl/device: Implement basic mailbox (8.2.8.4) Ben Widawsky
2021-02-02 14:58   ` Jonathan Cameron
2021-02-11 17:46     ` Jonathan Cameron
2021-02-18  0:55       ` Ben Widawsky
2021-02-18 16:50         ` Jonathan Cameron
2021-02-11 18:09   ` Jonathan Cameron
2021-02-02  0:59 ` [RFC PATCH v3 06/31] hw/cxl/device: Add memory device utilities Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 07/31] hw/cxl/device: Add cheap EVENTS implementation (8.2.9.1) Ben Widawsky
2021-02-02 13:44   ` Jonathan Cameron
2021-02-11 17:59   ` Jonathan Cameron
2021-02-02  0:59 ` [RFC PATCH v3 08/31] hw/cxl/device: Timestamp implementation (8.2.9.3) Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 09/31] hw/cxl/device: Add log commands (8.2.9.4) + CEL Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 10/31] hw/pxb: Use a type for realizing expanders Ben Widawsky
2021-02-02 13:50   ` Jonathan Cameron
2021-02-02  0:59 ` [RFC PATCH v3 11/31] hw/pci/cxl: Create a CXL bus type Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 12/31] hw/pxb: Allow creation of a CXL PXB (host bridge) Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 13/31] qtest: allow DSDT acpi table changes Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 14/31] acpi/pci: Consolidate host bridge setup Ben Widawsky
2021-02-02 13:56   ` Jonathan Cameron
2021-12-02 10:32   ` Jonathan Cameron via
2021-02-02  0:59 ` [RFC PATCH v3 15/31] tests/acpi: remove stale allowed tables Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 16/31] hw/pci: Plumb _UID through host bridges Ben Widawsky
2021-02-02 15:00   ` Jonathan Cameron
2021-02-02 15:24     ` Michael S. Tsirkin
2021-02-02 15:42       ` Ben Widawsky
2021-02-02 15:51         ` Michael S. Tsirkin
2021-02-02 16:20           ` Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 17/31] hw/cxl/component: Implement host bridge MMIO (8.2.5, table 142) Ben Widawsky
2021-02-02 19:21   ` Jonathan Cameron
2021-02-02 19:45     ` Ben Widawsky
2021-02-02 20:43       ` Jonathan Cameron
2021-02-02 21:03         ` Ben Widawsky
2021-02-02 22:06           ` Jonathan Cameron
2021-02-02  0:59 ` [RFC PATCH v3 18/31] acpi/pxb/cxl: Reserve host bridge MMIO Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 19/31] hw/pxb/cxl: Add "windows" for host bridges Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 20/31] hw/cxl/rp: Add a root port Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 21/31] hw/cxl/device: Add a memory device (8.2.8.5) Ben Widawsky
2021-02-02 14:26   ` Eric Blake
2021-02-02 15:06     ` Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 22/31] hw/cxl/device: Implement MMIO HDM decoding (8.2.5.12) Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 23/31] acpi/cxl: Add _OSC implementation (9.14.2) Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 24/31] tests/acpi: allow CEDT table addition Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 25/31] acpi/cxl: Create the CEDT (9.14.1) Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 26/31] tests/acpi: Add new CEDT files Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 27/31] hw/cxl/device: Add some trivial commands Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 28/31] hw/cxl/device: Plumb real LSA sizing Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 29/31] hw/cxl/device: Implement get/set LSA Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 30/31] qtest/cxl: Add very basic sanity tests Ben Widawsky
2021-02-02  0:59 ` [RFC PATCH v3 31/31] WIP: i386/cxl: Initialize a host bridge Ben Widawsky
2021-02-02  1:33 ` [RFC PATCH v3 00/31] CXL 2.0 Support no-reply
2021-02-03 17:42 ` Ben Widawsky
2021-02-11 18:51   ` Jonathan Cameron
2021-03-11 23:27 ` Ben Widawsky [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210311232716.3914084-1-ben.widawsky@intel.com \
    --to=ben.widawsky@intel.com \
    --cc=Jonathan.Cameron@huawei.com \
    --cc=cbrowy@avery-design.com \
    --cc=imammedo@redhat.com \
    --cc=mst@redhat.com \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).