Re: CXL 2.0 memory device design

[Ben Widawsky <ben@bwidawsk.net>]

On 21-03-17 14:40:58, Ben Widawsky wrote:
> Phil, Igor, Markus
> 
> TL;DR: What to do about multiple capacities in a single device, and what to do
> about interleave?
> 
> I've hacked together a basic CXL 2.0 implementation which exposes a CXL "Type 3"
> memory device (CXL 2.0 Chapter 2.3). For what we were trying to do this was
> sufficient. There are two main capabilities that CXL spec exposes which I've not
> implemented that I'd like to start working toward and am realizing that I what I
> have so far might not be able to carry forward to that next milestone.
> 
> Capability 1. A CXL memory device may have both a volatile, and a persistent
> 	      capacity. https://bwidawsk.net/HDM_decoders.svg (lower right
> 	      side). The current work only supports a single persistent
> 	      capacity.
> Capability 2. CXL topologies can be interleaved. Basic example:
>               https://bwidawsk.net/HDM_decoders.svg (lower left side)
> 
> Memory regions are configured via a CXL spec defined HDM decoder. The HDM
> decoder which is minimally implemented supports all the functionality mentioned
> above (base, size, interleave, type, etc.). A CXL component may have up to 10
> HDMs.
> 
> What I have today: https://bwidawsk.net/QEMU_objects.svg
> There's a single memory backend device for each host bridge. That backend is
> passed to any CXL component that is part of the hierarchy underneath that
> hostbridge. In the case of a Type 3 device memory capacity a subregion is
> created for that capacity from within the main backend. The device itself
> implements the TYPE_MEMORY_DEVICE interface. This allows me to utilize the
> existing memory region code to determine the next free address, and warn on
> overlaps. It hopefully will help when I'm ready to support hotplug.
> 
> Where I've gotten stuck: A Memory Device expects only to have one region of
> memory. Trying to add a second breaks pretty much everything.
> 
> I'm hoping to start the discussion about what the right way to emulate this in
> QEMU. Ideally something upstreamable would be great. I think adding a secondary
> (or more) capacity to a memory class device is doable, but probably not the
> right approach.
> 
> For context, I've posted v3 previously. Here's a link to v4 which has some minor
> changes as well as moving back to using subregions instead of aliases:
> https://gitlab.com/bwidawsk/qemu/-/tree/cxl-2.0v4
> 
> Thanks.
> Ben
> 

Hello.

I spent some time thinking a bit about this. Right now a have a CXL type 3
memory device which implements TYPE_MEMORY_DEVICE interface. Perhaps the easiest
solution would be to have that same device which doesn't implement
TYPE_MEMORY_DEVICE, but does object_initialize_child_with_props() a TYPE_PC_DIMM
and a TYPE_NVDIMM kind of thing. In the current design, those would be
subclassed (or simply rewritten) to not have their own memory backend, and carve
out from the main memory backend as I describe above.

Thanks. I'm looking forward to hearing some feedback or other suggestions.
Ben