From: "Philippe Mathieu-Daudé" <f4bug@amsat.org>
To: Ben Widawsky <ben@bwidawsk.net>, qemu-devel@nongnu.org
Cc: Vishal Verma <vishal.l.verma@intel.com>,
Dan Williams <dan.j.williams@intel.com>,
"Michael S. Tsirkin" <mst@redhat.com>
Subject: Re: [RFC PATCH 16/25] hw/pxb/cxl: Add "windows" for host bridges
Date: Mon, 23 Nov 2020 20:12:01 +0100 [thread overview]
Message-ID: <c51b000e-80db-40e9-d878-f260c49e4a2e@amsat.org> (raw)
In-Reply-To: <20201113004952.tm45qv2tukwnle52@mail.bwidawsk.net>
On 11/13/20 1:49 AM, Ben Widawsky wrote:
> On 20-11-10 21:47:15, Ben Widawsky wrote:
>> In a bare metal CXL capable system, system firmware will program
>> physical address ranges on the host. This is done by programming
>> internal registers that aren't typically known to OS. These address
>> ranges might be contiguous or interleaved across host bridges.
>>
>> For a QEMU guest a new construct is introduced allowing passing a memory
>> backend to the host bridge for this same purpose. Each memory backend
>> needs to be passed to the host bridge as well as any device that will be
>> emulating that memory (not implemented here).
>>
>> I'm hopeful the interleaving work in the link can be re-purposed here
>> (see Link).
>>
>> An example to create a host bridges with a 512M window at 0x4c0000000
>> -object memory-backend-file,id=cxl-mem1,share,mem-path=cxl-type3,size=512M
>> -device pxb-cxl,id=cxl.0,bus=pcie.0,bus_nr=52,uid=0,len-memory-base=1,memory-base\[0\]=0x4c0000000,memory\[0\]=cxl-mem1
>>
>> Link: https://lists.nongnu.org/archive/html/qemu-devel/2020-08/msg03680.html
>> Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
>
> Hi Phil, wanted to call you out specifically on this one.
>
> The newly released CXL 2.0 specification (which from a topology perspective can
> be thought of as very PCIe-like) allows for interleaving of memory access.
>
> Below is an example of two host bridges, each with two root ports, and 5 devices
> (two of switch are behind a switch).
>
> RP: Root Port
> USP: Upstream Port
> DSP: Downstream Port
> Type 3 device: Memory Device with persistent or volatile memory.
>
> +-------------------------+ +-------------------------+
> | | | |
> | CXL 2.0 Host Bridge | | CXL 2.0 Host Bridge |
> | | | |
> | +------+ +------+ | | +------+ +------+ |
> | | RP | | RP | | | | RP | | RP | |
> +--+------+-----+------+--+ +--+------+-----+------+--+
> | | | \--
> | | | +-------+-\--+------+
> +------+ +-------+ +-------+ | |USP | |
> |Type 3| |Type 3 | |Type 3 | | +----+ |
> |Device| |Device | |Device | | |
> +------+ +-------+ +-------+ | +----+ +----+ |
> | |DSP | |DSP | |
> +-+----+-----+----+-+
> | |
> +------+ +-------+
> |Type 3| |Type 3 |
> |Device| |Device |
> +------+ +-------+
>
> Considering this picture... interleaving of memory access can happen in all 3
> layers in the topology.
>
> - Memory access can be interleaved across host bridges (this is accomplished
> based on the physical address chosen for the devices, those address ranges are
> platform specific and not part of the 2.0 spec, for now).
>
> - Memory access can be interleaved across root ports in a host bridge.
>
> - Finally, memory access can be interleaved across downstream ports.
>
> I'd like to start the discussion about how this might overlap with the patch
> series you've last been working on to interleave memory. Do you have any
> thoughts or ideas on how I should go about doing this?
Main case:
+-------------------------+
| |
| CXL 2.0 Host Bridge |
| |
| +------+ +------+ |
| | RP | | RP | |
+--+------+-----+------+--+
| |
| |
+------+ +-------+
|Type 3| |Type 3 |
|Device| |Device |
+------+ +-------+
// cxl device state
s = qdev_create(TYPE_CXL20_HB_DEV)
cxl_memsize = 2 * memsize(Type3Dev);
// container for cxl
memory_region_init(&s->container, OBJECT(s),
"container", cxl_memsize);
// create 2 slots, interleaved each 2k
s->interleaver = qdev_create(INTERLEAVER_DEV,
slotsize=2k,
max_slots=2)
qdev_prop_set_uint64(s->interleaver, "size",
cxl_memsize);
// connect each device to the interleaver
object_property_set_link(OBJECT(interleaver),
"mr0", OBJECT(RP0))
object_property_set_link(OBJECT(interleaver),
"mr1", OBJECT(RP1))
sysbus_realize_and_unref(SYS_BUS_DEVICE(interleaver))
// we can probably avoid this container
memory_region_add_subregion(&s->container, 0,
sysbus_mmio_get_region(interleaver, 0));
For the 2nd case, USP can be created the same way than case 1
(as a 2nd interleaver) then the main CXL is created with the
minor difference of mr1 now being the USP:
object_property_set_link(OBJECT(interleaver),
"mr1", OBJECT(USP))
sysbus_realize_and_unref(SYS_BUS_DEVICE(interleaver))
Regards,
Phil.
next prev parent reply other threads:[~2020-11-23 19:13 UTC|newest]
Thread overview: 64+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-11 5:46 [RFC PATCH 00/25] Introduce CXL 2.0 Emulation Ben Widawsky
2020-11-11 5:47 ` [RFC PATCH 01/25] Temp: Add the PCI_EXT_ID_DVSEC definition to the qemu pci_regs.h copy Ben Widawsky
2020-11-11 5:47 ` [RFC PATCH 02/25] hw/pci/cxl: Add a CXL component type (interface) Ben Widawsky
2020-11-11 5:47 ` [RFC PATCH 03/25] hw/cxl/component: Introduce CXL components (8.1.x, 8.2.5) Ben Widawsky
2020-11-16 12:03 ` Jonathan Cameron
2020-11-16 19:19 ` Ben Widawsky
2020-11-17 12:29 ` Jonathan Cameron
2020-11-24 23:09 ` Ben Widawsky
2020-11-11 5:47 ` [RFC PATCH 04/25] hw/cxl/device: Introduce a CXL device (8.2.8) Ben Widawsky
2020-11-16 13:07 ` Jonathan Cameron
2020-11-16 21:11 ` Ben Widawsky
2020-11-17 14:21 ` Jonathan Cameron
2020-11-11 5:47 ` [RFC PATCH 05/25] hw/cxl/device: Implement the CAP array (8.2.8.1-2) Ben Widawsky
2020-11-16 13:11 ` Jonathan Cameron
2020-11-16 18:08 ` Ben Widawsky
2020-11-11 5:47 ` [RFC PATCH 06/25] hw/cxl/device: Add device status (8.2.8.3) Ben Widawsky
2020-11-16 13:16 ` Jonathan Cameron
2020-11-16 21:18 ` Ben Widawsky
2020-11-17 14:24 ` Jonathan Cameron
2020-11-11 5:47 ` [RFC PATCH 07/25] hw/cxl/device: Implement basic mailbox (8.2.8.4) Ben Widawsky
2020-11-16 13:46 ` Jonathan Cameron
2020-11-16 21:42 ` Ben Widawsky
2020-11-11 5:47 ` [RFC PATCH 08/25] hw/cxl/device: Add memory devices (8.2.8.5) Ben Widawsky
2020-11-16 16:37 ` Jonathan Cameron
2020-11-16 21:45 ` Ben Widawsky
2020-11-17 14:31 ` Jonathan Cameron
2020-11-11 5:47 ` [RFC PATCH 09/25] hw/pxb: Use a type for realizing expanders Ben Widawsky
2020-11-11 5:47 ` [RFC PATCH 10/25] hw/pci/cxl: Create a CXL bus type Ben Widawsky
2020-11-11 5:47 ` [RFC PATCH 11/25] hw/pxb: Allow creation of a CXL PXB (host bridge) Ben Widawsky
2020-11-16 16:44 ` Jonathan Cameron
2020-11-16 22:01 ` Ben Widawsky
2020-11-17 14:33 ` Jonathan Cameron
2020-11-11 5:47 ` [RFC PATCH 12/25] acpi/pci: Consolidate host bridge setup Ben Widawsky
2020-11-12 17:46 ` Ben Widawsky
2020-11-16 16:45 ` Jonathan Cameron
2020-11-11 5:47 ` [RFC PATCH 13/25] hw/pci: Plumb _UID through host bridges Ben Widawsky
2020-11-11 5:47 ` [RFC PATCH 14/25] hw/cxl/component: Implement host bridge MMIO (8.2.5, table 142) Ben Widawsky
2020-11-11 5:47 ` [RFC PATCH 15/25] acpi/pxb/cxl: Reserve host bridge MMIO Ben Widawsky
2020-11-16 16:54 ` Jonathan Cameron
2020-11-11 5:47 ` [RFC PATCH 16/25] hw/pxb/cxl: Add "windows" for host bridges Ben Widawsky
2020-11-13 0:49 ` Ben Widawsky
2020-11-23 19:12 ` Philippe Mathieu-Daudé [this message]
2020-11-11 5:47 ` [RFC PATCH 17/25] hw/cxl/rp: Add a root port Ben Widawsky
2020-11-11 5:47 ` [RFC PATCH 18/25] hw/cxl/device: Add a memory device (8.2.8.5) Ben Widawsky
2020-11-12 18:37 ` Eric Blake
2020-11-13 7:47 ` Markus Armbruster
2020-11-25 16:53 ` Ben Widawsky
2020-11-26 6:36 ` Markus Armbruster
2020-11-30 17:07 ` Ben Widawsky
2020-12-01 17:06 ` Markus Armbruster
2020-11-11 5:47 ` [RFC PATCH 19/25] hw/cxl/device: Implement MMIO HDM decoding (8.2.5.12) Ben Widawsky
2020-11-11 5:47 ` [RFC PATCH 20/25] acpi/cxl: Add _OSC implementation (9.14.2) Ben Widawsky
2020-11-11 5:47 ` [RFC PATCH 21/25] acpi/cxl: Introduce a compat-driver UUID for CXL _OSC Ben Widawsky
2020-11-11 5:47 ` [RFC PATCH 22/25] acpi/cxl: Create the CEDT (9.14.1) Ben Widawsky
2020-11-16 17:15 ` Jonathan Cameron
2020-11-16 22:05 ` Ben Widawsky
2020-11-11 5:47 ` [RFC PATCH 23/25] Temp: acpi/cxl: Add ACPI0017 (CEDT awareness) Ben Widawsky
2020-11-11 5:47 ` [RFC PATCH 24/25] WIP: i386/cxl: Initialize a host bridge Ben Widawsky
2020-11-11 5:47 ` [RFC PATCH 25/25] qtest/cxl: Add very basic sanity tests Ben Widawsky
2020-11-16 17:21 ` [RFC PATCH 00/25] Introduce CXL 2.0 Emulation Jonathan Cameron
2020-11-16 18:06 ` Ben Widawsky
2020-11-17 14:09 ` Jonathan Cameron
2020-11-25 18:29 ` Ben Widawsky
2020-12-04 14:27 ` Daniel P. Berrangé
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c51b000e-80db-40e9-d878-f260c49e4a2e@amsat.org \
--to=f4bug@amsat.org \
--cc=ben@bwidawsk.net \
--cc=dan.j.williams@intel.com \
--cc=mst@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=vishal.l.verma@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).