From: "Huang, Ying" <ying.huang@intel.com>
To: "Ho-Ren (Jack) Chuang" <horenchuang@bytedance.com>
Cc: Gregory Price <gourry.memverge@gmail.com>,
aneesh.kumar@linux.ibm.com, mhocko@suse.com, tj@kernel.org,
john@jagalactic.com, Eishan Mirakhur <emirakhur@micron.com>,
Vinicius Tavares Petrucci <vtavarespetr@micron.com>,
Ravis OpenSrc <Ravis.OpenSrc@micron.com>,
Alistair Popple <apopple@nvidia.com>,
"Rafael J. Wysocki" <rafael@kernel.org>,
Len Brown <lenb@kernel.org>,
Andrew Morton <akpm@linux-foundation.org>,
Dave Jiang <dave.jiang@intel.com>,
Dan Williams <dan.j.williams@intel.com>,
Jonathan Cameron <Jonathan.Cameron@huawei.com>,
linux-acpi@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-mm@kvack.org, "Ho-Ren (Jack) Chuang" <horenc@vt.edu>,
"Ho-Ren (Jack) Chuang" <horenchuang@gmail.com>,
linux-cxl@vger.kernel.org, qemu-devel@nongnu.org
Subject: Re: [External] Re: [PATCH v1 0/1] Improved Memory Tier Creation for CPUless NUMA Nodes
Date: Tue, 05 Mar 2024 14:34:36 +0800 [thread overview]
Message-ID: <87h6hl9og3.fsf@yhuang6-desk2.ccr.corp.intel.com> (raw)
In-Reply-To: <CAKPbEqr-0yPDW7qps24vJgVCtVOGy_Jm4kcc0FKUsL3d9APDsw@mail.gmail.com> (Ho-Ren Chuang's message of "Mon, 4 Mar 2024 22:22:59 -0800")
"Ho-Ren (Jack) Chuang" <horenchuang@bytedance.com> writes:
> On Sun, Mar 3, 2024 at 6:47 PM Huang, Ying <ying.huang@intel.com> wrote:
>>
>> "Ho-Ren (Jack) Chuang" <horenchuang@bytedance.com> writes:
>>
>> > The memory tiering component in the kernel is functionally useless for
>> > CPUless memory/non-DRAM devices like CXL1.1 type3 memory because the nodes
>> > are lumped together in the DRAM tier.
>> > https://lore.kernel.org/linux-mm/PH0PR08MB7955E9F08CCB64F23963B5C3A860A@PH0PR08MB7955.namprd08.prod.outlook.com/T/
>>
>> I think that it's unfair to call it "useless". Yes, it doesn't work if
>> the CXL memory device are not enumerate via drivers/dax/kmem.c. So,
>> please be specific about in which cases it doesn't work instead of too
>> general "useless".
>>
>
> Thank you and I didn't mean anything specific. I simply reused phrases
> we discussed
> earlier in the previous patchset. I will change them to the following in v2:
> "At boot time, current memory tiering assigns all detected memory nodes
> to the same DRAM tier. This results in CPUless memory/non-DRAM devices,
> such as CXL1.1 type3 memory, being unable to be assigned to the
> correct memory tier,
> leading to the inability to migrate pages between different types of memory."
>
> Please see if this looks more specific.
I don't think that the description above is accurate. In fact, there
are 2 ways to enumerate the memory device,
1. Mark it as reserved memory (E820_TYPE_SOFT_RESERVED, etc.) in E820
table or something similar.
2. Mark it as normal memory (E820_TYPE_RAM) in E820 table or something
similar
For 1, the memory device (including CXL memory) is onlined via
drivers/dax/kmem.c, so will be put in proper memory tiers. For 2, the
memory device is indistinguishable with normal DRAM with current
implementation. And this is what this patch is working on.
Right?
--
Best Regards,
Huang, Ying
>> > This patchset automatically resolves the issues. It delays the initialization
>> > of memory tiers for CPUless NUMA nodes until they obtain HMAT information
>> > at boot time, eliminating the need for user intervention.
>> > If no HMAT specified, it falls back to using `default_dram_type`.
>> >
>> > Example usecase:
>> > We have CXL memory on the host, and we create VMs with a new system memory
>> > device backed by host CXL memory. We inject CXL memory performance attributes
>> > through QEMU, and the guest now sees memory nodes with performance attributes
>> > in HMAT. With this change, we enable the guest kernel to construct
>> > the correct memory tiering for the memory nodes.
>> >
>> > Ho-Ren (Jack) Chuang (1):
>> > memory tier: acpi/hmat: create CPUless memory tiers after obtaining
>> > HMAT info
>> >
>> > drivers/acpi/numa/hmat.c | 3 ++
>> > include/linux/memory-tiers.h | 6 +++
>> > mm/memory-tiers.c | 76 ++++++++++++++++++++++++++++++++----
>> > 3 files changed, 77 insertions(+), 8 deletions(-)
>>
>> --
>> Best Regards,
>> Huang, Ying
next prev parent reply other threads:[~2024-03-05 6:36 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-01 8:22 [PATCH v1 0/1] Improved Memory Tier Creation for CPUless NUMA Nodes Ho-Ren (Jack) Chuang
2024-03-01 8:22 ` [PATCH v1 1/1] memory tier: acpi/hmat: create CPUless memory tiers after obtaining HMAT info Ho-Ren (Jack) Chuang
2024-03-04 2:40 ` Huang, Ying
2024-03-05 9:28 ` [External] " Ho-Ren (Jack) Chuang
2024-03-06 2:25 ` Huang, Ying
2024-03-06 3:55 ` Ho-Ren (Jack) Chuang
2024-03-04 2:45 ` [PATCH v1 0/1] Improved Memory Tier Creation for CPUless NUMA Nodes Huang, Ying
2024-03-05 6:22 ` [External] " Ho-Ren (Jack) Chuang
2024-03-05 6:34 ` Huang, Ying [this message]
2024-03-05 7:10 ` Ho-Ren (Jack) Chuang
2024-03-05 7:37 ` Huang, Ying
2024-03-04 3:07 ` fan
2024-03-04 8:40 ` [EXT] " Srinivasulu Opensrc
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87h6hl9og3.fsf@yhuang6-desk2.ccr.corp.intel.com \
--to=ying.huang@intel.com \
--cc=Jonathan.Cameron@huawei.com \
--cc=Ravis.OpenSrc@micron.com \
--cc=akpm@linux-foundation.org \
--cc=aneesh.kumar@linux.ibm.com \
--cc=apopple@nvidia.com \
--cc=dan.j.williams@intel.com \
--cc=dave.jiang@intel.com \
--cc=emirakhur@micron.com \
--cc=gourry.memverge@gmail.com \
--cc=horenc@vt.edu \
--cc=horenchuang@bytedance.com \
--cc=horenchuang@gmail.com \
--cc=john@jagalactic.com \
--cc=lenb@kernel.org \
--cc=linux-acpi@vger.kernel.org \
--cc=linux-cxl@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@suse.com \
--cc=qemu-devel@nongnu.org \
--cc=rafael@kernel.org \
--cc=tj@kernel.org \
--cc=vtavarespetr@micron.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).