Re: [PATCH v2] mm: Only re-generate demotion targets when a numa node changes its N_CPU state

From: "Huang, Ying" <ying.huang@intel.com>
To: Oscar Salvador <osalvador@suse.de>
Cc: Dave Hansen <dave.hansen@intel.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Abhishek Goel <huntbag@linux.vnet.ibm.com>,
	Baolin Wang <baolin.wang@linux.alibaba.com>,
	linux-mm@kvack.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2] mm: Only re-generate demotion targets when a numa node changes its N_CPU state
Date: Tue, 15 Mar 2022 14:31:34 +0800	[thread overview]
Message-ID: <87tubz3ewp.fsf@yhuang6-desk2.ccr.corp.intel.com> (raw)
In-Reply-To: <YjAukR2aPkZ0z7Z9@localhost.localdomain> (Oscar Salvador's message of "Tue, 15 Mar 2022 07:13:37 +0100")

Oscar Salvador <osalvador@suse.de> writes:

> On Mon, Mar 14, 2022 at 08:20:57AM -0700, Dave Hansen wrote:
>> Qemu, for instance, has a "mem-path" argument.  It's typically used for
>> using hugetlbfs as guest memory.  But, there's nothing stopping you from
>> pointing it to a DAX device or a file on a DAX filesystem that's backed
>> by pmem.
>
> Thanks Dave.
>
> But that is somehow different, is not it?
> When you use pmem backed memory as a RAM for the guest, the guest is not
> seeing that as PMEM, but just as a normal RAM, right?
> IOW, the guest cannot use that memory for demotion, as we can use it in
> the host when configured.
>
> I might be missing something, I am using this qemu cmdline:
>
>         $QEMU -enable-kvm -machine pc -smp 4 -cpu host -monitor pty -m 5G \
> 	-object memory-backend-file,id=pc.ram,size=5G,mem-path=/mnt/pmem,share=off -machine memory-backend=pc.ram \
> 	$IMAGE -boot c -vnc :0 
>
> (/mnt/pmem was mounted with "mount -o dax /dev/pmem1 /mnt/pmem/")
>
> My point is, if it is really true that the guest cannot use that memory for
> demotion, then we would still need CONFIG_MEMORY_HOTPLUG, as that is the
> only way to expose PMEM to any system to be used as a demotion option
> (via add_memory_driver_managed() through kmem driver).
>
> Or am I missing some qemu magic to use that memory as demotion in the
> guest as well?

You need to put PMEM to a NUMA node to use demotion, as follows,

qemu-system-x86_64 --enable-kvm \
-M pc,accel=kvm,nvdimm=on -smp 8 -m 160G,slots=18,maxmem=703G \
-object memory-backend-ram,id=mem0,size=32G \
-object memory-backend-file,id=mem1,share=on,mem-path=/dev/dax0.0,size=128G,align=2M \
-numa node,memdev=mem0,cpus=0-7,nodeid=0 \
-numa node,memdev=mem1,nodeid=1 \
$IMAGE

Best Regards,
Huang, Ying