[Qemu-devel] Memory Hotplug : Does QEmu support cold-plugged memory?

[Qemu-devel] Memory Hotplug : Does QEmu support cold-plugged memory?

[Dou Liyang]

Hi all,

I want to boot up a system with some removable memory.

So, i used '-object' and '-device' to cold-plug memory
in QEmu as following :

./x86_64-softmmu/qemu-system-x86_64 \
	...
	-object memory-backend-ram,id=mem3,size=128M \
	-device pc-dimm,id=dimm3,memdev=mem3 \
	...

then i want to hot-remove this memory by 'device-del' and
'object-del'.

But, unfortunately, I can't remove the memory device.

And, in guest OS, I found the memory's removable feature is 0,
But, in QEmu monitor with 'info memory-devices', I found the
hotplugged is false and hotpluggable is true.

Could you help me:

1, Can the ram based memory-backend be plugged like above?

2, Does the 'info memory-devices' command show a wrong info?

3, Can file memory backend(memory-backend-file) be cold-plugged?


PS: here is my whole script:
----
gdb --arg ./x86_64-softmmu/qemu-system-x86_64 \
	-hda /image/fedora.img \
	-m 256M,slots=4,maxmem=1G \
	-enable-kvm \
	-smp 2,maxcpus=16,sockets=2,cores=2,threads=2\
	-object memory-backend-ram,id=mem1,size=128M \
	-object memory-backend-ram,id=mem2,size=128M \
	-object memory-backend-ram,id=mem3,size=128M \
	-device pc-dimm,id=dimm3,memdev=mem3 \
	-device qemu64-x86_64-cpu,id=cpu2,socket-id=0,core-id=1,thread-id=0 \
	-device qemu64-x86_64-cpu,id=cpu3,socket-id=0,core-id=1,thread-id=1 \
	-numa node,memdev=mem1,nodeid=0 \
	-numa node,memdev=mem2,nodeid=1 \
	-serial stdio \
	-monitor telnet:127.0.0.1:4444,server,nowait \
	-kernel /home/douly/openSource/orignal_linux/arch/x86_64/boot/bzImage \
	-append "root=/dev/mapper/fedora_s3101490-root mem=256M movable_node 
nokaslr earlyprintk=serial,ttyS0,115200" \
	-initrd /home/douly/openSource/initramfs_image/4.12.0-rc3.img \
----

Thanks,
	dou.

Re: [Qemu-devel] Memory Hotplug : Does QEmu support cold-plugged memory?

[Igor Mammedov]

On Mon, 5 Jun 2017 15:53:02 +0800
Dou Liyang <douly.fnst@cn.fujitsu.com> wrote:

> Hi all,
> 
> I want to boot up a system with some removable memory.
> 
> So, i used '-object' and '-device' to cold-plug memory
> in QEmu as following :
> 
> ./x86_64-softmmu/qemu-system-x86_64 \
> 	...
> 	-object memory-backend-ram,id=mem3,size=128M \
> 	-device pc-dimm,id=dimm3,memdev=mem3 \
> 	...
> 
> then i want to hot-remove this memory by 'device-del' and
> 'object-del'.
> 
> But, unfortunately, I can't remove the memory device.
> 
> And, in guest OS, I found the memory's removable feature is 0,
> But, in QEmu monitor with 'info memory-devices', I found the
> hotplugged is false and hotpluggable is true.
It's issue in the guest side kernel.
In short it's not guarantied that guest kernel would be able
to hotremove DIMM module (even if hardware is capable of) since
kernel might use its memory as not MOVABLE, so DIMM might be
temporally or permanently pinned by kernel allocations.

There is ongoing refactoring/cleanup work upstream, that
tries to make memhotplug behavior consistent/reasonable.

If you are interested, I'd suggest to review/test changes that
Michal works on upstream, to make sure that refactoring won't
break your usecase.

You can find the latest series Michal posted at:
http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1410289.html


> 
> Could you help me:
> 
> 1, Can the ram based memory-backend be plugged like above?
yes, it can be used as above
 
> 2, Does the 'info memory-devices' command show a wrong info?
nope,
command shows hotpluggable status the way hw (qemu) sees it

> 3, Can file memory backend(memory-backend-file) be cold-plugged?
yes, it can.

[...]

Re: [Qemu-devel] Memory Hotplug : Does QEmu support cold-plugged memory?

[Michal Hocko]

On Mon 05-06-17 10:45:08, Igor Mammedov wrote:
> On Mon, 5 Jun 2017 15:53:02 +0800
> Dou Liyang <douly.fnst@cn.fujitsu.com> wrote:
> 
> > Hi all,
> > 
> > I want to boot up a system with some removable memory.
> > 
> > So, i used '-object' and '-device' to cold-plug memory
> > in QEmu as following :
> > 
> > ./x86_64-softmmu/qemu-system-x86_64 \
> > 	...
> > 	-object memory-backend-ram,id=mem3,size=128M \
> > 	-device pc-dimm,id=dimm3,memdev=mem3 \
> > 	...
> > 
> > then i want to hot-remove this memory by 'device-del' and
> > 'object-del'.
> > 
> > But, unfortunately, I can't remove the memory device.
> > 
> > And, in guest OS, I found the memory's removable feature is 0,
> > But, in QEmu monitor with 'info memory-devices', I found the
> > hotplugged is false and hotpluggable is true.
> It's issue in the guest side kernel.
> In short it's not guarantied that guest kernel would be able
> to hotremove DIMM module (even if hardware is capable of) since
> kernel might use its memory as not MOVABLE, so DIMM might be
> temporally or permanently pinned by kernel allocations.

Yes and that will be always the case as long as you allow kernel
allocations to use that memory. I do not know of any other way to work
this around than online the specific memory range as movable.

-- 
Michal Hocko
SUSE Labs

Re: [Qemu-devel] Memory Hotplug : Does QEmu support cold-plugged memory?

[Dou Liyang]

Hi Igor,

At 06/05/2017 04:45 PM, Igor Mammedov wrote:
> On Mon, 5 Jun 2017 15:53:02 +0800
> Dou Liyang <douly.fnst@cn.fujitsu.com> wrote:
>
>> Hi all,
>>
>> I want to boot up a system with some removable memory.
>>
>> So, i used '-object' and '-device' to cold-plug memory
>> in QEmu as following :
>>
>> ./x86_64-softmmu/qemu-system-x86_64 \
>> 	...
>> 	-object memory-backend-ram,id=mem3,size=128M \
>> 	-device pc-dimm,id=dimm3,memdev=mem3 \
>> 	...
>>
>> then i want to hot-remove this memory by 'device-del' and
>> 'object-del'.
>>
>> But, unfortunately, I can't remove the memory device.
>>
>> And, in guest OS, I found the memory's removable feature is 0,
>> But, in QEmu monitor with 'info memory-devices', I found the
>> hotplugged is false and hotpluggable is true.
> It's issue in the guest side kernel.
> In short it's not guarantied that guest kernel would be able
> to hotremove DIMM module (even if hardware is capable of) since
> kernel might use its memory as not MOVABLE, so DIMM might be
> temporally or permanently pinned by kernel allocations.
>

I see.

> There is ongoing refactoring/cleanup work upstream, that
> tries to make memhotplug behavior consistent/reasonable.
>
> If you are interested, I'd suggest to review/test changes that
> Michal works on upstream, to make sure that refactoring won't
> break your usecase.

OK, I will.

>
> You can find the latest series Michal posted at:
> http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1410289.html
>

um, I found the patchset, and I am reading the discussions and test
cases you gave in v2.

>
>>
>> Could you help me:
>>
>> 1, Can the ram based memory-backend be plugged like above?
> yes, it can be used as above
>
>> 2, Does the 'info memory-devices' command show a wrong info?
> nope,
> command shows hotpluggable status the way hw (qemu) sees it
>
>> 3, Can file memory backend(memory-backend-file) be cold-plugged?
> yes, it can.
>
> [...]
>

Thanks a lot ;)

Thanks,
	dou.

>
>