[Qemu-devel] Persistent Main Memory in QEmu

[Qemu-devel] Persistent Main Memory in QEmu

[Li, Tianyou]

[-- Attachment #1: Type: text/plain, Size: 727 bytes --]

Hi,

Currently we are trying to implement below functionalities in QEmu: main memory in guest can be logically viewed as persistent and its content can be survived through reboot or shutdown/powerup.

I have looked into the QEmu memory management code include memory.c, exec.c and other related source, unfortunately I do not have the chance to get clue of how to make QEmu main memory persistent. I found that pmemsave<http://doc.opensuse.org/documentation/html/openSUSE_114/opensuse-kvm/cha.qemu.monitor.html#id547777> could dump physical memory of guest, but I could not find how to restore the dump file before VM startup to execution.

Could anyone provide some hints to me? Thanks in advance!

Regards,
Tianyou

[-- Attachment #2: Type: text/html, Size: 2885 bytes --]

Re: [Qemu-devel] Persistent Main Memory in QEmu

[Artyom Tarasenko]

Hi Tianyou,

On Mon, Apr 18, 2016 at 5:50 AM, Li, Tianyou <tianyou.li@intel.com> wrote:
> Currently we are trying to implement below functionalities in QEmu: main
> memory in guest can be logically viewed as persistent and its content can be
> survived through reboot or shutdown/powerup.
>
> I have looked into the QEmu memory management code include memory.c, exec.c
> and other related source, unfortunately I do not have the chance to get clue
> of how to make QEmu main memory persistent. I found that pmemsave could dump
> physical memory of guest, but I could not find how to restore the dump file
> before VM startup to execution.
>
>
>
> Could anyone provide some hints to me? Thanks in advance!

Is the option "-mem-path=/path/to/mem-file" what you are looking for?

Stefan wrote a nice post about QEMU RAM internals:
http://blog.vmsplice.net/2016/01/qemu-internals-how-guest-physical-ram.html

Regards,
Artyom


-- 
Regards,
Artyom Tarasenko

SPARC and PPC PReP under qemu blog: http://tyom.blogspot.com/search/label/qemu

Re: [Qemu-devel] Persistent Main Memory in QEmu

[Li, Tianyou]

Hi Artyom,

Thanks for your pointer! I have tried the -mem-path option, and right now looking into the code to see if the content of the file will be used during the guest Linux running next time. Will let you know the result. 

Stefan's post is definitely helpful, thanks for letting me know. Could you please let me know if you have more QEmu memory management documentation about internals? I have googled around, but more likely to hear from your advice. Thanks.

Regards,
Tianyou

-----Original Message-----
From: Artyom Tarasenko [mailto:atar4qemu@gmail.com] 
Sent: Tuesday, April 19, 2016 4:20 AM
To: Li, Tianyou <tianyou.li@intel.com>
Cc: qemu-devel@nongnu.org
Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu

Hi Tianyou,

On Mon, Apr 18, 2016 at 5:50 AM, Li, Tianyou <tianyou.li@intel.com> wrote:
> Currently we are trying to implement below functionalities in QEmu: 
> main memory in guest can be logically viewed as persistent and its 
> content can be survived through reboot or shutdown/powerup.
>
> I have looked into the QEmu memory management code include memory.c, 
> exec.c and other related source, unfortunately I do not have the 
> chance to get clue of how to make QEmu main memory persistent. I found 
> that pmemsave could dump physical memory of guest, but I could not 
> find how to restore the dump file before VM startup to execution.
>
>
>
> Could anyone provide some hints to me? Thanks in advance!

Is the option "-mem-path=/path/to/mem-file" what you are looking for?

Stefan wrote a nice post about QEMU RAM internals:
http://blog.vmsplice.net/2016/01/qemu-internals-how-guest-physical-ram.html

Regards,
Artyom


--
Regards,
Artyom Tarasenko

SPARC and PPC PReP under qemu blog: http://tyom.blogspot.com/search/label/qemu

Re: [Qemu-devel] Persistent Main Memory in QEmu

[Dr. David Alan Gilbert]

* Li, Tianyou (tianyou.li@intel.com) wrote:
> Hi,
> 
> Currently we are trying to implement below functionalities in QEmu: main memory in guest can be logically viewed as persistent and its content can be survived through reboot or shutdown/powerup.
> 
> I have looked into the QEmu memory management code include memory.c, exec.c and other related source, unfortunately I do not have the chance to get clue of how to make QEmu main memory persistent. I found that pmemsave<http://doc.opensuse.org/documentation/html/openSUSE_114/opensuse-kvm/cha.qemu.monitor.html#id547777> could dump physical memory of guest, but I could not find how to restore the dump file before VM startup to execution.

Can you explain what you mean by 'persistent' - where do you intend
to store the guests memory?
Also, remember that you'll need to save/load the device state
as well as the rest of RAM.  If you've got a way to preserve
RAM then maybe hte xen-save-devices-state qemu command could be
used to store the rest of devices.

Dave

> 
> Could anyone provide some hints to me? Thanks in advance!
> 
> Regards,
> Tianyou
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] Persistent Main Memory in QEmu

[Li, Tianyou]

Hi Dave,

Thanks for your response. Below are my explanations:

> Can you explain what you mean by 'persistent' - where do you intend to store the guests memory?

There could be a file or memory region that can survive across guest shutdown/reboot. Seems Artyom has pointed out the right direction and I have verified by looking into the code throughout the call stack from pc_memory_init to qemu_ram_alloc_from_file. I plan to write something like kernel module to verify the persistency characteristics from guest point of view.

> Also, remember that you'll need to save/load the device state as well as the rest of RAM.  

Device state handling could be done from two different aspects: 1. From host perspective or, 2. From guest perspective. From host, qemu will always know the state of guest devices so that we can use qemu command to checkpoint states, as you point out (very appreciated that, I do not know the command xen-save-devices-state before). From guest, it can be something like suspend to RAM or S3 for PC to checkpoint the current state of PC and restore them when wakeup. Currently I will prefer the #2. 

In summary, I'd like to have the functionality in qemu that can save & restore PC main memory at shutdown/power-on phase. Thanks.

Regards,
Tianyou


-----Original Message-----
From: Dr. David Alan Gilbert [mailto:dgilbert@redhat.com] 
Sent: Wednesday, April 20, 2016 4:36 PM
To: Li, Tianyou <tianyou.li@intel.com>
Cc: qemu-devel@nongnu.org
Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu

* Li, Tianyou (tianyou.li@intel.com) wrote:
> Hi,
> 
> Currently we are trying to implement below functionalities in QEmu: main memory in guest can be logically viewed as persistent and its content can be survived through reboot or shutdown/powerup.
> 
> I have looked into the QEmu memory management code include memory.c, exec.c and other related source, unfortunately I do not have the chance to get clue of how to make QEmu main memory persistent. I found that pmemsave<http://doc.opensuse.org/documentation/html/openSUSE_114/opensuse-kvm/cha.qemu.monitor.html#id547777> could dump physical memory of guest, but I could not find how to restore the dump file before VM startup to execution.

Can you explain what you mean by 'persistent' - where do you intend to store the guests memory?
Also, remember that you'll need to save/load the device state as well as the rest of RAM.  If you've got a way to preserve RAM then maybe hte xen-save-devices-state qemu command could be used to store the rest of devices.

Dave

> 
> Could anyone provide some hints to me? Thanks in advance!
> 
> Regards,
> Tianyou
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] Persistent Main Memory in QEmu

[Dr. David Alan Gilbert]

* Li, Tianyou (tianyou.li@intel.com) wrote:
> Hi Dave,
> 
> Thanks for your response. Below are my explanations:
> 
> > Can you explain what you mean by 'persistent' - where do you intend to store the guests memory?
> 
> There could be a file or memory region that can survive across guest shutdown/reboot. Seems Artyom has pointed out the right direction and I have verified by looking into the code throughout the call stack from pc_memory_init to qemu_ram_alloc_from_file. I plan to write something like kernel module to verify the persistency characteristics from guest point of view.

Maybe it's worth checking the stuff in docs/memory-hotplug.txt - that shows how to create a
memory region backed by a file (in that case using a hugepagefs - but I think it's general).
Note, I don't think there's a way to use that at the moment for main PC memory.

> 
> > Also, remember that you'll need to save/load the device state as well as the rest of RAM.  
> 
> Device state handling could be done from two different aspects: 1. From host perspective or, 2. From guest perspective. From host, qemu will always know the state of guest devices so that we can use qemu command to checkpoint states, as you point out (very appreciated that, I do not know the command xen-save-devices-state before). From guest, it can be something like suspend to RAM or S3 for PC to checkpoint the current state of PC and restore them when wakeup. Currently I will prefer the #2. 
> 
> In summary, I'd like to have the functionality in qemu that can save & restore PC main memory at shutdown/power-on phase. Thanks.

Dave

> 
> Regards,
> Tianyou
> 
> 
> -----Original Message-----
> From: Dr. David Alan Gilbert [mailto:dgilbert@redhat.com] 
> Sent: Wednesday, April 20, 2016 4:36 PM
> To: Li, Tianyou <tianyou.li@intel.com>
> Cc: qemu-devel@nongnu.org
> Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu
> 
> * Li, Tianyou (tianyou.li@intel.com) wrote:
> > Hi,
> > 
> > Currently we are trying to implement below functionalities in QEmu: main memory in guest can be logically viewed as persistent and its content can be survived through reboot or shutdown/powerup.
> > 
> > I have looked into the QEmu memory management code include memory.c, exec.c and other related source, unfortunately I do not have the chance to get clue of how to make QEmu main memory persistent. I found that pmemsave<http://doc.opensuse.org/documentation/html/openSUSE_114/opensuse-kvm/cha.qemu.monitor.html#id547777> could dump physical memory of guest, but I could not find how to restore the dump file before VM startup to execution.
> 
> Can you explain what you mean by 'persistent' - where do you intend to store the guests memory?
> Also, remember that you'll need to save/load the device state as well as the rest of RAM.  If you've got a way to preserve RAM then maybe hte xen-save-devices-state qemu command could be used to store the rest of devices.
> 
> Dave
> 
> > 
> > Could anyone provide some hints to me? Thanks in advance!
> > 
> > Regards,
> > Tianyou
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] Persistent Main Memory in QEmu

[Li, Tianyou]

Hi Artyom, Dave & others,

An update: I have tried to use -mem-path option to make a file-backed memory. I have tried to write a particular physical address in guest Linux OS with specific value to verify the persistency characteristics. The result is not as we expected. Below is more details.


1. guest os launch: qemu-system-x86_64 -monitor stdio -machine pc -enable-kvm -smp 1  -m 8G  -mem-path ${RAM_FILE}  -device e1000,netdev=user.0 -netdev user,id=user.0,hostfwd=tcp::5555-:22 ${SRC_DIR}/../images/snapshot/arch-snapshot1.img
2. install fmem kernel module in guest linux, the code is here (quick & dirty workaround for mem_write, please ignore): https://github.com/TianyouLi/fmem
3. run printf "worldhello" | dd of=/dev/fmem bs=1 count=10 seek=6294967296 conv=notrunc
4. run dd if=/dev/fmem bs=1 count=10 seek=6294967296 of=out
5. cat out   -------> here the result is "worldhello"
6. poweroff ---> guest shutdown
7. re-launch guest use #1
8. run #4, get memory content of same address
9. the result is not as expected, all zero there


I am looking into the code in numa.c of function allocate_system_memory_nonnuma, which do init memory region and call qemu_ram_mmap in mmap-alloc.c. Seems the mmap with fd has been setup correctly so that memory data read/write should be flushed to file and survive next time of boot. I am not quite clear why step #9 failed to get the value we previous set in step #4. There could be the test methodology problem or can be my in-correct understanding of qemu feature. Could you please elaborate more details or give me some hints? Thanks.


Regards,
Tianyou

-----Original Message-----
From: Li, Tianyou 
Sent: Tuesday, April 19, 2016 10:53 AM
To: Artyom Tarasenko <atar4qemu@gmail.com>
Cc: qemu-devel@nongnu.org
Subject: RE: [Qemu-devel] Persistent Main Memory in QEmu

Hi Artyom,

Thanks for your pointer! I have tried the -mem-path option, and right now looking into the code to see if the content of the file will be used during the guest Linux running next time. Will let you know the result. 

Stefan's post is definitely helpful, thanks for letting me know. Could you please let me know if you have more QEmu memory management documentation about internals? I have googled around, but more likely to hear from your advice. Thanks.

Regards,
Tianyou

-----Original Message-----
From: Artyom Tarasenko [mailto:atar4qemu@gmail.com]
Sent: Tuesday, April 19, 2016 4:20 AM
To: Li, Tianyou <tianyou.li@intel.com>
Cc: qemu-devel@nongnu.org
Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu

Hi Tianyou,

On Mon, Apr 18, 2016 at 5:50 AM, Li, Tianyou <tianyou.li@intel.com> wrote:
> Currently we are trying to implement below functionalities in QEmu: 
> main memory in guest can be logically viewed as persistent and its 
> content can be survived through reboot or shutdown/powerup.
>
> I have looked into the QEmu memory management code include memory.c, 
> exec.c and other related source, unfortunately I do not have the 
> chance to get clue of how to make QEmu main memory persistent. I found 
> that pmemsave could dump physical memory of guest, but I could not 
> find how to restore the dump file before VM startup to execution.
>
>
>
> Could anyone provide some hints to me? Thanks in advance!

Is the option "-mem-path=/path/to/mem-file" what you are looking for?

Stefan wrote a nice post about QEMU RAM internals:
http://blog.vmsplice.net/2016/01/qemu-internals-how-guest-physical-ram.html

Regards,
Artyom


--
Regards,
Artyom Tarasenko

SPARC and PPC PReP under qemu blog: http://tyom.blogspot.com/search/label/qemu

Re: [Qemu-devel] Persistent Main Memory in QEmu

[Dr. David Alan Gilbert]

* Li, Tianyou (tianyou.li@intel.com) wrote:
> Hi Artyom, Dave & others,
> 
> An update: I have tried to use -mem-path option to make a file-backed memory. I have tried to write a particular physical address in guest Linux OS with specific value to verify the persistency characteristics. The result is not as we expected. Below is more details.
> 
> 
> 1. guest os launch: qemu-system-x86_64 -monitor stdio -machine pc -enable-kvm -smp 1  -m 8G  -mem-path ${RAM_FILE}  -device e1000,netdev=user.0 -netdev user,id=user.0,hostfwd=tcp::5555-:22 ${SRC_DIR}/../images/snapshot/arch-snapshot1.img
> 2. install fmem kernel module in guest linux, the code is here (quick & dirty workaround for mem_write, please ignore): https://github.com/TianyouLi/fmem
> 3. run printf "worldhello" | dd of=/dev/fmem bs=1 count=10 seek=6294967296 conv=notrunc
> 4. run dd if=/dev/fmem bs=1 count=10 seek=6294967296 of=out
> 5. cat out   -------> here the result is "worldhello"
> 6. poweroff ---> guest shutdown
> 7. re-launch guest use #1
> 8. run #4, get memory content of same address
> 9. the result is not as expected, all zero there
> 
> 
> I am looking into the code in numa.c of function allocate_system_memory_nonnuma, which do init memory region and call qemu_ram_mmap in mmap-alloc.c. Seems the mmap with fd has been setup correctly so that memory data read/write should be flushed to file and survive next time of boot. I am not quite clear why step #9 failed to get the value we previous set in step #4. There could be the test methodology problem or can be my in-correct understanding of qemu feature. Could you please elaborate more details or give me some hints? Thanks.

I wonder if QEMU or the guest (BIOS? Kernel?)  is zeroing the memory ?  For normal memory
I'd expect it to zero it.

Dave

> 
> 
> Regards,
> Tianyou
> 
> -----Original Message-----
> From: Li, Tianyou 
> Sent: Tuesday, April 19, 2016 10:53 AM
> To: Artyom Tarasenko <atar4qemu@gmail.com>
> Cc: qemu-devel@nongnu.org
> Subject: RE: [Qemu-devel] Persistent Main Memory in QEmu
> 
> Hi Artyom,
> 
> Thanks for your pointer! I have tried the -mem-path option, and right now looking into the code to see if the content of the file will be used during the guest Linux running next time. Will let you know the result. 
> 
> Stefan's post is definitely helpful, thanks for letting me know. Could you please let me know if you have more QEmu memory management documentation about internals? I have googled around, but more likely to hear from your advice. Thanks.
> 
> Regards,
> Tianyou
> 
> -----Original Message-----
> From: Artyom Tarasenko [mailto:atar4qemu@gmail.com]
> Sent: Tuesday, April 19, 2016 4:20 AM
> To: Li, Tianyou <tianyou.li@intel.com>
> Cc: qemu-devel@nongnu.org
> Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu
> 
> Hi Tianyou,
> 
> On Mon, Apr 18, 2016 at 5:50 AM, Li, Tianyou <tianyou.li@intel.com> wrote:
> > Currently we are trying to implement below functionalities in QEmu: 
> > main memory in guest can be logically viewed as persistent and its 
> > content can be survived through reboot or shutdown/powerup.
> >
> > I have looked into the QEmu memory management code include memory.c, 
> > exec.c and other related source, unfortunately I do not have the 
> > chance to get clue of how to make QEmu main memory persistent. I found 
> > that pmemsave could dump physical memory of guest, but I could not 
> > find how to restore the dump file before VM startup to execution.
> >
> >
> >
> > Could anyone provide some hints to me? Thanks in advance!
> 
> Is the option "-mem-path=/path/to/mem-file" what you are looking for?
> 
> Stefan wrote a nice post about QEMU RAM internals:
> http://blog.vmsplice.net/2016/01/qemu-internals-how-guest-physical-ram.html
> 
> Regards,
> Artyom
> 
> 
> --
> Regards,
> Artyom Tarasenko
> 
> SPARC and PPC PReP under qemu blog: http://tyom.blogspot.com/search/label/qemu
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] Persistent Main Memory in QEmu

[Li, Tianyou]

> I wonder if QEMU or the guest (BIOS? Kernel?)  is zeroing the memory ?  For normal memory I'd expect it to zero it.

Zeroing page probably happens when allocating memory, I am not sure if it did happen once release the page. And, I am trying to find specific pattern of the content I write to the physical address of the guest, in the memory backed file in host, using bgrep. The result is no that kind of data was written into the memory backed file specified in -mem-path option. Could someone help on elaborating more details? Thanks.

Regards,
Tianyou

-----Original Message-----
From: Dr. David Alan Gilbert [mailto:dgilbert@redhat.com] 
Sent: Thursday, April 21, 2016 6:52 PM
To: Li, Tianyou <tianyou.li@intel.com>
Cc: Artyom Tarasenko <atar4qemu@gmail.com>; qemu-devel@nongnu.org
Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu

* Li, Tianyou (tianyou.li@intel.com) wrote:
> Hi Artyom, Dave & others,
> 
> An update: I have tried to use -mem-path option to make a file-backed memory. I have tried to write a particular physical address in guest Linux OS with specific value to verify the persistency characteristics. The result is not as we expected. Below is more details.
> 
> 
> 1. guest os launch: qemu-system-x86_64 -monitor stdio -machine pc 
> -enable-kvm -smp 1  -m 8G  -mem-path ${RAM_FILE}  -device 
> e1000,netdev=user.0 -netdev user,id=user.0,hostfwd=tcp::5555-:22 
> ${SRC_DIR}/../images/snapshot/arch-snapshot1.img
> 2. install fmem kernel module in guest linux, the code is here (quick 
> & dirty workaround for mem_write, please ignore): 
> https://github.com/TianyouLi/fmem 3. run printf "worldhello" | dd of=/dev/fmem bs=1 count=10 seek=6294967296 conv=notrunc 4. run dd if=/dev/fmem bs=1 count=10 seek=6294967296 of=out
> 5. cat out   -------> here the result is "worldhello"
> 6. poweroff ---> guest shutdown
> 7. re-launch guest use #1
> 8. run #4, get memory content of same address 9. the result is not as 
> expected, all zero there
> 
> 
> I am looking into the code in numa.c of function allocate_system_memory_nonnuma, which do init memory region and call qemu_ram_mmap in mmap-alloc.c. Seems the mmap with fd has been setup correctly so that memory data read/write should be flushed to file and survive next time of boot. I am not quite clear why step #9 failed to get the value we previous set in step #4. There could be the test methodology problem or can be my in-correct understanding of qemu feature. Could you please elaborate more details or give me some hints? Thanks.

I wonder if QEMU or the guest (BIOS? Kernel?)  is zeroing the memory ?  For normal memory I'd expect it to zero it.

Dave

> 
> 
> Regards,
> Tianyou
> 
> -----Original Message-----
> From: Li, Tianyou
> Sent: Tuesday, April 19, 2016 10:53 AM
> To: Artyom Tarasenko <atar4qemu@gmail.com>
> Cc: qemu-devel@nongnu.org
> Subject: RE: [Qemu-devel] Persistent Main Memory in QEmu
> 
> Hi Artyom,
> 
> Thanks for your pointer! I have tried the -mem-path option, and right now looking into the code to see if the content of the file will be used during the guest Linux running next time. Will let you know the result. 
> 
> Stefan's post is definitely helpful, thanks for letting me know. Could you please let me know if you have more QEmu memory management documentation about internals? I have googled around, but more likely to hear from your advice. Thanks.
> 
> Regards,
> Tianyou
> 
> -----Original Message-----
> From: Artyom Tarasenko [mailto:atar4qemu@gmail.com]
> Sent: Tuesday, April 19, 2016 4:20 AM
> To: Li, Tianyou <tianyou.li@intel.com>
> Cc: qemu-devel@nongnu.org
> Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu
> 
> Hi Tianyou,
> 
> On Mon, Apr 18, 2016 at 5:50 AM, Li, Tianyou <tianyou.li@intel.com> wrote:
> > Currently we are trying to implement below functionalities in QEmu: 
> > main memory in guest can be logically viewed as persistent and its 
> > content can be survived through reboot or shutdown/powerup.
> >
> > I have looked into the QEmu memory management code include memory.c, 
> > exec.c and other related source, unfortunately I do not have the 
> > chance to get clue of how to make QEmu main memory persistent. I 
> > found that pmemsave could dump physical memory of guest, but I could 
> > not find how to restore the dump file before VM startup to execution.
> >
> >
> >
> > Could anyone provide some hints to me? Thanks in advance!
> 
> Is the option "-mem-path=/path/to/mem-file" what you are looking for?
> 
> Stefan wrote a nice post about QEMU RAM internals:
> http://blog.vmsplice.net/2016/01/qemu-internals-how-guest-physical-ram
> .html
> 
> Regards,
> Artyom
> 
> 
> --
> Regards,
> Artyom Tarasenko
> 
> SPARC and PPC PReP under qemu blog: 
> http://tyom.blogspot.com/search/label/qemu
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] Persistent Main Memory in QEmu

[Li, Tianyou]

I found the mmap was invoked in qemu with option MAP_ANONYMOUS, which will zero pages. After a simple workaround right now pc.ram region will keep its content through reboot/shutdown. 

Thanks Dave and Artyom for the help!

Regards,
Tianyou

-----Original Message-----
From: Li, Tianyou 
Sent: Friday, April 22, 2016 11:09 AM
To: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Artyom Tarasenko <atar4qemu@gmail.com>; qemu-devel@nongnu.org
Subject: RE: [Qemu-devel] Persistent Main Memory in QEmu

> I wonder if QEMU or the guest (BIOS? Kernel?)  is zeroing the memory ?  For normal memory I'd expect it to zero it.

Zeroing page probably happens when allocating memory, I am not sure if it did happen once release the page. And, I am trying to find specific pattern of the content I write to the physical address of the guest, in the memory backed file in host, using bgrep. The result is no that kind of data was written into the memory backed file specified in -mem-path option. Could someone help on elaborating more details? Thanks.

Regards,
Tianyou

-----Original Message-----
From: Dr. David Alan Gilbert [mailto:dgilbert@redhat.com]
Sent: Thursday, April 21, 2016 6:52 PM
To: Li, Tianyou <tianyou.li@intel.com>
Cc: Artyom Tarasenko <atar4qemu@gmail.com>; qemu-devel@nongnu.org
Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu

* Li, Tianyou (tianyou.li@intel.com) wrote:
> Hi Artyom, Dave & others,
> 
> An update: I have tried to use -mem-path option to make a file-backed memory. I have tried to write a particular physical address in guest Linux OS with specific value to verify the persistency characteristics. The result is not as we expected. Below is more details.
> 
> 
> 1. guest os launch: qemu-system-x86_64 -monitor stdio -machine pc 
> -enable-kvm -smp 1  -m 8G  -mem-path ${RAM_FILE}  -device
> e1000,netdev=user.0 -netdev user,id=user.0,hostfwd=tcp::5555-:22
> ${SRC_DIR}/../images/snapshot/arch-snapshot1.img
> 2. install fmem kernel module in guest linux, the code is here (quick 
> & dirty workaround for mem_write, please ignore):
> https://github.com/TianyouLi/fmem 3. run printf "worldhello" | dd of=/dev/fmem bs=1 count=10 seek=6294967296 conv=notrunc 4. run dd if=/dev/fmem bs=1 count=10 seek=6294967296 of=out
> 5. cat out   -------> here the result is "worldhello"
> 6. poweroff ---> guest shutdown
> 7. re-launch guest use #1
> 8. run #4, get memory content of same address 9. the result is not as 
> expected, all zero there
> 
> 
> I am looking into the code in numa.c of function allocate_system_memory_nonnuma, which do init memory region and call qemu_ram_mmap in mmap-alloc.c. Seems the mmap with fd has been setup correctly so that memory data read/write should be flushed to file and survive next time of boot. I am not quite clear why step #9 failed to get the value we previous set in step #4. There could be the test methodology problem or can be my in-correct understanding of qemu feature. Could you please elaborate more details or give me some hints? Thanks.

I wonder if QEMU or the guest (BIOS? Kernel?)  is zeroing the memory ?  For normal memory I'd expect it to zero it.

Dave

> 
> 
> Regards,
> Tianyou
> 
> -----Original Message-----
> From: Li, Tianyou
> Sent: Tuesday, April 19, 2016 10:53 AM
> To: Artyom Tarasenko <atar4qemu@gmail.com>
> Cc: qemu-devel@nongnu.org
> Subject: RE: [Qemu-devel] Persistent Main Memory in QEmu
> 
> Hi Artyom,
> 
> Thanks for your pointer! I have tried the -mem-path option, and right now looking into the code to see if the content of the file will be used during the guest Linux running next time. Will let you know the result. 
> 
> Stefan's post is definitely helpful, thanks for letting me know. Could you please let me know if you have more QEmu memory management documentation about internals? I have googled around, but more likely to hear from your advice. Thanks.
> 
> Regards,
> Tianyou
> 
> -----Original Message-----
> From: Artyom Tarasenko [mailto:atar4qemu@gmail.com]
> Sent: Tuesday, April 19, 2016 4:20 AM
> To: Li, Tianyou <tianyou.li@intel.com>
> Cc: qemu-devel@nongnu.org
> Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu
> 
> Hi Tianyou,
> 
> On Mon, Apr 18, 2016 at 5:50 AM, Li, Tianyou <tianyou.li@intel.com> wrote:
> > Currently we are trying to implement below functionalities in QEmu: 
> > main memory in guest can be logically viewed as persistent and its 
> > content can be survived through reboot or shutdown/powerup.
> >
> > I have looked into the QEmu memory management code include memory.c, 
> > exec.c and other related source, unfortunately I do not have the 
> > chance to get clue of how to make QEmu main memory persistent. I 
> > found that pmemsave could dump physical memory of guest, but I could 
> > not find how to restore the dump file before VM startup to execution.
> >
> >
> >
> > Could anyone provide some hints to me? Thanks in advance!
> 
> Is the option "-mem-path=/path/to/mem-file" what you are looking for?
> 
> Stefan wrote a nice post about QEMU RAM internals:
> http://blog.vmsplice.net/2016/01/qemu-internals-how-guest-physical-ram
> .html
> 
> Regards,
> Artyom
> 
> 
> --
> Regards,
> Artyom Tarasenko
> 
> SPARC and PPC PReP under qemu blog: 
> http://tyom.blogspot.com/search/label/qemu
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] Persistent Main Memory in QEmu

[Artyom Tarasenko]

Hi  Tianyou,

On Fri, Apr 22, 2016 at 5:35 PM, Li, Tianyou <tianyou.li@intel.com> wrote:
> I found the mmap was invoked in qemu with option MAP_ANONYMOUS, which will zero pages. After a simple workaround right now pc.ram region will keep its content through reboot/shutdown.

Can you publish the workaround? I think it would be nice to implement
a persistent RAM as a feature, specified by a command line switch.

Regards,
Artyom

-- 
Regards,
Artyom Tarasenko

SPARC and PPC PReP under qemu blog: http://tyom.blogspot.com/search/label/qemu

Re: [Qemu-devel] Persistent Main Memory in QEmu

[Li, Tianyou]

Sure, that will be great! How about give me a few days so I can cleanup the code and provide a better workaround as patch for you all to review? 

Regards,
Tianyou

-----Original Message-----
From: Artyom Tarasenko [mailto:atar4qemu@gmail.com] 
Sent: Friday, April 22, 2016 11:50 PM
To: Li, Tianyou <tianyou.li@intel.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>; qemu-devel@nongnu.org; Stefan Hajnoczi <stefanha@gmail.com>
Subject: Re: [Qemu-devel] Persistent Main Memory in QEmu

Hi  Tianyou,

On Fri, Apr 22, 2016 at 5:35 PM, Li, Tianyou <tianyou.li@intel.com> wrote:
> I found the mmap was invoked in qemu with option MAP_ANONYMOUS, which will zero pages. After a simple workaround right now pc.ram region will keep its content through reboot/shutdown.

Can you publish the workaround? I think it would be nice to implement a persistent RAM as a feature, specified by a command line switch.

Regards,
Artyom

--
Regards,
Artyom Tarasenko

SPARC and PPC PReP under qemu blog: http://tyom.blogspot.com/search/label/qemu