Re: questions of vm save/restore on arm64

[Chenxiao Zhao <chenxiao.zhao@gmail.com>]

On 6/2/2016 5:29 AM, Julien Grall wrote:
> Hello,
>
> On 01/06/16 01:28, Chenxiao Zhao wrote:
>>
>>
>> On 5/30/2016 4:40 AM, Stefano Stabellini wrote:
>>> On Fri, 27 May 2016, Chenxiao Zhao wrote:
>>>> Hi,
>>>>
>>>> My board is Hikey on which have octa-core of arm cortex-a53. I have
>>>> applied patches [1] to try vm save/restore on arm.
>>>> These patches originally do not working on arm64. I have made some
>>>> changes based on patch set [2].
>>>
>>> Hello Chenxiao,
>>>
>>> thanks for your interest in Xen on ARM save/restore.
>>
>> Hi Stefano,
>>
>> Thanks for your advice.
>>
>> I found a possible reason that cause the restore failure is that xen
>> always failed on p2m_lookup for guest domain.
>
> Who is calling p2m_lookup?

It call by handle_hvm_params(tools/libxc/xc_sr_restore_arm.c) while 
restoring HVM_PARAM_STORE_PFN.

>
>>
>> I called dump_p2m_lookup in p2m_look() and get the output like below:
>>
>> (XEN) dom1 IPA 0x0000000039001000
>
> Looking at the memory layout (see include/public/arch-arm.h) 0x39001000
> is part of the magic region. It contains pages for the console,
> xenstore, memaccess (see the list in tools/libxc/xc_dom_arm.c).

yes, I also noticed that.

>
> 0x39001000 is the base address of the xenstore page.
>
>> (XEN) P2M @ 0000000801e7ce80 mfn:0x79f3a
>> (XEN) Using concatenated root table 0
>> (XEN) 1ST[0x0] = 0x0040000079f3c77f
>> (XEN) 2ND[0x1c8] = 0x0000000000000000
>>
>> My question is:
>>
>> 1. who is responsible for restoring p2m table, Xen or guest kernel?
>
> AFAIK, the toolstack is restoring the most of the memory.
>
>> 2. After restore, the vm always get zero memory space, but there is no
>
> What do you mean by "zero memory space"?

I mean xl does not assign any memory to the restored VM.

Name                                        ID   Mem VCPUs      State 
Time(s)
Domain-0                                     0  1024     8     r----- 
   76.3
guest                                        2     0     1     r----- 
    1.9


>
>> error reported on the restore progress. Does the memory requested by the
>> guest kernel or should be allocated early by hypervisor?
>
> Bear in mind that the patch series you are working on is an RFC and the
> ARM64 was not supported. There might be some issue in the save/restore
> path.
>
> For instance xc_clear_domain_page (tools/libxc/xc_sr_restore_arm.c) main
> return an error if the memory is not mapped. But the caller does not
> check the return value.
>
> Regards,

I finally found out that the problem is that the toolstack did not get 
corret p2m_size while sending all pages on save(always be zero). After I 
fixed that, the guest could be restored but guest kernel caught 
handle_mm_fault().

where do you think I'm going to investigate, guest kernel hibernation 
restore or xen?

Best regards.

>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel