[BUG REPORT] soft_reset (kexec/kdump) does not work with mainline xen

[BUG REPORT] soft_reset (kexec/kdump) does not work with mainline xen

[Dongli Zhang]

Hello,

This is to report that the soft_reset (kexec/kdump) has not been working for me
since long time ago.

I have tested again with the most recent mainline xen and the most recent
mainline kernel.

While it works with my old xen version, it does not work with mainline xen.


This is the log of my HVM guest.

Waiting for domain test-vm (domid 1) to die [pid 1265]
Domain 1 has shut down, reason code 5 0x5
Action for shutdown reason code 5 is soft-reset
Done. Rebooting now
xc: error: Failed to set d1's policy (err leaf 0xffffffff, subleaf 0xffffffff, msr 0xffffffff) (17 = File exists): Internal error
libxl: error: libxl_cpuid.c:490:libxl__cpuid_legacy: Domain 1:Failed to apply CPUID policy: File exists
libxl: error: libxl_create.c:1613:domcreate_rebuild_done: Domain 1:cannot (re-)build domain: -3
libxl: error: libxl_xshelp.c:201:libxl__xs_read_mandatory: xenstore read failed: `/libxl/1/type': No such file or directory
libxl: warning: libxl_dom.c:51:libxl__domain_type: unable to get domain type for domid=1, assuming HVM


Neither of below works.

# kexec -l /boot/vmlinuz-5.17.0-rc4xen-00054-gf71077a4d84b-dirty --initrd=/boot/initrd.img-5.17.0-rc4xen-00054-gf71077a4d84b-dirty --reuse-cmdline
# kexec -e

or

# taskset -c 0 echo c > /proc/sysrq-trigger


Thank you very much!

Dongli Zhang

Re: [BUG REPORT] soft_reset (kexec/kdump) does not work with mainline xen

[Jan Beulich]

On 24.02.2022 23:27, Dongli Zhang wrote:
> Hello,
> 
> This is to report that the soft_reset (kexec/kdump) has not been working for me
> since long time ago.
> 
> I have tested again with the most recent mainline xen and the most recent
> mainline kernel.
> 
> While it works with my old xen version, it does not work with mainline xen.
> 
> 
> This is the log of my HVM guest.
> 
> Waiting for domain test-vm (domid 1) to die [pid 1265]
> Domain 1 has shut down, reason code 5 0x5
> Action for shutdown reason code 5 is soft-reset
> Done. Rebooting now
> xc: error: Failed to set d1's policy (err leaf 0xffffffff, subleaf 0xffffffff, msr 0xffffffff) (17 = File exists): Internal error

I don't suppose you tried you track down the origin of this EEXIST? I think
it's pretty obvious, as in the handling of XEN_DOMCTL_set_cpu_policy we have

        if ( d->creation_finished )
            ret = -EEXIST; /* No changing once the domain is running. */

Question is how to address it: One approach could be to clear
d->creation_finished in domain_soft_reset(). But I think it would be more
clean if the tool stack avoided trying to set the CPUID policy (again) on
the guest when it soft-resets, as it's still the same guest after all.
Cc-ing Andrew and Anthony for possible thoughts.

Jan

Re: [BUG REPORT] soft_reset (kexec/kdump) does not work with mainline xen

[Dongli Zhang]

Hi Jan,

On 2/24/22 11:15 PM, Jan Beulich wrote:
> On 24.02.2022 23:27, Dongli Zhang wrote:
>> Hello,
>>
>> This is to report that the soft_reset (kexec/kdump) has not been working for me
>> since long time ago.
>>
>> I have tested again with the most recent mainline xen and the most recent
>> mainline kernel.
>>
>> While it works with my old xen version, it does not work with mainline xen.
>>
>>
>> This is the log of my HVM guest.
>>
>> Waiting for domain test-vm (domid 1) to die [pid 1265]
>> Domain 1 has shut down, reason code 5 0x5
>> Action for shutdown reason code 5 is soft-reset
>> Done. Rebooting now
>> xc: error: Failed to set d1's policy (err leaf 0xffffffff, subleaf 0xffffffff, msr 0xffffffff) (17 = File exists): Internal error
> 
> I don't suppose you tried you track down the origin of this EEXIST? I think
> it's pretty obvious, as in the handling of XEN_DOMCTL_set_cpu_policy we have
> 
>         if ( d->creation_finished )
>             ret = -EEXIST; /* No changing once the domain is running. */
> 
> Question is how to address it: One approach could be to clear
> d->creation_finished in domain_soft_reset(). But I think it would be more
> clean if the tool stack avoided trying to set the CPUID policy (again) on
> the guest when it soft-resets, as it's still the same guest after all.
> Cc-ing Andrew and Anthony for possible thoughts.
> 

The soft_reset on HVM is successful after I reset d->creation_finished at the
beginning of domain_soft_reset(). So far I am able to use this as workaround to
test kexec/kdump.

However, while my image's console works well on old xen versions, the console on
mainline xen version does not work well.

I connect to the console with "xl console <domid>" immediately after the domU is
panic (and kdump is triggered). I am not able to have the syslogs of kdump
kernel on mainline xen. The same image works on old xen version.

Thank you very much!

Dongli Zhang