[PATCH linux 0/8] xen: pvhvm: support bootup on secondary vCPUs

[PATCH linux 0/8] xen: pvhvm: support bootup on secondary vCPUs

[Vitaly Kuznetsov]

It may happen that Xen's and Linux's ideas of vCPU id diverge. In
particular, when we crash on a secondary vCPU we may want to do kdump
and unlike plain kexec where we do migrate_to_reboot_cpu() we try booting
on the vCPU which crashed. This doesn't work very well for PVHVM guests as
we have a number of hypercalls where we pass vCPU id as a parameter. These
hypercalls either fail or do something unexpected. To solve the issue we
need to have a mapping between Linux's and Xen's vCPU ids.

This series solves the issue for x86 PVHVM guests. PV guests don't (and
probably won't) support kdump so I always assume Xen's vCPU id == Linux's
vCPU id. ARM guests will probably need to get proper mapping once we start
supporting kexec/kdump there.

Vitaly Kuznetsov (8):
  x86/xen: update cpuid.h from Xen-4.7
  xen: introduce xen_vcpu_id mapping
  x86/xen: use xen_vcpu_id mapping for HYPERVISOR_vcpu_op
  x86/xen: use xen_vcpu_id mapping when pointing vcpu_info to the
    shared_info page
  xen/events: use xen_vcpu_id mapping in events_base
  xen/events: fifo: use xen_vcpu_id mapping
  xen/evtchn: use xen_vcpu_id mapping
  xen/pvhvm: run xen_vcpu_setup() for the boot CPU

 arch/arm/xen/enlighten.c             | 10 ++++++++++
 arch/x86/include/asm/xen/cpuid.h     |  5 ++++-
 arch/x86/include/asm/xen/hypercall.h | 11 ++++++++++-
 arch/x86/xen/enlighten.c             | 35 ++++++++++++++++++++++++++++-------
 arch/x86/xen/smp.c                   |  7 +++++++
 arch/x86/xen/xen-ops.h               |  1 +
 drivers/xen/events/events_base.c     | 10 +++++-----
 drivers/xen/events/events_fifo.c     |  2 +-
 drivers/xen/evtchn.c                 |  2 +-
 include/xen/xen-ops.h                |  1 +
 10 files changed, 68 insertions(+), 16 deletions(-)

-- 
2.5.5


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH linux 0/8] xen: pvhvm: support bootup on secondary vCPUs

[Vitaly Kuznetsov]

David Vrabel <david.vrabel@citrix.com> writes:

> On 29/06/16 10:16, Vitaly Kuznetsov wrote:
>> David Vrabel <david.vrabel@citrix.com> writes:
>> 
>>> On 28/06/16 17:47, Vitaly Kuznetsov wrote:
>>>> It may happen that Xen's and Linux's ideas of vCPU id diverge. In
>>>> particular, when we crash on a secondary vCPU we may want to do kdump
>>>> and unlike plain kexec where we do migrate_to_reboot_cpu() we try booting
>>>> on the vCPU which crashed. This doesn't work very well for PVHVM guests as
>>>> we have a number of hypercalls where we pass vCPU id as a parameter. These
>>>> hypercalls either fail or do something unexpected. To solve the issue we
>>>> need to have a mapping between Linux's and Xen's vCPU ids.
>>>
>>> Could the soft-reboot hypercall (optionally) return on vcpu 0?
>>>
>> 
>> In theory, yes, I think we can re-arrange vCPUs inside the hypervisor so
>> Linux will get them in the natural order after soft reset.
>
> The series is straight forwards and the concept of the guest having to
> map its idea of CPU to VCPU is fine, so unless you think a hypervisor
> based solution is better we can take this series once it's fixed up.

I'm afraid that kdump is not the only case where the mapping may come
handy. Linux CPU ids are not guaranteed to always match the order of
CPUs in MADT, e.g. if we fail to add a CPU ids for the rest will get
shifted. This could easily be emulated by adding 'disable_cpu_apicid='
kernel parameter. I didn't try but I'm pretty sure that disabling any
CPU will break guest bootup. That said, I think maintaining the
Xen-Linux vCPU id mapping is preferred till we introduce an option to
address vCPUs in hypercalls by some other ids (vLAPIC id, for example).

-- 
  Vitaly

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH linux 0/8] xen: pvhvm: support bootup on secondary vCPUs

[David Vrabel]

On 29/06/16 10:16, Vitaly Kuznetsov wrote:
> David Vrabel <david.vrabel@citrix.com> writes:
> 
>> On 28/06/16 17:47, Vitaly Kuznetsov wrote:
>>> It may happen that Xen's and Linux's ideas of vCPU id diverge. In
>>> particular, when we crash on a secondary vCPU we may want to do kdump
>>> and unlike plain kexec where we do migrate_to_reboot_cpu() we try booting
>>> on the vCPU which crashed. This doesn't work very well for PVHVM guests as
>>> we have a number of hypercalls where we pass vCPU id as a parameter. These
>>> hypercalls either fail or do something unexpected. To solve the issue we
>>> need to have a mapping between Linux's and Xen's vCPU ids.
>>
>> Could the soft-reboot hypercall (optionally) return on vcpu 0?
>>
> 
> In theory, yes, I think we can re-arrange vCPUs inside the hypervisor so
> Linux will get them in the natural order after soft reset.

The series is straight forwards and the concept of the guest having to
map its idea of CPU to VCPU is fine, so unless you think a hypervisor
based solution is better we can take this series once it's fixed up.

David

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH linux 0/8] xen: pvhvm: support bootup on secondary vCPUs

[Vitaly Kuznetsov]

David Vrabel <david.vrabel@citrix.com> writes:

> On 28/06/16 17:47, Vitaly Kuznetsov wrote:
>> It may happen that Xen's and Linux's ideas of vCPU id diverge. In
>> particular, when we crash on a secondary vCPU we may want to do kdump
>> and unlike plain kexec where we do migrate_to_reboot_cpu() we try booting
>> on the vCPU which crashed. This doesn't work very well for PVHVM guests as
>> we have a number of hypercalls where we pass vCPU id as a parameter. These
>> hypercalls either fail or do something unexpected. To solve the issue we
>> need to have a mapping between Linux's and Xen's vCPU ids.
>
> Could the soft-reboot hypercall (optionally) return on vcpu 0?
>

In theory, yes, I think we can re-arrange vCPUs inside the hypervisor so
Linux will get them in the natural order after soft reset.

-- 
  Vitaly

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH linux 0/8] xen: pvhvm: support bootup on secondary vCPUs

[David Vrabel]

On 28/06/16 17:47, Vitaly Kuznetsov wrote:
> It may happen that Xen's and Linux's ideas of vCPU id diverge. In
> particular, when we crash on a secondary vCPU we may want to do kdump
> and unlike plain kexec where we do migrate_to_reboot_cpu() we try booting
> on the vCPU which crashed. This doesn't work very well for PVHVM guests as
> we have a number of hypercalls where we pass vCPU id as a parameter. These
> hypercalls either fail or do something unexpected. To solve the issue we
> need to have a mapping between Linux's and Xen's vCPU ids.

Could the soft-reboot hypercall (optionally) return on vcpu 0?

David

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel