[PATCH linux v2 0/9] xen: pvhvm: support bootup on secondary vCPUs

[PATCH linux v2 0/9] xen: pvhvm: support bootup on secondary vCPUs

[Vitaly Kuznetsov]

It may happen that Xen's and Linux's ideas of vCPU id diverge. In
particular, when we crash on a secondary vCPU we may want to do kdump
and unlike plain kexec where we do migrate_to_reboot_cpu() we try booting
on the vCPU which crashed. This doesn't work very well for PVHVM guests as
we have a number of hypercalls where we pass vCPU id as a parameter. These
hypercalls either fail or do something unexpected. To solve the issue we
need to have a mapping between Linux's and Xen's vCPU ids.

This series solves the issue for x86 PVHVM guests. PV guests don't (and
probably won't) support kdump so I always assume Xen's vCPU id == Linux's
vCPU id. ARM guests will probably need to get proper mapping once we start
supporting kexec/kdump there.

Changes since v1:
- "x86/acpi: store ACPI ids from MADT for future usage" patch added.
- Use ACPI ids instead of vLAPIC ids/2 [Andrew Cooper, Jan Beulich]
- Introduce xen_vcpu_nr() helper [David Vrabel].
- Modify all callers of HYPERVISOR_vcpu_op() instead of modifying
  HYPERVISOR_vcpu_op() [David Vrabel]

Vitaly Kuznetsov (9):
  x86/xen: update cpuid.h from Xen-4.7
  x86/acpi: store ACPI ids from MADT for future usage
  xen: introduce xen_vcpu_id mapping
  x86/xen: use xen_vcpu_id mapping for HYPERVISOR_vcpu_op
  x86/xen: use xen_vcpu_id mapping when pointing vcpu_info to the
    shared_info page
  xen/events: use xen_vcpu_id mapping in events_base
  xen/events: fifo: use xen_vcpu_id mapping
  xen/evtchn: use xen_vcpu_id mapping
  xen/pvhvm: run xen_vcpu_setup() for the boot CPU

 arch/arm/xen/enlighten.c         | 13 +++++++++++-
 arch/x86/include/asm/cpu.h       |  1 +
 arch/x86/include/asm/smp.h       |  2 ++
 arch/x86/include/asm/xen/cpuid.h |  5 ++++-
 arch/x86/kernel/acpi/boot.c      | 16 ++++++++++----
 arch/x86/kernel/apic/apic.c      |  2 ++
 arch/x86/kernel/setup_percpu.c   |  3 +++
 arch/x86/xen/enlighten.c         | 45 +++++++++++++++++++++++++++++++---------
 arch/x86/xen/irq.c               |  3 ++-
 arch/x86/xen/smp.c               | 18 +++++++++++-----
 arch/x86/xen/time.c              | 18 ++++++++++------
 arch/x86/xen/xen-ops.h           |  1 +
 drivers/xen/events/events_base.c | 13 ++++++------
 drivers/xen/events/events_fifo.c |  2 +-
 drivers/xen/evtchn.c             |  3 ++-
 drivers/xen/time.c               |  2 +-
 include/xen/xen-ops.h            |  6 ++++++
 17 files changed, 116 insertions(+), 37 deletions(-)

-- 
2.5.5


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH linux v2 0/9] xen: pvhvm: support bootup on secondary vCPUs

[Vitaly Kuznetsov]

Stefano Stabellini <sstabellini@kernel.org> writes:

> On Mon, 25 Jul 2016, Vitaly Kuznetsov wrote:
>> 
>> While we're not obliged to have the same type for xen_vcpu_id on all
>> arches I see no point in diverging without a reason. I can do v3 making
>> the mapping uint32 
>
> I agree that making the mapping uint32_t would be desirable. It would
> even make sense from the int types point of view in Linux.
>
>>and indicating the missing value as U32_MAX-1 if nobody is against the
>>idea.
>
> Why U32_MAX-1? (int)-1 is (unsigned)U32_MAX. Even
> XEN_INVALID_MAX_VCPU_ID is defined as (~0U).

Yes, my bad, of course it should be U32_MAX.

-- 
  Vitaly

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH linux v2 0/9] xen: pvhvm: support bootup on secondary vCPUs

[Stefano Stabellini]

On Mon, 25 Jul 2016, Julien Grall wrote:
> On 25/07/16 14:39, Vitaly Kuznetsov wrote:
> > Julien Grall <julien.grall@arm.com> writes:
> > 
> > > Hi David,
> > > 
> > > On 25/07/16 13:38, David Vrabel wrote:
> > > > On 30/06/16 16:56, Vitaly Kuznetsov wrote:
> > > > > It may happen that Xen's and Linux's ideas of vCPU id diverge. In
> > > > > particular, when we crash on a secondary vCPU we may want to do kdump
> > > > > and unlike plain kexec where we do migrate_to_reboot_cpu() we try
> > > > > booting
> > > > > on the vCPU which crashed. This doesn't work very well for PVHVM
> > > > > guests as
> > > > > we have a number of hypercalls where we pass vCPU id as a parameter.
> > > > > These
> > > > > hypercalls either fail or do something unexpected. To solve the issue
> > > > > we
> > > > > need to have a mapping between Linux's and Xen's vCPU ids.
> > > > > 
> > > > > This series solves the issue for x86 PVHVM guests. PV guests don't
> > > > > (and
> > > > > probably won't) support kdump so I always assume Xen's vCPU id ==
> > > > > Linux's
> > > > > vCPU id. ARM guests will probably need to get proper mapping once we
> > > > > start
> > > > > supporting kexec/kdump there.
> > > > 
> > > > Applied to for-linus-4.8, thanks.
> > > 
> > > It would have been nice to send a ping before applying. This patch
> > > series is containing Xen ARM code which has not been acked by Stefano,
> > > nor had feedback from ARM side.
> > > 
> > > For instance given that all the hypercalls are representing a "vcpu
> > > id" using "uint32_t" it is a bit weird to use "int" to define
> > > xen_vcpu_id (see patch #3).
> > 
> > CPU id is usually 'int' in linux and now we pass it to all
> > hypercalls as it is.
> 
> Well, we need to differentiate between the internal representation of the CPU
> which is based on the boot order and the logical CPU ID. For instance on ARM,
> the logical CPU ID may not be contiguous nor 0 for the first CPU.
> 
> From my understanding, the macros in patch #3 will be used at the last minute
> when prepare the hypercall data. IHMO this is very similar to a logical ID and
> defined as uint32_t by the hypercall ABI.
> 
> Although, I agree that currently we use the internal CPU id on ARM which is
> very unfortunate because this value is based on the order of the nodes in the
> device tree.
> 
> One way to abolish it on ARM would be to use the MPIDR (or at least a part)
> for the VCPU ID.

That's a good idea -- much more reliable than the device tree cpu order.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH linux v2 0/9] xen: pvhvm: support bootup on secondary vCPUs

[Stefano Stabellini]

On Mon, 25 Jul 2016, David Vrabel wrote:
> On 25/07/16 14:17, Julien Grall wrote:
> > Hi David,
> > 
> > On 25/07/16 13:38, David Vrabel wrote:
> >> On 30/06/16 16:56, Vitaly Kuznetsov wrote:
> >>> It may happen that Xen's and Linux's ideas of vCPU id diverge. In
> >>> particular, when we crash on a secondary vCPU we may want to do kdump
> >>> and unlike plain kexec where we do migrate_to_reboot_cpu() we try
> >>> booting
> >>> on the vCPU which crashed. This doesn't work very well for PVHVM
> >>> guests as
> >>> we have a number of hypercalls where we pass vCPU id as a parameter.
> >>> These
> >>> hypercalls either fail or do something unexpected. To solve the issue we
> >>> need to have a mapping between Linux's and Xen's vCPU ids.
> >>>
> >>> This series solves the issue for x86 PVHVM guests. PV guests don't (and
> >>> probably won't) support kdump so I always assume Xen's vCPU id ==
> >>> Linux's
> >>> vCPU id. ARM guests will probably need to get proper mapping once we
> >>> start
> >>> supporting kexec/kdump there.
> >>
> >> Applied to for-linus-4.8, thanks.
> > 
> > It would have been nice to send a ping before applying. This patch
> > series is containing Xen ARM code which has not been acked by Stefano,
> > nor had feedback from ARM side.
> 
> The ARM parts are trivial and such a small part of this series, I saw no
> need to wait.

That's OK, no worries. In the future it would be nice to wait a few more
hours, until morning PST.

Cheers,

Stefano

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH linux v2 0/9] xen: pvhvm: support bootup on secondary vCPUs

[Stefano Stabellini]

On Mon, 25 Jul 2016, Vitaly Kuznetsov wrote:
> Julien Grall <julien.grall@arm.com> writes:
> 
> > Hello,
> >
> > On 25/07/16 14:39, Vitaly Kuznetsov wrote:
> >> Julien Grall <julien.grall@arm.com> writes:
> >>
> >>> Hi David,
> >>>
> >>> On 25/07/16 13:38, David Vrabel wrote:
> >>>> On 30/06/16 16:56, Vitaly Kuznetsov wrote:
> >>>>> It may happen that Xen's and Linux's ideas of vCPU id diverge. In
> >>>>> particular, when we crash on a secondary vCPU we may want to do kdump
> >>>>> and unlike plain kexec where we do migrate_to_reboot_cpu() we try booting
> >>>>> on the vCPU which crashed. This doesn't work very well for PVHVM guests as
> >>>>> we have a number of hypercalls where we pass vCPU id as a parameter. These
> >>>>> hypercalls either fail or do something unexpected. To solve the issue we
> >>>>> need to have a mapping between Linux's and Xen's vCPU ids.
> >>>>>
> >>>>> This series solves the issue for x86 PVHVM guests. PV guests don't (and
> >>>>> probably won't) support kdump so I always assume Xen's vCPU id == Linux's
> >>>>> vCPU id. ARM guests will probably need to get proper mapping once we start
> >>>>> supporting kexec/kdump there.
> >>>>
> >>>> Applied to for-linus-4.8, thanks.
> >>>
> >>> It would have been nice to send a ping before applying. This patch
> >>> series is containing Xen ARM code which has not been acked by Stefano,
> >>> nor had feedback from ARM side.
> >>>
> >>> For instance given that all the hypercalls are representing a "vcpu
> >>> id" using "uint32_t" it is a bit weird to use "int" to define
> >>> xen_vcpu_id (see patch #3).
> >>
> >> CPU id is usually 'int' in linux and now we pass it to all
> >> hypercalls as it is.
> >
> > Well, we need to differentiate between the internal representation of
> > the CPU which is based on the boot order and the logical CPU ID. For
> > instance on ARM, the logical CPU ID may not be contiguous nor 0 for
> > the first CPU.
> >
> > From my understanding, the macros in patch #3 will be used at the last
> > minute when prepare the hypercall data. IHMO this is very similar to a
> > logical ID and defined as uint32_t by the hypercall ABI.
> >
> > Although, I agree that currently we use the internal CPU id on ARM
> > which is very unfortunate because this value is based on the order of
> > the nodes in the device tree.
> >
> > One way to abolish it on ARM would be to use the MPIDR (or at least a
> > part) for the VCPU ID.
> >
> 
> I probably know too little about ARM but it seems that unlike x86 we
> don't need the knowledge of _other_ vCPU ids before we start them so
> MPIDR looks very suitable.
> 
> >> It is a bit more convenient in the mapping I
> >> introduce as we can set it to a negative value to indicate there is no
> >> mapping available. I can definitely change that and use something like
> >> U32_MAX-1 to instead but I'm not sure it is worth it...
> >
> > I looked at the definition of cpu_acpi_id on x86 which return
> > x86_cpu_to_acpiid that has been defined to an uint32_t.
> >
> > So you are assuming that it will never be possible to have an ID >
> > 0x80000000.
> >
> > Also, this may not be true on ARM depending how we define the VCPU
> > mapping. We could decide to use the MPIDR which is in this case may be
> > considered as "negative".
> 
> While we're not obliged to have the same type for xen_vcpu_id on all
> arches I see no point in diverging without a reason. I can do v3 making
> the mapping uint32 

I agree that making the mapping uint32_t would be desirable. It would
even make sense from the int types point of view in Linux.


>and indicating the missing value as U32_MAX-1 if nobody is against the
>idea.

Why U32_MAX-1? (int)-1 is (unsigned)U32_MAX. Even
XEN_INVALID_MAX_VCPU_ID is defined as (~0U).

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH linux v2 0/9] xen: pvhvm: support bootup on secondary vCPUs

[Vitaly Kuznetsov]

Julien Grall <julien.grall@arm.com> writes:

> Hello,
>
> On 25/07/16 14:39, Vitaly Kuznetsov wrote:
>> Julien Grall <julien.grall@arm.com> writes:
>>
>>> Hi David,
>>>
>>> On 25/07/16 13:38, David Vrabel wrote:
>>>> On 30/06/16 16:56, Vitaly Kuznetsov wrote:
>>>>> It may happen that Xen's and Linux's ideas of vCPU id diverge. In
>>>>> particular, when we crash on a secondary vCPU we may want to do kdump
>>>>> and unlike plain kexec where we do migrate_to_reboot_cpu() we try booting
>>>>> on the vCPU which crashed. This doesn't work very well for PVHVM guests as
>>>>> we have a number of hypercalls where we pass vCPU id as a parameter. These
>>>>> hypercalls either fail or do something unexpected. To solve the issue we
>>>>> need to have a mapping between Linux's and Xen's vCPU ids.
>>>>>
>>>>> This series solves the issue for x86 PVHVM guests. PV guests don't (and
>>>>> probably won't) support kdump so I always assume Xen's vCPU id == Linux's
>>>>> vCPU id. ARM guests will probably need to get proper mapping once we start
>>>>> supporting kexec/kdump there.
>>>>
>>>> Applied to for-linus-4.8, thanks.
>>>
>>> It would have been nice to send a ping before applying. This patch
>>> series is containing Xen ARM code which has not been acked by Stefano,
>>> nor had feedback from ARM side.
>>>
>>> For instance given that all the hypercalls are representing a "vcpu
>>> id" using "uint32_t" it is a bit weird to use "int" to define
>>> xen_vcpu_id (see patch #3).
>>
>> CPU id is usually 'int' in linux and now we pass it to all
>> hypercalls as it is.
>
> Well, we need to differentiate between the internal representation of
> the CPU which is based on the boot order and the logical CPU ID. For
> instance on ARM, the logical CPU ID may not be contiguous nor 0 for
> the first CPU.
>
> From my understanding, the macros in patch #3 will be used at the last
> minute when prepare the hypercall data. IHMO this is very similar to a
> logical ID and defined as uint32_t by the hypercall ABI.
>
> Although, I agree that currently we use the internal CPU id on ARM
> which is very unfortunate because this value is based on the order of
> the nodes in the device tree.
>
> One way to abolish it on ARM would be to use the MPIDR (or at least a
> part) for the VCPU ID.
>

I probably know too little about ARM but it seems that unlike x86 we
don't need the knowledge of _other_ vCPU ids before we start them so
MPIDR looks very suitable.

>> It is a bit more convenient in the mapping I
>> introduce as we can set it to a negative value to indicate there is no
>> mapping available. I can definitely change that and use something like
>> U32_MAX-1 to instead but I'm not sure it is worth it...
>
> I looked at the definition of cpu_acpi_id on x86 which return
> x86_cpu_to_acpiid that has been defined to an uint32_t.
>
> So you are assuming that it will never be possible to have an ID >
> 0x80000000.
>
> Also, this may not be true on ARM depending how we define the VCPU
> mapping. We could decide to use the MPIDR which is in this case may be
> considered as "negative".

While we're not obliged to have the same type for xen_vcpu_id on all
arches I see no point in diverging without a reason. I can do v3 making
the mapping uint32 and indicating the missing value as U32_MAX-1 if
nobody is against the idea.

-- 
  Vitaly

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH linux v2 0/9] xen: pvhvm: support bootup on secondary vCPUs

[David Vrabel]

On 25/07/16 15:01, Julien Grall wrote:
> Hello,
> 
> On 25/07/16 14:39, Vitaly Kuznetsov wrote:
>> Julien Grall <julien.grall@arm.com> writes:
>>
>>> Hi David,
>>>
>>> On 25/07/16 13:38, David Vrabel wrote:
>>>> On 30/06/16 16:56, Vitaly Kuznetsov wrote:
>>>>> It may happen that Xen's and Linux's ideas of vCPU id diverge. In
>>>>> particular, when we crash on a secondary vCPU we may want to do kdump
>>>>> and unlike plain kexec where we do migrate_to_reboot_cpu() we try
>>>>> booting
>>>>> on the vCPU which crashed. This doesn't work very well for PVHVM
>>>>> guests as
>>>>> we have a number of hypercalls where we pass vCPU id as a
>>>>> parameter. These
>>>>> hypercalls either fail or do something unexpected. To solve the
>>>>> issue we
>>>>> need to have a mapping between Linux's and Xen's vCPU ids.
>>>>>
>>>>> This series solves the issue for x86 PVHVM guests. PV guests don't
>>>>> (and
>>>>> probably won't) support kdump so I always assume Xen's vCPU id ==
>>>>> Linux's
>>>>> vCPU id. ARM guests will probably need to get proper mapping once
>>>>> we start
>>>>> supporting kexec/kdump there.
>>>>
>>>> Applied to for-linus-4.8, thanks.
>>>
>>> It would have been nice to send a ping before applying. This patch
>>> series is containing Xen ARM code which has not been acked by Stefano,
>>> nor had feedback from ARM side.
>>>
>>> For instance given that all the hypercalls are representing a "vcpu
>>> id" using "uint32_t" it is a bit weird to use "int" to define
>>> xen_vcpu_id (see patch #3).
>>
>> CPU id is usually 'int' in linux and now we pass it to all
>> hypercalls as it is.
> 
> Well, we need to differentiate between the internal representation of
> the CPU which is based on the boot order and the logical CPU ID. For
> instance on ARM, the logical CPU ID may not be contiguous nor 0 for the
> first CPU.
> 
> From my understanding, the macros in patch #3 will be used at the last
> minute when prepare the hypercall data. IHMO this is very similar to a
> logical ID and defined as uint32_t by the hypercall ABI.
> 
> Although, I agree that currently we use the internal CPU id on ARM which
> is very unfortunate because this value is based on the order of the
> nodes in the device tree.
> 
> One way to abolish it on ARM would be to use the MPIDR (or at least a
> part) for the VCPU ID.
> 
>> It is a bit more convenient in the mapping I
>> introduce as we can set it to a negative value to indicate there is no
>> mapping available. I can definitely change that and use something like
>> U32_MAX-1 to instead but I'm not sure it is worth it...
> 
> I looked at the definition of cpu_acpi_id on x86 which return
> x86_cpu_to_acpiid that has been defined to an uint32_t.
> 
> So you are assuming that it will never be possible to have an ID >
> 0x80000000.
> 
> Also, this may not be true on ARM depending how we define the VCPU
> mapping. We could decide to use the MPIDR which is in this case may be
> considered as "negative".

If you want to change how you number vCPUs on ARM in the future, you can
(if necessary) change the type of this per-cpu variable as well.

David

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH linux v2 0/9] xen: pvhvm: support bootup on secondary vCPUs

[Julien Grall]

Hello,

On 25/07/16 14:39, Vitaly Kuznetsov wrote:
> Julien Grall <julien.grall@arm.com> writes:
>
>> Hi David,
>>
>> On 25/07/16 13:38, David Vrabel wrote:
>>> On 30/06/16 16:56, Vitaly Kuznetsov wrote:
>>>> It may happen that Xen's and Linux's ideas of vCPU id diverge. In
>>>> particular, when we crash on a secondary vCPU we may want to do kdump
>>>> and unlike plain kexec where we do migrate_to_reboot_cpu() we try booting
>>>> on the vCPU which crashed. This doesn't work very well for PVHVM guests as
>>>> we have a number of hypercalls where we pass vCPU id as a parameter. These
>>>> hypercalls either fail or do something unexpected. To solve the issue we
>>>> need to have a mapping between Linux's and Xen's vCPU ids.
>>>>
>>>> This series solves the issue for x86 PVHVM guests. PV guests don't (and
>>>> probably won't) support kdump so I always assume Xen's vCPU id == Linux's
>>>> vCPU id. ARM guests will probably need to get proper mapping once we start
>>>> supporting kexec/kdump there.
>>>
>>> Applied to for-linus-4.8, thanks.
>>
>> It would have been nice to send a ping before applying. This patch
>> series is containing Xen ARM code which has not been acked by Stefano,
>> nor had feedback from ARM side.
>>
>> For instance given that all the hypercalls are representing a "vcpu
>> id" using "uint32_t" it is a bit weird to use "int" to define
>> xen_vcpu_id (see patch #3).
>
> CPU id is usually 'int' in linux and now we pass it to all
> hypercalls as it is.

Well, we need to differentiate between the internal representation of 
the CPU which is based on the boot order and the logical CPU ID. For 
instance on ARM, the logical CPU ID may not be contiguous nor 0 for the 
first CPU.

 From my understanding, the macros in patch #3 will be used at the last 
minute when prepare the hypercall data. IHMO this is very similar to a 
logical ID and defined as uint32_t by the hypercall ABI.

Although, I agree that currently we use the internal CPU id on ARM which 
is very unfortunate because this value is based on the order of the 
nodes in the device tree.

One way to abolish it on ARM would be to use the MPIDR (or at least a 
part) for the VCPU ID.

> It is a bit more convenient in the mapping I
> introduce as we can set it to a negative value to indicate there is no
> mapping available. I can definitely change that and use something like
> U32_MAX-1 to instead but I'm not sure it is worth it...

I looked at the definition of cpu_acpi_id on x86 which return 
x86_cpu_to_acpiid that has been defined to an uint32_t.

So you are assuming that it will never be possible to have an ID > 
0x80000000.

Also, this may not be true on ARM depending how we define the VCPU 
mapping. We could decide to use the MPIDR which is in this case may be 
considered as "negative".

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH linux v2 0/9] xen: pvhvm: support bootup on secondary vCPUs

[David Vrabel]

On 25/07/16 14:17, Julien Grall wrote:
> Hi David,
> 
> On 25/07/16 13:38, David Vrabel wrote:
>> On 30/06/16 16:56, Vitaly Kuznetsov wrote:
>>> It may happen that Xen's and Linux's ideas of vCPU id diverge. In
>>> particular, when we crash on a secondary vCPU we may want to do kdump
>>> and unlike plain kexec where we do migrate_to_reboot_cpu() we try
>>> booting
>>> on the vCPU which crashed. This doesn't work very well for PVHVM
>>> guests as
>>> we have a number of hypercalls where we pass vCPU id as a parameter.
>>> These
>>> hypercalls either fail or do something unexpected. To solve the issue we
>>> need to have a mapping between Linux's and Xen's vCPU ids.
>>>
>>> This series solves the issue for x86 PVHVM guests. PV guests don't (and
>>> probably won't) support kdump so I always assume Xen's vCPU id ==
>>> Linux's
>>> vCPU id. ARM guests will probably need to get proper mapping once we
>>> start
>>> supporting kexec/kdump there.
>>
>> Applied to for-linus-4.8, thanks.
> 
> It would have been nice to send a ping before applying. This patch
> series is containing Xen ARM code which has not been acked by Stefano,
> nor had feedback from ARM side.

The ARM parts are trivial and such a small part of this series, I saw no
need to wait.

David

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH linux v2 0/9] xen: pvhvm: support bootup on secondary vCPUs

[Vitaly Kuznetsov]

Julien Grall <julien.grall@arm.com> writes:

> Hi David,
>
> On 25/07/16 13:38, David Vrabel wrote:
>> On 30/06/16 16:56, Vitaly Kuznetsov wrote:
>>> It may happen that Xen's and Linux's ideas of vCPU id diverge. In
>>> particular, when we crash on a secondary vCPU we may want to do kdump
>>> and unlike plain kexec where we do migrate_to_reboot_cpu() we try booting
>>> on the vCPU which crashed. This doesn't work very well for PVHVM guests as
>>> we have a number of hypercalls where we pass vCPU id as a parameter. These
>>> hypercalls either fail or do something unexpected. To solve the issue we
>>> need to have a mapping between Linux's and Xen's vCPU ids.
>>>
>>> This series solves the issue for x86 PVHVM guests. PV guests don't (and
>>> probably won't) support kdump so I always assume Xen's vCPU id == Linux's
>>> vCPU id. ARM guests will probably need to get proper mapping once we start
>>> supporting kexec/kdump there.
>>
>> Applied to for-linus-4.8, thanks.
>
> It would have been nice to send a ping before applying. This patch
> series is containing Xen ARM code which has not been acked by Stefano,
> nor had feedback from ARM side.
>
> For instance given that all the hypercalls are representing a "vcpu
> id" using "uint32_t" it is a bit weird to use "int" to define
> xen_vcpu_id (see patch #3).

CPU id is usually 'int' in linux and now we pass it to all
hypercalls as it is. It is a bit more convenient in the mapping I
introduce as we can set it to a negative value to indicate there is no
mapping available. I can definitely change that and use something like
U32_MAX-1 to instead but I'm not sure it is worth it...

-- 
  Vitaly

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH linux v2 0/9] xen: pvhvm: support bootup on secondary vCPUs

[Julien Grall]

Hi David,

On 25/07/16 13:38, David Vrabel wrote:
> On 30/06/16 16:56, Vitaly Kuznetsov wrote:
>> It may happen that Xen's and Linux's ideas of vCPU id diverge. In
>> particular, when we crash on a secondary vCPU we may want to do kdump
>> and unlike plain kexec where we do migrate_to_reboot_cpu() we try booting
>> on the vCPU which crashed. This doesn't work very well for PVHVM guests as
>> we have a number of hypercalls where we pass vCPU id as a parameter. These
>> hypercalls either fail or do something unexpected. To solve the issue we
>> need to have a mapping between Linux's and Xen's vCPU ids.
>>
>> This series solves the issue for x86 PVHVM guests. PV guests don't (and
>> probably won't) support kdump so I always assume Xen's vCPU id == Linux's
>> vCPU id. ARM guests will probably need to get proper mapping once we start
>> supporting kexec/kdump there.
>
> Applied to for-linus-4.8, thanks.

It would have been nice to send a ping before applying. This patch 
series is containing Xen ARM code which has not been acked by Stefano, 
nor had feedback from ARM side.

For instance given that all the hypercalls are representing a "vcpu id" 
using "uint32_t" it is a bit weird to use "int" to define xen_vcpu_id 
(see patch #3).

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH linux v2 0/9] xen: pvhvm: support bootup on secondary vCPUs

[David Vrabel]

On 30/06/16 16:56, Vitaly Kuznetsov wrote:
> It may happen that Xen's and Linux's ideas of vCPU id diverge. In
> particular, when we crash on a secondary vCPU we may want to do kdump
> and unlike plain kexec where we do migrate_to_reboot_cpu() we try booting
> on the vCPU which crashed. This doesn't work very well for PVHVM guests as
> we have a number of hypercalls where we pass vCPU id as a parameter. These
> hypercalls either fail or do something unexpected. To solve the issue we
> need to have a mapping between Linux's and Xen's vCPU ids.
> 
> This series solves the issue for x86 PVHVM guests. PV guests don't (and
> probably won't) support kdump so I always assume Xen's vCPU id == Linux's
> vCPU id. ARM guests will probably need to get proper mapping once we start
> supporting kexec/kdump there.

Applied to for-linus-4.8, thanks.

David

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH linux v2 0/9] xen: pvhvm: support bootup on secondary vCPUs

[Vitaly Kuznetsov]

Vitaly Kuznetsov <vkuznets@redhat.com> writes:

> It may happen that Xen's and Linux's ideas of vCPU id diverge. In
> particular, when we crash on a secondary vCPU we may want to do kdump
> and unlike plain kexec where we do migrate_to_reboot_cpu() we try booting
> on the vCPU which crashed. This doesn't work very well for PVHVM guests as
> we have a number of hypercalls where we pass vCPU id as a parameter. These
> hypercalls either fail or do something unexpected. To solve the issue we
> need to have a mapping between Linux's and Xen's vCPU ids.
>
> This series solves the issue for x86 PVHVM guests. PV guests don't (and
> probably won't) support kdump so I always assume Xen's vCPU id == Linux's
> vCPU id. ARM guests will probably need to get proper mapping once we start
> supporting kexec/kdump there.
>
> Changes since v1:
> - "x86/acpi: store ACPI ids from MADT for future usage" patch added.
> - Use ACPI ids instead of vLAPIC ids/2 [Andrew Cooper, Jan Beulich]
> - Introduce xen_vcpu_nr() helper [David Vrabel].
> - Modify all callers of HYPERVISOR_vcpu_op() instead of modifying
>   HYPERVISOR_vcpu_op() [David Vrabel]

David, Boris,

I didn't get any feedback for the v2 of this patchset, is it ready to go
in for 4.8 or is there anything else I need to do to make this happen?

BTW, the patch describing the de facto "ACPI id == Xen's VCPU id" policy
is in xen.git:

commit ea210c52abb6458e39f5365f7f2c3abb9c191c47
Author: Vitaly Kuznetsov <vkuznets@redhat.com>
Date:   Tue Jul 12 13:44:42 2016 +0200

    x86, hvm: document the de facto policy for vCPU ids

Thanks,

-- 
  Vitaly

Re: [PATCH linux v2 0/9] xen: pvhvm: support bootup on secondary vCPUs

[Vitaly Kuznetsov]

Vitaly Kuznetsov <vkuznets@redhat.com> writes:

> It may happen that Xen's and Linux's ideas of vCPU id diverge. In
> particular, when we crash on a secondary vCPU we may want to do kdump
> and unlike plain kexec where we do migrate_to_reboot_cpu() we try booting
> on the vCPU which crashed. This doesn't work very well for PVHVM guests as
> we have a number of hypercalls where we pass vCPU id as a parameter. These
> hypercalls either fail or do something unexpected. To solve the issue we
> need to have a mapping between Linux's and Xen's vCPU ids.
>
> This series solves the issue for x86 PVHVM guests. PV guests don't (and
> probably won't) support kdump so I always assume Xen's vCPU id == Linux's
> vCPU id. ARM guests will probably need to get proper mapping once we start
> supporting kexec/kdump there.
>
> Changes since v1:
> - "x86/acpi: store ACPI ids from MADT for future usage" patch added.
> - Use ACPI ids instead of vLAPIC ids/2 [Andrew Cooper, Jan Beulich]
> - Introduce xen_vcpu_nr() helper [David Vrabel].
> - Modify all callers of HYPERVISOR_vcpu_op() instead of modifying
>   HYPERVISOR_vcpu_op() [David Vrabel]

David, Boris,

I didn't get any feedback for the v2 of this patchset, is it ready to go
in for 4.8 or is there anything else I need to do to make this happen?

BTW, the patch describing the de facto "ACPI id == Xen's VCPU id" policy
is in xen.git:

commit ea210c52abb6458e39f5365f7f2c3abb9c191c47
Author: Vitaly Kuznetsov <vkuznets@redhat.com>
Date:   Tue Jul 12 13:44:42 2016 +0200

    x86, hvm: document the de facto policy for vCPU ids

Thanks,

-- 
  Vitaly

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH linux v2 0/9] xen: pvhvm: support bootup on secondary vCPUs

[Vitaly Kuznetsov]

It may happen that Xen's and Linux's ideas of vCPU id diverge. In
particular, when we crash on a secondary vCPU we may want to do kdump
and unlike plain kexec where we do migrate_to_reboot_cpu() we try booting
on the vCPU which crashed. This doesn't work very well for PVHVM guests as
we have a number of hypercalls where we pass vCPU id as a parameter. These
hypercalls either fail or do something unexpected. To solve the issue we
need to have a mapping between Linux's and Xen's vCPU ids.

This series solves the issue for x86 PVHVM guests. PV guests don't (and
probably won't) support kdump so I always assume Xen's vCPU id == Linux's
vCPU id. ARM guests will probably need to get proper mapping once we start
supporting kexec/kdump there.

Changes since v1:
- "x86/acpi: store ACPI ids from MADT for future usage" patch added.
- Use ACPI ids instead of vLAPIC ids/2 [Andrew Cooper, Jan Beulich]
- Introduce xen_vcpu_nr() helper [David Vrabel].
- Modify all callers of HYPERVISOR_vcpu_op() instead of modifying
  HYPERVISOR_vcpu_op() [David Vrabel]

Vitaly Kuznetsov (9):
  x86/xen: update cpuid.h from Xen-4.7
  x86/acpi: store ACPI ids from MADT for future usage
  xen: introduce xen_vcpu_id mapping
  x86/xen: use xen_vcpu_id mapping for HYPERVISOR_vcpu_op
  x86/xen: use xen_vcpu_id mapping when pointing vcpu_info to the
    shared_info page
  xen/events: use xen_vcpu_id mapping in events_base
  xen/events: fifo: use xen_vcpu_id mapping
  xen/evtchn: use xen_vcpu_id mapping
  xen/pvhvm: run xen_vcpu_setup() for the boot CPU

 arch/arm/xen/enlighten.c         | 13 +++++++++++-
 arch/x86/include/asm/cpu.h       |  1 +
 arch/x86/include/asm/smp.h       |  2 ++
 arch/x86/include/asm/xen/cpuid.h |  5 ++++-
 arch/x86/kernel/acpi/boot.c      | 16 ++++++++++----
 arch/x86/kernel/apic/apic.c      |  2 ++
 arch/x86/kernel/setup_percpu.c   |  3 +++
 arch/x86/xen/enlighten.c         | 45 +++++++++++++++++++++++++++++++---------
 arch/x86/xen/irq.c               |  3 ++-
 arch/x86/xen/smp.c               | 18 +++++++++++-----
 arch/x86/xen/time.c              | 18 ++++++++++------
 arch/x86/xen/xen-ops.h           |  1 +
 drivers/xen/events/events_base.c | 13 ++++++------
 drivers/xen/events/events_fifo.c |  2 +-
 drivers/xen/evtchn.c             |  3 ++-
 drivers/xen/time.c               |  2 +-
 include/xen/xen-ops.h            |  6 ++++++
 17 files changed, 116 insertions(+), 37 deletions(-)

-- 
2.5.5