[RFC] Improving CPU hotplug for 4.3 with qemu-xen

[RFC] Improving CPU hotplug for 4.3 with qemu-xen

[Anthony PERARD]

Hi,

The cpu hotplug that have been backported from QEMU to qemu-xen (1.5 to
1.3) is incomplete. Especially, there is no way to know if a CPU is
already marked as online by qemu. This result by those error messages
from libxl:

libxl: error: libxl_qmp.c:289:qmp_handle_error_response: received an
error message from QMP server: Unable to add CPU: 0, it already exists
libxl: error: libxl_qmp.c:289:qmp_handle_error_response: received an
error message from QMP server: Unable to add CPU: 1, it already exists

So I tried to backport more patches so it will become possible for libxl
to get the list of online cpu.

But this result in *a lot* of patches.

Here is the git tree and see the list of patches at the end of the mail:
git://xenbits.xen.org/people/aperard/qemu-dm.git  backport-cpu-device

There is still some work to be done like making sure everything is
working properly and making some patches to use it with libxl.

So George, is it worth getting those patches into Xen 4.3 to get ride of
the error message ?



cpu: Change parent type to Device
qdev: Fold state enum into bool realized
qdev: Prepare "realized" property
cpu: Prepare QOM realizefn
target-i386: Update X86CPU to QOM realizefn
cpu: Introduce cpu_resume(), for single CPU
target-i386: Split out CPU creation and features parsing
cpu: Resume CPU from DeviceClass::realize() if hot-plugged
Revert "xen: Implement hot_add_cpu hook."
Revert "acpi_piix4: Add infrastructure to send CPU hot-plug GPE to guest"
cpu: Introduce CPU hot-plug notifier
cpu: Add helper cpu_exists(), to check if CPU with specified id exists
acpi_piix4: Add infrastructure to send CPU hot-plug GPE to guest
target-i386: Introduce feat2prop() for CPU properties
target-i386: Introduce apic-id CPU property
target-i386: Do not allow to set apic-id once CPU is realized
target-i386: Replace MSI_SPACE_SIZE with APIC_SPACE_SIZE
qapi/qmp-registry.c: Include headers it needs
qga/channel-posix.c: Include headers it needs
user: Move *-user/qemu-types.h to main directory
user: Rename qemu-types.h to qemu-user-types.h
Create qemu-types.h for struct typedefs
qlist.h: Do not include qemu-common.h
sysbus: make SysBusDeviceClass::init optional
target-i386: Introduce ICC bus/device/bridge
target-i386: Attach ICC bus to CPU on its creation
a fix for cpu_x86_create
fix machine initialization.
extract/unify the constant 0xfee00000 as APIC_DEFAULT_ADDRESS
kvmvapic: Make dependency on sysbus.h explicit
target-i386: Move APIC to ICC bus
pc: Implement QEMUMachine::hot_add_cpu hook



-- 
Anthony PERARD

Re: [RFC] Improving CPU hotplug for 4.3 with qemu-xen

[Stefano Stabellini]

On Fri, 19 Jul 2013, Anthony PERARD wrote:
> Hi,
> 
> The cpu hotplug that have been backported from QEMU to qemu-xen (1.5 to
> 1.3) is incomplete. Especially, there is no way to know if a CPU is
> already marked as online by qemu. This result by those error messages
> from libxl:
> 
> libxl: error: libxl_qmp.c:289:qmp_handle_error_response: received an
> error message from QMP server: Unable to add CPU: 0, it already exists
> libxl: error: libxl_qmp.c:289:qmp_handle_error_response: received an
> error message from QMP server: Unable to add CPU: 1, it already exists
> 
> So I tried to backport more patches so it will become possible for libxl
> to get the list of online cpu.
> 
> But this result in *a lot* of patches.
> 
> Here is the git tree and see the list of patches at the end of the mail:
> git://xenbits.xen.org/people/aperard/qemu-dm.git  backport-cpu-device
> 
> There is still some work to be done like making sure everything is
> working properly and making some patches to use it with libxl.
> 
> So George, is it worth getting those patches into Xen 4.3 to get ride of
> the error message ?
> 
> 
> 
> cpu: Change parent type to Device
> qdev: Fold state enum into bool realized
> qdev: Prepare "realized" property
> cpu: Prepare QOM realizefn
> target-i386: Update X86CPU to QOM realizefn
> cpu: Introduce cpu_resume(), for single CPU
> target-i386: Split out CPU creation and features parsing
> cpu: Resume CPU from DeviceClass::realize() if hot-plugged
> Revert "xen: Implement hot_add_cpu hook."
> Revert "acpi_piix4: Add infrastructure to send CPU hot-plug GPE to guest"
> cpu: Introduce CPU hot-plug notifier
> cpu: Add helper cpu_exists(), to check if CPU with specified id exists
> acpi_piix4: Add infrastructure to send CPU hot-plug GPE to guest
> target-i386: Introduce feat2prop() for CPU properties
> target-i386: Introduce apic-id CPU property
> target-i386: Do not allow to set apic-id once CPU is realized
> target-i386: Replace MSI_SPACE_SIZE with APIC_SPACE_SIZE
> qapi/qmp-registry.c: Include headers it needs
> qga/channel-posix.c: Include headers it needs
> user: Move *-user/qemu-types.h to main directory
> user: Rename qemu-types.h to qemu-user-types.h
> Create qemu-types.h for struct typedefs
> qlist.h: Do not include qemu-common.h
> sysbus: make SysBusDeviceClass::init optional
> target-i386: Introduce ICC bus/device/bridge
> target-i386: Attach ICC bus to CPU on its creation
> a fix for cpu_x86_create
> fix machine initialization.

These two commits worry me.
I have a vague recollection that we hit both issues at some point in
the past and we fixed them properly. I would rather have the proper fix
than these two hacks if it's possible.


> extract/unify the constant 0xfee00000 as APIC_DEFAULT_ADDRESS
> kvmvapic: Make dependency on sysbus.h explicit
> target-i386: Move APIC to ICC bus
> pc: Implement QEMUMachine::hot_add_cpu hook

Re: [RFC] Improving CPU hotplug for 4.3 with qemu-xen

[George Dunlap]

On Fri, Jul 19, 2013 at 4:26 PM, Anthony PERARD
<anthony.perard@citrix.com> wrote:
> Hi,
>
> The cpu hotplug that have been backported from QEMU to qemu-xen (1.5 to
> 1.3) is incomplete. Especially, there is no way to know if a CPU is
> already marked as online by qemu. This result by those error messages
> from libxl:
>
> libxl: error: libxl_qmp.c:289:qmp_handle_error_response: received an
> error message from QMP server: Unable to add CPU: 0, it already exists
> libxl: error: libxl_qmp.c:289:qmp_handle_error_response: received an
> error message from QMP server: Unable to add CPU: 1, it already exists
>
> So I tried to backport more patches so it will become possible for libxl
> to get the list of online cpu.
>
> But this result in *a lot* of patches.
>
> Here is the git tree and see the list of patches at the end of the mail:
> git://xenbits.xen.org/people/aperard/qemu-dm.git  backport-cpu-device
>
> There is still some work to be done like making sure everything is
> working properly and making some patches to use it with libxl.
>
> So George, is it worth getting those patches into Xen 4.3 to get ride of
> the error message ?

I think updates to the stable tree are now IanJ's decision.  It seems
like this is an awful lot to backport.

 -George

Re: [RFC] Improving CPU hotplug for 4.3 with qemu-xen

[Ian Jackson]

George Dunlap writes ("Re: [Xen-devel] [RFC] Improving CPU hotplug for 4.3 with qemu-xen"):
> On Fri, Jul 19, 2013 at 4:26 PM, Anthony PERARD
> <anthony.perard@citrix.com> wrote:
> > The cpu hotplug that have been backported from QEMU to qemu-xen (1.5 to
> > 1.3) is incomplete. Especially, there is no way to know if a CPU is
> > already marked as online by qemu. This result by those error messages
> > from libxl:
> >
> > libxl: error: libxl_qmp.c:289:qmp_handle_error_response: received an
> > error message from QMP server: Unable to add CPU: 0, it already exists
> > libxl: error: libxl_qmp.c:289:qmp_handle_error_response: received an
> > error message from QMP server: Unable to add CPU: 1, it already exists
> >
> > So I tried to backport more patches so it will become possible for libxl
> > to get the list of online cpu.
> >
> > But this result in *a lot* of patches.
> >
> > Here is the git tree and see the list of patches at the end of the mail:
> > git://xenbits.xen.org/people/aperard/qemu-dm.git  backport-cpu-device

Blimey.

> I think updates to the stable tree are now IanJ's decision.  It seems
> like this is an awful lot to backport.

I agree.  I don't think this is really backport material.
Stefano, do you have an opinion ?

Ian.

Re: [RFC] Improving CPU hotplug for 4.3 with qemu-xen

[Stefano Stabellini]

On Wed, 7 Aug 2013, Ian Jackson wrote:
> George Dunlap writes ("Re: [Xen-devel] [RFC] Improving CPU hotplug for 4.3 with qemu-xen"):
> > On Fri, Jul 19, 2013 at 4:26 PM, Anthony PERARD
> > <anthony.perard@citrix.com> wrote:
> > > The cpu hotplug that have been backported from QEMU to qemu-xen (1.5 to
> > > 1.3) is incomplete. Especially, there is no way to know if a CPU is
> > > already marked as online by qemu. This result by those error messages
> > > from libxl:
> > >
> > > libxl: error: libxl_qmp.c:289:qmp_handle_error_response: received an
> > > error message from QMP server: Unable to add CPU: 0, it already exists
> > > libxl: error: libxl_qmp.c:289:qmp_handle_error_response: received an
> > > error message from QMP server: Unable to add CPU: 1, it already exists
> > >
> > > So I tried to backport more patches so it will become possible for libxl
> > > to get the list of online cpu.
> > >
> > > But this result in *a lot* of patches.
> > >
> > > Here is the git tree and see the list of patches at the end of the mail:
> > > git://xenbits.xen.org/people/aperard/qemu-dm.git  backport-cpu-device
> 
> Blimey.
> 
> > I think updates to the stable tree are now IanJ's decision.  It seems
> > like this is an awful lot to backport.
> 
> I agree.  I don't think this is really backport material.
> Stefano, do you have an opinion ?

I would prefer to avoid backporting all those patches.

Especially if the errors above only show if a user asks for a vcpu to be
plugged in when it's actually already present.