Adding pcie-root-port devices via QMP apparently isn't possible

Adding pcie-root-port devices via QMP apparently isn't possible

[Thomas Parrott]

[-- Attachment #1: Type: text/plain, Size: 1262 bytes --]

Due to QEMU moving towards a QMP configuration mechanism and away from
config file support, the LXD team are currently in the process of migrating
to using QMP to add devices to VMs (so that we can support the use of QEMU
6.0).

Currently we are using the `-S` flag to freeze CPU at startup, then using
QMP to add NIC devices via the `device_add` command, and then using the
`cont` command to start the VM guest.

However we have found that it is apparently not possible to add a
pcie-root-port device via QMP.

Initially we tried using something like:

device_add
{"id":"qemu_pcie5","driver":"pcie-root-port","bus":"pcie.0","addr":"1.5","chassis":5,"multifunction":"on"}

Which was a straight conversion from the current config file we use.

However this gave the error:

Error: Bus 'pcie.0' does not support hotplugging

Then I found the `--preconfig` flag which sounded interesting, so we
removed the use of `-daemonize` which prevents the use of --preconfig` and
then tried adding the pcie-root-port devices in the preconfig stage.

But this resulted in the error:

The command 'device_add' isn't permitted in 'preconfig' state

So we wondered is this a bug, or if not, what is the correct way to be
adding pcie-root-ports going forward?

Thanks
Tom Parrott

[-- Attachment #2: Type: text/html, Size: 1781 bytes --]

Re: Adding pcie-root-port devices via QMP apparently isn't possible

[Thomas Parrott]

[-- Attachment #1: Type: text/plain, Size: 1552 bytes --]

I meant to also add that it has been suggested we could discuss this
further on one of the forthcoming weekly community calls.

Thanks
Tom

On Tue, 18 May 2021 at 13:28, Thomas Parrott <thomas.parrott@canonical.com>
wrote:

> Due to QEMU moving towards a QMP configuration mechanism and away from
> config file support, the LXD team are currently in the process of migrating
> to using QMP to add devices to VMs (so that we can support the use of QEMU
> 6.0).
>
> Currently we are using the `-S` flag to freeze CPU at startup, then using
> QMP to add NIC devices via the `device_add` command, and then using the
> `cont` command to start the VM guest.
>
> However we have found that it is apparently not possible to add a
> pcie-root-port device via QMP.
>
> Initially we tried using something like:
>
> device_add
> {"id":"qemu_pcie5","driver":"pcie-root-port","bus":"pcie.0","addr":"1.5","chassis":5,"multifunction":"on"}
>
> Which was a straight conversion from the current config file we use.
>
> However this gave the error:
>
> Error: Bus 'pcie.0' does not support hotplugging
>
> Then I found the `--preconfig` flag which sounded interesting, so we
> removed the use of `-daemonize` which prevents the use of --preconfig` and
> then tried adding the pcie-root-port devices in the preconfig stage.
>
> But this resulted in the error:
>
> The command 'device_add' isn't permitted in 'preconfig' state
>
> So we wondered is this a bug, or if not, what is the correct way to be
> adding pcie-root-ports going forward?
>
> Thanks
> Tom Parrott
>
>
>

[-- Attachment #2: Type: text/html, Size: 2342 bytes --]

Re: Adding pcie-root-port devices via QMP apparently isn't possible

[Marcel Apfelbaum]

Hi Thomas,


On Tue, May 18, 2021 at 3:29 PM Thomas Parrott
<thomas.parrott@canonical.com> wrote:
>
> Due to QEMU moving towards a QMP configuration mechanism and away from config file support, the LXD team are currently in the process of migrating to using QMP to add devices to VMs (so that we can support the use of QEMU 6.0).
>
> Currently we are using the `-S` flag to freeze CPU at startup, then using QMP to add NIC devices via the `device_add` command, and then using the `cont` command to start the VM guest.
>
> However we have found that it is apparently not possible to add a pcie-root-port device via QMP.
>
> Initially we tried using something like:
>
> device_add {"id":"qemu_pcie5","driver":"pcie-root-port","bus":"pcie.0","addr":"1.5","chassis":5,"multifunction":"on"}
>
> Which was a straight conversion from the current config file we use.
>
> However this gave the error:
>
> Error: Bus 'pcie.0' does not support hotplugging
>
> Then I found the `--preconfig` flag which sounded interesting, so we removed the use of `-daemonize` which prevents the use of --preconfig` and then tried adding the pcie-root-port devices in the preconfig stage.
>
> But this resulted in the error:
>
> The command 'device_add' isn't permitted in 'preconfig' state
>
> So we wondered is this a bug, or if not, what is the correct way to be adding pcie-root-ports going forward?

The pcie-root-ports cannot be hot-plugged, you will need to
pre-allocate enough pcie-root-ports for your needs.
PCI(e) devices can be hot-plugged into the pre-allocated pcie-root-ports.

Thanks,
Marcel

>
> Thanks
> Tom Parrott
>
>

Re: Adding pcie-root-port devices via QMP apparently isn't possible

[Thomas Parrott]

[-- Attachment #1: Type: text/plain, Size: 1972 bytes --]

Thanks for confirming Marcel. Now we understand that -readconfig isn't
going away we can continue to preallocate the pcie-root-ports via the
config file as you suggest.

Thanks
Tom

On Tue, 18 May 2021 at 15:39, Marcel Apfelbaum <mapfelba@redhat.com> wrote:

> Hi Thomas,
>
>
> On Tue, May 18, 2021 at 3:29 PM Thomas Parrott
> <thomas.parrott@canonical.com> wrote:
> >
> > Due to QEMU moving towards a QMP configuration mechanism and away from
> config file support, the LXD team are currently in the process of migrating
> to using QMP to add devices to VMs (so that we can support the use of QEMU
> 6.0).
> >
> > Currently we are using the `-S` flag to freeze CPU at startup, then
> using QMP to add NIC devices via the `device_add` command, and then using
> the `cont` command to start the VM guest.
> >
> > However we have found that it is apparently not possible to add a
> pcie-root-port device via QMP.
> >
> > Initially we tried using something like:
> >
> > device_add
> {"id":"qemu_pcie5","driver":"pcie-root-port","bus":"pcie.0","addr":"1.5","chassis":5,"multifunction":"on"}
> >
> > Which was a straight conversion from the current config file we use.
> >
> > However this gave the error:
> >
> > Error: Bus 'pcie.0' does not support hotplugging
> >
> > Then I found the `--preconfig` flag which sounded interesting, so we
> removed the use of `-daemonize` which prevents the use of --preconfig` and
> then tried adding the pcie-root-port devices in the preconfig stage.
> >
> > But this resulted in the error:
> >
> > The command 'device_add' isn't permitted in 'preconfig' state
> >
> > So we wondered is this a bug, or if not, what is the correct way to be
> adding pcie-root-ports going forward?
>
> The pcie-root-ports cannot be hot-plugged, you will need to
> pre-allocate enough pcie-root-ports for your needs.
> PCI(e) devices can be hot-plugged into the pre-allocated pcie-root-ports.
>
> Thanks,
> Marcel
>
> >
> > Thanks
> > Tom Parrott
> >
> >
>
>

[-- Attachment #2: Type: text/html, Size: 2714 bytes --]