Adding devices via QMP's device_add don't have their bootindex setting respected

Adding devices via QMP's device_add don't have their bootindex setting respected

[Thomas Parrott]

[-- Attachment #1: Type: text/plain, Size: 1176 bytes --]

Due to QEMU moving towards a QMP configuration mechanism and away from
config file support, the LXD team are currently in the process of migrating
to using QMP to add devices to VMs (so that we can support the use of QEMU
6.0).

Currently we are using the `-S` flag to freeze CPU at startup, then using
QMP to add NIC devices via the `device_add` command, and then using the
`cont` command to start the VM guest.

This is working mostly fine, but there is one issue; the provided
"bootindex" property is not respected.

E.g.

device_add
{"addr":"00.0","bootindex":"0","bus":"qemu_pcie4","driver":"virtio-net-pci","id":"dev-lxd_eth0","mac":"00:16:3e:0c:69:e7","mq":"on","multifunction":"off","netdev":"lxd_eth0","vectors":"6"}

The device is seen within the VM guest and the VM BIOS, but its boot order
is last rather than first.

We've also tried using a non-zero bootindex of 1 and that has the same
effect.

After discussions on #qemu IRC channel, we found that running
`system_reset` after adding the devices allowed the `bootindex` property to
be respected.

So this looks like bug. Perhaps we can discuss it in one of the forthcoming
community calls?

Thanks
Tom Parrott

[-- Attachment #2: Type: text/html, Size: 1665 bytes --]

Re: Fwd: Adding devices via QMP's device_add don't have their bootindex setting respected

[Paolo Bonzini]

On 18/05/21 14:22, Thomas Parrott wrote:
> Due to QEMU moving towards a QMP configuration mechanism and away from 
> config file support, the LXD team are currently in the process of 
> migrating to using QMP to add devices to VMs (so that we can support the 
> use of QEMU 6.0).

If the only issue you have is with -spice, that's just a bug that we 
need to fix.  -readconfig is still supposed to work, even though 
-writeconfig is deprecated and I'd like to remove it as soon as 6.1.

In particular, the issue you reported below is not a bug.  Devices that 
are added after startup are considered hotplugged, and the boot index 
list is frozen until the next reset.  QMP-based configuration of 
coldplugged devices is in the works, but unfortunately it is not yet ready.

Paolo

> Currently we are using the `-S` flag to freeze CPU at startup, then 
> using QMP to add NIC devices via the `device_add` command, and then 
> using the `cont` command to start the VM guest.
> 
> This is working mostly fine, but there is one issue; the provided 
> "bootindex" property is not respected.
> 
> E.g.
> 
> device_add 
> {"addr":"00.0","bootindex":"0","bus":"qemu_pcie4","driver":"virtio-net-pci","id":"dev-lxd_eth0","mac":"00:16:3e:0c:69:e7","mq":"on","multifunction":"off","netdev":"lxd_eth0","vectors":"6"}
> 
> The device is seen within the VM guest and the VM BIOS, but its boot 
> order is last rather than first.
> 
> We've also tried using a non-zero bootindex of 1 and that has the same 
> effect.
> 
> After discussions on #qemu IRC channel, we found that running 
> `system_reset` after adding the devices allowed the `bootindex` property 
> to be respected.
> 
> So this looks like bug. Perhaps we can discuss it in one of the 
> forthcoming community calls?
> 
> Thanks
> Tom Parrott

Re: Fwd: Adding devices via QMP's device_add don't have their bootindex setting respected

[Thomas Parrott]

[-- Attachment #1: Type: text/plain, Size: 2434 bytes --]

Thanks Paolo for confirming that `-readconfig` isn't planned for removal
and that its just `-writeconfig` that is being deprecated, and that
non-hotplug device add via QMP is planned for development in the future.

We've had some reports from the Arch community that LXD on QEMU 6.0 is
broken, and had understood that config file support was being removed,
which is why we were focussing on switching to QMP.

We'll get some more details on what isn't working with LXD and QEMU 6.0
shortly.

Thanks
Tom

On Tue, 18 May 2021 at 13:55, Paolo Bonzini <pbonzini@redhat.com> wrote:

> On 18/05/21 14:22, Thomas Parrott wrote:
> > Due to QEMU moving towards a QMP configuration mechanism and away from
> > config file support, the LXD team are currently in the process of
> > migrating to using QMP to add devices to VMs (so that we can support the
> > use of QEMU 6.0).
>
> If the only issue you have is with -spice, that's just a bug that we
> need to fix.  -readconfig is still supposed to work, even though
> -writeconfig is deprecated and I'd like to remove it as soon as 6.1.
>
> In particular, the issue you reported below is not a bug.  Devices that
> are added after startup are considered hotplugged, and the boot index
> list is frozen until the next reset.  QMP-based configuration of
> coldplugged devices is in the works, but unfortunately it is not yet ready.
>
> Paolo
>
> > Currently we are using the `-S` flag to freeze CPU at startup, then
> > using QMP to add NIC devices via the `device_add` command, and then
> > using the `cont` command to start the VM guest.
> >
> > This is working mostly fine, but there is one issue; the provided
> > "bootindex" property is not respected.
> >
> > E.g.
> >
> > device_add
> >
> {"addr":"00.0","bootindex":"0","bus":"qemu_pcie4","driver":"virtio-net-pci","id":"dev-lxd_eth0","mac":"00:16:3e:0c:69:e7","mq":"on","multifunction":"off","netdev":"lxd_eth0","vectors":"6"}
> >
> > The device is seen within the VM guest and the VM BIOS, but its boot
> > order is last rather than first.
> >
> > We've also tried using a non-zero bootindex of 1 and that has the same
> > effect.
> >
> > After discussions on #qemu IRC channel, we found that running
> > `system_reset` after adding the devices allowed the `bootindex` property
> > to be respected.
> >
> > So this looks like bug. Perhaps we can discuss it in one of the
> > forthcoming community calls?
> >
> > Thanks
> > Tom Parrott
>
>

[-- Attachment #2: Type: text/html, Size: 3313 bytes --]