[Qemu-devel] should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Qemu-devel] should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Peter Maydell]

On 8 June 2015 at 11:32, Igor Mammedov <imammedo@redhat.com> wrote:
> On Thu, 4 Jun 2015 18:17:39 +0100
> Peter Maydell <peter.maydell@linaro.org> wrote:
>> On 4 June 2015 at 17:40, Shlomo Pongratz <shlomopongratz@gmail.com> wrote:
>>  In order for it to work correctly we must use MPIDR values in
>>  the device tree which match the MPIDR values the kernel has picked
>>  for the vcpus, so ask KVM what those values are.

> Could we set QEMU's generated mpidr in kernel instead of pulling it from kernel,
> like we do with APIC ID in x86 and fix kernel not to reset it its own value
> (i.e. untie mpidr from vcpuid)?
>
> Then later we could move setting mpidr from kvm_arch_init_vcpu() into
> board code which should be setting it, since it knows/defines what topology it has.

This is a question better asked on the kvmarm list (which I have cc'd),
because that is where the kernel folks hang out...

-- PMM

should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Peter Maydell]

On 8 June 2015 at 11:32, Igor Mammedov <imammedo@redhat.com> wrote:
> On Thu, 4 Jun 2015 18:17:39 +0100
> Peter Maydell <peter.maydell@linaro.org> wrote:
>> On 4 June 2015 at 17:40, Shlomo Pongratz <shlomopongratz@gmail.com> wrote:
>>  In order for it to work correctly we must use MPIDR values in
>>  the device tree which match the MPIDR values the kernel has picked
>>  for the vcpus, so ask KVM what those values are.

> Could we set QEMU's generated mpidr in kernel instead of pulling it from kernel,
> like we do with APIC ID in x86 and fix kernel not to reset it its own value
> (i.e. untie mpidr from vcpuid)?
>
> Then later we could move setting mpidr from kvm_arch_init_vcpu() into
> board code which should be setting it, since it knows/defines what topology it has.

This is a question better asked on the kvmarm list (which I have cc'd),
because that is where the kernel folks hang out...

-- PMM

Re: [Qemu-devel] should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Marc Zyngier]

On 08/06/15 11:52, Peter Maydell wrote:
> On 8 June 2015 at 11:32, Igor Mammedov <imammedo@redhat.com> wrote:
>> On Thu, 4 Jun 2015 18:17:39 +0100
>> Peter Maydell <peter.maydell@linaro.org> wrote:
>>> On 4 June 2015 at 17:40, Shlomo Pongratz <shlomopongratz@gmail.com> wrote:
>>>  In order for it to work correctly we must use MPIDR values in
>>>  the device tree which match the MPIDR values the kernel has picked
>>>  for the vcpus, so ask KVM what those values are.
> 
>> Could we set QEMU's generated mpidr in kernel instead of pulling it from kernel,
>> like we do with APIC ID in x86 and fix kernel not to reset it its own value
>> (i.e. untie mpidr from vcpuid)?
>>
>> Then later we could move setting mpidr from kvm_arch_init_vcpu() into
>> board code which should be setting it, since it knows/defines what topology it has.
> 
> This is a question better asked on the kvmarm list (which I have cc'd),
> because that is where the kernel folks hang out...

Care to provide some context? Why is this required?

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

Re: should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Marc Zyngier]

On 08/06/15 11:52, Peter Maydell wrote:
> On 8 June 2015 at 11:32, Igor Mammedov <imammedo@redhat.com> wrote:
>> On Thu, 4 Jun 2015 18:17:39 +0100
>> Peter Maydell <peter.maydell@linaro.org> wrote:
>>> On 4 June 2015 at 17:40, Shlomo Pongratz <shlomopongratz@gmail.com> wrote:
>>>  In order for it to work correctly we must use MPIDR values in
>>>  the device tree which match the MPIDR values the kernel has picked
>>>  for the vcpus, so ask KVM what those values are.
> 
>> Could we set QEMU's generated mpidr in kernel instead of pulling it from kernel,
>> like we do with APIC ID in x86 and fix kernel not to reset it its own value
>> (i.e. untie mpidr from vcpuid)?
>>
>> Then later we could move setting mpidr from kvm_arch_init_vcpu() into
>> board code which should be setting it, since it knows/defines what topology it has.
> 
> This is a question better asked on the kvmarm list (which I have cc'd),
> because that is where the kernel folks hang out...

Care to provide some context? Why is this required?

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

Re: [Qemu-devel] should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Peter Maydell]

On 9 June 2015 at 11:52, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On 08/06/15 11:52, Peter Maydell wrote:
>> On 8 June 2015 at 11:32, Igor Mammedov <imammedo@redhat.com> wrote:
>>> On Thu, 4 Jun 2015 18:17:39 +0100
>>> Peter Maydell <peter.maydell@linaro.org> wrote:
>>>> On 4 June 2015 at 17:40, Shlomo Pongratz <shlomopongratz@gmail.com> wrote:
>>>>  In order for it to work correctly we must use MPIDR values in
>>>>  the device tree which match the MPIDR values the kernel has picked
>>>>  for the vcpus, so ask KVM what those values are.
>>
>>> Could we set QEMU's generated mpidr in kernel instead of pulling it from kernel,
>>> like we do with APIC ID in x86 and fix kernel not to reset it its own value
>>> (i.e. untie mpidr from vcpuid)?
>>>
>>> Then later we could move setting mpidr from kvm_arch_init_vcpu() into
>>> board code which should be setting it, since it knows/defines what topology it has.
>>
>> This is a question better asked on the kvmarm list (which I have cc'd),
>> because that is where the kernel folks hang out...
>
> Care to provide some context? Why is this required?

The device tree spec says we have to tell the guest kernel the
affinity information for each CPU, in the reg field of the CPU node.
This device tree blob is created by userspace; the MPIDR the guest
actually reads is currently created by the kernel. The kernel's
vgic v3 code may also make assumptions about vcpu_id => affinity
mappings, I'm not sure.

In current code:
 * the kernel has an opinion about vcpu_id => MPIDR mappings
 * QEMU does too, but it's not the same!
   (luckily for 8 or fewer CPUs they agree, but this will not
   do for GICv3 support)

So either:
 * QEMU needs to tell the kernel the MPIDR for each vCPU
 * QEMU needs to ask the kernel the MPIDR for each vCPU

Which is better? The latter is simpler and will work with
existing kernels. The former would let us (for instance)
use KVM when we're modelling (real world) boards which have
particular cluster configurations (which might not match the
kernel's current simplistic "always 16 CPUs at Aff0" setup).

thanks
-- PMM

Re: should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Peter Maydell]

On 9 June 2015 at 11:52, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On 08/06/15 11:52, Peter Maydell wrote:
>> On 8 June 2015 at 11:32, Igor Mammedov <imammedo@redhat.com> wrote:
>>> On Thu, 4 Jun 2015 18:17:39 +0100
>>> Peter Maydell <peter.maydell@linaro.org> wrote:
>>>> On 4 June 2015 at 17:40, Shlomo Pongratz <shlomopongratz@gmail.com> wrote:
>>>>  In order for it to work correctly we must use MPIDR values in
>>>>  the device tree which match the MPIDR values the kernel has picked
>>>>  for the vcpus, so ask KVM what those values are.
>>
>>> Could we set QEMU's generated mpidr in kernel instead of pulling it from kernel,
>>> like we do with APIC ID in x86 and fix kernel not to reset it its own value
>>> (i.e. untie mpidr from vcpuid)?
>>>
>>> Then later we could move setting mpidr from kvm_arch_init_vcpu() into
>>> board code which should be setting it, since it knows/defines what topology it has.
>>
>> This is a question better asked on the kvmarm list (which I have cc'd),
>> because that is where the kernel folks hang out...
>
> Care to provide some context? Why is this required?

The device tree spec says we have to tell the guest kernel the
affinity information for each CPU, in the reg field of the CPU node.
This device tree blob is created by userspace; the MPIDR the guest
actually reads is currently created by the kernel. The kernel's
vgic v3 code may also make assumptions about vcpu_id => affinity
mappings, I'm not sure.

In current code:
 * the kernel has an opinion about vcpu_id => MPIDR mappings
 * QEMU does too, but it's not the same!
   (luckily for 8 or fewer CPUs they agree, but this will not
   do for GICv3 support)

So either:
 * QEMU needs to tell the kernel the MPIDR for each vCPU
 * QEMU needs to ask the kernel the MPIDR for each vCPU

Which is better? The latter is simpler and will work with
existing kernels. The former would let us (for instance)
use KVM when we're modelling (real world) boards which have
particular cluster configurations (which might not match the
kernel's current simplistic "always 16 CPUs at Aff0" setup).

thanks
-- PMM

Re: [Qemu-devel] should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Igor Mammedov]

On Tue, 9 Jun 2015 12:24:13 +0100
Peter Maydell <peter.maydell@linaro.org> wrote:

> On 9 June 2015 at 11:52, Marc Zyngier <marc.zyngier@arm.com> wrote:
> > On 08/06/15 11:52, Peter Maydell wrote:
> >> On 8 June 2015 at 11:32, Igor Mammedov <imammedo@redhat.com> wrote:
> >>> On Thu, 4 Jun 2015 18:17:39 +0100
> >>> Peter Maydell <peter.maydell@linaro.org> wrote:
> >>>> On 4 June 2015 at 17:40, Shlomo Pongratz
> >>>> <shlomopongratz@gmail.com> wrote: In order for it to work
> >>>> correctly we must use MPIDR values in the device tree which
> >>>> match the MPIDR values the kernel has picked for the vcpus, so
> >>>> ask KVM what those values are.
> >>
> >>> Could we set QEMU's generated mpidr in kernel instead of pulling
> >>> it from kernel, like we do with APIC ID in x86 and fix kernel not
> >>> to reset it its own value (i.e. untie mpidr from vcpuid)?
> >>>
> >>> Then later we could move setting mpidr from kvm_arch_init_vcpu()
> >>> into board code which should be setting it, since it
> >>> knows/defines what topology it has.
> >>
> >> This is a question better asked on the kvmarm list (which I have
> >> cc'd), because that is where the kernel folks hang out...
> >
> > Care to provide some context? Why is this required?
> 
> The device tree spec says we have to tell the guest kernel the
> affinity information for each CPU, in the reg field of the CPU node.
> This device tree blob is created by userspace; the MPIDR the guest
> actually reads is currently created by the kernel. The kernel's
> vgic v3 code may also make assumptions about vcpu_id => affinity
> mappings, I'm not sure.
> 
> In current code:
>  * the kernel has an opinion about vcpu_id => MPIDR mappings
>  * QEMU does too, but it's not the same!
>    (luckily for 8 or fewer CPUs they agree, but this will not
>    do for GICv3 support)
> 
> So either:
>  * QEMU needs to tell the kernel the MPIDR for each vCPU
that's what we do for KVM on x86 with APIC ID.
x86 KVM doesn't assume that vcpu_id is ACPI ID anymore,
it's just opaque id nowdays.

it's possible to make it working with existing kernels with
a little bit of compat code in QEMU. Newer kernels could advertise
KVM_CAP_SET_MPIDR feature and in absence of it userspace could revert
to old codepath.

>  * QEMU needs to ask the kernel the MPIDR for each vCPU
This would need some interface to feed topology info to kernel
so it could makeup MPIDR value, when we come to that point.

> Which is better? The latter is simpler and will work with
> existing kernels. The former would let us (for instance)
> use KVM when we're modelling (real world) boards which have
> particular cluster configurations (which might not match the
> kernel's current simplistic "always 16 CPUs at Aff0" setup).
> 
> thanks
> -- PMM

Re: should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Igor Mammedov]

On Tue, 9 Jun 2015 12:24:13 +0100
Peter Maydell <peter.maydell@linaro.org> wrote:

> On 9 June 2015 at 11:52, Marc Zyngier <marc.zyngier@arm.com> wrote:
> > On 08/06/15 11:52, Peter Maydell wrote:
> >> On 8 June 2015 at 11:32, Igor Mammedov <imammedo@redhat.com> wrote:
> >>> On Thu, 4 Jun 2015 18:17:39 +0100
> >>> Peter Maydell <peter.maydell@linaro.org> wrote:
> >>>> On 4 June 2015 at 17:40, Shlomo Pongratz
> >>>> <shlomopongratz@gmail.com> wrote: In order for it to work
> >>>> correctly we must use MPIDR values in the device tree which
> >>>> match the MPIDR values the kernel has picked for the vcpus, so
> >>>> ask KVM what those values are.
> >>
> >>> Could we set QEMU's generated mpidr in kernel instead of pulling
> >>> it from kernel, like we do with APIC ID in x86 and fix kernel not
> >>> to reset it its own value (i.e. untie mpidr from vcpuid)?
> >>>
> >>> Then later we could move setting mpidr from kvm_arch_init_vcpu()
> >>> into board code which should be setting it, since it
> >>> knows/defines what topology it has.
> >>
> >> This is a question better asked on the kvmarm list (which I have
> >> cc'd), because that is where the kernel folks hang out...
> >
> > Care to provide some context? Why is this required?
> 
> The device tree spec says we have to tell the guest kernel the
> affinity information for each CPU, in the reg field of the CPU node.
> This device tree blob is created by userspace; the MPIDR the guest
> actually reads is currently created by the kernel. The kernel's
> vgic v3 code may also make assumptions about vcpu_id => affinity
> mappings, I'm not sure.
> 
> In current code:
>  * the kernel has an opinion about vcpu_id => MPIDR mappings
>  * QEMU does too, but it's not the same!
>    (luckily for 8 or fewer CPUs they agree, but this will not
>    do for GICv3 support)
> 
> So either:
>  * QEMU needs to tell the kernel the MPIDR for each vCPU
that's what we do for KVM on x86 with APIC ID.
x86 KVM doesn't assume that vcpu_id is ACPI ID anymore,
it's just opaque id nowdays.

it's possible to make it working with existing kernels with
a little bit of compat code in QEMU. Newer kernels could advertise
KVM_CAP_SET_MPIDR feature and in absence of it userspace could revert
to old codepath.

>  * QEMU needs to ask the kernel the MPIDR for each vCPU
This would need some interface to feed topology info to kernel
so it could makeup MPIDR value, when we come to that point.

> Which is better? The latter is simpler and will work with
> existing kernels. The former would let us (for instance)
> use KVM when we're modelling (real world) boards which have
> particular cluster configurations (which might not match the
> kernel's current simplistic "always 16 CPUs at Aff0" setup).
> 
> thanks
> -- PMM

Re: [Qemu-devel] should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Marc Zyngier]

On 09/06/15 12:24, Peter Maydell wrote:
> On 9 June 2015 at 11:52, Marc Zyngier <marc.zyngier@arm.com> wrote:
>> On 08/06/15 11:52, Peter Maydell wrote:
>>> On 8 June 2015 at 11:32, Igor Mammedov <imammedo@redhat.com> wrote:
>>>> On Thu, 4 Jun 2015 18:17:39 +0100
>>>> Peter Maydell <peter.maydell@linaro.org> wrote:
>>>>> On 4 June 2015 at 17:40, Shlomo Pongratz <shlomopongratz@gmail.com> wrote:
>>>>>  In order for it to work correctly we must use MPIDR values in
>>>>>  the device tree which match the MPIDR values the kernel has picked
>>>>>  for the vcpus, so ask KVM what those values are.
>>>
>>>> Could we set QEMU's generated mpidr in kernel instead of pulling it from kernel,
>>>> like we do with APIC ID in x86 and fix kernel not to reset it its own value
>>>> (i.e. untie mpidr from vcpuid)?
>>>>
>>>> Then later we could move setting mpidr from kvm_arch_init_vcpu() into
>>>> board code which should be setting it, since it knows/defines what topology it has.
>>>
>>> This is a question better asked on the kvmarm list (which I have cc'd),
>>> because that is where the kernel folks hang out...
>>
>> Care to provide some context? Why is this required?
> 
> The device tree spec says we have to tell the guest kernel the
> affinity information for each CPU, in the reg field of the CPU node.
> This device tree blob is created by userspace; the MPIDR the guest
> actually reads is currently created by the kernel. The kernel's
> vgic v3 code may also make assumptions about vcpu_id => affinity
> mappings, I'm not sure.

It definitely does.

> In current code:
>  * the kernel has an opinion about vcpu_id => MPIDR mappings
>  * QEMU does too, but it's not the same!
>    (luckily for 8 or fewer CPUs they agree, but this will not
>    do for GICv3 support)
> 
> So either:
>  * QEMU needs to tell the kernel the MPIDR for each vCPU
>  * QEMU needs to ask the kernel the MPIDR for each vCPU
> 
> Which is better? The latter is simpler and will work with
> existing kernels. The former would let us (for instance)
> use KVM when we're modelling (real world) boards which have
> particular cluster configurations (which might not match the
> kernel's current simplistic "always 16 CPUs at Aff0" setup).

Given that it is userspace that provides the description of the
platform, it feels natural to let userspace set MPIDR accordingly.

This will require some interesting rework in the kernel, but this
doesn't feel too bad.

	M.
-- 
Jazz is not dead. It just smells funny...

Re: should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Marc Zyngier]

On 09/06/15 12:24, Peter Maydell wrote:
> On 9 June 2015 at 11:52, Marc Zyngier <marc.zyngier@arm.com> wrote:
>> On 08/06/15 11:52, Peter Maydell wrote:
>>> On 8 June 2015 at 11:32, Igor Mammedov <imammedo@redhat.com> wrote:
>>>> On Thu, 4 Jun 2015 18:17:39 +0100
>>>> Peter Maydell <peter.maydell@linaro.org> wrote:
>>>>> On 4 June 2015 at 17:40, Shlomo Pongratz <shlomopongratz@gmail.com> wrote:
>>>>>  In order for it to work correctly we must use MPIDR values in
>>>>>  the device tree which match the MPIDR values the kernel has picked
>>>>>  for the vcpus, so ask KVM what those values are.
>>>
>>>> Could we set QEMU's generated mpidr in kernel instead of pulling it from kernel,
>>>> like we do with APIC ID in x86 and fix kernel not to reset it its own value
>>>> (i.e. untie mpidr from vcpuid)?
>>>>
>>>> Then later we could move setting mpidr from kvm_arch_init_vcpu() into
>>>> board code which should be setting it, since it knows/defines what topology it has.
>>>
>>> This is a question better asked on the kvmarm list (which I have cc'd),
>>> because that is where the kernel folks hang out...
>>
>> Care to provide some context? Why is this required?
> 
> The device tree spec says we have to tell the guest kernel the
> affinity information for each CPU, in the reg field of the CPU node.
> This device tree blob is created by userspace; the MPIDR the guest
> actually reads is currently created by the kernel. The kernel's
> vgic v3 code may also make assumptions about vcpu_id => affinity
> mappings, I'm not sure.

It definitely does.

> In current code:
>  * the kernel has an opinion about vcpu_id => MPIDR mappings
>  * QEMU does too, but it's not the same!
>    (luckily for 8 or fewer CPUs they agree, but this will not
>    do for GICv3 support)
> 
> So either:
>  * QEMU needs to tell the kernel the MPIDR for each vCPU
>  * QEMU needs to ask the kernel the MPIDR for each vCPU
> 
> Which is better? The latter is simpler and will work with
> existing kernels. The former would let us (for instance)
> use KVM when we're modelling (real world) boards which have
> particular cluster configurations (which might not match the
> kernel's current simplistic "always 16 CPUs at Aff0" setup).

Given that it is userspace that provides the description of the
platform, it feels natural to let userspace set MPIDR accordingly.

This will require some interesting rework in the kernel, but this
doesn't feel too bad.

	M.
-- 
Jazz is not dead. It just smells funny...

Re: [Qemu-devel] should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Peter Maydell]

On 9 June 2015 at 13:27, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On 09/06/15 12:24, Peter Maydell wrote:
>> So either:
>>  * QEMU needs to tell the kernel the MPIDR for each vCPU
>>  * QEMU needs to ask the kernel the MPIDR for each vCPU
>>
>> Which is better? The latter is simpler and will work with
>> existing kernels. The former would let us (for instance)
>> use KVM when we're modelling (real world) boards which have
>> particular cluster configurations (which might not match the
>> kernel's current simplistic "always 16 CPUs at Aff0" setup).
>
> Given that it is userspace that provides the description of the
> platform, it feels natural to let userspace set MPIDR accordingly.
>
> This will require some interesting rework in the kernel, but this
> doesn't feel too bad.

On the other hand QEMU needs to keep working with current kernels,
so we need the "ask the kernel" code at least for the moment.
Probably needs a capability for "kernel allows userspace to
set MPIDR" so we can adapt.

-- PMM

Re: should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Peter Maydell]

On 9 June 2015 at 13:27, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On 09/06/15 12:24, Peter Maydell wrote:
>> So either:
>>  * QEMU needs to tell the kernel the MPIDR for each vCPU
>>  * QEMU needs to ask the kernel the MPIDR for each vCPU
>>
>> Which is better? The latter is simpler and will work with
>> existing kernels. The former would let us (for instance)
>> use KVM when we're modelling (real world) boards which have
>> particular cluster configurations (which might not match the
>> kernel's current simplistic "always 16 CPUs at Aff0" setup).
>
> Given that it is userspace that provides the description of the
> platform, it feels natural to let userspace set MPIDR accordingly.
>
> This will require some interesting rework in the kernel, but this
> doesn't feel too bad.

On the other hand QEMU needs to keep working with current kernels,
so we need the "ask the kernel" code at least for the moment.
Probably needs a capability for "kernel allows userspace to
set MPIDR" so we can adapt.

-- PMM

Re: [Qemu-devel] should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Marc Zyngier]

On 09/06/15 14:16, Peter Maydell wrote:
> On 9 June 2015 at 13:27, Marc Zyngier <marc.zyngier@arm.com> wrote:
>> On 09/06/15 12:24, Peter Maydell wrote:
>>> So either:
>>>  * QEMU needs to tell the kernel the MPIDR for each vCPU
>>>  * QEMU needs to ask the kernel the MPIDR for each vCPU
>>>
>>> Which is better? The latter is simpler and will work with
>>> existing kernels. The former would let us (for instance)
>>> use KVM when we're modelling (real world) boards which have
>>> particular cluster configurations (which might not match the
>>> kernel's current simplistic "always 16 CPUs at Aff0" setup).
>>
>> Given that it is userspace that provides the description of the
>> platform, it feels natural to let userspace set MPIDR accordingly.
>>
>> This will require some interesting rework in the kernel, but this
>> doesn't feel too bad.
> 
> On the other hand QEMU needs to keep working with current kernels,
> so we need the "ask the kernel" code at least for the moment.
> Probably needs a capability for "kernel allows userspace to
> set MPIDR" so we can adapt.

Yeah, what I had in mind was something along the lines of:
- kernel computes its "default MPDIR"
- kernel exposes a new capability "KVM_ARM_ALLOW_MPIDR_OVERRIDE" (or
something along those lines)
- userspace does the right thing.

	M.
-- 
Jazz is not dead. It just smells funny...

Re: should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Marc Zyngier]

On 09/06/15 14:16, Peter Maydell wrote:
> On 9 June 2015 at 13:27, Marc Zyngier <marc.zyngier@arm.com> wrote:
>> On 09/06/15 12:24, Peter Maydell wrote:
>>> So either:
>>>  * QEMU needs to tell the kernel the MPIDR for each vCPU
>>>  * QEMU needs to ask the kernel the MPIDR for each vCPU
>>>
>>> Which is better? The latter is simpler and will work with
>>> existing kernels. The former would let us (for instance)
>>> use KVM when we're modelling (real world) boards which have
>>> particular cluster configurations (which might not match the
>>> kernel's current simplistic "always 16 CPUs at Aff0" setup).
>>
>> Given that it is userspace that provides the description of the
>> platform, it feels natural to let userspace set MPIDR accordingly.
>>
>> This will require some interesting rework in the kernel, but this
>> doesn't feel too bad.
> 
> On the other hand QEMU needs to keep working with current kernels,
> so we need the "ask the kernel" code at least for the moment.
> Probably needs a capability for "kernel allows userspace to
> set MPIDR" so we can adapt.

Yeah, what I had in mind was something along the lines of:
- kernel computes its "default MPDIR"
- kernel exposes a new capability "KVM_ARM_ALLOW_MPIDR_OVERRIDE" (or
something along those lines)
- userspace does the right thing.

	M.
-- 
Jazz is not dead. It just smells funny...

Re: [Qemu-devel] should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Peter Maydell]

On 9 June 2015 at 15:00, Marc Zyngier <marc.zyngier@arm.com> wrote:
>
> Yeah, what I had in mind was something along the lines of:
> - kernel computes its "default MPDIR"
> - kernel exposes a new capability "KVM_ARM_ALLOW_MPIDR_OVERRIDE" (or
> something along those lines)
> - userspace does the right thing.

You forgot the "???" step :-)

-- PMM

Re: should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Peter Maydell]

On 9 June 2015 at 15:00, Marc Zyngier <marc.zyngier@arm.com> wrote:
>
> Yeah, what I had in mind was something along the lines of:
> - kernel computes its "default MPDIR"
> - kernel exposes a new capability "KVM_ARM_ALLOW_MPIDR_OVERRIDE" (or
> something along those lines)
> - userspace does the right thing.

You forgot the "???" step :-)

-- PMM

Re: [Qemu-devel] should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Marc Zyngier]

On 09/06/15 15:01, Peter Maydell wrote:
> On 9 June 2015 at 15:00, Marc Zyngier <marc.zyngier@arm.com> wrote:
>>
>> Yeah, what I had in mind was something along the lines of:
>> - kernel computes its "default MPDIR"
>> - kernel exposes a new capability "KVM_ARM_ALLOW_MPIDR_OVERRIDE" (or
>> something along those lines)
>> - userspace does the right thing.
> 
> You forgot the "???" step :-)

Indeed. I also missed the step that says "kernel is able to convert
arbitrary MPIDR to vcpu_id in an efficient manner...". GICv3 is
definitely going to require this.

	N,
-- 
Jazz is not dead. It just smells funny...

Re: should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Marc Zyngier]

On 09/06/15 15:01, Peter Maydell wrote:
> On 9 June 2015 at 15:00, Marc Zyngier <marc.zyngier@arm.com> wrote:
>>
>> Yeah, what I had in mind was something along the lines of:
>> - kernel computes its "default MPDIR"
>> - kernel exposes a new capability "KVM_ARM_ALLOW_MPIDR_OVERRIDE" (or
>> something along those lines)
>> - userspace does the right thing.
> 
> You forgot the "???" step :-)

Indeed. I also missed the step that says "kernel is able to convert
arbitrary MPIDR to vcpu_id in an efficient manner...". GICv3 is
definitely going to require this.

	N,
-- 
Jazz is not dead. It just smells funny...

Re: [Qemu-devel] should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Igor Mammedov]

On Tue, 09 Jun 2015 15:35:21 +0100
Marc Zyngier <marc.zyngier@arm.com> wrote:

> On 09/06/15 15:01, Peter Maydell wrote:
> > On 9 June 2015 at 15:00, Marc Zyngier <marc.zyngier@arm.com> wrote:
> >>
> >> Yeah, what I had in mind was something along the lines of:
> >> - kernel computes its "default MPDIR"
> >> - kernel exposes a new capability "KVM_ARM_ALLOW_MPIDR_OVERRIDE" (or
> >> something along those lines)
> >> - userspace does the right thing.
> > 
> > You forgot the "???" step :-)
> 
> Indeed. I also missed the step that says "kernel is able to convert
> arbitrary MPIDR to vcpu_id in an efficient manner...". GICv3 is
> definitely going to require this.
x86 probably already has code that does this for APIC ID -> vcpu_id

Re: should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Igor Mammedov]

On Tue, 09 Jun 2015 15:35:21 +0100
Marc Zyngier <marc.zyngier@arm.com> wrote:

> On 09/06/15 15:01, Peter Maydell wrote:
> > On 9 June 2015 at 15:00, Marc Zyngier <marc.zyngier@arm.com> wrote:
> >>
> >> Yeah, what I had in mind was something along the lines of:
> >> - kernel computes its "default MPDIR"
> >> - kernel exposes a new capability "KVM_ARM_ALLOW_MPIDR_OVERRIDE" (or
> >> something along those lines)
> >> - userspace does the right thing.
> > 
> > You forgot the "???" step :-)
> 
> Indeed. I also missed the step that says "kernel is able to convert
> arbitrary MPIDR to vcpu_id in an efficient manner...". GICv3 is
> definitely going to require this.
x86 probably already has code that does this for APIC ID -> vcpu_id

Re: [Qemu-devel] should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Marc Zyngier]

On 10/06/15 10:54, Igor Mammedov wrote:
> On Tue, 09 Jun 2015 15:35:21 +0100
> Marc Zyngier <marc.zyngier@arm.com> wrote:
> 
>> On 09/06/15 15:01, Peter Maydell wrote:
>>> On 9 June 2015 at 15:00, Marc Zyngier <marc.zyngier@arm.com> wrote:
>>>>
>>>> Yeah, what I had in mind was something along the lines of:
>>>> - kernel computes its "default MPDIR"
>>>> - kernel exposes a new capability "KVM_ARM_ALLOW_MPIDR_OVERRIDE" (or
>>>> something along those lines)
>>>> - userspace does the right thing.
>>>
>>> You forgot the "???" step :-)
>>
>> Indeed. I also missed the step that says "kernel is able to convert
>> arbitrary MPIDR to vcpu_id in an efficient manner...". GICv3 is
>> definitely going to require this.
> x86 probably already has code that does this for APIC ID -> vcpu_id

Apparently not. kvm_irq_delivery_to_apic seems to iterate over the vcpus
to find a match, and kvm_irq_delivery_to_apic_fast seems to rely on
knowing some form of topology (and some more iteration).

Overall, this looks awfully architecture specific, so it seems unlikely
we can reuse that aspect. I'm inclined to go for an rbtree mapping
MPIDRs to vcpus. As this is likely to be on the fast path, I'd like this
to me as lockless as possible though, which probably means that MPIDR
would become RO as soon as any vcpu has started executing.

	M.
-- 
Jazz is not dead. It just smells funny...

Re: should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Marc Zyngier]

On 10/06/15 10:54, Igor Mammedov wrote:
> On Tue, 09 Jun 2015 15:35:21 +0100
> Marc Zyngier <marc.zyngier@arm.com> wrote:
> 
>> On 09/06/15 15:01, Peter Maydell wrote:
>>> On 9 June 2015 at 15:00, Marc Zyngier <marc.zyngier@arm.com> wrote:
>>>>
>>>> Yeah, what I had in mind was something along the lines of:
>>>> - kernel computes its "default MPDIR"
>>>> - kernel exposes a new capability "KVM_ARM_ALLOW_MPIDR_OVERRIDE" (or
>>>> something along those lines)
>>>> - userspace does the right thing.
>>>
>>> You forgot the "???" step :-)
>>
>> Indeed. I also missed the step that says "kernel is able to convert
>> arbitrary MPIDR to vcpu_id in an efficient manner...". GICv3 is
>> definitely going to require this.
> x86 probably already has code that does this for APIC ID -> vcpu_id

Apparently not. kvm_irq_delivery_to_apic seems to iterate over the vcpus
to find a match, and kvm_irq_delivery_to_apic_fast seems to rely on
knowing some form of topology (and some more iteration).

Overall, this looks awfully architecture specific, so it seems unlikely
we can reuse that aspect. I'm inclined to go for an rbtree mapping
MPIDRs to vcpus. As this is likely to be on the fast path, I'd like this
to me as lockless as possible though, which probably means that MPIDR
would become RO as soon as any vcpu has started executing.

	M.
-- 
Jazz is not dead. It just smells funny...

Re: [Qemu-devel] should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Radim]

2015-06-10 11:31+0100, Marc Zyngier:
>On 10/06/15 10:54, Igor Mammedov wrote:
>> On Tue, 09 Jun 2015 15:35:21 +0100
>> Marc Zyngier <marc.zyngier@arm.com> wrote:
>>> Indeed. I also missed the step that says "kernel is able to convert
>>> arbitrary MPIDR to vcpu_id in an efficient manner...". GICv3 is
>>> definitely going to require this.
>> x86 probably already has code that does this for APIC ID -> vcpu_id
> 
> Apparently not. kvm_irq_delivery_to_apic seems to iterate over the vcpus
> to find a match, and kvm_irq_delivery_to_apic_fast seems to rely on
> knowing some form of topology (and some more iteration).

kvm_irq_delivery_to_apic_fast optimizes APIC ID -> vcpu_id, by using
simple lookup arrays for various modes, with fallback to iteration for
broadcast and unlikely configurations.

Fast path only does iteration for multicast.  The APIC ID space is
sliced by 4(clustered xAPIC)/8(flat xAPIC)/16(x2APIC) APICs and any
number of APICs in a slice can be selected for delivery.
The guest can configure topology (xAPIC) or it is set by hardware
(x2APIC) and matches "real" topology only in rare cases.  ("Real"
topology is encoded in x2APIC ID by reserving bit ranges for "levels".)

> Overall, this looks awfully architecture specific, so it seems unlikely
> we can reuse that aspect.

Yes, the most that can be reused is an idea for a structure that gets
recomputed every time MPIDR -> vcpu_id mapping changes.

>                           I'm inclined to go for an rbtree mapping
> MPIDRs to vcpus. As this is likely to be on the fast path, I'd like this
> to me as lockless as possible though, which probably means that MPIDR
> would become RO as soon as any vcpu has started executing.

x86 KVM uses RCU protected structure for it, and even though APIC IDs
can be sparse, x86 prefers a single array.  (IDs usually aren't very
sparse, so a more complicated structure would be slower, harder to
handle, and take up more memory.)

Re: should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Radim]

2015-06-10 11:31+0100, Marc Zyngier:
>On 10/06/15 10:54, Igor Mammedov wrote:
>> On Tue, 09 Jun 2015 15:35:21 +0100
>> Marc Zyngier <marc.zyngier@arm.com> wrote:
>>> Indeed. I also missed the step that says "kernel is able to convert
>>> arbitrary MPIDR to vcpu_id in an efficient manner...". GICv3 is
>>> definitely going to require this.
>> x86 probably already has code that does this for APIC ID -> vcpu_id
> 
> Apparently not. kvm_irq_delivery_to_apic seems to iterate over the vcpus
> to find a match, and kvm_irq_delivery_to_apic_fast seems to rely on
> knowing some form of topology (and some more iteration).

kvm_irq_delivery_to_apic_fast optimizes APIC ID -> vcpu_id, by using
simple lookup arrays for various modes, with fallback to iteration for
broadcast and unlikely configurations.

Fast path only does iteration for multicast.  The APIC ID space is
sliced by 4(clustered xAPIC)/8(flat xAPIC)/16(x2APIC) APICs and any
number of APICs in a slice can be selected for delivery.
The guest can configure topology (xAPIC) or it is set by hardware
(x2APIC) and matches "real" topology only in rare cases.  ("Real"
topology is encoded in x2APIC ID by reserving bit ranges for "levels".)

> Overall, this looks awfully architecture specific, so it seems unlikely
> we can reuse that aspect.

Yes, the most that can be reused is an idea for a structure that gets
recomputed every time MPIDR -> vcpu_id mapping changes.

>                           I'm inclined to go for an rbtree mapping
> MPIDRs to vcpus. As this is likely to be on the fast path, I'd like this
> to me as lockless as possible though, which probably means that MPIDR
> would become RO as soon as any vcpu has started executing.

x86 KVM uses RCU protected structure for it, and even though APIC IDs
can be sparse, x86 prefers a single array.  (IDs usually aren't very
sparse, so a more complicated structure would be slower, harder to
handle, and take up more memory.)

Re: [Qemu-devel] should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Christoffer Dall]

Hi,

[sorry for reviving this thread late]

On Tue, Jun 09, 2015 at 12:24:13PM +0100, Peter Maydell wrote:
> On 9 June 2015 at 11:52, Marc Zyngier <marc.zyngier@arm.com> wrote:
> > On 08/06/15 11:52, Peter Maydell wrote:
> >> On 8 June 2015 at 11:32, Igor Mammedov <imammedo@redhat.com> wrote:
> >>> On Thu, 4 Jun 2015 18:17:39 +0100
> >>> Peter Maydell <peter.maydell@linaro.org> wrote:
> >>>> On 4 June 2015 at 17:40, Shlomo Pongratz <shlomopongratz@gmail.com> wrote:
> >>>>  In order for it to work correctly we must use MPIDR values in
> >>>>  the device tree which match the MPIDR values the kernel has picked
> >>>>  for the vcpus, so ask KVM what those values are.
> >>
> >>> Could we set QEMU's generated mpidr in kernel instead of pulling it from kernel,
> >>> like we do with APIC ID in x86 and fix kernel not to reset it its own value
> >>> (i.e. untie mpidr from vcpuid)?
> >>>
> >>> Then later we could move setting mpidr from kvm_arch_init_vcpu() into
> >>> board code which should be setting it, since it knows/defines what topology it has.
> >>
> >> This is a question better asked on the kvmarm list (which I have cc'd),
> >> because that is where the kernel folks hang out...
> >
> > Care to provide some context? Why is this required?
> 
> The device tree spec says we have to tell the guest kernel the
> affinity information for each CPU, in the reg field of the CPU node.
> This device tree blob is created by userspace; the MPIDR the guest
> actually reads is currently created by the kernel. The kernel's
> vgic v3 code may also make assumptions about vcpu_id => affinity
> mappings, I'm not sure.
> 
> In current code:
>  * the kernel has an opinion about vcpu_id => MPIDR mappings
>  * QEMU does too, but it's not the same!
>    (luckily for 8 or fewer CPUs they agree, but this will not
>    do for GICv3 support)
> 
> So either:
>  * QEMU needs to tell the kernel the MPIDR for each vCPU

Does this option work for all configurations?  What if you're running on
a hardware with two clusters [8, 8] and QEMU asks KVM for a [12, 2]
configuration, for example.  Will it not be impossible for KVM to
maintain correct semantics for coherency guarantees across what seems to
the guest to be the same cluster, but really is two different clusters?

Of course, KVM can deny an unsupported configuration, but I am wondering
if we really think anybody will care about the 'model such specific
hardware' aspect with KVM, or if we should only consider the 'I want a
VM with x VCPUs' scenario, in which case the second option below seems
simpler to me.

Are there any concerns to think about wrt. migration here as well?

Thanks,
-Christoffer

>  * QEMU needs to ask the kernel the MPIDR for each vCPU
> 
> Which is better? The latter is simpler and will work with
> existing kernels. The former would let us (for instance)
> use KVM when we're modelling (real world) boards which have
> particular cluster configurations (which might not match the
> kernel's current simplistic "always 16 CPUs at Aff0" setup).
> 
> thanks
> -- PMM
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

Re: should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Christoffer Dall]

Hi,

[sorry for reviving this thread late]

On Tue, Jun 09, 2015 at 12:24:13PM +0100, Peter Maydell wrote:
> On 9 June 2015 at 11:52, Marc Zyngier <marc.zyngier@arm.com> wrote:
> > On 08/06/15 11:52, Peter Maydell wrote:
> >> On 8 June 2015 at 11:32, Igor Mammedov <imammedo@redhat.com> wrote:
> >>> On Thu, 4 Jun 2015 18:17:39 +0100
> >>> Peter Maydell <peter.maydell@linaro.org> wrote:
> >>>> On 4 June 2015 at 17:40, Shlomo Pongratz <shlomopongratz@gmail.com> wrote:
> >>>>  In order for it to work correctly we must use MPIDR values in
> >>>>  the device tree which match the MPIDR values the kernel has picked
> >>>>  for the vcpus, so ask KVM what those values are.
> >>
> >>> Could we set QEMU's generated mpidr in kernel instead of pulling it from kernel,
> >>> like we do with APIC ID in x86 and fix kernel not to reset it its own value
> >>> (i.e. untie mpidr from vcpuid)?
> >>>
> >>> Then later we could move setting mpidr from kvm_arch_init_vcpu() into
> >>> board code which should be setting it, since it knows/defines what topology it has.
> >>
> >> This is a question better asked on the kvmarm list (which I have cc'd),
> >> because that is where the kernel folks hang out...
> >
> > Care to provide some context? Why is this required?
> 
> The device tree spec says we have to tell the guest kernel the
> affinity information for each CPU, in the reg field of the CPU node.
> This device tree blob is created by userspace; the MPIDR the guest
> actually reads is currently created by the kernel. The kernel's
> vgic v3 code may also make assumptions about vcpu_id => affinity
> mappings, I'm not sure.
> 
> In current code:
>  * the kernel has an opinion about vcpu_id => MPIDR mappings
>  * QEMU does too, but it's not the same!
>    (luckily for 8 or fewer CPUs they agree, but this will not
>    do for GICv3 support)
> 
> So either:
>  * QEMU needs to tell the kernel the MPIDR for each vCPU

Does this option work for all configurations?  What if you're running on
a hardware with two clusters [8, 8] and QEMU asks KVM for a [12, 2]
configuration, for example.  Will it not be impossible for KVM to
maintain correct semantics for coherency guarantees across what seems to
the guest to be the same cluster, but really is two different clusters?

Of course, KVM can deny an unsupported configuration, but I am wondering
if we really think anybody will care about the 'model such specific
hardware' aspect with KVM, or if we should only consider the 'I want a
VM with x VCPUs' scenario, in which case the second option below seems
simpler to me.

Are there any concerns to think about wrt. migration here as well?

Thanks,
-Christoffer

>  * QEMU needs to ask the kernel the MPIDR for each vCPU
> 
> Which is better? The latter is simpler and will work with
> existing kernels. The former would let us (for instance)
> use KVM when we're modelling (real world) boards which have
> particular cluster configurations (which might not match the
> kernel's current simplistic "always 16 CPUs at Aff0" setup).
> 
> thanks
> -- PMM
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

Re: [Qemu-devel] should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Peter Maydell]

On 25 June 2015 at 09:00, Christoffer Dall <christoffer.dall@linaro.org> wrote:
> Of course, KVM can deny an unsupported configuration, but I am wondering
> if we really think anybody will care about the 'model such specific
> hardware' aspect with KVM, or if we should only consider the 'I want a
> VM with x VCPUs' scenario, in which case the second option below seems
> simpler to me.

I agree it's not very likely anybody cares about the specific cluster
topology. However if we don't want to support arbitrary topologies
then QEMU is going to end up in the business of editing the user
supplied device tree blob to make its cpu definitions match up with
whatever the kernel provides, which could be pretty tedious.

-- PMM

Re: should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Peter Maydell]

On 25 June 2015 at 09:00, Christoffer Dall <christoffer.dall@linaro.org> wrote:
> Of course, KVM can deny an unsupported configuration, but I am wondering
> if we really think anybody will care about the 'model such specific
> hardware' aspect with KVM, or if we should only consider the 'I want a
> VM with x VCPUs' scenario, in which case the second option below seems
> simpler to me.

I agree it's not very likely anybody cares about the specific cluster
topology. However if we don't want to support arbitrary topologies
then QEMU is going to end up in the business of editing the user
supplied device tree blob to make its cpu definitions match up with
whatever the kernel provides, which could be pretty tedious.

-- PMM

Re: [Qemu-devel] should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Christoffer Dall]

On Thu, Jun 25, 2015 at 10:06:20AM +0100, Peter Maydell wrote:
> On 25 June 2015 at 09:00, Christoffer Dall <christoffer.dall@linaro.org> wrote:
> > Of course, KVM can deny an unsupported configuration, but I am wondering
> > if we really think anybody will care about the 'model such specific
> > hardware' aspect with KVM, or if we should only consider the 'I want a
> > VM with x VCPUs' scenario, in which case the second option below seems
> > simpler to me.
> 
> I agree it's not very likely anybody cares about the specific cluster
> topology. However if we don't want to support arbitrary topologies
> then QEMU is going to end up in the business of editing the user
> supplied device tree blob to make its cpu definitions match up with
> whatever the kernel provides, which could be pretty tedious.
> 
I see, then you can't easily contruct a machine and a DT in one go
before talking to KVM.

Oh well, I don't feel strongly one way or the other.

-Christoffer

Re: should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Christoffer Dall]

On Thu, Jun 25, 2015 at 10:06:20AM +0100, Peter Maydell wrote:
> On 25 June 2015 at 09:00, Christoffer Dall <christoffer.dall@linaro.org> wrote:
> > Of course, KVM can deny an unsupported configuration, but I am wondering
> > if we really think anybody will care about the 'model such specific
> > hardware' aspect with KVM, or if we should only consider the 'I want a
> > VM with x VCPUs' scenario, in which case the second option below seems
> > simpler to me.
> 
> I agree it's not very likely anybody cares about the specific cluster
> topology. However if we don't want to support arbitrary topologies
> then QEMU is going to end up in the business of editing the user
> supplied device tree blob to make its cpu definitions match up with
> whatever the kernel provides, which could be pretty tedious.
> 
I see, then you can't easily contruct a machine and a DT in one go
before talking to KVM.

Oh well, I don't feel strongly one way or the other.

-Christoffer

Re: [Qemu-devel] should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Peter Maydell]

On 25 June 2015 at 13:41, Christoffer Dall <christoffer.dall@linaro.org> wrote:
> On Thu, Jun 25, 2015 at 10:06:20AM +0100, Peter Maydell wrote:
>> I agree it's not very likely anybody cares about the specific cluster
>> topology. However if we don't want to support arbitrary topologies
>> then QEMU is going to end up in the business of editing the user
>> supplied device tree blob to make its cpu definitions match up with
>> whatever the kernel provides, which could be pretty tedious.
>>
> I see, then you can't easily contruct a machine and a DT in one go
> before talking to KVM.

Also the "use upstream's kernel and its associated dtb with a
model that matches real hardware" use case would require either
DTB editing or getting the kernel to handle the topology of
the thing we're modelling.

-- PMM

Re: should KVM or userspace be the one which decides what MIPIDR/affinity values to assign to vcpus?

[Peter Maydell]

On 25 June 2015 at 13:41, Christoffer Dall <christoffer.dall@linaro.org> wrote:
> On Thu, Jun 25, 2015 at 10:06:20AM +0100, Peter Maydell wrote:
>> I agree it's not very likely anybody cares about the specific cluster
>> topology. However if we don't want to support arbitrary topologies
>> then QEMU is going to end up in the business of editing the user
>> supplied device tree blob to make its cpu definitions match up with
>> whatever the kernel provides, which could be pretty tedious.
>>
> I see, then you can't easily contruct a machine and a DT in one go
> before talking to KVM.

Also the "use upstream's kernel and its associated dtb with a
model that matches real hardware" use case would require either
DTB editing or getting the kernel to handle the topology of
the thing we're modelling.

-- PMM