Re: [Qemu-devel] [libvirt] Problem setting CPU topology

Re: [Qemu-devel] [libvirt] Problem setting CPU topology

[Eduardo Habkost]

On Tue, Jul 10, 2012 at 11:54:05AM +0200, Christophe Fergeau wrote:
> On Sat, Jul 07, 2012 at 07:10:53PM +0300, Zeeshan Ali (Khattak) wrote:
> > Hi,
> >    I'm trying to set exact CPU topology to qemu-kvm domains to match
> > host's topology. In my case, host topology is: 1 socket, 2 cores and 2
> > threads. If I set the XML like this:
> > 
> > <domain type='kvm'>
> >   ..
> >   <vcpu placement='static'>4</vcpu>
> >   <os>
> >     <type arch='i686' machine='pc-0.15'>hvm</type>
> >     <boot dev='hd'/>
> >   </os>
> >   <cpu mode='host-model'>
> >     <model fallback='allow'/>
> >     <topology sockets='1' cores='2' threads='2'/>
> >   </cpu>
> > ..
> > 
> > The qemu commandline launched for this domain looks like this:
> > 
> > /usr/bin/qemu-kvm -name fedora17-2 -S -M pc-0.15 -cpu
> > core2duo,+lahf_lm,+rdtscp,+aes,+popcnt,+sse4.2,+sse4.1,+pdcm,+xtpr,+cx16,+tm2,+est,+smx,+vmx,+ds_cpl,+dtes64,+pclmuldq,+pbe,+tm,+ht,+ss,+acpi,+ds
> > -enable-kvm -m 1152 -smp 4,sockets=1,cores=2,threads=2 -uuid
> > c573342b-2876-05b8-098e-6d5314cab062 -nodefconfig -nodefaults -chardev
> > socket,id=charmonitor,path=/home/zeenix/.config/libvirt/qemu/lib/fedora17-2.monitor,server,nowait
> > -mon chardev=charmonitor,id=monitor,mode=control -rtc
> > base=utc,driftfix=slew -no-kvm-pit-reinjection -no-shutdown -no-acpi
> 
> I debugged this together with Zeeshan, and the issue comes from this -no-acpi flag
> which libvirt added because the domain XML was missing
> <features><acpi/></features>
> So in the end that was a bug in Boxes, not really a libvirt/qemu issue.
> Unless libvirt/qemu should be taught that CPU topology won't work without ACPI
> and complain about it?

I don't think it's really impossible for any guest OS to recognize the
CPU topology or boot the other CPUs without ACPI. It looks like it's
just a limitation of (most?) guest OSes. If that's the case, libvirt or
QEMU can't prevent the user from running a (possibly valid)
configuration just because it's not very common.

-- 
Eduardo

Re: [Qemu-devel] [libvirt] Problem setting CPU topology

[Doug Goldstein]

On Tue, Jul 10, 2012 at 2:02 PM, Eduardo Habkost <ehabkost@redhat.com> wrote:
> On Tue, Jul 10, 2012 at 11:54:05AM +0200, Christophe Fergeau wrote:
>> On Sat, Jul 07, 2012 at 07:10:53PM +0300, Zeeshan Ali (Khattak) wrote:
>> > Hi,
>> >    I'm trying to set exact CPU topology to qemu-kvm domains to match
>> > host's topology. In my case, host topology is: 1 socket, 2 cores and 2
>> > threads. If I set the XML like this:
>> >
>> > <domain type='kvm'>
>> >   ..
>> >   <vcpu placement='static'>4</vcpu>
>> >   <os>
>> >     <type arch='i686' machine='pc-0.15'>hvm</type>
>> >     <boot dev='hd'/>
>> >   </os>
>> >   <cpu mode='host-model'>
>> >     <model fallback='allow'/>
>> >     <topology sockets='1' cores='2' threads='2'/>
>> >   </cpu>
>> > ..
>> >
>> > The qemu commandline launched for this domain looks like this:
>> >
>> > /usr/bin/qemu-kvm -name fedora17-2 -S -M pc-0.15 -cpu
>> > core2duo,+lahf_lm,+rdtscp,+aes,+popcnt,+sse4.2,+sse4.1,+pdcm,+xtpr,+cx16,+tm2,+est,+smx,+vmx,+ds_cpl,+dtes64,+pclmuldq,+pbe,+tm,+ht,+ss,+acpi,+ds
>> > -enable-kvm -m 1152 -smp 4,sockets=1,cores=2,threads=2 -uuid
>> > c573342b-2876-05b8-098e-6d5314cab062 -nodefconfig -nodefaults -chardev
>> > socket,id=charmonitor,path=/home/zeenix/.config/libvirt/qemu/lib/fedora17-2.monitor,server,nowait
>> > -mon chardev=charmonitor,id=monitor,mode=control -rtc
>> > base=utc,driftfix=slew -no-kvm-pit-reinjection -no-shutdown -no-acpi
>>
>> I debugged this together with Zeeshan, and the issue comes from this -no-acpi flag
>> which libvirt added because the domain XML was missing
>> <features><acpi/></features>
>> So in the end that was a bug in Boxes, not really a libvirt/qemu issue.
>> Unless libvirt/qemu should be taught that CPU topology won't work without ACPI
>> and complain about it?
>
> I don't think it's really impossible for any guest OS to recognize the
> CPU topology or boot the other CPUs without ACPI. It looks like it's
> just a limitation of (most?) guest OSes. If that's the case, libvirt or
> QEMU can't prevent the user from running a (possibly valid)
> configuration just because it's not very common.
>
> --
> Eduardo
>

Isn't MPS and APIC support required for a guest OS to really handle
the topology correctly? Now days ACPI provides most of that
information so its likely that QEMU doesn't support plain old MPS +
APIC. Its also possible that QEMU does support this but the kernel got
confused because I see the CPUID +acpi specified in the command line
while -no-acpi is on the command line as you noted.

-- 
Doug Goldstein

Re: [Qemu-devel] [libvirt] Problem setting CPU topology

[Eduardo Habkost]

On Tue, Jul 10, 2012 at 09:19:37PM -0500, Doug Goldstein wrote:
> On Tue, Jul 10, 2012 at 2:02 PM, Eduardo Habkost <ehabkost@redhat.com> wrote:
> > On Tue, Jul 10, 2012 at 11:54:05AM +0200, Christophe Fergeau wrote:
> >> On Sat, Jul 07, 2012 at 07:10:53PM +0300, Zeeshan Ali (Khattak) wrote:
> >> > Hi,
> >> >    I'm trying to set exact CPU topology to qemu-kvm domains to match
> >> > host's topology. In my case, host topology is: 1 socket, 2 cores and 2
> >> > threads. If I set the XML like this:
> >> >
> >> > <domain type='kvm'>
> >> >   ..
> >> >   <vcpu placement='static'>4</vcpu>
> >> >   <os>
> >> >     <type arch='i686' machine='pc-0.15'>hvm</type>
> >> >     <boot dev='hd'/>
> >> >   </os>
> >> >   <cpu mode='host-model'>
> >> >     <model fallback='allow'/>
> >> >     <topology sockets='1' cores='2' threads='2'/>
> >> >   </cpu>
> >> > ..
> >> >
> >> > The qemu commandline launched for this domain looks like this:
> >> >
> >> > /usr/bin/qemu-kvm -name fedora17-2 -S -M pc-0.15 -cpu
> >> > core2duo,+lahf_lm,+rdtscp,+aes,+popcnt,+sse4.2,+sse4.1,+pdcm,+xtpr,+cx16,+tm2,+est,+smx,+vmx,+ds_cpl,+dtes64,+pclmuldq,+pbe,+tm,+ht,+ss,+acpi,+ds
> >> > -enable-kvm -m 1152 -smp 4,sockets=1,cores=2,threads=2 -uuid
> >> > c573342b-2876-05b8-098e-6d5314cab062 -nodefconfig -nodefaults -chardev
> >> > socket,id=charmonitor,path=/home/zeenix/.config/libvirt/qemu/lib/fedora17-2.monitor,server,nowait
> >> > -mon chardev=charmonitor,id=monitor,mode=control -rtc
> >> > base=utc,driftfix=slew -no-kvm-pit-reinjection -no-shutdown -no-acpi
> >>
> >> I debugged this together with Zeeshan, and the issue comes from this -no-acpi flag
> >> which libvirt added because the domain XML was missing
> >> <features><acpi/></features>
> >> So in the end that was a bug in Boxes, not really a libvirt/qemu issue.
> >> Unless libvirt/qemu should be taught that CPU topology won't work without ACPI
> >> and complain about it?
> >
> > I don't think it's really impossible for any guest OS to recognize the
> > CPU topology or boot the other CPUs without ACPI. It looks like it's
> > just a limitation of (most?) guest OSes. If that's the case, libvirt or
> > QEMU can't prevent the user from running a (possibly valid)
> > configuration just because it's not very common.
> >
> > --
> > Eduardo
> >
> 
> Isn't MPS and APIC support required for a guest OS to really handle
> the topology correctly? Now days ACPI provides most of that
> information so its likely that QEMU doesn't support plain old MPS +
> APIC. Its also possible that QEMU does support this but the kernel got
> confused because I see the CPUID +acpi specified in the command line
> while -no-acpi is on the command line as you noted.
> 

QEMU + Seabios have plain old MPS working, and from my tests it looks
like:

- Seabios is able to init all VCPUs correctly.
- The MP-table contains only 1 entry for each CPU package (not for each
  core+thread). I don't understand why exactly, but I am assuming that
  this is a feature, not a bug (there's some discussion about it at
  [1]).
- The CPUID information exposed by QEMU reflect the topology correctly.

With that information, nothing really prevents the guest from trying to
init the other cores/threads in each package after parsing the MP-table.
On the other hand, I don't think there's any spec that recommends an
algorithm for that (as the MPS spec is too old to have any mention of
multi-core or hyper-threading).

[1] http://www.coreboot.org/pipermail/coreboot/2009-November/054119.html

-- 
Eduardo