[Qemu-devel] 3.1.0-rc{0,1} doesn't start

[Qemu-devel] 3.1.0-rc{0,1} doesn't start

[balducci]

hello

I'm building qemu from source and happily using it since a bit
(2.3.0)

Since 3.1.0-rc0 (including latest 3.1.0-rc1) I'm no more able to start
qemu, getting:

    ----8<----
    install:115> qemu
    qemu: error: failed to set MSR 0x10a to 0x0
    qemu: /home/balducci/tmp/install-us-d/qemu-3.1.0-rc1.d/qemu-3.1.0-rc0/target/i386/kvm.c:2185: kvm_put_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.
    Aborted
    ---->8----

I have no idea about what the reason might be, apologies.

Actually, I have found a recent (2018-10-16) post which might be
related to this (it mentions the same error message from qemu):
https://lkml.org/lkml/2018/10/16/440; but I'm not in the position to
go through. AFAICS, the commit mentioned in the link is present in the
4.19.2 kernel I'm using, so...?

I can add that 3.0.0 works nicely (everything else unchanged,
including running kernel 4.19.2)

OTOH, 3.1.0-rc0 dumps the same error message if I boot into 4.17.14 or
4.18.16 kernels.

I enclose my specs hoping that somebody can spot where the problem
might be. I will be happy to send any other detail which might be
useful.

I suspect that this might be some problem on my side, as I couldn't
find any similar report (apart some old (qemu-2.8.50) threads, that
didn't help)


thanks a lot in advance for any hint/help

ciao
gabriele


Here are my specs:

# --------------------------------
# command to run qemu is:
qemu -m 2G /opt/windog  \
    -accel kvm,thread=multi \
    -netdev user,id=net0,smb=/home/balducci \
    -device rtl8139,netdev=net0

# --------------------------------
# qemu build configuration:
--prefix=/opt/stow.d/versions/qemu-3.1.0-rc1/usr
--libdir=/opt/stow.d/versions/qemu-3.1.0-rc1/usr/lib64
--sysconfdir=/opt/stow.d/versions/qemu-3.1.0-rc1/etc
--localstatedir=/var/run
--docdir=/opt/stow.d/versions/qemu-3.1.0-rc1/usr/share/doc/qemu
--target-list=x86_64-softmmu
--audio-drv-list=alsa


# --------------------------------
install:154> uname -sr
Linux 4.19.2

# --------------------------------
install:155> cat /proc/cpuinfo
processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 21
model           : 48
model name      : AMD Athlon(tm) X4 860K Quad Core Processor
stepping        : 1
microcode       : 0x6003106
cpu MHz         : 3473.492
cache size      : 2048 KB
physical id     : 0
siblings        : 4
core id         : 0
cpu cores       : 2
apicid          : 16
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb bpext ptsc cpb hw_pstate ssbd vmmcall fsgsbase bmi1 xsaveopt arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov
bugs            : fxsave_leak sysret_ss_attrs null_seg spectre_v1 spectre_v2 spec_store_bypass
bogomips        : 7380.73
TLB size        : 1536 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro [13]

[...cpus 1 2 3 omitted...]


# --------------------------------
install:156> egrep KVM .config-4.19.2
CONFIG_HAVE_KVM=y
CONFIG_HAVE_KVM_IRQCHIP=y
CONFIG_HAVE_KVM_IRQFD=y
CONFIG_HAVE_KVM_IRQ_ROUTING=y
CONFIG_HAVE_KVM_EVENTFD=y
CONFIG_KVM_MMIO=y
CONFIG_KVM_ASYNC_PF=y
CONFIG_HAVE_KVM_MSI=y
CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT=y
CONFIG_KVM_VFIO=y
CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT=y
CONFIG_KVM_COMPAT=y
CONFIG_HAVE_KVM_IRQ_BYPASS=y
CONFIG_KVM=y
# CONFIG_KVM_INTEL is not set
CONFIG_KVM_AMD=y

[of course, I can send the whole kernel configuration file, if needed]

Re: [Qemu-devel] 3.1.0-rc{0,1} doesn't start

[Dr. David Alan Gilbert]

cc'ing in Paolo, Eduardo and Robert

* balducci@units.it (balducci@units.it) wrote:
> hello
> 
> I'm building qemu from source and happily using it since a bit
> (2.3.0)
> 
> Since 3.1.0-rc0 (including latest 3.1.0-rc1) I'm no more able to start
> qemu, getting:
> 
>     ----8<----
>     install:115> qemu
>     qemu: error: failed to set MSR 0x10a to 0x0
>     qemu: /home/balducci/tmp/install-us-d/qemu-3.1.0-rc1.d/qemu-3.1.0-rc0/target/i386/kvm.c:2185: kvm_put_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.
>     Aborted
>     ---->8----
> 
> I have no idea about what the reason might be, apologies.

Thanks for the detailed error report.
MSR 0x10a is one of the new ones from the recent round of security
things:
    #define MSR_IA32_ARCH_CAPABILITIES      0x10a

That only went in recently so is probably why you're seeing it
in 3.1.

> Actually, I have found a recent (2018-10-16) post which might be
> related to this (it mentions the same error message from qemu):
> https://lkml.org/lkml/2018/10/16/440; but I'm not in the position to
> go through. AFAICS, the commit mentioned in the link is present in the
> 4.19.2 kernel I'm using, so...?

That's a different MSR number so is probably unrelated.

> I can add that 3.0.0 works nicely (everything else unchanged,
> including running kernel 4.19.2)
> 
> OTOH, 3.1.0-rc0 dumps the same error message if I boot into 4.17.14 or
> 4.18.16 kernels.
> 
> I enclose my specs hoping that somebody can spot where the problem
> might be. I will be happy to send any other detail which might be
> useful.
> 
> I suspect that this might be some problem on my side, as I couldn't
> find any similar report (apart some old (qemu-2.8.50) threads, that
> didn't help)

Not necessarily; can you tell me:
  a) At what point does it fail - immediately when booting the guest?
     Some time during the boot? Later?
  b) What guest does it happen on?

> thanks a lot in advance for any hint/help
> 
> ciao
> gabriele
> 
> 
> Here are my specs:
> 
> # --------------------------------
> # command to run qemu is:
> qemu -m 2G /opt/windog  \
>     -accel kvm,thread=multi \
>     -netdev user,id=net0,smb=/home/balducci \
>     -device rtl8139,netdev=net0
> 
> # --------------------------------
> # qemu build configuration:
> --prefix=/opt/stow.d/versions/qemu-3.1.0-rc1/usr
> --libdir=/opt/stow.d/versions/qemu-3.1.0-rc1/usr/lib64
> --sysconfdir=/opt/stow.d/versions/qemu-3.1.0-rc1/etc
> --localstatedir=/var/run
> --docdir=/opt/stow.d/versions/qemu-3.1.0-rc1/usr/share/doc/qemu
> --target-list=x86_64-softmmu
> --audio-drv-list=alsa
> 
> 
> # --------------------------------
> install:154> uname -sr
> Linux 4.19.2
> 
> # --------------------------------
> install:155> cat /proc/cpuinfo
> processor       : 0
> vendor_id       : AuthenticAMD
> cpu family      : 21
> model           : 48
> model name      : AMD Athlon(tm) X4 860K Quad Core Processor

OK, older AMD cpu, so a little less tested perhaps, maybe why
others haven't sene it.

> stepping        : 1
> microcode       : 0x6003106
> cpu MHz         : 3473.492
> cache size      : 2048 KB
> physical id     : 0
> siblings        : 4
> core id         : 0
> cpu cores       : 2
> apicid          : 16
> initial apicid  : 0
> fpu             : yes
> fpu_exception   : yes
> cpuid level     : 13
> wp              : yes
> flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb bpext ptsc cpb hw_pstate ssbd vmmcall fsgsbase bmi1 xsaveopt arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov
> bugs            : fxsave_leak sysret_ss_attrs null_seg spectre_v1 spectre_v2 spec_store_bypass
> bogomips        : 7380.73
> TLB size        : 1536 4K pages
> clflush size    : 64
> cache_alignment : 64
> address sizes   : 48 bits physical, 48 bits virtual
> power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro [13]
> 
> [...cpus 1 2 3 omitted...]
> 
> 
> # --------------------------------
> install:156> egrep KVM .config-4.19.2
> CONFIG_HAVE_KVM=y
> CONFIG_HAVE_KVM_IRQCHIP=y
> CONFIG_HAVE_KVM_IRQFD=y
> CONFIG_HAVE_KVM_IRQ_ROUTING=y
> CONFIG_HAVE_KVM_EVENTFD=y
> CONFIG_KVM_MMIO=y
> CONFIG_KVM_ASYNC_PF=y
> CONFIG_HAVE_KVM_MSI=y
> CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT=y
> CONFIG_KVM_VFIO=y
> CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT=y
> CONFIG_KVM_COMPAT=y
> CONFIG_HAVE_KVM_IRQ_BYPASS=y
> CONFIG_KVM=y
> # CONFIG_KVM_INTEL is not set
> CONFIG_KVM_AMD=y
> 
> [of course, I can send the whole kernel configuration file, if needed]

Dave

> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] 3.1.0-rc{0,1} doesn't start

[balducci]

hi

thanks for taking the time to reply

Dr. David Alan Gilbert writes:
> > I suspect that this might be some problem on my side, as I couldn't
> > find any similar report (apart some old (qemu-2.8.50) threads, that
> > didn't help)
>
> Not necessarily; can you tell me:
>   a) At what point does it fail - immediately when booting the guest?
>      Some time during the boot? Later?
>   b) What guest does it happen on?

a) the error happens almost immediately; I mean: when I run qemu from an
   xterm, it doesn't even popup its window: it just dumps the error
   message to the terminal and stops
b) the guest is an old windows XP OS; but, as I say above, all goes as
   if qemu doesn't even load the OS image (at least this is my
   impression)

Meantime, I have tried to (quick&dirty) disable the error
catching/asserting in i386/kvm.c:

install:41> diff ./qemu-3.1.0-rc1/target/i386/kvm.c.MSR_HACK ./qemu-3.1.0-rc1/target/i386/kvm.c
2205c2205
<     if (ret < cpu->kvm_msr_buf->nmsrs) {
---
>     if (1==0) {
2211c2211
<     assert(ret == cpu->kvm_msr_buf->nmsrs);
---
>     assert(1==1);
2524c2524
<     if (ret < cpu->kvm_msr_buf->nmsrs) {
---
>     if (1==0) {
2530c2530
<     assert(ret == cpu->kvm_msr_buf->nmsrs);
---
>     assert(1==1);

and that makes qemu start and work without apparent problems.
Of course, that is a crude and risky (I guess) workaround...

thanks again

ciao
-gabriele

Re: [Qemu-devel] 3.1.0-rc{0,1} doesn't start

[Dr. David Alan Gilbert]

* balducci@units.it (balducci@units.it) wrote:
> hi
> 
> thanks for taking the time to reply
> 
> Dr. David Alan Gilbert writes:
> > > I suspect that this might be some problem on my side, as I couldn't
> > > find any similar report (apart some old (qemu-2.8.50) threads, that
> > > didn't help)
> >
> > Not necessarily; can you tell me:
> >   a) At what point does it fail - immediately when booting the guest?
> >      Some time during the boot? Later?
> >   b) What guest does it happen on?
> 
> a) the error happens almost immediately; I mean: when I run qemu from an
>    xterm, it doesn't even popup its window: it just dumps the error
>    message to the terminal and stops
> b) the guest is an old windows XP OS; but, as I say above, all goes as
>    if qemu doesn't even load the OS image (at least this is my
>    impression)

Thanks.

> Meantime, I have tried to (quick&dirty) disable the error
> catching/asserting in i386/kvm.c:
> 
> install:41> diff ./qemu-3.1.0-rc1/target/i386/kvm.c.MSR_HACK ./qemu-3.1.0-rc1/target/i386/kvm.c
> 2205c2205
> <     if (ret < cpu->kvm_msr_buf->nmsrs) {
> ---
> >     if (1==0) {
> 2211c2211
> <     assert(ret == cpu->kvm_msr_buf->nmsrs);
> ---
> >     assert(1==1);
> 2524c2524
> <     if (ret < cpu->kvm_msr_buf->nmsrs) {
> ---
> >     if (1==0) {
> 2530c2530
> <     assert(ret == cpu->kvm_msr_buf->nmsrs);
> ---
> >     assert(1==1);
> 
> and that makes qemu start and work without apparent problems.
> Of course, that is a crude and risky (I guess) workaround...

Yes, the gotcha is that the call writes a whole bunch of MSRs
and when one fails all the ones after it don't get written either;
so you get lots of weird problems due to whichever other MSRs
are wrong.

Dave

> thanks again
> 
> ciao
> -gabriele
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] 3.1.0-rc{0,1} doesn't start

[Dr. David Alan Gilbert]

* balducci@units.it (balducci@units.it) wrote:
> hi
> 
> thanks for taking the time to reply
> 
> Dr. David Alan Gilbert writes:
> > > I suspect that this might be some problem on my side, as I couldn't
> > > find any similar report (apart some old (qemu-2.8.50) threads, that
> > > didn't help)
> >
> > Not necessarily; can you tell me:
> >   a) At what point does it fail - immediately when booting the guest?
> >      Some time during the boot? Later?
> >   b) What guest does it happen on?
> 
> a) the error happens almost immediately; I mean: when I run qemu from an
>    xterm, it doesn't even popup its window: it just dumps the error
>    message to the terminal and stops
> b) the guest is an old windows XP OS; but, as I say above, all goes as
>    if qemu doesn't even load the OS image (at least this is my
>    impression)
> 
A colleague has confirmed this on his FX-8320 on Fedora 29 with the
virt-next repo;  so it's nothing that's special about your machine;
it's 3.1 that really doesn't like the old AMDs.

Dave

> Meantime, I have tried to (quick&dirty) disable the error
> catching/asserting in i386/kvm.c:
> 
> install:41> diff ./qemu-3.1.0-rc1/target/i386/kvm.c.MSR_HACK ./qemu-3.1.0-rc1/target/i386/kvm.c
> 2205c2205
> <     if (ret < cpu->kvm_msr_buf->nmsrs) {
> ---
> >     if (1==0) {
> 2211c2211
> <     assert(ret == cpu->kvm_msr_buf->nmsrs);
> ---
> >     assert(1==1);
> 2524c2524
> <     if (ret < cpu->kvm_msr_buf->nmsrs) {
> ---
> >     if (1==0) {
> 2530c2530
> <     assert(ret == cpu->kvm_msr_buf->nmsrs);
> ---
> >     assert(1==1);
> 
> and that makes qemu start and work without apparent problems.
> Of course, that is a crude and risky (I guess) workaround...
> 
> thanks again
> 
> ciao
> -gabriele
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] 3.1.0-rc{0,1} doesn't start

[balducci]

> A colleague has confirmed this on his FX-8320 on Fedora 29 with the
> virt-next repo;  so it's nothing that's special about your machine;
> it's 3.1 that really doesn't like the old AMDs.

ouch! does this mean that I must stop upgrading qemu or do you think
that some development work will be directed towards this "back
compatibility" issue?

(apologies for being totally unable to help with patches/code...I know
zero about all the qemu business)

thanks a lot
ciao
-gabriele

Re: [Qemu-devel] 3.1.0-rc{0,1} doesn't start

[Dr. David Alan Gilbert]

* balducci@units.it (balducci@units.it) wrote:
> > A colleague has confirmed this on his FX-8320 on Fedora 29 with the
> > virt-next repo;  so it's nothing that's special about your machine;
> > it's 3.1 that really doesn't like the old AMDs.
> 
> ouch! does this mean that I must stop upgrading qemu or do you think
> that some development work will be directed towards this "back
> compatibility" issue?

It'll probably be fixed; thanks to your report I've added it to the
known issues part of the 3.1rc list.

> (apologies for being totally unable to help with patches/code...I know
> zero about all the qemu business)

Those bits of the MSRs are pretty specialised as well; so I've asked
a few people more familiar with them to jump in.

Dave

> thanks a lot
> ciao
> -gabriele
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] 3.1.0-rc{0,1} doesn't start

[Cole Robinson]

On 11/19/2018 01:35 PM, Dr. David Alan Gilbert wrote:
> * balducci@units.it (balducci@units.it) wrote:
>> hi
>>
>> thanks for taking the time to reply
>>
>> Dr. David Alan Gilbert writes:
>>>> I suspect that this might be some problem on my side, as I couldn't
>>>> find any similar report (apart some old (qemu-2.8.50) threads, that
>>>> didn't help)
>>>
>>> Not necessarily; can you tell me:
>>>    a) At what point does it fail - immediately when booting the guest?
>>>       Some time during the boot? Later?
>>>    b) What guest does it happen on?
>>
>> a) the error happens almost immediately; I mean: when I run qemu from an
>>     xterm, it doesn't even popup its window: it just dumps the error
>>     message to the terminal and stops
>> b) the guest is an old windows XP OS; but, as I say above, all goes as
>>     if qemu doesn't even load the OS image (at least this is my
>>     impression)
>>
> A colleague has confirmed this on his FX-8320 on Fedora 29 with the
> virt-next repo;  so it's nothing that's special about your machine;
> it's 3.1 that really doesn't like the old AMDs.
> 

Also we received a similar report in Fedora bugzilla:

https://bugzilla.redhat.com/show_bug.cgi?id=1651021

Thanks,
Cole

Re: [Qemu-devel] 3.1.0-rc{0,1} doesn't start

[Bandan Das]

balducci@units.it writes:

> hello
>
> I'm building qemu from source and happily using it since a bit
> (2.3.0)
>
> Since 3.1.0-rc0 (including latest 3.1.0-rc1) I'm no more able to start
> qemu, getting:
>
>     ----8<----
>     install:115> qemu
>     qemu: error: failed to set MSR 0x10a to 0x0
>     qemu: /home/balducci/tmp/install-us-d/qemu-3.1.0-rc1.d/qemu-3.1.0-rc0/target/i386/kvm.c:2185: kvm_put_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.
>     Aborted
>     ---->8----
>
I believe the check on whether MSR_IA32_ARCH_CAPABILITIES is present is
incomplete because it can return 0 for data. Can you try this:

diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index f524e7d929..4878ffb90b 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -2002,14 +2002,9 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
 #endif
 
     /* If host supports feature MSR, write down. */
-    if (kvm_feature_msrs) {
-        int i;
-        for (i = 0; i < kvm_feature_msrs->nmsrs; i++)
-            if (kvm_feature_msrs->indices[i] == MSR_IA32_ARCH_CAPABILITIES) {
-                kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
+    if (kvm_arch_get_supported_msr_feature(kvm_state, MSR_IA32_ARCH_CAPABILITIES)) {
+        kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
                               env->features[FEAT_ARCH_CAPABILITIES]);
-                break;
-            }
     }
 
     /*


> I have no idea about what the reason might be, apologies.
>
> Actually, I have found a recent (2018-10-16) post which might be
> related to this (it mentions the same error message from qemu):
> https://lkml.org/lkml/2018/10/16/440; but I'm not in the position to
> go through. AFAICS, the commit mentioned in the link is present in the
> 4.19.2 kernel I'm using, so...?
>
> I can add that 3.0.0 works nicely (everything else unchanged,
> including running kernel 4.19.2)
>
> OTOH, 3.1.0-rc0 dumps the same error message if I boot into 4.17.14 or
> 4.18.16 kernels.
>
> I enclose my specs hoping that somebody can spot where the problem
> might be. I will be happy to send any other detail which might be
> useful.
>
> I suspect that this might be some problem on my side, as I couldn't
> find any similar report (apart some old (qemu-2.8.50) threads, that
> didn't help)
>
>
> thanks a lot in advance for any hint/help
>
> ciao
> gabriele
>
>
> Here are my specs:
>
> # --------------------------------
> # command to run qemu is:
> qemu -m 2G /opt/windog  \
>     -accel kvm,thread=multi \
>     -netdev user,id=net0,smb=/home/balducci \
>     -device rtl8139,netdev=net0
>
> # --------------------------------
> # qemu build configuration:
> --prefix=/opt/stow.d/versions/qemu-3.1.0-rc1/usr
> --libdir=/opt/stow.d/versions/qemu-3.1.0-rc1/usr/lib64
> --sysconfdir=/opt/stow.d/versions/qemu-3.1.0-rc1/etc
> --localstatedir=/var/run
> --docdir=/opt/stow.d/versions/qemu-3.1.0-rc1/usr/share/doc/qemu
> --target-list=x86_64-softmmu
> --audio-drv-list=alsa
>
>
> # --------------------------------
> install:154> uname -sr
> Linux 4.19.2
>
> # --------------------------------
> install:155> cat /proc/cpuinfo
> processor       : 0
> vendor_id       : AuthenticAMD
> cpu family      : 21
> model           : 48
> model name      : AMD Athlon(tm) X4 860K Quad Core Processor
> stepping        : 1
> microcode       : 0x6003106
> cpu MHz         : 3473.492
> cache size      : 2048 KB
> physical id     : 0
> siblings        : 4
> core id         : 0
> cpu cores       : 2
> apicid          : 16
> initial apicid  : 0
> fpu             : yes
> fpu_exception   : yes
> cpuid level     : 13
> wp              : yes
> flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb bpext ptsc cpb hw_pstate ssbd vmmcall fsgsbase bmi1 xsaveopt arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov
> bugs            : fxsave_leak sysret_ss_attrs null_seg spectre_v1 spectre_v2 spec_store_bypass
> bogomips        : 7380.73
> TLB size        : 1536 4K pages
> clflush size    : 64
> cache_alignment : 64
> address sizes   : 48 bits physical, 48 bits virtual
> power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro [13]
>
> [...cpus 1 2 3 omitted...]
>
>
> # --------------------------------
> install:156> egrep KVM .config-4.19.2
> CONFIG_HAVE_KVM=y
> CONFIG_HAVE_KVM_IRQCHIP=y
> CONFIG_HAVE_KVM_IRQFD=y
> CONFIG_HAVE_KVM_IRQ_ROUTING=y
> CONFIG_HAVE_KVM_EVENTFD=y
> CONFIG_KVM_MMIO=y
> CONFIG_KVM_ASYNC_PF=y
> CONFIG_HAVE_KVM_MSI=y
> CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT=y
> CONFIG_KVM_VFIO=y
> CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT=y
> CONFIG_KVM_COMPAT=y
> CONFIG_HAVE_KVM_IRQ_BYPASS=y
> CONFIG_KVM=y
> # CONFIG_KVM_INTEL is not set
> CONFIG_KVM_AMD=y
>
> [of course, I can send the whole kernel configuration file, if needed]

Re: [Qemu-devel] 3.1.0-rc{0,1} doesn't start

[Eduardo Habkost]

On Mon, Nov 19, 2018 at 04:55:13PM -0500, Bandan Das wrote:
> balducci@units.it writes:
> 
> > hello
> >
> > I'm building qemu from source and happily using it since a bit
> > (2.3.0)
> >
> > Since 3.1.0-rc0 (including latest 3.1.0-rc1) I'm no more able to start
> > qemu, getting:
> >
> >     ----8<----
> >     install:115> qemu
> >     qemu: error: failed to set MSR 0x10a to 0x0
> >     qemu: /home/balducci/tmp/install-us-d/qemu-3.1.0-rc1.d/qemu-3.1.0-rc0/target/i386/kvm.c:2185: kvm_put_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.
> >     Aborted
> >     ---->8----
> >
> I believe the check on whether MSR_IA32_ARCH_CAPABILITIES is present is
> incomplete because it can return 0 for data. Can you try this:
> 
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index f524e7d929..4878ffb90b 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -2002,14 +2002,9 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
>  #endif
>  
>      /* If host supports feature MSR, write down. */
> -    if (kvm_feature_msrs) {
> -        int i;
> -        for (i = 0; i < kvm_feature_msrs->nmsrs; i++)
> -            if (kvm_feature_msrs->indices[i] == MSR_IA32_ARCH_CAPABILITIES) {
> -                kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
> +    if (kvm_arch_get_supported_msr_feature(kvm_state, MSR_IA32_ARCH_CAPABILITIES)) {
> +        kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
>                                env->features[FEAT_ARCH_CAPABILITIES]);

kvm_arch_get_supported_msr_feature() will return the value of the
MSR on the host side (kvm/x86.c:kvm_get_msr_feature()).  Having
it return non-zero doesn't mean KVM's
svm_set_msr(MSR_IA32_ARCH_CAPABILITIES) will work.

If the MSR doesn't work on KVM_SET_MSRS, it is not supposed to
appear on KVM_GET_MSR_INDEX_LIST (even if it appears on
KVM_GET_MSR_FEATURE_INDEX_LIST).  QEMU must check
KVM_GET_MSR_INDEX_LIST too before including the MSR on the
KVM_SET_MSRS call.

> -                break;
> -            }
>      }
>  
>      /*
> 
> 
> > I have no idea about what the reason might be, apologies.
> >
> > Actually, I have found a recent (2018-10-16) post which might be
> > related to this (it mentions the same error message from qemu):
> > https://lkml.org/lkml/2018/10/16/440; but I'm not in the position to
> > go through. AFAICS, the commit mentioned in the link is present in the
> > 4.19.2 kernel I'm using, so...?
> >
> > I can add that 3.0.0 works nicely (everything else unchanged,
> > including running kernel 4.19.2)
> >
> > OTOH, 3.1.0-rc0 dumps the same error message if I boot into 4.17.14 or
> > 4.18.16 kernels.
> >
> > I enclose my specs hoping that somebody can spot where the problem
> > might be. I will be happy to send any other detail which might be
> > useful.
> >
> > I suspect that this might be some problem on my side, as I couldn't
> > find any similar report (apart some old (qemu-2.8.50) threads, that
> > didn't help)
> >
> >
> > thanks a lot in advance for any hint/help
> >
> > ciao
> > gabriele
> >
> >
> > Here are my specs:
> >
> > # --------------------------------
> > # command to run qemu is:
> > qemu -m 2G /opt/windog  \
> >     -accel kvm,thread=multi \
> >     -netdev user,id=net0,smb=/home/balducci \
> >     -device rtl8139,netdev=net0
> >
> > # --------------------------------
> > # qemu build configuration:
> > --prefix=/opt/stow.d/versions/qemu-3.1.0-rc1/usr
> > --libdir=/opt/stow.d/versions/qemu-3.1.0-rc1/usr/lib64
> > --sysconfdir=/opt/stow.d/versions/qemu-3.1.0-rc1/etc
> > --localstatedir=/var/run
> > --docdir=/opt/stow.d/versions/qemu-3.1.0-rc1/usr/share/doc/qemu
> > --target-list=x86_64-softmmu
> > --audio-drv-list=alsa
> >
> >
> > # --------------------------------
> > install:154> uname -sr
> > Linux 4.19.2
> >
> > # --------------------------------
> > install:155> cat /proc/cpuinfo
> > processor       : 0
> > vendor_id       : AuthenticAMD
> > cpu family      : 21
> > model           : 48
> > model name      : AMD Athlon(tm) X4 860K Quad Core Processor
> > stepping        : 1
> > microcode       : 0x6003106
> > cpu MHz         : 3473.492
> > cache size      : 2048 KB
> > physical id     : 0
> > siblings        : 4
> > core id         : 0
> > cpu cores       : 2
> > apicid          : 16
> > initial apicid  : 0
> > fpu             : yes
> > fpu_exception   : yes
> > cpuid level     : 13
> > wp              : yes
> > flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb bpext ptsc cpb hw_pstate ssbd vmmcall fsgsbase bmi1 xsaveopt arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold overflow_recov
> > bugs            : fxsave_leak sysret_ss_attrs null_seg spectre_v1 spectre_v2 spec_store_bypass
> > bogomips        : 7380.73
> > TLB size        : 1536 4K pages
> > clflush size    : 64
> > cache_alignment : 64
> > address sizes   : 48 bits physical, 48 bits virtual
> > power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro [13]
> >
> > [...cpus 1 2 3 omitted...]
> >
> >
> > # --------------------------------
> > install:156> egrep KVM .config-4.19.2
> > CONFIG_HAVE_KVM=y
> > CONFIG_HAVE_KVM_IRQCHIP=y
> > CONFIG_HAVE_KVM_IRQFD=y
> > CONFIG_HAVE_KVM_IRQ_ROUTING=y
> > CONFIG_HAVE_KVM_EVENTFD=y
> > CONFIG_KVM_MMIO=y
> > CONFIG_KVM_ASYNC_PF=y
> > CONFIG_HAVE_KVM_MSI=y
> > CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT=y
> > CONFIG_KVM_VFIO=y
> > CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT=y
> > CONFIG_KVM_COMPAT=y
> > CONFIG_HAVE_KVM_IRQ_BYPASS=y
> > CONFIG_KVM=y
> > # CONFIG_KVM_INTEL is not set
> > CONFIG_KVM_AMD=y
> >
> > [of course, I can send the whole kernel configuration file, if needed]

-- 
Eduardo

Re: [Qemu-devel] 3.1.0-rc{0,1} doesn't start

[balducci]

hello

> incomplete because it can return 0 for data. Can you try this:
>
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index f524e7d929..4878ffb90b 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -2002,14 +2002,9 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
>  #endif
>  
>      /* If host supports feature MSR, write down. */
> -    if (kvm_feature_msrs) {
> -        int i;
> -        for (i = 0; i < kvm_feature_msrs->nmsrs; i++)
> -            if (kvm_feature_msrs->indices[i] == MSR_IA32_ARCH_CAPABILITIES) 
> {
> -                kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
> +    if (kvm_arch_get_supported_msr_feature(kvm_state, MSR_IA32_ARCH_CAPABILI
> TIES)) {
> +        kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
>                                env->features[FEAT_ARCH_CAPABILITIES]);
> -                break;
> -            }
>      }
>  
>      /*

I'm getting the same error message at start time (patch applied to
official  qemu-3.1.0-rc1 tarball):

----8<----
qemu: error: failed to set MSR 0x10a to 0x0
qemu: /home/balducci/tmp/install-us-d/qemu-3.1.0-rc1.d/qemu-3.1.0-rc1/target/i386/kvm.c:2206: kvm_put_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.
Aborted
---->8----

thanks a lot
ciao
-gabriele

Re: [Qemu-devel] 3.1.0-rc{0,1} doesn't start

[Paolo Bonzini]

On 20/11/18 00:00, Eduardo Habkost wrote:
> On Mon, Nov 19, 2018 at 04:55:13PM -0500, Bandan Das wrote:
>> balducci@units.it writes:
>>
>>> hello
>>>
>>> I'm building qemu from source and happily using it since a bit
>>> (2.3.0)
>>>
>>> Since 3.1.0-rc0 (including latest 3.1.0-rc1) I'm no more able to start
>>> qemu, getting:
>>>
>>>     ----8<----
>>>     install:115> qemu
>>>     qemu: error: failed to set MSR 0x10a to 0x0
>>>     qemu: /home/balducci/tmp/install-us-d/qemu-3.1.0-rc1.d/qemu-3.1.0-rc0/target/i386/kvm.c:2185: kvm_put_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.
>>>     Aborted
>>>     ---->8----
>>>
>> I believe the check on whether MSR_IA32_ARCH_CAPABILITIES is present is
>> incomplete because it can return 0 for data. Can you try this:
>>
>> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
>> index f524e7d929..4878ffb90b 100644
>> --- a/target/i386/kvm.c
>> +++ b/target/i386/kvm.c
>> @@ -2002,14 +2002,9 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
>>  #endif
>>  
>>      /* If host supports feature MSR, write down. */
>> -    if (kvm_feature_msrs) {
>> -        int i;
>> -        for (i = 0; i < kvm_feature_msrs->nmsrs; i++)
>> -            if (kvm_feature_msrs->indices[i] == MSR_IA32_ARCH_CAPABILITIES) {
>> -                kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
>> +    if (kvm_arch_get_supported_msr_feature(kvm_state, MSR_IA32_ARCH_CAPABILITIES)) {
>> +        kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
>>                                env->features[FEAT_ARCH_CAPABILITIES]);
> 
> kvm_arch_get_supported_msr_feature() will return the value of the
> MSR on the host side (kvm/x86.c:kvm_get_msr_feature()).  Having
> it return non-zero doesn't mean KVM's
> svm_set_msr(MSR_IA32_ARCH_CAPABILITIES) will work.
> 
> If the MSR doesn't work on KVM_SET_MSRS, it is not supposed to
> appear on KVM_GET_MSR_INDEX_LIST (even if it appears on
> KVM_GET_MSR_FEATURE_INDEX_LIST).  QEMU must check
> KVM_GET_MSR_INDEX_LIST too before including the MSR on the
> KVM_SET_MSRS call.

Yes, this is a KVM bug.  For 3.1, making it "writable if nonzero" is a
valid workaround, because AMD processors always return 0. It's not the
prettiest thing, but it works.

Paolo

Re: [Qemu-devel] 3.1.0-rc{0,1} doesn't start

[Eduardo Habkost]

On Tue, Nov 20, 2018 at 07:40:01PM +0100, Paolo Bonzini wrote:
> On 20/11/18 00:00, Eduardo Habkost wrote:
> > On Mon, Nov 19, 2018 at 04:55:13PM -0500, Bandan Das wrote:
> >> balducci@units.it writes:
> >>
> >>> hello
> >>>
> >>> I'm building qemu from source and happily using it since a bit
> >>> (2.3.0)
> >>>
> >>> Since 3.1.0-rc0 (including latest 3.1.0-rc1) I'm no more able to start
> >>> qemu, getting:
> >>>
> >>>     ----8<----
> >>>     install:115> qemu
> >>>     qemu: error: failed to set MSR 0x10a to 0x0
> >>>     qemu: /home/balducci/tmp/install-us-d/qemu-3.1.0-rc1.d/qemu-3.1.0-rc0/target/i386/kvm.c:2185: kvm_put_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.
> >>>     Aborted
> >>>     ---->8----
> >>>
> >> I believe the check on whether MSR_IA32_ARCH_CAPABILITIES is present is
> >> incomplete because it can return 0 for data. Can you try this:
> >>
> >> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> >> index f524e7d929..4878ffb90b 100644
> >> --- a/target/i386/kvm.c
> >> +++ b/target/i386/kvm.c
> >> @@ -2002,14 +2002,9 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
> >>  #endif
> >>  
> >>      /* If host supports feature MSR, write down. */
> >> -    if (kvm_feature_msrs) {
> >> -        int i;
> >> -        for (i = 0; i < kvm_feature_msrs->nmsrs; i++)
> >> -            if (kvm_feature_msrs->indices[i] == MSR_IA32_ARCH_CAPABILITIES) {
> >> -                kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
> >> +    if (kvm_arch_get_supported_msr_feature(kvm_state, MSR_IA32_ARCH_CAPABILITIES)) {
> >> +        kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
> >>                                env->features[FEAT_ARCH_CAPABILITIES]);
> > 
> > kvm_arch_get_supported_msr_feature() will return the value of the
> > MSR on the host side (kvm/x86.c:kvm_get_msr_feature()).  Having
> > it return non-zero doesn't mean KVM's
> > svm_set_msr(MSR_IA32_ARCH_CAPABILITIES) will work.
> > 
> > If the MSR doesn't work on KVM_SET_MSRS, it is not supposed to
> > appear on KVM_GET_MSR_INDEX_LIST (even if it appears on
> > KVM_GET_MSR_FEATURE_INDEX_LIST).  QEMU must check
> > KVM_GET_MSR_INDEX_LIST too before including the MSR on the
> > KVM_SET_MSRS call.
> 
> Yes, this is a KVM bug.  For 3.1, making it "writable if nonzero" is a
> valid workaround, because AMD processors always return 0. It's not the
> prettiest thing, but it works.

So it looks like we have 2 bugs?  Is KVM incorrectly returning
the MSR on KVM_GET_MSR_INDEX_LIST?

I thought the bug was only in QEMU, which is supposed to be
checking KVM_GET_MSR_INDEX_LIST before setting the MSR (it is
only checking KVM_GET_MSR_FEATURE_INDEX_LIST).

-- 
Eduardo

Re: [Qemu-devel] 3.1.0-rc{0,1} doesn't start

[Paolo Bonzini]

On 20/11/18 19:48, Eduardo Habkost wrote:
> On Tue, Nov 20, 2018 at 07:40:01PM +0100, Paolo Bonzini wrote:
>> On 20/11/18 00:00, Eduardo Habkost wrote:
>>> On Mon, Nov 19, 2018 at 04:55:13PM -0500, Bandan Das wrote:
>>>> balducci@units.it writes:
>>>>
>>>>> hello
>>>>>
>>>>> I'm building qemu from source and happily using it since a bit
>>>>> (2.3.0)
>>>>>
>>>>> Since 3.1.0-rc0 (including latest 3.1.0-rc1) I'm no more able to start
>>>>> qemu, getting:
>>>>>
>>>>>     ----8<----
>>>>>     install:115> qemu
>>>>>     qemu: error: failed to set MSR 0x10a to 0x0
>>>>>     qemu: /home/balducci/tmp/install-us-d/qemu-3.1.0-rc1.d/qemu-3.1.0-rc0/target/i386/kvm.c:2185: kvm_put_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.
>>>>>     Aborted
>>>>>     ---->8----
>>>>>
>>>> I believe the check on whether MSR_IA32_ARCH_CAPABILITIES is present is
>>>> incomplete because it can return 0 for data. Can you try this:
>>>>
>>>> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
>>>> index f524e7d929..4878ffb90b 100644
>>>> --- a/target/i386/kvm.c
>>>> +++ b/target/i386/kvm.c
>>>> @@ -2002,14 +2002,9 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
>>>>  #endif
>>>>  
>>>>      /* If host supports feature MSR, write down. */
>>>> -    if (kvm_feature_msrs) {
>>>> -        int i;
>>>> -        for (i = 0; i < kvm_feature_msrs->nmsrs; i++)
>>>> -            if (kvm_feature_msrs->indices[i] == MSR_IA32_ARCH_CAPABILITIES) {
>>>> -                kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
>>>> +    if (kvm_arch_get_supported_msr_feature(kvm_state, MSR_IA32_ARCH_CAPABILITIES)) {
>>>> +        kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
>>>>                                env->features[FEAT_ARCH_CAPABILITIES]);
>>>
>>> kvm_arch_get_supported_msr_feature() will return the value of the
>>> MSR on the host side (kvm/x86.c:kvm_get_msr_feature()).  Having
>>> it return non-zero doesn't mean KVM's
>>> svm_set_msr(MSR_IA32_ARCH_CAPABILITIES) will work.
>>>
>>> If the MSR doesn't work on KVM_SET_MSRS, it is not supposed to
>>> appear on KVM_GET_MSR_INDEX_LIST (even if it appears on
>>> KVM_GET_MSR_FEATURE_INDEX_LIST).  QEMU must check
>>> KVM_GET_MSR_INDEX_LIST too before including the MSR on the
>>> KVM_SET_MSRS call.
>>
>> Yes, this is a KVM bug.  For 3.1, making it "writable if nonzero" is a
>> valid workaround, because AMD processors always return 0. It's not the
>> prettiest thing, but it works.
> 
> So it looks like we have 2 bugs?  Is KVM incorrectly returning
> the MSR on KVM_GET_MSR_INDEX_LIST?
> 
> I thought the bug was only in QEMU, which is supposed to be
> checking KVM_GET_MSR_INDEX_LIST before setting the MSR (it is
> only checking KVM_GET_MSR_FEATURE_INDEX_LIST).

Oh, wait.  I didn't understand that.  Then yeah, fixing QEMU is enough.
 KVM is not reporting the MSR in the list on AMD, but it should (and it
should implement the MSR, of course!).

Paolo

Re: [Qemu-devel] 3.1.0-rc{0,1} doesn't start

[Bandan Das]

Paolo Bonzini <pbonzini@redhat.com> writes:
...
>>>>> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
>>>>> index f524e7d929..4878ffb90b 100644
>>>>> --- a/target/i386/kvm.c
>>>>> +++ b/target/i386/kvm.c
>>>>> @@ -2002,14 +2002,9 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
>>>>>  #endif
>>>>>  
>>>>>      /* If host supports feature MSR, write down. */
>>>>> -    if (kvm_feature_msrs) {
>>>>> -        int i;
>>>>> -        for (i = 0; i < kvm_feature_msrs->nmsrs; i++)
>>>>> -            if (kvm_feature_msrs->indices[i] == MSR_IA32_ARCH_CAPABILITIES) {
>>>>> -                kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
>>>>> +    if (kvm_arch_get_supported_msr_feature(kvm_state, MSR_IA32_ARCH_CAPABILITIES)) {
>>>>> +        kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
>>>>>                                env->features[FEAT_ARCH_CAPABILITIES]);
>>>>
>>>> kvm_arch_get_supported_msr_feature() will return the value of the
>>>> MSR on the host side (kvm/x86.c:kvm_get_msr_feature()).  Having
>>>> it return non-zero doesn't mean KVM's
>>>> svm_set_msr(MSR_IA32_ARCH_CAPABILITIES) will work.
>>>>
>>>> If the MSR doesn't work on KVM_SET_MSRS, it is not supposed to
>>>> appear on KVM_GET_MSR_INDEX_LIST (even if it appears on
>>>> KVM_GET_MSR_FEATURE_INDEX_LIST).  QEMU must check
>>>> KVM_GET_MSR_INDEX_LIST too before including the MSR on the
>>>> KVM_SET_MSRS call.
>>>
>>> Yes, this is a KVM bug.  For 3.1, making it "writable if nonzero" is a
>>> valid workaround, because AMD processors always return 0. It's not the
>>> prettiest thing, but it works.
>> 
>> So it looks like we have 2 bugs?  Is KVM incorrectly returning
>> the MSR on KVM_GET_MSR_INDEX_LIST?
>> 
>> I thought the bug was only in QEMU, which is supposed to be
>> checking KVM_GET_MSR_INDEX_LIST before setting the MSR (it is
>> only checking KVM_GET_MSR_FEATURE_INDEX_LIST).
>
> Oh, wait.  I didn't understand that.  Then yeah, fixing QEMU is enough.
>  KVM is not reporting the MSR in the list on AMD, but it should (and it
> should implement the MSR, of course!).
>

Until AMD implements somethinig, on the KVM side, shouldn't we let
kvm_x86_ops->get_msr_feature(msr)) handle MSR_IA32_ARCH_CAPABILITIES ?

> Paolo

Re: [Qemu-devel] 3.1.0-rc{0,1} doesn't start

[Bandan Das]

balducci@units.it writes:

> hello
>
>> incomplete because it can return 0 for data. Can you try this:
>>
>> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
>> index f524e7d929..4878ffb90b 100644
>> --- a/target/i386/kvm.c
>> +++ b/target/i386/kvm.c
>> @@ -2002,14 +2002,9 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
>>  #endif
>>  
>>      /* If host supports feature MSR, write down. */
>> -    if (kvm_feature_msrs) {
>> -        int i;
>> -        for (i = 0; i < kvm_feature_msrs->nmsrs; i++)
>> -            if (kvm_feature_msrs->indices[i] == MSR_IA32_ARCH_CAPABILITIES) 
>> {
>> -                kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
>> +    if (kvm_arch_get_supported_msr_feature(kvm_state, MSR_IA32_ARCH_CAPABILI
>> TIES)) {
>> +        kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
>>                                env->features[FEAT_ARCH_CAPABILITIES]);
>> -                break;
>> -            }
>>      }
>>  
>>      /*
>
Sorry Gabriele, Eduardo and Paolo pointed out that the patch was incomplete.
I am not sure however how you are getting non zero data with the rdmsr on an AMD.
Can you please try this instead ?

diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index f524e7d929..cfec559aa3 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -95,6 +95,7 @@ static bool has_msr_xss;
 static bool has_msr_spec_ctrl;
 static bool has_msr_virt_ssbd;
 static bool has_msr_smi_count;
+static bool has_msr_arch_capabs;
 
 static uint32_t has_architectural_pmu_version;
 static uint32_t num_architectural_pmu_gp_counters;
@@ -1481,6 +1482,9 @@ static int kvm_get_supported_msrs(KVMState *s)
                 case MSR_VIRT_SSBD:
                     has_msr_virt_ssbd = true;
                     break;
+                case MSR_IA32_ARCH_CAPABILITIES:
+                    has_msr_arch_capabs = true;
+                    break;
                 }
             }
         }
@@ -2002,14 +2006,10 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
 #endif
 
     /* If host supports feature MSR, write down. */
-    if (kvm_feature_msrs) {
-        int i;
-        for (i = 0; i < kvm_feature_msrs->nmsrs; i++)
-            if (kvm_feature_msrs->indices[i] == MSR_IA32_ARCH_CAPABILITIES) {
+    if (kvm_arch_get_supported_msr_feature(kvm_state, MSR_IA32_ARCH_CAPABILITIES)
+        && has_msr_arch_capabs) {
                 kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
                               env->features[FEAT_ARCH_CAPABILITIES]);
-                break;
-            }
     }
 
     /*

Re: [Qemu-devel] 3.1.0-rc{0,1} doesn't start

[balducci]

> Sorry Gabriele, Eduardo and Paolo pointed out that the patch was incomplete.
> I am not sure however how you are getting non zero data with the rdmsr on an 
> AMD.
> Can you please try this instead ?
>
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index f524e7d929..cfec559aa3 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -95,6 +95,7 @@ static bool has_msr_xss;
>  static bool has_msr_spec_ctrl;
>  static bool has_msr_virt_ssbd;
>  static bool has_msr_smi_count;
> +static bool has_msr_arch_capabs;
>  
>  static uint32_t has_architectural_pmu_version;
>  static uint32_t num_architectural_pmu_gp_counters;
> @@ -1481,6 +1482,9 @@ static int kvm_get_supported_msrs(KVMState *s)
>                  case MSR_VIRT_SSBD:
>                      has_msr_virt_ssbd = true;
>                      break;
> +                case MSR_IA32_ARCH_CAPABILITIES:
> +                    has_msr_arch_capabs = true;
> +                    break;
>                  }
>              }
>          }
> @@ -2002,14 +2006,10 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
>  #endif
>  
>      /* If host supports feature MSR, write down. */
> -    if (kvm_feature_msrs) {
> -        int i;
> -        for (i = 0; i < kvm_feature_msrs->nmsrs; i++)
> -            if (kvm_feature_msrs->indices[i] == MSR_IA32_ARCH_CAPABILITIES) 
> {
> +    if (kvm_arch_get_supported_msr_feature(kvm_state, MSR_IA32_ARCH_CAPABILI
> TIES)
> +        && has_msr_arch_capabs) {
>                  kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
>                                env->features[FEAT_ARCH_CAPABILITIES]);
> -                break;
> -            }
>      }
>  
>      /*

this one works: qemu starts without any complaint and behaves as usual
(AFAICS)

thank you very much for working on this

ciao
gabriele

Re: [Qemu-devel] 3.1.0-rc{0,1} doesn't start

[Eduardo Habkost]

On Tue, Nov 20, 2018 at 11:28:59PM -0500, Bandan Das wrote:
> balducci@units.it writes:
> 
> > hello
> >
> >> incomplete because it can return 0 for data. Can you try this:
> >>
> >> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> >> index f524e7d929..4878ffb90b 100644
> >> --- a/target/i386/kvm.c
> >> +++ b/target/i386/kvm.c
> >> @@ -2002,14 +2002,9 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
> >>  #endif
> >>  
> >>      /* If host supports feature MSR, write down. */
> >> -    if (kvm_feature_msrs) {
> >> -        int i;
> >> -        for (i = 0; i < kvm_feature_msrs->nmsrs; i++)
> >> -            if (kvm_feature_msrs->indices[i] == MSR_IA32_ARCH_CAPABILITIES) 
> >> {
> >> -                kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
> >> +    if (kvm_arch_get_supported_msr_feature(kvm_state, MSR_IA32_ARCH_CAPABILI
> >> TIES)) {
> >> +        kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
> >>                                env->features[FEAT_ARCH_CAPABILITIES]);
> >> -                break;
> >> -            }
> >>      }
> >>  
> >>      /*
> >
> Sorry Gabriele, Eduardo and Paolo pointed out that the patch was incomplete.
> I am not sure however how you are getting non zero data with the rdmsr on an AMD.
> Can you please try this instead ?
> 
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index f524e7d929..cfec559aa3 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -95,6 +95,7 @@ static bool has_msr_xss;
>  static bool has_msr_spec_ctrl;
>  static bool has_msr_virt_ssbd;
>  static bool has_msr_smi_count;
> +static bool has_msr_arch_capabs;
>  
>  static uint32_t has_architectural_pmu_version;
>  static uint32_t num_architectural_pmu_gp_counters;
> @@ -1481,6 +1482,9 @@ static int kvm_get_supported_msrs(KVMState *s)
>                  case MSR_VIRT_SSBD:
>                      has_msr_virt_ssbd = true;
>                      break;
> +                case MSR_IA32_ARCH_CAPABILITIES:
> +                    has_msr_arch_capabs = true;
> +                    break;
>                  }
>              }
>          }
> @@ -2002,14 +2006,10 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
>  #endif
>  
>      /* If host supports feature MSR, write down. */
> -    if (kvm_feature_msrs) {
> -        int i;
> -        for (i = 0; i < kvm_feature_msrs->nmsrs; i++)
> -            if (kvm_feature_msrs->indices[i] == MSR_IA32_ARCH_CAPABILITIES) {
> +    if (kvm_arch_get_supported_msr_feature(kvm_state, MSR_IA32_ARCH_CAPABILITIES)
> +        && has_msr_arch_capabs) {

Unless we have a KVM bug too, has_msr_arch_capabs should be
enough.

If has_msr_arch_capabs is not enough, we'll need to add a comment
explaining why, or we risk reintroducing the same bug later.


>                  kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
>                                env->features[FEAT_ARCH_CAPABILITIES]);
> -                break;
> -            }
>      }
>  
>      /*
> 
> 

-- 
Eduardo

Re: [Qemu-devel] 3.1.0-rc{0,1} doesn't start

[Paolo Bonzini]

On 21/11/18 05:05, Bandan Das wrote:
>> Oh, wait.  I didn't understand that.  Then yeah, fixing QEMU is enough.
>>  KVM is not reporting the MSR in the list on AMD, but it should (and it
>> should implement the MSR, of course!).
>>
> Until AMD implements somethinig, on the KVM side, shouldn't we let
> kvm_x86_ops->get_msr_feature(msr)) handle MSR_IA32_ARCH_CAPABILITIES ?
> 

Either is okay.  But the MSR is not really vendor dependent, and can be
added even to older models.  So it may make sense to move it to generic
code.

Paolo

[Qemu-devel] [PATCH] kvm: Use KVM_GET_MSR_INDEX_LIST for MSR_IA32_ARCH_CAPABILITIES support

[Bandan Das]

When writing to guest's MSR_IA32_ARCH_CAPABILITIES, check whether it's
supported in the guest using the KVM_GET_MSR_INDEX_LIST ioctl.

Fixes: d86f963694df27f11b3681ffd225c9362de1b634
Suggested-by: Eduardo Habkost <ehabkost@redhat.com>
Tested-by: balducci@units.it
Signed-off-by: Bandan Das <bsd@redhat.com>
---
 target/i386/kvm.c | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)

diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index f524e7d929..3d6739a2b2 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -95,6 +95,7 @@ static bool has_msr_xss;
 static bool has_msr_spec_ctrl;
 static bool has_msr_virt_ssbd;
 static bool has_msr_smi_count;
+static bool has_msr_arch_capabs;
 
 static uint32_t has_architectural_pmu_version;
 static uint32_t num_architectural_pmu_gp_counters;
@@ -1481,6 +1482,9 @@ static int kvm_get_supported_msrs(KVMState *s)
                 case MSR_VIRT_SSBD:
                     has_msr_virt_ssbd = true;
                     break;
+                case MSR_IA32_ARCH_CAPABILITIES:
+                    has_msr_arch_capabs = true;
+                    break;
                 }
             }
         }
@@ -2002,14 +2006,9 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
 #endif
 
     /* If host supports feature MSR, write down. */
-    if (kvm_feature_msrs) {
-        int i;
-        for (i = 0; i < kvm_feature_msrs->nmsrs; i++)
-            if (kvm_feature_msrs->indices[i] == MSR_IA32_ARCH_CAPABILITIES) {
-                kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
-                              env->features[FEAT_ARCH_CAPABILITIES]);
-                break;
-            }
+    if (has_msr_arch_capabs) {
+        kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
+                          env->features[FEAT_ARCH_CAPABILITIES]);
     }
 
     /*
-- 
2.14.5

Re: [Qemu-devel] [PATCH] kvm: Use KVM_GET_MSR_INDEX_LIST for MSR_IA32_ARCH_CAPABILITIES support

[Eduardo Habkost]

On Sun, Nov 25, 2018 at 11:17:28PM -0500, Bandan Das wrote:
> 
> When writing to guest's MSR_IA32_ARCH_CAPABILITIES, check whether it's
> supported in the guest using the KVM_GET_MSR_INDEX_LIST ioctl.
> 
> Fixes: d86f963694df27f11b3681ffd225c9362de1b634
> Suggested-by: Eduardo Habkost <ehabkost@redhat.com>
> Tested-by: balducci@units.it
> Signed-off-by: Bandan Das <bsd@redhat.com>

Thanks!  Queued for 3.1.

-- 
Eduardo

Re: [Qemu-devel] [PATCH] kvm: Use KVM_GET_MSR_INDEX_LIST for MSR_IA32_ARCH_CAPABILITIES support

[Hu, Robert]

Thanks for fixing. Sorry I missed the whole discussion for I hadn't checked my
linux.intel.com account for a long time.

Best Regards,
Robert Hoo

> -----Original Message-----
> From: Qemu-devel <qemu-devel-bounces+robert.hu=intel.com@nongnu.org>
> On Behalf Of Bandan Das
> Sent: Monday, November 26, 2018 12:17
> To: Eduardo Habkost <ehabkost@redhat.com>; Paolo Bonzini
> <pbonzini@redhat.com>
> Cc: balducci@units.it; Cole Robinson <crobinso@redhat.com>; qemu-
> devel@nongnu.org; Laine Stump <laine@redhat.com>; David Alan Gilbert
> <dgilbert@redhat.com>
> Subject: [Qemu-devel] [PATCH] kvm: Use KVM_GET_MSR_INDEX_LIST for
> MSR_IA32_ARCH_CAPABILITIES support
> 
> 
> When writing to guest's MSR_IA32_ARCH_CAPABILITIES, check whether it's
> supported in the guest using the KVM_GET_MSR_INDEX_LIST ioctl.
> 
> Fixes: d86f963694df27f11b3681ffd225c9362de1b634
> Suggested-by: Eduardo Habkost <ehabkost@redhat.com>
> Tested-by: balducci@units.it
> Signed-off-by: Bandan Das <bsd@redhat.com>
> ---
>  target/i386/kvm.c | 15 +++++++--------
>  1 file changed, 7 insertions(+), 8 deletions(-)
> 
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c index
> f524e7d929..3d6739a2b2 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -95,6 +95,7 @@ static bool has_msr_xss;  static bool has_msr_spec_ctrl;
> static bool has_msr_virt_ssbd;  static bool has_msr_smi_count;
> +static bool has_msr_arch_capabs;
> 
>  static uint32_t has_architectural_pmu_version;  static uint32_t
> num_architectural_pmu_gp_counters;
> @@ -1481,6 +1482,9 @@ static int kvm_get_supported_msrs(KVMState *s)
>                  case MSR_VIRT_SSBD:
>                      has_msr_virt_ssbd = true;
>                      break;
> +                case MSR_IA32_ARCH_CAPABILITIES:
> +                    has_msr_arch_capabs = true;
> +                    break;
>                  }
>              }
>          }
> @@ -2002,14 +2006,9 @@ static int kvm_put_msrs(X86CPU *cpu, int level)
> #endif
> 
>      /* If host supports feature MSR, write down. */
> -    if (kvm_feature_msrs) {
> -        int i;
> -        for (i = 0; i < kvm_feature_msrs->nmsrs; i++)
> -            if (kvm_feature_msrs->indices[i] == MSR_IA32_ARCH_CAPABILITIES) {
> -                kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
> -                              env->features[FEAT_ARCH_CAPABILITIES]);
> -                break;
> -            }
> +    if (has_msr_arch_capabs) {

[Robert Hoo] 
Why not as your original "if (kvm_arch_get_supported_msr_feature(kvm_state, MSR_IA32_ARCH_CAPABILITIES))"?
checking if there is the MSR feature?

> +        kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
> +                          env->features[FEAT_ARCH_CAPABILITIES]);
>      }
> 
>      /*
> --
> 2.14.5
>