linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* qemu-kvm loops after kernel udpate
@ 2012-09-11 18:11 Jiri Slaby
  2012-09-11 19:03 ` Marcelo Tosatti
  0 siblings, 1 reply; 16+ messages in thread
From: Jiri Slaby @ 2012-09-11 18:11 UTC (permalink / raw)
  To: Avi Kivity, Marcelo Tosatti, kvm, LKML, Jiri Slaby

Hi,

it looks like an update from next-20120824 to next-20120910 makes kvm
defunct. When I try to run qemu, it loops forever without printing
anything on the monitor.

-no-kvm makes it indeed work.

Cmdline I use:
qemu-kvm -k en-us -usbdevice tablet -balloon virtio -hda IMAGE -smp 2 -m
1000M -net user -net nic,model=e1000 -usb -serial pty

Before I start investigating further, like biscection, is this a known
issue already and fix available somewhere?

thanks,
-- 
js
suse labs

^ permalink raw reply	[flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate
  2012-09-11 18:11 qemu-kvm loops after kernel udpate Jiri Slaby
@ 2012-09-11 19:03 ` Marcelo Tosatti
  2012-09-11 19:41   ` Jiri Slaby
  0 siblings, 1 reply; 16+ messages in thread
From: Marcelo Tosatti @ 2012-09-11 19:03 UTC (permalink / raw)
  To: Jiri Slaby; +Cc: Avi Kivity, kvm, LKML, Jiri Slaby

On Tue, Sep 11, 2012 at 08:11:36PM +0200, Jiri Slaby wrote:
> Hi,
> 
> it looks like an update from next-20120824 to next-20120910 makes kvm
> defunct. When I try to run qemu, it loops forever without printing
> anything on the monitor.
> 
> -no-kvm makes it indeed work.
> 
> Cmdline I use:
> qemu-kvm -k en-us -usbdevice tablet -balloon virtio -hda IMAGE -smp 2 -m
> 1000M -net user -net nic,model=e1000 -usb -serial pty
> 
> Before I start investigating further, like biscection, is this a known
> issue already and fix available somewhere?
> 
> thanks,

Its not a known issue. 'info registers' (for both vcpus) and
'x /20i $eip' might help track it down. 


^ permalink raw reply	[flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate
  2012-09-11 19:03 ` Marcelo Tosatti
@ 2012-09-11 19:41   ` Jiri Slaby
  2012-09-12  8:06     ` Avi Kivity
  0 siblings, 1 reply; 16+ messages in thread
From: Jiri Slaby @ 2012-09-11 19:41 UTC (permalink / raw)
  To: Marcelo Tosatti; +Cc: Avi Kivity, kvm, LKML, Jiri Slaby

On 09/11/2012 09:03 PM, Marcelo Tosatti wrote:
> On Tue, Sep 11, 2012 at 08:11:36PM +0200, Jiri Slaby wrote:
>> Hi,
>>
>> it looks like an update from next-20120824 to next-20120910 makes kvm
>> defunct. When I try to run qemu, it loops forever without printing
>> anything on the monitor.
>>
>> -no-kvm makes it indeed work.
>>
>> Cmdline I use:
>> qemu-kvm -k en-us -usbdevice tablet -balloon virtio -hda IMAGE -smp 2 -m
>> 1000M -net user -net nic,model=e1000 -usb -serial pty
>>
>> Before I start investigating further, like biscection, is this a known
>> issue already and fix available somewhere?
> 
> Its not a known issue. 'info registers' (for both vcpus) and
> 'x /20i $eip' might help track it down. 

I don't think it's in a state with something loaded.

(qemu) cpu 0
(qemu) info registers
EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000623
ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000
EIP=0000fff0 EFL=00010002 [-------] CPL=3 II=0 A20=1 SMM=0 HLT=0
ES =0000 00000000 0000ffff 00009300
CS =f000 000f0000 0000ffff 0000f300
SS =0000 00000000 0000ffff 0000f300
DS =0000 00000000 0000ffff 00009300
FS =0000 00000000 0000ffff 00009300
GS =0000 00000000 0000ffff 00009300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT=     00000000 0000ffff
IDT=     00000000 0000ffff
CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000
DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
XMM00=00000000000000000000000000000000
XMM01=00000000000000000000000000000000
XMM02=00000000000000000000000000000000
XMM03=00000000000000000000000000000000
XMM04=00000000000000000000000000000000
XMM05=00000000000000000000000000000000
XMM06=00000000000000000000000000000000
XMM07=00000000000000000000000000000000
(qemu) cpu 1
(qemu) info registers
EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000623
ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000
EIP=0000fff0 EFL=00000002 [-------] CPL=3 II=0 A20=1 SMM=0 HLT=0
ES =0000 00000000 0000ffff 00009300
CS =f000 000f0000 0000ffff 0000f300
SS =0000 00000000 0000ffff 0000f300
DS =0000 00000000 0000ffff 00009300
FS =0000 00000000 0000ffff 00009300
GS =0000 00000000 0000ffff 00009300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT=     00000000 0000ffff
IDT=     00000000 0000ffff
CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000
DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
XMM00=00000000000000000000000000000000
XMM01=00000000000000000000000000000000
XMM02=00000000000000000000000000000000
XMM03=00000000000000000000000000000000
XMM04=00000000000000000000000000000000
XMM05=00000000000000000000000000000000
XMM06=00000000000000000000000000000000
XMM07=00000000000000000000000000000000

(qemu) x /20i $eip
0x000000000000fff0:  add    %al,(%bx,%si)
0x000000000000fff2:  add    %al,(%bx,%si)
0x000000000000fff4:  add    %al,(%bx,%si)
0x000000000000fff6:  add    %al,(%bx,%si)
0x000000000000fff8:  add    %al,(%bx,%si)
0x000000000000fffa:  add    %al,(%bx,%si)
0x000000000000fffc:  add    %al,(%bx,%si)
0x000000000000fffe:  add    %al,(%bx,%si)
0x0000000000010000:  add    %al,(%bx,%si)
0x0000000000010002:  add    %al,(%bx,%si)
0x0000000000010004:  add    %al,(%bx,%si)
0x0000000000010006:  add    %al,(%bx,%si)
0x0000000000010008:  add    %al,(%bx,%si)
0x000000000001000a:  add    %al,(%bx,%si)
0x000000000001000c:  add    %al,(%bx,%si)
0x000000000001000e:  add    %al,(%bx,%si)
0x0000000000010010:  add    %al,(%bx,%si)
0x0000000000010012:  add    %al,(%bx,%si)
0x0000000000010014:  add    %al,(%bx,%si)
0x0000000000010016:  add    %al,(%bx,%si)

thanks,
-- 
js
suse labs

^ permalink raw reply	[flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate
  2012-09-11 19:41   ` Jiri Slaby
@ 2012-09-12  8:06     ` Avi Kivity
  2012-09-12  8:13       ` Jiri Slaby
  0 siblings, 1 reply; 16+ messages in thread
From: Avi Kivity @ 2012-09-12  8:06 UTC (permalink / raw)
  To: Jiri Slaby; +Cc: Marcelo Tosatti, kvm, LKML, Jiri Slaby

On 09/11/2012 10:41 PM, Jiri Slaby wrote:
> On 09/11/2012 09:03 PM, Marcelo Tosatti wrote:
>> On Tue, Sep 11, 2012 at 08:11:36PM +0200, Jiri Slaby wrote:
>>> Hi,
>>>
>>> it looks like an update from next-20120824 to next-20120910 makes kvm
>>> defunct. When I try to run qemu, it loops forever without printing
>>> anything on the monitor.
>>>
>>> -no-kvm makes it indeed work.
>>>
>>> Cmdline I use:
>>> qemu-kvm -k en-us -usbdevice tablet -balloon virtio -hda IMAGE -smp 2 -m
>>> 1000M -net user -net nic,model=e1000 -usb -serial pty
>>>
>>> Before I start investigating further, like biscection, is this a known
>>> issue already and fix available somewhere?
>> 
>> Its not a known issue. 'info registers' (for both vcpus) and
>> 'x /20i $eip' might help track it down. 
> 
> I don't think it's in a state with something loaded.
> 
> (qemu) cpu 0
> (qemu) info registers
> EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000623
> ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000
> EIP=0000fff0 EFL=00010002 [-------] CPL=3 II=0 A20=1 SMM=0 HLT=0
> ES =0000 00000000 0000ffff 00009300
> CS =f000 000f0000 0000ffff 0000f300

This is before the first instruction is executed.

You're on an Intel host, yes?  Please provide the output of vmxcap
(http://goo.gl/c5lUO), and a snapshot of kvm_stat while the guest is hung.


-- 
error compiling committee.c: too many arguments to function

^ permalink raw reply	[flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate
  2012-09-12  8:06     ` Avi Kivity
@ 2012-09-12  8:13       ` Jiri Slaby
  2012-09-12  8:18         ` Avi Kivity
  0 siblings, 1 reply; 16+ messages in thread
From: Jiri Slaby @ 2012-09-12  8:13 UTC (permalink / raw)
  To: Avi Kivity; +Cc: Jiri Slaby, Marcelo Tosatti, kvm, LKML

On 09/12/2012 10:06 AM, Avi Kivity wrote:
> On 09/11/2012 10:41 PM, Jiri Slaby wrote:
>> On 09/11/2012 09:03 PM, Marcelo Tosatti wrote:
>>> On Tue, Sep 11, 2012 at 08:11:36PM +0200, Jiri Slaby wrote:
>>>> Hi,
>>>>
>>>> it looks like an update from next-20120824 to next-20120910 makes kvm
>>>> defunct. When I try to run qemu, it loops forever without printing
>>>> anything on the monitor.
>>>>
>>>> -no-kvm makes it indeed work.
>>>>
>>>> Cmdline I use:
>>>> qemu-kvm -k en-us -usbdevice tablet -balloon virtio -hda IMAGE -smp 2 -m
>>>> 1000M -net user -net nic,model=e1000 -usb -serial pty
>>>>
>>>> Before I start investigating further, like biscection, is this a known
>>>> issue already and fix available somewhere?
>>>
>>> Its not a known issue. 'info registers' (for both vcpus) and
>>> 'x /20i $eip' might help track it down. 
>>
>> I don't think it's in a state with something loaded.
>>
>> (qemu) cpu 0
>> (qemu) info registers
>> EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000623
>> ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000
>> EIP=0000fff0 EFL=00010002 [-------] CPL=3 II=0 A20=1 SMM=0 HLT=0
>> ES =0000 00000000 0000ffff 00009300
>> CS =f000 000f0000 0000ffff 0000f300
> 
> This is before the first instruction is executed.
> 
> You're on an Intel host, yes?

Yes.

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 15
model name      : Intel(R) Core(TM)2 Duo CPU     E6850  @ 3.00GHz
stepping        : 11
microcode       : 0xba
...
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
syscall nx lm constant_tsc arch_perfmon pebs bts nopl aperfmperf pni
dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm lahf_lm
dtherm tpr_shadow vnmi flexpriority

>  Please provide the output of vmxcap
> (http://goo.gl/c5lUO),

pin-based controls
  External interrupt exiting               yes
  NMI exiting                              yes
  Virtual NMIs                             yes
  Activate VMX-preemption timer            no
primary processor-based controls
  Interrupt window exiting                 yes
  Use TSC offsetting                       yes
  HLT exiting                              yes
  INVLPG exiting                           yes
  MWAIT exiting                            yes
  RDPMC exiting                            yes
  RDTSC exiting                            yes
  CR3-load exiting                         forced
  CR3-store exiting                        forced
  CR8-load exiting                         yes
  CR8-store exiting                        yes
  Use TPR shadow                           yes
  NMI-window exiting                       yes
  MOV-DR exiting                           yes
  Unconditional I/O exiting                yes
  Use I/O bitmaps                          yes
  Monitor trap flag                        no
  Use MSR bitmaps                          yes
  MONITOR exiting                          yes
  PAUSE exiting                            yes
  Activate secondary control               yes
secondary processor-based controls
  Virtualize APIC accesses                 yes
  Enable EPT                               no
  Descriptor-table exiting                 no
  Virtualize x2APIC mode                   no
  Enable VPID                              no
  WBINVD exiting                           no
  Unrestricted guest                       no
  PAUSE-loop exiting                       no
  RDRAND exiting                           no
  Enable INVPCID                           no
  Enable VM functions                      no
VM-Exit controls
  Save debug controls                      forced
  Host address-space size                  yes
  Load IA32_PERF_GLOBAL_CTRL               no
  Acknowledge interrupt on exit            yes
  Save IA32_PAT                            no
  Load IA32_PAT                            no
  Save IA32_EFER                           no
  Load IA32_EFER                           no
  Save VMX-preemption timer value          no
VM-Entry controls
  Load debug controls                      forced
  IA-64 mode guest                         yes
  Entry to SMM                             yes
  Deactivate dual-monitor treatment        yes
  Load IA32_PERF_GLOBAL_CTRL               no
  Load IA32_PAT                            no
  Load IA32_EFER                           no
Miscellaneous data
  VMX-preemption timer scale (log2)        0
  Store EFER.LMA into IA-32e mode guest control no
  HLT activity state                       yes
  Shutdown activity state                  yes
  Wait-for-SIPI activity state             yes
  Number of CR3-target values              4
  MSR-load/store count recommenation       0
  IA32_SMM_MONITOR_CTL[2] can be set to 1  no
  MSEG revision identifier                 0
VPID and EPT capabilities
  Execute-only EPT translations            no
  Page-walk length 4                       no
  Paging-structure memory type UC          no
  Paging-structure memory type WB          no
  2MB EPT pages                            no
  1GB EPT pages                            no
  INVEPT supported                         no
  EPT accessed and dirty flags             no
  Single-context INVEPT                    no
  All-context INVEPT                       no
  INVVPID supported                        no
  Individual-address INVVPID               no
  Single-context INVVPID                   no
  All-context INVVPID                      no
  Single-context-retaining-globals INVVPID no
VM Functions
  EPTP Switching                           no

> and a snapshot of kvm_stat while the guest is hung.

kvm statistics

 exits                                      6778198  615942
 host_state_reload                             1988     187
 irq_exits                                     1523     138
 mmu_cache_miss                                   4       0
 fpu_reload                                       1       0

thanks,
-- 
js
suse labs

^ permalink raw reply	[flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate
  2012-09-12  8:13       ` Jiri Slaby
@ 2012-09-12  8:18         ` Avi Kivity
  2012-09-12  8:20           ` Jiri Slaby
  2012-09-12 18:11           ` Jiri Slaby
  0 siblings, 2 replies; 16+ messages in thread
From: Avi Kivity @ 2012-09-12  8:18 UTC (permalink / raw)
  To: Jiri Slaby; +Cc: Jiri Slaby, Marcelo Tosatti, kvm, LKML

On 09/12/2012 11:13 AM, Jiri Slaby wrote:
> 
>>  Please provide the output of vmxcap
>> (http://goo.gl/c5lUO),
> 
>   Unrestricted guest                       no

The big real mode fixes.


> 
>> and a snapshot of kvm_stat while the guest is hung.
> 
> kvm statistics
> 
>  exits                                      6778198  615942
>  host_state_reload                             1988     187
>  irq_exits                                     1523     138
>  mmu_cache_miss                                   4       0
>  fpu_reload                                       1       0

Please run this as root so we get the tracepoint based output; and press
'x' when it's running so we get more detailed output.


-- 
error compiling committee.c: too many arguments to function

^ permalink raw reply	[flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate
  2012-09-12  8:18         ` Avi Kivity
@ 2012-09-12  8:20           ` Jiri Slaby
  2012-09-12 18:11           ` Jiri Slaby
  1 sibling, 0 replies; 16+ messages in thread
From: Jiri Slaby @ 2012-09-12  8:20 UTC (permalink / raw)
  To: Avi Kivity; +Cc: Jiri Slaby, Marcelo Tosatti, kvm, LKML

On 09/12/2012 10:18 AM, Avi Kivity wrote:
> On 09/12/2012 11:13 AM, Jiri Slaby wrote:
>> kvm statistics
>>
>>  exits                                      6778198  615942
>>  host_state_reload                             1988     187
>>  irq_exits                                     1523     138
>>  mmu_cache_miss                                   4       0
>>  fpu_reload                                       1       0
> 
> Please run this as root so we get the tracepoint based output; and press
> 'x' when it's running so we get more detailed output.

I need to build a kernel with trace points enabled first... I will
return later today.

thanks,
-- 
js
suse labs

^ permalink raw reply	[flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate
  2012-09-12  8:18         ` Avi Kivity
  2012-09-12  8:20           ` Jiri Slaby
@ 2012-09-12 18:11           ` Jiri Slaby
  2012-09-13  9:59             ` Avi Kivity
  1 sibling, 1 reply; 16+ messages in thread
From: Jiri Slaby @ 2012-09-12 18:11 UTC (permalink / raw)
  To: Avi Kivity; +Cc: Jiri Slaby, Marcelo Tosatti, kvm, LKML

On 09/12/2012 10:18 AM, Avi Kivity wrote:
> On 09/12/2012 11:13 AM, Jiri Slaby wrote:
>>
>>>  Please provide the output of vmxcap
>>> (http://goo.gl/c5lUO),
>>
>>   Unrestricted guest                       no
> 
> The big real mode fixes.
> 
> 
>>
>>> and a snapshot of kvm_stat while the guest is hung.
>>
>> kvm statistics
>>
>>  exits                                      6778198  615942
>>  host_state_reload                             1988     187
>>  irq_exits                                     1523     138
>>  mmu_cache_miss                                   4       0
>>  fpu_reload                                       1       0
> 
> Please run this as root so we get the tracepoint based output; and press
> 'x' when it's running so we get more detailed output.

kvm statistics

 kvm_exit                                  13798699  330708
 kvm_entry                                 13799110  330708
 kvm_page_fault                            13793650  330604
 kvm_exit(EXCEPTION_NMI)                    6188458  330604
 kvm_exit(EXTERNAL_INTERRUPT)                  2169     105
 kvm_exit(TPR_BELOW_THRESHOLD)                   82       0
 kvm_exit(IO_INSTRUCTION)                         6       0
 kvm_exit(PAUSE_INSTRUCTION)                      5       0
 kvm_exit(MCE_DURING_VMENTRY)                     5       0
 kvm_exit(DR_ACCESS)                              5       0
 kvm_exit(VMPTRLD)                                5       0
 kvm_exit(VMLAUNCH)                               5       0
 kvm_exit(INVLPG)                                 5       0
 kvm_exit(VMCALL)                                 5       0
 kvm_exit(WBINVD)                                 5       0
 kvm_exit(MONITOR_INSTRUCTION)                    5       0
 kvm_exit(MWAIT_INSTRUCTION)                      4       0
 kvm_exit(NMI_WINDOW)                             4       0
 kvm_exit(VMREAD)                                 4       0
 kvm_exit(XSETBV)                                 4       0
 kvm_exit(RDPMC)                                  4       0
 kvm_exit(CPUID)                                  4       0
 kvm_exit(VMCLEAR)                                4       0
 kvm_exit(PENDING_INTERRUPT)                      4       0
 kvm_exit(VMRESUME)                               4       0
 kvm_exit(APIC_ACCESS)                            4       0
 kvm_exit(INVALID_STATE)                          4       0
 kvm_exit(MSR_READ)                               4       0
 kvm_exit(VMOFF)                                  4       0
 kvm_exit(RDTSC)                                  4       0
 kvm_exit(MSR_WRITE)                              4       0
 kvm_exit(VMPTRST)                                4       0
 kvm_exit(VMWRITE)                                4       0
 kvm_exit(HLT)                                    3       0
 kvm_exit(EPT_VIOLATION)                          3       0
 kvm_exit(TASK_SWITCH)                            3       0
 kvm_exit(CR_ACCESS)                              3       0
 kvm_exit(TRIPLE_FAULT)                           3       0
 kvm_exit(VMON)                                   3       0
 kvm_exit(EPT_MISCONFIG)                          3       0

thanks,
-- 
js
suse labs

^ permalink raw reply	[flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate
  2012-09-12 18:11           ` Jiri Slaby
@ 2012-09-13  9:59             ` Avi Kivity
  2012-09-13 18:32               ` Jiri Slaby
  2012-09-18 19:40               ` Jiri Slaby
  0 siblings, 2 replies; 16+ messages in thread
From: Avi Kivity @ 2012-09-13  9:59 UTC (permalink / raw)
  To: Jiri Slaby; +Cc: Jiri Slaby, Marcelo Tosatti, kvm, LKML

On 09/12/2012 09:11 PM, Jiri Slaby wrote:
> On 09/12/2012 10:18 AM, Avi Kivity wrote:
>> On 09/12/2012 11:13 AM, Jiri Slaby wrote:
>>>
>>>>  Please provide the output of vmxcap
>>>> (http://goo.gl/c5lUO),
>>>
>>>   Unrestricted guest                       no
>> 
>> The big real mode fixes.
>> 
>> 
>>>
>>>> and a snapshot of kvm_stat while the guest is hung.
>>>
>>> kvm statistics
>>>
>>>  exits                                      6778198  615942
>>>  host_state_reload                             1988     187
>>>  irq_exits                                     1523     138
>>>  mmu_cache_miss                                   4       0
>>>  fpu_reload                                       1       0
>> 
>> Please run this as root so we get the tracepoint based output; and press
>> 'x' when it's running so we get more detailed output.
> 
> kvm statistics
> 
>  kvm_exit                                  13798699  330708
>  kvm_entry                                 13799110  330708
>  kvm_page_fault                            13793650  330604
>  kvm_exit(EXCEPTION_NMI)                    6188458  330604
>  kvm_exit(EXTERNAL_INTERRUPT)                  2169     105
>  kvm_exit(TPR_BELOW_THRESHOLD)                   82       0
>  kvm_exit(IO_INSTRUCTION)                         6       0

Strange, it's unable to fault in the very first page.

Please provide a trace as per http://www.linux-kvm.org/page/Tracing (but
append -e kvmmmu to the command line).



-- 
error compiling committee.c: too many arguments to function

^ permalink raw reply	[flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate
  2012-09-13  9:59             ` Avi Kivity
@ 2012-09-13 18:32               ` Jiri Slaby
  2012-09-18 19:40               ` Jiri Slaby
  1 sibling, 0 replies; 16+ messages in thread
From: Jiri Slaby @ 2012-09-13 18:32 UTC (permalink / raw)
  To: Avi Kivity; +Cc: Jiri Slaby, Marcelo Tosatti, kvm, LKML

[-- Attachment #1: Type: text/plain, Size: 1702 bytes --]

On 09/13/2012 11:59 AM, Avi Kivity wrote:
> On 09/12/2012 09:11 PM, Jiri Slaby wrote:
>> On 09/12/2012 10:18 AM, Avi Kivity wrote:
>>> On 09/12/2012 11:13 AM, Jiri Slaby wrote:
>>>>
>>>>>  Please provide the output of vmxcap
>>>>> (http://goo.gl/c5lUO),
>>>>
>>>>   Unrestricted guest                       no
>>>
>>> The big real mode fixes.
>>>
>>>
>>>>
>>>>> and a snapshot of kvm_stat while the guest is hung.
>>>>
>>>> kvm statistics
>>>>
>>>>  exits                                      6778198  615942
>>>>  host_state_reload                             1988     187
>>>>  irq_exits                                     1523     138
>>>>  mmu_cache_miss                                   4       0
>>>>  fpu_reload                                       1       0
>>>
>>> Please run this as root so we get the tracepoint based output; and press
>>> 'x' when it's running so we get more detailed output.
>>
>> kvm statistics
>>
>>  kvm_exit                                  13798699  330708
>>  kvm_entry                                 13799110  330708
>>  kvm_page_fault                            13793650  330604
>>  kvm_exit(EXCEPTION_NMI)                    6188458  330604
>>  kvm_exit(EXTERNAL_INTERRUPT)                  2169     105
>>  kvm_exit(TPR_BELOW_THRESHOLD)                   82       0
>>  kvm_exit(IO_INSTRUCTION)                         6       0
> 
> Strange, it's unable to fault in the very first page.
> 
> Please provide a trace as per http://www.linux-kvm.org/page/Tracing (but
> append -e kvmmmu to the command line).

Attached. Does it make sense? It wrote things like:
  failed to read event print fmt for kvm_mmu_unsync_page
to the stderr.

thanks,
-- 
js
suse labs

[-- Attachment #2: report --]
[-- Type: text/plain, Size: 1627 bytes --]

version = 6
CPU 0 is empty
cpus=2
        qemu-kvm-6170  [001]   457.811896: kvm_mmu_get_page:     [FAILED TO PARSE] gfn=0 role=122882 root_count=0 unsync=0 created=1
        qemu-kvm-6170  [001]   457.811899: kvm_mmu_get_page:     [FAILED TO PARSE] gfn=262144 role=122882 root_count=0 unsync=0 created=1
        qemu-kvm-6170  [001]   457.811900: kvm_mmu_get_page:     [FAILED TO PARSE] gfn=524288 role=122882 root_count=0 unsync=0 created=1
        qemu-kvm-6170  [001]   457.811902: kvm_mmu_get_page:     [FAILED TO PARSE] gfn=786432 role=122882 root_count=0 unsync=0 created=1
        qemu-kvm-6171  [001]   462.416705: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=786432 role=122882 root_count=1 unsync=0
        qemu-kvm-6171  [001]   462.416712: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=524288 role=122882 root_count=1 unsync=0
        qemu-kvm-6171  [001]   462.416715: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=262144 role=122882 root_count=1 unsync=0
        qemu-kvm-6171  [001]   462.416717: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=0 role=122882 root_count=1 unsync=0
        qemu-kvm-6171  [001]   462.485197: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=0 role=253954 root_count=0 unsync=0
        qemu-kvm-6171  [001]   462.485202: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=262144 role=253954 root_count=0 unsync=0
        qemu-kvm-6171  [001]   462.485205: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=524288 role=253954 root_count=0 unsync=0
        qemu-kvm-6171  [001]   462.485209: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=786432 role=253954 root_count=0 unsync=0

^ permalink raw reply	[flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate
  2012-09-13  9:59             ` Avi Kivity
  2012-09-13 18:32               ` Jiri Slaby
@ 2012-09-18 19:40               ` Jiri Slaby
  2012-09-18 19:46                 ` Andrew Morton
  2012-09-19 14:18                 ` [PATCH] mm: call invalidate_range_end in do_wp_page even for zero pages Haggai Eran
  1 sibling, 2 replies; 16+ messages in thread
From: Jiri Slaby @ 2012-09-18 19:40 UTC (permalink / raw)
  To: Avi Kivity
  Cc: Jiri Slaby, Marcelo Tosatti, kvm, LKML, Haggai Eran,
	Andrew Morton, linux-mm

On 09/13/2012 11:59 AM, Avi Kivity wrote:
> On 09/12/2012 09:11 PM, Jiri Slaby wrote:
>> On 09/12/2012 10:18 AM, Avi Kivity wrote:
>>> On 09/12/2012 11:13 AM, Jiri Slaby wrote:
>>>>
>>>>>   Please provide the output of vmxcap
>>>>> (http://goo.gl/c5lUO),
>>>>
>>>>    Unrestricted guest                       no
>>>
>>> The big real mode fixes.
>>>
>>>
>>>>
>>>>> and a snapshot of kvm_stat while the guest is hung.
>>>>
>>>> kvm statistics
>>>>
>>>>   exits                                      6778198  615942
>>>>   host_state_reload                             1988     187
>>>>   irq_exits                                     1523     138
>>>>   mmu_cache_miss                                   4       0
>>>>   fpu_reload                                       1       0
>>>
>>> Please run this as root so we get the tracepoint based output; and press
>>> 'x' when it's running so we get more detailed output.
>>
>> kvm statistics
>>
>>   kvm_exit                                  13798699  330708
>>   kvm_entry                                 13799110  330708
>>   kvm_page_fault                            13793650  330604
>>   kvm_exit(EXCEPTION_NMI)                    6188458  330604
>>   kvm_exit(EXTERNAL_INTERRUPT)                  2169     105
>>   kvm_exit(TPR_BELOW_THRESHOLD)                   82       0
>>   kvm_exit(IO_INSTRUCTION)                         6       0
>
> Strange, it's unable to fault in the very first page.

I bisected that. Note the bisection log. I have never seen something 
like that :D:
git bisect start
git bisect bad 3de9d1a1500472bc80478bd75e33fa9c1eba1422
git bisect good fea7a08acb13524b47711625eebea40a0ede69a0
git bisect good 95a2fe4baa1ad444df5f94bfc9416fc6b4b34cef
git bisect good f42c0d57a5a60da03c705bdea9fbba381112dd60
git bisect good 31a2e241a9e37a133278959044960c229acc5714
git bisect good f15fb01c5593fa1b58cc7a8a9c59913e2625bf2e
git bisect good 16d21ff46f5d50e311d07406c31f96916e5e8e1a
git bisect good 0b84592f458b4e8567aa7d803aff382c1d3b64fd
git bisect bad b955428e7f14cd29fe9d8059efa3ea4be679c83d
git bisect bad 20c4da4f68fcade05eda9c9b7dbad0a78cc5efe8
git bisect bad 31b90ed2a90f80fb528ac55ee357a815e1dedc36
git bisect bad b273fe14ee5b38cecc7bce94f7777f35a0bf9ee4
git bisect bad de426dbe9a60706b91b40397f69f819a39a06b6b
git bisect bad 6b998094ec50248e72b9f251d0607b58b18dba38
git bisect bad cf9b81d47a89f5d404a0cd8013b461617751e520

=== 8< ===

Reverting cf9b81d47a89 (mm: wrap calls to set_pte_at_notify with 
invalidate_range_start and invalidate_range_end) on the top of today's 
-next fixes the issue.

thanks,
-- 
js
suse labs

^ permalink raw reply	[flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate
  2012-09-18 19:40               ` Jiri Slaby
@ 2012-09-18 19:46                 ` Andrew Morton
  2012-09-19  0:00                   ` Stephen Rothwell
  2012-09-19 14:18                 ` [PATCH] mm: call invalidate_range_end in do_wp_page even for zero pages Haggai Eran
  1 sibling, 1 reply; 16+ messages in thread
From: Andrew Morton @ 2012-09-18 19:46 UTC (permalink / raw)
  To: Jiri Slaby
  Cc: Avi Kivity, Jiri Slaby, Marcelo Tosatti, kvm, LKML, Haggai Eran,
	linux-mm, Sagi Grimberg, Shachar Raindel, Liran Liss

On Tue, 18 Sep 2012 21:40:31 +0200
Jiri Slaby <jslaby@suse.cz> wrote:

> On 09/13/2012 11:59 AM, Avi Kivity wrote:
> > On 09/12/2012 09:11 PM, Jiri Slaby wrote:
> >> On 09/12/2012 10:18 AM, Avi Kivity wrote:
> >>> On 09/12/2012 11:13 AM, Jiri Slaby wrote:
> >>>>
> >>>>>   Please provide the output of vmxcap
> >>>>> (http://goo.gl/c5lUO),
> >>>>
> >>>>    Unrestricted guest                       no
> >>>
> >>> The big real mode fixes.
> >>>
> >>>
> >>>>
> >>>>> and a snapshot of kvm_stat while the guest is hung.
> >>>>
> >>>> kvm statistics
> >>>>
> >>>>   exits                                      6778198  615942
> >>>>   host_state_reload                             1988     187
> >>>>   irq_exits                                     1523     138
> >>>>   mmu_cache_miss                                   4       0
> >>>>   fpu_reload                                       1       0
> >>>
> >>> Please run this as root so we get the tracepoint based output; and press
> >>> 'x' when it's running so we get more detailed output.
> >>
> >> kvm statistics
> >>
> >>   kvm_exit                                  13798699  330708
> >>   kvm_entry                                 13799110  330708
> >>   kvm_page_fault                            13793650  330604
> >>   kvm_exit(EXCEPTION_NMI)                    6188458  330604
> >>   kvm_exit(EXTERNAL_INTERRUPT)                  2169     105
> >>   kvm_exit(TPR_BELOW_THRESHOLD)                   82       0
> >>   kvm_exit(IO_INSTRUCTION)                         6       0
> >
> > Strange, it's unable to fault in the very first page.
> 
> I bisected that. Note the bisection log. I have never seen something 
> like that :D:
> git bisect start
> git bisect bad 3de9d1a1500472bc80478bd75e33fa9c1eba1422
> git bisect good fea7a08acb13524b47711625eebea40a0ede69a0
> git bisect good 95a2fe4baa1ad444df5f94bfc9416fc6b4b34cef
> git bisect good f42c0d57a5a60da03c705bdea9fbba381112dd60
> git bisect good 31a2e241a9e37a133278959044960c229acc5714
> git bisect good f15fb01c5593fa1b58cc7a8a9c59913e2625bf2e
> git bisect good 16d21ff46f5d50e311d07406c31f96916e5e8e1a
> git bisect good 0b84592f458b4e8567aa7d803aff382c1d3b64fd
> git bisect bad b955428e7f14cd29fe9d8059efa3ea4be679c83d
> git bisect bad 20c4da4f68fcade05eda9c9b7dbad0a78cc5efe8
> git bisect bad 31b90ed2a90f80fb528ac55ee357a815e1dedc36
> git bisect bad b273fe14ee5b38cecc7bce94f7777f35a0bf9ee4
> git bisect bad de426dbe9a60706b91b40397f69f819a39a06b6b
> git bisect bad 6b998094ec50248e72b9f251d0607b58b18dba38
> git bisect bad cf9b81d47a89f5d404a0cd8013b461617751e520
> 
> === 8< ===
> 
> Reverting cf9b81d47a89 (mm: wrap calls to set_pte_at_notify with 
> invalidate_range_start and invalidate_range_end) on the top of today's 
> -next fixes the issue.

hm, thanks.  This will probably take some time to resolve so I think
I'll drop

mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock.patch
mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix.patch
mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix-fix.patch
mm-wrap-calls-to-set_pte_at_notify-with-invalidate_range_start-and-invalidate_range_end.patch
 

^ permalink raw reply	[flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate
  2012-09-18 19:46                 ` Andrew Morton
@ 2012-09-19  0:00                   ` Stephen Rothwell
  2012-09-19  0:20                     ` Andrew Morton
  0 siblings, 1 reply; 16+ messages in thread
From: Stephen Rothwell @ 2012-09-19  0:00 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Jiri Slaby, Avi Kivity, Jiri Slaby, Marcelo Tosatti, kvm, LKML,
	Haggai Eran, linux-mm, Sagi Grimberg, Shachar Raindel,
	Liran Liss

[-- Attachment #1: Type: text/plain, Size: 721 bytes --]

Hi Andrew,

On Tue, 18 Sep 2012 12:46:46 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
>
> hm, thanks.  This will probably take some time to resolve so I think
> I'll drop
> 
> mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock.patch
> mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix.patch
> mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix-fix.patch
> mm-wrap-calls-to-set_pte_at_notify-with-invalidate_range_start-and-invalidate_range_end.patch

Should I attempt to remove these from the akpm tree in linux-next today?
Or should I just wait for a new mmotm?

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate
  2012-09-19  0:00                   ` Stephen Rothwell
@ 2012-09-19  0:20                     ` Andrew Morton
  2012-09-19  1:36                       ` Stephen Rothwell
  0 siblings, 1 reply; 16+ messages in thread
From: Andrew Morton @ 2012-09-19  0:20 UTC (permalink / raw)
  To: Stephen Rothwell
  Cc: Jiri Slaby, Avi Kivity, Jiri Slaby, Marcelo Tosatti, kvm, LKML,
	Haggai Eran, linux-mm, Sagi Grimberg, Shachar Raindel,
	Liran Liss

On Wed, 19 Sep 2012 10:00:34 +1000 Stephen Rothwell <sfr@canb.auug.org.au> wrote:

> Hi Andrew,
> 
> On Tue, 18 Sep 2012 12:46:46 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
> >
> > hm, thanks.  This will probably take some time to resolve so I think
> > I'll drop
> > 
> > mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock.patch
> > mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix.patch
> > mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix-fix.patch
> > mm-wrap-calls-to-set_pte_at_notify-with-invalidate_range_start-and-invalidate_range_end.patch
> 
> Should I attempt to remove these from the akpm tree in linux-next today?

That would be best - there's no point in having people test (and debug)
dead stuff.

> Or should I just wait for a new mmotm?

You could be brave and test http://ozlabs.org/~akpm/mmots/ for me :)

^ permalink raw reply	[flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate
  2012-09-19  0:20                     ` Andrew Morton
@ 2012-09-19  1:36                       ` Stephen Rothwell
  0 siblings, 0 replies; 16+ messages in thread
From: Stephen Rothwell @ 2012-09-19  1:36 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Jiri Slaby, Avi Kivity, Jiri Slaby, Marcelo Tosatti, kvm, LKML,
	Haggai Eran, linux-mm, Sagi Grimberg, Shachar Raindel,
	Liran Liss

[-- Attachment #1: Type: text/plain, Size: 1174 bytes --]

Hi Andrew,

On Tue, 18 Sep 2012 17:20:29 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
>
> On Wed, 19 Sep 2012 10:00:34 +1000 Stephen Rothwell <sfr@canb.auug.org.au> wrote:
> 
> > On Tue, 18 Sep 2012 12:46:46 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
> > >
> > > hm, thanks.  This will probably take some time to resolve so I think
> > > I'll drop
> > > 
> > > mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock.patch
> > > mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix.patch
> > > mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix-fix.patch
> > > mm-wrap-calls-to-set_pte_at_notify-with-invalidate_range_start-and-invalidate_range_end.patch
> > 
> > Should I attempt to remove these from the akpm tree in linux-next today?
> 
> That would be best - there's no point in having people test (and debug)
> dead stuff.

OK, I removed them.

> > Or should I just wait for a new mmotm?
> 
> You could be brave and test http://ozlabs.org/~akpm/mmots/ for me :)

Brave? maybe.  Stupid? no :-)

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread
* [PATCH] mm: call invalidate_range_end in do_wp_page even for zero pages
  2012-09-18 19:40               ` Jiri Slaby
  2012-09-18 19:46                 ` Andrew Morton
@ 2012-09-19 14:18                 ` Haggai Eran
  1 sibling, 0 replies; 16+ messages in thread
From: Haggai Eran @ 2012-09-19 14:18 UTC (permalink / raw)
  To: Jiri Slaby
  Cc: linux-kernel, kvm, linux-mm, Shachar Raindel, Haggai Eran,
	Avi Kivity, Andrew Morton

The previous patch "mm: wrap calls to set_pte_at_notify with
invalidate_range_start and invalidate_range_end" only called the
invalidate_range_end mmu notifier function in do_wp_page when the new_page
variable wasn't NULL. This was done in order to only call invalidate_range_end
after invalidate_range_start was called. Unfortunately, there are situations
where new_page is NULL and invalidate_range_start is called. This caused
invalidate_range_start to be called without a matching invalidate_range_end,
causing kvm to loop indefinitely on the first page fault.

This patch adds a flag variable to do_wp_page that marks whether the
invalidate_range_start notifier was called. invalidate_range_end is then
called if the flag is true.

Reported-by: Jiri Slaby <jslaby@suse.cz>
Cc: Avi Kivity <avi@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Haggai Eran <haggaie@mellanox.com>
---
I tested this patch against yesterday's linux-next (next-20120918), and it
seems to solve the problem with kvm. I used the same command line you reported:

  qemu-kvm -k en-us -usbdevice tablet -balloon virtio -hda IMAGE -smp 2 \
  -m 1000M -net user -net nic,model=e1000 -usb -serial pty

I was hoping you could also test it yourself, and see that it also works for
you, if you don't mind.

 mm/memory.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/mm/memory.c b/mm/memory.c
index 1a92d87..76ec199 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -2529,6 +2529,7 @@ static int do_wp_page(struct mm_struct *mm, struct vm_area_struct *vma,
 	struct page *dirty_page = NULL;
 	unsigned long mmun_start;	/* For mmu_notifiers */
 	unsigned long mmun_end;		/* For mmu_notifiers */
+	bool mmun_called = false;	/* For mmu_notifiers */
 
 	old_page = vm_normal_page(vma, address, orig_pte);
 	if (!old_page) {
@@ -2706,8 +2707,9 @@ gotten:
 	if (mem_cgroup_newpage_charge(new_page, mm, GFP_KERNEL))
 		goto oom_free_new;
 
-	mmun_start = address & PAGE_MASK;
-	mmun_end   = (address & PAGE_MASK) + PAGE_SIZE;
+	mmun_start  = address & PAGE_MASK;
+	mmun_end    = (address & PAGE_MASK) + PAGE_SIZE;
+	mmun_called = true;
 	mmu_notifier_invalidate_range_start(mm, mmun_start, mmun_end);
 
 	/*
@@ -2776,8 +2778,7 @@ gotten:
 		page_cache_release(new_page);
 unlock:
 	pte_unmap_unlock(page_table, ptl);
-	if (new_page)
-		/* Only call the end notifier if the begin was called. */
+	if (mmun_called)
 		mmu_notifier_invalidate_range_end(mm, mmun_start, mmun_end);
 	if (old_page) {
 		/*
-- 
1.7.11.2


^ permalink raw reply related	[flat|nested] 16+ messages in thread
end of thread, other threads:[~2012-09-19 14:19 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-09-11 18:11 qemu-kvm loops after kernel udpate Jiri Slaby
2012-09-11 19:03 ` Marcelo Tosatti
2012-09-11 19:41   ` Jiri Slaby
2012-09-12  8:06     ` Avi Kivity
2012-09-12  8:13       ` Jiri Slaby
2012-09-12  8:18         ` Avi Kivity
2012-09-12  8:20           ` Jiri Slaby
2012-09-12 18:11           ` Jiri Slaby
2012-09-13  9:59             ` Avi Kivity
2012-09-13 18:32               ` Jiri Slaby
2012-09-18 19:40               ` Jiri Slaby
2012-09-18 19:46                 ` Andrew Morton
2012-09-19  0:00                   ` Stephen Rothwell
2012-09-19  0:20                     ` Andrew Morton
2012-09-19  1:36                       ` Stephen Rothwell
2012-09-19 14:18                 ` [PATCH] mm: call invalidate_range_end in do_wp_page even for zero pages Haggai Eran

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).