All of lore.kernel.org
 help / color / mirror / Atom feed
* hypervisor crash with latest xen-unstable
@ 2010-11-25 10:27 Juergen Gross
  2010-11-25 17:14 ` Keir Fraser
  2010-11-26 10:12 ` Keir Fraser
  0 siblings, 2 replies; 5+ messages in thread
From: Juergen Gross @ 2010-11-25 10:27 UTC (permalink / raw)
  To: xen-devel

Hi,

latest xen-unstable crashed when I tried to remove a cpu from a cpupool via

xl cpupool-cpu-remove

Serial log shows:

(XEN) Assertion '!(((*({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) : 
"0"(&per_cpu____preempt_count)); (typeof(&per_cpu____preempt_count)) (__ptr + 
(get_cpu_info()->per_cpu_offset)); }))) || 
((irq_stat[(((get_cpu_info()->processor_id)))].__local_irq_count) != 0) || 
!local_irq_is_enabled())' failed at softirq.c:61
(XEN) ----[ Xen-4.1-unstable  x86_64  debug=y  Tainted:    C ]----
(XEN) CPU:    5
(XEN) RIP:    e008:[<ffff82c4801212f9>] do_softirq+0x51/0x7a
(XEN) RFLAGS: 0000000000010202   CONTEXT: hypervisor
(XEN) rax: ffff82c4802ca2a0   rbx: ffff8300780da000   rcx: ffff82c4802a7080
(XEN) rdx: 00000043b9aefd80   rsi: ffff830839db7ec0   rdi: ffff82c48024d0e0
(XEN) rbp: ffff830839db7f08   rsp: ffff830839db7f08   r8:  0000000000000005
(XEN) r9:  0000ffff0000ffff   r10: 00000000deadbeef   r11: 0f0f0f0f0f0f0f0f
(XEN) r12: ffff8803f8f61440   r13: ffff8803fdf4fe58   r14: 0000000000000000
(XEN) r15: 00007fff335d7150   cr0: 0000000080050033   cr4: 00000000000026f0
(XEN) cr3: 0000000846990000   cr2: 0000000000627048
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen stack trace from rsp=ffff830839db7f08:
(XEN)    00007cf7c62480c7 ffff82c480202736 00007fff335d7150 0000000000000000
(XEN)    ffff8803fdf4fe58 ffff8803f8f61440 ffffffffffffffda ffff8803fd491438
(XEN)    0000000000000216 000000000061e004 000000000061d050 00007f6d701d9f6c
(XEN)    0000000000000000 ffffffff8000346a 000000000061c2e0 0000000000624004
(XEN)    00000000deadbeef 0000010000000000 ffffffff8000346a 000000000000e033
(XEN)    0000000000000216 ffff8803fdf4fd80 000000000000e02b 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000005
(XEN)    ffff8300780da000 00000043b9aefd80 0000000000000000
(XEN) Xen call trace:
(XEN)    [<ffff82c4801212f9>] do_softirq+0x51/0x7a
(XEN)

I suspect the waitqueue stuff broke the continue_hypercall_on_cpu() stuff.


Juergen

-- 
Juergen Gross                 Principal Developer Operating Systems
TSP ES&S SWE OS6                       Telephone: +49 (0) 89 3222 2967
Fujitsu Technology Solutions              e-mail: juergen.gross@ts.fujitsu.com
Domagkstr. 28                           Internet: ts.fujitsu.com
D-80807 Muenchen                 Company details: ts.fujitsu.com/imprint.html

^ permalink raw reply	[flat|nested] 5+ messages in thread
* Re: hypervisor crash with latest xen-unstable
  2010-11-25 10:27 hypervisor crash with latest xen-unstable Juergen Gross
@ 2010-11-25 17:14 ` Keir Fraser
  2010-11-26  5:32   ` Juergen Gross
  2010-11-26 10:12 ` Keir Fraser
  1 sibling, 1 reply; 5+ messages in thread
From: Keir Fraser @ 2010-11-25 17:14 UTC (permalink / raw)
  To: Juergen Gross, xen-devel

On 25/11/2010 10:27, "Juergen Gross" <juergen.gross@ts.fujitsu.com> wrote:

> (XEN) Xen call trace:
> (XEN)    [<ffff82c4801212f9>] do_softirq+0x51/0x7a
> (XEN)
> 
> I suspect the waitqueue stuff broke the continue_hypercall_on_cpu() stuff.

Then please bisect to confirm that.

 -- Keir

^ permalink raw reply	[flat|nested] 5+ messages in thread
* Re: hypervisor crash with latest xen-unstable
  2010-11-25 17:14 ` Keir Fraser
@ 2010-11-26  5:32   ` Juergen Gross
  0 siblings, 0 replies; 5+ messages in thread
From: Juergen Gross @ 2010-11-26  5:32 UTC (permalink / raw)
  To: Keir Fraser; +Cc: xen-devel

On 11/25/10 18:14, Keir Fraser wrote:
> On 25/11/2010 10:27, "Juergen Gross"<juergen.gross@ts.fujitsu.com>  wrote:
>
>> (XEN) Xen call trace:
>> (XEN)    [<ffff82c4801212f9>] do_softirq+0x51/0x7a
>> (XEN)
>>
>> I suspect the waitqueue stuff broke the continue_hypercall_on_cpu() stuff.
>
> Then please bisect to confirm that.

Removing the ASSERT(!in_atomic()) in do_softirq() and schedule() makes it
work again. And these assertions were added with cs22395.

The question now is, whether these assertions are wrong or if it is just
working by chance without them.
It seems to be necessary to be able to call schedule() in this case, as adding

if (in_atomic()) return;

to do_softirq() made the hypercall hang.


Juergen

-- 
Juergen Gross                 Principal Developer Operating Systems
TSP ES&S SWE OS6                       Telephone: +49 (0) 89 3222 2967
Fujitsu Technology Solutions              e-mail: juergen.gross@ts.fujitsu.com
Domagkstr. 28                           Internet: ts.fujitsu.com
D-80807 Muenchen                 Company details: ts.fujitsu.com/imprint.html

^ permalink raw reply	[flat|nested] 5+ messages in thread
* Re: hypervisor crash with latest xen-unstable
  2010-11-25 10:27 hypervisor crash with latest xen-unstable Juergen Gross
  2010-11-25 17:14 ` Keir Fraser
@ 2010-11-26 10:12 ` Keir Fraser
  2010-11-26 10:24   ` Juergen Gross
  1 sibling, 1 reply; 5+ messages in thread
From: Keir Fraser @ 2010-11-26 10:12 UTC (permalink / raw)
  To: Juergen Gross, xen-devel

Fixed by xen-unstable:22342.

 K.

On 25/11/2010 10:27, "Juergen Gross" <juergen.gross@ts.fujitsu.com> wrote:

> Hi,
> 
> latest xen-unstable crashed when I tried to remove a cpu from a cpupool via
> 
> xl cpupool-cpu-remove
> 
> Serial log shows:
> 
> (XEN) Assertion '!(((*({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) :
> "0"(&per_cpu____preempt_count)); (typeof(&per_cpu____preempt_count)) (__ptr +
> (get_cpu_info()->per_cpu_offset)); }))) ||
> ((irq_stat[(((get_cpu_info()->processor_id)))].__local_irq_count) != 0) ||
> !local_irq_is_enabled())' failed at softirq.c:61
> (XEN) ----[ Xen-4.1-unstable  x86_64  debug=y  Tainted:    C ]----
> (XEN) CPU:    5
> (XEN) RIP:    e008:[<ffff82c4801212f9>] do_softirq+0x51/0x7a
> (XEN) RFLAGS: 0000000000010202   CONTEXT: hypervisor
> (XEN) rax: ffff82c4802ca2a0   rbx: ffff8300780da000   rcx: ffff82c4802a7080
> (XEN) rdx: 00000043b9aefd80   rsi: ffff830839db7ec0   rdi: ffff82c48024d0e0
> (XEN) rbp: ffff830839db7f08   rsp: ffff830839db7f08   r8:  0000000000000005
> (XEN) r9:  0000ffff0000ffff   r10: 00000000deadbeef   r11: 0f0f0f0f0f0f0f0f
> (XEN) r12: ffff8803f8f61440   r13: ffff8803fdf4fe58   r14: 0000000000000000
> (XEN) r15: 00007fff335d7150   cr0: 0000000080050033   cr4: 00000000000026f0
> (XEN) cr3: 0000000846990000   cr2: 0000000000627048
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
> (XEN) Xen stack trace from rsp=ffff830839db7f08:
> (XEN)    00007cf7c62480c7 ffff82c480202736 00007fff335d7150 0000000000000000
> (XEN)    ffff8803fdf4fe58 ffff8803f8f61440 ffffffffffffffda ffff8803fd491438
> (XEN)    0000000000000216 000000000061e004 000000000061d050 00007f6d701d9f6c
> (XEN)    0000000000000000 ffffffff8000346a 000000000061c2e0 0000000000624004
> (XEN)    00000000deadbeef 0000010000000000 ffffffff8000346a 000000000000e033
> (XEN)    0000000000000216 ffff8803fdf4fd80 000000000000e02b 0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000005
> (XEN)    ffff8300780da000 00000043b9aefd80 0000000000000000
> (XEN) Xen call trace:
> (XEN)    [<ffff82c4801212f9>] do_softirq+0x51/0x7a
> (XEN)
> 
> I suspect the waitqueue stuff broke the continue_hypercall_on_cpu() stuff.
> 
> 
> Juergen

^ permalink raw reply	[flat|nested] 5+ messages in thread
* Re: hypervisor crash with latest xen-unstable
  2010-11-26 10:12 ` Keir Fraser
@ 2010-11-26 10:24   ` Juergen Gross
  0 siblings, 0 replies; 5+ messages in thread
From: Juergen Gross @ 2010-11-26 10:24 UTC (permalink / raw)
  To: Keir Fraser; +Cc: xen-devel

Thanks, is working again!


Juergen

On 11/26/10 11:12, Keir Fraser wrote:
> Fixed by xen-unstable:22342.
>
>   K.
>
> On 25/11/2010 10:27, "Juergen Gross"<juergen.gross@ts.fujitsu.com>  wrote:
>
>> Hi,
>>
>> latest xen-unstable crashed when I tried to remove a cpu from a cpupool via
>>
>> xl cpupool-cpu-remove
>>
>> Serial log shows:
>>
>> (XEN) Assertion '!(((*({ unsigned long __ptr; __asm__ ("" : "=r"(__ptr) :
>> "0"(&per_cpu____preempt_count)); (typeof(&per_cpu____preempt_count)) (__ptr +
>> (get_cpu_info()->per_cpu_offset)); }))) ||
>> ((irq_stat[(((get_cpu_info()->processor_id)))].__local_irq_count) != 0) ||
>> !local_irq_is_enabled())' failed at softirq.c:61
>> (XEN) ----[ Xen-4.1-unstable  x86_64  debug=y  Tainted:    C ]----
>> (XEN) CPU:    5
>> (XEN) RIP:    e008:[<ffff82c4801212f9>] do_softirq+0x51/0x7a
>> (XEN) RFLAGS: 0000000000010202   CONTEXT: hypervisor
>> (XEN) rax: ffff82c4802ca2a0   rbx: ffff8300780da000   rcx: ffff82c4802a7080
>> (XEN) rdx: 00000043b9aefd80   rsi: ffff830839db7ec0   rdi: ffff82c48024d0e0
>> (XEN) rbp: ffff830839db7f08   rsp: ffff830839db7f08   r8:  0000000000000005
>> (XEN) r9:  0000ffff0000ffff   r10: 00000000deadbeef   r11: 0f0f0f0f0f0f0f0f
>> (XEN) r12: ffff8803f8f61440   r13: ffff8803fdf4fe58   r14: 0000000000000000
>> (XEN) r15: 00007fff335d7150   cr0: 0000000080050033   cr4: 00000000000026f0
>> (XEN) cr3: 0000000846990000   cr2: 0000000000627048
>> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
>> (XEN) Xen stack trace from rsp=ffff830839db7f08:
>> (XEN)    00007cf7c62480c7 ffff82c480202736 00007fff335d7150 0000000000000000
>> (XEN)    ffff8803fdf4fe58 ffff8803f8f61440 ffffffffffffffda ffff8803fd491438
>> (XEN)    0000000000000216 000000000061e004 000000000061d050 00007f6d701d9f6c
>> (XEN)    0000000000000000 ffffffff8000346a 000000000061c2e0 0000000000624004
>> (XEN)    00000000deadbeef 0000010000000000 ffffffff8000346a 000000000000e033
>> (XEN)    0000000000000216 ffff8803fdf4fd80 000000000000e02b 0000000000000000
>> (XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000005
>> (XEN)    ffff8300780da000 00000043b9aefd80 0000000000000000
>> (XEN) Xen call trace:
>> (XEN)    [<ffff82c4801212f9>] do_softirq+0x51/0x7a
>> (XEN)
>>
>> I suspect the waitqueue stuff broke the continue_hypercall_on_cpu() stuff.
>>
>>
>> Juergen
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>


-- 
Juergen Gross                 Principal Developer Operating Systems
TSP ES&S SWE OS6                       Telephone: +49 (0) 89 3222 2967
Fujitsu Technology Solutions              e-mail: juergen.gross@ts.fujitsu.com
Domagkstr. 28                           Internet: ts.fujitsu.com
D-80807 Muenchen                 Company details: ts.fujitsu.com/imprint.html

^ permalink raw reply	[flat|nested] 5+ messages in thread
end of thread, other threads:[~2010-11-26 10:24 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-11-25 10:27 hypervisor crash with latest xen-unstable Juergen Gross
2010-11-25 17:14 ` Keir Fraser
2010-11-26  5:32   ` Juergen Gross
2010-11-26 10:12 ` Keir Fraser
2010-11-26 10:24   ` Juergen Gross

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.