All of lore.kernel.org
 help / color / mirror / Atom feed
* [Qemu-devel] Regression: qemu crash of hvm domUs with spice (backtrace included)
@ 2015-04-20 14:10 Fabio Fantoni
  2015-04-21 10:49 ` Stefano Stabellini
  2015-04-21 10:49 ` [Qemu-devel] " Stefano Stabellini
  0 siblings, 2 replies; 30+ messages in thread
From: Fabio Fantoni @ 2015-04-20 14:10 UTC (permalink / raw)
  To: xen-devel
  Cc: Anthony PERARD, spice-devel, Gerd Hoffmann, qemu-devel,
	Stefano Stabellini

[-- Attachment #1: Type: text/plain, Size: 1257 bytes --]

I updated xen and qemu from xen 4.5.0 with its upstream qemu included to 
xen 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk to 
use revision "master").
After few minutes I booted windows 7 64 bit domU qemu crash, tried 2 
times with same result.

In the domU's qemu log:
> qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion `(old_top == 
> (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - 
> __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || 
> ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof 
> (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & 
> ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && 
> ((unsigned long)old_end & pagemask) == 0)' failed.
> Killing all inferiors

In attachment the full backtrace of qemu crash.

With a fast search after I saw the backtrace I found a probable cause of 
regression (I'm not sure):
http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa
spice: make sure we don't overflow ssd->buf

Added also qemu-devel and spice-devel as cc.

If you need more informations/tests tell me and I'll post them.

Thanks for any reply and sorry for my bad english.



[-- Attachment #2: qemu crash.log --]
[-- Type: text/plain, Size: 7540 bytes --]

Program received signal SIGABRT, Aborted.
[Switching to Thread 5234]
0x00007ffff3905165 in raise () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) bt full
#0  0x00007ffff3905165 in raise () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#1  0x00007ffff39083e0 in abort () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#2  0x00007ffff3948dea in ?? () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#3  0x00007ffff394bd13 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#4  0x00007ffff394da70 in malloc () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5  0x00007ffff4d38550 in spice_malloc (n_bytes=1184900) at mem.c:93
        mem = <optimized out>
        __FUNCTION__ = "spice_malloc"
#6  0x00007ffff4d389be in spice_chunks_linearize (chunks=0x7fffdc1fb6b0)
    at mem.c:226
        data = <optimized out>
        p = <optimized out>
        i = <optimized out>
#7  0x00007ffff4d16b56 in canvas_bitmap_to_surface (
    canvas=canvas@entry=0x555556719de0, bitmap=bitmap@entry=0x7fffdc1a2c08, 
    palette=0x0, want_original=1) at ../spice-common/common/canvas_base.c:635
        src = <optimized out>
        image = <optimized out>
        format = <optimized out>
        __FUNCTION__ = "canvas_bitmap_to_surface"
---Type <return> to continue, or q <return> to quit---
#8  0x00007ffff4d16ce2 in canvas_get_bits (want_original=<optimized out>, 
    bitmap=0x7fffdc1a2c08, canvas=0x555556719de0)
    at ../spice-common/common/canvas_base.c:964
        palette = <optimized out>
#9  canvas_get_image_internal (canvas=canvas@entry=0x555556719de0, 
    image=0x7fffdc1a2bf0, want_original=<optimized out>, 
    want_original@entry=0, real_get=real_get@entry=1)
    at ../spice-common/common/canvas_base.c:1141
        descriptor = 0x7fffdc1a2bf0
        surface = <optimized out>
        converted = <optimized out>
        wanted_format = 1
        surface_format = <optimized out>
        saved_want_original = <optimized out>
        __FUNCTION__ = "canvas_get_image_internal"
#10 0x00007ffff4d173ba in canvas_get_image (
    canvas=canvas@entry=0x555556719de0, image=<optimized out>, 
    want_original=want_original@entry=0)
    at ../spice-common/common/canvas_base.c:1285
No locals.
#11 0x00007ffff4d1970e in canvas_draw_copy (spice_canvas=0x555556719de0, 
    bbox=0x7fffdc207a50, clip=<optimized out>, copy=0x7fffe4dfc320)
    at ../spice-common/common/canvas_base.c:2258
        canvas = 0x555556719de0
        dest_region = {extents = {x1 = 0, y1 = 708, x2 = 425, y2 = 728}, 
---Type <return> to continue, or q <return> to quit---
          data = 0x0}
        surface_canvas = <optimized out>
        src_image = <optimized out>
        rop = SPICE_ROP_COPY
        __FUNCTION__ = "canvas_draw_copy"
#12 0x00007ffff4cecffc in red_draw_qxl_drawable (
    worker=worker@entry=0x7fffe4423010, 
    drawable=drawable@entry=0x7fffe45d6a88) at red_worker.c:4394
        copy = {src_bitmap = 0x7fffdc1a2bf0, src_area = {left = 0, top = 677, 
            right = 425, bottom = 697}, rop_descriptor = 8, 
          scale_mode = 1 '\001', mask = {flags = 245 '\365', pos = {
              x = -173079809, y = -173079809}, bitmap = 0x0}}
        img1 = {descriptor = {id = 93825007287960, type = 48 '0', 
            flags = 193 '\301', width = 21845, height = 4210421981}, u = {
            bitmap = {format = 55 '7', flags = 10 '\n', x = 0, 
              y = 3867565524, stride = 32767, palette = 0x7fffe805fffc, 
              palette_id = 606579, data = 0x7fffdc000078}, quic = {
              data_size = 2615, data = 0x7fffe6865dd4}, surface = {
              surface_id = 2615}, lz_rgb = {data_size = 2615, 
              data = 0x7fffe6865dd4}, lz_plt = {flags = 55 '7', 
              data_size = 0, palette = 0x7fffe6865dd4, 
              palette_id = 140737086095356, data = 0x94173}, jpeg = {
              data_size = 2615, data = 0x7fffe6865dd4}, zlib_glz = {
              glz_data_size = 2615, data_size = 0, data = 0x7fffe6865dd4}, 
            jpeg_alpha = {flags = 55 '7', jpeg_size = 0, 
---Type <return> to continue, or q <return> to quit---
              data_size = 3867565524, data = 0x7fffe805fffc}}}
        img2 = {descriptor = {id = 140737060953572, type = 236 '\354', 
            flags = 93 ']', width = 32767, height = 0}, u = {bitmap = {
              format = 69 'E', flags = 105 'i', x = 32767, y = 128, 
              stride = 0, palette = 0x555556438fc0, 
              palette_id = 140737060952556, data = 0x9b5}, quic = {
              data_size = 4107102533, data = 0x80}, surface = {
              surface_id = 4107102533}, lz_rgb = {data_size = 4107102533, 
              data = 0x80}, lz_plt = {flags = 69 'E', data_size = 32767, 
              palette = 0x80, palette_id = 93825007849408, 
              data = 0x7fffe68659ec}, jpeg = {data_size = 4107102533, 
              data = 0x80}, zlib_glz = {glz_data_size = 4107102533, 
              data_size = 32767, data = 0x80}, jpeg_alpha = {flags = 69 'E', 
              jpeg_size = 32767, data_size = 128, data = 0x555556438fc0}}}
        surface = 0x7fffe44232f0
        canvas = 0x555556719de0
        clip = {type = 0 '\000', rects = 0x0}
        __FUNCTION__ = "red_draw_qxl_drawable"
#13 0x00007ffff4cf9295 in red_draw_drawable (drawable=0x7fffe45d6a88, 
    worker=0x7fffe4423010) at red_worker.c:4507
No locals.
#14 red_update_area (worker=worker@entry=0x7fffe4423010, 
    area=area@entry=0x7fffe4dfcb60, surface_id=surface_id@entry=0)
    at red_worker.c:4760
        container = <optimized out>
---Type <return> to continue, or q <return> to quit---
        surface = 0x7fffe44232f0
        ring = 0x7fffe4423308
        ring_item = <optimized out>
        rgn = {extents = {x1 = 0, y1 = 0, x2 = 1366, y2 = 768}, data = 0x0}
        last = 0x7fffe45d7898
        now = 0x7fffe45d6a88
        __FUNCTION__ = "red_update_area"
#15 0x00007ffff4d04d76 in handle_dev_update_async (opaque=0x7fffe4423010, 
    payload=<optimized out>) at red_worker.c:10847
        worker = 0x7fffe4423010
        msg = <optimized out>
        rect = {left = 0, top = 0, right = 1366, bottom = 768}
        qxl_dirty_rects = <optimized out>
        num_dirty_rects = <optimized out>
        surface = <optimized out>
        surface_id = 0
        qxl_area = {top = 0, left = 0, bottom = 768, right = 1366}
        clear_dirty_region = 1
        __FUNCTION__ = "handle_dev_update_async"
        __func__ = "handle_dev_update_async"
#16 0x00007ffff4ce5044 in dispatcher_handle_single_read (
    dispatcher=0x555556428db8) at dispatcher.c:139
        ret = <optimized out>
        type = <optimized out>
        msg = 0x5555563cbff0
---Type <return> to continue, or q <return> to quit---
        ack = 4294967295
        payload = 0x5555563a57d0 "P0\032\334\377\177"
#17 dispatcher_handle_recv_read (dispatcher=0x555556428db8)
    at dispatcher.c:162
No locals.
#18 0x00007ffff4d072dc in red_worker_main (arg=<optimized out>)
    at red_worker.c:12021
        events = <optimized out>
        i = <optimized out>
        num_events = 1
        timers_queue_timeout = <optimized out>
        worker = 0x7fffe4423010
        __FUNCTION__ = "red_worker_main"
#19 0x00007ffff3c64b50 in start_thread ()
   from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#20 0x00007ffff39ae95d in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#21 0x0000000000000000 in ?? ()
No symbol table info available.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: [Qemu-devel] Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-04-20 14:10 [Qemu-devel] Regression: qemu crash of hvm domUs with spice (backtrace included) Fabio Fantoni
  2015-04-21 10:49 ` Stefano Stabellini
@ 2015-04-21 10:49 ` Stefano Stabellini
  2015-04-21 11:38   ` Fabio Fantoni
  2015-04-21 11:38   ` [Qemu-devel] " Fabio Fantoni
  1 sibling, 2 replies; 30+ messages in thread
From: Stefano Stabellini @ 2015-04-21 10:49 UTC (permalink / raw)
  To: Fabio Fantoni
  Cc: Stefano Stabellini, qemu-devel, xen-devel, Gerd Hoffmann,
	Anthony PERARD, spice-devel

On Mon, 20 Apr 2015, Fabio Fantoni wrote:
> I updated xen and qemu from xen 4.5.0 with its upstream qemu included to xen
> 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk to use
> revision "master").
> After few minutes I booted windows 7 64 bit domU qemu crash, tried 2 times
> with same result.
> 
> In the domU's qemu log:
> > qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion `(old_top ==
> > (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof
> > (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long)
> > (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk,
> > fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) -
> > 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) ==
> > 0)' failed.
> > Killing all inferiors
> 
> In attachment the full backtrace of qemu crash.
> 
> With a fast search after I saw the backtrace I found a probable cause of
> regression (I'm not sure):
> http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa
> spice: make sure we don't overflow ssd->buf
> 
> Added also qemu-devel and spice-devel as cc.
> 
> If you need more informations/tests tell me and I'll post them.
 
Maybe you could try to revert the offending commit
(5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better bisect the
crash?

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-04-20 14:10 [Qemu-devel] Regression: qemu crash of hvm domUs with spice (backtrace included) Fabio Fantoni
@ 2015-04-21 10:49 ` Stefano Stabellini
  2015-04-21 10:49 ` [Qemu-devel] " Stefano Stabellini
  1 sibling, 0 replies; 30+ messages in thread
From: Stefano Stabellini @ 2015-04-21 10:49 UTC (permalink / raw)
  To: Fabio Fantoni
  Cc: Stefano Stabellini, qemu-devel, xen-devel, Gerd Hoffmann,
	Anthony PERARD, spice-devel

On Mon, 20 Apr 2015, Fabio Fantoni wrote:
> I updated xen and qemu from xen 4.5.0 with its upstream qemu included to xen
> 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk to use
> revision "master").
> After few minutes I booted windows 7 64 bit domU qemu crash, tried 2 times
> with same result.
> 
> In the domU's qemu log:
> > qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion `(old_top ==
> > (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof
> > (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long)
> > (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk,
> > fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) -
> > 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) ==
> > 0)' failed.
> > Killing all inferiors
> 
> In attachment the full backtrace of qemu crash.
> 
> With a fast search after I saw the backtrace I found a probable cause of
> regression (I'm not sure):
> http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa
> spice: make sure we don't overflow ssd->buf
> 
> Added also qemu-devel and spice-devel as cc.
> 
> If you need more informations/tests tell me and I'll post them.
 
Maybe you could try to revert the offending commit
(5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better bisect the
crash?

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: [Qemu-devel] Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-04-21 10:49 ` [Qemu-devel] " Stefano Stabellini
  2015-04-21 11:38   ` Fabio Fantoni
@ 2015-04-21 11:38   ` Fabio Fantoni
  2015-04-21 12:53     ` Stefano Stabellini
  2015-04-21 12:53     ` [Qemu-devel] " Stefano Stabellini
  1 sibling, 2 replies; 30+ messages in thread
From: Fabio Fantoni @ 2015-04-21 11:38 UTC (permalink / raw)
  To: Stefano Stabellini
  Cc: Anthony PERARD, spice-devel, Gerd Hoffmann, qemu-devel, xen-devel

Il 21/04/2015 12:49, Stefano Stabellini ha scritto:
> On Mon, 20 Apr 2015, Fabio Fantoni wrote:
>> I updated xen and qemu from xen 4.5.0 with its upstream qemu included to xen
>> 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk to use
>> revision "master").
>> After few minutes I booted windows 7 64 bit domU qemu crash, tried 2 times
>> with same result.
>>
>> In the domU's qemu log:
>>> qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion `(old_top ==
>>> (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof
>>> (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long)
>>> (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk,
>>> fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) -
>>> 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) ==
>>> 0)' failed.
>>> Killing all inferiors
>> In attachment the full backtrace of qemu crash.
>>
>> With a fast search after I saw the backtrace I found a probable cause of
>> regression (I'm not sure):
>> http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa
>> spice: make sure we don't overflow ssd->buf
>>
>> Added also qemu-devel and spice-devel as cc.
>>
>> If you need more informations/tests tell me and I'll post them.
>   
> Maybe you could try to revert the offending commit
> (5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better bisect the
> crash?
Thanks for your reply.

I reverted to 4.5.0 on dom0 for now on that system because I'm busy 
trying to found another problem that cause very bad performance without 
errors or nothing in logs :( I don't know if if xen related, kernel 
related or other for now.

About this regression with spice I'll do further tests in next days 
(probably starting reverting the spice patch in qemu) but any help is 
appreciated.
Based on data I have for now is possible that the problem is that qemu 
try to allocate other ram or videoram after domU create but with xen is 
not possible?
In the spice related patch I saw something about dynamic allocation for 
example.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-04-21 10:49 ` [Qemu-devel] " Stefano Stabellini
@ 2015-04-21 11:38   ` Fabio Fantoni
  2015-04-21 11:38   ` [Qemu-devel] " Fabio Fantoni
  1 sibling, 0 replies; 30+ messages in thread
From: Fabio Fantoni @ 2015-04-21 11:38 UTC (permalink / raw)
  To: Stefano Stabellini
  Cc: Anthony PERARD, spice-devel, Gerd Hoffmann, qemu-devel, xen-devel

Il 21/04/2015 12:49, Stefano Stabellini ha scritto:
> On Mon, 20 Apr 2015, Fabio Fantoni wrote:
>> I updated xen and qemu from xen 4.5.0 with its upstream qemu included to xen
>> 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk to use
>> revision "master").
>> After few minutes I booted windows 7 64 bit domU qemu crash, tried 2 times
>> with same result.
>>
>> In the domU's qemu log:
>>> qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion `(old_top ==
>>> (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof
>>> (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long)
>>> (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk,
>>> fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) -
>>> 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) ==
>>> 0)' failed.
>>> Killing all inferiors
>> In attachment the full backtrace of qemu crash.
>>
>> With a fast search after I saw the backtrace I found a probable cause of
>> regression (I'm not sure):
>> http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa
>> spice: make sure we don't overflow ssd->buf
>>
>> Added also qemu-devel and spice-devel as cc.
>>
>> If you need more informations/tests tell me and I'll post them.
>   
> Maybe you could try to revert the offending commit
> (5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better bisect the
> crash?
Thanks for your reply.

I reverted to 4.5.0 on dom0 for now on that system because I'm busy 
trying to found another problem that cause very bad performance without 
errors or nothing in logs :( I don't know if if xen related, kernel 
related or other for now.

About this regression with spice I'll do further tests in next days 
(probably starting reverting the spice patch in qemu) but any help is 
appreciated.
Based on data I have for now is possible that the problem is that qemu 
try to allocate other ram or videoram after domU create but with xen is 
not possible?
In the spice related patch I saw something about dynamic allocation for 
example.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: [Qemu-devel] Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-04-21 11:38   ` [Qemu-devel] " Fabio Fantoni
  2015-04-21 12:53     ` Stefano Stabellini
@ 2015-04-21 12:53     ` Stefano Stabellini
  2015-05-11 15:04       ` Fabio Fantoni
  2015-05-11 15:04       ` Fabio Fantoni
  1 sibling, 2 replies; 30+ messages in thread
From: Stefano Stabellini @ 2015-04-21 12:53 UTC (permalink / raw)
  To: Fabio Fantoni
  Cc: Stefano Stabellini, qemu-devel, xen-devel, Gerd Hoffmann,
	Anthony PERARD, spice-devel

On Tue, 21 Apr 2015, Fabio Fantoni wrote:
> Il 21/04/2015 12:49, Stefano Stabellini ha scritto:
> > On Mon, 20 Apr 2015, Fabio Fantoni wrote:
> > > I updated xen and qemu from xen 4.5.0 with its upstream qemu included to
> > > xen
> > > 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk to use
> > > revision "master").
> > > After few minutes I booted windows 7 64 bit domU qemu crash, tried 2 times
> > > with same result.
> > > 
> > > In the domU's qemu log:
> > > > qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion `(old_top ==
> > > > (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) -
> > > > __builtin_offsetof
> > > > (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long)
> > > > (old_size) >= (unsigned long)((((__builtin_offsetof (struct
> > > > malloc_chunk,
> > > > fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) -
> > > > 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask)
> > > > ==
> > > > 0)' failed.
> > > > Killing all inferiors
> > > In attachment the full backtrace of qemu crash.
> > > 
> > > With a fast search after I saw the backtrace I found a probable cause of
> > > regression (I'm not sure):
> > > http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa
> > > spice: make sure we don't overflow ssd->buf
> > > 
> > > Added also qemu-devel and spice-devel as cc.
> > > 
> > > If you need more informations/tests tell me and I'll post them.
> >   Maybe you could try to revert the offending commit
> > (5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better bisect the
> > crash?
> Thanks for your reply.
> 
> I reverted to 4.5.0 on dom0 for now on that system because I'm busy trying to
> found another problem that cause very bad performance without errors or
> nothing in logs :( I don't know if if xen related, kernel related or other for
> now.
> 
> About this regression with spice I'll do further tests in next days (probably
> starting reverting the spice patch in qemu) but any help is appreciated.
> Based on data I have for now is possible that the problem is that qemu try to
> allocate other ram or videoram after domU create but with xen is not possible?
> In the spice related patch I saw something about dynamic allocation for
> example.

It is probably caused by a commit in the range:

1ebb75b1fee779621b63e84fefa7b07354c43a99..0b8fb1ec3d666d1eb8bbff56c76c5e6daa2789e4

there are only 10 commits in that range. By using git bisect you should
be able to narrow it down in just 3 tests.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-04-21 11:38   ` [Qemu-devel] " Fabio Fantoni
@ 2015-04-21 12:53     ` Stefano Stabellini
  2015-04-21 12:53     ` [Qemu-devel] " Stefano Stabellini
  1 sibling, 0 replies; 30+ messages in thread
From: Stefano Stabellini @ 2015-04-21 12:53 UTC (permalink / raw)
  To: Fabio Fantoni
  Cc: Stefano Stabellini, qemu-devel, xen-devel, Gerd Hoffmann,
	Anthony PERARD, spice-devel

On Tue, 21 Apr 2015, Fabio Fantoni wrote:
> Il 21/04/2015 12:49, Stefano Stabellini ha scritto:
> > On Mon, 20 Apr 2015, Fabio Fantoni wrote:
> > > I updated xen and qemu from xen 4.5.0 with its upstream qemu included to
> > > xen
> > > 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk to use
> > > revision "master").
> > > After few minutes I booted windows 7 64 bit domU qemu crash, tried 2 times
> > > with same result.
> > > 
> > > In the domU's qemu log:
> > > > qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion `(old_top ==
> > > > (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) -
> > > > __builtin_offsetof
> > > > (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long)
> > > > (old_size) >= (unsigned long)((((__builtin_offsetof (struct
> > > > malloc_chunk,
> > > > fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) -
> > > > 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask)
> > > > ==
> > > > 0)' failed.
> > > > Killing all inferiors
> > > In attachment the full backtrace of qemu crash.
> > > 
> > > With a fast search after I saw the backtrace I found a probable cause of
> > > regression (I'm not sure):
> > > http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa
> > > spice: make sure we don't overflow ssd->buf
> > > 
> > > Added also qemu-devel and spice-devel as cc.
> > > 
> > > If you need more informations/tests tell me and I'll post them.
> >   Maybe you could try to revert the offending commit
> > (5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better bisect the
> > crash?
> Thanks for your reply.
> 
> I reverted to 4.5.0 on dom0 for now on that system because I'm busy trying to
> found another problem that cause very bad performance without errors or
> nothing in logs :( I don't know if if xen related, kernel related or other for
> now.
> 
> About this regression with spice I'll do further tests in next days (probably
> starting reverting the spice patch in qemu) but any help is appreciated.
> Based on data I have for now is possible that the problem is that qemu try to
> allocate other ram or videoram after domU create but with xen is not possible?
> In the spice related patch I saw something about dynamic allocation for
> example.

It is probably caused by a commit in the range:

1ebb75b1fee779621b63e84fefa7b07354c43a99..0b8fb1ec3d666d1eb8bbff56c76c5e6daa2789e4

there are only 10 commits in that range. By using git bisect you should
be able to narrow it down in just 3 tests.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: [Qemu-devel] Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-04-21 12:53     ` [Qemu-devel] " Stefano Stabellini
@ 2015-05-11 15:04       ` Fabio Fantoni
  2015-05-12  9:23         ` Fabio Fantoni
  2015-05-12  9:23         ` [Qemu-devel] " Fabio Fantoni
  2015-05-11 15:04       ` Fabio Fantoni
  1 sibling, 2 replies; 30+ messages in thread
From: Fabio Fantoni @ 2015-05-11 15:04 UTC (permalink / raw)
  To: Stefano Stabellini
  Cc: Anthony PERARD, spice-devel, Gerd Hoffmann, qemu-devel, xen-devel

Il 21/04/2015 14:53, Stefano Stabellini ha scritto:
> On Tue, 21 Apr 2015, Fabio Fantoni wrote:
>> Il 21/04/2015 12:49, Stefano Stabellini ha scritto:
>>> On Mon, 20 Apr 2015, Fabio Fantoni wrote:
>>>> I updated xen and qemu from xen 4.5.0 with its upstream qemu included to
>>>> xen
>>>> 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk to use
>>>> revision "master").
>>>> After few minutes I booted windows 7 64 bit domU qemu crash, tried 2 times
>>>> with same result.
>>>>
>>>> In the domU's qemu log:
>>>>> qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion `(old_top ==
>>>>> (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) -
>>>>> __builtin_offsetof
>>>>> (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long)
>>>>> (old_size) >= (unsigned long)((((__builtin_offsetof (struct
>>>>> malloc_chunk,
>>>>> fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) -
>>>>> 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask)
>>>>> ==
>>>>> 0)' failed.
>>>>> Killing all inferiors
>>>> In attachment the full backtrace of qemu crash.
>>>>
>>>> With a fast search after I saw the backtrace I found a probable cause of
>>>> regression (I'm not sure):
>>>> http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa
>>>> spice: make sure we don't overflow ssd->buf
>>>>
>>>> Added also qemu-devel and spice-devel as cc.
>>>>
>>>> If you need more informations/tests tell me and I'll post them.
>>>    Maybe you could try to revert the offending commit
>>> (5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better bisect the
>>> crash?
>> Thanks for your reply.
>>
>> I reverted to 4.5.0 on dom0 for now on that system because I'm busy trying to
>> found another problem that cause very bad performance without errors or
>> nothing in logs :( I don't know if if xen related, kernel related or other for
>> now.
>>
>> About this regression with spice I'll do further tests in next days (probably
>> starting reverting the spice patch in qemu) but any help is appreciated.
>> Based on data I have for now is possible that the problem is that qemu try to
>> allocate other ram or videoram after domU create but with xen is not possible?
>> In the spice related patch I saw something about dynamic allocation for
>> example.
> It is probably caused by a commit in the range:
>
> 1ebb75b1fee779621b63e84fefa7b07354c43a99..0b8fb1ec3d666d1eb8bbff56c76c5e6daa2789e4
>
> there are only 10 commits in that range. By using git bisect you should
> be able to narrow it down in just 3 tests.

Sorry for delay, I was busy with many things, today I retried with 
updated stable-4.5 and also reverting "spice: make sure we don't 
overflow ssd->buf" (in a second test) but in both case regression remain :(
Tomorrow probably I'll do other tests.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-04-21 12:53     ` [Qemu-devel] " Stefano Stabellini
  2015-05-11 15:04       ` Fabio Fantoni
@ 2015-05-11 15:04       ` Fabio Fantoni
  1 sibling, 0 replies; 30+ messages in thread
From: Fabio Fantoni @ 2015-05-11 15:04 UTC (permalink / raw)
  To: Stefano Stabellini
  Cc: Anthony PERARD, spice-devel, Gerd Hoffmann, qemu-devel, xen-devel

Il 21/04/2015 14:53, Stefano Stabellini ha scritto:
> On Tue, 21 Apr 2015, Fabio Fantoni wrote:
>> Il 21/04/2015 12:49, Stefano Stabellini ha scritto:
>>> On Mon, 20 Apr 2015, Fabio Fantoni wrote:
>>>> I updated xen and qemu from xen 4.5.0 with its upstream qemu included to
>>>> xen
>>>> 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk to use
>>>> revision "master").
>>>> After few minutes I booted windows 7 64 bit domU qemu crash, tried 2 times
>>>> with same result.
>>>>
>>>> In the domU's qemu log:
>>>>> qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion `(old_top ==
>>>>> (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) -
>>>>> __builtin_offsetof
>>>>> (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long)
>>>>> (old_size) >= (unsigned long)((((__builtin_offsetof (struct
>>>>> malloc_chunk,
>>>>> fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) -
>>>>> 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask)
>>>>> ==
>>>>> 0)' failed.
>>>>> Killing all inferiors
>>>> In attachment the full backtrace of qemu crash.
>>>>
>>>> With a fast search after I saw the backtrace I found a probable cause of
>>>> regression (I'm not sure):
>>>> http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa
>>>> spice: make sure we don't overflow ssd->buf
>>>>
>>>> Added also qemu-devel and spice-devel as cc.
>>>>
>>>> If you need more informations/tests tell me and I'll post them.
>>>    Maybe you could try to revert the offending commit
>>> (5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better bisect the
>>> crash?
>> Thanks for your reply.
>>
>> I reverted to 4.5.0 on dom0 for now on that system because I'm busy trying to
>> found another problem that cause very bad performance without errors or
>> nothing in logs :( I don't know if if xen related, kernel related or other for
>> now.
>>
>> About this regression with spice I'll do further tests in next days (probably
>> starting reverting the spice patch in qemu) but any help is appreciated.
>> Based on data I have for now is possible that the problem is that qemu try to
>> allocate other ram or videoram after domU create but with xen is not possible?
>> In the spice related patch I saw something about dynamic allocation for
>> example.
> It is probably caused by a commit in the range:
>
> 1ebb75b1fee779621b63e84fefa7b07354c43a99..0b8fb1ec3d666d1eb8bbff56c76c5e6daa2789e4
>
> there are only 10 commits in that range. By using git bisect you should
> be able to narrow it down in just 3 tests.

Sorry for delay, I was busy with many things, today I retried with 
updated stable-4.5 and also reverting "spice: make sure we don't 
overflow ssd->buf" (in a second test) but in both case regression remain :(
Tomorrow probably I'll do other tests.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: [Qemu-devel] Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-05-11 15:04       ` Fabio Fantoni
  2015-05-12  9:23         ` Fabio Fantoni
@ 2015-05-12  9:23         ` Fabio Fantoni
  2015-05-12 10:26           ` Fabio Fantoni
  2015-05-12 10:26           ` Fabio Fantoni
  1 sibling, 2 replies; 30+ messages in thread
From: Fabio Fantoni @ 2015-05-12  9:23 UTC (permalink / raw)
  To: Stefano Stabellini
  Cc: qemu-devel, xen-devel, Gerd Hoffmann, Jan Beulich,
	Anthony PERARD, spice-devel

Il 11/05/2015 17:04, Fabio Fantoni ha scritto:
> Il 21/04/2015 14:53, Stefano Stabellini ha scritto:
>> On Tue, 21 Apr 2015, Fabio Fantoni wrote:
>>> Il 21/04/2015 12:49, Stefano Stabellini ha scritto:
>>>> On Mon, 20 Apr 2015, Fabio Fantoni wrote:
>>>>> I updated xen and qemu from xen 4.5.0 with its upstream qemu 
>>>>> included to
>>>>> xen
>>>>> 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk to 
>>>>> use
>>>>> revision "master").
>>>>> After few minutes I booted windows 7 64 bit domU qemu crash, tried 
>>>>> 2 times
>>>>> with same result.
>>>>>
>>>>> In the domU's qemu log:
>>>>>> qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion `(old_top ==
>>>>>> (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) -
>>>>>> __builtin_offsetof
>>>>>> (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long)
>>>>>> (old_size) >= (unsigned long)((((__builtin_offsetof (struct
>>>>>> malloc_chunk,
>>>>>> fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * 
>>>>>> (sizeof(size_t))) -
>>>>>> 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & 
>>>>>> pagemask)
>>>>>> ==
>>>>>> 0)' failed.
>>>>>> Killing all inferiors
>>>>> In attachment the full backtrace of qemu crash.
>>>>>
>>>>> With a fast search after I saw the backtrace I found a probable 
>>>>> cause of
>>>>> regression (I'm not sure):
>>>>> http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa 
>>>>>
>>>>> spice: make sure we don't overflow ssd->buf
>>>>>
>>>>> Added also qemu-devel and spice-devel as cc.
>>>>>
>>>>> If you need more informations/tests tell me and I'll post them.
>>>>    Maybe you could try to revert the offending commit
>>>> (5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better bisect the
>>>> crash?
>>> Thanks for your reply.
>>>
>>> I reverted to 4.5.0 on dom0 for now on that system because I'm busy 
>>> trying to
>>> found another problem that cause very bad performance without errors or
>>> nothing in logs :( I don't know if if xen related, kernel related or 
>>> other for
>>> now.
>>>
>>> About this regression with spice I'll do further tests in next days 
>>> (probably
>>> starting reverting the spice patch in qemu) but any help is 
>>> appreciated.
>>> Based on data I have for now is possible that the problem is that 
>>> qemu try to
>>> allocate other ram or videoram after domU create but with xen is not 
>>> possible?
>>> In the spice related patch I saw something about dynamic allocation for
>>> example.
>> It is probably caused by a commit in the range:
>>
>> 1ebb75b1fee779621b63e84fefa7b07354c43a99..0b8fb1ec3d666d1eb8bbff56c76c5e6daa2789e4 
>>
>>
>> there are only 10 commits in that range. By using git bisect you should
>> be able to narrow it down in just 3 tests.
>
> Sorry for delay, I was busy with many things, today I retried with 
> updated stable-4.5 and also reverting "spice: make sure we don't 
> overflow ssd->buf" (in a second test) but in both case regression 
> remain :(
> Tomorrow probably I'll do other tests.

I did another test, reverting this instead:
http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commit;h=c9ac5f816bf3a8b56f836b078711dcef6e5c90b8
And now seems I'm unable to reproduce the regression, before happen 
after few seconds up to 1-2 minutes, now I use the same domU 15-20 
minutes without problem.
Probably is the cause of regression even if seems strange that on 
unstable with same patch on tests of some days ago didn't happen.

Any ideas?

Thanks for any reply and sorry for my bad english.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-05-11 15:04       ` Fabio Fantoni
@ 2015-05-12  9:23         ` Fabio Fantoni
  2015-05-12  9:23         ` [Qemu-devel] " Fabio Fantoni
  1 sibling, 0 replies; 30+ messages in thread
From: Fabio Fantoni @ 2015-05-12  9:23 UTC (permalink / raw)
  To: Stefano Stabellini
  Cc: qemu-devel, xen-devel, Gerd Hoffmann, Jan Beulich,
	Anthony PERARD, spice-devel

Il 11/05/2015 17:04, Fabio Fantoni ha scritto:
> Il 21/04/2015 14:53, Stefano Stabellini ha scritto:
>> On Tue, 21 Apr 2015, Fabio Fantoni wrote:
>>> Il 21/04/2015 12:49, Stefano Stabellini ha scritto:
>>>> On Mon, 20 Apr 2015, Fabio Fantoni wrote:
>>>>> I updated xen and qemu from xen 4.5.0 with its upstream qemu 
>>>>> included to
>>>>> xen
>>>>> 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk to 
>>>>> use
>>>>> revision "master").
>>>>> After few minutes I booted windows 7 64 bit domU qemu crash, tried 
>>>>> 2 times
>>>>> with same result.
>>>>>
>>>>> In the domU's qemu log:
>>>>>> qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion `(old_top ==
>>>>>> (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) -
>>>>>> __builtin_offsetof
>>>>>> (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long)
>>>>>> (old_size) >= (unsigned long)((((__builtin_offsetof (struct
>>>>>> malloc_chunk,
>>>>>> fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * 
>>>>>> (sizeof(size_t))) -
>>>>>> 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & 
>>>>>> pagemask)
>>>>>> ==
>>>>>> 0)' failed.
>>>>>> Killing all inferiors
>>>>> In attachment the full backtrace of qemu crash.
>>>>>
>>>>> With a fast search after I saw the backtrace I found a probable 
>>>>> cause of
>>>>> regression (I'm not sure):
>>>>> http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa 
>>>>>
>>>>> spice: make sure we don't overflow ssd->buf
>>>>>
>>>>> Added also qemu-devel and spice-devel as cc.
>>>>>
>>>>> If you need more informations/tests tell me and I'll post them.
>>>>    Maybe you could try to revert the offending commit
>>>> (5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better bisect the
>>>> crash?
>>> Thanks for your reply.
>>>
>>> I reverted to 4.5.0 on dom0 for now on that system because I'm busy 
>>> trying to
>>> found another problem that cause very bad performance without errors or
>>> nothing in logs :( I don't know if if xen related, kernel related or 
>>> other for
>>> now.
>>>
>>> About this regression with spice I'll do further tests in next days 
>>> (probably
>>> starting reverting the spice patch in qemu) but any help is 
>>> appreciated.
>>> Based on data I have for now is possible that the problem is that 
>>> qemu try to
>>> allocate other ram or videoram after domU create but with xen is not 
>>> possible?
>>> In the spice related patch I saw something about dynamic allocation for
>>> example.
>> It is probably caused by a commit in the range:
>>
>> 1ebb75b1fee779621b63e84fefa7b07354c43a99..0b8fb1ec3d666d1eb8bbff56c76c5e6daa2789e4 
>>
>>
>> there are only 10 commits in that range. By using git bisect you should
>> be able to narrow it down in just 3 tests.
>
> Sorry for delay, I was busy with many things, today I retried with 
> updated stable-4.5 and also reverting "spice: make sure we don't 
> overflow ssd->buf" (in a second test) but in both case regression 
> remain :(
> Tomorrow probably I'll do other tests.

I did another test, reverting this instead:
http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commit;h=c9ac5f816bf3a8b56f836b078711dcef6e5c90b8
And now seems I'm unable to reproduce the regression, before happen 
after few seconds up to 1-2 minutes, now I use the same domU 15-20 
minutes without problem.
Probably is the cause of regression even if seems strange that on 
unstable with same patch on tests of some days ago didn't happen.

Any ideas?

Thanks for any reply and sorry for my bad english.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: [Qemu-devel] Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-05-12  9:23         ` [Qemu-devel] " Fabio Fantoni
@ 2015-05-12 10:26           ` Fabio Fantoni
  2015-05-12 13:54             ` Fabio Fantoni
  2015-05-12 13:54             ` [Qemu-devel] " Fabio Fantoni
  2015-05-12 10:26           ` Fabio Fantoni
  1 sibling, 2 replies; 30+ messages in thread
From: Fabio Fantoni @ 2015-05-12 10:26 UTC (permalink / raw)
  To: Stefano Stabellini
  Cc: qemu-devel, xen-devel, Gerd Hoffmann, Jan Beulich,
	Anthony PERARD, spice-devel

[-- Attachment #1: Type: text/plain, Size: 4151 bytes --]

Il 12/05/2015 11:23, Fabio Fantoni ha scritto:
> Il 11/05/2015 17:04, Fabio Fantoni ha scritto:
>> Il 21/04/2015 14:53, Stefano Stabellini ha scritto:
>>> On Tue, 21 Apr 2015, Fabio Fantoni wrote:
>>>> Il 21/04/2015 12:49, Stefano Stabellini ha scritto:
>>>>> On Mon, 20 Apr 2015, Fabio Fantoni wrote:
>>>>>> I updated xen and qemu from xen 4.5.0 with its upstream qemu 
>>>>>> included to
>>>>>> xen
>>>>>> 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk 
>>>>>> to use
>>>>>> revision "master").
>>>>>> After few minutes I booted windows 7 64 bit domU qemu crash, 
>>>>>> tried 2 times
>>>>>> with same result.
>>>>>>
>>>>>> In the domU's qemu log:
>>>>>>> qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion `(old_top ==
>>>>>>> (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) -
>>>>>>> __builtin_offsetof
>>>>>>> (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long)
>>>>>>> (old_size) >= (unsigned long)((((__builtin_offsetof (struct
>>>>>>> malloc_chunk,
>>>>>>> fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * 
>>>>>>> (sizeof(size_t))) -
>>>>>>> 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & 
>>>>>>> pagemask)
>>>>>>> ==
>>>>>>> 0)' failed.
>>>>>>> Killing all inferiors
>>>>>> In attachment the full backtrace of qemu crash.
>>>>>>
>>>>>> With a fast search after I saw the backtrace I found a probable 
>>>>>> cause of
>>>>>> regression (I'm not sure):
>>>>>> http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa 
>>>>>>
>>>>>> spice: make sure we don't overflow ssd->buf
>>>>>>
>>>>>> Added also qemu-devel and spice-devel as cc.
>>>>>>
>>>>>> If you need more informations/tests tell me and I'll post them.
>>>>>    Maybe you could try to revert the offending commit
>>>>> (5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better bisect the
>>>>> crash?
>>>> Thanks for your reply.
>>>>
>>>> I reverted to 4.5.0 on dom0 for now on that system because I'm busy 
>>>> trying to
>>>> found another problem that cause very bad performance without 
>>>> errors or
>>>> nothing in logs :( I don't know if if xen related, kernel related 
>>>> or other for
>>>> now.
>>>>
>>>> About this regression with spice I'll do further tests in next days 
>>>> (probably
>>>> starting reverting the spice patch in qemu) but any help is 
>>>> appreciated.
>>>> Based on data I have for now is possible that the problem is that 
>>>> qemu try to
>>>> allocate other ram or videoram after domU create but with xen is 
>>>> not possible?
>>>> In the spice related patch I saw something about dynamic allocation 
>>>> for
>>>> example.
>>> It is probably caused by a commit in the range:
>>>
>>> 1ebb75b1fee779621b63e84fefa7b07354c43a99..0b8fb1ec3d666d1eb8bbff56c76c5e6daa2789e4 
>>>
>>>
>>> there are only 10 commits in that range. By using git bisect you should
>>> be able to narrow it down in just 3 tests.
>>
>> Sorry for delay, I was busy with many things, today I retried with 
>> updated stable-4.5 and also reverting "spice: make sure we don't 
>> overflow ssd->buf" (in a second test) but in both case regression 
>> remain :(
>> Tomorrow probably I'll do other tests.
>
> I did another test, reverting this instead:
> http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commit;h=c9ac5f816bf3a8b56f836b078711dcef6e5c90b8 
>
> And now seems I'm unable to reproduce the regression, before happen 
> after few seconds up to 1-2 minutes, now I use the same domU 15-20 
> minutes without problem.
> Probably is the cause of regression even if seems strange that on 
> unstable with same patch on tests of some days ago didn't happen.
>
> Any ideas?
>
> Thanks for any reply and sorry for my bad english.

Bad news, qemu crash still happen even if this time in qemu log there is 
another output, see attachment.
After take a look on the other patches I saw:
http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commitdiff;h=7154fba0e51ec985ef621965d1b7120ad424fcbf
With "Conflicts: hw/display/vga.c" in description I'll try to revert it 
instead.

Or someone can tell me another probable test I can try?

[-- Attachment #2: qemu-dm-W7.log --]
[-- Type: text/plain, Size: 52916 bytes --]

main_channel_link: add main channel client
main_channel_handle_parsed: net test: latency 7.814000 ms, bitrate 5417989417 bps (5166.997354 Mbps)
inputs_connect: inputs channel client create
red_dispatcher_set_cursor_peer: 
main_channel_handle_parsed: agent start
main_channel_handle_parsed: agent start
red_channel_client_disconnect: rcc=0x7fa861b2eb60 (channel=0x7fa861b07e80 type=3 id=0)
red_channel_client_disconnect: rcc=0x7fa862349720 (channel=0x7fa861bc8b80 type=2 id=0)
red_channel_client_disconnect: rcc=0x7fa861ca6580 (channel=0x7fa861bb3990 type=4 id=0)
red_channel_client_disconnect_dummy: rcc=0x7fa861ecae20 (channel=0x7fa861c22c40 type=6 id=0)
snd_channel_put: SndChannel=0x7fa861e6f620 freed
red_channel_client_disconnect_dummy: rcc=0x7fa861ec6be0 (channel=0x7fa861b8c890 type=5 id=0)
snd_channel_put: SndChannel=0x7fa861d97ab0 freed
red_channel_client_disconnect: rcc=0x7fa861fbf120 (channel=0x7fa861bd57b0 type=9 id=0)
red_channel_client_disconnect: rcc=0x7fa861fbaee0 (channel=0x7fa861baff50 type=9 id=1)
red_channel_client_disconnect: rcc=0x7fa861eff170 (channel=0x7fa861c79f90 type=9 id=2)
red_channel_client_disconnect: rcc=0x7fa861f044b0 (channel=0x7fa861c7a7b0 type=9 id=3)
red_channel_client_disconnect: rcc=0x7fa861b52ad0 (channel=0x7fa861afc3f0 type=1 id=0)
main_channel_client_on_disconnect: rcc=0x7fa861b52ad0
red_client_destroy: destroy client 0x7fa861efe960 with #channels=6
red_dispatcher_disconnect_cursor_peer: 
red_dispatcher_disconnect_display_peer: 
main_channel_link: add main channel client
main_channel_handle_parsed: agent start
main_channel_handle_parsed: net test: latency 6.413000 ms, bitrate 1550340651 bps (1478.520061 Mbps)
*** glibc detected *** /usr/lib/xen/bin/qemu-system-i386: double free or corruption (!prev): 0x00007fa844d9c8c0 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x75be6)[0x7fa85bb86be6]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x6c)[0x7fa85bb8b98c]
/usr/lib/x86_64-linux-gnu/libpixman-1.so.0(+0x4344b)[0x7fa85c89a44b]
/usr/lib/x86_64-linux-gnu/libpixman-1.so.0(pixman_image_unref+0x9)[0x7fa85c89a399]
/usr/lib/xen/bin/qemu-system-i386(+0x30175b)[0x7fa86075b75b]
/usr/lib/xen/bin/qemu-system-i386(+0x32343b)[0x7fa86077d43b]
/usr/lib/xen/bin/qemu-system-i386(+0x2fad9d)[0x7fa860754d9d]
/usr/lib/xen/bin/qemu-system-i386(+0x19eb4a)[0x7fa8605f8b4a]
/usr/lib/xen/bin/qemu-system-i386(qxl_render_update_area_bh+0x41)[0x7fa8605f8e19]
/usr/lib/xen/bin/qemu-system-i386(+0xa2ba0)[0x7fa8604fcba0]
/usr/lib/xen/bin/qemu-system-i386(+0xa27f9)[0x7fa8604fc7f9]
/usr/lib/xen/bin/qemu-system-i386(+0xa2fc2)[0x7fa8604fcfc2]
/lib/x86_64-linux-gnu/libglib-2.0.so.0(g_main_context_dispatch+0x135)[0x7fa85f056355]
/usr/lib/xen/bin/qemu-system-i386(+0x28303e)[0x7fa8606dd03e]
/usr/lib/xen/bin/qemu-system-i386(+0x28313e)[0x7fa8606dd13e]
/usr/lib/xen/bin/qemu-system-i386(+0x283211)[0x7fa8606dd211]
/usr/lib/xen/bin/qemu-system-i386(+0x32ee14)[0x7fa860788e14]
/usr/lib/xen/bin/qemu-system-i386(main+0x3e76)[0x7fa8607908cf]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd)[0x7fa85bb2fead]
/usr/lib/xen/bin/qemu-system-i386(+0xa2169)[0x7fa8604fc169]
======= Memory map: ========
7fa82bf00000-7fa830000000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa830000000-7fa8336f9000 rw-p 00000000 00:00 0 
7fa8336f9000-7fa834000000 ---p 00000000 00:00 0 
7fa8345e1000-7fa8346e1000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8346e1000-7fa834ae2000 rw-p 00000000 00:00 0 
7fa834ae4000-7fa834be4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa834be4000-7fa834ce4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa834ce4000-7fa834de4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa834de4000-7fa834ee4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa834ee4000-7fa834fe4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa834fe4000-7fa8350e4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8350e4000-7fa8351e4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8351e4000-7fa8352e4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8352e4000-7fa8353e4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8353e4000-7fa8354e4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8354e4000-7fa8355e4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8355e4000-7fa8356e4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8356e4000-7fa8357e4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8357e4000-7fa8358e4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8358e4000-7fa8359e4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8359e4000-7fa835ae4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa835ae4000-7fa835be4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa835be4000-7fa835ce4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa835ce4000-7fa835de4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa835de4000-7fa835ee4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa835ee4000-7fa835fe4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa841ffc000-7fa841ffd000 ---p 00000000 00:00 0 
7fa841ffd000-7fa8427fd000 rw-p 00000000 00:00 0                          [stack:6334]
7fa8427fd000-7fa8427fe000 ---p 00000000 00:00 0 
7fa8427fe000-7fa842ffe000 rw-p 00000000 00:00 0 
7fa842ffe000-7fa842fff000 ---p 00000000 00:00 0 
7fa842fff000-7fa8437ff000 rw-p 00000000 00:00 0 
7fa8437ff000-7fa843800000 ---p 00000000 00:00 0 
7fa843800000-7fa844000000 rw-p 00000000 00:00 0 
7fa844000000-7fa84519e000 rw-p 00000000 00:00 0 
7fa84519e000-7fa848000000 ---p 00000000 00:00 0 
7fa8480f7000-7fa8481f7000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8481f7000-7fa8482f7000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8482f7000-7fa8483f7000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8487f8000-7fa8488f8000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8488f8000-7fa8489f8000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8489f8000-7fa848af8000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa848af8000-7fa848bf8000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa849bfa000-7fa849cfa000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa849cfa000-7fa849dfa000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa849dfa000-7fa849efa000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa849efa000-7fa849ffa000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa849ffa000-7fa84a0fa000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84a0fa000-7fa84a1fa000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84a1fa000-7fa84a7fc000 rw-p 00000000 00:00 0 
7fa84a7fd000-7fa84a8fd000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84a8fd000-7fa84a9fd000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84a9fd000-7fa84aafd000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84aafd000-7fa84adfe000 rw-p 00000000 00:00 0 
7fa84b619000-7fa84b719000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84b719000-7fa84b819000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84b819000-7fa84b919000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84b919000-7fa84ba19000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84ba73000-7fa84bb73000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84bb73000-7fa84bc73000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84bc73000-7fa84bc74000 ---p 00000000 00:00 0 
7fa84bc74000-7fa84c474000 rw-p 00000000 00:00 0                          [stack:3674]
7fa84c474000-7fa84c574000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84c574000-7fa84c674000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84c674000-7fa84c84e000 rw-p 00000000 00:00 0 
7fa84c84e000-7fa84c84f000 ---p 00000000 00:00 0 
7fa84c84f000-7fa84d04f000 rw-p 00000000 00:00 0                          [stack:3673]
7fa84d04f000-7fa84d14f000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84d17c000-7fa84d27c000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84d27c000-7fa85127c000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa85187d000-7fa85197d000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa85197d000-7fa851a7d000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa851a7d000-7fa851b7e000 rw-p 00000000 00:00 0 
7fa851b7e000-7fa851b7f000 ---p 00000000 00:00 0 
7fa851b7f000-7fa85237f000 rw-p 00000000 00:00 0                          [stack:3668]
7fa85237f000-7fa852380000 ---p 00000000 00:00 0 
7fa852380000-7fa852d01000 rw-p 00000000 00:00 0                          [stack:3664]
7fa852d01000-7fa852d06000 r-xp 00000000 fe:00 257374                     /usr/lib/x86_64-linux-gnu/sasl2/libcrammd5.so.2.0.25
7fa852d06000-7fa852f05000 ---p 00005000 fe:00 257374                     /usr/lib/x86_64-linux-gnu/sasl2/libcrammd5.so.2.0.25
7fa852f05000-7fa852f06000 r--p 00004000 fe:00 257374                     /usr/lib/x86_64-linux-gnu/sasl2/libcrammd5.so.2.0.25
7fa852f06000-7fa852f07000 rw-p 00005000 fe:00 257374                     /usr/lib/x86_64-linux-gnu/sasl2/libcrammd5.so.2.0.25
7fa852f07000-7fa853081000 r-xp 00000000 fe:00 85596                      /usr/lib/x86_64-linux-gnu/libdb-5.1.so
7fa853081000-7fa853281000 ---p 0017a000 fe:00 85596                      /usr/lib/x86_64-linux-gnu/libdb-5.1.so
7fa853281000-7fa853287000 r--p 0017a000 fe:00 85596                      /usr/lib/x86_64-linux-gnu/libdb-5.1.so
7fa853287000-7fa85328a000 rw-p 00180000 fe:00 85596                      /usr/lib/x86_64-linux-gnu/libdb-5.1.so
7fa85328a000-7fa853290000 r-xp 00000000 fe:00 257385                     /usr/lib/x86_64-linux-gnu/sasl2/libsasldb.so.2.0.25
7fa853290000-7fa85348f000 ---p 00006000 fe:00 257385                     /usr/lib/x86_64-linux-gnu/sasl2/libsasldb.so.2.0.25
7fa85348f000-7fa853490000 r--p 00005000 fe:00 257385                     /usr/lib/x86_64-linux-gnu/sasl2/libsasldb.so.2.0.25
7fa853490000-7fa853491000 rw-p 00006000 fe:00 257385                     /usr/lib/x86_64-linux-gnu/sasl2/libsasldb.so.2.0.25
7fa853491000-7fa853495000 r-xp 00000000 fe:00 257372                     /usr/lib/x86_64-linux-gnu/sasl2/libplain.so.2.0.25
7fa853495000-7fa853694000 ---p 00004000 fe:00 257372                     /usr/lib/x86_64-linux-gnu/sasl2/libplain.so.2.0.25
7fa853694000-7fa853695000 r--p 00003000 fe:00 257372                     /usr/lib/x86_64-linux-gnu/sasl2/libplain.so.2.0.25
7fa853695000-7fa853696000 rw-p 00004000 fe:00 257372                     /usr/lib/x86_64-linux-gnu/sasl2/libplain.so.2.0.25
7fa853696000-7fa8536a3000 r-xp 00000000 fe:00 257387                     /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so.2.0.25
7fa8536a3000-7fa8538a2000 ---p 0000d000 fe:00 257387                     /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so.2.0.25
7fa8538a2000-7fa8538a3000 r--p 0000c000 fe:00 257387                     /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so.2.0.25
7fa8538a3000-7fa8538a4000 rw-p 0000d000 fe:00 257387                     /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so.2.0.25
7fa8538a4000-7fa8538ac000 r-xp 00000000 fe:00 258635                     /lib/x86_64-linux-gnu/libcrypt-2.13.so
7fa8538ac000-7fa853aab000 ---p 00008000 fe:00 258635                     /lib/x86_64-linux-gnu/libcrypt-2.13.so
7fa853aab000-7fa853aac000 r--p 00007000 fe:00 258635                     /lib/x86_64-linux-gnu/libcrypt-2.13.so
7fa853aac000-7fa853aad000 rw-p 00008000 fe:00 258635                     /lib/x86_64-linux-gnu/libcrypt-2.13.so
7fa853aad000-7fa853adb000 rw-p 00000000 00:00 0 
7fa853adb000-7fa853adf000 r-xp 00000000 fe:00 257373                     /usr/lib/x86_64-linux-gnu/sasl2/liblogin.so.2.0.25
7fa853adf000-7fa853cde000 ---p 00004000 fe:00 257373                     /usr/lib/x86_64-linux-gnu/sasl2/liblogin.so.2.0.25
7fa853cde000-7fa853cdf000 r--p 00003000 fe:00 257373                     /usr/lib/x86_64-linux-gnu/sasl2/liblogin.so.2.0.25
7fa853cdf000-7fa853ce0000 rw-p 00004000 fe:00 257373                     /usr/lib/x86_64-linux-gnu/sasl2/liblogin.so.2.0.25
7fa853ce0000-7fa853ce4000 r-xp 00000000 fe:00 257378                     /usr/lib/x86_64-linux-gnu/sasl2/libanonymous.so.2.0.25
7fa853ce4000-7fa853ee3000 ---p 00004000 fe:00 257378                     /usr/lib/x86_64-linux-gnu/sasl2/libanonymous.so.2.0.25
7fa853ee3000-7fa853ee4000 r--p 00003000 fe:00 257378                     /usr/lib/x86_64-linux-gnu/sasl2/libanonymous.so.2.0.25
7fa853ee4000-7fa853ee5000 rw-p 00004000 fe:00 257378                     /usr/lib/x86_64-linux-gnu/sasl2/libanonymous.so.2.0.25
7fa853ee5000-7fa853eed000 r-xp 00000000 fe:00 257377                     /usr/lib/x86_64-linux-gnu/sasl2/libntlm.so.2.0.25
7fa853eed000-7fa8540ec000 ---p 00008000 fe:00 257377                     /usr/lib/x86_64-linux-gnu/sasl2/libntlm.so.2.0.25
7fa8540ec000-7fa8540ed000 r--p 00007000 fe:00 257377                     /usr/lib/x86_64-linux-gnu/sasl2/libntlm.so.2.0.25
7fa8540ed000-7fa8540ee000 rw-p 00008000 fe:00 257377                     /usr/lib/x86_64-linux-gnu/sasl2/libntlm.so.2.0.25
7fa8540ee000-7fa8540f4000 r-xp 00000000 fe:00 85642                      /usr/lib/x86_64-linux-gnu/libogg.so.0.8.0
7fa8540f4000-7fa8542f3000 ---p 00006000 fe:00 85642                      /usr/lib/x86_64-linux-gnu/libogg.so.0.8.0
7fa8542f3000-7fa8542f4000 rw-p 00005000 fe:00 85642                      /usr/lib/x86_64-linux-gnu/libogg.so.0.8.0
7fa8542f4000-7fa854320000 r-xp 00000000 fe:00 85239                      /usr/lib/x86_64-linux-gnu/libvorbis.so.0.4.5
7fa854320000-7fa85451f000 ---p 0002c000 fe:00 85239                      /usr/lib/x86_64-linux-gnu/libvorbis.so.0.4.5
7fa85451f000-7fa854520000 r--p 0002b000 fe:00 85239                      /usr/lib/x86_64-linux-gnu/libvorbis.so.0.4.5
7fa854520000-7fa854521000 rw-p 0002c000 fe:00 85239                      /usr/lib/x86_64-linux-gnu/libvorbis.so.0.4.5
7fa854521000-7fa8547d4000 r-xp 00000000 fe:00 85355                      /usr/lib/x86_64-linux-gnu/libvorbisenc.so.2.0.8
7fa8547d4000-7fa8549d3000 ---p 002b3000 fe:00 85355                      /usr/lib/x86_64-linux-gnu/libvorbisenc.so.2.0.8
7fa8549d3000-7fa8549ef000 r--p 002b2000 fe:00 85355                      /usr/lib/x86_64-linux-gnu/libvorbisenc.so.2.0.8
7fa8549ef000-7fa8549f0000 rw-p 002ce000 fe:00 85355                      /usr/lib/x86_64-linux-gnu/libvorbisenc.so.2.0.8
7fa8549f0000-7fa854a3c000 r-xp 00000000 fe:00 85555                      /usr/lib/x86_64-linux-gnu/libFLAC.so.8.2.0
7fa854a3c000-7fa854c3c000 ---p 0004c000 fe:00 85555                      /usr/lib/x86_64-linux-gnu/libFLAC.so.8.2.0
7fa854c3c000-7fa854c3d000 r--p 0004c000 fe:00 85555                      /usr/lib/x86_64-linux-gnu/libFLAC.so.8.2.0
7fa854c3d000-7fa854c3e000 rw-p 0004d000 fe:00 85555                      /usr/lib/x86_64-linux-gnu/libFLAC.so.8.2.0
7fa854c3e000-7fa854c53000 r-xp 00000000 fe:00 258289                     /lib/x86_64-linux-gnu/libnsl-2.13.so
7fa854c53000-7fa854e52000 ---p 00015000 fe:00 258289                     /lib/x86_64-linux-gnu/libnsl-2.13.so
7fa854e52000-7fa854e53000 r--p 00014000 fe:00 258289                     /lib/x86_64-linux-gnu/libnsl-2.13.so
7fa854e53000-7fa854e54000 rw-p 00015000 fe:00 258289                     /lib/x86_64-linux-gnu/libnsl-2.13.so
7fa854e54000-7fa854e56000 rw-p 00000000 00:00 0 
7fa854e56000-7fa854e64000 r-xp 00000000 fe:00 85131                      /usr/lib/x86_64-linux-gnu/libXi.so.6.1.0
7fa854e64000-7fa855064000 ---p 0000e000 fe:00 85131                      /usr/lib/x86_64-linux-gnu/libXi.so.6.1.0
7fa855064000-7fa855065000 rw-p 0000e000 fe:00 85131                      /usr/lib/x86_64-linux-gnu/libXi.so.6.1.0
7fa855065000-7fa855069000 r-xp 00000000 fe:00 269641                     /lib/x86_64-linux-gnu/libattr.so.1.1.0
7fa855069000-7fa855268000 ---p 00004000 fe:00 269641                     /lib/x86_64-linux-gnu/libattr.so.1.1.0
7fa855268000-7fa855269000 r--p 00003000 fe:00 269641                     /lib/x86_64-linux-gnu/libattr.so.1.1.0
7fa855269000-7fa85526a000 rw-p 00004000 fe:00 269641                     /lib/x86_64-linux-gnu/libattr.so.1.1.0
7fa85526a000-7fa85526f000 r-xp 00000000 fe:00 85661                      /usr/lib/x86_64-linux-gnu/libasyncns.so.0.3.1
7fa85526f000-7fa85546e000 ---p 00005000 fe:00 85661                      /usr/lib/x86_64-linux-gnu/libasyncns.so.0.3.1
7fa85546e000-7fa85546f000 rw-p 00004000 fe:00 85661                      /usr/lib/x86_64-linux-gnu/libasyncns.so.0.3.1
7fa85546f000-7fa8554d0000 r-xp 00000000 fe:00 85625                      /usr/lib/x86_64-linux-gnu/libsndfile.so.1.0.25
7fa8554d0000-7fa8556cf000 ---p 00061000 fe:00 85625                      /usr/lib/x86_64-linux-gnu/libsndfile.so.1.0.25
7fa8556cf000-7fa8556d1000 r--p 00060000 fe:00 85625                      /usr/lib/x86_64-linux-gnu/libsndfile.so.1.0.25
7fa8556d1000-7fa8556d2000 rw-p 00062000 fe:00 85625                      /usr/lib/x86_64-linux-gnu/libsndfile.so.1.0.25
7fa8556d2000-7fa8556d6000 rw-p 00000000 00:00 0 
7fa8556d6000-7fa8556df000 r-xp 00000000 fe:00 269460                     /lib/x86_64-linux-gnu/libwrap.so.0.7.6
7fa8556df000-7fa8558de000 ---p 00009000 fe:00 269460                     /lib/x86_64-linux-gnu/libwrap.so.0.7.6
7fa8558de000-7fa8558df000 r--p 00008000 fe:00 269460                     /lib/x86_64-linux-gnu/libwrap.so.0.7.6
7fa8558df000-7fa8558e0000 rw-p 00009000 fe:00 269460                     /lib/x86_64-linux-gnu/libwrap.so.0.7.6
7fa8558e0000-7fa8558e1000 rw-p 00000000 00:00 0 
7fa8558e1000-7fa8558e6000 r-xp 00000000 fe:00 85735                      /usr/lib/x86_64-linux-gnu/libXtst.so.6.1.0
7fa8558e6000-7fa855ae5000 ---p 00005000 fe:00 85735                      /usr/lib/x86_64-linux-gnu/libXtst.so.6.1.0
7fa855ae5000-7fa855ae6000 rw-p 00004000 fe:00 85735                      /usr/lib/x86_64-linux-gnu/libXtst.so.6.1.0
7fa855ae6000-7fa855aed000 r-xp 00000000 fe:00 85885                      /usr/lib/x86_64-linux-gnu/libSM.so.6.0.1
7fa855aed000-7fa855cec000 ---p 00007000 fe:00 85885                      /usr/lib/x86_64-linux-gnu/libSM.so.6.0.1
7fa855cec000-7fa855ced000 rw-p 00006000 fe:00 85885                      /usr/lib/x86_64-linux-gnu/libSM.so.6.0.1
7fa855ced000-7fa855d04000 r-xp 00000000 fe:00 85856                      /usr/lib/x86_64-linux-gnu/libICE.so.6.3.0
7fa855d04000-7fa855f03000 ---p 00017000 fe:00 85856                      /usr/lib/x86_64-linux-gnu/libICE.so.6.3.0
7fa855f03000-7fa855f05000 rw-p 00016000 fe:00 85856                      /usr/lib/x86_64-linux-gnu/libICE.so.6.3.0
7fa855f05000-7fa855f08000 rw-p 00000000 00:00 0 
7fa855f08000-7fa855f09000 r-xp 00000000 fe:00 2179                       /usr/lib/x86_64-linux-gnu/libX11-xcb.so.1.0.0
7fa855f09000-7fa856108000 ---p 00001000 fe:00 2179                       /usr/lib/x86_64-linux-gnu/libX11-xcb.so.1.0.0
7fa856108000-7fa856109000 rw-p 00000000 fe:00 2179                       /usr/lib/x86_64-linux-gnu/libX11-xcb.so.1.0.0
7fa856109000-7fa85610e000 r-xp 00000000 fe:00 85077                      /usr/lib/x86_64-linux-gnu/libXdmcp.so.6.0.0
7fa85610e000-7fa85630d000 ---p 00005000 fe:00 85077                      /usr/lib/x86_64-linux-gnu/libXdmcp.so.6.0.0
7fa85630d000-7fa85630e000 rw-p 00004000 fe:00 85077                      /usr/lib/x86_64-linux-gnu/libXdmcp.so.6.0.0
7fa85630e000-7fa856310000 r-xp 00000000 fe:00 85430                      /usr/lib/x86_64-linux-gnu/libXau.so.6.0.0
7fa856310000-7fa856510000 ---p 00002000 fe:00 85430                      /usr/lib/x86_64-linux-gnu/libXau.so.6.0.0
7fa856510000-7fa856511000 rw-p 00002000 fe:00 85430                      /usr/lib/x86_64-linux-gnu/libXau.so.6.0.0
7fa856511000-7fa856540000 r-xp 00000000 fe:00 269557                     /lib/x86_64-linux-gnu/libncursesw.so.5.9
7fa856540000-7fa85673f000 ---p 0002f000 fe:00 269557                     /lib/x86_64-linux-gnu/libncursesw.so.5.9
7fa85673f000-7fa856740000 r--p 0002e000 fe:00 269557                     /lib/x86_64-linux-gnu/libncursesw.so.5.9
7fa856740000-7fa856741000 rw-p 0002f000 fe:00 269557                     /lib/x86_64-linux-gnu/libncursesw.so.5.9
7fa856741000-7fa856856000 r-xp 00000000 fe:00 269619                     /lib/x86_64-linux-gnu/libslang.so.2.2.4
7fa856856000-7fa856a55000 ---p 00115000 fe:00 269619                     /lib/x86_64-linux-gnu/libslang.so.2.2.4
7fa856a55000-7fa856a59000 r--p 00114000 fe:00 269619                     /lib/x86_64-linux-gnu/libslang.so.2.2.4
7fa856a59000-7fa856a73000 rw-p 00118000 fe:00 269619                     /lib/x86_64-linux-gnu/libslang.so.2.2.4
7fa856a73000-7fa856ad7000 rw-p 00000000 00:00 0 
7fa856ad7000-7fa856b1c000 r-xp 00000000 fe:00 269510                     /lib/x86_64-linux-gnu/libdbus-1.so.3.7.2
7fa856b1c000-7fa856d1b000 ---p 00045000 fe:00 269510                     /lib/x86_64-linux-gnu/libdbus-1.so.3.7.2
7fa856d1b000-7fa856d1c000 r--p 00044000 fe:00 269510                     /lib/x86_64-linux-gnu/libdbus-1.so.3.7.2
7fa856d1c000-7fa856d1d000 rw-p 00045000 fe:00 269510                     /lib/x86_64-linux-gnu/libdbus-1.so.3.7.2
7fa856d1d000-7fa856d26000 r-xp 00000000 fe:00 269538                     /lib/x86_64-linux-gnu/libjson.so.0.1.0
7fa856d26000-7fa856f25000 ---p 00009000 fe:00 269538                     /lib/x86_64-linux-gnu/libjson.so.0.1.0
7fa856f25000-7fa856f26000 r--p 00008000 fe:00 269538                     /lib/x86_64-linux-gnu/libjson.so.0.1.0
7fa856f26000-7fa856f27000 rw-p 00009000 fe:00 269538                     /lib/x86_64-linux-gnu/libjson.so.0.1.0
7fa856f27000-7fa856f2b000 r-xp 00000000 fe:00 269503                     /lib/x86_64-linux-gnu/libcap.so.2.22
7fa856f2b000-7fa85712a000 ---p 00004000 fe:00 269503                     /lib/x86_64-linux-gnu/libcap.so.2.22
7fa85712a000-7fa85712b000 rw-p 00003000 fe:00 269503                     /lib/x86_64-linux-gnu/libcap.so.2.22
7fa85712b000-7fa85718c000 r-xp 00000000 fe:00 85203                      /usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-2.0.so
7fa85718c000-7fa85738b000 ---p 00061000 fe:00 85203                      /usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-2.0.so
7fa85738b000-7fa85738c000 r--p 00060000 fe:00 85203                      /usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-2.0.so
7fa85738c000-7fa85738e000 rw-p 00061000 fe:00 85203                      /usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-2.0.so
7fa85738e000-7fa857391000 r-xp 00000000 fe:00 269620                     /lib/x86_64-linux-gnu/libkeyutils.so.1.4
7fa857391000-7fa857590000 ---p 00003000 fe:00 269620                     /lib/x86_64-linux-gnu/libkeyutils.so.1.4
7fa857590000-7fa857591000 r--p 00002000 fe:00 269620                     /lib/x86_64-linux-gnu/libkeyutils.so.1.4
7fa857591000-7fa857592000 rw-p 00003000 fe:00 269620                     /lib/x86_64-linux-gnu/libkeyutils.so.1.4
7fa857592000-7fa85759a000 r-xp 00000000 fe:00 8184                       /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1
7fa85759a000-7fa857799000 ---p 00008000 fe:00 8184                       /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1
7fa857799000-7fa85779a000 r--p 00007000 fe:00 8184                       /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1
7fa85779a000-7fa85779b000 rw-p 00008000 fe:00 8184                       /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1
7fa85779b000-7fa85779e000 r-xp 00000000 fe:00 275449                     /lib/x86_64-linux-gnu/libcom_err.so.2.1
7fa85779e000-7fa85799d000 ---p 00003000 fe:00 275449                     /lib/x86_64-linux-gnu/libcom_err.so.2.1
7fa85799d000-7fa85799e000 r--p 00002000 fe:00 275449                     /lib/x86_64-linux-gnu/libcom_err.so.2.1
7fa85799e000-7fa85799f000 rw-p 00003000 fe:00 275449                     /lib/x86_64-linux-gnu/libcom_err.so.2.1
7fa85799f000-7fa8579c5000 r-xp 00000000 fe:00 20322                      /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1
7fa8579c5000-7fa857bc5000 ---p 00026000 fe:00 20322                      /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1
7fa857bc5000-7fa857bc6000 r--p 00026000 fe:00 20322                      /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1
7fa857bc6000-7fa857bc7000 rw-p 00027000 fe:00 20322                      /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1
7fa857bc7000-7fa857bc8000 rw-p 00000000 00:00 0 
7fa857bc8000-7fa857c91000 r-xp 00000000 fe:00 20391                      /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3
7fa857c91000-7fa857e90000 ---p 000c9000 fe:00 20391                      /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3
7fa857e90000-7fa857e9a000 r--p 000c8000 fe:00 20391                      /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3
7fa857e9a000-7fa857e9c000 rw-p 000d2000 fe:00 20391                      /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3
7fa857e9c000-7fa857ea9000 r-xp 00000000 fe:00 269631                     /lib/x86_64-linux-gnu/libudev.so.0.13.0
7fa857ea9000-7fa8580a9000 ---p 0000d000 fe:00 269631                     /lib/x86_64-linux-gnu/libudev.so.0.13.0
7fa8580a9000-7fa8580aa000 r--p 0000d000 fe:00 269631                     /lib/x86_64-linux-gnu/libudev.so.0.13.0
7fa8580aa000-7fa8580ab000 rw-p 0000e000 fe:00 269631                     /lib/x86_64-linux-gnu/libudev.so.0.13.0
7fa8580ab000-7fa8580b2000 r-xp 00000000 fe:00 85685                      /usr/lib/x86_64-linux-gnu/liblz4.so.1.3.0
7fa8580b2000-7fa8582b1000 ---p 00007000 fe:00 85685                      /usr/lib/x86_64-linux-gnu/liblz4.so.1.3.0
7fa8582b1000-7fa8582b2000 r--p 00006000 fe:00 85685                      /usr/lib/x86_64-linux-gnu/liblz4.so.1.3.0
7fa8582b2000-7fa8582b3000 rw-p 00007000 fe:00 85685                      /usr/lib/x86_64-linux-gnu/liblz4.so.1.3.0
7fa8582b3000-7fa8582fb000 r-xp 00000000 fe:00 85743                      /usr/lib/x86_64-linux-gnu/libopus.so.0.5.0
7fa8582fb000-7fa8584fb000 ---p 00048000 fe:00 85743                      /usr/lib/x86_64-linux-gnu/libopus.so.0.5.0
7fa8584fb000-7fa8584fc000 r--p 00048000 fe:00 85743                      /usr/lib/x86_64-linux-gnu/libopus.so.0.5.0
7fa8584fc000-7fa8584fd000 rw-p 00049000 fe:00 85743                      /usr/lib/x86_64-linux-gnu/libopus.so.0.5.0
7fa8584fd000-7fa85851f000 r-xp 00000000 fe:00 269545                     /lib/x86_64-linux-gnu/liblzma.so.5.0.0
7fa85851f000-7fa85871e000 ---p 00022000 fe:00 269545                     /lib/x86_64-linux-gnu/liblzma.so.5.0.0
7fa85871e000-7fa85871f000 r--p 00021000 fe:00 269545                     /lib/x86_64-linux-gnu/liblzma.so.5.0.0
7fa85871f000-7fa858720000 rw-p 00022000 fe:00 269545                     /lib/x86_64-linux-gnu/liblzma.so.5.0.0
7fa858720000-7fa85872f000 r-xp 00000000 fe:00 269555                     /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7fa85872f000-7fa85892e000 ---p 0000f000 fe:00 269555                     /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7fa85892e000-7fa85892f000 r--p 0000e000 fe:00 269555                     /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7fa85892f000-7fa858930000 rw-p 0000f000 fe:00 269555                     /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7fa858930000-7fa85894f000 r-xp 00000000 fe:00 85679                      /usr/lib/x86_64-linux-gnu/libxcb.so.1.1.0
7fa85894f000-7fa858b4e000 ---p 0001f000 fe:00 85679                      /usr/lib/x86_64-linux-gnu/libxcb.so.1.1.0
7fa858b4e000-7fa858b4f000 r--p 0001e000 fe:00 85679                      /usr/lib/x86_64-linux-gnu/libxcb.so.1.1.0
7fa858b4f000-7fa858b50000 rw-p 0001f000 fe:00 85679                      /usr/lib/x86_64-linux-gnu/libxcb.so.1.1.0
7fa858b50000-7fa858b52000 r-xp 00000000 fe:00 85904                      /usr/lib/x86_64-linux-gnu/libts-0.0.so.0.1.1
7fa858b52000-7fa858d52000 ---p 00002000 fe:00 85904                      /usr/lib/x86_64-linux-gnu/libts-0.0.so.0.1.1
7fa858d52000-7fa858d53000 rw-p 00002000 fe:00 85904                      /usr/lib/x86_64-linux-gnu/libts-0.0.so.0.1.1
7fa858d53000-7fa858d6a000 r-xp 00000000 fe:00 85649                      /usr/lib/x86_64-linux-gnu/libdirect-1.2.so.9.0.1
7fa858d6a000-7fa858f69000 ---p 00017000 fe:00 85649                      /usr/lib/x86_64-linux-gnu/libdirect-1.2.so.9.0.1
7fa858f69000-7fa858f6a000 rw-p 00016000 fe:00 85649                      /usr/lib/x86_64-linux-gnu/libdirect-1.2.so.9.0.1
7fa858f6a000-7fa858f6b000 rw-p 00000000 00:00 0 
7fa858f6b000-7fa858f74000 r-xp 00000000 fe:00 85270                      /usr/lib/x86_64-linux-gnu/libfusion-1.2.so.9.0.1
7fa858f74000-7fa859174000 ---p 00009000 fe:00 85270                      /usr/lib/x86_64-linux-gnu/libfusion-1.2.so.9.0.1
7fa859174000-7fa859175000 rw-p 00009000 fe:00 85270                      /usr/lib/x86_64-linux-gnu/libfusion-1.2.so.9.0.1
7fa859175000-7fa8591f6000 r-xp 00000000 fe:00 85874                      /usr/lib/x86_64-linux-gnu/libdirectfb-1.2.so.9.0.1
7fa8591f6000-7fa8593f6000 ---p 00081000 fe:00 85874                      /usr/lib/x86_64-linux-gnu/libdirectfb-1.2.so.9.0.1
7fa8593f6000-7fa8593fa000 rw-p 00081000 fe:00 85874                      /usr/lib/x86_64-linux-gnu/libdirectfb-1.2.so.9.0.1
7fa8593fa000-7fa85940b000 r-xp 00000000 fe:00 85563                      /usr/lib/x86_64-linux-gnu/libXext.so.6.4.0
7fa85940b000-7fa85960b000 ---p 00011000 fe:00 85563                      /usr/lib/x86_64-linux-gnu/libXext.so.6.4.0
7fa85960b000-7fa85960c000 rw-p 00011000 fe:00 85563                      /usr/lib/x86_64-linux-gnu/libXext.so.6.4.0
7fa85960c000-7fa859655000 r-xp 00000000 fe:00 85864                      /usr/lib/x86_64-linux-gnu/libpulse.so.0.14.2
7fa859655000-7fa859854000 ---p 00049000 fe:00 85864                      /usr/lib/x86_64-linux-gnu/libpulse.so.0.14.2
7fa859854000-7fa859855000 r--p 00048000 fe:00 85864                      /usr/lib/x86_64-linux-gnu/libpulse.so.0.14.2
7fa859855000-7fa859856000 rw-p 00049000 fe:00 85864                      /usr/lib/x86_64-linux-gnu/libpulse.so.0.14.2
7fa859856000-7fa859859000 r-xp 00000000 fe:00 85149                      /usr/lib/x86_64-linux-gnu/libpulse-simple.so.0.0.3
7fa859859000-7fa859a59000 ---p 00003000 fe:00 85149                      /usr/lib/x86_64-linux-gnu/libpulse-simple.so.0.0.3
7fa859a59000-7fa859a5a000 r--p 00003000 fe:00 85149                      /usr/lib/x86_64-linux-gnu/libpulse-simple.so.0.0.3
7fa859a5a000-7fa859a5b000 rw-p 00004000 fe:00 85149                      /usr/lib/x86_64-linux-gnu/libpulse-simple.so.0.0.3
7fa859a5b000-7fa859b47000 r-xp 00000000 fe:00 85458                      /usr/lib/x86_64-linux-gnu/libasound.so.2.0.0
7fa859b47000-7fa859d47000 ---p 000ec000 fe:00 85458                      /usr/lib/x86_64-linux-gnu/libasound.so.2.0.0
7fa859d47000-7fa859d4d000 r--p 000ec000 fe:00 85458                      /usr/lib/x86_64-linux-gnu/libasound.so.2.0.0
7fa859d4d000-7fa859d4f000 rw-p 000f2000 fe:00 85458                      /usr/lib/x86_64-linux-gnu/libasound.so.2.0.0
7fa859d4f000-7fa859d60000 r-xp 00000000 fe:00 85796                      /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.0.0
7fa859d60000-7fa859f5f000 ---p 00011000 fe:00 85796                      /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.0.0
7fa859f5f000-7fa859f60000 r--p 00010000 fe:00 85796                      /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.0.0
7fa859f60000-7fa859f61000 rw-p 00011000 fe:00 85796                      /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.0.0
7fa859f61000-7fa859f71000 r-xp 00000000 fe:00 19965                      /usr/lib/x86_64-linux-gnu/libtasn1.so.3.1.16
7fa859f71000-7fa85a170000 ---p 00010000 fe:00 19965                      /usr/lib/x86_64-linux-gnu/libtasn1.so.3.1.16
7fa85a170000-7fa85a171000 r--p 0000f000 fe:00 19965                      /usr/lib/x86_64-linux-gnu/libtasn1.so.3.1.16
7fa85a171000-7fa85a172000 rw-p 00010000 fe:00 19965                      /usr/lib/x86_64-linux-gnu/libtasn1.so.3.1.16
7fa85a172000-7fa85a185000 r-xp 00000000 fe:00 258649                     /lib/x86_64-linux-gnu/libresolv-2.13.so
7fa85a185000-7fa85a384000 ---p 00013000 fe:00 258649                     /lib/x86_64-linux-gnu/libresolv-2.13.so
7fa85a384000-7fa85a385000 r--p 00012000 fe:00 258649                     /lib/x86_64-linux-gnu/libresolv-2.13.so
7fa85a385000-7fa85a386000 rw-p 00013000 fe:00 258649                     /lib/x86_64-linux-gnu/libresolv-2.13.so
7fa85a386000-7fa85a388000 rw-p 00000000 00:00 0 
7fa85a388000-7fa85a38a000 r-xp 00000000 fe:00 258633                     /lib/x86_64-linux-gnu/libdl-2.13.so
7fa85a38a000-7fa85a58a000 ---p 00002000 fe:00 258633                     /lib/x86_64-linux-gnu/libdl-2.13.so
7fa85a58a000-7fa85a58b000 r--p 00002000 fe:00 258633                     /lib/x86_64-linux-gnu/libdl-2.13.so
7fa85a58b000-7fa85a58c000 rw-p 00003000 fe:00 258633                     /lib/x86_64-linux-gnu/libdl-2.13.so
7fa85a58c000-7fa85a5c8000 r-xp 00000000 fe:00 269550                     /lib/x86_64-linux-gnu/libpcre.so.3.13.1
7fa85a5c8000-7fa85a7c8000 ---p 0003c000 fe:00 269550                     /lib/x86_64-linux-gnu/libpcre.so.3.13.1
7fa85a7c8000-7fa85a7c9000 rw-p 0003c000 fe:00 269550                     /lib/x86_64-linux-gnu/libpcre.so.3.13.1
7fa85a7c9000-7fa85a7cc000 r-xp 00000000 fe:00 269443                     /lib/x86_64-linux-gnu/libgpg-error.so.0.8.0
7fa85a7cc000-7fa85a9cb000 ---p 00003000 fe:00 269443                     /lib/x86_64-linux-gnu/libgpg-error.so.0.8.0
7fa85a9cb000-7fa85a9cc000 rw-p 00002000 fe:00 269443                     /lib/x86_64-linux-gnu/libgpg-error.so.0.8.0
7fa85a9cc000-7fa85a9e5000 r-xp 00000000 fe:00 85932                      /usr/lib/x86_64-linux-gnu/librtmp.so.0
7fa85a9e5000-7fa85abe5000 ---p 00019000 fe:00 85932                      /usr/lib/x86_64-linux-gnu/librtmp.so.0
7fa85abe5000-7fa85abe6000 rw-p 00019000 fe:00 85932                      /usr/lib/x86_64-linux-gnu/librtmp.so.0
7fa85abe6000-7fa85adb0000 r-xp 00000000 fe:00 7494                       /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7fa85adb0000-7fa85afb0000 ---p 001ca000 fe:00 7494                       /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7fa85afb0000-7fa85afcb000 r--p 001ca000 fe:00 7494                       /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7fa85afcb000-7fa85afda000 rw-p 001e5000 fe:00 7494                       /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7fa85afda000-7fa85afde000 rw-p 00000000 00:00 0 
7fa85afde000-7fa85b034000 r-xp 00000000 fe:00 7511                       /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
7fa85b034000-7fa85b234000 ---p 00056000 fe:00 7511                       /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
7fa85b234000-7fa85b237000 r--p 00056000 fe:00 7511                       /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
7fa85b237000-7fa85b23e000 rw-p 00059000 fe:00 7511                       /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
7fa85b23e000-7fa85b27a000 r-xp 00000000 fe:00 20354                      /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2
7fa85b27a000-7fa85b47a000 ---p 0003c000 fe:00 20354                      /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2
7fa85b47a000-7fa85b47b000 r--p 0003c000 fe:00 20354                      /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2
7fa85b47b000-7fa85b47d000 rw-p 0003d000 fe:00 20354                      /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2
7fa85b47d000-7fa85b4c9000 r-xp 00000000 fe:00 19738                      /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.8.3
7fa85b4c9000-7fa85b6c9000 ---p 0004c000 fe:00 19738                      /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.8.3
7fa85b6c9000-7fa85b6cb000 r--p 0004c000 fe:00 19738                      /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.8.3
7fa85b6cb000-7fa85b6cc000 rw-p 0004e000 fe:00 19738                      /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.8.3
7fa85b6cc000-7fa85b6ce000 rw-p 00000000 00:00 0 
7fa85b6ce000-7fa85b6dc000 r-xp 00000000 fe:00 19723                      /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.8.3
7fa85b6dc000-7fa85b8db000 ---p 0000e000 fe:00 19723                      /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.8.3
7fa85b8db000-7fa85b8dc000 r--p 0000d000 fe:00 19723                      /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.8.3
7fa85b8dc000-7fa85b8dd000 rw-p 0000e000 fe:00 19723                      /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.8.3
7fa85b8dd000-7fa85b90f000 r-xp 00000000 fe:00 85264                      /usr/lib/x86_64-linux-gnu/libidn.so.11.6.8
7fa85b90f000-7fa85bb0f000 ---p 00032000 fe:00 85264                      /usr/lib/x86_64-linux-gnu/libidn.so.11.6.8
7fa85bb0f000-7fa85bb10000 r--p 00032000 fe:00 85264                      /usr/lib/x86_64-linux-gnu/libidn.so.11.6.8
7fa85bb10000-7fa85bb11000 rw-p 00033000 fe:00 85264                      /usr/lib/x86_64-linux-gnu/libidn.so.11.6.8
7fa85bb11000-7fa85bc92000 r-xp 00000000 fe:00 258617                     /lib/x86_64-linux-gnu/libc-2.13.so
7fa85bc92000-7fa85be92000 ---p 00181000 fe:00 258617                     /lib/x86_64-linux-gnu/libc-2.13.so
7fa85be92000-7fa85be96000 r--p 00181000 fe:00 258617                     /lib/x86_64-linux-gnu/libc-2.13.so
7fa85be96000-7fa85be97000 rw-p 00185000 fe:00 258617                     /lib/x86_64-linux-gnu/libc-2.13.so
7fa85be97000-7fa85be9c000 rw-p 00000000 00:00 0 
7fa85be9c000-7fa85beb3000 r-xp 00000000 fe:00 258653                     /lib/x86_64-linux-gnu/libpthread-2.13.so
7fa85beb3000-7fa85c0b2000 ---p 00017000 fe:00 258653                     /lib/x86_64-linux-gnu/libpthread-2.13.so
7fa85c0b2000-7fa85c0b3000 r--p 00016000 fe:00 258653                     /lib/x86_64-linux-gnu/libpthread-2.13.so
7fa85c0b3000-7fa85c0b4000 rw-p 00017000 fe:00 258653                     /lib/x86_64-linux-gnu/libpthread-2.13.so
7fa85c0b4000-7fa85c0b8000 rw-p 00000000 00:00 0 
7fa85c0b8000-7fa85c0cd000 r-xp 00000000 fe:00 269446                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7fa85c0cd000-7fa85c2cd000 ---p 00015000 fe:00 269446                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7fa85c2cd000-7fa85c2ce000 rw-p 00015000 fe:00 269446                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7fa85c2ce000-7fa85c34f000 r-xp 00000000 fe:00 258285                     /lib/x86_64-linux-gnu/libm-2.13.so
7fa85c34f000-7fa85c54e000 ---p 00081000 fe:00 258285                     /lib/x86_64-linux-gnu/libm-2.13.so
7fa85c54e000-7fa85c54f000 r--p 00080000 fe:00 258285                     /lib/x86_64-linux-gnu/libm-2.13.so
7fa85c54f000-7fa85c550000 rw-p 00081000 fe:00 258285                     /lib/x86_64-linux-gnu/libm-2.13.so
7fa85c550000-7fa85c638000 r-xp 00000000 fe:00 85315                      /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17
7fa85c638000-7fa85c838000 ---p 000e8000 fe:00 85315                      /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17
7fa85c838000-7fa85c840000 r--p 000e8000 fe:00 85315                      /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17
7fa85c840000-7fa85c842000 rw-p 000f0000 fe:00 85315                      /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17
7fa85c842000-7fa85c857000 rw-p 00000000 00:00 0 
7fa85c857000-7fa85c8d7000 r-xp 00000000 fe:00 85381                      /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.26.0
7fa85c8d7000-7fa85cad7000 ---p 00080000 fe:00 85381                      /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.26.0
7fa85cad7000-7fa85cadd000 rw-p 00080000 fe:00 85381                      /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.26.0
7fa85cadd000-7fa85cae4000 r-xp 00000000 fe:00 85294                      /usr/lib/x86_64-linux-gnu/libusbredirparser.so.1.0.0
7fa85cae4000-7fa85cce3000 ---p 00007000 fe:00 85294                      /usr/lib/x86_64-linux-gnu/libusbredirparser.so.1.0.0
7fa85cce3000-7fa85cce4000 r--p 00006000 fe:00 85294                      /usr/lib/x86_64-linux-gnu/libusbredirparser.so.1.0.0
7fa85cce4000-7fa85cce5000 rw-p 00007000 fe:00 85294                      /usr/lib/x86_64-linux-gnu/libusbredirparser.so.1.0.0
7fa85cce5000-7fa85ccfb000 r-xp 00000000 fe:00 269567                     /lib/x86_64-linux-gnu/libusb-1.0.so.0.1.0
7fa85ccfb000-7fa85cefb000 ---p 00016000 fe:00 269567                     /lib/x86_64-linux-gnu/libusb-1.0.so.0.1.0
7fa85cefb000-7fa85cefc000 r--p 00016000 fe:00 269567                     /lib/x86_64-linux-gnu/libusb-1.0.so.0.1.0
7fa85cefc000-7fa85cefd000 rw-p 00017000 fe:00 269567                     /lib/x86_64-linux-gnu/libusb-1.0.so.0.1.0
7fa85cefd000-7fa85d01b000 r-xp 00000000 fe:00 11054                      /usr/lib/x86_64-linux-gnu/libspice-server.so.1.9.0
7fa85d01b000-7fa85d21a000 ---p 0011e000 fe:00 11054                      /usr/lib/x86_64-linux-gnu/libspice-server.so.1.9.0
7fa85d21a000-7fa85d21c000 r--p 0011d000 fe:00 11054                      /usr/lib/x86_64-linux-gnu/libspice-server.so.1.9.0
7fa85d21c000-7fa85d21d000 rw-p 0011f000 fe:00 11054                      /usr/lib/x86_64-linux-gnu/libspice-server.so.1.9.0
7fa85d21d000-7fa85d22b000 rw-p 00000000 00:00 0 
7fa85d22b000-7fa85d258000 r-xp 00000000 fe:00 20863                      /usr/lib/libxenguest.so.4.5.0
7fa85d258000-7fa85d457000 ---p 0002d000 fe:00 20863                      /usr/lib/libxenguest.so.4.5.0
7fa85d457000-7fa85d458000 rw-p 0002c000 fe:00 20863                      /usr/lib/libxenguest.so.4.5.0
7fa85d458000-7fa85d481000 r-xp 00000000 fe:00 20840                      /usr/lib/libxenctrl.so.4.5.0
7fa85d481000-7fa85d681000 ---p 00029000 fe:00 20840                      /usr/lib/libxenctrl.so.4.5.0
7fa85d681000-7fa85d682000 rw-p 00029000 fe:00 20840                      /usr/lib/libxenctrl.so.4.5.0
7fa85d682000-7fa85d688000 r-xp 00000000 fe:00 20888                      /usr/lib/libxenstore.so.3.0.3
7fa85d688000-7fa85d887000 ---p 00006000 fe:00 20888                      /usr/lib/libxenstore.so.3.0.3
7fa85d887000-7fa85d888000 rw-p 00005000 fe:00 20888                      /usr/lib/libxenstore.so.3.0.3
7fa85d888000-7fa85d88b000 rw-p 00000000 00:00 0 
7fa85d88b000-7fa85d9c0000 r-xp 00000000 fe:00 2377                       /usr/lib/x86_64-linux-gnu/libX11.so.6.3.0
7fa85d9c0000-7fa85dbc0000 ---p 00135000 fe:00 2377                       /usr/lib/x86_64-linux-gnu/libX11.so.6.3.0
7fa85dbc0000-7fa85dbc6000 rw-p 00135000 fe:00 2377                       /usr/lib/x86_64-linux-gnu/libX11.so.6.3.0
7fa85dbc6000-7fa85dc32000 r-xp 00000000 fe:00 85332                      /usr/lib/x86_64-linux-gnu/libSDL-1.2.so.0.11.4
7fa85dc32000-7fa85de32000 ---p 0006c000 fe:00 85332                      /usr/lib/x86_64-linux-gnu/libSDL-1.2.so.0.11.4
7fa85de32000-7fa85de33000 r--p 0006c000 fe:00 85332                      /usr/lib/x86_64-linux-gnu/libSDL-1.2.so.0.11.4
7fa85de33000-7fa85de35000 rw-p 0006d000 fe:00 85332                      /usr/lib/x86_64-linux-gnu/libSDL-1.2.so.0.11.4
7fa85de35000-7fa85de62000 rw-p 00000000 00:00 0 
7fa85de62000-7fa85df1b000 r-xp 00000000 fe:00 11464                      /usr/lib/x86_64-linux-gnu/libgnutls.so.26.22.4
7fa85df1b000-7fa85e11a000 ---p 000b9000 fe:00 11464                      /usr/lib/x86_64-linux-gnu/libgnutls.so.26.22.4
7fa85e11a000-7fa85e120000 r--p 000b8000 fe:00 11464                      /usr/lib/x86_64-linux-gnu/libgnutls.so.26.22.4
7fa85e120000-7fa85e122000 rw-p 000be000 fe:00 11464                      /usr/lib/x86_64-linux-gnu/libgnutls.so.26.22.4
7fa85e122000-7fa85e13c000 r-xp 00000000 fe:00 85121                      /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25
7fa85e13c000-7fa85e33b000 ---p 0001a000 fe:00 85121                      /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25
7fa85e33b000-7fa85e33c000 r--p 00019000 fe:00 85121                      /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25
7fa85e33c000-7fa85e33d000 rw-p 0001a000 fe:00 85121                      /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25
7fa85e33d000-7fa85e381000 r-xp 00000000 fe:00 85726                      /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2
7fa85e381000-7fa85e580000 ---p 00044000 fe:00 85726                      /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2
7fa85e580000-7fa85e581000 r--p 00043000 fe:00 85726                      /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2
7fa85e581000-7fa85e582000 rw-p 00044000 fe:00 85726                      /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2
7fa85e582000-7fa85e592000 rw-p 00000000 00:00 0 
7fa85e592000-7fa85e5b8000 r-xp 00000000 fe:00 269455                     /lib/x86_64-linux-gnu/libpng12.so.0.49.0
7fa85e5b8000-7fa85e7b7000 ---p 00026000 fe:00 269455                     /lib/x86_64-linux-gnu/libpng12.so.0.49.0
7fa85e7b7000-7fa85e7b8000 r--p 00025000 fe:00 269455                     /lib/x86_64-linux-gnu/libpng12.so.0.49.0
7fa85e7b8000-7fa85e7b9000 rw-p 00026000 fe:00 269455                     /lib/x86_64-linux-gnu/libpng12.so.0.49.0
7fa85e7b9000-7fa85e7bd000 r-xp 00000000 fe:00 269543                     /lib/x86_64-linux-gnu/libuuid.so.1.3.0
7fa85e7bd000-7fa85e9bc000 ---p 00004000 fe:00 269543                     /lib/x86_64-linux-gnu/libuuid.so.1.3.0
7fa85e9bc000-7fa85e9bd000 r--p 00003000 fe:00 269543                     /lib/x86_64-linux-gnu/libuuid.so.1.3.0
7fa85e9bd000-7fa85e9be000 rw-p 00004000 fe:00 269543                     /lib/x86_64-linux-gnu/libuuid.so.1.3.0
7fa85e9be000-7fa85e9e3000 r-xp 00000000 fe:00 269539                     /lib/x86_64-linux-gnu/libtinfo.so.5.9
7fa85e9e3000-7fa85ebe2000 ---p 00025000 fe:00 269539                     /lib/x86_64-linux-gnu/libtinfo.so.5.9
7fa85ebe2000-7fa85ebe6000 r--p 00024000 fe:00 269539                     /lib/x86_64-linux-gnu/libtinfo.so.5.9
7fa85ebe6000-7fa85ebe7000 rw-p 00028000 fe:00 269539                     /lib/x86_64-linux-gnu/libtinfo.so.5.9
7fa85ebe7000-7fa85ec08000 r-xp 00000000 fe:00 269458                     /lib/x86_64-linux-gnu/libncurses.so.5.9
7fa85ec08000-7fa85ee07000 ---p 00021000 fe:00 269458                     /lib/x86_64-linux-gnu/libncurses.so.5.9
7fa85ee07000-7fa85ee08000 r--p 00020000 fe:00 269458                     /lib/x86_64-linux-gnu/libncurses.so.5.9
7fa85ee08000-7fa85ee09000 rw-p 00021000 fe:00 269458                     /lib/x86_64-linux-gnu/libncurses.so.5.9
7fa85ee09000-7fa85ee0b000 r-xp 00000000 fe:00 258625                     /lib/x86_64-linux-gnu/libutil-2.13.so
7fa85ee0b000-7fa85f00a000 ---p 00002000 fe:00 258625                     /lib/x86_64-linux-gnu/libutil-2.13.so
7fa85f00a000-7fa85f00b000 r--p 00001000 fe:00 258625                     /lib/x86_64-linux-gnu/libutil-2.13.so
7fa85f00b000-7fa85f00c000 rw-p 00002000 fe:00 258625                     /lib/x86_64-linux-gnu/libutil-2.13.so
7fa85f00c000-7fa85f101000 r-xp 00000000 fe:00 269472                     /lib/x86_64-linux-gnu/libglib-2.0.so.0.3200.4
7fa85f101000-7fa85f301000 ---p 000f5000 fe:00 269472                     /lib/x86_64-linux-gnu/libglib-2.0.so.0.3200.4
7fa85f301000-7fa85f302000 r--p 000f5000 fe:00 269472                     /lib/x86_64-linux-gnu/libglib-2.0.so.0.3200.4
7fa85f302000-7fa85f303000 rw-p 000f6000 fe:00 269472                     /lib/x86_64-linux-gnu/libglib-2.0.so.0.3200.4
7fa85f303000-7fa85f304000 rw-p 00000000 00:00 0 
7fa85f304000-7fa85f305000 r-xp 00000000 fe:00 85942                      /usr/lib/x86_64-linux-gnu/libgthread-2.0.so.0.3200.4
7fa85f305000-7fa85f504000 ---p 00001000 fe:00 85942                      /usr/lib/x86_64-linux-gnu/libgthread-2.0.so.0.3200.4
7fa85f504000-7fa85f505000 r--p 00000000 fe:00 85942                      /usr/lib/x86_64-linux-gnu/libgthread-2.0.so.0.3200.4
7fa85f505000-7fa85f506000 rw-p 00001000 fe:00 85942                      /usr/lib/x86_64-linux-gnu/libgthread-2.0.so.0.3200.4
7fa85f506000-7fa85f50d000 r-xp 00000000 fe:00 258652                     /lib/x86_64-linux-gnu/librt-2.13.so
7fa85f50d000-7fa85f70c000 ---p 00007000 fe:00 258652                     /lib/x86_64-linux-gnu/librt-2.13.so
7fa85f70c000-7fa85f70d000 r--p 00006000 fe:00 258652                     /lib/x86_64-linux-gnu/librt-2.13.so
7fa85f70d000-7fa85f70e000 rw-p 00007000 fe:00 258652                     /lib/x86_64-linux-gnu/librt-2.13.so
7fa85f70e000-7fa85f724000 r-xp 00000000 fe:00 269535                     /lib/x86_64-linux-gnu/libz.so.1.2.7
7fa85f724000-7fa85f923000 ---p 00016000 fe:00 269535                     /lib/x86_64-linux-gnu/libz.so.1.2.7
7fa85f923000-7fa85f924000 r--p 00015000 fe:00 269535                     /lib/x86_64-linux-gnu/libz.so.1.2.7
7fa85f924000-7fa85f925000 rw-p 00016000 fe:00 269535                     /lib/x86_64-linux-gnu/libz.so.1.2.7
7fa85f925000-7fa85f94c000 r-xp 00000000 fe:00 1795                       /usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1
7fa85f94c000-7fa85fb4c000 ---p 00027000 fe:00 1795                       /usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1
7fa85fb4c000-7fa85fb4d000 r--p 00027000 fe:00 1795                       /usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1
7fa85fb4d000-7fa85fb4e000 rw-p 00028000 fe:00 1795                       /usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1
7fa85fb4e000-7fa85fbc9000 r-xp 00000000 fe:00 269893                     /lib/x86_64-linux-gnu/libgcrypt.so.11.7.0
7fa85fbc9000-7fa85fdc9000 ---p 0007b000 fe:00 269893                     /lib/x86_64-linux-gnu/libgcrypt.so.11.7.0
7fa85fdc9000-7fa85fdca000 r--p 0007b000 fe:00 269893                     /lib/x86_64-linux-gnu/libgcrypt.so.11.7.0
7fa85fdca000-7fa85fdcd000 rw-p 0007c000 fe:00 269893                     /lib/x86_64-linux-gnu/libgcrypt.so.11.7.0
7fa85fdcd000-7fa85fe33000 r-xp 00000000 fe:00 20515                      /usr/lib/x86_64-linux-gnu/libcurl.so.4.2.0
7fa85fe33000-7fa860032000 ---p 00066000 fe:00 20515                      /usr/lib/x86_64-linux-gnu/libcurl.so.4.2.0
7fa860032000-7fa860035000 r--p 00065000 fe:00 20515                      /usr/lib/x86_64-linux-gnu/libcurl.so.4.2.0
7fa860035000-7fa860036000 rw-p 00068000 fe:00 20515                      /usr/lib/x86_64-linux-gnu/libcurl.so.4.2.0
7fa860036000-7fa860037000 r-xp 00000000 fe:00 269613                     /lib/x86_64-linux-gnu/libaio.so.1.0.1
7fa860037000-7fa860236000 ---p 00001000 fe:00 269613                     /lib/x86_64-linux-gnu/libaio.so.1.0.1
7fa860236000-7fa860237000 r--p 00000000 fe:00 269613                     /lib/x86_64-linux-gnu/libaio.so.1.0.1
7fa860237000-7fa860238000 rw-p 00001000 fe:00 269613                     /lib/x86_64-linux-gnu/libaio.so.1.0.1
7fa860238000-7fa860258000 r-xp 00000000 fe:00 258647                     /lib/x86_64-linux-gnu/ld-2.13.so
7fa8602ab000-7fa86036a000 rw-p 00000000 00:00 0 
7fa86036a000-7fa860431000 r-xp 00000000 fe:00 85784                      /usr/lib/x86_64-linux-gnu/libcaca.so.0.99.18
7fa860431000-7fa860433000 rw-p 000c6000 fe:00 85784                      /usr/lib/x86_64-linux-gnu/libcaca.so.0.99.18
7fa860433000-7fa860448000 rw-p 00000000 00:00 0 
7fa86044a000-7fa86044b000 -w-s 00001000 00:05 14843                      /dev/xen/gntdev
7fa86044b000-7fa86044c000 rw-s 00000000 00:05 14843                      /dev/xen/gntdev
7fa86044c000-7fa86044d000 rw-p 00000000 00:00 0 
7fa86044d000-7fa86044e000 ---p 00000000 00:00 0 
7fa86044e000-7fa860451000 rw-p 00000000 00:00 0                          [stack:3658]
7fa860451000-7fa860452000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa860452000-7fa860453000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa860453000-7fa860454000 rw-p 00000000 00:00 0 
7fa860454000-7fa860455000 rw-s 00000000 00:11 13783                      /run/shm/spice.3652
7fa860455000-7fa860457000 rw-p 00000000 00:00 0 
7fa860457000-7fa860458000 r--p 0001f000 fe:00 258647                     /lib/x86_64-linux-gnu/ld-2.13.so
7fa860458000-7fa860459000 rw-p 00020000 fe:00 258647                     /lib/x86_64-linux-gnu/ld-2.13.so
7fa860459000-7fa86045a000 rw-p 00000000 00:00 0 
7fa86045a000-7fa8609ff000 r-xp 00000000 fe:00 20995                      /usr/lib/xen/bin/qemu-system-i386
7fa860bfe000-7fa860cc6000 r--p 005a4000 fe:00 20995                      /usr/lib/xen/bin/qemu-system-i386
7fa860cc6000-7fa860d32000 rw-p 0066c000 fe:00 20995                      /usr/lib/xen/bin/qemu-system-i386
7fa860d32000-7fa8611bf000 rw-p 00000000 00:00 0 
7fa861ab6000-7fa864162000 rw-p 00000000 00:00 0                          [heap]
7fff4f3d5000-7fff4f3f6000 rw-p 00000000 00:00 0                          [stack]
7fff4f3fa000-7fff4f3fc000 r-xp 00000000 00:00 0                          [vdso]
7fff4f3fc000-7fff4f3fe000 r--p 00000000 00:00 0                          [vvar]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-05-12  9:23         ` [Qemu-devel] " Fabio Fantoni
  2015-05-12 10:26           ` Fabio Fantoni
@ 2015-05-12 10:26           ` Fabio Fantoni
  1 sibling, 0 replies; 30+ messages in thread
From: Fabio Fantoni @ 2015-05-12 10:26 UTC (permalink / raw)
  To: Stefano Stabellini
  Cc: qemu-devel, xen-devel, Gerd Hoffmann, Jan Beulich,
	Anthony PERARD, spice-devel

[-- Attachment #1: Type: text/plain, Size: 4151 bytes --]

Il 12/05/2015 11:23, Fabio Fantoni ha scritto:
> Il 11/05/2015 17:04, Fabio Fantoni ha scritto:
>> Il 21/04/2015 14:53, Stefano Stabellini ha scritto:
>>> On Tue, 21 Apr 2015, Fabio Fantoni wrote:
>>>> Il 21/04/2015 12:49, Stefano Stabellini ha scritto:
>>>>> On Mon, 20 Apr 2015, Fabio Fantoni wrote:
>>>>>> I updated xen and qemu from xen 4.5.0 with its upstream qemu 
>>>>>> included to
>>>>>> xen
>>>>>> 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk 
>>>>>> to use
>>>>>> revision "master").
>>>>>> After few minutes I booted windows 7 64 bit domU qemu crash, 
>>>>>> tried 2 times
>>>>>> with same result.
>>>>>>
>>>>>> In the domU's qemu log:
>>>>>>> qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion `(old_top ==
>>>>>>> (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) -
>>>>>>> __builtin_offsetof
>>>>>>> (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long)
>>>>>>> (old_size) >= (unsigned long)((((__builtin_offsetof (struct
>>>>>>> malloc_chunk,
>>>>>>> fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * 
>>>>>>> (sizeof(size_t))) -
>>>>>>> 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & 
>>>>>>> pagemask)
>>>>>>> ==
>>>>>>> 0)' failed.
>>>>>>> Killing all inferiors
>>>>>> In attachment the full backtrace of qemu crash.
>>>>>>
>>>>>> With a fast search after I saw the backtrace I found a probable 
>>>>>> cause of
>>>>>> regression (I'm not sure):
>>>>>> http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa 
>>>>>>
>>>>>> spice: make sure we don't overflow ssd->buf
>>>>>>
>>>>>> Added also qemu-devel and spice-devel as cc.
>>>>>>
>>>>>> If you need more informations/tests tell me and I'll post them.
>>>>>    Maybe you could try to revert the offending commit
>>>>> (5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better bisect the
>>>>> crash?
>>>> Thanks for your reply.
>>>>
>>>> I reverted to 4.5.0 on dom0 for now on that system because I'm busy 
>>>> trying to
>>>> found another problem that cause very bad performance without 
>>>> errors or
>>>> nothing in logs :( I don't know if if xen related, kernel related 
>>>> or other for
>>>> now.
>>>>
>>>> About this regression with spice I'll do further tests in next days 
>>>> (probably
>>>> starting reverting the spice patch in qemu) but any help is 
>>>> appreciated.
>>>> Based on data I have for now is possible that the problem is that 
>>>> qemu try to
>>>> allocate other ram or videoram after domU create but with xen is 
>>>> not possible?
>>>> In the spice related patch I saw something about dynamic allocation 
>>>> for
>>>> example.
>>> It is probably caused by a commit in the range:
>>>
>>> 1ebb75b1fee779621b63e84fefa7b07354c43a99..0b8fb1ec3d666d1eb8bbff56c76c5e6daa2789e4 
>>>
>>>
>>> there are only 10 commits in that range. By using git bisect you should
>>> be able to narrow it down in just 3 tests.
>>
>> Sorry for delay, I was busy with many things, today I retried with 
>> updated stable-4.5 and also reverting "spice: make sure we don't 
>> overflow ssd->buf" (in a second test) but in both case regression 
>> remain :(
>> Tomorrow probably I'll do other tests.
>
> I did another test, reverting this instead:
> http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commit;h=c9ac5f816bf3a8b56f836b078711dcef6e5c90b8 
>
> And now seems I'm unable to reproduce the regression, before happen 
> after few seconds up to 1-2 minutes, now I use the same domU 15-20 
> minutes without problem.
> Probably is the cause of regression even if seems strange that on 
> unstable with same patch on tests of some days ago didn't happen.
>
> Any ideas?
>
> Thanks for any reply and sorry for my bad english.

Bad news, qemu crash still happen even if this time in qemu log there is 
another output, see attachment.
After take a look on the other patches I saw:
http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commitdiff;h=7154fba0e51ec985ef621965d1b7120ad424fcbf
With "Conflicts: hw/display/vga.c" in description I'll try to revert it 
instead.

Or someone can tell me another probable test I can try?

[-- Attachment #2: qemu-dm-W7.log --]
[-- Type: text/plain, Size: 52916 bytes --]

main_channel_link: add main channel client
main_channel_handle_parsed: net test: latency 7.814000 ms, bitrate 5417989417 bps (5166.997354 Mbps)
inputs_connect: inputs channel client create
red_dispatcher_set_cursor_peer: 
main_channel_handle_parsed: agent start
main_channel_handle_parsed: agent start
red_channel_client_disconnect: rcc=0x7fa861b2eb60 (channel=0x7fa861b07e80 type=3 id=0)
red_channel_client_disconnect: rcc=0x7fa862349720 (channel=0x7fa861bc8b80 type=2 id=0)
red_channel_client_disconnect: rcc=0x7fa861ca6580 (channel=0x7fa861bb3990 type=4 id=0)
red_channel_client_disconnect_dummy: rcc=0x7fa861ecae20 (channel=0x7fa861c22c40 type=6 id=0)
snd_channel_put: SndChannel=0x7fa861e6f620 freed
red_channel_client_disconnect_dummy: rcc=0x7fa861ec6be0 (channel=0x7fa861b8c890 type=5 id=0)
snd_channel_put: SndChannel=0x7fa861d97ab0 freed
red_channel_client_disconnect: rcc=0x7fa861fbf120 (channel=0x7fa861bd57b0 type=9 id=0)
red_channel_client_disconnect: rcc=0x7fa861fbaee0 (channel=0x7fa861baff50 type=9 id=1)
red_channel_client_disconnect: rcc=0x7fa861eff170 (channel=0x7fa861c79f90 type=9 id=2)
red_channel_client_disconnect: rcc=0x7fa861f044b0 (channel=0x7fa861c7a7b0 type=9 id=3)
red_channel_client_disconnect: rcc=0x7fa861b52ad0 (channel=0x7fa861afc3f0 type=1 id=0)
main_channel_client_on_disconnect: rcc=0x7fa861b52ad0
red_client_destroy: destroy client 0x7fa861efe960 with #channels=6
red_dispatcher_disconnect_cursor_peer: 
red_dispatcher_disconnect_display_peer: 
main_channel_link: add main channel client
main_channel_handle_parsed: agent start
main_channel_handle_parsed: net test: latency 6.413000 ms, bitrate 1550340651 bps (1478.520061 Mbps)
*** glibc detected *** /usr/lib/xen/bin/qemu-system-i386: double free or corruption (!prev): 0x00007fa844d9c8c0 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x75be6)[0x7fa85bb86be6]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x6c)[0x7fa85bb8b98c]
/usr/lib/x86_64-linux-gnu/libpixman-1.so.0(+0x4344b)[0x7fa85c89a44b]
/usr/lib/x86_64-linux-gnu/libpixman-1.so.0(pixman_image_unref+0x9)[0x7fa85c89a399]
/usr/lib/xen/bin/qemu-system-i386(+0x30175b)[0x7fa86075b75b]
/usr/lib/xen/bin/qemu-system-i386(+0x32343b)[0x7fa86077d43b]
/usr/lib/xen/bin/qemu-system-i386(+0x2fad9d)[0x7fa860754d9d]
/usr/lib/xen/bin/qemu-system-i386(+0x19eb4a)[0x7fa8605f8b4a]
/usr/lib/xen/bin/qemu-system-i386(qxl_render_update_area_bh+0x41)[0x7fa8605f8e19]
/usr/lib/xen/bin/qemu-system-i386(+0xa2ba0)[0x7fa8604fcba0]
/usr/lib/xen/bin/qemu-system-i386(+0xa27f9)[0x7fa8604fc7f9]
/usr/lib/xen/bin/qemu-system-i386(+0xa2fc2)[0x7fa8604fcfc2]
/lib/x86_64-linux-gnu/libglib-2.0.so.0(g_main_context_dispatch+0x135)[0x7fa85f056355]
/usr/lib/xen/bin/qemu-system-i386(+0x28303e)[0x7fa8606dd03e]
/usr/lib/xen/bin/qemu-system-i386(+0x28313e)[0x7fa8606dd13e]
/usr/lib/xen/bin/qemu-system-i386(+0x283211)[0x7fa8606dd211]
/usr/lib/xen/bin/qemu-system-i386(+0x32ee14)[0x7fa860788e14]
/usr/lib/xen/bin/qemu-system-i386(main+0x3e76)[0x7fa8607908cf]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd)[0x7fa85bb2fead]
/usr/lib/xen/bin/qemu-system-i386(+0xa2169)[0x7fa8604fc169]
======= Memory map: ========
7fa82bf00000-7fa830000000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa830000000-7fa8336f9000 rw-p 00000000 00:00 0 
7fa8336f9000-7fa834000000 ---p 00000000 00:00 0 
7fa8345e1000-7fa8346e1000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8346e1000-7fa834ae2000 rw-p 00000000 00:00 0 
7fa834ae4000-7fa834be4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa834be4000-7fa834ce4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa834ce4000-7fa834de4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa834de4000-7fa834ee4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa834ee4000-7fa834fe4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa834fe4000-7fa8350e4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8350e4000-7fa8351e4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8351e4000-7fa8352e4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8352e4000-7fa8353e4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8353e4000-7fa8354e4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8354e4000-7fa8355e4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8355e4000-7fa8356e4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8356e4000-7fa8357e4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8357e4000-7fa8358e4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8358e4000-7fa8359e4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8359e4000-7fa835ae4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa835ae4000-7fa835be4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa835be4000-7fa835ce4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa835ce4000-7fa835de4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa835de4000-7fa835ee4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa835ee4000-7fa835fe4000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa841ffc000-7fa841ffd000 ---p 00000000 00:00 0 
7fa841ffd000-7fa8427fd000 rw-p 00000000 00:00 0                          [stack:6334]
7fa8427fd000-7fa8427fe000 ---p 00000000 00:00 0 
7fa8427fe000-7fa842ffe000 rw-p 00000000 00:00 0 
7fa842ffe000-7fa842fff000 ---p 00000000 00:00 0 
7fa842fff000-7fa8437ff000 rw-p 00000000 00:00 0 
7fa8437ff000-7fa843800000 ---p 00000000 00:00 0 
7fa843800000-7fa844000000 rw-p 00000000 00:00 0 
7fa844000000-7fa84519e000 rw-p 00000000 00:00 0 
7fa84519e000-7fa848000000 ---p 00000000 00:00 0 
7fa8480f7000-7fa8481f7000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8481f7000-7fa8482f7000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8482f7000-7fa8483f7000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8487f8000-7fa8488f8000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8488f8000-7fa8489f8000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa8489f8000-7fa848af8000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa848af8000-7fa848bf8000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa849bfa000-7fa849cfa000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa849cfa000-7fa849dfa000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa849dfa000-7fa849efa000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa849efa000-7fa849ffa000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa849ffa000-7fa84a0fa000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84a0fa000-7fa84a1fa000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84a1fa000-7fa84a7fc000 rw-p 00000000 00:00 0 
7fa84a7fd000-7fa84a8fd000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84a8fd000-7fa84a9fd000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84a9fd000-7fa84aafd000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84aafd000-7fa84adfe000 rw-p 00000000 00:00 0 
7fa84b619000-7fa84b719000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84b719000-7fa84b819000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84b819000-7fa84b919000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84b919000-7fa84ba19000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84ba73000-7fa84bb73000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84bb73000-7fa84bc73000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84bc73000-7fa84bc74000 ---p 00000000 00:00 0 
7fa84bc74000-7fa84c474000 rw-p 00000000 00:00 0                          [stack:3674]
7fa84c474000-7fa84c574000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84c574000-7fa84c674000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84c674000-7fa84c84e000 rw-p 00000000 00:00 0 
7fa84c84e000-7fa84c84f000 ---p 00000000 00:00 0 
7fa84c84f000-7fa84d04f000 rw-p 00000000 00:00 0                          [stack:3673]
7fa84d04f000-7fa84d14f000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84d17c000-7fa84d27c000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa84d27c000-7fa85127c000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa85187d000-7fa85197d000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa85197d000-7fa851a7d000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa851a7d000-7fa851b7e000 rw-p 00000000 00:00 0 
7fa851b7e000-7fa851b7f000 ---p 00000000 00:00 0 
7fa851b7f000-7fa85237f000 rw-p 00000000 00:00 0                          [stack:3668]
7fa85237f000-7fa852380000 ---p 00000000 00:00 0 
7fa852380000-7fa852d01000 rw-p 00000000 00:00 0                          [stack:3664]
7fa852d01000-7fa852d06000 r-xp 00000000 fe:00 257374                     /usr/lib/x86_64-linux-gnu/sasl2/libcrammd5.so.2.0.25
7fa852d06000-7fa852f05000 ---p 00005000 fe:00 257374                     /usr/lib/x86_64-linux-gnu/sasl2/libcrammd5.so.2.0.25
7fa852f05000-7fa852f06000 r--p 00004000 fe:00 257374                     /usr/lib/x86_64-linux-gnu/sasl2/libcrammd5.so.2.0.25
7fa852f06000-7fa852f07000 rw-p 00005000 fe:00 257374                     /usr/lib/x86_64-linux-gnu/sasl2/libcrammd5.so.2.0.25
7fa852f07000-7fa853081000 r-xp 00000000 fe:00 85596                      /usr/lib/x86_64-linux-gnu/libdb-5.1.so
7fa853081000-7fa853281000 ---p 0017a000 fe:00 85596                      /usr/lib/x86_64-linux-gnu/libdb-5.1.so
7fa853281000-7fa853287000 r--p 0017a000 fe:00 85596                      /usr/lib/x86_64-linux-gnu/libdb-5.1.so
7fa853287000-7fa85328a000 rw-p 00180000 fe:00 85596                      /usr/lib/x86_64-linux-gnu/libdb-5.1.so
7fa85328a000-7fa853290000 r-xp 00000000 fe:00 257385                     /usr/lib/x86_64-linux-gnu/sasl2/libsasldb.so.2.0.25
7fa853290000-7fa85348f000 ---p 00006000 fe:00 257385                     /usr/lib/x86_64-linux-gnu/sasl2/libsasldb.so.2.0.25
7fa85348f000-7fa853490000 r--p 00005000 fe:00 257385                     /usr/lib/x86_64-linux-gnu/sasl2/libsasldb.so.2.0.25
7fa853490000-7fa853491000 rw-p 00006000 fe:00 257385                     /usr/lib/x86_64-linux-gnu/sasl2/libsasldb.so.2.0.25
7fa853491000-7fa853495000 r-xp 00000000 fe:00 257372                     /usr/lib/x86_64-linux-gnu/sasl2/libplain.so.2.0.25
7fa853495000-7fa853694000 ---p 00004000 fe:00 257372                     /usr/lib/x86_64-linux-gnu/sasl2/libplain.so.2.0.25
7fa853694000-7fa853695000 r--p 00003000 fe:00 257372                     /usr/lib/x86_64-linux-gnu/sasl2/libplain.so.2.0.25
7fa853695000-7fa853696000 rw-p 00004000 fe:00 257372                     /usr/lib/x86_64-linux-gnu/sasl2/libplain.so.2.0.25
7fa853696000-7fa8536a3000 r-xp 00000000 fe:00 257387                     /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so.2.0.25
7fa8536a3000-7fa8538a2000 ---p 0000d000 fe:00 257387                     /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so.2.0.25
7fa8538a2000-7fa8538a3000 r--p 0000c000 fe:00 257387                     /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so.2.0.25
7fa8538a3000-7fa8538a4000 rw-p 0000d000 fe:00 257387                     /usr/lib/x86_64-linux-gnu/sasl2/libdigestmd5.so.2.0.25
7fa8538a4000-7fa8538ac000 r-xp 00000000 fe:00 258635                     /lib/x86_64-linux-gnu/libcrypt-2.13.so
7fa8538ac000-7fa853aab000 ---p 00008000 fe:00 258635                     /lib/x86_64-linux-gnu/libcrypt-2.13.so
7fa853aab000-7fa853aac000 r--p 00007000 fe:00 258635                     /lib/x86_64-linux-gnu/libcrypt-2.13.so
7fa853aac000-7fa853aad000 rw-p 00008000 fe:00 258635                     /lib/x86_64-linux-gnu/libcrypt-2.13.so
7fa853aad000-7fa853adb000 rw-p 00000000 00:00 0 
7fa853adb000-7fa853adf000 r-xp 00000000 fe:00 257373                     /usr/lib/x86_64-linux-gnu/sasl2/liblogin.so.2.0.25
7fa853adf000-7fa853cde000 ---p 00004000 fe:00 257373                     /usr/lib/x86_64-linux-gnu/sasl2/liblogin.so.2.0.25
7fa853cde000-7fa853cdf000 r--p 00003000 fe:00 257373                     /usr/lib/x86_64-linux-gnu/sasl2/liblogin.so.2.0.25
7fa853cdf000-7fa853ce0000 rw-p 00004000 fe:00 257373                     /usr/lib/x86_64-linux-gnu/sasl2/liblogin.so.2.0.25
7fa853ce0000-7fa853ce4000 r-xp 00000000 fe:00 257378                     /usr/lib/x86_64-linux-gnu/sasl2/libanonymous.so.2.0.25
7fa853ce4000-7fa853ee3000 ---p 00004000 fe:00 257378                     /usr/lib/x86_64-linux-gnu/sasl2/libanonymous.so.2.0.25
7fa853ee3000-7fa853ee4000 r--p 00003000 fe:00 257378                     /usr/lib/x86_64-linux-gnu/sasl2/libanonymous.so.2.0.25
7fa853ee4000-7fa853ee5000 rw-p 00004000 fe:00 257378                     /usr/lib/x86_64-linux-gnu/sasl2/libanonymous.so.2.0.25
7fa853ee5000-7fa853eed000 r-xp 00000000 fe:00 257377                     /usr/lib/x86_64-linux-gnu/sasl2/libntlm.so.2.0.25
7fa853eed000-7fa8540ec000 ---p 00008000 fe:00 257377                     /usr/lib/x86_64-linux-gnu/sasl2/libntlm.so.2.0.25
7fa8540ec000-7fa8540ed000 r--p 00007000 fe:00 257377                     /usr/lib/x86_64-linux-gnu/sasl2/libntlm.so.2.0.25
7fa8540ed000-7fa8540ee000 rw-p 00008000 fe:00 257377                     /usr/lib/x86_64-linux-gnu/sasl2/libntlm.so.2.0.25
7fa8540ee000-7fa8540f4000 r-xp 00000000 fe:00 85642                      /usr/lib/x86_64-linux-gnu/libogg.so.0.8.0
7fa8540f4000-7fa8542f3000 ---p 00006000 fe:00 85642                      /usr/lib/x86_64-linux-gnu/libogg.so.0.8.0
7fa8542f3000-7fa8542f4000 rw-p 00005000 fe:00 85642                      /usr/lib/x86_64-linux-gnu/libogg.so.0.8.0
7fa8542f4000-7fa854320000 r-xp 00000000 fe:00 85239                      /usr/lib/x86_64-linux-gnu/libvorbis.so.0.4.5
7fa854320000-7fa85451f000 ---p 0002c000 fe:00 85239                      /usr/lib/x86_64-linux-gnu/libvorbis.so.0.4.5
7fa85451f000-7fa854520000 r--p 0002b000 fe:00 85239                      /usr/lib/x86_64-linux-gnu/libvorbis.so.0.4.5
7fa854520000-7fa854521000 rw-p 0002c000 fe:00 85239                      /usr/lib/x86_64-linux-gnu/libvorbis.so.0.4.5
7fa854521000-7fa8547d4000 r-xp 00000000 fe:00 85355                      /usr/lib/x86_64-linux-gnu/libvorbisenc.so.2.0.8
7fa8547d4000-7fa8549d3000 ---p 002b3000 fe:00 85355                      /usr/lib/x86_64-linux-gnu/libvorbisenc.so.2.0.8
7fa8549d3000-7fa8549ef000 r--p 002b2000 fe:00 85355                      /usr/lib/x86_64-linux-gnu/libvorbisenc.so.2.0.8
7fa8549ef000-7fa8549f0000 rw-p 002ce000 fe:00 85355                      /usr/lib/x86_64-linux-gnu/libvorbisenc.so.2.0.8
7fa8549f0000-7fa854a3c000 r-xp 00000000 fe:00 85555                      /usr/lib/x86_64-linux-gnu/libFLAC.so.8.2.0
7fa854a3c000-7fa854c3c000 ---p 0004c000 fe:00 85555                      /usr/lib/x86_64-linux-gnu/libFLAC.so.8.2.0
7fa854c3c000-7fa854c3d000 r--p 0004c000 fe:00 85555                      /usr/lib/x86_64-linux-gnu/libFLAC.so.8.2.0
7fa854c3d000-7fa854c3e000 rw-p 0004d000 fe:00 85555                      /usr/lib/x86_64-linux-gnu/libFLAC.so.8.2.0
7fa854c3e000-7fa854c53000 r-xp 00000000 fe:00 258289                     /lib/x86_64-linux-gnu/libnsl-2.13.so
7fa854c53000-7fa854e52000 ---p 00015000 fe:00 258289                     /lib/x86_64-linux-gnu/libnsl-2.13.so
7fa854e52000-7fa854e53000 r--p 00014000 fe:00 258289                     /lib/x86_64-linux-gnu/libnsl-2.13.so
7fa854e53000-7fa854e54000 rw-p 00015000 fe:00 258289                     /lib/x86_64-linux-gnu/libnsl-2.13.so
7fa854e54000-7fa854e56000 rw-p 00000000 00:00 0 
7fa854e56000-7fa854e64000 r-xp 00000000 fe:00 85131                      /usr/lib/x86_64-linux-gnu/libXi.so.6.1.0
7fa854e64000-7fa855064000 ---p 0000e000 fe:00 85131                      /usr/lib/x86_64-linux-gnu/libXi.so.6.1.0
7fa855064000-7fa855065000 rw-p 0000e000 fe:00 85131                      /usr/lib/x86_64-linux-gnu/libXi.so.6.1.0
7fa855065000-7fa855069000 r-xp 00000000 fe:00 269641                     /lib/x86_64-linux-gnu/libattr.so.1.1.0
7fa855069000-7fa855268000 ---p 00004000 fe:00 269641                     /lib/x86_64-linux-gnu/libattr.so.1.1.0
7fa855268000-7fa855269000 r--p 00003000 fe:00 269641                     /lib/x86_64-linux-gnu/libattr.so.1.1.0
7fa855269000-7fa85526a000 rw-p 00004000 fe:00 269641                     /lib/x86_64-linux-gnu/libattr.so.1.1.0
7fa85526a000-7fa85526f000 r-xp 00000000 fe:00 85661                      /usr/lib/x86_64-linux-gnu/libasyncns.so.0.3.1
7fa85526f000-7fa85546e000 ---p 00005000 fe:00 85661                      /usr/lib/x86_64-linux-gnu/libasyncns.so.0.3.1
7fa85546e000-7fa85546f000 rw-p 00004000 fe:00 85661                      /usr/lib/x86_64-linux-gnu/libasyncns.so.0.3.1
7fa85546f000-7fa8554d0000 r-xp 00000000 fe:00 85625                      /usr/lib/x86_64-linux-gnu/libsndfile.so.1.0.25
7fa8554d0000-7fa8556cf000 ---p 00061000 fe:00 85625                      /usr/lib/x86_64-linux-gnu/libsndfile.so.1.0.25
7fa8556cf000-7fa8556d1000 r--p 00060000 fe:00 85625                      /usr/lib/x86_64-linux-gnu/libsndfile.so.1.0.25
7fa8556d1000-7fa8556d2000 rw-p 00062000 fe:00 85625                      /usr/lib/x86_64-linux-gnu/libsndfile.so.1.0.25
7fa8556d2000-7fa8556d6000 rw-p 00000000 00:00 0 
7fa8556d6000-7fa8556df000 r-xp 00000000 fe:00 269460                     /lib/x86_64-linux-gnu/libwrap.so.0.7.6
7fa8556df000-7fa8558de000 ---p 00009000 fe:00 269460                     /lib/x86_64-linux-gnu/libwrap.so.0.7.6
7fa8558de000-7fa8558df000 r--p 00008000 fe:00 269460                     /lib/x86_64-linux-gnu/libwrap.so.0.7.6
7fa8558df000-7fa8558e0000 rw-p 00009000 fe:00 269460                     /lib/x86_64-linux-gnu/libwrap.so.0.7.6
7fa8558e0000-7fa8558e1000 rw-p 00000000 00:00 0 
7fa8558e1000-7fa8558e6000 r-xp 00000000 fe:00 85735                      /usr/lib/x86_64-linux-gnu/libXtst.so.6.1.0
7fa8558e6000-7fa855ae5000 ---p 00005000 fe:00 85735                      /usr/lib/x86_64-linux-gnu/libXtst.so.6.1.0
7fa855ae5000-7fa855ae6000 rw-p 00004000 fe:00 85735                      /usr/lib/x86_64-linux-gnu/libXtst.so.6.1.0
7fa855ae6000-7fa855aed000 r-xp 00000000 fe:00 85885                      /usr/lib/x86_64-linux-gnu/libSM.so.6.0.1
7fa855aed000-7fa855cec000 ---p 00007000 fe:00 85885                      /usr/lib/x86_64-linux-gnu/libSM.so.6.0.1
7fa855cec000-7fa855ced000 rw-p 00006000 fe:00 85885                      /usr/lib/x86_64-linux-gnu/libSM.so.6.0.1
7fa855ced000-7fa855d04000 r-xp 00000000 fe:00 85856                      /usr/lib/x86_64-linux-gnu/libICE.so.6.3.0
7fa855d04000-7fa855f03000 ---p 00017000 fe:00 85856                      /usr/lib/x86_64-linux-gnu/libICE.so.6.3.0
7fa855f03000-7fa855f05000 rw-p 00016000 fe:00 85856                      /usr/lib/x86_64-linux-gnu/libICE.so.6.3.0
7fa855f05000-7fa855f08000 rw-p 00000000 00:00 0 
7fa855f08000-7fa855f09000 r-xp 00000000 fe:00 2179                       /usr/lib/x86_64-linux-gnu/libX11-xcb.so.1.0.0
7fa855f09000-7fa856108000 ---p 00001000 fe:00 2179                       /usr/lib/x86_64-linux-gnu/libX11-xcb.so.1.0.0
7fa856108000-7fa856109000 rw-p 00000000 fe:00 2179                       /usr/lib/x86_64-linux-gnu/libX11-xcb.so.1.0.0
7fa856109000-7fa85610e000 r-xp 00000000 fe:00 85077                      /usr/lib/x86_64-linux-gnu/libXdmcp.so.6.0.0
7fa85610e000-7fa85630d000 ---p 00005000 fe:00 85077                      /usr/lib/x86_64-linux-gnu/libXdmcp.so.6.0.0
7fa85630d000-7fa85630e000 rw-p 00004000 fe:00 85077                      /usr/lib/x86_64-linux-gnu/libXdmcp.so.6.0.0
7fa85630e000-7fa856310000 r-xp 00000000 fe:00 85430                      /usr/lib/x86_64-linux-gnu/libXau.so.6.0.0
7fa856310000-7fa856510000 ---p 00002000 fe:00 85430                      /usr/lib/x86_64-linux-gnu/libXau.so.6.0.0
7fa856510000-7fa856511000 rw-p 00002000 fe:00 85430                      /usr/lib/x86_64-linux-gnu/libXau.so.6.0.0
7fa856511000-7fa856540000 r-xp 00000000 fe:00 269557                     /lib/x86_64-linux-gnu/libncursesw.so.5.9
7fa856540000-7fa85673f000 ---p 0002f000 fe:00 269557                     /lib/x86_64-linux-gnu/libncursesw.so.5.9
7fa85673f000-7fa856740000 r--p 0002e000 fe:00 269557                     /lib/x86_64-linux-gnu/libncursesw.so.5.9
7fa856740000-7fa856741000 rw-p 0002f000 fe:00 269557                     /lib/x86_64-linux-gnu/libncursesw.so.5.9
7fa856741000-7fa856856000 r-xp 00000000 fe:00 269619                     /lib/x86_64-linux-gnu/libslang.so.2.2.4
7fa856856000-7fa856a55000 ---p 00115000 fe:00 269619                     /lib/x86_64-linux-gnu/libslang.so.2.2.4
7fa856a55000-7fa856a59000 r--p 00114000 fe:00 269619                     /lib/x86_64-linux-gnu/libslang.so.2.2.4
7fa856a59000-7fa856a73000 rw-p 00118000 fe:00 269619                     /lib/x86_64-linux-gnu/libslang.so.2.2.4
7fa856a73000-7fa856ad7000 rw-p 00000000 00:00 0 
7fa856ad7000-7fa856b1c000 r-xp 00000000 fe:00 269510                     /lib/x86_64-linux-gnu/libdbus-1.so.3.7.2
7fa856b1c000-7fa856d1b000 ---p 00045000 fe:00 269510                     /lib/x86_64-linux-gnu/libdbus-1.so.3.7.2
7fa856d1b000-7fa856d1c000 r--p 00044000 fe:00 269510                     /lib/x86_64-linux-gnu/libdbus-1.so.3.7.2
7fa856d1c000-7fa856d1d000 rw-p 00045000 fe:00 269510                     /lib/x86_64-linux-gnu/libdbus-1.so.3.7.2
7fa856d1d000-7fa856d26000 r-xp 00000000 fe:00 269538                     /lib/x86_64-linux-gnu/libjson.so.0.1.0
7fa856d26000-7fa856f25000 ---p 00009000 fe:00 269538                     /lib/x86_64-linux-gnu/libjson.so.0.1.0
7fa856f25000-7fa856f26000 r--p 00008000 fe:00 269538                     /lib/x86_64-linux-gnu/libjson.so.0.1.0
7fa856f26000-7fa856f27000 rw-p 00009000 fe:00 269538                     /lib/x86_64-linux-gnu/libjson.so.0.1.0
7fa856f27000-7fa856f2b000 r-xp 00000000 fe:00 269503                     /lib/x86_64-linux-gnu/libcap.so.2.22
7fa856f2b000-7fa85712a000 ---p 00004000 fe:00 269503                     /lib/x86_64-linux-gnu/libcap.so.2.22
7fa85712a000-7fa85712b000 rw-p 00003000 fe:00 269503                     /lib/x86_64-linux-gnu/libcap.so.2.22
7fa85712b000-7fa85718c000 r-xp 00000000 fe:00 85203                      /usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-2.0.so
7fa85718c000-7fa85738b000 ---p 00061000 fe:00 85203                      /usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-2.0.so
7fa85738b000-7fa85738c000 r--p 00060000 fe:00 85203                      /usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-2.0.so
7fa85738c000-7fa85738e000 rw-p 00061000 fe:00 85203                      /usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-2.0.so
7fa85738e000-7fa857391000 r-xp 00000000 fe:00 269620                     /lib/x86_64-linux-gnu/libkeyutils.so.1.4
7fa857391000-7fa857590000 ---p 00003000 fe:00 269620                     /lib/x86_64-linux-gnu/libkeyutils.so.1.4
7fa857590000-7fa857591000 r--p 00002000 fe:00 269620                     /lib/x86_64-linux-gnu/libkeyutils.so.1.4
7fa857591000-7fa857592000 rw-p 00003000 fe:00 269620                     /lib/x86_64-linux-gnu/libkeyutils.so.1.4
7fa857592000-7fa85759a000 r-xp 00000000 fe:00 8184                       /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1
7fa85759a000-7fa857799000 ---p 00008000 fe:00 8184                       /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1
7fa857799000-7fa85779a000 r--p 00007000 fe:00 8184                       /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1
7fa85779a000-7fa85779b000 rw-p 00008000 fe:00 8184                       /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1
7fa85779b000-7fa85779e000 r-xp 00000000 fe:00 275449                     /lib/x86_64-linux-gnu/libcom_err.so.2.1
7fa85779e000-7fa85799d000 ---p 00003000 fe:00 275449                     /lib/x86_64-linux-gnu/libcom_err.so.2.1
7fa85799d000-7fa85799e000 r--p 00002000 fe:00 275449                     /lib/x86_64-linux-gnu/libcom_err.so.2.1
7fa85799e000-7fa85799f000 rw-p 00003000 fe:00 275449                     /lib/x86_64-linux-gnu/libcom_err.so.2.1
7fa85799f000-7fa8579c5000 r-xp 00000000 fe:00 20322                      /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1
7fa8579c5000-7fa857bc5000 ---p 00026000 fe:00 20322                      /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1
7fa857bc5000-7fa857bc6000 r--p 00026000 fe:00 20322                      /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1
7fa857bc6000-7fa857bc7000 rw-p 00027000 fe:00 20322                      /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1
7fa857bc7000-7fa857bc8000 rw-p 00000000 00:00 0 
7fa857bc8000-7fa857c91000 r-xp 00000000 fe:00 20391                      /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3
7fa857c91000-7fa857e90000 ---p 000c9000 fe:00 20391                      /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3
7fa857e90000-7fa857e9a000 r--p 000c8000 fe:00 20391                      /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3
7fa857e9a000-7fa857e9c000 rw-p 000d2000 fe:00 20391                      /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3
7fa857e9c000-7fa857ea9000 r-xp 00000000 fe:00 269631                     /lib/x86_64-linux-gnu/libudev.so.0.13.0
7fa857ea9000-7fa8580a9000 ---p 0000d000 fe:00 269631                     /lib/x86_64-linux-gnu/libudev.so.0.13.0
7fa8580a9000-7fa8580aa000 r--p 0000d000 fe:00 269631                     /lib/x86_64-linux-gnu/libudev.so.0.13.0
7fa8580aa000-7fa8580ab000 rw-p 0000e000 fe:00 269631                     /lib/x86_64-linux-gnu/libudev.so.0.13.0
7fa8580ab000-7fa8580b2000 r-xp 00000000 fe:00 85685                      /usr/lib/x86_64-linux-gnu/liblz4.so.1.3.0
7fa8580b2000-7fa8582b1000 ---p 00007000 fe:00 85685                      /usr/lib/x86_64-linux-gnu/liblz4.so.1.3.0
7fa8582b1000-7fa8582b2000 r--p 00006000 fe:00 85685                      /usr/lib/x86_64-linux-gnu/liblz4.so.1.3.0
7fa8582b2000-7fa8582b3000 rw-p 00007000 fe:00 85685                      /usr/lib/x86_64-linux-gnu/liblz4.so.1.3.0
7fa8582b3000-7fa8582fb000 r-xp 00000000 fe:00 85743                      /usr/lib/x86_64-linux-gnu/libopus.so.0.5.0
7fa8582fb000-7fa8584fb000 ---p 00048000 fe:00 85743                      /usr/lib/x86_64-linux-gnu/libopus.so.0.5.0
7fa8584fb000-7fa8584fc000 r--p 00048000 fe:00 85743                      /usr/lib/x86_64-linux-gnu/libopus.so.0.5.0
7fa8584fc000-7fa8584fd000 rw-p 00049000 fe:00 85743                      /usr/lib/x86_64-linux-gnu/libopus.so.0.5.0
7fa8584fd000-7fa85851f000 r-xp 00000000 fe:00 269545                     /lib/x86_64-linux-gnu/liblzma.so.5.0.0
7fa85851f000-7fa85871e000 ---p 00022000 fe:00 269545                     /lib/x86_64-linux-gnu/liblzma.so.5.0.0
7fa85871e000-7fa85871f000 r--p 00021000 fe:00 269545                     /lib/x86_64-linux-gnu/liblzma.so.5.0.0
7fa85871f000-7fa858720000 rw-p 00022000 fe:00 269545                     /lib/x86_64-linux-gnu/liblzma.so.5.0.0
7fa858720000-7fa85872f000 r-xp 00000000 fe:00 269555                     /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7fa85872f000-7fa85892e000 ---p 0000f000 fe:00 269555                     /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7fa85892e000-7fa85892f000 r--p 0000e000 fe:00 269555                     /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7fa85892f000-7fa858930000 rw-p 0000f000 fe:00 269555                     /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7fa858930000-7fa85894f000 r-xp 00000000 fe:00 85679                      /usr/lib/x86_64-linux-gnu/libxcb.so.1.1.0
7fa85894f000-7fa858b4e000 ---p 0001f000 fe:00 85679                      /usr/lib/x86_64-linux-gnu/libxcb.so.1.1.0
7fa858b4e000-7fa858b4f000 r--p 0001e000 fe:00 85679                      /usr/lib/x86_64-linux-gnu/libxcb.so.1.1.0
7fa858b4f000-7fa858b50000 rw-p 0001f000 fe:00 85679                      /usr/lib/x86_64-linux-gnu/libxcb.so.1.1.0
7fa858b50000-7fa858b52000 r-xp 00000000 fe:00 85904                      /usr/lib/x86_64-linux-gnu/libts-0.0.so.0.1.1
7fa858b52000-7fa858d52000 ---p 00002000 fe:00 85904                      /usr/lib/x86_64-linux-gnu/libts-0.0.so.0.1.1
7fa858d52000-7fa858d53000 rw-p 00002000 fe:00 85904                      /usr/lib/x86_64-linux-gnu/libts-0.0.so.0.1.1
7fa858d53000-7fa858d6a000 r-xp 00000000 fe:00 85649                      /usr/lib/x86_64-linux-gnu/libdirect-1.2.so.9.0.1
7fa858d6a000-7fa858f69000 ---p 00017000 fe:00 85649                      /usr/lib/x86_64-linux-gnu/libdirect-1.2.so.9.0.1
7fa858f69000-7fa858f6a000 rw-p 00016000 fe:00 85649                      /usr/lib/x86_64-linux-gnu/libdirect-1.2.so.9.0.1
7fa858f6a000-7fa858f6b000 rw-p 00000000 00:00 0 
7fa858f6b000-7fa858f74000 r-xp 00000000 fe:00 85270                      /usr/lib/x86_64-linux-gnu/libfusion-1.2.so.9.0.1
7fa858f74000-7fa859174000 ---p 00009000 fe:00 85270                      /usr/lib/x86_64-linux-gnu/libfusion-1.2.so.9.0.1
7fa859174000-7fa859175000 rw-p 00009000 fe:00 85270                      /usr/lib/x86_64-linux-gnu/libfusion-1.2.so.9.0.1
7fa859175000-7fa8591f6000 r-xp 00000000 fe:00 85874                      /usr/lib/x86_64-linux-gnu/libdirectfb-1.2.so.9.0.1
7fa8591f6000-7fa8593f6000 ---p 00081000 fe:00 85874                      /usr/lib/x86_64-linux-gnu/libdirectfb-1.2.so.9.0.1
7fa8593f6000-7fa8593fa000 rw-p 00081000 fe:00 85874                      /usr/lib/x86_64-linux-gnu/libdirectfb-1.2.so.9.0.1
7fa8593fa000-7fa85940b000 r-xp 00000000 fe:00 85563                      /usr/lib/x86_64-linux-gnu/libXext.so.6.4.0
7fa85940b000-7fa85960b000 ---p 00011000 fe:00 85563                      /usr/lib/x86_64-linux-gnu/libXext.so.6.4.0
7fa85960b000-7fa85960c000 rw-p 00011000 fe:00 85563                      /usr/lib/x86_64-linux-gnu/libXext.so.6.4.0
7fa85960c000-7fa859655000 r-xp 00000000 fe:00 85864                      /usr/lib/x86_64-linux-gnu/libpulse.so.0.14.2
7fa859655000-7fa859854000 ---p 00049000 fe:00 85864                      /usr/lib/x86_64-linux-gnu/libpulse.so.0.14.2
7fa859854000-7fa859855000 r--p 00048000 fe:00 85864                      /usr/lib/x86_64-linux-gnu/libpulse.so.0.14.2
7fa859855000-7fa859856000 rw-p 00049000 fe:00 85864                      /usr/lib/x86_64-linux-gnu/libpulse.so.0.14.2
7fa859856000-7fa859859000 r-xp 00000000 fe:00 85149                      /usr/lib/x86_64-linux-gnu/libpulse-simple.so.0.0.3
7fa859859000-7fa859a59000 ---p 00003000 fe:00 85149                      /usr/lib/x86_64-linux-gnu/libpulse-simple.so.0.0.3
7fa859a59000-7fa859a5a000 r--p 00003000 fe:00 85149                      /usr/lib/x86_64-linux-gnu/libpulse-simple.so.0.0.3
7fa859a5a000-7fa859a5b000 rw-p 00004000 fe:00 85149                      /usr/lib/x86_64-linux-gnu/libpulse-simple.so.0.0.3
7fa859a5b000-7fa859b47000 r-xp 00000000 fe:00 85458                      /usr/lib/x86_64-linux-gnu/libasound.so.2.0.0
7fa859b47000-7fa859d47000 ---p 000ec000 fe:00 85458                      /usr/lib/x86_64-linux-gnu/libasound.so.2.0.0
7fa859d47000-7fa859d4d000 r--p 000ec000 fe:00 85458                      /usr/lib/x86_64-linux-gnu/libasound.so.2.0.0
7fa859d4d000-7fa859d4f000 rw-p 000f2000 fe:00 85458                      /usr/lib/x86_64-linux-gnu/libasound.so.2.0.0
7fa859d4f000-7fa859d60000 r-xp 00000000 fe:00 85796                      /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.0.0
7fa859d60000-7fa859f5f000 ---p 00011000 fe:00 85796                      /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.0.0
7fa859f5f000-7fa859f60000 r--p 00010000 fe:00 85796                      /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.0.0
7fa859f60000-7fa859f61000 rw-p 00011000 fe:00 85796                      /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.0.0
7fa859f61000-7fa859f71000 r-xp 00000000 fe:00 19965                      /usr/lib/x86_64-linux-gnu/libtasn1.so.3.1.16
7fa859f71000-7fa85a170000 ---p 00010000 fe:00 19965                      /usr/lib/x86_64-linux-gnu/libtasn1.so.3.1.16
7fa85a170000-7fa85a171000 r--p 0000f000 fe:00 19965                      /usr/lib/x86_64-linux-gnu/libtasn1.so.3.1.16
7fa85a171000-7fa85a172000 rw-p 00010000 fe:00 19965                      /usr/lib/x86_64-linux-gnu/libtasn1.so.3.1.16
7fa85a172000-7fa85a185000 r-xp 00000000 fe:00 258649                     /lib/x86_64-linux-gnu/libresolv-2.13.so
7fa85a185000-7fa85a384000 ---p 00013000 fe:00 258649                     /lib/x86_64-linux-gnu/libresolv-2.13.so
7fa85a384000-7fa85a385000 r--p 00012000 fe:00 258649                     /lib/x86_64-linux-gnu/libresolv-2.13.so
7fa85a385000-7fa85a386000 rw-p 00013000 fe:00 258649                     /lib/x86_64-linux-gnu/libresolv-2.13.so
7fa85a386000-7fa85a388000 rw-p 00000000 00:00 0 
7fa85a388000-7fa85a38a000 r-xp 00000000 fe:00 258633                     /lib/x86_64-linux-gnu/libdl-2.13.so
7fa85a38a000-7fa85a58a000 ---p 00002000 fe:00 258633                     /lib/x86_64-linux-gnu/libdl-2.13.so
7fa85a58a000-7fa85a58b000 r--p 00002000 fe:00 258633                     /lib/x86_64-linux-gnu/libdl-2.13.so
7fa85a58b000-7fa85a58c000 rw-p 00003000 fe:00 258633                     /lib/x86_64-linux-gnu/libdl-2.13.so
7fa85a58c000-7fa85a5c8000 r-xp 00000000 fe:00 269550                     /lib/x86_64-linux-gnu/libpcre.so.3.13.1
7fa85a5c8000-7fa85a7c8000 ---p 0003c000 fe:00 269550                     /lib/x86_64-linux-gnu/libpcre.so.3.13.1
7fa85a7c8000-7fa85a7c9000 rw-p 0003c000 fe:00 269550                     /lib/x86_64-linux-gnu/libpcre.so.3.13.1
7fa85a7c9000-7fa85a7cc000 r-xp 00000000 fe:00 269443                     /lib/x86_64-linux-gnu/libgpg-error.so.0.8.0
7fa85a7cc000-7fa85a9cb000 ---p 00003000 fe:00 269443                     /lib/x86_64-linux-gnu/libgpg-error.so.0.8.0
7fa85a9cb000-7fa85a9cc000 rw-p 00002000 fe:00 269443                     /lib/x86_64-linux-gnu/libgpg-error.so.0.8.0
7fa85a9cc000-7fa85a9e5000 r-xp 00000000 fe:00 85932                      /usr/lib/x86_64-linux-gnu/librtmp.so.0
7fa85a9e5000-7fa85abe5000 ---p 00019000 fe:00 85932                      /usr/lib/x86_64-linux-gnu/librtmp.so.0
7fa85abe5000-7fa85abe6000 rw-p 00019000 fe:00 85932                      /usr/lib/x86_64-linux-gnu/librtmp.so.0
7fa85abe6000-7fa85adb0000 r-xp 00000000 fe:00 7494                       /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7fa85adb0000-7fa85afb0000 ---p 001ca000 fe:00 7494                       /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7fa85afb0000-7fa85afcb000 r--p 001ca000 fe:00 7494                       /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7fa85afcb000-7fa85afda000 rw-p 001e5000 fe:00 7494                       /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7fa85afda000-7fa85afde000 rw-p 00000000 00:00 0 
7fa85afde000-7fa85b034000 r-xp 00000000 fe:00 7511                       /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
7fa85b034000-7fa85b234000 ---p 00056000 fe:00 7511                       /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
7fa85b234000-7fa85b237000 r--p 00056000 fe:00 7511                       /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
7fa85b237000-7fa85b23e000 rw-p 00059000 fe:00 7511                       /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
7fa85b23e000-7fa85b27a000 r-xp 00000000 fe:00 20354                      /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2
7fa85b27a000-7fa85b47a000 ---p 0003c000 fe:00 20354                      /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2
7fa85b47a000-7fa85b47b000 r--p 0003c000 fe:00 20354                      /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2
7fa85b47b000-7fa85b47d000 rw-p 0003d000 fe:00 20354                      /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2
7fa85b47d000-7fa85b4c9000 r-xp 00000000 fe:00 19738                      /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.8.3
7fa85b4c9000-7fa85b6c9000 ---p 0004c000 fe:00 19738                      /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.8.3
7fa85b6c9000-7fa85b6cb000 r--p 0004c000 fe:00 19738                      /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.8.3
7fa85b6cb000-7fa85b6cc000 rw-p 0004e000 fe:00 19738                      /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.8.3
7fa85b6cc000-7fa85b6ce000 rw-p 00000000 00:00 0 
7fa85b6ce000-7fa85b6dc000 r-xp 00000000 fe:00 19723                      /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.8.3
7fa85b6dc000-7fa85b8db000 ---p 0000e000 fe:00 19723                      /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.8.3
7fa85b8db000-7fa85b8dc000 r--p 0000d000 fe:00 19723                      /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.8.3
7fa85b8dc000-7fa85b8dd000 rw-p 0000e000 fe:00 19723                      /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.8.3
7fa85b8dd000-7fa85b90f000 r-xp 00000000 fe:00 85264                      /usr/lib/x86_64-linux-gnu/libidn.so.11.6.8
7fa85b90f000-7fa85bb0f000 ---p 00032000 fe:00 85264                      /usr/lib/x86_64-linux-gnu/libidn.so.11.6.8
7fa85bb0f000-7fa85bb10000 r--p 00032000 fe:00 85264                      /usr/lib/x86_64-linux-gnu/libidn.so.11.6.8
7fa85bb10000-7fa85bb11000 rw-p 00033000 fe:00 85264                      /usr/lib/x86_64-linux-gnu/libidn.so.11.6.8
7fa85bb11000-7fa85bc92000 r-xp 00000000 fe:00 258617                     /lib/x86_64-linux-gnu/libc-2.13.so
7fa85bc92000-7fa85be92000 ---p 00181000 fe:00 258617                     /lib/x86_64-linux-gnu/libc-2.13.so
7fa85be92000-7fa85be96000 r--p 00181000 fe:00 258617                     /lib/x86_64-linux-gnu/libc-2.13.so
7fa85be96000-7fa85be97000 rw-p 00185000 fe:00 258617                     /lib/x86_64-linux-gnu/libc-2.13.so
7fa85be97000-7fa85be9c000 rw-p 00000000 00:00 0 
7fa85be9c000-7fa85beb3000 r-xp 00000000 fe:00 258653                     /lib/x86_64-linux-gnu/libpthread-2.13.so
7fa85beb3000-7fa85c0b2000 ---p 00017000 fe:00 258653                     /lib/x86_64-linux-gnu/libpthread-2.13.so
7fa85c0b2000-7fa85c0b3000 r--p 00016000 fe:00 258653                     /lib/x86_64-linux-gnu/libpthread-2.13.so
7fa85c0b3000-7fa85c0b4000 rw-p 00017000 fe:00 258653                     /lib/x86_64-linux-gnu/libpthread-2.13.so
7fa85c0b4000-7fa85c0b8000 rw-p 00000000 00:00 0 
7fa85c0b8000-7fa85c0cd000 r-xp 00000000 fe:00 269446                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7fa85c0cd000-7fa85c2cd000 ---p 00015000 fe:00 269446                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7fa85c2cd000-7fa85c2ce000 rw-p 00015000 fe:00 269446                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7fa85c2ce000-7fa85c34f000 r-xp 00000000 fe:00 258285                     /lib/x86_64-linux-gnu/libm-2.13.so
7fa85c34f000-7fa85c54e000 ---p 00081000 fe:00 258285                     /lib/x86_64-linux-gnu/libm-2.13.so
7fa85c54e000-7fa85c54f000 r--p 00080000 fe:00 258285                     /lib/x86_64-linux-gnu/libm-2.13.so
7fa85c54f000-7fa85c550000 rw-p 00081000 fe:00 258285                     /lib/x86_64-linux-gnu/libm-2.13.so
7fa85c550000-7fa85c638000 r-xp 00000000 fe:00 85315                      /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17
7fa85c638000-7fa85c838000 ---p 000e8000 fe:00 85315                      /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17
7fa85c838000-7fa85c840000 r--p 000e8000 fe:00 85315                      /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17
7fa85c840000-7fa85c842000 rw-p 000f0000 fe:00 85315                      /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17
7fa85c842000-7fa85c857000 rw-p 00000000 00:00 0 
7fa85c857000-7fa85c8d7000 r-xp 00000000 fe:00 85381                      /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.26.0
7fa85c8d7000-7fa85cad7000 ---p 00080000 fe:00 85381                      /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.26.0
7fa85cad7000-7fa85cadd000 rw-p 00080000 fe:00 85381                      /usr/lib/x86_64-linux-gnu/libpixman-1.so.0.26.0
7fa85cadd000-7fa85cae4000 r-xp 00000000 fe:00 85294                      /usr/lib/x86_64-linux-gnu/libusbredirparser.so.1.0.0
7fa85cae4000-7fa85cce3000 ---p 00007000 fe:00 85294                      /usr/lib/x86_64-linux-gnu/libusbredirparser.so.1.0.0
7fa85cce3000-7fa85cce4000 r--p 00006000 fe:00 85294                      /usr/lib/x86_64-linux-gnu/libusbredirparser.so.1.0.0
7fa85cce4000-7fa85cce5000 rw-p 00007000 fe:00 85294                      /usr/lib/x86_64-linux-gnu/libusbredirparser.so.1.0.0
7fa85cce5000-7fa85ccfb000 r-xp 00000000 fe:00 269567                     /lib/x86_64-linux-gnu/libusb-1.0.so.0.1.0
7fa85ccfb000-7fa85cefb000 ---p 00016000 fe:00 269567                     /lib/x86_64-linux-gnu/libusb-1.0.so.0.1.0
7fa85cefb000-7fa85cefc000 r--p 00016000 fe:00 269567                     /lib/x86_64-linux-gnu/libusb-1.0.so.0.1.0
7fa85cefc000-7fa85cefd000 rw-p 00017000 fe:00 269567                     /lib/x86_64-linux-gnu/libusb-1.0.so.0.1.0
7fa85cefd000-7fa85d01b000 r-xp 00000000 fe:00 11054                      /usr/lib/x86_64-linux-gnu/libspice-server.so.1.9.0
7fa85d01b000-7fa85d21a000 ---p 0011e000 fe:00 11054                      /usr/lib/x86_64-linux-gnu/libspice-server.so.1.9.0
7fa85d21a000-7fa85d21c000 r--p 0011d000 fe:00 11054                      /usr/lib/x86_64-linux-gnu/libspice-server.so.1.9.0
7fa85d21c000-7fa85d21d000 rw-p 0011f000 fe:00 11054                      /usr/lib/x86_64-linux-gnu/libspice-server.so.1.9.0
7fa85d21d000-7fa85d22b000 rw-p 00000000 00:00 0 
7fa85d22b000-7fa85d258000 r-xp 00000000 fe:00 20863                      /usr/lib/libxenguest.so.4.5.0
7fa85d258000-7fa85d457000 ---p 0002d000 fe:00 20863                      /usr/lib/libxenguest.so.4.5.0
7fa85d457000-7fa85d458000 rw-p 0002c000 fe:00 20863                      /usr/lib/libxenguest.so.4.5.0
7fa85d458000-7fa85d481000 r-xp 00000000 fe:00 20840                      /usr/lib/libxenctrl.so.4.5.0
7fa85d481000-7fa85d681000 ---p 00029000 fe:00 20840                      /usr/lib/libxenctrl.so.4.5.0
7fa85d681000-7fa85d682000 rw-p 00029000 fe:00 20840                      /usr/lib/libxenctrl.so.4.5.0
7fa85d682000-7fa85d688000 r-xp 00000000 fe:00 20888                      /usr/lib/libxenstore.so.3.0.3
7fa85d688000-7fa85d887000 ---p 00006000 fe:00 20888                      /usr/lib/libxenstore.so.3.0.3
7fa85d887000-7fa85d888000 rw-p 00005000 fe:00 20888                      /usr/lib/libxenstore.so.3.0.3
7fa85d888000-7fa85d88b000 rw-p 00000000 00:00 0 
7fa85d88b000-7fa85d9c0000 r-xp 00000000 fe:00 2377                       /usr/lib/x86_64-linux-gnu/libX11.so.6.3.0
7fa85d9c0000-7fa85dbc0000 ---p 00135000 fe:00 2377                       /usr/lib/x86_64-linux-gnu/libX11.so.6.3.0
7fa85dbc0000-7fa85dbc6000 rw-p 00135000 fe:00 2377                       /usr/lib/x86_64-linux-gnu/libX11.so.6.3.0
7fa85dbc6000-7fa85dc32000 r-xp 00000000 fe:00 85332                      /usr/lib/x86_64-linux-gnu/libSDL-1.2.so.0.11.4
7fa85dc32000-7fa85de32000 ---p 0006c000 fe:00 85332                      /usr/lib/x86_64-linux-gnu/libSDL-1.2.so.0.11.4
7fa85de32000-7fa85de33000 r--p 0006c000 fe:00 85332                      /usr/lib/x86_64-linux-gnu/libSDL-1.2.so.0.11.4
7fa85de33000-7fa85de35000 rw-p 0006d000 fe:00 85332                      /usr/lib/x86_64-linux-gnu/libSDL-1.2.so.0.11.4
7fa85de35000-7fa85de62000 rw-p 00000000 00:00 0 
7fa85de62000-7fa85df1b000 r-xp 00000000 fe:00 11464                      /usr/lib/x86_64-linux-gnu/libgnutls.so.26.22.4
7fa85df1b000-7fa85e11a000 ---p 000b9000 fe:00 11464                      /usr/lib/x86_64-linux-gnu/libgnutls.so.26.22.4
7fa85e11a000-7fa85e120000 r--p 000b8000 fe:00 11464                      /usr/lib/x86_64-linux-gnu/libgnutls.so.26.22.4
7fa85e120000-7fa85e122000 rw-p 000be000 fe:00 11464                      /usr/lib/x86_64-linux-gnu/libgnutls.so.26.22.4
7fa85e122000-7fa85e13c000 r-xp 00000000 fe:00 85121                      /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25
7fa85e13c000-7fa85e33b000 ---p 0001a000 fe:00 85121                      /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25
7fa85e33b000-7fa85e33c000 r--p 00019000 fe:00 85121                      /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25
7fa85e33c000-7fa85e33d000 rw-p 0001a000 fe:00 85121                      /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25
7fa85e33d000-7fa85e381000 r-xp 00000000 fe:00 85726                      /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2
7fa85e381000-7fa85e580000 ---p 00044000 fe:00 85726                      /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2
7fa85e580000-7fa85e581000 r--p 00043000 fe:00 85726                      /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2
7fa85e581000-7fa85e582000 rw-p 00044000 fe:00 85726                      /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2
7fa85e582000-7fa85e592000 rw-p 00000000 00:00 0 
7fa85e592000-7fa85e5b8000 r-xp 00000000 fe:00 269455                     /lib/x86_64-linux-gnu/libpng12.so.0.49.0
7fa85e5b8000-7fa85e7b7000 ---p 00026000 fe:00 269455                     /lib/x86_64-linux-gnu/libpng12.so.0.49.0
7fa85e7b7000-7fa85e7b8000 r--p 00025000 fe:00 269455                     /lib/x86_64-linux-gnu/libpng12.so.0.49.0
7fa85e7b8000-7fa85e7b9000 rw-p 00026000 fe:00 269455                     /lib/x86_64-linux-gnu/libpng12.so.0.49.0
7fa85e7b9000-7fa85e7bd000 r-xp 00000000 fe:00 269543                     /lib/x86_64-linux-gnu/libuuid.so.1.3.0
7fa85e7bd000-7fa85e9bc000 ---p 00004000 fe:00 269543                     /lib/x86_64-linux-gnu/libuuid.so.1.3.0
7fa85e9bc000-7fa85e9bd000 r--p 00003000 fe:00 269543                     /lib/x86_64-linux-gnu/libuuid.so.1.3.0
7fa85e9bd000-7fa85e9be000 rw-p 00004000 fe:00 269543                     /lib/x86_64-linux-gnu/libuuid.so.1.3.0
7fa85e9be000-7fa85e9e3000 r-xp 00000000 fe:00 269539                     /lib/x86_64-linux-gnu/libtinfo.so.5.9
7fa85e9e3000-7fa85ebe2000 ---p 00025000 fe:00 269539                     /lib/x86_64-linux-gnu/libtinfo.so.5.9
7fa85ebe2000-7fa85ebe6000 r--p 00024000 fe:00 269539                     /lib/x86_64-linux-gnu/libtinfo.so.5.9
7fa85ebe6000-7fa85ebe7000 rw-p 00028000 fe:00 269539                     /lib/x86_64-linux-gnu/libtinfo.so.5.9
7fa85ebe7000-7fa85ec08000 r-xp 00000000 fe:00 269458                     /lib/x86_64-linux-gnu/libncurses.so.5.9
7fa85ec08000-7fa85ee07000 ---p 00021000 fe:00 269458                     /lib/x86_64-linux-gnu/libncurses.so.5.9
7fa85ee07000-7fa85ee08000 r--p 00020000 fe:00 269458                     /lib/x86_64-linux-gnu/libncurses.so.5.9
7fa85ee08000-7fa85ee09000 rw-p 00021000 fe:00 269458                     /lib/x86_64-linux-gnu/libncurses.so.5.9
7fa85ee09000-7fa85ee0b000 r-xp 00000000 fe:00 258625                     /lib/x86_64-linux-gnu/libutil-2.13.so
7fa85ee0b000-7fa85f00a000 ---p 00002000 fe:00 258625                     /lib/x86_64-linux-gnu/libutil-2.13.so
7fa85f00a000-7fa85f00b000 r--p 00001000 fe:00 258625                     /lib/x86_64-linux-gnu/libutil-2.13.so
7fa85f00b000-7fa85f00c000 rw-p 00002000 fe:00 258625                     /lib/x86_64-linux-gnu/libutil-2.13.so
7fa85f00c000-7fa85f101000 r-xp 00000000 fe:00 269472                     /lib/x86_64-linux-gnu/libglib-2.0.so.0.3200.4
7fa85f101000-7fa85f301000 ---p 000f5000 fe:00 269472                     /lib/x86_64-linux-gnu/libglib-2.0.so.0.3200.4
7fa85f301000-7fa85f302000 r--p 000f5000 fe:00 269472                     /lib/x86_64-linux-gnu/libglib-2.0.so.0.3200.4
7fa85f302000-7fa85f303000 rw-p 000f6000 fe:00 269472                     /lib/x86_64-linux-gnu/libglib-2.0.so.0.3200.4
7fa85f303000-7fa85f304000 rw-p 00000000 00:00 0 
7fa85f304000-7fa85f305000 r-xp 00000000 fe:00 85942                      /usr/lib/x86_64-linux-gnu/libgthread-2.0.so.0.3200.4
7fa85f305000-7fa85f504000 ---p 00001000 fe:00 85942                      /usr/lib/x86_64-linux-gnu/libgthread-2.0.so.0.3200.4
7fa85f504000-7fa85f505000 r--p 00000000 fe:00 85942                      /usr/lib/x86_64-linux-gnu/libgthread-2.0.so.0.3200.4
7fa85f505000-7fa85f506000 rw-p 00001000 fe:00 85942                      /usr/lib/x86_64-linux-gnu/libgthread-2.0.so.0.3200.4
7fa85f506000-7fa85f50d000 r-xp 00000000 fe:00 258652                     /lib/x86_64-linux-gnu/librt-2.13.so
7fa85f50d000-7fa85f70c000 ---p 00007000 fe:00 258652                     /lib/x86_64-linux-gnu/librt-2.13.so
7fa85f70c000-7fa85f70d000 r--p 00006000 fe:00 258652                     /lib/x86_64-linux-gnu/librt-2.13.so
7fa85f70d000-7fa85f70e000 rw-p 00007000 fe:00 258652                     /lib/x86_64-linux-gnu/librt-2.13.so
7fa85f70e000-7fa85f724000 r-xp 00000000 fe:00 269535                     /lib/x86_64-linux-gnu/libz.so.1.2.7
7fa85f724000-7fa85f923000 ---p 00016000 fe:00 269535                     /lib/x86_64-linux-gnu/libz.so.1.2.7
7fa85f923000-7fa85f924000 r--p 00015000 fe:00 269535                     /lib/x86_64-linux-gnu/libz.so.1.2.7
7fa85f924000-7fa85f925000 rw-p 00016000 fe:00 269535                     /lib/x86_64-linux-gnu/libz.so.1.2.7
7fa85f925000-7fa85f94c000 r-xp 00000000 fe:00 1795                       /usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1
7fa85f94c000-7fa85fb4c000 ---p 00027000 fe:00 1795                       /usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1
7fa85fb4c000-7fa85fb4d000 r--p 00027000 fe:00 1795                       /usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1
7fa85fb4d000-7fa85fb4e000 rw-p 00028000 fe:00 1795                       /usr/lib/x86_64-linux-gnu/libssh2.so.1.0.1
7fa85fb4e000-7fa85fbc9000 r-xp 00000000 fe:00 269893                     /lib/x86_64-linux-gnu/libgcrypt.so.11.7.0
7fa85fbc9000-7fa85fdc9000 ---p 0007b000 fe:00 269893                     /lib/x86_64-linux-gnu/libgcrypt.so.11.7.0
7fa85fdc9000-7fa85fdca000 r--p 0007b000 fe:00 269893                     /lib/x86_64-linux-gnu/libgcrypt.so.11.7.0
7fa85fdca000-7fa85fdcd000 rw-p 0007c000 fe:00 269893                     /lib/x86_64-linux-gnu/libgcrypt.so.11.7.0
7fa85fdcd000-7fa85fe33000 r-xp 00000000 fe:00 20515                      /usr/lib/x86_64-linux-gnu/libcurl.so.4.2.0
7fa85fe33000-7fa860032000 ---p 00066000 fe:00 20515                      /usr/lib/x86_64-linux-gnu/libcurl.so.4.2.0
7fa860032000-7fa860035000 r--p 00065000 fe:00 20515                      /usr/lib/x86_64-linux-gnu/libcurl.so.4.2.0
7fa860035000-7fa860036000 rw-p 00068000 fe:00 20515                      /usr/lib/x86_64-linux-gnu/libcurl.so.4.2.0
7fa860036000-7fa860037000 r-xp 00000000 fe:00 269613                     /lib/x86_64-linux-gnu/libaio.so.1.0.1
7fa860037000-7fa860236000 ---p 00001000 fe:00 269613                     /lib/x86_64-linux-gnu/libaio.so.1.0.1
7fa860236000-7fa860237000 r--p 00000000 fe:00 269613                     /lib/x86_64-linux-gnu/libaio.so.1.0.1
7fa860237000-7fa860238000 rw-p 00001000 fe:00 269613                     /lib/x86_64-linux-gnu/libaio.so.1.0.1
7fa860238000-7fa860258000 r-xp 00000000 fe:00 258647                     /lib/x86_64-linux-gnu/ld-2.13.so
7fa8602ab000-7fa86036a000 rw-p 00000000 00:00 0 
7fa86036a000-7fa860431000 r-xp 00000000 fe:00 85784                      /usr/lib/x86_64-linux-gnu/libcaca.so.0.99.18
7fa860431000-7fa860433000 rw-p 000c6000 fe:00 85784                      /usr/lib/x86_64-linux-gnu/libcaca.so.0.99.18
7fa860433000-7fa860448000 rw-p 00000000 00:00 0 
7fa86044a000-7fa86044b000 -w-s 00001000 00:05 14843                      /dev/xen/gntdev
7fa86044b000-7fa86044c000 rw-s 00000000 00:05 14843                      /dev/xen/gntdev
7fa86044c000-7fa86044d000 rw-p 00000000 00:00 0 
7fa86044d000-7fa86044e000 ---p 00000000 00:00 0 
7fa86044e000-7fa860451000 rw-p 00000000 00:00 0                          [stack:3658]
7fa860451000-7fa860452000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa860452000-7fa860453000 rw-s 00000000 00:16 4                          /proc/xen/privcmd
7fa860453000-7fa860454000 rw-p 00000000 00:00 0 
7fa860454000-7fa860455000 rw-s 00000000 00:11 13783                      /run/shm/spice.3652
7fa860455000-7fa860457000 rw-p 00000000 00:00 0 
7fa860457000-7fa860458000 r--p 0001f000 fe:00 258647                     /lib/x86_64-linux-gnu/ld-2.13.so
7fa860458000-7fa860459000 rw-p 00020000 fe:00 258647                     /lib/x86_64-linux-gnu/ld-2.13.so
7fa860459000-7fa86045a000 rw-p 00000000 00:00 0 
7fa86045a000-7fa8609ff000 r-xp 00000000 fe:00 20995                      /usr/lib/xen/bin/qemu-system-i386
7fa860bfe000-7fa860cc6000 r--p 005a4000 fe:00 20995                      /usr/lib/xen/bin/qemu-system-i386
7fa860cc6000-7fa860d32000 rw-p 0066c000 fe:00 20995                      /usr/lib/xen/bin/qemu-system-i386
7fa860d32000-7fa8611bf000 rw-p 00000000 00:00 0 
7fa861ab6000-7fa864162000 rw-p 00000000 00:00 0                          [heap]
7fff4f3d5000-7fff4f3f6000 rw-p 00000000 00:00 0                          [stack]
7fff4f3fa000-7fff4f3fc000 r-xp 00000000 00:00 0                          [vdso]
7fff4f3fc000-7fff4f3fe000 r--p 00000000 00:00 0                          [vvar]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: [Qemu-devel] Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-05-12 10:26           ` Fabio Fantoni
  2015-05-12 13:54             ` Fabio Fantoni
@ 2015-05-12 13:54             ` Fabio Fantoni
  2015-05-12 14:38               ` Stefano Stabellini
  2015-05-12 14:38               ` [Qemu-devel] " Stefano Stabellini
  1 sibling, 2 replies; 30+ messages in thread
From: Fabio Fantoni @ 2015-05-12 13:54 UTC (permalink / raw)
  To: Stefano Stabellini
  Cc: qemu-devel, xen-devel, Gerd Hoffmann, Jan Beulich,
	Anthony PERARD, spice-devel

[-- Attachment #1: Type: text/plain, Size: 4609 bytes --]

Il 12/05/2015 12:26, Fabio Fantoni ha scritto:
> Il 12/05/2015 11:23, Fabio Fantoni ha scritto:
>> Il 11/05/2015 17:04, Fabio Fantoni ha scritto:
>>> Il 21/04/2015 14:53, Stefano Stabellini ha scritto:
>>>> On Tue, 21 Apr 2015, Fabio Fantoni wrote:
>>>>> Il 21/04/2015 12:49, Stefano Stabellini ha scritto:
>>>>>> On Mon, 20 Apr 2015, Fabio Fantoni wrote:
>>>>>>> I updated xen and qemu from xen 4.5.0 with its upstream qemu 
>>>>>>> included to
>>>>>>> xen
>>>>>>> 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk 
>>>>>>> to use
>>>>>>> revision "master").
>>>>>>> After few minutes I booted windows 7 64 bit domU qemu crash, 
>>>>>>> tried 2 times
>>>>>>> with same result.
>>>>>>>
>>>>>>> In the domU's qemu log:
>>>>>>>> qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion `(old_top ==
>>>>>>>> (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) -
>>>>>>>> __builtin_offsetof
>>>>>>>> (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long)
>>>>>>>> (old_size) >= (unsigned long)((((__builtin_offsetof (struct
>>>>>>>> malloc_chunk,
>>>>>>>> fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * 
>>>>>>>> (sizeof(size_t))) -
>>>>>>>> 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & 
>>>>>>>> pagemask)
>>>>>>>> ==
>>>>>>>> 0)' failed.
>>>>>>>> Killing all inferiors
>>>>>>> In attachment the full backtrace of qemu crash.
>>>>>>>
>>>>>>> With a fast search after I saw the backtrace I found a probable 
>>>>>>> cause of
>>>>>>> regression (I'm not sure):
>>>>>>> http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa 
>>>>>>>
>>>>>>> spice: make sure we don't overflow ssd->buf
>>>>>>>
>>>>>>> Added also qemu-devel and spice-devel as cc.
>>>>>>>
>>>>>>> If you need more informations/tests tell me and I'll post them.
>>>>>>    Maybe you could try to revert the offending commit
>>>>>> (5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better bisect 
>>>>>> the
>>>>>> crash?
>>>>> Thanks for your reply.
>>>>>
>>>>> I reverted to 4.5.0 on dom0 for now on that system because I'm 
>>>>> busy trying to
>>>>> found another problem that cause very bad performance without 
>>>>> errors or
>>>>> nothing in logs :( I don't know if if xen related, kernel related 
>>>>> or other for
>>>>> now.
>>>>>
>>>>> About this regression with spice I'll do further tests in next 
>>>>> days (probably
>>>>> starting reverting the spice patch in qemu) but any help is 
>>>>> appreciated.
>>>>> Based on data I have for now is possible that the problem is that 
>>>>> qemu try to
>>>>> allocate other ram or videoram after domU create but with xen is 
>>>>> not possible?
>>>>> In the spice related patch I saw something about dynamic 
>>>>> allocation for
>>>>> example.
>>>> It is probably caused by a commit in the range:
>>>>
>>>> 1ebb75b1fee779621b63e84fefa7b07354c43a99..0b8fb1ec3d666d1eb8bbff56c76c5e6daa2789e4 
>>>>
>>>>
>>>> there are only 10 commits in that range. By using git bisect you 
>>>> should
>>>> be able to narrow it down in just 3 tests.
>>>
>>> Sorry for delay, I was busy with many things, today I retried with 
>>> updated stable-4.5 and also reverting "spice: make sure we don't 
>>> overflow ssd->buf" (in a second test) but in both case regression 
>>> remain :(
>>> Tomorrow probably I'll do other tests.
>>
>> I did another test, reverting this instead:
>> http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commit;h=c9ac5f816bf3a8b56f836b078711dcef6e5c90b8 
>>
>> And now seems I'm unable to reproduce the regression, before happen 
>> after few seconds up to 1-2 minutes, now I use the same domU 15-20 
>> minutes without problem.
>> Probably is the cause of regression even if seems strange that on 
>> unstable with same patch on tests of some days ago didn't happen.
>>
>> Any ideas?
>>
>> Thanks for any reply and sorry for my bad english.
>
> Bad news, qemu crash still happen even if this time in qemu log there 
> is another output, see attachment.
> After take a look on the other patches I saw:
> http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commitdiff;h=7154fba0e51ec985ef621965d1b7120ad424fcbf 
>
> With "Conflicts: hw/display/vga.c" in description I'll try to revert 
> it instead.
>
> Or someone can tell me another probable test I can try?

Tried also to revet the patch above with same result, so I retried with 
qemu from 4.5.0 and seems the crash happen also in this case...I'm going 
crazy :(

In attachment full gdb log.

Any ideas on how to found the problem please?

Thanks for any reply and sorry for my bad english.

[-- Attachment #2: qemu-xen-gdb.log --]
[-- Type: text/plain, Size: 18174 bytes --]

Full backtrace:
#0  0x00007ffff36e8165 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
        pid = <optimized out>
        selftid = <optimized out>
#1  0x00007ffff36eb3e0 in *__GI_abort () at abort.c:92
        act = {__sigaction_handler = {sa_handler = 0x555558ddeba0, sa_sigaction = 0x555558ddeba0}, sa_mask = {__val = {140737278660816, 140737014337136, 4, 140737014337376, 140737277706678, 206158430256, 140737014337416, 140737014337168, 87, 226653584, 140737351936019, 140737488348083, 140737278647399, 140737278651152, 3096, 140737277299604}}, sa_flags = -474017696, sa_restorer = 0x7ffff36b9c60}
        sigs = {__val = {32, 0 <repeats 15 times>}}
#2  0x00007ffff372bdea in __malloc_assert (assertion=<optimized out>, file=<optimized out>, line=<optimized out>, function=<optimized out>) at malloc.c:351
No locals.
#3  0x00007ffff372ed13 in sYSMALLOc (av=<optimized out>, nb=<optimized out>) at malloc.c:3093
        snd_brk = <optimized out>
        front_misalign = <optimized out>
        remainder = <optimized out>
        tried_mmap = false
        old_size = <optimized out>
        size = <optimized out>
        old_end = 0x555558ddeba0 ""
        correction = <optimized out>
        end_misalign = <optimized out>
        aligned_brk = <optimized out>
        p = <optimized out>
        pagemask = 4095
#4  _int_malloc (av=0x7ffff3a3ce40, bytes=<optimized out>) at malloc.c:4776
        p = <optimized out>
        iters = <optimized out>
        nb = 4196368
        idx = <optimized out>
        bin = <optimized out>
        victim = 0x555558ddeba0
        size = 0
        victim_index = <optimized out>
        remainder = <optimized out>
        remainder_size = <optimized out>
        block = 4
        bit = <optimized out>
        map = 2138988542
        fwd = <optimized out>
        bck = <optimized out>
        errstr = <optimized out>
        __func__ = "_int_malloc"
#5  0x00007ffff3730a70 in *__GI___libc_malloc (bytes=4196352) at malloc.c:3660
        ar_ptr = 0x7ffff3a3ce40
        victim = 0x400800
        __func__ = "__libc_malloc"
#6  0x00007ffff4b1d280 in spice_malloc (n_bytes=4196352) at mem.c:93
        mem = <optimized out>
        __FUNCTION__ = "spice_malloc"
#7  0x00007ffff4b1d6ee in spice_chunks_linearize (chunks=0x7fffdc088fa0) at mem.c:226
        data = <optimized out>
        p = <optimized out>
        i = <optimized out>
#8  0x00007ffff4afb4e6 in canvas_bitmap_to_surface (canvas=canvas@entry=0x7fffdc0eef70, bitmap=bitmap@entry=0x7fffdc044e88, palette=0x0, want_original=1) at ../spice-common/common/canvas_base.c:738
        src = <optimized out>
        image = <optimized out>
        format = <optimized out>
        __FUNCTION__ = "canvas_bitmap_to_surface"
#9  0x00007ffff4afb698 in canvas_get_bits (want_original=<optimized out>, bitmap=0x7fffdc044e88, canvas=0x7fffdc0eef70) at ../spice-common/common/canvas_base.c:1067
        palette = <optimized out>
#10 canvas_get_image_internal (canvas=canvas@entry=0x7fffdc0eef70, image=0x7fffdc044e70, want_original=<optimized out>, want_original@entry=0, real_get=real_get@entry=1) at ../spice-common/common/canvas_base.c:1253
        descriptor = 0x7fffdc044e70
        surface = <optimized out>
        converted = <optimized out>
        wanted_format = 3691966320
        surface_format = <optimized out>
        saved_want_original = <optimized out>
        __FUNCTION__ = "canvas_get_image_internal"
#11 0x00007ffff4afc0da in canvas_get_image (canvas=canvas@entry=0x7fffdc0eef70, image=<optimized out>, want_original=want_original@entry=0) at ../spice-common/common/canvas_base.c:1397
No locals.
#12 0x00007ffff4afe42e in canvas_draw_copy (spice_canvas=0x7fffdc0eef70, bbox=0x7fffdc0e9c80, clip=<optimized out>, copy=0x7fffe3bf1320) at ../spice-common/common/canvas_base.c:2370
        canvas = 0x7fffdc0eef70
        dest_region = {extents = {x1 = 5, y1 = 5, x2 = 161, y2 = 33}, data = 0x0}
        surface_canvas = <optimized out>
        src_image = <optimized out>
        rop = SPICE_ROP_COPY
        __FUNCTION__ = "canvas_draw_copy"
#13 0x00007ffff4ad120c in red_draw_qxl_drawable (worker=worker@entry=0x7fffe3218010, drawable=drawable@entry=0x7fffe33d8a58) at red_worker.c:4421
        copy = {src_bitmap = 0x7fffdc044e70, src_area = {left = 5, top = 5, right = 161, bottom = 33}, rop_descriptor = 8, scale_mode = 1 '\001', mask = {flags = 194 '\302', pos = {x = -1027423550, y = -1027423550}, bitmap = 0x0}}
        img1 = {descriptor = {id = 140736884378864, type = 80 'P', flags = 21 '\025', width = 32767, height = 4294902015}, u = {bitmap = {format = 112 'p', flags = 10 '\n', x = 32767, y = 3865334188, stride = 32767, palette = 0x7fffe66451c4, palette_id = 138783, data = 0x0}, quic = {data_size = 4084402800, data = 0x7fffe66451ac}, surface = {surface_id = 4084402800}, lz_rgb = {data_size = 4084402800, data = 0x7fffe66451ac}, lz_plt = {flags = 112 'p', data_size = 32767, palette = 0x7fffe66451ac, palette_id = 140737058722244, data = 0x21e1f}, jpeg = {data_size = 4084402800, data = 0x7fffe66451ac}, lz4 = {data_size = 4084402800, data = 0x7fffe66451ac}, zlib_glz = {glz_data_size = 4084402800, data_size = 32767, data = 0x7fffe66451ac}, jpeg_alpha = {flags = 112 'p', jpeg_size = 32767, data_size = 3865334188, data = 0x7fffe66451c4}}}
        img2 = {descriptor = {id = 140737058722236, type = 196 '\304', flags = 81 'Q', width = 32767, height = 3786207248}, u = {bitmap = {format = 53 '5', flags = 156 '\234', x = 32767, y = 22176, stride = 0, palette = 0x5555566fc920, palette_id = 140737058708356, data = 0x539}, quic = {data_size = 4104887349, data = 0x56a0}, surface = {surface_id = 4104887349}, lz_rgb = {data_size = 4104887349, data = 0x56a0}, lz_plt = {flags = 53 '5', data_size = 32767, palette = 0x56a0, palette_id = 93825010747680, data = 0x7fffe6641b84}, jpeg = {data_size = 4104887349, data = 0x56a0}, lz4 = {data_size = 4104887349, data = 0x56a0}, zlib_glz = {glz_data_size = 4104887349, data_size = 32767, data = 0x56a0}, jpeg_alpha = {flags = 53 '5', jpeg_size = 32767, data_size = 22176, data = 0x5555566fc920}}}
        surface = 0x7fffe32182f0
        canvas = 0x7fffdc0eef70
        clip = {type = 0 '\000', rects = 0x0}
        __FUNCTION__ = "red_draw_qxl_drawable"
#14 0x00007ffff4adfad5 in red_draw_drawable (drawable=0x7fffe33d8a58, worker=0x7fffe3218010) at red_worker.c:4534
No locals.
#15 red_update_area (worker=worker@entry=0x7fffe3218010, area=area@entry=0x7fffe3bf1b60, surface_id=surface_id@entry=0) at red_worker.c:4787
        container = <optimized out>
        surface = 0x7fffe32182f0
        ring = 0x7fffe3218308
        ring_item = <optimized out>
        rgn = {extents = {x1 = 0, y1 = 0, x2 = 1366, y2 = 768}, data = 0x0}
        last = 0x7fffe33c9cd8
        now = 0x7fffe33d8a58
        __FUNCTION__ = "red_update_area"
#16 0x00007ffff4ae9644 in handle_dev_update_async (opaque=0x7fffe3218010, payload=<optimized out>) at red_worker.c:10992
        worker = 0x7fffe3218010
        msg = <optimized out>
        rect = {left = 0, top = 0, right = 1366, bottom = 768}
        qxl_dirty_rects = <optimized out>
        num_dirty_rects = <optimized out>
        surface = <optimized out>
        surface_id = 0
        qxl_area = {top = 0, left = 0, bottom = 768, right = 1366}
        clear_dirty_region = 1
        __FUNCTION__ = "handle_dev_update_async"
        __func__ = "handle_dev_update_async"
#17 0x00007ffff4ac85d4 in dispatcher_handle_single_read (dispatcher=0x555556415e28) at dispatcher.c:139
        ret = <optimized out>
        type = <optimized out>
        msg = 0x5555563cd330
        ack = 4294967295
        payload = 0x5555563daf20 "P\177=VUU"
#18 dispatcher_handle_recv_read (dispatcher=0x555556415e28) at dispatcher.c:162
No locals.
#19 0x00007ffff4aebc5c in red_worker_main (arg=<optimized out>) at red_worker.c:12175
        events = <optimized out>
        i = <optimized out>
        num_events = 1
        timers_queue_timeout = <optimized out>
        worker = 0x7fffe3218010
        __FUNCTION__ = "red_worker_main"
#20 0x00007ffff3a47b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
        __res = <optimized out>
        pd = 0x7fffe3bf2700
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737014343424, -7079689987510599981, 140737281073696, 140737014344128, 140737354125376, 3, 7079698237168975571, 7079663285551136467}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#21 0x00007ffff379195d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
No locals.
#22 0x0000000000000000 in ?? ()
No symbol table info available.


Registers:
rax            0x0	0
rbx            0x7ffff3a3ce98	140737280986776
rcx            0xffffffffffffffff	-1
rdx            0x6	6
rsi            0x1067	4199
rdi            0x1053	4179
rbp            0x7ffff3a3ce40	0x7ffff3a3ce40
rsp            0x7fffe3bf0e38	0x7fffe3bf0e38
r8             0x7fffe3bf2700	140737014343424
r9             0x656e6769736e7528	7308892947874739496
r10            0x8	8
r11            0x206	518
r12            0x555558ddeba0	93825051519904
r13            0x0	0
r14            0x7f7e5ffe	2138988542
r15            0x4	4
rip            0x7ffff36e8165	0x7ffff36e8165 <*__GI_raise+53>
eflags         0x206	[ PF IF ]
cs             0x33	51
ss             0x2b	43
ds             0x0	0
es             0x0	0
fs             0x0	0
gs             0x0	0


Current instructions:
=> 0x7ffff36e8165 <*__GI_raise+53>:	cmp    $0xfffffffffffff000,%rax
   0x7ffff36e816b <*__GI_raise+59>:	ja     0x7ffff36e8182 <*__GI_raise+82>
   0x7ffff36e816d <*__GI_raise+61>:	repz retq 
   0x7ffff36e816f <*__GI_raise+63>:	nop
   0x7ffff36e8170 <*__GI_raise+64>:	test   %eax,%eax
   0x7ffff36e8172 <*__GI_raise+66>:	jg     0x7ffff36e8155 <*__GI_raise+37>
   0x7ffff36e8174 <*__GI_raise+68>:	test   $0x7fffffff,%eax
   0x7ffff36e8179 <*__GI_raise+73>:	jne    0x7ffff36e8192 <*__GI_raise+98>
   0x7ffff36e817b <*__GI_raise+75>:	mov    %esi,%eax
   0x7ffff36e817d <*__GI_raise+77>:	nopl   (%rax)
   0x7ffff36e8180 <*__GI_raise+80>:	jmp    0x7ffff36e8155 <*__GI_raise+37>
   0x7ffff36e8182 <*__GI_raise+82>:	mov    0x352c8f(%rip),%rdx        # 0x7ffff3a3ae18
   0x7ffff36e8189 <*__GI_raise+89>:	neg    %eax
   0x7ffff36e818b <*__GI_raise+91>:	mov    %eax,%fs:(%rdx)
   0x7ffff36e818e <*__GI_raise+94>:	or     $0xffffffff,%eax
   0x7ffff36e8191 <*__GI_raise+97>:	retq   


Threads backtrace:

Thread 9 (Thread 0x7fffdbfff700 (LWP 4313)):
#0  sem_timedwait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_timedwait.S:106
#1  0x00005555559b38cc in qemu_sem_timedwait (sem=0x5555563dcea8, ms=10000) at util/qemu-thread-posix.c:257
#2  0x0000555555849892 in worker_thread (opaque=0x5555563dce10) at thread-pool.c:97
#3  0x00007ffff3a47b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#4  0x00007ffff379195d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#5  0x0000000000000000 in ?? ()

Thread 8 (Thread 0x7fffe0acb700 (LWP 4312)):
#0  sem_timedwait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_timedwait.S:106
#1  0x00005555559b38cc in qemu_sem_timedwait (sem=0x5555563dcea8, ms=10000) at util/qemu-thread-posix.c:257
#2  0x0000555555849892 in worker_thread (opaque=0x5555563dce10) at thread-pool.c:97
#3  0x00007ffff3a47b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#4  0x00007ffff379195d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#5  0x0000000000000000 in ?? ()

Thread 7 (Thread 0x7fffe3017700 (LWP 4200)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162
#1  0x00005555559b360a in qemu_cond_wait (cond=0x5555564202a0, mutex=0x5555564202d0) at util/qemu-thread-posix.c:135
#2  0x00005555558731d6 in vnc_worker_thread_loop (queue=0x5555564202a0) at ui/vnc-jobs.c:222
#3  0x0000555555873739 in vnc_worker_thread (arg=0x5555564202a0) at ui/vnc-jobs.c:323
#4  0x00007ffff3a47b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#5  0x00007ffff379195d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#6  0x0000000000000000 in ?? ()

Thread 6 (Thread 0x7fffe3bf2700 (LWP 4199)):
#0  0x00007ffff36e8165 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1  0x00007ffff36eb3e0 in *__GI_abort () at abort.c:92
#2  0x00007ffff372bdea in __malloc_assert (assertion=<optimized out>, file=<optimized out>, line=<optimized out>, function=<optimized out>) at malloc.c:351
#3  0x00007ffff372ed13 in sYSMALLOc (av=<optimized out>, nb=<optimized out>) at malloc.c:3093
#4  _int_malloc (av=0x7ffff3a3ce40, bytes=<optimized out>) at malloc.c:4776
#5  0x00007ffff3730a70 in *__GI___libc_malloc (bytes=4196352) at malloc.c:3660
#6  0x00007ffff4b1d280 in spice_malloc (n_bytes=4196352) at mem.c:93
#7  0x00007ffff4b1d6ee in spice_chunks_linearize (chunks=0x7fffdc088fa0) at mem.c:226
#8  0x00007ffff4afb4e6 in canvas_bitmap_to_surface (canvas=canvas@entry=0x7fffdc0eef70, bitmap=bitmap@entry=0x7fffdc044e88, palette=0x0, want_original=1) at ../spice-common/common/canvas_base.c:738
#9  0x00007ffff4afb698 in canvas_get_bits (want_original=<optimized out>, bitmap=0x7fffdc044e88, canvas=0x7fffdc0eef70) at ../spice-common/common/canvas_base.c:1067
#10 canvas_get_image_internal (canvas=canvas@entry=0x7fffdc0eef70, image=0x7fffdc044e70, want_original=<optimized out>, want_original@entry=0, real_get=real_get@entry=1) at ../spice-common/common/canvas_base.c:1253
#11 0x00007ffff4afc0da in canvas_get_image (canvas=canvas@entry=0x7fffdc0eef70, image=<optimized out>, want_original=want_original@entry=0) at ../spice-common/common/canvas_base.c:1397
#12 0x00007ffff4afe42e in canvas_draw_copy (spice_canvas=0x7fffdc0eef70, bbox=0x7fffdc0e9c80, clip=<optimized out>, copy=0x7fffe3bf1320) at ../spice-common/common/canvas_base.c:2370
#13 0x00007ffff4ad120c in red_draw_qxl_drawable (worker=worker@entry=0x7fffe3218010, drawable=drawable@entry=0x7fffe33d8a58) at red_worker.c:4421
#14 0x00007ffff4adfad5 in red_draw_drawable (drawable=0x7fffe33d8a58, worker=0x7fffe3218010) at red_worker.c:4534
#15 red_update_area (worker=worker@entry=0x7fffe3218010, area=area@entry=0x7fffe3bf1b60, surface_id=surface_id@entry=0) at red_worker.c:4787
#16 0x00007ffff4ae9644 in handle_dev_update_async (opaque=0x7fffe3218010, payload=<optimized out>) at red_worker.c:10992
#17 0x00007ffff4ac85d4 in dispatcher_handle_single_read (dispatcher=0x555556415e28) at dispatcher.c:139
#18 dispatcher_handle_recv_read (dispatcher=0x555556415e28) at dispatcher.c:162
#19 0x00007ffff4aebc5c in red_worker_main (arg=<optimized out>) at red_worker.c:12175
#20 0x00007ffff3a47b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#21 0x00007ffff379195d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#22 0x0000000000000000 in ?? ()

Thread 5 (Thread 0x7fffe9621700 (LWP 4198)):
#0  sem_timedwait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_timedwait.S:106
#1  0x00005555559b38cc in qemu_sem_timedwait (sem=0x5555563dcea8, ms=10000) at util/qemu-thread-posix.c:257
#2  0x0000555555849892 in worker_thread (opaque=0x5555563dce10) at thread-pool.c:97
#3  0x00007ffff3a47b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#4  0x00007ffff379195d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#5  0x0000000000000000 in ?? ()

Thread 4 (Thread 0x7fffe9f23700 (LWP 4197)):
#0  do_sigwait (set=0x7fffe9f22c50, sig=0x7fffe9f22c40) at ../nptl/sysdeps/unix/sysv/linux/../../../../../sysdeps/unix/sysv/linux/sigwait.c:65
#1  0x00007ffff3a4fe67 in __sigwait (set=<optimized out>, sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/../../../../../sysdeps/unix/sysv/linux/sigwait.c:100
#2  0x0000555555893e0a in qemu_dummy_cpu_thread_fn (arg=0x55555636fa30) at /mnt/raid-vm/RW/source/xen/Xen-stable/tools/qemu-xen-dir/cpus.c:911
#3  0x00007ffff3a47b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#4  0x00007ffff379195d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#5  0x0000000000000000 in ?? ()

Thread 3 (Thread 0x7fffea724700 (LWP 4196)):
#0  do_sigwait (set=0x7fffea723c50, sig=0x7fffea723c40) at ../nptl/sysdeps/unix/sysv/linux/../../../../../sysdeps/unix/sysv/linux/sigwait.c:65
#1  0x00007ffff3a4fe67 in __sigwait (set=<optimized out>, sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/../../../../../sysdeps/unix/sysv/linux/sigwait.c:100
#2  0x0000555555893e0a in qemu_dummy_cpu_thread_fn (arg=0x55555635ecf0) at /mnt/raid-vm/RW/source/xen/Xen-stable/tools/qemu-xen-dir/cpus.c:911
#3  0x00007ffff3a47b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#4  0x00007ffff379195d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#5  0x0000000000000000 in ?? ()

Thread 2 (Thread 0x7ffff7ff1700 (LWP 4184)):
#0  0x00007ffff3a4f1fd in read () at ../sysdeps/unix/syscall-template.S:82
#1  0x00007ffff5229626 in read_all (fd=26, data=0x7fffdc096cc0, data@entry=0x20, len=len@entry=16, nonblocking=nonblocking@entry=0) at xs.c:378
#2  0x00007ffff5229743 in read_message (h=h@entry=0x55555632ee90, nonblocking=nonblocking@entry=0) at xs.c:1150
#3  0x00007ffff522a05e in read_thread (arg=0x55555632ee90) at xs.c:1222
#4  0x00007ffff3a47b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#5  0x00007ffff379195d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#6  0x0000000000000000 in ?? ()

Thread 1 (Thread 0x7ffff7ef6900 (LWP 4179)):
#0  0x00007ffff3786de1 in ppoll (fds=<optimized out>, nfds=<optimized out>, timeout=<optimized out>, sigmask=<optimized out>) at ../sysdeps/unix/sysv/linux/ppoll.c:58
#1  0x0000555555822f96 in qemu_poll_ns (fds=0x5555566fbe00, nfds=19, timeout=63041) at qemu-timer.c:316
#2  0x00005555557d74b0 in os_host_main_loop_wait (timeout=63041) at main-loop.c:229
#3  0x00005555557d7599 in main_loop_wait (nonblocking=0) at main-loop.c:484
#4  0x00005555558830a4 in main_loop () at vl.c:2056
#5  0x000055555588ab5f in main (argc=68, argv=0x7fffffffdf98, envp=0x7fffffffe1c0) at vl.c:4535

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-05-12 10:26           ` Fabio Fantoni
@ 2015-05-12 13:54             ` Fabio Fantoni
  2015-05-12 13:54             ` [Qemu-devel] " Fabio Fantoni
  1 sibling, 0 replies; 30+ messages in thread
From: Fabio Fantoni @ 2015-05-12 13:54 UTC (permalink / raw)
  To: Stefano Stabellini
  Cc: qemu-devel, xen-devel, Gerd Hoffmann, Jan Beulich,
	Anthony PERARD, spice-devel

[-- Attachment #1: Type: text/plain, Size: 4609 bytes --]

Il 12/05/2015 12:26, Fabio Fantoni ha scritto:
> Il 12/05/2015 11:23, Fabio Fantoni ha scritto:
>> Il 11/05/2015 17:04, Fabio Fantoni ha scritto:
>>> Il 21/04/2015 14:53, Stefano Stabellini ha scritto:
>>>> On Tue, 21 Apr 2015, Fabio Fantoni wrote:
>>>>> Il 21/04/2015 12:49, Stefano Stabellini ha scritto:
>>>>>> On Mon, 20 Apr 2015, Fabio Fantoni wrote:
>>>>>>> I updated xen and qemu from xen 4.5.0 with its upstream qemu 
>>>>>>> included to
>>>>>>> xen
>>>>>>> 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk 
>>>>>>> to use
>>>>>>> revision "master").
>>>>>>> After few minutes I booted windows 7 64 bit domU qemu crash, 
>>>>>>> tried 2 times
>>>>>>> with same result.
>>>>>>>
>>>>>>> In the domU's qemu log:
>>>>>>>> qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion `(old_top ==
>>>>>>>> (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) -
>>>>>>>> __builtin_offsetof
>>>>>>>> (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long)
>>>>>>>> (old_size) >= (unsigned long)((((__builtin_offsetof (struct
>>>>>>>> malloc_chunk,
>>>>>>>> fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * 
>>>>>>>> (sizeof(size_t))) -
>>>>>>>> 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & 
>>>>>>>> pagemask)
>>>>>>>> ==
>>>>>>>> 0)' failed.
>>>>>>>> Killing all inferiors
>>>>>>> In attachment the full backtrace of qemu crash.
>>>>>>>
>>>>>>> With a fast search after I saw the backtrace I found a probable 
>>>>>>> cause of
>>>>>>> regression (I'm not sure):
>>>>>>> http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa 
>>>>>>>
>>>>>>> spice: make sure we don't overflow ssd->buf
>>>>>>>
>>>>>>> Added also qemu-devel and spice-devel as cc.
>>>>>>>
>>>>>>> If you need more informations/tests tell me and I'll post them.
>>>>>>    Maybe you could try to revert the offending commit
>>>>>> (5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better bisect 
>>>>>> the
>>>>>> crash?
>>>>> Thanks for your reply.
>>>>>
>>>>> I reverted to 4.5.0 on dom0 for now on that system because I'm 
>>>>> busy trying to
>>>>> found another problem that cause very bad performance without 
>>>>> errors or
>>>>> nothing in logs :( I don't know if if xen related, kernel related 
>>>>> or other for
>>>>> now.
>>>>>
>>>>> About this regression with spice I'll do further tests in next 
>>>>> days (probably
>>>>> starting reverting the spice patch in qemu) but any help is 
>>>>> appreciated.
>>>>> Based on data I have for now is possible that the problem is that 
>>>>> qemu try to
>>>>> allocate other ram or videoram after domU create but with xen is 
>>>>> not possible?
>>>>> In the spice related patch I saw something about dynamic 
>>>>> allocation for
>>>>> example.
>>>> It is probably caused by a commit in the range:
>>>>
>>>> 1ebb75b1fee779621b63e84fefa7b07354c43a99..0b8fb1ec3d666d1eb8bbff56c76c5e6daa2789e4 
>>>>
>>>>
>>>> there are only 10 commits in that range. By using git bisect you 
>>>> should
>>>> be able to narrow it down in just 3 tests.
>>>
>>> Sorry for delay, I was busy with many things, today I retried with 
>>> updated stable-4.5 and also reverting "spice: make sure we don't 
>>> overflow ssd->buf" (in a second test) but in both case regression 
>>> remain :(
>>> Tomorrow probably I'll do other tests.
>>
>> I did another test, reverting this instead:
>> http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commit;h=c9ac5f816bf3a8b56f836b078711dcef6e5c90b8 
>>
>> And now seems I'm unable to reproduce the regression, before happen 
>> after few seconds up to 1-2 minutes, now I use the same domU 15-20 
>> minutes without problem.
>> Probably is the cause of regression even if seems strange that on 
>> unstable with same patch on tests of some days ago didn't happen.
>>
>> Any ideas?
>>
>> Thanks for any reply and sorry for my bad english.
>
> Bad news, qemu crash still happen even if this time in qemu log there 
> is another output, see attachment.
> After take a look on the other patches I saw:
> http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commitdiff;h=7154fba0e51ec985ef621965d1b7120ad424fcbf 
>
> With "Conflicts: hw/display/vga.c" in description I'll try to revert 
> it instead.
>
> Or someone can tell me another probable test I can try?

Tried also to revet the patch above with same result, so I retried with 
qemu from 4.5.0 and seems the crash happen also in this case...I'm going 
crazy :(

In attachment full gdb log.

Any ideas on how to found the problem please?

Thanks for any reply and sorry for my bad english.

[-- Attachment #2: qemu-xen-gdb.log --]
[-- Type: text/plain, Size: 18176 bytes --]

Full backtrace:
#0  0x00007ffff36e8165 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
        pid = <optimized out>
        selftid = <optimized out>
#1  0x00007ffff36eb3e0 in *__GI_abort () at abort.c:92
        act = {__sigaction_handler = {sa_handler = 0x555558ddeba0, sa_sigaction = 0x555558ddeba0}, sa_mask = {__val = {140737278660816, 140737014337136, 4, 140737014337376, 140737277706678, 206158430256, 140737014337416, 140737014337168, 87, 226653584, 140737351936019, 140737488348083, 140737278647399, 140737278651152, 3096, 140737277299604}}, sa_flags = -474017696, sa_restorer = 0x7ffff36b9c60}
        sigs = {__val = {32, 0 <repeats 15 times>}}
#2  0x00007ffff372bdea in __malloc_assert (assertion=<optimized out>, file=<optimized out>, line=<optimized out>, function=<optimized out>) at malloc.c:351
No locals.
#3  0x00007ffff372ed13 in sYSMALLOc (av=<optimized out>, nb=<optimized out>) at malloc.c:3093
        snd_brk = <optimized out>
        front_misalign = <optimized out>
        remainder = <optimized out>
        tried_mmap = false
        old_size = <optimized out>
        size = <optimized out>
        old_end = 0x555558ddeba0 ""
        correction = <optimized out>
        end_misalign = <optimized out>
        aligned_brk = <optimized out>
        p = <optimized out>
        pagemask = 4095
#4  _int_malloc (av=0x7ffff3a3ce40, bytes=<optimized out>) at malloc.c:4776
        p = <optimized out>
        iters = <optimized out>
        nb = 4196368
        idx = <optimized out>
        bin = <optimized out>
        victim = 0x555558ddeba0
        size = 0
        victim_index = <optimized out>
        remainder = <optimized out>
        remainder_size = <optimized out>
        block = 4
        bit = <optimized out>
        map = 2138988542
        fwd = <optimized out>
        bck = <optimized out>
        errstr = <optimized out>
        __func__ = "_int_malloc"
#5  0x00007ffff3730a70 in *__GI___libc_malloc (bytes=4196352) at malloc.c:3660
        ar_ptr = 0x7ffff3a3ce40
        victim = 0x400800
        __func__ = "__libc_malloc"
#6  0x00007ffff4b1d280 in spice_malloc (n_bytes=4196352) at mem.c:93
        mem = <optimized out>
        __FUNCTION__ = "spice_malloc"
#7  0x00007ffff4b1d6ee in spice_chunks_linearize (chunks=0x7fffdc088fa0) at mem.c:226
        data = <optimized out>
        p = <optimized out>
        i = <optimized out>
#8  0x00007ffff4afb4e6 in canvas_bitmap_to_surface (canvas=canvas@entry=0x7fffdc0eef70, bitmap=bitmap@entry=0x7fffdc044e88, palette=0x0, want_original=1) at ../spice-common/common/canvas_base.c:738
        src = <optimized out>
        image = <optimized out>
        format = <optimized out>
        __FUNCTION__ = "canvas_bitmap_to_surface"
#9  0x00007ffff4afb698 in canvas_get_bits (want_original=<optimized out>, bitmap=0x7fffdc044e88, canvas=0x7fffdc0eef70) at ../spice-common/common/canvas_base.c:1067
        palette = <optimized out>
#10 canvas_get_image_internal (canvas=canvas@entry=0x7fffdc0eef70, image=0x7fffdc044e70, want_original=<optimized out>, want_original@entry=0, real_get=real_get@entry=1) at ../spice-common/common/canvas_base.c:1253
        descriptor = 0x7fffdc044e70
        surface = <optimized out>
        converted = <optimized out>
        wanted_format = 3691966320
        surface_format = <optimized out>
        saved_want_original = <optimized out>
        __FUNCTION__ = "canvas_get_image_internal"
#11 0x00007ffff4afc0da in canvas_get_image (canvas=canvas@entry=0x7fffdc0eef70, image=<optimized out>, want_original=want_original@entry=0) at ../spice-common/common/canvas_base.c:1397
No locals.
#12 0x00007ffff4afe42e in canvas_draw_copy (spice_canvas=0x7fffdc0eef70, bbox=0x7fffdc0e9c80, clip=<optimized out>, copy=0x7fffe3bf1320) at ../spice-common/common/canvas_base.c:2370
        canvas = 0x7fffdc0eef70
        dest_region = {extents = {x1 = 5, y1 = 5, x2 = 161, y2 = 33}, data = 0x0}
        surface_canvas = <optimized out>
        src_image = <optimized out>
        rop = SPICE_ROP_COPY
        __FUNCTION__ = "canvas_draw_copy"
#13 0x00007ffff4ad120c in red_draw_qxl_drawable (worker=worker@entry=0x7fffe3218010, drawable=drawable@entry=0x7fffe33d8a58) at red_worker.c:4421
        copy = {src_bitmap = 0x7fffdc044e70, src_area = {left = 5, top = 5, right = 161, bottom = 33}, rop_descriptor = 8, scale_mode = 1 '\001', mask = {flags = 194 '\302', pos = {x = -1027423550, y = -1027423550}, bitmap = 0x0}}
        img1 = {descriptor = {id = 140736884378864, type = 80 'P', flags = 21 '\025', width = 32767, height = 4294902015}, u = {bitmap = {format = 112 'p', flags = 10 '\n', x = 32767, y = 3865334188, stride = 32767, palette = 0x7fffe66451c4, palette_id = 138783, data = 0x0}, quic = {data_size = 4084402800, data = 0x7fffe66451ac}, surface = {surface_id = 4084402800}, lz_rgb = {data_size = 4084402800, data = 0x7fffe66451ac}, lz_plt = {flags = 112 'p', data_size = 32767, palette = 0x7fffe66451ac, palette_id = 140737058722244, data = 0x21e1f}, jpeg = {data_size = 4084402800, data = 0x7fffe66451ac}, lz4 = {data_size = 4084402800, data = 0x7fffe66451ac}, zlib_glz = {glz_data_size = 4084402800, data_size = 32767, data = 0x7fffe66451ac}, jpeg_alpha = {flags = 112 'p', jpeg_size = 32767, data_size 
 = 3865334188, data = 0x7fffe66451c4}}}
        img2 = {descriptor = {id = 140737058722236, type = 196 '\304', flags = 81 'Q', width = 32767, height = 3786207248}, u = {bitmap = {format = 53 '5', flags = 156 '\234', x = 32767, y = 22176, stride = 0, palette = 0x5555566fc920, palette_id = 140737058708356, data = 0x539}, quic = {data_size = 4104887349, data = 0x56a0}, surface = {surface_id = 4104887349}, lz_rgb = {data_size = 4104887349, data = 0x56a0}, lz_plt = {flags = 53 '5', data_size = 32767, palette = 0x56a0, palette_id = 93825010747680, data = 0x7fffe6641b84}, jpeg = {data_size = 4104887349, data = 0x56a0}, lz4 = {data_size = 4104887349, data = 0x56a0}, zlib_glz = {glz_data_size = 4104887349, data_size = 32767, data = 0x56a0}, jpeg_alpha = {flags = 53 '5', jpeg_size = 32767, data_size = 22176, data = 0x5555566fc920}}}
        surface = 0x7fffe32182f0
        canvas = 0x7fffdc0eef70
        clip = {type = 0 '\000', rects = 0x0}
        __FUNCTION__ = "red_draw_qxl_drawable"
#14 0x00007ffff4adfad5 in red_draw_drawable (drawable=0x7fffe33d8a58, worker=0x7fffe3218010) at red_worker.c:4534
No locals.
#15 red_update_area (worker=worker@entry=0x7fffe3218010, area=area@entry=0x7fffe3bf1b60, surface_id=surface_id@entry=0) at red_worker.c:4787
        container = <optimized out>
        surface = 0x7fffe32182f0
        ring = 0x7fffe3218308
        ring_item = <optimized out>
        rgn = {extents = {x1 = 0, y1 = 0, x2 = 1366, y2 = 768}, data = 0x0}
        last = 0x7fffe33c9cd8
        now = 0x7fffe33d8a58
        __FUNCTION__ = "red_update_area"
#16 0x00007ffff4ae9644 in handle_dev_update_async (opaque=0x7fffe3218010, payload=<optimized out>) at red_worker.c:10992
        worker = 0x7fffe3218010
        msg = <optimized out>
        rect = {left = 0, top = 0, right = 1366, bottom = 768}
        qxl_dirty_rects = <optimized out>
        num_dirty_rects = <optimized out>
        surface = <optimized out>
        surface_id = 0
        qxl_area = {top = 0, left = 0, bottom = 768, right = 1366}
        clear_dirty_region = 1
        __FUNCTION__ = "handle_dev_update_async"
        __func__ = "handle_dev_update_async"
#17 0x00007ffff4ac85d4 in dispatcher_handle_single_read (dispatcher=0x555556415e28) at dispatcher.c:139
        ret = <optimized out>
        type = <optimized out>
        msg = 0x5555563cd330
        ack = 4294967295
        payload = 0x5555563daf20 "P\177=VUU"
#18 dispatcher_handle_recv_read (dispatcher=0x555556415e28) at dispatcher.c:162
No locals.
#19 0x00007ffff4aebc5c in red_worker_main (arg=<optimized out>) at red_worker.c:12175
        events = <optimized out>
        i = <optimized out>
        num_events = 1
        timers_queue_timeout = <optimized out>
        worker = 0x7fffe3218010
        __FUNCTION__ = "red_worker_main"
#20 0x00007ffff3a47b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
        __res = <optimized out>
        pd = 0x7fffe3bf2700
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737014343424, -7079689987510599981, 140737281073696, 140737014344128, 140737354125376, 3, 7079698237168975571, 7079663285551136467}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#21 0x00007ffff379195d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
No locals.
#22 0x0000000000000000 in ?? ()
No symbol table info available.


Registers:
rax            0x0	0
rbx            0x7ffff3a3ce98	140737280986776
rcx            0xffffffffffffffff	-1
rdx            0x6	6
rsi            0x1067	4199
rdi            0x1053	4179
rbp            0x7ffff3a3ce40	0x7ffff3a3ce40
rsp            0x7fffe3bf0e38	0x7fffe3bf0e38
r8             0x7fffe3bf2700	140737014343424
r9             0x656e6769736e7528	7308892947874739496
r10            0x8	8
r11            0x206	518
r12            0x555558ddeba0	93825051519904
r13            0x0	0
r14            0x7f7e5ffe	2138988542
r15            0x4	4
rip            0x7ffff36e8165	0x7ffff36e8165 <*__GI_raise+53>
eflags         0x206	[ PF IF ]
cs             0x33	51
ss             0x2b	43
ds             0x0	0
es             0x0	0
fs             0x0	0
gs             0x0	0


Current instructions:
=> 0x7ffff36e8165 <*__GI_raise+53>:	cmp    $0xfffffffffffff000,%rax
   0x7ffff36e816b <*__GI_raise+59>:	ja     0x7ffff36e8182 <*__GI_raise+82>
   0x7ffff36e816d <*__GI_raise+61>:	repz retq 
   0x7ffff36e816f <*__GI_raise+63>:	nop
   0x7ffff36e8170 <*__GI_raise+64>:	test   %eax,%eax
   0x7ffff36e8172 <*__GI_raise+66>:	jg     0x7ffff36e8155 <*__GI_raise+37>
   0x7ffff36e8174 <*__GI_raise+68>:	test   $0x7fffffff,%eax
   0x7ffff36e8179 <*__GI_raise+73>:	jne    0x7ffff36e8192 <*__GI_raise+98>
   0x7ffff36e817b <*__GI_raise+75>:	mov    %esi,%eax
   0x7ffff36e817d <*__GI_raise+77>:	nopl   (%rax)
   0x7ffff36e8180 <*__GI_raise+80>:	jmp    0x7ffff36e8155 <*__GI_raise+37>
   0x7ffff36e8182 <*__GI_raise+82>:	mov    0x352c8f(%rip),%rdx        # 0x7ffff3a3ae18
   0x7ffff36e8189 <*__GI_raise+89>:	neg    %eax
   0x7ffff36e818b <*__GI_raise+91>:	mov    %eax,%fs:(%rdx)
   0x7ffff36e818e <*__GI_raise+94>:	or     $0xffffffff,%eax
   0x7ffff36e8191 <*__GI_raise+97>:	retq   


Threads backtrace:

Thread 9 (Thread 0x7fffdbfff700 (LWP 4313)):
#0  sem_timedwait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_timedwait.S:106
#1  0x00005555559b38cc in qemu_sem_timedwait (sem=0x5555563dcea8, ms=10000) at util/qemu-thread-posix.c:257
#2  0x0000555555849892 in worker_thread (opaque=0x5555563dce10) at thread-pool.c:97
#3  0x00007ffff3a47b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#4  0x00007ffff379195d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#5  0x0000000000000000 in ?? ()

Thread 8 (Thread 0x7fffe0acb700 (LWP 4312)):
#0  sem_timedwait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_timedwait.S:106
#1  0x00005555559b38cc in qemu_sem_timedwait (sem=0x5555563dcea8, ms=10000) at util/qemu-thread-posix.c:257
#2  0x0000555555849892 in worker_thread (opaque=0x5555563dce10) at thread-pool.c:97
#3  0x00007ffff3a47b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#4  0x00007ffff379195d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#5  0x0000000000000000 in ?? ()

Thread 7 (Thread 0x7fffe3017700 (LWP 4200)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162
#1  0x00005555559b360a in qemu_cond_wait (cond=0x5555564202a0, mutex=0x5555564202d0) at util/qemu-thread-posix.c:135
#2  0x00005555558731d6 in vnc_worker_thread_loop (queue=0x5555564202a0) at ui/vnc-jobs.c:222
#3  0x0000555555873739 in vnc_worker_thread (arg=0x5555564202a0) at ui/vnc-jobs.c:323
#4  0x00007ffff3a47b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#5  0x00007ffff379195d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#6  0x0000000000000000 in ?? ()

Thread 6 (Thread 0x7fffe3bf2700 (LWP 4199)):
#0  0x00007ffff36e8165 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1  0x00007ffff36eb3e0 in *__GI_abort () at abort.c:92
#2  0x00007ffff372bdea in __malloc_assert (assertion=<optimized out>, file=<optimized out>, line=<optimized out>, function=<optimized out>) at malloc.c:351
#3  0x00007ffff372ed13 in sYSMALLOc (av=<optimized out>, nb=<optimized out>) at malloc.c:3093
#4  _int_malloc (av=0x7ffff3a3ce40, bytes=<optimized out>) at malloc.c:4776
#5  0x00007ffff3730a70 in *__GI___libc_malloc (bytes=4196352) at malloc.c:3660
#6  0x00007ffff4b1d280 in spice_malloc (n_bytes=4196352) at mem.c:93
#7  0x00007ffff4b1d6ee in spice_chunks_linearize (chunks=0x7fffdc088fa0) at mem.c:226
#8  0x00007ffff4afb4e6 in canvas_bitmap_to_surface (canvas=canvas@entry=0x7fffdc0eef70, bitmap=bitmap@entry=0x7fffdc044e88, palette=0x0, want_original=1) at ../spice-common/common/canvas_base.c:738
#9  0x00007ffff4afb698 in canvas_get_bits (want_original=<optimized out>, bitmap=0x7fffdc044e88, canvas=0x7fffdc0eef70) at ../spice-common/common/canvas_base.c:1067
#10 canvas_get_image_internal (canvas=canvas@entry=0x7fffdc0eef70, image=0x7fffdc044e70, want_original=<optimized out>, want_original@entry=0, real_get=real_get@entry=1) at ../spice-common/common/canvas_base.c:1253
#11 0x00007ffff4afc0da in canvas_get_image (canvas=canvas@entry=0x7fffdc0eef70, image=<optimized out>, want_original=want_original@entry=0) at ../spice-common/common/canvas_base.c:1397
#12 0x00007ffff4afe42e in canvas_draw_copy (spice_canvas=0x7fffdc0eef70, bbox=0x7fffdc0e9c80, clip=<optimized out>, copy=0x7fffe3bf1320) at ../spice-common/common/canvas_base.c:2370
#13 0x00007ffff4ad120c in red_draw_qxl_drawable (worker=worker@entry=0x7fffe3218010, drawable=drawable@entry=0x7fffe33d8a58) at red_worker.c:4421
#14 0x00007ffff4adfad5 in red_draw_drawable (drawable=0x7fffe33d8a58, worker=0x7fffe3218010) at red_worker.c:4534
#15 red_update_area (worker=worker@entry=0x7fffe3218010, area=area@entry=0x7fffe3bf1b60, surface_id=surface_id@entry=0) at red_worker.c:4787
#16 0x00007ffff4ae9644 in handle_dev_update_async (opaque=0x7fffe3218010, payload=<optimized out>) at red_worker.c:10992
#17 0x00007ffff4ac85d4 in dispatcher_handle_single_read (dispatcher=0x555556415e28) at dispatcher.c:139
#18 dispatcher_handle_recv_read (dispatcher=0x555556415e28) at dispatcher.c:162
#19 0x00007ffff4aebc5c in red_worker_main (arg=<optimized out>) at red_worker.c:12175
#20 0x00007ffff3a47b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#21 0x00007ffff379195d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#22 0x0000000000000000 in ?? ()

Thread 5 (Thread 0x7fffe9621700 (LWP 4198)):
#0  sem_timedwait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_timedwait.S:106
#1  0x00005555559b38cc in qemu_sem_timedwait (sem=0x5555563dcea8, ms=10000) at util/qemu-thread-posix.c:257
#2  0x0000555555849892 in worker_thread (opaque=0x5555563dce10) at thread-pool.c:97
#3  0x00007ffff3a47b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#4  0x00007ffff379195d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#5  0x0000000000000000 in ?? ()

Thread 4 (Thread 0x7fffe9f23700 (LWP 4197)):
#0  do_sigwait (set=0x7fffe9f22c50, sig=0x7fffe9f22c40) at ../nptl/sysdeps/unix/sysv/linux/../../../../../sysdeps/unix/sysv/linux/sigwait.c:65
#1  0x00007ffff3a4fe67 in __sigwait (set=<optimized out>, sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/../../../../../sysdeps/unix/sysv/linux/sigwait.c:100
#2  0x0000555555893e0a in qemu_dummy_cpu_thread_fn (arg=0x55555636fa30) at /mnt/raid-vm/RW/source/xen/Xen-stable/tools/qemu-xen-dir/cpus.c:911
#3  0x00007ffff3a47b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#4  0x00007ffff379195d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#5  0x0000000000000000 in ?? ()

Thread 3 (Thread 0x7fffea724700 (LWP 4196)):
#0  do_sigwait (set=0x7fffea723c50, sig=0x7fffea723c40) at ../nptl/sysdeps/unix/sysv/linux/../../../../../sysdeps/unix/sysv/linux/sigwait.c:65
#1  0x00007ffff3a4fe67 in __sigwait (set=<optimized out>, sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/../../../../../sysdeps/unix/sysv/linux/sigwait.c:100
#2  0x0000555555893e0a in qemu_dummy_cpu_thread_fn (arg=0x55555635ecf0) at /mnt/raid-vm/RW/source/xen/Xen-stable/tools/qemu-xen-dir/cpus.c:911
#3  0x00007ffff3a47b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#4  0x00007ffff379195d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#5  0x0000000000000000 in ?? ()

Thread 2 (Thread 0x7ffff7ff1700 (LWP 4184)):
#0  0x00007ffff3a4f1fd in read () at ../sysdeps/unix/syscall-template.S:82
#1  0x00007ffff5229626 in read_all (fd=26, data=0x7fffdc096cc0, data@entry=0x20, len=len@entry=16, nonblocking=nonblocking@entry=0) at xs.c:378
#2  0x00007ffff5229743 in read_message (h=h@entry=0x55555632ee90, nonblocking=nonblocking@entry=0) at xs.c:1150
#3  0x00007ffff522a05e in read_thread (arg=0x55555632ee90) at xs.c:1222
#4  0x00007ffff3a47b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#5  0x00007ffff379195d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#6  0x0000000000000000 in ?? ()

Thread 1 (Thread 0x7ffff7ef6900 (LWP 4179)):
#0  0x00007ffff3786de1 in ppoll (fds=<optimized out>, nfds=<optimized out>, timeout=<optimized out>, sigmask=<optimized out>) at ../sysdeps/unix/sysv/linux/ppoll.c:58
#1  0x0000555555822f96 in qemu_poll_ns (fds=0x5555566fbe00, nfds=19, timeout=63041) at qemu-timer.c:316
#2  0x00005555557d74b0 in os_host_main_loop_wait (timeout=63041) at main-loop.c:229
#3  0x00005555557d7599 in main_loop_wait (nonblocking=0) at main-loop.c:484
#4  0x00005555558830a4 in main_loop () at vl.c:2056
#5  0x000055555588ab5f in main (argc=68, argv=0x7fffffffdf98, envp=0x7fffffffe1c0) at vl.c:4535

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: [Qemu-devel] Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-05-12 13:54             ` [Qemu-devel] " Fabio Fantoni
  2015-05-12 14:38               ` Stefano Stabellini
@ 2015-05-12 14:38               ` Stefano Stabellini
  2015-05-12 14:44                 ` Stefano Stabellini
  2015-05-12 14:44                 ` Regression: qemu crash of hvm domUs with spice (backtrace included) Stefano Stabellini
  1 sibling, 2 replies; 30+ messages in thread
From: Stefano Stabellini @ 2015-05-12 14:38 UTC (permalink / raw)
  To: Fabio Fantoni
  Cc: Stefano Stabellini, qemu-devel, xen-devel, Gerd Hoffmann,
	Jan Beulich, Anthony PERARD, spice-devel

On Tue, 12 May 2015, Fabio Fantoni wrote:
> Il 12/05/2015 12:26, Fabio Fantoni ha scritto:
> > Il 12/05/2015 11:23, Fabio Fantoni ha scritto:
> > > Il 11/05/2015 17:04, Fabio Fantoni ha scritto:
> > > > Il 21/04/2015 14:53, Stefano Stabellini ha scritto:
> > > > > On Tue, 21 Apr 2015, Fabio Fantoni wrote:
> > > > > > Il 21/04/2015 12:49, Stefano Stabellini ha scritto:
> > > > > > > On Mon, 20 Apr 2015, Fabio Fantoni wrote:
> > > > > > > > I updated xen and qemu from xen 4.5.0 with its upstream qemu
> > > > > > > > included to
> > > > > > > > xen
> > > > > > > > 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk
> > > > > > > > to use
> > > > > > > > revision "master").
> > > > > > > > After few minutes I booted windows 7 64 bit domU qemu crash,
> > > > > > > > tried 2 times
> > > > > > > > with same result.
> > > > > > > > 
> > > > > > > > In the domU's qemu log:
> > > > > > > > > qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion
> > > > > > > > > `(old_top ==
> > > > > > > > > (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) -
> > > > > > > > > __builtin_offsetof
> > > > > > > > > (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned
> > > > > > > > > long)
> > > > > > > > > (old_size) >= (unsigned long)((((__builtin_offsetof (struct
> > > > > > > > > malloc_chunk,
> > > > > > > > > fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 *
> > > > > > > > > (sizeof(size_t))) -
> > > > > > > > > 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end &
> > > > > > > > > pagemask)
> > > > > > > > > ==
> > > > > > > > > 0)' failed.
> > > > > > > > > Killing all inferiors
> > > > > > > > In attachment the full backtrace of qemu crash.
> > > > > > > > 
> > > > > > > > With a fast search after I saw the backtrace I found a probable
> > > > > > > > cause of
> > > > > > > > regression (I'm not sure):
> > > > > > > > http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa 
> > > > > > > > spice: make sure we don't overflow ssd->buf
> > > > > > > > 
> > > > > > > > Added also qemu-devel and spice-devel as cc.
> > > > > > > > 
> > > > > > > > If you need more informations/tests tell me and I'll post them.
> > > > > > >    Maybe you could try to revert the offending commit
> > > > > > > (5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better bisect
> > > > > > > the
> > > > > > > crash?
> > > > > > Thanks for your reply.
> > > > > > 
> > > > > > I reverted to 4.5.0 on dom0 for now on that system because I'm busy
> > > > > > trying to
> > > > > > found another problem that cause very bad performance without errors
> > > > > > or
> > > > > > nothing in logs :( I don't know if if xen related, kernel related or
> > > > > > other for
> > > > > > now.
> > > > > > 
> > > > > > About this regression with spice I'll do further tests in next days
> > > > > > (probably
> > > > > > starting reverting the spice patch in qemu) but any help is
> > > > > > appreciated.
> > > > > > Based on data I have for now is possible that the problem is that
> > > > > > qemu try to
> > > > > > allocate other ram or videoram after domU create but with xen is not
> > > > > > possible?
> > > > > > In the spice related patch I saw something about dynamic allocation
> > > > > > for
> > > > > > example.
> > > > > It is probably caused by a commit in the range:
> > > > > 
> > > > > 1ebb75b1fee779621b63e84fefa7b07354c43a99..0b8fb1ec3d666d1eb8bbff56c76c5e6daa2789e4 
> > > > > 
> > > > > there are only 10 commits in that range. By using git bisect you
> > > > > should
> > > > > be able to narrow it down in just 3 tests.
> > > > 
> > > > Sorry for delay, I was busy with many things, today I retried with
> > > > updated stable-4.5 and also reverting "spice: make sure we don't
> > > > overflow ssd->buf" (in a second test) but in both case regression remain
> > > > :(
> > > > Tomorrow probably I'll do other tests.
> > > 
> > > I did another test, reverting this instead:
> > > http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commit;h=c9ac5f816bf3a8b56f836b078711dcef6e5c90b8 
> > > And now seems I'm unable to reproduce the regression, before happen after
> > > few seconds up to 1-2 minutes, now I use the same domU 15-20 minutes
> > > without problem.
> > > Probably is the cause of regression even if seems strange that on unstable
> > > with same patch on tests of some days ago didn't happen.
> > > 
> > > Any ideas?
> > > 
> > > Thanks for any reply and sorry for my bad english.
> > 
> > Bad news, qemu crash still happen even if this time in qemu log there is
> > another output, see attachment.
> > After take a look on the other patches I saw:
> > http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commitdiff;h=7154fba0e51ec985ef621965d1b7120ad424fcbf 
> > With "Conflicts: hw/display/vga.c" in description I'll try to revert it
> > instead.
> > 
> > Or someone can tell me another probable test I can try?
> 
> Tried also to revet the patch above with same result, so I retried with qemu
> from 4.5.0 and seems the crash happen also in this case...I'm going crazy :(
> 
> In attachment full gdb log.
> 
> Any ideas on how to found the problem please?

Hi Fabio,

Don't worry, bisecting 10 commits should be pretty straightforward.
Just use the command "git bisect" on the QEMU repository:

git bisect start
git bisect bad
git bisect good 1ebb75b1fee779621b63e84fefa7b07354c43a99

These 3 commands tell git that 1ebb75b1fee779621b63e84fefa7b07354c43a99
was working correctly but the current head is broken. git bisect will
select a commit, that you need to that, somewhere in the middle of the
range. Once you tested it, you do

git bisect good

if it worked, or

git bisect bad

if it didn't work as expected. git bisect will automatically select
another commit for you to test. After about 3 tests, git bisect will
find what was the exact cause of the issue.

Let me know if anything is not clear.

Cheers,

Stefano

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-05-12 13:54             ` [Qemu-devel] " Fabio Fantoni
@ 2015-05-12 14:38               ` Stefano Stabellini
  2015-05-12 14:38               ` [Qemu-devel] " Stefano Stabellini
  1 sibling, 0 replies; 30+ messages in thread
From: Stefano Stabellini @ 2015-05-12 14:38 UTC (permalink / raw)
  To: Fabio Fantoni
  Cc: Stefano Stabellini, qemu-devel, xen-devel, Gerd Hoffmann,
	Jan Beulich, Anthony PERARD, spice-devel

On Tue, 12 May 2015, Fabio Fantoni wrote:
> Il 12/05/2015 12:26, Fabio Fantoni ha scritto:
> > Il 12/05/2015 11:23, Fabio Fantoni ha scritto:
> > > Il 11/05/2015 17:04, Fabio Fantoni ha scritto:
> > > > Il 21/04/2015 14:53, Stefano Stabellini ha scritto:
> > > > > On Tue, 21 Apr 2015, Fabio Fantoni wrote:
> > > > > > Il 21/04/2015 12:49, Stefano Stabellini ha scritto:
> > > > > > > On Mon, 20 Apr 2015, Fabio Fantoni wrote:
> > > > > > > > I updated xen and qemu from xen 4.5.0 with its upstream qemu
> > > > > > > > included to
> > > > > > > > xen
> > > > > > > > 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk
> > > > > > > > to use
> > > > > > > > revision "master").
> > > > > > > > After few minutes I booted windows 7 64 bit domU qemu crash,
> > > > > > > > tried 2 times
> > > > > > > > with same result.
> > > > > > > > 
> > > > > > > > In the domU's qemu log:
> > > > > > > > > qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion
> > > > > > > > > `(old_top ==
> > > > > > > > > (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) -
> > > > > > > > > __builtin_offsetof
> > > > > > > > > (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned
> > > > > > > > > long)
> > > > > > > > > (old_size) >= (unsigned long)((((__builtin_offsetof (struct
> > > > > > > > > malloc_chunk,
> > > > > > > > > fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 *
> > > > > > > > > (sizeof(size_t))) -
> > > > > > > > > 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end &
> > > > > > > > > pagemask)
> > > > > > > > > ==
> > > > > > > > > 0)' failed.
> > > > > > > > > Killing all inferiors
> > > > > > > > In attachment the full backtrace of qemu crash.
> > > > > > > > 
> > > > > > > > With a fast search after I saw the backtrace I found a probable
> > > > > > > > cause of
> > > > > > > > regression (I'm not sure):
> > > > > > > > http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa 
> > > > > > > > spice: make sure we don't overflow ssd->buf
> > > > > > > > 
> > > > > > > > Added also qemu-devel and spice-devel as cc.
> > > > > > > > 
> > > > > > > > If you need more informations/tests tell me and I'll post them.
> > > > > > >    Maybe you could try to revert the offending commit
> > > > > > > (5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better bisect
> > > > > > > the
> > > > > > > crash?
> > > > > > Thanks for your reply.
> > > > > > 
> > > > > > I reverted to 4.5.0 on dom0 for now on that system because I'm busy
> > > > > > trying to
> > > > > > found another problem that cause very bad performance without errors
> > > > > > or
> > > > > > nothing in logs :( I don't know if if xen related, kernel related or
> > > > > > other for
> > > > > > now.
> > > > > > 
> > > > > > About this regression with spice I'll do further tests in next days
> > > > > > (probably
> > > > > > starting reverting the spice patch in qemu) but any help is
> > > > > > appreciated.
> > > > > > Based on data I have for now is possible that the problem is that
> > > > > > qemu try to
> > > > > > allocate other ram or videoram after domU create but with xen is not
> > > > > > possible?
> > > > > > In the spice related patch I saw something about dynamic allocation
> > > > > > for
> > > > > > example.
> > > > > It is probably caused by a commit in the range:
> > > > > 
> > > > > 1ebb75b1fee779621b63e84fefa7b07354c43a99..0b8fb1ec3d666d1eb8bbff56c76c5e6daa2789e4 
> > > > > 
> > > > > there are only 10 commits in that range. By using git bisect you
> > > > > should
> > > > > be able to narrow it down in just 3 tests.
> > > > 
> > > > Sorry for delay, I was busy with many things, today I retried with
> > > > updated stable-4.5 and also reverting "spice: make sure we don't
> > > > overflow ssd->buf" (in a second test) but in both case regression remain
> > > > :(
> > > > Tomorrow probably I'll do other tests.
> > > 
> > > I did another test, reverting this instead:
> > > http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commit;h=c9ac5f816bf3a8b56f836b078711dcef6e5c90b8 
> > > And now seems I'm unable to reproduce the regression, before happen after
> > > few seconds up to 1-2 minutes, now I use the same domU 15-20 minutes
> > > without problem.
> > > Probably is the cause of regression even if seems strange that on unstable
> > > with same patch on tests of some days ago didn't happen.
> > > 
> > > Any ideas?
> > > 
> > > Thanks for any reply and sorry for my bad english.
> > 
> > Bad news, qemu crash still happen even if this time in qemu log there is
> > another output, see attachment.
> > After take a look on the other patches I saw:
> > http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commitdiff;h=7154fba0e51ec985ef621965d1b7120ad424fcbf 
> > With "Conflicts: hw/display/vga.c" in description I'll try to revert it
> > instead.
> > 
> > Or someone can tell me another probable test I can try?
> 
> Tried also to revet the patch above with same result, so I retried with qemu
> from 4.5.0 and seems the crash happen also in this case...I'm going crazy :(
> 
> In attachment full gdb log.
> 
> Any ideas on how to found the problem please?

Hi Fabio,

Don't worry, bisecting 10 commits should be pretty straightforward.
Just use the command "git bisect" on the QEMU repository:

git bisect start
git bisect bad
git bisect good 1ebb75b1fee779621b63e84fefa7b07354c43a99

These 3 commands tell git that 1ebb75b1fee779621b63e84fefa7b07354c43a99
was working correctly but the current head is broken. git bisect will
select a commit, that you need to that, somewhere in the middle of the
range. Once you tested it, you do

git bisect good

if it worked, or

git bisect bad

if it didn't work as expected. git bisect will automatically select
another commit for you to test. After about 3 tests, git bisect will
find what was the exact cause of the issue.

Let me know if anything is not clear.

Cheers,

Stefano

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: [Qemu-devel] Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-05-12 14:38               ` [Qemu-devel] " Stefano Stabellini
@ 2015-05-12 14:44                 ` Stefano Stabellini
  2015-05-13 13:29                   ` Fabio Fantoni
  2015-05-13 13:29                   ` [Qemu-devel] " Fabio Fantoni
  2015-05-12 14:44                 ` Regression: qemu crash of hvm domUs with spice (backtrace included) Stefano Stabellini
  1 sibling, 2 replies; 30+ messages in thread
From: Stefano Stabellini @ 2015-05-12 14:44 UTC (permalink / raw)
  To: Stefano Stabellini
  Cc: qemu-devel, xen-devel, Fabio Fantoni, Gerd Hoffmann, Jan Beulich,
	Anthony PERARD, spice-devel

On Tue, 12 May 2015, Stefano Stabellini wrote:
> On Tue, 12 May 2015, Fabio Fantoni wrote:
> > Il 12/05/2015 12:26, Fabio Fantoni ha scritto:
> > > Il 12/05/2015 11:23, Fabio Fantoni ha scritto:
> > > > Il 11/05/2015 17:04, Fabio Fantoni ha scritto:
> > > > > Il 21/04/2015 14:53, Stefano Stabellini ha scritto:
> > > > > > On Tue, 21 Apr 2015, Fabio Fantoni wrote:
> > > > > > > Il 21/04/2015 12:49, Stefano Stabellini ha scritto:
> > > > > > > > On Mon, 20 Apr 2015, Fabio Fantoni wrote:
> > > > > > > > > I updated xen and qemu from xen 4.5.0 with its upstream qemu
> > > > > > > > > included to
> > > > > > > > > xen
> > > > > > > > > 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk
> > > > > > > > > to use
> > > > > > > > > revision "master").
> > > > > > > > > After few minutes I booted windows 7 64 bit domU qemu crash,
> > > > > > > > > tried 2 times
> > > > > > > > > with same result.
> > > > > > > > > 
> > > > > > > > > In the domU's qemu log:
> > > > > > > > > > qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion
> > > > > > > > > > `(old_top ==
> > > > > > > > > > (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) -
> > > > > > > > > > __builtin_offsetof
> > > > > > > > > > (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned
> > > > > > > > > > long)
> > > > > > > > > > (old_size) >= (unsigned long)((((__builtin_offsetof (struct
> > > > > > > > > > malloc_chunk,
> > > > > > > > > > fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 *
> > > > > > > > > > (sizeof(size_t))) -
> > > > > > > > > > 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end &
> > > > > > > > > > pagemask)
> > > > > > > > > > ==
> > > > > > > > > > 0)' failed.
> > > > > > > > > > Killing all inferiors
> > > > > > > > > In attachment the full backtrace of qemu crash.
> > > > > > > > > 
> > > > > > > > > With a fast search after I saw the backtrace I found a probable
> > > > > > > > > cause of
> > > > > > > > > regression (I'm not sure):
> > > > > > > > > http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa 
> > > > > > > > > spice: make sure we don't overflow ssd->buf
> > > > > > > > > 
> > > > > > > > > Added also qemu-devel and spice-devel as cc.
> > > > > > > > > 
> > > > > > > > > If you need more informations/tests tell me and I'll post them.
> > > > > > > >    Maybe you could try to revert the offending commit
> > > > > > > > (5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better bisect
> > > > > > > > the
> > > > > > > > crash?
> > > > > > > Thanks for your reply.
> > > > > > > 
> > > > > > > I reverted to 4.5.0 on dom0 for now on that system because I'm busy
> > > > > > > trying to
> > > > > > > found another problem that cause very bad performance without errors
> > > > > > > or
> > > > > > > nothing in logs :( I don't know if if xen related, kernel related or
> > > > > > > other for
> > > > > > > now.
> > > > > > > 
> > > > > > > About this regression with spice I'll do further tests in next days
> > > > > > > (probably
> > > > > > > starting reverting the spice patch in qemu) but any help is
> > > > > > > appreciated.
> > > > > > > Based on data I have for now is possible that the problem is that
> > > > > > > qemu try to
> > > > > > > allocate other ram or videoram after domU create but with xen is not
> > > > > > > possible?
> > > > > > > In the spice related patch I saw something about dynamic allocation
> > > > > > > for
> > > > > > > example.
> > > > > > It is probably caused by a commit in the range:
> > > > > > 
> > > > > > 1ebb75b1fee779621b63e84fefa7b07354c43a99..0b8fb1ec3d666d1eb8bbff56c76c5e6daa2789e4 
> > > > > > 
> > > > > > there are only 10 commits in that range. By using git bisect you
> > > > > > should
> > > > > > be able to narrow it down in just 3 tests.
> > > > > 
> > > > > Sorry for delay, I was busy with many things, today I retried with
> > > > > updated stable-4.5 and also reverting "spice: make sure we don't
> > > > > overflow ssd->buf" (in a second test) but in both case regression remain
> > > > > :(
> > > > > Tomorrow probably I'll do other tests.
> > > > 
> > > > I did another test, reverting this instead:
> > > > http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commit;h=c9ac5f816bf3a8b56f836b078711dcef6e5c90b8 
> > > > And now seems I'm unable to reproduce the regression, before happen after
> > > > few seconds up to 1-2 minutes, now I use the same domU 15-20 minutes
> > > > without problem.
> > > > Probably is the cause of regression even if seems strange that on unstable
> > > > with same patch on tests of some days ago didn't happen.
> > > > 
> > > > Any ideas?
> > > > 
> > > > Thanks for any reply and sorry for my bad english.
> > > 
> > > Bad news, qemu crash still happen even if this time in qemu log there is
> > > another output, see attachment.
> > > After take a look on the other patches I saw:
> > > http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commitdiff;h=7154fba0e51ec985ef621965d1b7120ad424fcbf 
> > > With "Conflicts: hw/display/vga.c" in description I'll try to revert it
> > > instead.
> > > 
> > > Or someone can tell me another probable test I can try?
> > 
> > Tried also to revet the patch above with same result, so I retried with qemu
> > from 4.5.0 and seems the crash happen also in this case...I'm going crazy :(

Sorry, I missed this bit before. The only thing I could suggest at this
point, would be to make sure that you have a clean test environment.
Usually this happens when you have some "leftovers" from previous broken
tests.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-05-12 14:38               ` [Qemu-devel] " Stefano Stabellini
  2015-05-12 14:44                 ` Stefano Stabellini
@ 2015-05-12 14:44                 ` Stefano Stabellini
  1 sibling, 0 replies; 30+ messages in thread
From: Stefano Stabellini @ 2015-05-12 14:44 UTC (permalink / raw)
  To: Stefano Stabellini
  Cc: qemu-devel, xen-devel, Fabio Fantoni, Gerd Hoffmann, Jan Beulich,
	Anthony PERARD, spice-devel

On Tue, 12 May 2015, Stefano Stabellini wrote:
> On Tue, 12 May 2015, Fabio Fantoni wrote:
> > Il 12/05/2015 12:26, Fabio Fantoni ha scritto:
> > > Il 12/05/2015 11:23, Fabio Fantoni ha scritto:
> > > > Il 11/05/2015 17:04, Fabio Fantoni ha scritto:
> > > > > Il 21/04/2015 14:53, Stefano Stabellini ha scritto:
> > > > > > On Tue, 21 Apr 2015, Fabio Fantoni wrote:
> > > > > > > Il 21/04/2015 12:49, Stefano Stabellini ha scritto:
> > > > > > > > On Mon, 20 Apr 2015, Fabio Fantoni wrote:
> > > > > > > > > I updated xen and qemu from xen 4.5.0 with its upstream qemu
> > > > > > > > > included to
> > > > > > > > > xen
> > > > > > > > > 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk
> > > > > > > > > to use
> > > > > > > > > revision "master").
> > > > > > > > > After few minutes I booted windows 7 64 bit domU qemu crash,
> > > > > > > > > tried 2 times
> > > > > > > > > with same result.
> > > > > > > > > 
> > > > > > > > > In the domU's qemu log:
> > > > > > > > > > qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion
> > > > > > > > > > `(old_top ==
> > > > > > > > > > (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) -
> > > > > > > > > > __builtin_offsetof
> > > > > > > > > > (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned
> > > > > > > > > > long)
> > > > > > > > > > (old_size) >= (unsigned long)((((__builtin_offsetof (struct
> > > > > > > > > > malloc_chunk,
> > > > > > > > > > fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 *
> > > > > > > > > > (sizeof(size_t))) -
> > > > > > > > > > 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end &
> > > > > > > > > > pagemask)
> > > > > > > > > > ==
> > > > > > > > > > 0)' failed.
> > > > > > > > > > Killing all inferiors
> > > > > > > > > In attachment the full backtrace of qemu crash.
> > > > > > > > > 
> > > > > > > > > With a fast search after I saw the backtrace I found a probable
> > > > > > > > > cause of
> > > > > > > > > regression (I'm not sure):
> > > > > > > > > http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa 
> > > > > > > > > spice: make sure we don't overflow ssd->buf
> > > > > > > > > 
> > > > > > > > > Added also qemu-devel and spice-devel as cc.
> > > > > > > > > 
> > > > > > > > > If you need more informations/tests tell me and I'll post them.
> > > > > > > >    Maybe you could try to revert the offending commit
> > > > > > > > (5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better bisect
> > > > > > > > the
> > > > > > > > crash?
> > > > > > > Thanks for your reply.
> > > > > > > 
> > > > > > > I reverted to 4.5.0 on dom0 for now on that system because I'm busy
> > > > > > > trying to
> > > > > > > found another problem that cause very bad performance without errors
> > > > > > > or
> > > > > > > nothing in logs :( I don't know if if xen related, kernel related or
> > > > > > > other for
> > > > > > > now.
> > > > > > > 
> > > > > > > About this regression with spice I'll do further tests in next days
> > > > > > > (probably
> > > > > > > starting reverting the spice patch in qemu) but any help is
> > > > > > > appreciated.
> > > > > > > Based on data I have for now is possible that the problem is that
> > > > > > > qemu try to
> > > > > > > allocate other ram or videoram after domU create but with xen is not
> > > > > > > possible?
> > > > > > > In the spice related patch I saw something about dynamic allocation
> > > > > > > for
> > > > > > > example.
> > > > > > It is probably caused by a commit in the range:
> > > > > > 
> > > > > > 1ebb75b1fee779621b63e84fefa7b07354c43a99..0b8fb1ec3d666d1eb8bbff56c76c5e6daa2789e4 
> > > > > > 
> > > > > > there are only 10 commits in that range. By using git bisect you
> > > > > > should
> > > > > > be able to narrow it down in just 3 tests.
> > > > > 
> > > > > Sorry for delay, I was busy with many things, today I retried with
> > > > > updated stable-4.5 and also reverting "spice: make sure we don't
> > > > > overflow ssd->buf" (in a second test) but in both case regression remain
> > > > > :(
> > > > > Tomorrow probably I'll do other tests.
> > > > 
> > > > I did another test, reverting this instead:
> > > > http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commit;h=c9ac5f816bf3a8b56f836b078711dcef6e5c90b8 
> > > > And now seems I'm unable to reproduce the regression, before happen after
> > > > few seconds up to 1-2 minutes, now I use the same domU 15-20 minutes
> > > > without problem.
> > > > Probably is the cause of regression even if seems strange that on unstable
> > > > with same patch on tests of some days ago didn't happen.
> > > > 
> > > > Any ideas?
> > > > 
> > > > Thanks for any reply and sorry for my bad english.
> > > 
> > > Bad news, qemu crash still happen even if this time in qemu log there is
> > > another output, see attachment.
> > > After take a look on the other patches I saw:
> > > http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commitdiff;h=7154fba0e51ec985ef621965d1b7120ad424fcbf 
> > > With "Conflicts: hw/display/vga.c" in description I'll try to revert it
> > > instead.
> > > 
> > > Or someone can tell me another probable test I can try?
> > 
> > Tried also to revet the patch above with same result, so I retried with qemu
> > from 4.5.0 and seems the crash happen also in this case...I'm going crazy :(

Sorry, I missed this bit before. The only thing I could suggest at this
point, would be to make sure that you have a clean test environment.
Usually this happens when you have some "leftovers" from previous broken
tests.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: [Qemu-devel] Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-05-12 14:44                 ` Stefano Stabellini
  2015-05-13 13:29                   ` Fabio Fantoni
@ 2015-05-13 13:29                   ` Fabio Fantoni
  2015-05-15 10:26                     ` Stefano Stabellini
                                       ` (2 more replies)
  1 sibling, 3 replies; 30+ messages in thread
From: Fabio Fantoni @ 2015-05-13 13:29 UTC (permalink / raw)
  To: Stefano Stabellini
  Cc: qemu-devel, xen-devel, Gerd Hoffmann, Jan Beulich,
	Anthony PERARD, spice-devel

Il 12/05/2015 16:44, Stefano Stabellini ha scritto:
> On Tue, 12 May 2015, Stefano Stabellini wrote:
>> On Tue, 12 May 2015, Fabio Fantoni wrote:
>>> Il 12/05/2015 12:26, Fabio Fantoni ha scritto:
>>>> Il 12/05/2015 11:23, Fabio Fantoni ha scritto:
>>>>> Il 11/05/2015 17:04, Fabio Fantoni ha scritto:
>>>>>> Il 21/04/2015 14:53, Stefano Stabellini ha scritto:
>>>>>>> On Tue, 21 Apr 2015, Fabio Fantoni wrote:
>>>>>>>> Il 21/04/2015 12:49, Stefano Stabellini ha scritto:
>>>>>>>>> On Mon, 20 Apr 2015, Fabio Fantoni wrote:
>>>>>>>>>> I updated xen and qemu from xen 4.5.0 with its upstream qemu
>>>>>>>>>> included to
>>>>>>>>>> xen
>>>>>>>>>> 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk
>>>>>>>>>> to use
>>>>>>>>>> revision "master").
>>>>>>>>>> After few minutes I booted windows 7 64 bit domU qemu crash,
>>>>>>>>>> tried 2 times
>>>>>>>>>> with same result.
>>>>>>>>>>
>>>>>>>>>> In the domU's qemu log:
>>>>>>>>>>> qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion
>>>>>>>>>>> `(old_top ==
>>>>>>>>>>> (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) -
>>>>>>>>>>> __builtin_offsetof
>>>>>>>>>>> (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned
>>>>>>>>>>> long)
>>>>>>>>>>> (old_size) >= (unsigned long)((((__builtin_offsetof (struct
>>>>>>>>>>> malloc_chunk,
>>>>>>>>>>> fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 *
>>>>>>>>>>> (sizeof(size_t))) -
>>>>>>>>>>> 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end &
>>>>>>>>>>> pagemask)
>>>>>>>>>>> ==
>>>>>>>>>>> 0)' failed.
>>>>>>>>>>> Killing all inferiors
>>>>>>>>>> In attachment the full backtrace of qemu crash.
>>>>>>>>>>
>>>>>>>>>> With a fast search after I saw the backtrace I found a probable
>>>>>>>>>> cause of
>>>>>>>>>> regression (I'm not sure):
>>>>>>>>>> http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa
>>>>>>>>>> spice: make sure we don't overflow ssd->buf
>>>>>>>>>>
>>>>>>>>>> Added also qemu-devel and spice-devel as cc.
>>>>>>>>>>
>>>>>>>>>> If you need more informations/tests tell me and I'll post them.
>>>>>>>>>     Maybe you could try to revert the offending commit
>>>>>>>>> (5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better bisect
>>>>>>>>> the
>>>>>>>>> crash?
>>>>>>>> Thanks for your reply.
>>>>>>>>
>>>>>>>> I reverted to 4.5.0 on dom0 for now on that system because I'm busy
>>>>>>>> trying to
>>>>>>>> found another problem that cause very bad performance without errors
>>>>>>>> or
>>>>>>>> nothing in logs :( I don't know if if xen related, kernel related or
>>>>>>>> other for
>>>>>>>> now.
>>>>>>>>
>>>>>>>> About this regression with spice I'll do further tests in next days
>>>>>>>> (probably
>>>>>>>> starting reverting the spice patch in qemu) but any help is
>>>>>>>> appreciated.
>>>>>>>> Based on data I have for now is possible that the problem is that
>>>>>>>> qemu try to
>>>>>>>> allocate other ram or videoram after domU create but with xen is not
>>>>>>>> possible?
>>>>>>>> In the spice related patch I saw something about dynamic allocation
>>>>>>>> for
>>>>>>>> example.
>>>>>>> It is probably caused by a commit in the range:
>>>>>>>
>>>>>>> 1ebb75b1fee779621b63e84fefa7b07354c43a99..0b8fb1ec3d666d1eb8bbff56c76c5e6daa2789e4
>>>>>>>
>>>>>>> there are only 10 commits in that range. By using git bisect you
>>>>>>> should
>>>>>>> be able to narrow it down in just 3 tests.
>>>>>> Sorry for delay, I was busy with many things, today I retried with
>>>>>> updated stable-4.5 and also reverting "spice: make sure we don't
>>>>>> overflow ssd->buf" (in a second test) but in both case regression remain
>>>>>> :(
>>>>>> Tomorrow probably I'll do other tests.
>>>>> I did another test, reverting this instead:
>>>>> http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commit;h=c9ac5f816bf3a8b56f836b078711dcef6e5c90b8
>>>>> And now seems I'm unable to reproduce the regression, before happen after
>>>>> few seconds up to 1-2 minutes, now I use the same domU 15-20 minutes
>>>>> without problem.
>>>>> Probably is the cause of regression even if seems strange that on unstable
>>>>> with same patch on tests of some days ago didn't happen.
>>>>>
>>>>> Any ideas?
>>>>>
>>>>> Thanks for any reply and sorry for my bad english.
>>>> Bad news, qemu crash still happen even if this time in qemu log there is
>>>> another output, see attachment.
>>>> After take a look on the other patches I saw:
>>>> http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commitdiff;h=7154fba0e51ec985ef621965d1b7120ad424fcbf
>>>> With "Conflicts: hw/display/vga.c" in description I'll try to revert it
>>>> instead.
>>>>
>>>> Or someone can tell me another probable test I can try?
>>> Tried also to revet the patch above with same result, so I retried with qemu
>>> from 4.5.0 and seems the crash happen also in this case...I'm going crazy :(
> Sorry, I missed this bit before. The only thing I could suggest at this
> point, would be to make sure that you have a clean test environment.
> Usually this happens when you have some "leftovers" from previous broken
> tests.

I use make debball to be sure to track and remove all files on package 
update.
Now I retried with latest xen-unstable and the qemu crash didn't happen, 
more exactly I used this:
https://github.com/Fantu/Xen/commits/rebase/m2r-staging
Latest test with regression based on latest stable-4.5, more exactly:
https://github.com/Fantu/Xen/commits/rebase/m2r-testing
Some days ago on same dom0 and domU I tried with latest stable version 
(that I use on only 2 production servers for now but I not saw the 
regression), more exactly:
https://github.com/Fantu/Xen/commits/rebase/m2r-stable-4.5
Dom0 debian 7 with kernel 3.16 from backports, seabios 1.8.1-2 from 
unstable and this xen configure:
./configure --prefix=/usr --disable-blktap1 --disable-qemu-traditional 
--disable-rombios --with-system-seabios=/usr/share/seabios/bios-256k.bin 
--with-extra-qemuu-configure-args="--enable-spice --enable-usb-redir" 
--disable-blktap2

I suppose that there is unexpected case caused by a backports or missed 
patch/es to backports from unstable.
I not found with a fast look rilevant patch to try to revert, can anyone 
suggest me the more probable point/s for bisect and/or patch to revert 
or I must try full bisect 4.5.0->stable-4.5?

Thanks for any reply and sorry for my bad english.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-05-12 14:44                 ` Stefano Stabellini
@ 2015-05-13 13:29                   ` Fabio Fantoni
  2015-05-13 13:29                   ` [Qemu-devel] " Fabio Fantoni
  1 sibling, 0 replies; 30+ messages in thread
From: Fabio Fantoni @ 2015-05-13 13:29 UTC (permalink / raw)
  To: Stefano Stabellini
  Cc: qemu-devel, xen-devel, Gerd Hoffmann, Jan Beulich,
	Anthony PERARD, spice-devel

Il 12/05/2015 16:44, Stefano Stabellini ha scritto:
> On Tue, 12 May 2015, Stefano Stabellini wrote:
>> On Tue, 12 May 2015, Fabio Fantoni wrote:
>>> Il 12/05/2015 12:26, Fabio Fantoni ha scritto:
>>>> Il 12/05/2015 11:23, Fabio Fantoni ha scritto:
>>>>> Il 11/05/2015 17:04, Fabio Fantoni ha scritto:
>>>>>> Il 21/04/2015 14:53, Stefano Stabellini ha scritto:
>>>>>>> On Tue, 21 Apr 2015, Fabio Fantoni wrote:
>>>>>>>> Il 21/04/2015 12:49, Stefano Stabellini ha scritto:
>>>>>>>>> On Mon, 20 Apr 2015, Fabio Fantoni wrote:
>>>>>>>>>> I updated xen and qemu from xen 4.5.0 with its upstream qemu
>>>>>>>>>> included to
>>>>>>>>>> xen
>>>>>>>>>> 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk
>>>>>>>>>> to use
>>>>>>>>>> revision "master").
>>>>>>>>>> After few minutes I booted windows 7 64 bit domU qemu crash,
>>>>>>>>>> tried 2 times
>>>>>>>>>> with same result.
>>>>>>>>>>
>>>>>>>>>> In the domU's qemu log:
>>>>>>>>>>> qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion
>>>>>>>>>>> `(old_top ==
>>>>>>>>>>> (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) -
>>>>>>>>>>> __builtin_offsetof
>>>>>>>>>>> (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned
>>>>>>>>>>> long)
>>>>>>>>>>> (old_size) >= (unsigned long)((((__builtin_offsetof (struct
>>>>>>>>>>> malloc_chunk,
>>>>>>>>>>> fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 *
>>>>>>>>>>> (sizeof(size_t))) -
>>>>>>>>>>> 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end &
>>>>>>>>>>> pagemask)
>>>>>>>>>>> ==
>>>>>>>>>>> 0)' failed.
>>>>>>>>>>> Killing all inferiors
>>>>>>>>>> In attachment the full backtrace of qemu crash.
>>>>>>>>>>
>>>>>>>>>> With a fast search after I saw the backtrace I found a probable
>>>>>>>>>> cause of
>>>>>>>>>> regression (I'm not sure):
>>>>>>>>>> http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa
>>>>>>>>>> spice: make sure we don't overflow ssd->buf
>>>>>>>>>>
>>>>>>>>>> Added also qemu-devel and spice-devel as cc.
>>>>>>>>>>
>>>>>>>>>> If you need more informations/tests tell me and I'll post them.
>>>>>>>>>     Maybe you could try to revert the offending commit
>>>>>>>>> (5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better bisect
>>>>>>>>> the
>>>>>>>>> crash?
>>>>>>>> Thanks for your reply.
>>>>>>>>
>>>>>>>> I reverted to 4.5.0 on dom0 for now on that system because I'm busy
>>>>>>>> trying to
>>>>>>>> found another problem that cause very bad performance without errors
>>>>>>>> or
>>>>>>>> nothing in logs :( I don't know if if xen related, kernel related or
>>>>>>>> other for
>>>>>>>> now.
>>>>>>>>
>>>>>>>> About this regression with spice I'll do further tests in next days
>>>>>>>> (probably
>>>>>>>> starting reverting the spice patch in qemu) but any help is
>>>>>>>> appreciated.
>>>>>>>> Based on data I have for now is possible that the problem is that
>>>>>>>> qemu try to
>>>>>>>> allocate other ram or videoram after domU create but with xen is not
>>>>>>>> possible?
>>>>>>>> In the spice related patch I saw something about dynamic allocation
>>>>>>>> for
>>>>>>>> example.
>>>>>>> It is probably caused by a commit in the range:
>>>>>>>
>>>>>>> 1ebb75b1fee779621b63e84fefa7b07354c43a99..0b8fb1ec3d666d1eb8bbff56c76c5e6daa2789e4
>>>>>>>
>>>>>>> there are only 10 commits in that range. By using git bisect you
>>>>>>> should
>>>>>>> be able to narrow it down in just 3 tests.
>>>>>> Sorry for delay, I was busy with many things, today I retried with
>>>>>> updated stable-4.5 and also reverting "spice: make sure we don't
>>>>>> overflow ssd->buf" (in a second test) but in both case regression remain
>>>>>> :(
>>>>>> Tomorrow probably I'll do other tests.
>>>>> I did another test, reverting this instead:
>>>>> http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commit;h=c9ac5f816bf3a8b56f836b078711dcef6e5c90b8
>>>>> And now seems I'm unable to reproduce the regression, before happen after
>>>>> few seconds up to 1-2 minutes, now I use the same domU 15-20 minutes
>>>>> without problem.
>>>>> Probably is the cause of regression even if seems strange that on unstable
>>>>> with same patch on tests of some days ago didn't happen.
>>>>>
>>>>> Any ideas?
>>>>>
>>>>> Thanks for any reply and sorry for my bad english.
>>>> Bad news, qemu crash still happen even if this time in qemu log there is
>>>> another output, see attachment.
>>>> After take a look on the other patches I saw:
>>>> http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commitdiff;h=7154fba0e51ec985ef621965d1b7120ad424fcbf
>>>> With "Conflicts: hw/display/vga.c" in description I'll try to revert it
>>>> instead.
>>>>
>>>> Or someone can tell me another probable test I can try?
>>> Tried also to revet the patch above with same result, so I retried with qemu
>>> from 4.5.0 and seems the crash happen also in this case...I'm going crazy :(
> Sorry, I missed this bit before. The only thing I could suggest at this
> point, would be to make sure that you have a clean test environment.
> Usually this happens when you have some "leftovers" from previous broken
> tests.

I use make debball to be sure to track and remove all files on package 
update.
Now I retried with latest xen-unstable and the qemu crash didn't happen, 
more exactly I used this:
https://github.com/Fantu/Xen/commits/rebase/m2r-staging
Latest test with regression based on latest stable-4.5, more exactly:
https://github.com/Fantu/Xen/commits/rebase/m2r-testing
Some days ago on same dom0 and domU I tried with latest stable version 
(that I use on only 2 production servers for now but I not saw the 
regression), more exactly:
https://github.com/Fantu/Xen/commits/rebase/m2r-stable-4.5
Dom0 debian 7 with kernel 3.16 from backports, seabios 1.8.1-2 from 
unstable and this xen configure:
./configure --prefix=/usr --disable-blktap1 --disable-qemu-traditional 
--disable-rombios --with-system-seabios=/usr/share/seabios/bios-256k.bin 
--with-extra-qemuu-configure-args="--enable-spice --enable-usb-redir" 
--disable-blktap2

I suppose that there is unexpected case caused by a backports or missed 
patch/es to backports from unstable.
I not found with a fast look rilevant patch to try to revert, can anyone 
suggest me the more probable point/s for bisect and/or patch to revert 
or I must try full bisect 4.5.0->stable-4.5?

Thanks for any reply and sorry for my bad english.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: [Qemu-devel] Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-05-13 13:29                   ` [Qemu-devel] " Fabio Fantoni
  2015-05-15 10:26                     ` Stefano Stabellini
@ 2015-05-15 10:26                     ` Stefano Stabellini
  2015-10-08 15:58                     ` Andreas Kinzler
  2 siblings, 0 replies; 30+ messages in thread
From: Stefano Stabellini @ 2015-05-15 10:26 UTC (permalink / raw)
  To: Fabio Fantoni
  Cc: Stefano Stabellini, qemu-devel, xen-devel, Gerd Hoffmann,
	Jan Beulich, Anthony PERARD, spice-devel

On Wed, 13 May 2015, Fabio Fantoni wrote:
> Il 12/05/2015 16:44, Stefano Stabellini ha scritto:
> > On Tue, 12 May 2015, Stefano Stabellini wrote:
> > > On Tue, 12 May 2015, Fabio Fantoni wrote:
> > > > Il 12/05/2015 12:26, Fabio Fantoni ha scritto:
> > > > > Il 12/05/2015 11:23, Fabio Fantoni ha scritto:
> > > > > > Il 11/05/2015 17:04, Fabio Fantoni ha scritto:
> > > > > > > Il 21/04/2015 14:53, Stefano Stabellini ha scritto:
> > > > > > > > On Tue, 21 Apr 2015, Fabio Fantoni wrote:
> > > > > > > > > Il 21/04/2015 12:49, Stefano Stabellini ha scritto:
> > > > > > > > > > On Mon, 20 Apr 2015, Fabio Fantoni wrote:
> > > > > > > > > > > I updated xen and qemu from xen 4.5.0 with its upstream
> > > > > > > > > > > qemu
> > > > > > > > > > > included to
> > > > > > > > > > > xen
> > > > > > > > > > > 4.5.1-pre with qemu upstream from stable-4.5 (changed
> > > > > > > > > > > Config.mk
> > > > > > > > > > > to use
> > > > > > > > > > > revision "master").
> > > > > > > > > > > After few minutes I booted windows 7 64 bit domU qemu
> > > > > > > > > > > crash,
> > > > > > > > > > > tried 2 times
> > > > > > > > > > > with same result.
> > > > > > > > > > > 
> > > > > > > > > > > In the domU's qemu log:
> > > > > > > > > > > > qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion
> > > > > > > > > > > > `(old_top ==
> > > > > > > > > > > > (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) -
> > > > > > > > > > > > __builtin_offsetof
> > > > > > > > > > > > (struct malloc_chunk, fd)))) && old_size == 0) ||
> > > > > > > > > > > > ((unsigned
> > > > > > > > > > > > long)
> > > > > > > > > > > > (old_size) >= (unsigned long)((((__builtin_offsetof
> > > > > > > > > > > > (struct
> > > > > > > > > > > > malloc_chunk,
> > > > > > > > > > > > fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 *
> > > > > > > > > > > > (sizeof(size_t))) -
> > > > > > > > > > > > 1))) && ((old_top)->size & 0x1) && ((unsigned
> > > > > > > > > > > > long)old_end &
> > > > > > > > > > > > pagemask)
> > > > > > > > > > > > ==
> > > > > > > > > > > > 0)' failed.
> > > > > > > > > > > > Killing all inferiors
> > > > > > > > > > > In attachment the full backtrace of qemu crash.
> > > > > > > > > > > 
> > > > > > > > > > > With a fast search after I saw the backtrace I found a
> > > > > > > > > > > probable
> > > > > > > > > > > cause of
> > > > > > > > > > > regression (I'm not sure):
> > > > > > > > > > > http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa
> > > > > > > > > > > spice: make sure we don't overflow ssd->buf
> > > > > > > > > > > 
> > > > > > > > > > > Added also qemu-devel and spice-devel as cc.
> > > > > > > > > > > 
> > > > > > > > > > > If you need more informations/tests tell me and I'll post
> > > > > > > > > > > them.
> > > > > > > > > >     Maybe you could try to revert the offending commit
> > > > > > > > > > (5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better
> > > > > > > > > > bisect
> > > > > > > > > > the
> > > > > > > > > > crash?
> > > > > > > > > Thanks for your reply.
> > > > > > > > > 
> > > > > > > > > I reverted to 4.5.0 on dom0 for now on that system because I'm
> > > > > > > > > busy
> > > > > > > > > trying to
> > > > > > > > > found another problem that cause very bad performance without
> > > > > > > > > errors
> > > > > > > > > or
> > > > > > > > > nothing in logs :( I don't know if if xen related, kernel
> > > > > > > > > related or
> > > > > > > > > other for
> > > > > > > > > now.
> > > > > > > > > 
> > > > > > > > > About this regression with spice I'll do further tests in next
> > > > > > > > > days
> > > > > > > > > (probably
> > > > > > > > > starting reverting the spice patch in qemu) but any help is
> > > > > > > > > appreciated.
> > > > > > > > > Based on data I have for now is possible that the problem is
> > > > > > > > > that
> > > > > > > > > qemu try to
> > > > > > > > > allocate other ram or videoram after domU create but with xen
> > > > > > > > > is not
> > > > > > > > > possible?
> > > > > > > > > In the spice related patch I saw something about dynamic
> > > > > > > > > allocation
> > > > > > > > > for
> > > > > > > > > example.
> > > > > > > > It is probably caused by a commit in the range:
> > > > > > > > 
> > > > > > > > 1ebb75b1fee779621b63e84fefa7b07354c43a99..0b8fb1ec3d666d1eb8bbff56c76c5e6daa2789e4
> > > > > > > > 
> > > > > > > > there are only 10 commits in that range. By using git bisect you
> > > > > > > > should
> > > > > > > > be able to narrow it down in just 3 tests.
> > > > > > > Sorry for delay, I was busy with many things, today I retried with
> > > > > > > updated stable-4.5 and also reverting "spice: make sure we don't
> > > > > > > overflow ssd->buf" (in a second test) but in both case regression
> > > > > > > remain
> > > > > > > :(
> > > > > > > Tomorrow probably I'll do other tests.
> > > > > > I did another test, reverting this instead:
> > > > > > http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commit;h=c9ac5f816bf3a8b56f836b078711dcef6e5c90b8
> > > > > > And now seems I'm unable to reproduce the regression, before happen
> > > > > > after
> > > > > > few seconds up to 1-2 minutes, now I use the same domU 15-20 minutes
> > > > > > without problem.
> > > > > > Probably is the cause of regression even if seems strange that on
> > > > > > unstable
> > > > > > with same patch on tests of some days ago didn't happen.
> > > > > > 
> > > > > > Any ideas?
> > > > > > 
> > > > > > Thanks for any reply and sorry for my bad english.
> > > > > Bad news, qemu crash still happen even if this time in qemu log there
> > > > > is
> > > > > another output, see attachment.
> > > > > After take a look on the other patches I saw:
> > > > > http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commitdiff;h=7154fba0e51ec985ef621965d1b7120ad424fcbf
> > > > > With "Conflicts: hw/display/vga.c" in description I'll try to revert
> > > > > it
> > > > > instead.
> > > > > 
> > > > > Or someone can tell me another probable test I can try?
> > > > Tried also to revet the patch above with same result, so I retried with
> > > > qemu
> > > > from 4.5.0 and seems the crash happen also in this case...I'm going
> > > > crazy :(
> > Sorry, I missed this bit before. The only thing I could suggest at this
> > point, would be to make sure that you have a clean test environment.
> > Usually this happens when you have some "leftovers" from previous broken
> > tests.
> 
> I use make debball to be sure to track and remove all files on package update.
> Now I retried with latest xen-unstable and the qemu crash didn't happen, more
> exactly I used this:
> https://github.com/Fantu/Xen/commits/rebase/m2r-staging
> Latest test with regression based on latest stable-4.5, more exactly:
> https://github.com/Fantu/Xen/commits/rebase/m2r-testing
> Some days ago on same dom0 and domU I tried with latest stable version (that I
> use on only 2 production servers for now but I not saw the regression), more
> exactly:
> https://github.com/Fantu/Xen/commits/rebase/m2r-stable-4.5
> Dom0 debian 7 with kernel 3.16 from backports, seabios 1.8.1-2 from unstable
> and this xen configure:
> ./configure --prefix=/usr --disable-blktap1 --disable-qemu-traditional
> --disable-rombios --with-system-seabios=/usr/share/seabios/bios-256k.bin
> --with-extra-qemuu-configure-args="--enable-spice --enable-usb-redir"
> --disable-blktap2
> 
> I suppose that there is unexpected case caused by a backports or missed
> patch/es to backports from unstable.
> I not found with a fast look rilevant patch to try to revert, can anyone
> suggest me the more probable point/s for bisect and/or patch to revert or I
> must try full bisect 4.5.0->stable-4.5?

It is possible that this is an intermittent bug: it only shows once
every so many tests. I think you need to go over the full range
4.5.0->stable-4.5. Also it is important that you start from a working
baseline: you need to be sure that 4.5.0 works as expected.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-05-13 13:29                   ` [Qemu-devel] " Fabio Fantoni
@ 2015-05-15 10:26                     ` Stefano Stabellini
  2015-05-15 10:26                     ` [Qemu-devel] " Stefano Stabellini
  2015-10-08 15:58                     ` Andreas Kinzler
  2 siblings, 0 replies; 30+ messages in thread
From: Stefano Stabellini @ 2015-05-15 10:26 UTC (permalink / raw)
  To: Fabio Fantoni
  Cc: Stefano Stabellini, qemu-devel, xen-devel, Gerd Hoffmann,
	Jan Beulich, Anthony PERARD, spice-devel

On Wed, 13 May 2015, Fabio Fantoni wrote:
> Il 12/05/2015 16:44, Stefano Stabellini ha scritto:
> > On Tue, 12 May 2015, Stefano Stabellini wrote:
> > > On Tue, 12 May 2015, Fabio Fantoni wrote:
> > > > Il 12/05/2015 12:26, Fabio Fantoni ha scritto:
> > > > > Il 12/05/2015 11:23, Fabio Fantoni ha scritto:
> > > > > > Il 11/05/2015 17:04, Fabio Fantoni ha scritto:
> > > > > > > Il 21/04/2015 14:53, Stefano Stabellini ha scritto:
> > > > > > > > On Tue, 21 Apr 2015, Fabio Fantoni wrote:
> > > > > > > > > Il 21/04/2015 12:49, Stefano Stabellini ha scritto:
> > > > > > > > > > On Mon, 20 Apr 2015, Fabio Fantoni wrote:
> > > > > > > > > > > I updated xen and qemu from xen 4.5.0 with its upstream
> > > > > > > > > > > qemu
> > > > > > > > > > > included to
> > > > > > > > > > > xen
> > > > > > > > > > > 4.5.1-pre with qemu upstream from stable-4.5 (changed
> > > > > > > > > > > Config.mk
> > > > > > > > > > > to use
> > > > > > > > > > > revision "master").
> > > > > > > > > > > After few minutes I booted windows 7 64 bit domU qemu
> > > > > > > > > > > crash,
> > > > > > > > > > > tried 2 times
> > > > > > > > > > > with same result.
> > > > > > > > > > > 
> > > > > > > > > > > In the domU's qemu log:
> > > > > > > > > > > > qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion
> > > > > > > > > > > > `(old_top ==
> > > > > > > > > > > > (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) -
> > > > > > > > > > > > __builtin_offsetof
> > > > > > > > > > > > (struct malloc_chunk, fd)))) && old_size == 0) ||
> > > > > > > > > > > > ((unsigned
> > > > > > > > > > > > long)
> > > > > > > > > > > > (old_size) >= (unsigned long)((((__builtin_offsetof
> > > > > > > > > > > > (struct
> > > > > > > > > > > > malloc_chunk,
> > > > > > > > > > > > fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 *
> > > > > > > > > > > > (sizeof(size_t))) -
> > > > > > > > > > > > 1))) && ((old_top)->size & 0x1) && ((unsigned
> > > > > > > > > > > > long)old_end &
> > > > > > > > > > > > pagemask)
> > > > > > > > > > > > ==
> > > > > > > > > > > > 0)' failed.
> > > > > > > > > > > > Killing all inferiors
> > > > > > > > > > > In attachment the full backtrace of qemu crash.
> > > > > > > > > > > 
> > > > > > > > > > > With a fast search after I saw the backtrace I found a
> > > > > > > > > > > probable
> > > > > > > > > > > cause of
> > > > > > > > > > > regression (I'm not sure):
> > > > > > > > > > > http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa
> > > > > > > > > > > spice: make sure we don't overflow ssd->buf
> > > > > > > > > > > 
> > > > > > > > > > > Added also qemu-devel and spice-devel as cc.
> > > > > > > > > > > 
> > > > > > > > > > > If you need more informations/tests tell me and I'll post
> > > > > > > > > > > them.
> > > > > > > > > >     Maybe you could try to revert the offending commit
> > > > > > > > > > (5c3402816aaddb15156c69df73c54abe4e1c76aa)? Or even better
> > > > > > > > > > bisect
> > > > > > > > > > the
> > > > > > > > > > crash?
> > > > > > > > > Thanks for your reply.
> > > > > > > > > 
> > > > > > > > > I reverted to 4.5.0 on dom0 for now on that system because I'm
> > > > > > > > > busy
> > > > > > > > > trying to
> > > > > > > > > found another problem that cause very bad performance without
> > > > > > > > > errors
> > > > > > > > > or
> > > > > > > > > nothing in logs :( I don't know if if xen related, kernel
> > > > > > > > > related or
> > > > > > > > > other for
> > > > > > > > > now.
> > > > > > > > > 
> > > > > > > > > About this regression with spice I'll do further tests in next
> > > > > > > > > days
> > > > > > > > > (probably
> > > > > > > > > starting reverting the spice patch in qemu) but any help is
> > > > > > > > > appreciated.
> > > > > > > > > Based on data I have for now is possible that the problem is
> > > > > > > > > that
> > > > > > > > > qemu try to
> > > > > > > > > allocate other ram or videoram after domU create but with xen
> > > > > > > > > is not
> > > > > > > > > possible?
> > > > > > > > > In the spice related patch I saw something about dynamic
> > > > > > > > > allocation
> > > > > > > > > for
> > > > > > > > > example.
> > > > > > > > It is probably caused by a commit in the range:
> > > > > > > > 
> > > > > > > > 1ebb75b1fee779621b63e84fefa7b07354c43a99..0b8fb1ec3d666d1eb8bbff56c76c5e6daa2789e4
> > > > > > > > 
> > > > > > > > there are only 10 commits in that range. By using git bisect you
> > > > > > > > should
> > > > > > > > be able to narrow it down in just 3 tests.
> > > > > > > Sorry for delay, I was busy with many things, today I retried with
> > > > > > > updated stable-4.5 and also reverting "spice: make sure we don't
> > > > > > > overflow ssd->buf" (in a second test) but in both case regression
> > > > > > > remain
> > > > > > > :(
> > > > > > > Tomorrow probably I'll do other tests.
> > > > > > I did another test, reverting this instead:
> > > > > > http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commit;h=c9ac5f816bf3a8b56f836b078711dcef6e5c90b8
> > > > > > And now seems I'm unable to reproduce the regression, before happen
> > > > > > after
> > > > > > few seconds up to 1-2 minutes, now I use the same domU 15-20 minutes
> > > > > > without problem.
> > > > > > Probably is the cause of regression even if seems strange that on
> > > > > > unstable
> > > > > > with same patch on tests of some days ago didn't happen.
> > > > > > 
> > > > > > Any ideas?
> > > > > > 
> > > > > > Thanks for any reply and sorry for my bad english.
> > > > > Bad news, qemu crash still happen even if this time in qemu log there
> > > > > is
> > > > > another output, see attachment.
> > > > > After take a look on the other patches I saw:
> > > > > http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.5-testing.git;a=commitdiff;h=7154fba0e51ec985ef621965d1b7120ad424fcbf
> > > > > With "Conflicts: hw/display/vga.c" in description I'll try to revert
> > > > > it
> > > > > instead.
> > > > > 
> > > > > Or someone can tell me another probable test I can try?
> > > > Tried also to revet the patch above with same result, so I retried with
> > > > qemu
> > > > from 4.5.0 and seems the crash happen also in this case...I'm going
> > > > crazy :(
> > Sorry, I missed this bit before. The only thing I could suggest at this
> > point, would be to make sure that you have a clean test environment.
> > Usually this happens when you have some "leftovers" from previous broken
> > tests.
> 
> I use make debball to be sure to track and remove all files on package update.
> Now I retried with latest xen-unstable and the qemu crash didn't happen, more
> exactly I used this:
> https://github.com/Fantu/Xen/commits/rebase/m2r-staging
> Latest test with regression based on latest stable-4.5, more exactly:
> https://github.com/Fantu/Xen/commits/rebase/m2r-testing
> Some days ago on same dom0 and domU I tried with latest stable version (that I
> use on only 2 production servers for now but I not saw the regression), more
> exactly:
> https://github.com/Fantu/Xen/commits/rebase/m2r-stable-4.5
> Dom0 debian 7 with kernel 3.16 from backports, seabios 1.8.1-2 from unstable
> and this xen configure:
> ./configure --prefix=/usr --disable-blktap1 --disable-qemu-traditional
> --disable-rombios --with-system-seabios=/usr/share/seabios/bios-256k.bin
> --with-extra-qemuu-configure-args="--enable-spice --enable-usb-redir"
> --disable-blktap2
> 
> I suppose that there is unexpected case caused by a backports or missed
> patch/es to backports from unstable.
> I not found with a fast look rilevant patch to try to revert, can anyone
> suggest me the more probable point/s for bisect and/or patch to revert or I
> must try full bisect 4.5.0->stable-4.5?

It is possible that this is an intermittent bug: it only shows once
every so many tests. I think you need to go over the full range
4.5.0->stable-4.5. Also it is important that you start from a working
baseline: you need to be sure that 4.5.0 works as expected.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-05-13 13:29                   ` [Qemu-devel] " Fabio Fantoni
  2015-05-15 10:26                     ` Stefano Stabellini
  2015-05-15 10:26                     ` [Qemu-devel] " Stefano Stabellini
@ 2015-10-08 15:58                     ` Andreas Kinzler
  2015-10-08 16:24                       ` Andreas Kinzler
  2015-10-09  7:56                       ` Fabio Fantoni
  2 siblings, 2 replies; 30+ messages in thread
From: Andreas Kinzler @ 2015-10-08 15:58 UTC (permalink / raw)
  To: xen-devel, stefano.stabellini, fabio.fantoni

Is this still current? I made an interesting observation:

I had no problems with SPICE and vanilla Xen 4.5.1 when using it on 
Gentoo with glibc 2.19/gcc 4.6.4.
Segfaults started when I switched to glibc 2.20/gcc 4.9.3 - I did not 
change Xen source code at all.
All this might be related to: 
https://lists.nongnu.org/archive/html/qemu-devel/2015-06/msg02764.html

Andreas

> https://github.com/Fantu/Xen/commits/rebase/m2r-staging
> Latest test with regression based on latest stable-4.5, more exactly:
> https://github.com/Fantu/Xen/commits/rebase/m2r-testing
> Some days ago on same dom0 and domU I tried with latest stable version 
> (that I use on only 2 production servers for now but I not saw the 
> regression), more exactly:
> https://github.com/Fantu/Xen/commits/rebase/m2r-stable-4.5
> Dom0 debian 7 with kernel 3.16 from backports, seabios 1.8.1-2 from 
> unstable and this xen configure:
> ./configure --prefix=/usr --disable-blktap1 --disable-qemu-traditional 
> --disable-rombios 
> --with-system-seabios=/usr/share/seabios/bios-256k.bin 
> --with-extra-qemuu-configure-args="--enable-spice --enable-usb-redir" 
> --disable-blktap2
>
> I suppose that there is unexpected case caused by a backports or 
> missed patch/es to backports from unstable.
> I not found with a fast look rilevant patch to try to revert, can 
> anyone suggest me the more probable point/s for bisect and/or patch to 
> revert or I must try full bisect 4.5.0->stable-4.5?
>

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-10-08 15:58                     ` Andreas Kinzler
@ 2015-10-08 16:24                       ` Andreas Kinzler
  2015-10-09  7:56                       ` Fabio Fantoni
  1 sibling, 0 replies; 30+ messages in thread
From: Andreas Kinzler @ 2015-10-08 16:24 UTC (permalink / raw)
  To: xen-devel

Is this still current?

I made an interesting observation: I had no problems with SPICE and 
vanilla Xen 4.5.1 when using it on Gentoo with glibc 2.19/gcc 4.6.4.
Segfaults started when I switched to glibc 2.20/gcc 4.9.3 - I did not 
change Xen source code at all.
All this might be related to: 
https://lists.nongnu.org/archive/html/qemu-devel/2015-06/msg02764.html

Andreas

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Regression: qemu crash of hvm domUs with spice (backtrace included)
  2015-10-08 15:58                     ` Andreas Kinzler
  2015-10-08 16:24                       ` Andreas Kinzler
@ 2015-10-09  7:56                       ` Fabio Fantoni
  2015-10-16 12:09                         ` Regression: qemu crash of hvm domUs with spice (backtrace included) - patch backport propose Fabio Fantoni
  1 sibling, 1 reply; 30+ messages in thread
From: Fabio Fantoni @ 2015-10-09  7:56 UTC (permalink / raw)
  To: Andreas Kinzler, xen-devel, stefano.stabellini

Il 08/10/2015 17:58, Andreas Kinzler ha scritto:
> Is this still current? I made an interesting observation:
>
> I had no problems with SPICE and vanilla Xen 4.5.1 when using it on 
> Gentoo with glibc 2.19/gcc 4.6.4.
> Segfaults started when I switched to glibc 2.20/gcc 4.9.3 - I did not 
> change Xen source code at all.
> All this might be related to: 
> https://lists.nongnu.org/archive/html/qemu-devel/2015-06/msg02764.html
>
> Andreas

Thanks for your mail.
The problem I had seems different, I not found the exactly cause but I 
solved using newer qemu (2.2 from unstable - now 4.6) with xen 4.5.1.
Big distro I saw already use newer qemu and should be ok.
I still using glibc < 2.20 in my debian servers, this is probably 
because I not had your problem but I think that backport the patch you 
linked can be useful for solve a qemu crash case, I'll test it in my 
next build and if I'll not found regression I'll require the backport 
for xen qemu gits.

>
>> https://github.com/Fantu/Xen/commits/rebase/m2r-staging
>> Latest test with regression based on latest stable-4.5, more exactly:
>> https://github.com/Fantu/Xen/commits/rebase/m2r-testing
>> Some days ago on same dom0 and domU I tried with latest stable 
>> version (that I use on only 2 production servers for now but I not 
>> saw the regression), more exactly:
>> https://github.com/Fantu/Xen/commits/rebase/m2r-stable-4.5
>> Dom0 debian 7 with kernel 3.16 from backports, seabios 1.8.1-2 from 
>> unstable and this xen configure:
>> ./configure --prefix=/usr --disable-blktap1 
>> --disable-qemu-traditional --disable-rombios 
>> --with-system-seabios=/usr/share/seabios/bios-256k.bin 
>> --with-extra-qemuu-configure-args="--enable-spice --enable-usb-redir" 
>> --disable-blktap2
>>
>> I suppose that there is unexpected case caused by a backports or 
>> missed patch/es to backports from unstable.
>> I not found with a fast look rilevant patch to try to revert, can 
>> anyone suggest me the more probable point/s for bisect and/or patch 
>> to revert or I must try full bisect 4.5.0->stable-4.5?
>>
>

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Regression: qemu crash of hvm domUs with spice (backtrace included) - patch backport propose
  2015-10-09  7:56                       ` Fabio Fantoni
@ 2015-10-16 12:09                         ` Fabio Fantoni
  2015-10-16 14:01                           ` Stefano Stabellini
  0 siblings, 1 reply; 30+ messages in thread
From: Fabio Fantoni @ 2015-10-16 12:09 UTC (permalink / raw)
  To: Andreas Kinzler, xen-devel, stefano.stabellini

Il 09/10/2015 09:56, Fabio Fantoni ha scritto:
> Il 08/10/2015 17:58, Andreas Kinzler ha scritto:
>> Is this still current? I made an interesting observation:
>>
>> I had no problems with SPICE and vanilla Xen 4.5.1 when using it on 
>> Gentoo with glibc 2.19/gcc 4.6.4.
>> Segfaults started when I switched to glibc 2.20/gcc 4.9.3 - I did not 
>> change Xen source code at all.
>> All this might be related to: 
>> https://lists.nongnu.org/archive/html/qemu-devel/2015-06/msg02764.html
>>
>> Andreas
>
> Thanks for your mail.
> The problem I had seems different, I not found the exactly cause but I 
> solved using newer qemu (2.2 from unstable - now 4.6) with xen 4.5.1.
> Big distro I saw already use newer qemu and should be ok.
> I still using glibc < 2.20 in my debian servers, this is probably 
> because I not had your problem but I think that backport the patch you 
> linked can be useful for solve a qemu crash case, I'll test it in my 
> next build and if I'll not found regression I'll require the backport 
> for xen qemu gits.
>
>>
>>> https://github.com/Fantu/Xen/commits/rebase/m2r-staging
>>> Latest test with regression based on latest stable-4.5, more exactly:
>>> https://github.com/Fantu/Xen/commits/rebase/m2r-testing
>>> Some days ago on same dom0 and domU I tried with latest stable 
>>> version (that I use on only 2 production servers for now but I not 
>>> saw the regression), more exactly:
>>> https://github.com/Fantu/Xen/commits/rebase/m2r-stable-4.5
>>> Dom0 debian 7 with kernel 3.16 from backports, seabios 1.8.1-2 from 
>>> unstable and this xen configure:
>>> ./configure --prefix=/usr --disable-blktap1 
>>> --disable-qemu-traditional --disable-rombios 
>>> --with-system-seabios=/usr/share/seabios/bios-256k.bin 
>>> --with-extra-qemuu-configure-args="--enable-spice 
>>> --enable-usb-redir" --disable-blktap2
>>>
>>> I suppose that there is unexpected case caused by a backports or 
>>> missed patch/es to backports from unstable.
>>> I not found with a fast look rilevant patch to try to revert, can 
>>> anyone suggest me the more probable point/s for bisect and/or patch 
>>> to revert or I must try full bisect 4.5.0->stable-4.5?
>>>
>>
>

I tried to use xen 4.6 with its qemu plus cherry-pick of this patch:
http://git.qemu.org/?p=qemu.git;a=commit;h=c6e484707f28b3e115e64122a0570f6b3c585489 
- spice-display: fix segfault in qemu_spice_create_update
Used some days without see regression.
Probably is useful apply it to qemu-xen unstable and 4.6 (about older 
versions I don't know and I not tested).
@Stefano Stabellini: can you take a look to it please?

About qemu used in unstable I think is good update to 2.4.0.1 now that 
xen 4.7 devel is started, I think we keep upstream qemu updated with too 
many latency and there is a more high bug risk with xen for major of 
distro that use newer qemu version (following stable version of both).

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Regression: qemu crash of hvm domUs with spice (backtrace included) - patch backport propose
  2015-10-16 12:09                         ` Regression: qemu crash of hvm domUs with spice (backtrace included) - patch backport propose Fabio Fantoni
@ 2015-10-16 14:01                           ` Stefano Stabellini
  2015-10-19 15:06                             ` Stefano Stabellini
  0 siblings, 1 reply; 30+ messages in thread
From: Stefano Stabellini @ 2015-10-16 14:01 UTC (permalink / raw)
  To: Fabio Fantoni; +Cc: stefano.stabellini, Andreas Kinzler, xen-devel

On Fri, 16 Oct 2015, Fabio Fantoni wrote:
> Il 09/10/2015 09:56, Fabio Fantoni ha scritto:
> > Il 08/10/2015 17:58, Andreas Kinzler ha scritto:
> > > Is this still current? I made an interesting observation:
> > > 
> > > I had no problems with SPICE and vanilla Xen 4.5.1 when using it on Gentoo
> > > with glibc 2.19/gcc 4.6.4.
> > > Segfaults started when I switched to glibc 2.20/gcc 4.9.3 - I did not
> > > change Xen source code at all.
> > > All this might be related to:
> > > https://lists.nongnu.org/archive/html/qemu-devel/2015-06/msg02764.html
> > > 
> > > Andreas
> > 
> > Thanks for your mail.
> > The problem I had seems different, I not found the exactly cause but I
> > solved using newer qemu (2.2 from unstable - now 4.6) with xen 4.5.1.
> > Big distro I saw already use newer qemu and should be ok.
> > I still using glibc < 2.20 in my debian servers, this is probably because I
> > not had your problem but I think that backport the patch you linked can be
> > useful for solve a qemu crash case, I'll test it in my next build and if
> > I'll not found regression I'll require the backport for xen qemu gits.
> > 
> > > 
> > > > https://github.com/Fantu/Xen/commits/rebase/m2r-staging
> > > > Latest test with regression based on latest stable-4.5, more exactly:
> > > > https://github.com/Fantu/Xen/commits/rebase/m2r-testing
> > > > Some days ago on same dom0 and domU I tried with latest stable version
> > > > (that I use on only 2 production servers for now but I not saw the
> > > > regression), more exactly:
> > > > https://github.com/Fantu/Xen/commits/rebase/m2r-stable-4.5
> > > > Dom0 debian 7 with kernel 3.16 from backports, seabios 1.8.1-2 from
> > > > unstable and this xen configure:
> > > > ./configure --prefix=/usr --disable-blktap1 --disable-qemu-traditional
> > > > --disable-rombios --with-system-seabios=/usr/share/seabios/bios-256k.bin
> > > > --with-extra-qemuu-configure-args="--enable-spice --enable-usb-redir"
> > > > --disable-blktap2
> > > > 
> > > > I suppose that there is unexpected case caused by a backports or missed
> > > > patch/es to backports from unstable.
> > > > I not found with a fast look rilevant patch to try to revert, can anyone
> > > > suggest me the more probable point/s for bisect and/or patch to revert
> > > > or I must try full bisect 4.5.0->stable-4.5?
> > > > 
> > > 
> > 
> 
> I tried to use xen 4.6 with its qemu plus cherry-pick of this patch:
> http://git.qemu.org/?p=qemu.git;a=commit;h=c6e484707f28b3e115e64122a0570f6b3c585489
> - spice-display: fix segfault in qemu_spice_create_update
> Used some days without see regression.
> Probably is useful apply it to qemu-xen unstable and 4.6 (about older versions
> I don't know and I not tested).
> @Stefano Stabellini: can you take a look to it please?

It looks like a reasonable effort.  My test machines are unavailable at
the moment, when they are back I'll commit (assuming it passes the
tests).


> About qemu used in unstable I think is good update to 2.4.0.1 now that xen 4.7
> devel is started, I think we keep upstream qemu updated with too many latency
> and there is a more high bug risk with xen for major of distro that use newer
> qemu version (following stable version of both).

I'll upgrade QEMU as soon as possible.

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Re: Regression: qemu crash of hvm domUs with spice (backtrace included) - patch backport propose
  2015-10-16 14:01                           ` Stefano Stabellini
@ 2015-10-19 15:06                             ` Stefano Stabellini
  0 siblings, 0 replies; 30+ messages in thread
From: Stefano Stabellini @ 2015-10-19 15:06 UTC (permalink / raw)
  To: Stefano Stabellini; +Cc: Fabio Fantoni, Andreas Kinzler, xen-devel

On Fri, 16 Oct 2015, Stefano Stabellini wrote:
> On Fri, 16 Oct 2015, Fabio Fantoni wrote:
> > Il 09/10/2015 09:56, Fabio Fantoni ha scritto:
> > > Il 08/10/2015 17:58, Andreas Kinzler ha scritto:
> > > > Is this still current? I made an interesting observation:
> > > > 
> > > > I had no problems with SPICE and vanilla Xen 4.5.1 when using it on Gentoo
> > > > with glibc 2.19/gcc 4.6.4.
> > > > Segfaults started when I switched to glibc 2.20/gcc 4.9.3 - I did not
> > > > change Xen source code at all.
> > > > All this might be related to:
> > > > https://lists.nongnu.org/archive/html/qemu-devel/2015-06/msg02764.html
> > > > 
> > > > Andreas
> > > 
> > > Thanks for your mail.
> > > The problem I had seems different, I not found the exactly cause but I
> > > solved using newer qemu (2.2 from unstable - now 4.6) with xen 4.5.1.
> > > Big distro I saw already use newer qemu and should be ok.
> > > I still using glibc < 2.20 in my debian servers, this is probably because I
> > > not had your problem but I think that backport the patch you linked can be
> > > useful for solve a qemu crash case, I'll test it in my next build and if
> > > I'll not found regression I'll require the backport for xen qemu gits.
> > > 
> > > > 
> > > > > https://github.com/Fantu/Xen/commits/rebase/m2r-staging
> > > > > Latest test with regression based on latest stable-4.5, more exactly:
> > > > > https://github.com/Fantu/Xen/commits/rebase/m2r-testing
> > > > > Some days ago on same dom0 and domU I tried with latest stable version
> > > > > (that I use on only 2 production servers for now but I not saw the
> > > > > regression), more exactly:
> > > > > https://github.com/Fantu/Xen/commits/rebase/m2r-stable-4.5
> > > > > Dom0 debian 7 with kernel 3.16 from backports, seabios 1.8.1-2 from
> > > > > unstable and this xen configure:
> > > > > ./configure --prefix=/usr --disable-blktap1 --disable-qemu-traditional
> > > > > --disable-rombios --with-system-seabios=/usr/share/seabios/bios-256k.bin
> > > > > --with-extra-qemuu-configure-args="--enable-spice --enable-usb-redir"
> > > > > --disable-blktap2
> > > > > 
> > > > > I suppose that there is unexpected case caused by a backports or missed
> > > > > patch/es to backports from unstable.
> > > > > I not found with a fast look rilevant patch to try to revert, can anyone
> > > > > suggest me the more probable point/s for bisect and/or patch to revert
> > > > > or I must try full bisect 4.5.0->stable-4.5?
> > > > > 
> > > > 
> > > 
> > 
> > I tried to use xen 4.6 with its qemu plus cherry-pick of this patch:
> > http://git.qemu.org/?p=qemu.git;a=commit;h=c6e484707f28b3e115e64122a0570f6b3c585489
> > - spice-display: fix segfault in qemu_spice_create_update
> > Used some days without see regression.
> > Probably is useful apply it to qemu-xen unstable and 4.6 (about older versions
> > I don't know and I not tested).
> > @Stefano Stabellini: can you take a look to it please?
> 
> It looks like a reasonable effort.  My test machines are unavailable at
> the moment, when they are back I'll commit (assuming it passes the
> tests).

I have applied the commit to staging and 4.6-testing

 
> > About qemu used in unstable I think is good update to 2.4.0.1 now that xen 4.7
> > devel is started, I think we keep upstream qemu updated with too many latency
> > and there is a more high bug risk with xen for major of distro that use newer
> > qemu version (following stable version of both).
> 
> I'll upgrade QEMU as soon as possible.
> 

^ permalink raw reply	[flat|nested] 30+ messages in thread

* Regression: qemu crash of hvm domUs with spice (backtrace included)
@ 2015-04-20 14:10 Fabio Fantoni
  0 siblings, 0 replies; 30+ messages in thread
From: Fabio Fantoni @ 2015-04-20 14:10 UTC (permalink / raw)
  To: xen-devel
  Cc: Anthony PERARD, spice-devel, Gerd Hoffmann, qemu-devel,
	Stefano Stabellini

[-- Attachment #1: Type: text/plain, Size: 1257 bytes --]

I updated xen and qemu from xen 4.5.0 with its upstream qemu included to 
xen 4.5.1-pre with qemu upstream from stable-4.5 (changed Config.mk to 
use revision "master").
After few minutes I booted windows 7 64 bit domU qemu crash, tried 2 
times with same result.

In the domU's qemu log:
> qemu-system-i386: malloc.c:3096: sYSMALLOc: Assertion `(old_top == 
> (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - 
> __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || 
> ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof 
> (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & 
> ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && 
> ((unsigned long)old_end & pagemask) == 0)' failed.
> Killing all inferiors

In attachment the full backtrace of qemu crash.

With a fast search after I saw the backtrace I found a probable cause of 
regression (I'm not sure):
http://xenbits.xen.org/gitweb/?p=staging/qemu-upstream-4.5-testing.git;a=commit;h=5c3402816aaddb15156c69df73c54abe4e1c76aa
spice: make sure we don't overflow ssd->buf

Added also qemu-devel and spice-devel as cc.

If you need more informations/tests tell me and I'll post them.

Thanks for any reply and sorry for my bad english.



[-- Attachment #2: qemu crash.log --]
[-- Type: text/plain, Size: 7540 bytes --]

Program received signal SIGABRT, Aborted.
[Switching to Thread 5234]
0x00007ffff3905165 in raise () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) bt full
#0  0x00007ffff3905165 in raise () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#1  0x00007ffff39083e0 in abort () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#2  0x00007ffff3948dea in ?? () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#3  0x00007ffff394bd13 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#4  0x00007ffff394da70 in malloc () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#5  0x00007ffff4d38550 in spice_malloc (n_bytes=1184900) at mem.c:93
        mem = <optimized out>
        __FUNCTION__ = "spice_malloc"
#6  0x00007ffff4d389be in spice_chunks_linearize (chunks=0x7fffdc1fb6b0)
    at mem.c:226
        data = <optimized out>
        p = <optimized out>
        i = <optimized out>
#7  0x00007ffff4d16b56 in canvas_bitmap_to_surface (
    canvas=canvas@entry=0x555556719de0, bitmap=bitmap@entry=0x7fffdc1a2c08, 
    palette=0x0, want_original=1) at ../spice-common/common/canvas_base.c:635
        src = <optimized out>
        image = <optimized out>
        format = <optimized out>
        __FUNCTION__ = "canvas_bitmap_to_surface"
---Type <return> to continue, or q <return> to quit---
#8  0x00007ffff4d16ce2 in canvas_get_bits (want_original=<optimized out>, 
    bitmap=0x7fffdc1a2c08, canvas=0x555556719de0)
    at ../spice-common/common/canvas_base.c:964
        palette = <optimized out>
#9  canvas_get_image_internal (canvas=canvas@entry=0x555556719de0, 
    image=0x7fffdc1a2bf0, want_original=<optimized out>, 
    want_original@entry=0, real_get=real_get@entry=1)
    at ../spice-common/common/canvas_base.c:1141
        descriptor = 0x7fffdc1a2bf0
        surface = <optimized out>
        converted = <optimized out>
        wanted_format = 1
        surface_format = <optimized out>
        saved_want_original = <optimized out>
        __FUNCTION__ = "canvas_get_image_internal"
#10 0x00007ffff4d173ba in canvas_get_image (
    canvas=canvas@entry=0x555556719de0, image=<optimized out>, 
    want_original=want_original@entry=0)
    at ../spice-common/common/canvas_base.c:1285
No locals.
#11 0x00007ffff4d1970e in canvas_draw_copy (spice_canvas=0x555556719de0, 
    bbox=0x7fffdc207a50, clip=<optimized out>, copy=0x7fffe4dfc320)
    at ../spice-common/common/canvas_base.c:2258
        canvas = 0x555556719de0
        dest_region = {extents = {x1 = 0, y1 = 708, x2 = 425, y2 = 728}, 
---Type <return> to continue, or q <return> to quit---
          data = 0x0}
        surface_canvas = <optimized out>
        src_image = <optimized out>
        rop = SPICE_ROP_COPY
        __FUNCTION__ = "canvas_draw_copy"
#12 0x00007ffff4cecffc in red_draw_qxl_drawable (
    worker=worker@entry=0x7fffe4423010, 
    drawable=drawable@entry=0x7fffe45d6a88) at red_worker.c:4394
        copy = {src_bitmap = 0x7fffdc1a2bf0, src_area = {left = 0, top = 677, 
            right = 425, bottom = 697}, rop_descriptor = 8, 
          scale_mode = 1 '\001', mask = {flags = 245 '\365', pos = {
              x = -173079809, y = -173079809}, bitmap = 0x0}}
        img1 = {descriptor = {id = 93825007287960, type = 48 '0', 
            flags = 193 '\301', width = 21845, height = 4210421981}, u = {
            bitmap = {format = 55 '7', flags = 10 '\n', x = 0, 
              y = 3867565524, stride = 32767, palette = 0x7fffe805fffc, 
              palette_id = 606579, data = 0x7fffdc000078}, quic = {
              data_size = 2615, data = 0x7fffe6865dd4}, surface = {
              surface_id = 2615}, lz_rgb = {data_size = 2615, 
              data = 0x7fffe6865dd4}, lz_plt = {flags = 55 '7', 
              data_size = 0, palette = 0x7fffe6865dd4, 
              palette_id = 140737086095356, data = 0x94173}, jpeg = {
              data_size = 2615, data = 0x7fffe6865dd4}, zlib_glz = {
              glz_data_size = 2615, data_size = 0, data = 0x7fffe6865dd4}, 
            jpeg_alpha = {flags = 55 '7', jpeg_size = 0, 
---Type <return> to continue, or q <return> to quit---
              data_size = 3867565524, data = 0x7fffe805fffc}}}
        img2 = {descriptor = {id = 140737060953572, type = 236 '\354', 
            flags = 93 ']', width = 32767, height = 0}, u = {bitmap = {
              format = 69 'E', flags = 105 'i', x = 32767, y = 128, 
              stride = 0, palette = 0x555556438fc0, 
              palette_id = 140737060952556, data = 0x9b5}, quic = {
              data_size = 4107102533, data = 0x80}, surface = {
              surface_id = 4107102533}, lz_rgb = {data_size = 4107102533, 
              data = 0x80}, lz_plt = {flags = 69 'E', data_size = 32767, 
              palette = 0x80, palette_id = 93825007849408, 
              data = 0x7fffe68659ec}, jpeg = {data_size = 4107102533, 
              data = 0x80}, zlib_glz = {glz_data_size = 4107102533, 
              data_size = 32767, data = 0x80}, jpeg_alpha = {flags = 69 'E', 
              jpeg_size = 32767, data_size = 128, data = 0x555556438fc0}}}
        surface = 0x7fffe44232f0
        canvas = 0x555556719de0
        clip = {type = 0 '\000', rects = 0x0}
        __FUNCTION__ = "red_draw_qxl_drawable"
#13 0x00007ffff4cf9295 in red_draw_drawable (drawable=0x7fffe45d6a88, 
    worker=0x7fffe4423010) at red_worker.c:4507
No locals.
#14 red_update_area (worker=worker@entry=0x7fffe4423010, 
    area=area@entry=0x7fffe4dfcb60, surface_id=surface_id@entry=0)
    at red_worker.c:4760
        container = <optimized out>
---Type <return> to continue, or q <return> to quit---
        surface = 0x7fffe44232f0
        ring = 0x7fffe4423308
        ring_item = <optimized out>
        rgn = {extents = {x1 = 0, y1 = 0, x2 = 1366, y2 = 768}, data = 0x0}
        last = 0x7fffe45d7898
        now = 0x7fffe45d6a88
        __FUNCTION__ = "red_update_area"
#15 0x00007ffff4d04d76 in handle_dev_update_async (opaque=0x7fffe4423010, 
    payload=<optimized out>) at red_worker.c:10847
        worker = 0x7fffe4423010
        msg = <optimized out>
        rect = {left = 0, top = 0, right = 1366, bottom = 768}
        qxl_dirty_rects = <optimized out>
        num_dirty_rects = <optimized out>
        surface = <optimized out>
        surface_id = 0
        qxl_area = {top = 0, left = 0, bottom = 768, right = 1366}
        clear_dirty_region = 1
        __FUNCTION__ = "handle_dev_update_async"
        __func__ = "handle_dev_update_async"
#16 0x00007ffff4ce5044 in dispatcher_handle_single_read (
    dispatcher=0x555556428db8) at dispatcher.c:139
        ret = <optimized out>
        type = <optimized out>
        msg = 0x5555563cbff0
---Type <return> to continue, or q <return> to quit---
        ack = 4294967295
        payload = 0x5555563a57d0 "P0\032\334\377\177"
#17 dispatcher_handle_recv_read (dispatcher=0x555556428db8)
    at dispatcher.c:162
No locals.
#18 0x00007ffff4d072dc in red_worker_main (arg=<optimized out>)
    at red_worker.c:12021
        events = <optimized out>
        i = <optimized out>
        num_events = 1
        timers_queue_timeout = <optimized out>
        worker = 0x7fffe4423010
        __FUNCTION__ = "red_worker_main"
#19 0x00007ffff3c64b50 in start_thread ()
   from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#20 0x00007ffff39ae95d in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#21 0x0000000000000000 in ?? ()
No symbol table info available.

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 30+ messages in thread

end of thread, other threads:[~2015-10-19 15:06 UTC | newest]

Thread overview: 30+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-04-20 14:10 [Qemu-devel] Regression: qemu crash of hvm domUs with spice (backtrace included) Fabio Fantoni
2015-04-21 10:49 ` Stefano Stabellini
2015-04-21 10:49 ` [Qemu-devel] " Stefano Stabellini
2015-04-21 11:38   ` Fabio Fantoni
2015-04-21 11:38   ` [Qemu-devel] " Fabio Fantoni
2015-04-21 12:53     ` Stefano Stabellini
2015-04-21 12:53     ` [Qemu-devel] " Stefano Stabellini
2015-05-11 15:04       ` Fabio Fantoni
2015-05-12  9:23         ` Fabio Fantoni
2015-05-12  9:23         ` [Qemu-devel] " Fabio Fantoni
2015-05-12 10:26           ` Fabio Fantoni
2015-05-12 13:54             ` Fabio Fantoni
2015-05-12 13:54             ` [Qemu-devel] " Fabio Fantoni
2015-05-12 14:38               ` Stefano Stabellini
2015-05-12 14:38               ` [Qemu-devel] " Stefano Stabellini
2015-05-12 14:44                 ` Stefano Stabellini
2015-05-13 13:29                   ` Fabio Fantoni
2015-05-13 13:29                   ` [Qemu-devel] " Fabio Fantoni
2015-05-15 10:26                     ` Stefano Stabellini
2015-05-15 10:26                     ` [Qemu-devel] " Stefano Stabellini
2015-10-08 15:58                     ` Andreas Kinzler
2015-10-08 16:24                       ` Andreas Kinzler
2015-10-09  7:56                       ` Fabio Fantoni
2015-10-16 12:09                         ` Regression: qemu crash of hvm domUs with spice (backtrace included) - patch backport propose Fabio Fantoni
2015-10-16 14:01                           ` Stefano Stabellini
2015-10-19 15:06                             ` Stefano Stabellini
2015-05-12 14:44                 ` Regression: qemu crash of hvm domUs with spice (backtrace included) Stefano Stabellini
2015-05-12 10:26           ` Fabio Fantoni
2015-05-11 15:04       ` Fabio Fantoni
  -- strict thread matches above, loose matches on Subject: below --
2015-04-20 14:10 Fabio Fantoni

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.