All of lore.kernel.org
 help / color / mirror / Atom feed
* [xen-unstable test] 11946: regressions - FAIL
@ 2012-02-13 20:16 xen.org
  2012-02-14 10:44 ` Ian Campbell
  0 siblings, 1 reply; 24+ messages in thread
From: xen.org @ 2012-02-13 20:16 UTC (permalink / raw)
  To: xen-devel; +Cc: ian.jackson

flight 11946 xen-unstable real [real]
http://www.chiark.greenend.org.uk/~xensrcts/logs/11946/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-credit2    7 debian-install            fail REGR. vs. 11944

Regressions which are regarded as allowable (not blocking):
 test-i386-i386-win           14 guest-start.2                fail   like 11944

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-winxpsp3  7 windows-install          fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64  7 windows-install        fail never pass
 test-amd64-i386-qemuu-rhel6hvm-intel  7 redhat-install         fail never pass
 test-i386-i386-xl-qemuu-winxpsp3  7 windows-install            fail never pass
 test-amd64-amd64-xl-pcipt-intel  9 guest-start                 fail never pass
 test-amd64-i386-rhel6hvm-amd 11 leak-check/check             fail   never pass
 test-amd64-i386-rhel6hvm-intel 11 leak-check/check             fail never pass
 test-amd64-i386-qemuu-rhel6hvm-amd  7 redhat-install           fail never pass
 test-amd64-amd64-win         16 leak-check/check             fail   never pass
 test-amd64-i386-win-vcpus1   16 leak-check/check             fail   never pass
 test-amd64-amd64-xl-winxpsp3 13 guest-stop                   fail   never pass
 test-i386-i386-xl-winxpsp3   13 guest-stop                   fail   never pass
 test-amd64-i386-win          16 leak-check/check             fail   never pass
 test-amd64-amd64-xl-win7-amd64 13 guest-stop                   fail never pass
 test-amd64-i386-xl-winxpsp3-vcpus1 13 guest-stop               fail never pass
 test-amd64-amd64-xl-win      13 guest-stop                   fail   never pass
 test-amd64-i386-xend-winxpsp3 16 leak-check/check             fail  never pass
 test-i386-i386-xl-win        13 guest-stop                   fail   never pass
 test-amd64-i386-xl-win7-amd64 13 guest-stop                   fail  never pass
 test-amd64-i386-xl-win-vcpus1 13 guest-stop                   fail  never pass

version targeted for testing:
 xen                  9207cc3a0862
baseline version:
 xen                  9ad1e42c341b

------------------------------------------------------------
People who touched revisions under test:
  David Vrabel <david.vrabel@citrix.com>
  Ian Campbell <ian.campbell@citrix.com>
  Ian Jackson <ian.jackson@eu.citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Julian Pidancet <julian.pidancet@gmail.com>
  Keir Fraser <keir@xen.org>
  Stefano Stabellini <stefano.stabellini@eu.citrix.com>
  Tim Deegan <tim@xen.org>
  Yongjie Ren <yongjie.ren@intel.com>
------------------------------------------------------------

jobs:
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-oldkern                                          pass    
 build-i386-oldkern                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-i386-i386-xl                                            pass    
 test-amd64-i386-rhel6hvm-amd                                 fail    
 test-amd64-i386-qemuu-rhel6hvm-amd                           fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-amd64-xl-win7-amd64                               fail    
 test-amd64-i386-xl-win7-amd64                                fail    
 test-amd64-i386-xl-credit2                                   fail    
 test-amd64-amd64-xl-pcipt-intel                              fail    
 test-amd64-i386-rhel6hvm-intel                               fail    
 test-amd64-i386-qemuu-rhel6hvm-intel                         fail    
 test-amd64-i386-xl-multivcpu                                 pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         pass    
 test-i386-i386-pair                                          pass    
 test-amd64-amd64-xl-sedf-pin                                 pass    
 test-amd64-amd64-pv                                          pass    
 test-amd64-i386-pv                                           pass    
 test-i386-i386-pv                                            pass    
 test-amd64-amd64-xl-sedf                                     pass    
 test-amd64-i386-win-vcpus1                                   fail    
 test-amd64-i386-xl-win-vcpus1                                fail    
 test-amd64-i386-xl-winxpsp3-vcpus1                           fail    
 test-amd64-amd64-win                                         fail    
 test-amd64-i386-win                                          fail    
 test-i386-i386-win                                           fail    
 test-amd64-amd64-xl-win                                      fail    
 test-i386-i386-xl-win                                        fail    
 test-amd64-amd64-xl-qemuu-winxpsp3                           fail    
 test-i386-i386-xl-qemuu-winxpsp3                             fail    
 test-amd64-i386-xend-winxpsp3                                fail    
 test-amd64-amd64-xl-winxpsp3                                 fail    
 test-i386-i386-xl-winxpsp3                                   fail    


------------------------------------------------------------
sg-report-flight on woking.cam.xci-test.com
logs: /home/xc_osstest/logs
images: /home/xc_osstest/images

Logs, config files, etc. are available at
    http://www.chiark.greenend.org.uk/~xensrcts/logs

Test harness code can be found at
    http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
changeset:   24790:9207cc3a0862
tag:         tip
user:        David Vrabel <david.vrabel@citrix.com>
date:        Mon Feb 13 13:34:47 2012 +0000
    
    libfdt: add to build
    
    Signed-off-by: David Vrabel <david.vrabel@citrix.com>
    Acked-by: Tim Deegan <tim@xen.org>
    Committed-by: Keir Fraser <keir@xen.org>
    
    
changeset:   24789:e060d1bd7b60
user:        David Vrabel <david.vrabel@citrix.com>
date:        Mon Feb 13 13:34:08 2012 +0000
    
    libfdt: fixup libfdt_env.h for xen
    
    Signed-off-by: David Vrabel <david.vrabel@citrix.com>
    Acked-by: Tim Deegan <tim@xen.org>
    Committed-by: Keir Fraser <keir@xen.org>
    
    
changeset:   24788:fcc188f21e47
user:        David Vrabel <david.vrabel@citrix.com>
date:        Mon Feb 13 13:33:26 2012 +0000
    
    libfdt: add version 1.3.0
    
    Add libfdt 1.3.0 from http://git.jdl.com/gitweb/?p=dtc.git
    
    This will be used by Xen to parse the DTBs provided by bootloaders on
    ARM platforms.
    
    Signed-off-by: David Vrabel <david.vrabel@citrix.com>
    Acked-by: Tim Deegan <tim@xen.org>
    Committed-by: Keir Fraser <keir@xen.org>
    
    
changeset:   24787:bd0a11ed1a67
user:        Ian Campbell <ian.campbell@citrix.com>
date:        Mon Feb 13 12:53:28 2012 +0000
    
    MAINTAINERS: Add entry for ARM w/ virt extensions port
    
    Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
    Committed-by: Keir Fraser <keir@xen.org>
    
    
changeset:   24786:79fe73117c12
user:        Julian Pidancet <julian.pidancet@gmail.com>
date:        Mon Feb 13 12:50:46 2012 +0000
    
    firmware: Introduce CONFIG_ROMBIOS and CONFIG_SEABIOS options
    
    This patch introduces configuration options allowing to built either a
    rombios only or a seabios only hvmloader.
    
    Building option ROMs like vgabios or etherboot is only enabled for a
    rombios hvmloader, since SeaBIOS takes care or extracting option ROMs
    itself from the PCI devices (these option ROMs are provided by the
    device model and do not need to be built in hvmloader).
    
    The Makefile in tools/firmware/ now only checks for bcc if rombios is
    enabled.
    
    These two configuration options are left on by default to remain
    compatible.
    
    Signed-off-by: Julian Pidancet <julian.pidancet@gmail.com>
    Acked-by: Ian Campbell <ian.campbell@citrix.com>
    
    
changeset:   24785:e4d8d2524407
user:        Julian Pidancet <julian.pidancet@gmail.com>
date:        Mon Feb 13 12:50:04 2012 +0000
    
    hvmloader: Move option ROM loading into a separate optionnal file
    
    Make load_rom field in struct bios_config an optionnal callback rather
    than a boolean value. It allow BIOS specific code to implement it's
    own option ROM loading methods.
    
    Facilities to scan PCI devices, extract an deploy ROMs are moved into
    a separate file that can be compiled optionnaly.
    
    Signed-off-by: Julian Pidancet <julian.pidancet@gmail.com>
    Acked-by: Ian Campbell <ian.campbell@citrix.com>
    
    
changeset:   24784:ab47cfef2b0a
user:        Julian Pidancet <julian.pidancet@gmail.com>
date:        Mon Feb 13 12:49:06 2012 +0000
    
    firmware: Use mkhex from hvmloader directory for etherboot ROMs
    
    To remain consistent with how other ROMs are built into hvmloader,
    call mkhex on etherboot ROMs from the hvmloader directory, instead of
    the etherboot directory. In other words, eb-roms.h is not used any
    more.
    
    Introduce ETHERBOOT_NICS config option to choose which ROMs should be
    built (kept rtl8139 and 8086100e per default as before).
    
    Signed-off-by: Julian Pidancet <julian.pidancet@gmail.com>
    Acked-by: Ian Campbell <ian.campbell@citrix.com>
    
    
changeset:   24783:0fe9e2556e20
user:        Julian Pidancet <julian.pidancet@gmail.com>
date:        Mon Feb 13 12:48:20 2012 +0000
    
    hvmloader: Allow the mkhex command to take several file arguments
    Signed-off-by: Julian Pidancet <julian.pidancet@gmail.com>
    Acked-by: Ian Campbell <ian.campbell@citrix.com>
    
    
changeset:   24782:e1f10d12b9fe
user:        Julian Pidancet <julian.pidancet@gmail.com>
date:        Mon Feb 13 12:47:46 2012 +0000
    
    hvmloader: Only compile 32bitbios_support.c when rombios is enabled
    
    32bitbios_support.c only contains code specific to rombios, and should
    not be built-in when building hvmloader for SeaBIOS only (as for
    rombios.c).
    
    Signed-off-by: Julian Pidancet <julian.pidancet@gmail.com>
    Acked-by: Ian Campbell <ian.campbell@citrix.com>
    
    
changeset:   24781:6ae5506e49ab
user:        Jan Beulich <jbeulich@suse.com>
date:        Mon Feb 13 13:12:30 2012 +0100
    
    x86/vMCE: MC{G,i}_CTL handling adjustments
    
    - g_mcg_cap was read to determine whether MCG_CTL exists before it got
      initialized
    - h_mci_ctrl[] and dom_vmce()->mci_ctl[] both got initialized via
      memset() with an inappropriate size (hence causing a [minor?]
      information leak)
    
    Signed-off-by: Jan Beulich <jbeulich@suse.com>
    Acked-by: Keir Fraser <keir@xen.org>
    
    
changeset:   24780:e953d536d3c6
user:        Jan Beulich <jbeulich@suse.com>
date:        Mon Feb 13 13:09:02 2012 +0100
    
    x86/paging: use clear_guest() for zero-filling guest buffers
    
    While static arrays of all zeros may be tolerable (but are simply
    inefficient now that we have the necessary infrastructure), using on-
    stack arrays for this purpose (particularly when their size doesn't
    have an upper limit enforced) is calling for eventual problems (even
    if the code can be reached via administrative interfaces only).
    
    Signed-off-by: Jan Beulich <jbeulich@suse.com>
    Acked-by: Tim Deegan <tim@xen.org>
    
    
changeset:   24779:9ad1e42c341b
user:        Ian Campbell <ian.campbell@citrix.com>
date:        Fri Feb 10 17:24:50 2012 +0000
    
    xend: populate HVM guest grant table on boot
    
    Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
    Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
    
    
========================================
commit 8cc8a3651c9c5bc2d0086d12f4b870fc525b9387
Author: Jan Beulich <JBeulich@suse.com>
Date:   Tue Feb 7 18:42:56 2012 +0000

    qemu-dm: fix unregister_iomem()
    
    This function (introduced quite a long time ago in
    e7911109f4321e9ba0cc56a253b653600aa46bea - "disable qemu PCI
    devices in HVM domains") appears to be completely broken, causing
    the regression reported in
    http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1805 (due to
    the newly added caller of it in
    56d7747a3cf811910c4cf865e1ebcb8b82502005 - "qemu: clean up
    MSI-X table handling"). It's unclear how the function can ever have
    fulfilled its purpose: the value returned by iomem_index() is *not* an
    index into mmio[].
    
    Additionally, fix two problems:
    - unregister_iomem() must not clear mmio[].start, otherwise
      cpu_register_physical_memory() won't be able to re-use the previous
      slot, thus causing a leak
    - cpu_unregister_io_memory() must not check mmio[].size, otherwise it
      won't properly clean up entries (temporarily) squashed through
      unregister_iomem()
    
    Signed-off-by: Jan Beulich <jbeulich@suse.com>
    Tested-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
    Tested-by: Yongjie Ren <yongjie.ren@intel.com>

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-02-13 20:16 [xen-unstable test] 11946: regressions - FAIL xen.org
@ 2012-02-14 10:44 ` Ian Campbell
  2012-02-14 19:17   ` Daniel De Graaf
  2012-03-27 10:36   ` Ian Campbell
  0 siblings, 2 replies; 24+ messages in thread
From: Ian Campbell @ 2012-02-14 10:44 UTC (permalink / raw)
  To: xen.org; +Cc: Andrew Cooper, Daniel De Graaf, xen-devel, Keir Fraser

On Mon, 2012-02-13 at 20:16 +0000, xen.org wrote:
> flight 11946 xen-unstable real [real]
> http://www.chiark.greenend.org.uk/~xensrcts/logs/11946/
> 
> Regressions :-(
> 
> Tests which did not succeed and are blocking,
> including tests which could not be run:
>  test-amd64-i386-xl-credit2    7 debian-install            fail REGR. vs. 11944

Host crash:
http://www.chiark.greenend.org.uk/~xensrcts/logs/11946/test-amd64-i386-xl-credit2/serial-woodlouse.log

This is the debug Andrew Cooper added recently to track down the IRQ
assertion we've been seeing, sadly it looks like the debug code tries to
call xfree from interrupt context and therefore doesn't produce full
output :-(

Or is 24675:d82a1e3d3c65 ("xsm: Add security label to IRQ debug output")
at fault for adding the xfree in what may be an IRQ context? (are
keyhandlers run in IRQ context?)

A skanky quick "fix" follows.

        Feb 13 17:17:29.777522 (XEN) *** IRQ BUG found ***
        Feb 13 17:19:32.594539 (XEN) CPU0 -Testing vector 229 from bitmap 34,48,57,64,72,75,80,83,88,97,104-105,113,120-121,129,136,144,152,160,168,176,184,192,202
        Feb 13 17:19:32.617515 (XEN) Guest interrupt information:
        Feb 13 17:19:32.617536 (XEN)    IRQ:   0 affinity:001 vec:f0 type=IO-APIC-edge    status=00000000 mapped, unbound
        Feb 13 17:19:32.617567 (XEN) Assertion '!in_irq()' failed at xmalloc_tlsf.c:607
        Feb 13 17:19:32.626489 (XEN) ----[ Xen-4.2-unstable  x86_64  debug=y  Not tainted ]----
        Feb 13 17:19:32.626512 (XEN) CPU:    0
        Feb 13 17:19:32.626525 (XEN) RIP:    e008:[<ffff82c48012c842>] xfree+0x33/0x121
        Feb 13 17:19:32.641496 (XEN) RFLAGS: 0000000000010002   CONTEXT: hypervisor
        Feb 13 17:19:32.641519 (XEN) rax: ffff82c4802d0800   rbx: ffff8301a7e00080   rcx: 0000000000000000
        Feb 13 17:19:32.650560 (XEN) rdx: 0000000000000000   rsi: 0000000000000083   rdi: 0000000000000000
        Feb 13 17:19:32.665510 (XEN) rbp: ffff82c4802afd18   rsp: ffff82c4802afcf8   r8:  0000000000000004
        Feb 13 17:19:32.665550 (XEN) r9:  0000000000000000   r10: 0000000000000006   r11: ffff82c480224aa0
        Feb 13 17:19:32.673509 (XEN) r12: ffff8301a7e00580   r13: 0000000000000005   r14: ffff82c4802aff18
        Feb 13 17:19:32.685503 (XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4: 00000000000006f0
        Feb 13 17:19:32.685537 (XEN) cr3: 00000001a7f54000   cr2: 00000000c4b4ee84
        Feb 13 17:19:32.697505 (XEN) ds: 007b   es: 007b   fs: 00d8   gs: 0000   ss: 0000   cs: e008
        Feb 13 17:19:32.697540 (XEN) Xen stack trace from rsp=ffff82c4802afcf8:
        Feb 13 17:19:32.706513 (XEN)    ffff8301a7e00080 ffff8301a7e00580 0000000000000005 ffff82c4802aff18
        Feb 13 17:19:32.721495 (XEN)    ffff82c4802afd88 ffff82c4801658ee ffff82c4802afd38 ffff82c48010098a
        Feb 13 17:19:32.721531 (XEN)    00000400802afd68 0000000000000083 ffff8301a7e000a8 0000000000000000
        Feb 13 17:19:32.729495 (XEN)    00000000fffffffa 00000000000000e5 ffff8301a7e00580 0000000000000005
        Feb 13 17:19:32.738490 (XEN)    ffff82c4802aff18 ffff8301a7e005a8 ffff82c4802afe28 ffff82c480167781
        Feb 13 17:19:32.738515 (XEN)    ffff8301a7ece000 ffff82c4802afde8 0000000000000000 ffff82c4802aff18
        Feb 13 17:19:32.750497 (XEN)    ffff82c4802aff18 0000000000000002 ffff82c4802aff18 ffff82c4802fa060
        Feb 13 17:19:32.762568 (XEN)    000000e500000000 ffff82c4802fa060 ffff82c4802afe08 ffff82c48017bd51
        Feb 13 17:19:32.762596 (XEN)    ffff82c4802aff18 ffff82c4802aff18 ffff82c48025e380 ffff82c4802aff18
        Feb 13 17:19:32.773513 (XEN)    00000000ffffffff 0000000000000002 00007d3b7fd501a7 ffff82c4801525d0
        Feb 13 17:19:32.785503 (XEN)    0000000000000002 00000000ffffffff ffff82c4802aff18 ffff82c48025e380
        Feb 13 17:19:32.785539 (XEN)    ffff82c4802afee0 ffff82c4802aff18 0000001863058413 00000000000c0000
        Feb 13 17:19:32.794514 (XEN)    000000000e1ff99c 000000000000c701 ffff82c4802f9a90 0000000000000000
        Feb 13 17:19:32.809503 (XEN)    0000000000000000 ffff8301a7f5dc80 0000000000000000 0000002000000000
        Feb 13 17:19:32.809529 (XEN)    ffff82c4801581a9 000000000000e008 0000000000000246 ffff82c4802afee0
        Feb 13 17:19:32.814513 (XEN)    0000000000000000 ffff82c4802aff10 ffff82c48015a647 0000000000000000
        Feb 13 17:19:32.829506 (XEN)    ffff8300d7cfb000 ffff8300d7af9000 0000000000000000 ffff82c4802afd88
        Feb 13 17:19:32.829549 (XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
        Feb 13 17:19:32.841510 (XEN)    00000000dfc91f90 00000000deadbeef 0000000000000000 0000000000000000
        Feb 13 17:19:32.853508 (XEN)    0000000000000000 0000000000000000 0000000000000000 00000000deadbeef
        Feb 13 17:19:32.858496 (XEN) Xen call trace:
        Feb 13 17:19:32.858518 (XEN)    [<ffff82c48012c842>] xfree+0x33/0x121
        Feb 13 17:19:32.858547 (XEN)    [<ffff82c4801658ee>] dump_irqs+0x2a3/0x2ca
        Feb 13 17:19:32.870500 (XEN)    [<ffff82c480167781>] smp_irq_move_cleanup_interrupt+0x303/0x37b
        Feb 13 17:19:32.870554 (XEN)    [<ffff82c4801525d0>] irq_move_cleanup_interrupt+0x30/0x40
        Feb 13 17:19:32.885510 (XEN)    [<ffff82c4801581a9>] default_idle+0x99/0x9e
        Feb 13 17:19:32.885541 (XEN)    [<ffff82c48015a647>] idle_loop+0x6c/0x7c
        Feb 13 17:19:32.897496 (XEN)    
        Feb 13 17:19:32.897510 (XEN) 
        Feb 13 17:19:32.897520 (XEN) ****************************************
        Feb 13 17:19:32.897537 (XEN) Panic on CPU 0:
        Feb 13 17:19:32.905499 (XEN) Assertion '!in_irq()' failed at xmalloc_tlsf.c:607
        Feb 13 17:19:32.905522 (XEN) ****************************************
        Feb 13 17:19:32.913488 (XEN) 
        Feb 13 17:19:32.913506 (XEN) Reboot in five seconds...

# HG changeset patch
# User Ian Campbell <ian.campbell@citrix.com>
# Date 1329216241 0
# Node ID 738424a5e5a5053c75cfbe64f6675b5d756daf1b
# Parent  0ba87b95e80bae059fe70b4b117dcc409f2471ef
xen: don't try to print IRQ SSID in IRQ debug from irq context.

It is not possible to call xfree() in that context.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>

diff -r 0ba87b95e80b -r 738424a5e5a5 xen/arch/x86/irq.c
--- a/xen/arch/x86/irq.c	Mon Feb 13 17:26:08 2012 +0000
+++ b/xen/arch/x86/irq.c	Tue Feb 14 10:44:01 2012 +0000
@@ -2026,7 +2026,7 @@ static void dump_irqs(unsigned char key)
         if ( !irq_desc_initialized(desc) || desc->handler == &no_irq_type )
             continue;
 
-        ssid = xsm_show_irq_sid(irq);
+        ssid = in_irq() ? NULL : xsm_show_irq_sid(irq);
 
         spin_lock_irqsave(&desc->lock, flags);
 
@@ -2073,7 +2073,8 @@ static void dump_irqs(unsigned char key)
 
         spin_unlock_irqrestore(&desc->lock, flags);
 
-        xfree(ssid);
+        if ( ssid )
+                xfree(ssid);
     }
 
     dump_ioapic_irq_info();

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-02-14 10:44 ` Ian Campbell
@ 2012-02-14 19:17   ` Daniel De Graaf
  2012-03-27 10:36   ` Ian Campbell
  1 sibling, 0 replies; 24+ messages in thread
From: Daniel De Graaf @ 2012-02-14 19:17 UTC (permalink / raw)
  To: Ian Campbell; +Cc: Andrew Cooper, Keir Fraser, xen-devel, xen.org

On 02/14/2012 05:44 AM, Ian Campbell wrote:
> On Mon, 2012-02-13 at 20:16 +0000, xen.org wrote:
>> flight 11946 xen-unstable real [real]
>> http://www.chiark.greenend.org.uk/~xensrcts/logs/11946/
>>
>> Regressions :-(
>>
>> Tests which did not succeed and are blocking,
>> including tests which could not be run:
>>  test-amd64-i386-xl-credit2    7 debian-install            fail REGR. vs. 11944
> 
> Host crash:
> http://www.chiark.greenend.org.uk/~xensrcts/logs/11946/test-amd64-i386-xl-credit2/serial-woodlouse.log
> 
> This is the debug Andrew Cooper added recently to track down the IRQ
> assertion we've been seeing, sadly it looks like the debug code tries to
> call xfree from interrupt context and therefore doesn't produce full
> output :-(
> 
> Or is 24675:d82a1e3d3c65 ("xsm: Add security label to IRQ debug output")
> at fault for adding the xfree in what may be an IRQ context? (are
> keyhandlers run in IRQ context?)

Keyhandlers are not run in IRQ context (or at least, the primary methods of
invoking them don't run there - serial keypress, xl debug-key). The placement
of the xsm call and xfree was to avoid a similar backtrace from attempting
allocation while holding the irq's spinlock.

> A skanky quick "fix" follows.
> 
>         Feb 13 17:17:29.777522 (XEN) *** IRQ BUG found ***
>         Feb 13 17:19:32.594539 (XEN) CPU0 -Testing vector 229 from bitmap 34,48,57,64,72,75,80,83,88,97,104-105,113,120-121,129,136,144,152,160,168,176,184,192,202
>         Feb 13 17:19:32.617515 (XEN) Guest interrupt information:
>         Feb 13 17:19:32.617536 (XEN)    IRQ:   0 affinity:001 vec:f0 type=IO-APIC-edge    status=00000000 mapped, unbound
>         Feb 13 17:19:32.617567 (XEN) Assertion '!in_irq()' failed at xmalloc_tlsf.c:607
>         Feb 13 17:19:32.626489 (XEN) ----[ Xen-4.2-unstable  x86_64  debug=y  Not tainted ]----
>         Feb 13 17:19:32.626512 (XEN) CPU:    0
>         Feb 13 17:19:32.626525 (XEN) RIP:    e008:[<ffff82c48012c842>] xfree+0x33/0x121
>         Feb 13 17:19:32.641496 (XEN) RFLAGS: 0000000000010002   CONTEXT: hypervisor
>         Feb 13 17:19:32.641519 (XEN) rax: ffff82c4802d0800   rbx: ffff8301a7e00080   rcx: 0000000000000000
>         Feb 13 17:19:32.650560 (XEN) rdx: 0000000000000000   rsi: 0000000000000083   rdi: 0000000000000000
>         Feb 13 17:19:32.665510 (XEN) rbp: ffff82c4802afd18   rsp: ffff82c4802afcf8   r8:  0000000000000004
>         Feb 13 17:19:32.665550 (XEN) r9:  0000000000000000   r10: 0000000000000006   r11: ffff82c480224aa0
>         Feb 13 17:19:32.673509 (XEN) r12: ffff8301a7e00580   r13: 0000000000000005   r14: ffff82c4802aff18
>         Feb 13 17:19:32.685503 (XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4: 00000000000006f0
>         Feb 13 17:19:32.685537 (XEN) cr3: 00000001a7f54000   cr2: 00000000c4b4ee84
>         Feb 13 17:19:32.697505 (XEN) ds: 007b   es: 007b   fs: 00d8   gs: 0000   ss: 0000   cs: e008
>         Feb 13 17:19:32.697540 (XEN) Xen stack trace from rsp=ffff82c4802afcf8:
>         Feb 13 17:19:32.706513 (XEN)    ffff8301a7e00080 ffff8301a7e00580 0000000000000005 ffff82c4802aff18
>         Feb 13 17:19:32.721495 (XEN)    ffff82c4802afd88 ffff82c4801658ee ffff82c4802afd38 ffff82c48010098a
>         Feb 13 17:19:32.721531 (XEN)    00000400802afd68 0000000000000083 ffff8301a7e000a8 0000000000000000
>         Feb 13 17:19:32.729495 (XEN)    00000000fffffffa 00000000000000e5 ffff8301a7e00580 0000000000000005
>         Feb 13 17:19:32.738490 (XEN)    ffff82c4802aff18 ffff8301a7e005a8 ffff82c4802afe28 ffff82c480167781
>         Feb 13 17:19:32.738515 (XEN)    ffff8301a7ece000 ffff82c4802afde8 0000000000000000 ffff82c4802aff18
>         Feb 13 17:19:32.750497 (XEN)    ffff82c4802aff18 0000000000000002 ffff82c4802aff18 ffff82c4802fa060
>         Feb 13 17:19:32.762568 (XEN)    000000e500000000 ffff82c4802fa060 ffff82c4802afe08 ffff82c48017bd51
>         Feb 13 17:19:32.762596 (XEN)    ffff82c4802aff18 ffff82c4802aff18 ffff82c48025e380 ffff82c4802aff18
>         Feb 13 17:19:32.773513 (XEN)    00000000ffffffff 0000000000000002 00007d3b7fd501a7 ffff82c4801525d0
>         Feb 13 17:19:32.785503 (XEN)    0000000000000002 00000000ffffffff ffff82c4802aff18 ffff82c48025e380
>         Feb 13 17:19:32.785539 (XEN)    ffff82c4802afee0 ffff82c4802aff18 0000001863058413 00000000000c0000
>         Feb 13 17:19:32.794514 (XEN)    000000000e1ff99c 000000000000c701 ffff82c4802f9a90 0000000000000000
>         Feb 13 17:19:32.809503 (XEN)    0000000000000000 ffff8301a7f5dc80 0000000000000000 0000002000000000
>         Feb 13 17:19:32.809529 (XEN)    ffff82c4801581a9 000000000000e008 0000000000000246 ffff82c4802afee0
>         Feb 13 17:19:32.814513 (XEN)    0000000000000000 ffff82c4802aff10 ffff82c48015a647 0000000000000000
>         Feb 13 17:19:32.829506 (XEN)    ffff8300d7cfb000 ffff8300d7af9000 0000000000000000 ffff82c4802afd88
>         Feb 13 17:19:32.829549 (XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
>         Feb 13 17:19:32.841510 (XEN)    00000000dfc91f90 00000000deadbeef 0000000000000000 0000000000000000
>         Feb 13 17:19:32.853508 (XEN)    0000000000000000 0000000000000000 0000000000000000 00000000deadbeef
>         Feb 13 17:19:32.858496 (XEN) Xen call trace:
>         Feb 13 17:19:32.858518 (XEN)    [<ffff82c48012c842>] xfree+0x33/0x121
>         Feb 13 17:19:32.858547 (XEN)    [<ffff82c4801658ee>] dump_irqs+0x2a3/0x2ca
>         Feb 13 17:19:32.870500 (XEN)    [<ffff82c480167781>] smp_irq_move_cleanup_interrupt+0x303/0x37b
>         Feb 13 17:19:32.870554 (XEN)    [<ffff82c4801525d0>] irq_move_cleanup_interrupt+0x30/0x40
>         Feb 13 17:19:32.885510 (XEN)    [<ffff82c4801581a9>] default_idle+0x99/0x9e
>         Feb 13 17:19:32.885541 (XEN)    [<ffff82c48015a647>] idle_loop+0x6c/0x7c
>         Feb 13 17:19:32.897496 (XEN)    
>         Feb 13 17:19:32.897510 (XEN) 
>         Feb 13 17:19:32.897520 (XEN) ****************************************
>         Feb 13 17:19:32.897537 (XEN) Panic on CPU 0:
>         Feb 13 17:19:32.905499 (XEN) Assertion '!in_irq()' failed at xmalloc_tlsf.c:607
>         Feb 13 17:19:32.905522 (XEN) ****************************************
>         Feb 13 17:19:32.913488 (XEN) 
>         Feb 13 17:19:32.913506 (XEN) Reboot in five seconds...
> 
> # HG changeset patch
> # User Ian Campbell <ian.campbell@citrix.com>
> # Date 1329216241 0
> # Node ID 738424a5e5a5053c75cfbe64f6675b5d756daf1b
> # Parent  0ba87b95e80bae059fe70b4b117dcc409f2471ef
> xen: don't try to print IRQ SSID in IRQ debug from irq context.
> 
> It is not possible to call xfree() in that context.
> 
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> 
> diff -r 0ba87b95e80b -r 738424a5e5a5 xen/arch/x86/irq.c
> --- a/xen/arch/x86/irq.c	Mon Feb 13 17:26:08 2012 +0000
> +++ b/xen/arch/x86/irq.c	Tue Feb 14 10:44:01 2012 +0000
> @@ -2026,7 +2026,7 @@ static void dump_irqs(unsigned char key)
>          if ( !irq_desc_initialized(desc) || desc->handler == &no_irq_type )
>              continue;
>  
> -        ssid = xsm_show_irq_sid(irq);
> +        ssid = in_irq() ? NULL : xsm_show_irq_sid(irq);
>  
>          spin_lock_irqsave(&desc->lock, flags);
>  
> @@ -2073,7 +2073,8 @@ static void dump_irqs(unsigned char key)
>  
>          spin_unlock_irqrestore(&desc->lock, flags);
>  
> -        xfree(ssid);
> +        if ( ssid )
> +                xfree(ssid);
>      }
>  
>      dump_ioapic_irq_info();
> 
> 
> 

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-02-14 10:44 ` Ian Campbell
  2012-02-14 19:17   ` Daniel De Graaf
@ 2012-03-27 10:36   ` Ian Campbell
  2012-03-27 10:52     ` Jan Beulich
  2012-05-04 19:48     ` AP
  1 sibling, 2 replies; 24+ messages in thread
From: Ian Campbell @ 2012-03-27 10:36 UTC (permalink / raw)
  To: Ian Jackson; +Cc: Andrew Cooper, Daniel De Graaf, xen-devel, Keir (Xen.org)

On Tue, 2012-02-14 at 10:44 +0000, Ian Campbell wrote:
> On Mon, 2012-02-13 at 20:16 +0000, xen.org wrote:
> > flight 11946 xen-unstable real [real]
> > http://www.chiark.greenend.org.uk/~xensrcts/logs/11946/
> > 
> > Regressions :-(
> > 
> > Tests which did not succeed and are blocking,
> > including tests which could not be run:
> >  test-amd64-i386-xl-credit2    7 debian-install            fail REGR. vs. 11944
> 
> Host crash:
> http://www.chiark.greenend.org.uk/~xensrcts/logs/11946/test-amd64-i386-xl-credit2/serial-woodlouse.log
> 
> This is the debug Andrew Cooper added recently to track down the IRQ
> assertion we've been seeing, sadly it looks like the debug code tries to
> call xfree from interrupt context and therefore doesn't produce full
> output :-(

Are we still seeing the issue this debugging was intended to address? We
don't seem to be seeing the host crashes any more. Should the debug code
be patched up as in the following patch, otherwise when we do see it it
doesn't end up printing any useful info.

Someone recently reported bugs.debian.org/665433 to Debian, is this the
same underlying issue? That report is with Xen 4.0 FWIW.

> Or is 24675:d82a1e3d3c65 ("xsm: Add security label to IRQ debug output")
> at fault for adding the xfree in what may be an IRQ context? (are
> keyhandlers run in IRQ context?)
> 
> A skanky quick "fix" follows.
> 
>         Feb 13 17:17:29.777522 (XEN) *** IRQ BUG found ***
>         Feb 13 17:19:32.594539 (XEN) CPU0 -Testing vector 229 from bitmap 34,48,57,64,72,75,80,83,88,97,104-105,113,120-121,129,136,144,152,160,168,176,184,192,202
>         Feb 13 17:19:32.617515 (XEN) Guest interrupt information:
>         Feb 13 17:19:32.617536 (XEN)    IRQ:   0 affinity:001 vec:f0 type=IO-APIC-edge    status=00000000 mapped, unbound
>         Feb 13 17:19:32.617567 (XEN) Assertion '!in_irq()' failed at xmalloc_tlsf.c:607
>         Feb 13 17:19:32.626489 (XEN) ----[ Xen-4.2-unstable  x86_64  debug=y  Not tainted ]----
>         Feb 13 17:19:32.626512 (XEN) CPU:    0
>         Feb 13 17:19:32.626525 (XEN) RIP:    e008:[<ffff82c48012c842>] xfree+0x33/0x121
>         Feb 13 17:19:32.641496 (XEN) RFLAGS: 0000000000010002   CONTEXT: hypervisor
>         Feb 13 17:19:32.641519 (XEN) rax: ffff82c4802d0800   rbx: ffff8301a7e00080   rcx: 0000000000000000
>         Feb 13 17:19:32.650560 (XEN) rdx: 0000000000000000   rsi: 0000000000000083   rdi: 0000000000000000
>         Feb 13 17:19:32.665510 (XEN) rbp: ffff82c4802afd18   rsp: ffff82c4802afcf8   r8:  0000000000000004
>         Feb 13 17:19:32.665550 (XEN) r9:  0000000000000000   r10: 0000000000000006   r11: ffff82c480224aa0
>         Feb 13 17:19:32.673509 (XEN) r12: ffff8301a7e00580   r13: 0000000000000005   r14: ffff82c4802aff18
>         Feb 13 17:19:32.685503 (XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4: 00000000000006f0
>         Feb 13 17:19:32.685537 (XEN) cr3: 00000001a7f54000   cr2: 00000000c4b4ee84
>         Feb 13 17:19:32.697505 (XEN) ds: 007b   es: 007b   fs: 00d8   gs: 0000   ss: 0000   cs: e008
>         Feb 13 17:19:32.697540 (XEN) Xen stack trace from rsp=ffff82c4802afcf8:
>         Feb 13 17:19:32.706513 (XEN)    ffff8301a7e00080 ffff8301a7e00580 0000000000000005 ffff82c4802aff18
>         Feb 13 17:19:32.721495 (XEN)    ffff82c4802afd88 ffff82c4801658ee ffff82c4802afd38 ffff82c48010098a
>         Feb 13 17:19:32.721531 (XEN)    00000400802afd68 0000000000000083 ffff8301a7e000a8 0000000000000000
>         Feb 13 17:19:32.729495 (XEN)    00000000fffffffa 00000000000000e5 ffff8301a7e00580 0000000000000005
>         Feb 13 17:19:32.738490 (XEN)    ffff82c4802aff18 ffff8301a7e005a8 ffff82c4802afe28 ffff82c480167781
>         Feb 13 17:19:32.738515 (XEN)    ffff8301a7ece000 ffff82c4802afde8 0000000000000000 ffff82c4802aff18
>         Feb 13 17:19:32.750497 (XEN)    ffff82c4802aff18 0000000000000002 ffff82c4802aff18 ffff82c4802fa060
>         Feb 13 17:19:32.762568 (XEN)    000000e500000000 ffff82c4802fa060 ffff82c4802afe08 ffff82c48017bd51
>         Feb 13 17:19:32.762596 (XEN)    ffff82c4802aff18 ffff82c4802aff18 ffff82c48025e380 ffff82c4802aff18
>         Feb 13 17:19:32.773513 (XEN)    00000000ffffffff 0000000000000002 00007d3b7fd501a7 ffff82c4801525d0
>         Feb 13 17:19:32.785503 (XEN)    0000000000000002 00000000ffffffff ffff82c4802aff18 ffff82c48025e380
>         Feb 13 17:19:32.785539 (XEN)    ffff82c4802afee0 ffff82c4802aff18 0000001863058413 00000000000c0000
>         Feb 13 17:19:32.794514 (XEN)    000000000e1ff99c 000000000000c701 ffff82c4802f9a90 0000000000000000
>         Feb 13 17:19:32.809503 (XEN)    0000000000000000 ffff8301a7f5dc80 0000000000000000 0000002000000000
>         Feb 13 17:19:32.809529 (XEN)    ffff82c4801581a9 000000000000e008 0000000000000246 ffff82c4802afee0
>         Feb 13 17:19:32.814513 (XEN)    0000000000000000 ffff82c4802aff10 ffff82c48015a647 0000000000000000
>         Feb 13 17:19:32.829506 (XEN)    ffff8300d7cfb000 ffff8300d7af9000 0000000000000000 ffff82c4802afd88
>         Feb 13 17:19:32.829549 (XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
>         Feb 13 17:19:32.841510 (XEN)    00000000dfc91f90 00000000deadbeef 0000000000000000 0000000000000000
>         Feb 13 17:19:32.853508 (XEN)    0000000000000000 0000000000000000 0000000000000000 00000000deadbeef
>         Feb 13 17:19:32.858496 (XEN) Xen call trace:
>         Feb 13 17:19:32.858518 (XEN)    [<ffff82c48012c842>] xfree+0x33/0x121
>         Feb 13 17:19:32.858547 (XEN)    [<ffff82c4801658ee>] dump_irqs+0x2a3/0x2ca
>         Feb 13 17:19:32.870500 (XEN)    [<ffff82c480167781>] smp_irq_move_cleanup_interrupt+0x303/0x37b
>         Feb 13 17:19:32.870554 (XEN)    [<ffff82c4801525d0>] irq_move_cleanup_interrupt+0x30/0x40
>         Feb 13 17:19:32.885510 (XEN)    [<ffff82c4801581a9>] default_idle+0x99/0x9e
>         Feb 13 17:19:32.885541 (XEN)    [<ffff82c48015a647>] idle_loop+0x6c/0x7c
>         Feb 13 17:19:32.897496 (XEN)    
>         Feb 13 17:19:32.897510 (XEN) 
>         Feb 13 17:19:32.897520 (XEN) ****************************************
>         Feb 13 17:19:32.897537 (XEN) Panic on CPU 0:
>         Feb 13 17:19:32.905499 (XEN) Assertion '!in_irq()' failed at xmalloc_tlsf.c:607
>         Feb 13 17:19:32.905522 (XEN) ****************************************
>         Feb 13 17:19:32.913488 (XEN) 
>         Feb 13 17:19:32.913506 (XEN) Reboot in five seconds...
> 
> # HG changeset patch
> # User Ian Campbell <ian.campbell@citrix.com>
> # Date 1329216241 0
> # Node ID 738424a5e5a5053c75cfbe64f6675b5d756daf1b
> # Parent  0ba87b95e80bae059fe70b4b117dcc409f2471ef
> xen: don't try to print IRQ SSID in IRQ debug from irq context.
> 
> It is not possible to call xfree() in that context.
> 
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> 
> diff -r 0ba87b95e80b -r 738424a5e5a5 xen/arch/x86/irq.c
> --- a/xen/arch/x86/irq.c	Mon Feb 13 17:26:08 2012 +0000
> +++ b/xen/arch/x86/irq.c	Tue Feb 14 10:44:01 2012 +0000
> @@ -2026,7 +2026,7 @@ static void dump_irqs(unsigned char key)
>          if ( !irq_desc_initialized(desc) || desc->handler == &no_irq_type )
>              continue;
>  
> -        ssid = xsm_show_irq_sid(irq);
> +        ssid = in_irq() ? NULL : xsm_show_irq_sid(irq);
>  
>          spin_lock_irqsave(&desc->lock, flags);
>  
> @@ -2073,7 +2073,8 @@ static void dump_irqs(unsigned char key)
>  
>          spin_unlock_irqrestore(&desc->lock, flags);
>  
> -        xfree(ssid);
> +        if ( ssid )
> +                xfree(ssid);
>      }
>  
>      dump_ioapic_irq_info();
> 
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-03-27 10:36   ` Ian Campbell
@ 2012-03-27 10:52     ` Jan Beulich
  2012-05-04 19:48     ` AP
  1 sibling, 0 replies; 24+ messages in thread
From: Jan Beulich @ 2012-03-27 10:52 UTC (permalink / raw)
  To: Ian Campbell, Ian Jackson
  Cc: Andrew Cooper, Daniel De Graaf, xen-devel, Keir (Xen.org)

>>> On 27.03.12 at 12:36, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> # HG changeset patch
>> # User Ian Campbell <ian.campbell@citrix.com>
>> # Date 1329216241 0
>> # Node ID 738424a5e5a5053c75cfbe64f6675b5d756daf1b
>> # Parent  0ba87b95e80bae059fe70b4b117dcc409f2471ef
>> xen: don't try to print IRQ SSID in IRQ debug from irq context.
>> 
>> It is not possible to call xfree() in that context.
>> 
>> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
>> 
>> diff -r 0ba87b95e80b -r 738424a5e5a5 xen/arch/x86/irq.c
>> --- a/xen/arch/x86/irq.c	Mon Feb 13 17:26:08 2012 +0000
>> +++ b/xen/arch/x86/irq.c	Tue Feb 14 10:44:01 2012 +0000
>> @@ -2026,7 +2026,7 @@ static void dump_irqs(unsigned char key)
>>          if ( !irq_desc_initialized(desc) || desc->handler == &no_irq_type )
>>              continue;
>>  
>> -        ssid = xsm_show_irq_sid(irq);
>> +        ssid = in_irq() ? NULL : xsm_show_irq_sid(irq);
>>  
>>          spin_lock_irqsave(&desc->lock, flags);
>>  
>> @@ -2073,7 +2073,8 @@ static void dump_irqs(unsigned char key)
>>  
>>          spin_unlock_irqrestore(&desc->lock, flags);
>>  
>> -        xfree(ssid);
>> +        if ( ssid )
>> +                xfree(ssid);

But perhaps xfree(NULL) should be made usable in any context (i.e.
the assertion in there moved down)? Otherwise the construct above
is likely to get collapsed again at some point with "xfree(NULL) is
perfectly valid" in mind.

Jan

>>      }
>>  
>>      dump_ioapic_irq_info();

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-03-27 10:36   ` Ian Campbell
  2012-03-27 10:52     ` Jan Beulich
@ 2012-05-04 19:48     ` AP
  2012-05-04 20:11       ` Andrew Cooper
  1 sibling, 1 reply; 24+ messages in thread
From: AP @ 2012-05-04 19:48 UTC (permalink / raw)
  To: Ian Campbell
  Cc: Keir (Xen.org), Andrew Cooper, Daniel De Graaf, xen-devel, Ian Jackson

On Tue, Mar 27, 2012 at 3:36 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Tue, 2012-02-14 at 10:44 +0000, Ian Campbell wrote:
>> On Mon, 2012-02-13 at 20:16 +0000, xen.org wrote:
>> > flight 11946 xen-unstable real [real]
>> > http://www.chiark.greenend.org.uk/~xensrcts/logs/11946/
>> >
>> > Regressions :-(
>> >
>> > Tests which did not succeed and are blocking,
>> > including tests which could not be run:
>> >  test-amd64-i386-xl-credit2    7 debian-install            fail REGR. vs. 11944
>>
>> Host crash:
>> http://www.chiark.greenend.org.uk/~xensrcts/logs/11946/test-amd64-i386-xl-credit2/serial-woodlouse.log
>>
>> This is the debug Andrew Cooper added recently to track down the IRQ
>> assertion we've been seeing, sadly it looks like the debug code tries to
>> call xfree from interrupt context and therefore doesn't produce full
>> output :-(
>
> Are we still seeing the issue this debugging was intended to address? We
> don't seem to be seeing the host crashes any more. Should the debug code
> be patched up as in the following patch, otherwise when we do see it it
> doesn't end up printing any useful info.
>
> Someone recently reported bugs.debian.org/665433 to Debian, is this the
> same underlying issue? That report is with Xen 4.0 FWIW.

I saw the issue (xen-unstable 25256:9dda0efd8ce1) that the debugging
code added. Can the fix to the debugging code be checked in until the
original issue has been fixed?

Thanks,
AP

(XEN) *** IRQ BUG found ***
(XEN) CPU0 -Testing vector 236 from bitmap
41,47,49,57,64,72,80,88,96,100,104,120,136,152,160-161,168,171,192,200-201,208
(XEN) Guest interrupt information:
(XEN)    IRQ:   0 affinity:01 vec:f0 type=IO-APIC-edge
status=00000000 mapped, unbound
(XEN) Assertion '!in_irq()' failed at xmalloc_tlsf.c:607
(XEN) ----[ Xen-4.2-unstable  x86_64  debug=y  Tainted:    C ]----
(XEN) CPU:    0
(XEN) RIP:    e008:[<ffff82c48012cefb>] xfree+0x33/0x118
(XEN) RFLAGS: 0000000000010002   CONTEXT: hypervisor
(XEN) rax: 0000000000000000   rbx: ffff830214ac0080   rcx: 0000000000000000
(XEN) rdx: ffff82c4802d8880   rsi: 0000000000000083   rdi: 0000000000000000
(XEN) rbp: ffff82c4802b7c78   rsp: ffff82c4802b7c58   r8:  0000000000000004
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000010
(XEN) r12: ffff830214ac0c80   r13: 000000000000000c   r14: ffff830214ac0ca8
(XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4: 00000000000426f0
(XEN) cr3: 0000000168971000   cr2: 0000000001095e00
(XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen stack trace from rsp=ffff82c4802b7c58:
(XEN)    ffff830214ac0080 ffff830214ac0c80 000000000000000c ffff830214ac0ca8
(XEN)    ffff82c4802b7ce8 ffff82c4801664d4 ffff82c4802e214a ffff82c400000020
(XEN)    ffff82c4802b7cf8 0000000000000083 ffff830214ac00a8 0000000000000000
(XEN)    00000000000000ec 00000000000000ec ffff830214ac0c80 000000000000000c
(XEN)    ffff830214ac0ca8 ffff82c480302760 ffff82c4802b7d58 ffff82c480168000
(XEN)    ffff82c4802b7f18 ffff82c4802b7f18 000000ec00000000 ffff82c4802b7f18
(XEN)    0000000000000000 0000000000000000 ffff82c480302324 0000000000000020
(XEN)    ffff82c4802b7dd8 0000000000000003 0000000000000000 0000000000000000
(XEN)    ffff82c4802b7dc8 ffff82c4801683d3 ffff8300da991000 ffff8300da996000
(XEN)    0000000000000000 ffffffff802b7d90 ffff82c480159160 ffff82c4802b7e20
(XEN)    ffff82c48015d7db ffff82c4802b7f18 ffff8300da991000 0000000000000003
(XEN)    0000000000000000 0000000000000000 00007d3b7fd48207 ffff82c480160426
(XEN)    0000000000000000 0000000000000000 0000000000000003 ffff8300da991000
(XEN)    ffff82c4802b7ef8 ffff82c4802b7f18 0000000000000282 ffff82c4802319a0
(XEN)    00000000deadbeef 0000000000000000 ffff83021c0b8081 0000000000000000
(XEN)    0000000000000048 ffff8801d7227ec0 ffff8300da991000 0000002000000000
(XEN)    ffff82c4801865c1 000000000000e008 0000000000000202 ffff82c4802b7e88
(XEN)    000000000000e010 0000000000000003 ffff82c4802b7ef8 ffff82c4802230d8
(XEN)    ffff82c4802b7f18 0000000000000000 0000000000000246 ffffffff810013aa
(XEN)    0000000000000000 ffffffff810013aa 000000000000e030 0000000000000246
(XEN) Xen call trace:
(XEN)    [<ffff82c48012cefb>] xfree+0x33/0x118
(XEN)    [<ffff82c4801664d4>] dump_irqs+0x2a4/0x2e8
(XEN)    [<ffff82c480168000>] irq_move_cleanup_interrupt+0x29f/0x2db
(XEN)    [<ffff82c4801683d3>] do_IRQ+0x9e/0x5a4
(XEN)    [<ffff82c480160426>] common_interrupt+0x26/0x30
(XEN)    [<ffff82c4801865c1>] async_exception_cleanup+0x1/0x35a
(XEN)    [<ffff82c480228438>] syscall_enter+0xc8/0x122
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Assertion '!in_irq()' failed at xmalloc_tlsf.c:607
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-05-04 19:48     ` AP
@ 2012-05-04 20:11       ` Andrew Cooper
  2012-05-05  0:21         ` AP
  2012-05-05 10:33         ` Ian Campbell
  0 siblings, 2 replies; 24+ messages in thread
From: Andrew Cooper @ 2012-05-04 20:11 UTC (permalink / raw)
  To: AP, Ian Jackson, Keir (Xen.org); +Cc: Daniel De Graaf, xen-devel, Ian Campbell

[-- Attachment #1: Type: text/plain, Size: 5167 bytes --]

On 04/05/12 20:48, AP wrote:
> On Tue, Mar 27, 2012 at 3:36 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> On Tue, 2012-02-14 at 10:44 +0000, Ian Campbell wrote:
>>> On Mon, 2012-02-13 at 20:16 +0000, xen.org wrote:
>>>> flight 11946 xen-unstable real [real]
>>>> http://www.chiark.greenend.org.uk/~xensrcts/logs/11946/
>>>>
>>>> Regressions :-(
>>>>
>>>> Tests which did not succeed and are blocking,
>>>> including tests which could not be run:
>>>>  test-amd64-i386-xl-credit2    7 debian-install            fail REGR. vs. 11944
>>> Host crash:
>>> http://www.chiark.greenend.org.uk/~xensrcts/logs/11946/test-amd64-i386-xl-credit2/serial-woodlouse.log
>>>
>>> This is the debug Andrew Cooper added recently to track down the IRQ
>>> assertion we've been seeing, sadly it looks like the debug code tries to
>>> call xfree from interrupt context and therefore doesn't produce full
>>> output :-(
>> Are we still seeing the issue this debugging was intended to address? We
>> don't seem to be seeing the host crashes any more. Should the debug code
>> be patched up as in the following patch, otherwise when we do see it it
>> doesn't end up printing any useful info.
>>
>> Someone recently reported bugs.debian.org/665433 to Debian, is this the
>> same underlying issue? That report is with Xen 4.0 FWIW.
> I saw the issue (xen-unstable 25256:9dda0efd8ce1) that the debugging
> code added. Can the fix to the debugging code be checked in until the
> original issue has been fixed?
>
> Thanks,
> AP
>
> (XEN) *** IRQ BUG found ***
> (XEN) CPU0 -Testing vector 236 from bitmap
> 41,47,49,57,64,72,80,88,96,100,104,120,136,152,160-161,168,171,192,200-201,208
> (XEN) Guest interrupt information:
> (XEN)    IRQ:   0 affinity:01 vec:f0 type=IO-APIC-edge
> status=00000000 mapped, unbound
> (XEN) Assertion '!in_irq()' failed at xmalloc_tlsf.c:607
> (XEN) ----[ Xen-4.2-unstable  x86_64  debug=y  Tainted:    C ]----
> (XEN) CPU:    0
> (XEN) RIP:    e008:[<ffff82c48012cefb>] xfree+0x33/0x118
> (XEN) RFLAGS: 0000000000010002   CONTEXT: hypervisor
> (XEN) rax: 0000000000000000   rbx: ffff830214ac0080   rcx: 0000000000000000
> (XEN) rdx: ffff82c4802d8880   rsi: 0000000000000083   rdi: 0000000000000000
> (XEN) rbp: ffff82c4802b7c78   rsp: ffff82c4802b7c58   r8:  0000000000000004
> (XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000010
> (XEN) r12: ffff830214ac0c80   r13: 000000000000000c   r14: ffff830214ac0ca8
> (XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4: 00000000000426f0
> (XEN) cr3: 0000000168971000   cr2: 0000000001095e00
> (XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs: e008
> (XEN) Xen stack trace from rsp=ffff82c4802b7c58:
> (XEN)    ffff830214ac0080 ffff830214ac0c80 000000000000000c ffff830214ac0ca8
> (XEN)    ffff82c4802b7ce8 ffff82c4801664d4 ffff82c4802e214a ffff82c400000020
> (XEN)    ffff82c4802b7cf8 0000000000000083 ffff830214ac00a8 0000000000000000
> (XEN)    00000000000000ec 00000000000000ec ffff830214ac0c80 000000000000000c
> (XEN)    ffff830214ac0ca8 ffff82c480302760 ffff82c4802b7d58 ffff82c480168000
> (XEN)    ffff82c4802b7f18 ffff82c4802b7f18 000000ec00000000 ffff82c4802b7f18
> (XEN)    0000000000000000 0000000000000000 ffff82c480302324 0000000000000020
> (XEN)    ffff82c4802b7dd8 0000000000000003 0000000000000000 0000000000000000
> (XEN)    ffff82c4802b7dc8 ffff82c4801683d3 ffff8300da991000 ffff8300da996000
> (XEN)    0000000000000000 ffffffff802b7d90 ffff82c480159160 ffff82c4802b7e20
> (XEN)    ffff82c48015d7db ffff82c4802b7f18 ffff8300da991000 0000000000000003
> (XEN)    0000000000000000 0000000000000000 00007d3b7fd48207 ffff82c480160426
> (XEN)    0000000000000000 0000000000000000 0000000000000003 ffff8300da991000
> (XEN)    ffff82c4802b7ef8 ffff82c4802b7f18 0000000000000282 ffff82c4802319a0
> (XEN)    00000000deadbeef 0000000000000000 ffff83021c0b8081 0000000000000000
> (XEN)    0000000000000048 ffff8801d7227ec0 ffff8300da991000 0000002000000000
> (XEN)    ffff82c4801865c1 000000000000e008 0000000000000202 ffff82c4802b7e88
> (XEN)    000000000000e010 0000000000000003 ffff82c4802b7ef8 ffff82c4802230d8
> (XEN)    ffff82c4802b7f18 0000000000000000 0000000000000246 ffffffff810013aa
> (XEN)    0000000000000000 ffffffff810013aa 000000000000e030 0000000000000246
> (XEN) Xen call trace:
> (XEN)    [<ffff82c48012cefb>] xfree+0x33/0x118
> (XEN)    [<ffff82c4801664d4>] dump_irqs+0x2a4/0x2e8
> (XEN)    [<ffff82c480168000>] irq_move_cleanup_interrupt+0x29f/0x2db
> (XEN)    [<ffff82c4801683d3>] do_IRQ+0x9e/0x5a4
> (XEN)    [<ffff82c480160426>] common_interrupt+0x26/0x30
> (XEN)    [<ffff82c4801865c1>] async_exception_cleanup+0x1/0x35a
> (XEN)    [<ffff82c480228438>] syscall_enter+0xc8/0x122
> (XEN)
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 0:
> (XEN) Assertion '!in_irq()' failed at xmalloc_tlsf.c:607
> (XEN) ****************************************
> (XEN)
> (XEN) Reboot in five seconds...
The attached patch should prevent this panic, allowing for all the debug
information to be printed to the console.

-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com


[-- Attachment #2: irq-fix-dump_irqs.patch --]
[-- Type: text/x-patch, Size: 1504 bytes --]

# HG changeset patch
# Parent 3b563b2c79f991f226bd383d40402d96ddf9a168
x86/irq: Prevent call to xfree in dump_irqs while in an irq context.

Because of c/s 24707:96987c324a4f, dump_irqs() can now be called in an irq
context when a bug condition is encountered.  If this is the case, ignore the
call to xsm_show_irq_ssid() and the subsequent call to xfree.

This prevents an assertion failure in xfree(), and should allow all the debug
information to be dumped, before failing with a BUG() because of the underlying
race condition we are attempting to reproduce.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

diff -r 3b563b2c79f9 xen/arch/x86/irq.c
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -2039,7 +2039,7 @@ static void dump_irqs(unsigned char key)
     struct domain *d;
     const struct pirq *info;
     unsigned long flags;
-    char *ssid;
+    char *ssid = NULL;
 
     printk("Guest interrupt information:\n");
 
@@ -2051,7 +2051,8 @@ static void dump_irqs(unsigned char key)
         if ( !irq_desc_initialized(desc) || desc->handler == &no_irq_type )
             continue;
 
-        ssid = xsm_show_irq_sid(irq);
+        if ( ! in_irq() )
+            ssid = xsm_show_irq_sid(irq);
 
         spin_lock_irqsave(&desc->lock, flags);
 
@@ -2098,7 +2099,8 @@ static void dump_irqs(unsigned char key)
 
         spin_unlock_irqrestore(&desc->lock, flags);
 
-        xfree(ssid);
+        if ( ! in_irq() )
+            xfree(ssid);
     }
 
     dump_ioapic_irq_info();

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-05-04 20:11       ` Andrew Cooper
@ 2012-05-05  0:21         ` AP
  2012-05-05 11:04           ` Andrew Cooper
  2012-05-07  8:10           ` Jan Beulich
  2012-05-05 10:33         ` Ian Campbell
  1 sibling, 2 replies; 24+ messages in thread
From: AP @ 2012-05-05  0:21 UTC (permalink / raw)
  To: Andrew Cooper
  Cc: Keir (Xen.org), Daniel De Graaf, xen-devel, Ian Jackson, Ian Campbell


[-- Attachment #1.1: Type: text/plain, Size: 7035 bytes --]

On Fri, May 4, 2012 at 8:11 PM, Andrew Cooper <andrew.cooper3@citrix.com>
wrote:
>
> On 04/05/12 20:48, AP wrote:
> > On Tue, Mar 27, 2012 at 3:36 AM, Ian Campbell <Ian.Campbell@citrix.com>
> > wrote:
> >> On Tue, 2012-02-14 at 10:44 +0000, Ian Campbell wrote:
> >>> On Mon, 2012-02-13 at 20:16 +0000, xen.org wrote:
> >>>> flight 11946 xen-unstable real [real]
> >>>> http://www.chiark.greenend.org.uk/~xensrcts/logs/11946/
> >>>>
> >>>> Regressions :-(
> >>>>
> >>>> Tests which did not succeed and are blocking,
> >>>> including tests which could not be run:
> >>>>  test-amd64-i386-xl-credit2    7 debian-install            fail REGR.
> >>>> vs. 11944
> >>> Host crash:
> >>>
> >>>
http://www.chiark.greenend.org.uk/~xensrcts/logs/11946/test-amd64-i386-xl-credit2/serial-woodlouse.log
> >>>
> >>> This is the debug Andrew Cooper added recently to track down the IRQ
> >>> assertion we've been seeing, sadly it looks like the debug code tries
> >>> to
> >>> call xfree from interrupt context and therefore doesn't produce full
> >>> output :-(
> >> Are we still seeing the issue this debugging was intended to address?
> >> We
> >> don't seem to be seeing the host crashes any more. Should the debug
> >> code
> >> be patched up as in the following patch, otherwise when we do see it it
> >> doesn't end up printing any useful info.
> >>
> >> Someone recently reported bugs.debian.org/665433 to Debian, is this the
> >> same underlying issue? That report is with Xen 4.0 FWIW.
> > I saw the issue (xen-unstable 25256:9dda0efd8ce1) that the debugging
> > code added. Can the fix to the debugging code be checked in until the
> > original issue has been fixed?
> >
> > Thanks,
> > AP
> >
> > (XEN) *** IRQ BUG found ***
> > (XEN) CPU0 -Testing vector 236 from bitmap
> >
> >
41,47,49,57,64,72,80,88,96,100,104,120,136,152,160-161,168,171,192,200-201,208
> > (XEN) Guest interrupt information:
> > (XEN)    IRQ:   0 affinity:01 vec:f0 type=IO-APIC-edge
> > status=00000000 mapped, unbound
> > (XEN) Assertion '!in_irq()' failed at xmalloc_tlsf.c:607
> > (XEN) ----[ Xen-4.2-unstable  x86_64  debug=y  Tainted:    C ]----
> > (XEN) CPU:    0
> > (XEN) RIP:    e008:[<ffff82c48012cefb>] xfree+0x33/0x118
> > (XEN) RFLAGS: 0000000000010002   CONTEXT: hypervisor
> > (XEN) rax: 0000000000000000   rbx: ffff830214ac0080   rcx:
> > 0000000000000000
> > (XEN) rdx: ffff82c4802d8880   rsi: 0000000000000083   rdi:
> > 0000000000000000
> > (XEN) rbp: ffff82c4802b7c78   rsp: ffff82c4802b7c58   r8:
> >  0000000000000004
> > (XEN) r9:  0000000000000000   r10: 0000000000000000   r11:
> > 0000000000000010
> > (XEN) r12: ffff830214ac0c80   r13: 000000000000000c   r14:
> > ffff830214ac0ca8
> > (XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4:
> > 00000000000426f0
> > (XEN) cr3: 0000000168971000   cr2: 0000000001095e00
> > (XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs: e008
> > (XEN) Xen stack trace from rsp=ffff82c4802b7c58:
> > (XEN)    ffff830214ac0080 ffff830214ac0c80 000000000000000c
> > ffff830214ac0ca8
> > (XEN)    ffff82c4802b7ce8 ffff82c4801664d4 ffff82c4802e214a
> > ffff82c400000020
> > (XEN)    ffff82c4802b7cf8 0000000000000083 ffff830214ac00a8
> > 0000000000000000
> > (XEN)    00000000000000ec 00000000000000ec ffff830214ac0c80
> > 000000000000000c
> > (XEN)    ffff830214ac0ca8 ffff82c480302760 ffff82c4802b7d58
> > ffff82c480168000
> > (XEN)    ffff82c4802b7f18 ffff82c4802b7f18 000000ec00000000
> > ffff82c4802b7f18
> > (XEN)    0000000000000000 0000000000000000 ffff82c480302324
> > 0000000000000020
> > (XEN)    ffff82c4802b7dd8 0000000000000003 0000000000000000
> > 0000000000000000
> > (XEN)    ffff82c4802b7dc8 ffff82c4801683d3 ffff8300da991000
> > ffff8300da996000
> > (XEN)    0000000000000000 ffffffff802b7d90 ffff82c480159160
> > ffff82c4802b7e20
> > (XEN)    ffff82c48015d7db ffff82c4802b7f18 ffff8300da991000
> > 0000000000000003
> > (XEN)    0000000000000000 0000000000000000 00007d3b7fd48207
> > ffff82c480160426
> > (XEN)    0000000000000000 0000000000000000 0000000000000003
> > ffff8300da991000
> > (XEN)    ffff82c4802b7ef8 ffff82c4802b7f18 0000000000000282
> > ffff82c4802319a0
> > (XEN)    00000000deadbeef 0000000000000000 ffff83021c0b8081
> > 0000000000000000
> > (XEN)    0000000000000048 ffff8801d7227ec0 ffff8300da991000
> > 0000002000000000
> > (XEN)    ffff82c4801865c1 000000000000e008 0000000000000202
> > ffff82c4802b7e88
> > (XEN)    000000000000e010 0000000000000003 ffff82c4802b7ef8
> > ffff82c4802230d8
> > (XEN)    ffff82c4802b7f18 0000000000000000 0000000000000246
> > ffffffff810013aa
> > (XEN)    0000000000000000 ffffffff810013aa 000000000000e030
> > 0000000000000246
> > (XEN) Xen call trace:
> > (XEN)    [<ffff82c48012cefb>] xfree+0x33/0x118
> > (XEN)    [<ffff82c4801664d4>] dump_irqs+0x2a4/0x2e8
> > (XEN)    [<ffff82c480168000>] irq_move_cleanup_interrupt+0x29f/0x2db
> > (XEN)    [<ffff82c4801683d3>] do_IRQ+0x9e/0x5a4
> > (XEN)    [<ffff82c480160426>] common_interrupt+0x26/0x30
> > (XEN)    [<ffff82c4801865c1>] async_exception_cleanup+0x1/0x35a
> > (XEN)    [<ffff82c480228438>] syscall_enter+0xc8/0x122
> > (XEN)
> > (XEN)
> > (XEN) ****************************************
> > (XEN) Panic on CPU 0:
> > (XEN) Assertion '!in_irq()' failed at xmalloc_tlsf.c:607
> > (XEN) ****************************************
> > (XEN)
> > (XEN) Reboot in five seconds...
> The attached patch should prevent this panic, allowing for all the debug
> information to be printed to the console.

Thanks, that fixed it. Here is what I see now:

(XEN) *** IRQ BUG found ***
(XEN) CPU0 -Testing vector 236 from bitmap
37,41,49,51,64,72,80,88,96,104,120,136,145,152,158,160,168,175,182,192,200,211
(XEN) Guest interrupt information:
(XEN)    IRQ:   0 affinity:01 vec:f0 type=IO-APIC-edge    status=00000000
mapped, unbound
(XEN)    IRQ:   1 affinity:01 vec:d3 type=IO-APIC-edge    status=00000030
in-flight=0 domain-list=0:  1(-S--),
(XEN)    IRQ:   2 affinity:ff vec:e2 type=XT-PIC          status=00000000
mapped, unbound
(XEN)    IRQ:   3 affinity:01 vec:40 type=IO-APIC-edge    status=00000002
mapped, unbound
(XEN)    IRQ:   4 affinity:01 vec:48 type=IO-APIC-edge    status=00000002
mapped, unbound
(XEN)    IRQ:   5 affinity:01 vec:50 type=IO-APIC-edge    status=00000002
mapped, unbound
(XEN)    IRQ:   6 affinity:01 vec:58 type=IO-APIC-edge    status=00000002
mapped, unbound
(XEN)    IRQ:   7 affinity:01 vec:60 type=IO-APIC-edge    status=00000002
mapped, unbound
(XEN)    IRQ:   8 affinity:08 vec:29 type=IO-APIC-edge    status=00000030
in-flight=0 domain-list=0:  8(-S--),
(XEN)    IRQ:   9 affinity:02 vec:25 type=IO-APIC-level   status=00000030
in-flight=0 domain-list=0:  9(-S--),
(XEN)    IRQ:  10 affinity:01 vec:78 type=IO-APIC-edge    status=00000002
mapped, unbound
(XEN)    IRQ:  11 affinity:01 vec:88 type=IO-APIC-edge    status=00000002
mapped, unbound
[ 5129.737147] [drm:i915_hangcheck_ring_idle] *ERROR* Hangcheck timer
elapsed... blt ring idle [waiting on 1800652, at 1800652], missed IRQ?

Let me know if you need any more info.
Thanks,
AP

[-- Attachment #1.2: Type: text/html, Size: 8909 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-05-04 20:11       ` Andrew Cooper
  2012-05-05  0:21         ` AP
@ 2012-05-05 10:33         ` Ian Campbell
  2012-05-05 11:11           ` Andrew Cooper
  1 sibling, 1 reply; 24+ messages in thread
From: Ian Campbell @ 2012-05-05 10:33 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Ian Jackson, Daniel De Graaf, Keir (Xen.org), xen-devel

On Fri, 2012-05-04 at 21:11 +0100, Andrew Cooper wrote:
> On 04/05/12 20:48, AP wrote:
> > On Tue, Mar 27, 2012 at 3:36 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> >> On Tue, 2012-02-14 at 10:44 +0000, Ian Campbell wrote:
> >>> On Mon, 2012-02-13 at 20:16 +0000, xen.org wrote:
> >>>> flight 11946 xen-unstable real [real]
> >>>> http://www.chiark.greenend.org.uk/~xensrcts/logs/11946/
> >>>>
> >>>> Regressions :-(
> >>>>
> >>>> Tests which did not succeed and are blocking,
> >>>> including tests which could not be run:
> >>>>  test-amd64-i386-xl-credit2    7 debian-install            fail REGR. vs. 11944
> >>> Host crash:
> >>> http://www.chiark.greenend.org.uk/~xensrcts/logs/11946/test-amd64-i386-xl-credit2/serial-woodlouse.log
> >>>
> >>> This is the debug Andrew Cooper added recently to track down the IRQ
> >>> assertion we've been seeing, sadly it looks like the debug code tries to
> >>> call xfree from interrupt context and therefore doesn't produce full
> >>> output :-(
> >> Are we still seeing the issue this debugging was intended to address? We
> >> don't seem to be seeing the host crashes any more. Should the debug code
> >> be patched up as in the following patch, otherwise when we do see it it
> >> doesn't end up printing any useful info.
> >>
> >> Someone recently reported bugs.debian.org/665433 to Debian, is this the
> >> same underlying issue? That report is with Xen 4.0 FWIW.
> > I saw the issue (xen-unstable 25256:9dda0efd8ce1) that the debugging
> > code added. Can the fix to the debugging code be checked in until the
> > original issue has been fixed?
> >
> > Thanks,
> > AP
> >
> > (XEN) *** IRQ BUG found ***
> > (XEN) CPU0 -Testing vector 236 from bitmap
> > 41,47,49,57,64,72,80,88,96,100,104,120,136,152,160-161,168,171,192,200-201,208
> > (XEN) Guest interrupt information:
> > (XEN)    IRQ:   0 affinity:01 vec:f0 type=IO-APIC-edge
> > status=00000000 mapped, unbound
> > (XEN) Assertion '!in_irq()' failed at xmalloc_tlsf.c:607
> > (XEN) ----[ Xen-4.2-unstable  x86_64  debug=y  Tainted:    C ]----
> > (XEN) CPU:    0
> > (XEN) RIP:    e008:[<ffff82c48012cefb>] xfree+0x33/0x118
> > (XEN) RFLAGS: 0000000000010002   CONTEXT: hypervisor
> > (XEN) rax: 0000000000000000   rbx: ffff830214ac0080   rcx: 0000000000000000
> > (XEN) rdx: ffff82c4802d8880   rsi: 0000000000000083   rdi: 0000000000000000
> > (XEN) rbp: ffff82c4802b7c78   rsp: ffff82c4802b7c58   r8:  0000000000000004
> > (XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000010
> > (XEN) r12: ffff830214ac0c80   r13: 000000000000000c   r14: ffff830214ac0ca8
> > (XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4: 00000000000426f0
> > (XEN) cr3: 0000000168971000   cr2: 0000000001095e00
> > (XEN) ds: 002b   es: 002b   fs: 0000   gs: 0000   ss: e010   cs: e008
> > (XEN) Xen stack trace from rsp=ffff82c4802b7c58:
> > (XEN)    ffff830214ac0080 ffff830214ac0c80 000000000000000c ffff830214ac0ca8
> > (XEN)    ffff82c4802b7ce8 ffff82c4801664d4 ffff82c4802e214a ffff82c400000020
> > (XEN)    ffff82c4802b7cf8 0000000000000083 ffff830214ac00a8 0000000000000000
> > (XEN)    00000000000000ec 00000000000000ec ffff830214ac0c80 000000000000000c
> > (XEN)    ffff830214ac0ca8 ffff82c480302760 ffff82c4802b7d58 ffff82c480168000
> > (XEN)    ffff82c4802b7f18 ffff82c4802b7f18 000000ec00000000 ffff82c4802b7f18
> > (XEN)    0000000000000000 0000000000000000 ffff82c480302324 0000000000000020
> > (XEN)    ffff82c4802b7dd8 0000000000000003 0000000000000000 0000000000000000
> > (XEN)    ffff82c4802b7dc8 ffff82c4801683d3 ffff8300da991000 ffff8300da996000
> > (XEN)    0000000000000000 ffffffff802b7d90 ffff82c480159160 ffff82c4802b7e20
> > (XEN)    ffff82c48015d7db ffff82c4802b7f18 ffff8300da991000 0000000000000003
> > (XEN)    0000000000000000 0000000000000000 00007d3b7fd48207 ffff82c480160426
> > (XEN)    0000000000000000 0000000000000000 0000000000000003 ffff8300da991000
> > (XEN)    ffff82c4802b7ef8 ffff82c4802b7f18 0000000000000282 ffff82c4802319a0
> > (XEN)    00000000deadbeef 0000000000000000 ffff83021c0b8081 0000000000000000
> > (XEN)    0000000000000048 ffff8801d7227ec0 ffff8300da991000 0000002000000000
> > (XEN)    ffff82c4801865c1 000000000000e008 0000000000000202 ffff82c4802b7e88
> > (XEN)    000000000000e010 0000000000000003 ffff82c4802b7ef8 ffff82c4802230d8
> > (XEN)    ffff82c4802b7f18 0000000000000000 0000000000000246 ffffffff810013aa
> > (XEN)    0000000000000000 ffffffff810013aa 000000000000e030 0000000000000246
> > (XEN) Xen call trace:
> > (XEN)    [<ffff82c48012cefb>] xfree+0x33/0x118
> > (XEN)    [<ffff82c4801664d4>] dump_irqs+0x2a4/0x2e8
> > (XEN)    [<ffff82c480168000>] irq_move_cleanup_interrupt+0x29f/0x2db
> > (XEN)    [<ffff82c4801683d3>] do_IRQ+0x9e/0x5a4
> > (XEN)    [<ffff82c480160426>] common_interrupt+0x26/0x30
> > (XEN)    [<ffff82c4801865c1>] async_exception_cleanup+0x1/0x35a
> > (XEN)    [<ffff82c480228438>] syscall_enter+0xc8/0x122
> > (XEN)
> > (XEN)
> > (XEN) ****************************************
> > (XEN) Panic on CPU 0:
> > (XEN) Assertion '!in_irq()' failed at xmalloc_tlsf.c:607
> > (XEN) ****************************************
> > (XEN)
> > (XEN) Reboot in five seconds...
> The attached patch should prevent this panic

This is effectively the same as my patch from
<1332844592.25560.9.camel@zakaz.uk.xensource.com>. I think "if (ssid)
xfree(...)" is preferable to "if (in_irq()) xfree(...)" but not enough
to prevent me:

Acked-by: Ian Campbell <ian.campbell@citrix.com>

If the debug code is going to stay for 4.2 then IMHO we should also take
this patch to make it actually useful. Otherwise we should just revert
the original debug patch before the release.

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-05-05  0:21         ` AP
@ 2012-05-05 11:04           ` Andrew Cooper
  2012-05-05 18:41             ` AP
  2012-05-07  8:10           ` Jan Beulich
  1 sibling, 1 reply; 24+ messages in thread
From: Andrew Cooper @ 2012-05-05 11:04 UTC (permalink / raw)
  To: AP; +Cc: Keir (Xen.org), Daniel De Graaf, xen-devel, Ian Jackson, Ian Campbell


> Thanks, that fixed it. Here is what I see now:
>
> (XEN) *** IRQ BUG found ***
> (XEN) CPU0 -Testing vector 236 from bitmap
> 37,41,49,51,64,72,80,88,96,104,120,136,145,152,158,160,168,175,182,192,200,211
> (XEN) Guest interrupt information:
> (XEN)    IRQ:   0 affinity:01 vec:f0 type=IO-APIC-edge   
> status=00000000 mapped, unbound
> (XEN)    IRQ:   1 affinity:01 vec:d3 type=IO-APIC-edge   
> status=00000030 in-flight=0 domain-list=0:  1(-S--),
> (XEN)    IRQ:   2 affinity:ff vec:e2 type=XT-PIC         
> status=00000000 mapped, unbound
> (XEN)    IRQ:   3 affinity:01 vec:40 type=IO-APIC-edge   
> status=00000002 mapped, unbound
> (XEN)    IRQ:   4 affinity:01 vec:48 type=IO-APIC-edge   
> status=00000002 mapped, unbound
> (XEN)    IRQ:   5 affinity:01 vec:50 type=IO-APIC-edge   
> status=00000002 mapped, unbound
> (XEN)    IRQ:   6 affinity:01 vec:58 type=IO-APIC-edge   
> status=00000002 mapped, unbound
> (XEN)    IRQ:   7 affinity:01 vec:60 type=IO-APIC-edge   
> status=00000002 mapped, unbound
> (XEN)    IRQ:   8 affinity:08 vec:29 type=IO-APIC-edge   
> status=00000030 in-flight=0 domain-list=0:  8(-S--),
> (XEN)    IRQ:   9 affinity:02 vec:25 type=IO-APIC-level  
> status=00000030 in-flight=0 domain-list=0:  9(-S--),
> (XEN)    IRQ:  10 affinity:01 vec:78 type=IO-APIC-edge   
> status=00000002 mapped, unbound
> (XEN)    IRQ:  11 affinity:01 vec:88 type=IO-APIC-edge   
> status=00000002 mapped, unbound
> [ 5129.737147] [drm:i915_hangcheck_ring_idle] *ERROR* Hangcheck timer
> elapsed... blt ring idle [waiting on 1800652, at 1800652], missed IRQ?
>
> Let me know if you need any more info.
> Thanks,
> AP
>

There should be quite a lot more irq information dumped than just that. 
Was there any more on the console or had it given up by that point? It
might be worth trying to set synchronous console to get all of that
debug information?

How easy is this error to reproduce for you? I never managed to
reproduce it reliably enough to be able to debug?

If you could provide your Xen boot console log, that would be very useful

~Andrew

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-05-05 10:33         ` Ian Campbell
@ 2012-05-05 11:11           ` Andrew Cooper
  0 siblings, 0 replies; 24+ messages in thread
From: Andrew Cooper @ 2012-05-05 11:11 UTC (permalink / raw)
  To: Ian Campbell; +Cc: Ian Jackson, Daniel De Graaf, Keir (Xen.org), xen-devel

>> The attached patch should prevent this panic
> This is effectively the same as my patch from
> <1332844592.25560.9.camel@zakaz.uk.xensource.com>. I think "if (ssid)
> xfree(...)" is preferable to "if (in_irq()) xfree(...)" but not enough
> to prevent me:
>
> Acked-by: Ian Campbell <ian.campbell@citrix.com>
>
> If the debug code is going to stay for 4.2 then IMHO we should also take
> this patch to make it actually useful. Otherwise we should just revert
> the original debug patch before the release.
>
>

Yes - I was thinking the same.  I suggest that when xen-4.2-testing.hg
gets branched off unstable, this debugging gets put back to just being
an assert as before.  However, I am quite unsure as to what would happen
with interrupts following that failed assert.

I shall re-do the patch.  I think it is a fairly sensible patch to have
in even after the main debugging has been removed, especially if similar
debugging needs to be done in the future.

~Andrew

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-05-05 11:04           ` Andrew Cooper
@ 2012-05-05 18:41             ` AP
  2012-05-05 19:06               ` AP
  0 siblings, 1 reply; 24+ messages in thread
From: AP @ 2012-05-05 18:41 UTC (permalink / raw)
  To: Andrew Cooper
  Cc: Keir (Xen.org), Daniel De Graaf, xen-devel, Ian Jackson, Ian Campbell


[-- Attachment #1.1: Type: text/plain, Size: 2424 bytes --]

On Sat, May 5, 2012 at 4:04 AM, Andrew Cooper <andrew.cooper3@citrix.com>
wrote:
>
>
> > Thanks, that fixed it. Here is what I see now:
> >
> > (XEN) *** IRQ BUG found ***
> > (XEN) CPU0 -Testing vector 236 from bitmap
> >
37,41,49,51,64,72,80,88,96,104,120,136,145,152,158,160,168,175,182,192,200,211
> > (XEN) Guest interrupt information:
> > (XEN)    IRQ:   0 affinity:01 vec:f0 type=IO-APIC-edge
> > status=00000000 mapped, unbound
> > (XEN)    IRQ:   1 affinity:01 vec:d3 type=IO-APIC-edge
> > status=00000030 in-flight=0 domain-list=0:  1(-S--),
> > (XEN)    IRQ:   2 affinity:ff vec:e2 type=XT-PIC
> > status=00000000 mapped, unbound
> > (XEN)    IRQ:   3 affinity:01 vec:40 type=IO-APIC-edge
> > status=00000002 mapped, unbound
> > (XEN)    IRQ:   4 affinity:01 vec:48 type=IO-APIC-edge
> > status=00000002 mapped, unbound
> > (XEN)    IRQ:   5 affinity:01 vec:50 type=IO-APIC-edge
> > status=00000002 mapped, unbound
> > (XEN)    IRQ:   6 affinity:01 vec:58 type=IO-APIC-edge
> > status=00000002 mapped, unbound
> > (XEN)    IRQ:   7 affinity:01 vec:60 type=IO-APIC-edge
> > status=00000002 mapped, unbound
> > (XEN)    IRQ:   8 affinity:08 vec:29 type=IO-APIC-edge
> > status=00000030 in-flight=0 domain-list=0:  8(-S--),
> > (XEN)    IRQ:   9 affinity:02 vec:25 type=IO-APIC-level
> > status=00000030 in-flight=0 domain-list=0:  9(-S--),
> > (XEN)    IRQ:  10 affinity:01 vec:78 type=IO-APIC-edge
> > status=00000002 mapped, unbound
> > (XEN)    IRQ:  11 affinity:01 vec:88 type=IO-APIC-edge
> > status=00000002 mapped, unbound
> > [ 5129.737147] [drm:i915_hangcheck_ring_idle] *ERROR* Hangcheck timer
> > elapsed... blt ring idle [waiting on 1800652, at 1800652], missed IRQ?
> >
> > Let me know if you need any more info.
> > Thanks,
> > AP
> >
>
> There should be quite a lot more irq information dumped than just that.
> Was there any more on the console or had it given up by that point? It

There was nothing more on the console. The system was hung.

> might be worth trying to set synchronous console to get all of that
> debug information?

I was running with sync_console and console_to_ring options.

> How easy is this error to reproduce for you? I never managed to
> reproduce it reliably enough to be able to debug?

I cannot reproduce it easily either.

> If you could provide your Xen boot console log, that would be very useful

I will send full logs the next time I see the problem.

Thanks,
AP

[-- Attachment #1.2: Type: text/html, Size: 2978 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-05-05 18:41             ` AP
@ 2012-05-05 19:06               ` AP
  0 siblings, 0 replies; 24+ messages in thread
From: AP @ 2012-05-05 19:06 UTC (permalink / raw)
  To: Andrew Cooper
  Cc: Keir (Xen.org), Daniel De Graaf, xen-devel, Ian Jackson, Ian Campbell


[-- Attachment #1.1: Type: text/plain, Size: 2690 bytes --]

On Sat, May 5, 2012 at 11:41 AM, AP <apxeng@gmail.com> wrote:
>
> On Sat, May 5, 2012 at 4:04 AM, Andrew Cooper <andrew.cooper3@citrix.com>
wrote:
> >
> >
> > > Thanks, that fixed it. Here is what I see now:
> > >
> > > (XEN) *** IRQ BUG found ***
> > > (XEN) CPU0 -Testing vector 236 from bitmap
> > >
37,41,49,51,64,72,80,88,96,104,120,136,145,152,158,160,168,175,182,192,200,211
> > > (XEN) Guest interrupt information:
> > > (XEN)    IRQ:   0 affinity:01 vec:f0 type=IO-APIC-edge
> > > status=00000000 mapped, unbound
> > > (XEN)    IRQ:   1 affinity:01 vec:d3 type=IO-APIC-edge
> > > status=00000030 in-flight=0 domain-list=0:  1(-S--),
> > > (XEN)    IRQ:   2 affinity:ff vec:e2 type=XT-PIC
> > > status=00000000 mapped, unbound
> > > (XEN)    IRQ:   3 affinity:01 vec:40 type=IO-APIC-edge
> > > status=00000002 mapped, unbound
> > > (XEN)    IRQ:   4 affinity:01 vec:48 type=IO-APIC-edge
> > > status=00000002 mapped, unbound
> > > (XEN)    IRQ:   5 affinity:01 vec:50 type=IO-APIC-edge
> > > status=00000002 mapped, unbound
> > > (XEN)    IRQ:   6 affinity:01 vec:58 type=IO-APIC-edge
> > > status=00000002 mapped, unbound
> > > (XEN)    IRQ:   7 affinity:01 vec:60 type=IO-APIC-edge
> > > status=00000002 mapped, unbound
> > > (XEN)    IRQ:   8 affinity:08 vec:29 type=IO-APIC-edge
> > > status=00000030 in-flight=0 domain-list=0:  8(-S--),
> > > (XEN)    IRQ:   9 affinity:02 vec:25 type=IO-APIC-level
> > > status=00000030 in-flight=0 domain-list=0:  9(-S--),
> > > (XEN)    IRQ:  10 affinity:01 vec:78 type=IO-APIC-edge
> > > status=00000002 mapped, unbound
> > > (XEN)    IRQ:  11 affinity:01 vec:88 type=IO-APIC-edge
> > > status=00000002 mapped, unbound
> > > [ 5129.737147] [drm:i915_hangcheck_ring_idle] *ERROR* Hangcheck timer
> > > elapsed... blt ring idle [waiting on 1800652, at 1800652], missed IRQ?
> > >
> > > Let me know if you need any more info.
> > > Thanks,
> > > AP
> > >
> >
> > There should be quite a lot more irq information dumped than just that.
> > Was there any more on the console or had it given up by that point? It
>
> There was nothing more on the console. The system was hung.
>
> > might be worth trying to set synchronous console to get all of that
> > debug information?
>
> I was running with sync_console and console_to_ring options.
>
>
> > How easy is this error to reproduce for you? I never managed to
> > reproduce it reliably enough to be able to debug?
>
> I cannot reproduce it easily either.
>
>
> > If you could provide your Xen boot console log, that would be very
useful
>
> I will send full logs the next time I see the problem.

I have attached the full logs. I had a CentOS 5.6 and a Windows 7 HVM
domain running.

Thanks,
AP

[-- Attachment #1.2: Type: text/html, Size: 3462 bytes --]

[-- Attachment #2: irq_bug.log --]
[-- Type: application/octet-stream, Size: 72073 bytes --]

 __  __            _  _    ____                     _        _     _      
 \ \/ /___ _ __   | || |  |___ \    _   _ _ __  ___| |_ __ _| |__ | | ___ 
  \  // _ \ '_ \  | || |_   __) |__| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
  /  \  __/ | | | |__   _| / __/|__| |_| | | |N) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) Processor #0 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
(XEN) Processor #1 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] enabled)
(XEN) Processor #2 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] enabled)
(XEN) Processor #3 6:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x00] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x00] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x00] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x00] disabled)
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a301 base: 0xfed00000
(XEN) Table is not found!
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 8 CPUs (4 hotplug CPUs)
(XEN) IRQ limits: 24 GSI, 760 MSI/MSI-X
(XEN) Switched to APIC driver x2apic_cluster.
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2691.361 MHz processor.
(XEN) Initing memory sharing.
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) mce_intel.c:1239: MCA Capability: BCAST 1 SER 0 CMCI 1 firstbank 0 extended MCE MSR 0
(XEN) Intel machine check reporting enabled
(XEN) PCI: MCFG configuration 0: base f8000000 segment 0000 buses 00 - 3f
(XEN) PCI: MCFG area at f8000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-3f
(XEN) Intel VT-d Snoop Control not enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Shared EPT tables not enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) TSC deadline timer enabled
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 32 KiB.
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB [disabled]
(XEN) Brought up 4 CPUs
(XEN) ACPI sleep modes: S3
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) *** LOADING DOMAIN 0 ***
(XEN) elf_parse_binary: phdr: paddr=0x1000000 memsz=0xaa5000
(XEN) elf_parse_binary: phdr: paddr=0x1c00000 memsz=0xbac80
(XEN) elf_parse_binary: phdr: paddr=0x1cbb000 memsz=0xd60
(XEN) elf_parse_binary: phdr: paddr=0x1cbc000 memsz=0x13700
(XEN) elf_parse_binary: phdr: paddr=0x1cd0000 memsz=0x373000
(XEN) elf_parse_binary: memory: 0x1000000 -> 0x2043000
(XEN) elf_xen_parse_note: GUEST_OS = "linux"
(XEN) elf_xen_parse_note: GUEST_VERSION = "2.6"
(XEN) elf_xen_parse_note: XEN_VERSION = "xen-3.0"
(XEN) elf_xen_parse_note: VIRT_BASE = 0xffffffff80000000
(XEN) elf_xen_parse_note: ENTRY = 0xffffffff81cd0200
(XEN) elf_xen_parse_note: HYPERCALL_PAGE = 0xffffffff81001000
(XEN) elf_xen_parse_note: FEATURES = "!writable_page_tables|pae_pgdir_above_4gb"
(XEN) elf_xen_parse_note: PAE_MODE = "yes"
(XEN) elf_xen_parse_note: LOADER = "generic"
(XEN) elf_xen_parse_note: unknown xen elf note (0xd)
(XEN) elf_xen_parse_note: SUSPEND_CANCEL = 0x1
(XEN) elf_xen_parse_note: HV_START_LOW = 0xffff800000000000
(XEN) elf_xen_parse_note: PADDR_OFFSET = 0x0
(XEN) elf_xen_addr_calc_check: addresses:
(XEN)     virt_base        = 0xffffffff80000000
(XEN)     elf_paddr_offset = 0x0
(XEN)     virt_offset      = 0xffffffff80000000
(XEN)     virt_kstart      = 0xffffffff81000000
(XEN)     virt_kend        = 0xffffffff82043000
(XEN)     virt_entry       = 0xffffffff81cd0200
(XEN)     p2m_base         = 0xffffffffffffffff
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2043000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000210000000->0000000214000000 (1979465 pages to be allocated)
(XEN)  Init. ramdisk: 000000021c154000->000000021e5ffc00
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff82043000
(XEN)  Init. ramdisk: ffffffff82043000->ffffffff844eec00
(XEN)  Phys-Mach map: ffffffff844ef000->ffffffff8543b7a8
(XEN)  Start info:    ffffffff8543c000->ffffffff8543c4b4
(XEN)  Page tables:   ffffffff8543d000->ffffffff8546c000
(XEN)  Boot stack:    ffffffff8546c000->ffffffff8546d000
(XEN)  TOTAL:         ffffffff80000000->ffffffff85800000
(XEN)  ENTRY ADDRESS: ffffffff81cd0200
(XEN) Dom0 has maximum 4 VCPUs
(XEN) elf_load_binary: phdr 0 at 0xffffffff81000000 -> 0xffffffff81aa5000
(XEN) elf_load_binary: phdr 1 at 0xffffffff81c00000 -> 0xffffffff81cbac80
(XEN) elf_load_binary: phdr 2 at 0xffffffff81cbb000 -> 0xffffffff81cbbd60
(XEN) elf_load_binary: phdr 3 at 0xffffffff81cbc000 -> 0xffffffff81ccf700
(XEN) elf_load_binary: phdr 4 at 0xffffffff81cd0000 -> 0xffffffff81dba000
(XEN) Scrubbing Free RAM: .done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) **********************************************
(XEN) ******* WARNING: CONSOLE OUTPUT IS SYNCHRONOUS
(XEN) ******* This option is intended to aid debugging of Xen by ensuring
(XEN) ******* that all output is synchronously delivered on the serial line.
(XEN) ******* However it can introduce SIGNIFICANT latencies and affect
(XEN) ******* timekeeping. It is NOT recommended for production use!
(XEN) **********************************************
(XEN) 3... 2... 1... 
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 236kB init memory.
mapping kernel into physical memory
Xen: setup ISA identity maps
about to get started...
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version 3.0.0-17-generic (buildd@yellow) (gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3) ) #30-Ubuntu SMP Thu Mar 8 20:45:39 UTC 2012 (Ubuntu 3.0.0-17.30-generic 3.0.22)
[    0.000000] Command line: placeholder root=UUID=c7815afe-ae72-4cfa-b8b0-94b3035c0bb7 ro debug console=hvc0 earlyprintk=xen
[    0.000000] KERNEL supported cpus:
[    0.000000]   Intel GenuineIntel
[    0.000000]   AMD AuthenticAMD
[    0.000000]   Centaur CentaurHauls
[    0.000000] Disabled fast string operations
[    0.000000] xen_release_chunk: looking at area pfn dfa00-f8000: 99840 pages freed
[    0.000000] xen_release_chunk: looking at area pfn fc000-fec00: 11264 pages freed
[    0.000000] xen_release_chunk: looking at area pfn fec01-fed08: 263 pages freed
[    0.000000] xen_release_chunk: looking at area pfn fed09-fed10: 7 pages freed
[    0.000000] xen_release_chunk: looking at area pfn fed1a-fed1c: 2 pages freed
[    0.000000] xen_release_chunk: looking at area pfn fed20-fee00: 224 pages freed
[    0.000000] xen_release_chunk: looking at area pfn fee01-ffd20: 3871 pages freed
[    0.000000] released 115471 pages of unused memory
[    0.000000] 1-1 mapping on 9e->100
[    0.000000] 1-1 mapping on 20000->20200
[    0.000000] 1-1 mapping on 40000->40200
[    0.000000] 1-1 mapping on da99f->dafff
[    0.000000] 1-1 mapping on db000->100000
[    0.000000] 1-1 mapping on 21e600->21e800
[    0.000000] Set 154818 page(s) to 1-1 mapping.
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  Xen: 0000000000000000 - 000000000009d000 (usable)
[    0.000000]  Xen: 000000000009d800 - 0000000000100000 (reserved)
[    0.000000]  Xen: 0000000000100000 - 0000000020000000 (usable)
[    0.000000]  Xen: 0000000020000000 - 0000000020200000 (reserved)
[    0.000000]  Xen: 0000000020200000 - 0000000040000000 (usable)
[    0.000000]  Xen: 0000000040000000 - 0000000040200000 (reserved)
[    0.000000]  Xen: 0000000040200000 - 00000000da99f000 (usable)
[    0.000000]  Xen: 00000000da99f000 - 00000000dae9f000 (reserved)
[    0.000000]  Xen: 00000000dae9f000 - 00000000daf9f000 (ACPI NVS)
[    0.000000]  Xen: 00000000daf9f000 - 00000000dafff000 (ACPI data)
[    0.000000]  Xen: 00000000dafff000 - 00000000db000000 (usable)
[    0.000000]  Xen: 00000000db000000 - 00000000dfa00000 (reserved)
[    0.000000]  Xen: 00000000f8000000 - 00000000fc000000 (reserved)
[    0.000000]  Xen: 00000000fec00000 - 00000000fec01000 (reserved)
[    0.000000]  Xen: 00000000fed08000 - 00000000fed09000 (reserved)
[    0.000000]  Xen: 00000000fed10000 - 00000000fed1a000 (reserved)
[    0.000000]  Xen: 00000000fed1c000 - 00000000fed20000 (reserved)
[    0.000000]  Xen: 00000000fee00000 - 00000000fee01000 (reserved)
[    0.000000]  Xen: 00000000ffd20000 - 0000000100000000 (reserved)
[    0.000000]  Xen: 0000000100000000 - 00000001e98f5000 (usable)
[    0.000000]  Xen: 000000021e600000 - 000000021e800000 (reserved)
[    0.000000]  Xen: 000000021e800000 - 000000026f81a000 (usable)
[    0.000000] bootconsole [xenboot0] enabled
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI 2.6 present.
[    0.000000] DMI: LENOVO 4286CTO/4286CTO, BIOS 8DET58WW (1.28 ) 02/14/2012
[    0.000000] e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved)
[    0.000000] e820 remove range: 00000000000a0000 - 0000000000100000 (usable)
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x26f81a max_arch_pfn = 0x400000000
[    0.000000] x2apic enabled by BIOS, switching to x2apic ops
[    0.000000] last_pfn = 0xdb000 max_arch_pfn = 0x400000000
[    0.000000] initial memory mapped : 0 - 044ef000
[    0.000000] Base memory trampoline at [ffff880000098000] 98000 size 20480
[    0.000000] init_memory_mapping: 0000000000000000-00000000db000000
[    0.000000]  0000000000 - 00db000000 page 4k
[    0.000000] kernel direct mapping tables up to db000000 @ 923000-1000000
[    0.000000] xen: setting RW the range fce000 - 1000000
[    0.000000] init_memory_mapping: 0000000100000000-000000026f81a000
[    0.000000]  0100000000 - 026f81a000 page 4k
[    0.000000] kernel direct mapping tables up to 26f81a000 @ d9617000-da99f000
[    0.000000] xen: setting RW the range da19a000 - da99f000
[    0.000000] RAMDISK: 02043000 - 044ef000
[    0.000000] ACPI: RSDP 00000000000f00e0 00024 (v02 LENOVO)
[    0.000000] ACPI: XSDT 00000000daffe120 000AC (v01 LENOVO TP-8D    00001280 PTEC 00000002)
[    0.000000] ACPI: FACP 00000000dafe7000 000F4 (v04 LENOVO TP-8D    00001280 PTL  00000002)
[    0.000000] ACPI: DSDT 00000000dafea000 0F6D8 (v01 LENOVO TP-8D    00001280 INTL 20061109)
[    0.000000] ACPI: FACS 00000000daf2d000 00040
[    0.000000] ACPI: SLIC 00000000daffd000 00176 (v01 LENOVO TP-8D    00001280 PTEC 00000001)
[    0.000000] ACPI: SSDT 00000000daffc000 00249 (v01 LENOVO TP-SSDT2 00000200 INTL 20061109)
[    0.000000] ACPI: SSDT 00000000daffb000 00033 (v01 LENOVO TP-SSDT1 00000100 INTL 20061109)
[    0.000000] ACPI: SSDT 00000000daffa000 007D1 (v01 LENOVO SataAhci 00001000 INTL 20061109)
[    0.000000] ACPI: HPET 00000000dafe6000 00038 (v01 LENOVO TP-8D    00001280 PTL  00000002)
[    0.000000] ACPI: APIC 00000000dafe5000 00098 (v01 LENOVO TP-8D    00001280 PTL  00000002)
[    0.000000] ACPI: MCFG 00000000dafe4000 0003C (v01 LENOVO TP-8D    00001280 PTL  00000002)
[    0.000000] ACPI: ECDT 00000000dafe3000 00052 (v01 LENOVO TP-8D    00001280 PTL  00000002)
[    0.000000] ACPI: ASF! 00000000dafe9000 000A5 (v32 LENOVO TP-8D    00001280 PTL  00000002)
[    0.000000] ACPI: TCPA 00000000dafe2000 00032 (v02    PTL   LENOVO 06040000 LNVO 00000001)
[    0.000000] ACPI: SSDT 00000000dafe1000 00A69 (v01  PmRef  Cpu0Ist 00003000 INTL 20061109)
[    0.000000] ACPI: SSDT 00000000dafe0000 00996 (v01  PmRef    CpuPm 00003000 INTL 20061109)
[    0.000000] ACPI: XMAR 00000000dafdf000 000E8 (v01 INTEL      SNB  00000001 INTL 00000001)
[    0.000000] ACPI: UEFI 00000000dafde000 0003E (v01 LENOVO TP-8D    00001280 PTL  00000002)
[    0.000000] ACPI: UEFI 00000000dafdd000 00042 (v01 PTL      COMBUF 00000001 PTL  00000001)
[    0.000000] ACPI: UEFI 00000000dafdc000 00292 (v01 LENOVO TP-8D    00001280 PTL  00000002)
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] Setting APIC routing to cluster x2apic.
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-000000026f81a000
[    0.000000] Initmem setup node 0 0000000000000000-000000026f81a000
[    0.000000]   NODE_DATA [00000001e98f0000 - 00000001e98f4fff]
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   0x00100000 -> 0x0026f81a
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[7] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x0000009d
[    0.000000]     0: 0x00000100 -> 0x00020000
[    0.000000]     0: 0x00020200 -> 0x00040000
[    0.000000]     0: 0x00040200 -> 0x000da99f
[    0.000000]     0: 0x000dafff -> 0x000db000
[    0.000000]     0: 0x00100000 -> 0x001e98f5
[    0.000000]     0: 0x0021e800 -> 0x0026f81a
[    0.000000] On node 0 totalpages: 2182716
[    0.000000]   DMA zone: 56 pages used for memmap
[    0.000000]   DMA zone: 1712 pages reserved
[    0.000000]   DMA zone: 2213 pages, LIFO batch:0
[    0.000000]   DMA32 zone: 14280 pages used for memmap
[    0.000000]   DMA32 zone: 875992 pages, LIFO batch:31
[    0.000000]   Normal zone: 20581 pages used for memmap
[    0.000000]   Normal zone: 1267882 pages, LIFO batch:31
[    0.000000] ACPI: PM-Timer IO Port: 0x408
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x00] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x06] lapic_id[0x00] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x07] lapic_id[0x00] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x08] lapic_id[0x00] disabled)
[    0.000000] ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
[    0.000000] ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
[    0.000000] ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 2, version 255, address 0xfec00000, GSI 0-255
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] ACPI: IRQ0 used by override.
[    0.000000] ACPI: IRQ2 used by override.
[    0.000000] ACPI: IRQ9 used by override.
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0x8086a301 base: 0xfed00000
[    0.000000] SMP: Allowing 8 CPUs, 4 hotplug CPUs
[    0.000000] nr_irqs_gsi: 272
[    0.000000] PM: Registered nosave memory: 000000000009d000 - 000000000009e000
[    0.000000] PM: Registered nosave memory: 000000000009e000 - 0000000000100000
[    0.000000] PM: Registered nosave memory: 0000000020000000 - 0000000020200000
[    0.000000] PM: Registered nosave memory: 0000000040000000 - 0000000040200000
[    0.000000] PM: Registered nosave memory: 00000000da99f000 - 00000000dae9f000
[    0.000000] PM: Registered nosave memory: 00000000dae9f000 - 00000000daf9f000
[    0.000000] PM: Registered nosave memory: 00000000daf9f000 - 00000000dafff000
[    0.000000] PM: Registered nosave memory: 00000000db000000 - 00000000dfa00000
[    0.000000] PM: Registered nosave memory: 00000000dfa00000 - 00000000f8000000
[    0.000000] PM: Registered nosave memory: 00000000f8000000 - 00000000fc000000
[    0.000000] PM: Registered nosave memory: 00000000fc000000 - 00000000fec00000
[    0.000000] PM: Registered nosave memory: 00000000fec00000 - 00000000fec01000
[    0.000000] PM: Registered nosave memory: 00000000fec01000 - 00000000fed08000
[    0.000000] PM: Registered nosave memory: 00000000fed08000 - 00000000fed09000
[    0.000000] PM: Registered nosave memory: 00000000fed09000 - 00000000fed10000
[    0.000000] PM: Registered nosave memory: 00000000fed10000 - 00000000fed1a000
[    0.000000] PM: Registered nosave memory: 00000000fed1a000 - 00000000fed1c000
[    0.000000] PM: Registered nosave memory: 00000000fed1c000 - 00000000fed20000
[    0.000000] PM: Registered nosave memory: 00000000fed20000 - 00000000fee00000
[    0.000000] PM: Registered nosave memory: 00000000fee00000 - 00000000fee01000
[    0.000000] PM: Registered nosave memory: 00000000fee01000 - 00000000ffd20000
[    0.000000] PM: Registered nosave memory: 00000000ffd20000 - 0000000100000000
[    0.000000] PM: Registered nosave memory: 00000001e98f5000 - 000000021e600000
[    0.000000] PM: Registered nosave memory: 000000021e600000 - 000000021e800000
[    0.000000] Allocating PCI resources starting at dfa00000 (gap: dfa00000:18600000)
[    0.000000] Booting paravirtualized kernel on Xen
[    0.000000] Xen version: 4.2-unstable (preserve-AD)
[    0.000000] setup_percpu: NR_CPUS:256 nr_cpumask_bits:256 nr_cpu_ids:8 nr_node_ids:1
[    0.000000] PERCPU: Embedded 27 pages/cpu @ffff8801e97e0000 s79616 r8192 d22784 u110592
[    0.000000] pcpu-alloc: s79616 r8192 d22784 u110592 alloc=27*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5 [0] 6 [0] 7 
[    6.653356] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 2146087
[    6.661752] Policy zone: Normal
[    6.665011] Kernel command line: placeholder root=UUID=c7815afe-ae72-4cfa-b8b0-94b3035c0bb7 ro debug console=hvc0 earlyprintk=xen
[    6.677183] PID hash table entries: 4096 (order: 3, 32768 bytes)
(XEN) domain.c:698:d0 Attempt to change CR4 flags 00042660 -> 00002660
(XEN) domain.c:698:d0 Attempt to change CR4 flags 00042660 -> 00002660
(XEN) domain.c:698:d0 Attempt to change CR4 flags 00042660 -> 00002660
[    6.703128] xsave/xrstor: enabled xstate_bv 0x7, cntxt size 0x340
[    6.758517] Placing 64MB software IO TLB between ffff8801dc600000 - ffff8801e0600000
[    6.766504] software IO TLB at phys 0x1dc600000 - 0x1e0600000
[    6.796076] Memory: 7125060k/10215528k available (6140k kernel code, 1484664k absent, 1605804k reserved, 6894k data, 988k init)
[    6.808000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1
[    6.815917] Hierarchical RCU implementation.
[    6.820315] 	RCU dyntick-idle grace-period acceleration is enabled.
[    6.826832] NR_IRQS:16640 nr_irqs:2048 16
[    6.831047] xen: sci override: global_irq=9 trigger=0 polarity=0
[    6.837220] xen: registering gsi 9 triggering 0 polarity 0
[    6.842922] xen: --> pirq=9 -> irq=9 (gsi=9)
[    6.847367] xen: acpi sci 9
[    6.850229] xen: --> pirq=1 -> irq=1 (gsi=1)
[    6.854657] xen: --> pirq=2 -> irq=2 (gsi=2)
[    6.859078] xen: --> pirq=3 -> irq=3 (gsi=3)
[    6.863505] xen: --> pirq=4 -> irq=4 (gsi=4)
[    6.867931] xen: --> pirq=5 -> irq=5 (gsi=5)
[    6.872366] xen: --> pirq=6 -> irq=6 (gsi=6)
[    6.876798] xen: --> pirq=7 -> irq=7 (gsi=7)
[    6.881230] xen: --> pirq=8 -> irq=8 (gsi=8)
[    6.885652] xen_map_pirq_gsi: returning irq 9 for gsi 9
[    6.891069] xen: --> pirq=9 -> irq=9 (gsi=9)
[    6.895497] xen: --> pirq=10 -> irq=10 (gsi=10)
[    6.900196] xen: --> pirq=11 -> irq=11 (gsi=11)
[    6.904902] xen: --> pirq=12 -> irq=12 (gsi=12)
[    6.909603] xen: --> pirq=13 -> irq=13 (gsi=13)
[    6.914308] xen: --> pirq=14 -> irq=14 (gsi=14)
[    6.918999] xen: --> pirq=15 -> irq=15 (gsi=15)
[    6.924636] Console: colour VGA+ 80x25
[    6.928449] console [hvc0] enabled, bootconsole disabled
[    6.928449] console [hvc0] enabled, bootconsole disabled
[    6.947994] allocated 72351744 bytes of page_cgroup
[    6.953062] please try 'cgroup_disable=memory' option if you don't want memory cgroups
[    6.961428] Xen: using vcpuop timer interface
[    6.966006] installing Xen timer for CPU 0
[    6.970371] Detected 2691.360 MHz processor.
[    6.974857] Calibrating delay loop (skipped), value calculated using timer frequency.. 5382.72 BogoMIPS (lpj=10765440)
[    6.986049] pid_max: default: 32768 minimum: 301
[    6.990970] Security Framework initialized
[    6.995274] AppArmor: AppArmor initialized
[    6.999596] Yama: becoming mindful.
[    7.007825] Dentry cache hash table entries: 2097152 (order: 12, 16777216 bytes)
[    7.020293] Inode-cache hash table entries: 1048576 (order: 11, 8388608 bytes)
[    7.029087] Mount-cache hash table entries: 256
[    7.033972] Initializing cgroup subsys cpuacct
[    7.038588] Initializing cgroup subsys memory
[    7.043206] Initializing cgroup subsys devices
[    7.047903] Initializing cgroup subsys freezer
[    7.052594] Initializing cgroup subsys net_cls
[    7.057289] Initializing cgroup subsys blkio
[    7.061814] Initializing cgroup subsys perf_event
[    7.066824] Disabled fast string operations
[    7.071210] ENERGY_PERF_BIAS: Set to 'normal', was 'performance'
[    7.071211] ENERGY_PERF_BIAS: View and update with x86_energy_perf_policy(8)
[    7.084914] CPU: Physical Processor ID: 0
[    7.089163] CPU: Processor Core ID: 0
[    7.095238] ACPI: Core revision 20110413
[    7.128234] ftrace: allocating 25750 entries in 101 pages
[    7.140900] cpu 0 spinlock event irq 273
[    7.145046] Performance Events: unsupported p6 CPU model 42 no PMU driver, software events only.
[    7.154687] installing Xen timer for CPU 1
[    7.158988] cpu 1 spinlock event irq 279
(XEN) domain.c:698:d0 Attempt to change CR4 flags 00042660 -> 00002660
(XEN) domain.c:698:d0 Attempt to change CR4 flags 00042660 -> 00002660
(XEN) domain.c:698:d0 Attempt to change CR4 flags 00042660 -> 00002660
[    7.182805] Disabled fast string operations
[    7.182908] installing Xen timer for CPU 2
[    7.191668] cpu 2 spinlock event irq 285
(XEN) domain.c:698:d0 Attempt to change CR4 flags 00042660 -> 00002660
(XEN) domain.c:698:d0 Attempt to change CR4 flags 00042660 -> 00002660
(XEN) domain.c:698:d0 Attempt to change CR4 flags 00042660 -> 00002660
[    7.215501] Disabled fast string operations
[    7.215595] installing Xen timer for CPU 3
[    7.224345] cpu 3 spinlock event irq 291
(XEN) domain.c:698:d0 Attempt to change CR4 flags 00042660 -> 00002660
(XEN) domain.c:698:d0 Attempt to change CR4 flags 00042660 -> 00002660
(XEN) domain.c:698:d0 Attempt to change CR4 flags 00042660 -> 00002660
[    7.248180] Disabled fast string operations
[    7.248207] Brought up 4 CPUs
[    7.255999] devtmpfs: initialized
[    7.259709] PM: Registering ACPI NVS region at dae9f000 (1048576 bytes)
[    7.267758] Grant table initialized
[    7.271433] print_constraints: dummy: 
[    7.275435] Time: 10:58:24  Date: 05/05/12
[    7.279790] NET: Registered protocol family 16
[    7.284592] Trying to unpack rootfs image as initramfs...
[    7.294551] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it
[    7.302473] ACPI: bus type pci registered
[    7.307070] PCI: MMCONFIG for domain 0000 [bus 00-3f] at [mem 0xf8000000-0xfbffffff] (base 0xf8000000)
[    7.316808] PCI: MMCONFIG at [mem 0xf8000000-0xfbffffff] reserved in E820
[    7.332187] Freeing initrd memory: 37552k freed
[    7.348717] PCI: Using configuration type 1 for base access
[    7.355215] bio: create slab <bio-0> at 0
[    7.364180] ACPI: EC: EC description table is found, configuring boot EC
[    7.380879] [Firmware Bug]: ACPI: BIOS _OSI(Linux) query ignored
[    7.424825] ACPI: SSDT 00000000dae8c018 008C0 (v01  PmRef  Cpu0Cst 00003001 INTL 20061109)
[    7.433891] ACPI: Dynamic OEM Table Load:
[    7.438077] ACPI: SSDT           (null) 008C0 (v01  PmRef  Cpu0Cst 00003001 INTL 20061109)
[    7.471492] ACPI: SSDT 00000000dae8da98 00303 (v01  PmRef    ApIst 00003000 INTL 20061109)
[    7.480593] ACPI: Dynamic OEM Table Load:
[    7.484786] ACPI: SSDT           (null) 00303 (v01  PmRef    ApIst 00003000 INTL 20061109)
[    7.509932] ACPI: SSDT 00000000dae8bd98 00119 (v01  PmRef    ApCst 00003000 INTL 20061109)
[    7.519739] ACPI: Dynamic OEM Table Load:
[    7.523930] ACPI: SSDT           (null) 00119 (v01  PmRef    ApCst 00003000 INTL 20061109)
[    7.549702] ACPI: Interpreter enabled
[    7.553527] ACPI: (supports S0 S3 S4 S5)
[    7.557709] ACPI: Using IOAPIC for interrupt routing
[    7.587974] ACPI: Power Resource [PUBS] (on)
[    7.596697] ACPI: EC: GPE = 0x11, I/O: command/status = 0x66, data = 0x62
[    7.605522] ACPI: ACPI Dock Station Driver: 3 docks/bays found
[    7.611618] HEST: Table not found.
[    7.615252] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    7.625019] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-fe])
[    7.631534] pci_root PNP0A08:00: host bridge window [io  0x0000-0x0cf7]
[    7.638481] pci_root PNP0A08:00: host bridge window [io  0x0d00-0xffff]
[    7.645474] pci_root PNP0A08:00: host bridge window [mem 0x000a0000-0x000bffff]
[    7.653205] pci_root PNP0A08:00: host bridge window [mem 0xdfa00000-0xfebfffff]
[    7.660927] pci_root PNP0A08:00: host bridge window [mem 0xfed40000-0xfed4bfff]
[    7.668665] pci 0000:00:00.0: [8086:0104] type 0 class 0x000600
[    7.675035] pci 0000:00:02.0: [8086:0126] type 0 class 0x000300
[    7.681252] pci 0000:00:02.0: reg 10: [mem 0xf0000000-0xf03fffff 64bit]
[    7.688227] pci 0000:00:02.0: reg 18: [mem 0xe0000000-0xefffffff 64bit pref]
[    7.695669] pci 0000:00:02.0: reg 20: [io  0x5000-0x503f]
[    7.701547] pci 0000:00:16.0: [8086:1c3a] type 0 class 0x000780
[    7.707784] pci 0000:00:16.0: reg 10: [mem 0xf2625000-0xf262500f 64bit]
[    7.714894] pci 0000:00:16.0: PME# supported from D0 D3hot D3cold
[    7.721260] pci 0000:00:16.0: PME# disabled
[    7.725758] pci 0000:00:16.3: [8086:1c3d] type 0 class 0x000700
[    7.732015] pci 0000:00:16.3: reg 10: [io  0x50b0-0x50b7]
[    7.737713] pci 0000:00:16.3: reg 14: [mem 0xf262c000-0xf262cfff]
[    7.744399] pci 0000:00:19.0: [8086:1502] type 0 class 0x000200
[    7.750629] pci 0000:00:19.0: reg 10: [mem 0xf2600000-0xf261ffff]
[    7.757053] pci 0000:00:19.0: reg 14: [mem 0xf262b000-0xf262bfff]
[    7.763512] pci 0000:00:19.0: reg 18: [io  0x5080-0x509f]
[    7.769400] pci 0000:00:19.0: PME# supported from D0 D3hot D3cold
[    7.775769] pci 0000:00:19.0: PME# disabled
[    7.780264] pci 0000:00:1a.0: [8086:1c2d] type 0 class 0x000c03
[    7.786519] pci 0000:00:1a.0: reg 10: [mem 0xf262a000-0xf262a3ff]
[    7.793143] pci 0000:00:1a.0: PME# supported from D0 D3hot D3cold
[    7.799502] pci 0000:00:1a.0: PME# disabled
[    7.804010] pci 0000:00:1b.0: [8086:1c20] type 0 class 0x000403
[    7.810252] pci 0000:00:1b.0: reg 10: [mem 0xf2620000-0xf2623fff 64bit]
[    7.817409] pci 0000:00:1b.0: PME# supported from D0 D3hot D3cold
[    7.823766] pci 0000:00:1b.0: PME# disabled
[    7.828254] pci 0000:00:1c.0: [8086:1c10] type 1 class 0x000604
[    7.834676] pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold
[    7.841048] pci 0000:00:1c.0: PME# disabled
[    7.845539] pci 0000:00:1c.1: [8086:1c12] type 1 class 0x000604
[    7.851951] pci 0000:00:1c.1: PME# supported from D0 D3hot D3cold
[    7.858326] pci 0000:00:1c.1: PME# disabled
[    7.862823] pci 0000:00:1c.3: [8086:1c16] type 1 class 0x000604
[    7.869261] pci 0000:00:1c.3: PME# supported from D0 D3hot D3cold
[    7.875621] pci 0000:00:1c.3: PME# disabled
[    7.880121] pci 0000:00:1c.4: [8086:1c18] type 1 class 0x000604
[    7.886542] pci 0000:00:1c.4: PME# supported from D0 D3hot D3cold
[    7.892922] pci 0000:00:1c.4: PME# disabled
[    7.897412] pci 0000:00:1c.6: [8086:1c1c] type 1 class 0x000604
[    7.903838] pci 0000:00:1c.6: PME# supported from D0 D3hot D3cold
[    7.910205] pci 0000:00:1c.6: PME# disabled
[    7.914723] pci 0000:00:1d.0: [8086:1c26] type 0 class 0x000c03
[    7.920982] pci 0000:00:1d.0: reg 10: [mem 0xf2629000-0xf26293ff]
[    7.927601] pci 0000:00:1d.0: PME# supported from D0 D3hot D3cold
[    7.933977] pci 0000:00:1d.0: PME# disabled
[    7.938473] pci 0000:00:1f.0: [8086:1c4f] type 0 class 0x000601
[    7.944983] pci 0000:00:1f.2: [8086:1c03] type 0 class 0x000106
[    7.951206] pci 0000:00:1f.2: reg 10: [io  0x50a8-0x50af]
[    7.956891] pci 0000:00:1f.2: reg 14: [io  0x50bc-0x50bf]
[    7.962622] pci 0000:00:1f.2: reg 18: [io  0x50a0-0x50a7]
[    7.968338] pci 0000:00:1f.2: reg 1c: [io  0x50b8-0x50bb]
[    7.974050] pci 0000:00:1f.2: reg 20: [io  0x5060-0x507f]
[    7.979766] pci 0000:00:1f.2: reg 24: [mem 0xf2628000-0xf26287ff]
[    7.986332] pci 0000:00:1f.2: PME# supported from D3hot
[    7.991790] pci 0000:00:1f.2: PME# disabled
[    7.996263] pci 0000:00:1f.3: [8086:1c22] type 0 class 0x000c05
[    8.002537] pci 0000:00:1f.3: reg 10: [mem 0xf2624000-0xf26240ff 64bit]
[    8.009564] pci 0000:00:1f.3: reg 20: [io  0xefa0-0xefbf]
[    8.015417] pci 0000:00:1c.0: PCI bridge to [bus 02-02]
[    8.020887] pci 0000:00:1c.0:   bridge window [io  0xf000-0x0000] (disabled)
[    8.028323] pci 0000:00:1c.0:   bridge window [mem 0xfff00000-0x000fffff] (disabled)
[    8.036496] pci 0000:00:1c.0:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
[    8.045508] pci 0000:03:00.0: [8086:4238] type 0 class 0x000280
[    8.052061] pci 0000:03:00.0: reg 10: [mem 0xf2500000-0xf2501fff 64bit]
[    8.060771] pci 0000:03:00.0: PME# supported from D0 D3hot D3cold
[    8.067196] pci 0000:03:00.0: PME# disabled
[    8.076189] pci 0000:00:1c.1: PCI bridge to [bus 03-03]
[    8.081656] pci 0000:00:1c.1:   bridge window [io  0xf000-0x0000] (disabled)
[    8.089110] pci 0000:00:1c.1:   bridge window [mem 0xf2500000-0xf25fffff]
[    8.096302] pci 0000:00:1c.1:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
[    8.105058] pci 0000:05:00.0: [1415:c120] type 0 class 0x000700
[    8.111272] pci 0000:05:00.0: reg 10: [io  0x4000-0x4007]
[    8.117206] pci 0000:05:00.0: supports D1 D2
[    8.121659] pci 0000:05:00.0: PME# supported from D1 D2 D3hot D3cold
[    8.128390] pci 0000:05:00.0: PME# disabled
[    8.137011] pci 0000:00:1c.3: PCI bridge to [bus 05-0c]
[    8.142485] pci 0000:00:1c.3:   bridge window [io  0x4000-0x4fff]
[    8.148926] pci 0000:00:1c.3:   bridge window [mem 0xf1d00000-0xf24fffff]
[    8.156123] pci 0000:00:1c.3:   bridge window [mem 0xf0400000-0xf0bfffff 64bit pref]
[    8.164632] pci 0000:0d:00.0: [1180:e822] type 0 class 0x000880
[    8.170985] pci 0000:0d:00.0: reg 10: [mem 0xf1500000-0xf15000ff]
[    8.177802] pci 0000:0d:00.0: supports D1 D2
[    8.182262] pci 0000:0d:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    8.189401] pci 0000:0d:00.0: PME# disabled
[    8.198270] pci 0000:00:1c.4: PCI bridge to [bus 0d-0d]
[    8.203733] pci 0000:00:1c.4:   bridge window [io  0x3000-0x3fff]
[    8.210179] pci 0000:00:1c.4:   bridge window [mem 0xf1500000-0xf1cfffff]
[    8.217392] pci 0000:00:1c.4:   bridge window [mem 0xf0c00000-0xf13fffff 64bit pref]
[    8.225708] pci 0000:0e:00.0: [1033:0194] type 0 class 0x000c03
[    8.231947] pci 0000:0e:00.0: reg 10: [mem 0xf1400000-0xf1401fff 64bit]
[    8.239137] pci 0000:0e:00.0: PME# supported from D0 D3hot D3cold
[    8.245502] pci 0000:0e:00.0: PME# disabled
[    8.250043] pci 0000:00:1c.6: PCI bridge to [bus 0e-0e]
[    8.255503] pci 0000:00:1c.6:   bridge window [io  0xf000-0x0000] (disabled)
[    8.262955] pci 0000:00:1c.6:   bridge window [mem 0xf1400000-0xf14fffff]
[    8.270134] pci 0000:00:1c.6:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
[    8.278798] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
[    8.285085] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.EXP1._PRT]
[    8.291758] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.EXP2._PRT]
[    8.298484] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.EXP4._PRT]
[    8.305210] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.EXP5._PRT]
[    8.311933] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.EXP7._PRT]
[    8.318786]  pci0000:00: Requesting ACPI _OSC control (0x1d)
[    8.324930]  pci0000:00: ACPI _OSC request failed (AE_SUPPORT), returned control mask: 0x0d
[    8.333654] ACPI _OSC control for PCIe not granted, disabling ASPM
(XEN) PCI add device 0000:00:00.0
(XEN) PCI add device 0000:00:02.0
(XEN) PCI add device 0000:00:16.0
(XEN) PCI add device 0000:00:16.3
(XEN) PCI add device 0000:00:19.0
(XEN) PCI add device 0000:00:1a.0
(XEN) PCI add device 0000:00:1b.0
(XEN) PCI add device 0000:00:1c.0
(XEN) PCI add device 0000:00:1c.1
(XEN) PCI add device 0000:00:1c.3
(XEN) PCI add device 0000:00:1c.4
(XEN) PCI add device 0000:00:1c.6
(XEN) PCI add device 0000:00:1d.0
(XEN) PCI add device 0000:00:1f.0
(XEN) PCI add device 0000:00:1f.2
(XEN) PCI add device 0000:00:1f.3
(XEN) PCI add device 0000:03:00.0
(XEN) PCI add device 0000:05:00.0
(XEN) PCI add device 0000:0d:00.0
(XEN) PCI add device 0000:0e:00.0
[    8.405275] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 9 10 *11)
[    8.412079] ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 *7 9 10 11)
[    8.418885] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 *7 9 10 11)
[    8.425706] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 9 10 *11)
[    8.432509] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 9 *10 11)
[    8.439322] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 9 10 11) *0, disabled.
[    8.447313] ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 9 10 *11)
[    8.454132] ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 7 9 *10 11)
[    8.460917] xen/balloon: Initialising balloon driver.
[    8.466257] last_pfn = 0x26f81a max_arch_pfn = 0x400000000
[    8.474888] xen-balloon: Initialising balloon driver.
[    8.480262] vgaarb: device added: PCI:0000:00:02.0,decodes=io+mem,owns=io+mem,locks=none
[    8.488738] vgaarb: loaded
[    8.491638] vgaarb: bridge control possible 0000:00:02.0
[    8.497404] SCSI subsystem initialized
[    8.501413] libata version 3.00 loaded.
[    8.505463] usbcore: registered new interface driver usbfs
[    8.511249] usbcore: registered new interface driver hub
[    8.516913] usbcore: registered new device driver usb
[    8.522318] PCI: Using ACPI for IRQ routing
[    8.531461] PCI: pci_cache_line_size set to 64 bytes
[    8.537042] reserve RAM buffer: 000000000009d000 - 000000000009ffff 
[    8.543403] reserve RAM buffer: 00000000da99f000 - 00000000dbffffff 
[    8.550127] reserve RAM buffer: 00000000db000000 - 00000000dbffffff 
[    8.556852] reserve RAM buffer: 00000001e98f5000 - 00000001ebffffff 
[    8.563569] reserve RAM buffer: 000000026f81a000 - 000000026fffffff 
[    8.570382] NetLabel: Initializing
[    8.574213] NetLabel:  domain hash size = 128
[    8.578843] NetLabel:  protocols = UNLABELED CIPSOv4
[    8.584120] NetLabel:  unlabeled traffic allowed by default
[    8.590063] Switching to clocksource xen
[    8.598253] Switched to NOHz mode on CPU #0
[    8.601598] Switched to NOHz mode on CPU #3
[    8.601783] Switched to NOHz mode on CPU #1
[    8.604062] Switched to NOHz mode on CPU #2
[    8.616813] AppArmor: AppArmor Filesystem Enabled
[    8.621737] pnp: PnP ACPI init
[    8.625008] ACPI: bus type pnp registered
[    8.629716] pnp 00:00: [mem 0x00000000-0x0009ffff]
[    8.634715] pnp 00:00: [mem 0x000c0000-0x000c3fff]
[    8.639808] pnp 00:00: [mem 0x000c4000-0x000c7fff]
[    8.644907] pnp 00:00: [mem 0x000c8000-0x000cbfff]
[    8.649992] pnp 00:00: [mem 0x000cc000-0x000cffff]
[    8.655085] pnp 00:00: [mem 0x000d0000-0x000d3fff]
[    8.660175] pnp 00:00: [mem 0x000d4000-0x000d7fff]
[    8.665265] pnp 00:00: [mem 0x000d8000-0x000dbfff]
[    8.670348] pnp 00:00: [mem 0x000dc000-0x000dffff]
[    8.675442] pnp 00:00: [mem 0x000e0000-0x000e3fff]
[    8.680523] pnp 00:00: [mem 0x000e4000-0x000e7fff]
[    8.685624] pnp 00:00: [mem 0x000e8000-0x000ebfff]
[    8.690721] pnp 00:00: [mem 0x000ec000-0x000effff]
[    8.695807] pnp 00:00: [mem 0x000f0000-0x000fffff]
[    8.700896] pnp 00:00: [mem 0x00100000-0xdf9fffff]
[    8.705981] pnp 00:00: [mem 0xfec00000-0xfed3ffff]
[    8.711061] pnp 00:00: [mem 0xfed4c000-0xffffffff]
[    8.716209] system 00:00: [mem 0x00000000-0x0009ffff] could not be reserved
[    8.723512] system 00:00: [mem 0x000c0000-0x000c3fff] could not be reserved
[    8.730876] system 00:00: [mem 0x000c4000-0x000c7fff] could not be reserved
[    8.738246] system 00:00: [mem 0x000c8000-0x000cbfff] could not be reserved
[    8.745604] system 00:00: [mem 0x000cc000-0x000cffff] could not be reserved
[    8.752968] system 00:00: [mem 0x000d0000-0x000d3fff] could not be reserved
[    8.760332] system 00:00: [mem 0x000d4000-0x000d7fff] could not be reserved
[    8.767688] system 00:00: [mem 0x000d8000-0x000dbfff] could not be reserved
[    8.775053] system 00:00: [mem 0x000dc000-0x000dffff] could not be reserved
[    8.782408] system 00:00: [mem 0x000e0000-0x000e3fff] could not be reserved
[    8.789768] system 00:00: [mem 0x000e4000-0x000e7fff] could not be reserved
[    8.797126] system 00:00: [mem 0x000e8000-0x000ebfff] could not be reserved
[    8.804497] system 00:00: [mem 0x000ec000-0x000effff] could not be reserved
[    8.811859] system 00:00: [mem 0x000f0000-0x000fffff] could not be reserved
[    8.819222] system 00:00: [mem 0x00100000-0xdf9fffff] could not be reserved
[    8.826579] system 00:00: [mem 0xfec00000-0xfed3ffff] could not be reserved
[    8.833935] system 00:00: [mem 0xfed4c000-0xffffffff] could not be reserved
[    8.841295] system 00:00: Plug and Play ACPI device, IDs PNP0c01 (active)
[    8.848489] pnp 00:01: [bus 00-fe]
[    8.852107] pnp 00:01: [io  0x0cf8-0x0cff]
[    8.856464] pnp 00:01: [io  0x0000-0x0cf7 window]
[    8.861456] pnp 00:01: [io  0x0d00-0xffff window]
[    8.866449] pnp 00:01: [mem 0x000a0000-0x000bffff window]
[    8.872172] pnp 00:01: [mem 0x000c0000-0x000c3fff window]
[    8.877890] pnp 00:01: [mem 0x000c4000-0x000c7fff window]
[    8.883613] pnp 00:01: [mem 0x000c8000-0x000cbfff window]
[    8.889335] pnp 00:01: [mem 0x000cc000-0x000cffff window]
[    8.895059] pnp 00:01: [mem 0x000d0000-0x000d3fff window]
[    8.900775] pnp 00:01: [mem 0x000d4000-0x000d7fff window]
[    8.906500] pnp 00:01: [mem 0x000d8000-0x000dbfff window]
[    8.912229] pnp 00:01: [mem 0x000dc000-0x000dffff window]
[    8.917953] pnp 00:01: [mem 0x000e0000-0x000e3fff window]
[    8.923687] pnp 00:01: [mem 0x000e4000-0x000e7fff window]
[    8.929414] pnp 00:01: [mem 0x000e8000-0x000ebfff window]
[    8.935138] pnp 00:01: [mem 0x000ec000-0x000effff window]
[    8.940851] pnp 00:01: [mem 0xdfa00000-0xfebfffff window]
[    8.946585] pnp 00:01: [mem 0xfed40000-0xfed4bfff window]
[    8.952354] pnp 00:01: Plug and Play ACPI device, IDs PNP0a08 PNP0a03 (active)
[    8.960002] pnp 00:02: [io  0x0010-0x001f]
[    8.964298] pnp 00:02: [io  0x0090-0x009f]
[    8.968659] pnp 00:02: [io  0x0024-0x0025]
[    8.973014] pnp 00:02: [io  0x0028-0x0029]
[    8.977361] pnp 00:02: [io  0x002c-0x002d]
[    8.981725] pnp 00:02: [io  0x0030-0x0031]
[    8.986088] pnp 00:02: [io  0x0034-0x0035]
[    8.990452] pnp 00:02: [io  0x0038-0x0039]
[    8.994812] pnp 00:02: [io  0x003c-0x003d]
[    8.999176] pnp 00:02: [io  0x00a4-0x00a5]
[    9.003531] pnp 00:02: [io  0x00a8-0x00a9]
[    9.007892] pnp 00:02: [io  0x00ac-0x00ad]
[    9.012264] pnp 00:02: [io  0x00b0-0x00b5]
[    9.016627] pnp 00:02: [io  0x00b8-0x00b9]
[    9.020982] pnp 00:02: [io  0x00bc-0x00bd]
[    9.025343] pnp 00:02: [io  0x0050-0x0053]
[    9.029700] pnp 00:02: [io  0x0072-0x0077]
[    9.034064] pnp 00:02: [io  0x0400-0x047f]
[    9.038431] pnp 00:02: [io  0x0500-0x057f]
[    9.042798] pnp 00:02: [io  0x0800-0x080f]
[    9.047159] pnp 00:02: [io  0x15e0-0x15ef]
[    9.051518] pnp 00:02: [io  0x1600-0x167f]
[    9.055876] pnp 00:02: [mem 0xf8000000-0xfbffffff]
[    9.060961] pnp 00:02: [mem 0x00000000-0x00000fff]
[    9.066052] pnp 00:02: [mem 0xfed1c000-0xfed1ffff]
[    9.071138] pnp 00:02: [mem 0xfed10000-0xfed13fff]
[    9.076226] pnp 00:02: [mem 0xfed18000-0xfed18fff]
[    9.081318] pnp 00:02: [mem 0xfed19000-0xfed19fff]
[    9.086403] pnp 00:02: [mem 0xfed45000-0xfed4bfff]
[    9.091541] system 00:02: [io  0x0400-0x047f] has been reserved
[    9.097755] system 00:02: [io  0x0500-0x057f] has been reserved
[    9.104028] system 00:02: [io  0x0800-0x080f] has been reserved
[    9.110296] system 00:02: [io  0x15e0-0x15ef] has been reserved
[    9.116565] system 00:02: [io  0x1600-0x167f] has been reserved
[    9.122832] system 00:02: [mem 0xf8000000-0xfbffffff] has been reserved
[    9.129836] system 00:02: [mem 0x00000000-0x00000fff] could not be reserved
[    9.137205] system 00:02: [mem 0xfed1c000-0xfed1ffff] has been reserved
[    9.144202] system 00:02: [mem 0xfed10000-0xfed13fff] has been reserved
[    9.151199] system 00:02: [mem 0xfed18000-0xfed18fff] has been reserved
[    9.158189] system 00:02: [mem 0xfed19000-0xfed19fff] has been reserved
[    9.165186] system 00:02: [mem 0xfed45000-0xfed4bfff] has been reserved
[    9.172185] system 00:02: Plug and Play ACPI device, IDs PNP0c02 (active)
[    9.179397] pnp 00:03: [mem 0xfed00000-0xfed003ff]
[    9.184464] pnp 00:03: Plug and Play ACPI device, IDs PNP0103 (active)
[    9.191347] pnp 00:04: [io  0x0000-0x000f]
[    9.195676] pnp 00:04: [io  0x0080-0x008f]
[    9.200025] pnp 00:04: [io  0x00c0-0x00df]
[    9.204387] pnp 00:04: [dma 4]
[    9.207698] pnp 00:04: Plug and Play ACPI device, IDs PNP0200 (active)
[    9.214562] pnp 00:05: [io  0x0061]
[    9.218338] pnp 00:05: Plug and Play ACPI device, IDs PNP0800 (active)
[    9.225231] pnp 00:06: [io  0x00f0]
[    9.228954] xen: registering gsi 13 triggering 1 polarity 0
[    9.234842] xen_map_pirq_gsi: returning irq 13 for gsi 13
[    9.240566] xen: --> pirq=13 -> irq=13 (gsi=13)
[    9.245413] pnp 00:06: [irq 13]
[    9.248765] pnp 00:06: Plug and Play ACPI device, IDs PNP0c04 (active)
[    9.255674] pnp 00:07: [io  0x0070-0x0071]
[    9.260033] xen: registering gsi 8 triggering 1 polarity 0
[    9.265841] xen_map_pirq_gsi: returning irq 8 for gsi 8
[    9.271376] xen: --> pirq=8 -> irq=8 (gsi=8)
[    9.275943] pnp 00:07: [irq 8]
[    9.279215] pnp 00:07: Plug and Play ACPI device, IDs PNP0b00 (active)
[    9.286097] pnp 00:08: [io  0x0060]
[    9.289811] pnp 00:08: [io  0x0064]
[    9.293538] xen: registering gsi 1 triggering 1 polarity 0
[    9.299346] xen_map_pirq_gsi: returning irq 1 for gsi 1
[    9.304893] xen: --> pirq=1 -> irq=1 (gsi=1)
[    9.309449] pnp 00:08: [irq 1]
[    9.312721] pnp 00:08: Plug and Play ACPI device, IDs PNP0303 (active)
[    9.319618] xen: registering gsi 12 triggering 1 polarity 0
[    9.325521] xen_map_pirq_gsi: returning irq 12 for gsi 12
[    9.331246] xen: --> pirq=12 -> irq=12 (gsi=12)
[    9.336086] pnp 00:09: [irq 12]
[    9.339453] pnp 00:09: Plug and Play ACPI device, IDs LEN0020 PNP0f13 (active)
[    9.347082] pnp 00:0a: [mem 0xfed40000-0xfed44fff]
[    9.352143] pnp 00:0a: Plug and Play ACPI device, IDs SMO1200 PNP0c31 (active)
[    9.360409] pnp: PnP ACPI: found 11 devices
[    9.364782] ACPI: ACPI bus type pnp unregistered
[    9.375054] PM-Timer failed consistency check  (0x0xffffff) - aborting.
[    9.381987] PCI: max bus depth: 1 pci_try_num: 2
[    9.386958] pci 0000:00:1c.0: PCI bridge to [bus 02-02]
[    9.392414] pci 0000:00:1c.0:   bridge window [io  disabled]
[    9.398419] pci 0000:00:1c.0:   bridge window [mem disabled]
[    9.404417] pci 0000:00:1c.0:   bridge window [mem pref disabled]
[    9.410883] pci 0000:00:1c.1: PCI bridge to [bus 03-03]
[    9.416409] pci 0000:00:1c.1:   bridge window [io  disabled]
[    9.422416] pci 0000:00:1c.1:   bridge window [mem 0xf2500000-0xf25fffff]
[    9.429596] pci 0000:00:1c.1:   bridge window [mem pref disabled]
[    9.436031] pci 0000:00:1c.3: PCI bridge to [bus 05-0c]
[    9.441565] pci 0000:00:1c.3:   bridge window [io  0x4000-0x4fff]
[    9.448027] pci 0000:00:1c.3:   bridge window [mem 0xf1d00000-0xf24fffff]
[    9.455200] pci 0000:00:1c.3:   bridge window [mem 0xf0400000-0xf0bfffff 64bit pref]
[    9.463383] pci 0000:00:1c.4: PCI bridge to [bus 0d-0d]
[    9.468920] pci 0000:00:1c.4:   bridge window [io  0x3000-0x3fff]
[    9.475375] pci 0000:00:1c.4:   bridge window [mem 0xf1500000-0xf1cfffff]
[    9.482567] pci 0000:00:1c.4:   bridge window [mem 0xf0c00000-0xf13fffff 64bit pref]
[    9.490752] pci 0000:00:1c.6: PCI bridge to [bus 0e-0e]
[    9.496278] pci 0000:00:1c.6:   bridge window [io  disabled]
[    9.502291] pci 0000:00:1c.6:   bridge window [mem 0xf1400000-0xf14fffff]
[    9.509470] pci 0000:00:1c.6:   bridge window [mem pref disabled]
[    9.515943] xen: registering gsi 16 triggering 0 polarity 1
[    9.521848] xen: --> pirq=16 -> irq=16 (gsi=16)
[    9.526674] pci 0000:00:1c.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    9.533742] pci 0000:00:1c.0: setting latency timer to 64
[    9.539473] xen: registering gsi 17 triggering 0 polarity 1
[    9.545383] xen: --> pirq=17 -> irq=17 (gsi=17)
[    9.550210] pci 0000:00:1c.1: PCI INT B -> GSI 17 (level, low) -> IRQ 17
[    9.557296] pci 0000:00:1c.1: setting latency timer to 64
[    9.563033] xen: registering gsi 19 triggering 0 polarity 1
[    9.568945] xen: --> pirq=19 -> irq=19 (gsi=19)
[    9.573786] pci 0000:00:1c.3: PCI INT D -> GSI 19 (level, low) -> IRQ 19
[    9.580861] pci 0000:00:1c.3: setting latency timer to 64
[    9.586577] xen: registering gsi 16 triggering 0 polarity 1
[    9.592474] xen_map_pirq_gsi: returning irq 16 for gsi 16
[    9.598200] xen: --> pirq=16 -> irq=16 (gsi=16)
[    9.603014] Already setup the GSI :16
[    9.606903] pci 0000:00:1c.4: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    9.613991] pci 0000:00:1c.4: setting latency timer to 64
[    9.619703] xen: registering gsi 18 triggering 0 polarity 1
[    9.625607] xen: --> pirq=18 -> irq=18 (gsi=18)
[    9.630447] pci 0000:00:1c.6: PCI INT C -> GSI 18 (level, low) -> IRQ 18
[    9.637506] pci 0000:00:1c.6: setting latency timer to 64
[    9.643224] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7]
[    9.649118] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff]
[    9.655040] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff]
[    9.661678] pci_bus 0000:00: resource 7 [mem 0xdfa00000-0xfebfffff]
[    9.668310] pci_bus 0000:00: resource 8 [mem 0xfed40000-0xfed4bfff]
[    9.674939] pci_bus 0000:03: resource 1 [mem 0xf2500000-0xf25fffff]
[    9.681559] pci_bus 0000:05: resource 0 [io  0x4000-0x4fff]
[    9.687464] pci_bus 0000:05: resource 1 [mem 0xf1d00000-0xf24fffff]
[    9.694109] pci_bus 0000:05: resource 2 [mem 0xf0400000-0xf0bfffff 64bit pref]
[    9.701734] pci_bus 0000:0d: resource 0 [io  0x3000-0x3fff]
[    9.707639] pci_bus 0000:0d: resource 1 [mem 0xf1500000-0xf1cfffff]
[    9.714272] pci_bus 0000:0d: resource 2 [mem 0xf0c00000-0xf13fffff 64bit pref]
[    9.721895] pci_bus 0000:0e: resource 1 [mem 0xf1400000-0xf14fffff]
[    9.728569] NET: Registered protocol family 2
[    9.734339] IP route cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    9.746176] TCP established hash table entries: 524288 (order: 11, 8388608 bytes)
[    9.755331] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    9.762456] TCP: Hash tables configured (established 524288 bind 65536)
[    9.769388] TCP reno registered
[    9.772812] UDP hash table entries: 8192 (order: 6, 262144 bytes)
[    9.779331] UDP-Lite hash table entries: 8192 (order: 6, 262144 bytes)
[    9.786269] NET: Registered protocol family 1
[    9.790827] pci 0000:00:02.0: Boot video device
[    9.795968] PCI: CLS 64 bytes, default 64
[    9.800546] audit: initializing netlink socket (disabled)
[    9.806203] type=2000 audit(1336215505.802:1): initialized
[    9.836689] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    9.844526] VFS: Disk quotas dquot_6.5.2
[    9.848678] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    9.855871] fuse init (API version 7.16)
[    9.860038] msgmni has been set to 13989
[    9.864688] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
[    9.872449] io scheduler noop registered
[    9.876577] io scheduler deadline registered
[    9.881157] io scheduler cfq registered (default)
[    9.886730] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[    9.892563] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
[    9.899816] ACPI: Deprecated procfs I/F for AC is loaded, please retry with CONFIG_ACPI_PROCFS_POWER cleared
[    9.910298] ACPI: AC Adapter [AC] (on-line)
[    9.914889] input: Lid Switch as /devices/LNXSYSTM:00/device:00/PNP0C0D:00/input/input0
[    9.923616] ACPI: Lid Switch [LID]
[    9.927183] input: Sleep Button as /devices/LNXSYSTM:00/device:00/PNP0C0E:00/input/input1
[    9.935773] ACPI: Sleep Button [SLPB]
[    9.939721] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input2
[    9.947478] ACPI: Power Button [PWRF]
[    9.951417] ACPI: acpi_idle registered with cpuidle
[    9.960035] thermal LNXTHERM:00: registered as thermal_zone0
[    9.965939] ACPI: Thermal Zone [THM0] (60 C)
[    9.970496] ACPI: Deprecated procfs I/F for battery is loaded, please retry with CONFIG_ACPI_PROCFS_POWER cleared
[    9.981311] ERST: Table is not found!
[    9.985646] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled
[   10.011415] ACPI: Battery Slot [BAT0] (battery present)
[   10.062442] xen: registering gsi 19 triggering 0 polarity 1
[   10.068274] xen_map_pirq_gsi: returning irq 19 for gsi 19
[   10.073997] xen: --> pirq=19 -> irq=19 (gsi=19)
[   10.078831] Already setup the GSI :19
[   10.082734] serial 0000:00:16.3: PCI INT B -> GSI 19 (level, low) -> IRQ 19
[   10.110709] 0000:00:16.3: ttyS4 at I/O 0x50b0 (irq = 19) is a 16550A
[   10.117435] xen: registering gsi 19 triggering 0 polarity 1
[   10.123244] xen_map_pirq_gsi: returning irq 19 for gsi 19
[   10.128944] xen: --> pirq=19 -> irq=19 (gsi=19)
[   10.133757] Already setup the GSI :19
[   10.137635] serial 0000:05:00.0: PCI INT A -> GSI 19 (level, low) -> IRQ 19
[   10.145201] hpet_acpi_add: no address or irqs in _CRS
[   10.150492] Linux agpgart interface v0.103
[   10.154936] agpgart-intel 0000:00:00.0: Intel Sandybridge Chipset
[   10.161518] agpgart-intel 0000:00:00.0: detected gtt size: 2097152K total, 262144K mappable
[   10.171810] agpgart-intel 0000:00:00.0: detected 65536K stolen memory
[   10.178704] agpgart-intel 0000:00:00.0: AGP aperture is 256M @ 0xe0000000
[   10.186588] brd: module loaded
[   10.190096] loop: module loaded
[   10.193690] Fixed MDIO Bus: probed
[   10.197257] PPP generic driver version 2.4.2
[   10.201838] tun: Universal TUN/TAP device driver, 1.6
[   10.207167] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[   10.213776] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[   10.220683] ehci_hcd 0000:00:1a.0: power state changed by ACPI to D0
[   10.227365] ehci_hcd 0000:00:1a.0: power state changed by ACPI to D0
[   10.234087] xen: registering gsi 16 triggering 0 polarity 1
[   10.239971] xen_map_pirq_gsi: returning irq 16 for gsi 16
[   10.245680] xen: --> pirq=16 -> irq=16 (gsi=16)
[   10.250495] Already setup the GSI :16
[   10.254389] ehci_hcd 0000:00:1a.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[   10.261927] ehci_hcd 0000:00:1a.0: setting latency timer to 64
[   10.268069] ehci_hcd 0000:00:1a.0: EHCI Host Controller
[   10.273629] ehci_hcd 0000:00:1a.0: new USB bus registered, assigned bus number 1
[   10.281456] ehci_hcd 0000:00:1a.0: debug port 2
[   10.290091] ehci_hcd 0000:00:1a.0: cache line size of 64 is not supported
[   10.297213] ehci_hcd 0000:00:1a.0: irq 16, io mem 0xf262a000
[   10.307070] ehci_hcd 0000:00:1a.0: USB 2.0 started, EHCI 1.00
[   10.313170] hub 1-0:1.0: USB hub found
[   10.317089] hub 1-0:1.0: 3 ports detected
[   10.321406] ehci_hcd 0000:00:1d.0: power state changed by ACPI to D0
[   10.328074] ehci_hcd 0000:00:1d.0: power state changed by ACPI to D0
[   10.334805] xen: registering gsi 23 triggering 0 polarity 1
[   10.340707] xen: --> pirq=23 -> irq=23 (gsi=23)
[   10.345521] ehci_hcd 0000:00:1d.0: PCI INT A -> GSI 23 (level, low) -> IRQ 23
[   10.353064] ehci_hcd 0000:00:1d.0: setting latency timer to 64
[   10.359228] ehci_hcd 0000:00:1d.0: EHCI Host Controller
[   10.364792] ehci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2
[   10.372598] ehci_hcd 0000:00:1d.0: debug port 2
[   10.381242] ehci_hcd 0000:00:1d.0: cache line size of 64 is not supported
[   10.388365] ehci_hcd 0000:00:1d.0: irq 23, io mem 0xf2629000
[   10.398253] ehci_hcd 0000:00:1d.0: USB 2.0 started, EHCI 1.00
[   10.404341] hub 2-0:1.0: USB hub found
[   10.408265] hub 2-0:1.0: 3 ports detected
[   10.412572] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[   10.419084] uhci_hcd: USB Universal Host Controller Interface driver
[   10.425809] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
[   10.437457] serio: i8042 KBD port at 0x60,0x64 irq 1
[   10.442617] serio: i8042 AUX port at 0x60,0x64 irq 12
[   10.448053] mousedev: PS/2 mouse device common for all mice
[   10.454011] rtc_cmos 00:07: RTC can wake from S4
[   10.458985] rtc_cmos 00:07: rtc core: registered rtc_cmos as rtc0
[   10.465431] rtc0: alarms up to one month, y3k, 114 bytes nvram
[   10.471652] device-mapper: uevent: version 1.0.3
[   10.476527] device-mapper: ioctl: 4.20.0-ioctl (2011-02-02) initialised: dm-devel@redhat.com
[   10.485362] cpuidle: using governor ladder
[   10.489713] cpuidle: using governor menu
[   10.493877] EFI Variables Facility v0.08 2004-May-17
[   10.499399] TCP cubic registered
[   10.500937] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input3
[   10.511916] NET: Registered protocol family 10
[   10.516968] NET: Registered protocol family 17
[   10.521616] Registering the dns_resolver key type
[   10.526656] PM: Hibernation image not present or could not be loaded.
[   10.533382] registered taskstats version 1
[   10.549142]   Magic number: 12:868:981
[   10.553141] rtc_cmos 00:07: setting system clock to 2012-05-05 10:58:27 UTC (1336215507)
[   10.570385] BIOS EDD facility v0.16 2004-Jun-25, 0 devices found
[   10.576671] EDD information not available.
[   10.581469] Freeing unused kernel memory: 988k freed
[   10.586768] Write protecting the kernel read-only data: 12288k
[   10.598176] Freeing unused kernel memory: 2032k freed
[   10.604193] Freeing unused kernel memory: 1388k freed
Loading, please wait...
[   10.630269] usb 1-1: new high speed USB device number 2 using ehci_hcd
[   10.658925] udevd[87]: starting version 173
[   10.751323] e1000e: Intel(R) PRO/1000 Network Driver - 1.3.10-k2
[   10.757621] e1000e: Copyright(c) 1999 - 2011 Intel Corporation.
[   10.763939] xen: registering gsi 20 triggering 0 polarity 1
[   10.769826] xen: --> pirq=20 -> irq=20 (gsi=20)
[   10.774724] e1000e 0000:00:19.0: PCI INT A -> GSI 20 (level, low) -> IRQ 20
[   10.782776] hub 1-1:1.0: USB hub found
[   10.786826] hub 1-1:1.0: 6 ports detected
[   10.799416] sdhci: Secure Digital Host Controller Interface driver
[   10.805869] sdhci: Copyright(c) Pierre Ossman
[   10.810566] e1000e 0000:00:19.0: setting latency timer to 64
[   10.819028] xen: registering gsi 18 triggering 0 polarity 1
[   10.824839] xen_map_pirq_gsi: returning irq 18 for gsi 18
[   10.830559] xen: --> pirq=18 -> irq=18 (gsi=18)
[   10.835403] Already setup the GSI :18
[   10.839295] xhci_hcd 0000:0e:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
[   10.856774] sdhci-pci 0000:0d:00.0: SDHCI controller found [1180:e822] (rev 7)
[   10.864484] xen: registering gsi 16 triggering 0 polarity 1
[   10.870283] xen_map_pirq_gsi: returning irq 16 for gsi 16
[   10.875992] xen: --> pirq=16 -> irq=16 (gsi=16)
[   10.880837] Already setup the GSI :16
[   10.884694] sdhci-pci 0000:0d:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[   10.892584] xhci_hcd 0000:0e:00.0: setting latency timer to 64
[   10.898688] xhci_hcd 0000:0e:00.0: xHCI Host Controller
[   10.904739] xhci_hcd 0000:0e:00.0: new USB bus registered, assigned bus number 3
[   10.912556] usb 2-1: new high speed USB device number 2 using ehci_hcd
[   10.919757] sdhci-pci 0000:0d:00.0: setting latency timer to 64
[   10.926126] mmc0: no vmmc regulator found
[   10.930858] Registered led device: mmc0::
[   10.935570] xhci_hcd 0000:0e:00.0: irq 18, io mem 0xf1400000
[   10.942285] mmc0: SDHCI controller on PCI [0000:0d:00.0] using DMA
[   10.949534] xHCI xhci_add_endpoint called for root hub
[   10.954947] xHCI xhci_check_bandwidth called for root hub
[   10.960712] hub 3-0:1.0: USB hub found
[   10.964707] hub 3-0:1.0: 2 ports detected
[   10.969023] xhci_hcd 0000:0e:00.0: xHCI Host Controller
[   10.974551] xhci_hcd 0000:0e:00.0: new USB bus registered, assigned bus number 4
[   10.982573] xHCI xhci_add_endpoint called for root hub
[   10.987941] xHCI xhci_check_bandwidth called for root hub
[   10.993684] hub 4-0:1.0: USB hub found
[   10.997659] hub 4-0:1.0: 2 ports detected
[   11.048090] e1000e 0000:00:19.0: eth0: (PCI Express:2.5GT/s:Width x1) f0:de:f1:5b:b3:7d
[   11.056405] e1000e 0000:00:19.0: eth0: Intel(R) PRO/1000 Network Connection
[   11.062722] hub 2-1:1.0: USB hub found
[   11.062817] hub 2-1:1.0: 8 ports detected
[   11.072045] e1000e 0000:00:19.0: eth0: MAC: 10, PHY: 11, PBA No: 1000FF-0FF
[   11.079333] ahci 0000:00:1f.2: version 3.0
[   11.083677] xen: registering gsi 19 triggering 0 polarity 1
[   11.089507] xen_map_pirq_gsi: returning irq 19 for gsi 19
[   11.095188] xen: --> pirq=19 -> irq=19 (gsi=19)
[   11.099994] Already setup the GSI :19
[   11.103863] ahci 0000:00:1f.2: PCI INT B -> GSI 19 (level, low) -> IRQ 19
[   11.111276] ahci: SSS flag set, parallel bus scan disabled
[   11.126336] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x13 impl SATA mode
[   11.134890] ahci 0000:00:1f.2: flags: 64bit ncq sntf ilck stag pm led clo pio slum part ems sxs apst 
[   11.144611] ahci 0000:00:1f.2: setting latency timer to 64
[   11.154512] usb 1-1.3: new full speed USB device number 3 using ehci_hcd
[   11.166785] scsi0 : ahci
[   11.169558] scsi1 : ahci
[   11.172288] scsi2 : ahci
[   11.175034] scsi3 : ahci
[   11.177750] scsi4 : ahci
[   11.180472] scsi5 : ahci
[   11.183887] ata1: SATA max UDMA/133 abar m2048@0xf2628000 port 0xf2628100 irq 304
[   11.191674] ata2: SATA max UDMA/133 abar m2048@0xf2628000 port 0xf2628180 irq 304
[   11.199553] ata3: DUMMY
[   11.202178] ata4: DUMMY
[   11.204800] ata5: SATA max UDMA/133 abar m2048@0xf2628000 port 0xf2628300 irq 304
[   11.212700] ata6: DUMMY
[   11.322660] usb 1-1.4: new full speed USB device number 4 using ehci_hcd
[   11.534314] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   11.543407] ata1.00: ACPI cmd ef/02:00:00:00:00:a0 (SET FEATURES) succeeded
[   11.550690] ata1.00: ACPI cmd f5/00:00:00:00:00:a0 (SECURITY FREEZE LOCK) filtered out
[   11.559040] ata1.00: ACPI cmd ef/10:03:00:00:00:a0 (SET FEATURES) filtered out
[   11.569166] ata1.00: ATA-8: ST320LT000-9VL142, 0003LVM1, max UDMA/133
[   11.575916] ata1.00: 625142448 sectors, multi 16: LBA48 NCQ (depth 31/32)
[   11.586019] ata1.00: ACPI cmd ef/02:00:00:00:00:a0 (SET FEATURES) succeeded
[   11.593314] ata1.00: ACPI cmd f5/00:00:00:00:00:a0 (SECURITY FREEZE LOCK) filtered out
[   11.601662] ata1.00: ACPI cmd ef/10:03:00:00:00:a0 (SET FEATURES) filtered out
[   11.611800] ata1.00: configured for UDMA/133
[   11.616420] scsi 0:0:0:0: Direct-Access     ATA      ST320LT000-9VL14 0003 PQ: 0 ANSI: 5
[   11.625048] sd 0:0:0:0: Attached scsi generic sg0 type 0
[   11.625053] sd 0:0:0:0: [sda] 625142448 512-byte logical blocks: (320 GB/298 GiB)
[   11.625108] sd 0:0:0:0: [sda] Write Protect is off
[   11.625110] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[   11.625133] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   11.671973]  sda: sda1 sda2 sda3 sda4 < sda5 sda6 sda7 >
[   11.677867] sd 0:0:0:0: [sda] Attached SCSI disk
[   11.978299] ata2: SATA link down (SStatus 0 SControl 300)
[   12.306299] ata5: SATA link down (SStatus 0 SControl 300)
[   12.990703] EXT4-fs (sda7): mounted filesystem with ordered data mode. Opts: (null)
fsck from util-linux 2.19.1
fsck from util-linux 2.19.1
/dev/sda7: clean, 893789/14925824 files, 29160442/59677440 blocks (check in 4 mounts)
/dev/sda5: clean, 274/62248 files, 76545/248832 blocks (check in 4 mounts)
Skipping profile in /etc/apparmor.d/disable: usr.bin.firefox
 * Starting mDNS/DNS-SD daemon                                           [ OK ]
 * Starting AppArmor profiles       (XEN) Cannot bind IRQ19 to dom0. In use by 'ns16550'.
                                                                         [ OK ] 
 * Stopping Failsafe Boot Delay                                          [ OK ]
 * Stopping System V initialisation compatibility                        [ OK ]
 * Starting System V runlevel compatibility                              [ OK ]
 * Stopping automatic crash report generation                            [fail]
 * Starting LightDM Display Manager                                      [ OK ]
 * Starting save kernel messages                                         [ OK ]
 * Starting KVM                                                          [ OK ]
 * Starting anac(h)ronistic cron                                         [ OK ]
 * Starting ACPI daemon                                                  [ OK ]
 * Starting regular background program processing daemon                 [ OK ]
 * Starting deferred execution scheduler                                 [ OK ]
 * Starting CPU interrupts balancing daemon                              [ OK ]
 * Starting CPU interrupts balancing daemon                              [fail]
Starting oxenstored... * Stopping save kernel messages                   [ OK ]
[   31.653441] XENBUS: Unable to read cpu state
[   31.658014] XENBUS: Unable to read cpu state
[   31.662681] X
ENBUS: Unable toSetting domain 0 name... read cpu state

[   31.669990] XENBUS: Unable to read cpu state
Starting xenconsoled...
speech-dispatcher disabled; edit /etc/default/speech-dispatcher
Checking for running unattended-upgrades: 
 * Starting VirtualBox kernel modules                                    [ OK ] 
Starting domain watchdog daemon:  * xenwatchdogd startup

 * Starting bluetooth                                                    [ OK ] 
 * PulseAudio configured for per-user sessions
saned disabled; edit /etc/default/saned

 * Checking battery state...                                             [ OK ] 
(XEN) Cannot bind IRQ19 to dom0. In use by 'ns16550'.
(XEN) HVM1: HVM Loader
(XEN) HVM1: Detected Xen v4.2-unstable
(XEN) HVM1: Xenbus rings @0xfeffc000, event channel 3
(XEN) HVM1: System requested ROMBIOS
(XEN) HVM1: CPU speed is 2691 MHz
(XEN) irq.c:270: Dom1 PCI link 0 changed 0 -> 5
(XEN) HVM1: PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:270: Dom1 PCI link 1 changed 0 -> 10
(XEN) HVM1: PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:270: Dom1 PCI link 2 changed 0 -> 11
(XEN) HVM1: PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:270: Dom1 PCI link 3 changed 0 -> 5
(XEN) HVM1: PCI-ISA link 3 routed to IRQ5
(XEN) HVM1: pci dev 01:2 INTD->IRQ5
(XEN) HVM1: pci dev 01:3 INTA->IRQ10
(XEN) HVM1: pci dev 03:0 INTA->IRQ5
(XEN) HVM1: pci dev 02:0 bar 10 size 01000000: f0000008
(XEN) HVM1: pci dev 03:0 bar 14 size 01000000: f1000008
(XEN) HVM1: pci dev 03:0 bar 10 size 00000100: 0000c001
(XEN) HVM1: pci dev 01:2 bar 20 size 00000020: 0000c101
(XEN) HVM1: pci dev 01:1 bar 20 size 00000010: 0000c121
(XEN) HVM1: Multiprocessor initialisation:
(XEN) HVM1:  - CPU0 ... 36-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM1: Testing HVM environment:
(XEN) HVM1:  - REP INSB across page boundaries ... passed
(XEN) HVM1:  - GS base MSRs and SWAPGS ... passed
(XEN) HVM1: Passed 2 of 2 tests
(XEN) HVM1: Writing SMBIOS tables ...
(XEN) HVM1: Loading ROMBIOS ...
(XEN) HVM1: 12636 bytes of ROMBIOS high-memory extensions:
(XEN) HVM1:   Relocating to 0xfc001000-0xfc00415c ... done
(XEN) HVM1: Creating MP tables ...
(XEN) HVM1: Loading Standard VGABIOS ...
(XEN) HVM1: Option ROMs:
(XEN) HVM1:  c0000-c9fff: VGA BIOS
(XEN) HVM1: Loading ACPI ...
(XEN) HVM1: vm86 TSS at fc010280
(XEN) HVM1: BIOS map:
(XEN) HVM1:  f0000-fffff: Main BIOS
(XEN) HVM1: E820 table:
(XEN) HVM1:  [00]: 00000000:00000000 - 00000000:0009e000: RAM
(XEN) HVM1:  [01]: 00000000:0009e000 - 00000000:000a0000: RESERVED
(XEN) HVM1:  HOLE: 00000000:000a0000 - 00000000:000e0000
(XEN) HVM1:  [02]: 00000000:000e0000 - 00000000:00100000: RESERVED
(XEN) HVM1:  [03]: 00000000:00100000 - 00000000:1f000000: RAM
(XEN) HVM1:  HOLE: 00000000:1f000000 - 00000000:fc000000
(XEN) HVM1:  [04]: 00000000:fc000000 - 00000001:00000000: RESERVED
(XEN) HVM1: Invoking ROMBIOS ...
(XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) stdvga.c:147:d1 entering stdvga and caching modes
(XEN) HVM1: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert Exp $
(XEN) HVM1: VBE Bios $Id: vbe.c,v 1.60 2008/03/02 07:47:21 vruppert Exp $
(XEN) HVM1: Bochs BIOS - build: 06/23/99
(XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) HVM1: Options: apmbios pcibios eltorito PMM 
(XEN) HVM1: 
(XEN) HVM1: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
(XEN) HVM1: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (10000 MBytes)
(XEN) HVM1: IDE time out
(XEN) HVM1: 
(XEN) HVM1: 
(XEN) HVM1: 
(XEN) HVM1: Press F12 for boot menu.
(XEN) HVM1: 
(XEN) HVM1: Booting from Hard Disk...
(XEN) HVM1: Booting from 0000:7c00
(XEN) HVM1: int13_harddisk: function 41, unmapped device for ELDL=81
(XEN) HVM1: int13_harddisk: function 08, unmapped device for ELDL=81
(XEN) HVM1: *** int 15h function AX=00c0, BX=0000 not yet supported!
(XEN) HVM2: HVM Loader
(XEN) HVM2: Detected Xen v4.2-unstable
(XEN) HVM2: Xenbus rings @0xfeffc000, event channel 4
(XEN) HVM2: System requested ROMBIOS
(XEN) HVM2: CPU speed is 2691 MHz
(XEN) irq.c:270: Dom2 PCI link 0 changed 0 -> 5
(XEN) HVM2: PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:270: Dom2 PCI link 1 changed 0 -> 10
(XEN) HVM2: PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:270: Dom2 PCI link 2 changed 0 -> 11
(XEN) HVM2: PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:270: Dom2 PCI link 3 changed 0 -> 5
(XEN) HVM2: PCI-ISA link 3 routed to IRQ5
(XEN) HVM2: pci dev 01:2 INTD->IRQ5
(XEN) HVM2: pci dev 01:3 INTA->IRQ10
(XEN) HVM2: pci dev 03:0 INTA->IRQ5
(XEN) HVM2: pci dev 04:0 INTA->IRQ5
(XEN) HVM2: pci dev 02:0 bar 10 size 01000000: f0000008
(XEN) HVM2: pci dev 03:0 bar 14 size 01000000: f1000008
(XEN) HVM2: pci dev 03:0 bar 10 size 00000100: 0000c001
(XEN) HVM2: pci dev 04:0 bar 10 size 00000100: 0000c101
(XEN) HVM2: pci dev 04:0 bar 14 size 00000100: f2000000
(XEN) HVM2: pci dev 01:2 bar 20 size 00000020: 0000c201
(XEN) HVM2: pci dev 01:1 bar 20 size 00000010: 0000c221
(XEN) HVM2: Multiprocessor initialisation:
(XEN) HVM2:  - CPU0 ... 36-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM2:  - CPU1 ... 36-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done.
(XEN) HVM2: Testing HVM environment:
(XEN) HVM2:  - REP INSB across page boundaries ... passed
(XEN) HVM2:  - GS base MSRs and SWAPGS ... passed
(XEN) HVM2: Passed 2 of 2 tests
(XEN) HVM2: Writing SMBIOS tables ...
(XEN) HVM2: Loading ROMBIOS ...
(XEN) HVM2: 12636 bytes of ROMBIOS high-memory extensions:
(XEN) HVM2:   Relocating to 0xfc001000-0xfc00415c ... done
(XEN) HVM2: Creating MP tables ...
(XEN) HVM2: Loading Standard VGABIOS ...
(XEN) HVM2: Loading PCI Option ROM ...
(XEN) HVM2:  - Manufacturer: http://ipxe.org
(XEN) HVM2:  - Product name: iPXE
(XEN) HVM2: Option ROMs:
(XEN) HVM2:  c0000-c9fff: VGA BIOS
(XEN) HVM2:  ca000-d9fff: Etherboot ROM
(XEN) HVM2: Loading ACPI ...
(XEN) HVM2: vm86 TSS at fc010280
(XEN) HVM2: BIOS map:
(XEN) HVM2:  f0000-fffff: Main BIOS
(XEN) HVM2: E820 table:
(XEN) HVM2:  [00]: 00000000:00000000 - 00000000:0009e000: RAM
(XEN) HVM2:  [01]: 00000000:0009e000 - 00000000:000a0000: RESERVED
(XEN) HVM2:  HOLE: 00000000:000a0000 - 00000000:000e0000
(XEN) HVM2:  [02]: 00000000:000e0000 - 00000000:00100000: RESERVED
(XEN) HVM2:  [03]: 00000000:00100000 - 00000000:3f000000: RAM
(XEN) HVM2:  HOLE: 00000000:3f000000 - 00000000:fc000000
(XEN) HVM2:  [04]: 00000000:fc000000 - 00000001:00000000: RESERVED
(XEN) HVM2: Invoking ROMBIOS ...
(XEN) HVM2: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) stdvga.c:147:d2 entering stdvga and caching modes
(XEN) HVM2: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert Exp $
(XEN) HVM2: VBE Bios $Id: vbe.c,v 1.60 2008/03/02 07:47:21 vruppert Exp $
(XEN) HVM2: Bochs BIOS - build: 06/23/99
(XEN) HVM2: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) HVM2: Options: apmbios pcibios eltorito PMM 
(XEN) HVM2: 
(XEN) HVM2: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
(XEN) HVM2: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (20000 MBytes)
(XEN) HVM2: IDE time out
(XEN) HVM2: 
(XEN) HVM2: 
(XEN) HVM2: 
(XEN) HVM2: Press F12 for boot menu.
(XEN) HVM2: 
(XEN) HVM2: Booting from CD-Rom...
(XEN) HVM2: CDROM boot failure code : 0002
(XEN) HVM2: Boot from CD-Rom failed: could not read the boot disk
(XEN) HVM2: 
(XEN) HVM2: Booting from Hard Disk...
(XEN) HVM2: Booting from 0000:7c00
(XEN) HVM1: int13_harddisk: function 15, unmapped device for ELDL=81
(XEN) HVM1: *** int 15h function AX=ec00, BX=0002 not yet supported!
(XEN) HVM1: KBD: unsupported int 16h function 03
(XEN) HVM1: int13_harddisk: function 15, unmapped device for ELDL=81
(XEN) HVM1: int13_harddisk: function 02, unmapped device for ELDL=81
(XEN) HVM1: int13_harddisk: function 41, unmapped device for ELDL=81
(XEN) irq.c:270: Dom1 PCI link 0 changed 5 -> 0
(XEN) irq.c:270: Dom1 PCI link 1 changed 10 -> 0
(XEN) irq.c:270: Dom1 PCI link 2 changed 11 -> 0
(XEN) irq.c:270: Dom1 PCI link 3 changed 5 -> 0
(XEN) irq.c:350: Dom1 callback via changed to PCI INTx Dev 0x03 IntA
(XEN) irq.c:270: Dom2 PCI link 0 changed 5 -> 0
(XEN) irq.c:270: Dom2 PCI link 1 changed 10 -> 0
(XEN) irq.c:270: Dom2 PCI link 2 changed 11 -> 0
(XEN) irq.c:270: Dom2 PCI link 3 changed 5 -> 0
(XEN) *** IRQ BUG found ***
(XEN) CPU0 -Testing vector 236 from bitmap 41,49,64,72,80,82,84,88,96,98,104,120,123,136,145,152,160,168,192,200-201,223
(XEN) Guest interrupt information:
(XEN)    IRQ:   0 affinity:01 vec:f0 type=IO-APIC-edge    status=00000000 mapped, unbound
(XEN)    IRQ:   1 affinity:02 vec:7b type=IO-APIC-edge    status=00000030 in-flight=0 domain-list=0:  1(----),
(XEN)    IRQ:   2 affinity:ff vec:e2 type=XT-PIC          status=00000000 mapped, unbound
(XEN)    IRQ:   3 affinity:01 vec:40 type=IO-APIC-edge    status=00000002 mapped, unbound
(XEN)    IRQ:   4 affinity:01 vec:48 type=IO-APIC-edge    status=00000002 mapped, unbound
(XEN)    IRQ:   5 affinity:01 vec:50 type=IO-APIC-edge    status=00000002 mapped, unbound
(XEN)    IRQ:   6 affinity:01 vec:58 type=IO-APIC-edge    status=00000002 mapped, unbound
(XEN)    IRQ:   7 affinity:01 vec:60 type=IO-APIC-edge    status=00000002 mapped, unbound
(XEN)    IRQ:   8 affinity:08 vec:29 type=IO-APIC-edge    status=00000030 in-flight=0 domain-list=0:  8(----),
(XEN)    IRQ:   9 affinity:01 vec:91 type=IO-APIC-level   status=00000030 in-flight=0 domain-list=0:  9(----),
(XEN)    IRQ:  10 affinity:01 vec:78 type=IO-APIC-edge    status=00000002 mapped, unbound
(XEN)    IRQ:  11 affinity:01 vec:88 type=IO-APIC-edge    status=00000002 mapped, unbound



[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-05-05  0:21         ` AP
  2012-05-05 11:04           ` Andrew Cooper
@ 2012-05-07  8:10           ` Jan Beulich
  2012-05-07 11:50             ` Andrew Cooper
  1 sibling, 1 reply; 24+ messages in thread
From: Jan Beulich @ 2012-05-07  8:10 UTC (permalink / raw)
  To: Andrew Cooper, AP
  Cc: Keir (Xen.org), Daniel De Graaf, xen-devel, Ian Jackson, Ian Campbell

>>> On 05.05.12 at 02:21, AP <apxeng@gmail.com> wrote:
> (XEN) *** IRQ BUG found ***
> (XEN) CPU0 -Testing vector 236 from bitmap

236 = 0xec = FIRST_LEGACY_VECTOR + 0x0c, i.e. an IRQ12 coming
in through the 8259A. Something fundamentally fishy must be going
on here, and I would suppose the code in question shouldn't even be
reached for legacy vectors.

Furthermore, calling dump_irqs() from the debugging code with
desc->lock still held makes it impossible to get full output, as that
function wants to lock all initialized IRQ descriptors.

Jan

> 37,41,49,51,64,72,80,88,96,104,120,136,145,152,158,160,168,175,182,192,200,211
> (XEN) Guest interrupt information:
> (XEN)    IRQ:   0 affinity:01 vec:f0 type=IO-APIC-edge    status=00000000
> mapped, unbound
> (XEN)    IRQ:   1 affinity:01 vec:d3 type=IO-APIC-edge    status=00000030
> in-flight=0 domain-list=0:  1(-S--),
> (XEN)    IRQ:   2 affinity:ff vec:e2 type=XT-PIC          status=00000000
> mapped, unbound
> (XEN)    IRQ:   3 affinity:01 vec:40 type=IO-APIC-edge    status=00000002
> mapped, unbound
> (XEN)    IRQ:   4 affinity:01 vec:48 type=IO-APIC-edge    status=00000002
> mapped, unbound
> (XEN)    IRQ:   5 affinity:01 vec:50 type=IO-APIC-edge    status=00000002
> mapped, unbound
> (XEN)    IRQ:   6 affinity:01 vec:58 type=IO-APIC-edge    status=00000002
> mapped, unbound
> (XEN)    IRQ:   7 affinity:01 vec:60 type=IO-APIC-edge    status=00000002
> mapped, unbound
> (XEN)    IRQ:   8 affinity:08 vec:29 type=IO-APIC-edge    status=00000030
> in-flight=0 domain-list=0:  8(-S--),
> (XEN)    IRQ:   9 affinity:02 vec:25 type=IO-APIC-level   status=00000030
> in-flight=0 domain-list=0:  9(-S--),
> (XEN)    IRQ:  10 affinity:01 vec:78 type=IO-APIC-edge    status=00000002
> mapped, unbound
> (XEN)    IRQ:  11 affinity:01 vec:88 type=IO-APIC-edge    status=00000002
> mapped, unbound
> [ 5129.737147] [drm:i915_hangcheck_ring_idle] *ERROR* Hangcheck timer
> elapsed... blt ring idle [waiting on 1800652, at 1800652], missed IRQ?
> 
> Let me know if you need any more info.
> Thanks,
> AP

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-05-07  8:10           ` Jan Beulich
@ 2012-05-07 11:50             ` Andrew Cooper
  2012-05-07 13:34               ` Jan Beulich
  0 siblings, 1 reply; 24+ messages in thread
From: Andrew Cooper @ 2012-05-07 11:50 UTC (permalink / raw)
  To: Jan Beulich, AP
  Cc: Keir (Xen.org), Daniel De Graaf, xen-devel, Ian Jackson, Ian Campbell

On 07/05/2012 09:10, Jan Beulich wrote:
>>>> On 05.05.12 at 02:21, AP <apxeng@gmail.com> wrote:
>> (XEN) *** IRQ BUG found ***
>> (XEN) CPU0 -Testing vector 236 from bitmap
> 236 = 0xec = FIRST_LEGACY_VECTOR + 0x0c, i.e. an IRQ12 coming
> in through the 8259A. Something fundamentally fishy must be going
> on here, and I would suppose the code in question shouldn't even be
> reached for legacy vectors.
>
> Furthermore, calling dump_irqs() from the debugging code with
> desc->lock still held makes it impossible to get full output, as that
> function wants to lock all initialized IRQ descriptors.
>
> Jan

Yes - it has been vector 236 on each of the 3 reported failures from AP,
and I believe it was also vector 236 in the one case I managed to
reproduce the issue.

However, once we have set up the IO-APIC, the 8259A should not be used
any more.  The boot dmeg shows that io_ack_method is indeed "old" (which
was going to be my first suggestion), and that EOI Broadcast Suppression
is enabled, which I have already identified as a source of problems for
some customers.  As a 'fix', I provided the ability for
"io_ack_method=new" to prevent EOI Broadcast Suppression being enabled. 
This was upstreamed in c/s 24870:9bf3ec036bef, but apparently has not
completely fixed the customer problems - just made it substantially more
rare.

AP: Can you manually invoke the 'i' debug key and provide that - it will
help to see how Xen is setting up the IO-APIC(s) on your system.

~Andrew

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-05-07 11:50             ` Andrew Cooper
@ 2012-05-07 13:34               ` Jan Beulich
  2012-05-07 14:41                 ` Andrew Cooper
  2012-05-07 18:29                 ` AP
  0 siblings, 2 replies; 24+ messages in thread
From: Jan Beulich @ 2012-05-07 13:34 UTC (permalink / raw)
  To: Andrew Cooper, AP
  Cc: Keir (Xen.org), Daniel De Graaf, xen-devel, Ian Jackson, Ian Campbell

>>> On 07.05.12 at 13:50, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> On 07/05/2012 09:10, Jan Beulich wrote:
>>>>> On 05.05.12 at 02:21, AP <apxeng@gmail.com> wrote:
>>> (XEN) *** IRQ BUG found ***
>>> (XEN) CPU0 -Testing vector 236 from bitmap
>> 236 = 0xec = FIRST_LEGACY_VECTOR + 0x0c, i.e. an IRQ12 coming
>> in through the 8259A. Something fundamentally fishy must be going
>> on here, and I would suppose the code in question shouldn't even be
>> reached for legacy vectors.
>>
>> Furthermore, calling dump_irqs() from the debugging code with
>> desc->lock still held makes it impossible to get full output, as that
>> function wants to lock all initialized IRQ descriptors.
> 
> Yes - it has been vector 236 on each of the 3 reported failures from AP,
> and I believe it was also vector 236 in the one case I managed to
> reproduce the issue.
> 
> However, once we have set up the IO-APIC, the 8259A should not be used
> any more.  The boot dmeg shows that io_ack_method is indeed "old" (which
> was going to be my first suggestion), and that EOI Broadcast Suppression
> is enabled, which I have already identified as a source of problems for
> some customers.  As a 'fix', I provided the ability for
> "io_ack_method=new" to prevent EOI Broadcast Suppression being enabled. 
> This was upstreamed in c/s 24870:9bf3ec036bef, but apparently has not
> completely fixed the customer problems - just made it substantially more
> rare.
> 
> AP: Can you manually invoke the 'i' debug key and provide that - it will
> help to see how Xen is setting up the IO-APIC(s) on your system.

Seeing the 'z' output might also be helpful, especially to see whether
any of the IO-APICs' RTEs is an ExtINT one.

Further, checking that no 8259A IRQ got (or was left) enabled for
some reason might be useful as well (cached_irq_mask plus the raw
port 0x21 and 0xA1 values).

In any case the debugging code's locking should be fixed.

Jan

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-05-07 13:34               ` Jan Beulich
@ 2012-05-07 14:41                 ` Andrew Cooper
  2012-05-07 14:50                   ` Jan Beulich
  2012-05-07 14:54                   ` Jan Beulich
  2012-05-07 18:29                 ` AP
  1 sibling, 2 replies; 24+ messages in thread
From: Andrew Cooper @ 2012-05-07 14:41 UTC (permalink / raw)
  To: Jan Beulich
  Cc: xen-devel, Keir (Xen.org), Ian Campbell, Ian Jackson, Daniel De Graaf

On 07/05/2012 14:34, Jan Beulich wrote:
>>>> On 07.05.12 at 13:50, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>> On 07/05/2012 09:10, Jan Beulich wrote:
>>>>>> On 05.05.12 at 02:21, AP <apxeng@gmail.com> wrote:
>>>> (XEN) *** IRQ BUG found ***
>>>> (XEN) CPU0 -Testing vector 236 from bitmap
>>> 236 = 0xec = FIRST_LEGACY_VECTOR + 0x0c, i.e. an IRQ12 coming
>>> in through the 8259A. Something fundamentally fishy must be going
>>> on here, and I would suppose the code in question shouldn't even be
>>> reached for legacy vectors.
>>>
>>> Furthermore, calling dump_irqs() from the debugging code with
>>> desc->lock still held makes it impossible to get full output, as that
>>> function wants to lock all initialized IRQ descriptors.
>> Yes - it has been vector 236 on each of the 3 reported failures from AP,
>> and I believe it was also vector 236 in the one case I managed to
>> reproduce the issue.
>>
>> However, once we have set up the IO-APIC, the 8259A should not be used
>> any more.  The boot dmeg shows that io_ack_method is indeed "old" (which
>> was going to be my first suggestion), and that EOI Broadcast Suppression
>> is enabled, which I have already identified as a source of problems for
>> some customers.  As a 'fix', I provided the ability for
>> "io_ack_method=new" to prevent EOI Broadcast Suppression being enabled. 
>> This was upstreamed in c/s 24870:9bf3ec036bef, but apparently has not
>> completely fixed the customer problems - just made it substantially more
>> rare.
>>
>> AP: Can you manually invoke the 'i' debug key and provide that - it will
>> help to see how Xen is setting up the IO-APIC(s) on your system.
> Seeing the 'z' output might also be helpful, especially to see whether
> any of the IO-APICs' RTEs is an ExtINT one.
>
> Further, checking that no 8259A IRQ got (or was left) enabled for
> some reason might be useful as well (cached_irq_mask plus the raw
> port 0x21 and 0xA1 values).
>
> In any case the debugging code's locking should be fixed.
>
> Jan
>

It appears we have two functions to dump the IO-APIC state:
__print_IO_APIC() which gets called on boot and from 'z', and
dump_ioapic_irq_info() which gets called from the end of 'i'.  These
should probably be consolidated somehow.

As for the debugging, perhaps change the call to dump_irqs() with a call
to dump_ioapic_irq_info() instead.

Given that the legacy vectors cant migrate, is it wise including them in
the loop in irq_move_cleanup_interrupt()?  In fact, is it wise including
any vector above LAST_DYNAMIC_VECTOR?

~Andrew

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-05-07 14:41                 ` Andrew Cooper
@ 2012-05-07 14:50                   ` Jan Beulich
  2012-05-07 15:40                     ` Andrew Cooper
  2012-05-07 14:54                   ` Jan Beulich
  1 sibling, 1 reply; 24+ messages in thread
From: Jan Beulich @ 2012-05-07 14:50 UTC (permalink / raw)
  To: Andrew Cooper
  Cc: xen-devel, Keir (Xen.org), Ian Campbell, Ian Jackson, Daniel De Graaf

>>> On 07.05.12 at 16:41, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> Given that the legacy vectors cant migrate, is it wise including them in
> the loop in irq_move_cleanup_interrupt()?  In fact, is it wise including
> any vector above LAST_DYNAMIC_VECTOR?

Likely not, but then again this is the final piece of moving an interrupt,
so there must have been something earlier that incorrectly initiated a
move. In other words, rather than fixing the loop here, we should
make sure execution can't even make it there for legacy vectors.

And of course this is irrespective of the fact that no legacy interrupt
should occur in the first place, unless this is a very strange system.

Jan

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-05-07 14:41                 ` Andrew Cooper
  2012-05-07 14:50                   ` Jan Beulich
@ 2012-05-07 14:54                   ` Jan Beulich
  2012-05-07 15:51                     ` Andrew Cooper
  1 sibling, 1 reply; 24+ messages in thread
From: Jan Beulich @ 2012-05-07 14:54 UTC (permalink / raw)
  To: Andrew Cooper
  Cc: xen-devel, Keir (Xen.org), Ian Campbell, Ian Jackson, Daniel De Graaf

>>> On 07.05.12 at 16:41, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> It appears we have two functions to dump the IO-APIC state:
> __print_IO_APIC() which gets called on boot and from 'z', and
> dump_ioapic_irq_info() which gets called from the end of 'i'.  These
> should probably be consolidated somehow.

Rather not - 'z' provides information on the IO-APIC that isn't
directly related to specific interrupts, while 'i' (when it comes to
the IO-APIC) is exclusively interested in the RTEs. Unless
dump_ioapic_irq_info() is _fully_ redundant with 'z' (didn't check
in detail yet), in which case I'd vote for removing this function.

Jan

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-05-07 14:50                   ` Jan Beulich
@ 2012-05-07 15:40                     ` Andrew Cooper
  2012-05-07 15:43                       ` Jan Beulich
  0 siblings, 1 reply; 24+ messages in thread
From: Andrew Cooper @ 2012-05-07 15:40 UTC (permalink / raw)
  To: Jan Beulich
  Cc: xen-devel, Keir (Xen.org), Ian Campbell, Ian Jackson, Daniel De Graaf

On 07/05/2012 15:50, Jan Beulich wrote:
>>>> On 07.05.12 at 16:41, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>> Given that the legacy vectors cant migrate, is it wise including them in
>> the loop in irq_move_cleanup_interrupt()?  In fact, is it wise including
>> any vector above LAST_DYNAMIC_VECTOR?
> Likely not, but then again this is the final piece of moving an interrupt,
> so there must have been something earlier that incorrectly initiated a
> move. In other words, rather than fixing the loop here, we should
> make sure execution can't even make it there for legacy vectors.
>
> And of course this is irrespective of the fact that no legacy interrupt
> should occur in the first place, unless this is a very strange system.
>
> Jan
>

The only way to get to this point is if desc->arch.move_cleanup_count is
non 0, in which case, one of these functions:

hpet_msi_ack (hpet.c)
ack_edge_ioapic_irq (io_apci.c)
mask_and_ack_level_ioapic_irq (io_apic.c)
ack_nonmaskable_msi_irq (msi.c)
iommu_msi_mask (iommu_init.c)
dma_msi_mask (iommu.c)

has called irq_complete_move, after something has called
__assign_irq_vector() to move the irq to another CPU.

I would say something very fishy is going on - no desc used by any of
those functions should have a vector from the legacy region.

As for the loop, it is probably quite sensible to reduce that down to
LAST_DYNAMIC_VECTOR.  Leaving it at NR_VECTORS is just 32 wasted
iterations of the loop in interrupt context.

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-05-07 15:40                     ` Andrew Cooper
@ 2012-05-07 15:43                       ` Jan Beulich
  0 siblings, 0 replies; 24+ messages in thread
From: Jan Beulich @ 2012-05-07 15:43 UTC (permalink / raw)
  To: Andrew Cooper
  Cc: xen-devel, Keir (Xen.org), Ian Campbell, Ian Jackson, Daniel DeGraaf

>>> On 07.05.12 at 17:40, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> As for the loop, it is probably quite sensible to reduce that down to
> LAST_DYNAMIC_VECTOR.  Leaving it at NR_VECTORS is just 32 wasted
> iterations of the loop in interrupt context.

No, you can't leave there. You'd have to skip the legacy vectors, and
continue with the ones Xen itself may have in use.

Jan

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-05-07 14:54                   ` Jan Beulich
@ 2012-05-07 15:51                     ` Andrew Cooper
  0 siblings, 0 replies; 24+ messages in thread
From: Andrew Cooper @ 2012-05-07 15:51 UTC (permalink / raw)
  To: Jan Beulich
  Cc: xen-devel, Keir (Xen.org), Ian Campbell, Ian Jackson, Daniel De Graaf

On 07/05/2012 15:54, Jan Beulich wrote:
>>>> On 07.05.12 at 16:41, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>> It appears we have two functions to dump the IO-APIC state:
>> __print_IO_APIC() which gets called on boot and from 'z', and
>> dump_ioapic_irq_info() which gets called from the end of 'i'.  These
>> should probably be consolidated somehow.
> Rather not - 'z' provides information on the IO-APIC that isn't
> directly related to specific interrupts, while 'i' (when it comes to
> the IO-APIC) is exclusively interested in the RTEs. Unless
> dump_ioapic_irq_info() is _fully_ redundant with 'z' (didn't check
> in detail yet), in which case I'd vote for removing this function.
>
> Jan
>

dump_ioapic_irq_info() loops through nr_irqs_gsi and uses irq_2_pin to
work out which io-apic RTE to read and decode.

__print_IO_APIC() loop through nr_ioapics, then through each RTE and
decodes it.  At the end, it loops through nr_irqs_gsi and matches irqs
to ioapic:pin pairs.

So they are probably different enough to be worth keeping.

~Andrew

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-05-07 13:34               ` Jan Beulich
  2012-05-07 14:41                 ` Andrew Cooper
@ 2012-05-07 18:29                 ` AP
  2012-05-08  6:37                   ` Jan Beulich
  1 sibling, 1 reply; 24+ messages in thread
From: AP @ 2012-05-07 18:29 UTC (permalink / raw)
  To: Jan Beulich
  Cc: xen-devel, Keir (Xen.org),
	Ian Campbell, Andrew Cooper, Ian Jackson, Daniel De Graaf


[-- Attachment #1.1: Type: text/plain, Size: 10773 bytes --]

On Mon, May 7, 2012 at 1:34 PM, Jan Beulich <JBeulich@suse.com> wrote:
>
> >>> On 07.05.12 at 13:50, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> > On 07/05/2012 09:10, Jan Beulich wrote:
> >>>>> On 05.05.12 at 02:21, AP <apxeng@gmail.com> wrote:
> >>> (XEN) *** IRQ BUG found ***
> >>> (XEN) CPU0 -Testing vector 236 from bitmap
> >> 236 = 0xec = FIRST_LEGACY_VECTOR + 0x0c, i.e. an IRQ12 coming
> >> in through the 8259A. Something fundamentally fishy must be going
> >> on here, and I would suppose the code in question shouldn't even be
> >> reached for legacy vectors.
> >>
> >> Furthermore, calling dump_irqs() from the debugging code with
> >> desc->lock still held makes it impossible to get full output, as that
> >> function wants to lock all initialized IRQ descriptors.
> >
> > Yes - it has been vector 236 on each of the 3 reported failures from AP,
> > and I believe it was also vector 236 in the one case I managed to
> > reproduce the issue.
> >
> > However, once we have set up the IO-APIC, the 8259A should not be used
> > any more.  The boot dmeg shows that io_ack_method is indeed "old" (which
> > was going to be my first suggestion), and that EOI Broadcast Suppression
> > is enabled, which I have already identified as a source of problems for
> > some customers.  As a 'fix', I provided the ability for
> > "io_ack_method=new" to prevent EOI Broadcast Suppression being enabled.
> > This was upstreamed in c/s 24870:9bf3ec036bef, but apparently has not
> > completely fixed the customer problems - just made it substantially more
> > rare.
> >
> > AP: Can you manually invoke the 'i' debug key and provide that - it will
> > help to see how Xen is setting up the IO-APIC(s) on your system.

(XEN) Guest interrupt information:
(XEN)    IRQ:   0 affinity:01 vec:f0 type=IO-APIC-edge    status=00000000
mapped, unbound
(XEN)    IRQ:   1 affinity:02 vec:85 type=IO-APIC-edge    status=00000030
in-flight=0 domain-list=0:  1(----),
(XEN)    IRQ:   2 affinity:ff vec:e2 type=XT-PIC          status=00000000
mapped, unbound
(XEN)    IRQ:   3 affinity:01 vec:40 type=IO-APIC-edge    status=00000002
mapped, unbound
(XEN)    IRQ:   4 affinity:01 vec:48 type=IO-APIC-edge    status=00000002
mapped, unbound
(XEN)    IRQ:   5 affinity:01 vec:50 type=IO-APIC-edge    status=00000002
mapped, unbound
(XEN)    IRQ:   6 affinity:01 vec:58 type=IO-APIC-edge    status=00000002
mapped, unbound
(XEN)    IRQ:   7 affinity:01 vec:60 type=IO-APIC-edge    status=00000002
mapped, unbound
(XEN)    IRQ:   8 affinity:08 vec:29 type=IO-APIC-edge    status=00000030
in-flight=0 domain-list=0:  8(----),
(XEN)    IRQ:   9 affinity:02 vec:7f type=IO-APIC-level   status=00000010
in-flight=0 domain-list=0:  9(----),
(XEN)    IRQ:  10 affinity:01 vec:78 type=IO-APIC-edge    status=00000002
mapped, unbound
(XEN)    IRQ:  11 affinity:01 vec:88 type=IO-APIC-edge    status=00000002
mapped, unbound
(XEN)    IRQ:  12 affinity:08 vec:d4 type=IO-APIC-edge    status=00000030
in-flight=0 domain-list=0: 12(----),
(XEN)    IRQ:  13 affinity:0f vec:98 type=IO-APIC-edge    status=00000002
mapped, unbound
(XEN)    IRQ:  14 affinity:01 vec:a0 type=IO-APIC-edge    status=00000002
mapped, unbound
(XEN)    IRQ:  15 affinity:01 vec:a8 type=IO-APIC-edge    status=00000002
mapped, unbound
(XEN)    IRQ:  16 affinity:02 vec:a6 type=IO-APIC-level   status=00000030
in-flight=0 domain-list=0: 16(----),
(XEN)    IRQ:  17 affinity:0f vec:c0 type=IO-APIC-level   status=00000002
mapped, unbound
(XEN)    IRQ:  18 affinity:0f vec:c8 type=IO-APIC-level   status=00000002
mapped, unbound
(XEN)    IRQ:  19 affinity:0f vec:f1 type=IO-APIC-level   status=00000000
mapped, unbound
(XEN)    IRQ:  20 affinity:0f vec:61 type=IO-APIC-level   status=00000002
mapped, unbound
(XEN)    IRQ:  22 affinity:0f vec:32 type=IO-APIC-level   status=00000002
mapped, unbound
(XEN)    IRQ:  23 affinity:01 vec:ac type=IO-APIC-level   status=00000030
in-flight=0 domain-list=0: 23(----),
(XEN)    IRQ:  24 affinity:01 vec:28 type=DMA_MSI         status=00000000
mapped, unbound
(XEN)    IRQ:  25 affinity:01 vec:30 type=DMA_MSI         status=00000000
mapped, unbound
(XEN)    IRQ:  26 affinity:01 vec:31 type=PCI-MSI/-X      status=00000030
in-flight=0 domain-list=0:279(----),
(XEN)    IRQ:  27 affinity:01 vec:39 type=PCI-MSI/-X      status=00000030
in-flight=0 domain-list=0:278(----),
(XEN)    IRQ:  28 affinity:01 vec:41 type=PCI-MSI/-X      status=00000030
in-flight=0 domain-list=0:277(----),
(XEN)    IRQ:  29 affinity:01 vec:49 type=PCI-MSI/-X      status=00000030
in-flight=0 domain-list=0:276(----),
(XEN)    IRQ:  30 affinity:01 vec:51 type=PCI-MSI/-X      status=00000030
in-flight=0 domain-list=0:275(----),
(XEN)    IRQ:  31 affinity:04 vec:d7 type=PCI-MSI         status=00000030
in-flight=0 domain-list=0:274(----),
(XEN)    IRQ:  32 affinity:04 vec:df type=PCI-MSI         status=00000030
in-flight=0 domain-list=0:273(----),
(XEN)    IRQ:  33 affinity:02 vec:b0 type=PCI-MSI         status=00000010
in-flight=0 domain-list=0:272(----),
(XEN)    IRQ:  34 affinity:02 vec:a8 type=PCI-MSI         status=00000010
in-flight=0 domain-list=0:271(----),
(XEN)    IRQ:  35 affinity:04 vec:ad type=PCI-MSI         status=00000030
in-flight=0 domain-list=0:270(----),
(XEN) IO-APIC interrupt information:
(XEN)     IRQ  0 Vec240:
(XEN)       Apic 0x00, Pin  2: vec=f0 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN)     IRQ  1 Vec133:
(XEN)       Apic 0x00, Pin  1: vec=85 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN)     IRQ  3 Vec 64:
(XEN)       Apic 0x00, Pin  3: vec=40 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN)     IRQ  4 Vec 72:
(XEN)       Apic 0x00, Pin  4: vec=48 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN)     IRQ  5 Vec 80:
(XEN)       Apic 0x00, Pin  5: vec=50 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN)     IRQ  6 Vec 88:
(XEN)       Apic 0x00, Pin  6: vec=58 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN)     IRQ  7 Vec 96:
(XEN)       Apic 0x00, Pin  7: vec=60 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN)     IRQ  8 Vec 41:
(XEN)       Apic 0x00, Pin  8: vec=29 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN)     IRQ  9 Vec127:
(XEN)       Apic 0x00, Pin  9: vec=7f delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=L mask=0 dest_id:0
(XEN)     IRQ 10 Vec120:
(XEN)       Apic 0x00, Pin 10: vec=78 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN)     IRQ 11 Vec136:
(XEN)       Apic 0x00, Pin 11: vec=88 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN)     IRQ 12 Vec212:
(XEN)       Apic 0x00, Pin 12: vec=d4 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN)     IRQ 13 Vec152:
(XEN)       Apic 0x00, Pin 13: vec=98 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=1 dest_id:0
(XEN)     IRQ 14 Vec160:
(XEN)       Apic 0x00, Pin 14: vec=a0 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN)     IRQ 15 Vec168:
(XEN)       Apic 0x00, Pin 15: vec=a8 delivery=LoPri dest=L status=0
polarity=0 irr=0 trig=E mask=0 dest_id:0
(XEN)     IRQ 16 Vec166:
(XEN)       Apic 0x00, Pin 16: vec=a6 delivery=LoPri dest=L status=0
polarity=1 irr=0 trig=L mask=0 dest_id:0
(XEN)     IRQ 17 Vec192:
(XEN)       Apic 0x00, Pin 17: vec=c0 delivery=LoPri dest=L status=0
polarity=1 irr=0 trig=L mask=1 dest_id:0
(XEN)     IRQ 18 Vec200:
(XEN)       Apic 0x00, Pin 18: vec=c8 delivery=LoPri dest=L status=0
polarity=1 irr=0 trig=L mask=1 dest_id:0
(XEN)     IRQ 19 Vec241:
(XEN)       Apic 0x00, Pin 19: vec=f1 delivery=LoPri dest=L status=0
polarity=1 irr=0 trig=L mask=0 dest_id:0
(XEN)     IRQ 20 Vec 97:
(XEN)       Apic 0x00, Pin 20: vec=61 delivery=LoPri dest=L status=0
polarity=1 irr=0 trig=L mask=1 dest_id:0
(XEN)     IRQ 22 Vec 50:
(XEN)       Apic 0x00, Pin 22: vec=32 delivery=LoPri dest=L status=0
polarity=1 irr=0 trig=L mask=1 dest_id:0
(XEN)     IRQ 23 Vec172:
(XEN)       Apic 0x00, Pin 23: vec=ac delivery=LoPri dest=L status=0
polarity=1 irr=0 trig=L mask=0 dest_id:0

> Seeing the 'z' output might also be helpful, especially to see whether
> any of the IO-APICs' RTEs is an ExtINT one.

(XEN) number of MP IRQ sources: 15.
(XEN) number of IO-APIC #2 registers: 24.
(XEN) testing the IO APIC.......................
(XEN) IO APIC #2......
(XEN) .... register #00: 02000000
(XEN) .......    : physical APIC id: 02
(XEN) .......    : Delivery Type: 0
(XEN) .......    : LTS          : 0
(XEN) .... register #01: 00170020
(XEN) .......     : max redirection entries: 0017
(XEN) .......     : PRQ implemented: 0
(XEN) .......     : IO APIC version: 0020
(XEN) .... IRQ redirection table:
(XEN)  NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
(XEN)  00 000 00  1    0    0   0   0    0    0    00
(XEN)  01 000 00  0    0    0   0   0    1    1    85
(XEN)  02 000 00  0    0    0   0   0    1    1    F0
(XEN)  03 000 00  0    0    0   0   0    1    1    40
(XEN)  04 000 00  0    0    0   0   0    1    1    48
(XEN)  05 000 00  0    0    0   0   0    1    1    50
(XEN)  06 000 00  0    0    0   0   0    1    1    58
(XEN)  07 000 00  0    0    0   0   0    1    1    60
(XEN)  08 000 00  0    0    0   0   0    1    1    29
(XEN)  09 000 00  0    1    0   0   0    1    1    A7
(XEN)  0a 000 00  0    0    0   0   0    1    1    78
(XEN)  0b 000 00  0    0    0   0   0    1    1    88
(XEN)  0c 000 00  0    0    0   0   0    1    1    D4
(XEN)  0d 000 00  1    0    0   0   0    1    1    98
(XEN)  0e 000 00  0    0    0   0   0    1    1    A0
(XEN)  0f 000 00  0    0    0   0   0    1    1    A8
(XEN)  10 000 00  0    1    0   1   0    1    1    AE
(XEN)  11 000 00  1    1    0   1   0    1    1    C0
(XEN)  12 000 00  1    1    0   1   0    1    1    C8
(XEN)  13 000 00  0    1    0   1   0    1    1    F1
(XEN)  14 000 00  1    1    0   1   0    1    1    61
(XEN)  15 0CA 0A  1    0    0   0   0    1    2    71
(XEN)  16 000 00  1    1    0   1   0    1    1    32
(XEN)  17 000 00  0    1    0   1   0    1    1    AC
(XEN) Using vector-based indexing
(XEN) IRQ to pin mappings:
(XEN) IRQ240 -> 0:2
(XEN) IRQ133 -> 0:1
(XEN) IRQ64 -> 0:3
(XEN) IRQ72 -> 0:4
(XEN) IRQ80 -> 0:5
(XEN) IRQ88 -> 0:6
(XEN) IRQ96 -> 0:7
(XEN) IRQ41 -> 0:8
(XEN) IRQ167 -> 0:9
(XEN) IRQ120 -> 0:10
(XEN) IRQ136 -> 0:11
(XEN) IRQ212 -> 0:12
(XEN) IRQ152 -> 0:13
(XEN) IRQ160 -> 0:14
(XEN) IRQ168 -> 0:15
(XEN) IRQ174 -> 0:16
(XEN) IRQ192 -> 0:17
(XEN) IRQ200 -> 0:18
(XEN) IRQ241 -> 0:19
(XEN) IRQ97 -> 0:20
(XEN) IRQ50 -> 0:22
(XEN) IRQ172 -> 0:23
(XEN) .................................... done.

[-- Attachment #1.2: Type: text/html, Size: 11923 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [xen-unstable test] 11946: regressions - FAIL
  2012-05-07 18:29                 ` AP
@ 2012-05-08  6:37                   ` Jan Beulich
  0 siblings, 0 replies; 24+ messages in thread
From: Jan Beulich @ 2012-05-08  6:37 UTC (permalink / raw)
  To: AP
  Cc: xen-devel, Keir (Xen.org),
	Ian Campbell, Andrew Cooper, Ian Jackson, Daniel De Graaf

>>> On 07.05.12 at 20:29, AP <apxeng@gmail.com> wrote:
> On Mon, May 7, 2012 at 1:34 PM, Jan Beulich <JBeulich@suse.com> wrote:
>> Seeing the 'z' output might also be helpful, especially to see whether
>> any of the IO-APICs' RTEs is an ExtINT one.
> 
> (XEN) number of MP IRQ sources: 15.
> (XEN) number of IO-APIC #2 registers: 24.
> (XEN) testing the IO APIC.......................
> (XEN) IO APIC #2......
> (XEN) .... register #00: 02000000
> (XEN) .......    : physical APIC id: 02
> (XEN) .......    : Delivery Type: 0
> (XEN) .......    : LTS          : 0
> (XEN) .... register #01: 00170020
> (XEN) .......     : max redirection entries: 0017
> (XEN) .......     : PRQ implemented: 0
> (XEN) .......     : IO APIC version: 0020
> (XEN) .... IRQ redirection table:
> (XEN)  NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
> (XEN)  00 000 00  1    0    0   0   0    0    0    00
> (XEN)  01 000 00  0    0    0   0   0    1    1    85
> (XEN)  02 000 00  0    0    0   0   0    1    1    F0
> (XEN)  03 000 00  0    0    0   0   0    1    1    40
> (XEN)  04 000 00  0    0    0   0   0    1    1    48
> (XEN)  05 000 00  0    0    0   0   0    1    1    50
> (XEN)  06 000 00  0    0    0   0   0    1    1    58
> (XEN)  07 000 00  0    0    0   0   0    1    1    60
> (XEN)  08 000 00  0    0    0   0   0    1    1    29
> (XEN)  09 000 00  0    1    0   0   0    1    1    A7
> (XEN)  0a 000 00  0    0    0   0   0    1    1    78
> (XEN)  0b 000 00  0    0    0   0   0    1    1    88
> (XEN)  0c 000 00  0    0    0   0   0    1    1    D4
> (XEN)  0d 000 00  1    0    0   0   0    1    1    98
> (XEN)  0e 000 00  0    0    0   0   0    1    1    A0
> (XEN)  0f 000 00  0    0    0   0   0    1    1    A8
> (XEN)  10 000 00  0    1    0   1   0    1    1    AE
> (XEN)  11 000 00  1    1    0   1   0    1    1    C0
> (XEN)  12 000 00  1    1    0   1   0    1    1    C8
> (XEN)  13 000 00  0    1    0   1   0    1    1    F1
> (XEN)  14 000 00  1    1    0   1   0    1    1    61
> (XEN)  15 0CA 0A  1    0    0   0   0    1    2    71

This entry is definitely bogus (delivery mode is SMI, which is not
allowed in an IO-APIC RTE), but as it is masked it _shouldn't_
cause any harm.

> (XEN)  16 000 00  1    1    0   1   0    1    1    32
> (XEN)  17 000 00  0    1    0   1   0    1    1    AC

So we'll need to see the PIC (8259A) masks too. IRQ12 definitely
appears to get touched a lot (judging by the vector it uses), so while
this shouldn't be the case I would nevertheless consider the possibility
of a window where the 8259A interrupt gets temporarily unmasked.

Jan

^ permalink raw reply	[flat|nested] 24+ messages in thread

end of thread, other threads:[~2012-05-08  6:37 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-02-13 20:16 [xen-unstable test] 11946: regressions - FAIL xen.org
2012-02-14 10:44 ` Ian Campbell
2012-02-14 19:17   ` Daniel De Graaf
2012-03-27 10:36   ` Ian Campbell
2012-03-27 10:52     ` Jan Beulich
2012-05-04 19:48     ` AP
2012-05-04 20:11       ` Andrew Cooper
2012-05-05  0:21         ` AP
2012-05-05 11:04           ` Andrew Cooper
2012-05-05 18:41             ` AP
2012-05-05 19:06               ` AP
2012-05-07  8:10           ` Jan Beulich
2012-05-07 11:50             ` Andrew Cooper
2012-05-07 13:34               ` Jan Beulich
2012-05-07 14:41                 ` Andrew Cooper
2012-05-07 14:50                   ` Jan Beulich
2012-05-07 15:40                     ` Andrew Cooper
2012-05-07 15:43                       ` Jan Beulich
2012-05-07 14:54                   ` Jan Beulich
2012-05-07 15:51                     ` Andrew Cooper
2012-05-07 18:29                 ` AP
2012-05-08  6:37                   ` Jan Beulich
2012-05-05 10:33         ` Ian Campbell
2012-05-05 11:11           ` Andrew Cooper

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.