On Fri, Nov 19, 2010 at 11:17 AM, Boris Derzhavets wrote: > > You uncompress :- >   tar lxf linux-2.6.37-rc2.tar.bz2 > Mine :- >  tar jxvf linux-2.6.37-rc2.tar.bz2 > > Boris > > --- On Fri, 11/19/10, Bruce Edge wrote: > > From: Bruce Edge > Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request > To: "Boris Derzhavets" > Cc: "Jeremy Fitzhardinge" , xen-devel@lists.xensource.com, "Konrad Rzeszutek Wilk" > Date: Friday, November 19, 2010, 1:16 PM > > On Thu, Nov 18, 2010 at 11:12 PM, Boris Derzhavets wrote: > > I would guess, applying this patches to mainline 2.6.37-rc2 has nothing to do with specific fedora patches. It was done on Ubuntu 10.10 > 1. Kernel source extracted. > 2. Patches copied over LAN and applied. > > root@boris-System-P5Q3:~# cd Downloads/*-rc2 > root@boris-System-P5Q3:~/Downloads/linux-2.6.37-rc2# ls -l ../*.patch > -rw-r--r-- 1 root root 84442 2010-11-17 01:06 ../xen.next-2.6.37.patch > -rw-r--r-- 1 root root  4062 2010-11-17 01:06 ../xen.pcifront.fixes.patch > root@boris-System-P5Q3:~/Downloads/linux-2.6.37-rc2# patch -p1 -s -i ../xen.next-2.6.37.patch > root@boris-System-P5Q3:~/Downloads/linux-2.6.37-rc2# patch -p1 -s -i ../xen.pcifront.fixes.patch > root@boris-System-P5Q3:~/Downloads/linux-2.6.37-rc2# > > Boris. > P.S Directory /root/Downloads/linux-2.6.37-rc2 contains mainline's 2.6.37-rc2 > > I feel like I must be doing something unbelievably stupid: > > %> rm -rf linux-2.6.37 > %> tar lxf linux-2.6.37-rc2.tar.bz2 (This is http://www.kernel.org/pub/linux/kernel/v2.6/testing/linux-2.6.37-rc2.tar.bz2) > %> cd linux-2.6.37-rc2 > %> patch --dry-run -p1 -s -i  ../patches.2.6.37/xen.next-2.6.37.patch > > 1 out of 2 hunks FAILED -- saving rejects to file drivers/block/xen-blkfront.c.rej > 1 out of 1 hunk FAILED -- saving rejects to file arch/x86/include/asm/microcode.h.rej > 1 out of 1 hunk FAILED -- saving rejects to file arch/x86/kernel/Makefile.rej > 1 out of 1 hunk FAILED -- saving rejects to file arch/x86/xen/Kconfig.rej > 1 out of 2 hunks FAILED -- saving rejects to file arch/x86/include/asm/processor.h.rej > 1 out of 1 hunk FAILED -- saving rejects to file drivers/xen/gntdev.c.rej > > -Bruce > > > --- On Thu, 11/18/10, Bruce Edge wrote: > > From: Bruce Edge > Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request > To: "Boris Derzhavets" > Cc: "Jeremy Fitzhardinge" , xen-devel@lists.xensource.com, "Konrad Rzeszutek Wilk" > Date: Thursday, November 18, 2010, 3:05 PM > > > > On Thu, Nov 18, 2010 at 9:05 AM, Boris Derzhavets wrote: > > Bruce, > You should be able to apply patches to mainline 2.6.37-rc2 cleanly. This patches are > taken out from MY's kernel-2.6.37-rc2.git0.fc15.src.rpm. > I already applied them on Ubuntu 10.10 to uncompressed mainline rc2. > > I'm assuming fc=fedora core here? > Doesn't the fc kernel have a number of other patches already installed? > I'm using a kernel.org kernel, which has none of the fc patches so it's not surprising that they don't apply. > > -Bruce > > > I have also to notice, that > # mount IP-Dom0:/home/user1 /mnt/nfs > # cd /mnt/nfs > # ls -l > crashes DomU immediately in text mode. > In graphics mode it doesn't necessary happen every time. > DomU might survive this "hack" and crashed one > hour latter by another reason. > > Boris. > > --- On Thu, 11/18/10, Bruce Edge wrote: > > From: Bruce Edge > Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request > To: "Boris Derzhavets" > Cc: "Konrad Rzeszutek Wilk" , "Jeremy Fitzhardinge" , xen-devel@lists.xensource.com > Date: Thursday, November 18, 2010, 11:40 AM > > > > On Thu, Nov 18, 2010 at 2:34 AM, Boris Derzhavets wrote: > > Could you apply two attached patches on top of 2.6.37-rc2 and see > whether it gives some improvement or no ( with active NFS client at DomU) > > Boris > > Hi Boris, > > Are you using the mainline kernel or a pvops branch with these patches? > Maybe I'm doing something wrong, but they don't apply cleanly with 2.6.37-rc2: > > %> patch --dry-run <../patches.2.6.37/xen.next-2.6.37.patch > > patching file pgtable.h > Hunk #1 FAILED at 399. > 1 out of 1 hunk FAILED -- saving rejects to file pgtable.h.rej > patching file pgtable.c > Hunk #1 FAILED at 15. > 1 out of 1 hunk FAILED -- saving rejects to file pgtable.c.rej > patching file ttm_bo_vm.c > Hunk #1 FAILED at 273. > Hunk #2 FAILED at 288. > 2 out of 2 hunks FAILED -- saving rejects to file ttm_bo_vm.c.rej > ...... > >  %> patch --dry-run <../patches.2.6.37/xen.pcifront.fixes.patch > > patching file enlighten.c > Hunk #1 FAILED at 1090. > Hunk #2 FAILED at 1202. > 2 out of 2 hunks FAILED -- saving rejects to file enlighten.c.rej > patching file setup.c > Hunk #1 FAILED at 337. > Hunk #2 FAILED at 356. > 2 out of 2 hunks FAILED -- saving rejects to file setup.c.rej > > Same result for linux-2.6.37-xen-next branch. > > -Bruce > > > --- On Wed, 11/17/10, Bruce Edge wrote: > > From: Bruce Edge > Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request > To: "Boris Derzhavets" > Cc: "Konrad Rzeszutek Wilk" , "Jeremy Fitzhardinge" , xen-devel@lists.xensource.com > Date: Wednesday, November 17, 2010, 4:28 PM > > On Tue, Nov 16, 2010 at 1:49 PM, Boris Derzhavets wrote: > > Yes, here we are > > [  186.975228] ------------[ cut here ]------------ > [  186.975245] kernel BUG at mm/mmap.c:2399! > [  186.975254] invalid opcode: 0000 [#1] SMP > [  186.975269] last sysfs file: /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map > [  186.975284] CPU 0 > [  186.975290] Modules linked in: nfs fscache deflate zlib_deflate ctr camellia cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic twofish_x86_64 twofish_common ecb xcbc cbc sha256_generic sha512_generic des_generic cryptd aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_tunnel tunnel4 xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp xfrm6_tunnel tunnel6 af_key nfsd lockd nfs_acl auth_rpcgss exportfs sunrpc ipv6 uinput xen_netfront microcode xen_blkfront [last unloaded: scsi_wait_scan] > [  186.975507] > [  186.975515] Pid: 1562, comm: ls Not tainted 2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 / > [  186.975529] RIP: e030:[]  [] exit_mmap+0x10c/0x119 > [  186.975550] RSP: e02b:ffff8800781bde18  EFLAGS: 00010202 > [  186.975560] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000 > [  186.975573] RDX: 00000000914a9149 RSI: 0000000000000001 RDI: ffffea00000c0280 > [  186.975585] RBP: ffff8800781bde48 R08: ffffea00000c0280 R09: 0000000000000001 > [  186.975598] R10: ffffffff8100750f R11: ffffea0000967778 R12: ffff880076c68b00 > [  186.975610] R13: ffff88007f83f1e0 R14: ffff880076c68b68 R15: 0000000000000001 > [  186.975625] FS:  00007f8e471d97c0(0000) GS:ffff88007f831000(0000) knlGS:0000000000000000 > [  186.975639] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b > [  186.975650] CR2: 00007f8e464a9940 CR3: 0000000001a03000 CR4: 0000000000002660 > [  186.975663] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 > [  186.976012] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 > [  186.976012] Process ls (pid: 1562, threadinfo ffff8800781bc000, task ffff8800788223e0) > [  186.976012] Stack: > [  186.976012]  000000000000006b ffff88007f83f1e0 ffff8800781bde38 ffff880076c68b00 > [  186.976012]  ffff880076c68c40 ffff8800788229d0 ffff8800781bde68 ffffffff810505fc > [  186.976012]  ffff8800788223e0 ffff880076c68b00 ffff8800781bdeb8 ffffffff81056747 > [  186.976012] Call Trace: > [  186.976012]  [] mmput+0x65/0xd8 > [  186.976012]  [] exit_mm+0x13e/0x14b > [  186.976012]  [] do_exit+0x222/0x7c6 > [  186.976012]  [] ? xen_restore_fl_direct_end+0x0/0x1 > [  186.976012]  [] ? arch_local_irq_restore+0xb/0xd > [  186.976012]  [] ? lockdep_sys_exit_thunk+0x35/0x67 > [  186.976012]  [] do_group_exit+0x88/0xb6 > [  186.976012]  [] sys_exit_group+0x17/0x1b > [  186.976012]  [] system_call_fastpath+0x16/0x1b > [  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02 <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48 > [  186.976012] RIP  [] exit_mmap+0x10c/0x119 > [  186.976012]  RSP > [  186.976012] ---[ end trace c0f4eff4054a67e4 ]--- > [  186.976012] Fixing recursive fault but reboot is needed! > > Message from syslogd@fedora14 at Nov 17 00:47:40 ... >  kernel:[  186.975228] ------------[ cut here ]------------ > > Message from syslogd@fedora14 at Nov 17 00:47:40 ... >  kernel:[  186.975254] invalid opcode: 0000 [#1] SMP > > Message from syslogd@fedora14 at Nov 17 00:47:40 ... >  kernel:[  186.975269] last sysfs file: /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map > > Message from syslogd@fedora14 at Nov 17 00:47:40 ... >  kernel:[  186.976012] Stack: > > Message from syslogd@fedora14 at Nov 17 00:47:40 ... >  kernel:[  186.976012] Call Trace: > > Message from syslogd@fedora14 at Nov 17 00:47:40 ... >  kernel:[  186.976012] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02 <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48 > > --- On Tue, 11/16/10, Konrad Rzeszutek Wilk wrote: > > From: Konrad Rzeszutek Wilk > Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request > To: "Boris Derzhavets" > Cc: "Jeremy Fitzhardinge" , xen-devel@lists.xensource.com, "Bruce Edge" > Date: Tuesday, November 16, 2010, 4:15 PM > > On Tue, Nov 16, 2010 at 12:43:28PM -0800, Boris Derzhavets wrote: > > > Huh. I .. what? I am confused. I thought we established that the issue > > > was not related to Xen PCI front? You also seem to uncomment the > > > upstream.core.patches and the xen.pvhvm.patch - why? > > > > I cannot uncomment upstream.core.patches and the xen.pvhvm.patch > > it gives failed HUNKs > > Uhh.. I am even more confused. > > > > > Ok, they are.. v2.6.37-rc2 which came out today has the fixes > > > > I am pretty sure rc2 doesn't contain everything from xen.next-2.6.37.patch, > > gntdev's stuff for sure. I've built 2.6.37-rc2 kernel rpms and loaded > > kernel-2.6.27-rc2.git0.xendom0.x86_64 under Xen 4.0.1. > > Device /dev/xen/gntdev has not been created. I understand that it's > > unrelated to DomU ( related to Dom0) , but once again with rc2 in DomU i cannot > > get 3.2 GB copied over to DomU from NFS share at Dom0. > > So what I think you are saying is that you keep on getting the bug in DomU? > Is the stack-trace the same as in rc1? > > > > I haven't had much time to look into the broken/working version issues here, but I did confirm a couple of points: > 1) The 2.6.37-rc2 has the same problem still > 2) This problem goes away of one is not using NFS. > > Not staggeringly helpful I know, but it's one small data point. > > -Bruce > > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > > > > -----Inline Attachment Follows----- > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > > > I just checked the recently released 2.6.73-rc4, and while the BUG signature is different, it still fails under NFS accesses. This is 100% recreatable. 0 uss004-dpm  ~ #> [   60.303318] ------------[ cut here ]------------ [   60.303335] kernel BUG at arch/x86/xen/mmu.c:1831! [   60.303345] invalid opcode: 0000 [#1] SMP [   60.303362] last sysfs file: /sys/kernel/uevent_seqnum [   60.303371] CPU 0 [   60.303377] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront xen_blkfront [   60.303444] [   60.303453] Pid: 3965, comm: cron Not tainted 2.6.37-rc4-kernel.org-domu-6.0.nopvhvm.bedge-debug #1 / [   60.303468] RIP: e030:[]  [] pin_pagetable_pfn+0x31/0x37 [   60.303491] RSP: e02b:ffff880077d37ce8  EFLAGS: 00010282 [   60.303501] RAX: 00000000ffffffea RBX: 0000000000077da4 RCX: 0000000000000001 [   60.303512] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI: 00000000deadbeef [   60.303524] RBP: ffff880077d37d08 R08: 00003ffffffff000 R09: ffff880000000000 [   60.303536] R10: 00000000deadbeef R11: dead000000200200 R12: 0000000000000003 [   60.303548] R13: ffff880077d01598 R14: ffff880075dbd7e8 R15: ffff880076335070 [   60.303564] FS:  00007fd3d87aa7a0(0000) GS:ffff88007ff40000(0000) knlGS:0000000000000000 [   60.303577] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b [   60.303588] CR2: 00007fd3d663d398 CR3: 00000000776b1000 CR4: 0000000000002660 [   60.303600] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [   60.303612] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [   60.303624] Process cron (pid: 3965, threadinfo ffff880077d36000, task ffff880076335070) [   60.303636] Stack: [   60.303643]  ffff880000000000 0000000000047c6a ffff880076297800 0000000000077da4 [   60.303671]  ffff880077d37d28 ffffffff8100665c 0000000000077da4 ffff880076297800 [   60.303699]  ffff880077d37d38 ffffffff8100668f ffff880077d37d78 ffffffff810e0653 [   60.303726] Call Trace: [   60.303739]  [] xen_alloc_ptpage+0x68/0x6d [   60.303754]  [] xen_alloc_pte+0xe/0x10 [   60.303769]  [] __pte_alloc+0x72/0xd7 [   60.303783]  [] handle_mm_fault+0x131/0x8c1 [   60.303797]  [] ? mmap_region+0x391/0x4bf [   60.303812]  [] do_page_fault+0x374/0x396 [   60.303827]  [] ? pvclock_clocksource_read+0x4b/0xb4 [   60.303843]  [] ? __spin_time_accum+0x21/0x37 [   60.303858]  [] ? __xen_spin_lock+0xb7/0xcd [   60.303873]  [] page_fault+0x25/0x30 [   60.303883] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d e0 48 89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 67 cb ff ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74 30 48 [   60.304076] RIP  [] pin_pagetable_pfn+0x31/0x37 [   60.304076]  RSP [   60.304076] ---[ end trace 8bd01a369ffae446 ]--- [   64.355381] kmemleak: 4 new suspected memory leaks (see /sys/kernel/debug/kmemleak) Another domU boot yielded a slightly different crash: uss004-dpm login: [ 164.897190] ------------[ cut here ]------------ [ 164.897208] kernel BUG at arch/x86/xen/mmu.c:1831! [ 164.897221] invalid opcode: 0000 [#1] SMP [ 164.897238] last sysfs file: /sys/kernel/uevent_seqnum [ 164.897250] CPU 2 [ 164.897257] Modules linked in: ext4 mbcache jbd2 crc16 xen_fbfront fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xen_netfront xen_blkfront [ 164.897326] [ 164.897337] Pid: 3938, comm: cron Not tainted 2.6.37-rc4-kernel.org-domu-6.0.nopvhvm.bedge-debug #1 / [ 164.897352] RIP: e030:[] [] pin_pagetable_pfn+0x31/0x37 [ 164.897375] RSP: e02b:ffff8800781dbce8 EFLAGS: 00010282 [ 164.897386] RAX: 00000000ffffffea RBX: 0000000000077e3b RCX: 0000000000000001 [ 164.897398] RDX: 00000000deadbeef RSI: 00000000deadbeef RDI: 00000000deadbeef [ 164.897410] RBP: ffff8800781dbd08 R08: 00003ffffffff000 R09: ffff880000000000 [ 164.897421] R10: 00000000deadbeef R11: dead000000200200 R12: 0000000000000003 [ 164.897433] R13: ffff88007646a0f0 R14: ffff880076309678 R15: ffff880075c08ea0 [ 164.897451] FS: 00007fcd44d5e7a0(0000) GS:ffff88007ff78000(0000) knlGS:0000000000000000 [ 164.897464] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b [ 164.897474] CR2: 00007fcd43d55850 CR3: 0000000075e6e000 CR4: 0000000000002660 [ 164.897488] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 164.897500] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 164.897513] Process cron (pid: 3938, threadinfo ffff8800781da000, task ffff880075c08ea0) [ 164.897528] Stack: [ 164.897536] ffff880000000000 0000000000047bd3 ffff88007631c700 0000000000077e3b [ 164.897565] ffff8800781dbd28 ffffffff8100665c 0000000000077e3b ffff88007631c700 [ 164.897595] ffff8800781dbd38 ffffffff8100668f ffff8800781dbd78 ffffffff810e0653 [ 164.897625] Call Trace: [ 164.897639] [] xen_alloc_ptpage+0x68/0x6d [ 164.897655] [] xen_alloc_pte+0xe/0x10 [ 164.897670] [] __pte_alloc+0x72/0xd7 [ 164.897684] [] handle_mm_fault+0x131/0x8c1 [ 164.897699] [] ? xen_restore_fl_direct_end+0x0/0x1 [ 164.897714] [] ? xen_mc_flush+0x1cd/0x1f2 [ 164.897730] [] do_page_fault+0x374/0x396 [ 164.897744] [] ? check_events+0x12/0x20 [ 164.897758] [] ? xen_force_evtchn_callback+0xd/0xf [ 164.897772] [] ? check_events+0x12/0x20 [ 164.897786] [] ? xen_restore_fl_direct_end+0x0/0x1 [ 164.897804] [] ? T.1087+0x22/0x24 [ 164.897816] [] ? xen_clts+0x57/0x59 [ 164.897832] [] page_fault+0x25/0x30 [ 164.897842] Code: ec 20 89 7d e0 48 89 f7 e8 c0 ff ff ff 48 8d 7d e0 48 89 45 e8 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 67 cb ff ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74 30 48 [ 164.898062] RIP [] pin_pagetable_pfn+0x31/0x37 [ 164.898062] RSP [ 164.898062] ---[ end trace 85c4388599f45d39 ]--- The dom0 printed this on the serial console around the same time the domU crashed: 0 uss004  ~ #> (XEN) mm.c:2389:d5 Bad type (saw 7400000000000001 != exp 1000000000000000) for mfn 47c6a (pfn 77da4) (XEN) mm.c:2965:d5 Error while pinning mfn 47c6a Then, a few min later started spewing these to the dom0 console: (XEN) mm.c:2389:d5 Bad type (saw 7400000000000001 != exp 3000000000000000) for mfn 49c0a (pfn 75e04) (XEN) mm.c:964:d5 Attempt to create linear p.t. with write perms (XEN) mm.c:1505:d5 Failure in alloc_l4_table: entry 0 (XEN) mm.c:2142:d5 Error while validating mfn 484d4 (pfn 7753a) for type 4000000000000000: caf=8000000000000003 taf=4000000000000001 (XEN) mm.c:2965:d5 Error while pinning mfn 484d4 (XEN) mm.c:2389:d5 Bad type (saw 7400000000000001 != exp 3000000000000000) for mfn 49c0a (pfn 75e04) (XEN) mm.c:964:d5 Attempt to create linear p.t. with write perms (XEN) mm.c:1505:d5 Failure in alloc_l4_table: entry 0 (XEN) mm.c:2142:d5 Error while validating mfn 49b69 (pfn 75ea5) for type 4000000000000000: caf=8000000000000003 taf=4000000000000001 (XEN) mm.c:2965:d5 Error while pinning mfn 49b69 (XEN) mm.c:2389:d5 Bad type (saw 7400000000000001 != exp 3000000000000000) for mfn 49c0a (pfn 75e04) (XEN) mm.c:964:d5 Attempt to create linear p.t. with write perms (XEN) mm.c:1505:d5 Failure in alloc_l4_table: entry 0 (XEN) mm.c:2142:d5 Error while validating mfn 484d4 (pfn 7753a) for type 4000000000000000: caf=8000000000000003 taf=4000000000000001 (XEN) mm.c:2732:d5 Error while installing new baseptr 484d4 (XEN) mm.c:2389:d5 Bad type (saw 7400000000000001 != exp 3000000000000000) for mfn 49c0a (pfn 75e04) (XEN) mm.c:964:d5 Attempt to create linear p.t. with write perms (XEN) mm.c:1505:d5 Failure in alloc_l4_table: entry 0 (XEN) mm.c:2142:d5 Error while validating mfn 49b69 (pfn 75ea5) for type 4000000000000000: caf=8000000000000003 taf=4000000000000001 (XEN) mm.c:3057:d5 Error while installing new mfn 49b69 (XEN) mm.c:2389:d5 Bad type (saw 7400000000000001 != exp 3000000000000000) for mfn 49c0a (pfn 75e04) (XEN) mm.c:964:d5 Attempt to create linear p.t. with write perms (XEN) mm.c:1505:d5 Failure in alloc_l4_table: entry 0 (XEN) mm.c:2142:d5 Error while validating mfn 484d4 (pfn 7753a) for type 4000000000000000: caf=8000000000000003 taf=4000000000000001 (XEN) mm.c:2732:d5 Error while installing new baseptr 484d4 (XEN) mm.c:2389:d5 Bad type (saw 7400000000000001 != exp 3000000000000000) for mfn 49c0a (pfn 75e04) (XEN) mm.c:964:d5 Attempt to create linear p.t. with write perms (XEN) mm.c:1505:d5 Failure in alloc_l4_table: entry 0 (XEN) mm.c:2142:d5 Error while validating mfn 49b69 (pfn 75ea5) for type 4000000000000000: caf=8000000000000003 taf=4000000000000001 (XEN) mm.c:3057:d5 Error while installing new mfn 49b69 .......... These messages continued until I killed the domU. The hypervisor is xen-unstable 22241 Here's the complete domU boot log: Started domain dpm (id=5) [    0.000000] Initializing cgroup subsys cpuset [    0.000000] Initializing cgroup subsys cpu [    0.000000] Linux version 2.6.37-rc4-kernel.org-domu-6.0.nopvhvm.bedge-debug (root@opal) (gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #1 SMP Mon Nov 29 22:22:40 PST 2010 [    0.000000] Command line: root=UUID=e971c31a-7353-4ac7-90c4-d6eb7fc33735 ro   iommu=soft noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r ip=:127.0.255.255::::eth0:dhcp [    0.000000] ACPI in unprivileged domain disabled [    0.000000] released 0 pages of unused memory [    0.000000] BIOS-provided physical RAM map: [    0.000000]  Xen: 0000000000000000 - 00000000000a0000 (usable) [    0.000000]  Xen: 00000000000a0000 - 0000000000100000 (reserved) [    0.000000]  Xen: 0000000000100000 - 0000000080800000 (usable) [    0.000000] NX (Execute Disable) protection: active [    0.000000] DMI not present or invalid. [    0.000000] No AGP bridge found [    0.000000] last_pfn = 0x80800 max_arch_pfn = 0x400000000 [    0.000000] Scanning 0 areas for low memory corruption [    0.000000] init_memory_mapping: 0000000000000000-0000000080800000 [    0.000000] RAMDISK: 01d95000 - 02faf000 [    0.000000] No NUMA configuration found [    0.000000] Faking a node at 0000000000000000-0000000080800000 [    0.000000] Initmem setup node 0 0000000000000000-0000000080800000 [    0.000000]   NODE_DATA [000000007fffb000 - 000000007fffffff] [    0.000000] Zone PFN ranges: [    0.000000]   DMA      0x00000010 -> 0x00001000 [    0.000000]   DMA32    0x00001000 -> 0x00100000 [    0.000000]   Normal   empty [    0.000000] Movable zone start PFN for each node [    0.000000] early_node_map[2] active PFN ranges [    0.000000]     0: 0x00000010 -> 0x000000a0 [    0.000000]     0: 0x00000100 -> 0x00080800 [    0.000000] SMP: Allowing 6 CPUs, 0 hotplug CPUs [    0.000000] No local APIC present [    0.000000] APIC: disable apic facility [    0.000000] APIC: switched to apic NOOP [    0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000 [    0.000000] Allocating PCI resources starting at 80800000 (gap: 80800000:7f800000) [    0.000000] Booting paravirtualized kernel on Xen [    0.000000] Xen version: 4.1-unstable (preserve-AD) [    0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:6 nr_node_ids:1 [    0.000000] PERCPU: Embedded 28 pages/cpu @ffff88007ff40000 s82752 r8192 d23744 u114688 [    0.000000] pcpu-alloc: s82752 r8192 d23744 u114688 alloc=28*4096 [    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5 [    0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 519022 [    0.000000] Policy zone: DMA32 [    0.000000] Kernel command line: root=UUID=e971c31a-7353-4ac7-90c4-d6eb7fc33735 ro   iommu=soft noirqbalance noirqbalance console=hvc0,115200n8 --flow_control=r ip=:127.0.255.255::::eth0:dhcp [    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes) [    0.000000] Placing 64MB software IO TLB between ffff880079400000 - ffff88007d400000 [    0.000000] software IO TLB at phys 0x79400000 - 0x7d400000 [    0.000000] Memory: 1959872k/2105344k available (4402k kernel code, 448k absent, 145024k reserved, 6323k data, 756k init) [    0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, CPUs=6, Nodes=1 [    0.000000] Hierarchical RCU implementation. [    0.000000]  RCU-based detection of stalled CPUs is disabled. [    0.000000] NR_IRQS:4352 nr_irqs:320 16 [    0.000000] Console: colour dummy device 80x25 [    0.000000] console [tty0] enabled [    0.000000] console [hvc0] enabled [    0.000000] installing Xen timer for CPU 0 [    0.000000] Detected 2660.088 MHz processor. [    0.000999] Calibrating delay loop (skipped), value calculated using timer frequency.. 5320.17 BogoMIPS (lpj=2660088) [    0.000999] pid_max: default: 32768 minimum: 301 [    0.000999] Security Framework initialized [    0.000999] SELinux:  Initializing. [    0.000999] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes) [    0.001403] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes) [    0.001616] Mount-cache hash table entries: 256 [    0.002223] Initializing cgroup subsys ns [    0.002235] ns_cgroup deprecated: consider using the 'clone_children' flag without the ns_cgroup. [    0.002302] Initializing cgroup subsys cpuacct [    0.002321] Initializing cgroup subsys freezer [    0.002450] CPU: Unsupported number of siblings 16 [    0.002456] Performance Events: unsupported p6 CPU model 26 no PMU driver, software events only. [    0.002543] SMP alternatives: switching to UP code [    0.003119] cpu 0 spinlock event irq 318 [    0.003764] NMI watchdog failed to create perf event on cpu0: fffffffffffffffe [    0.004040] installing Xen timer for CPU 1 [    0.004147] cpu 1 spinlock event irq 312 [    0.004184] SMP alternatives: switching to SMP code [    0.000999] CPU: Unsupported number of siblings 16 [    0.005154] NMI watchdog failed to create perf event on cpu1: fffffffffffffffe [    0.005466] installing Xen timer for CPU 2 [    0.005570] cpu 2 spinlock event irq 306 [    0.000999] CPU: Unsupported number of siblings 16 [    0.005856] NMI watchdog failed to create perf event on cpu2: fffffffffffffffe [    0.006027] installing Xen timer for CPU 3 [    0.006068] cpu 3 spinlock event irq 300 [    0.000999] CPU: Unsupported number of siblings 16 [    0.006374] NMI watchdog failed to create perf event on cpu3: fffffffffffffffe [    0.006608] installing Xen timer for CPU 4 [    0.006716] cpu 4 spinlock event irq 294 [    0.000999] CPU: Unsupported number of siblings 16 [    0.006999] NMI watchdog failed to create perf event on cpu4: fffffffffffffffe [    0.007213] installing Xen timer for CPU 5 [    0.007316] cpu 5 spinlock event irq 288 [    0.000999] CPU: Unsupported number of siblings 16 [    0.007594] NMI watchdog failed to create perf event on cpu5: fffffffffffffffe [    0.007605] Brought up 6 CPUs [    0.008290] kworker/u:0 used greatest stack depth: 5696 bytes left [    0.011150] Grant table initialized [    0.030141] Time: 165:165:165  Date: 165/165/65 [    0.030386] NET: Registered protocol family 16 [    0.032157] kworker/u:0 used greatest stack depth: 5528 bytes left [    0.035695] PCI: setting up Xen PCI frontend stub [    0.063222] bio: create slab at 0 [    0.064255] ACPI: Interpreter disabled. [    0.065099] xen_balloon: Initialising balloon driver. [    0.065149] last_pfn = 0x80800 max_arch_pfn = 0x400000000 [    0.066259] vgaarb: loaded [    0.067215] usbcore: registered new interface driver usbfs [    0.068051] usbcore: registered new interface driver hub [    0.068163] usbcore: registered new device driver usb [    0.070102] PCI: System does not support PCI [    0.070147] PCI: System does not support PCI [    0.071161] NetLabel: Initializing [    0.071161] NetLabel:  domain hash size = 128 [    0.071161] NetLabel:  protocols = UNLABELED CIPSOv4 [    0.071276] NetLabel:  unlabeled traffic allowed by default [    0.072077] Switching to clocksource xen [    0.086791] pnp: PnP ACPI: disabled [    0.121206] NET: Registered protocol family 2 [    0.121669] IP route cache hash table entries: 65536 (order: 7, 524288 bytes) [    0.124289] TCP established hash table entries: 262144 (order: 10, 4194304 bytes) [    0.125702] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes) [    0.125921] TCP: Hash tables configured (established 262144 bind 65536) [    0.125934] TCP reno registered [    0.125984] UDP hash table entries: 1024 (order: 3, 32768 bytes) [    0.126043] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes) [    0.126722] NET: Registered protocol family 1 [    0.127683] RPC: Registered udp transport module. [    0.128057] RPC: Registered tcp transport module. [    0.128057] RPC: Registered tcp NFSv4.1 backchannel transport module. [    0.140312] Trying to unpack rootfs image as initramfs... [    0.181030] Freeing initrd memory: 18536k freed [    0.270706] DMA-API: preallocated 32768 debug entries [    0.270720] DMA-API: debugging enabled by kernel config [    0.272362] platform rtc_cmos: registered platform RTC device (no PNP device found) [    0.282120] Machine check injector initialized [    0.288555] microcode: CPU0 sig=0x106a4, pf=0x2, revision=0xa [    0.288614] microcode: CPU1 sig=0x106a4, pf=0x2, revision=0xa [    0.288697] microcode: CPU2 sig=0x106a4, pf=0x2, revision=0xa [    0.288738] microcode: CPU3 sig=0x106a4, pf=0x2, revision=0xa [    0.288806] microcode: CPU4 sig=0x106a4, pf=0x2, revision=0xa [    0.288849] microcode: CPU5 sig=0x106a4, pf=0x2, revision=0xa [    0.289277] microcode: Microcode Update Driver: v2.00 , Peter Oruba [    0.289295] Scanning for low memory corruption every 60 seconds [    0.290872] audit: initializing netlink socket (disabled) [    0.290949] type=2000 audit(1291237981.045:1): initialized [    0.307918] HugeTLB registered 2 MB page size, pre-allocated 0 pages [    0.346385] VFS: Disk quotas dquot_6.5.2 [    0.347173] Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [    0.352395] NTFS driver 2.1.29 [Flags: R/W]. [    0.354350] msgmni has been set to 3864 [    0.359310] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253) [    0.359327] io scheduler noop registered [    0.359336] io scheduler deadline registered [    0.359975] io scheduler cfq registered (default) [    0.361893] pci_hotplug: PCI Hot Plug PCI Core version: 0.5 [    0.376639] pcifront pci-0: Installing PCI frontend [    0.377039] pcifront pci-0: Creating PCI Frontend Bus 0000:00 [    0.387317] pcifront pci-0: claiming resource 0000:00:00.0/0 [    0.387317] pcifront pci-0: claiming resource 0000:00:00.0/2 [    0.387317] pcifront pci-0: claiming resource 0000:00:00.0/3 [    0.387317] pcifront pci-0: claiming resource 0000:00:00.0/4 [    0.387317] pcifront pci-0: claiming resource 0000:00:01.0/0 [    0.387317] pcifront pci-0: claiming resource 0000:00:01.0/2 [    0.387317] pcifront pci-0: claiming resource 0000:00:01.0/3 [    0.387317] pcifront pci-0: claiming resource 0000:00:01.0/4 [    0.387317] pcifront pci-0: claiming resource 0000:00:02.0/0 [    0.387317] pcifront pci-0: claiming resource 0000:00:02.0/2 [    0.387317] pcifront pci-0: claiming resource 0000:00:02.0/3 [    0.387317] pcifront pci-0: claiming resource 0000:00:02.0/4 [    0.387317] pcifront pci-0: claiming resource 0000:00:03.0/0 [    0.387317] pcifront pci-0: claiming resource 0000:00:03.0/2 [    0.387317] pcifront pci-0: claiming resource 0000:00:03.0/3 [    0.387317] pcifront pci-0: claiming resource 0000:00:03.0/4 [    0.683090] Non-volatile memory driver v1.3 [    0.683109] Linux agpgart interface v0.103 [    0.684495] [drm] Initialized drm 1.1.0 20060810 [    0.684516] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [    0.782067] brd: module loaded [    0.795822] loop: module loaded [    0.796963] Fixed MDIO Bus: probed [    0.799041] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver [    0.799696] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver [    0.800288] uhci_hcd: USB Universal Host Controller Interface driver [    0.802024] usbcore: registered new interface driver usblp [    0.802618] usbcore: registered new interface driver libusual [    0.804297] PNP: No PS/2 controller found. Probing ports directly. [    0.805147] i8042.c: No controller found. [    0.805764] mice: PS/2 mouse device common for all mice [    0.808307] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0 [    0.808937] rtc_cmos: probe of rtc_cmos failed with error -38 [    0.810975] cpuidle: using governor ladder [    0.810988] cpuidle: using governor menu [    0.811099] Netfilter messages via NETLINK v0.30. [    0.811187] nf_conntrack version 0.5.0 (16384 buckets, 65536 max) [    0.812949] ctnetlink v0.93: registering with nfnetlink. [    0.815366] ip_tables: (C) 2000-2006 Netfilter Core Team [    0.815763] TCP cubic registered [    0.815773] Initializing XFRM netlink socket [    0.818404] NET: Registered protocol family 10 [    0.822365] ip6_tables: (C) 2000-2006 Netfilter Core Team [    0.822752] IPv6 over IPv4 tunneling driver [    0.826707] NET: Registered protocol family 17 [    0.826938] Registering the dns_resolver key type [    0.829092] registered taskstats version 1 [    0.829381] kmemleak: Kernel memory leak detector initialized [    0.829400] kmemleak: Automatic memory scanning thread started [    0.830361] XENBUS: Device with no driver: device/vbd/51712 [    0.830361] XENBUS: Device with no driver: device/vif/0 [    0.830361] XENBUS: Device with no driver: device/console/0 [    0.830361]   Magic number: 1:252:3141 [   13.345029] Freeing unused kernel memory: 756k freed [   13.345264] Write protecting the kernel read-only data: 10240k [   13.355174] Freeing unused kernel memory: 1720k freed [   13.357141] Freeing unused kernel memory: 1988k freed [   13.413416] exe used greatest stack depth: 5360 bytes left [   13.454058] exe used greatest stack depth: 5296 bytes left [   13.480525] udevd (1111): /proc/1111/oom_adj is deprecated, please use /proc/1111/oom_score_adj instead. Begin: Loading essential drivers... ... [   13.503292] blkid used greatest stack depth: 5088 bytes left [   13.522790] blkfront: xvda: barriers enabled [   13.534570]  xvda: xvda1 xvda2 < xvda5 > [   13.606023] blkid used greatest stack depth: 4752 bytes left [   13.631992] Initialising Xen virtual ethernet driver. Done. Begin: Running /scripts/init-premount ... Done. Begin: Mounting root file system... ... Begin: Running /scripts/local-top ... Done. Begin: Running /scripts/local-premount ... Done. [   13.885288] EXT4-fs (xvda1): mounted filesystem with ordered data mode. Opts: (null) Begin: Running /scripts/local-bottom ... Done. Done. Begin: Running /scripts/init-bottom ... Done. [   14.704681] hwclock used greatest stack depth: 4592 bytes left init: ureadahead main process (1211) terminated with status 5 [   14.812832] plymouthd used greatest stack depth: 4352 bytes left [   15.402077] cp used greatest stack depth: 4304 bytes left fsck from util-linux-ng 2.17.2 e2fsck 1.41.11 (14-Mar-2010) /dev/xvda1: clean, 56385/288576 files, 309277/1153280 blocks  * Enable login on console                                               [ OK ]  * Check for hostname changes...                                         [ OK ]    ...done.  * Setting sensors limits          No sensors found! Make sure you loaded all the kernel drivers you need. Try sensors-detect to find out which these are. No sensors found! Make sure you loaded all the kernel drivers you need. Try sensors-detect to find out which these are.                                                                          [ OK ]  * Initializing random number generator...                               [ OK ]  * Setting up X server socket directory /tmp/.X11-unix...                [ OK ]  * Setting up ICE socket directory /tmp/.ICE-unix...                     [ OK ]  * Not starting fancontrol; run pwmconfig first.  * Starting internet superserver inetd                                   [ OK ] Starting openntpd: ntpd.  * Starting Tomcat servlet engine tomcat6          Using CATALINA_BASE:   /var/lib/tomcat6 Using CATALINA_HOME:   /usr/share/tomcat6 Using CATALINA_TMPDIR: /tmp/tomcat6-tmp Using JRE_HOME:        /usr/lib/jvm/java-6-openjdk Using CLASSPATH:       /usr/share/tomcat6/bin/bootstrap.jar                                                                          [ OK ]  * Recording successful boot for GRUB    ...done.  * Running local boot scripts (/etc/rc.local)                            [ OK ] mountall: Plymouth command failed mountall: Disconnected from Plymouth uss004-dpm login: root Password: Linux uss004-dpm 2.6.37-rc4-kernel.org-domu-6.0.nopvhvm.bedge-debug #1 SMP Mon Nov 29 22:22:40 PST 2010 x86_64 GNU/Linux DPM-8800 6.0.16-01 Welcome to Ubuntu!  * Documentation:  https://help.ubuntu.com/ uss004-dpm:~# zsh 0 uss004-dpm  ~ #> mount /dev/xvda1 on / type ext4 (rw,errors=remount-ro) proc on /proc type proc (rw,noexec,nosuid,nodev) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev) none on /sys type sysfs (rw,noexec,nosuid,nodev) none on /sys/kernel/debug type debugfs (rw) none on /dev type tmpfs (rw,mode=0755) none on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620) none on /dev/shm type tmpfs (rw,nosuid,nodev) none on /var/run type tmpfs (rw,nosuid,mode=0755) none on /var/lock type tmpfs (rw,noexec,nosuid,nodev) none on /lib/init/rw type tmpfs (rw,nosuid,mode=0755) sys:/common/var/dpm/log on /var/log type nfs (rw,nolock,rsize=32768,wsize=32768,sloppy,addr=135.149.78.230) sys:/common/var/dpm/core on /var/core type nfs (rw,nolock,rsize=32768,wsize=32768,sloppy,addr=135.149.78.230) DomU config is attached -Bruce