From mboxrd@z Thu Jan 1 00:00:00 1970 From: Boris Derzhavets Subject: Re: Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging request Date: Mon, 15 Nov 2010 00:06:16 -0800 (PST) Message-ID: <166134.36458.qm@web56104.mail.re3.yahoo.com> References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0242377734==" Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Sander Eikelenboom , Bruce Edge Cc: Jeremy Fitzhardinge , xen-devel@lists.xensource.com, Konrad Rzeszutek Wilk List-Id: xen-devel@lists.xenproject.org --===============0242377734== Content-Type: multipart/alternative; boundary="0-1366127094-1289808376=:36458" --0-1366127094-1289808376=:36458 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Stack trace on f14 when working with NFS mount [=A0 218.984818] ------------[ cut here ]------------ [=A0 218.984834] kernel BUG at mm/mmap.c:2399! [=A0 218.984844] invalid opcode: 0000 [#1] SMP=20 [=A0 218.984857] last sysfs file: /sys/devices/system/cpu/cpu1/cache/index2= /shared_cpu_map [=A0 218.984872] CPU 1=20 [=A0 218.984879] Modules linked in: nfs fscache deflate zlib_deflate ctr ca= mellia cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic twofis= h_x86_64 twofish_common ecb xcbc cbc sha256_generic sha512_generic des_gene= ric cryptd aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_t= unnel tunnel4 xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport x= frm6_mode_ro xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp x= frm6_tunnel tunnel6 af_key nfsd lockd nfs_acl auth_rpcgss exportfs sunrpc i= p6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables ipv6= uinput xen_netfront microcode xen_blkfront [last unloaded: scsi_wait_scan] [=A0 218.985011]=20 [=A0 218.985011] Pid: 1566, comm: ls Not tainted 2.6.37-0.1.rc1.git8.xendom= 0.fc14.x86_64 #1 / [=A0 218.985011] RIP: e030:[]=A0 [] exi= t_mmap+0x10c/0x119 [=A0 218.985011] RSP: e02b:ffff8800774a9e18=A0 EFLAGS: 00010202 [=A0 218.985011] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 002000000= 0000000 [=A0 218.985011] RDX: 0000000000100004 RSI: ffff8800770ea1b8 RDI: ffffea000= 1a00230 [=A0 218.985011] RBP: ffff8800774a9e48 R08: ffff88007d045108 R09: 000000000= 000005a [=A0 218.985011] R10: ffffffff8100750f R11: ffffea000182b7b0 R12: ffff88007= 7dc6300 [=A0 218.985011] R13: ffff88007fa1b1e0 R14: ffff880077dc6368 R15: 000000000= 0000001 [=A0 218.985011] FS:=A0 00007f4a38dd17c0(0000) GS:ffff88007fa0d000(0000) kn= lGS:0000000000000000 [=A0 218.985011] CS:=A0 e033 DS: 0000 ES: 0000 CR0: 000000008005003b [=A0 218.985011] CR2: 00007f4a380a1940 CR3: 0000000001a03000 CR4: 000000000= 0002660 [=A0 218.985011] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 000000000= 0000000 [=A0 218.985011] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 000000000= 0000400 [=A0 218.985011] Process ls (pid: 1566, threadinfo ffff8800774a8000, task f= fff880003ca47c0) [=A0 218.985011] Stack: [=A0 218.985011]=A0 000000000000006b ffff88007fa1b1e0 ffff8800774a9e38 ffff= 880077dc6300 [=A0 218.985011]=A0 ffff880077dc6440 ffff880003ca4db0 ffff8800774a9e68 ffff= ffff810505fc [=A0 218.985011]=A0 ffff880003ca47c0 ffff880077dc6300 ffff8800774a9eb8 ffff= ffff81056747 [=A0 218.985011] Call Trace: [=A0 218.985011]=A0 [] mmput+0x65/0xd8 [=A0 218.985011]=A0 [] exit_mm+0x13e/0x14b [=A0 218.985011]=A0 [] do_exit+0x222/0x7c6 [=A0 218.985011]=A0 [] ? xen_restore_fl_direct_end+0x0/0x= 1 [=A0 218.985011]=A0 [] ? arch_local_irq_restore+0xb/0xd [=A0 218.985011]=A0 [] ? lockdep_sys_exit_thunk+0x35/0x67 [=A0 218.985011]=A0 [] do_group_exit+0x88/0xb6 [=A0 218.985011]=A0 [] sys_exit_group+0x17/0x1b [=A0 218.985011]=A0 [] system_call_fastpath+0x16/0x1b [=A0 218.985011] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 8= 9 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 0= 2 <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48=20 [=A0 218.985011] RIP=A0 [] exit_mmap+0x10c/0x119 [=A0 218.985011]=A0 RSP [=A0 218.985011] ---[ end trace 99b09fa378e85262 ]--- [=A0 218.985011] Fixing recursive fault but reboot is needed! Message from syslogd@fedora14 at Nov 15 11:03:20 ... =A0kernel:[=A0 218.984818] ------------[ cut here ]------------ Message from syslogd@fedora14 at Nov 15 11:03:20 ... =A0kernel:[=A0 218.984844] invalid opcode: 0000 [#1] SMP=20 Message from syslogd@fedora14 at Nov 15 11:03:20 ... =A0kernel:[=A0 218.984857] last sysfs file: /sys/devices/system/cpu/cpu1/ca= che/index2/shared_cpu_map Message from syslogd@fedora14 at Nov 15 11:03:20 ... =A0kernel:[=A0 218.985011] Stack: Message from syslogd@fedora14 at Nov 15 11:03:20 ... =A0kernel:[=A0 218.985011] Call Trace: Message from syslogd@fedora14 at Nov 15 11:03:20 ... =A0kernel:[=A0 218.985011] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 = 00 00 48 89 df e8 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 = 00 00 74 02 <0f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 4= 8=20 [=A0 259.093423] BUG: unable to handle kernel paging request at ffff880077d= 352a8 [=A0 259.093441] IP: [] ptep_set_access_flags+0x2b/0x51 [=A0 259.093456] PGD 1a04067 PUD 59c9067 PMD 5b88067 PTE 8010000077d35065 [=A0 259.093472] Oops: 0003 [#2] SMP=20 [=A0 259.093481] last sysfs file: /sys/devices/system/cpu/cpu1/cache/index2= /shared_cpu_map [=A0 259.093493] CPU 1=20 [=A0 259.093498] Modules linked in: nfs fscache deflate zlib_deflate ctr ca= mellia cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic twofis= h_x86_64 twofish_common ecb xcbc cbc sha256_generic sha512_generic des_gene= ric cryptd aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_t= unnel tunnel4 xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport x= frm6_mode_ro xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp x= frm6_tunnel tunnel6 af_key nfsd lockd nfs_acl auth_rpcgss exportfs sunrpc i= p6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables ipv6= uinput xen_netfront microcode xen_blkfront [last unloaded: scsi_wait_scan] [=A0 259.093652]=20 [=A0 259.093658] Pid: 1567, comm: abrtd Tainted: G=A0=A0=A0=A0=A0 D=A0=A0= =A0=A0 2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 / [=A0 259.093669] RIP: e030:[]=A0 [] pte= p_set_access_flags+0x2b/0x51 [=A0 259.093683] RSP: e02b:ffff8800770e7bf8=A0 EFLAGS: 00010202 [=A0 259.093690] RAX: 80000001bf75f101 RBX: ffff880077521400 RCX: 80000001b= f75f167 [=A0 259.093699] RDX: ffff880077d352a8 RSI: 00007fb9b9255ad0 RDI: ffff88007= 7521400 [=A0 259.093708] RBP: ffff8800770e7c28 R08: 0000000000000001 R09: 158000000= 0000000 [=A0 259.093717] R10: ffffffff8100750f R11: ffff880077dc5800 R12: 00007fb9b= 9255ad0 [=A0 259.093726] R13: 0000000000000001 R14: ffff880003f2f9f8 R15: ffff88007= 7d352a8 [=A0 259.093737] FS:=A0 00007fb9b9255800(0000) GS:ffff88007fa0d000(0000) kn= lGS:0000000000000000 [=A0 259.093747] CS:=A0 e033 DS: 0000 ES: 0000 CR0: 000000008005003b [=A0 259.093755] CR2: ffff880077d352a8 CR3: 00000000043c8000 CR4: 000000000= 0002660 [=A0 259.093764] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 000000000= 0000000 [=A0 259.093773] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 000000000= 0000400 [=A0 259.093783] Process abrtd (pid: 1567, threadinfo ffff8800770e6000, tas= k ffff880003d2c7c0) [=A0 259.093800] Stack: [=A0 259.093807]=A0 ffffea00018382b0 0000000000000000 0000000000000034 0000= 000000000000 [=A0 259.093829]=A0 ffff880077521400 0000000000000000 ffff8800770e7cb8 ffff= ffff81104a57 [=A0 259.093851]=A0 ffffffff810050a3 ffffffff00000001 ffff880004307e48 ffff= 8800770e7ca8 [=A0 259.093873] Call Trace: [=A0 259.093885]=A0 [] do_wp_page+0x241/0x53d [=A0 259.093899]=A0 [] ? xen_pte_val+0x6a/0x6c [=A0 259.093911]=A0 [] ? __raw_callee_save_xen_pte_val+0x= 11/0x1e [=A0 259.093926]=A0 [] ? xen_restore_fl_direct_end+0x0/0x= 1 [=A0 259.093941]=A0 [] ? handle_mm_fault+0x6ea/0x7af [=A0 259.093954]=A0 [] handle_mm_fault+0x73b/0x7af [=A0 259.093969]=A0 [] ? down_read_trylock+0x44/0x4e [=A0 259.093983]=A0 [] do_page_fault+0x363/0x385 [=A0 259.093996]=A0 [] ? xen_force_evtchn_callback+0xd/0x= f [=A0 259.094011]=A0 [] ? check_events+0x12/0x20 [=A0 259.094025]=A0 [] ? trace_hardirqs_off_thunk+0x3a/0x= 3c [=A0 259.094039]=A0 [] page_fault+0x25/0x30 [=A0 259.094053]=A0 [] ? __put_user_4+0x1d/0x30 [=A0 259.094066]=A0 [] ? schedule_tail+0x61/0x65 [=A0 259.094079]=A0 [] ret_from_fork+0x13/0x80 [=A0 259.094089] Code: 55 48 89 e5 41 55 41 54 53 48 83 ec 18 0f 1f 44 00 0= 0 48 39 0a 48 89 fb 49 89 f4 0f 95 c0 45 85 c0 44 0f b6 e8 74 1c 84 c0 74 1= 8 <48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 bb=20 [=A0 259.094149] RIP=A0 [] ptep_set_access_flags+0x2b/0x5= 1 [=A0 259.094149]=A0 RSP [=A0 259.094149] CR2: ffff880077d352a8 [=A0 259.094149] ---[ end trace 99b09fa378e85263 ]--- Message from syslogd@fedora14 at Nov 15 11:04:00 ... =A0kernel:[=A0 259.093472] Oops: 0003 [#2] SMP=20 Message from syslogd@fedora14 at Nov 15 11:04:00 ... =A0kernel:[=A0 259.093481] last sysfs file: /sys/devices/system/cpu/cpu1/ca= che/index2/shared_cpu_map Message from syslogd@fedora14 at Nov 15 11:04:00 ... =A0kernel:[=A0 259.093800] Stack: Message from syslogd@fedora14 at Nov 15 11:04:00 ... =A0kernel:[=A0 259.093873] Call Trace: Message from syslogd@fedora14 at Nov 15 11:04:00 ... =A0kernel:[=A0 259.094089] Code: 55 48 89 e5 41 55 41 54 53 48 83 ec 18 0f = 1f 44 00 00 48 39 0a 48 89 fb 49 89 f4 0f 95 c0 45 85 c0 44 0f b6 e8 74 1c = 84 c0 74 18 <48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 b= b=20 Message from syslogd@fedora14 at Nov 15 11:04:00 ... =A0kernel:[=A0 259.094149] CR2: ffff880077d352a8 --- On Sun, 11/14/10, Bruce Edge wrote: From: Bruce Edge Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to hand= le kernel paging request To: "Sander Eikelenboom" Cc: "Boris Derzhavets" , xen-devel@lists.xensource.c= om, "Jeremy Fitzhardinge" , "Konrad Rzeszutek Wilk" Date: Sunday, November 14, 2010, 4:35 PM On Sun, Nov 14, 2010 at 8:56 AM, Sander Eikelenboom wrote: > Hmmm have you tried do do a lot of I/O with something else as NFS ? > That would perhaps pinpoint it to NFS doing something not completely comp= atible with Xen. I have my own suspicions regarding the more recent NFS clients. Post 10.04 Ubuntu variants do not tolerate large NFS transfers even without Xen. Any more than a few 100 Megs and you start getting 'task blocked for more than 120 sec..." messages along with stack traces showing part of the NFS call stack. Perhaps a parallel effort could be to test the 2.6.37-rc1 kernel with something other than NFS for remote filesystems. I'll see if I get the same problems with glusterfs. -Bruce > > I'm not using NFS (I still use file: based guests, and i use glusterfs (f= use based userspace cluster fs) to share diskspace to domU's via ethernet). > I tried NFS in the past, but had some troubles setting it up, and even mo= re problems with disconnects. > > I haven't seen any "unable to handle page request" problems with my mix o= f guest kernels, which includes some 2.6.37-rc1 kernels. > > -- > > Sander > > > > > > Sunday, November 14, 2010, 5:37:59 PM, you wrote: > >> I've tested F14 DomU (kernel vmlinuz-2.6.37-0.1.rc1.git8.xendom0.fc14.x8= 6_64) as NFS client and Xen 4.0.1 F14 Dom0 (kernel vmlinuz-2.6.32.25-172.xe= ndom0.fc14.x86_64) as NFS server . Copied 700 MB ISO images from NFS folder= at Dom0 to DomU and scp'ed them back to Dom0. During about 30 - 40 min Dom= U ran pretty stable , regardless kernel crash as "unable to handle page req= uest" was reported once by F14 DomU, but it didn't actually crash DomU. Sam= e excersises with replacement F14 by Ubuntu 10.04 Server results DomU crash= in about several minutes. Dom0's instances dual boot on same development b= ox ( Q9500,ASUS P5Q3,8GB) > >> Boris. > >> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk wro= te: > >> From: Konrad Rzeszutek Wilk >> Subject: Re: [Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to h= andle kernel paging request >> To: "Sander Eikelenboom" >> Cc: "Boris Derzhavets" , xen-devel@lists.xensourc= e.com, "Bruce Edge" , "Jeremy Fitzhardinge" >> Date: Friday, November 12, 2010, 12:01 PM > >> On Fri, Nov 12, 2010 at 05:27:43PM +0100, Sander Eikelenboom wrote: >>> Hi Bruce, >>> >>> Perhaps handpick some kernels before and after the pulls of the xen pat= ches (pv-on-hvm etc) to begin with ? >>> When you let git choose, especially with rc-1 kernels, you will end up = with kernels in between patch series, resulting in panics. > >> Well, just the bare-bone boot of PV guests with nothing fancy ought to w= ork. > >> But that is the theory and .. >>> > The git bisecting is slow going. I've never tried that before and I'm= a git >>> > rookie. >>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect range and my first 2 bisec= ts all >>> > panic at boot so I'm obviously doing something wrong. >>> > I'll RTFM a bit more and keep at it. > >> .. as Bruce experiences this is not the case. Hmm.. > >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel > > > >> > > > > -- > Best regards, > =A0Sander =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0mailto:l= inux@eikelenboom.it > > _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel =0A=0A=0A --0-1366127094-1289808376=:36458 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
Stack trace on f14 when working with NFS moun= t

[  218.984818] ------------[ cut here ]------------
[ = ; 218.984834] kernel BUG at mm/mmap.c:2399!
[  218.984844] invalid = opcode: 0000 [#1] SMP
[  218.984857] last sysfs file: /sys/devices= /system/cpu/cpu1/cache/index2/shared_cpu_map
[  218.984872] CPU 1 <= br>[  218.984879] Modules linked in: nfs fscache deflate zlib_deflate = ctr camellia cast5 rmd160 crypto_null ccm serpent blowfish twofish_generic = twofish_x86_64 twofish_common ecb xcbc cbc sha256_generic sha512_generic de= s_generic cryptd aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_mode_beet x= frm4_tunnel tunnel4 xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_trans= port xfrm6_mode_ro xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm_ip= comp xfrm6_tunnel tunnel6 af_key nfsd lockd nfs_acl auth_rpcgss exportfs sunrpc ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_ta= bles ipv6 uinput xen_netfront microcode xen_blkfront [last unloaded: scsi_w= ait_scan]
[  218.985011]
[  218.985011] Pid: 1566, comm: l= s Not tainted 2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 /
[  218.9= 85011] RIP: e030:[<ffffffff8110ada1>]  [<ffffffff8110ada1>= ] exit_mmap+0x10c/0x119
[  218.985011] RSP: e02b:ffff8800774a9e18&n= bsp; EFLAGS: 00010202
[  218.985011] RAX: 0000000000000000 RBX: 000= 0000000000000 RCX: 0020000000000000
[  218.985011] RDX: 00000000001= 00004 RSI: ffff8800770ea1b8 RDI: ffffea0001a00230
[  218.985011] RB= P: ffff8800774a9e48 R08: ffff88007d045108 R09: 000000000000005a
[  = 218.985011] R10: ffffffff8100750f R11: ffffea000182b7b0 R12: ffff880077dc63= 00
[  218.985011] R13: ffff88007fa1b1e0 R14: ffff880077dc6368 R15: = 0000000000000001
[  218.985011] FS:  00007f4a38dd17c0(0000) GS:ffff88007fa0d000(0000) knlGS:0000000000000000[  218.985011] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b=
[  218.985011] CR2: 00007f4a380a1940 CR3: 0000000001a03000 CR4: 00= 00000000002660
[  218.985011] DR0: 0000000000000000 DR1: 0000000000= 000000 DR2: 0000000000000000
[  218.985011] DR3: 0000000000000000 D= R6: 00000000ffff0ff0 DR7: 0000000000000400
[  218.985011] Process l= s (pid: 1566, threadinfo ffff8800774a8000, task ffff880003ca47c0)
[ = ; 218.985011] Stack:
[  218.985011]  000000000000006b ffff8800= 7fa1b1e0 ffff8800774a9e38 ffff880077dc6300
[  218.985011]  fff= f880077dc6440 ffff880003ca4db0 ffff8800774a9e68 ffffffff810505fc
[ = 218.985011]  ffff880003ca47c0 ffff880077dc6300 ffff8800774a9eb8 fffff= fff81056747
[  218.985011] Call Trace:
[  218.985011] = [<ffffffff810505fc>] mmput+0x65/0xd8
[  218.985011]  [<ffffffff81056747>] exit_mm+0x13e/0x14b
[ = ; 218.985011]  [<ffffffff81056976>] do_exit+0x222/0x7c6
[&nbs= p; 218.985011]  [<ffffffff8100750f>] ? xen_restore_fl_direct_end= +0x0/0x1
[  218.985011]  [<ffffffff8107ea7c>] ? arch_loc= al_irq_restore+0xb/0xd
[  218.985011]  [<ffffffff814b3949&g= t;] ? lockdep_sys_exit_thunk+0x35/0x67
[  218.985011]  [<ff= ffffff810571b0>] do_group_exit+0x88/0xb6
[  218.985011]  [&= lt;ffffffff810571f5>] sys_exit_group+0x17/0x1b
[  218.985011]&nb= sp; [<ffffffff8100acf2>] system_call_fastpath+0x16/0x1b
[  21= 8.985011] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89 df e8= 0d e9 ff ff 48 85 c0 48 89 c3 75 f0 49 83 bc 24 98 01 00 00 00 74 02 <0= f> 0b 48 83 c4 18 5b 41 5c 41 5d c9 c3 55 48 89 e5 41 54 53 48
[&nbs= p; 218.985011] RIP  [<ffffffff8110ada1>] exit_mmap+0x10c/0x119
[  218.985011]  RSP <ffff8800774a9e1= 8>
[  218.985011] ---[ end trace 99b09fa378e85262 ]---
[ = ; 218.985011] Fixing recursive fault but reboot is needed!

Message f= rom syslogd@fedora14 at Nov 15 11:03:20 ...
 kernel:[  218.984= 818] ------------[ cut here ]------------

Message from syslogd@fedor= a14 at Nov 15 11:03:20 ...
 kernel:[  218.984844] invalid opco= de: 0000 [#1] SMP

Message from syslogd@fedora14 at Nov 15 11:03:20 = ...
 kernel:[  218.984857] last sysfs file: /sys/devices/syste= m/cpu/cpu1/cache/index2/shared_cpu_map

Message from syslogd@fedora14= at Nov 15 11:03:20 ...
 kernel:[  218.985011] Stack:

M= essage from syslogd@fedora14 at Nov 15 11:03:20 ...
 kernel:[ = 218.985011] Call Trace:

Message from syslogd@fedora14 at Nov 15 11:= 03:20 ...
 kernel:[  218.985011] Code: 8d 7d 18 e8 c3 8a 00 00 41 c7 45 08 00 00 00 00 48 89 df e8 0d e9 ff ff 48 85 c0 48 89 c3= 75 f0 49 83 bc 24 98 01 00 00 00 74 02 <0f> 0b 48 83 c4 18 5b 41 5c = 41 5d c9 c3 55 48 89 e5 41 54 53 48
[  259.093423] BUG: unable to = handle kernel paging request at ffff880077d352a8
[  259.093441] IP:= [<ffffffff81037648>] ptep_set_access_flags+0x2b/0x51
[  259.= 093456] PGD 1a04067 PUD 59c9067 PMD 5b88067 PTE 8010000077d35065
[ = 259.093472] Oops: 0003 [#2] SMP
[  259.093481] last sysfs file: /= sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map
[  259.0934= 93] CPU 1
[  259.093498] Modules linked in: nfs fscache deflate zl= ib_deflate ctr camellia cast5 rmd160 crypto_null ccm serpent blowfish twofi= sh_generic twofish_x86_64 twofish_common ecb xcbc cbc sha256_generic sha512= _generic des_generic cryptd aes_x86_64 aes_generic ah6 ah4 esp6 esp4 xfrm4_= mode_beet xfrm4_tunnel tunnel4 xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro xfrm6_mode_beet xf= rm6_mode_tunnel ipcomp ipcomp6 xfrm_ipcomp xfrm6_tunnel tunnel6 af_key nfsd= lockd nfs_acl auth_rpcgss exportfs sunrpc ip6t_REJECT nf_conntrack_ipv6 nf= _defrag_ipv6 ip6table_filter ip6_tables ipv6 uinput xen_netfront microcode = xen_blkfront [last unloaded: scsi_wait_scan]
[  259.093652]
[&n= bsp; 259.093658] Pid: 1567, comm: abrtd Tainted: G    &= nbsp; D     2.6.37-0.1.rc1.git8.xendom0.fc14.x86_64 #1 = /
[  259.093669] RIP: e030:[<ffffffff81037648>]  [<ff= ffffff81037648>] ptep_set_access_flags+0x2b/0x51
[  259.093683] = RSP: e02b:ffff8800770e7bf8  EFLAGS: 00010202
[  259.093690] RA= X: 80000001bf75f101 RBX: ffff880077521400 RCX: 80000001bf75f167
[  = 259.093699] RDX: ffff880077d352a8 RSI: 00007fb9b9255ad0 RDI: ffff8800775214= 00
[  259.093708] RBP: ffff8800770e7c28 R08: 0000000000000001 R09: 1580000000000000
[  259.093717] R10: fffffff= f8100750f R11: ffff880077dc5800 R12: 00007fb9b9255ad0
[  259.093726= ] R13: 0000000000000001 R14: ffff880003f2f9f8 R15: ffff880077d352a8
[&nb= sp; 259.093737] FS:  00007fb9b9255800(0000) GS:ffff88007fa0d000(0000) = knlGS:0000000000000000
[  259.093747] CS:  e033 DS: 0000 ES: 0= 000 CR0: 000000008005003b
[  259.093755] CR2: ffff880077d352a8 CR3:= 00000000043c8000 CR4: 0000000000002660
[  259.093764] DR0: 0000000= 000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  259.093773= ] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[&nb= sp; 259.093783] Process abrtd (pid: 1567, threadinfo ffff8800770e6000, task= ffff880003d2c7c0)
[  259.093800] Stack:
[  259.093807]&nbs= p; ffffea00018382b0 0000000000000000 0000000000000034 0000000000000000
[=   259.093829]  ffff880077521400 0000000000000000 ffff8800770e7cb8 ffffffff81104a57
[  259.093851]  ffffffff810= 050a3 ffffffff00000001 ffff880004307e48 ffff8800770e7ca8
[  259.093= 873] Call Trace:
[  259.093885]  [<ffffffff81104a57>] do= _wp_page+0x241/0x53d
[  259.093899]  [<ffffffff810050a3>= ] ? xen_pte_val+0x6a/0x6c
[  259.093911]  [<ffffffff8100463= 5>] ? __raw_callee_save_xen_pte_val+0x11/0x1e
[  259.093926]&nbs= p; [<ffffffff8100750f>] ? xen_restore_fl_direct_end+0x0/0x1
[ = ; 259.093941]  [<ffffffff81106491>] ? handle_mm_fault+0x6ea/0x7a= f
[  259.093954]  [<ffffffff811064e2>] handle_mm_fault+0= x73b/0x7af
[  259.093969]  [<ffffffff81073597>] ? down_r= ead_trylock+0x44/0x4e
[  259.093983]  [<ffffffff814b7aa4>= ;] do_page_fault+0x363/0x385
[  259.093996]  [<ffffffff8100= 6f59>] ? xen_force_evtchn_callback+0xd/0xf
[  259.094011]  [<ffffffff81007522>] ? check_events+0x12/0x20
[=   259.094025]  [<ffffffff814b3912>] ? trace_hardirqs_off_th= unk+0x3a/0x3c
[  259.094039]  [<ffffffff814b4ad5>] page_= fault+0x25/0x30
[  259.094053]  [<ffffffff8125403d>] ? _= _put_user_4+0x1d/0x30
[  259.094066]  [<ffffffff8104bf66>= ;] ? schedule_tail+0x61/0x65
[  259.094079]  [<ffffffff8100= abf3>] ret_from_fork+0x13/0x80
[  259.094089] Code: 55 48 89 e5 = 41 55 41 54 53 48 83 ec 18 0f 1f 44 00 00 48 39 0a 48 89 fb 49 89 f4 0f 95 = c0 45 85 c0 44 0f b6 e8 74 1c 84 c0 74 18 <48> 89 0a 48 8b 3f 0f 1f 8= 0 00 00 00 00 4c 89 e6 48 89 df e8 bb
[  259.094149] RIP  [&l= t;ffffffff81037648>] ptep_set_access_flags+0x2b/0x51
[  259.0941= 49]  RSP <ffff8800770e7bf8>
[  259.094149] CR2: ffff8800= 77d352a8
[  259.094149] ---[ end trace 99b09fa378e85263 ]---

Message from syslogd@fedora14 at Nov 15 11:04:00 ...
 = kernel:[  259.093472] Oops: 0003 [#2] SMP

Message from syslogd= @fedora14 at Nov 15 11:04:00 ...
 kernel:[  259.093481] last s= ysfs file: /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map

= Message from syslogd@fedora14 at Nov 15 11:04:00 ...
 kernel:[ = ; 259.093800] Stack:

Message from syslogd@fedora14 at Nov 15 11:04:0= 0 ...
 kernel:[  259.093873] Call Trace:

Message from s= yslogd@fedora14 at Nov 15 11:04:00 ...
 kernel:[  259.094089] = Code: 55 48 89 e5 41 55 41 54 53 48 83 ec 18 0f 1f 44 00 00 48 39 0a 48 89 = fb 49 89 f4 0f 95 c0 45 85 c0 44 0f b6 e8 74 1c 84 c0 74 18 <48> 89 0= a 48 8b 3f 0f 1f 80 00 00 00 00 4c 89 e6 48 89 df e8 bb

Message fro= m syslogd@fedora14 at Nov 15 11:04:00 ...
 kernel:[  259.09414= 9] CR2: ffff880077d352a8


--- On Sun, 11/14/10, Bruce Edge <bruce.edge@gmail.com> wrote:

From: Bruce Edge <bruce.edge@gmail.com>
Subject: Re: [= Xen-devel] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel pagi= ng request
To: "Sander Eikelenboom" <linux@eikelenboom.it>
Cc: = "Boris Derzhavets" <bderzhavets@yahoo.com>, xen-devel@lists.xensource= .com, "Jeremy Fitzhardinge" <jeremy@goop.org>, "Konrad Rzeszutek Wilk= " <konrad.wilk@oracle.com>
Date: Sunday, November 14, 2010, 4:35 P= M

On Sun, Nov 14, 2010 at 8:56 AM, Sander E= ikelenboom
<linux@eikelenboom.it> wrote:
>= ; Hmmm have you tried do do a lot of I/O with something else as NFS ?
&g= t; That would perhaps pinpoint it to NFS doing something not completely compatible with Xen.

I have my own suspicions regarding the more re= cent NFS clients. Post
10.04 Ubuntu variants do not tolerate large NFS t= ransfers even without
Xen. Any more than a few 100 Megs and you start ge= tting 'task blocked
for more than 120 sec..." messages along with stack = traces showing
part of the NFS call stack.
Perhaps a parallel effort = could be to test the 2.6.37-rc1 kernel with
something other than NFS for= remote filesystems. I'll see if I get the
same problems with glusterfs.=

-Bruce

>
> I'm not using NFS (I still use file: bas= ed guests, and i use glusterfs (fuse based userspace cluster fs) to share d= iskspace to domU's via ethernet).
> I tried NFS in the past, but had = some troubles setting it up, and even more problems with disconnects.
&g= t;
> I haven't seen any "unable to handle page request" problems with= my mix of guest kernels, which includes some 2.6.37-rc1 kernels.
>
> --
>
> Sander
>
>
>=
>
>
> Sunday, November 14, 2010, 5:37:59 PM, you wrote:<= br>>
>> I've tested F14 DomU (kernel vmlinuz-2.6.37-0.1.rc1.git= 8.xendom0.fc14.x86_64) as NFS client and Xen 4.0.1 F14 Dom0 (kernel vmlinuz= -2.6.32.25-172.xendom0.fc14.x86_64) as NFS server . Copied 700 MB ISO image= s from NFS folder at Dom0 to DomU and scp'ed them back to Dom0. During abou= t 30 - 40 min DomU ran pretty stable , regardless kernel crash as "unable t= o handle page request" was reported once by F14 DomU, but it didn't actuall= y crash DomU. Same excersises with replacement F14 by Ubuntu 10.04 Server r= esults DomU crash in about several minutes. Dom0's instances dual boot on s= ame development box ( Q9500,ASUS P5Q3,8GB)
>
>> Boris.
&g= t;
>> --- On Fri, 11/12/10, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
>
>> From: Konrad Rzeszutek Wilk <
konrad.wilk@oracle.com>
>> Subject: Re: [Xen-devel= ] Re: 2.6.37-rc1 mainline domU - BUG: unable to handle kernel paging reques= t
>> To: "Sander Eikelenboom" <linux@eikelenboom= .it>
>> Cc: "Boris Derzhavets" <bderzha= vets@yahoo.com>, xen-devel@lists.xe= nsource.com, "Bruce Edge" <bruce.edge@gmail.com&g= t;, "Jeremy Fitzhardinge" <jeremy@goop.org>
>> Date: Friday, November 12, 20= 10, 12:01 PM
>
>> On Fri, Nov 12, 2010 at 05:27:43PM +0100, = Sander Eikelenboom wrote:
>>> Hi Bruce,
>>>
>= >> Perhaps handpick some kernels before and after the pulls of the xe= n patches (pv-on-hvm etc) to begin with ?
>>> When you let git = choose, especially with rc-1 kernels, you will end up with kernels in betwe= en patch series, resulting in panics.
>
>> Well, just the ba= re-bone boot of PV guests with nothing fancy ought to work.
>
>= > But that is the theory and ..
>>> > The git bisecting i= s slow going. I've never tried that before and I'm a git
>>> &g= t; rookie.
>>> > I picked 2.6.36 - 2.6.37-rc1 as the bisect = range and my first 2 bisects all
>>> > panic at boot so I'm obviously doing something wrong.
>>> > I'll RTFM a bit more= and keep at it.
>
>> .. as Bruce experiences this is not th= e case. Hmm..
>
>> _________________________________________= ______
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.co= m/xen-devel
>
>
>
>>
>
>
>=
> --
> Best regards,
>  Sander      =                     &nbs= p;mailto:linux@eikelenboom.it
>
>

__= _____________________________________________
Xen-devel mailing list
= Xen-devel@lists.xensource.com
http://lists.xe= nsource.com/xen-devel

=0A= =0A --0-1366127094-1289808376=:36458-- --===============0242377734== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --===============0242377734==--