* kernel BUG at arch/x86/xen/mmu.c:1860!
@ 2010-12-26 8:16 Teck Choon Giam
2010-12-27 15:53 ` Konrad Rzeszutek Wilk
` (2 more replies)
0 siblings, 3 replies; 85+ messages in thread
From: Teck Choon Giam @ 2010-12-26 8:16 UTC (permalink / raw)
To: xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 4396 bytes --]
Hi,
Information: CentOS 5.5 x86_64
dom0: latest xen/stable-2.6.32.x pvops git commit
75cc13f5aa29b4f3227d269ca165dfa8937c94fe
xen version: xen-4.0.2-rc1-pre from xen-4.0-testing.hg changeset 21422
While doing LVM snapshot for migration and get the following:
Dec 26 15:58:29 xen01 kernel: ------------[ cut here ]------------
Dec 26 15:58:29 xen01 kernel: kernel BUG at arch/x86/xen/mmu.c:1860!
Dec 26 15:58:29 xen01 kernel: invalid opcode: 0000 [#1] SMP
Dec 26 15:58:29 xen01 kernel: last sysfs file: /sys/block/dm-26/dev
Dec 26 15:58:29 xen01 kernel: CPU 0
Dec 26 15:58:29 xen01 kernel: Modules linked in: ipt_MASQUERADE iptable_nat
nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT
xt_tcpudp xt_physdev iptable_filter ip_tables x_tables bridge stp be2iscsi
iscsi_tcp bnx2i cnic uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi
scsi_transport_iscsi loop dm_multipath scsi_dh video backlight output sbs
sbshc power_meter hwmon battery acpi_memhotplug xen_acpi_memhotplug ac
parport_pc lp parport sg tpm_tis tpm tpm_bios button i2c_i801 i2c_core
iTCO_wdt e1000e shpchp pcspkr dm_snapshot dm_zero dm_mirror dm_region_hash
dm_log dm_mod usb_storage ahci libata sd_mod scsi_mod raid1 ext3 jbd
uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
Dec 26 15:58:29 xen01 kernel: Pid: 27998, comm: udevd Not tainted
2.6.32.27-0.xen.pvops.choon.centos5 #1 S3420GP
Dec 26 15:58:29 xen01 kernel: RIP: e030:[<ffffffff8100cb5b>]
[<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
Dec 26 15:58:29 xen01 kernel: RSP: e02b:ffff88003bc3bdc8 EFLAGS: 00010282
Dec 26 15:58:29 xen01 kernel: RAX: 00000000ffffffea RBX: 0000000000017605
RCX: 00000000000000bb
Dec 26 15:58:29 xen01 kernel: RDX: 00000000deadbeef RSI: 00000000deadbeef
RDI: 00000000deadbeef
Dec 26 15:58:29 xen01 kernel: RBP: ffff88003bc3bde8 R08: 0000000000000028
R09: ffff880000000000
Dec 26 15:58:29 xen01 kernel: R10: 00000000deadbeef R11: 00007fdb5665e600
R12: 0000000000000003
Dec 26 15:58:30 xen01 kernel: R13: 0000000000017605 R14: ffff880012ee0780
R15: 00007fdb56224268
Dec 26 15:58:30 xen01 kernel: FS: 00007fdb56fed710(0000)
GS:ffff88002804f000(0000) knlGS:0000000000000000
Dec 26 15:58:30 xen01 kernel: CS: e033 DS: 0000 ES: 0000 CR0:
000000008005003b
Dec 26 15:58:30 xen01 kernel: CR2: 00007fdb56224268 CR3: 000000003addb000
CR4: 0000000000002660
Dec 26 15:58:30 xen01 kernel: DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000
Dec 26 15:58:30 xen01 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
DR7: 0000000000000400
Dec 26 15:58:30 xen01 kernel: Process udevd (pid: 27998, threadinfo
ffff88003bc3a000, task ffff880012ee0780)
Dec 26 15:58:30 xen01 kernel: Stack:
Dec 26 15:58:30 xen01 kernel: 0000000000000000 0000000000424121
000000013f00ae20 0000000000017605
Dec 26 15:58:30 xen01 kernel: <0> ffff88003bc3be08 ffffffff8100e07c
ffff88003a3c2580 ffff880034bb6588
Dec 26 15:58:30 xen01 kernel: <0> ffff88003bc3be18 ffffffff8100e0af
ffff88003bc3be58 ffffffff810a402f
Dec 26 15:58:31 xen01 kernel: Call Trace:
Dec 26 15:58:31 xen01 kernel: [<ffffffff8100e07c>]
xen_alloc_ptpage+0x64/0x69
Dec 26 15:58:31 xen01 kernel: [<ffffffff8100e0af>] xen_alloc_pte+0xe/0x10
Dec 26 15:58:31 xen01 kernel: [<ffffffff810a402f>] __pte_alloc+0x70/0xce
Dec 26 15:58:31 xen01 kernel: [<ffffffff810a41cd>]
handle_mm_fault+0x140/0x8b9
Dec 26 15:58:31 xen01 kernel: [<ffffffff810d2ecc>] ? d_kill+0x3a/0x42
Dec 26 15:58:31 xen01 kernel: [<ffffffff810c4cd1>] ? __fput+0x1cb/0x1da
Dec 26 15:58:31 xen01 kernel: [<ffffffff8131be4d>]
do_page_fault+0x252/0x2e2
Dec 26 15:58:31 xen01 kernel: [<ffffffff81319dd5>] page_fault+0x25/0x30
Dec 26 15:58:31 xen01 kernel: Code: 48 b8 ff ff ff ff ff ff ff 7f 48 21 c2
48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 e9 c7 ff
ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74 5b
48
Dec 26 15:58:31 xen01 kernel: RIP [<ffffffff8100cb5b>]
pin_pagetable_pfn+0x53/0x59
Dec 26 15:58:31 xen01 kernel: RSP <ffff88003bc3bdc8>
Dec 26 15:58:31 xen01 kernel: ---[ end trace 540bcf6f0170242d ]---
Triggered BUG() in line 1860:
static void pin_pagetable_pfn(unsigned cmd, unsigned long pfn)
{
struct mmuext_op op;
op.cmd = cmd;
op.arg1.mfn = pfn_to_mfn(pfn);
if (HYPERVISOR_mmuext_op(&op, 1, NULL, DOMID_SELF))
BUG(); <<THIS ONE?
}
Any idea?
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 4731 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2010-12-26 8:16 kernel BUG at arch/x86/xen/mmu.c:1860! Teck Choon Giam
@ 2010-12-27 15:53 ` Konrad Rzeszutek Wilk
2010-12-27 22:14 ` Teck Choon Giam
2011-01-04 13:48 ` Ian Campbell
2011-01-13 14:28 ` tjaouen
2 siblings, 1 reply; 85+ messages in thread
From: Konrad Rzeszutek Wilk @ 2010-12-27 15:53 UTC (permalink / raw)
To: Teck Choon Giam; +Cc: xen-devel
On Sun, Dec 26, 2010 at 04:16:16PM +0800, Teck Choon Giam wrote:
> Hi,
>
> Information: CentOS 5.5 x86_64
> dom0: latest xen/stable-2.6.32.x pvops git commit
> 75cc13f5aa29b4f3227d269ca165dfa8937c94fe
> xen version: xen-4.0.2-rc1-pre from xen-4.0-testing.hg changeset 21422
>
> While doing LVM snapshot for migration and get the following:
>
> Dec 26 15:58:29 xen01 kernel: ------------[ cut here ]------------
> Dec 26 15:58:29 xen01 kernel: kernel BUG at arch/x86/xen/mmu.c:1860!
> Dec 26 15:58:29 xen01 kernel: invalid opcode: 0000 [#1] SMP
> Dec 26 15:58:29 xen01 kernel: last sysfs file: /sys/block/dm-26/dev
> Dec 26 15:58:29 xen01 kernel: CPU 0
> Dec 26 15:58:29 xen01 kernel: Modules linked in: ipt_MASQUERADE iptable_nat
> nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT
> xt_tcpudp xt_physdev iptable_filter ip_tables x_tables bridge stp be2iscsi
> iscsi_tcp bnx2i cnic uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi
> scsi_transport_iscsi loop dm_multipath scsi_dh video backlight output sbs
> sbshc power_meter hwmon battery acpi_memhotplug xen_acpi_memhotplug ac
> parport_pc lp parport sg tpm_tis tpm tpm_bios button i2c_i801 i2c_core
> iTCO_wdt e1000e shpchp pcspkr dm_snapshot dm_zero dm_mirror dm_region_hash
> dm_log dm_mod usb_storage ahci libata sd_mod scsi_mod raid1 ext3 jbd
> uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
> Dec 26 15:58:29 xen01 kernel: Pid: 27998, comm: udevd Not tainted
> 2.6.32.27-0.xen.pvops.choon.centos5 #1 S3420GP
> Dec 26 15:58:29 xen01 kernel: RIP: e030:[<ffffffff8100cb5b>]
> [<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
> Dec 26 15:58:29 xen01 kernel: RSP: e02b:ffff88003bc3bdc8 EFLAGS: 00010282
> Dec 26 15:58:29 xen01 kernel: RAX: 00000000ffffffea RBX: 0000000000017605
> RCX: 00000000000000bb
> Dec 26 15:58:29 xen01 kernel: RDX: 00000000deadbeef RSI: 00000000deadbeef
> RDI: 00000000deadbeef
> Dec 26 15:58:29 xen01 kernel: RBP: ffff88003bc3bde8 R08: 0000000000000028
> R09: ffff880000000000
> Dec 26 15:58:29 xen01 kernel: R10: 00000000deadbeef R11: 00007fdb5665e600
> R12: 0000000000000003
> Dec 26 15:58:30 xen01 kernel: R13: 0000000000017605 R14: ffff880012ee0780
> R15: 00007fdb56224268
> Dec 26 15:58:30 xen01 kernel: FS: 00007fdb56fed710(0000)
> GS:ffff88002804f000(0000) knlGS:0000000000000000
> Dec 26 15:58:30 xen01 kernel: CS: e033 DS: 0000 ES: 0000 CR0:
> 000000008005003b
> Dec 26 15:58:30 xen01 kernel: CR2: 00007fdb56224268 CR3: 000000003addb000
> CR4: 0000000000002660
> Dec 26 15:58:30 xen01 kernel: DR0: 0000000000000000 DR1: 0000000000000000
> DR2: 0000000000000000
> Dec 26 15:58:30 xen01 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
> DR7: 0000000000000400
> Dec 26 15:58:30 xen01 kernel: Process udevd (pid: 27998, threadinfo
> ffff88003bc3a000, task ffff880012ee0780)
> Dec 26 15:58:30 xen01 kernel: Stack:
> Dec 26 15:58:30 xen01 kernel: 0000000000000000 0000000000424121
> 000000013f00ae20 0000000000017605
> Dec 26 15:58:30 xen01 kernel: <0> ffff88003bc3be08 ffffffff8100e07c
> ffff88003a3c2580 ffff880034bb6588
> Dec 26 15:58:30 xen01 kernel: <0> ffff88003bc3be18 ffffffff8100e0af
> ffff88003bc3be58 ffffffff810a402f
> Dec 26 15:58:31 xen01 kernel: Call Trace:
> Dec 26 15:58:31 xen01 kernel: [<ffffffff8100e07c>]
> xen_alloc_ptpage+0x64/0x69
> Dec 26 15:58:31 xen01 kernel: [<ffffffff8100e0af>] xen_alloc_pte+0xe/0x10
> Dec 26 15:58:31 xen01 kernel: [<ffffffff810a402f>] __pte_alloc+0x70/0xce
> Dec 26 15:58:31 xen01 kernel: [<ffffffff810a41cd>]
> handle_mm_fault+0x140/0x8b9
> Dec 26 15:58:31 xen01 kernel: [<ffffffff810d2ecc>] ? d_kill+0x3a/0x42
> Dec 26 15:58:31 xen01 kernel: [<ffffffff810c4cd1>] ? __fput+0x1cb/0x1da
> Dec 26 15:58:31 xen01 kernel: [<ffffffff8131be4d>]
> do_page_fault+0x252/0x2e2
> Dec 26 15:58:31 xen01 kernel: [<ffffffff81319dd5>] page_fault+0x25/0x30
> Dec 26 15:58:31 xen01 kernel: Code: 48 b8 ff ff ff ff ff ff ff 7f 48 21 c2
> 48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 e9 c7 ff
> ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74 5b
> 48
> Dec 26 15:58:31 xen01 kernel: RIP [<ffffffff8100cb5b>]
> pin_pagetable_pfn+0x53/0x59
> Dec 26 15:58:31 xen01 kernel: RSP <ffff88003bc3bdc8>
> Dec 26 15:58:31 xen01 kernel: ---[ end trace 540bcf6f0170242d ]---
>
> Triggered BUG() in line 1860:
>
> static void pin_pagetable_pfn(unsigned cmd, unsigned long pfn)
> {
> struct mmuext_op op;
> op.cmd = cmd;
> op.arg1.mfn = pfn_to_mfn(pfn);
> if (HYPERVISOR_mmuext_op(&op, 1, NULL, DOMID_SELF))
> BUG(); <<THIS ONE?
Yup.
> }
>
> Any idea?
Do you get to see this every time you do LVM migrate?
>
> Thanks.
>
> Kindest regards,
> Giam Teck Choon
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2010-12-27 15:53 ` Konrad Rzeszutek Wilk
@ 2010-12-27 22:14 ` Teck Choon Giam
2010-12-28 10:42 ` Pasi Kärkkäinen
0 siblings, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2010-12-27 22:14 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 5740 bytes --]
On Mon, Dec 27, 2010 at 11:53 PM, Konrad Rzeszutek Wilk <
konrad.wilk@oracle.com> wrote:
> On Sun, Dec 26, 2010 at 04:16:16PM +0800, Teck Choon Giam wrote:
> > Hi,
> >
> > Information: CentOS 5.5 x86_64
> > dom0: latest xen/stable-2.6.32.x pvops git commit
> > 75cc13f5aa29b4f3227d269ca165dfa8937c94fe
> > xen version: xen-4.0.2-rc1-pre from xen-4.0-testing.hg changeset 21422
> >
> > While doing LVM snapshot for migration and get the following:
> >
> > Dec 26 15:58:29 xen01 kernel: ------------[ cut here ]------------
> > Dec 26 15:58:29 xen01 kernel: kernel BUG at arch/x86/xen/mmu.c:1860!
> > Dec 26 15:58:29 xen01 kernel: invalid opcode: 0000 [#1] SMP
> > Dec 26 15:58:29 xen01 kernel: last sysfs file: /sys/block/dm-26/dev
> > Dec 26 15:58:29 xen01 kernel: CPU 0
> > Dec 26 15:58:29 xen01 kernel: Modules linked in: ipt_MASQUERADE
> iptable_nat
> > nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT
> > xt_tcpudp xt_physdev iptable_filter ip_tables x_tables bridge stp
> be2iscsi
> > iscsi_tcp bnx2i cnic uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi
> > scsi_transport_iscsi loop dm_multipath scsi_dh video backlight output sbs
> > sbshc power_meter hwmon battery acpi_memhotplug xen_acpi_memhotplug ac
> > parport_pc lp parport sg tpm_tis tpm tpm_bios button i2c_i801 i2c_core
> > iTCO_wdt e1000e shpchp pcspkr dm_snapshot dm_zero dm_mirror
> dm_region_hash
> > dm_log dm_mod usb_storage ahci libata sd_mod scsi_mod raid1 ext3 jbd
> > uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
> > Dec 26 15:58:29 xen01 kernel: Pid: 27998, comm: udevd Not tainted
> > 2.6.32.27-0.xen.pvops.choon.centos5 #1 S3420GP
> > Dec 26 15:58:29 xen01 kernel: RIP: e030:[<ffffffff8100cb5b>]
> > [<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
> > Dec 26 15:58:29 xen01 kernel: RSP: e02b:ffff88003bc3bdc8 EFLAGS:
> 00010282
> > Dec 26 15:58:29 xen01 kernel: RAX: 00000000ffffffea RBX: 0000000000017605
> > RCX: 00000000000000bb
> > Dec 26 15:58:29 xen01 kernel: RDX: 00000000deadbeef RSI: 00000000deadbeef
> > RDI: 00000000deadbeef
> > Dec 26 15:58:29 xen01 kernel: RBP: ffff88003bc3bde8 R08: 0000000000000028
> > R09: ffff880000000000
> > Dec 26 15:58:29 xen01 kernel: R10: 00000000deadbeef R11: 00007fdb5665e600
> > R12: 0000000000000003
> > Dec 26 15:58:30 xen01 kernel: R13: 0000000000017605 R14: ffff880012ee0780
> > R15: 00007fdb56224268
> > Dec 26 15:58:30 xen01 kernel: FS: 00007fdb56fed710(0000)
> > GS:ffff88002804f000(0000) knlGS:0000000000000000
> > Dec 26 15:58:30 xen01 kernel: CS: e033 DS: 0000 ES: 0000 CR0:
> > 000000008005003b
> > Dec 26 15:58:30 xen01 kernel: CR2: 00007fdb56224268 CR3: 000000003addb000
> > CR4: 0000000000002660
> > Dec 26 15:58:30 xen01 kernel: DR0: 0000000000000000 DR1: 0000000000000000
> > DR2: 0000000000000000
> > Dec 26 15:58:30 xen01 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
> > DR7: 0000000000000400
> > Dec 26 15:58:30 xen01 kernel: Process udevd (pid: 27998, threadinfo
> > ffff88003bc3a000, task ffff880012ee0780)
> > Dec 26 15:58:30 xen01 kernel: Stack:
> > Dec 26 15:58:30 xen01 kernel: 0000000000000000 0000000000424121
> > 000000013f00ae20 0000000000017605
> > Dec 26 15:58:30 xen01 kernel: <0> ffff88003bc3be08 ffffffff8100e07c
> > ffff88003a3c2580 ffff880034bb6588
> > Dec 26 15:58:30 xen01 kernel: <0> ffff88003bc3be18 ffffffff8100e0af
> > ffff88003bc3be58 ffffffff810a402f
> > Dec 26 15:58:31 xen01 kernel: Call Trace:
> > Dec 26 15:58:31 xen01 kernel: [<ffffffff8100e07c>]
> > xen_alloc_ptpage+0x64/0x69
> > Dec 26 15:58:31 xen01 kernel: [<ffffffff8100e0af>]
> xen_alloc_pte+0xe/0x10
> > Dec 26 15:58:31 xen01 kernel: [<ffffffff810a402f>] __pte_alloc+0x70/0xce
> > Dec 26 15:58:31 xen01 kernel: [<ffffffff810a41cd>]
> > handle_mm_fault+0x140/0x8b9
> > Dec 26 15:58:31 xen01 kernel: [<ffffffff810d2ecc>] ? d_kill+0x3a/0x42
> > Dec 26 15:58:31 xen01 kernel: [<ffffffff810c4cd1>] ? __fput+0x1cb/0x1da
> > Dec 26 15:58:31 xen01 kernel: [<ffffffff8131be4d>]
> > do_page_fault+0x252/0x2e2
> > Dec 26 15:58:31 xen01 kernel: [<ffffffff81319dd5>] page_fault+0x25/0x30
> > Dec 26 15:58:31 xen01 kernel: Code: 48 b8 ff ff ff ff ff ff ff 7f 48 21
> c2
> > 48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 e9 c7
> ff
> > ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74
> 5b
> > 48
> > Dec 26 15:58:31 xen01 kernel: RIP [<ffffffff8100cb5b>]
> > pin_pagetable_pfn+0x53/0x59
> > Dec 26 15:58:31 xen01 kernel: RSP <ffff88003bc3bdc8>
> > Dec 26 15:58:31 xen01 kernel: ---[ end trace 540bcf6f0170242d ]---
> >
> > Triggered BUG() in line 1860:
> >
> > static void pin_pagetable_pfn(unsigned cmd, unsigned long pfn)
> > {
> > struct mmuext_op op;
> > op.cmd = cmd;
> > op.arg1.mfn = pfn_to_mfn(pfn);
> > if (HYPERVISOR_mmuext_op(&op, 1, NULL, DOMID_SELF))
> > BUG(); <<THIS ONE?
>
> Yup.
> > }
> >
> > Any idea?
>
> Do you get to see this every time you do LVM migrate?
>
My method of migration is to do below without using xen xm migrate... ...
1. LVM snapshot of domU
2. mount LVM snapshot domU
3. rsync over to the target host
4. umount LVM snapshot domU
5. remove LVM snapshot domU
6. shutdown domU
7. mount LVM domU
8. rsync mounted LVM domU over to target host
9. start domU in the new target host
Actually server will crash if I do daily LVM snapshot to backup domUs not
just for migration. And this happen almost daily :(
Method of backup domU:
1. LVM snapshot domU
2. mount LVM snapshot domU
3. rsync to disk as backup
4. umount LVM snapshot domU
5. remove LVM snapshot domU
Even I use combined ionice and nice... ... yet still crash... ...
Maybe it is time to roll back to XenLinux 2.6.18.8... ...
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 6812 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2010-12-27 22:14 ` Teck Choon Giam
@ 2010-12-28 10:42 ` Pasi Kärkkäinen
2010-12-28 18:01 ` Teck Choon Giam
2011-01-04 15:10 ` Christophe Saout
0 siblings, 2 replies; 85+ messages in thread
From: Pasi Kärkkäinen @ 2010-12-28 10:42 UTC (permalink / raw)
To: Teck Choon Giam; +Cc: xen-devel, Konrad Rzeszutek Wilk
On Tue, Dec 28, 2010 at 06:14:16AM +0800, Teck Choon Giam wrote:
> On Mon, Dec 27, 2010 at 11:53 PM, Konrad Rzeszutek Wilk
> <[1]konrad.wilk@oracle.com> wrote:
>
> On Sun, Dec 26, 2010 at 04:16:16PM +0800, Teck Choon Giam wrote:
> > Hi,
> >
> > Information: CentOS 5.5 x86_64
> > dom0: latest xen/stable-2.6.32.x pvops git commit
> > 75cc13f5aa29b4f3227d269ca165dfa8937c94fe
> > xen version: xen-4.0.2-rc1-pre from xen-4.0-testing.hg changeset 21422
> >
> > While doing LVM snapshot for migration and get the following:
> >
> > Dec 26 15:58:29 xen01 kernel: ------------[ cut here ]------------
> > Dec 26 15:58:29 xen01 kernel: kernel BUG at arch/x86/xen/mmu.c:1860!
> > Dec 26 15:58:29 xen01 kernel: invalid opcode: 0000 [#1] SMP
> > Dec 26 15:58:29 xen01 kernel: last sysfs file: /sys/block/dm-26/dev
> > Dec 26 15:58:29 xen01 kernel: CPU 0
> > Dec 26 15:58:29 xen01 kernel: Modules linked in: ipt_MASQUERADE
> iptable_nat
> > nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack
> ipt_REJECT
> > xt_tcpudp xt_physdev iptable_filter ip_tables x_tables bridge stp
> be2iscsi
> > iscsi_tcp bnx2i cnic uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi
> > scsi_transport_iscsi loop dm_multipath scsi_dh video backlight output
> sbs
> > sbshc power_meter hwmon battery acpi_memhotplug xen_acpi_memhotplug ac
> > parport_pc lp parport sg tpm_tis tpm tpm_bios button i2c_i801 i2c_core
> > iTCO_wdt e1000e shpchp pcspkr dm_snapshot dm_zero dm_mirror
> dm_region_hash
> > dm_log dm_mod usb_storage ahci libata sd_mod scsi_mod raid1 ext3 jbd
> > uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
> > Dec 26 15:58:29 xen01 kernel: Pid: 27998, comm: udevd Not tainted
> > 2.6.32.27-0.xen.pvops.choon.centos5 #1 S3420GP
> > Dec 26 15:58:29 xen01 kernel: RIP: e030:[<ffffffff8100cb5b>]
> > [<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
> > Dec 26 15:58:29 xen01 kernel: RSP: e02b:ffff88003bc3bdc8 EFLAGS:
> 00010282
> > Dec 26 15:58:29 xen01 kernel: RAX: 00000000ffffffea RBX:
> 0000000000017605
> > RCX: 00000000000000bb
> > Dec 26 15:58:29 xen01 kernel: RDX: 00000000deadbeef RSI:
> 00000000deadbeef
> > RDI: 00000000deadbeef
> > Dec 26 15:58:29 xen01 kernel: RBP: ffff88003bc3bde8 R08:
> 0000000000000028
> > R09: ffff880000000000
> > Dec 26 15:58:29 xen01 kernel: R10: 00000000deadbeef R11:
> 00007fdb5665e600
> > R12: 0000000000000003
> > Dec 26 15:58:30 xen01 kernel: R13: 0000000000017605 R14:
> ffff880012ee0780
> > R15: 00007fdb56224268
> > Dec 26 15:58:30 xen01 kernel: FS: 00007fdb56fed710(0000)
> > GS:ffff88002804f000(0000) knlGS:0000000000000000
> > Dec 26 15:58:30 xen01 kernel: CS: e033 DS: 0000 ES: 0000 CR0:
> > 000000008005003b
> > Dec 26 15:58:30 xen01 kernel: CR2: 00007fdb56224268 CR3:
> 000000003addb000
> > CR4: 0000000000002660
> > Dec 26 15:58:30 xen01 kernel: DR0: 0000000000000000 DR1:
> 0000000000000000
> > DR2: 0000000000000000
> > Dec 26 15:58:30 xen01 kernel: DR3: 0000000000000000 DR6:
> 00000000ffff0ff0
> > DR7: 0000000000000400
> > Dec 26 15:58:30 xen01 kernel: Process udevd (pid: 27998, threadinfo
> > ffff88003bc3a000, task ffff880012ee0780)
> > Dec 26 15:58:30 xen01 kernel: Stack:
> > Dec 26 15:58:30 xen01 kernel: 0000000000000000 0000000000424121
> > 000000013f00ae20 0000000000017605
> > Dec 26 15:58:30 xen01 kernel: <0> ffff88003bc3be08 ffffffff8100e07c
> > ffff88003a3c2580 ffff880034bb6588
> > Dec 26 15:58:30 xen01 kernel: <0> ffff88003bc3be18 ffffffff8100e0af
> > ffff88003bc3be58 ffffffff810a402f
> > Dec 26 15:58:31 xen01 kernel: Call Trace:
> > Dec 26 15:58:31 xen01 kernel: [<ffffffff8100e07c>]
> > xen_alloc_ptpage+0x64/0x69
> > Dec 26 15:58:31 xen01 kernel: [<ffffffff8100e0af>]
> xen_alloc_pte+0xe/0x10
> > Dec 26 15:58:31 xen01 kernel: [<ffffffff810a402f>]
> __pte_alloc+0x70/0xce
> > Dec 26 15:58:31 xen01 kernel: [<ffffffff810a41cd>]
> > handle_mm_fault+0x140/0x8b9
> > Dec 26 15:58:31 xen01 kernel: [<ffffffff810d2ecc>] ? d_kill+0x3a/0x42
> > Dec 26 15:58:31 xen01 kernel: [<ffffffff810c4cd1>] ?
> __fput+0x1cb/0x1da
> > Dec 26 15:58:31 xen01 kernel: [<ffffffff8131be4d>]
> > do_page_fault+0x252/0x2e2
> > Dec 26 15:58:31 xen01 kernel: [<ffffffff81319dd5>]
> page_fault+0x25/0x30
> > Dec 26 15:58:31 xen01 kernel: Code: 48 b8 ff ff ff ff ff ff ff 7f 48
> 21 c2
> > 48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 e9
> c7 ff
> > ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb
> 74 5b
> > 48
> > Dec 26 15:58:31 xen01 kernel: RIP [<ffffffff8100cb5b>]
> > pin_pagetable_pfn+0x53/0x59
> > Dec 26 15:58:31 xen01 kernel: RSP <ffff88003bc3bdc8>
> > Dec 26 15:58:31 xen01 kernel: ---[ end trace 540bcf6f0170242d ]---
> >
> > Triggered BUG() in line 1860:
> >
> > static void pin_pagetable_pfn(unsigned cmd, unsigned long pfn)
> > {
> > struct mmuext_op op;
> > op.cmd = cmd;
> > op.arg1.mfn = pfn_to_mfn(pfn);
> > if (HYPERVISOR_mmuext_op(&op, 1, NULL, DOMID_SELF))
> > BUG(); <<THIS ONE?
>
> Yup.
> > }
> >
> > Any idea?
>
> Do you get to see this every time you do LVM migrate?
>
> My method of migration is to do below without using xen xm migrate... ...
>
> 1. LVM snapshot of domU
> 2. mount LVM snapshot domU
> 3. rsync over to the target host
> 4. umount LVM snapshot domU
> 5. remove LVM snapshot domU
> 6. shutdown domU
> 7. mount LVM domU
> 8. rsync mounted LVM domU over to target host
> 9. start domU in the new target host
>
> Actually server will crash if I do daily LVM snapshot to backup domUs not
> just for migration. And this happen almost daily :(
>
> Method of backup domU:
>
> 1. LVM snapshot domU
> 2. mount LVM snapshot domU
> 3. rsync to disk as backup
> 4. umount LVM snapshot domU
> 5. remove LVM snapshot domU
>
> Even I use combined ionice and nice... ... yet still crash... ...
>
> Maybe it is time to roll back to XenLinux 2.6.18.8... ...
>
It would be very good to track this down and get it fixed..
hopefully you're able to help a bit and try some things to debug it.
Konrad maybe has some ideas to try..
-- Pasi
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2010-12-28 10:42 ` Pasi Kärkkäinen
@ 2010-12-28 18:01 ` Teck Choon Giam
2010-12-29 4:25 ` Teck Choon Giam
2011-01-04 15:10 ` Christophe Saout
1 sibling, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2010-12-28 18:01 UTC (permalink / raw)
To: Pasi Kärkkäinen; +Cc: xen-devel, Konrad Rzeszutek Wilk
[-- Attachment #1.1: Type: text/plain, Size: 18214 bytes --]
> It would be very good to track this down and get it fixed..
> hopefully you're able to help a bit and try some things to debug it.
>
> Konrad maybe has some ideas to try..
>
>
I would love to track this down and have it fixed or else this is a stopper
for majority of us to deploy pvops stable kernel 2.6.32.x in production.
Well at least for me... ...
More related logs and it seems all is related to LVM since all will have the
last sysfs file to /dev/block/dm-??/dev... ?
/var/log/messages.2:Dec 16 01:00:04 xen01 kernel: ------------[ cut here
]------------
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: kernel BUG at
arch/x86/xen/mmu.c:1860!
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: invalid opcode: 0000 [#1]
SMP
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: last sysfs file:
/sys/block/dm-17/dev
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: CPU 5
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: Modules linked in:
dm_snapshot ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4
nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp xt_physdev
iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic
uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi loop
dm_mirror dm_multipath scsi_dh video backlight output sbs sbshc power_meter
hwmon battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc lp parport
sg tpm_tis tpm tpm_bios button i2c_i801 i2c_core iTCO_wdt e1000e shpchp
pcspkr dm_region_hash dm_log dm_mod usb_storage ahci libata sd_mod scsi_mod
raid1 ext3 jbd uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: Pid: 26100, comm: udevd
Not tainted 2.6.32.26-3.xen.pvops.choon.centos5 #1 S3420GP
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: RIP:
e030:[<ffffffff8100cb5b>] [<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: RSP:
e02b:ffff8800086abdc8 EFLAGS: 00010282
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: RAX: 00000000ffffffea RBX:
000000000003cf00 RCX: 00000000000001e7
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: RDX: 00000000deadbeef RSI:
00000000deadbeef RDI: 00000000deadbeef
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: RBP: ffff8800086abde8 R08:
0000000000000800 R09: ffff880000000000
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: R10: 00000000deadbeef R11:
00007fe3d2a72600 R12: 0000000000000003
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: R13: 000000000003cf00 R14:
ffff88003384c140 R15: 00007fe3d2638268
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: FS:
00007fe3d3401710(0000) GS:ffff8800280e0000(0000) knlGS:0000000000000000
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: CS: e033 DS: 0000 ES:
0000 CR0: 000000008005003b
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: CR2: 00007fe3d2638268 CR3:
0000000014ff7000 CR4: 0000000000002660
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: Process udevd (pid: 26100,
threadinfo ffff8800086aa000, task ffff88003384c140)
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: Stack:
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: 0000000000000000
000000000024f026 000000013e766520 000000000003cf00
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: <0> ffff8800086abe08
ffffffff8100e07c ffff88003ad77ac0 ffff88003ce03498
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: <0> ffff8800086abe18
ffffffff8100e0af ffff8800086abe58 ffffffff810a4013
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: Call Trace:
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: [<ffffffff8100e07c>]
xen_alloc_ptpage+0x64/0x69
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: [<ffffffff8100e0af>]
xen_alloc_pte+0xe/0x10
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: [<ffffffff810a4013>]
__pte_alloc+0x70/0xce
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: [<ffffffff810a41b1>]
handle_mm_fault+0x140/0x8b9
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: [<ffffffff810d2ea0>] ?
d_kill+0x3a/0x42
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: [<ffffffff8131ba4d>]
do_page_fault+0x252/0x2e2
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: [<ffffffff813199d5>]
page_fault+0x25/0x30
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: Code: 48 b8 ff ff ff ff ff
ff ff 7f 48 21 c2 48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2 41 ba f0 7f
00 00 e8 e9 c7 ff ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5
53 48 89 fb 74 5b 48
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: RIP [<ffffffff8100cb5b>]
pin_pagetable_pfn+0x53/0x59
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: RSP <ffff8800086abdc8>
/var/log/messages.2-Dec 16 01:00:04 xen01 kernel: ---[ end trace
6828227b20a6a7c6 ]---
/var/log/messages.1:Dec 25 21:45:20 xen01 kernel: ------------[ cut here
]------------
/var/log/messages.1-Dec 25 21:45:20 xen01 kernel: kernel BUG at
arch/x86/xen/mmu.c:1860!
/var/log/messages.1-Dec 25 21:45:20 xen01 kernel: invalid opcode: 0000 [#1]
SMP
/var/log/messages.1-Dec 25 21:45:20 xen01 kernel: last sysfs file:
/sys/block/dm-26/dev
/var/log/messages.1-Dec 25 21:45:20 xen01 kernel: CPU 7
/var/log/messages.1-Dec 25 21:45:20 xen01 kernel: Modules linked in:
ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state
nf_conntrack ipt_REJECT xt_tcpudp xt_physdev iptable_filter ip_tables
x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic uio ipv6 cxgb3i cxgb3 mdio
libiscsi_tcp libiscsi scsi_transport_iscsi loop dm_multipath scsi_dh video
backlight output sbs sbshc power_meter hwmon battery acpi_memhotplug
xen_acpi_memhotplug ac parport_pc lp parport sg tpm_tis tpm tpm_bios button
i2c_i801 i2c_core iTCO_wdt shpchp e1000e pcspkr dm_snapshot dm_zero
dm_mirror dm_region_hash dm_log dm_mod usb_storage ahci libata sd_mod
scsi_mod raid1 ext3 jbd uhci_hcd ohci_hcd ehci_hcd [last unloaded:
microcode]
/var/log/messages.1-Dec 25 21:45:20 xen01 kernel: Pid: 20985, comm: udevd
Not tainted 2.6.32.26-3.xen.pvops.choon.centos5 #1 S3420GP
/var/log/messages.1-Dec 25 21:45:20 xen01 kernel: RIP:
e030:[<ffffffff8100cb5b>] [<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
/var/log/messages.1-Dec 25 21:45:20 xen01 kernel: RSP:
e02b:ffff880035c91dc8 EFLAGS: 00010282
/var/log/messages.1-Dec 25 21:45:20 xen01 kernel: RAX: 00000000ffffffea RBX:
0000000000036540 RCX: 00000000000001b2
/var/log/messages.1-Dec 25 21:45:20 xen01 kernel: RDX: 00000000deadbeef RSI:
00000000deadbeef RDI: 00000000deadbeef
/var/log/messages.1-Dec 25 21:45:20 xen01 kernel: RBP: ffff880035c91de8 R08:
0000000000000a00 R09: ffff880000000000
/var/log/messages.1-Dec 25 21:45:20 xen01 kernel: R10: 00000000deadbeef R11:
00007f8988e49600 R12: 0000000000000003
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: R13: 0000000000036540 R14:
ffff880036768540 R15: 00007f8988a0f268
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: FS:
00007f89897d8710(0000) GS:ffff88002811a000(0000) knlGS:0000000000000000
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: CS: e033 DS: 0000 ES:
0000 CR0: 000000008005003b
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: CR2: 00007f8988a0f268 CR3:
00000000375b8000 CR4: 0000000000002660
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: Process udevd (pid: 20985,
threadinfo ffff880035c90000, task ffff880036768540)
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: Stack:
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: 0000000000000000
00000000004431e6 000000013dba6e20 0000000000036540
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: <0> ffff880035c91e08
ffffffff8100e07c ffff8800375abac0 ffff88000665e228
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: <0> ffff880035c91e18
ffffffff8100e0af ffff880035c91e58 ffffffff810a4013
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: Call Trace:
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: [<ffffffff8100e07c>]
xen_alloc_ptpage+0x64/0x69
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: [<ffffffff8100e0af>]
xen_alloc_pte+0xe/0x10
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: [<ffffffff810a4013>]
__pte_alloc+0x70/0xce
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: [<ffffffff810a41b1>]
handle_mm_fault+0x140/0x8b9
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: [<ffffffff8131ba4d>]
do_page_fault+0x252/0x2e2
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: [<ffffffff813199d5>]
page_fault+0x25/0x30
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: Code: 48 b8 ff ff ff ff ff
ff ff 7f 48 21 c2 48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2 41 ba f0 7f
00 00 e8 e9 c7 ff ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5
53 48 89 fb 74 5b 48
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: RIP [<ffffffff8100cb5b>]
pin_pagetable_pfn+0x53/0x59
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: RSP <ffff880035c91dc8>
/var/log/messages.1-Dec 25 21:45:21 xen01 kernel: ---[ end trace
f394f31a52cfbac7 ]---
/var/log/messages:Dec 28 01:20:44 xen01 kernel: ------------[ cut here
]------------
/var/log/messages-Dec 28 01:20:44 xen01 kernel: kernel BUG at
arch/x86/xen/mmu.c:1860!
/var/log/messages-Dec 28 01:20:44 xen01 kernel: invalid opcode: 0000 [#1]
SMP
/var/log/messages-Dec 28 01:20:44 xen01 kernel: last sysfs file:
/sys/block/dm-26/dev
/var/log/messages-Dec 28 01:20:44 xen01 kernel: CPU 2
/var/log/messages-Dec 28 01:20:44 xen01 kernel: Modules linked in:
ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state
nf_conntrack ipt_REJECT xt_tcpudp xt_physdev iptable_filter ip_tables
x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic uio ipv6 cxgb3i cxgb3 mdio
libiscsi_tcp libiscsi scsi_transport_iscsi loop dm_multipath scsi_dh video
backlight output sbs sbshc power_meter hwmon battery acpi_memhotplug
xen_acpi_memhotplug ac parport_pc lp parport sg tpm_tis tpm tpm_bios button
i2c_i801 i2c_core iTCO_wdt e1000e shpchp pcspkr dm_snapshot dm_zero
dm_mirror dm_region_hash dm_log dm_mod usb_storage ahci libata sd_mod
scsi_mod raid1 ext3 jbd uhci_hcd ohci_hcd ehci_hcd [last unloaded:
microcode]
/var/log/messages-Dec 28 01:20:44 xen01 kernel: Pid: 30295, comm: sh Not
tainted 2.6.32.27-0.xen.pvops.choon.centos5 #1 S3420GP
/var/log/messages-Dec 28 01:20:44 xen01 kernel: RIP:
e030:[<ffffffff8100cb5b>] [<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
/var/log/messages-Dec 28 01:20:44 xen01 kernel: RSP: e02b:ffff880036661dc8
EFLAGS: 00010282
/var/log/messages-Dec 28 01:20:44 xen01 kernel: RAX: 00000000ffffffea RBX:
000000000003be66 RCX: 00000000000001df
/var/log/messages-Dec 28 01:20:44 xen01 kernel: RDX: 00000000deadbeef RSI:
00000000deadbeef RDI: 00000000deadbeef
/var/log/messages-Dec 28 01:20:44 xen01 kernel: RBP: ffff880036661de8 R08:
0000000000000330 R09: ffff880000000000
/var/log/messages-Dec 28 01:20:44 xen01 kernel: R10: 00000000deadbeef R11:
0000000000000246 R12: 0000000000000003
/var/log/messages-Dec 28 01:20:44 xen01 kernel: R13: 000000000003be66 R14:
ffff88001845e180 R15: 00000037e629a4d5
/var/log/messages-Dec 28 01:20:44 xen01 kernel: FS: 00007f240f1b96e0(0000)
GS:ffff880028089000(0000) knlGS:0000000000000000
/var/log/messages-Dec 28 01:20:44 xen01 kernel: CS: e033 DS: 0000 ES: 0000
CR0: 000000008005003b
/var/log/messages-Dec 28 01:20:44 xen01 kernel: CR2: 00000037e629a4d5 CR3:
0000000031765000 CR4: 0000000000002660
/var/log/messages-Dec 28 01:20:44 xen01 kernel: DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
/var/log/messages-Dec 28 01:20:44 xen01 kernel: DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
/var/log/messages-Dec 28 01:20:44 xen01 kernel: Process sh (pid: 30295,
threadinfo ffff880036660000, task ffff88001845e180)
/var/log/messages-Dec 28 01:20:44 xen01 kernel: Stack:
/var/log/messages-Dec 28 01:20:44 xen01 kernel: 0000000000000000
00000000004508c0 000000013dae4820 000000000003be66
/var/log/messages-Dec 28 01:20:44 xen01 kernel: <0> ffff880036661e08
ffffffff8100e07c ffff880035da1580 ffff880035c48988
/var/log/messages-Dec 28 01:20:44 xen01 kernel: <0> ffff880036661e18
ffffffff8100e0af ffff880036661e58 ffffffff810a402f
/var/log/messages-Dec 28 01:20:44 xen01 kernel: Call Trace:
/var/log/messages-Dec 28 01:20:44 xen01 kernel: [<ffffffff8100e07c>]
xen_alloc_ptpage+0x64/0x69
/var/log/messages-Dec 28 01:20:44 xen01 kernel: [<ffffffff8100e0af>]
xen_alloc_pte+0xe/0x10
/var/log/messages-Dec 28 01:20:44 xen01 kernel: [<ffffffff810a402f>]
__pte_alloc+0x70/0xce
/var/log/messages-Dec 28 01:20:44 xen01 kernel: [<ffffffff810a41cd>]
handle_mm_fault+0x140/0x8b9
/var/log/messages-Dec 28 01:20:44 xen01 kernel: [<ffffffff81319dd5>] ?
page_fault+0x25/0x30
/var/log/messages-Dec 28 01:20:44 xen01 kernel: [<ffffffff8131be4d>]
do_page_fault+0x252/0x2e2
/var/log/messages-Dec 28 01:20:44 xen01 kernel: [<ffffffff8116dd7d>] ?
__put_user_4+0x1d/0x30
/var/log/messages-Dec 28 01:20:44 xen01 kernel: [<ffffffff81319dd5>]
page_fault+0x25/0x30
/var/log/messages-Dec 28 01:20:44 xen01 kernel: Code: 48 b8 ff ff ff ff ff
ff ff 7f 48 21 c2 48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2 41 ba f0 7f
00 00 e8 e9 c7 ff ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5
53 48 89 fb 74 5b 48
/var/log/messages-Dec 28 01:20:45 xen01 kernel: RIP [<ffffffff8100cb5b>]
pin_pagetable_pfn+0x53/0x59
/var/log/messages-Dec 28 01:20:45 xen01 kernel: RSP <ffff880036661dc8>
/var/log/messages-Dec 28 01:20:45 xen01 kernel: ---[ end trace
b65ec9b025b586cf ]---
/var/log/messages:Dec 26 15:58:29 xen01 kernel: ------------[ cut here
]------------
/var/log/messages-Dec 26 15:58:29 xen01 kernel: kernel BUG at
arch/x86/xen/mmu.c:1860!
/var/log/messages-Dec 26 15:58:29 xen01 kernel: invalid opcode: 0000 [#1]
SMP
/var/log/messages-Dec 26 15:58:29 xen01 kernel: last sysfs file:
/sys/block/dm-26/dev
/var/log/messages-Dec 26 15:58:29 xen01 kernel: CPU 0
/var/log/messages-Dec 26 15:58:29 xen01 kernel: Modules linked in:
ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state
nf_conntrack ipt_REJECT xt_tcpudp xt_physdev iptable_filter ip_tables
x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic uio ipv6 cxgb3i cxgb3 mdio
libiscsi_tcp libiscsi scsi_transport_iscsi loop dm_multipath scsi_dh video
backlight output sbs sbshc power_meter hwmon battery acpi_memhotplug
xen_acpi_memhotplug ac parport_pc lp parport sg tpm_tis tpm tpm_bios button
i2c_i801 i2c_core iTCO_wdt e1000e shpchp pcspkr dm_snapshot dm_zero
dm_mirror dm_region_hash dm_log dm_mod usb_storage ahci libata sd_mod
scsi_mod raid1 ext3 jbd uhci_hcd ohci_hcd ehci_hcd [last unloaded:
microcode]
/var/log/messages-Dec 26 15:58:29 xen01 kernel: Pid: 27998, comm: udevd Not
tainted 2.6.32.27-0.xen.pvops.choon.centos5 #1 S3420GP
/var/log/messages-Dec 26 15:58:29 xen01 kernel: RIP:
e030:[<ffffffff8100cb5b>] [<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
/var/log/messages-Dec 26 15:58:29 xen01 kernel: RSP: e02b:ffff88003bc3bdc8
EFLAGS: 00010282
/var/log/messages-Dec 26 15:58:29 xen01 kernel: RAX: 00000000ffffffea RBX:
0000000000017605 RCX: 00000000000000bb
/var/log/messages-Dec 26 15:58:29 xen01 kernel: RDX: 00000000deadbeef RSI:
00000000deadbeef RDI: 00000000deadbeef
/var/log/messages-Dec 26 15:58:29 xen01 kernel: RBP: ffff88003bc3bde8 R08:
0000000000000028 R09: ffff880000000000
/var/log/messages-Dec 26 15:58:29 xen01 kernel: R10: 00000000deadbeef R11:
00007fdb5665e600 R12: 0000000000000003
/var/log/messages-Dec 26 15:58:30 xen01 kernel: R13: 0000000000017605 R14:
ffff880012ee0780 R15: 00007fdb56224268
/var/log/messages-Dec 26 15:58:30 xen01 kernel: FS: 00007fdb56fed710(0000)
GS:ffff88002804f000(0000) knlGS:0000000000000000
/var/log/messages-Dec 26 15:58:30 xen01 kernel: CS: e033 DS: 0000 ES: 0000
CR0: 000000008005003b
/var/log/messages-Dec 26 15:58:30 xen01 kernel: CR2: 00007fdb56224268 CR3:
000000003addb000 CR4: 0000000000002660
/var/log/messages-Dec 26 15:58:30 xen01 kernel: DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
/var/log/messages-Dec 26 15:58:30 xen01 kernel: DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
/var/log/messages-Dec 26 15:58:30 xen01 kernel: Process udevd (pid: 27998,
threadinfo ffff88003bc3a000, task ffff880012ee0780)
/var/log/messages-Dec 26 15:58:30 xen01 kernel: Stack:
/var/log/messages-Dec 26 15:58:30 xen01 kernel: 0000000000000000
0000000000424121 000000013f00ae20 0000000000017605
/var/log/messages-Dec 26 15:58:30 xen01 kernel: <0> ffff88003bc3be08
ffffffff8100e07c ffff88003a3c2580 ffff880034bb6588
/var/log/messages-Dec 26 15:58:30 xen01 kernel: <0> ffff88003bc3be18
ffffffff8100e0af ffff88003bc3be58 ffffffff810a402f
/var/log/messages-Dec 26 15:58:31 xen01 kernel: Call Trace:
/var/log/messages-Dec 26 15:58:31 xen01 kernel: [<ffffffff8100e07c>]
xen_alloc_ptpage+0x64/0x69
/var/log/messages-Dec 26 15:58:31 xen01 kernel: [<ffffffff8100e0af>]
xen_alloc_pte+0xe/0x10
/var/log/messages-Dec 26 15:58:31 xen01 kernel: [<ffffffff810a402f>]
__pte_alloc+0x70/0xce
/var/log/messages-Dec 26 15:58:31 xen01 kernel: [<ffffffff810a41cd>]
handle_mm_fault+0x140/0x8b9
/var/log/messages-Dec 26 15:58:31 xen01 kernel: [<ffffffff810d2ecc>] ?
d_kill+0x3a/0x42
/var/log/messages-Dec 26 15:58:31 xen01 kernel: [<ffffffff810c4cd1>] ?
__fput+0x1cb/0x1da
/var/log/messages-Dec 26 15:58:31 xen01 kernel: [<ffffffff8131be4d>]
do_page_fault+0x252/0x2e2
/var/log/messages-Dec 26 15:58:31 xen01 kernel: [<ffffffff81319dd5>]
page_fault+0x25/0x30
/var/log/messages-Dec 26 15:58:31 xen01 kernel: Code: 48 b8 ff ff ff ff ff
ff ff 7f 48 21 c2 48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2 41 ba f0 7f
00 00 e8 e9 c7 ff ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5
53 48 89 fb 74 5b 48
/var/log/messages-Dec 26 15:58:31 xen01 kernel: RIP [<ffffffff8100cb5b>]
pin_pagetable_pfn+0x53/0x59
/var/log/messages-Dec 26 15:58:31 xen01 kernel: RSP <ffff88003bc3bdc8>
/var/log/messages-Dec 26 15:58:31 xen01 kernel: ---[ end trace
540bcf6f0170242d ]---
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 19389 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2010-12-28 18:01 ` Teck Choon Giam
@ 2010-12-29 4:25 ` Teck Choon Giam
2010-12-29 4:58 ` Teck Choon Giam
0 siblings, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2010-12-29 4:25 UTC (permalink / raw)
To: Pasi Kärkkäinen; +Cc: xen-devel, Konrad Rzeszutek Wilk
[-- Attachment #1.1: Type: text/plain, Size: 20345 bytes --]
On Wed, Dec 29, 2010 at 2:01 AM, Teck Choon Giam <giamteckchoon@gmail.com>wrote:
>
> It would be very good to track this down and get it fixed..
>> hopefully you're able to help a bit and try some things to debug it.
>>
>> Konrad maybe has some ideas to try..
>>
>>
> I would love to track this down and have it fixed or else this is a stopper
> for majority of us to deploy pvops stable kernel 2.6.32.x in production.
> Well at least for me... ...
>
> More related logs and it seems all is related to LVM since all will have
> the last sysfs file to /dev/block/dm-??/dev... ?
>
While waiting for others to offer any assistance... I created a script to
basically test lvm create snapshot, mount, umount and remove snapshot... ...
the result is quite sad as I am able to reproduce the crash with loop count
100 easily on two of my servers with different specs... ...
If anyone is willing to run the script to test whether can reproduce the
crash on their deveopment xen servers or any LVM host servers would be
great. The below script assume that those snapshot LVM can be mounted as
personally I am running all domUs. If your LV Group Name is different, just
ammend the LVGroupName=VolGroup or anything to suit your environment.
----------8<----------8<----------8<----------8<----------8<----------8<----------8<----------
#!/bin/sh
#
# This script is to create lvm snapshot, mount it, umount it and remove in a
# specified number of loops to test whether it will crash the host server.
#
# Created by Giam Teck Choon
#
# The LV name and for this case we are using XenGroup
LVGroupName=XenGroup
# return 1 if is mounted otherwise return 0
check_mount() {
local checkdir=${1}
if [ -n "$checkdir" ] ; then
local check=`grep "$checkdir" /proc/mounts`
if [ -n "$check" ] ; then
return 1
fi
fi
return 0
}
do_lvm_create_remove() {
# number of loops default is 1
local loopcountlimit=${1:-1}
# snapshot size default is 1G
local snapshotsize=${2:-1G}
# implement a sleep between create, mount, umount and remove (default is
0 which is no pause)
local pauseinterval=${3:-0}
# We filter out control, snapshot and swap
local count=0
if [ -d "/dev/${LVGroupName}" ] ; then
while [ "$count" -lt "$loopcountlimit" ]
do
count=`expr $count + 1`
echo "${count} ... ... "
for i in `ls /dev/${LVGroupName} | grep -Ev 'snapshot$' | grep
-Ev 'swap$'`; do
if [ -h "/dev/${LVGroupName}/${i}" ] ; then
echo -n "lvcreate -s -v -n ${i}-snapshot -L
${snapshotsize} /dev/${LVGroupName}/${i} ... ... "
lvcreate -s -v -n ${i}-snapshot -L ${snapshotsize}
/dev/${LVGroupName}/${i}
echo "done."
sleep ${pauseinterval}
mkdir -p /mnt/testlvm/${i}
if [ -h "/dev/${LVGroupName}/${i}-snapshot" ] ; then
check_mount /mnt/testlvm/${i}
local ismount=$?
if [ "$ismount" -eq 0 ] ; then
echo -n "mount /dev/${LVGroupName}/${i}-snapshot
/mnt/testlvm/${i} ... ... "
mount /dev/${LVGroupName}/${i}-snapshot
/mnt/testlvm/${i}
echo "done."
sleep ${pauseinterval}
fi
check_mount /mnt/testlvm/${i}
local ismount2=$?
if [ "$ismount2" -eq 1 ] ; then
echo -n "umount /mnt/testlvm/${i} ... ... "
umount /mnt/testlvm/${i}
echo "done."
sleep ${pauseinterval}
fi
fi
rm -rf /mnt/testlvm/${i}
echo -n "lvremove -f /dev/${LVGroupName}/${i}-snapshot
... ... "
lvremove -f /dev/${LVGroupName}/${i}-snapshot
echo "done."
sleep ${pauseinterval}
fi
done
rm -fr /mnt/testlvm
done
else
echo "/dev/${LVGroupName} directory not found!"
exit 1
fi
}
case $1 in
loop) shift
do_lvm_create_remove "$@"
;;
*) echo "Usage: $0 loop loopcountlimit(default is 1)
snapshotsize(default is 1GB) pauseinterval (default is 0)"
;;
esac
----------8<----------8<----------8<----------8<----------8<----------8<----------8<----------
I name the above script as test.sh then run:
Below is without pause
# sh test.sh loop 100
Then I will get something printed to output as below:
Message from syslogd@ at Wed Dec 29 12:13:19 2010 ...
xen03 kernel: ------------[ cut here ]------------
Message from syslogd@ at Wed Dec 29 12:13:19 2010 ...
xen03 kernel: invalid opcode: 0000 [#1] SMP
Message from syslogd@ at Wed Dec 29 12:13:19 2010 ...
xen03 kernel: last sysfs file: /sys/block/dm-16/dev
Message from syslogd@ at Wed Dec 29 12:13:19 2010 ...
xen03 kernel: Stack:
Message from syslogd@ at Wed Dec 29 12:13:19 2010 ...
xen03 kernel: Call Trace:
Message from syslogd@ at Wed Dec 29 12:13:19 2010 ...
xen03 kernel: Code: 48 b8 ff ff ff ff ff ff ff 7f 48 21 c2 48 89 55 e8 48 8d
7d e0 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 e9 c7 ff ff 85 c0 74 04 <0f>
0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74 5b 48
Unable to read from event server.
Then I can run the below command to get the log after it is back online:
# grep -A 50 'cut here' /var/log/messages
Dec 29 11:47:32 xen03 kernel: ------------[ cut here ]------------
Dec 29 11:47:32 xen03 kernel: kernel BUG at arch/x86/xen/mmu.c:1860!
Dec 29 11:47:32 xen03 kernel: invalid opcode: 0000 [#2] SMP
Dec 29 11:47:32 xen03 kernel: last sysfs file: /sys/block/dm-16/dev
Dec 29 11:47:32 xen03 kernel: CPU 3
Dec 29 11:47:32 xen03 kernel: Modules linked in: ext4 jbd2 crc16 xt_physdev
iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic
uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi loop
dm_multipath scsi_dh video backlight output sbs sbshc power_meter hwmon
battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc lp parport sg tg3
libphy ide_cd_mod cdrom button serio_raw tpm_tis tpm tpm_bios pcspkr
iTCO_wdt i2c_i801 shpchp i2c_core dm_snapshot dm_zero dm_mirror
dm_region_hash dm_log dm_mod ata_piix libata sd_mod scsi_mod raid1 ext3 jbd
uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
Dec 29 11:47:32 xen03 kernel: Pid: 30576, comm: mpath_wait Tainted: G
D 2.6.32.27-0.xen.pvops.choon.centos5 #1 PowerEdge 860
Dec 29 11:47:32 xen03 kernel: RIP: e030:[<ffffffff8100cb5b>]
[<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
Dec 29 11:47:32 xen03 kernel: RSP: e02b:ffff880030a7fdc8 EFLAGS: 00010282
Dec 29 11:47:32 xen03 kernel: RAX: 00000000ffffffea RBX: 000000000002445a
RCX: 0000000000000122
Dec 29 11:47:32 xen03 kernel: RDX: 00000000deadbeef RSI: 00000000deadbeef
RDI: 00000000deadbeef
Dec 29 11:47:32 xen03 kernel: RBP: ffff880030a7fde8 R08: 00000000000002d0
R09: ffff880000000000
Dec 29 11:47:32 xen03 kernel: R10: 00000000deadbeef R11: 0000000000000246
R12: 0000000000000003
Dec 29 11:47:32 xen03 kernel: R13: 000000000002445a R14: ffff88002456e480
R15: 0000003db6c9a4d5
Dec 29 11:47:32 xen03 kernel: FS: 00007f45a01586e0(0000)
GS:ffff8800280a6000(0000) knlGS:0000000000000000
Dec 29 11:47:32 xen03 kernel: CS: e033 DS: 0000 ES: 0000 CR0:
000000008005003b
Dec 29 11:47:32 xen03 kernel: CR2: 0000003db6c9a4d5 CR3: 000000001cc2d000
CR4: 0000000000002660
Dec 29 11:47:32 xen03 kernel: DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000
Dec 29 11:47:32 xen03 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
DR7: 0000000000000400
Dec 29 11:47:32 xen03 kernel: Process mpath_wait (pid: 30576, threadinfo
ffff880030a7e000, task ffff88002456e480)
Dec 29 11:47:32 xen03 kernel: Stack:
Dec 29 11:47:32 xen03 kernel: 0000000000000000 00000000001f667f
000000013e6f3e18 000000000002445a
Dec 29 11:47:32 xen03 kernel: <0> ffff880030a7fe08 ffffffff8100e07c
ffff88003d2fc040 ffff8800246c9db0
Dec 29 11:47:32 xen03 kernel: <0> ffff880030a7fe18 ffffffff8100e0af
ffff880030a7fe58 ffffffff810a402f
Dec 29 11:47:32 xen03 kernel: Call Trace:
Dec 29 11:47:32 xen03 kernel: [<ffffffff8100e07c>]
xen_alloc_ptpage+0x64/0x69
Dec 29 11:47:32 xen03 kernel: [<ffffffff8100e0af>] xen_alloc_pte+0xe/0x10
Dec 29 11:47:32 xen03 kernel: [<ffffffff810a402f>] __pte_alloc+0x70/0xce
Dec 29 11:47:32 xen03 kernel: [<ffffffff810a41cd>]
handle_mm_fault+0x140/0x8b9
Dec 29 11:47:32 xen03 kernel: [<ffffffff81319dd5>] ? page_fault+0x25/0x30
Dec 29 11:47:32 xen03 kernel: [<ffffffff8131be4d>]
do_page_fault+0x252/0x2e2
Dec 29 11:47:32 xen03 kernel: [<ffffffff8116dd7d>] ? __put_user_4+0x1d/0x30
Dec 29 11:47:32 xen03 kernel: [<ffffffff81319dd5>] page_fault+0x25/0x30
Dec 29 11:47:32 xen03 kernel: Code: 48 b8 ff ff ff ff ff ff ff 7f 48 21 c2
48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 e9 c7 ff
ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74 5b
48
Dec 29 11:47:32 xen03 kernel: RIP [<ffffffff8100cb5b>]
pin_pagetable_pfn+0x53/0x59
Dec 29 11:47:32 xen03 kernel: RSP <ffff880030a7fdc8>
Dec 29 11:47:32 xen03 kernel: ---[ end trace 900e639a50e97057 ]---
Dec 29 11:50:38 xen03 syslogd 1.4.1: restart.
Dec 29 11:50:38 xen03 kernel: klogd 1.4.1, log source = /proc/kmsg started.
Dec 29 11:50:38 xen03 kernel: Linux version
2.6.32.27-0.xen.pvops.choon.centos5 (mockbuild@builder5.choon.net) (gcc
version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Sat Dec 25 09:19:44 SGT
2010
Dec 29 11:50:38 xen03 kernel: Command line: ro root=/dev/md1 panic=5
panic_timeout=5
Dec 29 11:50:38 xen03 kernel: KERNEL supported cpus:
Dec 29 11:50:38 xen03 kernel: Intel GenuineIntel
Dec 29 11:50:38 xen03 kernel: AMD AuthenticAMD
Dec 29 11:50:38 xen03 kernel: Centaur CentaurHauls
Dec 29 11:50:38 xen03 kernel: released 0 pages of unused memory
Dec 29 11:50:38 xen03 kernel: BIOS-provided physical RAM map:
Dec 29 11:50:38 xen03 kernel: Xen: 0000000000000000 - 00000000000a0000
(usable)
Dec 29 11:50:38 xen03 kernel: Xen: 00000000000a0000 - 0000000000100000
(reserved)
Dec 29 11:50:38 xen03 kernel: Xen: 0000000000100000 - 0000000040000000
(usable)
Dec 29 11:50:38 xen03 kernel: Xen: 00000000dffc0000 - 00000000dffcfc00
(ACPI data)
--
Dec 29 11:52:30 xen03 kernel: ------------[ cut here ]------------
Dec 29 11:52:30 xen03 kernel: kernel BUG at arch/x86/xen/mmu.c:1860!
Dec 29 11:52:30 xen03 kernel: invalid opcode: 0000 [#1] SMP
Dec 29 11:52:30 xen03 kernel: last sysfs file:
/sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_map
Dec 29 11:52:30 xen03 kernel: CPU 3
Dec 29 11:52:30 xen03 kernel: Modules linked in: xt_physdev iptable_filter
ip_tables x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic uio ipv6 cxgb3i
cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi loop dm_multipath
scsi_dh video backlight output sbs sbshc power_meter hwmon battery
acpi_memhotplug xen_acpi_memhotplug ac parport_pc lp parport sg tg3 libphy
ide_cd_mod cdrom serio_raw button tpm_tis tpm tpm_bios i2c_i801 i2c_core
iTCO_wdt pcspkr shpchp dm_snapshot dm_zero dm_mirror dm_region_hash dm_log
dm_mod ata_piix libata sd_mod scsi_mod raid1 ext3 jbd uhci_hcd ohci_hcd
ehci_hcd [last unloaded: microcode]
Dec 29 11:52:30 xen03 kernel: Pid: 8000, comm: udevd Not tainted
2.6.32.27-0.xen.pvops.choon.centos5 #1 PowerEdge 860
Dec 29 11:52:30 xen03 kernel: RIP: e030:[<ffffffff8100cb5b>]
[<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
Dec 29 11:52:30 xen03 kernel: RSP: e02b:ffff88001f0c9dc8 EFLAGS: 00010282
Dec 29 11:52:30 xen03 kernel: RAX: 00000000ffffffea RBX: 0000000000037fc2
RCX: 00000000000001bf
Dec 29 11:52:30 xen03 kernel: RDX: 00000000deadbeef RSI: 00000000deadbeef
RDI: 00000000deadbeef
Dec 29 11:52:30 xen03 kernel: RBP: ffff88001f0c9de8 R08: 0000000000000e10
R09: ffff880000000000
Dec 29 11:52:30 xen03 kernel: R10: 00000000deadbeef R11: 00007fb810051ce0
R12: 0000000000000003
Dec 29 11:52:30 xen03 kernel: R13: 0000000000037fc2 R14: ffff88003257e100
R15: 00007fb80fc14158
Dec 29 11:52:30 xen03 kernel: FS: 00007fb81024e710(0000)
GS:ffff8800280a6000(0000) knlGS:0000000000000000
Dec 29 11:52:30 xen03 kernel: CS: e033 DS: 0000 ES: 0000 CR0:
000000008005003b
Dec 29 11:52:30 xen03 kernel: CR2: 00007fb80fc14158 CR3: 0000000032ee7000
CR4: 0000000000002660
Dec 29 11:52:30 xen03 kernel: DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000
Dec 29 11:52:30 xen03 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
DR7: 0000000000000400
Dec 29 11:52:30 xen03 kernel: Process udevd (pid: 8000, threadinfo
ffff88001f0c8000, task ffff88003257e100)
Dec 29 11:52:30 xen03 kernel: Stack:
Dec 29 11:52:30 xen03 kernel: 0000000000000000 000000000020a314
000000013e6f3e18 0000000000037fc2
Dec 29 11:52:30 xen03 kernel: <0> ffff88001f0c9e08 ffffffff8100e07c
ffff880032c2a040 ffff880032aaa3f0
Dec 29 11:52:30 xen03 kernel: <0> ffff88001f0c9e18 ffffffff8100e0af
ffff88001f0c9e58 ffffffff810a402f
Dec 29 11:52:30 xen03 kernel: Call Trace:
Dec 29 11:52:30 xen03 kernel: [<ffffffff8100e07c>]
xen_alloc_ptpage+0x64/0x69
Dec 29 11:52:30 xen03 kernel: [<ffffffff8100e0af>] xen_alloc_pte+0xe/0x10
Dec 29 11:52:30 xen03 kernel: [<ffffffff810a402f>] __pte_alloc+0x70/0xce
Dec 29 11:52:30 xen03 kernel: [<ffffffff810a41cd>]
handle_mm_fault+0x140/0x8b9
Dec 29 11:52:30 xen03 kernel: [<ffffffff810d2ecc>] ? d_kill+0x3a/0x42
Dec 29 11:52:30 xen03 kernel: [<ffffffff810c4cd1>] ? __fput+0x1cb/0x1da
Dec 29 11:52:30 xen03 kernel: [<ffffffff8131be4d>]
do_page_fault+0x252/0x2e2
Dec 29 11:52:30 xen03 kernel: [<ffffffff81319dd5>] page_fault+0x25/0x30
Dec 29 11:52:30 xen03 kernel: Code: 48 b8 ff ff ff ff ff ff ff 7f 48 21 c2
48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 e9 c7 ff
ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74 5b
48
Dec 29 11:52:30 xen03 kernel: RIP [<ffffffff8100cb5b>]
pin_pagetable_pfn+0x53/0x59
Dec 29 11:52:30 xen03 kernel: RSP <ffff88001f0c9dc8>
Dec 29 11:52:30 xen03 kernel: ---[ end trace 2b8ce24b81556aa1 ]---
Dec 29 11:52:40 xen03 kernel: dmeventd[4328]: segfault at 90 ip
0000003db740bcd0 sp 00000000408080c8 error 4 in libpthread-2.5.so
[3db7400000+16000]
Dec 29 11:55:35 xen03 syslogd 1.4.1: restart.
Dec 29 11:55:35 xen03 kernel: klogd 1.4.1, log source = /proc/kmsg started.
Dec 29 11:55:35 xen03 kernel: Linux version
2.6.32.27-0.xen.pvops.choon.centos5 (mockbuild@builder5.choon.net) (gcc
version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Sat Dec 25 09:19:44 SGT
2010
Dec 29 11:55:35 xen03 kernel: Command line: ro root=/dev/md1 panic=5
panic_timeout=5
Dec 29 11:55:35 xen03 kernel: KERNEL supported cpus:
Dec 29 11:55:35 xen03 kernel: Intel GenuineIntel
Dec 29 11:55:35 xen03 kernel: AMD AuthenticAMD
Dec 29 11:55:35 xen03 kernel: Centaur CentaurHauls
Dec 29 11:55:35 xen03 kernel: released 0 pages of unused memory
Dec 29 11:55:35 xen03 kernel: BIOS-provided physical RAM map:
Dec 29 11:55:35 xen03 kernel: Xen: 0000000000000000 - 00000000000a0000
(usable)
Dec 29 11:55:35 xen03 kernel: Xen: 00000000000a0000 - 0000000000100000
(reserved)
Dec 29 11:55:35 xen03 kernel: Xen: 0000000000100000 - 0000000040000000
(usable)
--
Dec 29 12:13:19 xen03 kernel: ------------[ cut here ]------------
Dec 29 12:13:19 xen03 kernel: kernel BUG at arch/x86/xen/mmu.c:1860!
Dec 29 12:13:19 xen03 kernel: invalid opcode: 0000 [#1] SMP
Dec 29 12:13:19 xen03 kernel: last sysfs file: /sys/block/dm-16/dev
Dec 29 12:13:19 xen03 kernel: CPU 1
Dec 29 12:13:19 xen03 kernel: Modules linked in: ext4 jbd2 crc16 xt_physdev
iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic
uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi loop
dm_multipath scsi_dh video backlight output sbs sbshc power_meter hwmon
battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc lp parport sg
ide_cd_mod cdrom serio_raw tg3 button libphy tpm_tis tpm tpm_bios iTCO_wdt
i2c_i801 pcspkr i2c_core shpchp dm_snapshot dm_zero dm_mirror dm_region_hash
dm_log dm_mod ata_piix libata sd_mod scsi_mod raid1 ext3 jbd uhci_hcd
ohci_hcd ehci_hcd [last unloaded: microcode]
Dec 29 12:13:19 xen03 kernel: Pid: 15285, comm: udevd Not tainted
2.6.32.27-0.xen.pvops.choon.centos5 #1 PowerEdge 860
Dec 29 12:13:19 xen03 kernel: RIP: e030:[<ffffffff8100cb5b>]
[<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
Dec 29 12:13:19 xen03 kernel: RSP: e02b:ffff880025db5dc8 EFLAGS: 00010282
Dec 29 12:13:19 xen03 kernel: RAX: 00000000ffffffea RBX: 0000000000035093
RCX: 00000000000001a8
Dec 29 12:13:19 xen03 kernel: RDX: 00000000deadbeef RSI: 00000000deadbeef
RDI: 00000000deadbeef
Dec 29 12:13:19 xen03 kernel: RBP: ffff880025db5de8 R08: 0000000000000498
R09: ffff880000000000
Dec 29 12:13:19 xen03 kernel: R10: 00000000deadbeef R11: 00007f7824d70600
R12: 0000000000000003
Dec 29 12:13:19 xen03 kernel: R13: 0000000000035093 R14: ffff880039ff0540
R15: 00007f7824936268
Dec 29 12:13:19 xen03 kernel: FS: 00007f78256fe710(0000)
GS:ffff88002806c000(0000) knlGS:0000000000000000
Dec 29 12:13:19 xen03 kernel: CS: e033 DS: 0000 ES: 0000 CR0:
000000008005003b
Dec 29 12:13:19 xen03 kernel: CR2: 00007f7824936268 CR3: 0000000039fef000
CR4: 0000000000002660
Dec 29 12:13:19 xen03 kernel: DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000
Dec 29 12:13:19 xen03 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
DR7: 0000000000000400
Dec 29 12:13:19 xen03 kernel: Process udevd (pid: 15285, threadinfo
ffff880025db4000, task ffff880039ff0540)
Dec 29 12:13:19 xen03 kernel: Stack:
Dec 29 12:13:19 xen03 kernel: 0000000000000000 0000000000207243
000000013e64d518 0000000000035093
Dec 29 12:13:19 xen03 kernel: <0> ffff880025db5e08 ffffffff8100e07c
ffff88003a50fac0 ffff8800352a6920
Dec 29 12:13:19 xen03 kernel: <0> ffff880025db5e18 ffffffff8100e0af
ffff880025db5e58 ffffffff810a402f
Dec 29 12:13:19 xen03 kernel: Call Trace:
Dec 29 12:13:19 xen03 kernel: [<ffffffff8100e07c>]
xen_alloc_ptpage+0x64/0x69
Dec 29 12:13:19 xen03 kernel: [<ffffffff8100e0af>] xen_alloc_pte+0xe/0x10
Dec 29 12:13:19 xen03 kernel: [<ffffffff810a402f>] __pte_alloc+0x70/0xce
Dec 29 12:13:19 xen03 kernel: [<ffffffff810a41cd>]
handle_mm_fault+0x140/0x8b9
Dec 29 12:13:19 xen03 kernel: [<ffffffff813199ff>] ?
_spin_unlock_irqrestore+0x11/0x13
Dec 29 12:13:19 xen03 kernel: [<ffffffff8131be4d>]
do_page_fault+0x252/0x2e2
Dec 29 12:13:19 xen03 kernel: [<ffffffff81319dd5>] page_fault+0x25/0x30
Dec 29 12:13:19 xen03 kernel: Code: 48 b8 ff ff ff ff ff ff ff 7f 48 21 c2
48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 e9 c7 ff
ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74 5b
48
Dec 29 12:13:19 xen03 kernel: RIP [<ffffffff8100cb5b>]
pin_pagetable_pfn+0x53/0x59
Dec 29 12:13:19 xen03 kernel: RSP <ffff880025db5dc8>
Dec 29 12:13:19 xen03 kernel: ---[ end trace aa0f33d0cdc0d845 ]---
Dec 29 12:16:25 xen03 syslogd 1.4.1: restart.
Dec 29 12:16:25 xen03 kernel: klogd 1.4.1, log source = /proc/kmsg started.
Dec 29 12:16:25 xen03 kernel: Linux version
2.6.32.27-0.xen.pvops.choon.centos5 (mockbuild@builder5.choon.net) (gcc
version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Sat Dec 25 09:19:44 SGT
2010
Dec 29 12:16:25 xen03 kernel: Command line: ro root=/dev/md1 panic=5
panic_timeout=5
Dec 29 12:16:25 xen03 kernel: KERNEL supported cpus:
Dec 29 12:16:25 xen03 kernel: Intel GenuineIntel
Dec 29 12:16:25 xen03 kernel: AMD AuthenticAMD
Dec 29 12:16:25 xen03 kernel: Centaur CentaurHauls
Dec 29 12:16:25 xen03 kernel: released 0 pages of unused memory
Dec 29 12:16:25 xen03 kernel: BIOS-provided physical RAM map:
Dec 29 12:16:25 xen03 kernel: Xen: 0000000000000000 - 00000000000a0000
(usable)
Dec 29 12:16:25 xen03 kernel: Xen: 00000000000a0000 - 0000000000100000
(reserved)
Dec 29 12:16:25 xen03 kernel: Xen: 0000000000100000 - 0000000040000000
(usable)
Dec 29 12:16:25 xen03 kernel: Xen: 00000000dffc0000 - 00000000dffcfc00
(ACPI data)
Dec 29 12:16:25 xen03 kernel: Xen: 00000000dffcfc00 - 00000000dffff000
(reserved)
My next crash test for this will be with a pause... to see whether I can
reproduce the crash or not... then after that I will implement a sync before
each crash to test crash... ...
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 22634 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2010-12-29 4:25 ` Teck Choon Giam
@ 2010-12-29 4:58 ` Teck Choon Giam
2011-01-14 15:20 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2010-12-29 4:58 UTC (permalink / raw)
To: Pasi Kärkkäinen; +Cc: xen-devel, Konrad Rzeszutek Wilk
[-- Attachment #1.1: Type: text/plain, Size: 5201 bytes --]
Below is my latest test crash script:
----------8<----------8<----------8<----------8<----------8<----------8<----------8<----------8<----------
#!/bin/sh
#
# This script is to create lvm snapshot, mount it, umount it and remove in a
# specified number of loops to test whether it will crash the host server.
# All LVM snapshots assumed can be mounted like if you are running a PV
domU.
#
# Created by Giam Teck Choon
#
# The LV name and for this case we are using XenGroup
LVGroupName=XenGroup
# return 1 if is mounted otherwise return 0
check_mount() {
local checkdir=${1}
if [ -n "$checkdir" ] ; then
local check=`grep "$checkdir" /proc/mounts`
if [ -n "$check" ] ; then
return 1
fi
fi
return 0
}
do_lvm_create_remove() {
# number of loops default is 1
local loopcountlimit=${1:-1}
# snapshot size default is 1G
local snapshotsize=${2:-1G}
# implement a sleep between create, mount, umount and remove (default is
0 which is no pause)
local pauseinterval=${3:-0}
# execute commands after each pause/sleep such as sync or anything that
you want to test
local commands=${4}
# We filter out snapshot and swap
local count=0
if [ -d "/dev/${LVGroupName}" ] ; then
while [ "$count" -lt "$loopcountlimit" ]
do
count=`expr $count + 1`
echo "${count} ... ... "
for i in `ls /dev/${LVGroupName} | grep -Ev 'snapshot$' | grep
-Ev 'swap$'`; do
if [ -h "/dev/${LVGroupName}/${i}" ] ; then
echo -n "lvcreate -s -v -n ${i}-snapshot -L
${snapshotsize} /dev/${LVGroupName}/${i} ... ... "
lvcreate -s -v -n ${i}-snapshot -L ${snapshotsize}
/dev/${LVGroupName}/${i}
echo "done."
sleep ${pauseinterval}
if [ -n "$commands" ] ; then
echo -n "${commands} ... ... "
$commands
echo "done."
fi
mkdir -p /mnt/testlvm/${i}
if [ -h "/dev/${LVGroupName}/${i}-snapshot" ] ; then
check_mount /mnt/testlvm/${i}
local ismount=$?
if [ "$ismount" -eq 0 ] ; then
echo -n "mount /dev/${LVGroupName}/${i}-snapshot
/mnt/testlvm/${i} ... ... "
mount /dev/${LVGroupName}/${i}-snapshot
/mnt/testlvm/${i}
echo "done."
sleep ${pauseinterval}
if [ -n "$commands" ] ; then
echo -n "${commands} ... ... "
$commands
echo "done."
fi
fi
check_mount /mnt/testlvm/${i}
local ismount2=$?
if [ "$ismount2" -eq 1 ] ; then
echo -n "umount /mnt/testlvm/${i} ... ... "
umount /mnt/testlvm/${i}
echo "done."
sleep ${pauseinterval}
if [ -n "$commands" ] ; then
echo -n "${commands} ... ... "
$commands
echo "done."
fi
fi
fi
rm -rf /mnt/testlvm/${i}
echo -n "lvremove -f /dev/${LVGroupName}/${i}-snapshot
... ... "
lvremove -f /dev/${LVGroupName}/${i}-snapshot
echo "done."
sleep ${pauseinterval}
if [ -n "$commands" ] ; then
echo -n "${commands} ... ... "
$commands
echo "done."
fi
fi
done
rm -fr /mnt/testlvm
done
else
echo "/dev/${LVGroupName} directory not found!"
exit 1
fi
}
case $1 in
loop) shift
do_lvm_create_remove "$@"
;;
*) cat <<HELP
Usage: $0 loop loopcountlimit snapshotsize pauseinterval commands
Where:
loopcountlimit is default to 1
snapshotsize is default to 1G
pauseinterval is default to 0
commands is default to none
Example to run with 100 loops without pause/sleep:
$0 loop 100
Example to run with 100 loops with pause/sleep of 5 seconds:
$0 loop 100 1G 5
Example to run with 100 loops with snapshot size of 2G instead of 1G:
$0 loop 100 2G
Example to run with 50 loops, 1G snapshot size, 5 seconds pause and with
sync:
command with each pause/sleep
$0 loop 50 1G 5 sync
Example to run with 50 loops, 1G snapshot size, no pause and with sync:
command with each pause/sleep
$0 loop 50 1G 0 sync
Example to run your own commands:
$0 loop 100 1G 5 "echo hi && sync"
HELP
;;
esac
----------8<----------8<----------8<----------8<----------8<----------8<----------8<----------8<----------
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 6076 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2010-12-26 8:16 kernel BUG at arch/x86/xen/mmu.c:1860! Teck Choon Giam
2010-12-27 15:53 ` Konrad Rzeszutek Wilk
@ 2011-01-04 13:48 ` Ian Campbell
2011-01-04 19:24 ` Teck Choon Giam
2011-01-13 14:28 ` tjaouen
2 siblings, 1 reply; 85+ messages in thread
From: Ian Campbell @ 2011-01-04 13:48 UTC (permalink / raw)
To: Teck Choon Giam; +Cc: xen-devel
On Sun, 2010-12-26 at 08:16 +0000, Teck Choon Giam wrote:
>
> Triggered BUG() in line 1860:
>
> static void pin_pagetable_pfn(unsigned cmd, unsigned long pfn)
> {
> struct mmuext_op op;
> op.cmd = cmd;
> op.arg1.mfn = pfn_to_mfn(pfn);
> if (HYPERVISOR_mmuext_op(&op, 1, NULL, DOMID_SELF))
> BUG(); <<THIS ONE?
> }
A failure to pin/unpin is usually associated with a log message from the
hypervisor. Please can you attempt to capture the full host log, e.g.
using serial console.
See http://wiki.xen.org/xenwiki/XenParavirtOps under "Are there more
debugging options I could enable to troubleshoot booting problems?" for
some details.
Ian.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2010-12-28 10:42 ` Pasi Kärkkäinen
2010-12-28 18:01 ` Teck Choon Giam
@ 2011-01-04 15:10 ` Christophe Saout
2011-01-04 15:19 ` Christophe Saout
` (2 more replies)
1 sibling, 3 replies; 85+ messages in thread
From: Christophe Saout @ 2011-01-04 15:10 UTC (permalink / raw)
To: xen-devel; +Cc: Teck Choon Giam, Konrad Rzeszutek Wilk
Hello thread,
hijacking this thread since I am running into the same issue on a new
machine.
> > > While doing LVM snapshot for migration and get the following:
> > >
> > > Dec 26 15:58:29 xen01 kernel: ------------[ cut here ]------------
> > > Dec 26 15:58:29 xen01 kernel: kernel BUG at arch/x86/xen/mmu.c:1860!
> > > Dec 26 15:58:29 xen01 kernel: invalid opcode: 0000 [#1] SMP
> > > Dec 26 15:58:29 xen01 kernel: last sysfs file: /sys/block/dm-26/dev
> > > Dec 26 15:58:29 xen01 kernel: CPU 0
> > > Dec 26 15:58:29 xen01 kernel: Modules linked in: ipt_MASQUERADE
>
> It would be very good to track this down and get it fixed..
> hopefully you're able to help a bit and try some things to debug it.
>
> Konrad maybe has some ideas to try..
I am seeing this with an lvcreate here, so I guess it's somehow related
to device-mapper stuff in general.
It doesn't look like this has been resolved yet. Somewhere I saw a
request for the hypervisor message related to the pinning failure.
Here it is:
(XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 1000000000000000) for mfn 41114f (pfn d514f)
(XEN) mm.c:2733:d0 Error while pinning mfn 41114f
I have a bit of experience in debugging things, so if I can help someone
with more information...
Cheers,
Christophe
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-04 15:10 ` Christophe Saout
@ 2011-01-04 15:19 ` Christophe Saout
2011-01-04 15:37 ` benco
2011-01-04 18:40 ` Christophe Saout
2011-01-04 23:10 ` Christophe Saout
2011-01-05 10:51 ` Pasi Kärkkäinen
2 siblings, 2 replies; 85+ messages in thread
From: Christophe Saout @ 2011-01-04 15:19 UTC (permalink / raw)
To: xen-devel; +Cc: Teck Choon Giam, Konrad Rzeszutek Wilk
Hi again,
> > > > While doing LVM snapshot for migration and get the following:
> > > >
> > > > Dec 26 15:58:29 xen01 kernel: ------------[ cut here ]------------
> > > > Dec 26 15:58:29 xen01 kernel: kernel BUG at arch/x86/xen/mmu.c:1860!
> > > > Dec 26 15:58:29 xen01 kernel: invalid opcode: 0000 [#1] SMP
> > > > Dec 26 15:58:29 xen01 kernel: last sysfs file: /sys/block/dm-26/dev
> > > > Dec 26 15:58:29 xen01 kernel: CPU 0
> > > > Dec 26 15:58:29 xen01 kernel: Modules linked in: ipt_MASQUERADE
> >
> > It would be very good to track this down and get it fixed..
> > hopefully you're able to help a bit and try some things to debug it.
> >
> > Konrad maybe has some ideas to try..
>
> I am seeing this with an lvcreate here, so I guess it's somehow related
> to device-mapper stuff in general.
>
> It doesn't look like this has been resolved yet. Somewhere I saw a
> request for the hypervisor message related to the pinning failure.
>
> Here it is:
>
> (XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 1000000000000000) for mfn 41114f (pfn d514f)
> (XEN) mm.c:2733:d0 Error while pinning mfn 41114f
>
> I have a bit of experience in debugging things, so if I can help someone
> with more information...
Additional information: This happened with a number of commands now.
However, I am running a multipath setup and every time the crash seemed
to be caused in the process context of the multipath daemon. I think
the daemon listens to events from the device-mapper subsystem to watch
for changes and the problem somehow arises from there, since on another
machine with the same XEN/Dom0 version without such a daemon I never had
any troubles with LVM.
[<ffffffff810052e2>] pin_pagetable_pfn+0x52/0x60
[<ffffffff81006f5c>] xen_alloc_ptpage+0x9c/0xa0
[<ffffffff81006f8e>] xen_alloc_pte+0xe/0x10
[<ffffffff810decde>] __pte_alloc+0x7e/0xf0
[<ffffffff810e15c5>] handle_mm_fault+0x855/0x930
[<ffffffff8102dd9e>] ? pvclock_clocksource_read+0x4e/0x100
[<ffffffff810e734c>] ? do_mmap_pgoff+0x33c/0x380
[<ffffffff81452b96>] do_page_fault+0x116/0x3e0
[<ffffffff8144ff65>] page_fault+0x25/0x30
Cheers,
Christophe
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-04 15:19 ` Christophe Saout
@ 2011-01-04 15:37 ` benco
2011-01-04 18:40 ` Christophe Saout
1 sibling, 0 replies; 85+ messages in thread
From: benco @ 2011-01-04 15:37 UTC (permalink / raw)
To: Christophe Saout; +Cc: xen-devel, Teck Choon Giam, Konrad Rzeszutek Wilk
Hi,
the same issue on old IBM x335 servers with QLogic ISP2312 HBAs and
multipath setup. Newer hardware (x3550M2 + QLA2462 HBA) seems to be fine.
I've tried several versions of pvops kernel and xen hypervisor from 4.x line
with the same results.
Roman
On Tue, Jan 04, 2011 at 04:19:02PM +0100, Christophe Saout wrote:
> Hi again,
>
> > > > > While doing LVM snapshot for migration and get the following:
> > > > >
> > > > > Dec 26 15:58:29 xen01 kernel: ------------[ cut here ]------------
> > > > > Dec 26 15:58:29 xen01 kernel: kernel BUG at arch/x86/xen/mmu.c:1860!
> > > > > Dec 26 15:58:29 xen01 kernel: invalid opcode: 0000 [#1] SMP
> > > > > Dec 26 15:58:29 xen01 kernel: last sysfs file: /sys/block/dm-26/dev
> > > > > Dec 26 15:58:29 xen01 kernel: CPU 0
> > > > > Dec 26 15:58:29 xen01 kernel: Modules linked in: ipt_MASQUERADE
> > >
> > > It would be very good to track this down and get it fixed..
> > > hopefully you're able to help a bit and try some things to debug it.
> > >
> > > Konrad maybe has some ideas to try..
> >
> > I am seeing this with an lvcreate here, so I guess it's somehow related
> > to device-mapper stuff in general.
> >
> > It doesn't look like this has been resolved yet. Somewhere I saw a
> > request for the hypervisor message related to the pinning failure.
> >
> > Here it is:
> >
> > (XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 1000000000000000) for mfn 41114f (pfn d514f)
> > (XEN) mm.c:2733:d0 Error while pinning mfn 41114f
> >
> > I have a bit of experience in debugging things, so if I can help someone
> > with more information...
>
> Additional information: This happened with a number of commands now.
> However, I am running a multipath setup and every time the crash seemed
> to be caused in the process context of the multipath daemon. I think
> the daemon listens to events from the device-mapper subsystem to watch
> for changes and the problem somehow arises from there, since on another
> machine with the same XEN/Dom0 version without such a daemon I never had
> any troubles with LVM.
>
> [<ffffffff810052e2>] pin_pagetable_pfn+0x52/0x60
> [<ffffffff81006f5c>] xen_alloc_ptpage+0x9c/0xa0
> [<ffffffff81006f8e>] xen_alloc_pte+0xe/0x10
> [<ffffffff810decde>] __pte_alloc+0x7e/0xf0
> [<ffffffff810e15c5>] handle_mm_fault+0x855/0x930
> [<ffffffff8102dd9e>] ? pvclock_clocksource_read+0x4e/0x100
> [<ffffffff810e734c>] ? do_mmap_pgoff+0x33c/0x380
> [<ffffffff81452b96>] do_page_fault+0x116/0x3e0
> [<ffffffff8144ff65>] page_fault+0x25/0x30
>
> Cheers,
> Christophe
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
--
----------------------------------------------------------------------
,''`. [benco] | mailto: benco@acid.sk | silc: /msg benco
: :' : -------------------------------------------------------------
`. `' GPG publickey: http://www.acid.sk/pubkey.asc
`- KF = 0DF6 0592 74D2 F17A DACF A5C3 1720 CB7C F54C F429
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-04 15:19 ` Christophe Saout
2011-01-04 15:37 ` benco
@ 2011-01-04 18:40 ` Christophe Saout
2011-01-04 19:32 ` Teck Choon Giam
2011-01-14 15:22 ` Konrad Rzeszutek Wilk
1 sibling, 2 replies; 85+ messages in thread
From: Christophe Saout @ 2011-01-04 18:40 UTC (permalink / raw)
To: xen-devel; +Cc: Teck Choon Giam, Konrad Rzeszutek Wilk
Hi once more,
> > It doesn't look like this has been resolved yet. Somewhere I saw a
> > request for the hypervisor message related to the pinning failure.
> >
> > Here it is:
> >
> > (XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 1000000000000000) for mfn 41114f (pfn d514f)
> > (XEN) mm.c:2733:d0 Error while pinning mfn 41114f
> >
> > I have a bit of experience in debugging things, so if I can help someone
> > with more information...
> [<ffffffff810052e2>] pin_pagetable_pfn+0x52/0x60
> [<ffffffff81006f5c>] xen_alloc_ptpage+0x9c/0xa0
> [<ffffffff81006f8e>] xen_alloc_pte+0xe/0x10
> [<ffffffff810decde>] __pte_alloc+0x7e/0xf0
> [<ffffffff810e15c5>] handle_mm_fault+0x855/0x930
> [<ffffffff8102dd9e>] ? pvclock_clocksource_read+0x4e/0x100
> [<ffffffff810e734c>] ? do_mmap_pgoff+0x33c/0x380
> [<ffffffff81452b96>] do_page_fault+0x116/0x3e0
> [<ffffffff8144ff65>] page_fault+0x25/0x30
> Additional information: This happened with a number of commands now.
> However, I am running a multipath setup and every time the crash
> seemed to be caused in the process context of the multipath daemon.
> I think the daemon listens to events from the device-mapper subsystem
> to watch for changes and the problem somehow arises from there, since
> on another machine with the same XEN/Dom0 version without such a
> daemon I never had any troubles with LVM.
On further investigation is seems that most of the time the issue is not
caused by the daemon, but by the "multipath" tool, which is used a lot
by udev to identify properties of block devices.
When I start stracing udevd (following forks), I'm not able to reproduce
the crash anymore. So I was hoping to find out what the process was
doing before the crash occurs, but since my attempts to trace the
process masks the bug, I can't. :(
(without strace, the bug is very common, about every third "lvcreate"
command. Every lvcreate command triggers about 20 multipath
invocations)
Christophe
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-04 13:48 ` Ian Campbell
@ 2011-01-04 19:24 ` Teck Choon Giam
2011-01-05 15:30 ` Teck Choon Giam
0 siblings, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2011-01-04 19:24 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 962 bytes --]
On Tue, Jan 4, 2011 at 9:48 PM, Ian Campbell <Ian.Campbell@citrix.com>wrote:
> On Sun, 2010-12-26 at 08:16 +0000, Teck Choon Giam wrote:
> >
> > Triggered BUG() in line 1860:
> >
> > static void pin_pagetable_pfn(unsigned cmd, unsigned long pfn)
> > {
> > struct mmuext_op op;
> > op.cmd = cmd;
> > op.arg1.mfn = pfn_to_mfn(pfn);
> > if (HYPERVISOR_mmuext_op(&op, 1, NULL, DOMID_SELF))
> > BUG(); <<THIS ONE?
> > }
>
> A failure to pin/unpin is usually associated with a log message from the
> hypervisor. Please can you attempt to capture the full host log, e.g.
> using serial console.
>
> See http://wiki.xen.org/xenwiki/XenParavirtOps under "Are there more
> debugging options I could enable to troubleshoot booting problems?" for
> some details.
>
I will once I got the serial console cable and have a system to catch the
log during my next visit to the DC.
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 1462 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-04 18:40 ` Christophe Saout
@ 2011-01-04 19:32 ` Teck Choon Giam
2011-01-04 19:56 ` benco
2011-01-14 15:22 ` Konrad Rzeszutek Wilk
1 sibling, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2011-01-04 19:32 UTC (permalink / raw)
To: Christophe Saout; +Cc: xen-devel, Konrad Rzeszutek Wilk
[-- Attachment #1.1: Type: text/plain, Size: 2887 bytes --]
On Wed, Jan 5, 2011 at 2:40 AM, Christophe Saout <christophe@saout.de>wrote:
> Hi once more,
>
>
> > > It doesn't look like this has been resolved yet. Somewhere I saw a
> > > request for the hypervisor message related to the pinning failure.
> > >
> > > Here it is:
> > >
> > > (XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp
> 1000000000000000) for mfn 41114f (pfn d514f)
> > > (XEN) mm.c:2733:d0 Error while pinning mfn 41114f
> > >
> > > I have a bit of experience in debugging things, so if I can help
> someone
> > > with more information...
> > [<ffffffff810052e2>] pin_pagetable_pfn+0x52/0x60
> > [<ffffffff81006f5c>] xen_alloc_ptpage+0x9c/0xa0
> > [<ffffffff81006f8e>] xen_alloc_pte+0xe/0x10
> > [<ffffffff810decde>] __pte_alloc+0x7e/0xf0
> > [<ffffffff810e15c5>] handle_mm_fault+0x855/0x930
> > [<ffffffff8102dd9e>] ? pvclock_clocksource_read+0x4e/0x100
> > [<ffffffff810e734c>] ? do_mmap_pgoff+0x33c/0x380
> > [<ffffffff81452b96>] do_page_fault+0x116/0x3e0
> > [<ffffffff8144ff65>] page_fault+0x25/0x30
>
> > Additional information: This happened with a number of commands now.
> > However, I am running a multipath setup and every time the crash
> > seemed to be caused in the process context of the multipath daemon.
> > I think the daemon listens to events from the device-mapper subsystem
> > to watch for changes and the problem somehow arises from there, since
> > on another machine with the same XEN/Dom0 version without such a
> > daemon I never had any troubles with LVM.
>
> On further investigation is seems that most of the time the issue is not
> caused by the daemon, but by the "multipath" tool, which is used a lot
> by udev to identify properties of block devices.
>
> When I start stracing udevd (following forks), I'm not able to reproduce
> the crash anymore. So I was hoping to find out what the process was
> doing before the crash occurs, but since my attempts to trace the
> process masks the bug, I can't. :(
>
> (without strace, the bug is very common, about every third "lvcreate"
> command. Every lvcreate command triggers about 20 multipath
> invocations)
>
>
I am able to prevent that bug for 8 days (till now) by implementing sleep 5
seconds then syc then sleep 5 seconds then sync repeating this for 60
seconds while doing lvm snapshot for 10 domUs. I mean:
1. lvm snapshot domU (lvcreate)
2. mount lvm snapsho domUt
3. rsync to backup domU
4. umount lvm snapshot domU
5. remove lvm snapshot domU (lvremove)
6. sync (start countdown of 60 seconds and every 5 seconds interval doing
sync)
7. sleep 5
8. sync
9. sleep 5
10. sync
11. sleep 5
12. sync
.... until it hits 0 second countdown
Then next domU repeat the cycle.
Doing the above I am able to prevent such crash or bug to pop up for 8 days
(8 such daily LVM snapshot backup for all domUs) which I posted in this
thread.
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 3622 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-04 19:32 ` Teck Choon Giam
@ 2011-01-04 19:56 ` benco
0 siblings, 0 replies; 85+ messages in thread
From: benco @ 2011-01-04 19:56 UTC (permalink / raw)
To: Teck Choon Giam; +Cc: xen-devel, Christophe Saout, Konrad Rzeszutek Wilk
Hi,
I use the same workaround to start domUs, since the issue mentioned bellow
ocured also during domU start via xendomains script.
Another scenario causing this issue (at least for me) are cron scripts. I was
unable to find out which one is responsible (crash every 2-3 days), but the
issue went away with disabled cronscripts.
Roman
On Wed, Jan 05, 2011 at 03:32:58AM +0800, Teck Choon Giam wrote:
> On Wed, Jan 5, 2011 at 2:40 AM, Christophe Saout <christophe@saout.de>wrote:
>
> > Hi once more,
> >
> >
> > > > It doesn't look like this has been resolved yet. Somewhere I saw a
> > > > request for the hypervisor message related to the pinning failure.
> > > >
> > > > Here it is:
> > > >
> > > > (XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp
> > 1000000000000000) for mfn 41114f (pfn d514f)
> > > > (XEN) mm.c:2733:d0 Error while pinning mfn 41114f
> > > >
> > > > I have a bit of experience in debugging things, so if I can help
> > someone
> > > > with more information...
> > > [<ffffffff810052e2>] pin_pagetable_pfn+0x52/0x60
> > > [<ffffffff81006f5c>] xen_alloc_ptpage+0x9c/0xa0
> > > [<ffffffff81006f8e>] xen_alloc_pte+0xe/0x10
> > > [<ffffffff810decde>] __pte_alloc+0x7e/0xf0
> > > [<ffffffff810e15c5>] handle_mm_fault+0x855/0x930
> > > [<ffffffff8102dd9e>] ? pvclock_clocksource_read+0x4e/0x100
> > > [<ffffffff810e734c>] ? do_mmap_pgoff+0x33c/0x380
> > > [<ffffffff81452b96>] do_page_fault+0x116/0x3e0
> > > [<ffffffff8144ff65>] page_fault+0x25/0x30
> >
> > > Additional information: This happened with a number of commands now.
> > > However, I am running a multipath setup and every time the crash
> > > seemed to be caused in the process context of the multipath daemon.
> > > I think the daemon listens to events from the device-mapper subsystem
> > > to watch for changes and the problem somehow arises from there, since
> > > on another machine with the same XEN/Dom0 version without such a
> > > daemon I never had any troubles with LVM.
> >
> > On further investigation is seems that most of the time the issue is not
> > caused by the daemon, but by the "multipath" tool, which is used a lot
> > by udev to identify properties of block devices.
> >
> > When I start stracing udevd (following forks), I'm not able to reproduce
> > the crash anymore. So I was hoping to find out what the process was
> > doing before the crash occurs, but since my attempts to trace the
> > process masks the bug, I can't. :(
> >
> > (without strace, the bug is very common, about every third "lvcreate"
> > command. Every lvcreate command triggers about 20 multipath
> > invocations)
> >
> >
> I am able to prevent that bug for 8 days (till now) by implementing sleep 5
> seconds then syc then sleep 5 seconds then sync repeating this for 60
> seconds while doing lvm snapshot for 10 domUs. I mean:
>
> 1. lvm snapshot domU (lvcreate)
> 2. mount lvm snapsho domUt
> 3. rsync to backup domU
> 4. umount lvm snapshot domU
> 5. remove lvm snapshot domU (lvremove)
> 6. sync (start countdown of 60 seconds and every 5 seconds interval doing
> sync)
> 7. sleep 5
> 8. sync
> 9. sleep 5
> 10. sync
> 11. sleep 5
> 12. sync
> .... until it hits 0 second countdown
> Then next domU repeat the cycle.
>
> Doing the above I am able to prevent such crash or bug to pop up for 8 days
> (8 such daily LVM snapshot backup for all domUs) which I posted in this
> thread.
>
> Thanks.
>
> Kindest regards,
> Giam Teck Choon
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
--
----------------------------------------------------------------------
,''`. [benco] | mailto: benco@acid.sk | silc: /msg benco
: :' : -------------------------------------------------------------
`. `' GPG publickey: http://www.acid.sk/pubkey.asc
`- KF = 0DF6 0592 74D2 F17A DACF A5C3 1720 CB7C F54C F429
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-04 15:10 ` Christophe Saout
2011-01-04 15:19 ` Christophe Saout
@ 2011-01-04 23:10 ` Christophe Saout
2011-01-05 10:51 ` Pasi Kärkkäinen
2 siblings, 0 replies; 85+ messages in thread
From: Christophe Saout @ 2011-01-04 23:10 UTC (permalink / raw)
To: xen-devel; +Cc: Teck Choon Giam, Konrad Rzeszutek Wilk
Hi,
> > > > While doing LVM snapshot for migration and get the following:
> > > >
> > > > Dec 26 15:58:29 xen01 kernel: ------------[ cut here ]------------
> > > > Dec 26 15:58:29 xen01 kernel: kernel BUG at arch/x86/xen/mmu.c:1860!
> > > > Dec 26 15:58:29 xen01 kernel: invalid opcode: 0000 [#1] SMP
> > > > Dec 26 15:58:29 xen01 kernel: last sysfs file: /sys/block/dm-26/dev
> > > > Dec 26 15:58:29 xen01 kernel: CPU 0
> > > > Dec 26 15:58:29 xen01 kernel: Modules linked in: ipt_MASQUERADE
>
> [...]
> [<ffffffff810052e2>] pin_pagetable_pfn+0x52/0x60
> [<ffffffff81006f5c>] xen_alloc_ptpage+0x9c/0xa0
> [<ffffffff81006f8e>] xen_alloc_pte+0xe/0x10
> [<ffffffff810decde>] __pte_alloc+0x7e/0xf0
> [<ffffffff810e15c5>] handle_mm_fault+0x855/0x930
> [<ffffffff8102dd9e>] ? pvclock_clocksource_read+0x4e/0x100
> [<ffffffff810e734c>] ? do_mmap_pgoff+0x33c/0x380
> [<ffffffff81452b96>] do_page_fault+0x116/0x3e0
> [<ffffffff8144ff65>] page_fault+0x25/0x30
> [...]
> (XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 1000000000000000) for mfn 41114f (pfn d514f)
> (XEN) mm.c:2733:d0 Error while pinning mfn 41114f
Looking into the code, the Dom0 code ist attempting to pin what it thins
is a "PGT_l1_page_table", however the hypervisor returns -EINVAL because
it actually is a "PGT_writable_page".
After a few hours I managed to catch the crash while the offending
process is being straced. However the results where totally
inconclusive, because the last lines before the crash are:
16576 open("/lib/multipath/libcheckdirectio.so", O_RDONLY) = 4
16576 read(4, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\v\0\0\0\0\0\0"..., 832) = 832
16576 fstat(4, {st_mode=S_IFREG|0644, st_size=9344, ...}) = 0
16576 mmap(NULL, 2104672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x7fa6b36f6000
16576 mprotect(0x7fa6b36f8000, 2093056, PROT_NONE) = 0
16576 mmap(0x7fa6b38f7000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x1000) = 0x7fa6b38f7000
16576 close(4) = 0
A non-crashing execution would have continued with:
16667 open("/etc/ld.so.cache", O_RDONLY) = 4
16667 fstat(4, {st_mode=S_IFREG|0644, st_size=21739, ...}) = 0
16667 mmap(NULL, 21739, PROT_READ, MAP_PRIVATE, 4, 0) = 0x7f237de56000
16667 close(4) = 0
16667 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
16667 open("/lib/libaio.so.1", O_RDONLY) = 4
[...]
Which means that it crashed during the dynamic loading of a plugin
shared library and not while interacting with the device mapper.
(also, the device being investigated was /dev/sde and not some dm
device)
This leads me to believe that some device-mapper shared library has a
particular memory layout that tends to trigger this crash and it has
nothing to do with any device-mapper code at all. Also, the crash seems
to be timing-sensitive, so it might also be a race condition of some
sort. (on a side-note: this is a 24-core machine (!) and the kernel has
happens to have full preemption enabled).
I am trying to understand the code a bit. Can someone explain to me
what xen_alloc_ptpage is doing.
> /* This needs to make sure the new pte page is pinned iff its being
> attached to a pinned pagetable. */
> [...]
> if (PagePinned(virt_to_page(mm->pgd))) {
> [...]
> pin_pagetable_pfn(MMUEXT_PIN_L1_TABLE, pfn);
I must admit I don't know very much about memory handling in linux (so
please excuse me if I am interpreting total nonsense into this here,
still I'm intigued and would like to understand it a bit better), but
isn't `mm->pgd' supposed to point to the L1 page table and `pfn', being
a pte page a 3rd/4th level page? Is this a code path that is exercised
a lot?
Thanks,
Christophe
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-04 15:10 ` Christophe Saout
2011-01-04 15:19 ` Christophe Saout
2011-01-04 23:10 ` Christophe Saout
@ 2011-01-05 10:51 ` Pasi Kärkkäinen
2011-01-05 14:56 ` Teck Choon Giam
2 siblings, 1 reply; 85+ messages in thread
From: Pasi Kärkkäinen @ 2011-01-05 10:51 UTC (permalink / raw)
To: Christophe Saout; +Cc: xen-devel, Teck Choon Giam, Konrad Rzeszutek Wilk
On Tue, Jan 04, 2011 at 04:10:17PM +0100, Christophe Saout wrote:
> Hello thread,
>
> hijacking this thread since I am running into the same issue on a new
> machine.
>
> > > > While doing LVM snapshot for migration and get the following:
> > > >
> > > > Dec 26 15:58:29 xen01 kernel: ------------[ cut here ]------------
> > > > Dec 26 15:58:29 xen01 kernel: kernel BUG at arch/x86/xen/mmu.c:1860!
> > > > Dec 26 15:58:29 xen01 kernel: invalid opcode: 0000 [#1] SMP
> > > > Dec 26 15:58:29 xen01 kernel: last sysfs file: /sys/block/dm-26/dev
> > > > Dec 26 15:58:29 xen01 kernel: CPU 0
> > > > Dec 26 15:58:29 xen01 kernel: Modules linked in: ipt_MASQUERADE
> >
> > It would be very good to track this down and get it fixed..
> > hopefully you're able to help a bit and try some things to debug it.
> >
> > Konrad maybe has some ideas to try..
>
> I am seeing this with an lvcreate here, so I guess it's somehow related
> to device-mapper stuff in general.
>
Sorry if this was already stated earlier..
what are the exact steps to reproduce? I could try reproducing it at some point..
-- Pasi
> It doesn't look like this has been resolved yet. Somewhere I saw a
> request for the hypervisor message related to the pinning failure.
>
> Here it is:
>
> (XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 1000000000000000) for mfn 41114f (pfn d514f)
> (XEN) mm.c:2733:d0 Error while pinning mfn 41114f
>
> I have a bit of experience in debugging things, so if I can help someone
> with more information...
>
> Cheers,
> Christophe
>
>
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-05 10:51 ` Pasi Kärkkäinen
@ 2011-01-05 14:56 ` Teck Choon Giam
2011-01-14 15:24 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2011-01-05 14:56 UTC (permalink / raw)
To: Pasi Kärkkäinen
Cc: xen-devel, Christophe Saout, Konrad Rzeszutek Wilk
[-- Attachment #1.1: Type: text/plain, Size: 8774 bytes --]
On Wed, Jan 5, 2011 at 6:51 PM, Pasi Kärkkäinen <pasik@iki.fi> wrote:
> On Tue, Jan 04, 2011 at 04:10:17PM +0100, Christophe Saout wrote:
> > Hello thread,
> >
> > hijacking this thread since I am running into the same issue on a new
> > machine.
> >
> > > > > While doing LVM snapshot for migration and get the following:
> > > > >
> > > > > Dec 26 15:58:29 xen01 kernel: ------------[ cut here
> ]------------
> > > > > Dec 26 15:58:29 xen01 kernel: kernel BUG at
> arch/x86/xen/mmu.c:1860!
> > > > > Dec 26 15:58:29 xen01 kernel: invalid opcode: 0000 [#1] SMP
> > > > > Dec 26 15:58:29 xen01 kernel: last sysfs file:
> /sys/block/dm-26/dev
> > > > > Dec 26 15:58:29 xen01 kernel: CPU 0
> > > > > Dec 26 15:58:29 xen01 kernel: Modules linked in:
> ipt_MASQUERADE
> > >
> > > It would be very good to track this down and get it fixed..
> > > hopefully you're able to help a bit and try some things to debug it.
> > >
> > > Konrad maybe has some ideas to try..
> >
> > I am seeing this with an lvcreate here, so I guess it's somehow related
> > to device-mapper stuff in general.
> >
>
> Sorry if this was already stated earlier..
> what are the exact steps to reproduce? I could try reproducing it at some
> point..
>
> -- Pasi
>
Did you try my posted script? Provided you have existing LV for domUs in
your VG which can be easily created if not there.
The idea is to create snapshot, mount it, umount it, remove snapshot and
repeat this cycle in loop will catch this BUG!!!
Here are the latest crash with serial console output as it doesn't take long
with sh test.sh loop 100 to produce this:
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
(XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 1000000000000000)
for mfn 1f7e29 (pfn 25cb4)
(XEN) mm.c:2733:d0 Error while pinning mfn 1f7e29
------------[ cut here ]------------
kernel BUG at arch/x86/xen/mmu.c:1860!
invalid opcode: 0000 [#1] SMP
last sysfs file:
/sys/devices/pci0000:00/0000:00:1e.0/0000:05:05.0/local_cpus
CPU 0
Modules linked in: ext4 jbd2 crc16 gfs2 dlm configfs xt_physdev
iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_]
Pid: 6526, comm: dmsetup Not tainted 2.6.32.27-0.xen.pvops.choon.centos5 #1
PowerEdge 860
RIP: e030:[<ffffffff8100cb5b>] [<ffffffff8100cb5b>]
pin_pagetable_pfn+0x53/0x59
RSP: e02b:ffff88001d8dfdc8 EFLAGS: 00010282
RAX: 00000000ffffffea RBX: 0000000000025cb4 RCX: 000000000000012e
RDX: 00000000deadbeef RSI: 00000000deadbeef RDI: 00000000deadbeef
RBP: ffff88001d8dfde8 R08: 00000000000005a0 R09: ffff880000000000
R10: 00000000deadbeef R11: 0000003db6814e00 R12: 0000000000000003
R13: 0000000000025cb4 R14: ffff88002ffe8440 R15: 0000003db7616250
FS: 00007fb54068b710(0000) GS:ffff88002804f000(0000) knlGS:0000000000000000
CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000003db7616250 CR3: 000000002ff88000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process dmsetup (pid: 6526, threadinfo ffff88001d8de000, task
ffff88002ffe8440)
Stack:
0000000000000000 00000000001f7e29 000000013f009e18 0000000000025cb4
<0> ffff88001d8dfe08 ffffffff8100e07c ffff88001dc3d040 ffff88001dc6cdd8
<0> ffff88001d8dfe18 ffffffff8100e0af ffff88001d8dfe58 ffffffff810a402f
Call Trace:
[<ffffffff8100e07c>] xen_alloc_ptpage+0x64/0x69
[<ffffffff8100e0af>] xen_alloc_pte+0xe/0x10
[<ffffffff810a402f>] __pte_alloc+0x70/0xce
[<ffffffff810a41cd>] handle_mm_fault+0x140/0x8b9
[<ffffffff8131be4d>] do_page_fault+0x252/0x2e2
[<ffffffff81319dd5>] page_fault+0x25/0x30
Code: 48 b8 ff ff ff ff ff ff ff 7f 48 21 c2 48 89 55 e8 48 8d 7d e0 be 01
00 00 00 31 d2 41 ba f0 7f 00 00 e8 e9 c7 ff ff 8
RIP [<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
RSP <ffff88001d8dfdc8>
---[ end trace b0a2643219f652eb ]---
BUG: soft lockup - CPU#0 stuck for 61s! [dmsetup:6526]
Modules linked in: ext4 jbd2 crc16 gfs2 dlm configfs xt_physdev
iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_]
CPU 0:
Modules linked in: ext4 jbd2 crc16 gfs2 dlm configfs xt_physdev
iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_]
Pid: 6526, comm: dmsetup Tainted: G D
2.6.32.27-0.xen.pvops.choon.centos5 #1 PowerEdge 860
RIP: e030:[<ffffffff813199d3>] [<ffffffff813199d3>] _spin_lock+0x19/0x20
RSP: e02b:ffff88001d8dfa68 EFLAGS: 00000297
RAX: 0000000000000023 RBX: 0000000025f91000 RCX: 0000000000000004
RDX: 0000000000000022 RSI: 0000000000000004 RDI: ffff88001dc3d0c0
RBP: ffff88001d8dfa68 R08: 0000000000000000 R09: ffffffff816dd100
R10: ffff88003e7424c8 R11: 0000000000000020 R12: ffff88001dc3d040
R13: 0000000000000004 R14: ffff88001dc3d0a0 R15: ffffffff816dd100
FS: 00007fb54068b710(0000) GS:ffff88002804f000(0000) knlGS:0000000000000000
CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000003db7616250 CR3: 0000000001001000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff811660dd>] ? free_cpumask_var+0x9/0xb
[<ffffffff8100dde1>] xen_exit_mmap+0x199/0x1d7
[<ffffffff810a8137>] exit_mmap+0x5f/0x14b
[<ffffffff81048648>] mmput+0x46/0xb2
[<ffffffff8104c552>] exit_mm+0xfd/0x108
[<ffffffff8100f799>] ? xen_irq_enable_direct_end+0x0/0x7
[<ffffffff8104d7ee>] do_exit+0x1f3/0x67b
[<ffffffff8131a908>] oops_end+0xba/0xc2
[<ffffffff810163a1>] die+0x55/0x5e
[<ffffffff8131a192>] do_trap+0x110/0x11f
[<ffffffff810142c8>] do_invalid_op+0x97/0xa0
[<ffffffff8100cb5b>] ? pin_pagetable_pfn+0x53/0x59
[<ffffffff810138bb>] invalid_op+0x1b/0x20
[<ffffffff8100cb5b>] ? pin_pagetable_pfn+0x53/0x59
[<ffffffff8100cb57>] ? pin_pagetable_pfn+0x4f/0x59
[<ffffffff8100e07c>] xen_alloc_ptpage+0x64/0x69
[<ffffffff8100e0af>] xen_alloc_pte+0xe/0x10
[<ffffffff810a402f>] __pte_alloc+0x70/0xce
[<ffffffff810a41cd>] handle_mm_fault+0x140/0x8b9
[<ffffffff8131be4d>] do_page_fault+0x252/0x2e2
[<ffffffff81319dd5>] page_fault+0x25/0x30
Kernel panic - not syncing: softlockup: hung tasks
Pid: 6526, comm: dmsetup Tainted: G D
2.6.32.27-0.xen.pvops.choon.centos5 #1
Call Trace:
<IRQ> [<ffffffff8104aa97>] panic+0xa0/0x15f
[<ffffffff81319dd5>] ? page_fault+0x25/0x30
[<ffffffff8101640f>] ? show_trace_log_lvl+0x4c/0x58
[<ffffffff8101642b>] ? show_trace+0x10/0x12
[<ffffffff81011755>] ? show_regs+0x44/0x48
[<ffffffff8107f202>] softlockup_tick+0x173/0x182
[<ffffffff810539bf>] run_local_timers+0x18/0x1a
[<ffffffff81053bde>] update_process_times+0x30/0x54
[<ffffffff81068821>] tick_sched_timer+0x70/0x99
[<ffffffff8105f52e>] __run_hrtimer+0x53/0xb3
[<ffffffff8105f772>] hrtimer_interrupt+0xae/0x192
[<ffffffff8100f3a3>] xen_timer_interrupt+0x37/0x181
[<ffffffff81082898>] ? check_for_new_grace_period+0x97/0xa5
[<ffffffff811c870f>] ? unmask_evtchn+0x34/0xd6
[<ffffffff8108318c>] ? __rcu_process_callbacks+0xf2/0x2ae
[<ffffffff8107f708>] handle_IRQ_event+0x2d/0xb7
[<ffffffff81081079>] handle_percpu_irq+0x3c/0x69
[<ffffffff811c8640>] __xen_evtchn_do_upcall+0xe1/0x168
[<ffffffff811c92d1>] xen_evtchn_do_upcall+0x2e/0x41
[<ffffffff81013c7e>] xen_do_hypervisor_callback+0x1e/0x30
<EOI> [<ffffffff813199d3>] ? _spin_lock+0x19/0x20
[<ffffffff811660dd>] ? free_cpumask_var+0x9/0xb
[<ffffffff8100dde1>] ? xen_exit_mmap+0x199/0x1d7
[<ffffffff810a8137>] ? exit_mmap+0x5f/0x14b
[<ffffffff81048648>] ? mmput+0x46/0xb2
[<ffffffff8104c552>] ? exit_mm+0xfd/0x108
[<ffffffff8100f799>] ? xen_irq_enable_direct_end+0x0/0x7
[<ffffffff8104d7ee>] ? do_exit+0x1f3/0x67b
[<ffffffff8131a908>] ? oops_end+0xba/0xc2
[<ffffffff810163a1>] ? die+0x55/0x5e
[<ffffffff8131a192>] ? do_trap+0x110/0x11f
[<ffffffff810142c8>] ? do_invalid_op+0x97/0xa0
[<ffffffff8100cb5b>] ? pin_pagetable_pfn+0x53/0x59
[<ffffffff810138bb>] ? invalid_op+0x1b/0x20
[<ffffffff8100cb5b>] ? pin_pagetable_pfn+0x53/0x59
[<ffffffff8100cb57>] ? pin_pagetable_pfn+0x4f/0x59
[<ffffffff8100e07c>] ? xen_alloc_ptpage+0x64/0x69
[<ffffffff8100e0af>] ? xen_alloc_pte+0xe/0x10
[<ffffffff810a402f>] ? __pte_alloc+0x70/0xce
[<ffffffff810a41cd>] ? handle_mm_fault+0x140/0x8b9
[<ffffffff8131be4d>] ? do_page_fault+0x252/0x2e2
[<ffffffff81319dd5>] ? page_fault+0x25/0x30
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 10087 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-04 19:24 ` Teck Choon Giam
@ 2011-01-05 15:30 ` Teck Choon Giam
0 siblings, 0 replies; 85+ messages in thread
From: Teck Choon Giam @ 2011-01-05 15:30 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 11155 bytes --]
On Wed, Jan 5, 2011 at 3:24 AM, Teck Choon Giam <giamteckchoon@gmail.com>wrote:
>
>
> On Tue, Jan 4, 2011 at 9:48 PM, Ian Campbell <Ian.Campbell@citrix.com>wrote:
>
>> On Sun, 2010-12-26 at 08:16 +0000, Teck Choon Giam wrote:
>> >
>> > Triggered BUG() in line 1860:
>> >
>> > static void pin_pagetable_pfn(unsigned cmd, unsigned long pfn)
>> > {
>> > struct mmuext_op op;
>> > op.cmd = cmd;
>> > op.arg1.mfn = pfn_to_mfn(pfn);
>> > if (HYPERVISOR_mmuext_op(&op, 1, NULL, DOMID_SELF))
>> > BUG(); <<THIS ONE?
>> > }
>>
>> A failure to pin/unpin is usually associated with a log message from the
>> hypervisor. Please can you attempt to capture the full host log, e.g.
>> using serial console.
>>
>> See http://wiki.xen.org/xenwiki/XenParavirtOps under "Are there more
>> debugging options I could enable to troubleshoot booting problems?" for
>> some details.
>>
>
> I will once I got the serial console cable and have a system to catch the
> log during my next visit to the DC.
>
>
Hi Ian,
Here is another console output besides the other one which I posted:
EXT3-fs warning: maximal mount count reached, running e2fsck is
recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is
recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is
recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is
recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is
recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is
recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is
recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is
recommended
INIT: Id "s1" respawning too fast: disabled for 5
minutes
EXT3-fs warning: maximal mount count reached, running e2fsck is
recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is
recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is
recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is
recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
hrtimer: interrupt took 3096797 ns
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
(XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 1000000000000000)
for mfn 1ee744 (pfn 1c399)
(XEN) mm.c:2733:d0 Error while pinning mfn 1ee744
------------[ cut here ]------------
kernel BUG at arch/x86/xen/mmu.c:1860!
invalid opcode: 0000 [#1] SMP
last sysfs file: /sys/block/dm-17/dev
CPU 1
Modules linked in: ext4 jbd2 crc16 gfs2 dlm configfs xt_physdev
iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_]
Pid: 19758, comm: dmsetup Not tainted 2.6.32.27-0.xen.pvops.choon.centos5 #1
PowerEdge 860
RIP: e030:[<ffffffff8100cb5b>] [<ffffffff8100cb5b>]
pin_pagetable_pfn+0x53/0x59
RSP: e02b:ffff88003a615dc8 EFLAGS: 00010282
RAX: 00000000ffffffea RBX: 000000000001c399 RCX: 00000000000000e1
RDX: 00000000deadbeef RSI: 00000000deadbeef RDI: 00000000deadbeef
RBP: ffff88003a615de8 R08: 0000000000000cc8 R09: ffff880000000000
R10: 00000000deadbeef R11: 0000000000000246 R12: 0000000000000003
R13: 000000000001c399 R14: ffff88001c1d86c0 R15: 0000003db7400258
FS: 00007f6cbd5b6710(0000) GS:ffff88002806c000(0000) knlGS:0000000000000000
CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000003db7400258 CR3: 000000003a5ed000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process dmsetup (pid: 19758, threadinfo ffff88003a614000, task
ffff88001c1d86c0)
Stack:
0000000000000000 00000000001ee744 000000013e64d518 000000000001c399
<0> ffff88003a615e08 ffffffff8100e07c ffff880027a2c580 ffff88003a56fdd0
<0> ffff88003a615e18 ffffffff8100e0af ffff88003a615e58 ffffffff810a402f
Call Trace:
[<ffffffff8100e07c>] xen_alloc_ptpage+0x64/0x69
[<ffffffff8100e0af>] xen_alloc_pte+0xe/0x10
[<ffffffff810a402f>] __pte_alloc+0x70/0xce
[<ffffffff810a41cd>] handle_mm_fault+0x140/0x8b9
[<ffffffff8131be4d>] do_page_fault+0x252/0x2e2
[<ffffffff81319dd5>] page_fault+0x25/0x30
Code: 48 b8 ff ff ff ff ff ff ff 7f 48 21 c2 48 89 55 e8 48 8d 7d e0 be 01
00 00 00 31 d2 41 ba f0 7f 00 00 e8 e9 c7 ff ff 8
RIP [<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
RSP <ffff88003a615dc8>
---[ end trace 63676fea977b3461 ]---
BUG: soft lockup - CPU#1 stuck for 61s! [dmsetup:19758]
Modules linked in: ext4 jbd2 crc16 gfs2 dlm configfs xt_physdev
iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_]
CPU 1:
Modules linked in: ext4 jbd2 crc16 gfs2 dlm configfs xt_physdev
iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_]
Pid: 19758, comm: dmsetup Tainted: G D
2.6.32.27-0.xen.pvops.choon.centos5 #1 PowerEdge 860
RIP: e030:[<ffffffff813199d3>] [<ffffffff813199d3>] _spin_lock+0x19/0x20
RSP: e02b:ffff88003a615a68 EFLAGS: 00000297
RAX: 0000000000000025 RBX: 000000003d2fd000 RCX: 0000000000000004
RDX: 0000000000000024 RSI: 0000000000000004 RDI: ffff880027a2c600
RBP: ffff88003a615a68 R08: 0000000000000000 R09: ffffffff816dd100
R10: 3030303030303030 R11: 0000000000000120 R12: ffff880027a2c580
R13: 0000000000000004 R14: ffff880027a2c5e0 R15: ffffffff816dd100
FS: 00007f720a0f36e0(0000) GS:ffff88002806c000(0000) knlGS:0000000000000000
CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f7209c9c898 CR3: 0000000001001000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff811660dd>] ? free_cpumask_var+0x9/0xb
[<ffffffff8100dde1>] xen_exit_mmap+0x199/0x1d7
[<ffffffff810a8137>] exit_mmap+0x5f/0x14b
[<ffffffff81048648>] mmput+0x46/0xb2
[<ffffffff8104c552>] exit_mm+0xfd/0x108
[<ffffffff8100f799>] ? xen_irq_enable_direct_end+0x0/0x7
[<ffffffff8104d7ee>] do_exit+0x1f3/0x67b
[<ffffffff8131a908>] oops_end+0xba/0xc2
[<ffffffff810163a1>] die+0x55/0x5e
[<ffffffff8131a192>] do_trap+0x110/0x11f
[<ffffffff810142c8>] do_invalid_op+0x97/0xa0
[<ffffffff8100cb5b>] ? pin_pagetable_pfn+0x53/0x59
[<ffffffff810138bb>] invalid_op+0x1b/0x20
[<ffffffff8100cb5b>] ? pin_pagetable_pfn+0x53/0x59
[<ffffffff8100cb57>] ? pin_pagetable_pfn+0x4f/0x59
[<ffffffff8100e07c>] xen_alloc_ptpage+0x64/0x69
[<ffffffff8100e0af>] xen_alloc_pte+0xe/0x10
[<ffffffff810a402f>] __pte_alloc+0x70/0xce
[<ffffffff810a41cd>] handle_mm_fault+0x140/0x8b9
[<ffffffff8131be4d>] do_page_fault+0x252/0x2e2
[<ffffffff81319dd5>] page_fault+0x25/0x30
Kernel panic - not syncing: softlockup: hung tasks
Pid: 19758, comm: dmsetup Tainted: G D
2.6.32.27-0.xen.pvops.choon.centos5 #1
Call Trace:
<IRQ> [<ffffffff8104aa97>] panic+0xa0/0x15f
[<ffffffff81319dd5>] ? page_fault+0x25/0x30
[<ffffffff8101640f>] ? show_trace_log_lvl+0x4c/0x58
[<ffffffff8101642b>] ? show_trace+0x10/0x12
[<ffffffff81011755>] ? show_regs+0x44/0x48
[<ffffffff8107f202>] softlockup_tick+0x173/0x182
[<ffffffff810539bf>] run_local_timers+0x18/0x1a
[<ffffffff81053bde>] update_process_times+0x30/0x54
[<ffffffff81068821>] tick_sched_timer+0x70/0x99
[<ffffffff8105f52e>] __run_hrtimer+0x53/0xb3
[<ffffffff8105f772>] hrtimer_interrupt+0xae/0x192
[<ffffffff8100f3a3>] xen_timer_interrupt+0x37/0x181
[<ffffffff81082898>] ? check_for_new_grace_period+0x97/0xa5
[<ffffffff811c870f>] ? unmask_evtchn+0x34/0xd6
[<ffffffff8108318c>] ? __rcu_process_callbacks+0xf2/0x2ae
[<ffffffff8107f708>] handle_IRQ_event+0x2d/0xb7
[<ffffffff81081079>] handle_percpu_irq+0x3c/0x69
[<ffffffff811c8640>] __xen_evtchn_do_upcall+0xe1/0x168
[<ffffffff811c92d1>] xen_evtchn_do_upcall+0x2e/0x41
[<ffffffff81013c7e>] xen_do_hypervisor_callback+0x1e/0x30
<EOI> [<ffffffff813199d3>] ? _spin_lock+0x19/0x20
[<ffffffff811660dd>] ? free_cpumask_var+0x9/0xb
[<ffffffff8100dde1>] ? xen_exit_mmap+0x199/0x1d7
[<ffffffff810a8137>] ? exit_mmap+0x5f/0x14b
[<ffffffff81048648>] ? mmput+0x46/0xb2
[<ffffffff8104c552>] ? exit_mm+0xfd/0x108
[<ffffffff8100f799>] ? xen_irq_enable_direct_end+0x0/0x7
[<ffffffff8104d7ee>] ? do_exit+0x1f3/0x67b
[<ffffffff8131a908>] ? oops_end+0xba/0xc2
[<ffffffff810163a1>] ? die+0x55/0x5e
[<ffffffff8131a192>] ? do_trap+0x110/0x11f
[<ffffffff810142c8>] ? do_invalid_op+0x97/0xa0
[<ffffffff8100cb5b>] ? pin_pagetable_pfn+0x53/0x59
[<ffffffff810138bb>] ? invalid_op+0x1b/0x20
[<ffffffff8100cb5b>] ? pin_pagetable_pfn+0x53/0x59
[<ffffffff8100cb57>] ? pin_pagetable_pfn+0x4f/0x59
[<ffffffff8100e07c>] ? xen_alloc_ptpage+0x64/0x69
[<ffffffff8100e0af>] ? xen_alloc_pte+0xe/0x10
[<ffffffff810a402f>] ? __pte_alloc+0x70/0xce
[<ffffffff810a41cd>] ? handle_mm_fault+0x140/0x8b9
[<ffffffff8131be4d>] ? do_page_fault+0x252/0x2e2
[<ffffffff81319dd5>] ? page_fault+0x25/0x30
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 13622 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2010-12-26 8:16 kernel BUG at arch/x86/xen/mmu.c:1860! Teck Choon Giam
2010-12-27 15:53 ` Konrad Rzeszutek Wilk
2011-01-04 13:48 ` Ian Campbell
@ 2011-01-13 14:28 ` tjaouen
2011-01-14 14:47 ` Konrad Rzeszutek Wilk
2 siblings, 1 reply; 85+ messages in thread
From: tjaouen @ 2011-01-13 14:28 UTC (permalink / raw)
To: xen-devel
Same issue:
Backup DomU with lvsnapshot ...
Linux xencave 2.6.32-5-xen-amd64 #1 SMP Fri Dec 10 17:41:50 UTC 2010 x86_64
GNU/Linux
Jan 13 04:01:51 xencave kernel: [223872.758977] ------------[ cut here
]------------
Jan 13 04:01:51 xencave kernel: [223872.759006] kernel BUG at
/build/buildd-linux-2.6_2.6.32-29-amd64-xcs37n/linux-2.6-2.6.32/debian/build/source_amd64_xen/arch/x86/xen/mmu.c:1649!
Jan 13 04:01:51 xencave kernel: [223872.759047] invalid opcode: 0000 [#2]
SMP
Jan 13 04:01:51 xencave kernel: [223872.759076] last sysfs file:
/sys/devices/virtual/block/dm-30/dm/suspended
Jan 13 04:01:51 xencave kernel: [223872.759099] CPU 1
Jan 13 04:01:51 xencave kernel: [223872.759121] Modules linked in:
dm_snapshot nfs lockd fscache nfs_acl auth_rpcgss sunrpc nf_conntrack_ipv4
nf_defrag_ipv4 xt_state nf_conntrack xt_physdev iptable_filter ip_tables
x_tables xen_evtchn xenfs bridge stp coretemp it87 hwmon_vid fuse loop i915
drm_kms_helper processor acpi_processor drm i2c_algo_bit button video
snd_pcsp rng_core evdev i2c_i801 serio_raw snd_pcm snd_timer output i2c_core
snd soundcore snd_page_alloc usbhid hid ext3 jbd mbcache dm_mod raid1 md_mod
sg sr_mod cdrom sd_mod crc_t10dif ata_generic ata_piix libata scsi_mod
ehci_hcd uhci_hcd r8169 mii usbcore nls_base thermal fan thermal_sys [last
unloaded: scsi_wait_scan]
Jan 13 04:01:51 xencave kernel: [223872.759599] Pid: 9244, comm: dmsetup_env
Tainted: G D 2.6.32-5-xen-amd64 #1 945GM-S2
Jan 13 04:01:51 xencave kernel: [223872.759636] RIP:
e030:[<ffffffff8100c694>] [<ffffffff8100c694>] pin_pagetable_pfn+0x2d/0x36
Jan 13 04:01:51 xencave kernel: [223872.759683] RSP: e02b:ffff88001d695e08
EFLAGS: 00010282
Jan 13 04:01:51 xencave kernel: [223872.759707] RAX: 00000000ffffffea RBX:
000000000001e9df RCX: 0000000000000001
Jan 13 04:01:51 xencave kernel: [223872.759742] RDX: 0000000000000000 RSI:
0000000000000001 RDI: ffff88001d695e08
Jan 13 04:01:51 xencave kernel: [223872.759777] RBP: ffff88001f2b4700 R08:
0000000000000ef8 R09: ffffea00006b28c8
Jan 13 04:01:51 xencave kernel: [223872.759812] R10: 0000000000007ff0 R11:
ffffea00005858c8 R12: ffff88001a3ee010
Jan 13 04:01:51 xencave kernel: [223872.759846] R13: ffff8800020a8000 R14:
ffff880002089530 R15: ffff88001a3ee010
Jan 13 04:01:51 xencave kernel: [223872.759887] FS: 00007f81d5709700(0000)
GS:ffff88000308c000(0000) knlGS:0000000000000000
Jan 13 04:01:51 xencave kernel: [223872.759925] CS: e033 DS: 0000 ES: 0000
CR0: 000000008005003b
Jan 13 04:01:51 xencave kernel: [223872.759948] CR2: 000000000043c56a CR3:
000000001a38c000 CR4: 0000000000002660
Jan 13 04:01:51 xencave kernel: [223872.759985] DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
Jan 13 04:01:51 xencave kernel: [223872.760022] DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
Jan 13 04:01:51 xencave kernel: [223872.760057] Process dmsetup_env (pid:
9244, threadinfo ffff88001d694000, task ffff880002089530)
Jan 13 04:01:51 xencave kernel: [223872.760094] Stack:
Jan 13 04:01:51 xencave kernel: [223872.760112] 0000000000000000
000000000009e90d ffffea00006b28c8 000000000001e9df
Jan 13 04:01:51 xencave kernel: [223872.760149] <0> ffff88001f2b4700
ffffffff810cd4a1 ffff88001a3dc000 000000000043c56a
Jan 13 04:01:51 xencave kernel: [223872.760203] <0> 000000001a3dc000
ffffffff810cb34c ffff8800020a8000 000000000043c56a
Jan 13 04:01:51 xencave kernel: [223872.760270] Call Trace:
Jan 13 04:01:51 xencave kernel: [223872.760295] [<ffffffff810cd4a1>] ?
__pte_alloc+0x6b/0xc6
Jan 13 04:01:51 xencave kernel: [223872.760320] [<ffffffff810cb34c>] ?
pmd_alloc+0x28/0x5b
Jan 13 04:01:51 xencave kernel: [223872.760344] [<ffffffff810cd5ca>] ?
handle_mm_fault+0xce/0x80f
Jan 13 04:01:51 xencave kernel: [223872.760372] [<ffffffff8130cc35>] ?
page_fault+0x25/0x30
Jan 13 04:01:51 xencave kernel: [223872.760397] [<ffffffff8130ed96>] ?
do_page_fault+0x2e0/0x2fc
Jan 13 04:01:51 xencave kernel: [223872.760422] [<ffffffff8130cc35>] ?
page_fault+0x25/0x30
Jan 13 04:01:51 xencave kernel: [223872.760443] Code: ec 28 89 3c 24 48 89
f7 e8 a2 fd ff ff 48 89 e7 48 89 44 24 08 be 01 00 00 00 31 d2 41 ba f0 7f
00 00 e8 b0 cc ff ff 85 c0 74 04 <0f> 0b eb fe 48 83 c4 28 c3 55 49 89 ca 48
89 d5 40 88 f1 48 89
Jan 13 04:01:51 xencave kernel: [223872.760761] RIP [<ffffffff8100c694>]
pin_pagetable_pfn+0x2d/0x36
Jan 13 04:01:51 xencave kernel: [223872.760790] RSP <ffff88001d695e08>
Jan 13 04:01:51 xencave kernel: [223872.761008] ---[ end trace
b143b5bdb343412e ]---
help!
--
View this message in context: http://xen.1045712.n5.nabble.com/kernel-BUG-at-arch-x86-xen-mmu-c-1860-tp3318567p3339844.html
Sent from the Xen - Dev mailing list archive at Nabble.com.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-13 14:28 ` tjaouen
@ 2011-01-14 14:47 ` Konrad Rzeszutek Wilk
0 siblings, 0 replies; 85+ messages in thread
From: Konrad Rzeszutek Wilk @ 2011-01-14 14:47 UTC (permalink / raw)
To: tjaouen; +Cc: xen-devel
On Thu, Jan 13, 2011 at 06:28:22AM -0800, tjaouen wrote:
>
> Same issue:
>
> Backup DomU with lvsnapshot ...
Ok. I just got some free time so let me take a look at this.
>
> Linux xencave 2.6.32-5-xen-amd64 #1 SMP Fri Dec 10 17:41:50 UTC 2010 x86_64
> GNU/Linux
>
> Jan 13 04:01:51 xencave kernel: [223872.758977] ------------[ cut here
> ]------------
> Jan 13 04:01:51 xencave kernel: [223872.759006] kernel BUG at
> /build/buildd-linux-2.6_2.6.32-29-amd64-xcs37n/linux-2.6-2.6.32/debian/build/source_amd64_xen/arch/x86/xen/mmu.c:1649!
> Jan 13 04:01:51 xencave kernel: [223872.759047] invalid opcode: 0000 [#2]
> SMP
> Jan 13 04:01:51 xencave kernel: [223872.759076] last sysfs file:
> /sys/devices/virtual/block/dm-30/dm/suspended
> Jan 13 04:01:51 xencave kernel: [223872.759099] CPU 1
> Jan 13 04:01:51 xencave kernel: [223872.759121] Modules linked in:
> dm_snapshot nfs lockd fscache nfs_acl auth_rpcgss sunrpc nf_conntrack_ipv4
> nf_defrag_ipv4 xt_state nf_conntrack xt_physdev iptable_filter ip_tables
> x_tables xen_evtchn xenfs bridge stp coretemp it87 hwmon_vid fuse loop i915
> drm_kms_helper processor acpi_processor drm i2c_algo_bit button video
> snd_pcsp rng_core evdev i2c_i801 serio_raw snd_pcm snd_timer output i2c_core
> snd soundcore snd_page_alloc usbhid hid ext3 jbd mbcache dm_mod raid1 md_mod
> sg sr_mod cdrom sd_mod crc_t10dif ata_generic ata_piix libata scsi_mod
> ehci_hcd uhci_hcd r8169 mii usbcore nls_base thermal fan thermal_sys [last
> unloaded: scsi_wait_scan]
> Jan 13 04:01:51 xencave kernel: [223872.759599] Pid: 9244, comm: dmsetup_env
> Tainted: G D 2.6.32-5-xen-amd64 #1 945GM-S2
> Jan 13 04:01:51 xencave kernel: [223872.759636] RIP:
> e030:[<ffffffff8100c694>] [<ffffffff8100c694>] pin_pagetable_pfn+0x2d/0x36
> Jan 13 04:01:51 xencave kernel: [223872.759683] RSP: e02b:ffff88001d695e08
> EFLAGS: 00010282
> Jan 13 04:01:51 xencave kernel: [223872.759707] RAX: 00000000ffffffea RBX:
> 000000000001e9df RCX: 0000000000000001
> Jan 13 04:01:51 xencave kernel: [223872.759742] RDX: 0000000000000000 RSI:
> 0000000000000001 RDI: ffff88001d695e08
> Jan 13 04:01:51 xencave kernel: [223872.759777] RBP: ffff88001f2b4700 R08:
> 0000000000000ef8 R09: ffffea00006b28c8
> Jan 13 04:01:51 xencave kernel: [223872.759812] R10: 0000000000007ff0 R11:
> ffffea00005858c8 R12: ffff88001a3ee010
> Jan 13 04:01:51 xencave kernel: [223872.759846] R13: ffff8800020a8000 R14:
> ffff880002089530 R15: ffff88001a3ee010
> Jan 13 04:01:51 xencave kernel: [223872.759887] FS: 00007f81d5709700(0000)
> GS:ffff88000308c000(0000) knlGS:0000000000000000
> Jan 13 04:01:51 xencave kernel: [223872.759925] CS: e033 DS: 0000 ES: 0000
> CR0: 000000008005003b
> Jan 13 04:01:51 xencave kernel: [223872.759948] CR2: 000000000043c56a CR3:
> 000000001a38c000 CR4: 0000000000002660
> Jan 13 04:01:51 xencave kernel: [223872.759985] DR0: 0000000000000000 DR1:
> 0000000000000000 DR2: 0000000000000000
> Jan 13 04:01:51 xencave kernel: [223872.760022] DR3: 0000000000000000 DR6:
> 00000000ffff0ff0 DR7: 0000000000000400
> Jan 13 04:01:51 xencave kernel: [223872.760057] Process dmsetup_env (pid:
> 9244, threadinfo ffff88001d694000, task ffff880002089530)
> Jan 13 04:01:51 xencave kernel: [223872.760094] Stack:
> Jan 13 04:01:51 xencave kernel: [223872.760112] 0000000000000000
> 000000000009e90d ffffea00006b28c8 000000000001e9df
> Jan 13 04:01:51 xencave kernel: [223872.760149] <0> ffff88001f2b4700
> ffffffff810cd4a1 ffff88001a3dc000 000000000043c56a
> Jan 13 04:01:51 xencave kernel: [223872.760203] <0> 000000001a3dc000
> ffffffff810cb34c ffff8800020a8000 000000000043c56a
> Jan 13 04:01:51 xencave kernel: [223872.760270] Call Trace:
> Jan 13 04:01:51 xencave kernel: [223872.760295] [<ffffffff810cd4a1>] ?
> __pte_alloc+0x6b/0xc6
> Jan 13 04:01:51 xencave kernel: [223872.760320] [<ffffffff810cb34c>] ?
> pmd_alloc+0x28/0x5b
> Jan 13 04:01:51 xencave kernel: [223872.760344] [<ffffffff810cd5ca>] ?
> handle_mm_fault+0xce/0x80f
> Jan 13 04:01:51 xencave kernel: [223872.760372] [<ffffffff8130cc35>] ?
> page_fault+0x25/0x30
> Jan 13 04:01:51 xencave kernel: [223872.760397] [<ffffffff8130ed96>] ?
> do_page_fault+0x2e0/0x2fc
> Jan 13 04:01:51 xencave kernel: [223872.760422] [<ffffffff8130cc35>] ?
> page_fault+0x25/0x30
> Jan 13 04:01:51 xencave kernel: [223872.760443] Code: ec 28 89 3c 24 48 89
> f7 e8 a2 fd ff ff 48 89 e7 48 89 44 24 08 be 01 00 00 00 31 d2 41 ba f0 7f
> 00 00 e8 b0 cc ff ff 85 c0 74 04 <0f> 0b eb fe 48 83 c4 28 c3 55 49 89 ca 48
> 89 d5 40 88 f1 48 89
> Jan 13 04:01:51 xencave kernel: [223872.760761] RIP [<ffffffff8100c694>]
> pin_pagetable_pfn+0x2d/0x36
> Jan 13 04:01:51 xencave kernel: [223872.760790] RSP <ffff88001d695e08>
> Jan 13 04:01:51 xencave kernel: [223872.761008] ---[ end trace
> b143b5bdb343412e ]---
>
>
> help!
> --
> View this message in context: http://xen.1045712.n5.nabble.com/kernel-BUG-at-arch-x86-xen-mmu-c-1860-tp3318567p3339844.html
> Sent from the Xen - Dev mailing list archive at Nabble.com.
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2010-12-29 4:58 ` Teck Choon Giam
@ 2011-01-14 15:20 ` Konrad Rzeszutek Wilk
2011-01-14 19:25 ` Teck Choon Giam
0 siblings, 1 reply; 85+ messages in thread
From: Konrad Rzeszutek Wilk @ 2011-01-14 15:20 UTC (permalink / raw)
To: Teck Choon Giam; +Cc: xen-devel
On Wed, Dec 29, 2010 at 12:58:15PM +0800, Teck Choon Giam wrote:
> Below is my latest test crash script:
This test script is still valid? Or do you have a more updated one?
>
> ----------8<----------8<----------8<----------8<----------8<----------8<----------8<----------8<----------
> #!/bin/sh
> #
> # This script is to create lvm snapshot, mount it, umount it and remove in a
> # specified number of loops to test whether it will crash the host server.
> # All LVM snapshots assumed can be mounted like if you are running a PV
> domU.
> #
> # Created by Giam Teck Choon
> #
>
> # The LV name and for this case we are using XenGroup
> LVGroupName=XenGroup
>
> # return 1 if is mounted otherwise return 0
> check_mount() {
> local checkdir=${1}
> if [ -n "$checkdir" ] ; then
> local check=`grep "$checkdir" /proc/mounts`
> if [ -n "$check" ] ; then
> return 1
> fi
> fi
> return 0
> }
>
> do_lvm_create_remove() {
> # number of loops default is 1
> local loopcountlimit=${1:-1}
> # snapshot size default is 1G
> local snapshotsize=${2:-1G}
> # implement a sleep between create, mount, umount and remove (default is
> 0 which is no pause)
> local pauseinterval=${3:-0}
> # execute commands after each pause/sleep such as sync or anything that
> you want to test
> local commands=${4}
> # We filter out snapshot and swap
> local count=0
> if [ -d "/dev/${LVGroupName}" ] ; then
> while [ "$count" -lt "$loopcountlimit" ]
> do
> count=`expr $count + 1`
> echo "${count} ... ... "
> for i in `ls /dev/${LVGroupName} | grep -Ev 'snapshot$' | grep
> -Ev 'swap$'`; do
> if [ -h "/dev/${LVGroupName}/${i}" ] ; then
> echo -n "lvcreate -s -v -n ${i}-snapshot -L
> ${snapshotsize} /dev/${LVGroupName}/${i} ... ... "
> lvcreate -s -v -n ${i}-snapshot -L ${snapshotsize}
> /dev/${LVGroupName}/${i}
> echo "done."
> sleep ${pauseinterval}
> if [ -n "$commands" ] ; then
> echo -n "${commands} ... ... "
> $commands
> echo "done."
> fi
> mkdir -p /mnt/testlvm/${i}
> if [ -h "/dev/${LVGroupName}/${i}-snapshot" ] ; then
> check_mount /mnt/testlvm/${i}
> local ismount=$?
> if [ "$ismount" -eq 0 ] ; then
> echo -n "mount /dev/${LVGroupName}/${i}-snapshot
> /mnt/testlvm/${i} ... ... "
> mount /dev/${LVGroupName}/${i}-snapshot
> /mnt/testlvm/${i}
> echo "done."
> sleep ${pauseinterval}
> if [ -n "$commands" ] ; then
> echo -n "${commands} ... ... "
> $commands
> echo "done."
> fi
> fi
> check_mount /mnt/testlvm/${i}
> local ismount2=$?
> if [ "$ismount2" -eq 1 ] ; then
> echo -n "umount /mnt/testlvm/${i} ... ... "
> umount /mnt/testlvm/${i}
> echo "done."
> sleep ${pauseinterval}
> if [ -n "$commands" ] ; then
> echo -n "${commands} ... ... "
> $commands
> echo "done."
> fi
> fi
> fi
> rm -rf /mnt/testlvm/${i}
> echo -n "lvremove -f /dev/${LVGroupName}/${i}-snapshot
> ... ... "
> lvremove -f /dev/${LVGroupName}/${i}-snapshot
> echo "done."
> sleep ${pauseinterval}
> if [ -n "$commands" ] ; then
> echo -n "${commands} ... ... "
> $commands
> echo "done."
> fi
> fi
> done
> rm -fr /mnt/testlvm
> done
> else
> echo "/dev/${LVGroupName} directory not found!"
> exit 1
> fi
> }
>
> case $1 in
> loop) shift
> do_lvm_create_remove "$@"
> ;;
> *) cat <<HELP
> Usage: $0 loop loopcountlimit snapshotsize pauseinterval commands
> Where:
> loopcountlimit is default to 1
> snapshotsize is default to 1G
> pauseinterval is default to 0
> commands is default to none
>
> Example to run with 100 loops without pause/sleep:
> $0 loop 100
>
> Example to run with 100 loops with pause/sleep of 5 seconds:
> $0 loop 100 1G 5
>
> Example to run with 100 loops with snapshot size of 2G instead of 1G:
> $0 loop 100 2G
>
> Example to run with 50 loops, 1G snapshot size, 5 seconds pause and with
> sync:
> command with each pause/sleep
> $0 loop 50 1G 5 sync
>
> Example to run with 50 loops, 1G snapshot size, no pause and with sync:
> command with each pause/sleep
> $0 loop 50 1G 0 sync
>
> Example to run your own commands:
> $0 loop 100 1G 5 "echo hi && sync"
>
> HELP
> ;;
> esac
> ----------8<----------8<----------8<----------8<----------8<----------8<----------8<----------8<----------
>
> Thanks.
>
> Kindest regards,
> Giam Teck Choon
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-04 18:40 ` Christophe Saout
2011-01-04 19:32 ` Teck Choon Giam
@ 2011-01-14 15:22 ` Konrad Rzeszutek Wilk
2011-01-14 15:33 ` Christophe Saout
1 sibling, 1 reply; 85+ messages in thread
From: Konrad Rzeszutek Wilk @ 2011-01-14 15:22 UTC (permalink / raw)
To: Christophe Saout; +Cc: xen-devel, Teck Choon Giam
> (without strace, the bug is very common, about every third "lvcreate"
> command. Every lvcreate command triggers about 20 multipath
Something must be busted with your udev rules for this to go in effect.
You should have multipathd process those and not run multipath for
every device mapper call.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-05 14:56 ` Teck Choon Giam
@ 2011-01-14 15:24 ` Konrad Rzeszutek Wilk
2011-01-14 19:31 ` Teck Choon Giam
0 siblings, 1 reply; 85+ messages in thread
From: Konrad Rzeszutek Wilk @ 2011-01-14 15:24 UTC (permalink / raw)
To: Teck Choon Giam; +Cc: xen-devel, Christophe Saout
> The idea is to create snapshot, mount it, umount it, remove snapshot and
> repeat this cycle in loop will catch this BUG!!!
What is the hypervisor version? Is it xen-unstable.hg?
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-14 15:22 ` Konrad Rzeszutek Wilk
@ 2011-01-14 15:33 ` Christophe Saout
0 siblings, 0 replies; 85+ messages in thread
From: Christophe Saout @ 2011-01-14 15:33 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel, Teck Choon Giam
Hi Konrad,
> > (without strace, the bug is very common, about every third "lvcreate"
> > command. Every lvcreate command triggers about 20 multipath
>
> Something must be busted with your udev rules for this to go in effect.
> You should have multipathd process those and not run multipath for
> every device mapper call.
This is a standard Debian squeeze. I think udev is just calling those
to figure out if a device should be handled by multipath or not (part of
the device identification). I'd rather not fiddle with it, as it "works"
even if Debian is shipping suboptimal rules. Anyhow, changing this would
simply reduce the risk of running into the kernel issue, not remove it
entirely, so I prefer the actual fix. :)
Christophe
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-14 15:20 ` Konrad Rzeszutek Wilk
@ 2011-01-14 19:25 ` Teck Choon Giam
2011-01-14 19:44 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2011-01-14 19:25 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 7158 bytes --]
On Fri, Jan 14, 2011 at 11:20 PM, Konrad Rzeszutek Wilk <
konrad.wilk@oracle.com> wrote:
> On Wed, Dec 29, 2010 at 12:58:15PM +0800, Teck Choon Giam wrote:
> > Below is my latest test crash script:
>
> This test script is still valid? Or do you have a more updated one?
>
Below is the more updated one. Please note that this will mostly not
support VG in CLVM since clustered LVM doesn't support snapshot in my CentOS
5 testing... ...
----------8<----------8<----------8<----------8<----------8<----------8<----------8<----------8<----------8<----------
#!/bin/sh
#
# This script is to create lvm snapshot, mount it, umount it and remove in a
# specified number of loops to test whether it will crash the host server.
# All LVM snapshots assumed can be mounted like if you are running a PV
domU.
#
# Created by Giam Teck Choon
#
# The LV name and for this case we are using the first in vgdisplay output.
# Change the variable if you want other VG Name or change head -n 1 to
# tail -n 1 if you prefer to use last VG instead of first if you happen to
have
# more than one VG
LVGroupName=`vgdisplay | grep 'VG Name' | awk '{print $3}' | head -n 1`
if [ ! -n "$LVGroupName" ] && [ ! -d "/dev/${LVGroupName}" ] ; then
echo "Unable to detect VG Name!"
exit 1
fi
# return 1 if is mounted otherwise return 0
check_mount() {
local checkdir=${1}
if [ -n "$checkdir" ] ; then
local check=`grep "$checkdir" /proc/mounts`
if [ -n "$check" ] ; then
return 1
fi
fi
return 0
}
# We will create 5 testcrash LV in $LVGroupName each with 5GB size
# and format it as ext3
do_lvm_create_testcrash() {
local lvname=${1:-testcrash}
local lvsize=${2:-5G}
local count=1
local limit=5
while [ "$count" -le "$limit" ]
do
if [ ! -h "/dev/${LVGroupName}/${lvname}${count}" ] ; then
echo "lvcreate -v -n ${lvname}${count} -L ${lvsize}
${LVGroupName} ... ... "
lvcreate -v -n ${lvname}${count} -L ${lvsize} ${LVGroupName}
echo "lvcreate -v -n ${lvname}${count} -L ${lvsize}
${LVGroupName} completed!"
if [ -h "/dev/${LVGroupName}/${lvname}${count}" ] ; then
echo "mke2fs -F -j /dev/${LVGroupName}/${lvname}${count} ...
... "
mke2fs -F -j /dev/${LVGroupName}/${lvname}${count}
echo "mke2fs -F -j /dev/${LVGroupName}/${lvname}${count}
completed!"
else
echo "/dev/${LVGroupName}/${lvname}${count} not found!"
fi
fi
count=`expr $count + 1`
done
}
do_lvm_create_remove() {
# number of loops default is 1
local loopcountlimit=${1:-1}
# snapshot size default is 1G
local snapshotsize=${2:-1G}
# implement a sleep between create, mount, umount and remove (default is
0 which is no pause)
local pauseinterval=${3:-0}
# execute commands after each pause/sleep such as sync or anything that
you want to test
local commands=${4}
# We filter out snapshot and swap
local count=0
if [ -d "/dev/${LVGroupName}" ] ; then
while [ "$count" -lt "$loopcountlimit" ]
do
count=`expr $count + 1`
echo "${count} ... ... "
for i in `ls /dev/${LVGroupName} | grep -Ev 'snapshot$' | grep
-Ev 'swap$'`; do
if [ -h "/dev/${LVGroupName}/${i}" ] ; then
echo -n "lvcreate -s -v -n ${i}-snapshot -L
${snapshotsize} /dev/${LVGroupName}/${i} ... ... "
lvcreate -s -v -n ${i}-snapshot -L ${snapshotsize}
/dev/${LVGroupName}/${i}
echo "done."
sleep ${pauseinterval}
if [ -n "$commands" ] ; then
echo -n "${commands} ... ... "
$commands
echo "done."
fi
mkdir -p /mnt/testlvm/${i}
if [ -h "/dev/${LVGroupName}/${i}-snapshot" ] ; then
check_mount /mnt/testlvm/${i}
local ismount=$?
if [ "$ismount" -eq 0 ] ; then
echo -n "mount /dev/${LVGroupName}/${i}-snapshot
/mnt/testlvm/${i} ... ... "
mount /dev/${LVGroupName}/${i}-snapshot
/mnt/testlvm/${i}
echo "done."
sleep ${pauseinterval}
if [ -n "$commands" ] ; then
echo -n "${commands} ... ... "
$commands
echo "done."
fi
fi
check_mount /mnt/testlvm/${i}
local ismount2=$?
if [ "$ismount2" -eq 1 ] ; then
echo -n "umount /mnt/testlvm/${i} ... ... "
umount /mnt/testlvm/${i}
echo "done."
sleep ${pauseinterval}
if [ -n "$commands" ] ; then
echo -n "${commands} ... ... "
$commands
echo "done."
fi
fi
fi
rm -rf /mnt/testlvm/${i}
echo -n "lvremove -f /dev/${LVGroupName}/${i}-snapshot
... ... "
lvremove -f /dev/${LVGroupName}/${i}-snapshot
echo "done."
sleep ${pauseinterval}
if [ -n "$commands" ] ; then
echo -n "${commands} ... ... "
$commands
echo "done."
fi
fi
done
rm -fr /mnt/testlvm
done
else
echo "/dev/${LVGroupName} directory not found!"
exit 1
fi
}
case $1 in
setup) shift
do_lvm_create_testcrash "$@"
;;
loop) shift
do_lvm_create_remove "$@"
;;
*) cat <<HELP
Usage: $0 loop loopcountlimit snapshotsize pauseinterval commands
Where:
loopcountlimit is default to 1
snapshotsize is default to 1G
pauseinterval is default to 0
commands is default to none
Example to run with 100 loops without pause/sleep:
$0 loop 100
Example to run with 100 loops with pause/sleep of 5 seconds:
$0 loop 100 1G 5
Example to run with 100 loops with snapshot size of 2G instead of 1G:
$0 loop 100 2G
Example to run with 50 loops, 1G snapshot size, 5 seconds pause and with
sync:
command with each pause/sleep
$0 loop 50 1G 5 sync
Example to run your own commands:
$0 loop 100 1G 5 "echo hi && sync"
If this is the first time you are running and do not have any LV in your VG,
run:
$0 setup
This will create 5 testcrash LV in your VG with 5GB size each (default) and
format
to ext3.
HELP
;;
esac
----------8<----------8<----------8<----------8<----------8<----------8<----------8<----------8<----------8<----------
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 8667 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-14 15:24 ` Konrad Rzeszutek Wilk
@ 2011-01-14 19:31 ` Teck Choon Giam
0 siblings, 0 replies; 85+ messages in thread
From: Teck Choon Giam @ 2011-01-14 19:31 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel, Christophe Saout
[-- Attachment #1.1: Type: text/plain, Size: 403 bytes --]
On Fri, Jan 14, 2011 at 11:24 PM, Konrad Rzeszutek Wilk <
konrad.wilk@oracle.com> wrote:
> > The idea is to create snapshot, mount it, umount it, remove snapshot and
> > repeat this cycle in loop will catch this BUG!!!
>
> What is the hypervisor version? Is it xen-unstable.hg?
>
xen-4.0-testing changeset 214xx even with 21436
PVOPS latest stable/2.6.32.x
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 744 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-14 19:25 ` Teck Choon Giam
@ 2011-01-14 19:44 ` Konrad Rzeszutek Wilk
2011-01-14 20:09 ` Teck Choon Giam
0 siblings, 1 reply; 85+ messages in thread
From: Konrad Rzeszutek Wilk @ 2011-01-14 19:44 UTC (permalink / raw)
To: Teck Choon Giam; +Cc: xen-devel
On Sat, Jan 15, 2011 at 03:25:55AM +0800, Teck Choon Giam wrote:
> On Fri, Jan 14, 2011 at 11:20 PM, Konrad Rzeszutek Wilk <
> konrad.wilk@oracle.com> wrote:
>
> > On Wed, Dec 29, 2010 at 12:58:15PM +0800, Teck Choon Giam wrote:
> > > Below is my latest test crash script:
> >
> > This test script is still valid? Or do you have a more updated one?
> >
>
> Below is the more updated one. Please note that this will mostly not
Can you send it as attachment? The copy-n-paste broke it.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-14 19:44 ` Konrad Rzeszutek Wilk
@ 2011-01-14 20:09 ` Teck Choon Giam
2011-01-14 20:32 ` Teck Choon Giam
0 siblings, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2011-01-14 20:09 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 654 bytes --]
On Sat, Jan 15, 2011 at 3:44 AM, Konrad Rzeszutek Wilk <
konrad.wilk@oracle.com> wrote:
> On Sat, Jan 15, 2011 at 03:25:55AM +0800, Teck Choon Giam wrote:
> > On Fri, Jan 14, 2011 at 11:20 PM, Konrad Rzeszutek Wilk <
> > konrad.wilk@oracle.com> wrote:
> >
> > > On Wed, Dec 29, 2010 at 12:58:15PM +0800, Teck Choon Giam wrote:
> > > > Below is my latest test crash script:
> > >
> > > This test script is still valid? Or do you have a more updated one?
> > >
> >
> > Below is the more updated one. Please note that this will mostly not
>
> Can you send it as attachment? The copy-n-paste broke it.
>
Sure ;)
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 1092 bytes --]
[-- Attachment #2: testcrash.sh --]
[-- Type: application/x-sh, Size: 5146 bytes --]
[-- Attachment #3: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-14 20:09 ` Teck Choon Giam
@ 2011-01-14 20:32 ` Teck Choon Giam
2011-01-24 1:42 ` Teck Choon Giam
0 siblings, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2011-01-14 20:32 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 4229 bytes --]
My latest test crash with a little different result:
Jan 15 04:27:16 xen06 kernel: last sysfs file:
/sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_map
Normally is related to LVM/DM but now it is related to shared_cpu_map cache?
Jan 15 04:27:16 xen06 kernel: ------------[ cut here ]------------
Jan 15 04:27:16 xen06 kernel: kernel BUG at arch/x86/xen/mmu.c:1860!
Jan 15 04:27:16 xen06 kernel: invalid opcode: 0000 [#1] SMP
Jan 15 04:27:16 xen06 kernel: last sysfs file:
/sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_map << SEE THIS?
Jan 15 04:27:16 xen06 kernel: CPU 1
Jan 15 04:27:16 xen06 kernel: Modules linked in: dm_snapshot ipt_MASQUERADE
iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack
ipt_REJECT xt_tcpudp gfs2 dlm configfs xt_physdev iptable_filter ip_tables
x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic uio ipv6 cxgb3i cxgb3 mdio
libiscsi_tcp libiscsi scsi_transport_iscsi dm_mirror dm_multipath scsi_dh
video backlight output sbs sbshc power_meter hwmon battery acpi_memhotplug
xen_acpi_memhotplug ac parport_pc lp parport sg tg3 libphy ide_cd_mod cdrom
serio_raw button tpm_tis tpm tpm_bios i2c_i801 pcspkr i2c_core shpchp
iTCO_wdt dm_region_hash dm_log dm_mod ata_piix libata sd_mod scsi_mod raid1
ext3 jbd uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
Jan 15 04:27:16 xen06 kernel: Pid: 26392, comm: mpath_wait Not tainted
2.6.32.27-0.xen.pvops.choon.centos5 #1 PowerEdge 860
Jan 15 04:27:16 xen06 kernel: RIP: e030:[<ffffffff8100cb5b>]
[<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
Jan 15 04:27:16 xen06 kernel: RSP: e02b:ffff88001aae5dc8 EFLAGS: 00010282
Jan 15 04:27:16 xen06 kernel: RAX: 00000000ffffffea RBX: 000000000003cf04
RCX: 00000000000001e7
Jan 15 04:27:16 xen06 kernel: RDX: 00000000deadbeef RSI: 00000000deadbeef
RDI: 00000000deadbeef
Jan 15 04:27:16 xen06 kernel: RBP: ffff88001aae5de8 R08: 0000000000000820
R09: ffff880000000000
Jan 15 04:27:16 xen06 kernel: R10: 00000000deadbeef R11: 0000000000000246
R12: 0000000000000003
Jan 15 04:27:16 xen06 kernel: R13: 000000000003cf04 R14: ffff88001aa6a7c0
R15: 00000033e1a9a4d5
Jan 15 04:27:16 xen06 kernel: FS: 00007f9b8dbdf6e0(0000)
GS:ffff88002806c000(0000) knlGS:0000000000000000
Jan 15 04:27:16 xen06 kernel: CS: e033 DS: 0000 ES: 0000 CR0:
000000008005003b
Jan 15 04:27:16 xen06 kernel: CR2: 00000033e1a9a4d5 CR3: 000000001ab61000
CR4: 0000000000002660
Jan 15 04:27:16 xen06 kernel: DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000
Jan 15 04:27:16 xen06 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
DR7: 0000000000000400
Jan 15 04:27:16 xen06 kernel: Process mpath_wait (pid: 26392, threadinfo
ffff88001aae4000, task ffff88001aa6a7c0)
Jan 15 04:27:16 xen06 kernel: Stack:
Jan 15 04:27:16 xen06 kernel: 0000000000000000 000000000020f3d2
000000013e64d518 000000000003cf04
Jan 15 04:27:16 xen06 kernel: <0> ffff88001aae5e08 ffffffff8100e07c
ffff880025db0040 ffff880025fda868
Jan 15 04:27:16 xen06 kernel: <0> ffff88001aae5e18 ffffffff8100e0af
ffff88001aae5e58 ffffffff810a402f
Jan 15 04:27:16 xen06 kernel: Call Trace:
Jan 15 04:27:16 xen06 kernel: [<ffffffff8100e07c>]
xen_alloc_ptpage+0x64/0x69
Jan 15 04:27:16 xen06 kernel: [<ffffffff8100e0af>] xen_alloc_pte+0xe/0x10
Jan 15 04:27:16 xen06 kernel: [<ffffffff810a402f>] __pte_alloc+0x70/0xce
Jan 15 04:27:16 xen06 kernel: [<ffffffff810a41cd>]
handle_mm_fault+0x140/0x8b9
Jan 15 04:27:16 xen06 kernel: [<ffffffff81319dd5>] ? page_fault+0x25/0x30
Jan 15 04:27:16 xen06 kernel: [<ffffffff8131be4d>]
do_page_fault+0x252/0x2e2
Jan 15 04:27:16 xen06 kernel: [<ffffffff8116dd7d>] ? __put_user_4+0x1d/0x30
Jan 15 04:27:16 xen06 kernel: [<ffffffff81319dd5>] page_fault+0x25/0x30
Jan 15 04:27:16 xen06 kernel: Code: 48 b8 ff ff ff ff ff ff ff 7f 48 21 c2
48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 e9 c7 ff
ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74 5b
48
Jan 15 04:27:16 xen06 kernel: RIP [<ffffffff8100cb5b>]
pin_pagetable_pfn+0x53/0x59
Jan 15 04:27:16 xen06 kernel: RSP <ffff88001aae5dc8>
Jan 15 04:27:16 xen06 kernel: ---[ end trace 865f1d440d090f4f ]---
Sorry, this server doesn't have serial console though :(
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 4517 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-14 20:32 ` Teck Choon Giam
@ 2011-01-24 1:42 ` Teck Choon Giam
2011-01-24 14:36 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2011-01-24 1:42 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 205 bytes --]
Sorry, does anyone able to solve this bug? My prevention doing sleep 5 and
sync method only last me longest 20+- days of uptime then the same bug will
appear :(
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 225 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-24 1:42 ` Teck Choon Giam
@ 2011-01-24 14:36 ` Konrad Rzeszutek Wilk
2011-01-24 15:56 ` Teck Choon Giam
0 siblings, 1 reply; 85+ messages in thread
From: Konrad Rzeszutek Wilk @ 2011-01-24 14:36 UTC (permalink / raw)
To: Teck Choon Giam; +Cc: xen-devel
On Mon, Jan 24, 2011 at 09:42:02AM +0800, Teck Choon Giam wrote:
> Sorry, does anyone able to solve this bug? My prevention doing sleep 5 and
> sync method only last me longest 20+- days of uptime then the same bug will
> appear :(
You have to give more details. Which kernel and what does the back-trace look like?
>
> Thanks.
>
> Kindest regards,
> Giam Teck Choon
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-24 14:36 ` Konrad Rzeszutek Wilk
@ 2011-01-24 15:56 ` Teck Choon Giam
2011-01-25 14:48 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2011-01-24 15:56 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 1050 bytes --]
On Mon, Jan 24, 2011 at 10:36 PM, Konrad Rzeszutek Wilk <
konrad.wilk@oracle.com> wrote:
> On Mon, Jan 24, 2011 at 09:42:02AM +0800, Teck Choon Giam wrote:
> > Sorry, does anyone able to solve this bug? My prevention doing sleep 5
> and
> > sync method only last me longest 20+- days of uptime then the same bug
> will
> > appear :(
>
> You have to give more details. Which kernel and what does the back-trace
> look like?
>
Thanks for your prompt reply.
Kernel is from
http://git.kernel.org/?p=linux/kernel/git/jeremy/xen.git;a=summary latest
stable/2.6.32.x
Xen version is from
http://xenbits.xensource.com/staging/xen-4.0-testing.hglatest
changeset 21439 in fact any changeset 214xx I believe.
I have posted console output for one of my xen servers and maybe my serial
console configuration not right... ...
http://lists.xensource.com/archives/html/xen-devel/2011-01/msg00177.html
How do I back-trace? You mean I should execute my test crash script with
strace and post the output during crash?
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 1658 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-24 15:56 ` Teck Choon Giam
@ 2011-01-25 14:48 ` Konrad Rzeszutek Wilk
2011-01-26 14:31 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 85+ messages in thread
From: Konrad Rzeszutek Wilk @ 2011-01-25 14:48 UTC (permalink / raw)
To: Teck Choon Giam; +Cc: xen-devel
On Mon, Jan 24, 2011 at 11:56:21PM +0800, Teck Choon Giam wrote:
> On Mon, Jan 24, 2011 at 10:36 PM, Konrad Rzeszutek Wilk <
> konrad.wilk@oracle.com> wrote:
>
> > On Mon, Jan 24, 2011 at 09:42:02AM +0800, Teck Choon Giam wrote:
> > > Sorry, does anyone able to solve this bug? My prevention doing sleep 5
> > and
> > > sync method only last me longest 20+- days of uptime then the same bug
> > will
> > > appear :(
> >
> > You have to give more details. Which kernel and what does the back-trace
> > look like?
> >
>
> Thanks for your prompt reply.
>
> Kernel is from
> http://git.kernel.org/?p=linux/kernel/git/jeremy/xen.git;a=summary latest
> stable/2.6.32.x
>
> Xen version is from
> http://xenbits.xensource.com/staging/xen-4.0-testing.hglatest
> changeset 21439 in fact any changeset 214xx I believe.
>
> I have posted console output for one of my xen servers and maybe my serial
> console configuration not right... ...
>
> http://lists.xensource.com/archives/html/xen-devel/2011-01/msg00177.html
>
> How do I back-trace? You mean I should execute my test crash script with
> strace and post the output during crash?
No that is OK. I now remember this one - I am poking at the code to get an idea
of what might be happening. No data yet.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-25 14:48 ` Konrad Rzeszutek Wilk
@ 2011-01-26 14:31 ` Konrad Rzeszutek Wilk
2011-01-27 17:17 ` Teck Choon Giam
0 siblings, 1 reply; 85+ messages in thread
From: Konrad Rzeszutek Wilk @ 2011-01-26 14:31 UTC (permalink / raw)
To: Teck Choon Giam; +Cc: xen-devel
On Tue, Jan 25, 2011 at 09:48:09AM -0500, Konrad Rzeszutek Wilk wrote:
> On Mon, Jan 24, 2011 at 11:56:21PM +0800, Teck Choon Giam wrote:
> > On Mon, Jan 24, 2011 at 10:36 PM, Konrad Rzeszutek Wilk <
> > konrad.wilk@oracle.com> wrote:
> >
> > > On Mon, Jan 24, 2011 at 09:42:02AM +0800, Teck Choon Giam wrote:
> > > > Sorry, does anyone able to solve this bug? My prevention doing sleep 5
> > > and
> > > > sync method only last me longest 20+- days of uptime then the same bug
> > > will
> > > > appear :(
> > >
> > > You have to give more details. Which kernel and what does the back-trace
> > > look like?
> > >
> >
> > Thanks for your prompt reply.
> >
> > Kernel is from
> > http://git.kernel.org/?p=linux/kernel/git/jeremy/xen.git;a=summary latest
> > stable/2.6.32.x
What does git log show? What is the latest commit you have there?
> >
> > Xen version is from
> > http://xenbits.xensource.com/staging/xen-4.0-testing.hglatest
> > changeset 21439 in fact any changeset 214xx I believe.
> >
> > I have posted console output for one of my xen servers and maybe my serial
> > console configuration not right... ...
> >
> > http://lists.xensource.com/archives/html/xen-devel/2011-01/msg00177.html
> >
> > How do I back-trace? You mean I should execute my test crash script with
> > strace and post the output during crash?
>
>
> No that is OK. I now remember this one - I am poking at the code to get an idea
> of what might be happening. No data yet.
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-26 14:31 ` Konrad Rzeszutek Wilk
@ 2011-01-27 17:17 ` Teck Choon Giam
2011-01-27 20:32 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2011-01-27 17:17 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 2575 bytes --]
On Wed, Jan 26, 2011 at 10:31 PM, Konrad Rzeszutek Wilk <
konrad.wilk@oracle.com> wrote:
> On Tue, Jan 25, 2011 at 09:48:09AM -0500, Konrad Rzeszutek Wilk wrote:
> > On Mon, Jan 24, 2011 at 11:56:21PM +0800, Teck Choon Giam wrote:
> > > On Mon, Jan 24, 2011 at 10:36 PM, Konrad Rzeszutek Wilk <
> > > konrad.wilk@oracle.com> wrote:
> > >
> > > > On Mon, Jan 24, 2011 at 09:42:02AM +0800, Teck Choon Giam wrote:
> > > > > Sorry, does anyone able to solve this bug? My prevention doing
> sleep 5
> > > > and
> > > > > sync method only last me longest 20+- days of uptime then the same
> bug
> > > > will
> > > > > appear :(
> > > >
> > > > You have to give more details. Which kernel and what does the
> back-trace
> > > > look like?
> > > >
> > >
> > > Thanks for your prompt reply.
> > >
> > > Kernel is from
> > > http://git.kernel.org/?p=linux/kernel/git/jeremy/xen.git;a=summarylatest
> > > stable/2.6.32.x
>
> What does git log show? What is the latest commit you have there?
>
commit 75cc13f5aa29b4f3227d269ca165dfa8937c94fe
Merge: 2607c07 a386bf7
Author: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Date: Thu Dec 9 17:16:16 2010 -0800
Merge commit 'v2.6.32.27' into xen/next-2.6.32
* commit 'v2.6.32.27': (128 commits)
Linux 2.6.32.27
x86: uv: xpc NULL deref when mesq becomes empty
X86: uv: xpc_make_first_contact hang due to not accepting ACTIVE state
x86: uv: XPC receive message reuse triggers invalid BUG_ON()
UV - XPC: pass nasid instead of nid to gru_create_message_queue
net sched: fix some kernel memory leaks
act_nat: use stack variable
nmi: fix clock comparator revalidation
net: Limit socket I/O iovec total length to INT_MAX.
net: Truncate recvfrom and sendto length to INT_MAX.
rds: Integer overflow in RDS cmsg handling
econet: fix CVE-2010-3850
econet: disallow NULL remote addr for sendmsg(), fixes CVE-2010-3849
x86-32: Fix dummy trampoline-related inline stubs
x86, mm: Fix CONFIG_VMSPLIT_1G and 2G_OPT trampoline
x86-32: Separate 1:1 pagetables from swapper_pg_dir
crypto: padlock - Fix AES-CBC handling on odd-block-sized input
x25: Prevent crashing when parsing bad X.25 facilities
V4L/DVB: ivtvfb: prevent reading uninitialized stack memory
can-bcm: fix minor heap overflow
...
Conflicts:
drivers/xen/events.c
Anything that I can test or you need me to test... feel free to ask so that
I can speed up in tracking/hunting this bug down :)
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 3450 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-27 17:17 ` Teck Choon Giam
@ 2011-01-27 20:32 ` Konrad Rzeszutek Wilk
2011-01-27 22:20 ` Teck Choon Giam
0 siblings, 1 reply; 85+ messages in thread
From: Konrad Rzeszutek Wilk @ 2011-01-27 20:32 UTC (permalink / raw)
To: Teck Choon Giam; +Cc: xen-devel
> > What does git log show? What is the latest commit you have there?
> >
>
> commit 75cc13f5aa29b4f3227d269ca165dfa8937c94fe
> Merge: 2607c07 a386bf7
> Author: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
> Date: Thu Dec 9 17:16:16 2010 -0800
>
> Merge commit 'v2.6.32.27' into xen/next-2.6.32
>
> * commit 'v2.6.32.27': (128 commits)
> Linux 2.6.32.27
.. snip..
>
> Anything that I can test or you need me to test... feel free to ask so that
> I can speed up in tracking/hunting this bug down :)
You got the latest one so you should have the fixes. But something is not
working.. just to make sure I am not missing anything can you send me your .config
file?
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-27 20:32 ` Konrad Rzeszutek Wilk
@ 2011-01-27 22:20 ` Teck Choon Giam
2011-02-26 12:03 ` Teck Choon Giam
0 siblings, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2011-01-27 22:20 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 893 bytes --]
On Fri, Jan 28, 2011 at 4:32 AM, Konrad Rzeszutek Wilk <
konrad.wilk@oracle.com> wrote:
> > > What does git log show? What is the latest commit you have there?
> > >
> >
> > commit 75cc13f5aa29b4f3227d269ca165dfa8937c94fe
> > Merge: 2607c07 a386bf7
> > Author: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
> > Date: Thu Dec 9 17:16:16 2010 -0800
> >
> > Merge commit 'v2.6.32.27' into xen/next-2.6.32
> >
> > * commit 'v2.6.32.27': (128 commits)
> > Linux 2.6.32.27
>
> .. snip..
> >
> > Anything that I can test or you need me to test... feel free to ask so
> that
> > I can speed up in tracking/hunting this bug down :)
>
> You got the latest one so you should have the fixes. But something is not
> working.. just to make sure I am not missing anything can you send me your
> .config
> file?
>
Ok... please see attached.
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 1415 bytes --]
[-- Attachment #2: config-2.6.32.27-0.xen.pvops.choon.centos5 --]
[-- Type: application/octet-stream, Size: 73091 bytes --]
#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.32.27-0.xen.pvops.choon.centos5
# Sat Dec 25 09:16:08 2010
#
CONFIG_64BIT=y
# CONFIG_X86_32 is not set
CONFIG_X86_64=y
CONFIG_X86=y
CONFIG_OUTPUT_FORMAT="elf64-x86-64"
CONFIG_ARCH_DEFCONFIG="arch/x86/configs/x86_64_defconfig"
CONFIG_GENERIC_TIME=y
CONFIG_GENERIC_CMOS_UPDATE=y
CONFIG_CLOCKSOURCE_WATCHDOG=y
CONFIG_GENERIC_CLOCKEVENTS=y
CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_HAVE_LATENCYTOP_SUPPORT=y
CONFIG_MMU=y
CONFIG_ZONE_DMA=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
CONFIG_GENERIC_BUG=y
CONFIG_GENERIC_BUG_RELATIVE_POINTERS=y
CONFIG_GENERIC_HWEIGHT=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
# CONFIG_RWSEM_GENERIC_SPINLOCK is not set
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_ARCH_HAS_CPU_IDLE_WAIT=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_GENERIC_TIME_VSYSCALL=y
CONFIG_ARCH_HAS_CPU_RELAX=y
CONFIG_ARCH_HAS_DEFAULT_IDLE=y
CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y
CONFIG_HAVE_SETUP_PER_CPU_AREA=y
CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK=y
CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK=y
CONFIG_HAVE_CPUMASK_OF_CPU_MAP=y
CONFIG_ARCH_HIBERNATION_POSSIBLE=y
CONFIG_ARCH_SUSPEND_POSSIBLE=y
CONFIG_ZONE_DMA32=y
CONFIG_ARCH_POPULATES_NODE_MAP=y
CONFIG_AUDIT_ARCH=y
CONFIG_ARCH_SUPPORTS_OPTIMIZED_INLINING=y
CONFIG_ARCH_SUPPORTS_DEBUG_PAGEALLOC=y
CONFIG_GENERIC_HARDIRQS=y
CONFIG_GENERIC_HARDIRQS_NO__DO_IRQ=y
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_GENERIC_PENDING_IRQ=y
CONFIG_USE_GENERIC_SMP_HELPERS=y
CONFIG_X86_64_SMP=y
CONFIG_X86_HT=y
CONFIG_X86_TRAMPOLINE=y
# CONFIG_KTIME_SCALAR is not set
CONFIG_ARCH_CPU_PROBE_RELEASE=y
CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"
CONFIG_CONSTRUCTORS=y
#
# General setup
#
CONFIG_EXPERIMENTAL=y
CONFIG_LOCK_KERNEL=y
CONFIG_INIT_ENV_ARG_LIMIT=32
CONFIG_LOCALVERSION=""
# CONFIG_LOCALVERSION_AUTO is not set
CONFIG_HAVE_KERNEL_GZIP=y
CONFIG_HAVE_KERNEL_BZIP2=y
CONFIG_HAVE_KERNEL_LZMA=y
CONFIG_KERNEL_GZIP=y
# CONFIG_KERNEL_BZIP2 is not set
# CONFIG_KERNEL_LZMA is not set
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_SYSVIPC_SYSCTL=y
CONFIG_POSIX_MQUEUE=y
CONFIG_POSIX_MQUEUE_SYSCTL=y
CONFIG_BSD_PROCESS_ACCT=y
# CONFIG_BSD_PROCESS_ACCT_V3 is not set
CONFIG_TASKSTATS=y
CONFIG_TASK_DELAY_ACCT=y
# CONFIG_TASK_XACCT is not set
CONFIG_AUDIT=y
CONFIG_AUDITSYSCALL=y
CONFIG_AUDIT_TREE=y
#
# RCU Subsystem
#
CONFIG_TREE_RCU=y
# CONFIG_TREE_PREEMPT_RCU is not set
# CONFIG_RCU_TRACE is not set
CONFIG_RCU_FANOUT=64
# CONFIG_RCU_FANOUT_EXACT is not set
# CONFIG_TREE_RCU_TRACE is not set
# CONFIG_IKCONFIG is not set
CONFIG_LOG_BUF_SHIFT=17
CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y
# CONFIG_GROUP_SCHED is not set
# CONFIG_CGROUPS is not set
CONFIG_SYSFS_DEPRECATED=y
CONFIG_SYSFS_DEPRECATED_V2=y
CONFIG_RELAY=y
CONFIG_NAMESPACES=y
CONFIG_UTS_NS=y
CONFIG_IPC_NS=y
# CONFIG_USER_NS is not set
# CONFIG_PID_NS is not set
# CONFIG_NET_NS is not set
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE=""
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
CONFIG_CC_OPTIMIZE_FOR_SIZE=y
CONFIG_SYSCTL=y
CONFIG_ANON_INODES=y
# CONFIG_EMBEDDED is not set
CONFIG_UID16=y
CONFIG_SYSCTL_SYSCALL=y
CONFIG_KALLSYMS=y
CONFIG_KALLSYMS_ALL=y
CONFIG_KALLSYMS_EXTRA_PASS=y
CONFIG_HOTPLUG=y
CONFIG_PRINTK=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
CONFIG_PCSPKR_PLATFORM=y
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_SIGNALFD=y
CONFIG_TIMERFD=y
CONFIG_EVENTFD=y
CONFIG_SHMEM=y
CONFIG_AIO=y
CONFIG_HAVE_PERF_EVENTS=y
#
# Kernel Performance Events And Counters
#
CONFIG_PERF_EVENTS=y
CONFIG_PERF_COUNTERS=y
# CONFIG_DEBUG_PERF_USE_VMALLOC is not set
CONFIG_VM_EVENT_COUNTERS=y
CONFIG_PCI_QUIRKS=y
# CONFIG_COMPAT_BRK is not set
CONFIG_SLAB=y
# CONFIG_SLUB is not set
# CONFIG_SLOB is not set
CONFIG_PROFILING=y
CONFIG_OPROFILE=m
# CONFIG_OPROFILE_IBS is not set
# CONFIG_OPROFILE_EVENT_MULTIPLEX is not set
CONFIG_HAVE_OPROFILE=y
CONFIG_KPROBES=y
CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
CONFIG_KRETPROBES=y
CONFIG_HAVE_IOREMAP_PROT=y
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_HAVE_ARCH_TRACEHOOK=y
CONFIG_HAVE_DMA_ATTRS=y
CONFIG_HAVE_DMA_API_DEBUG=y
#
# GCOV-based kernel profiling
#
# CONFIG_GCOV_KERNEL is not set
CONFIG_SLOW_WORK=y
# CONFIG_SLOW_WORK_DEBUG is not set
# CONFIG_HAVE_GENERIC_DMA_COHERENT is not set
CONFIG_SLABINFO=y
CONFIG_RT_MUTEXES=y
CONFIG_BASE_SMALL=0
CONFIG_MODULES=y
# CONFIG_MODULE_FORCE_LOAD is not set
CONFIG_MODULE_UNLOAD=y
# CONFIG_MODULE_FORCE_UNLOAD is not set
CONFIG_MODVERSIONS=y
CONFIG_MODULE_SRCVERSION_ALL=y
CONFIG_STOP_MACHINE=y
CONFIG_BLOCK=y
# CONFIG_BLK_DEV_BSG is not set
# CONFIG_BLK_DEV_INTEGRITY is not set
CONFIG_BLOCK_COMPAT=y
#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
# CONFIG_DEFAULT_AS is not set
# CONFIG_DEFAULT_DEADLINE is not set
CONFIG_DEFAULT_CFQ=y
# CONFIG_DEFAULT_NOOP is not set
CONFIG_DEFAULT_IOSCHED="cfq"
CONFIG_FREEZER=y
#
# Processor type and features
#
CONFIG_TICK_ONESHOT=y
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y
CONFIG_GENERIC_CLOCKEVENTS_BUILD=y
CONFIG_SMP=y
# CONFIG_SPARSE_IRQ is not set
CONFIG_X86_MPPARSE=y
CONFIG_X86_EXTENDED_PLATFORM=y
# CONFIG_X86_VSMP is not set
CONFIG_X86_SUPPORTS_MEMORY_FAILURE=y
CONFIG_SCHED_OMIT_FRAME_POINTER=y
CONFIG_PARAVIRT_GUEST=y
CONFIG_XEN=y
CONFIG_XEN_PVHVM=y
CONFIG_XEN_MAX_DOMAIN_MEMORY=128
CONFIG_XEN_SAVE_RESTORE=y
# CONFIG_XEN_DEBUG_FS is not set
CONFIG_SWIOTLB_XEN=y
CONFIG_MICROCODE_XEN=y
CONFIG_XEN_DOM0=y
CONFIG_XEN_PRIVILEGED_GUEST=y
CONFIG_XEN_DOM0_PCI=y
CONFIG_XEN_PCI_PASSTHROUGH=y
# CONFIG_KVM_CLOCK is not set
# CONFIG_KVM_GUEST is not set
CONFIG_PARAVIRT=y
# CONFIG_PARAVIRT_SPINLOCKS is not set
CONFIG_PARAVIRT_CLOCK=y
# CONFIG_PARAVIRT_DEBUG is not set
CONFIG_MEMTEST=y
# CONFIG_M386 is not set
# CONFIG_M486 is not set
# CONFIG_M586 is not set
# CONFIG_M586TSC is not set
# CONFIG_M586MMX is not set
# CONFIG_M686 is not set
# CONFIG_MPENTIUMII is not set
# CONFIG_MPENTIUMIII is not set
# CONFIG_MPENTIUMM is not set
# CONFIG_MPENTIUM4 is not set
# CONFIG_MK6 is not set
# CONFIG_MK7 is not set
# CONFIG_MK8 is not set
# CONFIG_MCRUSOE is not set
# CONFIG_MEFFICEON is not set
# CONFIG_MWINCHIPC6 is not set
# CONFIG_MWINCHIP3D is not set
# CONFIG_MGEODEGX1 is not set
# CONFIG_MGEODE_LX is not set
# CONFIG_MCYRIXIII is not set
# CONFIG_MVIAC3_2 is not set
# CONFIG_MVIAC7 is not set
# CONFIG_MPSC is not set
# CONFIG_MCORE2 is not set
# CONFIG_MATOM is not set
CONFIG_GENERIC_CPU=y
CONFIG_X86_CPU=y
CONFIG_X86_L1_CACHE_BYTES=64
CONFIG_X86_INTERNODE_CACHE_BYTES=64
CONFIG_X86_CMPXCHG=y
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_X86_XADD=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_TSC=y
CONFIG_X86_CMPXCHG64=y
CONFIG_X86_CMOV=y
CONFIG_X86_MINIMUM_CPU_FAMILY=64
CONFIG_X86_DEBUGCTLMSR=y
CONFIG_CPU_SUP_INTEL=y
CONFIG_CPU_SUP_AMD=y
CONFIG_CPU_SUP_CENTAUR=y
# CONFIG_X86_DS is not set
CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
CONFIG_DMI=y
CONFIG_GART_IOMMU=y
CONFIG_CALGARY_IOMMU=y
# CONFIG_CALGARY_IOMMU_ENABLED_BY_DEFAULT is not set
CONFIG_AMD_IOMMU=y
# CONFIG_AMD_IOMMU_STATS is not set
CONFIG_SWIOTLB=y
CONFIG_IOMMU_HELPER=y
CONFIG_IOMMU_API=y
CONFIG_MAXSMP=y
CONFIG_NR_CPUS=4096
CONFIG_SCHED_SMT=y
CONFIG_SCHED_MC=y
# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_REROUTE_FOR_BROKEN_BOOT_IRQS=y
CONFIG_X86_MCE=y
CONFIG_X86_MCE_INTEL=y
CONFIG_X86_MCE_AMD=y
CONFIG_X86_MCE_THRESHOLD=y
# CONFIG_X86_MCE_INJECT is not set
CONFIG_X86_THERMAL_VECTOR=y
# CONFIG_I8K is not set
CONFIG_MICROCODE=m
CONFIG_MICROCODE_INTEL=y
CONFIG_MICROCODE_AMD=y
CONFIG_MICROCODE_OLD_INTERFACE=y
CONFIG_X86_MSR=y
CONFIG_X86_CPUID=y
CONFIG_ARCH_PHYS_ADDR_T_64BIT=y
CONFIG_DIRECT_GBPAGES=y
CONFIG_NUMA=y
CONFIG_K8_NUMA=y
CONFIG_X86_64_ACPI_NUMA=y
CONFIG_NODES_SPAN_OTHER_NODES=y
# CONFIG_NUMA_EMU is not set
CONFIG_NODES_SHIFT=9
CONFIG_ARCH_PROC_KCORE_TEXT=y
CONFIG_ARCH_SPARSEMEM_DEFAULT=y
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SELECT_MEMORY_MODEL=y
CONFIG_ARCH_MEMORY_PROBE=y
CONFIG_ILLEGAL_POINTER_VALUE=0xdead000000000000
CONFIG_SELECT_MEMORY_MODEL=y
# CONFIG_FLATMEM_MANUAL is not set
# CONFIG_DISCONTIGMEM_MANUAL is not set
CONFIG_SPARSEMEM_MANUAL=y
CONFIG_SPARSEMEM=y
CONFIG_NEED_MULTIPLE_NODES=y
CONFIG_HAVE_MEMORY_PRESENT=y
CONFIG_SPARSEMEM_EXTREME=y
CONFIG_SPARSEMEM_VMEMMAP_ENABLE=y
CONFIG_SPARSEMEM_VMEMMAP=y
CONFIG_MEMORY_HOTPLUG=y
CONFIG_MEMORY_HOTPLUG_SPARSE=y
# CONFIG_MEMORY_HOTREMOVE is not set
CONFIG_PAGEFLAGS_EXTENDED=y
CONFIG_SPLIT_PTLOCK_CPUS=4
CONFIG_MIGRATION=y
CONFIG_PHYS_ADDR_T_64BIT=y
CONFIG_ZONE_DMA_FLAG=1
CONFIG_BOUNCE=y
CONFIG_VIRT_TO_BUS=y
CONFIG_HAVE_MLOCK=y
CONFIG_HAVE_MLOCKED_PAGE_BIT=y
CONFIG_MMU_NOTIFIER=y
CONFIG_KSM=y
CONFIG_DEFAULT_MMAP_MIN_ADDR=4096
CONFIG_ARCH_SUPPORTS_MEMORY_FAILURE=y
# CONFIG_MEMORY_FAILURE is not set
CONFIG_X86_CHECK_BIOS_CORRUPTION=y
# CONFIG_X86_BOOTPARAM_MEMORY_CORRUPTION_CHECK is not set
# CONFIG_X86_RESERVE_LOW_64K is not set
CONFIG_MTRR=y
# CONFIG_MTRR_SANITIZER is not set
CONFIG_X86_PAT=y
CONFIG_ARCH_USES_PG_UNCACHED=y
# CONFIG_EFI is not set
# CONFIG_SECCOMP is not set
# CONFIG_CC_STACKPROTECTOR is not set
# CONFIG_HZ_100 is not set
# CONFIG_HZ_250 is not set
# CONFIG_HZ_300 is not set
CONFIG_HZ_1000=y
CONFIG_HZ=1000
CONFIG_SCHED_HRTICK=y
CONFIG_KEXEC=y
CONFIG_CRASH_DUMP=y
CONFIG_PHYSICAL_START=0x200000
CONFIG_RELOCATABLE=y
CONFIG_PHYSICAL_ALIGN=0x1000000
CONFIG_HOTPLUG_CPU=y
# CONFIG_COMPAT_VDSO is not set
# CONFIG_CMDLINE_BOOL is not set
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
CONFIG_ARCH_ENABLE_MEMORY_HOTREMOVE=y
CONFIG_HAVE_ARCH_EARLY_PFN_TO_NID=y
#
# Power management and ACPI options
#
CONFIG_PM=y
# CONFIG_PM_DEBUG is not set
CONFIG_PM_SLEEP_SMP=y
CONFIG_PM_SLEEP=y
# CONFIG_SUSPEND is not set
# CONFIG_HIBERNATION is not set
# CONFIG_PM_RUNTIME is not set
CONFIG_ACPI=y
CONFIG_ACPI_PROCFS=y
# CONFIG_ACPI_PROCFS_POWER is not set
CONFIG_ACPI_POWER_METER=m
# CONFIG_ACPI_SYSFS_POWER is not set
CONFIG_ACPI_PROC_EVENT=y
CONFIG_ACPI_AC=m
CONFIG_ACPI_BATTERY=m
CONFIG_ACPI_BUTTON=m
CONFIG_ACPI_VIDEO=m
CONFIG_ACPI_FAN=y
CONFIG_ACPI_DOCK=y
CONFIG_ACPI_PROCESSOR=y
CONFIG_ACPI_HOTPLUG_CPU=y
CONFIG_ACPI_PROCESSOR_AGGREGATOR=m
CONFIG_ACPI_THERMAL=y
CONFIG_ACPI_NUMA=y
# CONFIG_ACPI_CUSTOM_DSDT is not set
CONFIG_ACPI_BLACKLIST_YEAR=0
# CONFIG_ACPI_DEBUG is not set
# CONFIG_ACPI_PCI_SLOT is not set
CONFIG_X86_PM_TIMER=y
CONFIG_ACPI_CONTAINER=y
CONFIG_ACPI_HOTPLUG_MEMORY=m
CONFIG_ACPI_SBS=m
# CONFIG_SFI is not set
#
# CPU Frequency scaling
#
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_TABLE=y
CONFIG_CPU_FREQ_DEBUG=y
CONFIG_CPU_FREQ_STAT=m
CONFIG_CPU_FREQ_STAT_DETAILS=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_POWERSAVE is not set
CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE is not set
CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=m
CONFIG_CPU_FREQ_GOV_USERSPACE=y
CONFIG_CPU_FREQ_GOV_ONDEMAND=m
CONFIG_CPU_FREQ_GOV_CONSERVATIVE=m
#
# CPUFreq processor drivers
#
CONFIG_X86_ACPI_CPUFREQ=m
CONFIG_X86_POWERNOW_K8=y
CONFIG_X86_SPEEDSTEP_CENTRINO=y
CONFIG_X86_P4_CLOCKMOD=m
#
# shared options
#
CONFIG_X86_SPEEDSTEP_LIB=m
CONFIG_CPU_IDLE=y
CONFIG_CPU_IDLE_GOV_LADDER=y
CONFIG_CPU_IDLE_GOV_MENU=y
#
# Memory power savings
#
# CONFIG_I7300_IDLE is not set
#
# Bus options (PCI etc.)
#
CONFIG_PCI=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
CONFIG_PCI_XEN=y
CONFIG_PCI_DOMAINS=y
# CONFIG_DMAR is not set
# CONFIG_INTR_REMAP is not set
CONFIG_PCIEPORTBUS=y
CONFIG_HOTPLUG_PCI_PCIE=m
# CONFIG_PCIEAER is not set
# CONFIG_PCIEASPM is not set
CONFIG_ARCH_SUPPORTS_MSI=y
CONFIG_PCI_MSI=y
CONFIG_PCI_LEGACY=y
# CONFIG_PCI_DEBUG is not set
# CONFIG_PCI_STUB is not set
CONFIG_XEN_PCIDEV_FRONTEND=y
# CONFIG_HT_IRQ is not set
# CONFIG_PCI_IOV is not set
CONFIG_ISA_DMA_API=y
CONFIG_K8_NB=y
CONFIG_PCCARD=y
# CONFIG_PCMCIA_DEBUG is not set
CONFIG_PCMCIA=y
CONFIG_PCMCIA_LOAD_CIS=y
CONFIG_PCMCIA_IOCTL=y
CONFIG_CARDBUS=y
#
# PC-card bridges
#
CONFIG_YENTA=y
CONFIG_YENTA_O2=y
CONFIG_YENTA_RICOH=y
CONFIG_YENTA_TI=y
CONFIG_YENTA_ENE_TUNE=y
CONFIG_YENTA_TOSHIBA=y
CONFIG_PD6729=m
# CONFIG_I82092 is not set
CONFIG_PCCARD_NONSTATIC=y
CONFIG_HOTPLUG_PCI=y
CONFIG_HOTPLUG_PCI_FAKE=m
CONFIG_HOTPLUG_PCI_ACPI=m
CONFIG_HOTPLUG_PCI_ACPI_IBM=m
# CONFIG_HOTPLUG_PCI_CPCI is not set
CONFIG_HOTPLUG_PCI_SHPC=m
#
# Executable file formats / Emulations
#
CONFIG_BINFMT_ELF=y
CONFIG_COMPAT_BINFMT_ELF=y
# CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set
# CONFIG_HAVE_AOUT is not set
CONFIG_BINFMT_MISC=y
CONFIG_IA32_EMULATION=y
# CONFIG_IA32_AOUT is not set
CONFIG_COMPAT=y
CONFIG_COMPAT_FOR_U64_ALIGNMENT=y
CONFIG_SYSVIPC_COMPAT=y
CONFIG_NET=y
#
# Networking options
#
CONFIG_PACKET=y
CONFIG_PACKET_MMAP=y
CONFIG_UNIX=y
CONFIG_XFRM=y
CONFIG_XFRM_USER=y
# CONFIG_XFRM_SUB_POLICY is not set
# CONFIG_XFRM_MIGRATE is not set
# CONFIG_XFRM_STATISTICS is not set
CONFIG_XFRM_IPCOMP=m
CONFIG_NET_KEY=m
# CONFIG_NET_KEY_MIGRATE is not set
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_ASK_IP_FIB_HASH=y
# CONFIG_IP_FIB_TRIE is not set
CONFIG_IP_FIB_HASH=y
CONFIG_IP_MULTIPLE_TABLES=y
CONFIG_IP_ROUTE_MULTIPATH=y
CONFIG_IP_ROUTE_VERBOSE=y
# CONFIG_IP_PNP is not set
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE=m
CONFIG_NET_IPGRE_BROADCAST=y
CONFIG_IP_MROUTE=y
CONFIG_IP_PIMSM_V1=y
CONFIG_IP_PIMSM_V2=y
# CONFIG_ARPD is not set
CONFIG_SYN_COOKIES=y
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
CONFIG_INET_IPCOMP=m
CONFIG_INET_XFRM_TUNNEL=m
CONFIG_INET_TUNNEL=m
CONFIG_INET_XFRM_MODE_TRANSPORT=m
CONFIG_INET_XFRM_MODE_TUNNEL=m
CONFIG_INET_XFRM_MODE_BEET=m
CONFIG_INET_LRO=y
CONFIG_INET_DIAG=m
CONFIG_INET_TCP_DIAG=m
CONFIG_TCP_CONG_ADVANCED=y
CONFIG_TCP_CONG_BIC=y
CONFIG_TCP_CONG_CUBIC=m
CONFIG_TCP_CONG_WESTWOOD=m
CONFIG_TCP_CONG_HTCP=m
CONFIG_TCP_CONG_HSTCP=m
CONFIG_TCP_CONG_HYBLA=m
CONFIG_TCP_CONG_VEGAS=m
CONFIG_TCP_CONG_SCALABLE=m
CONFIG_TCP_CONG_LP=m
CONFIG_TCP_CONG_VENO=m
CONFIG_TCP_CONG_YEAH=m
CONFIG_TCP_CONG_ILLINOIS=m
CONFIG_DEFAULT_BIC=y
# CONFIG_DEFAULT_CUBIC is not set
# CONFIG_DEFAULT_HTCP is not set
# CONFIG_DEFAULT_VEGAS is not set
# CONFIG_DEFAULT_WESTWOOD is not set
# CONFIG_DEFAULT_RENO is not set
CONFIG_DEFAULT_TCP_CONG="bic"
# CONFIG_TCP_MD5SIG is not set
CONFIG_IPV6=m
CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_IPV6_ROUTE_INFO=y
CONFIG_IPV6_OPTIMISTIC_DAD=y
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_IPCOMP=m
# CONFIG_IPV6_MIP6 is not set
CONFIG_INET6_XFRM_TUNNEL=m
CONFIG_INET6_TUNNEL=m
CONFIG_INET6_XFRM_MODE_TRANSPORT=m
CONFIG_INET6_XFRM_MODE_TUNNEL=m
CONFIG_INET6_XFRM_MODE_BEET=m
# CONFIG_INET6_XFRM_MODE_ROUTEOPTIMIZATION is not set
CONFIG_IPV6_SIT=m
CONFIG_IPV6_NDISC_NODETYPE=y
CONFIG_IPV6_TUNNEL=m
CONFIG_IPV6_MULTIPLE_TABLES=y
# CONFIG_IPV6_SUBTREES is not set
# CONFIG_IPV6_MROUTE is not set
CONFIG_NETLABEL=y
CONFIG_NETWORK_SECMARK=y
CONFIG_NETFILTER=y
# CONFIG_NETFILTER_DEBUG is not set
CONFIG_NETFILTER_ADVANCED=y
CONFIG_BRIDGE_NETFILTER=y
#
# Core Netfilter Configuration
#
CONFIG_NETFILTER_NETLINK=m
CONFIG_NETFILTER_NETLINK_QUEUE=m
CONFIG_NETFILTER_NETLINK_LOG=m
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CT_ACCT=y
CONFIG_NF_CONNTRACK_MARK=y
CONFIG_NF_CONNTRACK_SECMARK=y
# CONFIG_NF_CONNTRACK_EVENTS is not set
# CONFIG_NF_CT_PROTO_DCCP is not set
CONFIG_NF_CT_PROTO_GRE=m
# CONFIG_NF_CT_PROTO_SCTP is not set
# CONFIG_NF_CT_PROTO_UDPLITE is not set
CONFIG_NF_CONNTRACK_AMANDA=m
CONFIG_NF_CONNTRACK_FTP=m
# CONFIG_NF_CONNTRACK_H323 is not set
CONFIG_NF_CONNTRACK_IRC=m
# CONFIG_NF_CONNTRACK_NETBIOS_NS is not set
CONFIG_NF_CONNTRACK_PPTP=m
# CONFIG_NF_CONNTRACK_SANE is not set
# CONFIG_NF_CONNTRACK_SIP is not set
CONFIG_NF_CONNTRACK_TFTP=m
# CONFIG_NF_CT_NETLINK is not set
# CONFIG_NETFILTER_TPROXY is not set
CONFIG_NETFILTER_XTABLES=m
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
CONFIG_NETFILTER_XT_TARGET_CONNMARK=m
CONFIG_NETFILTER_XT_TARGET_CONNSECMARK=m
CONFIG_NETFILTER_XT_TARGET_DSCP=m
CONFIG_NETFILTER_XT_TARGET_HL=m
CONFIG_NETFILTER_XT_TARGET_LED=m
CONFIG_NETFILTER_XT_TARGET_MARK=m
CONFIG_NETFILTER_XT_TARGET_NFLOG=m
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
CONFIG_NETFILTER_XT_TARGET_NOTRACK=m
CONFIG_NETFILTER_XT_TARGET_RATEEST=m
CONFIG_NETFILTER_XT_TARGET_TRACE=m
CONFIG_NETFILTER_XT_TARGET_SECMARK=m
CONFIG_NETFILTER_XT_TARGET_TCPMSS=m
# CONFIG_NETFILTER_XT_TARGET_TCPOPTSTRIP is not set
CONFIG_NETFILTER_XT_MATCH_CLUSTER=m
CONFIG_NETFILTER_XT_MATCH_COMMENT=m
CONFIG_NETFILTER_XT_MATCH_CONNBYTES=m
CONFIG_NETFILTER_XT_MATCH_CONNLIMIT=m
CONFIG_NETFILTER_XT_MATCH_CONNMARK=m
CONFIG_NETFILTER_XT_MATCH_CONNTRACK=m
CONFIG_NETFILTER_XT_MATCH_DCCP=m
CONFIG_NETFILTER_XT_MATCH_DSCP=m
CONFIG_NETFILTER_XT_MATCH_ESP=m
CONFIG_NETFILTER_XT_MATCH_HASHLIMIT=m
CONFIG_NETFILTER_XT_MATCH_HELPER=m
CONFIG_NETFILTER_XT_MATCH_HL=m
CONFIG_NETFILTER_XT_MATCH_IPRANGE=m
CONFIG_NETFILTER_XT_MATCH_LENGTH=m
CONFIG_NETFILTER_XT_MATCH_LIMIT=m
CONFIG_NETFILTER_XT_MATCH_MAC=m
CONFIG_NETFILTER_XT_MATCH_MARK=m
CONFIG_NETFILTER_XT_MATCH_MULTIPORT=m
CONFIG_NETFILTER_XT_MATCH_OWNER=m
CONFIG_NETFILTER_XT_MATCH_POLICY=m
CONFIG_NETFILTER_XT_MATCH_PHYSDEV=m
CONFIG_NETFILTER_XT_MATCH_PKTTYPE=m
CONFIG_NETFILTER_XT_MATCH_QUOTA=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
CONFIG_NETFILTER_XT_MATCH_RECENT_PROC_COMPAT=y
CONFIG_NETFILTER_XT_MATCH_SCTP=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_NETFILTER_XT_MATCH_TCPMSS=m
CONFIG_NETFILTER_XT_MATCH_TIME=m
CONFIG_NETFILTER_XT_MATCH_U32=m
CONFIG_NETFILTER_XT_MATCH_OSF=m
CONFIG_IP_VS=m
# CONFIG_IP_VS_IPV6 is not set
# CONFIG_IP_VS_DEBUG is not set
CONFIG_IP_VS_TAB_BITS=12
#
# IPVS transport protocol load balancing support
#
CONFIG_IP_VS_PROTO_TCP=y
CONFIG_IP_VS_PROTO_UDP=y
CONFIG_IP_VS_PROTO_AH_ESP=y
CONFIG_IP_VS_PROTO_ESP=y
CONFIG_IP_VS_PROTO_AH=y
#
# IPVS scheduler
#
CONFIG_IP_VS_RR=m
CONFIG_IP_VS_WRR=m
CONFIG_IP_VS_LC=m
CONFIG_IP_VS_WLC=m
CONFIG_IP_VS_LBLC=m
CONFIG_IP_VS_LBLCR=m
CONFIG_IP_VS_DH=m
CONFIG_IP_VS_SH=m
CONFIG_IP_VS_SED=m
CONFIG_IP_VS_NQ=m
#
# IPVS application helper
#
CONFIG_IP_VS_FTP=m
#
# IP: Netfilter Configuration
#
CONFIG_NF_DEFRAG_IPV4=m
CONFIG_NF_CONNTRACK_IPV4=m
CONFIG_NF_CONNTRACK_PROC_COMPAT=y
CONFIG_IP_NF_QUEUE=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_ADDRTYPE=m
CONFIG_IP_NF_MATCH_AH=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
CONFIG_IP_NF_TARGET_LOG=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_NF_NAT=m
CONFIG_NF_NAT_NEEDED=y
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_TARGET_NETMAP=m
CONFIG_IP_NF_TARGET_REDIRECT=m
CONFIG_NF_NAT_SNMP_BASIC=m
CONFIG_NF_NAT_PROTO_GRE=m
CONFIG_NF_NAT_FTP=m
CONFIG_NF_NAT_IRC=m
CONFIG_NF_NAT_TFTP=m
CONFIG_NF_NAT_AMANDA=m
CONFIG_NF_NAT_PPTP=m
# CONFIG_NF_NAT_H323 is not set
# CONFIG_NF_NAT_SIP is not set
CONFIG_IP_NF_MANGLE=m
CONFIG_IP_NF_TARGET_CLUSTERIP=m
CONFIG_IP_NF_TARGET_ECN=m
CONFIG_IP_NF_TARGET_TTL=m
CONFIG_IP_NF_RAW=m
# CONFIG_IP_NF_SECURITY is not set
CONFIG_IP_NF_ARPTABLES=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m
#
# IPv6: Netfilter Configuration
#
CONFIG_NF_CONNTRACK_IPV6=m
CONFIG_IP6_NF_QUEUE=m
CONFIG_IP6_NF_IPTABLES=m
CONFIG_IP6_NF_MATCH_AH=m
CONFIG_IP6_NF_MATCH_EUI64=m
CONFIG_IP6_NF_MATCH_FRAG=m
CONFIG_IP6_NF_MATCH_OPTS=m
CONFIG_IP6_NF_MATCH_HL=m
CONFIG_IP6_NF_MATCH_IPV6HEADER=m
CONFIG_IP6_NF_MATCH_MH=m
CONFIG_IP6_NF_MATCH_RT=m
CONFIG_IP6_NF_TARGET_HL=m
CONFIG_IP6_NF_TARGET_LOG=m
CONFIG_IP6_NF_FILTER=m
CONFIG_IP6_NF_TARGET_REJECT=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_RAW=m
# CONFIG_IP6_NF_SECURITY is not set
CONFIG_BRIDGE_NF_EBTABLES=m
CONFIG_BRIDGE_EBT_BROUTE=m
CONFIG_BRIDGE_EBT_T_FILTER=m
CONFIG_BRIDGE_EBT_T_NAT=m
CONFIG_BRIDGE_EBT_802_3=m
CONFIG_BRIDGE_EBT_AMONG=m
CONFIG_BRIDGE_EBT_ARP=m
CONFIG_BRIDGE_EBT_IP=m
CONFIG_BRIDGE_EBT_IP6=m
CONFIG_BRIDGE_EBT_LIMIT=m
CONFIG_BRIDGE_EBT_MARK=m
CONFIG_BRIDGE_EBT_PKTTYPE=m
CONFIG_BRIDGE_EBT_STP=m
CONFIG_BRIDGE_EBT_VLAN=m
CONFIG_BRIDGE_EBT_ARPREPLY=m
CONFIG_BRIDGE_EBT_DNAT=m
CONFIG_BRIDGE_EBT_MARK_T=m
CONFIG_BRIDGE_EBT_REDIRECT=m
CONFIG_BRIDGE_EBT_SNAT=m
CONFIG_BRIDGE_EBT_LOG=m
CONFIG_BRIDGE_EBT_ULOG=m
CONFIG_BRIDGE_EBT_NFLOG=m
CONFIG_IP_DCCP=m
CONFIG_INET_DCCP_DIAG=m
#
# DCCP CCIDs Configuration (EXPERIMENTAL)
#
# CONFIG_IP_DCCP_CCID2_DEBUG is not set
# CONFIG_IP_DCCP_CCID3 is not set
#
# DCCP Kernel Hacking
#
# CONFIG_IP_DCCP_DEBUG is not set
# CONFIG_NET_DCCPPROBE is not set
CONFIG_IP_SCTP=m
# CONFIG_SCTP_DBG_MSG is not set
# CONFIG_SCTP_DBG_OBJCNT is not set
# CONFIG_SCTP_HMAC_NONE is not set
# CONFIG_SCTP_HMAC_SHA1 is not set
CONFIG_SCTP_HMAC_MD5=y
# CONFIG_RDS is not set
CONFIG_TIPC=m
# CONFIG_TIPC_ADVANCED is not set
# CONFIG_TIPC_DEBUG is not set
CONFIG_ATM=m
CONFIG_ATM_CLIP=m
# CONFIG_ATM_CLIP_NO_ICMP is not set
CONFIG_ATM_LANE=m
# CONFIG_ATM_MPOA is not set
CONFIG_ATM_BR2684=m
# CONFIG_ATM_BR2684_IPFILTER is not set
CONFIG_STP=m
CONFIG_BRIDGE=m
# CONFIG_NET_DSA is not set
CONFIG_VLAN_8021Q=m
# CONFIG_VLAN_8021Q_GVRP is not set
# CONFIG_DECNET is not set
CONFIG_LLC=y
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
# CONFIG_PHONET is not set
# CONFIG_IEEE802154 is not set
CONFIG_NET_SCHED=y
#
# Queueing/Scheduling
#
CONFIG_NET_SCH_CBQ=m
CONFIG_NET_SCH_HTB=m
CONFIG_NET_SCH_HFSC=m
CONFIG_NET_SCH_ATM=m
CONFIG_NET_SCH_PRIO=m
CONFIG_NET_SCH_MULTIQ=m
CONFIG_NET_SCH_RED=m
CONFIG_NET_SCH_SFQ=m
CONFIG_NET_SCH_TEQL=m
CONFIG_NET_SCH_TBF=m
CONFIG_NET_SCH_GRED=m
CONFIG_NET_SCH_DSMARK=m
CONFIG_NET_SCH_NETEM=m
# CONFIG_NET_SCH_DRR is not set
CONFIG_NET_SCH_INGRESS=m
# CONFIG_NET_SCH_PLUG is not set
#
# Classification
#
CONFIG_NET_CLS=y
CONFIG_NET_CLS_BASIC=m
CONFIG_NET_CLS_TCINDEX=m
CONFIG_NET_CLS_ROUTE4=m
CONFIG_NET_CLS_ROUTE=y
CONFIG_NET_CLS_FW=m
CONFIG_NET_CLS_U32=m
CONFIG_CLS_U32_PERF=y
CONFIG_CLS_U32_MARK=y
CONFIG_NET_CLS_RSVP=m
CONFIG_NET_CLS_RSVP6=m
CONFIG_NET_CLS_FLOW=m
CONFIG_NET_EMATCH=y
CONFIG_NET_EMATCH_STACK=32
CONFIG_NET_EMATCH_CMP=m
CONFIG_NET_EMATCH_NBYTE=m
CONFIG_NET_EMATCH_U32=m
CONFIG_NET_EMATCH_META=m
CONFIG_NET_EMATCH_TEXT=m
CONFIG_NET_CLS_ACT=y
CONFIG_NET_ACT_POLICE=m
CONFIG_NET_ACT_GACT=m
CONFIG_GACT_PROB=y
CONFIG_NET_ACT_MIRRED=m
CONFIG_NET_ACT_IPT=m
CONFIG_NET_ACT_NAT=m
CONFIG_NET_ACT_PEDIT=m
CONFIG_NET_ACT_SIMP=m
# CONFIG_NET_ACT_SKBEDIT is not set
CONFIG_NET_CLS_IND=y
CONFIG_NET_SCH_FIFO=y
# CONFIG_DCB is not set
#
# Network testing
#
CONFIG_NET_PKTGEN=m
# CONFIG_NET_TCPPROBE is not set
# CONFIG_HAMRADIO is not set
# CONFIG_CAN is not set
# CONFIG_IRDA is not set
CONFIG_BT=m
CONFIG_BT_L2CAP=m
CONFIG_BT_SCO=m
CONFIG_BT_RFCOMM=m
CONFIG_BT_RFCOMM_TTY=y
CONFIG_BT_BNEP=m
CONFIG_BT_BNEP_MC_FILTER=y
CONFIG_BT_BNEP_PROTO_FILTER=y
CONFIG_BT_HIDP=m
#
# Bluetooth device drivers
#
# CONFIG_BT_HCIBTUSB is not set
CONFIG_BT_HCIUART=m
CONFIG_BT_HCIUART_H4=y
CONFIG_BT_HCIUART_BCSP=y
# CONFIG_BT_HCIUART_LL is not set
CONFIG_BT_HCIBCM203X=m
CONFIG_BT_HCIBPA10X=m
CONFIG_BT_HCIBFUSB=m
CONFIG_BT_HCIDTL1=m
CONFIG_BT_HCIBT3C=m
CONFIG_BT_HCIBLUECARD=m
CONFIG_BT_HCIBTUART=m
CONFIG_BT_HCIVHCI=m
# CONFIG_BT_MRVL is not set
# CONFIG_AF_RXRPC is not set
CONFIG_FIB_RULES=y
# CONFIG_WIRELESS is not set
# CONFIG_WIMAX is not set
CONFIG_RFKILL=m
CONFIG_RFKILL_LEDS=y
CONFIG_RFKILL_INPUT=y
# CONFIG_NET_9P is not set
#
# Device Drivers
#
#
# Generic Driver Options
#
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_DEVTMPFS is not set
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y
CONFIG_FW_LOADER=y
CONFIG_FIRMWARE_IN_KERNEL=y
CONFIG_EXTRA_FIRMWARE=""
# CONFIG_DEBUG_DRIVER is not set
# CONFIG_DEBUG_DEVRES is not set
CONFIG_SYS_HYPERVISOR=y
CONFIG_CONNECTOR=y
CONFIG_PROC_EVENTS=y
CONFIG_MTD=m
# CONFIG_MTD_DEBUG is not set
# CONFIG_MTD_TESTS is not set
CONFIG_MTD_CONCAT=m
CONFIG_MTD_PARTITIONS=y
CONFIG_MTD_REDBOOT_PARTS=m
CONFIG_MTD_REDBOOT_DIRECTORY_BLOCK=-1
# CONFIG_MTD_REDBOOT_PARTS_UNALLOCATED is not set
# CONFIG_MTD_REDBOOT_PARTS_READONLY is not set
# CONFIG_MTD_AR7_PARTS is not set
#
# User Modules And Translation Layers
#
CONFIG_MTD_CHAR=m
CONFIG_MTD_BLKDEVS=m
CONFIG_MTD_BLOCK=m
CONFIG_MTD_BLOCK_RO=m
CONFIG_FTL=m
CONFIG_NFTL=m
CONFIG_NFTL_RW=y
# CONFIG_INFTL is not set
CONFIG_RFD_FTL=m
# CONFIG_SSFDC is not set
# CONFIG_MTD_OOPS is not set
#
# RAM/ROM/Flash chip drivers
#
CONFIG_MTD_CFI=m
CONFIG_MTD_JEDECPROBE=m
CONFIG_MTD_GEN_PROBE=m
# CONFIG_MTD_CFI_ADV_OPTIONS is not set
CONFIG_MTD_MAP_BANK_WIDTH_1=y
CONFIG_MTD_MAP_BANK_WIDTH_2=y
CONFIG_MTD_MAP_BANK_WIDTH_4=y
# CONFIG_MTD_MAP_BANK_WIDTH_8 is not set
# CONFIG_MTD_MAP_BANK_WIDTH_16 is not set
# CONFIG_MTD_MAP_BANK_WIDTH_32 is not set
CONFIG_MTD_CFI_I1=y
CONFIG_MTD_CFI_I2=y
# CONFIG_MTD_CFI_I4 is not set
# CONFIG_MTD_CFI_I8 is not set
CONFIG_MTD_CFI_INTELEXT=m
CONFIG_MTD_CFI_AMDSTD=m
CONFIG_MTD_CFI_STAA=m
CONFIG_MTD_CFI_UTIL=m
CONFIG_MTD_RAM=m
CONFIG_MTD_ROM=m
CONFIG_MTD_ABSENT=m
#
# Mapping drivers for chip access
#
# CONFIG_MTD_COMPLEX_MAPPINGS is not set
# CONFIG_MTD_PHYSMAP is not set
CONFIG_MTD_SC520CDP=m
CONFIG_MTD_NETSC520=m
CONFIG_MTD_TS5500=m
# CONFIG_MTD_AMD76XROM is not set
# CONFIG_MTD_ICHXROM is not set
# CONFIG_MTD_ESB2ROM is not set
# CONFIG_MTD_CK804XROM is not set
CONFIG_MTD_SCB2_FLASH=m
# CONFIG_MTD_NETtel is not set
# CONFIG_MTD_L440GX is not set
# CONFIG_MTD_INTEL_VR_NOR is not set
# CONFIG_MTD_PLATRAM is not set
#
# Self-contained MTD device drivers
#
# CONFIG_MTD_PMC551 is not set
# CONFIG_MTD_SLRAM is not set
# CONFIG_MTD_PHRAM is not set
CONFIG_MTD_MTDRAM=m
CONFIG_MTDRAM_TOTAL_SIZE=4096
CONFIG_MTDRAM_ERASE_SIZE=128
CONFIG_MTD_BLOCK2MTD=m
#
# Disk-On-Chip Device Drivers
#
# CONFIG_MTD_DOC2000 is not set
# CONFIG_MTD_DOC2001 is not set
# CONFIG_MTD_DOC2001PLUS is not set
CONFIG_MTD_NAND=m
# CONFIG_MTD_NAND_VERIFY_WRITE is not set
CONFIG_MTD_NAND_ECC_SMC=y
# CONFIG_MTD_NAND_MUSEUM_IDS is not set
CONFIG_MTD_NAND_IDS=m
CONFIG_MTD_NAND_DISKONCHIP=m
# CONFIG_MTD_NAND_DISKONCHIP_PROBE_ADVANCED is not set
CONFIG_MTD_NAND_DISKONCHIP_PROBE_ADDRESS=0
# CONFIG_MTD_NAND_DISKONCHIP_BBTWRITE is not set
# CONFIG_MTD_NAND_CAFE is not set
CONFIG_MTD_NAND_NANDSIM=m
# CONFIG_MTD_NAND_PLATFORM is not set
# CONFIG_MTD_ALAUDA is not set
# CONFIG_MTD_ONENAND is not set
#
# LPDDR flash memory drivers
#
# CONFIG_MTD_LPDDR is not set
#
# UBI - Unsorted block images
#
# CONFIG_MTD_UBI is not set
CONFIG_PARPORT=m
CONFIG_PARPORT_PC=m
CONFIG_PARPORT_SERIAL=m
# CONFIG_PARPORT_PC_FIFO is not set
# CONFIG_PARPORT_PC_SUPERIO is not set
CONFIG_PARPORT_PC_PCMCIA=m
# CONFIG_PARPORT_GSC is not set
# CONFIG_PARPORT_AX88796 is not set
CONFIG_PARPORT_1284=y
CONFIG_PARPORT_NOT_PC=y
CONFIG_PNP=y
# CONFIG_PNP_DEBUG_MESSAGES is not set
#
# Protocols
#
CONFIG_PNPACPI=y
CONFIG_BLK_DEV=y
CONFIG_BLK_DEV_FD=m
CONFIG_PARIDE=m
#
# Parallel IDE high-level drivers
#
CONFIG_PARIDE_PD=m
CONFIG_PARIDE_PCD=m
CONFIG_PARIDE_PF=m
CONFIG_PARIDE_PT=m
CONFIG_PARIDE_PG=m
#
# Parallel IDE protocol modules
#
CONFIG_PARIDE_ATEN=m
CONFIG_PARIDE_BPCK=m
CONFIG_PARIDE_COMM=m
CONFIG_PARIDE_DSTR=m
CONFIG_PARIDE_FIT2=m
CONFIG_PARIDE_FIT3=m
CONFIG_PARIDE_EPAT=m
CONFIG_PARIDE_EPATC8=y
CONFIG_PARIDE_EPIA=m
CONFIG_PARIDE_FRIQ=m
CONFIG_PARIDE_FRPW=m
CONFIG_PARIDE_KBIC=m
CONFIG_PARIDE_KTTI=m
CONFIG_PARIDE_ON20=m
CONFIG_PARIDE_ON26=m
CONFIG_BLK_CPQ_DA=m
CONFIG_BLK_CPQ_CISS_DA=m
CONFIG_CISS_SCSI_TAPE=y
CONFIG_BLK_DEV_DAC960=m
# CONFIG_BLK_DEV_UMEM is not set
# CONFIG_BLK_DEV_COW_COMMON is not set
CONFIG_BLK_DEV_LOOP=m
CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_OSD=m
CONFIG_BLK_DEV_SX8=m
# CONFIG_BLK_DEV_UB is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_COUNT=16
CONFIG_BLK_DEV_RAM_SIZE=16384
# CONFIG_BLK_DEV_XIP is not set
CONFIG_CDROM_PKTCDVD=m
CONFIG_CDROM_PKTCDVD_BUFFERS=8
# CONFIG_CDROM_PKTCDVD_WCACHE is not set
CONFIG_ATA_OVER_ETH=m
CONFIG_XEN_BLKDEV_FRONTEND=y
# CONFIG_BLK_DEV_HD is not set
# CONFIG_MISC_DEVICES is not set
CONFIG_HAVE_IDE=y
CONFIG_IDE=y
#
# Please see Documentation/ide/ide.txt for help/info on IDE drives
#
CONFIG_IDE_XFER_MODE=y
CONFIG_IDE_TIMINGS=y
CONFIG_IDE_ATAPI=y
# CONFIG_BLK_DEV_IDE_SATA is not set
CONFIG_IDE_GD=y
CONFIG_IDE_GD_ATA=y
CONFIG_IDE_GD_ATAPI=y
CONFIG_BLK_DEV_IDECS=m
# CONFIG_BLK_DEV_DELKIN is not set
CONFIG_BLK_DEV_IDECD=m
CONFIG_BLK_DEV_IDECD_VERBOSE_ERRORS=y
# CONFIG_BLK_DEV_IDETAPE is not set
CONFIG_BLK_DEV_IDEACPI=y
CONFIG_IDE_TASK_IOCTL=y
CONFIG_IDE_PROC_FS=y
#
# IDE chipset support/bugfixes
#
CONFIG_IDE_GENERIC=y
# CONFIG_BLK_DEV_PLATFORM is not set
# CONFIG_BLK_DEV_CMD640 is not set
CONFIG_BLK_DEV_IDEPNP=y
CONFIG_BLK_DEV_IDEDMA_SFF=y
#
# PCI IDE chipsets support
#
CONFIG_BLK_DEV_IDEPCI=y
CONFIG_IDEPCI_PCIBUS_ORDER=y
# CONFIG_BLK_DEV_OFFBOARD is not set
CONFIG_BLK_DEV_GENERIC=y
# CONFIG_BLK_DEV_OPTI621 is not set
# CONFIG_BLK_DEV_RZ1000 is not set
CONFIG_BLK_DEV_IDEDMA_PCI=y
CONFIG_BLK_DEV_AEC62XX=y
CONFIG_BLK_DEV_ALI15X3=y
CONFIG_BLK_DEV_AMD74XX=y
CONFIG_BLK_DEV_ATIIXP=y
CONFIG_BLK_DEV_CMD64X=y
# CONFIG_BLK_DEV_TRIFLEX is not set
# CONFIG_BLK_DEV_CS5520 is not set
# CONFIG_BLK_DEV_CS5530 is not set
CONFIG_BLK_DEV_HPT366=y
# CONFIG_BLK_DEV_JMICRON is not set
# CONFIG_BLK_DEV_SC1200 is not set
CONFIG_BLK_DEV_PIIX=y
CONFIG_BLK_DEV_IT8172=m
CONFIG_BLK_DEV_IT8213=m
CONFIG_BLK_DEV_IT821X=y
# CONFIG_BLK_DEV_NS87415 is not set
CONFIG_BLK_DEV_PDC202XX_OLD=y
CONFIG_BLK_DEV_PDC202XX_NEW=y
CONFIG_BLK_DEV_SVWKS=y
CONFIG_BLK_DEV_SIIMAGE=y
CONFIG_BLK_DEV_SIS5513=y
# CONFIG_BLK_DEV_SLC90E66 is not set
# CONFIG_BLK_DEV_TRM290 is not set
CONFIG_BLK_DEV_VIA82CXXX=y
# CONFIG_BLK_DEV_TC86C001 is not set
CONFIG_BLK_DEV_IDEDMA=y
#
# SCSI device support
#
CONFIG_RAID_ATTRS=m
CONFIG_SCSI=m
CONFIG_SCSI_DMA=y
# CONFIG_SCSI_TGT is not set
CONFIG_SCSI_NETLINK=y
CONFIG_SCSI_PROC_FS=y
#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=m
CONFIG_CHR_DEV_ST=m
CONFIG_CHR_DEV_OSST=m
CONFIG_BLK_DEV_SR=m
CONFIG_BLK_DEV_SR_VENDOR=y
CONFIG_CHR_DEV_SG=m
CONFIG_CHR_DEV_SCH=m
CONFIG_SCSI_MULTI_LUN=y
CONFIG_SCSI_CONSTANTS=y
CONFIG_SCSI_LOGGING=y
# CONFIG_SCSI_SCAN_ASYNC is not set
CONFIG_SCSI_WAIT_SCAN=m
#
# SCSI Transports
#
CONFIG_SCSI_SPI_ATTRS=m
CONFIG_SCSI_FC_ATTRS=m
CONFIG_SCSI_ISCSI_ATTRS=m
CONFIG_SCSI_SAS_ATTRS=m
CONFIG_SCSI_SAS_LIBSAS=m
CONFIG_SCSI_SAS_ATA=y
CONFIG_SCSI_SAS_HOST_SMP=y
# CONFIG_SCSI_SAS_LIBSAS_DEBUG is not set
CONFIG_SCSI_SRP_ATTRS=m
CONFIG_SCSI_LOWLEVEL=y
CONFIG_ISCSI_TCP=m
CONFIG_SCSI_CXGB3_ISCSI=m
CONFIG_SCSI_BNX2_ISCSI=m
CONFIG_BE2ISCSI=m
CONFIG_SCSI_ISCSI_TRGT=m
CONFIG_BLK_DEV_3W_XXXX_RAID=m
CONFIG_SCSI_3W_9XXX=m
CONFIG_SCSI_ACARD=m
CONFIG_SCSI_AACRAID=m
CONFIG_SCSI_AIC7XXX=m
CONFIG_AIC7XXX_CMDS_PER_DEVICE=4
CONFIG_AIC7XXX_RESET_DELAY_MS=15000
# CONFIG_AIC7XXX_DEBUG_ENABLE is not set
CONFIG_AIC7XXX_DEBUG_MASK=0
# CONFIG_AIC7XXX_REG_PRETTY_PRINT is not set
CONFIG_SCSI_AIC7XXX_OLD=m
CONFIG_SCSI_AIC79XX=m
CONFIG_AIC79XX_CMDS_PER_DEVICE=4
CONFIG_AIC79XX_RESET_DELAY_MS=15000
# CONFIG_AIC79XX_DEBUG_ENABLE is not set
CONFIG_AIC79XX_DEBUG_MASK=0
# CONFIG_AIC79XX_REG_PRETTY_PRINT is not set
CONFIG_SCSI_AIC94XX=m
# CONFIG_AIC94XX_DEBUG is not set
CONFIG_SCSI_MVSAS=m
# CONFIG_SCSI_MVSAS_DEBUG is not set
# CONFIG_SCSI_DPT_I2O is not set
# CONFIG_SCSI_ADVANSYS is not set
CONFIG_SCSI_ARCMSR=m
CONFIG_MEGARAID_NEWGEN=y
CONFIG_MEGARAID_MM=m
CONFIG_MEGARAID_MAILBOX=m
CONFIG_MEGARAID_LEGACY=m
CONFIG_MEGARAID_SAS=m
# CONFIG_SCSI_MPT2SAS is not set
CONFIG_SCSI_HPTIOP=m
# CONFIG_SCSI_BUSLOGIC is not set
# CONFIG_LIBFC is not set
# CONFIG_LIBFCOE is not set
# CONFIG_FCOE is not set
# CONFIG_FCOE_FNIC is not set
# CONFIG_SCSI_DMX3191D is not set
# CONFIG_SCSI_EATA is not set
# CONFIG_SCSI_FUTURE_DOMAIN is not set
CONFIG_SCSI_GDTH=m
CONFIG_SCSI_IPS=m
CONFIG_SCSI_INITIO=m
# CONFIG_SCSI_INIA100 is not set
CONFIG_SCSI_PPA=m
CONFIG_SCSI_IMM=m
# CONFIG_SCSI_IZIP_EPP16 is not set
# CONFIG_SCSI_IZIP_SLOW_CTR is not set
CONFIG_SCSI_STEX=m
CONFIG_SCSI_SYM53C8XX_2=m
CONFIG_SCSI_SYM53C8XX_DMA_ADDRESSING_MODE=1
CONFIG_SCSI_SYM53C8XX_DEFAULT_TAGS=16
CONFIG_SCSI_SYM53C8XX_MAX_TAGS=64
CONFIG_SCSI_SYM53C8XX_MMIO=y
# CONFIG_SCSI_IPR is not set
CONFIG_SCSI_QLOGIC_1280=m
CONFIG_SCSI_QLA_FC=m
CONFIG_SCSI_QLA_ISCSI=m
CONFIG_SCSI_LPFC=m
# CONFIG_SCSI_LPFC_DEBUG_FS is not set
CONFIG_SCSI_DC395x=m
# CONFIG_SCSI_DC390T is not set
# CONFIG_SCSI_DEBUG is not set
CONFIG_SCSI_PMCRAID=m
# CONFIG_SCSI_SRP is not set
CONFIG_SCSI_BFA_FC=m
# CONFIG_SCSI_LOWLEVEL_PCMCIA is not set
CONFIG_SCSI_DH=m
# CONFIG_SCSI_DH_RDAC is not set
# CONFIG_SCSI_DH_HP_SW is not set
# CONFIG_SCSI_DH_EMC is not set
# CONFIG_SCSI_DH_ALUA is not set
CONFIG_SCSI_OSD_INITIATOR=m
CONFIG_SCSI_OSD_ULD=m
CONFIG_SCSI_OSD_DPRINT_SENSE=1
# CONFIG_SCSI_OSD_DEBUG is not set
CONFIG_ATA=m
# CONFIG_ATA_NONSTANDARD is not set
CONFIG_ATA_VERBOSE_ERROR=y
CONFIG_ATA_ACPI=y
# CONFIG_SATA_PMP is not set
CONFIG_SATA_AHCI=m
CONFIG_SATA_SIL24=m
CONFIG_ATA_SFF=y
CONFIG_SATA_SVW=m
CONFIG_ATA_PIIX=m
CONFIG_SATA_MV=m
CONFIG_SATA_NV=m
CONFIG_PDC_ADMA=m
CONFIG_SATA_QSTOR=m
CONFIG_SATA_PROMISE=m
CONFIG_SATA_SX4=m
CONFIG_SATA_SIL=m
CONFIG_SATA_SIS=m
CONFIG_SATA_ULI=m
CONFIG_SATA_VIA=m
CONFIG_SATA_VITESSE=m
CONFIG_SATA_INIC162X=m
# CONFIG_PATA_ACPI is not set
# CONFIG_PATA_ALI is not set
# CONFIG_PATA_AMD is not set
# CONFIG_PATA_ARTOP is not set
CONFIG_PATA_ATP867X=m
# CONFIG_PATA_ATIIXP is not set
# CONFIG_PATA_CMD640_PCI is not set
# CONFIG_PATA_CMD64X is not set
# CONFIG_PATA_CS5520 is not set
# CONFIG_PATA_CS5530 is not set
# CONFIG_PATA_CYPRESS is not set
# CONFIG_PATA_EFAR is not set
# CONFIG_ATA_GENERIC is not set
# CONFIG_PATA_HPT366 is not set
# CONFIG_PATA_HPT37X is not set
# CONFIG_PATA_HPT3X2N is not set
# CONFIG_PATA_HPT3X3 is not set
# CONFIG_PATA_IT821X is not set
# CONFIG_PATA_IT8213 is not set
# CONFIG_PATA_JMICRON is not set
# CONFIG_PATA_TRIFLEX is not set
CONFIG_PATA_MARVELL=m
# CONFIG_PATA_MPIIX is not set
# CONFIG_PATA_OLDPIIX is not set
# CONFIG_PATA_NETCELL is not set
# CONFIG_PATA_NINJA32 is not set
# CONFIG_PATA_NS87410 is not set
# CONFIG_PATA_NS87415 is not set
# CONFIG_PATA_OPTI is not set
# CONFIG_PATA_OPTIDMA is not set
# CONFIG_PATA_PCMCIA is not set
# CONFIG_PATA_PDC_OLD is not set
# CONFIG_PATA_RADISYS is not set
CONFIG_PATA_RDC=m
# CONFIG_PATA_RZ1000 is not set
# CONFIG_PATA_SC1200 is not set
# CONFIG_PATA_SERVERWORKS is not set
CONFIG_PATA_PDC2027X=m
# CONFIG_PATA_SIL680 is not set
CONFIG_PATA_SIS=m
# CONFIG_PATA_VIA is not set
# CONFIG_PATA_WINBOND is not set
CONFIG_PATA_SCH=m
CONFIG_MD=y
CONFIG_BLK_DEV_MD=y
CONFIG_MD_AUTODETECT=y
CONFIG_MD_LINEAR=m
CONFIG_MD_RAID0=m
CONFIG_MD_RAID1=m
CONFIG_MD_RAID10=m
CONFIG_MD_RAID456=m
# CONFIG_MULTICORE_RAID456 is not set
CONFIG_MD_RAID6_PQ=m
# CONFIG_ASYNC_RAID6_TEST is not set
CONFIG_MD_MULTIPATH=m
CONFIG_MD_FAULTY=m
CONFIG_BLK_DEV_DM=m
# CONFIG_DM_DEBUG is not set
CONFIG_DM_CRYPT=m
CONFIG_DM_SNAPSHOT=m
CONFIG_DM_MIRROR=m
# CONFIG_DM_LOG_USERSPACE is not set
CONFIG_DM_ZERO=m
CONFIG_DM_MULTIPATH=m
# CONFIG_DM_MULTIPATH_QL is not set
# CONFIG_DM_MULTIPATH_ST is not set
# CONFIG_DM_DELAY is not set
CONFIG_DM_UEVENT=y
CONFIG_FUSION=y
CONFIG_FUSION_SPI=m
CONFIG_FUSION_FC=m
CONFIG_FUSION_SAS=m
CONFIG_FUSION_MAX_SGE=128
CONFIG_FUSION_CTL=m
CONFIG_FUSION_LAN=m
CONFIG_FUSION_LOGGING=y
#
# IEEE 1394 (FireWire) support
#
#
# You can enable one or both FireWire driver stacks.
#
#
# See the help texts for more information.
#
CONFIG_FIREWIRE=m
CONFIG_FIREWIRE_OHCI=m
CONFIG_FIREWIRE_OHCI_DEBUG=y
CONFIG_FIREWIRE_SBP2=m
# CONFIG_FIREWIRE_NET is not set
# CONFIG_IEEE1394 is not set
CONFIG_I2O=m
# CONFIG_I2O_LCT_NOTIFY_ON_CHANGES is not set
CONFIG_I2O_EXT_ADAPTEC=y
CONFIG_I2O_EXT_ADAPTEC_DMA64=y
CONFIG_I2O_CONFIG=m
CONFIG_I2O_CONFIG_OLD_IOCTL=y
CONFIG_I2O_BUS=m
CONFIG_I2O_BLOCK=m
CONFIG_I2O_SCSI=m
CONFIG_I2O_PROC=m
# CONFIG_MACINTOSH_DRIVERS is not set
CONFIG_NETDEVICES=y
CONFIG_IFB=m
CONFIG_DUMMY=m
CONFIG_BONDING=m
# CONFIG_MACVLAN is not set
# CONFIG_EQUALIZER is not set
CONFIG_TUN=m
# CONFIG_VETH is not set
# CONFIG_NET_SB1000 is not set
# CONFIG_ARCNET is not set
CONFIG_PHYLIB=m
#
# MII PHY device drivers
#
CONFIG_MARVELL_PHY=m
CONFIG_DAVICOM_PHY=m
CONFIG_QSEMI_PHY=m
CONFIG_LXT_PHY=m
CONFIG_CICADA_PHY=m
CONFIG_VITESSE_PHY=m
CONFIG_SMSC_PHY=m
CONFIG_BROADCOM_PHY=m
CONFIG_ICPLUS_PHY=m
CONFIG_REALTEK_PHY=m
CONFIG_NATIONAL_PHY=m
CONFIG_STE10XP=m
CONFIG_LSI_ET1011C_PHY=m
# CONFIG_MDIO_BITBANG is not set
CONFIG_NET_ETHERNET=y
CONFIG_MII=m
CONFIG_HAPPYMEAL=m
CONFIG_SUNGEM=m
CONFIG_CASSINI=m
CONFIG_NET_VENDOR_3COM=y
CONFIG_VORTEX=m
CONFIG_TYPHOON=m
CONFIG_ETHOC=m
CONFIG_DNET=m
CONFIG_NET_TULIP=y
CONFIG_DE2104X=m
CONFIG_DE2104X_DSL=0
CONFIG_TULIP=m
# CONFIG_TULIP_MWI is not set
CONFIG_TULIP_MMIO=y
# CONFIG_TULIP_NAPI is not set
CONFIG_DE4X5=m
CONFIG_WINBOND_840=m
CONFIG_DM9102=m
CONFIG_ULI526X=m
CONFIG_PCMCIA_XIRCOM=m
# CONFIG_HP100 is not set
# CONFIG_IBM_NEW_EMAC_ZMII is not set
# CONFIG_IBM_NEW_EMAC_RGMII is not set
# CONFIG_IBM_NEW_EMAC_TAH is not set
# CONFIG_IBM_NEW_EMAC_EMAC4 is not set
# CONFIG_IBM_NEW_EMAC_NO_FLOW_CTRL is not set
# CONFIG_IBM_NEW_EMAC_MAL_CLR_ICINTSTAT is not set
# CONFIG_IBM_NEW_EMAC_MAL_COMMON_ERR is not set
CONFIG_NET_PCI=y
CONFIG_PCNET32=m
CONFIG_AMD8111_ETH=m
CONFIG_ADAPTEC_STARFIRE=m
CONFIG_B44=m
CONFIG_B44_PCI_AUTOSELECT=y
CONFIG_B44_PCICORE_AUTOSELECT=y
CONFIG_B44_PCI=y
CONFIG_FORCEDETH=m
# CONFIG_FORCEDETH_NAPI is not set
CONFIG_E100=m
CONFIG_FEALNX=m
CONFIG_NATSEMI=m
CONFIG_NE2K_PCI=m
CONFIG_8139CP=m
CONFIG_8139TOO=m
# CONFIG_8139TOO_PIO is not set
# CONFIG_8139TOO_TUNE_TWISTER is not set
CONFIG_8139TOO_8129=y
# CONFIG_8139_OLD_RX_RESET is not set
# CONFIG_R6040 is not set
CONFIG_SIS900=m
CONFIG_EPIC100=m
CONFIG_SMSC9420=m
CONFIG_SUNDANCE=m
# CONFIG_SUNDANCE_MMIO is not set
CONFIG_TLAN=m
CONFIG_KS8842=m
CONFIG_KS8851_MLL=m
CONFIG_VIA_RHINE=m
CONFIG_VIA_RHINE_MMIO=y
# CONFIG_SC92031 is not set
CONFIG_NET_POCKET=y
# CONFIG_ATP is not set
# CONFIG_DE600 is not set
# CONFIG_DE620 is not set
CONFIG_ATL2=m
CONFIG_NETDEV_1000=y
CONFIG_ACENIC=m
# CONFIG_ACENIC_OMIT_TIGON_I is not set
CONFIG_DL2K=m
CONFIG_E1000=m
CONFIG_E1000E=m
CONFIG_IP1000=m
CONFIG_IGB=m
CONFIG_IGBVF=m
CONFIG_NS83820=m
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
CONFIG_R8169=m
CONFIG_R8169_VLAN=y
CONFIG_SIS190=m
CONFIG_SKGE=m
# CONFIG_SKGE_DEBUG is not set
CONFIG_SKY2=m
# CONFIG_SKY2_DEBUG is not set
CONFIG_VIA_VELOCITY=m
CONFIG_TIGON3=m
CONFIG_BNX2=m
CONFIG_CNIC=m
CONFIG_QLA3XXX=m
# CONFIG_ATL1 is not set
# CONFIG_ATL1E is not set
# CONFIG_ATL1C is not set
CONFIG_JME=m
CONFIG_NETDEV_10000=y
CONFIG_MDIO=m
CONFIG_CHELSIO_T1=m
# CONFIG_CHELSIO_T1_1G is not set
CONFIG_CHELSIO_T3_DEPENDS=y
CONFIG_CHELSIO_T3=m
CONFIG_ENIC=m
CONFIG_IXGBE=m
CONFIG_IXGB=m
CONFIG_S2IO=m
CONFIG_VXGE=m
# CONFIG_VXGE_DEBUG_TRACE_ALL is not set
CONFIG_MYRI10GE=m
CONFIG_NETXEN_NIC=m
CONFIG_NIU=m
CONFIG_MLX4_EN=m
CONFIG_MLX4_CORE=m
CONFIG_MLX4_DEBUG=y
CONFIG_TEHUTI=m
CONFIG_BNX2X=m
CONFIG_QLGE=m
# CONFIG_SFC is not set
# CONFIG_BE2NET is not set
CONFIG_TR=y
CONFIG_IBMOL=m
CONFIG_3C359=m
# CONFIG_TMS380TR is not set
CONFIG_WLAN=y
# CONFIG_WLAN_PRE80211 is not set
# CONFIG_WLAN_80211 is not set
#
# Enable WiMAX (Networking options) to see the WiMAX drivers
#
#
# USB Network Adapters
#
CONFIG_USB_CATC=m
CONFIG_USB_KAWETH=m
CONFIG_USB_PEGASUS=m
CONFIG_USB_RTL8150=m
CONFIG_USB_USBNET=m
CONFIG_USB_NET_AX8817X=m
CONFIG_USB_NET_CDCETHER=m
# CONFIG_USB_NET_CDC_EEM is not set
CONFIG_USB_NET_DM9601=m
# CONFIG_USB_NET_SMSC95XX is not set
CONFIG_USB_NET_GL620A=m
CONFIG_USB_NET_NET1080=m
CONFIG_USB_NET_PLUSB=m
# CONFIG_USB_NET_MCS7830 is not set
CONFIG_USB_NET_RNDIS_HOST=m
CONFIG_USB_NET_CDC_SUBSET=m
CONFIG_USB_ALI_M5632=y
CONFIG_USB_AN2720=y
CONFIG_USB_BELKIN=y
CONFIG_USB_ARMLINUX=y
CONFIG_USB_EPSON2888=y
# CONFIG_USB_KC2190 is not set
CONFIG_USB_NET_ZAURUS=m
# CONFIG_USB_HSO is not set
# CONFIG_USB_NET_INT51X1 is not set
CONFIG_NET_PCMCIA=y
CONFIG_PCMCIA_3C589=m
CONFIG_PCMCIA_3C574=m
CONFIG_PCMCIA_FMVJ18X=m
CONFIG_PCMCIA_PCNET=m
CONFIG_PCMCIA_NMCLAN=m
CONFIG_PCMCIA_SMC91C92=m
CONFIG_PCMCIA_XIRC2PS=m
CONFIG_PCMCIA_AXNET=m
CONFIG_PCMCIA_IBMTR=m
# CONFIG_WAN is not set
# CONFIG_ATM_DRIVERS is not set
CONFIG_XEN_NETDEV_FRONTEND=y
CONFIG_FDDI=y
# CONFIG_DEFXX is not set
# CONFIG_SKFP is not set
# CONFIG_HIPPI is not set
# CONFIG_PLIP is not set
CONFIG_PPP=m
CONFIG_PPP_MULTILINK=y
CONFIG_PPP_FILTER=y
CONFIG_PPP_ASYNC=m
CONFIG_PPP_SYNC_TTY=m
CONFIG_PPP_DEFLATE=m
# CONFIG_PPP_BSDCOMP is not set
CONFIG_PPP_MPPE=m
CONFIG_PPPOE=m
CONFIG_PPPOATM=m
# CONFIG_PPPOL2TP is not set
CONFIG_SLIP=m
CONFIG_SLIP_COMPRESSED=y
CONFIG_SLHC=m
CONFIG_SLIP_SMART=y
# CONFIG_SLIP_MODE_SLIP6 is not set
CONFIG_NET_FC=y
CONFIG_NETCONSOLE=m
# CONFIG_NETCONSOLE_DYNAMIC is not set
CONFIG_NETPOLL=y
CONFIG_NETPOLL_TRAP=y
CONFIG_NET_POLL_CONTROLLER=y
# CONFIG_VMXNET3 is not set
# CONFIG_ISDN is not set
# CONFIG_PHONE is not set
#
# Input device support
#
CONFIG_INPUT=y
CONFIG_INPUT_FF_MEMLESS=y
CONFIG_INPUT_POLLDEV=m
#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
# CONFIG_INPUT_MOUSEDEV_PSAUX is not set
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
# CONFIG_INPUT_JOYDEV is not set
CONFIG_INPUT_EVDEV=y
# CONFIG_INPUT_EVBUG is not set
CONFIG_XEN_KBDDEV_FRONTEND=y
#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
# CONFIG_KEYBOARD_ADP5588 is not set
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_QT2160 is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_LM8323 is not set
# CONFIG_KEYBOARD_MAX7359 is not set
# CONFIG_KEYBOARD_NEWTON is not set
# CONFIG_KEYBOARD_OPENCORES is not set
# CONFIG_KEYBOARD_STOWAWAY is not set
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=y
CONFIG_MOUSE_PS2_ALPS=y
CONFIG_MOUSE_PS2_LOGIPS2PP=y
CONFIG_MOUSE_PS2_SYNAPTICS=y
CONFIG_MOUSE_PS2_LIFEBOOK=y
CONFIG_MOUSE_PS2_TRACKPOINT=y
# CONFIG_MOUSE_PS2_ELANTECH is not set
# CONFIG_MOUSE_PS2_SENTELIC is not set
# CONFIG_MOUSE_PS2_TOUCHKIT is not set
CONFIG_MOUSE_SERIAL=m
# CONFIG_MOUSE_APPLETOUCH is not set
# CONFIG_MOUSE_BCM5974 is not set
CONFIG_MOUSE_VSXXXAA=m
# CONFIG_MOUSE_SYNAPTICS_I2C is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TABLET is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
CONFIG_INPUT_MISC=y
CONFIG_INPUT_PCSPKR=m
# CONFIG_INPUT_APANEL is not set
# CONFIG_INPUT_ATLAS_BTNS is not set
# CONFIG_INPUT_ATI_REMOTE is not set
# CONFIG_INPUT_ATI_REMOTE2 is not set
# CONFIG_INPUT_KEYSPAN_REMOTE is not set
# CONFIG_INPUT_POWERMATE is not set
# CONFIG_INPUT_YEALINK is not set
# CONFIG_INPUT_CM109 is not set
CONFIG_INPUT_UINPUT=m
# CONFIG_INPUT_WINBOND_CIR is not set
#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_SERIO_SERPORT=y
# CONFIG_SERIO_CT82C710 is not set
# CONFIG_SERIO_PARKBD is not set
# CONFIG_SERIO_PCIPS2 is not set
CONFIG_SERIO_LIBPS2=y
CONFIG_SERIO_RAW=m
CONFIG_GAMEPORT=m
CONFIG_GAMEPORT_NS558=m
CONFIG_GAMEPORT_L4=m
CONFIG_GAMEPORT_EMU10K1=m
CONFIG_GAMEPORT_FM801=m
#
# Character devices
#
CONFIG_VT=y
CONFIG_CONSOLE_TRANSLATIONS=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
CONFIG_VT_HW_CONSOLE_BINDING=y
# CONFIG_DEVKMEM is not set
CONFIG_SERIAL_NONSTANDARD=y
# CONFIG_COMPUTONE is not set
# CONFIG_ROCKETPORT is not set
CONFIG_CYCLADES=m
# CONFIG_CYZ_INTR is not set
# CONFIG_DIGIEPCA is not set
# CONFIG_MOXA_INTELLIO is not set
# CONFIG_MOXA_SMARTIO is not set
# CONFIG_ISI is not set
CONFIG_SYNCLINK=m
CONFIG_SYNCLINKMP=m
CONFIG_SYNCLINK_GT=m
CONFIG_N_HDLC=m
# CONFIG_RISCOM8 is not set
# CONFIG_SPECIALIX is not set
# CONFIG_STALDRV is not set
# CONFIG_NOZOMI is not set
#
# Serial drivers
#
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_FIX_EARLYCON_MEM=y
CONFIG_SERIAL_8250_PCI=y
CONFIG_SERIAL_8250_PNP=y
CONFIG_SERIAL_8250_CS=m
CONFIG_SERIAL_8250_NR_UARTS=32
CONFIG_SERIAL_8250_RUNTIME_UARTS=4
CONFIG_SERIAL_8250_EXTENDED=y
CONFIG_SERIAL_8250_MANY_PORTS=y
CONFIG_SERIAL_8250_SHARE_IRQ=y
CONFIG_SERIAL_8250_DETECT_IRQ=y
CONFIG_SERIAL_8250_RSA=y
#
# Non-8250 serial port support
#
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
CONFIG_SERIAL_JSM=m
CONFIG_UNIX98_PTYS=y
# CONFIG_DEVPTS_MULTIPLE_INSTANCES is not set
# CONFIG_LEGACY_PTYS is not set
CONFIG_PRINTER=m
CONFIG_LP_CONSOLE=y
CONFIG_PPDEV=m
CONFIG_HVC_DRIVER=y
CONFIG_HVC_IRQ=y
CONFIG_HVC_XEN=y
CONFIG_IPMI_HANDLER=m
CONFIG_IPMI_PANIC_EVENT=y
CONFIG_IPMI_PANIC_STRING=y
CONFIG_IPMI_DEVICE_INTERFACE=m
CONFIG_IPMI_SI=m
CONFIG_IPMI_WATCHDOG=m
CONFIG_IPMI_POWEROFF=m
CONFIG_HW_RANDOM=y
CONFIG_HW_RANDOM_TIMERIOMEM=m
CONFIG_HW_RANDOM_INTEL=m
CONFIG_HW_RANDOM_AMD=m
CONFIG_HW_RANDOM_VIA=m
CONFIG_NVRAM=y
CONFIG_RTC=y
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
#
# PCMCIA character devices
#
# CONFIG_SYNCLINK_CS is not set
CONFIG_CARDMAN_4000=m
CONFIG_CARDMAN_4040=m
# CONFIG_IPWIRELESS is not set
# CONFIG_MWAVE is not set
CONFIG_PC8736x_GPIO=m
CONFIG_NSC_GPIO=m
CONFIG_RAW_DRIVER=y
CONFIG_MAX_RAW_DEVS=8192
CONFIG_HPET=y
# CONFIG_HPET_MMAP is not set
CONFIG_HANGCHECK_TIMER=m
CONFIG_TCG_TPM=m
CONFIG_TCG_TIS=m
CONFIG_TCG_NSC=m
CONFIG_TCG_ATMEL=m
CONFIG_TCG_INFINEON=m
CONFIG_TELCLOCK=m
CONFIG_DEVPORT=y
CONFIG_I2C=m
CONFIG_I2C_BOARDINFO=y
CONFIG_I2C_COMPAT=y
CONFIG_I2C_CHARDEV=m
CONFIG_I2C_HELPER_AUTO=y
CONFIG_I2C_ALGOBIT=m
#
# I2C Hardware Bus support
#
#
# PC SMBus host controller drivers
#
# CONFIG_I2C_ALI1535 is not set
# CONFIG_I2C_ALI1563 is not set
# CONFIG_I2C_ALI15X3 is not set
CONFIG_I2C_AMD756=m
CONFIG_I2C_AMD756_S4882=m
CONFIG_I2C_AMD8111=m
CONFIG_I2C_I801=m
CONFIG_I2C_ISCH=m
CONFIG_I2C_PIIX4=m
CONFIG_I2C_NFORCE2=m
CONFIG_I2C_NFORCE2_S4985=m
# CONFIG_I2C_SIS5595 is not set
# CONFIG_I2C_SIS630 is not set
CONFIG_I2C_SIS96X=m
CONFIG_I2C_VIA=m
CONFIG_I2C_VIAPRO=m
#
# ACPI drivers
#
# CONFIG_I2C_SCMI is not set
#
# I2C system bus drivers (mostly embedded / system-on-chip)
#
# CONFIG_I2C_OCORES is not set
# CONFIG_I2C_SIMTEC is not set
#
# External I2C/SMBus adapter drivers
#
CONFIG_I2C_PARPORT=m
CONFIG_I2C_PARPORT_LIGHT=m
# CONFIG_I2C_TAOS_EVM is not set
# CONFIG_I2C_TINY_USB is not set
#
# Graphics adapter I2C/DDC channel drivers
#
CONFIG_I2C_VOODOO3=m
#
# Other I2C/SMBus bus drivers
#
# CONFIG_I2C_PCA_PLATFORM is not set
CONFIG_I2C_STUB=m
#
# Miscellaneous I2C Chip support
#
# CONFIG_DS1682 is not set
# CONFIG_SENSORS_TSL2550 is not set
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# CONFIG_I2C_DEBUG_CHIP is not set
# CONFIG_SPI is not set
#
# PPS support
#
# CONFIG_PPS is not set
CONFIG_ARCH_WANT_OPTIONAL_GPIOLIB=y
# CONFIG_GPIOLIB is not set
# CONFIG_W1 is not set
# CONFIG_POWER_SUPPLY is not set
CONFIG_HWMON=m
CONFIG_HWMON_VID=m
# CONFIG_HWMON_DEBUG_CHIP is not set
#
# Native drivers
#
CONFIG_SENSORS_ABITUGURU=m
CONFIG_SENSORS_ABITUGURU3=m
CONFIG_SENSORS_AD7414=m
CONFIG_SENSORS_AD7418=m
CONFIG_SENSORS_ADM1021=m
CONFIG_SENSORS_ADM1025=m
CONFIG_SENSORS_ADM1026=m
CONFIG_SENSORS_ADM1029=m
CONFIG_SENSORS_ADM1031=m
CONFIG_SENSORS_ADM9240=m
CONFIG_SENSORS_ADT7462=m
CONFIG_SENSORS_ADT7470=m
CONFIG_SENSORS_ADT7473=m
CONFIG_SENSORS_ADT7475=m
CONFIG_SENSORS_K8TEMP=m
CONFIG_SENSORS_ASB100=m
CONFIG_SENSORS_ATXP1=m
CONFIG_SENSORS_DS1621=m
CONFIG_SENSORS_I5K_AMB=m
CONFIG_SENSORS_F71805F=m
CONFIG_SENSORS_F71882FG=m
CONFIG_SENSORS_F75375S=m
CONFIG_SENSORS_FSCHMD=m
CONFIG_SENSORS_G760A=m
CONFIG_SENSORS_GL518SM=m
CONFIG_SENSORS_GL520SM=m
CONFIG_SENSORS_CORETEMP=m
CONFIG_SENSORS_IBMAEM=m
CONFIG_SENSORS_IBMPEX=m
CONFIG_SENSORS_IT87=m
CONFIG_SENSORS_LM63=m
CONFIG_SENSORS_LM75=m
CONFIG_SENSORS_LM77=m
CONFIG_SENSORS_LM78=m
CONFIG_SENSORS_LM80=m
CONFIG_SENSORS_LM83=m
CONFIG_SENSORS_LM85=m
CONFIG_SENSORS_LM87=m
CONFIG_SENSORS_LM90=m
CONFIG_SENSORS_LM92=m
CONFIG_SENSORS_LM93=m
CONFIG_SENSORS_LTC4215=m
CONFIG_SENSORS_LTC4245=m
CONFIG_SENSORS_LM95241=m
CONFIG_SENSORS_MAX1619=m
CONFIG_SENSORS_MAX6650=m
CONFIG_SENSORS_PC87360=m
CONFIG_SENSORS_PC87427=m
CONFIG_SENSORS_PCF8591=m
CONFIG_SENSORS_SIS5595=m
CONFIG_SENSORS_DME1737=m
CONFIG_SENSORS_SMSC47M1=m
CONFIG_SENSORS_SMSC47M192=m
CONFIG_SENSORS_SMSC47B397=m
CONFIG_SENSORS_ADS7828=m
CONFIG_SENSORS_THMC50=m
CONFIG_SENSORS_TMP401=m
CONFIG_SENSORS_TMP421=m
CONFIG_SENSORS_VIA686A=m
CONFIG_SENSORS_VT1211=m
CONFIG_SENSORS_VT8231=m
CONFIG_SENSORS_W83781D=m
CONFIG_SENSORS_W83791D=m
CONFIG_SENSORS_W83792D=m
CONFIG_SENSORS_W83793=m
CONFIG_SENSORS_W83L785TS=m
CONFIG_SENSORS_W83L786NG=m
CONFIG_SENSORS_W83627HF=m
CONFIG_SENSORS_W83627EHF=m
CONFIG_SENSORS_HDAPS=m
CONFIG_SENSORS_APPLESMC=m
#
# ACPI drivers
#
CONFIG_SENSORS_ATK0110=m
CONFIG_SENSORS_LIS3LV02D=m
CONFIG_THERMAL=y
CONFIG_WATCHDOG=y
# CONFIG_WATCHDOG_NOWAYOUT is not set
#
# Watchdog Device Drivers
#
CONFIG_SOFT_WATCHDOG=m
# CONFIG_ACQUIRE_WDT is not set
# CONFIG_ADVANTECH_WDT is not set
CONFIG_ALIM1535_WDT=m
CONFIG_ALIM7101_WDT=m
# CONFIG_SC520_WDT is not set
# CONFIG_SBC_FITPC2_WATCHDOG is not set
# CONFIG_EUROTECH_WDT is not set
# CONFIG_IB700_WDT is not set
CONFIG_IBMASR=m
# CONFIG_WAFER_WDT is not set
CONFIG_I6300ESB_WDT=m
CONFIG_ITCO_WDT=m
# CONFIG_ITCO_VENDOR_SUPPORT is not set
CONFIG_IT8712F_WDT=m
CONFIG_IT87_WDT=m
# CONFIG_HP_WATCHDOG is not set
# CONFIG_SC1200_WDT is not set
# CONFIG_PC87413_WDT is not set
# CONFIG_60XX_WDT is not set
# CONFIG_SBC8360_WDT is not set
# CONFIG_CPU5_WDT is not set
CONFIG_SMSC_SCH311X_WDT=m
CONFIG_SMSC37B787_WDT=m
CONFIG_W83627HF_WDT=m
CONFIG_W83697HF_WDT=m
CONFIG_W83697UG_WDT=m
CONFIG_W83877F_WDT=m
CONFIG_W83977F_WDT=m
CONFIG_MACHZ_WDT=m
# CONFIG_SBC_EPX_C3_WATCHDOG is not set
# CONFIG_XEN_WDT is not set
#
# PCI-based Watchdog Cards
#
CONFIG_PCIPCWATCHDOG=m
CONFIG_WDTPCI=m
#
# USB-based Watchdog Cards
#
CONFIG_USBPCWATCHDOG=m
CONFIG_SSB_POSSIBLE=y
#
# Sonics Silicon Backplane
#
CONFIG_SSB=m
CONFIG_SSB_SPROM=y
CONFIG_SSB_PCIHOST_POSSIBLE=y
CONFIG_SSB_PCIHOST=y
# CONFIG_SSB_B43_PCI_BRIDGE is not set
CONFIG_SSB_PCMCIAHOST_POSSIBLE=y
# CONFIG_SSB_PCMCIAHOST is not set
# CONFIG_SSB_DEBUG is not set
CONFIG_SSB_DRIVER_PCICORE_POSSIBLE=y
CONFIG_SSB_DRIVER_PCICORE=y
#
# Multifunction device drivers
#
# CONFIG_MFD_CORE is not set
# CONFIG_MFD_SM501 is not set
# CONFIG_HTC_PASIC3 is not set
# CONFIG_MFD_TMIO is not set
# CONFIG_MFD_WM8400 is not set
# CONFIG_MFD_WM831X is not set
# CONFIG_MFD_WM8350_I2C is not set
# CONFIG_MFD_PCF50633 is not set
# CONFIG_AB3100_CORE is not set
# CONFIG_REGULATOR is not set
# CONFIG_MEDIA_SUPPORT is not set
#
# Graphics support
#
CONFIG_AGP=y
CONFIG_AGP_AMD64=y
CONFIG_AGP_INTEL=y
CONFIG_AGP_SIS=y
CONFIG_AGP_VIA=y
CONFIG_VGA_ARB=y
CONFIG_DRM=m
CONFIG_DRM_KMS_HELPER=m
CONFIG_DRM_TTM=m
# CONFIG_DRM_TDFX is not set
CONFIG_DRM_R128=m
CONFIG_DRM_RADEON=m
CONFIG_DRM_I810=m
CONFIG_DRM_I830=m
CONFIG_DRM_I915=m
# CONFIG_DRM_I915_KMS is not set
CONFIG_DRM_MGA=m
# CONFIG_DRM_SIS is not set
CONFIG_DRM_VIA=m
CONFIG_DRM_SAVAGE=m
CONFIG_VGASTATE=m
CONFIG_VIDEO_OUTPUT_CONTROL=m
CONFIG_FB=y
# CONFIG_FIRMWARE_EDID is not set
CONFIG_FB_DDC=m
CONFIG_FB_BOOT_VESA_SUPPORT=y
CONFIG_FB_CFB_FILLRECT=y
CONFIG_FB_CFB_COPYAREA=y
CONFIG_FB_CFB_IMAGEBLIT=y
# CONFIG_FB_CFB_REV_PIXELS_IN_BYTE is not set
CONFIG_FB_SYS_FILLRECT=y
CONFIG_FB_SYS_COPYAREA=y
CONFIG_FB_SYS_IMAGEBLIT=y
# CONFIG_FB_FOREIGN_ENDIAN is not set
CONFIG_FB_SYS_FOPS=y
CONFIG_FB_DEFERRED_IO=y
CONFIG_FB_SVGALIB=m
# CONFIG_FB_MACMODES is not set
# CONFIG_FB_BACKLIGHT is not set
CONFIG_FB_MODE_HELPERS=y
CONFIG_FB_TILEBLITTING=y
#
# Frame buffer hardware drivers
#
CONFIG_FB_CIRRUS=m
# CONFIG_FB_PM2 is not set
# CONFIG_FB_CYBER2000 is not set
# CONFIG_FB_ARC is not set
# CONFIG_FB_ASILIANT is not set
# CONFIG_FB_IMSTT is not set
CONFIG_FB_VGA16=m
# CONFIG_FB_UVESA is not set
CONFIG_FB_VESA=y
# CONFIG_FB_N411 is not set
# CONFIG_FB_HGA is not set
# CONFIG_FB_S1D13XXX is not set
CONFIG_FB_NVIDIA=m
CONFIG_FB_NVIDIA_I2C=y
# CONFIG_FB_NVIDIA_DEBUG is not set
# CONFIG_FB_NVIDIA_BACKLIGHT is not set
CONFIG_FB_RIVA=m
# CONFIG_FB_RIVA_I2C is not set
# CONFIG_FB_RIVA_DEBUG is not set
# CONFIG_FB_RIVA_BACKLIGHT is not set
CONFIG_FB_LE80578=m
CONFIG_FB_CARILLO_RANCH=m
# CONFIG_FB_MATROX is not set
# CONFIG_FB_RADEON is not set
# CONFIG_FB_ATY128 is not set
# CONFIG_FB_ATY is not set
# CONFIG_FB_S3 is not set
CONFIG_FB_SAVAGE=m
CONFIG_FB_SAVAGE_I2C=y
CONFIG_FB_SAVAGE_ACCEL=y
# CONFIG_FB_SIS is not set
# CONFIG_FB_VIA is not set
# CONFIG_FB_NEOMAGIC is not set
CONFIG_FB_KYRO=m
# CONFIG_FB_3DFX is not set
# CONFIG_FB_VOODOO1 is not set
CONFIG_FB_VT8623=m
# CONFIG_FB_TRIDENT is not set
# CONFIG_FB_ARK is not set
# CONFIG_FB_PM3 is not set
CONFIG_FB_CARMINE=m
CONFIG_FB_CARMINE_DRAM_EVAL=y
# CONFIG_CARMINE_DRAM_CUSTOM is not set
# CONFIG_FB_GEODE is not set
# CONFIG_FB_VIRTUAL is not set
CONFIG_XEN_FBDEV_FRONTEND=y
# CONFIG_FB_METRONOME is not set
# CONFIG_FB_MB862XX is not set
# CONFIG_FB_BROADSHEET is not set
CONFIG_BACKLIGHT_LCD_SUPPORT=y
CONFIG_LCD_CLASS_DEVICE=m
# CONFIG_LCD_ILI9320 is not set
# CONFIG_LCD_PLATFORM is not set
CONFIG_BACKLIGHT_CLASS_DEVICE=m
# CONFIG_BACKLIGHT_GENERIC is not set
# CONFIG_BACKLIGHT_PROGEAR is not set
# CONFIG_BACKLIGHT_CARILLO_RANCH is not set
# CONFIG_BACKLIGHT_MBP_NVIDIA is not set
# CONFIG_BACKLIGHT_SAHARA is not set
#
# Display device support
#
CONFIG_DISPLAY_SUPPORT=m
#
# Display hardware drivers
#
#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
CONFIG_VGACON_SOFT_SCROLLBACK=y
CONFIG_VGACON_SOFT_SCROLLBACK_SIZE=64
CONFIG_DUMMY_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE=y
# CONFIG_FRAMEBUFFER_CONSOLE_DETECT_PRIMARY is not set
CONFIG_FRAMEBUFFER_CONSOLE_ROTATION=y
# CONFIG_FONTS is not set
CONFIG_FONT_8x8=y
CONFIG_FONT_8x16=y
CONFIG_LOGO=y
# CONFIG_LOGO_LINUX_MONO is not set
# CONFIG_LOGO_LINUX_VGA16 is not set
CONFIG_LOGO_LINUX_CLUT224=y
# CONFIG_SOUND is not set
CONFIG_HID_SUPPORT=y
CONFIG_HID=y
# CONFIG_HIDRAW is not set
#
# USB Input Devices
#
CONFIG_USB_HID=y
CONFIG_HID_PID=y
CONFIG_USB_HIDDEV=y
#
# Special HID drivers
#
CONFIG_HID_A4TECH=y
CONFIG_HID_APPLE=y
CONFIG_HID_BELKIN=y
CONFIG_HID_CHERRY=y
CONFIG_HID_CHICONY=y
CONFIG_HID_CYPRESS=y
CONFIG_HID_DRAGONRISE=y
# CONFIG_DRAGONRISE_FF is not set
CONFIG_HID_EZKEY=y
CONFIG_HID_KYE=y
CONFIG_HID_GYRATION=y
CONFIG_HID_TWINHAN=y
CONFIG_HID_KENSINGTON=y
CONFIG_HID_LOGITECH=y
CONFIG_LOGITECH_FF=y
# CONFIG_LOGIRUMBLEPAD2_FF is not set
CONFIG_HID_MICROSOFT=y
CONFIG_HID_MONTEREY=y
CONFIG_HID_NTRIG=y
CONFIG_HID_PANTHERLORD=y
# CONFIG_PANTHERLORD_FF is not set
CONFIG_HID_PETALYNX=y
CONFIG_HID_SAMSUNG=y
CONFIG_HID_SONY=y
CONFIG_HID_SUNPLUS=y
CONFIG_HID_GREENASIA=y
# CONFIG_GREENASIA_FF is not set
CONFIG_HID_SMARTJOYPLUS=y
# CONFIG_SMARTJOYPLUS_FF is not set
CONFIG_HID_TOPSEED=y
CONFIG_HID_THRUSTMASTER=y
CONFIG_THRUSTMASTER_FF=y
CONFIG_HID_WACOM=m
CONFIG_HID_ZEROPLUS=y
# CONFIG_ZEROPLUS_FF is not set
CONFIG_USB_SUPPORT=y
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB_ARCH_HAS_OHCI=y
CONFIG_USB_ARCH_HAS_EHCI=y
CONFIG_USB=y
# CONFIG_USB_DEBUG is not set
# CONFIG_USB_ANNOUNCE_NEW_DEVICES is not set
#
# Miscellaneous USB options
#
CONFIG_USB_DEVICEFS=y
CONFIG_USB_DEVICE_CLASS=y
# CONFIG_USB_DYNAMIC_MINORS is not set
# CONFIG_USB_SUSPEND is not set
# CONFIG_USB_OTG is not set
CONFIG_USB_MON=y
# CONFIG_USB_WUSB is not set
# CONFIG_USB_WUSB_CBAF is not set
#
# USB Host Controller Drivers
#
# CONFIG_USB_C67X00_HCD is not set
# CONFIG_USB_XHCI_HCD is not set
CONFIG_USB_EHCI_HCD=m
CONFIG_USB_EHCI_ROOT_HUB_TT=y
CONFIG_USB_EHCI_TT_NEWSCHED=y
# CONFIG_USB_OXU210HP_HCD is not set
CONFIG_USB_ISP116X_HCD=m
# CONFIG_USB_ISP1760_HCD is not set
# CONFIG_USB_ISP1362_HCD is not set
CONFIG_USB_OHCI_HCD=m
# CONFIG_USB_OHCI_HCD_SSB is not set
# CONFIG_USB_OHCI_BIG_ENDIAN_DESC is not set
# CONFIG_USB_OHCI_BIG_ENDIAN_MMIO is not set
CONFIG_USB_OHCI_LITTLE_ENDIAN=y
CONFIG_USB_UHCI_HCD=m
CONFIG_USB_SL811_HCD=m
CONFIG_USB_SL811_CS=m
# CONFIG_USB_R8A66597_HCD is not set
# CONFIG_USB_WHCI_HCD is not set
# CONFIG_USB_HWA_HCD is not set
#
# USB Device Class drivers
#
CONFIG_USB_ACM=m
CONFIG_USB_PRINTER=m
# CONFIG_USB_WDM is not set
# CONFIG_USB_TMC is not set
#
# NOTE: USB_STORAGE depends on SCSI but BLK_DEV_SD may
#
#
# also be needed; see USB_STORAGE Help for more info
#
CONFIG_USB_STORAGE=m
# CONFIG_USB_STORAGE_DEBUG is not set
CONFIG_USB_STORAGE_DATAFAB=m
CONFIG_USB_STORAGE_FREECOM=m
CONFIG_USB_STORAGE_ISD200=m
CONFIG_USB_STORAGE_USBAT=m
CONFIG_USB_STORAGE_SDDR09=m
CONFIG_USB_STORAGE_SDDR55=m
CONFIG_USB_STORAGE_JUMPSHOT=m
CONFIG_USB_STORAGE_ALAUDA=m
# CONFIG_USB_STORAGE_ONETOUCH is not set
# CONFIG_USB_STORAGE_KARMA is not set
# CONFIG_USB_STORAGE_CYPRESS_ATACB is not set
# CONFIG_USB_LIBUSUAL is not set
#
# USB Imaging devices
#
CONFIG_USB_MDC800=m
CONFIG_USB_MICROTEK=m
#
# USB port drivers
#
CONFIG_USB_USS720=m
CONFIG_USB_SERIAL=m
CONFIG_USB_EZUSB=y
CONFIG_USB_SERIAL_GENERIC=y
# CONFIG_USB_SERIAL_AIRCABLE is not set
CONFIG_USB_SERIAL_ARK3116=m
CONFIG_USB_SERIAL_BELKIN=m
# CONFIG_USB_SERIAL_CH341 is not set
CONFIG_USB_SERIAL_WHITEHEAT=m
CONFIG_USB_SERIAL_DIGI_ACCELEPORT=m
# CONFIG_USB_SERIAL_CP210X is not set
CONFIG_USB_SERIAL_CYPRESS_M8=m
CONFIG_USB_SERIAL_EMPEG=m
CONFIG_USB_SERIAL_FTDI_SIO=m
CONFIG_USB_SERIAL_FUNSOFT=m
CONFIG_USB_SERIAL_VISOR=m
CONFIG_USB_SERIAL_IPAQ=m
CONFIG_USB_SERIAL_IR=m
CONFIG_USB_SERIAL_EDGEPORT=m
CONFIG_USB_SERIAL_EDGEPORT_TI=m
CONFIG_USB_SERIAL_GARMIN=m
CONFIG_USB_SERIAL_IPW=m
# CONFIG_USB_SERIAL_IUU is not set
CONFIG_USB_SERIAL_KEYSPAN_PDA=m
CONFIG_USB_SERIAL_KEYSPAN=m
# CONFIG_USB_SERIAL_KEYSPAN_MPR is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA28 is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA28X is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA28XA is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA28XB is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA19 is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA18X is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA19W is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA19QW is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA19QI is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA49W is not set
# CONFIG_USB_SERIAL_KEYSPAN_USA49WLC is not set
CONFIG_USB_SERIAL_KLSI=m
CONFIG_USB_SERIAL_KOBIL_SCT=m
CONFIG_USB_SERIAL_MCT_U232=m
# CONFIG_USB_SERIAL_MOS7720 is not set
# CONFIG_USB_SERIAL_MOS7840 is not set
# CONFIG_USB_SERIAL_MOTOROLA is not set
CONFIG_USB_SERIAL_NAVMAN=m
CONFIG_USB_SERIAL_PL2303=m
# CONFIG_USB_SERIAL_OTI6858 is not set
# CONFIG_USB_SERIAL_QUALCOMM is not set
# CONFIG_USB_SERIAL_SPCP8X5 is not set
CONFIG_USB_SERIAL_HP4X=m
CONFIG_USB_SERIAL_SAFE=m
CONFIG_USB_SERIAL_SAFE_PADDED=y
# CONFIG_USB_SERIAL_SIEMENS_MPI is not set
CONFIG_USB_SERIAL_SIERRAWIRELESS=m
# CONFIG_USB_SERIAL_SYMBOL is not set
CONFIG_USB_SERIAL_TI=m
CONFIG_USB_SERIAL_CYBERJACK=m
CONFIG_USB_SERIAL_XIRCOM=m
CONFIG_USB_SERIAL_OPTION=m
CONFIG_USB_SERIAL_OMNINET=m
# CONFIG_USB_SERIAL_OPTICON is not set
# CONFIG_USB_SERIAL_DEBUG is not set
#
# USB Miscellaneous drivers
#
CONFIG_USB_EMI62=m
CONFIG_USB_EMI26=m
# CONFIG_USB_ADUTUX is not set
# CONFIG_USB_SEVSEG is not set
CONFIG_USB_RIO500=m
CONFIG_USB_LEGOTOWER=m
CONFIG_USB_LCD=m
# CONFIG_USB_BERRY_CHARGE is not set
CONFIG_USB_LED=m
# CONFIG_USB_CYPRESS_CY7C63 is not set
# CONFIG_USB_CYTHERM is not set
CONFIG_USB_IDMOUSE=m
# CONFIG_USB_FTDI_ELAN is not set
CONFIG_USB_APPLEDISPLAY=m
CONFIG_USB_SISUSBVGA=m
CONFIG_USB_SISUSBVGA_CON=y
CONFIG_USB_LD=m
# CONFIG_USB_TRANCEVIBRATOR is not set
# CONFIG_USB_IOWARRIOR is not set
CONFIG_USB_TEST=m
# CONFIG_USB_ISIGHTFW is not set
# CONFIG_USB_VST is not set
CONFIG_USB_ATM=m
CONFIG_USB_SPEEDTOUCH=m
CONFIG_USB_CXACRU=m
CONFIG_USB_UEAGLEATM=m
CONFIG_USB_XUSBATM=m
# CONFIG_USB_GADGET is not set
#
# OTG and related infrastructure
#
CONFIG_USB_OTG_UTILS=y
CONFIG_NOP_USB_XCEIV=m
# CONFIG_UWB is not set
# CONFIG_MMC is not set
# CONFIG_MEMSTICK is not set
CONFIG_NEW_LEDS=y
CONFIG_LEDS_CLASS=y
#
# LED drivers
#
# CONFIG_LEDS_ALIX2 is not set
CONFIG_LEDS_PCA9532=m
# CONFIG_LEDS_LP3944 is not set
# CONFIG_LEDS_CLEVO_MAIL is not set
CONFIG_LEDS_PCA955X=m
CONFIG_LEDS_BD2802=m
#
# LED Triggers
#
CONFIG_LEDS_TRIGGERS=y
CONFIG_LEDS_TRIGGER_TIMER=m
CONFIG_LEDS_TRIGGER_IDE_DISK=y
CONFIG_LEDS_TRIGGER_HEARTBEAT=m
# CONFIG_LEDS_TRIGGER_BACKLIGHT is not set
CONFIG_LEDS_TRIGGER_DEFAULT_ON=m
#
# iptables trigger is under Netfilter config (LED target)
#
# CONFIG_ACCESSIBILITY is not set
# CONFIG_INFINIBAND is not set
# CONFIG_EDAC is not set
# CONFIG_RTC_CLASS is not set
# CONFIG_DMADEVICES is not set
# CONFIG_AUXDISPLAY is not set
CONFIG_UIO=m
# CONFIG_UIO_CIF is not set
# CONFIG_UIO_PDRV is not set
# CONFIG_UIO_PDRV_GENIRQ is not set
# CONFIG_UIO_SMX is not set
# CONFIG_UIO_AEC is not set
# CONFIG_UIO_SERCOS3 is not set
# CONFIG_UIO_PCI_GENERIC is not set
#
# TI VLYNQ
#
CONFIG_XEN_BALLOON=y
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_XEN_DEV_EVTCHN=y
CONFIG_XEN_BACKEND=y
CONFIG_XEN_NETDEV_BACKEND=y
CONFIG_XEN_BLKDEV_BACKEND=y
CONFIG_XEN_BLKDEV_TAP=y
CONFIG_XEN_BLKBACK_PAGEMAP=y
CONFIG_XEN_PCIDEV_BACKEND=y
CONFIG_XEN_PCIDEV_BACKEND_VPCI=y
# CONFIG_XEN_PCIDEV_BACKEND_PASS is not set
# CONFIG_XEN_PCIDEV_BACKEND_SLOT is not set
# CONFIG_XEN_PCIDEV_BACKEND_CONTROLLER is not set
# CONFIG_XEN_PCIDEV_BE_DEBUG is not set
CONFIG_XENFS=y
CONFIG_XEN_COMPAT_XENFS=y
CONFIG_XEN_SYS_HYPERVISOR=y
CONFIG_XEN_MCE=y
CONFIG_XEN_XENBUS_FRONTEND=y
CONFIG_XEN_GNTDEV=y
CONFIG_XEN_S3=y
CONFIG_ACPI_PROCESSOR_XEN=y
CONFIG_XEN_PLATFORM_PCI=m
# CONFIG_STAGING is not set
# CONFIG_X86_PLATFORM_DEVICES is not set
#
# Firmware Drivers
#
CONFIG_EDD=m
CONFIG_EDD_OFF=y
CONFIG_FIRMWARE_MEMMAP=y
CONFIG_DELL_RBU=m
CONFIG_DCDBAS=m
# CONFIG_DMIID is not set
CONFIG_ISCSI_IBFT_FIND=y
CONFIG_ISCSI_IBFT=m
#
# File systems
#
CONFIG_EXT2_FS=y
CONFIG_EXT2_FS_XATTR=y
CONFIG_EXT2_FS_POSIX_ACL=y
CONFIG_EXT2_FS_SECURITY=y
CONFIG_EXT2_FS_XIP=y
CONFIG_EXT3_FS=m
CONFIG_EXT3_DEFAULTS_TO_ORDERED=y
CONFIG_EXT3_FS_XATTR=y
CONFIG_EXT3_FS_POSIX_ACL=y
CONFIG_EXT3_FS_SECURITY=y
CONFIG_EXT4_FS=m
CONFIG_EXT4_FS_XATTR=y
CONFIG_EXT4_FS_POSIX_ACL=y
CONFIG_EXT4_FS_SECURITY=y
# CONFIG_EXT4_DEBUG is not set
CONFIG_FS_XIP=y
CONFIG_JBD=m
# CONFIG_JBD_DEBUG is not set
CONFIG_JBD2=m
# CONFIG_JBD2_DEBUG is not set
CONFIG_FS_MBCACHE=y
# CONFIG_REISERFS_FS is not set
# CONFIG_JFS_FS is not set
CONFIG_FS_POSIX_ACL=y
# CONFIG_XFS_FS is not set
CONFIG_GFS2_FS=m
CONFIG_GFS2_FS_LOCKING_DLM=y
CONFIG_OCFS2_FS=m
CONFIG_OCFS2_FS_O2CB=m
CONFIG_OCFS2_FS_USERSPACE_CLUSTER=m
CONFIG_OCFS2_FS_STATS=y
CONFIG_OCFS2_DEBUG_MASKLOG=y
# CONFIG_OCFS2_DEBUG_FS is not set
# CONFIG_OCFS2_FS_POSIX_ACL is not set
# CONFIG_BTRFS_FS is not set
# CONFIG_NILFS2_FS is not set
CONFIG_FILE_LOCKING=y
CONFIG_FSNOTIFY=y
CONFIG_DNOTIFY=y
CONFIG_INOTIFY=y
CONFIG_INOTIFY_USER=y
CONFIG_QUOTA=y
CONFIG_QUOTA_NETLINK_INTERFACE=y
# CONFIG_PRINT_QUOTA_WARNING is not set
CONFIG_QUOTA_TREE=y
# CONFIG_QFMT_V1 is not set
CONFIG_QFMT_V2=y
CONFIG_QUOTACTL=y
# CONFIG_AUTOFS_FS is not set
CONFIG_AUTOFS4_FS=m
CONFIG_FUSE_FS=m
CONFIG_CUSE=m
#
# Caches
#
CONFIG_FSCACHE=m
CONFIG_FSCACHE_STATS=y
# CONFIG_FSCACHE_HISTOGRAM is not set
# CONFIG_FSCACHE_DEBUG is not set
# CONFIG_FSCACHE_OBJECT_LIST is not set
CONFIG_CACHEFILES=m
CONFIG_CACHEFILES_DEBUG=y
# CONFIG_CACHEFILES_HISTOGRAM is not set
#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_UDF_FS=m
CONFIG_UDF_NLS=y
#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=m
CONFIG_MSDOS_FS=m
CONFIG_VFAT_FS=m
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="ascii"
CONFIG_NTFS_FS=m
# CONFIG_NTFS_DEBUG is not set
CONFIG_NTFS_RW=y
#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_PROC_VMCORE=y
CONFIG_PROC_SYSCTL=y
CONFIG_PROC_PAGE_MONITOR=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
# CONFIG_TMPFS_POSIX_ACL is not set
CONFIG_HUGETLBFS=y
CONFIG_HUGETLB_PAGE=y
CONFIG_CONFIGFS_FS=m
CONFIG_MISC_FILESYSTEMS=y
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_ECRYPT_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_JFFS2_FS is not set
# CONFIG_CRAMFS is not set
# CONFIG_SQUASHFS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_MINIX_FS is not set
# CONFIG_OMFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_ROMFS_FS is not set
# CONFIG_SYSV_FS is not set
CONFIG_UFS_FS=m
CONFIG_UFS_FS_WRITE=y
# CONFIG_UFS_DEBUG is not set
# CONFIG_EXOFS_FS is not set
CONFIG_NETWORK_FILESYSTEMS=y
CONFIG_NFS_FS=m
CONFIG_NFS_V3=y
CONFIG_NFS_V3_ACL=y
CONFIG_NFS_V4=y
CONFIG_NFS_V4_1=y
# CONFIG_NFS_FSCACHE is not set
CONFIG_NFSD=m
CONFIG_NFSD_V2_ACL=y
CONFIG_NFSD_V3=y
CONFIG_NFSD_V3_ACL=y
CONFIG_NFSD_V4=y
CONFIG_LOCKD=m
CONFIG_LOCKD_V4=y
CONFIG_EXPORTFS=m
CONFIG_NFS_ACL_SUPPORT=m
CONFIG_NFS_COMMON=y
CONFIG_SUNRPC=m
CONFIG_SUNRPC_GSS=m
CONFIG_RPCSEC_GSS_KRB5=m
CONFIG_RPCSEC_GSS_SPKM3=m
# CONFIG_SMB_FS is not set
CONFIG_CIFS=m
# CONFIG_CIFS_STATS is not set
CONFIG_CIFS_WEAK_PW_HASH=y
# CONFIG_CIFS_UPCALL is not set
CONFIG_CIFS_XATTR=y
CONFIG_CIFS_POSIX=y
# CONFIG_CIFS_DEBUG2 is not set
# CONFIG_CIFS_DFS_UPCALL is not set
# CONFIG_CIFS_EXPERIMENTAL is not set
# CONFIG_NCP_FS is not set
# CONFIG_CODA_FS is not set
# CONFIG_AFS_FS is not set
#
# Partition Types
#
CONFIG_PARTITION_ADVANCED=y
# CONFIG_ACORN_PARTITION is not set
CONFIG_OSF_PARTITION=y
CONFIG_AMIGA_PARTITION=y
# CONFIG_ATARI_PARTITION is not set
CONFIG_MAC_PARTITION=y
CONFIG_MSDOS_PARTITION=y
CONFIG_BSD_DISKLABEL=y
CONFIG_MINIX_SUBPARTITION=y
CONFIG_SOLARIS_X86_PARTITION=y
CONFIG_UNIXWARE_DISKLABEL=y
# CONFIG_LDM_PARTITION is not set
CONFIG_SGI_PARTITION=y
# CONFIG_ULTRIX_PARTITION is not set
CONFIG_SUN_PARTITION=y
CONFIG_KARMA_PARTITION=y
CONFIG_EFI_PARTITION=y
# CONFIG_SYSV68_PARTITION is not set
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="utf8"
CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_CODEPAGE_737=m
CONFIG_NLS_CODEPAGE_775=m
CONFIG_NLS_CODEPAGE_850=m
CONFIG_NLS_CODEPAGE_852=m
CONFIG_NLS_CODEPAGE_855=m
CONFIG_NLS_CODEPAGE_857=m
CONFIG_NLS_CODEPAGE_860=m
CONFIG_NLS_CODEPAGE_861=m
CONFIG_NLS_CODEPAGE_862=m
CONFIG_NLS_CODEPAGE_863=m
CONFIG_NLS_CODEPAGE_864=m
CONFIG_NLS_CODEPAGE_865=m
CONFIG_NLS_CODEPAGE_866=m
CONFIG_NLS_CODEPAGE_869=m
CONFIG_NLS_CODEPAGE_936=m
CONFIG_NLS_CODEPAGE_950=m
CONFIG_NLS_CODEPAGE_932=m
CONFIG_NLS_CODEPAGE_949=m
CONFIG_NLS_CODEPAGE_874=m
CONFIG_NLS_ISO8859_8=m
CONFIG_NLS_CODEPAGE_1250=m
CONFIG_NLS_CODEPAGE_1251=m
CONFIG_NLS_ASCII=y
CONFIG_NLS_ISO8859_1=m
CONFIG_NLS_ISO8859_2=m
CONFIG_NLS_ISO8859_3=m
CONFIG_NLS_ISO8859_4=m
CONFIG_NLS_ISO8859_5=m
CONFIG_NLS_ISO8859_6=m
CONFIG_NLS_ISO8859_7=m
CONFIG_NLS_ISO8859_9=m
CONFIG_NLS_ISO8859_13=m
CONFIG_NLS_ISO8859_14=m
CONFIG_NLS_ISO8859_15=m
CONFIG_NLS_KOI8_R=m
CONFIG_NLS_KOI8_U=m
CONFIG_NLS_UTF8=m
CONFIG_DLM=m
CONFIG_DLM_DEBUG=y
#
# Kernel hacking
#
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
# CONFIG_PRINTK_TIME is not set
# CONFIG_ENABLE_WARN_DEPRECATED is not set
# CONFIG_ENABLE_MUST_CHECK is not set
CONFIG_FRAME_WARN=2048
CONFIG_MAGIC_SYSRQ=y
CONFIG_STRIP_ASM_SYMS=y
# CONFIG_UNUSED_SYMBOLS is not set
CONFIG_DEBUG_FS=y
# CONFIG_HEADERS_CHECK is not set
CONFIG_DEBUG_KERNEL=y
# CONFIG_DEBUG_SHIRQ is not set
CONFIG_DETECT_SOFTLOCKUP=y
CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC=y
CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC_VALUE=1
CONFIG_DETECT_HUNG_TASK=y
CONFIG_BOOTPARAM_HUNG_TASK_PANIC=y
CONFIG_BOOTPARAM_HUNG_TASK_PANIC_VALUE=1
CONFIG_SCHED_DEBUG=y
CONFIG_SCHEDSTATS=y
# CONFIG_TIMER_STATS is not set
# CONFIG_DEBUG_OBJECTS is not set
# CONFIG_DEBUG_SLAB is not set
# CONFIG_DEBUG_RT_MUTEXES is not set
# CONFIG_RT_MUTEX_TESTER is not set
# CONFIG_DEBUG_SPINLOCK is not set
# CONFIG_DEBUG_MUTEXES is not set
# CONFIG_DEBUG_LOCK_ALLOC is not set
# CONFIG_PROVE_LOCKING is not set
# CONFIG_LOCK_STAT is not set
# CONFIG_DEBUG_SPINLOCK_SLEEP is not set
# CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
CONFIG_STACKTRACE=y
# CONFIG_DEBUG_KOBJECT is not set
CONFIG_DEBUG_BUGVERBOSE=y
CONFIG_DEBUG_INFO=y
# CONFIG_DEBUG_VM is not set
# CONFIG_DEBUG_VIRTUAL is not set
# CONFIG_DEBUG_WRITECOUNT is not set
CONFIG_DEBUG_MEMORY_INIT=y
CONFIG_DEBUG_LIST=y
# CONFIG_DEBUG_SG is not set
# CONFIG_DEBUG_NOTIFIERS is not set
# CONFIG_DEBUG_CREDENTIALS is not set
CONFIG_ARCH_WANT_FRAME_POINTERS=y
CONFIG_FRAME_POINTER=y
# CONFIG_BOOT_PRINTK_DELAY is not set
# CONFIG_RCU_TORTURE_TEST is not set
# CONFIG_RCU_CPU_STALL_DETECTOR is not set
# CONFIG_KPROBES_SANITY_TEST is not set
# CONFIG_BACKTRACE_SELF_TEST is not set
# CONFIG_DEBUG_BLOCK_EXT_DEVT is not set
# CONFIG_DEBUG_FORCE_WEAK_PER_CPU is not set
# CONFIG_LKDTM is not set
# CONFIG_FAULT_INJECTION is not set
CONFIG_LATENCYTOP=y
CONFIG_SYSCTL_SYSCALL_CHECK=y
# CONFIG_DEBUG_PAGEALLOC is not set
CONFIG_USER_STACKTRACE_SUPPORT=y
CONFIG_HAVE_FUNCTION_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_FP_TEST=y
CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
CONFIG_RING_BUFFER=y
CONFIG_RING_BUFFER_ALLOW_SWAP=y
CONFIG_TRACING_SUPPORT=y
# CONFIG_FTRACE is not set
# CONFIG_PROVIDE_OHCI1394_DMA_INIT is not set
# CONFIG_FIREWIRE_OHCI_REMOTE_DMA is not set
# CONFIG_DYNAMIC_DEBUG is not set
# CONFIG_DMA_API_DEBUG is not set
# CONFIG_SAMPLES is not set
CONFIG_HAVE_ARCH_KGDB=y
# CONFIG_KGDB is not set
CONFIG_HAVE_ARCH_KMEMCHECK=y
CONFIG_STRICT_DEVMEM=y
# CONFIG_X86_VERBOSE_BOOTUP is not set
CONFIG_EARLY_PRINTK=y
# CONFIG_EARLY_PRINTK_DBGP is not set
CONFIG_DEBUG_STACKOVERFLOW=y
# CONFIG_DEBUG_STACK_USAGE is not set
# CONFIG_DEBUG_PER_CPU_MAPS is not set
# CONFIG_X86_PTDUMP is not set
CONFIG_DEBUG_RODATA=y
# CONFIG_DEBUG_RODATA_TEST is not set
# CONFIG_DEBUG_NX_TEST is not set
# CONFIG_IOMMU_DEBUG is not set
# CONFIG_IOMMU_STRESS is not set
CONFIG_HAVE_MMIOTRACE_SUPPORT=y
CONFIG_IO_DELAY_TYPE_0X80=0
CONFIG_IO_DELAY_TYPE_0XED=1
CONFIG_IO_DELAY_TYPE_UDELAY=2
CONFIG_IO_DELAY_TYPE_NONE=3
CONFIG_IO_DELAY_0X80=y
# CONFIG_IO_DELAY_0XED is not set
# CONFIG_IO_DELAY_UDELAY is not set
# CONFIG_IO_DELAY_NONE is not set
CONFIG_DEFAULT_IO_DELAY_TYPE=0
# CONFIG_DEBUG_BOOT_PARAMS is not set
# CONFIG_CPA_DEBUG is not set
# CONFIG_OPTIMIZE_INLINING is not set
#
# Security options
#
CONFIG_KEYS=y
CONFIG_KEYS_DEBUG_PROC_KEYS=y
CONFIG_SECURITY=y
CONFIG_SECURITYFS=y
CONFIG_SECURITY_NETWORK=y
CONFIG_SECURITY_NETWORK_XFRM=y
# CONFIG_SECURITY_PATH is not set
# CONFIG_SECURITY_FILE_CAPABILITIES is not set
# CONFIG_SECURITY_ROOTPLUG is not set
CONFIG_LSM_MMAP_MIN_ADDR=65536
CONFIG_SECURITY_SELINUX=y
CONFIG_SECURITY_SELINUX_BOOTPARAM=y
CONFIG_SECURITY_SELINUX_BOOTPARAM_VALUE=1
CONFIG_SECURITY_SELINUX_DISABLE=y
CONFIG_SECURITY_SELINUX_DEVELOP=y
CONFIG_SECURITY_SELINUX_AVC_STATS=y
CONFIG_SECURITY_SELINUX_CHECKREQPROT_VALUE=1
# CONFIG_SECURITY_SELINUX_POLICYDB_VERSION_MAX is not set
# CONFIG_SECURITY_SMACK is not set
# CONFIG_SECURITY_TOMOYO is not set
# CONFIG_IMA is not set
CONFIG_XOR_BLOCKS=m
CONFIG_ASYNC_CORE=m
CONFIG_ASYNC_MEMCPY=m
CONFIG_ASYNC_XOR=m
CONFIG_ASYNC_PQ=m
CONFIG_ASYNC_RAID6_RECOV=m
CONFIG_CRYPTO=y
#
# Crypto core or helper
#
CONFIG_CRYPTO_FIPS=y
CONFIG_CRYPTO_ALGAPI=y
CONFIG_CRYPTO_ALGAPI2=y
CONFIG_CRYPTO_AEAD=m
CONFIG_CRYPTO_AEAD2=y
CONFIG_CRYPTO_BLKCIPHER=m
CONFIG_CRYPTO_BLKCIPHER2=y
CONFIG_CRYPTO_HASH=y
CONFIG_CRYPTO_HASH2=y
CONFIG_CRYPTO_RNG=m
CONFIG_CRYPTO_RNG2=y
CONFIG_CRYPTO_PCOMP=y
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_MANAGER2=y
CONFIG_CRYPTO_GF128MUL=m
CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_WORKQUEUE=y
CONFIG_CRYPTO_CRYPTD=m
CONFIG_CRYPTO_AUTHENC=m
# CONFIG_CRYPTO_TEST is not set
#
# Authenticated Encryption with Associated Data
#
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_GCM=m
CONFIG_CRYPTO_SEQIV=m
#
# Block modes
#
CONFIG_CRYPTO_CBC=m
CONFIG_CRYPTO_CTR=m
CONFIG_CRYPTO_CTS=m
CONFIG_CRYPTO_ECB=m
# CONFIG_CRYPTO_LRW is not set
CONFIG_CRYPTO_PCBC=m
# CONFIG_CRYPTO_XTS is not set
CONFIG_CRYPTO_FPU=m
#
# Hash modes
#
CONFIG_CRYPTO_HMAC=y
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_VMAC=m
#
# Digest
#
CONFIG_CRYPTO_CRC32C=y
CONFIG_CRYPTO_CRC32C_INTEL=m
CONFIG_CRYPTO_GHASH=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MD5=m
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_RMD128=m
CONFIG_CRYPTO_RMD160=m
CONFIG_CRYPTO_RMD256=m
CONFIG_CRYPTO_RMD320=m
CONFIG_CRYPTO_SHA1=y
CONFIG_CRYPTO_SHA256=m
CONFIG_CRYPTO_SHA512=m
CONFIG_CRYPTO_TGR192=m
CONFIG_CRYPTO_WP512=m
#
# Ciphers
#
CONFIG_CRYPTO_AES=m
CONFIG_CRYPTO_AES_X86_64=m
CONFIG_CRYPTO_AES_NI_INTEL=m
CONFIG_CRYPTO_ANUBIS=m
CONFIG_CRYPTO_ARC4=m
CONFIG_CRYPTO_BLOWFISH=m
CONFIG_CRYPTO_CAMELLIA=m
CONFIG_CRYPTO_CAST5=m
CONFIG_CRYPTO_CAST6=m
CONFIG_CRYPTO_DES=m
CONFIG_CRYPTO_FCRYPT=m
CONFIG_CRYPTO_KHAZAD=m
# CONFIG_CRYPTO_SALSA20 is not set
# CONFIG_CRYPTO_SALSA20_X86_64 is not set
CONFIG_CRYPTO_SEED=m
CONFIG_CRYPTO_SERPENT=m
CONFIG_CRYPTO_TEA=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_TWOFISH_COMMON=m
CONFIG_CRYPTO_TWOFISH_X86_64=m
#
# Compression
#
CONFIG_CRYPTO_DEFLATE=m
CONFIG_CRYPTO_ZLIB=m
CONFIG_CRYPTO_LZO=m
#
# Random Number Generation
#
CONFIG_CRYPTO_ANSI_CPRNG=m
CONFIG_CRYPTO_HW=y
CONFIG_CRYPTO_DEV_PADLOCK=m
CONFIG_CRYPTO_DEV_PADLOCK_AES=m
CONFIG_CRYPTO_DEV_PADLOCK_SHA=m
# CONFIG_CRYPTO_DEV_HIFN_795X is not set
CONFIG_HAVE_KVM=y
# CONFIG_VIRTUALIZATION is not set
# CONFIG_BINARY_PRINTF is not set
#
# Library routines
#
CONFIG_BITREVERSE=y
CONFIG_GENERIC_FIND_FIRST_BIT=y
CONFIG_GENERIC_FIND_NEXT_BIT=y
CONFIG_GENERIC_FIND_LAST_BIT=y
CONFIG_CRC_CCITT=m
CONFIG_CRC16=m
CONFIG_CRC_T10DIF=m
CONFIG_CRC_ITU_T=m
CONFIG_CRC32=y
CONFIG_CRC7=m
CONFIG_LIBCRC32C=y
CONFIG_ZLIB_INFLATE=y
CONFIG_ZLIB_DEFLATE=m
CONFIG_LZO_COMPRESS=m
CONFIG_LZO_DECOMPRESS=m
CONFIG_DECOMPRESS_GZIP=y
CONFIG_DECOMPRESS_BZIP2=y
CONFIG_DECOMPRESS_LZMA=y
CONFIG_REED_SOLOMON=m
CONFIG_REED_SOLOMON_DEC16=y
CONFIG_TEXTSEARCH=y
CONFIG_TEXTSEARCH_KMP=m
CONFIG_TEXTSEARCH_BM=m
CONFIG_TEXTSEARCH_FSM=m
CONFIG_HAS_IOMEM=y
CONFIG_HAS_IOPORT=y
CONFIG_HAS_DMA=y
CONFIG_CPUMASK_OFFSTACK=y
CONFIG_NLATTR=y
[-- Attachment #3: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-01-27 22:20 ` Teck Choon Giam
@ 2011-02-26 12:03 ` Teck Choon Giam
2011-02-28 16:20 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2011-02-26 12:03 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 12472 bytes --]
On Fri, Jan 28, 2011 at 6:20 AM, Teck Choon Giam <giamteckchoon@gmail.com>wrote:
>
>
> On Fri, Jan 28, 2011 at 4:32 AM, Konrad Rzeszutek Wilk <
> konrad.wilk@oracle.com> wrote:
>
>> > > What does git log show? What is the latest commit you have there?
>> > >
>> >
>> > commit 75cc13f5aa29b4f3227d269ca165dfa8937c94fe
>> > Merge: 2607c07 a386bf7
>> > Author: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
>> > Date: Thu Dec 9 17:16:16 2010 -0800
>> >
>> > Merge commit 'v2.6.32.27' into xen/next-2.6.32
>> >
>> > * commit 'v2.6.32.27': (128 commits)
>> > Linux 2.6.32.27
>>
>> .. snip..
>> >
>> > Anything that I can test or you need me to test... feel free to ask so
>> that
>> > I can speed up in tracking/hunting this bug down :)
>>
>> You got the latest one so you should have the fixes. But something is not
>> working.. just to make sure I am not missing anything can you send me your
>> .config
>> file?
>>
>
> Ok... please see attached.
>
>
> Thanks.
>
> Kindest regards,
> Giam Teck Choon
>
>
Hi Konrad,
This is just an update from my testing.
I tried with latest xen/next-2.6.32.x and same bug still there:
Feb 23 22:56:56 xen06 kernel: ------------[ cut here ]------------
Feb 23 22:56:56 xen06 kernel: kernel BUG at arch/x86/xen/mmu.c:1872!
Feb 23 22:56:56 xen06 kernel: invalid opcode: 0000 [#1] SMP
Feb 23 22:56:56 xen06 kernel: last sysfs file:
/sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_map
Feb 23 22:56:56 xen06 kernel: CPU 0
Feb 23 22:56:56 xen06 kernel: Modules linked in: dlm configfs xt_physdev
iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic
uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi
dm_multipath scsi_dh video backlight output sbs sbshc power_meter hwmon
battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc lp parport sg
ide_cd_mod cdrom serio_raw tg3 libphy button tpm_tis tpm tpm_bios iTCO_wdt
i2c_i801 i2c_core pcspkr shpchp dm_snapshot dm_zero dm_mirror dm_region_hash
dm_log dm_mod ata_piix libata sd_mod scsi_mod raid1 ext3 jbd uhci_hcd
ohci_hcd ehci_hcd [last unloaded: microcode]
Feb 23 22:56:56 xen06 kernel: Pid: 15222, comm: mpath_wait Not tainted
2.6.32.28-2.xen.pvops.choon.centos5 #1 PowerEdge 860
Feb 23 22:56:56 xen06 kernel: RIP: e030:[<ffffffff8100cb5b>]
[<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
Feb 23 22:56:56 xen06 kernel: RSP: e02b:ffff8800265b9dc8 EFLAGS: 00010282
Feb 23 22:56:56 xen06 kernel: RAX: 00000000ffffffea RBX: 00000000000306d7
RCX: 0000000000000183
Feb 23 22:56:56 xen06 kernel: RDX: 00000000deadbeef RSI: 00000000deadbeef
RDI: 00000000deadbeef
Feb 23 22:56:56 xen06 kernel: RBP: ffff8800265b9de8 R08: 00000000000006b8
R09: ffff880000000000
Feb 23 22:56:56 xen06 kernel: R10: 00000000deadbeef R11: 0000000000000246
R12: 0000000000000003
Feb 23 22:56:56 xen06 kernel: R13: 00000000000306d7 R14: ffff88003b940200
R15: 00007f7655df6258
Feb 23 22:56:56 xen06 kernel: FS: 00007f76560016e0(0000)
GS:ffff88002804f000(0000) knlGS:0000000000000000
Feb 23 22:56:56 xen06 kernel: CS: e033 DS: 0000 ES: 0000 CR0:
000000008005003b
Feb 23 22:56:56 xen06 kernel: CR2: 00007f7655df6258 CR3: 000000002559b000
CR4: 0000000000002660
Feb 23 22:56:56 xen06 lvm[3984]: Monitoring snapshot
XenGroup-testcrash1--snapshot
Feb 23 22:56:56 xen06 kernel: DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000
Feb 23 22:56:56 xen06 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
DR7: 0000000000000400
Feb 23 22:56:56 xen06 kernel: Process mpath_wait (pid: 15222, threadinfo
ffff8800265b8000, task ffff88003b940200)
Feb 23 22:56:56 xen06 kernel: Stack:
Feb 23 22:56:56 xen06 kernel: 0000000000000000 0000000000202bff
000000013f009e18 00000000000306d7
Feb 23 22:56:56 xen06 kernel: <0> ffff8800265b9e08 ffffffff8100e07c
ffff8800267bdac0 ffff88002679f570
Feb 23 22:56:56 xen06 kernel: <0> ffff8800265b9e18 ffffffff8100e0af
ffff8800265b9e58 ffffffff810a408b
Feb 23 22:56:56 xen06 kernel: Call Trace:
Feb 23 22:56:56 xen06 kernel: [<ffffffff8100e07c>]
xen_alloc_ptpage+0x64/0x69
Feb 23 22:56:56 xen06 kernel: [<ffffffff8100e0af>] xen_alloc_pte+0xe/0x10
Feb 23 22:56:56 xen06 kernel: [<ffffffff810a408b>] __pte_alloc+0x70/0xce
Feb 23 22:56:56 xen06 kernel: [<ffffffff810a4229>]
handle_mm_fault+0x140/0x8b9
Feb 23 22:56:56 xen06 kernel: [<ffffffff8131be2d>]
do_page_fault+0x252/0x2e2
Feb 23 22:56:56 xen06 kernel: [<ffffffff81319db5>] page_fault+0x25/0x30
Feb 23 22:56:56 xen06 kernel: Code: 48 b8 ff ff ff ff ff ff ff 7f 48 21 c2
48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 e9 c7 ff
ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74 5b
48
Feb 23 22:56:56 xen06 kernel: RIP [<ffffffff8100cb5b>]
pin_pagetable_pfn+0x53/0x59
Feb 23 22:56:56 xen06 kernel: RSP <ffff8800265b9dc8>
Feb 23 22:56:56 xen06 kernel: ---[ end trace fd2f141edfc37649 ]---
Feb 23 22:56:56 xen06 kernel: kjournald starting. Commit interval 5 seconds
Feb 23 22:56:56 xen06 kernel: EXT3 FS on dm-11, internal journal
Feb 23 22:56:56 xen06 kernel: EXT3-fs: mounted filesystem with ordered data
mode.
Feb 23 23:00:01 xen06 syslogd 1.4.1: restart.
Feb 23 23:00:01 xen06 kernel: klogd 1.4.1, log source = /proc/kmsg started.
Feb 23 23:00:01 xen06 kernel: Linux version
2.6.32.28-2.xen.pvops.choon.centos5 (mockbuild@builder5.choon.net) (gcc
version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Wed Feb 23 22:29:39 SGT
2011
Feb 23 23:00:01 xen06 kernel: Command line: ro root=/dev/md1 panic=5
panic_timeout=5
Feb 23 23:00:01 xen06 kernel: KERNEL supported cpus:
Feb 23 23:00:01 xen06 kernel: Intel GenuineIntel
Feb 23 23:00:01 xen06 kernel: AMD AuthenticAMD
Feb 23 23:00:01 xen06 kernel: Centaur CentaurHauls
Feb 23 23:00:01 xen06 kernel: released 0 pages of unused memory
Feb 23 23:00:01 xen06 kernel: BIOS-provided physical RAM map:
Feb 23 23:00:01 xen06 kernel: Xen: 0000000000000000 - 00000000000a0000
(usable)
Feb 23 23:00:01 xen06 kernel: Xen: 00000000000a0000 - 0000000000100000
(reserved)
Feb 23 23:00:01 xen06 kernel: Xen: 0000000000100000 - 0000000040000000
(usable)
Feb 23 23:00:01 xen06 kernel: Xen: 00000000dffc0000 - 00000000dffcfc00
(ACPI data)
Feb 23 23:00:01 xen06 kernel: Xen: 00000000dffcfc00 - 00000000dffff000
(reserved)
Feb 23 23:00:01 xen06 kernel: Xen: 00000000f0000000 - 00000000f4000000
(reserved)
Feb 23 23:00:01 xen06 kernel: Xen: 00000000fec00000 - 00000000fed00400
(reserved)
Feb 23 23:00:01 xen06 kernel: Xen: 00000000fed13000 - 00000000feda0000
(reserved)
Feb 23 23:00:01 xen06 kernel: Xen: 00000000fee00000 - 00000000fee10000
(reserved)
Feb 23 23:00:01 xen06 kernel: Xen: 00000000ffb00000 - 0000000100000000
(reserved)
Feb 23 23:00:01 xen06 kernel: Xen: 00000001ffffe000 - 0000000200000000
(reserved)
Feb 23 23:00:01 xen06 kernel: Xen: 0000000200000000 - 00000003bffbe000
(usable)
Even I patch with latest 2.6.32.29:
Feb 24 01:03:17 xen06 kernel: ------------[ cut here ]------------
Feb 24 01:03:17 xen06 kernel: kernel BUG at arch/x86/xen/mmu.c:1872!
Feb 24 01:03:17 xen06 kernel: invalid opcode: 0000 [#2] SMP
Feb 24 01:03:17 xen06 kernel: last sysfs file: /sys/block/dm-13/dev
Feb 24 01:03:17 xen06 kernel: CPU 2
Feb 24 01:03:17 xen06 kernel: Modules linked in: dlm configfs xt_physdev
iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic
uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi
dm_multipath scsi_dh video backlight output sbs sbshc power_meter hwmon
battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc lp parport sg
ide_cd_mod cdrom tg3 serio_raw libphy button tpm_tis tpm tpm_bios pcspkr
shpchp i2c_i801 i2c_core iTCO_wdt dm_snapshot dm_zero dm_mirror
dm_region_hash dm_log dm_mod ata_piix libata sd_mod scsi_mod raid1 ext3 jbd
uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
Feb 24 01:03:17 xen06 kernel: Pid: 509, comm: dmsetup Tainted: G D
2.6.32.29-0.xen.pvops.choon.centos5 #1 PowerEdge 860
Feb 24 01:03:17 xen06 kernel: RIP: e030:[<ffffffff8100cb5b>]
[<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
Feb 24 01:03:17 xen06 kernel: RSP: e02b:ffff88003ced5dc8 EFLAGS: 00010282
Feb 24 01:03:17 xen06 kernel: RAX: 00000000ffffffea RBX: 0000000000030395
RCX: 0000000000000181
Feb 24 01:03:17 xen06 kernel: RDX: 00000000deadbeef RSI: 00000000deadbeef
RDI: 00000000deadbeef
Feb 24 01:03:17 xen06 kernel: RBP: ffff88003ced5de8 R08: 0000000000000ca8
R09: ffff880000000000
Feb 24 01:03:17 xen06 kernel: R10: 00000000deadbeef R11: 0000000000000246
R12: 0000000000000003
Feb 24 01:03:17 xen06 kernel: R13: 0000000000030395 R14: ffff88003025c4c0
R15: 00000033e1e00258
Feb 24 01:03:17 xen06 kernel: FS: 00007ff20eb366e0(0000)
GS:ffff880028089000(0000) knlGS:0000000000000000
Feb 24 01:03:17 xen06 kernel: CS: e033 DS: 0000 ES: 0000 CR0:
000000008005003b
Feb 24 01:03:17 xen06 kernel: CR2: 00000033e1e00258 CR3: 000000003b57b000
CR4: 0000000000002660
Feb 24 01:03:17 xen06 kernel: DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000
Feb 24 01:03:17 xen06 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
DR7: 0000000000000400
Feb 24 01:03:17 xen06 kernel: Process dmsetup (pid: 509, threadinfo
ffff88003ced4000, task ffff88003025c4c0)
Feb 24 01:03:17 xen06 kernel: Stack:
Feb 24 01:03:17 xen06 kernel: 0000000000000000 0000000000202741
000000013e6e5a18 0000000000030395
Feb 24 01:03:17 xen06 kernel: <0> ffff88003ced5e08 ffffffff8100e07c
ffff88003000aac0 ffff880036bb1878
Feb 24 01:03:17 xen06 kernel: <0> ffff88003ced5e18 ffffffff8100e0af
ffff88003ced5e58 ffffffff810a4309
Feb 24 01:03:17 xen06 kernel: Call Trace:
Feb 24 01:03:17 xen06 kernel: [<ffffffff8100e07c>]
xen_alloc_ptpage+0x64/0x69
Feb 24 01:03:17 xen06 kernel: [<ffffffff8100e0af>] xen_alloc_pte+0xe/0x10
Feb 24 01:03:17 xen06 kernel: [<ffffffff810a4309>] __pte_alloc+0x70/0xce
Feb 24 01:03:17 xen06 kernel: [<ffffffff810a44a7>]
handle_mm_fault+0x140/0x8b9
Feb 24 01:03:17 xen06 kernel: [<ffffffff8131c1fd>]
do_page_fault+0x252/0x2e2
Feb 24 01:03:17 xen06 kernel: [<ffffffff8131a185>] page_fault+0x25/0x30
Feb 24 01:03:18 xen06 kernel: Code: 48 b8 ff ff ff ff ff ff ff 7f 48 21 c2
48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 e9 c7 ff
ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74 5b
48
Feb 24 01:03:18 xen06 kernel: RIP [<ffffffff8100cb5b>]
pin_pagetable_pfn+0x53/0x59
Feb 24 01:03:18 xen06 kernel: RSP <ffff88003ced5dc8>
Feb 24 01:03:18 xen06 kernel: ---[ end trace 5ea31e622470b519 ]---
Feb 24 01:06:21 xen06 syslogd 1.4.1: restart.
Feb 24 01:06:21 xen06 kernel: klogd 1.4.1, log source = /proc/kmsg started.
Feb 24 01:06:21 xen06 kernel: Linux version
2.6.32.29-0.xen.pvops.choon.centos5 (mockbuild@builder5.choon.net) (gcc
version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Thu Feb 24 00:21:24 SGT
2011
Feb 24 01:06:21 xen06 kernel: Command line: ro root=/dev/md1 panic=5
panic_timeout=5
Feb 24 01:06:21 xen06 kernel: KERNEL supported cpus:
Feb 24 01:06:21 xen06 kernel: Intel GenuineIntel
Feb 24 01:06:21 xen06 kernel: AMD AuthenticAMD
Feb 24 01:06:21 xen06 kernel: Centaur CentaurHauls
Feb 24 01:06:21 xen06 kernel: released 0 pages of unused memory
Feb 24 01:06:21 xen06 kernel: BIOS-provided physical RAM map:
Feb 24 01:06:21 xen06 kernel: Xen: 0000000000000000 - 00000000000a0000
(usable)
Feb 24 01:06:21 xen06 kernel: Xen: 00000000000a0000 - 0000000000100000
(reserved)
Feb 24 01:06:21 xen06 kernel: Xen: 0000000000100000 - 0000000040000000
(usable)
Feb 24 01:06:21 xen06 kernel: Xen: 00000000dffc0000 - 00000000dffcfc00
(ACPI data)
Feb 24 01:06:21 xen06 kernel: Xen: 00000000dffcfc00 - 00000000dffff000
(reserved)
Feb 24 01:06:21 xen06 kernel: Xen: 00000000f0000000 - 00000000f4000000
(reserved)
Feb 24 01:06:21 xen06 kernel: Xen: 00000000fec00000 - 00000000fed00400
(reserved)
Feb 24 01:06:21 xen06 kernel: Xen: 00000000fed13000 - 00000000feda0000
(reserved)
Feb 24 01:06:21 xen06 kernel: Xen: 00000000fee00000 - 00000000fee10000
(reserved)
Feb 24 01:06:21 xen06 kernel: Xen: 00000000ffb00000 - 0000000100000000
(reserved)
Feb 24 01:06:21 xen06 kernel: Xen: 00000001ffffe000 - 0000000200000000
(reserved)
Feb 24 01:06:21 xen06 kernel: Xen: 0000000200000000 - 00000003bffbe000
(usable)
Feb 24 01:06:21 xen06 kernel: DMI 2.4 present.
Feb 24 01:06:21 xen06 kernel: last_pfn = 0x3bffbe max_arch_pfn = 0x400000000
Feb 24 01:06:21 xen06 kernel: x86 PAT enabled: cpu 0, old 0x50100070406, new
0x7010600070106
Feb 24 01:06:21 xen06 kernel: last_pfn = 0x40000 max_arch_pfn = 0x400000000
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 14086 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-02-26 12:03 ` Teck Choon Giam
@ 2011-02-28 16:20 ` Konrad Rzeszutek Wilk
2011-03-01 9:59 ` Teck Choon Giam
0 siblings, 1 reply; 85+ messages in thread
From: Konrad Rzeszutek Wilk @ 2011-02-28 16:20 UTC (permalink / raw)
To: Teck Choon Giam; +Cc: xen-devel
> This is just an update from my testing.
>
> I tried with latest xen/next-2.6.32.x and same bug still there:
Grrrrrrrr..
Let me setup a machine to reproduce this. Is the distro you are using
still CentOS 5?
>
> Feb 23 22:56:56 xen06 kernel: ------------[ cut here ]------------
> Feb 23 22:56:56 xen06 kernel: kernel BUG at arch/x86/xen/mmu.c:1872!
> Feb 23 22:56:56 xen06 kernel: invalid opcode: 0000 [#1] SMP
> Feb 23 22:56:56 xen06 kernel: last sysfs file:
> /sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_map
> Feb 23 22:56:56 xen06 kernel: CPU 0
> Feb 23 22:56:56 xen06 kernel: Modules linked in: dlm configfs xt_physdev
> iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic
> uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi
> dm_multipath scsi_dh video backlight output sbs sbshc power_meter hwmon
> battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc lp parport sg
> ide_cd_mod cdrom serio_raw tg3 libphy button tpm_tis tpm tpm_bios iTCO_wdt
> i2c_i801 i2c_core pcspkr shpchp dm_snapshot dm_zero dm_mirror dm_region_hash
> dm_log dm_mod ata_piix libata sd_mod scsi_mod raid1 ext3 jbd uhci_hcd
> ohci_hcd ehci_hcd [last unloaded: microcode]
> Feb 23 22:56:56 xen06 kernel: Pid: 15222, comm: mpath_wait Not tainted
> 2.6.32.28-2.xen.pvops.choon.centos5 #1 PowerEdge 860
> Feb 23 22:56:56 xen06 kernel: RIP: e030:[<ffffffff8100cb5b>]
> [<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
> Feb 23 22:56:56 xen06 kernel: RSP: e02b:ffff8800265b9dc8 EFLAGS: 00010282
> Feb 23 22:56:56 xen06 kernel: RAX: 00000000ffffffea RBX: 00000000000306d7
> RCX: 0000000000000183
> Feb 23 22:56:56 xen06 kernel: RDX: 00000000deadbeef RSI: 00000000deadbeef
> RDI: 00000000deadbeef
> Feb 23 22:56:56 xen06 kernel: RBP: ffff8800265b9de8 R08: 00000000000006b8
> R09: ffff880000000000
> Feb 23 22:56:56 xen06 kernel: R10: 00000000deadbeef R11: 0000000000000246
> R12: 0000000000000003
> Feb 23 22:56:56 xen06 kernel: R13: 00000000000306d7 R14: ffff88003b940200
> R15: 00007f7655df6258
> Feb 23 22:56:56 xen06 kernel: FS: 00007f76560016e0(0000)
> GS:ffff88002804f000(0000) knlGS:0000000000000000
> Feb 23 22:56:56 xen06 kernel: CS: e033 DS: 0000 ES: 0000 CR0:
> 000000008005003b
> Feb 23 22:56:56 xen06 kernel: CR2: 00007f7655df6258 CR3: 000000002559b000
> CR4: 0000000000002660
> Feb 23 22:56:56 xen06 lvm[3984]: Monitoring snapshot
> XenGroup-testcrash1--snapshot
> Feb 23 22:56:56 xen06 kernel: DR0: 0000000000000000 DR1: 0000000000000000
> DR2: 0000000000000000
> Feb 23 22:56:56 xen06 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
> DR7: 0000000000000400
> Feb 23 22:56:56 xen06 kernel: Process mpath_wait (pid: 15222, threadinfo
> ffff8800265b8000, task ffff88003b940200)
> Feb 23 22:56:56 xen06 kernel: Stack:
> Feb 23 22:56:56 xen06 kernel: 0000000000000000 0000000000202bff
> 000000013f009e18 00000000000306d7
> Feb 23 22:56:56 xen06 kernel: <0> ffff8800265b9e08 ffffffff8100e07c
> ffff8800267bdac0 ffff88002679f570
> Feb 23 22:56:56 xen06 kernel: <0> ffff8800265b9e18 ffffffff8100e0af
> ffff8800265b9e58 ffffffff810a408b
> Feb 23 22:56:56 xen06 kernel: Call Trace:
> Feb 23 22:56:56 xen06 kernel: [<ffffffff8100e07c>]
> xen_alloc_ptpage+0x64/0x69
> Feb 23 22:56:56 xen06 kernel: [<ffffffff8100e0af>] xen_alloc_pte+0xe/0x10
> Feb 23 22:56:56 xen06 kernel: [<ffffffff810a408b>] __pte_alloc+0x70/0xce
> Feb 23 22:56:56 xen06 kernel: [<ffffffff810a4229>]
> handle_mm_fault+0x140/0x8b9
> Feb 23 22:56:56 xen06 kernel: [<ffffffff8131be2d>]
> do_page_fault+0x252/0x2e2
> Feb 23 22:56:56 xen06 kernel: [<ffffffff81319db5>] page_fault+0x25/0x30
> Feb 23 22:56:56 xen06 kernel: Code: 48 b8 ff ff ff ff ff ff ff 7f 48 21 c2
> 48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 e9 c7 ff
> ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74 5b
> 48
> Feb 23 22:56:56 xen06 kernel: RIP [<ffffffff8100cb5b>]
> pin_pagetable_pfn+0x53/0x59
> Feb 23 22:56:56 xen06 kernel: RSP <ffff8800265b9dc8>
> Feb 23 22:56:56 xen06 kernel: ---[ end trace fd2f141edfc37649 ]---
> Feb 23 22:56:56 xen06 kernel: kjournald starting. Commit interval 5 seconds
> Feb 23 22:56:56 xen06 kernel: EXT3 FS on dm-11, internal journal
> Feb 23 22:56:56 xen06 kernel: EXT3-fs: mounted filesystem with ordered data
> mode.
> Feb 23 23:00:01 xen06 syslogd 1.4.1: restart.
> Feb 23 23:00:01 xen06 kernel: klogd 1.4.1, log source = /proc/kmsg started.
> Feb 23 23:00:01 xen06 kernel: Linux version
> 2.6.32.28-2.xen.pvops.choon.centos5 (mockbuild@builder5.choon.net) (gcc
> version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Wed Feb 23 22:29:39 SGT
> 2011
> Feb 23 23:00:01 xen06 kernel: Command line: ro root=/dev/md1 panic=5
> panic_timeout=5
> Feb 23 23:00:01 xen06 kernel: KERNEL supported cpus:
> Feb 23 23:00:01 xen06 kernel: Intel GenuineIntel
> Feb 23 23:00:01 xen06 kernel: AMD AuthenticAMD
> Feb 23 23:00:01 xen06 kernel: Centaur CentaurHauls
> Feb 23 23:00:01 xen06 kernel: released 0 pages of unused memory
> Feb 23 23:00:01 xen06 kernel: BIOS-provided physical RAM map:
> Feb 23 23:00:01 xen06 kernel: Xen: 0000000000000000 - 00000000000a0000
> (usable)
> Feb 23 23:00:01 xen06 kernel: Xen: 00000000000a0000 - 0000000000100000
> (reserved)
> Feb 23 23:00:01 xen06 kernel: Xen: 0000000000100000 - 0000000040000000
> (usable)
> Feb 23 23:00:01 xen06 kernel: Xen: 00000000dffc0000 - 00000000dffcfc00
> (ACPI data)
> Feb 23 23:00:01 xen06 kernel: Xen: 00000000dffcfc00 - 00000000dffff000
> (reserved)
> Feb 23 23:00:01 xen06 kernel: Xen: 00000000f0000000 - 00000000f4000000
> (reserved)
> Feb 23 23:00:01 xen06 kernel: Xen: 00000000fec00000 - 00000000fed00400
> (reserved)
> Feb 23 23:00:01 xen06 kernel: Xen: 00000000fed13000 - 00000000feda0000
> (reserved)
> Feb 23 23:00:01 xen06 kernel: Xen: 00000000fee00000 - 00000000fee10000
> (reserved)
> Feb 23 23:00:01 xen06 kernel: Xen: 00000000ffb00000 - 0000000100000000
> (reserved)
> Feb 23 23:00:01 xen06 kernel: Xen: 00000001ffffe000 - 0000000200000000
> (reserved)
> Feb 23 23:00:01 xen06 kernel: Xen: 0000000200000000 - 00000003bffbe000
> (usable)
>
> Even I patch with latest 2.6.32.29:
>
> Feb 24 01:03:17 xen06 kernel: ------------[ cut here ]------------
> Feb 24 01:03:17 xen06 kernel: kernel BUG at arch/x86/xen/mmu.c:1872!
> Feb 24 01:03:17 xen06 kernel: invalid opcode: 0000 [#2] SMP
> Feb 24 01:03:17 xen06 kernel: last sysfs file: /sys/block/dm-13/dev
> Feb 24 01:03:17 xen06 kernel: CPU 2
> Feb 24 01:03:17 xen06 kernel: Modules linked in: dlm configfs xt_physdev
> iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic
> uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi
> dm_multipath scsi_dh video backlight output sbs sbshc power_meter hwmon
> battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc lp parport sg
> ide_cd_mod cdrom tg3 serio_raw libphy button tpm_tis tpm tpm_bios pcspkr
> shpchp i2c_i801 i2c_core iTCO_wdt dm_snapshot dm_zero dm_mirror
> dm_region_hash dm_log dm_mod ata_piix libata sd_mod scsi_mod raid1 ext3 jbd
> uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
> Feb 24 01:03:17 xen06 kernel: Pid: 509, comm: dmsetup Tainted: G D
> 2.6.32.29-0.xen.pvops.choon.centos5 #1 PowerEdge 860
> Feb 24 01:03:17 xen06 kernel: RIP: e030:[<ffffffff8100cb5b>]
> [<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
> Feb 24 01:03:17 xen06 kernel: RSP: e02b:ffff88003ced5dc8 EFLAGS: 00010282
> Feb 24 01:03:17 xen06 kernel: RAX: 00000000ffffffea RBX: 0000000000030395
> RCX: 0000000000000181
> Feb 24 01:03:17 xen06 kernel: RDX: 00000000deadbeef RSI: 00000000deadbeef
> RDI: 00000000deadbeef
> Feb 24 01:03:17 xen06 kernel: RBP: ffff88003ced5de8 R08: 0000000000000ca8
> R09: ffff880000000000
> Feb 24 01:03:17 xen06 kernel: R10: 00000000deadbeef R11: 0000000000000246
> R12: 0000000000000003
> Feb 24 01:03:17 xen06 kernel: R13: 0000000000030395 R14: ffff88003025c4c0
> R15: 00000033e1e00258
> Feb 24 01:03:17 xen06 kernel: FS: 00007ff20eb366e0(0000)
> GS:ffff880028089000(0000) knlGS:0000000000000000
> Feb 24 01:03:17 xen06 kernel: CS: e033 DS: 0000 ES: 0000 CR0:
> 000000008005003b
> Feb 24 01:03:17 xen06 kernel: CR2: 00000033e1e00258 CR3: 000000003b57b000
> CR4: 0000000000002660
> Feb 24 01:03:17 xen06 kernel: DR0: 0000000000000000 DR1: 0000000000000000
> DR2: 0000000000000000
> Feb 24 01:03:17 xen06 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
> DR7: 0000000000000400
> Feb 24 01:03:17 xen06 kernel: Process dmsetup (pid: 509, threadinfo
> ffff88003ced4000, task ffff88003025c4c0)
> Feb 24 01:03:17 xen06 kernel: Stack:
> Feb 24 01:03:17 xen06 kernel: 0000000000000000 0000000000202741
> 000000013e6e5a18 0000000000030395
> Feb 24 01:03:17 xen06 kernel: <0> ffff88003ced5e08 ffffffff8100e07c
> ffff88003000aac0 ffff880036bb1878
> Feb 24 01:03:17 xen06 kernel: <0> ffff88003ced5e18 ffffffff8100e0af
> ffff88003ced5e58 ffffffff810a4309
> Feb 24 01:03:17 xen06 kernel: Call Trace:
> Feb 24 01:03:17 xen06 kernel: [<ffffffff8100e07c>]
> xen_alloc_ptpage+0x64/0x69
> Feb 24 01:03:17 xen06 kernel: [<ffffffff8100e0af>] xen_alloc_pte+0xe/0x10
> Feb 24 01:03:17 xen06 kernel: [<ffffffff810a4309>] __pte_alloc+0x70/0xce
> Feb 24 01:03:17 xen06 kernel: [<ffffffff810a44a7>]
> handle_mm_fault+0x140/0x8b9
> Feb 24 01:03:17 xen06 kernel: [<ffffffff8131c1fd>]
> do_page_fault+0x252/0x2e2
> Feb 24 01:03:17 xen06 kernel: [<ffffffff8131a185>] page_fault+0x25/0x30
> Feb 24 01:03:18 xen06 kernel: Code: 48 b8 ff ff ff ff ff ff ff 7f 48 21 c2
> 48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8 e9 c7 ff
> ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40 f6 c7 01 48 89 e5 53 48 89 fb 74 5b
> 48
> Feb 24 01:03:18 xen06 kernel: RIP [<ffffffff8100cb5b>]
> pin_pagetable_pfn+0x53/0x59
> Feb 24 01:03:18 xen06 kernel: RSP <ffff88003ced5dc8>
> Feb 24 01:03:18 xen06 kernel: ---[ end trace 5ea31e622470b519 ]---
> Feb 24 01:06:21 xen06 syslogd 1.4.1: restart.
> Feb 24 01:06:21 xen06 kernel: klogd 1.4.1, log source = /proc/kmsg started.
> Feb 24 01:06:21 xen06 kernel: Linux version
> 2.6.32.29-0.xen.pvops.choon.centos5 (mockbuild@builder5.choon.net) (gcc
> version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Thu Feb 24 00:21:24 SGT
> 2011
> Feb 24 01:06:21 xen06 kernel: Command line: ro root=/dev/md1 panic=5
> panic_timeout=5
> Feb 24 01:06:21 xen06 kernel: KERNEL supported cpus:
> Feb 24 01:06:21 xen06 kernel: Intel GenuineIntel
> Feb 24 01:06:21 xen06 kernel: AMD AuthenticAMD
> Feb 24 01:06:21 xen06 kernel: Centaur CentaurHauls
> Feb 24 01:06:21 xen06 kernel: released 0 pages of unused memory
> Feb 24 01:06:21 xen06 kernel: BIOS-provided physical RAM map:
> Feb 24 01:06:21 xen06 kernel: Xen: 0000000000000000 - 00000000000a0000
> (usable)
> Feb 24 01:06:21 xen06 kernel: Xen: 00000000000a0000 - 0000000000100000
> (reserved)
> Feb 24 01:06:21 xen06 kernel: Xen: 0000000000100000 - 0000000040000000
> (usable)
> Feb 24 01:06:21 xen06 kernel: Xen: 00000000dffc0000 - 00000000dffcfc00
> (ACPI data)
> Feb 24 01:06:21 xen06 kernel: Xen: 00000000dffcfc00 - 00000000dffff000
> (reserved)
> Feb 24 01:06:21 xen06 kernel: Xen: 00000000f0000000 - 00000000f4000000
> (reserved)
> Feb 24 01:06:21 xen06 kernel: Xen: 00000000fec00000 - 00000000fed00400
> (reserved)
> Feb 24 01:06:21 xen06 kernel: Xen: 00000000fed13000 - 00000000feda0000
> (reserved)
> Feb 24 01:06:21 xen06 kernel: Xen: 00000000fee00000 - 00000000fee10000
> (reserved)
> Feb 24 01:06:21 xen06 kernel: Xen: 00000000ffb00000 - 0000000100000000
> (reserved)
> Feb 24 01:06:21 xen06 kernel: Xen: 00000001ffffe000 - 0000000200000000
> (reserved)
> Feb 24 01:06:21 xen06 kernel: Xen: 0000000200000000 - 00000003bffbe000
> (usable)
> Feb 24 01:06:21 xen06 kernel: DMI 2.4 present.
> Feb 24 01:06:21 xen06 kernel: last_pfn = 0x3bffbe max_arch_pfn = 0x400000000
> Feb 24 01:06:21 xen06 kernel: x86 PAT enabled: cpu 0, old 0x50100070406, new
> 0x7010600070106
> Feb 24 01:06:21 xen06 kernel: last_pfn = 0x40000 max_arch_pfn = 0x400000000
>
> Thanks.
>
> Kindest regards,
> Giam Teck Choon
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-02-28 16:20 ` Konrad Rzeszutek Wilk
@ 2011-03-01 9:59 ` Teck Choon Giam
2011-03-03 22:16 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2011-03-01 9:59 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 364 bytes --]
On Tue, Mar 1, 2011 at 12:20 AM, Konrad Rzeszutek Wilk <
konrad.wilk@oracle.com> wrote:
> > This is just an update from my testing.
> >
> > I tried with latest xen/next-2.6.32.x and same bug still there:
>
> Grrrrrrrr..
>
> Let me setup a machine to reproduce this. Is the distro you are using
> still CentOS 5?
>
Yes.
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 708 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-01 9:59 ` Teck Choon Giam
@ 2011-03-03 22:16 ` Konrad Rzeszutek Wilk
2011-03-04 5:30 ` Teck Choon Giam
0 siblings, 1 reply; 85+ messages in thread
From: Konrad Rzeszutek Wilk @ 2011-03-03 22:16 UTC (permalink / raw)
To: Teck Choon Giam; +Cc: xen-devel
On Tue, Mar 01, 2011 at 05:59:54PM +0800, Teck Choon Giam wrote:
> On Tue, Mar 1, 2011 at 12:20 AM, Konrad Rzeszutek Wilk <
> konrad.wilk@oracle.com> wrote:
>
> > > This is just an update from my testing.
> > >
> > > I tried with latest xen/next-2.6.32.x and same bug still there:
> >
> > Grrrrrrrr..
> >
> > Let me setup a machine to reproduce this. Is the distro you are using
> > still CentOS 5?
> >
>
> Yes.
OK, I got a machine with CentOS 5.5 installed. Had some trouble getting
the kernel to boot - a different issue that the one you are seeing however.
What arguments are you using for your dom0 and Xen hypervisor?
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-03 22:16 ` Konrad Rzeszutek Wilk
@ 2011-03-04 5:30 ` Teck Choon Giam
2011-03-04 6:15 ` Fajar A. Nugraha
0 siblings, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2011-03-04 5:30 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 1142 bytes --]
On Fri, Mar 4, 2011 at 6:16 AM, Konrad Rzeszutek Wilk <
konrad.wilk@oracle.com> wrote:
> On Tue, Mar 01, 2011 at 05:59:54PM +0800, Teck Choon Giam wrote:
> > On Tue, Mar 1, 2011 at 12:20 AM, Konrad Rzeszutek Wilk <
> > konrad.wilk@oracle.com> wrote:
> >
> > > > This is just an update from my testing.
> > > >
> > > > I tried with latest xen/next-2.6.32.x and same bug still there:
> > >
> > > Grrrrrrrr..
> > >
> > > Let me setup a machine to reproduce this. Is the distro you are using
> > > still CentOS 5?
> > >
> >
> > Yes.
>
> OK, I got a machine with CentOS 5.5 installed. Had some trouble getting
> the kernel to boot - a different issue that the one you are seeing however.
>
> What arguments are you using for your dom0 and Xen hypervisor?
>
Example in one of my test server:
title CentOS (2.6.32.29-0.xen.pvops.choon.centos5)
root (hd0,0)
kernel /xen.gz dom0_mem=1024M loglvl=all guest_loglvl=all cpuidle=0
cpufreq=none
module /vmlinuz-2.6.32.29-0.xen.pvops.choon.centos5 ro root=/dev/md1
panic=5 panic_timeout=5
module /initrd-2.6.32.29-0.xen.pvops.choon.centos5.img
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 1661 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-04 5:30 ` Teck Choon Giam
@ 2011-03-04 6:15 ` Fajar A. Nugraha
2011-03-04 6:33 ` Teck Choon Giam
0 siblings, 1 reply; 85+ messages in thread
From: Fajar A. Nugraha @ 2011-03-04 6:15 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel, Teck Choon Giam
[-- Attachment #1.1: Type: text/plain, Size: 1385 bytes --]
On Fri, Mar 4, 2011 at 12:30 PM, Teck Choon Giam <giamteckchoon@gmail.com>wrote:
>
>
> On Fri, Mar 4, 2011 at 6:16 AM, Konrad Rzeszutek Wilk <
> konrad.wilk@oracle.com> wrote:
>
>> On Tue, Mar 01, 2011 at 05:59:54PM +0800, Teck Choon Giam wrote:
>> > On Tue, Mar 1, 2011 at 12:20 AM, Konrad Rzeszutek Wilk <
>> > konrad.wilk@oracle.com> wrote:
>> >
>> > > > This is just an update from my testing.
>> > > >
>> > > > I tried with latest xen/next-2.6.32.x and same bug still there:
>> > >
>> > > Grrrrrrrr..
>> > >
>> > > Let me setup a machine to reproduce this. Is the distro you are using
>> > > still CentOS 5?
>> > >
>> >
>> > Yes.
>>
>> OK, I got a machine with CentOS 5.5 installed. Had some trouble getting
>> the kernel to boot - a different issue that the one you are seeing
>> however.
>>
>> What arguments are you using for your dom0 and Xen hypervisor?
>>
>
>
> Example in one of my test server:
>
> title CentOS (2.6.32.29-0.xen.pvops.choon.centos5)
> root (hd0,0)
> kernel /xen.gz dom0_mem=1024M loglvl=all guest_loglvl=all cpuidle=0
> cpufreq=none
> module /vmlinuz-2.6.32.29-0.xen.pvops.choon.centos5 ro root=/dev/md1
> panic=5 panic_timeout=5
> module /initrd-2.6.32.29-0.xen.pvops.choon.centos5.img
>
RHEL/Centos5 also
needs CONFIG_SYSFS_DEPRECATED=y, CONFIG_SYSFS_DEPRECATED_V2=y for 2.6.32
kernel to work correctly with userland tools.
--
Fajar
[-- Attachment #1.2: Type: text/html, Size: 2179 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-04 6:15 ` Fajar A. Nugraha
@ 2011-03-04 6:33 ` Teck Choon Giam
2011-03-08 19:29 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2011-03-04 6:33 UTC (permalink / raw)
To: Fajar A. Nugraha; +Cc: xen-devel, Konrad Rzeszutek Wilk
[-- Attachment #1.1: Type: text/plain, Size: 1710 bytes --]
On Fri, Mar 4, 2011 at 2:15 PM, Fajar A. Nugraha <list@fajar.net> wrote:
> On Fri, Mar 4, 2011 at 12:30 PM, Teck Choon Giam <giamteckchoon@gmail.com>wrote:
>
>>
>>
>> On Fri, Mar 4, 2011 at 6:16 AM, Konrad Rzeszutek Wilk <
>> konrad.wilk@oracle.com> wrote:
>>
>>> On Tue, Mar 01, 2011 at 05:59:54PM +0800, Teck Choon Giam wrote:
>>> > On Tue, Mar 1, 2011 at 12:20 AM, Konrad Rzeszutek Wilk <
>>> > konrad.wilk@oracle.com> wrote:
>>> >
>>> > > > This is just an update from my testing.
>>> > > >
>>> > > > I tried with latest xen/next-2.6.32.x and same bug still there:
>>> > >
>>> > > Grrrrrrrr..
>>> > >
>>> > > Let me setup a machine to reproduce this. Is the distro you are using
>>> > > still CentOS 5?
>>> > >
>>> >
>>> > Yes.
>>>
>>> OK, I got a machine with CentOS 5.5 installed. Had some trouble getting
>>> the kernel to boot - a different issue that the one you are seeing
>>> however.
>>>
>>> What arguments are you using for your dom0 and Xen hypervisor?
>>>
>>
>>
>> Example in one of my test server:
>>
>> title CentOS (2.6.32.29-0.xen.pvops.choon.centos5)
>> root (hd0,0)
>> kernel /xen.gz dom0_mem=1024M loglvl=all guest_loglvl=all cpuidle=0
>> cpufreq=none
>> module /vmlinuz-2.6.32.29-0.xen.pvops.choon.centos5 ro root=/dev/md1
>> panic=5 panic_timeout=5
>> module /initrd-2.6.32.29-0.xen.pvops.choon.centos5.img
>>
>
>
> RHEL/Centos5 also
> needs CONFIG_SYSFS_DEPRECATED=y, CONFIG_SYSFS_DEPRECATED_V2=y for 2.6.32
> kernel to work correctly with userland tools.
>
Thanks Fajar ;)
Yes. All are set in my kernel config and I believed Konrad has my kernel
config. I didn't use nomodeset as my servers don't have those graphic
cards... ...
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 2797 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-04 6:33 ` Teck Choon Giam
@ 2011-03-08 19:29 ` Konrad Rzeszutek Wilk
2011-03-08 20:10 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 85+ messages in thread
From: Konrad Rzeszutek Wilk @ 2011-03-08 19:29 UTC (permalink / raw)
To: Teck Choon Giam; +Cc: xen-devel, Fajar A. Nugraha
[-- Attachment #1: Type: text/plain, Size: 2582 bytes --]
> Yes. All are set in my kernel config and I believed Konrad has my kernel
> config. I didn't use nomodeset as my servers don't have those graphic
> cards... ...
I am having a hard-time reproducing this. I made six LV's:
[root@tst011 ~]# lvs
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
LogVol00 VolGroup00 -wi-ao 17.56G
LogVol01 VolGroup00 -wi-ao 5.81G
data-1.ext3 XenGroup -wi-ao 10.00G
data-2.ext3 XenGroup owi-ao 10.00G
data-2.ext3-snapshot XenGroup swi-a- 1.00G data-2.ext3 2.41
data-3.ext3 XenGroup -wi-a- 10.00G
scratch-1.ext3 XenGroup -wi-ao 10.00G
scratch-2.ext3 XenGroup -wi-ao 10.00G
scratch-3.ext3 XenGroup -wi-a- 10.00G
[root@tst011 ~]#
Where each scratch-X/data-X was prepared with 'mkfs.ext3'
And ran two guests, where each guest configuration looks as so:
kernel="/mnt/lab/latest/vmlinuz"
ramdisk="/mnt/lab/latest/initramfs.cpio.gz"
extra="console=hvc0 debug"
memory=768
vcpus=4
on_crash="preserve"
#vif = [ 'mac=00:0f:4b:00:00:68, bridge=switch' ]
vfb = [ 'vnc=1, vnclisten=0.0.0.0,vncunused=1']
disk = [ 'phy:/dev/XenGroup/scratch-1.ext3,xvda,w', 'phy:/dev/XenGroup/data-1.ext3,xvdb,w']
(the other is using -2 obviously).
And each guest is running 'mount /dev/xvda /mnt-1;(cd /mnt-1;fio iometer-file-access-server)'
to produce I/Os on the xvdb/xvda disks.
As those guests are chugging along, I ran your script with:
root 6250 4438 0 10:12 pts/4 00:00:00 /bin/sh ./lvm-test.sh loop 100 1G 5
(I tried 100 1G 0)
and so far it is running.... how long should I wait until I hit this problem?
This is on CentOS 5.5 and also on Fedora Core 13. Dom0 and DomU are all x86_64.
Dom0 is:
commit 892d2f052e979cf1916647c752b94cf62ec1c6dc
Merge: 35e2e28... 376faec...
Author: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Date: Fri Feb 11 13:31:03 2011 -0800
Merge commit 'v2.6.32.28' into xen/next-2.6.32
(plus one patch I just posted - but that is to fix the serial console, so
it is not relevant to this problem).
DomU is 2.6.38 kernel, but I can swap over to the same as Dom0..
Attached is the script I am using.
[-- Attachment #2: iometer-file-access-server --]
[-- Type: text/plain, Size: 439 bytes --]
# This job file tries to mimic the Intel IOMeter File Server Access Pattern
[global]
description=Emulation of Intel IOmeter File Server Access Pattern
[iometer]
bssplit=512/10:1k/5:2k/5:4k/60:8k/2:16k/4:32k/4:64k/10
rw=randrw
rwmixread=80
direct=1
size=4g
ioengine=libaio
# IOMeter defines the server loads as the following:
# iodepth=1 Linear
# iodepth=4 Very Light
# iodepth=8 Light
# iodepth=64 Moderate
# iodepth=256 Heavy
iodepth=64
[-- Attachment #3: lvm-test.sh --]
[-- Type: application/x-sh, Size: 3237 bytes --]
[-- Attachment #4: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-08 19:29 ` Konrad Rzeszutek Wilk
@ 2011-03-08 20:10 ` Konrad Rzeszutek Wilk
2011-03-08 20:20 ` Teck Choon Giam
0 siblings, 1 reply; 85+ messages in thread
From: Konrad Rzeszutek Wilk @ 2011-03-08 20:10 UTC (permalink / raw)
To: Teck Choon Giam; +Cc: xen-devel, Fajar A. Nugraha
On Tue, Mar 08, 2011 at 02:29:50PM -0500, Konrad Rzeszutek Wilk wrote:
> > Yes. All are set in my kernel config and I believed Konrad has my kernel
> > config. I didn't use nomodeset as my servers don't have those graphic
> > cards... ...
>
> I am having a hard-time reproducing this. I made six LV's:
Aha!
respawning too fast: disabled for 5 minutes
INIT: Id "co" respawning too fast: disabled for 5 minutes
INIT: Id "co" respawning too fast: disabled for 5 minutes
INIT: Id "co" respawning too fast: disabled for 5 minutes
(XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 3000000000000000) for mfn 354e0 (pfn c76d7)
(XEN) mm.c:942:d0 Attempt : entry 255
(XEN) mm.c:2117:d0 Error while validating mfn 35620 (pfn c7417) for type 4000000000000000: caf=8000000000000003 taf=4000000000000001
(XEN) mm.c:2733:d0 Error while pinning mfn 35620
(XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 3000000000000000) for mfn 354e0 (pfn c76d7)
(XEN) mm.c:942:d0 Attempt to create linear p.t. with write perms
(XEN) mm.c:1487:d0 Failure in alloc_l4_table: entry 255
(XEN) mm.c:2117:d0 Error while validating mfn 3b286 (pfn c187d) for type 4000000000000000: caf=8000000000000003 taf=4000000000000001
(XEN) mm.c:2733:d0 Error while pinning mfn 3b286
(XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 3000000000000000) for mfn 354e0 (pfn c76d7)
(XEN) mm.c:942:d0 Attempt to create linear p.t. with write perms
(XEN) mm.c:1487:d0 Failure in alloc_l4_table: entry 255
(XEN) mm.c:2117:d0 Error while validating mfn 35620 (pfn c7417) for type 4000000000000000: caf=8000000000000003 taf=4000000000000001
(XEN) mm.c:2500:d0 Error while installing new baseptr 35620
(XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 3000000000000000) for mfn 354e0 (pfn c76d7)
(XEN) mm.c:942:d0 Attempt to create linear p.t. with write perms
(XEN) mm.c:1487:d0 Failure in alloc_l4_table: entry 255
(XEN) mm.c:2117:d0 Error while validating mfn 3b286 (pfn c187d) for type 4000000000000000: caf=8000000000000003 taf=4000000000000001
(XEN) mm.c:2825:d0 Error while installing new mfn 3b286
(XEN) mm.c:2794:d0 Mfn 35620 not pinned
(XEN) mm.c:2794:d0 Mfn 3b286 not pinned
Takes a bit of time to reproduce it, but I can reproduce it on my CentOS 5.5 OS.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-08 20:10 ` Konrad Rzeszutek Wilk
@ 2011-03-08 20:20 ` Teck Choon Giam
2011-03-08 20:45 ` Guido Hecken
` (2 more replies)
0 siblings, 3 replies; 85+ messages in thread
From: Teck Choon Giam @ 2011-03-08 20:20 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel, Fajar A. Nugraha
[-- Attachment #1.1: Type: text/plain, Size: 2932 bytes --]
On Wed, Mar 9, 2011 at 4:10 AM, Konrad Rzeszutek Wilk <
konrad.wilk@oracle.com> wrote:
> On Tue, Mar 08, 2011 at 02:29:50PM -0500, Konrad Rzeszutek Wilk wrote:
> > > Yes. All are set in my kernel config and I believed Konrad has my
> kernel
> > > config. I didn't use nomodeset as my servers don't have those graphic
> > > cards... ...
> >
> > I am having a hard-time reproducing this. I made six LV's:
>
> Aha!
>
> respawning too fast: disabled for 5 minutes
> INIT: Id "co" respawning too fast: disabled for 5 minutes
> INIT: Id "co" respawning too fast: disabled for 5 minutes
> INIT: Id "co" respawning too fast: disabled for 5 minutes
> (XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 3000000000000000)
> for mfn 354e0 (pfn c76d7)
> (XEN) mm.c:942:d0 Attempt : entry 255
> (XEN) mm.c:2117:d0 Error while validating mfn 35620 (pfn c7417) for type
> 4000000000000000: caf=8000000000000003 taf=4000000000000001
> (XEN) mm.c:2733:d0 Error while pinning mfn 35620
> (XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 3000000000000000)
> for mfn 354e0 (pfn c76d7)
> (XEN) mm.c:942:d0 Attempt to create linear p.t. with write perms
> (XEN) mm.c:1487:d0 Failure in alloc_l4_table: entry 255
> (XEN) mm.c:2117:d0 Error while validating mfn 3b286 (pfn c187d) for type
> 4000000000000000: caf=8000000000000003 taf=4000000000000001
> (XEN) mm.c:2733:d0 Error while pinning mfn 3b286
> (XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 3000000000000000)
> for mfn 354e0 (pfn c76d7)
> (XEN) mm.c:942:d0 Attempt to create linear p.t. with write perms
> (XEN) mm.c:1487:d0 Failure in alloc_l4_table: entry 255
> (XEN) mm.c:2117:d0 Error while validating mfn 35620 (pfn c7417) for type
> 4000000000000000: caf=8000000000000003 taf=4000000000000001
> (XEN) mm.c:2500:d0 Error while installing new baseptr 35620
> (XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 3000000000000000)
> for mfn 354e0 (pfn c76d7)
> (XEN) mm.c:942:d0 Attempt to create linear p.t. with write perms
> (XEN) mm.c:1487:d0 Failure in alloc_l4_table: entry 255
> (XEN) mm.c:2117:d0 Error while validating mfn 3b286 (pfn c187d) for type
> 4000000000000000: caf=8000000000000003 taf=4000000000000001
> (XEN) mm.c:2825:d0 Error while installing new mfn 3b286
> (XEN) mm.c:2794:d0 Mfn 35620 not pinned
> (XEN) mm.c:2794:d0 Mfn 3b286 not pinned
>
> Takes a bit of time to reproduce it, but I can reproduce it on my CentOS
> 5.5 OS.
>
Yep... it takes sometime to hit this bug and if you run my test script
without any guest running will be faster to reproduce it.
May I know what is your normal development platform/distribution for xen? I
hope this isn't just distribution specific issue for PVOPS
stable/xen-2.6.32.x Dom0... ... mostly not as from this thread alone I see
someone saying they hit this bug as well and I believe they are not running
CentOS 5.x... guess is debian based distribution.
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 3438 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-08 20:20 ` Teck Choon Giam
@ 2011-03-08 20:45 ` Guido Hecken
2011-03-08 20:50 ` [SPAM] " tjaouen
2011-03-09 0:41 ` kernel BUG at arch/x86/xen/mmu.c:1860! Konrad Rzeszutek Wilk
2 siblings, 0 replies; 85+ messages in thread
From: Guido Hecken @ 2011-03-08 20:45 UTC (permalink / raw)
To: Teck Choon Giam, Konrad Rzeszutek Wilk; +Cc: xen-devel, Fajar A. Nugraha
[-- Attachment #1.1: Type: text/plain, Size: 3500 bytes --]
Hi,
maybe I've got the same problem on debian squeeze.
I opened a bug report some time ago but diddn't get any feedback till now.
Perhaps there were not enough informations...
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=614400
Guido
Von: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] Im Auftrag von Teck Choon Giam
Gesendet: Dienstag, 8. März 2011 21:21
An: Konrad Rzeszutek Wilk
Cc: xen-devel@lists.xensource.com; Fajar A. Nugraha
Betreff: Re: [Xen-devel] kernel BUG at arch/x86/xen/mmu.c:1860!
On Wed, Mar 9, 2011 at 4:10 AM, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
On Tue, Mar 08, 2011 at 02:29:50PM -0500, Konrad Rzeszutek Wilk wrote:
> > Yes. All are set in my kernel config and I believed Konrad has my kernel
> > config. I didn't use nomodeset as my servers don't have those graphic
> > cards... ...
>
> I am having a hard-time reproducing this. I made six LV's:
Aha!
respawning too fast: disabled for 5 minutes
INIT: Id "co" respawning too fast: disabled for 5 minutes
INIT: Id "co" respawning too fast: disabled for 5 minutes
INIT: Id "co" respawning too fast: disabled for 5 minutes
(XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 3000000000000000) for mfn 354e0 (pfn c76d7)
(XEN) mm.c:942:d0 Attempt : entry 255
(XEN) mm.c:2117:d0 Error while validating mfn 35620 (pfn c7417) for type 4000000000000000: caf=8000000000000003 taf=4000000000000001
(XEN) mm.c:2733:d0 Error while pinning mfn 35620
(XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 3000000000000000) for mfn 354e0 (pfn c76d7)
(XEN) mm.c:942:d0 Attempt to create linear p.t. with write perms
(XEN) mm.c:1487:d0 Failure in alloc_l4_table: entry 255
(XEN) mm.c:2117:d0 Error while validating mfn 3b286 (pfn c187d) for type 4000000000000000: caf=8000000000000003 taf=4000000000000001
(XEN) mm.c:2733:d0 Error while pinning mfn 3b286
(XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 3000000000000000) for mfn 354e0 (pfn c76d7)
(XEN) mm.c:942:d0 Attempt to create linear p.t. with write perms
(XEN) mm.c:1487:d0 Failure in alloc_l4_table: entry 255
(XEN) mm.c:2117:d0 Error while validating mfn 35620 (pfn c7417) for type 4000000000000000: caf=8000000000000003 taf=4000000000000001
(XEN) mm.c:2500:d0 Error while installing new baseptr 35620
(XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 3000000000000000) for mfn 354e0 (pfn c76d7)
(XEN) mm.c:942:d0 Attempt to create linear p.t. with write perms
(XEN) mm.c:1487:d0 Failure in alloc_l4_table: entry 255
(XEN) mm.c:2117:d0 Error while validating mfn 3b286 (pfn c187d) for type 4000000000000000: caf=8000000000000003 taf=4000000000000001
(XEN) mm.c:2825:d0 Error while installing new mfn 3b286
(XEN) mm.c:2794:d0 Mfn 35620 not pinned
(XEN) mm.c:2794:d0 Mfn 3b286 not pinned
Takes a bit of time to reproduce it, but I can reproduce it on my CentOS 5.5 OS.
Yep... it takes sometime to hit this bug and if you run my test script without any guest running will be faster to reproduce it.
May I know what is your normal development platform/distribution for xen? I hope this isn't just distribution specific issue for PVOPS stable/xen-2.6.32.x Dom0... ... mostly not as from this thread alone I see someone saying they hit this bug as well and I believe they are not running CentOS 5.x... guess is debian based distribution.
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 9217 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-08 20:20 ` Teck Choon Giam
2011-03-08 20:45 ` Guido Hecken
@ 2011-03-08 20:50 ` tjaouen
2011-03-09 0:06 ` Andreas Olsowski
2011-03-09 0:43 ` [SPAM] " Konrad Rzeszutek Wilk
2011-03-09 0:41 ` kernel BUG at arch/x86/xen/mmu.c:1860! Konrad Rzeszutek Wilk
2 siblings, 2 replies; 85+ messages in thread
From: tjaouen @ 2011-03-08 20:50 UTC (permalink / raw)
To: xen-devel
Teck Choon Giam wrote:
>
>
> I hope this isn't just distribution specific issue for PVOPS
> stable/xen-2.6.32.x Dom0... ... mostly not as from this thread alone I see
> someone saying they hit this bug as well and I believe they are not
> running
> CentOS 5.x... guess is debian based distribution.
>
>
Debian/Squeeze ""Stable""
Debian GNU/Linux 6.0
Linux xencave 2.6.32-5-xen-amd64 #1 SMP Wed Jan 12 05:46:49 UTC 2011 x86_64
GNU/Linux
March 1, I enter this command:
lvcreate -s -L 5G -n databank_snap /dev/vgraid1/databank
.... and BUG in /var/log/syslog:
Mar 1 21:23:56 xencave kernel: [1840078.405905] ------------[ cut here
]------------
Mar 1 21:23:56 xencave kernel: [1840078.405937] kernel BUG at
/build/buildd-linux-2.6_2.6.32-30-amd64-d4MbNM/linux-2.6-2.6.32/debian/build/source_amd64_xen/arch/x86/xen/mmu.c:1649!
Mar 1 21:23:56 xencave kernel: [1840078.405981] invalid opcode: 0000 [#1]
SMP
Mar 1 21:23:56 xencave kernel: [1840078.406010] last sysfs file:
/sys/devices/virtual/bdi/253:35/uevent
Mar 1 21:23:56 xencave kernel: [1840078.406037] CPU 0
Mar 1 21:23:56 xencave kernel: [1840078.406065] Modules linked in:
dm_snapshot nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack
xt_physdev iptable_filter ip_tables x_tables xen_evtchn xenfs bridge stp
coretemp it87 hwmon_vid fuse loop i915 drm_kms_helper drm snd_pcsp processor
i2c_algo_bit button video acpi_processor i2c_i801 snd_pcm snd_timer rng_core
serio_raw i2c_core evdev snd output soundcore snd_page_alloc usbhid hid ext3
jbd mbcache dm_mod raid1 md_mod sg sr_mod cdrom sd_mod crc_t10dif
ata_generic uhci_hcd ata_piix libata r8169 mii thermal thermal_sys ehci_hcd
scsi_mod usbcore nls_base [last unloaded: scsi_wait_scan]
Mar 1 21:23:56 xencave kernel: [1840078.406505] Pid: 14525, comm: udevd Not
tainted 2.6.32-5-xen-amd64 #1 945GM-S2
Mar 1 21:23:56 xencave kernel: [1840078.406539] RIP: e030:[] []
pin_pagetable_pfn+0x2d/0x36
Mar 1 21:23:56 xencave kernel: [1840078.406586] RSP: e02b:ffff88000df33e08
EFLAGS: 00010282
Mar 1 21:23:56 xencave kernel: [1840078.406608] RAX: 00000000ffffffea RBX:
000000000000df31 RCX: 0000000000000001
Mar 1 21:23:56 xencave kernel: [1840078.406642] RDX: 0000000000000000 RSI:
0000000000000001 RDI: ffff88000df33e08
Mar 1 21:23:56 xencave kernel: [1840078.406677] RBP: ffff8800028bb480 R08:
0000000000000988 R09: ffffea000030d2b8
Mar 1 21:23:56 xencave kernel: [1840078.406711] R10: 0000000000007ff0 R11:
ffff880000000838 R12: ffff88004f026a48
Mar 1 21:23:56 xencave kernel: [1840078.406745] R13: ffff880002be3f18 R14:
ffff8800513f2a60 R15: ffff88004f026a48
Mar 1 21:23:56 xencave kernel: [1840078.406784] FS: 00007ff6a99077a0(0000)
GS:ffff88000390b000(0000) knlGS:0000000000000000
Mar 1 21:23:56 xencave kernel: [1840078.406820] CS: e033 DS: 0000 ES: 0000
CR0: 000000008005003b
Mar 1 21:23:56 xencave kernel: [1840078.406842] CR2: 00007ff6a9215876 CR3:
000000004af1a000 CR4: 0000000000002660
Mar 1 21:23:56 xencave kernel: [1840078.406878] DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
Mar 1 21:23:56 xencave kernel: [1840078.406913] DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
Mar 1 21:23:56 xencave kernel: [1840078.406948] Process udevd (pid: 14525,
threadinfo ffff88000df32000, task ffff8800513f2a60)
Mar 1 21:23:56 xencave kernel: [1840078.406985] Stack:
Mar 1 21:23:56 xencave kernel: [1840078.407003] 0000000000000000
00000000000af260 ffffea000030d2b8 000000000000df31
Mar 1 21:23:56 xencave kernel: [1840078.407042] <0> ffff8800028bb480
ffffffff810cd4e2 ffff88004af24ed0 00007ff6a9215876
Mar 1 21:23:56 xencave kernel: [1840078.407095] <0> 000000004af24000
ffffffff810cb394 ffff880002be3f18 00007ff6a9215876
Mar 1 21:23:56 xencave kernel: [1840078.407164] Call Trace:
Mar 1 21:23:56 xencave kernel: [1840078.407187] [] ? __pte_alloc+0x6b/0xc6
Mar 1 21:23:56 xencave kernel: [1840078.407212] [] ? pmd_alloc+0x28/0x5b
Mar 1 21:23:56 xencave kernel: [1840078.407235] [] ?
handle_mm_fault+0xce/0x80f
Mar 1 21:23:56 xencave kernel: [1840078.407262] [] ? page_fault+0x25/0x30
Mar 1 21:23:56 xencave kernel: [1840078.407287] [] ? error_exit+0x2a/0x60
Mar 1 21:23:56 xencave kernel: [1840078.407311] [] ?
retint_restore_args+0x5/0x6
Mar 1 21:23:56 xencave kernel: [1840078.407336] [] ?
do_page_fault+0x2e0/0x2fc
Mar 1 21:23:56 xencave kernel: [1840078.407361] [] ? page_fault+0x25/0x30
Mar 1 21:23:56 xencave kernel: [1840078.407383] Code: ec 28 89 3c 24 48 89
f7 e8 a2 fd ff ff 48 89 e7 48 89 44 24 08 be 01 00 00 00 31 d2 41 ba f0 7f
00 00 e8 b0 cc ff ff 85 c0 74 04 <0f> 0b eb fe 48 83 c4 28 c3 55 49 89 ca 48
89 d5 40 88 f1 48 89
Mar 1 21:23:56 xencave kernel: [1840078.407702] RIP []
pin_pagetable_pfn+0x2d/0x36
Mar 1 21:23:56 xencave kernel: [1840078.407731] RSP
--
View this message in context: http://xen.1045712.n5.nabble.com/kernel-BUG-at-arch-x86-xen-mmu-c-1860-tp3318567p3414620.html
Sent from the Xen - Dev mailing list archive at Nabble.com.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-08 20:50 ` [SPAM] " tjaouen
@ 2011-03-09 0:06 ` Andreas Olsowski
2011-03-11 18:38 ` benco
2011-03-09 0:43 ` [SPAM] " Konrad Rzeszutek Wilk
1 sibling, 1 reply; 85+ messages in thread
From: Andreas Olsowski @ 2011-03-09 0:06 UTC (permalink / raw)
To: xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 5206 bytes --]
Well, this is too bad.
I encountered this bug when xen 4.0 was released, around the time
development on 2.6.31 was halted.
That is why i stuck with 2.6.31 when everyone else went with 2.6.32,
because i determined 2.6.32 was not fit for duty and im guessing it
still isnt today.
The bug occures on 2.6.32 xen kernels ( maybe even newer ones) and is
distribution unrelated, i was running debian 5.0 then, i am running 6.0
testing now and even have tried compiling all the userland stuff myself.
This is error can be encountered during a number of different actions:
1.) any action with lvm (start, stop, create, delete)
2.) while starting multipathd (restarting too, of course)
Sometimes the box only hangs there and no further device mapper
interactions are possible. This is where i got my syslog entry from.
Back in 2010 i had to serial console the server and stuff like that to
see the whole error.
my guess is everything one does with the device mapper can and will
trigger this sooner or later.
Does anybody have any kind of insight on what the problem may be?
------------
Here is my syslog part when i ran "/etc/init.d/multipath-tools restart":
Mar 9 00:24:10 memoryana multipathd: mpatha: stop event checker thread
(140606587918080)
Mar 9 00:24:10 memoryana multipathd: mpathb: stop event checker thread
(140606587885312)
Mar 9 00:24:10 memoryana multipathd: mpathc: stop event checker thread
(140606587852544)
Mar 9 00:24:10 memoryana kernel: ------------[ cut here ]------------
Mar 9 00:24:10 memoryana kernel: kernel BUG at arch/x86/xen/mmu.c:1872!
Mar 9 00:24:10 memoryana kernel: invalid opcode: 0000 [#1] SMP
Mar 9 00:24:10 memoryana kernel: last sysfs file:
/sys/devices/pci0000:00/0000:00:07.0/0000:04:00.1/host3/rport-3:0-2/target3:0:2/3:0:2:0/state
Mar 9 00:24:10 memoryana kernel: CPU 1
Mar 9 00:24:10 memoryana kernel: Modules linked in: dm_round_robin
dm_multipath qla2xxx
Mar 9 00:24:10 memoryana kernel: Pid: 10662, comm: multipath-tools Not
tainted 2.6.32.28-xen0 #4 PowerEdge R610
Mar 9 00:24:10 memoryana kernel: RIP: e030:[<ffffffff8100d471>]
[<ffffffff8100d471>] pin_pagetable_pfn+0x31/0x60
Mar 9 00:24:10 memoryana kernel: RSP: e02b:ffff8800c3101df8 EFLAGS:
00010282
Mar 9 00:24:10 memoryana kernel: RAX: 00000000ffffffea RBX:
ffff8800cc4c3400 RCX: 0000000000000003
Mar 9 00:24:10 memoryana kernel: RDX: 0000000000000000 RSI:
0000000000000001 RDI: ffff8800c3101df8
Mar 9 00:24:10 memoryana kernel: RBP: ffff8800c3135b60 R08:
00003ffffffff000 R09: ffff880000000000
Mar 9 00:24:10 memoryana kernel: R10: 0000000000007ff0 R11:
0000000000000246 R12: 00000000000cc302
Mar 9 00:24:10 memoryana kernel: R13: 0000000000000000 R14:
ffff8800c374cc60 R15: ffff8800c374cc60
Mar 9 00:24:10 memoryana kernel: FS: 00007f60add15700(0000)
GS:ffff880028055000(0000) knlGS:0000000000000000
Mar 9 00:24:10 memoryana kernel: CS: e033 DS: 0000 ES: 0000 CR0:
000000008005003b
Mar 9 00:24:10 memoryana kernel: CR2: 00007f60ad841876 CR3:
00000000cef79000 CR4: 0000000000002660
Mar 9 00:24:10 memoryana kernel: DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
Mar 9 00:24:10 memoryana kernel: DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
Mar 9 00:24:10 memoryana kernel: Process multipath-tools (pid: 10662,
threadinfo ffff8800c3100000, task ffff8800cc01cbc0)
Mar 9 00:24:10 memoryana kernel: Stack:
Mar 9 00:24:10 memoryana kernel: 0000000000000000 00000000008e8302
ffff8800cc4c3400 ffff8800c3135b60
Mar 9 00:24:10 memoryana kernel: <0> 00000000000cc302 ffffffff810b0382
00007f60ad841876 ffff8800c30b4c10
Mar 9 00:24:10 memoryana kernel: <0> 00000000000100e0 0000000000000000
ffff8800c374cc60 ffffffff810b3595
Mar 9 00:24:10 memoryana kernel: Call Trace:
Mar 9 00:24:10 memoryana kernel: [<ffffffff810b0382>] ?
__pte_alloc+0xf2/0x120
Mar 9 00:24:10 memoryana kernel: [<ffffffff810b3595>] ?
handle_mm_fault+0xa45/0xab0
Mar 9 00:24:10 memoryana kernel: [<ffffffff8153cfe5>] ?
page_fault+0x25/0x30
Mar 9 00:24:10 memoryana kernel: [<ffffffff8153d21a>] ?
error_exit+0x2a/0x60
Mar 9 00:24:10 memoryana kernel: [<ffffffff8101481d>] ?
retint_restore_args+0x5/0x6
Mar 9 00:24:10 memoryana kernel: [<ffffffff81038631>] ?
do_page_fault+0x121/0x3c0
Mar 9 00:24:10 memoryana kernel: [<ffffffff812a2e0d>] ?
__put_user_4+0x1d/0x30
Mar 9 00:24:10 memoryana kernel: [<ffffffff8153cfe5>] ?
page_fault+0x25/0x30
Mar 9 00:24:10 memoryana kernel: Code: 57 c7 75 00 00 48 89 f0 89 3c 24
74 27 48 89 44 24 08 48 89 e7 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8
d3 be ff ff 85 c0 74 04 <0f> 0b eb fe 48 83 c4 28 c3 48 89 f7 e8 6e f7
ff ff 48 83 f8 ff
Mar 9 00:24:10 memoryana kernel: RIP [<ffffffff8100d471>]
pin_pagetable_pfn+0x31/0x60
Mar 9 00:24:10 memoryana kernel: RSP <ffff8800c3101df8>
Mar 9 00:24:10 memoryana kernel: ---[ end trace f4eae184c1a9f532 ]---
Mar 9 00:24:11 memoryana multipathd: --------shut down-------
--
Andreas Olsowski
Leuphana Universität Lüneburg
Rechen- und Medienzentrum
Scharnhorststraße 1, C7.015
21335 Lüneburg
Tel: ++49 4131 677 1309
[-- Attachment #1.2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 6595 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-08 20:20 ` Teck Choon Giam
2011-03-08 20:45 ` Guido Hecken
2011-03-08 20:50 ` [SPAM] " tjaouen
@ 2011-03-09 0:41 ` Konrad Rzeszutek Wilk
2 siblings, 0 replies; 85+ messages in thread
From: Konrad Rzeszutek Wilk @ 2011-03-09 0:41 UTC (permalink / raw)
To: Teck Choon Giam; +Cc: xen-devel, Fajar A. Nugraha
> > 4000000000000000: caf=8000000000000003 taf=4000000000000001
> > (XEN) mm.c:2825:d0 Error while installing new mfn 3b286
> > (XEN) mm.c:2794:d0 Mfn 35620 not pinned
> > (XEN) mm.c:2794:d0 Mfn 3b286 not pinned
> >
> > Takes a bit of time to reproduce it, but I can reproduce it on my CentOS
> > 5.5 OS.
> >
>
> Yep... it takes sometime to hit this bug and if you run my test script
> without any guest running will be faster to reproduce it.
>
> May I know what is your normal development platform/distribution for xen? I
It used to be Fedora Core 13, but then I switched to Ubuntu 10.10 and 11.04.
> hope this isn't just distribution specific issue for PVOPS
> stable/xen-2.6.32.x Dom0... ... mostly not as from this thread alone I see
> someone saying they hit this bug as well and I believe they are not running
> CentOS 5.x... guess is debian based distribution.
I am trying to eliminate this being a userspace tool version issue. I am running a
similar test on a Fedora Core 13 box.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-08 20:50 ` [SPAM] " tjaouen
2011-03-09 0:06 ` Andreas Olsowski
@ 2011-03-09 0:43 ` Konrad Rzeszutek Wilk
2011-03-09 6:58 ` Andreas Olsowski
1 sibling, 1 reply; 85+ messages in thread
From: Konrad Rzeszutek Wilk @ 2011-03-09 0:43 UTC (permalink / raw)
To: tjaouen; +Cc: xen-devel
On Tue, Mar 08, 2011 at 12:50:07PM -0800, tjaouen wrote:
>
> Teck Choon Giam wrote:
> >
> >
> > I hope this isn't just distribution specific issue for PVOPS
> > stable/xen-2.6.32.x Dom0... ... mostly not as from this thread alone I see
> > someone saying they hit this bug as well and I believe they are not
> > running
> > CentOS 5.x... guess is debian based distribution.
> >
> >
>
> Debian/Squeeze ""Stable""
>
> Debian GNU/Linux 6.0
> Linux xencave 2.6.32-5-xen-amd64 #1 SMP Wed Jan 12 05:46:49 UTC 2011 x86_64
> GNU/Linux
>
>
> March 1, I enter this command:
>
> lvcreate -s -L 5G -n databank_snap /dev/vgraid1/databank
>
> .... and BUG in /var/log/syslog:
Yup, looks like the same thing.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-09 0:43 ` [SPAM] " Konrad Rzeszutek Wilk
@ 2011-03-09 6:58 ` Andreas Olsowski
2011-03-09 15:00 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 85+ messages in thread
From: Andreas Olsowski @ 2011-03-09 6:58 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk, xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 5222 bytes --]
I encountered this bug in 2010 when xen 4.0 was released, around the
time development on 2.6.31 was halted.
That is why i stuck with 2.6.31 when everyone else went with 2.6.32,
because i determined 2.6.32 was not stable and im guessing it still isnt
today.
The bug occures on 2.6.32 xen kernels ( maybe even newer ones) and is
distribution unrelated, i was running debian 5.0 then, i am running 6.0
testing now and even have tried compiling all the userland stuff myself.
This is error can be encountered during a number of different actions:
1.) any action with lvm (start, stop, create, delete)
2.) while starting multipathd (restarting too, of course)
Sometimes the box only hangs there and no further device interactions
are possible. This is where i got my syslog entry from.
Some other times, processes like vgchange just ... hang, for no
particular reason.
Back in 2010 i had to serial console the server and stuff like that to
see the whole error.
If you tried to use anything that did sth with device mapper, it would
just ... hang. xm list for example.
my guess is everything one does with the device mapper can and will
trigger this sooner or later.
Does anybody have any kind of insight on what the problem may be?
------------
Here is my syslog part when i ran "/etc/init.d/multipath-tools restart":
Mar 9 00:24:10 memoryana multipathd: mpatha: stop event checker thread
(140606587918080)
Mar 9 00:24:10 memoryana multipathd: mpathb: stop event checker thread
(140606587885312)
Mar 9 00:24:10 memoryana multipathd: mpathc: stop event checker thread
(140606587852544)
Mar 9 00:24:10 memoryana kernel: ------------[ cut here ]------------
Mar 9 00:24:10 memoryana kernel: kernel BUG at arch/x86/xen/mmu.c:1872!
Mar 9 00:24:10 memoryana kernel: invalid opcode: 0000 [#1] SMP
Mar 9 00:24:10 memoryana kernel: last sysfs file:
/sys/devices/pci0000:00/0000:00:07.0/0000:04:00.1/host3/rport-3:0-2/target3:0:2/3:0:2:0/state
Mar 9 00:24:10 memoryana kernel: CPU 1
Mar 9 00:24:10 memoryana kernel: Modules linked in: dm_round_robin
dm_multipath qla2xxx
Mar 9 00:24:10 memoryana kernel: Pid: 10662, comm: multipath-tools Not
tainted 2.6.32.28-xen0 #4 PowerEdge R610
Mar 9 00:24:10 memoryana kernel: RIP: e030:[<ffffffff8100d471>]
[<ffffffff8100d471>] pin_pagetable_pfn+0x31/0x60
Mar 9 00:24:10 memoryana kernel: RSP: e02b:ffff8800c3101df8 EFLAGS:
00010282
Mar 9 00:24:10 memoryana kernel: RAX: 00000000ffffffea RBX:
ffff8800cc4c3400 RCX: 0000000000000003
Mar 9 00:24:10 memoryana kernel: RDX: 0000000000000000 RSI:
0000000000000001 RDI: ffff8800c3101df8
Mar 9 00:24:10 memoryana kernel: RBP: ffff8800c3135b60 R08:
00003ffffffff000 R09: ffff880000000000
Mar 9 00:24:10 memoryana kernel: R10: 0000000000007ff0 R11:
0000000000000246 R12: 00000000000cc302
Mar 9 00:24:10 memoryana kernel: R13: 0000000000000000 R14:
ffff8800c374cc60 R15: ffff8800c374cc60
Mar 9 00:24:10 memoryana kernel: FS: 00007f60add15700(0000)
GS:ffff880028055000(0000) knlGS:0000000000000000
Mar 9 00:24:10 memoryana kernel: CS: e033 DS: 0000 ES: 0000 CR0:
000000008005003b
Mar 9 00:24:10 memoryana kernel: CR2: 00007f60ad841876 CR3:
00000000cef79000 CR4: 0000000000002660
Mar 9 00:24:10 memoryana kernel: DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
Mar 9 00:24:10 memoryana kernel: DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
Mar 9 00:24:10 memoryana kernel: Process multipath-tools (pid: 10662,
threadinfo ffff8800c3100000, task ffff8800cc01cbc0)
Mar 9 00:24:10 memoryana kernel: Stack:
Mar 9 00:24:10 memoryana kernel: 0000000000000000 00000000008e8302
ffff8800cc4c3400 ffff8800c3135b60
Mar 9 00:24:10 memoryana kernel: <0> 00000000000cc302 ffffffff810b0382
00007f60ad841876 ffff8800c30b4c10
Mar 9 00:24:10 memoryana kernel: <0> 00000000000100e0 0000000000000000
ffff8800c374cc60 ffffffff810b3595
Mar 9 00:24:10 memoryana kernel: Call Trace:
Mar 9 00:24:10 memoryana kernel: [<ffffffff810b0382>] ?
__pte_alloc+0xf2/0x120
Mar 9 00:24:10 memoryana kernel: [<ffffffff810b3595>] ?
handle_mm_fault+0xa45/0xab0
Mar 9 00:24:10 memoryana kernel: [<ffffffff8153cfe5>] ?
page_fault+0x25/0x30
Mar 9 00:24:10 memoryana kernel: [<ffffffff8153d21a>] ?
error_exit+0x2a/0x60
Mar 9 00:24:10 memoryana kernel: [<ffffffff8101481d>] ?
retint_restore_args+0x5/0x6
Mar 9 00:24:10 memoryana kernel: [<ffffffff81038631>] ?
do_page_fault+0x121/0x3c0
Mar 9 00:24:10 memoryana kernel: [<ffffffff812a2e0d>] ?
__put_user_4+0x1d/0x30
Mar 9 00:24:10 memoryana kernel: [<ffffffff8153cfe5>] ?
page_fault+0x25/0x30
Mar 9 00:24:10 memoryana kernel: Code: 57 c7 75 00 00 48 89 f0 89 3c 24
74 27 48 89 44 24 08 48 89 e7 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8
d3 be ff ff 85 c0 74 04 <0f> 0b eb fe 48 83 c4 28 c3 48 89 f7 e8 6e f7
ff ff 48 83 f8 ff
Mar 9 00:24:10 memoryana kernel: RIP [<ffffffff8100d471>]
pin_pagetable_pfn+0x31/0x60
Mar 9 00:24:10 memoryana kernel: RSP <ffff8800c3101df8>
Mar 9 00:24:10 memoryana kernel: ---[ end trace f4eae184c1a9f532 ]---
Mar 9 00:24:11 memoryana multipathd: --------shut down-------
[-- Attachment #1.2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 6595 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-09 6:58 ` Andreas Olsowski
@ 2011-03-09 15:00 ` Konrad Rzeszutek Wilk
2011-03-09 19:59 ` Andreas Olsowski
0 siblings, 1 reply; 85+ messages in thread
From: Konrad Rzeszutek Wilk @ 2011-03-09 15:00 UTC (permalink / raw)
To: Andreas Olsowski; +Cc: xen-devel
On Wed, Mar 09, 2011 at 07:58:39AM +0100, Andreas Olsowski wrote:
>
> I encountered this bug in 2010 when xen 4.0 was released, around the
> time development on 2.6.31 was halted.
That is interesting data. Can you give more details on what 2.6.31 kernel
and hypervisor you are using? Have you tried to rev the hypervisor
up to Xen 4.1.0-rc7-pre for example?
>
> That is why i stuck with 2.6.31 when everyone else went with 2.6.32,
> because i determined 2.6.32 was not stable and im guessing it still
> isnt today.
>
> The bug occures on 2.6.32 xen kernels ( maybe even newer ones) and
Good point. Let me check 2.6.38.
> is distribution unrelated, i was running debian 5.0 then, i am
> running 6.0 testing now and even have tried compiling all the
> userland stuff myself.
<nods>The overnight tests on Fedora Core 13 showed the same failure
@ loop 97.
>
> This is error can be encountered during a number of different actions:
> 1.) any action with lvm (start, stop, create, delete)
> 2.) while starting multipathd (restarting too, of course)
>
> Sometimes the box only hangs there and no further device
> interactions are possible. This is where i got my syslog entry from.
> Some other times, processes like vgchange just ... hang, for no
> particular reason.
>
> Back in 2010 i had to serial console the server and stuff like that
> to see the whole error.
> If you tried to use anything that did sth with device mapper, it
> would just ... hang. xm list for example.
>
>
> my guess is everything one does with the device mapper can and will
> trigger this sooner or later.
>
> Does anybody have any kind of insight on what the problem may be?
The problem is that when an user application dies, the page-table is
discarded (which is a process of unpinning the pages) and we let the
pages be re-used. When another application is launched we construct a new
page-table (and pin the page-table) and when it exits we do the same thing.
The issue is that during the construction (the application just forked)
we encounter a page that used to belong to now a discarded page-table and Xen
(rightly) tells us that we are trying to pin an already pinned page.
Pinning here is the process of letting Xen inspect the pagetable so that
it can assert that there are no machine addresses that point to the
hypervisor or another guest.
Back to the problem. Xen tell us that we are trying to pin an
already pinned page _way_ after we had discarded (or thought we had)
the old page-tables - so finding the culprit of _why_ we missed a page
is difficult as it had happend in the past. Jeremy had gone over
with a fine comb over the code that deals with construction/deconstruction
and made sure there are no races but there is obviously something amiss.
>
>
> ------------
> Here is my syslog part when i ran "/etc/init.d/multipath-tools restart":
>
> Mar 9 00:24:10 memoryana multipathd: mpatha: stop event checker
> thread (140606587918080)
> Mar 9 00:24:10 memoryana multipathd: mpathb: stop event checker
> thread (140606587885312)
> Mar 9 00:24:10 memoryana multipathd: mpathc: stop event checker
> thread (140606587852544)
> Mar 9 00:24:10 memoryana kernel: ------------[ cut here ]------------
> Mar 9 00:24:10 memoryana kernel: kernel BUG at arch/x86/xen/mmu.c:1872!
> Mar 9 00:24:10 memoryana kernel: invalid opcode: 0000 [#1] SMP
> Mar 9 00:24:10 memoryana kernel: last sysfs file: /sys/devices/pci0000:00/0000:00:07.0/0000:04:00.1/host3/rport-3:0-2/target3:0:2/3:0:2:0/state
> Mar 9 00:24:10 memoryana kernel: CPU 1
> Mar 9 00:24:10 memoryana kernel: Modules linked in: dm_round_robin
> dm_multipath qla2xxx
> Mar 9 00:24:10 memoryana kernel: Pid: 10662, comm: multipath-tools
> Not tainted 2.6.32.28-xen0 #4 PowerEdge R610
> Mar 9 00:24:10 memoryana kernel: RIP: e030:[<ffffffff8100d471>]
> [<ffffffff8100d471>] pin_pagetable_pfn+0x31/0x60
> Mar 9 00:24:10 memoryana kernel: RSP: e02b:ffff8800c3101df8
> EFLAGS: 00010282
> Mar 9 00:24:10 memoryana kernel: RAX: 00000000ffffffea RBX:
> ffff8800cc4c3400 RCX: 0000000000000003
> Mar 9 00:24:10 memoryana kernel: RDX: 0000000000000000 RSI:
> 0000000000000001 RDI: ffff8800c3101df8
> Mar 9 00:24:10 memoryana kernel: RBP: ffff8800c3135b60 R08:
> 00003ffffffff000 R09: ffff880000000000
> Mar 9 00:24:10 memoryana kernel: R10: 0000000000007ff0 R11:
> 0000000000000246 R12: 00000000000cc302
> Mar 9 00:24:10 memoryana kernel: R13: 0000000000000000 R14:
> ffff8800c374cc60 R15: ffff8800c374cc60
> Mar 9 00:24:10 memoryana kernel: FS: 00007f60add15700(0000)
> GS:ffff880028055000(0000) knlGS:0000000000000000
> Mar 9 00:24:10 memoryana kernel: CS: e033 DS: 0000 ES: 0000 CR0:
> 000000008005003b
> Mar 9 00:24:10 memoryana kernel: CR2: 00007f60ad841876 CR3:
> 00000000cef79000 CR4: 0000000000002660
> Mar 9 00:24:10 memoryana kernel: DR0: 0000000000000000 DR1:
> 0000000000000000 DR2: 0000000000000000
> Mar 9 00:24:10 memoryana kernel: DR3: 0000000000000000 DR6:
> 00000000ffff0ff0 DR7: 0000000000000400
> Mar 9 00:24:10 memoryana kernel: Process multipath-tools (pid:
> 10662, threadinfo ffff8800c3100000, task ffff8800cc01cbc0)
> Mar 9 00:24:10 memoryana kernel: Stack:
> Mar 9 00:24:10 memoryana kernel: 0000000000000000 00000000008e8302
> ffff8800cc4c3400 ffff8800c3135b60
> Mar 9 00:24:10 memoryana kernel: <0> 00000000000cc302
> ffffffff810b0382 00007f60ad841876 ffff8800c30b4c10
> Mar 9 00:24:10 memoryana kernel: <0> 00000000000100e0
> 0000000000000000 ffff8800c374cc60 ffffffff810b3595
> Mar 9 00:24:10 memoryana kernel: Call Trace:
> Mar 9 00:24:10 memoryana kernel: [<ffffffff810b0382>] ?
> __pte_alloc+0xf2/0x120
> Mar 9 00:24:10 memoryana kernel: [<ffffffff810b3595>] ?
> handle_mm_fault+0xa45/0xab0
> Mar 9 00:24:10 memoryana kernel: [<ffffffff8153cfe5>] ?
> page_fault+0x25/0x30
> Mar 9 00:24:10 memoryana kernel: [<ffffffff8153d21a>] ?
> error_exit+0x2a/0x60
> Mar 9 00:24:10 memoryana kernel: [<ffffffff8101481d>] ?
> retint_restore_args+0x5/0x6
> Mar 9 00:24:10 memoryana kernel: [<ffffffff81038631>] ?
> do_page_fault+0x121/0x3c0
> Mar 9 00:24:10 memoryana kernel: [<ffffffff812a2e0d>] ?
> __put_user_4+0x1d/0x30
> Mar 9 00:24:10 memoryana kernel: [<ffffffff8153cfe5>] ?
> page_fault+0x25/0x30
> Mar 9 00:24:10 memoryana kernel: Code: 57 c7 75 00 00 48 89 f0 89
> 3c 24 74 27 48 89 44 24 08 48 89 e7 be 01 00 00 00 31 d2 41 ba f0 7f
> 00 00 e8 d3 be ff ff 85 c0 74 04 <0f> 0b eb fe 48 83 c4 28 c3 48 89
> f7 e8 6e f7 ff ff 48 83 f8 ff
> Mar 9 00:24:10 memoryana kernel: RIP [<ffffffff8100d471>]
> pin_pagetable_pfn+0x31/0x60
> Mar 9 00:24:10 memoryana kernel: RSP <ffff8800c3101df8>
> Mar 9 00:24:10 memoryana kernel: ---[ end trace f4eae184c1a9f532 ]---
> Mar 9 00:24:11 memoryana multipathd: --------shut down-------
>
>
>
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-09 15:00 ` Konrad Rzeszutek Wilk
@ 2011-03-09 19:59 ` Andreas Olsowski
2011-03-10 7:20 ` Andreas Olsowski
2011-03-10 13:45 ` Andreas Olsowski
0 siblings, 2 replies; 85+ messages in thread
From: Andreas Olsowski @ 2011-03-09 19:59 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk, xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 1715 bytes --]
> That is interesting data. Can you give more details on what 2.6.31 kernel
> and hypervisor you are using? Have you tried to rev the hypervisor
> up to Xen 4.1.0-rc7-pre for example?
I can do you one more, here are links to, kernel tarball as built by me
(2.6.31.14), its configuration and my xen4.0.0 directory.
Tomorrow i will test 4.1.0 and 2.6.38, allthough i have to say, if one
kernel works and the other doesnt, to me it seems like a kernel problem,
not one of the hypervisor. Of course, since i dont know the inner
workings of xen, that point of view might be ... flawed.
It would be nice if you could define a set of parameters that could
prove beneficial in provoking the bug as it would make it easier for me
to test different scenarios.
Or better yet a script that definitly will provoke it within a given
timeframe.
So far even restarting multipathd _could_ trigger it for me.
It also occured during bootup and when it did it kept on happening until
i did a cold restart of the server.
Does that make sense? Does data remain in the memory modules when i
reboot a system? (init 6)
And if that is so, whats the "Scrubbing memory ....." for that i see
when xen is loading?
One thing i would like to verify is that the bug only occurs when
running the kernel under xen and not when its running on its own.
I cant quite remember if i tried that in 2010.
Today i ran a loop of 300 lvcreate,snapshot,lvdelete on a standalone
2.6.32-xen0 kernel and did not receive an error. I didnt really have the
time to try and catch it that way running under Xen. I will do that
tomorrow.
I will report my findings.
best regards,
--
Andreas Olsowski
[-- Attachment #1.2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 6595 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-09 19:59 ` Andreas Olsowski
@ 2011-03-10 7:20 ` Andreas Olsowski
2011-03-10 13:45 ` Andreas Olsowski
1 sibling, 0 replies; 85+ messages in thread
From: Andreas Olsowski @ 2011-03-10 7:20 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk, xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 396 bytes --]
Whoops, forgot to put in the links.
These, in combination, will not produce the error:
http://141.39.208.101/linux-2.6.31.13-xen0.config
http://141.39.208.101/linux-2.6.31.13-xen0.tar.bz2
http://141.39.208.101/xen-4.0.0.tar.bz2
--
Andreas Olsowski
Leuphana Universität Lüneburg
Rechen- und Medienzentrum
Scharnhorststraße 1, C7.015
21335 Lüneburg
Tel: ++49 4131 677 1309
[-- Attachment #1.2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 6595 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-09 19:59 ` Andreas Olsowski
2011-03-10 7:20 ` Andreas Olsowski
@ 2011-03-10 13:45 ` Andreas Olsowski
2011-03-11 18:05 ` Konrad Rzeszutek Wilk
2011-03-16 15:52 ` [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860! - ideas Konrad Rzeszutek Wilk
1 sibling, 2 replies; 85+ messages in thread
From: Andreas Olsowski @ 2011-03-10 13:45 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk, xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 4892 bytes --]
All xen 4.1.0 test were done on server1 (netcatarina).
All but one test with xen 4.0.1 were made on server2 (memoryana).
Why i had to rerun one of the test for server2 on server1 is explained
below.
Here are my test results:
======================================================
Kernel 2.6.32.28 without XEN:
about 50 successful runs of Teck Choon Giams "test.sh" script.
(modified for handling 10 test volumes and sleeping 2 seconds)
multipathd restarted succesfully s
multipath module loaded/unloaded successfully
lvm2 restarted successfully
======================================================
Kernel 2.6.38 without XEN:
about 20 successful runs of "test.sh"
multipathd restarted succesfully s
multipath module loaded/unloaded successfully
lvm2 restarted successfully
======================================================
Kernel 2.6.32.28 with XEN 4.0.1:
at about loop 2 for volume 7 of "test.sh" it stopped doing ... well anything
there has been no output on the screen and neitehr syslog nor dmesg entry.
I left it hanging for about 15 Minutes until i decided to write this one
off as a side effect of the same underlying problem.
All lvm2 tools stopped working and i couldnt shut it down.
Killing the hangig process ended it properly.
I did a cold reset of the server, as i wanted to see the discussed BUG
again. But i failed here.
It would seem like my server2 has some kind of addressing error:
pci 000:04:00.1: BAR 6: address space collision of device ....
0000:04:00.1: is one of my QLogic HBAs
And since i use centralized FC storage ... who knows what side effects
happened here.
Interesting enough i had no problems with kernel 2.6.38 on this machine.
So i downgraded server1 that did never show this message to xen 4.0.1
and ran the test:
after 2 loops at volume 5 i hit "kernel BUG at arch/x86/xen/mmu.c" again.
======================================================
Kernel 2.6.38 with XEN 4.0.1:
100 runs of test.sh without error
multipathd restarted successfully
multipath module loaded/unloaded successfully
lvm2 stop/start ok
======================================================
Kernel 2.6.32.28 with XEN 4.1.0-rc7:
booted at first:
crash afer only 5 iterations of "test.sh"
http://pastebin.com/uNL7ehZ8
later, after having booted 2.6.38 on this server to test it with xen
4.1, i encountered different error at boottime:
BUG: unable to handle kernel paging request at ffff8800cc3e5f48
Only have pictures of it:
http://141.39.208.101/err1.png
http://141.39.208.101/err2.png
I then did a cold boot of the server, as this has proven to make it boot
in the past.
When this did not help, i stopped the test.sh running on my other
server, because the hang came when lvm2 was started and the servers use
shared storage.
Apparently this helped, the server booted fine after another cold reset.
After that i encountered an error again at loop 10 of "test.sh", but not
with the "kernel BUG at arch/x86/xen/mmu.c", but again, with
"BUG: unable to handle kernel paging request at ffff8800cc61ce010"
http://141.39.208.101/err3.png
http://141.39.208.101/err4.png
======================================================
Kernel 2.6.38 with XEN 4.1.0-rc7:
100 runs of test.sh without error
multipathd restarted successfully
multipath module loaded/unloaded successfully
lvm2 stop/start ok
======================================================
Summary
======================================================
So thats two different errors i have encountered,
one is the "kernel BUG at arch/x86/xen/mmu.c", the other is
"BUG: unablte to handle kernel paging request"
Both only apply to 2.6.32 when running under eitehr xen4.0.1 or 4.1.
On its own the kernel works fine.
Kernel 2.6.38 ran fine on both hypervisors as well as on its own.
One other issue occured that i didnt expect:
With the same .config (make oldconfig), 2.6.38 left my screen black
after loading the kernel, on both hypervisors.
The servers worked just fine, i just didnt see any output on their VGA
ports.
I hope this information helps you to hunt this bug down as it
effectively makes the "default" Xen unusable in server situations where
the device mapper is involved.
It is puzzling to me why noone did notice it last year, am i the only
one running xen on server hardware (Dell R610, 710 and 2950) with
centralized storage (FibreChannel or iSCSI) and using it as environment
for production.
Is multipathing two links to a centralized storage and using LVM2 to
split it up for virtual machines running on two or more servers really
such a rare thing to find Xen running on?
Btw, who is currently working on the remus implementation?
If you should need any more testing from me, feel free to ask.
Best regards.
--
Andreas Olsowski
[-- Attachment #1.2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 6595 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-10 13:45 ` Andreas Olsowski
@ 2011-03-11 18:05 ` Konrad Rzeszutek Wilk
2011-03-14 10:25 ` Ian Campbell
2011-03-16 15:52 ` [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860! - ideas Konrad Rzeszutek Wilk
1 sibling, 1 reply; 85+ messages in thread
From: Konrad Rzeszutek Wilk @ 2011-03-11 18:05 UTC (permalink / raw)
To: Andreas Olsowski; +Cc: xen-devel
> Both only apply to 2.6.32 when running under eitehr xen4.0.1 or 4.1.
OK, so Xen hypervisor is not at fault here.
> On its own the kernel works fine.
>
> Kernel 2.6.38 ran fine on both hypervisors as well as on its own.
Great!
>
> One other issue occured that i didnt expect:
> With the same .config (make oldconfig), 2.6.38 left my screen black
> after loading the kernel, on both hypervisors.
Um, that is surprising. If you have a radeon VGA driver, what happens
if you do 'radeon.modeset=0'?
> The servers worked just fine, i just didnt see any output on their
> VGA ports.
>
>
> I hope this information helps you to hunt this bug down as it
> effectively makes the "default" Xen unusable in server situations
> where the device mapper is involved.
>
> It is puzzling to me why noone did notice it last year, am i the
> only one running xen on server hardware (Dell R610, 710 and 2950)
> with centralized storage (FibreChannel or iSCSI) and using it as
> environment for production.
>
> Is multipathing two links to a centralized storage and using LVM2 to
> split it up for virtual machines running on two or more servers
> really such a rare thing to find Xen running on?
It is for kernel engineers. That equipment isn't cheap.
It is very difficult for us to fix bugs we don't see on our equipment.
>
> Btw, who is currently working on the remus implementation?
Frank Pan <frankpzh@gmail.com>
>
>
>
> If you should need any more testing from me, feel free to ask.
Ok.
>
> Best regards.
>
>
> --
> Andreas Olsowski
>
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-09 0:06 ` Andreas Olsowski
@ 2011-03-11 18:38 ` benco
2011-03-11 19:59 ` Sander Eikelenboom
0 siblings, 1 reply; 85+ messages in thread
From: benco @ 2011-03-11 18:38 UTC (permalink / raw)
To: Andreas Olsowski; +Cc: xen-devel
Hello,
I can confirm this bug, I'm using very similar configuration - Debian
Lenny/Squeeze on several servers connected to FC storage with enabled
multipathing.
One more thing - it was quite long ago, but I did not see this bug with Xen
4.0 pre-release versions with 2.6.31 kernel from Jeremy's tree. I'm
regularly updating my system to up-to-date version of Xen/PV-OPS kernel and
this bug is here across whole development line from Xen 4.0/2.6.32 kernel
until now:(
Roman
On Wed, Mar 09, 2011 at 01:06:39AM +0100, Andreas Olsowski wrote:
> Well, this is too bad.
>
> I encountered this bug when xen 4.0 was released, around the time
> development on 2.6.31 was halted.
>
> That is why i stuck with 2.6.31 when everyone else went with 2.6.32,
> because i determined 2.6.32 was not fit for duty and im guessing it
> still isnt today.
>
> The bug occures on 2.6.32 xen kernels ( maybe even newer ones) and is
> distribution unrelated, i was running debian 5.0 then, i am running 6.0
> testing now and even have tried compiling all the userland stuff myself.
>
> This is error can be encountered during a number of different actions:
> 1.) any action with lvm (start, stop, create, delete)
> 2.) while starting multipathd (restarting too, of course)
>
> Sometimes the box only hangs there and no further device mapper
> interactions are possible. This is where i got my syslog entry from.
>
> Back in 2010 i had to serial console the server and stuff like that to
> see the whole error.
>
>
> my guess is everything one does with the device mapper can and will
> trigger this sooner or later.
>
> Does anybody have any kind of insight on what the problem may be?
>
> ------------
> Here is my syslog part when i ran "/etc/init.d/multipath-tools restart":
>
> Mar 9 00:24:10 memoryana multipathd: mpatha: stop event checker thread
> (140606587918080)
> Mar 9 00:24:10 memoryana multipathd: mpathb: stop event checker thread
> (140606587885312)
> Mar 9 00:24:10 memoryana multipathd: mpathc: stop event checker thread
> (140606587852544)
> Mar 9 00:24:10 memoryana kernel: ------------[ cut here ]------------
> Mar 9 00:24:10 memoryana kernel: kernel BUG at arch/x86/xen/mmu.c:1872!
> Mar 9 00:24:10 memoryana kernel: invalid opcode: 0000 [#1] SMP
> Mar 9 00:24:10 memoryana kernel: last sysfs file:
> /sys/devices/pci0000:00/0000:00:07.0/0000:04:00.1/host3/rport-3:0-2/target3:0:2/3:0:2:0/state
> Mar 9 00:24:10 memoryana kernel: CPU 1
> Mar 9 00:24:10 memoryana kernel: Modules linked in: dm_round_robin
> dm_multipath qla2xxx
> Mar 9 00:24:10 memoryana kernel: Pid: 10662, comm: multipath-tools Not
> tainted 2.6.32.28-xen0 #4 PowerEdge R610
> Mar 9 00:24:10 memoryana kernel: RIP: e030:[<ffffffff8100d471>]
> [<ffffffff8100d471>] pin_pagetable_pfn+0x31/0x60
> Mar 9 00:24:10 memoryana kernel: RSP: e02b:ffff8800c3101df8 EFLAGS:
> 00010282
> Mar 9 00:24:10 memoryana kernel: RAX: 00000000ffffffea RBX:
> ffff8800cc4c3400 RCX: 0000000000000003
> Mar 9 00:24:10 memoryana kernel: RDX: 0000000000000000 RSI:
> 0000000000000001 RDI: ffff8800c3101df8
> Mar 9 00:24:10 memoryana kernel: RBP: ffff8800c3135b60 R08:
> 00003ffffffff000 R09: ffff880000000000
> Mar 9 00:24:10 memoryana kernel: R10: 0000000000007ff0 R11:
> 0000000000000246 R12: 00000000000cc302
> Mar 9 00:24:10 memoryana kernel: R13: 0000000000000000 R14:
> ffff8800c374cc60 R15: ffff8800c374cc60
> Mar 9 00:24:10 memoryana kernel: FS: 00007f60add15700(0000)
> GS:ffff880028055000(0000) knlGS:0000000000000000
> Mar 9 00:24:10 memoryana kernel: CS: e033 DS: 0000 ES: 0000 CR0:
> 000000008005003b
> Mar 9 00:24:10 memoryana kernel: CR2: 00007f60ad841876 CR3:
> 00000000cef79000 CR4: 0000000000002660
> Mar 9 00:24:10 memoryana kernel: DR0: 0000000000000000 DR1:
> 0000000000000000 DR2: 0000000000000000
> Mar 9 00:24:10 memoryana kernel: DR3: 0000000000000000 DR6:
> 00000000ffff0ff0 DR7: 0000000000000400
> Mar 9 00:24:10 memoryana kernel: Process multipath-tools (pid: 10662,
> threadinfo ffff8800c3100000, task ffff8800cc01cbc0)
> Mar 9 00:24:10 memoryana kernel: Stack:
> Mar 9 00:24:10 memoryana kernel: 0000000000000000 00000000008e8302
> ffff8800cc4c3400 ffff8800c3135b60
> Mar 9 00:24:10 memoryana kernel: <0> 00000000000cc302 ffffffff810b0382
> 00007f60ad841876 ffff8800c30b4c10
> Mar 9 00:24:10 memoryana kernel: <0> 00000000000100e0 0000000000000000
> ffff8800c374cc60 ffffffff810b3595
> Mar 9 00:24:10 memoryana kernel: Call Trace:
> Mar 9 00:24:10 memoryana kernel: [<ffffffff810b0382>] ?
> __pte_alloc+0xf2/0x120
> Mar 9 00:24:10 memoryana kernel: [<ffffffff810b3595>] ?
> handle_mm_fault+0xa45/0xab0
> Mar 9 00:24:10 memoryana kernel: [<ffffffff8153cfe5>] ?
> page_fault+0x25/0x30
> Mar 9 00:24:10 memoryana kernel: [<ffffffff8153d21a>] ?
> error_exit+0x2a/0x60
> Mar 9 00:24:10 memoryana kernel: [<ffffffff8101481d>] ?
> retint_restore_args+0x5/0x6
> Mar 9 00:24:10 memoryana kernel: [<ffffffff81038631>] ?
> do_page_fault+0x121/0x3c0
> Mar 9 00:24:10 memoryana kernel: [<ffffffff812a2e0d>] ?
> __put_user_4+0x1d/0x30
> Mar 9 00:24:10 memoryana kernel: [<ffffffff8153cfe5>] ?
> page_fault+0x25/0x30
> Mar 9 00:24:10 memoryana kernel: Code: 57 c7 75 00 00 48 89 f0 89 3c 24
> 74 27 48 89 44 24 08 48 89 e7 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8
> d3 be ff ff 85 c0 74 04 <0f> 0b eb fe 48 83 c4 28 c3 48 89 f7 e8 6e f7
> ff ff 48 83 f8 ff
> Mar 9 00:24:10 memoryana kernel: RIP [<ffffffff8100d471>]
> pin_pagetable_pfn+0x31/0x60
> Mar 9 00:24:10 memoryana kernel: RSP <ffff8800c3101df8>
> Mar 9 00:24:10 memoryana kernel: ---[ end trace f4eae184c1a9f532 ]---
> Mar 9 00:24:11 memoryana multipathd: --------shut down-------
>
> --
> Andreas Olsowski
> Leuphana Universität Lüneburg
> Rechen- und Medienzentrum
> Scharnhorststraße 1, C7.015
> 21335 Lüneburg
>
> Tel: ++49 4131 677 1309
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
--
----------------------------------------------------------------------
,''`. [benco] | mailto: benco@acid.sk | silc: /msg benco
: :' : -------------------------------------------------------------
`. `' GPG publickey: http://www.acid.sk/pubkey.asc
`- KF = 0DF6 0592 74D2 F17A DACF A5C3 1720 CB7C F54C F429
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-11 18:38 ` benco
@ 2011-03-11 19:59 ` Sander Eikelenboom
2011-03-11 20:29 ` Teck Choon Giam
2011-03-11 20:45 ` Teck Choon Giam
0 siblings, 2 replies; 85+ messages in thread
From: Sander Eikelenboom @ 2011-03-11 19:59 UTC (permalink / raw)
To: benco; +Cc: Andreas Olsowski, xen-devel
Haven't seen this one on my systen. I do use LVM, but i'm only using LVM as device mapper, without software raid etc.
Also using debian lenny/squeeze together with custom compiled xen and xen-2.6.32.x pvops kernel
So I would suspect the multipathing / raid part.
--
Sander
Friday, March 11, 2011, 7:38:00 PM, you wrote:
> Hello,
> I can confirm this bug, I'm using very similar configuration - Debian
> Lenny/Squeeze on several servers connected to FC storage with enabled
> multipathing.
> One more thing - it was quite long ago, but I did not see this bug with Xen
> 4.0 pre-release versions with 2.6.31 kernel from Jeremy's tree. I'm
> regularly updating my system to up-to-date version of Xen/PV-OPS kernel and
> this bug is here across whole development line from Xen 4.0/2.6.32 kernel
> until now:(
> Roman
> On Wed, Mar 09, 2011 at 01:06:39AM +0100, Andreas Olsowski wrote:
>> Well, this is too bad.
>>
>> I encountered this bug when xen 4.0 was released, around the time
>> development on 2.6.31 was halted.
>>
>> That is why i stuck with 2.6.31 when everyone else went with 2.6.32,
>> because i determined 2.6.32 was not fit for duty and im guessing it
>> still isnt today.
>>
>> The bug occures on 2.6.32 xen kernels ( maybe even newer ones) and is
>> distribution unrelated, i was running debian 5.0 then, i am running 6.0
>> testing now and even have tried compiling all the userland stuff myself.
>>
>> This is error can be encountered during a number of different actions:
>> 1.) any action with lvm (start, stop, create, delete)
>> 2.) while starting multipathd (restarting too, of course)
>>
>> Sometimes the box only hangs there and no further device mapper
>> interactions are possible. This is where i got my syslog entry from.
>>
>> Back in 2010 i had to serial console the server and stuff like that to
>> see the whole error.
>>
>>
>> my guess is everything one does with the device mapper can and will
>> trigger this sooner or later.
>>
>> Does anybody have any kind of insight on what the problem may be?
>>
>> ------------
>> Here is my syslog part when i ran "/etc/init.d/multipath-tools restart":
>>
>> Mar 9 00:24:10 memoryana multipathd: mpatha: stop event checker thread
>> (140606587918080)
>> Mar 9 00:24:10 memoryana multipathd: mpathb: stop event checker thread
>> (140606587885312)
>> Mar 9 00:24:10 memoryana multipathd: mpathc: stop event checker thread
>> (140606587852544)
>> Mar 9 00:24:10 memoryana kernel: ------------[ cut here ]------------
>> Mar 9 00:24:10 memoryana kernel: kernel BUG at arch/x86/xen/mmu.c:1872!
>> Mar 9 00:24:10 memoryana kernel: invalid opcode: 0000 [#1] SMP
>> Mar 9 00:24:10 memoryana kernel: last sysfs file:
>> /sys/devices/pci0000:00/0000:00:07.0/0000:04:00.1/host3/rport-3:0-2/target3:0:2/3:0:2:0/state
>> Mar 9 00:24:10 memoryana kernel: CPU 1
>> Mar 9 00:24:10 memoryana kernel: Modules linked in: dm_round_robin
>> dm_multipath qla2xxx
>> Mar 9 00:24:10 memoryana kernel: Pid: 10662, comm: multipath-tools Not
>> tainted 2.6.32.28-xen0 #4 PowerEdge R610
>> Mar 9 00:24:10 memoryana kernel: RIP: e030:[<ffffffff8100d471>]
>> [<ffffffff8100d471>] pin_pagetable_pfn+0x31/0x60
>> Mar 9 00:24:10 memoryana kernel: RSP: e02b:ffff8800c3101df8 EFLAGS:
>> 00010282
>> Mar 9 00:24:10 memoryana kernel: RAX: 00000000ffffffea RBX:
>> ffff8800cc4c3400 RCX: 0000000000000003
>> Mar 9 00:24:10 memoryana kernel: RDX: 0000000000000000 RSI:
>> 0000000000000001 RDI: ffff8800c3101df8
>> Mar 9 00:24:10 memoryana kernel: RBP: ffff8800c3135b60 R08:
>> 00003ffffffff000 R09: ffff880000000000
>> Mar 9 00:24:10 memoryana kernel: R10: 0000000000007ff0 R11:
>> 0000000000000246 R12: 00000000000cc302
>> Mar 9 00:24:10 memoryana kernel: R13: 0000000000000000 R14:
>> ffff8800c374cc60 R15: ffff8800c374cc60
>> Mar 9 00:24:10 memoryana kernel: FS: 00007f60add15700(0000)
>> GS:ffff880028055000(0000) knlGS:0000000000000000
>> Mar 9 00:24:10 memoryana kernel: CS: e033 DS: 0000 ES: 0000 CR0:
>> 000000008005003b
>> Mar 9 00:24:10 memoryana kernel: CR2: 00007f60ad841876 CR3:
>> 00000000cef79000 CR4: 0000000000002660
>> Mar 9 00:24:10 memoryana kernel: DR0: 0000000000000000 DR1:
>> 0000000000000000 DR2: 0000000000000000
>> Mar 9 00:24:10 memoryana kernel: DR3: 0000000000000000 DR6:
>> 00000000ffff0ff0 DR7: 0000000000000400
>> Mar 9 00:24:10 memoryana kernel: Process multipath-tools (pid: 10662,
>> threadinfo ffff8800c3100000, task ffff8800cc01cbc0)
>> Mar 9 00:24:10 memoryana kernel: Stack:
>> Mar 9 00:24:10 memoryana kernel: 0000000000000000 00000000008e8302
>> ffff8800cc4c3400 ffff8800c3135b60
>> Mar 9 00:24:10 memoryana kernel: <0> 00000000000cc302 ffffffff810b0382
>> 00007f60ad841876 ffff8800c30b4c10
>> Mar 9 00:24:10 memoryana kernel: <0> 00000000000100e0 0000000000000000
>> ffff8800c374cc60 ffffffff810b3595
>> Mar 9 00:24:10 memoryana kernel: Call Trace:
>> Mar 9 00:24:10 memoryana kernel: [<ffffffff810b0382>] ?
>> __pte_alloc+0xf2/0x120
>> Mar 9 00:24:10 memoryana kernel: [<ffffffff810b3595>] ?
>> handle_mm_fault+0xa45/0xab0
>> Mar 9 00:24:10 memoryana kernel: [<ffffffff8153cfe5>] ?
>> page_fault+0x25/0x30
>> Mar 9 00:24:10 memoryana kernel: [<ffffffff8153d21a>] ?
>> error_exit+0x2a/0x60
>> Mar 9 00:24:10 memoryana kernel: [<ffffffff8101481d>] ?
>> retint_restore_args+0x5/0x6
>> Mar 9 00:24:10 memoryana kernel: [<ffffffff81038631>] ?
>> do_page_fault+0x121/0x3c0
>> Mar 9 00:24:10 memoryana kernel: [<ffffffff812a2e0d>] ?
>> __put_user_4+0x1d/0x30
>> Mar 9 00:24:10 memoryana kernel: [<ffffffff8153cfe5>] ?
>> page_fault+0x25/0x30
>> Mar 9 00:24:10 memoryana kernel: Code: 57 c7 75 00 00 48 89 f0 89 3c 24
>> 74 27 48 89 44 24 08 48 89 e7 be 01 00 00 00 31 d2 41 ba f0 7f 00 00 e8
>> d3 be ff ff 85 c0 74 04 <0f> 0b eb fe 48 83 c4 28 c3 48 89 f7 e8 6e f7
>> ff ff 48 83 f8 ff
>> Mar 9 00:24:10 memoryana kernel: RIP [<ffffffff8100d471>]
>> pin_pagetable_pfn+0x31/0x60
>> Mar 9 00:24:10 memoryana kernel: RSP <ffff8800c3101df8>
>> Mar 9 00:24:10 memoryana kernel: ---[ end trace f4eae184c1a9f532 ]---
>> Mar 9 00:24:11 memoryana multipathd: --------shut down-------
>>
>> --
>> Andreas Olsowski
>> Leuphana Universität Lüneburg
>> Rechen- und Medienzentrum
>> Scharnhorststraße 1, C7.015
>> 21335 Lüneburg
>>
>> Tel: ++49 4131 677 1309
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
--
Best regards,
Sander mailto:linux@eikelenboom.it
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-11 19:59 ` Sander Eikelenboom
@ 2011-03-11 20:29 ` Teck Choon Giam
2011-03-11 20:45 ` Teck Choon Giam
1 sibling, 0 replies; 85+ messages in thread
From: Teck Choon Giam @ 2011-03-11 20:29 UTC (permalink / raw)
To: Sander Eikelenboom; +Cc: Andreas Olsowski, xen-devel, benco
[-- Attachment #1.1: Type: text/plain, Size: 39231 bytes --]
On Sat, Mar 12, 2011 at 3:59 AM, Sander Eikelenboom <linux@eikelenboom.it>wrote:
> Haven't seen this one on my systen. I do use LVM, but i'm only using LVM as
> device mapper, without software raid etc.
> Also using debian lenny/squeeze together with custom compiled xen and
> xen-2.6.32.x pvops kernel
>
> So I would suspect the multipathing / raid part.
>
>From one of my test server, below is grep output:
# grep -A 45 'cut here' /var/log/messages*|grep 'kernel: Process'
/var/log/messages-Mar 10 00:53:00 xen06 kernel: Process mpath_wait (pid:
9513, threadinfo ffff880010f38000, task ffff880010c4c100)
/var/log/messages-Mar 10 02:52:33 xen06 kernel: Process clvmd (pid: 3240,
threadinfo ffff880035a14000, task ffff88003b49c340)
/var/log/messages-Mar 10 02:52:33 xen06 kernel: Process syslogd (pid: 24279,
threadinfo ffff88003bb3e000, task ffff880016f3a1c0)
/var/log/messages-Mar 10 03:40:52 xen06 kernel: Process mpath_wait (pid:
16487, threadinfo ffff88003becc000, task ffff88003bdb8480)
/var/log/messages.2-Feb 23 22:56:56 xen06 kernel: Process mpath_wait (pid:
15222, threadinfo ffff8800265b8000, task ffff88003b940200)
/var/log/messages.2-Feb 24 01:03:17 xen06 kernel: Process dmsetup (pid: 509,
threadinfo ffff88003ced4000, task ffff88003025c4c0)
/var/log/messages.2-Feb 26 23:20:22 xen06 kernel: Process mpath_wait (pid:
10154, threadinfo ffff880025604000, task ffff8800254804c0)
/var/log/messages.4-Feb 8 04:30:10 xen06 kernel: Process mpath_wait (pid:
11646, threadinfo ffff88001e722000, task ffff880025222440)
/var/log/messages.4-Feb 9 16:43:11 xen06 kernel: Process lvremove (pid:
24469, threadinfo ffff88003b930000, task ffff880025ab03c0)
/var/log/messages.4-Feb 9 22:02:09 xen06 kernel: Process grep (pid: 10327,
threadinfo ffff880025056000, task ffff880024814240)
/var/log/messages.4-Feb 9 22:13:37 xen06 kernel: Process dmsetup (pid:
6287, threadinfo ffff88003cf64000, task ffff880025fe8740)
/var/log/messages.4-Feb 9 22:16:55 xen06 kernel: Process clvmd (pid: 3204,
threadinfo ffff88003ccb8000, task ffff88003a436440)
Whereby output related to 'kernel: BUG: unable to handle kernel paging
request at':
# grep -B 20 -A 50 'kernel: BUG: unable to handle kernel paging request at'
/var/log/messages*
/var/log/messages.2-Feb 24 00:59:13 xen06 lvm[2379]: No longer monitoring
snapshot XenGroup-testcrash5--snapshot
/var/log/messages.2-Feb 24 00:59:14 xen06 lvm[2379]: Monitoring snapshot
XenGroup-testcrash1--snapshot
/var/log/messages.2-Feb 24 00:59:14 xen06 kernel: kjournald starting.
Commit interval 5 seconds
/var/log/messages.2-Feb 24 00:59:14 xen06 kernel: EXT3 FS on dm-0, internal
journal
/var/log/messages.2-Feb 24 00:59:14 xen06 kernel: EXT3-fs: mounted
filesystem with ordered data mode.
/var/log/messages.2-Feb 24 00:59:14 xen06 lvm[2379]: No longer monitoring
snapshot XenGroup-testcrash1--snapshot
/var/log/messages.2-Feb 24 00:59:15 xen06 lvm[2379]: Monitoring snapshot
XenGroup-testcrash2--snapshot
/var/log/messages.2-Feb 24 00:59:16 xen06 kernel: kjournald starting.
Commit interval 5 seconds
/var/log/messages.2-Feb 24 00:59:16 xen06 kernel: EXT3 FS on dm-0, internal
journal
/var/log/messages.2-Feb 24 00:59:16 xen06 kernel: EXT3-fs: mounted
filesystem with ordered data mode.
/var/log/messages.2-Feb 24 00:59:16 xen06 lvm[2379]: No longer monitoring
snapshot XenGroup-testcrash2--snapshot
/var/log/messages.2-Feb 24 00:59:17 xen06 lvm[2379]: Monitoring snapshot
XenGroup-testcrash3--snapshot
/var/log/messages.2-Feb 24 00:59:17 xen06 kernel: kjournald starting.
Commit interval 5 seconds
/var/log/messages.2-Feb 24 00:59:17 xen06 kernel: EXT3 FS on dm-0, internal
journal
/var/log/messages.2-Feb 24 00:59:17 xen06 kernel: EXT3-fs: mounted
filesystem with ordered data mode.
/var/log/messages.2-Feb 24 00:59:18 xen06 lvm[2379]: No longer monitoring
snapshot XenGroup-testcrash3--snapshot
/var/log/messages.2-Feb 24 00:59:19 xen06 lvm[2379]: Monitoring snapshot
XenGroup-testcrash4--snapshot
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: kjournald starting.
Commit interval 5 seconds
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: EXT3 FS on dm-0, internal
journal
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: EXT3-fs: mounted
filesystem with ordered data mode.
/var/log/messages.2:Feb 24 00:59:19 xen06 kernel: BUG: unable to handle
kernel paging request at ffff88002561f3f8
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: IP: [<ffffffff8100e2d4>]
xen_set_pmd+0x16/0x2b
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: PGD 1002067 PUD 1006067
PMD 1f1067 PTE 801000002561f065
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: Oops: 0003 [#1] SMP
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: last sysfs file:
/sys/block/dm-13/dev
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: CPU 1
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: Modules linked in: dlm
configfs xt_physdev iptable_filter ip_tables x_tables bridge stp be2iscsi
iscsi_tcp bnx2i cnic uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi
scsi_transport_iscsi dm_multipath scsi_dh video backlight output sbs sbshc
power_meter hwmon battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc
lp parport sg ide_cd_mod cdrom tg3 serio_raw libphy button tpm_tis tpm
tpm_bios pcspkr shpchp i2c_i801 i2c_core iTCO_wdt dm_snapshot dm_zero
dm_mirror dm_region_hash dm_log dm_mod ata_piix libata sd_mod scsi_mod raid1
ext3 jbd uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: Pid: 16520, comm: udevd
Not tainted 2.6.32.29-0.xen.pvops.choon.centos5 #1 PowerEdge 860
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: RIP:
e030:[<ffffffff8100e2d4>] [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: RSP:
e02b:ffff880025655b88 EFLAGS: 00010246
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: RAX: 0000000000000000 RBX:
ffff88002561f3f8 RCX: 00007f9a10c63000
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: RDX: ffffea0000000000 RSI:
0000000000000000 RDI: ffff88002561f3f8
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: RBP: ffff880025655b98 R08:
00007f9a10fc0000 R09: ffff88003e64d609
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: R10: ffff88003cce22c0 R11:
ffffffff813391a0 R12: 0000000000000000
/var/log/messages.2-Feb 24 00:59:19 xen06 lvm[2379]: No longer monitoring
snapshot XenGroup-testcrash4--snapshot
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: R13: 00007f9a10000000 R14:
ffff88003cc6a7f8 R15: ffff88002550c340
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: FS:
00007f9a10a41710(0000) GS:ffff88002806c000(0000) knlGS:0000000000000000
/var/log/messages.2-Feb 24 00:59:19 xen06 udevd-event[16460]: run_program:
'/sbin/dmsetup' abnormal exit
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: CS: e033 DS: 0000 ES:
0000 CR0: 000000008005003b
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: CR2: ffff88002561f3f8 CR3:
000000003bb46000 CR4: 0000000000002660
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: Process udevd (pid: 16520,
threadinfo ffff880025654000, task ffff88003cce22c0)
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: Stack:
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: ffff88002561f3f8
0000000025690067 ffff880025655c48 ffffffff810a60b0
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: <0> ffff880025655ac8
000000000000001b 00007f9a10e00000 0000000000000000
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: <0> 00007f9a10c63000
ffff88002807a5c0 00007f9a10c63000 ffff880025655cb8
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: Call Trace:
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: [<ffffffff810a60b0>]
free_pgd_range+0x3b3/0x401
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: [<ffffffff810a63b1>]
free_pgtables+0xb6/0xce
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: [<ffffffff810a84a4>]
exit_mmap+0xf0/0x14b
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: [<ffffffff81048888>]
mmput+0x46/0xb2
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: [<ffffffff810c8cd9>]
flush_old_exec+0x463/0x536
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: [<ffffffff810fbf99>]
load_elf_binary+0x365/0x171b
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: [<ffffffff810faa30>] ?
load_misc_binary+0x61/0x322
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: [<ffffffff810a50dd>] ?
get_user_pages+0x44/0x46
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: [<ffffffff810c795e>] ?
put_arg_page+0x9/0xb
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: [<ffffffff810c7e0b>]
search_binary_handler+0xde/0x268
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: [<ffffffff810c9208>]
do_execve+0x193/0x262
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: [<ffffffff810113af>]
sys_execve+0x3e/0x58
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: [<ffffffff81012f1a>]
stub_execve+0x6a/0xc0
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: Code: 8b 3c 25 70 c5 00 00
ff 14 25 68 a2 5c 81 48 83 c4 18 5b c9 c3 55 48 89 e5 41 54 49 89 f4 53 48
89 fb e8 79 e3 ff ff 84 c0 75 05 <4c> 89 23 eb 0b 4c 89 e6 48 89 df e8 69 ff
ff ff 5b 41 5c c9 c3
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: RIP [<ffffffff8100e2d4>]
xen_set_pmd+0x16/0x2b
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: RSP <ffff880025655b88>
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: CR2: ffff88002561f3f8
/var/log/messages.2-Feb 24 00:59:19 xen06 kernel: ---[ end trace
5ea31e622470b518 ]---
/var/log/messages.2-Feb 24 00:59:20 xen06 lvm[2379]: Monitoring snapshot
XenGroup-testcrash5--snapshot
/var/log/messages.2-Feb 24 00:59:21 xen06 kernel: kjournald starting.
Commit interval 5 seconds
/var/log/messages.2-Feb 24 00:59:21 xen06 kernel: EXT3 FS on dm-0, internal
journal
/var/log/messages.2-Feb 24 00:59:21 xen06 kernel: EXT3-fs: mounted
filesystem with ordered data mode.
/var/log/messages.2-Feb 24 00:59:21 xen06 lvm[2379]: No longer monitoring
snapshot XenGroup-testcrash5--snapshot
--
/var/log/messages.4-Feb 8 03:58:30 xen06 lvm[3961]: Monitoring snapshot
XenGroup-testcrash4--snapshot
/var/log/messages.4-Feb 8 03:58:30 xen06 kernel: kjournald starting.
Commit interval 5 seconds
/var/log/messages.4-Feb 8 03:58:30 xen06 kernel: EXT3 FS on dm-11, internal
journal
/var/log/messages.4-Feb 8 03:58:30 xen06 kernel: EXT3-fs: mounted
filesystem with ordered data mode.
/var/log/messages.4-Feb 8 03:58:30 xen06 lvm[3961]: No longer monitoring
snapshot XenGroup-testcrash4--snapshot
/var/log/messages.4-Feb 8 03:58:31 xen06 lvm[3961]: Monitoring snapshot
XenGroup-testcrash5--snapshot
/var/log/messages.4-Feb 8 03:58:31 xen06 kernel: kjournald starting.
Commit interval 5 seconds
/var/log/messages.4-Feb 8 03:58:31 xen06 kernel: EXT3 FS on dm-11, internal
journal
/var/log/messages.4-Feb 8 03:58:31 xen06 kernel: EXT3-fs: mounted
filesystem with ordered data mode.
/var/log/messages.4-Feb 8 03:58:32 xen06 lvm[3961]: No longer monitoring
snapshot XenGroup-testcrash5--snapshot
/var/log/messages.4-Feb 8 03:58:33 xen06 lvm[3961]: Monitoring snapshot
XenGroup-testcrash1--snapshot
/var/log/messages.4-Feb 8 03:58:33 xen06 kernel: kjournald starting.
Commit interval 5 seconds
/var/log/messages.4-Feb 8 03:58:33 xen06 kernel: EXT3 FS on dm-11, internal
journal
/var/log/messages.4-Feb 8 03:58:33 xen06 kernel: EXT3-fs: mounted
filesystem with ordered data mode.
/var/log/messages.4-Feb 8 03:58:33 xen06 lvm[3961]: No longer monitoring
snapshot XenGroup-testcrash1--snapshot
/var/log/messages.4-Feb 8 03:58:34 xen06 lvm[3961]: Monitoring snapshot
XenGroup-testcrash2--snapshot
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: kjournald starting.
Commit interval 5 seconds
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: EXT3 FS on dm-11, internal
journal
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: EXT3-fs: mounted
filesystem with ordered data mode.
/var/log/messages.4-Feb 8 03:58:34 xen06 lvm[3961]: No longer monitoring
snapshot XenGroup-testcrash2--snapshot
/var/log/messages.4:Feb 8 03:58:34 xen06 kernel: BUG: unable to handle
kernel paging request at ffff880030081010
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: IP: [<ffffffff8100e2d4>]
xen_set_pmd+0x16/0x2b
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: PGD 1002067 PUD 1006067
PMD 246067 PTE 8010000030081065
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: Oops: 0003 [#1] SMP
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: last sysfs file:
/sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_map
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: CPU 3
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: Modules linked in:
ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state
nf_conntrack ipt_REJECT xt_tcpudp gfs2 dlm configfs xt_physdev
iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic
uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi
dm_multipath scsi_dh video backlight output sbs sbshc power_meter hwmon
battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc lp parport sg
ide_cd_mod cdrom serio_raw tg3 button libphy tpm_tis tpm tpm_bios i2c_i801
pcspkr i2c_core iTCO_wdt shpchp dm_snapshot dm_zero dm_mirror dm_region_hash
dm_log dm_mod ata_piix libata sd_mod scsi_mod raid1 ext3 jbd uhci_hcd
ohci_hcd ehci_hcd [last unloaded: microcode]
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: Pid: 32051, comm:
mpath_wait Not tainted 2.6.32.28-0.xen.pvops.choon.centos5 #1 PowerEdge 860
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: RIP:
e030:[<ffffffff8100e2d4>] [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: RSP:
e02b:ffff880030c19b88 EFLAGS: 00010246
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: RAX: 0000000000000000 RBX:
ffff880030081010 RCX: 00000000008c3000
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: RDX: ffffea0000000000 RSI:
0000000000000000 RDI: ffff880030081010
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: RBP: ffff880030c19b98 R08:
0000000002150000 R09: ffff88003e6f3e89
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: R10: ffff880030e78040 R11:
ffffffff813381a0 R12: 0000000000000000
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: R13: 0000000000600000 R14:
ffff880036dc5000 R15: ffff880030994000
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: FS:
00007f147d9606e0(0000) GS:ffff8800280a6000(0000) knlGS:0000000000000000
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: CS: e033 DS: 0000 ES:
0000 CR0: 000000008005003b
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: CR2: ffff880030081010 CR3:
000000003ba55000 CR4: 0000000000002660
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: Process mpath_wait (pid:
32051, threadinfo ffff880030c18000, task ffff88003c93c280)
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: Stack:
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: ffff880030081010
000000003b7e4067 ffff880030c19c48 ffffffff810a5e32
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: <0> ffff880030c19ac8
000000000000001f 0000000002000000 0000000000000000
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: <0> 00000000008c3000
ffff8800280b45c0 00000000008c3000 ffff880030c19cb8
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: Call Trace:
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: [<ffffffff810a5e32>]
free_pgd_range+0x3b3/0x401
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: [<ffffffff810a6133>]
free_pgtables+0xb6/0xce
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: [<ffffffff810a8224>]
exit_mmap+0xf0/0x14b
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: [<ffffffff81048648>]
mmput+0x46/0xb2
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: [<ffffffff810c8a56>]
flush_old_exec+0x460/0x533
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: [<ffffffff810fbd15>]
load_elf_binary+0x365/0x171c
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: [<ffffffff810fa7ac>] ?
load_misc_binary+0x61/0x322
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: [<ffffffff810a4e5f>] ?
get_user_pages+0x44/0x46
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: [<ffffffff810c76de>] ?
put_arg_page+0x9/0xb
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: [<ffffffff810c7b8b>]
search_binary_handler+0xde/0x268
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: [<ffffffff810c8f85>]
do_execve+0x193/0x262
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: [<ffffffff810113af>]
sys_execve+0x3e/0x58
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: [<ffffffff81012f1a>]
stub_execve+0x6a/0xc0
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: Code: 8b 3c 25 70 c5 00 00
ff 14 25 68 82 5c 81 48 83 c4 18 5b c9 c3 55 48 89 e5 41 54 49 89 f4 53 48
89 fb e8 79 e3 ff ff 84 c0 75 05 <4c> 89 23 eb 0b 4c 89 e6 48 89 df e8 69 ff
ff ff 5b 41 5c c9 c3
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: RIP [<ffffffff8100e2d4>]
xen_set_pmd+0x16/0x2b
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: RSP <ffff880030c19b88>
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: CR2: ffff880030081010
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: ---[ end trace
f2d7fcc13c4cdf6a ]---
/var/log/messages.4:Feb 8 03:58:34 xen06 kernel: BUG: unable to handle
kernel paging request at ffff8800313e1b00
/var/log/messages.4-Feb 8 03:58:34 xen06 kernel: IP: [<ffffffff810377f2>]
ptep_set_access_flags+0x27/0x4a
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: PGD 1002067 PUD 1006067
PMD 24f067 PTE 80100000313e1065
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: Oops: 0003 [#2] SMP
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: last sysfs file:
/sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_map
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: CPU 1
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: Modules linked in:
ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state
nf_conntrack ipt_REJECT xt_tcpudp gfs2 dlm configfs xt_physdev
iptable_filter ip_tables x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic
uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi
dm_multipath scsi_dh video backlight output sbs sbshc power_meter hwmon
battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc lp parport sg
ide_cd_mod cdrom serio_raw tg3 button libphy tpm_tis tpm tpm_bios i2c_i801
pcspkr i2c_core iTCO_wdt shpchp dm_snapshot dm_zero dm_mirror dm_region_hash
dm_log dm_mod ata_piix libata sd_mod scsi_mod raid1 ext3 jbd uhci_hcd
ohci_hcd ehci_hcd [last unloaded: microcode]
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: Pid: 32053, comm:
mpath_wait Tainted: G D 2.6.32.28-0.xen.pvops.choon.centos5 #1
PowerEdge 860
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: RIP:
e030:[<ffffffff810377f2>] [<ffffffff810377f2>]
ptep_set_access_flags+0x27/0x4a
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: RSP:
e02b:ffff8800307c7cb8 EFLAGS: 00210202
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: RAX: ffff8800304b4870 RBX:
00007f147d960770 RCX: 80000001f6d6f167
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: RDX: ffff8800313e1b00 RSI:
00007f147d960770 RDI: ffff8800304b4870
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: RBP: ffff8800307c7cd8 R08:
0000000000000001 R09: ffffea0000ac5948
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: R10: 3138666666666666 R11:
000000000002496a R12: 0000000000000001
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: R13: ffff8800304b4870 R14:
0000000000000008 R15: 0000000000000000
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: FS:
00007f147d9606e0(0000) GS:ffff88002806c000(0000) knlGS:0000000000000000
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: CS: e033 DS: 0000 ES:
0000 CR0: 000000008005003b
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: CR2: ffff8800313e1b00 CR3:
0000000031261000 CR4: 0000000000002660
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: Process mpath_wait (pid:
32053, threadinfo ffff8800307c6000, task ffff880030ece200)
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: Stack:
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: 80000001f6d6f145
0000000000000000 ffffea0000ac5948 ffffea0000800f30
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: <0> ffff8800307c7d88
ffffffff810a2a19 0000000000000001 ffff8800307c7de8
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: <0> ffffffff8100f0b1
ffff8800307c7d98 ffff8800305a8f60 ffff8800313e1b00
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: Call Trace:
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: [<ffffffff810a2a19>]
do_wp_page+0x2fd/0x717
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: [<ffffffff8100f0b1>] ?
xen_force_evtchn_callback+0xd/0xf
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: [<ffffffff8100cde0>] ?
xen_pte_val+0x64/0x6e
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: [<ffffffff8100c3e9>] ?
__raw_callee_save_xen_pte_val+0x15/0x23
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: [<ffffffff810a4933>]
handle_mm_fault+0x84a/0x8b9
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: [<ffffffff8131be2d>]
do_page_fault+0x252/0x2e2
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: [<ffffffff8100f7f2>] ?
check_events+0x12/0x20
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: [<ffffffff81319db5>]
page_fault+0x25/0x30
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: [<ffffffff8116dc9d>] ?
__put_user_4+0x1d/0x30
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: [<ffffffff81047d2c>] ?
schedule_tail+0x98/0x9d
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: [<ffffffff810129c3>]
ret_from_fork+0x13/0x80
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: Code: 41 5d c9 c3 55 48 89
e5 41 55 49 89 fd 41 54 45 31 e4 53 48 89 f3 48 83 ec 08 48 39 0a 41 0f 95
c4 45 85 e4 74 1d 45 85 c0 74 18 <48> 89 0a 48 8b 3f 0f 1f 80 00 00 00 00 48
89 de 4c 89 ef e8 37
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: RIP [<ffffffff810377f2>]
ptep_set_access_flags+0x27/0x4a
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: RSP <ffff8800307c7cb8>
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: CR2: ffff8800313e1b00
/var/log/messages.4-Feb 8 03:58:35 xen06 kernel: ---[ end trace
f2d7fcc13c4cdf6b ]---
/var/log/messages.4-Feb 8 04:26:35 xen06 syslogd 1.4.1: restart.
/var/log/messages.4-Feb 8 04:26:35 xen06 kernel: klogd 1.4.1, log source =
/proc/kmsg started.
/var/log/messages.4-Feb 8 04:26:35 xen06 kernel: Linux version
2.6.32.28-0.xen.pvops.choon.centos5 (mockbuild@builder5.choon.net) (gcc
version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Tue Feb 8 03:26:17 SGT
2011
/var/log/messages.4-Feb 8 04:26:35 xen06 kernel: Command line: ro
root=/dev/md1 panic=5 panic_timeout=5
/var/log/messages.4-Feb 8 04:26:35 xen06 kernel: KERNEL supported cpus:
/var/log/messages.4-Feb 8 04:26:35 xen06 kernel: Intel GenuineIntel
/var/log/messages.4-Feb 8 04:26:35 xen06 kernel: AMD AuthenticAMD
/var/log/messages.4-Feb 8 04:26:35 xen06 kernel: Centaur CentaurHauls
/var/log/messages.4-Feb 8 04:26:35 xen06 kernel: released 0 pages of unused
memory
--
/var/log/messages.4-Feb 9 07:44:11 xen06 lvm[3802]: Monitoring snapshot
XenGroup-testcrash2--snapshot
/var/log/messages.4-Feb 9 07:44:11 xen06 kernel: kjournald starting.
Commit interval 5 seconds
/var/log/messages.4-Feb 9 07:44:11 xen06 kernel: EXT3 FS on dm-11, internal
journal
/var/log/messages.4-Feb 9 07:44:11 xen06 kernel: EXT3-fs: mounted
filesystem with ordered data mode.
/var/log/messages.4-Feb 9 07:44:11 xen06 lvm[3802]: No longer monitoring
snapshot XenGroup-testcrash2--snapshot
/var/log/messages.4-Feb 9 07:44:12 xen06 lvm[3802]: Monitoring snapshot
XenGroup-testcrash3--snapshot
/var/log/messages.4-Feb 9 07:44:12 xen06 kernel: kjournald starting.
Commit interval 5 seconds
/var/log/messages.4-Feb 9 07:44:12 xen06 kernel: EXT3 FS on dm-11, internal
journal
/var/log/messages.4-Feb 9 07:44:12 xen06 kernel: EXT3-fs: mounted
filesystem with ordered data mode.
/var/log/messages.4-Feb 9 07:44:12 xen06 lvm[3802]: No longer monitoring
snapshot XenGroup-testcrash3--snapshot
/var/log/messages.4-Feb 9 07:44:13 xen06 lvm[3802]: Monitoring snapshot
XenGroup-testcrash4--snapshot
/var/log/messages.4-Feb 9 07:44:13 xen06 kernel: kjournald starting.
Commit interval 5 seconds
/var/log/messages.4-Feb 9 07:44:13 xen06 kernel: EXT3 FS on dm-11, internal
journal
/var/log/messages.4-Feb 9 07:44:13 xen06 kernel: EXT3-fs: mounted
filesystem with ordered data mode.
/var/log/messages.4-Feb 9 07:44:13 xen06 lvm[3802]: No longer monitoring
snapshot XenGroup-testcrash4--snapshot
/var/log/messages.4-Feb 9 07:44:14 xen06 lvm[3802]: Monitoring snapshot
XenGroup-testcrash5--snapshot
/var/log/messages.4-Feb 9 07:44:14 xen06 kernel: kjournald starting.
Commit interval 5 seconds
/var/log/messages.4-Feb 9 07:44:15 xen06 kernel: EXT3 FS on dm-11, internal
journal
/var/log/messages.4-Feb 9 07:44:15 xen06 kernel: EXT3-fs: mounted
filesystem with ordered data mode.
/var/log/messages.4-Feb 9 07:44:15 xen06 lvm[3802]: No longer monitoring
snapshot XenGroup-testcrash5--snapshot
/var/log/messages.4:Feb 9 07:53:59 xen06 kernel: BUG: unable to handle
kernel paging request at ffff88003cc0c010
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: IP: [<ffffffff8100e2d4>]
xen_set_pmd+0x16/0x2b
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: PGD 1002067 PUD 1006067
PMD 2ac067 PTE 801000003cc0c065
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: Oops: 0003 [#1] SMP
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: last sysfs file:
/sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_map
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: CPU 0
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: Modules linked in: dlm
configfs xt_physdev iptable_filter ip_tables x_tables bridge stp be2iscsi
iscsi_tcp bnx2i cnic uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi
scsi_transport_iscsi dm_multipath scsi_dh video backlight output sbs sbshc
power_meter hwmon battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc
lp parport sg ide_cd_mod cdrom serio_raw tg3 libphy button tpm_tis tpm
tpm_bios pcspkr i2c_i801 shpchp i2c_core iTCO_wdt dm_snapshot dm_zero
dm_mirror dm_region_hash dm_log dm_mod ata_piix libata sd_mod scsi_mod raid1
ext3 jbd uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: Pid: 11005, comm:
mpath_wait Not tainted 2.6.32.28-0.xen.pvops.choon.centos5 #1 PowerEdge 860
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: RIP:
e030:[<ffffffff8100e2d4>] [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: RSP:
e02b:ffff88003b7d3b88 EFLAGS: 00010246
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: RAX: 0000000000000000 RBX:
ffff88003cc0c010 RCX: 00000000008c3000
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: RDX: ffffea0000000000 RSI:
0000000000000000 RDI: ffff88003cc0c010
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: RBP: ffff88003b7d3b98 R08:
0000000001244000 R09: ffff88003f009f09
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: R10: ffff8800269e3ac0 R11:
ffffffff813381a0 R12: 0000000000000000
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: R13: 0000000000600000 R14:
ffff8800269ca000 R15: ffff8800268bb000
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: FS:
00007fa7f8de76e0(0000) GS:ffff88002804f000(0000) knlGS:0000000000000000
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: CS: e033 DS: 0000 ES:
0000 CR0: 000000008005003b
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: CR2: ffff88003cc0c010 CR3:
00000000306b0000 CR4: 0000000000002660
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: Process mpath_wait (pid:
11005, threadinfo ffff88003b7d2000, task ffff8800307aa480)
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: Stack:
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: ffff88003cc0c010
000000003cfee067 ffff88003b7d3c48 ffffffff810a5e32
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: <0> 000000000000e02b
0000000000000060 0000000001200000 0000000000000000
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: <0> 00000000008c3000
ffff88002805d5c0 00000000008c3000 ffff88003b7d3cb8
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: Call Trace:
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: [<ffffffff810a5e32>]
free_pgd_range+0x3b3/0x401
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: [<ffffffff810a6133>]
free_pgtables+0xb6/0xce
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: [<ffffffff810a8224>]
exit_mmap+0xf0/0x14b
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: [<ffffffff81048648>]
mmput+0x46/0xb2
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: [<ffffffff810c8a56>]
flush_old_exec+0x460/0x533
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: [<ffffffff810fbd15>]
load_elf_binary+0x365/0x171c
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: [<ffffffff810fa7ac>] ?
load_misc_binary+0x61/0x322
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: [<ffffffff810a4e5f>] ?
get_user_pages+0x44/0x46
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: [<ffffffff810c76de>] ?
put_arg_page+0x9/0xb
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: [<ffffffff810c7b8b>]
search_binary_handler+0xde/0x268
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: [<ffffffff810c8f85>]
do_execve+0x193/0x262
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: [<ffffffff810113af>]
sys_execve+0x3e/0x58
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: [<ffffffff81012f1a>]
stub_execve+0x6a/0xc0
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: Code: 8b 3c 25 70 c5 00 00
ff 14 25 68 82 5c 81 48 83 c4 18 5b c9 c3 55 48 89 e5 41 54 49 89 f4 53 48
89 fb e8 79 e3 ff ff 84 c0 75 05 <4c> 89 23 eb 0b 4c 89 e6 48 89 df e8 69 ff
ff ff 5b 41 5c c9 c3
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: RIP [<ffffffff8100e2d4>]
xen_set_pmd+0x16/0x2b
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: RSP <ffff88003b7d3b88>
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: CR2: ffff88003cc0c010
/var/log/messages.4-Feb 9 07:53:59 xen06 kernel: ---[ end trace
b5fffc59f6c8f974 ]---
/var/log/messages.4-Feb 9 16:31:21 xen06 lvm[3802]: Monitoring snapshot
XenGroup-testcrash1--snapshot
/var/log/messages.4-Feb 9 16:31:21 xen06 kernel: kjournald starting.
Commit interval 5 seconds
/var/log/messages.4-Feb 9 16:31:21 xen06 kernel: EXT3 FS on dm-11, internal
journal
/var/log/messages.4-Feb 9 16:31:21 xen06 kernel: EXT3-fs: mounted
filesystem with ordered data mode.
/var/log/messages.4-Feb 9 16:31:21 xen06 lvm[3802]: No longer monitoring
snapshot XenGroup-testcrash1--snapshot
/var/log/messages.4-Feb 9 16:31:22 xen06 lvm[3802]: Monitoring snapshot
XenGroup-testcrash2--snapshot
/var/log/messages.4-Feb 9 16:31:22 xen06 kernel: kjournald starting.
Commit interval 5 seconds
--
/var/log/messages.4-Feb 9 16:35:34 xen06 lvm[3802]: Monitoring snapshot
XenGroup-testcrash1--snapshot
/var/log/messages.4-Feb 9 16:35:34 xen06 kernel: kjournald starting.
Commit interval 5 seconds
/var/log/messages.4-Feb 9 16:35:34 xen06 kernel: EXT3 FS on dm-11, internal
journal
/var/log/messages.4-Feb 9 16:35:34 xen06 kernel: EXT3-fs: mounted
filesystem with ordered data mode.
/var/log/messages.4-Feb 9 16:35:35 xen06 lvm[3802]: No longer monitoring
snapshot XenGroup-testcrash1--snapshot
/var/log/messages.4-Feb 9 16:35:36 xen06 lvm[3802]: Monitoring snapshot
XenGroup-testcrash2--snapshot
/var/log/messages.4-Feb 9 16:35:36 xen06 kernel: kjournald starting.
Commit interval 5 seconds
/var/log/messages.4-Feb 9 16:35:36 xen06 kernel: EXT3 FS on dm-11, internal
journal
/var/log/messages.4-Feb 9 16:35:36 xen06 kernel: EXT3-fs: mounted
filesystem with ordered data mode.
/var/log/messages.4-Feb 9 16:35:36 xen06 lvm[3802]: No longer monitoring
snapshot XenGroup-testcrash2--snapshot
/var/log/messages.4-Feb 9 16:35:37 xen06 lvm[3802]: Monitoring snapshot
XenGroup-testcrash3--snapshot
/var/log/messages.4-Feb 9 16:35:37 xen06 kernel: kjournald starting.
Commit interval 5 seconds
/var/log/messages.4-Feb 9 16:35:37 xen06 kernel: EXT3 FS on dm-11, internal
journal
/var/log/messages.4-Feb 9 16:35:37 xen06 kernel: EXT3-fs: mounted
filesystem with ordered data mode.
/var/log/messages.4-Feb 9 16:35:37 xen06 lvm[3802]: No longer monitoring
snapshot XenGroup-testcrash3--snapshot
/var/log/messages.4-Feb 9 16:35:38 xen06 lvm[3802]: Monitoring snapshot
XenGroup-testcrash4--snapshot
/var/log/messages.4-Feb 9 16:35:38 xen06 kernel: kjournald starting.
Commit interval 5 seconds
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: EXT3 FS on dm-11, internal
journal
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: EXT3-fs: mounted
filesystem with ordered data mode.
/var/log/messages.4-Feb 9 16:35:39 xen06 lvm[3802]: No longer monitoring
snapshot XenGroup-testcrash4--snapshot
/var/log/messages.4:Feb 9 16:35:39 xen06 kernel: BUG: unable to handle
kernel paging request at ffff880026b00418
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: IP: [<ffffffff8100e2d4>]
xen_set_pmd+0x16/0x2b
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: PGD 1002067 PUD 1006067
PMD 1fb067 PTE 8010000026b00065
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: Oops: 0003 [#2] SMP
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: last sysfs file:
/sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_map
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: CPU 2
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: Modules linked in: dlm
configfs xt_physdev iptable_filter ip_tables x_tables bridge stp be2iscsi
iscsi_tcp bnx2i cnic uio ipv6 cxgb3i cxgb3 mdio libiscsi_tcp libiscsi
scsi_transport_iscsi dm_multipath scsi_dh video backlight output sbs sbshc
power_meter hwmon battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc
lp parport sg ide_cd_mod cdrom serio_raw tg3 libphy button tpm_tis tpm
tpm_bios pcspkr i2c_i801 shpchp i2c_core iTCO_wdt dm_snapshot dm_zero
dm_mirror dm_region_hash dm_log dm_mod ata_piix libata sd_mod scsi_mod raid1
ext3 jbd uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: Pid: 25953, comm: udevd
Tainted: G D 2.6.32.28-0.xen.pvops.choon.centos5 #1 PowerEdge 860
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: RIP:
e030:[<ffffffff8100e2d4>] [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: RSP:
e02b:ffff88002682fb88 EFLAGS: 00010246
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: RAX: 0000000000000000 RBX:
ffff880026b00418 RCX: 00007f71114ac000
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: RDX: ffffea0000000000 RSI:
0000000000000000 RDI: ffff880026b00418
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: RBP: ffff88002682fb98 R08:
00007f7112d14000 R09: ffff88003e6e5a89
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: R10: ffff88003d2d0580 R11:
ffffffff813381a0 R12: 0000000000000000
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: R13: 00007f7110800000 R14:
ffff880026b047f0 R15: ffff880025943e20
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: FS:
00007f711128a710(0000) GS:ffff880028089000(0000) knlGS:0000000000000000
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: CS: e033 DS: 0000 ES:
0000 CR0: 000000008005003b
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: CR2: ffff880026b00418 CR3:
0000000026946000 CR4: 0000000000002660
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: Process udevd (pid: 25953,
threadinfo ffff88002682e000, task ffff8800269ae380)
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: Stack:
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: ffff880026b00418
000000003d32d067 ffff88002682fc48 ffffffff810a5e32
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: <0> ffff88002682fac8
000000000000001c 00007f7112c00000 0000000000000000
/var/log/messages.4-Feb 9 16:35:39 xen06 udevd-event[25941]: run_program:
'/sbin/dmsetup' abnormal exit
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: <0> 00007f71114ac000
ffff8800280975c0 00007f71114ac000 ffff88002682fcb8
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: Call Trace:
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: [<ffffffff810a5e32>]
free_pgd_range+0x3b3/0x401
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: [<ffffffff810a6133>]
free_pgtables+0xb6/0xce
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: [<ffffffff810a8224>]
exit_mmap+0xf0/0x14b
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: [<ffffffff81048648>]
mmput+0x46/0xb2
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: [<ffffffff810c8a56>]
flush_old_exec+0x460/0x533
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: [<ffffffff810fbd15>]
load_elf_binary+0x365/0x171c
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: [<ffffffff810fa7ac>] ?
load_misc_binary+0x61/0x322
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: [<ffffffff810a4e5f>] ?
get_user_pages+0x44/0x46
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: [<ffffffff810c76de>] ?
put_arg_page+0x9/0xb
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: [<ffffffff810c7b8b>]
search_binary_handler+0xde/0x268
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: [<ffffffff810c8f85>]
do_execve+0x193/0x262
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: [<ffffffff810113af>]
sys_execve+0x3e/0x58
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: [<ffffffff81012f1a>]
stub_execve+0x6a/0xc0
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: Code: 8b 3c 25 70 c5 00 00
ff 14 25 68 82 5c 81 48 83 c4 18 5b c9 c3 55 48 89 e5 41 54 49 89 f4 53 48
89 fb e8 79 e3 ff ff 84 c0 75 05 <4c> 89 23 eb 0b 4c 89 e6 48 89 df e8 69 ff
ff ff 5b 41 5c c9 c3
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: RIP [<ffffffff8100e2d4>]
xen_set_pmd+0x16/0x2b
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: RSP <ffff88002682fb88>
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: CR2: ffff880026b00418
/var/log/messages.4-Feb 9 16:35:39 xen06 kernel: ---[ end trace
b5fffc59f6c8f975 ]---
/var/log/messages.4-Feb 9 16:35:40 xen06 lvm[3802]: Monitoring snapshot
XenGroup-testcrash5--snapshot
/var/log/messages.4-Feb 9 16:35:40 xen06 kernel: kjournald starting.
Commit interval 5 seconds
/var/log/messages.4-Feb 9 16:35:40 xen06 kernel: EXT3 FS on dm-11, internal
journal
/var/log/messages.4-Feb 9 16:35:40 xen06 kernel: EXT3-fs: mounted
filesystem with ordered data mode.
/var/log/messages.4-Feb 9 16:35:40 xen06 lvm[3802]: No longer monitoring
snapshot XenGroup-testcrash5--snapshot
/var/log/messages.4-Feb 9 16:35:41 xen06 lvm[3802]: Monitoring snapshot
XenGroup-testcrash1--snapshot
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 41656 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-11 19:59 ` Sander Eikelenboom
2011-03-11 20:29 ` Teck Choon Giam
@ 2011-03-11 20:45 ` Teck Choon Giam
2011-03-11 21:02 ` Sander Eikelenboom
1 sibling, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2011-03-11 20:45 UTC (permalink / raw)
To: Sander Eikelenboom; +Cc: Andreas Olsowski, xen-devel, benco
[-- Attachment #1.1: Type: text/plain, Size: 569 bytes --]
On Sat, Mar 12, 2011 at 3:59 AM, Sander Eikelenboom <linux@eikelenboom.it>wrote:
> Haven't seen this one on my systen. I do use LVM, but i'm only using LVM as
> device mapper, without software raid etc.
> Also using debian lenny/squeeze together with custom compiled xen and
> xen-2.6.32.x pvops kernel
>
> So I would suspect the multipathing / raid part.
>
If you do not use LVM snapshot create/delete then you mostly won't hit this
BUG. LVM create/delete might but very rare but I hit this before... so
unlucky for me :(
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 879 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-11 20:45 ` Teck Choon Giam
@ 2011-03-11 21:02 ` Sander Eikelenboom
2011-03-11 21:15 ` Teck Choon Giam
0 siblings, 1 reply; 85+ messages in thread
From: Sander Eikelenboom @ 2011-03-11 21:02 UTC (permalink / raw)
To: Teck Choon Giam; +Cc: Andreas Olsowski, xen-devel, benco
You did have some test script for creating/delete snapshots that triggers it ?
Could try to test it with 2.6.38 as dom0 now, i you could resend that.
Friday, March 11, 2011, 9:45:31 PM, you wrote:
> On Sat, Mar 12, 2011 at 3:59 AM, Sander Eikelenboom <linux@eikelenboom.it>wrote:
>> Haven't seen this one on my systen. I do use LVM, but i'm only using LVM as
>> device mapper, without software raid etc.
>> Also using debian lenny/squeeze together with custom compiled xen and
>> xen-2.6.32.x pvops kernel
>>
>> So I would suspect the multipathing / raid part.
>>
> If you do not use LVM snapshot create/delete then you mostly won't hit this
> BUG. LVM create/delete might but very rare but I hit this before... so
> unlucky for me :(
> Thanks.
> Kindest regards,
> Giam Teck Choon
--
Best regards,
Sander mailto:linux@eikelenboom.it
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-11 21:02 ` Sander Eikelenboom
@ 2011-03-11 21:15 ` Teck Choon Giam
0 siblings, 0 replies; 85+ messages in thread
From: Teck Choon Giam @ 2011-03-11 21:15 UTC (permalink / raw)
To: Sander Eikelenboom; +Cc: Andreas Olsowski, xen-devel, benco
[-- Attachment #1.1: Type: text/plain, Size: 401 bytes --]
On Sat, Mar 12, 2011 at 5:02 AM, Sander Eikelenboom <linux@eikelenboom.it>wrote:
> You did have some test script for creating/delete snapshots that triggers
> it ?
>
Yes, I do and it is posted in this thread.
>
> Could try to test it with 2.6.38 as dom0 now, i you could resend that.
>
According to Andreas Olsowski testing, 2.6.38 doesn't suffer this.
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 864 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-11 18:05 ` Konrad Rzeszutek Wilk
@ 2011-03-14 10:25 ` Ian Campbell
2011-03-14 10:36 ` Teck Choon Giam
0 siblings, 1 reply; 85+ messages in thread
From: Ian Campbell @ 2011-03-14 10:25 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: Andreas Olsowski, xen-devel
On Fri, 2011-03-11 at 18:05 +0000, Konrad Rzeszutek Wilk wrote:
> > Btw, who is currently working on the remus implementation?
>
> Frank Pan <frankpzh@gmail.com>
Interesting, I didn't know Frank was working on Remus stuff (I thought
he was looking at more general suspend/restore/migrate stuff).
However Shriram Rajagopalan <rshriram@cs.ubc.ca> recently stepped up to
be Remus maintainer.
Ian.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-03-14 10:25 ` Ian Campbell
@ 2011-03-14 10:36 ` Teck Choon Giam
0 siblings, 0 replies; 85+ messages in thread
From: Teck Choon Giam @ 2011-03-14 10:36 UTC (permalink / raw)
To: xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 196 bytes --]
Hi,
Just in case anyone need my testcrash.sh/test.sh script, attached is my
latest version. For usage, simply execute testcrash.sh without any option.
Thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 266 bytes --]
[-- Attachment #2: testcrash.sh --]
[-- Type: application/x-sh, Size: 5634 bytes --]
[-- Attachment #3: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860! - ideas.
2011-03-10 13:45 ` Andreas Olsowski
2011-03-11 18:05 ` Konrad Rzeszutek Wilk
@ 2011-03-16 15:52 ` Konrad Rzeszutek Wilk
2011-03-16 16:26 ` Teck Choon Giam
2011-03-28 11:37 ` Andreas Olsowski
1 sibling, 2 replies; 85+ messages in thread
From: Konrad Rzeszutek Wilk @ 2011-03-16 15:52 UTC (permalink / raw)
To: Andreas Olsowski, giamteckchoon; +Cc: xen-devel
> ======================================================
> Kernel 2.6.32.28 without XEN:
> about 50 successful runs of Teck Choon Giams "test.sh" script.
> (modified for handling 10 test volumes and sleeping 2 seconds)
> multipathd restarted succesfully s
> multipath module loaded/unloaded successfully
> lvm2 restarted successfully
.. snip..
> ======================================================
> Kernel 2.6.32.28 with XEN 4.0.1:
> at about loop 2 for volume 7 of "test.sh" it stopped doing ... well anything
> there has been no output on the screen and neitehr syslog nor dmesg entry.
> I left it hanging for about 15 Minutes until i decided to write this
> one off as a side effect of the same underlying problem.
> All lvm2 tools stopped working and i couldnt shut it down.
> Killing the hangig process ended it properly.
Jeremy and I were brainstorming this yesterday and couple of things
that we thought might be interesting are to:
- turn on CONFIG_DEBUG_PAGEALLOC
- turn on CONFIG_DEBUG_LIST
- turn on CONFIG_DEBUG_KMEMLEAK
- turn on CONFIG_JBD_DEBUG, CONFIG_JBD2_DEBUG
- turn on CONFIG_SLUB_DEBUG_ON
And see if anything starts coming out.
Also looking in the changes for the drivers/dm/ between 2.6.32 and 2.6.38 and
see if we just hitting some memory leak bugs that hadn't been back-ported.
(Still busy with the upstream effort, can't work on this).
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860! - ideas.
2011-03-16 15:52 ` [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860! - ideas Konrad Rzeszutek Wilk
@ 2011-03-16 16:26 ` Teck Choon Giam
2011-03-16 16:40 ` Konrad Rzeszutek Wilk
2011-03-28 11:37 ` Andreas Olsowski
1 sibling, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2011-03-16 16:26 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: Andreas Olsowski, xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 2108 bytes --]
On Wed, Mar 16, 2011 at 11:52 PM, Konrad Rzeszutek Wilk <
konrad.wilk@oracle.com> wrote:
> > ======================================================
> > Kernel 2.6.32.28 without XEN:
> > about 50 successful runs of Teck Choon Giams "test.sh" script.
> > (modified for handling 10 test volumes and sleeping 2 seconds)
> > multipathd restarted succesfully s
> > multipath module loaded/unloaded successfully
> > lvm2 restarted successfully
> .. snip..
> > ======================================================
> > Kernel 2.6.32.28 with XEN 4.0.1:
> > at about loop 2 for volume 7 of "test.sh" it stopped doing ... well
> anything
> > there has been no output on the screen and neitehr syslog nor dmesg
> entry.
> > I left it hanging for about 15 Minutes until i decided to write this
> > one off as a side effect of the same underlying problem.
> > All lvm2 tools stopped working and i couldnt shut it down.
> > Killing the hangig process ended it properly.
>
> Jeremy and I were brainstorming this yesterday and couple of things
> that we thought might be interesting are to:
>
> - turn on CONFIG_DEBUG_PAGEALLOC
> - turn on CONFIG_DEBUG_LIST
> - turn on CONFIG_DEBUG_KMEMLEAK
> - turn on CONFIG_JBD_DEBUG, CONFIG_JBD2_DEBUG
> - turn on CONFIG_SLUB_DEBUG_ON
>
> And see if anything starts coming out.
>
Thanks a lot for both of you spending time to do so. It isn't easy as I
believe this is something related to kernel 2.6.32.x and just wondering is
there something related to *sched_domains? I read recent mails in LKML
about rebuild_sched_domains consider dangerous issues... and that is about
recent kernels but won't know what recent kernels that refer to... ...
I will do those config changes in one of my test server when time permit and
will post results/output here when done.
>
> Also looking in the changes for the drivers/dm/ between 2.6.32 and 2.6.38
> and
> see if we just hitting some memory leak bugs that hadn't been back-ported.
>
> (Still busy with the upstream effort, can't work on this).
>
You mean 2.6.39 merge window?
Once again, thanks.
Kindest regards,
Giam Teck Choon
[-- Attachment #1.2: Type: text/html, Size: 2728 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860! - ideas.
2011-03-16 16:26 ` Teck Choon Giam
@ 2011-03-16 16:40 ` Konrad Rzeszutek Wilk
2011-03-24 11:57 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 85+ messages in thread
From: Konrad Rzeszutek Wilk @ 2011-03-16 16:40 UTC (permalink / raw)
To: Teck Choon Giam; +Cc: Andreas Olsowski, xen-devel
> > - turn on CONFIG_DEBUG_PAGEALLOC
> > - turn on CONFIG_DEBUG_LIST
> > - turn on CONFIG_DEBUG_KMEMLEAK
> > - turn on CONFIG_JBD_DEBUG, CONFIG_JBD2_DEBUG
> > - turn on CONFIG_SLUB_DEBUG_ON
> >
> > And see if anything starts coming out.
> >
>
> Thanks a lot for both of you spending time to do so. It isn't easy as I
> believe this is something related to kernel 2.6.32.x and just wondering is
> there something related to *sched_domains? I read recent mails in LKML
Hmmm.. no idea.
> about rebuild_sched_domains consider dangerous issues... and that is about
> recent kernels but won't know what recent kernels that refer to... ...
>
> I will do those config changes in one of my test server when time permit and
> will post results/output here when done.
OK. Thank you!
>
>
> >
> > Also looking in the changes for the drivers/dm/ between 2.6.32 and 2.6.38
> > and
> > see if we just hitting some memory leak bugs that hadn't been back-ported.
> >
> > (Still busy with the upstream effort, can't work on this).
> >
>
> You mean 2.6.39 merge window?
Yes.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860! - ideas.
2011-03-16 16:40 ` Konrad Rzeszutek Wilk
@ 2011-03-24 11:57 ` Konrad Rzeszutek Wilk
2011-03-24 21:28 ` Teck Choon Giam
2011-03-25 3:57 ` Teck Choon Giam
0 siblings, 2 replies; 85+ messages in thread
From: Konrad Rzeszutek Wilk @ 2011-03-24 11:57 UTC (permalink / raw)
To: Teck Choon Giam; +Cc: Andreas Olsowski, xen-devel
On Wed, Mar 16, 2011 at 12:40:01PM -0400, Konrad Rzeszutek Wilk wrote:
> > > - turn on CONFIG_DEBUG_PAGEALLOC
> > > - turn on CONFIG_DEBUG_LIST
> > > - turn on CONFIG_DEBUG_KMEMLEAK
> > > - turn on CONFIG_JBD_DEBUG, CONFIG_JBD2_DEBUG
> > > - turn on CONFIG_SLUB_DEBUG_ON
> > >
> > > And see if anything starts coming out.
> > >
> >
> > Thanks a lot for both of you spending time to do so. It isn't easy as I
> > believe this is something related to kernel 2.6.32.x and just wondering is
> > there something related to *sched_domains? I read recent mails in LKML
>
> Hmmm.. no idea.
> > about rebuild_sched_domains consider dangerous issues... and that is about
> > recent kernels but won't know what recent kernels that refer to... ...
> >
> > I will do those config changes in one of my test server when time permit and
> > will post results/output here when done.
>
> OK. Thank you!
I've been using Jermey's latest tree: 2.6.32.32 (there is even a 2.6.32.33)
and I can't hit this bug anymore. Would appreciate your input if you still see this.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860! - ideas.
2011-03-24 11:57 ` Konrad Rzeszutek Wilk
@ 2011-03-24 21:28 ` Teck Choon Giam
2011-03-25 3:57 ` Teck Choon Giam
1 sibling, 0 replies; 85+ messages in thread
From: Teck Choon Giam @ 2011-03-24 21:28 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: Andreas Olsowski, xen-devel
On Thu, Mar 24, 2011 at 7:57 PM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
> On Wed, Mar 16, 2011 at 12:40:01PM -0400, Konrad Rzeszutek Wilk wrote:
>> > > - turn on CONFIG_DEBUG_PAGEALLOC
>> > > - turn on CONFIG_DEBUG_LIST
>> > > - turn on CONFIG_DEBUG_KMEMLEAK
>> > > - turn on CONFIG_JBD_DEBUG, CONFIG_JBD2_DEBUG
>> > > - turn on CONFIG_SLUB_DEBUG_ON
>> > >
>> > > And see if anything starts coming out.
>> > >
>> >
>> > Thanks a lot for both of you spending time to do so. It isn't easy as I
>> > believe this is something related to kernel 2.6.32.x and just wondering is
>> > there something related to *sched_domains? I read recent mails in LKML
>>
>> Hmmm.. no idea.
>> > about rebuild_sched_domains consider dangerous issues... and that is about
>> > recent kernels but won't know what recent kernels that refer to... ...
>> >
>> > I will do those config changes in one of my test server when time permit and
>> > will post results/output here when done.
>>
>> OK. Thank you!
>
> I've been using Jermey's latest tree: 2.6.32.32 (there is even a 2.6.32.33)
> and I can't hit this bug anymore. Would appreciate your input if you still see this.
>
I still able to unfortunately :(
git url = git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
git branch = xen/next-2.6.32
git commit = ba5fcb8b8ac91d8b65c8a9c2545bf3d416fdc151
I will be compiling the latest commit 2.6.32.35 and try... ... maybe I
am crazy... it doesn't satisfy me if past one cycle of 100 loop test.
In fact, I will run with loop 1000 a few times and this can take days.
XenLinux aka linux-2.6.18-xen.hg tree pass this test though... ...
unfortunately there are issues if I insist to use this old kernel due
to drivers etc. such as e1000e, ahci... ... although now in production
I use this xenkernel but need manually patch drivers etc... pain in
ass :p
Sorry, still haven't find time to change the configuration to those
and test and report back though as those build/compilation/packaging
are all via handled by my customized script. I will try to do so
about the configuration change and test this weekend if possible.
Thanks.
Kindest regards,
Giam Teck Choon
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860! - ideas.
2011-03-24 11:57 ` Konrad Rzeszutek Wilk
2011-03-24 21:28 ` Teck Choon Giam
@ 2011-03-25 3:57 ` Teck Choon Giam
2011-03-27 10:16 ` Teck Choon Giam
1 sibling, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2011-03-25 3:57 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: Andreas Olsowski, xen-devel
On Thu, Mar 24, 2011 at 7:57 PM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
> On Wed, Mar 16, 2011 at 12:40:01PM -0400, Konrad Rzeszutek Wilk wrote:
>> > > - turn on CONFIG_DEBUG_PAGEALLOC
>> > > - turn on CONFIG_DEBUG_LIST
>> > > - turn on CONFIG_DEBUG_KMEMLEAK
>> > > - turn on CONFIG_JBD_DEBUG, CONFIG_JBD2_DEBUG
>> > > - turn on CONFIG_SLUB_DEBUG_ON
>> > >
>> > > And see if anything starts coming out.
>> > >
>> >
>> > Thanks a lot for both of you spending time to do so. It isn't easy as I
>> > believe this is something related to kernel 2.6.32.x and just wondering is
>> > there something related to *sched_domains? I read recent mails in LKML
>>
>> Hmmm.. no idea.
>> > about rebuild_sched_domains consider dangerous issues... and that is about
>> > recent kernels but won't know what recent kernels that refer to... ...
>> >
>> > I will do those config changes in one of my test server when time permit and
>> > will post results/output here when done.
>>
>> OK. Thank you!
>
> I've been using Jermey's latest tree: 2.6.32.32 (there is even a 2.6.32.33)
> and I can't hit this bug anymore. Would appreciate your input if you still see this.
>
This is a report back to you that unfortunately I still able to hit this bug :(
git url = git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
git branch = xen/next-2.6.32
git commit = 4306ea8f6db3d83a5a2bbfe5448dd78e6846475a
Thanks.
Kindest regards,
Giam Teck Choon
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860! - ideas.
2011-03-25 3:57 ` Teck Choon Giam
@ 2011-03-27 10:16 ` Teck Choon Giam
0 siblings, 0 replies; 85+ messages in thread
From: Teck Choon Giam @ 2011-03-27 10:16 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: Andreas Olsowski, xen-devel
On Fri, Mar 25, 2011 at 11:57 AM, Teck Choon Giam
<giamteckchoon@gmail.com> wrote:
> On Thu, Mar 24, 2011 at 7:57 PM, Konrad Rzeszutek Wilk
> <konrad.wilk@oracle.com> wrote:
>> On Wed, Mar 16, 2011 at 12:40:01PM -0400, Konrad Rzeszutek Wilk wrote:
>>> > > - turn on CONFIG_DEBUG_PAGEALLOC
>>> > > - turn on CONFIG_DEBUG_LIST
>>> > > - turn on CONFIG_DEBUG_KMEMLEAK
>>> > > - turn on CONFIG_JBD_DEBUG, CONFIG_JBD2_DEBUG
>>> > > - turn on CONFIG_SLUB_DEBUG_ON
>>> > >
>>> > > And see if anything starts coming out.
>>> > >
>>> >
>>> > Thanks a lot for both of you spending time to do so. It isn't easy as I
>>> > believe this is something related to kernel 2.6.32.x and just wondering is
>>> > there something related to *sched_domains? I read recent mails in LKML
>>>
>>> Hmmm.. no idea.
>>> > about rebuild_sched_domains consider dangerous issues... and that is about
>>> > recent kernels but won't know what recent kernels that refer to... ...
>>> >
>>> > I will do those config changes in one of my test server when time permit and
>>> > will post results/output here when done.
>>>
>>> OK. Thank you!
>>
>> I've been using Jermey's latest tree: 2.6.32.32 (there is even a 2.6.32.33)
>> and I can't hit this bug anymore. Would appreciate your input if you still see this.
>>
>
> This is a report back to you that unfortunately I still able to hit this bug :(
>
> git url = git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
> git branch = xen/next-2.6.32
> git commit = 4306ea8f6db3d83a5a2bbfe5448dd78e6846475a
>
> Thanks.
>
> Kindest regards,
> Giam Teck Choon
>
Maybe this is good news ;)
This is my report about various suggested kernel configuration options
suggested by Konrad and Jeremy. I think I caught the cause or prevent
this same BUG from happening so that Konrad or Jeremy have fewer place
to look into it. Sorry, this will be little lengthy and sorry for my
poor English. I am using the following:
git url = git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
git branch = xen/next-2.6.32
git commit = df3a5560166da5a05de93f2fc36b718cc43c6c3c
hg_root = http://xenbits.xensource.com/xen-4.0-testing.hg
hg_changeset = 21465
With my old kernel config, I still hit this BUG with testcrash loop
100 easily. In fact, with loop below 30 I mostly will hit this same
BUG. My two test servers setup with at least 20 x 5GB LV. So each
loop cycle will have at least 20 lvcreate/lvremove
snapshots/mount/umount.
With the suggested CONFIGURATION changes by Konrad and Jeremy, I am
unable to reproduce this same BUG for testcrash.sh loop 1000 for two
of my test servers. The following are the summary/short note:
> - turn on CONFIG_DEBUG_PAGEALLOC
Ok, set.
> - turn on CONFIG_DEBUG_LIST
Already set originally.
> - turn on CONFIG_DEBUG_KMEMLEAK
Don't think I can enable this with x86_64 as there isn't an option for
it in x86_64 arch. However, I can see this option in x86_32 arch so I
guess it is dependent in x86_32. Anyway, I don't think this is
important for my case... ... why... read on... ... :P
> - turn on CONFIG_JBD_DEBUG, CONFIG_JBD2_DEBUG
Ok, set.
> - turn on CONFIG_SLUB_DEBUG_ON
Ok, set as I need to change from CONFIG_SLAB to CONFIG_SLUB instead
which also set CONFIG_SLUB_DEBUG=y besides CONFIG_SLUB_DEBUG_ON=y.
So from the testcrash results for two of my servers, I know there must
be related to the kernel CONFIGURATION changes and one of them is the
cause to prevent hitting this BUG. Now I am testing to set one of the
mentioned CONFIG at a time then run the same testcrash again to
determine which is the only CONFIG option that will not trigger this
same BUG. The results as below all using my old config as base with
*only one CONFIG option change at a time* to run testcrash loop 100:
With CONFIG_DEBUG_PAGEALLOC=y: Result : Think should be this one to
prevent hitting this same BUG as one of my test server already past
testcrash loop cycle 100... ... now testing testcrash loop 10000 :P
With CONFIG_SLUB=y and CONFIG_SLUB_DEBUG=y: Result : CRASH
With CONFIG_SLUB=y, CONFIG_SLUB_DEBUG=y and CONFIG_SLUB_DEBUG_ON=y:
Result : CRASH
With CONFIG_JBD_DEBUG=y: Result : CRASH
With CONFIG_JBD2_DEBUG=y: Result : CRASH
Can others who hit this same BUG reconfirm that your kernel config is
without CONFIG_DEBUG_PAGEALLOC being set/on? I think most production
servers will not have this config option enable in default. If so,
can test with CONFIG_DEBUG_PAGEALLOC=y instead? Sorry, currently
still in testing phrase for such configuration and hopefully can pass
this testcrash with loop 10000 for one of my server (am I crazy? LOL).
If this is really the case (I hope) then I guess there must be some
conditional difference for CONFIG_PAGEALLOC as without
CONFIG_DEBUG_PAGEALLOC set it will hit this BUG but with it set to on
it won't (at least during my composing of this mail reply/report)...
...
I will report back my final testcrash loop 10000 result when finish...
... keeping fingers crossed!!!
Can anyone test with kernel version 2.6.38 PVOPS tree with
CONFIG_DEBUG_PAGEALLOC not set and set to see whether such BUG exists
in 2.6.38?
I hope this report is useful especially to Konrad and Jeremy... ... ;)
Thanks.
Kindest regards,
Giam Teck Choon
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860! - ideas.
2011-03-16 15:52 ` [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860! - ideas Konrad Rzeszutek Wilk
2011-03-16 16:26 ` Teck Choon Giam
@ 2011-03-28 11:37 ` Andreas Olsowski
2011-03-28 12:29 ` Teck Choon Giam
1 sibling, 1 reply; 85+ messages in thread
From: Andreas Olsowski @ 2011-03-28 11:37 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel, giamteckchoon
[-- Attachment #1.1.1: Type: text/plain, Size: 1366 bytes --]
> - turn on CONFIG_DEBUG_PAGEALLOC
> - turn on CONFIG_DEBUG_LIST
> - turn on CONFIG_DEBUG_KMEMLEAK
> - turn on CONFIG_JBD_DEBUG, CONFIG_JBD2_DEBUG
> - turn on CONFIG_SLUB_DEBUG_ON
After i enabled those options (i dont use SLUB, i use SLAB) i do no
longer encounter any errors.
I completed 1000 loops of snapshot/mount/umoun/removesnapshot.
Without those options in 2.6.32.35 i hit a different bug earlier today:
But you really have to be patient to see some output, because lvremove
will hang quite a while:
(a "while" beeing the a a roughly the time it takes for: wait 5 min for
error, leave office, get coffee, smoke cigarette, goto restroom, return
to office, finally see error)
kernel: BUG: unable to handle kernel paging request
...
kernel: RIP [<ffffffff8100f2bf>] xen_set_pmd+0x2f/0xb0
syslog/dmesg output is attached as crash.2.6.32.35-xen_01 or available at:
http://pastebin.com/Ad8MhUzD
After that happened i did a kernel recompile without rebooting the
machine first and encoundeterd system_call_fastpath as last call once
more as shown in crash.2.6.32.35-xen_02 or http://pastebin.com/kB38W5mp
Maybe this helps, but i think, if anything, this makes it worse as the
debug options actually supressed the problem that needs to be debugged.
with best regards
--
Andreas "andiolsi "Olsowski
[-- Attachment #1.1.2: crash.2.6.32.35-xen_01 --]
[-- Type: text/plain, Size: 5292 bytes --]
Mar 28 08:07:59 netcatarina udevd[1019]: worker [5115] unexpectedly returned with status 0x0100
Mar 28 08:07:59 netcatarina udevd[1019]: worker [5115] failed while handling '/devices/virtual/block/dm-35'
Mar 28 08:08:02 netcatarina kernel: BUG: unable to handle kernel paging request at ffff8800c5d0d678
Mar 28 08:08:02 netcatarina kernel: IP: [<ffffffff8100f2bf>] xen_set_pmd+0x2f/0xb0
Mar 28 08:08:02 netcatarina kernel: PGD 1002067 PUD 6e8067 PMD 717067 PTE 100000c5d0d065
Mar 28 08:08:02 netcatarina kernel: Oops: 0003 [#1] SMP
Message from syslogd@netcatarina at Mar 28 08:08:02 ...
kernel:Oops: 0003 [#1] SMP
Mar 28 08:08:02 netcatarina kernel: last sysfs file: /sys/devices/pci0000:00/0000:00:07.0/0000:04:00.1/host2/rport-2:0-2/target2:0:2/2:0:2:0/state
Message from syslogd@netcatarina at Mar 28 08:08:02 ...
kernel:last sysfs file: /sys/devices/pci0000:00/0000:00:07.0/0000:04:00.1/host2/rport-2:0-2/target2:0:2/2:0:2:0/state
Mar 28 08:08:02 netcatarina kernel: CPU 2
Mar 28 08:08:02 netcatarina kernel: Modules linked in: iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi dm_round_robin dm_multipath qla2xxx
Mar 28 08:08:02 netcatarina kernel: Pid: 5182, comm: udevd Not tainted 2.6.32.35-xen #1 PowerEdge R610
Mar 28 08:08:02 netcatarina kernel: RIP: e030:[<ffffffff8100f2bf>] [<ffffffff8100f2bf>] xen_set_pmd+0x2f/0xb0
Mar 28 08:08:02 netcatarina kernel: RSP: e02b:ffff8800c280fd48 EFLAGS: 00010246
Mar 28 08:08:02 netcatarina kernel: RAX: 0000000000000000 RBX: ffff8800c5d0d678 RCX: ffff880000000558
Mar 28 08:08:02 netcatarina kernel: RDX: ffffea0000000000 RSI: 0000000000000000 RDI: ffff8800c5d0d678
Mar 28 08:08:02 netcatarina kernel: RBP: 0000000000000000 R08: 00007faadb57c000 R09: dead000000200200
Mar 28 08:08:02 netcatarina kernel: R10: dead000000100100 R11: 0000000000000000 R12: ffff8800c5d0d678
Mar 28 08:08:02 netcatarina kernel: R13: 00000000c2b01067 R14: ffff880028079fc0 R15: 00007faadb1fbfff
Mar 28 08:08:02 netcatarina kernel: FS: 00007faadafcd7a0(0000) GS:ffff88002806d000(0000) knlGS:0000000000000000
Mar 28 08:08:02 netcatarina kernel: CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
Mar 28 08:08:02 netcatarina kernel: CR2: ffff8800c5d0d678 CR3: 0000000001001000 CR4: 0000000000002660
Mar 28 08:08:02 netcatarina kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Mar 28 08:08:02 netcatarina kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Mar 28 08:08:02 netcatarina kernel: Process udevd (pid: 5182, threadinfo ffff8800c280e000, task ffff8800c2b4e630)
Mar 28 08:08:02 netcatarina kernel: Stack:
Message from syslogd@netcatarina at Mar 28 08:08:02 ...
kernel:Stack:
Mar 28 08:08:02 netcatarina kernel: 00007faad9e01000 00007faada000000 00007faadb1fc000 ffffffff8109e613
Mar 28 08:08:02 netcatarina kernel: <0> ffffffff81030dad ffffea0002a8c630 00007faad9e00000 ffff8800c285a558
Mar 28 08:08:02 netcatarina kernel: <0> 00007faadb1fc000 00007faad9e00000 00007faadb1fbfff ffff8800c52217f8
Mar 28 08:08:02 netcatarina kernel: Call Trace:
Message from syslogd@netcatarina at Mar 28 08:08:02 ...
kernel:Call Trace:
Mar 28 08:08:02 netcatarina kernel: [<ffffffff8109e613>] ? free_pgd_range+0x253/0x490
Mar 28 08:08:02 netcatarina kernel: [<ffffffff81030dad>] ? pvclock_clocksource_read+0x4d/0xb0
Mar 28 08:08:02 netcatarina kernel: [<ffffffff8109e8e9>] ? free_pgtables+0x99/0xb0
Mar 28 08:08:02 netcatarina kernel: [<ffffffff810a3ca0>] ? exit_mmap+0xd0/0x170
Mar 28 08:08:02 netcatarina kernel: [<ffffffff81049902>] ? mmput+0x32/0x100
Mar 28 08:08:02 netcatarina kernel: [<ffffffff8104de52>] ? exit_mm+0x102/0x140
Mar 28 08:08:02 netcatarina kernel: [<ffffffff8150519c>] ? _spin_lock_irq+0xc/0x10
Mar 28 08:08:02 netcatarina kernel: [<ffffffff81078e31>] ? acct_collect+0x161/0x1b0
Mar 28 08:08:02 netcatarina kernel: [<ffffffff8104fe2e>] ? do_exit+0x67e/0x750
Mar 28 08:08:02 netcatarina kernel: [<ffffffff8103490b>] ? do_page_fault+0x10b/0x310
Mar 28 08:08:02 netcatarina kernel: [<ffffffff8104ff50>] ? do_group_exit+0x50/0xc0
Mar 28 08:08:02 netcatarina kernel: [<ffffffff8104ffd2>] ? sys_exit_group+0x12/0x20
Mar 28 08:08:02 netcatarina kernel: [<ffffffff8150587a>] ? error_exit+0x2a/0x60
Mar 28 08:08:02 netcatarina kernel: [<ffffffff81013f02>] ? system_call_fastpath+0x16/0x1b
Mar 28 08:08:02 netcatarina kernel: Code: 48 89 5c 24 08 48 89 6c 24 10 48 89 fb 48 89 f5 80 3d 01 af 7b 00 00 75 53 ff 05 45 ae 7b 00 48 89 df e8 55 d7 ff ff 84 c0 75 19 <48> 89 2b 48 8b 5c 24 08 48 8b 6c 24 10 48 83 c4 18 c3 0f 1f 80
Message from syslogd@netcatarina at Mar 28 08:08:02 ...
kernel:Code: 48 89 5c 24 08 48 89 6c 24 10 48 89 fb 48 89 f5 80 3d 01 af 7b 00 00 75 53 ff 05 45 ae 7b 00 48 89 df e8 55 d7 ff ff 84 c0 75 19 <48> 89 2b 48 8b 5c 24 08 48 8b 6c 24 10 48 83 c4 18 c3 0f 1f 80
Mar 28 08:08:02 netcatarina kernel: RIP [<ffffffff8100f2bf>] xen_set_pmd+0x2f/0xb0
Mar 28 08:08:02 netcatarina kernel: RSP <ffff8800c280fd48>
Mar 28 08:08:02 netcatarina kernel: CR2: ffff8800c5d0d678
Message from syslogd@netcatarina at Mar 28 08:08:02 ...
kernel:CR2: ffff8800c5d0d678
Mar 28 08:08:02 netcatarina kernel: ---[ end trace cc5a29f516acc76b ]---
Mar 28 08:08:02 netcatarina kernel: Fixing recursive fault but reboot is needed!
[-- Attachment #1.1.3: crash.2.6.32.35-xen_02 --]
[-- Type: text/plain, Size: 4216 bytes --]
perl invoked oom-killer: gfp_mask=0xd0, order=1, oom_adj=0
Pid: 7146, comm: perl Tainted: G D 2.6.32.35-xen #1
Call Trace:
[<ffffffff810894cd>] ? T.437+0xcd/0x290
[<ffffffff810897a3>] ? __out_of_memory+0x113/0x180
[<ffffffff810898ea>] ? out_of_memory+0xda/0x160
[<ffffffff8108c790>] ? __alloc_pages_nodemask+0x680/0x690
[<ffffffff810b7d4b>] ? kmem_getpages+0x4b/0x140
[<ffffffff810b878b>] ? fallback_alloc+0x14b/0x1e0
[<ffffffff810b8e92>] ? kmem_cache_alloc+0xd2/0x110
[<ffffffff8104a328>] ? copy_process+0xb8/0x1170
[<ffffffff81030dad>] ? pvclock_clocksource_read+0x4d/0xb0
[<ffffffff8104b466>] ? do_fork+0x86/0x3b0
[<ffffffff81505235>] ? _spin_lock+0x5/0x10
[<ffffffff810c8d72>] ? do_pipe_flags+0xc2/0x110
[<ffffffff810141a3>] ? stub_clone+0x13/0x20
[<ffffffff81013f02>] ? system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
CPU 4: hi: 0, btch: 1 usd: 0
CPU 5: hi: 0, btch: 1 usd: 0
CPU 6: hi: 0, btch: 1 usd: 0
CPU 7: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
CPU 4: hi: 186, btch: 31 usd: 0
CPU 5: hi: 186, btch: 31 usd: 0
CPU 6: hi: 186, btch: 31 usd: 0
CPU 7: hi: 186, btch: 31 usd: 0
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
CPU 4: hi: 186, btch: 31 usd: 0
CPU 5: hi: 186, btch: 31 usd: 0
CPU 6: hi: 186, btch: 31 usd: 0
CPU 7: hi: 186, btch: 31 usd: 0
active_anon:269 inactive_anon:121 isolated_anon:96
active_file:0 inactive_file:8 isolated_file:33
unevictable:4282 dirty:0 writeback:73 unstable:0
free:79907 slab_reclaimable:32585 slab_unreclaimable:318961
mapped:990 shmem:6 pagetables:718 bounce:0
Node 0 DMA free:9416kB min:4kB low:4kB high:4kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:9224kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 3243 39603 39603
Node 0 DMA32 free:300480kB min:2084kB low:2604kB high:3124kB active_anon:1076kB inactive_anon:484kB active_file:0kB inactive_file:32kB unevictable:17128kB isolated(anon):384kB isolated(file):132kB present:3321540kB mlocked:17128kB dirty:0kB writeback:292kB mapped:3960kB shmem:24kB slab_reclaimable:129184kB slab_unreclaimable:1275724kB kernel_stack:791040kB pagetables:2872kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:138 all_unreclaimable? no
lowmem_reserve[]: 0 0 36360 36360
Node 0 Normal free:9732kB min:23380kB low:29224kB high:35068kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:37232640kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:1156kB slab_unreclaimable:120kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 2*4kB 0*8kB 2*16kB 1*32kB 2*64kB 2*128kB 1*256kB 1*512kB 0*1024kB 0*2048kB 2*4096kB = 9416kB
Node 0 DMA32: 74880*4kB 141*8kB 0*16kB 1*32kB 1*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 301000kB
Node 0 Normal: 11*4kB 5*8kB 1*16kB 1*32kB 2*64kB 0*128kB 3*256kB 3*512kB 1*1024kB 3*2048kB 0*4096kB = 9732kB
1295 total pagecache pages
235 pages in swap cache
Swap cache stats: add 199389, delete 199154, find 38746/51504
Free swap = 3882788kB
Total swap = 3943416kB
10289152 pages RAM
9620405 pages reserved
9071 pages shared
458700 pages non-shared
Out of memory: kill process 2701 (dbus-daemon) score 5815 or a child
Killed process 2701 (dbus-daemon)
[-- Attachment #1.2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 6595 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860! - ideas.
2011-03-28 11:37 ` Andreas Olsowski
@ 2011-03-28 12:29 ` Teck Choon Giam
2011-04-05 22:01 ` Dave Hunter
0 siblings, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2011-03-28 12:29 UTC (permalink / raw)
To: Andreas Olsowski; +Cc: xen-devel, Konrad Rzeszutek Wilk
On Mon, Mar 28, 2011 at 7:37 PM, Andreas Olsowski
<andreas.olsowski@leuphana.de> wrote:
>
>> - turn on CONFIG_DEBUG_PAGEALLOC
>> - turn on CONFIG_DEBUG_LIST
>> - turn on CONFIG_DEBUG_KMEMLEAK
>> - turn on CONFIG_JBD_DEBUG, CONFIG_JBD2_DEBUG
>> - turn on CONFIG_SLUB_DEBUG_ON
>
> After i enabled those options (i dont use SLUB, i use SLAB) i do no longer
> encounter any errors.
>
> I completed 1000 loops of snapshot/mount/umoun/removesnapshot.
Did you try with just CONFIG_DEBUG_PAGEALLOC=y and leave the rest
unchange of your config? My testing all narrow down to
CONFIG_DEBUG_PAGEALLOC=y to prevent this BUG.
>
>
> Without those options in 2.6.32.35 i hit a different bug earlier today:
>
> But you really have to be patient to see some output, because lvremove will
> hang quite a while:
> (a "while" beeing the a a roughly the time it takes for: wait 5 min for
> error, leave office, get coffee, smoke cigarette, goto restroom, return to
> office, finally see error)
>
> kernel: BUG: unable to handle kernel paging request
> ...
> kernel: RIP [<ffffffff8100f2bf>] xen_set_pmd+0x2f/0xb0
> syslog/dmesg output is attached as crash.2.6.32.35-xen_01 or available at:
> http://pastebin.com/Ad8MhUzD
I hit this before:
# grep 'xen_set_pmd' /var/log/messages*
/var/log/messages:Mar 27 09:31:14 xen05 kernel: IP:
[<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages:Mar 27 09:31:14 xen05 kernel: RIP:
e030:[<ffffffff8100e2d4>] [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages:Mar 27 09:31:14 xen05 kernel: RIP
[<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages:Mar 27 09:06:10 xen05 kernel: IP:
[<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages:Mar 27 09:06:10 xen05 kernel: RIP:
e030:[<ffffffff8100e2d4>] [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages:Mar 27 09:06:10 xen05 kernel: RIP
[<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages:Mar 27 15:18:57 xen05 kernel: IP:
[<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages:Mar 27 15:18:57 xen05 kernel: RIP:
e030:[<ffffffff8100e2d4>] [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages:Mar 27 15:18:57 xen05 kernel: RIP
[<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages.1:Mar 23 11:00:16 xen05 kernel: IP:
[<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages.1:Mar 23 11:00:16 xen05 kernel: RIP:
e030:[<ffffffff8100e2d4>] [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
/var/log/messages.1:Mar 23 11:00:17 xen05 kernel: RIP
[<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
But unable to reproduce when CONFIG_DEBUG_PAGEALLOC=y.
>
> After that happened i did a kernel recompile without rebooting the machine
> first and encoundeterd system_call_fastpath as last call once more as shown
> in crash.2.6.32.35-xen_02 or http://pastebin.com/kB38W5mp
I hit this at least once but unable to when CONFIG_DEBUG_PAGEALLOC=y:
/var/log/messages-Mar 27 17:04:39 xen05 kernel: ------------[ cut here
]------------
/var/log/messages-Mar 27 17:04:39 xen05 kernel: kernel BUG at
arch/x86/xen/mmu.c:1872!
/var/log/messages-Mar 27 17:04:39 xen05 kernel: invalid opcode: 0000 [#1] SMP
/var/log/messages-Mar 27 17:04:39 xen05 kernel: last sysfs file:
/sys/block/sdd/dev
/var/log/messages-Mar 27 17:04:39 xen05 kernel: CPU 2
/var/log/messages-Mar 27 17:04:39 xen05 kernel: Modules linked in:
ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4
xt_state nf_conntrack ipt_REJECT xt_tcpudp xt_physdev iptable_filter
ip_tables x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic uio ipv6
cxgb3i cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi
dm_multipath scsi_dh video backlight output sbs sbshc power_meter
hwmon battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc lp
parport tg3 libphy sg ide_cd_mod cdrom serio_raw button tpm_tis tpm
tpm_bios i2c_i801 i2c_core shpchp iTCO_wdt pcspkr dm_snapshot dm_zero
dm_mirror dm_region_hash dm_log dm_mod ata_piix libata sd_mod scsi_mod
raid1 ext3 jbd uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
/var/log/messages-Mar 27 17:04:39 xen05 kernel: Pid: 5874, comm:
lvcreate Not tainted 2.6.32.35-4.xen.pvops.choon.centos5 #1 PowerEdge
860
/var/log/messages-Mar 27 17:04:39 xen05 kernel: RIP:
e030:[<ffffffff8100cb5b>] [<ffffffff8100cb5b>]
pin_pagetable_pfn+0x53/0x59
/var/log/messages-Mar 27 17:04:39 xen05 kernel: RSP:
e02b:ffff8800303d1c28 EFLAGS: 00010282
/var/log/messages-Mar 27 17:04:39 xen05 kernel: RAX: 00000000ffffffea
RBX: 000000000003032d RCX: 0000000000000181
/var/log/messages-Mar 27 17:04:39 xen05 kernel: RDX: 00000000deadbeef
RSI: 00000000deadbeef RDI: 00000000deadbeef
/var/log/messages-Mar 27 17:04:39 xen05 kernel: RBP: ffff8800303d1c48
R08: 0000000000000968 R09: ffff880000000000
/var/log/messages-Mar 27 17:04:39 xen05 kernel: R10: 00000000deadbeef
R11: ffff8800303d1d08 R12: 0000000000000003
/var/log/messages-Mar 27 17:04:39 xen05 kernel: R13: 000000000003032d
R14: ffff880030360000 R15: 00007fd324a00000
/var/log/messages-Mar 27 17:04:39 xen05 kernel: FS:
00007fd327d2e710(0000) GS:ffff880028089000(0000)
knlGS:0000000000000000
/var/log/messages-Mar 27 17:04:39 xen05 kernel: CS: e033 DS: 0000 ES:
0000 CR0: 000000008005003b
/var/log/messages-Mar 27 17:04:39 xen05 kernel: CR2: 00000000004612f0
CR3: 000000003a025000 CR4: 0000000000002660
/var/log/messages-Mar 27 17:04:39 xen05 kernel: DR0: 0000000000000000
DR1: 0000000000000000 DR2: 0000000000000000
/var/log/messages-Mar 27 17:04:39 xen05 kernel: DR3: 0000000000000000
DR6: 00000000ffff0ff0 DR7: 0000000000000400
/var/log/messages-Mar 27 17:04:39 xen05 kernel: Process lvcreate (pid:
5874, threadinfo ffff8800303d0000, task ffff880030360000)
/var/log/messages-Mar 27 17:04:39 xen05 kernel: Stack:
/var/log/messages-Mar 27 17:04:39 xen05 kernel: 0000000000000000
00000000002027a9 000000013eb43318 000000000003032d
/var/log/messages-Mar 27 17:04:39 xen05 kernel: <0> ffff8800303d1c68
ffffffff8100e07c ffff880032be05c0 ffff880032aa9928
/var/log/messages-Mar 27 17:04:39 xen05 kernel: <0> ffff8800303d1c78
ffffffff8100e0af ffff8800303d1cb8 ffffffff810a4433
/var/log/messages-Mar 27 17:04:39 xen05 kernel: Call Trace:
/var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff8100e07c>]
xen_alloc_ptpage+0x64/0x69
/var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff8100e0af>]
xen_alloc_pte+0xe/0x10
/var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff810a4433>]
__pte_alloc+0x70/0xce
/var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff810a45d1>]
handle_mm_fault+0x140/0x8b9
/var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff810a50c9>]
__get_user_pages+0x37f/0x479
/var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff810a76ca>]
__mlock_vma_pages_range+0xc0/0x16f
/var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff8131c03f>]
? _spin_unlock_irqrestore+0x11/0x13
/var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff810a78db>]
mlock_fixup+0x162/0x199
/var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff810a7989>]
do_mlockall+0x77/0x8d
/var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff81139016>]
? security_capable+0x27/0x29
/var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff810a7ce2>]
sys_mlockall+0x8f/0xb9
/var/log/messages:Mar 27 17:04:39 xen05 kernel: [<ffffffff81012ac2>]
system_call_fastpath+0x16/0x1b
/var/log/messages-Mar 27 17:04:39 xen05 kernel: Code: 48 b8 ff ff ff
ff ff ff ff 7f 48 21 c2 48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2
41 ba f0 7f 00 00 e8 e9 c7 ff ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40
f6 c7 01 48 89 e5 53 48 89 fb 74 5b 48
/var/log/messages-Mar 27 17:04:39 xen05 kernel: RIP
[<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
/var/log/messages-Mar 27 17:04:39 xen05 kernel: RSP <ffff8800303d1c28>
/var/log/messages-Mar 27 17:04:39 xen05 kernel: ---[ end trace
bf36c55d2ecd52e5 ]---
>
>
> Maybe this helps, but i think, if anything, this makes it worse as the debug
> options actually supressed the problem that needs to be debugged.
True. At least now we know/narrow down to just related to
CONFIG_DEBUG_PAGEALLOC. Maybe Konrad or Jeremy can have a closer look
in the related codes... ...
Thanks.
Kindest regards,
Giam Teck Choon
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860! - ideas.
2011-03-28 12:29 ` Teck Choon Giam
@ 2011-04-05 22:01 ` Dave Hunter
2011-04-05 22:15 ` Teck Choon Giam
0 siblings, 1 reply; 85+ messages in thread
From: Dave Hunter @ 2011-04-05 22:01 UTC (permalink / raw)
To: xen-devel
Hi guys,
This thread has gone quiet for a while and I was wondering if a solution
had been found?
I'm currently running the packaged version of Xen 4.0.1 in Debian
Squeeze and everything runs well, except for the random crashing when
using LVM.
I use LVM for the disk partitions, and use live snapshots as part of our
backup routine. That is, create snapshot -> mount snapshot -> rsync ->
umount snapshot -> remove snapshot.
Cheers,
Dave Hunter.
On Mon, 2011-03-28 at 20:29 +0800, Teck Choon Giam wrote:
> On Mon, Mar 28, 2011 at 7:37 PM, Andreas Olsowski
> <andreas.olsowski@leuphana.de> wrote:
> >
> >> - turn on CONFIG_DEBUG_PAGEALLOC
> >> - turn on CONFIG_DEBUG_LIST
> >> - turn on CONFIG_DEBUG_KMEMLEAK
> >> - turn on CONFIG_JBD_DEBUG, CONFIG_JBD2_DEBUG
> >> - turn on CONFIG_SLUB_DEBUG_ON
> >
> > After i enabled those options (i dont use SLUB, i use SLAB) i do no longer
> > encounter any errors.
> >
> > I completed 1000 loops of snapshot/mount/umoun/removesnapshot.
>
> Did you try with just CONFIG_DEBUG_PAGEALLOC=y and leave the rest
> unchange of your config? My testing all narrow down to
> CONFIG_DEBUG_PAGEALLOC=y to prevent this BUG.
>
> >
> >
> > Without those options in 2.6.32.35 i hit a different bug earlier today:
> >
> > But you really have to be patient to see some output, because lvremove will
> > hang quite a while:
> > (a "while" beeing the a a roughly the time it takes for: wait 5 min for
> > error, leave office, get coffee, smoke cigarette, goto restroom, return to
> > office, finally see error)
> >
> > kernel: BUG: unable to handle kernel paging request
> > ...
> > kernel: RIP [<ffffffff8100f2bf>] xen_set_pmd+0x2f/0xb0
> > syslog/dmesg output is attached as crash.2.6.32.35-xen_01 or available at:
> > http://pastebin.com/Ad8MhUzD
>
> I hit this before:
>
> # grep 'xen_set_pmd' /var/log/messages*
> /var/log/messages:Mar 27 09:31:14 xen05 kernel: IP:
> [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
> /var/log/messages:Mar 27 09:31:14 xen05 kernel: RIP:
> e030:[<ffffffff8100e2d4>] [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
> /var/log/messages:Mar 27 09:31:14 xen05 kernel: RIP
> [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
> /var/log/messages:Mar 27 09:06:10 xen05 kernel: IP:
> [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
> /var/log/messages:Mar 27 09:06:10 xen05 kernel: RIP:
> e030:[<ffffffff8100e2d4>] [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
> /var/log/messages:Mar 27 09:06:10 xen05 kernel: RIP
> [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
> /var/log/messages:Mar 27 15:18:57 xen05 kernel: IP:
> [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
> /var/log/messages:Mar 27 15:18:57 xen05 kernel: RIP:
> e030:[<ffffffff8100e2d4>] [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
> /var/log/messages:Mar 27 15:18:57 xen05 kernel: RIP
> [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
> /var/log/messages.1:Mar 23 11:00:16 xen05 kernel: IP:
> [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
> /var/log/messages.1:Mar 23 11:00:16 xen05 kernel: RIP:
> e030:[<ffffffff8100e2d4>] [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
> /var/log/messages.1:Mar 23 11:00:17 xen05 kernel: RIP
> [<ffffffff8100e2d4>] xen_set_pmd+0x16/0x2b
>
> But unable to reproduce when CONFIG_DEBUG_PAGEALLOC=y.
>
> >
> > After that happened i did a kernel recompile without rebooting the machine
> > first and encoundeterd system_call_fastpath as last call once more as shown
> > in crash.2.6.32.35-xen_02 or http://pastebin.com/kB38W5mp
>
> I hit this at least once but unable to when CONFIG_DEBUG_PAGEALLOC=y:
>
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: ------------[ cut here
> ]------------
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: kernel BUG at
> arch/x86/xen/mmu.c:1872!
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: invalid opcode: 0000 [#1] SMP
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: last sysfs file:
> /sys/block/sdd/dev
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: CPU 2
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: Modules linked in:
> ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4
> xt_state nf_conntrack ipt_REJECT xt_tcpudp xt_physdev iptable_filter
> ip_tables x_tables bridge stp be2iscsi iscsi_tcp bnx2i cnic uio ipv6
> cxgb3i cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi
> dm_multipath scsi_dh video backlight output sbs sbshc power_meter
> hwmon battery acpi_memhotplug xen_acpi_memhotplug ac parport_pc lp
> parport tg3 libphy sg ide_cd_mod cdrom serio_raw button tpm_tis tpm
> tpm_bios i2c_i801 i2c_core shpchp iTCO_wdt pcspkr dm_snapshot dm_zero
> dm_mirror dm_region_hash dm_log dm_mod ata_piix libata sd_mod scsi_mod
> raid1 ext3 jbd uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: Pid: 5874, comm:
> lvcreate Not tainted 2.6.32.35-4.xen.pvops.choon.centos5 #1 PowerEdge
> 860
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: RIP:
> e030:[<ffffffff8100cb5b>] [<ffffffff8100cb5b>]
> pin_pagetable_pfn+0x53/0x59
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: RSP:
> e02b:ffff8800303d1c28 EFLAGS: 00010282
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: RAX: 00000000ffffffea
> RBX: 000000000003032d RCX: 0000000000000181
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: RDX: 00000000deadbeef
> RSI: 00000000deadbeef RDI: 00000000deadbeef
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: RBP: ffff8800303d1c48
> R08: 0000000000000968 R09: ffff880000000000
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: R10: 00000000deadbeef
> R11: ffff8800303d1d08 R12: 0000000000000003
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: R13: 000000000003032d
> R14: ffff880030360000 R15: 00007fd324a00000
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: FS:
> 00007fd327d2e710(0000) GS:ffff880028089000(0000)
> knlGS:0000000000000000
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: CS: e033 DS: 0000 ES:
> 0000 CR0: 000000008005003b
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: CR2: 00000000004612f0
> CR3: 000000003a025000 CR4: 0000000000002660
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: DR0: 0000000000000000
> DR1: 0000000000000000 DR2: 0000000000000000
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: DR3: 0000000000000000
> DR6: 00000000ffff0ff0 DR7: 0000000000000400
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: Process lvcreate (pid:
> 5874, threadinfo ffff8800303d0000, task ffff880030360000)
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: Stack:
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: 0000000000000000
> 00000000002027a9 000000013eb43318 000000000003032d
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: <0> ffff8800303d1c68
> ffffffff8100e07c ffff880032be05c0 ffff880032aa9928
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: <0> ffff8800303d1c78
> ffffffff8100e0af ffff8800303d1cb8 ffffffff810a4433
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: Call Trace:
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff8100e07c>]
> xen_alloc_ptpage+0x64/0x69
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff8100e0af>]
> xen_alloc_pte+0xe/0x10
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff810a4433>]
> __pte_alloc+0x70/0xce
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff810a45d1>]
> handle_mm_fault+0x140/0x8b9
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff810a50c9>]
> __get_user_pages+0x37f/0x479
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff810a76ca>]
> __mlock_vma_pages_range+0xc0/0x16f
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff8131c03f>]
> ? _spin_unlock_irqrestore+0x11/0x13
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff810a78db>]
> mlock_fixup+0x162/0x199
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff810a7989>]
> do_mlockall+0x77/0x8d
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff81139016>]
> ? security_capable+0x27/0x29
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: [<ffffffff810a7ce2>]
> sys_mlockall+0x8f/0xb9
> /var/log/messages:Mar 27 17:04:39 xen05 kernel: [<ffffffff81012ac2>]
> system_call_fastpath+0x16/0x1b
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: Code: 48 b8 ff ff ff
> ff ff ff ff 7f 48 21 c2 48 89 55 e8 48 8d 7d e0 be 01 00 00 00 31 d2
> 41 ba f0 7f 00 00 e8 e9 c7 ff ff 85 c0 74 04 <0f> 0b eb fe c9 c3 55 40
> f6 c7 01 48 89 e5 53 48 89 fb 74 5b 48
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: RIP
> [<ffffffff8100cb5b>] pin_pagetable_pfn+0x53/0x59
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: RSP <ffff8800303d1c28>
> /var/log/messages-Mar 27 17:04:39 xen05 kernel: ---[ end trace
> bf36c55d2ecd52e5 ]---
>
> >
> >
> > Maybe this helps, but i think, if anything, this makes it worse as the debug
> > options actually supressed the problem that needs to be debugged.
>
> True. At least now we know/narrow down to just related to
> CONFIG_DEBUG_PAGEALLOC. Maybe Konrad or Jeremy can have a closer look
> in the related codes... ...
>
> Thanks.
>
> Kindest regards,
> Giam Teck Choon
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860! - ideas.
2011-04-05 22:01 ` Dave Hunter
@ 2011-04-05 22:15 ` Teck Choon Giam
2011-04-05 23:20 ` Dave Hunter
0 siblings, 1 reply; 85+ messages in thread
From: Teck Choon Giam @ 2011-04-05 22:15 UTC (permalink / raw)
To: Dave Hunter; +Cc: xen-devel
On Wed, Apr 6, 2011 at 6:01 AM, Dave Hunter <dave@ivt.com.au> wrote:
> Hi guys,
>
> This thread has gone quiet for a while and I was wondering if a solution
> had been found?
>
> I'm currently running the packaged version of Xen 4.0.1 in Debian
> Squeeze and everything runs well, except for the random crashing when
> using LVM.
>
> I use LVM for the disk partitions, and use live snapshots as part of our
> backup routine. That is, create snapshot -> mount snapshot -> rsync ->
> umount snapshot -> remove snapshot.
>
> Cheers,
>
> Dave Hunter.
Solution is to compile your PVOPS 2.6.32.x kernel with CONFIG_DEBUG_PAGEALLOC=y.
With CONFIG_DEBUG_PAGEALLOC=y, it pass my testcrash loop 10000.
Thanks.
Kindest regards,
Giam Teck Choon
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860! - ideas.
2011-04-05 22:15 ` Teck Choon Giam
@ 2011-04-05 23:20 ` Dave Hunter
2011-04-06 7:53 ` Ian Campbell
0 siblings, 1 reply; 85+ messages in thread
From: Dave Hunter @ 2011-04-05 23:20 UTC (permalink / raw)
To: Teck Choon Giam; +Cc: xen-devel
Thanks!
Is it likely that Debian would release an updated kernel in squeeze with
this configuration? (sorry, this might not be the place to ask).
Dave.
On Wed, 2011-04-06 at 06:15 +0800, Teck Choon Giam wrote:
> On Wed, Apr 6, 2011 at 6:01 AM, Dave Hunter <dave@ivt.com.au> wrote:
> > Hi guys,
> >
> > This thread has gone quiet for a while and I was wondering if a solution
> > had been found?
> >
> > I'm currently running the packaged version of Xen 4.0.1 in Debian
> > Squeeze and everything runs well, except for the random crashing when
> > using LVM.
> >
> > I use LVM for the disk partitions, and use live snapshots as part of our
> > backup routine. That is, create snapshot -> mount snapshot -> rsync ->
> > umount snapshot -> remove snapshot.
> >
> > Cheers,
> >
> > Dave Hunter.
>
> Solution is to compile your PVOPS 2.6.32.x kernel with CONFIG_DEBUG_PAGEALLOC=y.
>
> With CONFIG_DEBUG_PAGEALLOC=y, it pass my testcrash loop 10000.
>
> Thanks.
>
> Kindest regards,
> Giam Teck Choon
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860! - ideas.
2011-04-05 23:20 ` Dave Hunter
@ 2011-04-06 7:53 ` Ian Campbell
2011-04-06 21:52 ` Jeremy Fitzhardinge
0 siblings, 1 reply; 85+ messages in thread
From: Ian Campbell @ 2011-04-06 7:53 UTC (permalink / raw)
To: Dave Hunter; +Cc: xen-devel, Teck Choon Giam
Please don't top post.
On Wed, 2011-04-06 at 00:20 +0100, Dave Hunter wrote:
> Is it likely that Debian would release an updated kernel in squeeze with
> this configuration? (sorry, this might not be the place to ask).
I doubt they will, enabling DEBUG_PAGEALLOC seems very much like a
workaround not a solution to me.
Ian.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860! - ideas.
2011-04-06 7:53 ` Ian Campbell
@ 2011-04-06 21:52 ` Jeremy Fitzhardinge
2011-04-07 13:16 ` Teck Choon Giam
0 siblings, 1 reply; 85+ messages in thread
From: Jeremy Fitzhardinge @ 2011-04-06 21:52 UTC (permalink / raw)
To: Ian Campbell; +Cc: Dave Hunter, Teck Choon Giam, xen-devel
On 04/06/2011 12:53 AM, Ian Campbell wrote:
> Please don't top post.
>
> On Wed, 2011-04-06 at 00:20 +0100, Dave Hunter wrote:
>> Is it likely that Debian would release an updated kernel in squeeze with
>> this configuration? (sorry, this might not be the place to ask).
> I doubt they will, enabling DEBUG_PAGEALLOC seems very much like a
> workaround not a solution to me.
Yes, it will impose a pretty large performance overhead.
J
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860! - ideas.
2011-04-06 21:52 ` Jeremy Fitzhardinge
@ 2011-04-07 13:16 ` Teck Choon Giam
0 siblings, 0 replies; 85+ messages in thread
From: Teck Choon Giam @ 2011-04-07 13:16 UTC (permalink / raw)
To: Jeremy Fitzhardinge; +Cc: xen-devel, Dave Hunter, Ian Campbell
On Thu, Apr 7, 2011 at 5:52 AM, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
> On 04/06/2011 12:53 AM, Ian Campbell wrote:
>> Please don't top post.
>>
>> On Wed, 2011-04-06 at 00:20 +0100, Dave Hunter wrote:
>>> Is it likely that Debian would release an updated kernel in squeeze with
>>> this configuration? (sorry, this might not be the place to ask).
>> I doubt they will, enabling DEBUG_PAGEALLOC seems very much like a
>> workaround not a solution to me.
>
> Yes, it will impose a pretty large performance overhead.
>
> J
I understand that option will impose a pretty large performance
overhead but it is currently a working trade-off/solution/workaround
;) It is better to have that trade-off than having an unstable host
server for those of us who need LVM snapshots related... IMO.
Hopefully you or Konrad can have a look into the related codes and if
need some testers... I always will try my best to test any new patches
if available to test its stability in this... ...
Thanks.
Kindest regards,
Giam Teck Choon
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
@ 2011-04-15 7:37 Joerg Stephan
0 siblings, 0 replies; 85+ messages in thread
From: Joerg Stephan @ 2011-04-15 7:37 UTC (permalink / raw)
To: xen-devel
Hey guys,
i just wanne report that our debian squeeze xen machines are also
influenced by this bug, and we tried the workaround with the debug option
by building our own
linux-image-2.6.32-5-xen-amd64_2.6.32-31_amd64.deb
but this isnt a workaround really, we tested it on an 40 vmachines dom0
and it just dont work, of course the lvm works but the machine is slow as
hell and runs into several kernel errors within a few hours
[46566.052925] BUG: soft lockup - CPU#0 stuck for 61s! [qemu-dm:6292]
in this case xentop shows an 500% CPU usage by the dom0, we experience an
85%st value.
So, i think this should be fixed soon, so what is the status of the issue?
Best whishes
Joerg
^ permalink raw reply [flat|nested] 85+ messages in thread
* RE: kernel BUG at arch/x86/xen/mmu.c:1860!
2011-04-08 11:24 ` MaoXiaoyun
@ 2011-04-08 11:46 ` MaoXiaoyun
0 siblings, 0 replies; 85+ messages in thread
From: MaoXiaoyun @ 2011-04-08 11:46 UTC (permalink / raw)
To: xen devel; +Cc: jeremy, dave, giamteckchoon, ian.campbell, konrad.wilk
[-- Attachment #1.1: Type: text/plain, Size: 3144 bytes --]
HI:
As I go through the code with log, I noticed that the log:
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
is from xen/common/grant_table.c:266, which is in function _set_status_v1()
so it looks like kernel 2.6.32 use grant table version 1.
While in 2.6.31. driver/xen/grant-table.c, I noticed function gnttab_request_version()
which looks like 2.6.31 require grant version 2. But this function cannot be found
in 2.6.32.
Is this correct?
Thanks.
>--------------------------------------------------------------------------------
>From: tinnycloud@hotmail.com
>To: xen-devel@lists.xensource.com
>CC: dave@ivt.com.au; ian.campbell@citrix.com; giamteckchoon@gmail.com; konrad.wilk@oracle.com; jeremy@goop.org
>Subject: Re: kernel BUG at arch/x86/xen/mmu.c:1860!
>Date: Fri, 8 Apr 2011 19:24:35 +0800
>
>Hi:
> Unfortunately I met the exactly same bug today. With pvops kernel 2.6.32.36, and xen 4.0.1.
> Kernel Panic and serial log attached.
>
> Our test cases is quite simple, on a single physical host, we start 12 HVMS(windows 2003),
>each of the HVM reboot every 10minutes.
>
> The bug is easy to hit on our 48G machine(in hours). But We haven't hit the bug in our 24G
>machine(we have three 24G machine, all works fine.) -----Is is possible related to Memory capacity?
>
>Taking a look at the serial output, the Dom0 code is attempting to pin what it thins
>is a "PGT_l3_page_table", however the hypervisor returns -EINVAL because it actually is a "PGT_writable_page".
>
>(XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 4000 0000 0000 0000) for mfn 898a41 (pfn 9ca41)
>(XEN) mm.c:2733:d0 Error while pinning mfn 898a41
>
>And before that quite a lot abnormal grant table log like :
>
>(XEN) grant_table.c:1717:d0 Bad grant reference 4294965983
>(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
>(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
>(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
>(XEN) grant_table.c:1717:d0 Bad grant reference 4294965888
>(XEN) grant_table.c:1717:d0 Bad grant reference 4294965983
>
>It looks like something wrong with grant table.
>
>Many thanks.
>
>> From: Jeremy Fitzhardinge <jeremy@goop.org>
>> Subject: Re: [Xen-devel] [SPAM] Re: kernel BUG at
>> arch/x86/xen/mmu.c:1860! - ideas.
>> To: Ian Campbell <Ian.Campbell@citrix.com>
>> Cc: Dave Hunter <dave@ivt.com.au>, Teck Choon Giam
>> <giamteckchoon@gmail.com>, "xen-devel@lists.xensource.com"
>> xen-devel@lists.xensource.com
>
>> On 04/06/2011 12:53 AM, Ian Campbell wrote:
>> > Please don't top post.
>> >
>> > On Wed, 2011-04-06 at 00:20 +0100, Dave Hunter wrote:
>> >> Is it likely that Debian would release an updated kernel in squeeze with
>> >> this configuration? (sorry, this might not be the place to ask).
>> > I doubt they will, enabling DEBUG_PAGEALLOC seems very much like a
>> > workaround not a solution to me.
>>
>> Yes, it will impose a pretty large performance overhead.
>>
>> J
>>
>>
>
[-- Attachment #1.2: Type: text/html, Size: 4560 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: kernel BUG at arch/x86/xen/mmu.c:1860!
[not found] <COL0-MC1-F14hmBzxHs00230882@col0-mc1-f14.Col0.hotmail.com>
@ 2011-04-08 11:24 ` MaoXiaoyun
2011-04-08 11:46 ` MaoXiaoyun
0 siblings, 1 reply; 85+ messages in thread
From: MaoXiaoyun @ 2011-04-08 11:24 UTC (permalink / raw)
To: xen devel; +Cc: jeremy, dave, giamteckchoon, ian.campbell, konrad.wilk
[-- Attachment #1.1: Type: text/plain, Size: 2223 bytes --]
Hi:
Unfortunately I met the exactly same bug today. With pvops kernel 2.6.32.36, and xen 4.0.1.
Kernel Panic and serial log attached.
Our test cases is quite simple, on a single physical host, we start 12 HVMS(windows 2003),
each of the HVM reboot every 10minutes.
The bug is easy to hit on our 48G machine(in hours). But We haven't hit the bug in our 24G
machine(we have three 24G machine, all works fine.) -----Is is possible related to Memory capacity?
Taking a look at the serial output, the Dom0 code is attempting to pin what it thins
is a "PGT_l3_page_table", however the hypervisor returns -EINVAL because it actually is a "PGT_writable_page".
(XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 4000 0000 0000 0000) for mfn 898a41 (pfn 9ca41)
(XEN) mm.c:2733:d0 Error while pinning mfn 898a41
And before that quite a lot abnormal grant table log like :
(XEN) grant_table.c:1717:d0 Bad grant reference 4294965983
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) grant_table.c:1717:d0 Bad grant reference 4294965888
(XEN) grant_table.c:1717:d0 Bad grant reference 4294965983
It looks like something wrong with grant table.
Many thanks.
> From: Jeremy Fitzhardinge <jeremy@goop.org>
> Subject: Re: [Xen-devel] [SPAM] Re: kernel BUG at
> arch/x86/xen/mmu.c:1860! - ideas.
> To: Ian Campbell <Ian.Campbell@citrix.com>
> Cc: Dave Hunter <dave@ivt.com.au>, Teck Choon Giam
> <giamteckchoon@gmail.com>, "xen-devel@lists.xensource.com"
> xen-devel@lists.xensource.com
> On 04/06/2011 12:53 AM, Ian Campbell wrote:
> > Please don't top post.
> >
> > On Wed, 2011-04-06 at 00:20 +0100, Dave Hunter wrote:
> >> Is it likely that Debian would release an updated kernel in squeeze with
> >> this configuration? (sorry, this might not be the place to ask).
> > I doubt they will, enabling DEBUG_PAGEALLOC seems very much like a
> > workaround not a solution to me.
>
> Yes, it will impose a pretty large performance overhead.
>
> J
>
>
[-- Attachment #1.2: Type: text/html, Size: 3064 bytes --]
[-- Attachment #2: kernel.txt --]
[-- Type: text/plain, Size: 4390 bytes --]
Apr 8 12:19:47 r14a11017 kernel: ------------[ cut here ]------------
Apr 8 12:19:47 r14a11017 kernel: kernel BUG at arch/x86/xen/mmu.c:1872!
Apr 8 12:19:47 r14a11017 kernel: invalid opcode: 0000 [#1] SMP
Apr 8 12:19:47 r14a11017 kernel: last sysfs file: /sys/hypervisor/properties/capabilities
Apr 8 12:19:47 r14a11017 kernel: CPU 0
Apr 8 12:19:47 r14a11017 kernel: Modules linked in: 8021q garp blktap xen_netback xen_blkback blkback_pagemap nbd bridge stp llc autofs4 ipmi_devintf ipmi_si ipmi_msghandler lockd sunrpc bonding ipv6 xenfs dm_multipath video output sbs sbshc parport_pc lp parport ses enclosure snd_seq_dummy snd_seq_oss bnx2 snd_seq_midi_event serio_raw snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer i2c_i801 iTCO_wdt i2c_core snd soundcore snd_page_alloc iTCO_vendor_support pata_acpi ata_generic pcspkr ata_piix shpchp mptsas mptscsih mptbase [last unloaded: freq_table]
Apr 8 12:19:47 r14a11017 kernel: Pid: 15769, comm: sh Not tainted 2.6.32.36xen #1 Tecal RH2285
Apr 8 12:19:47 r14a11017 kernel: RIP: e030:[<ffffffff8100cebc>] [<ffffffff8100cebc>] pin_pagetable_pfn+0x36/0x3c
Apr 8 12:19:47 r14a11017 kernel: RSP: e02b:ffff88001eb7baa8 EFLAGS: 00010282
Apr 8 12:19:47 r14a11017 kernel: RAX: 00000000ffffffea RBX: 000000000007b307 RCX: 0000000000000001
Apr 8 12:19:47 r14a11017 kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88001eb7baa8
Apr 8 12:19:47 r14a11017 kernel: RBP: ffff88001eb7bac8 R08: 0000000000000420 R09: ffff880000000000
Apr 8 12:19:47 r14a11017 kernel: R10: 0000000000007ff0 R11: ffff88008fc97248 R12: ffff88002840b000
Apr 8 12:19:47 r14a11017 kernel: R13: 000000000007b484 R14: 0000000000000003 R15: ffff88009b090000
Apr 8 12:19:47 r14a11017 kernel: FS: 00007fe8bbc656e0(0000) GS:ffff88002803b000(0000) knlGS:0000000000000000
Apr 8 12:19:47 r14a11017 kernel: CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
Apr 8 12:19:47 r14a11017 kernel: CR2: 00000000006bb338 CR3: 000000007b307000 CR4: 0000000000002660
Apr 8 12:19:47 r14a11017 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Apr 8 12:19:47 r14a11017 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Apr 8 12:19:47 r14a11017 kernel: Process sh (pid: 15769, threadinfo ffff88001eb7a000, task ffff88009b090000)
Apr 8 12:19:47 r14a11017 kernel: Stack:
Apr 8 12:19:47 r14a11017 kernel: 0000000000000000 00000000004b7484 000000011eb7bac8 000000000007b307
Apr 8 12:19:47 r14a11017 kernel: <0> ffff88001eb7baf8 ffffffff8100e8ef ffff88012e4fb100 ffff88000fb5e018
Apr 8 12:19:47 r14a11017 kernel: <0> 000000000007b484 00000000006bb338 ffff88001eb7bb08 ffffffff8100e935
Apr 8 12:19:47 r14a11017 kernel: Call Trace:
Apr 8 12:19:47 r14a11017 kernel: [<ffffffff8100e8ef>] xen_alloc_ptpage+0x8d/0x96
Apr 8 12:19:47 r14a11017 kernel: [<ffffffff8100e935>] xen_alloc_pte+0x13/0x15
Apr 8 12:19:47 r14a11017 kernel: [<ffffffff810eb702>] __pte_alloc+0x7f/0xdc
Apr 8 12:19:47 r14a11017 kernel: [<ffffffff810e90bd>] ? pmd_offset+0x13/0x3c
Apr 8 12:19:47 r14a11017 kernel: [<ffffffff810eb818>] handle_mm_fault+0xb9/0x771
Apr 8 12:19:47 r14a11017 kernel: [<ffffffff810f08fd>] ? vma_link+0x7c/0xa4
Apr 8 12:19:47 r14a11017 kernel: [<ffffffff810f13b0>] ? mmap_region+0x322/0x42b
Apr 8 12:19:47 r14a11017 kernel: [<ffffffff8100f169>] ? xen_force_evtchn_callback+0xd/0xf
Apr 8 12:19:47 r14a11017 kernel: [<ffffffff81449701>] do_page_fault+0x21c/0x288
Apr 8 12:19:47 r14a11017 kernel: [<ffffffff81447695>] page_fault+0x25/0x30
Apr 8 12:19:47 r14a11017 kernel: [<ffffffff81222a39>] ? __clear_user+0x33/0x55
Apr 8 12:19:47 r14a11017 kernel: [<ffffffff81222a1d>] ? __clear_user+0x17/0x55
Apr 8 12:19:47 r14a11017 kernel: [<ffffffff81222a8b>] clear_user+0x30/0x38
Apr 8 12:19:47 r14a11017 kernel: [<ffffffff8115139a>] load_elf_binary+0x5d5/0x17ef
Apr 8 12:19:47 r14a11017 kernel: [<ffffffff811f4648>] ? process_measurement+0xc0/0xd7
Apr 8 12:19:47 r14a11017 kernel: [<ffffffff81150dc5>] ? load_elf_binary+0x0/0x17ef
Apr 8 12:19:47 r14a11017 kernel: [<ffffffff81113094>] search_binary_handler+0xc8/0x255
Apr 8 12:19:47 r14a11017 kernel: [<ffffffff81114362>] do_execve+0x1c3/0x29e
Apr 8 12:19:47 r14a11017 kernel: [<ffffffff8101155d>] sys_execve+0x43/0x5d
Apr 8 12:19:47 r14a11017 kernel: [<ffffffff810131ca>] stub_execve+0x6a/0xc0
[-- Attachment #3: serial.txt --]
[-- Type: text/plain, Size: 4360 bytes --]
(XEN) grant_table.c:1717:d0 Bad grant reference 4294965983
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) grant_table.c:1717:d0 Bad grant reference 4294965888
(XEN) grant_table.c:1717:d0 Bad grant reference 4294965983
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) grant_table.c:1717:d0 Bad grant reference 4294965983
(XEN) grant_table.c:1717:d0 Bad grant reference 4294965983
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) printk: 1 messages suppressed.
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) printk: 1 messages suppressed.
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) printk: 5 messages suppressed.
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) printk: 14 messages suppressed.
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) printk: 7 messages suppressed.
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) printk: 1 messages suppressed.
(XEN) grant_table.c:266:d0 Bad flags (0) or dom (0). (expected dom 0)
(XEN) grant_table.c:1717:d0 Bad grant reference 4294901760
(XEN) printk: 13 messages suppressed.
(XEN) grant_table.c:1717:d0 Bad grant reference 4294901760
(XEN) printk: 59 messages suppressed.
(XEN) grant_table.c:1717:d0 Bad grant reference 4294901760
(XEN) printk: 81 messages suppressed.
(XEN) grant_table.c:1717:d0 Bad grant reference 4294901760
(XEN) printk: 75 messages suppressed.
(XEN) grant_table.c:1717:d0 Bad grant reference 4294901760
(XEN) printk: 79 messages suppressed.
(XEN) grant_table.c:1717:d0 Bad grant reference 4294901760
(XEN) printk: 81 messages suppressed.
(XEN) grant_table.c:1717:d0 Bad grant reference 4294901760
(XEN) printk: 33 messages suppressed.
(XEN) grant_table.c:1717:d0 Bad grant reference 4294901760
(XEN) printk: 9 messages suppressed.
(XEN) grant_table.c:1717:d0 Bad grant reference 4294901765
(XEN) printk: 7 messages suppressed.
(XEN) grant_table.c:1717:d0 Bad grant reference 4294901760
(XEN) printk: 1 messages suppressed.
(XEN) grant_table.c:1717:d0 Bad grant reference 4294901760
(XEN) grant_table.c:1717:d0 Bad grant reference 4294901765
(XEN) grant_table.c:1717:d0 Bad grant reference 4294901760
(XEN) grant_table.c:1717:d0 Bad grant reference 4294901765
(XEN) grant_table.c:1717:d0 Bad grant reference 4294901760
(XEN) grant_table.c:1717:d0 Bad grant reference 4294901765
(XEN) grant_table.c:1717:d0 Bad grant reference 4294901760
(XEN) grant_table.c:1717:d0 Bad grant reference 4294901765
(XEN) printk: 10 messages suppressed.
(XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 4000 0000 0000 0000) for mfn 898a41 (pfn 9ca41)
(XEN) mm.c:2733:d0 Error while pinning mfn 898a41
8000000000000000
(XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 4000000000000000) for mfn 871443 (pfn 75443)
(XEN) mm.c:2733:d0 Error while pinning mfn 871443
(XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 4000000000000000) for mfn 898a41 (pfn 9ca41)
(XEN) mm.c:2500:d0 Error while installing new baseptr 898a41
(XEN) mm.c:2364:d0 Bad type (saw 7400000000000001 != exp 4000000000000000) for mfn 871443 (pfn 75443)
(XEN) mm.c:2825:d0 Error while installing new mfn 871443
(XEN) mm.c:2364:d0 Bad type (saw 4400000000000001 != exp 7000000000000000) for mfn 899551 (pfn 9d551)
(XEN) mm.c:860:d0 Error getting mfn 899551 (pfn 9d551) from L1 entry 8000000899551063 for l1e_owner=0, pg_owner=0
[-- Attachment #4: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 85+ messages in thread
end of thread, other threads:[~2011-04-15 7:37 UTC | newest]
Thread overview: 85+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-12-26 8:16 kernel BUG at arch/x86/xen/mmu.c:1860! Teck Choon Giam
2010-12-27 15:53 ` Konrad Rzeszutek Wilk
2010-12-27 22:14 ` Teck Choon Giam
2010-12-28 10:42 ` Pasi Kärkkäinen
2010-12-28 18:01 ` Teck Choon Giam
2010-12-29 4:25 ` Teck Choon Giam
2010-12-29 4:58 ` Teck Choon Giam
2011-01-14 15:20 ` Konrad Rzeszutek Wilk
2011-01-14 19:25 ` Teck Choon Giam
2011-01-14 19:44 ` Konrad Rzeszutek Wilk
2011-01-14 20:09 ` Teck Choon Giam
2011-01-14 20:32 ` Teck Choon Giam
2011-01-24 1:42 ` Teck Choon Giam
2011-01-24 14:36 ` Konrad Rzeszutek Wilk
2011-01-24 15:56 ` Teck Choon Giam
2011-01-25 14:48 ` Konrad Rzeszutek Wilk
2011-01-26 14:31 ` Konrad Rzeszutek Wilk
2011-01-27 17:17 ` Teck Choon Giam
2011-01-27 20:32 ` Konrad Rzeszutek Wilk
2011-01-27 22:20 ` Teck Choon Giam
2011-02-26 12:03 ` Teck Choon Giam
2011-02-28 16:20 ` Konrad Rzeszutek Wilk
2011-03-01 9:59 ` Teck Choon Giam
2011-03-03 22:16 ` Konrad Rzeszutek Wilk
2011-03-04 5:30 ` Teck Choon Giam
2011-03-04 6:15 ` Fajar A. Nugraha
2011-03-04 6:33 ` Teck Choon Giam
2011-03-08 19:29 ` Konrad Rzeszutek Wilk
2011-03-08 20:10 ` Konrad Rzeszutek Wilk
2011-03-08 20:20 ` Teck Choon Giam
2011-03-08 20:45 ` Guido Hecken
2011-03-08 20:50 ` [SPAM] " tjaouen
2011-03-09 0:06 ` Andreas Olsowski
2011-03-11 18:38 ` benco
2011-03-11 19:59 ` Sander Eikelenboom
2011-03-11 20:29 ` Teck Choon Giam
2011-03-11 20:45 ` Teck Choon Giam
2011-03-11 21:02 ` Sander Eikelenboom
2011-03-11 21:15 ` Teck Choon Giam
2011-03-09 0:43 ` [SPAM] " Konrad Rzeszutek Wilk
2011-03-09 6:58 ` Andreas Olsowski
2011-03-09 15:00 ` Konrad Rzeszutek Wilk
2011-03-09 19:59 ` Andreas Olsowski
2011-03-10 7:20 ` Andreas Olsowski
2011-03-10 13:45 ` Andreas Olsowski
2011-03-11 18:05 ` Konrad Rzeszutek Wilk
2011-03-14 10:25 ` Ian Campbell
2011-03-14 10:36 ` Teck Choon Giam
2011-03-16 15:52 ` [SPAM] Re: kernel BUG at arch/x86/xen/mmu.c:1860! - ideas Konrad Rzeszutek Wilk
2011-03-16 16:26 ` Teck Choon Giam
2011-03-16 16:40 ` Konrad Rzeszutek Wilk
2011-03-24 11:57 ` Konrad Rzeszutek Wilk
2011-03-24 21:28 ` Teck Choon Giam
2011-03-25 3:57 ` Teck Choon Giam
2011-03-27 10:16 ` Teck Choon Giam
2011-03-28 11:37 ` Andreas Olsowski
2011-03-28 12:29 ` Teck Choon Giam
2011-04-05 22:01 ` Dave Hunter
2011-04-05 22:15 ` Teck Choon Giam
2011-04-05 23:20 ` Dave Hunter
2011-04-06 7:53 ` Ian Campbell
2011-04-06 21:52 ` Jeremy Fitzhardinge
2011-04-07 13:16 ` Teck Choon Giam
2011-03-09 0:41 ` kernel BUG at arch/x86/xen/mmu.c:1860! Konrad Rzeszutek Wilk
2011-01-04 15:10 ` Christophe Saout
2011-01-04 15:19 ` Christophe Saout
2011-01-04 15:37 ` benco
2011-01-04 18:40 ` Christophe Saout
2011-01-04 19:32 ` Teck Choon Giam
2011-01-04 19:56 ` benco
2011-01-14 15:22 ` Konrad Rzeszutek Wilk
2011-01-14 15:33 ` Christophe Saout
2011-01-04 23:10 ` Christophe Saout
2011-01-05 10:51 ` Pasi Kärkkäinen
2011-01-05 14:56 ` Teck Choon Giam
2011-01-14 15:24 ` Konrad Rzeszutek Wilk
2011-01-14 19:31 ` Teck Choon Giam
2011-01-04 13:48 ` Ian Campbell
2011-01-04 19:24 ` Teck Choon Giam
2011-01-05 15:30 ` Teck Choon Giam
2011-01-13 14:28 ` tjaouen
2011-01-14 14:47 ` Konrad Rzeszutek Wilk
[not found] <COL0-MC1-F14hmBzxHs00230882@col0-mc1-f14.Col0.hotmail.com>
2011-04-08 11:24 ` MaoXiaoyun
2011-04-08 11:46 ` MaoXiaoyun
2011-04-15 7:37 Joerg Stephan
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.