* [BUG] xfstests #68 with data=journal hang against 3.8-rc7 and 'dev' branch
@ 2013-02-17 17:22 Zheng Liu
2013-02-18 5:04 ` Theodore Ts'o
` (2 more replies)
0 siblings, 3 replies; 7+ messages in thread
From: Zheng Liu @ 2013-02-17 17:22 UTC (permalink / raw)
To: linux-ext4; +Cc: Theodore Ts'o
Hi all,
Xfstests #68 will hang with data=journal in 3.8-rc7 and 'dev' branch. I
remember that there has a patch for ext4 to fix filesystem freeze bug
but I am not sure whether it can fix this bug and it has been applied
into 'dev' branch. So I file this bug here.
Regards,
- Zheng
FSTYP -- ext4
PLATFORM -- Linux/x86_64 lz-desktop 3.8.0-rc3+
MKFS_OPTIONS -- /dev/sda2
MOUNT_OPTIONS -- -o acl,user_xattr,data=journal /dev/sda2 /mnt/sda2
068 46s ...
wenqing: run xfstest 068
kernel: EXT4-fs (sda2): mounted filesystem with journalled data mode.
Opts: acl,user_xattr,data=journal
kernel: ------------[ cut here ]------------
kernel: kernel BUG at fs/jbd2/transaction.c:2011!
kernel: invalid opcode: 0000 [#1] SMP
kernel: Modules linked in: ext4 jbd2 crc16 cpufreq_ondemand acpi_cpufreq
mperf ipv6 dm_mirror dm_region_hash dm_log dm_mod parport_pc parport
dcdbas pcspkr serio_raw i2c_i801 i2c_core sg button ehci_pci ehci_hcd
e1000e ext3 jbd sd_mod ahci libahci libata scsi_mod uhci_hcd
kernel: CPU 1
kernel: Pid: 10646, comm: fstest Not tainted 3.8.0-rc3+ #2 Dell Inc.
OptiPlex 780 /0V4W66
kernel: RIP: 0010:[<ffffffffa01ee9de>] [<ffffffffa01ee9de>]
jbd2_journal_invalidatepage+0x265/0x2ec [jbd2]
kernel: RSP: 0018:ffff88010cf4d948 EFLAGS: 00010206
kernel: RAX: 0000000000000000 RBX: ffff88010f8e2000 RCX: 0000000000000000
kernel: RDX: 00000000001c4025 RSI: 0000000000000376 RDI: ffff88010f8e2028
kernel: RBP: ffff88010cf4d9b8 R08: 0000000000000002 R09: ffff88010d79d6e0
kernel: R10: ffffea00038738a8 R11: 0000000000000002 R12: ffffea00038738a8
kernel: R13: ffff88010d79d6e0 R14: ffff88010a8b13c8 R15: ffff88010f8e2678
kernel: FS: 00007f7d834ac700(0000) GS:ffff880112a00000(0000) knlGS:0000000000000000
kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
kernel: CR2: 00007f7d834aa000 CR3: 00000001179ca000 CR4: 00000000000407e0
kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
kernel: Process fstest (pid: 10646, threadinfo ffff88010cf4c000, task ffff8801015289c0)
kernel: Stack:
kernel: ffff8801015289c0 0000000000000000 ffff88010d79d6e0 00001000015289c0
kernel: ffff88010d79d6e0 0000000000000000 000000010b419bb0 ffff88010f8e2028
kernel: ffff88010cf4d9a8 ffffea00038738a8 ffffea00038738a8 0000000000000000
kernel: Call Trace:
kernel: [<ffffffffa0218435>] __ext4_journalled_invalidatepage+0x7a/0x83 [ext4]
kernel: [<ffffffffa0219c51>] ext4_journalled_invalidatepage+0xe/0x25 [ext4]
kernel: [<ffffffff820d2b64>] do_invalidatepage+0x28/0x2a
kernel: [<ffffffff820d30bf>] truncate_inode_page+0x4b/0x6c
kernel: [<ffffffff820d31cb>] truncate_inode_pages_range+0xeb/0x3a5
kernel: [<ffffffff823c9742>] ? __mutex_unlock_slowpath+0x110/0x11c
kernel: [<ffffffff820d34f8>] truncate_inode_pages+0x12/0x14
kernel: [<ffffffff820d353e>] truncate_pagecache+0x44/0x5e
kernel: [<ffffffffa0219ae9>] ext4_setattr+0x55a/0x5ff [ext4]
kernel: [<ffffffff8211f8e8>] notify_change+0x1fc/0x2de
kernel: [<ffffffff821081a4>] ? generic_file_llseek_size+0x50/0xf7
kernel: [<ffffffff821069cc>] do_truncate+0x6c/0x89
kernel: [<ffffffff821142d9>] do_last+0xbcc/0xbea
kernel: [<ffffffff821143ca>] path_openat+0xd3/0x481
kernel: [<ffffffff8206320e>] ? sched_clock_cpu+0xc3/0xd1
kernel: [<ffffffff82114887>] do_filp_open+0x3d/0x89
kernel: [<ffffffff82121fac>] ? __alloc_fd+0x1c5/0x1d7
kernel: [<ffffffff82105aba>] do_sys_open+0x119/0x1ab
kernel: [<ffffffff82105b83>] sys_open+0x21/0x23
kernel: [<ffffffff823d45c2>] system_call_fastpath+0x16/0x1b
kernel: Code: 0f 0b eb fe f0 41 80 65 02 ef 48 8b 7d c8 89 45 98 e8 a0 da 1d
e2 8b 45 98 f0 41 80 65 00 fd 49 8b 55 00 f7 c2 00 00 08 00 74 04 <0f> 0b eb
fe f0 41 80 65 00 df f0 41 80 65 00 f7 f0 41 80 65 00
kernel: RIP [<ffffffffa01ee9de>] jbd2_journal_invalidatepage+0x265/0x2ec [jbd2]
kernel: RSP <ffff88010cf4d948>
kernel: ---[ end trace 6314db6c587d0c29 ]---
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [BUG] xfstests #68 with data=journal hang against 3.8-rc7 and 'dev' branch
2013-02-17 17:22 [BUG] xfstests #68 with data=journal hang against 3.8-rc7 and 'dev' branch Zheng Liu
@ 2013-02-18 5:04 ` Theodore Ts'o
2013-02-18 17:33 ` Zheng Liu
2013-02-19 8:55 ` Jan Kara
2013-02-22 19:46 ` Theodore Ts'o
2 siblings, 1 reply; 7+ messages in thread
From: Theodore Ts'o @ 2013-02-18 5:04 UTC (permalink / raw)
To: linux-ext4
On Mon, Feb 18, 2013 at 01:22:08AM +0800, Zheng Liu wrote:
> Hi all,
>
> Xfstests #68 will hang with data=journal in 3.8-rc7 and 'dev' branch. I
> remember that there has a patch for ext4 to fix filesystem freeze bug
> but I am not sure whether it can fix this bug and it has been applied
> into 'dev' branch. So I file this bug here.
Hmm.... I have never seen this test fail with my tests (although I'm
just using the dev branch; I have not tried merging in 3.8-rc7 into
the dev branch for any of my tests).
Is it failing for you reliably?
The commit which I think you are referring to fixed a bug where a user
was trying to freeze a file systme which was mounted over a FUSE
partition. It's certainly in dev; it's not marked for inclusion for
the stable patch series since it's an optimization change, and it was
not intended to fix something which is inherently dangerous (and if it
works, it's purely by luck, since the FUSE userspace will get frozen
before the file system is necessarily done doing its journal freeze
operation; the change may have affected the timing so that the user
would get lucky, but it's a sheer chance thing, and there's no
guarantee the user will lose later).
- Ted
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [BUG] xfstests #68 with data=journal hang against 3.8-rc7 and 'dev' branch
2013-02-18 5:04 ` Theodore Ts'o
@ 2013-02-18 17:33 ` Zheng Liu
2013-02-19 19:57 ` Eric Whitney
0 siblings, 1 reply; 7+ messages in thread
From: Zheng Liu @ 2013-02-18 17:33 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: linux-ext4
On Mon, Feb 18, 2013 at 12:04:22AM -0500, Theodore Ts'o wrote:
> On Mon, Feb 18, 2013 at 01:22:08AM +0800, Zheng Liu wrote:
> > Hi all,
> >
> > Xfstests #68 will hang with data=journal in 3.8-rc7 and 'dev' branch. I
> > remember that there has a patch for ext4 to fix filesystem freeze bug
> > but I am not sure whether it can fix this bug and it has been applied
> > into 'dev' branch. So I file this bug here.
>
> Hmm.... I have never seen this test fail with my tests (although I'm
> just using the dev branch; I have not tried merging in 3.8-rc7 into
> the dev branch for any of my tests).
>
> Is it failing for you reliably?
I dig this bug again and I always reproduce it in my own machine with
SSD disk in 3.8-rc7 and dev branch. But, interestingly, I run the same
test case in another machine, which is the same as previous one except
that it only has a HDD, and I never trigger this bug. So it seems that
this bug could be triggered in a high-speed device, but we need to
verify this conclusion. Sorry I haven't another server with SSD/Flash
device to run test case. That would be great if others could give it a
try.
Thanks,
- Zheng
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [BUG] xfstests #68 with data=journal hang against 3.8-rc7 and 'dev' branch
2013-02-17 17:22 [BUG] xfstests #68 with data=journal hang against 3.8-rc7 and 'dev' branch Zheng Liu
2013-02-18 5:04 ` Theodore Ts'o
@ 2013-02-19 8:55 ` Jan Kara
2013-02-22 19:46 ` Theodore Ts'o
2 siblings, 0 replies; 7+ messages in thread
From: Jan Kara @ 2013-02-19 8:55 UTC (permalink / raw)
To: Zheng Liu; +Cc: linux-ext4, Theodore Ts'o
Hi,
On Mon 18-02-13 01:22:08, Zheng Liu wrote:
> Xfstests #68 will hang with data=journal in 3.8-rc7 and 'dev' branch. I
> remember that there has a patch for ext4 to fix filesystem freeze bug
> but I am not sure whether it can fix this bug and it has been applied
> into 'dev' branch. So I file this bug here.
Yes, I also recently hit this bug. I found that the root cause is that in
data=journal mode there are dirty pages even after syncing the whole
filesystem. I didn't yet found out why that happens but dirty page handling
is complex in data=journal mode as pages get marked dirty at the moment
transaction is committing and buffers are filed into checkpoint lists so I
find it quite possible there is some case I didn't think about where
syncing results in non-clean filesystem.
Honza
--
Jan Kara <jack@suse.cz>
SUSE Labs, CR
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [BUG] xfstests #68 with data=journal hang against 3.8-rc7 and 'dev' branch
2013-02-18 17:33 ` Zheng Liu
@ 2013-02-19 19:57 ` Eric Whitney
0 siblings, 0 replies; 7+ messages in thread
From: Eric Whitney @ 2013-02-19 19:57 UTC (permalink / raw)
To: Zheng Liu; +Cc: linux-ext4
* Zheng Liu <gnehzuil.liu@gmail.com>:
> On Mon, Feb 18, 2013 at 12:04:22AM -0500, Theodore Ts'o wrote:
> > On Mon, Feb 18, 2013 at 01:22:08AM +0800, Zheng Liu wrote:
> > > Hi all,
> > >
> > > Xfstests #68 will hang with data=journal in 3.8-rc7 and 'dev' branch. I
> > > remember that there has a patch for ext4 to fix filesystem freeze bug
> > > but I am not sure whether it can fix this bug and it has been applied
> > > into 'dev' branch. So I file this bug here.
> >
> > Hmm.... I have never seen this test fail with my tests (although I'm
> > just using the dev branch; I have not tried merging in 3.8-rc7 into
> > the dev branch for any of my tests).
> >
> > Is it failing for you reliably?
>
> I dig this bug again and I always reproduce it in my own machine with
> SSD disk in 3.8-rc7 and dev branch. But, interestingly, I run the same
> test case in another machine, which is the same as previous one except
> that it only has a HDD, and I never trigger this bug. So it seems that
> this bug could be triggered in a high-speed device, but we need to
> verify this conclusion. Sorry I haven't another server with SSD/Flash
> device to run test case. That would be great if others could give it a
> try.
>
> Thanks,
> - Zheng
I saw a similar hang (stall) of 068 on ARM in one of two runs I made
with the xfstests auto group on 3.8-rc7.
The test system was a Pandaboard using a USB-attached SATA disk. I've not yet
seen it on x86-64 using a virtual disk.
I've made three sets of xfstest 068 runs on a fresh 3.8 kernel on the same
system to get a sense of repeatability, terminating each set at the first
occurrence of a hang. The hang occurred at the eighth run in the first set,
second run in the second set, and third run in the third set. So, it's
possible to see the problem reasonably easily on a slow test system.
Example trace follows.
Regards,
Eric
FSTYP -- ext4
PLATFORM -- Linux/armv7l panda 3.8.0
MKFS_OPTIONS -- -q /dev/sda2
MOUNT_OPTIONS -- -o acl,user_xattr -o block_validity,data=journal /dev/sda2 /vdc
068 88s ... [13:58:56]
<test stalls>
kernel: [ 742.672546] kernel BUG at fs/jbd2/transaction.c:1986!
kernel: [ 742.677856] Internal error: Oops - BUG: 0 [#1] SMP ARM
kernel: [ 742.683227] Modules linked in:
kernel: [ 742.686431] CPU: 0 Tainted: G W (3.8.0 #1)
kernel: [ 742.692016] PC is at jbd2_journal_invalidatepage+0x370/0x38c
kernel: [ 742.697967] LR is at jbd2_journal_invalidatepage+0x1c0/0x38c
kernel: [ 742.703887] pc : [<c01ffb68>] lr : [<c01ff9b8>] psr: 00000113
kernel: [ 742.703887] sp : e8399ba8 ip : 00000000 fp : e8399bec
kernel: [ 742.715942] r10: eca09788 r9 : 00080000 r8 : eca09788
kernel: [ 742.721405] r7 : 00000000 r6 : 00000000 r5 : eca09788 r4 : 00001000
kernel: [ 742.728240] r3 : 00000002 r2 : 001c4025 r1 : eca09788 r0 : 00000000
kernel: [ 742.735076] Flags: nzcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
kernel: [ 742.742584] Control: 10c5387d Table: a21b404a DAC: 00000015
kernel: [ 742.748596] Process fstest (pid: 3909, stack limit = 0xe8398240)
kernel: [ 742.754882] Stack: (0xe8399ba8 to 0xe839a000)
kernel: [ 742.759460] 9ba0: e8399bd4 eca63488 c1f39040 e30cc328 e30cc000 00000001
kernel: [ 742.768035] 9bc0: 00000000 c1f39040 00000000 c1f39040 e30cc000 e41b123c 000000df 00000000
kernel: [ 742.776611] 9be0: e8399c0c e8399bf0 c01ad7f0 c01ff804 c1f39040 e41b123c 0000000d c1f39040
kernel: [ 742.785186] 9c00: e8399c1c e8399c10 c01aed44 c01ad7a4 e8399c44 e8399c20 c00e5cd8 c01aed38
kernel: [ 742.793762] 9c20: e8399c44 e8399c30 c00d8a84 c005c3cc ffffffff e8399c58 e8399cc4 e8399c48
kernel: [ 742.802337] 9c40: c00e5dfc c00e5c2c e41b127c e41b1280 00000000 00000000 0000000e 00000000
kernel: [ 742.810913] 9c60: c1f60460 c1f60440 c1f60420 c1f60400 c1f297e0 c1f297c0 c1f297a0 c1f29780
kernel: [ 742.819458] 9c80: c1f29760 c1f29740 c1f390a0 c1f39080 c1f39060 c1f39040 c05a5108 ffffffff
kernel: [ 742.828033] 9ca0: ffffffff 00000000 00000000 00000001 00000000 e41b123c e8399ce4 e8399cc8
kernel: [ 742.836608] 9cc0: c00e6118 c00e5cec ffffffff ffffffff 00000000 00000000 e8399d1c e8399ce8
kernel: [ 742.845184] 9ce0: c00e6190 c00e60fc 00000000 00000000 00000001 00000000 c1f4b440 e30cc014
kernel: [ 742.853759] 9d00: e41b1118 e8399db0 00000000 0000a068 e8399d6c e8399d20 c01b2e54 c00e612c
kernel: [ 742.862335] 9d20: 00000000 00000000 00100000 00000000 e8399db0 00000000 00000001 e30cc000
kernel: [ 742.870910] 9d40: c003f0b8 00000000 00000001 ecadd660 0000a068 e8399db0 000081a4 e41b1118
kernel: [ 742.879486] 9d60: e8399dac e8399d70 c0131c50 c01b2a64 22222222 22222222 5123cb94 22c98098
kernel: [ 742.888031] 9d80: e41b1118 ecadd660 00000001 e41b1230 e41b1118 e8399e98 00000026 e8399f60
kernel: [ 742.896606] 9da0: e8399dfc e8399db0 c0115dbc c0131a74 0000a068 c00e3a64 ee29eb40 e8399e94
kernel: [ 742.905181] 9dc0: 00000000 00000000 5123cb94 22c98098 5123cb94 22c98098 5123cb94 22c98098
kernel: [ 742.913757] 9de0: ecc279c0 c0115814 e8399ee0 00000001 e8399e74 e8399e00 c01254f0 c0115d44
kernel: [ 742.922332] 9e00: 00008060 ecc279c0 c00653b0 e8399ee0 00000051 e440a010 00000001 ec96b118
kernel: [ 742.930908] 9e20: 00000000 ecadd660 00000001 00000000 ecadd078 ecc279c0 00cfb4c9 e41b1118
kernel: [ 742.939483] 9e40: ee1260d0 ecadd078 00000020 ecc279c0 e8399ee0 e8399f60 e8399e98 e8398000
kernel: [ 742.948059] 9e60: 00000000 00000000 e8399ed4 e8399e78 c0125b34 c0124eb4 e8399e94 00000000
kernel: [ 742.956634] 9e80: 60000113 e8398000 e8399ee4 e8399e98 00000000 00000002 ee1260d0 ecadd660
kernel: [ 742.965179] 9ea0: 00000000 c01335ec 00000000 e8399f60 00000001 e440a000 ffffff9c 00000001
kernel: [ 742.973754] 9ec0: e8398000 00000000 e8399f54 e8399ed8 c0126244 c0125a88 00000041 02320231
kernel: [ 742.982330] 9ee0: ee1260d0 ecadd660 90cfb4c9 00000005 e440a02c c02e3690 00000000 ed5cd7b0
kernel: [ 742.990905] 9f00: e41b1118 00000301 00000004 00000000 00000000 00020242 00000300 00020242
kernel: [ 742.999481] 9f20: e440a000 000120a4 ffffff9c 00000001 e8398000 00000000 00020242 e440a000
kernel: [ 743.008056] 9f40: 00000003 ffffff9c e8399f94 e8399f58 c0116da8 c0126214 e8399f84 e8399f68
kernel: [ 743.016632] 9f60: 00020242 c00881a4 00000026 00000300 bec54b14 000120a4 000120a4 00000005
kernel: [ 743.025207] 9f80: c000e628 00000000 e8399fa4 e8399f98 c0116e60 c0116cc0 00000000 e8399fa8
kernel: [ 743.033752] 9fa0: c000e3c0 c0116e40 bec54b14 000120a4 bec54b14 00020242 000001a4 00000000
kernel: [ 743.042327] 9fc0: bec54b14 000120a4 000120a4 00000005 00000003 00000017 b6d0f000 00000001
kernel: [ 743.050903] 9fe0: 00000005 bec54698 b6e95b5d b6e261e6 20000130 bec54b14 00000000 00000000
kernel: [ 743.059509] [<c01ffb68>] (jbd2_journal_invalidatepage+0x370/0x38c) from [<c01ad7f0>] (__ext4_journalled_invalidatepage+0x58/0xa0)
kernel: [ 743.071716] [<c01ad7f0>] (__ext4_journalled_invalidatepage+0x58/0xa0) from [<c01aed44>] (ext4_journalled_invalidatepage+0x18/0x34)
kernel: [ 743.084045] [<c01aed44>] (ext4_journalled_invalidatepage+0x18/0x34) from [<c00e5cd8>] (truncate_inode_page+0xb8/0xc0)
kernel: [ 743.095153] [<c00e5cd8>] (truncate_inode_page+0xb8/0xc0) from [<c00e5dfc>] (truncate_inode_pages_range+0x11c/0x35c)
kernel: [ 743.106109] [<c00e5dfc>] (truncate_inode_pages_range+0x11c/0x35c) from [<c00e6118>] (truncate_inode_pages+0x28/0x30)
kernel: [ 743.117126] [<c00e6118>] (truncate_inode_pages+0x28/0x30) from [<c00e6190>] (truncate_pagecache+0x70/0x90)
kernel: [ 743.127258] [<c00e6190>] (truncate_pagecache+0x70/0x90) from [<c01b2e54>] (ext4_setattr+0x3fc/0x67c)
kernel: [ 743.136840] [<c01b2e54>] (ext4_setattr+0x3fc/0x67c) from [<c0131c50>] (notify_change+0x1e8/0x334)
kernel: [ 743.146148] [<c0131c50>] (notify_change+0x1e8/0x334) from [<c0115dbc>] (do_truncate+0x84/0xa8)
kernel: [ 743.155181] [<c0115dbc>] (do_truncate+0x84/0xa8) from [<c01254f0>] (do_last.isra.28+0x648/0xbd4)
kernel: [ 743.164367] [<c01254f0>] (do_last.isra.28+0x648/0xbd4) from [<c0125b34>] (path_openat+0xb8/0x49c)
kernel: [ 743.173675] [<c0125b34>] (path_openat+0xb8/0x49c) from [<c0126244>] (do_filp_open+0x3c/0x90)
kernel: [ 743.182525] [<c0126244>] (do_filp_open+0x3c/0x90) from [<c0116da8>] (do_sys_open+0xf4/0x180)
kernel: [ 743.191375] [<c0116da8>] (do_sys_open+0xf4/0x180) from [<c0116e60>] (sys_open+0x2c/0x30)
kernel: [ 743.199890] [<c0116e60>] (sys_open+0x2c/0x30) from [<c000e3c0>] (ret_fast_syscall+0x0/0x3c)
kernel: [ 743.208648] Code: e3a07001 eaffff74 e7f001f2 e7f001f2 (e7f001f2)
kernel: [ 743.250122] ---[ end trace 3cd8ea773e9935b2 ]---
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [BUG] xfstests #68 with data=journal hang against 3.8-rc7 and 'dev' branch
2013-02-17 17:22 [BUG] xfstests #68 with data=journal hang against 3.8-rc7 and 'dev' branch Zheng Liu
2013-02-18 5:04 ` Theodore Ts'o
2013-02-19 8:55 ` Jan Kara
@ 2013-02-22 19:46 ` Theodore Ts'o
2013-02-26 18:49 ` Jan Kara
2 siblings, 1 reply; 7+ messages in thread
From: Theodore Ts'o @ 2013-02-22 19:46 UTC (permalink / raw)
To: linux-ext4
On Mon, Feb 18, 2013 at 01:22:08AM +0800, Zheng Liu wrote:
> Hi all,
>
> Xfstests #68 will hang with data=journal in 3.8-rc7 and 'dev' branch. I
> remember that there has a patch for ext4 to fix filesystem freeze bug
> but I am not sure whether it can fix this bug and it has been applied
> into 'dev' branch. So I file this bug here.
I've confirmed that I can reproduce this by using tmpfs imagea files
under KVM. I can replicate the bug as far back as the 3.0 kernel, so
this is definitely not a recent regression.
- Ted
kernel BUG at /tyt/linux/ext4/fs/jbd2/transaction.c:1986!
invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
Modules linked in:
Pid: 3399, comm: fstest Not tainted 3.8.0-rc3-00026-ge7b04ac #54 Bochs Bochs
EIP: 0060:[<c02b0bb7>] EFLAGS: 00010206 CPU: 0
EIP is at jbd2_journal_invalidatepage+0x1bb/0x238
EAX: 001c4025 EBX: f5f7ab38 ECX: 00000000 EDX: 00000246
ESI: f481d3c8 EDI: f5b87800 EBP: f4d3dcac ESP: f4d3dc7c
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
CR0: 80050033 CR2: b7666000 CR3: 34cef000 CR4: 000006f0
DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
DR6: ffff0ff0 DR7: 00000400
Process fstest (pid: 3399, ti=f4d3c000 task=f4d208a0 task.ti=f4d3c000)
Stack:
00001000 f5f7ab38 f5f7ab38 00000001 f5b87b80 f5b87814 00000000 f7bbc788
00000000 f7bbc788 f5f5f794 00000000 f4d3dcc4 c02741b1 f5b87800 c02315a0
f5f5f794 00000030 f4d3dccc c027524d f4d3dcd8 c01e9cab f7bbc788 f4d3dce8
Call Trace:
[<c02741b1>] __ext4_journalled_invalidatepage+0x60/0x66
[<c02315a0>] ? sync_mapping_buffers+0x1e7/0x1e7
[<c027524d>] ext4_journalled_invalidatepage+0xd/0x22
[<c01e9cab>] do_invalidatepage+0x21/0x24
[<c01e9cfd>] truncate_inode_page+0x4f/0x70
[<c01e9dc6>] truncate_inode_pages_range+0xa8/0x206
[<c01e9fd3>] truncate_inode_pages+0x11/0x15
[<c01ea01f>] truncate_pagecache+0x48/0x64
[<c0277f2f>] ext4_setattr+0x3cc/0x464
[<c0277b63>] ? ext4_mark_inode_dirty+0x1b3/0x1b3
[<c02200fd>] notify_change+0x1b1/0x272
[<c08077ea>] ? mutex_lock_nested+0x26/0x2f
[<c020c576>] do_truncate+0x69/0x82
[<c0217ae7>] do_last+0x8af/0x8d6
[<c0215aca>] ? inode_permission+0x45/0x47
[<c0215b66>] ? link_path_walk+0x9a/0x3ab
[<c0217bab>] path_openat+0x9d/0x2bc
[<c0199afe>] ? lock_release_holdtime.part.21+0x5d/0x63
[<c0199693>] ? trace_hardirqs_off+0xb/0xd
[<c021800f>] do_filp_open+0x26/0x62
[<c022116d>] ? __alloc_fd+0xbd/0xc8
[<c020d0b5>] do_sys_open+0x58/0xd1
[<c020d154>] sys_open+0x26/0x2e
[<c0809748>] syscall_call+0x7/0xb
[<c0800000>] ? no_context+0x67/0x1a5
Code: 68 7d 00 00 8b 45 e0 e8 da 88 55 00 89 d8 e8 39 ee ff ff 8b 45 e4 e8 67 83 55 00 89 d8 e8 c6 ec ff ff 8b 03 a9 00 00 08 00 74 02 <0f> 0b f0 80 23 df f0 8
3 bf f0 80 63 01 fd f0 80
EIP: [<c02b0bb7>] jbd2_journal_invalidatepage+0x1bb/0x238 SS:ESP 0068:f4d3dc7c
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [BUG] xfstests #68 with data=journal hang against 3.8-rc7 and 'dev' branch
2013-02-22 19:46 ` Theodore Ts'o
@ 2013-02-26 18:49 ` Jan Kara
0 siblings, 0 replies; 7+ messages in thread
From: Jan Kara @ 2013-02-26 18:49 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: linux-ext4
On Fri 22-02-13 14:46:21, Ted Tso wrote:
> On Mon, Feb 18, 2013 at 01:22:08AM +0800, Zheng Liu wrote:
> > Hi all,
> >
> > Xfstests #68 will hang with data=journal in 3.8-rc7 and 'dev' branch. I
> > remember that there has a patch for ext4 to fix filesystem freeze bug
> > but I am not sure whether it can fix this bug and it has been applied
> > into 'dev' branch. So I file this bug here.
>
> I've confirmed that I can reproduce this by using tmpfs imagea files
> under KVM. I can replicate the bug as far back as the 3.0 kernel, so
> this is definitely not a recent regression.
Yeah, I was thinking about it for a while now and I think I understand
what's going on. As I already mentioned, the problem with data=journal mode
is that when a transaction containing page data is committed, corresponding
buffers (and thus the page) is marked dirty so that flusher thread (or
checkpointing code) can do checkpoint. So after one iteration of
inode syncing, we have a plenty of dirty pages still around. Now syncing
happens in two rounds - the first in WB_SYNC_NONE mode and the second in
WB_SYNC_ALL mode so usually we perform writeback needed for checkpoint in
the second round. But if for some reason in the first round we skipped the
page (it was locked, under writeback or so) we have a problem and the page
remains dirty after sync.
Another variation of the problem is that ext4_sync_fs() just starts a
transaction commit if wait == 0 and pages are marked dirty only at the end
of transaction commit so the second syncing round in WB_SYNC_ALL mode may
miss some pages which will be marked dirty later.
The question is what to do with these races. We could sync the inodes again
in ext4_sync_fs() after waiting for transaction commit to flush
data needing checkpoint but that looks as an overkill...
And BTW, the trace below looks as a different problem. We fail on:
J_ASSERT_BH(bh, !buffer_jbddirty(bh));
which shouldn't really happen (__dispose_buffer() should have cleared
that). I'll try my luck with RAM based images as well...
Honza
> kernel BUG at /tyt/linux/ext4/fs/jbd2/transaction.c:1986!
> invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
> Modules linked in:
> Pid: 3399, comm: fstest Not tainted 3.8.0-rc3-00026-ge7b04ac #54 Bochs Bochs
> EIP: 0060:[<c02b0bb7>] EFLAGS: 00010206 CPU: 0
> EIP is at jbd2_journal_invalidatepage+0x1bb/0x238
> EAX: 001c4025 EBX: f5f7ab38 ECX: 00000000 EDX: 00000246
> ESI: f481d3c8 EDI: f5b87800 EBP: f4d3dcac ESP: f4d3dc7c
> DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
> CR0: 80050033 CR2: b7666000 CR3: 34cef000 CR4: 000006f0
> DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
> DR6: ffff0ff0 DR7: 00000400
> Process fstest (pid: 3399, ti=f4d3c000 task=f4d208a0 task.ti=f4d3c000)
> Stack:
> 00001000 f5f7ab38 f5f7ab38 00000001 f5b87b80 f5b87814 00000000 f7bbc788
> 00000000 f7bbc788 f5f5f794 00000000 f4d3dcc4 c02741b1 f5b87800 c02315a0
> f5f5f794 00000030 f4d3dccc c027524d f4d3dcd8 c01e9cab f7bbc788 f4d3dce8
> Call Trace:
> [<c02741b1>] __ext4_journalled_invalidatepage+0x60/0x66
> [<c02315a0>] ? sync_mapping_buffers+0x1e7/0x1e7
> [<c027524d>] ext4_journalled_invalidatepage+0xd/0x22
> [<c01e9cab>] do_invalidatepage+0x21/0x24
> [<c01e9cfd>] truncate_inode_page+0x4f/0x70
> [<c01e9dc6>] truncate_inode_pages_range+0xa8/0x206
> [<c01e9fd3>] truncate_inode_pages+0x11/0x15
> [<c01ea01f>] truncate_pagecache+0x48/0x64
> [<c0277f2f>] ext4_setattr+0x3cc/0x464
> [<c0277b63>] ? ext4_mark_inode_dirty+0x1b3/0x1b3
> [<c02200fd>] notify_change+0x1b1/0x272
> [<c08077ea>] ? mutex_lock_nested+0x26/0x2f
> [<c020c576>] do_truncate+0x69/0x82
> [<c0217ae7>] do_last+0x8af/0x8d6
> [<c0215aca>] ? inode_permission+0x45/0x47
> [<c0215b66>] ? link_path_walk+0x9a/0x3ab
> [<c0217bab>] path_openat+0x9d/0x2bc
> [<c0199afe>] ? lock_release_holdtime.part.21+0x5d/0x63
> [<c0199693>] ? trace_hardirqs_off+0xb/0xd
> [<c021800f>] do_filp_open+0x26/0x62
> [<c022116d>] ? __alloc_fd+0xbd/0xc8
> [<c020d0b5>] do_sys_open+0x58/0xd1
> [<c020d154>] sys_open+0x26/0x2e
> [<c0809748>] syscall_call+0x7/0xb
> [<c0800000>] ? no_context+0x67/0x1a5
> Code: 68 7d 00 00 8b 45 e0 e8 da 88 55 00 89 d8 e8 39 ee ff ff 8b 45 e4 e8 67 83 55 00 89 d8 e8 c6 ec ff ff 8b 03 a9 00 00 08 00 74 02 <0f> 0b f0 80 23 df f0 8
> 3 bf f0 80 63 01 fd f0 80
> EIP: [<c02b0bb7>] jbd2_journal_invalidatepage+0x1bb/0x238 SS:ESP 0068:f4d3dc7c
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Jan Kara <jack@suse.cz>
SUSE Labs, CR
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2013-02-26 18:49 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-02-17 17:22 [BUG] xfstests #68 with data=journal hang against 3.8-rc7 and 'dev' branch Zheng Liu
2013-02-18 5:04 ` Theodore Ts'o
2013-02-18 17:33 ` Zheng Liu
2013-02-19 19:57 ` Eric Whitney
2013-02-19 8:55 ` Jan Kara
2013-02-22 19:46 ` Theodore Ts'o
2013-02-26 18:49 ` Jan Kara
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.