From: Dave Jones <davej@codemonkey.org.uk>
To: Chris Mason <clm@fb.com>, Al Viro <viro@ZenIV.linux.org.uk>,
Josef Bacik <jbacik@fb.com>, David Sterba <dsterba@suse.com>,
linux-btrfs@vger.kernel.org,
Linux Kernel <linux-kernel@vger.kernel.org>
Subject: Re: btrfs bio linked list corruption.
Date: Wed, 12 Oct 2016 10:40:12 -0400 [thread overview]
Message-ID: <20161012144012.7vvfehceoykswmun@codemonkey.org.uk> (raw)
In-Reply-To: <20161012134717.n74tww5eywc7dqp7@codemonkey.org.uk>
On Wed, Oct 12, 2016 at 09:47:17AM -0400, Dave Jones wrote:
> On Tue, Oct 11, 2016 at 11:54:09AM -0400, Chris Mason wrote:
> >
> >
> > On 10/11/2016 10:45 AM, Dave Jones wrote:
> > > This is from Linus' current tree, with Al's iovec fixups on top.
> > >
> > > ------------[ cut here ]------------
> > > WARNING: CPU: 1 PID: 3673 at lib/list_debug.c:33 __list_add+0x89/0xb0
> > > list_add corruption. prev->next should be next (ffffe8ffff806648), but was ffffc9000067fcd8. (prev=ffff880503878b80).
> > > CPU: 1 PID: 3673 Comm: trinity-c0 Not tainted 4.8.0-think+ #13
> > > ffffc90000d87458 ffffffff8d32007c ffffc90000d874a8 0000000000000000
> > > ffffc90000d87498 ffffffff8d07a6c1 0000002100000246 ffff88050388e880
>
> I hit this again overnight, it's the same trace, the only difference
> being slightly different addresses in the list pointers:
>
> [42572.777196] list_add corruption. prev->next should be next (ffffe8ffff806648), but was ffffc90000647cd8. (prev=ffff880503a0ba00).
>
> I'm actually a little surprised that ->next was the same across two
> reboots on two different kernel builds. That might be a sign this is
> more repeatable than I'd thought, even if it does take hours of runtime
> right now to trigger it. I'll try and narrow the scope of what trinity
> is doing to see if I can make it happen faster.
.. and of course the first thing that happens is a completely different
btrfs trace..
WARNING: CPU: 1 PID: 21706 at fs/btrfs/transaction.c:489 start_transaction+0x40a/0x440 [btrfs]
CPU: 1 PID: 21706 Comm: trinity-c16 Not tainted 4.8.0-think+ #14
ffffc900019076a8 ffffffffb731ff3c 0000000000000000 0000000000000000
ffffc900019076e8 ffffffffb707a6c1 000001e9f5806ce0 ffff8804f74c4d98
0000000000000801 ffff880501cfa2a8 000000000000008a 000000000000008a
Call Trace:
[<ffffffffb731ff3c>] dump_stack+0x4f/0x73
[<ffffffffb707a6c1>] __warn+0xc1/0xe0
[<ffffffffb707a7e8>] warn_slowpath_null+0x18/0x20
[<ffffffffc01d312a>] start_transaction+0x40a/0x440 [btrfs]
[<ffffffffc01a6215>] ? btrfs_alloc_path+0x15/0x20 [btrfs]
[<ffffffffc01d31b2>] btrfs_join_transaction+0x12/0x20 [btrfs]
[<ffffffffc01d92cf>] cow_file_range_inline+0xef/0x830 [btrfs]
[<ffffffffc01d9d75>] cow_file_range.isra.64+0x365/0x480 [btrfs]
[<ffffffffb77c24cc>] ? _raw_spin_unlock+0x2c/0x50
[<ffffffffc01f229f>] ? release_extent_buffer+0x9f/0x110 [btrfs]
[<ffffffffc01da299>] run_delalloc_nocow+0x409/0xbd0 [btrfs]
[<ffffffffb70c6109>] ? get_lock_stats+0x19/0x50
[<ffffffffc01dadea>] run_delalloc_range+0x38a/0x3e0 [btrfs]
[<ffffffffc01f4aba>] writepage_delalloc.isra.47+0x10a/0x190 [btrfs]
[<ffffffffc01f7678>] __extent_writepage+0xd8/0x2c0 [btrfs]
[<ffffffffc01f7b2e>] extent_write_cache_pages.isra.44.constprop.63+0x2ce/0x430 [btrfs]
[<ffffffffb733e497>] ? debug_smp_processor_id+0x17/0x20
[<ffffffffb70c6109>] ? get_lock_stats+0x19/0x50
[<ffffffffc01f8278>] extent_writepages+0x58/0x80 [btrfs]
[<ffffffffc01d7a80>] ? btrfs_releasepage+0x40/0x40 [btrfs]
[<ffffffffc01d4a63>] btrfs_writepages+0x23/0x30 [btrfs]
[<ffffffffb7162e9c>] do_writepages+0x1c/0x30
[<ffffffffb71550f1>] __filemap_fdatawrite_range+0xc1/0x100
[<ffffffffb71551ee>] filemap_fdatawrite_range+0xe/0x10
[<ffffffffc01eb2fb>] btrfs_fdatawrite_range+0x1b/0x50 [btrfs]
[<ffffffffc01f0820>] btrfs_wait_ordered_range+0x40/0x100 [btrfs]
[<ffffffffc01eb5d5>] btrfs_sync_file+0x285/0x390 [btrfs]
[<ffffffffb7207626>] vfs_fsync_range+0x46/0xa0
[<ffffffffb72076d8>] do_fsync+0x38/0x60
[<ffffffffb720795b>] SyS_fsync+0xb/0x10
[<ffffffffb700259c>] do_syscall_64+0x5c/0x170
[<ffffffffb77c2e4b>] entry_SYSCALL64_slow_path+0x25/0x25
next prev parent reply other threads:[~2016-10-12 14:40 UTC|newest]
Thread overview: 118+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-10-11 14:45 btrfs bio linked list corruption Dave Jones
2016-10-11 15:11 ` Al Viro
2016-10-11 15:19 ` Dave Jones
2016-10-11 15:20 ` Chris Mason
2016-10-11 15:49 ` Dave Jones
2016-10-11 15:54 ` Chris Mason
2016-10-11 16:25 ` Dave Jones
2016-10-12 13:47 ` Dave Jones
2016-10-12 14:40 ` Dave Jones [this message]
2016-10-12 14:42 ` Chris Mason
2016-10-13 18:16 ` Dave Jones
2016-10-13 21:18 ` Chris Mason
2016-10-13 21:56 ` Dave Jones
2016-10-16 0:42 ` Dave Jones
2016-10-18 1:07 ` Chris Mason
2016-10-18 22:42 ` Dave Jones
2016-10-18 23:12 ` Jens Axboe
2016-10-18 23:31 ` Chris Mason
2016-10-18 23:36 ` Jens Axboe
2016-10-18 23:39 ` Linus Torvalds
2016-10-18 23:42 ` Chris Mason
2016-10-19 0:10 ` Linus Torvalds
2016-10-19 0:19 ` Chris Mason
2016-10-19 0:28 ` Linus Torvalds
2016-10-20 22:48 ` Dave Jones
2016-10-19 1:05 ` Andy Lutomirski
2016-10-20 22:50 ` Dave Jones
2016-10-20 23:01 ` Andy Lutomirski
2016-10-20 23:03 ` Dave Jones
2016-10-20 23:23 ` Andy Lutomirski
2016-10-21 20:02 ` Dave Jones
2016-10-21 20:17 ` Chris Mason
2016-10-21 20:23 ` Dave Jones
2016-10-21 20:38 ` Chris Mason
2016-10-21 20:41 ` Josef Bacik
2016-10-21 21:11 ` Dave Jones
2016-10-22 15:20 ` Dave Jones
2016-10-23 21:32 ` Chris Mason
2016-10-24 4:40 ` Dave Jones
2016-10-24 13:42 ` Chris Mason
2016-10-26 0:27 ` Dave Jones
2016-10-26 1:33 ` Linus Torvalds
2016-10-26 1:39 ` Linus Torvalds
2016-10-26 16:30 ` Dave Jones
2016-10-26 16:48 ` Linus Torvalds
2016-10-26 18:18 ` Dave Jones
2016-10-26 18:42 ` Dave Jones
2016-10-26 19:06 ` Linus Torvalds
2016-10-26 20:00 ` Chris Mason
2016-10-26 21:52 ` Chris Mason
2016-10-26 22:21 ` Linus Torvalds
2016-10-26 22:40 ` Dave Jones
2016-10-26 22:51 ` Linus Torvalds
2016-10-26 22:55 ` Jens Axboe
2016-10-26 22:58 ` Linus Torvalds
2016-10-26 23:03 ` Jens Axboe
2016-10-26 23:07 ` Dave Jones
2016-10-26 23:08 ` Linus Torvalds
2016-10-26 23:20 ` Jens Axboe
2016-10-26 23:38 ` Chris Mason
2016-10-26 23:47 ` Dave Jones
2016-10-27 0:00 ` Jens Axboe
2016-10-27 13:33 ` Chris Mason
2016-10-31 18:55 ` Dave Jones
2016-10-31 19:35 ` Linus Torvalds
2016-10-31 19:44 ` Chris Mason
2016-11-06 16:55 ` btrfs btree_ctree_super fault Dave Jones
2016-11-08 14:59 ` Dave Jones
2016-11-08 15:08 ` Chris Mason
2016-11-10 14:35 ` Dave Jones
2016-11-10 15:27 ` Chris Mason
2016-11-23 19:34 ` bio linked list corruption Dave Jones
2016-11-23 19:58 ` Dave Jones
2016-12-01 15:32 ` btrfs_destroy_inode warn (outstanding extents) Dave Jones
2016-12-03 16:48 ` Dave Jones
2016-12-07 16:15 ` Dave Jones
2016-12-09 21:12 ` Steven Rostedt
2016-12-04 23:04 ` bio linked list corruption Vegard Nossum
2016-12-05 11:10 ` Vegard Nossum
2016-12-05 17:09 ` Vegard Nossum
2016-12-05 17:21 ` Dave Jones
2016-12-05 17:55 ` Linus Torvalds
2016-12-05 19:11 ` Vegard Nossum
2016-12-05 20:10 ` Linus Torvalds
2016-12-05 20:35 ` Linus Torvalds
2016-12-05 21:33 ` Vegard Nossum
2016-12-06 8:42 ` Vegard Nossum
2016-12-06 8:16 ` Peter Zijlstra
2016-12-06 8:36 ` Ingo Molnar
2016-12-06 16:33 ` Linus Torvalds
2016-12-05 20:10 ` Vegard Nossum
2016-12-05 18:11 ` Andy Lutomirski
2016-12-05 18:25 ` Linus Torvalds
2016-12-05 18:26 ` Vegard Nossum
2016-10-26 23:19 ` Chris Mason
2016-10-26 23:21 ` Jens Axboe
2016-10-27 6:33 ` Christoph Hellwig
2016-10-27 16:34 ` Linus Torvalds
2016-10-27 16:36 ` Jens Axboe
2016-10-26 23:01 ` Dave Jones
2016-10-26 23:05 ` Jens Axboe
2016-10-26 22:52 ` Jens Axboe
2016-10-26 22:07 ` Linus Torvalds
2016-10-26 22:54 ` Chris Mason
2016-10-27 5:41 ` Dave Chinner
2016-10-27 17:23 ` Dave Jones
2016-10-24 20:06 ` Andy Lutomirski
2016-10-24 20:46 ` Linus Torvalds
2016-10-24 21:17 ` Linus Torvalds
2016-10-24 21:50 ` Linus Torvalds
2016-10-24 22:02 ` Chris Mason
2016-10-24 22:42 ` Andy Lutomirski
2016-10-25 0:00 ` Linus Torvalds
2016-10-25 1:09 ` Andy Lutomirski
2016-10-19 17:09 ` Philipp Hahn
2016-10-19 17:43 ` Linus Torvalds
2016-10-20 6:52 ` Ingo Molnar
2016-10-20 7:17 ` Thomas Gleixner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20161012144012.7vvfehceoykswmun@codemonkey.org.uk \
--to=davej@codemonkey.org.uk \
--cc=clm@fb.com \
--cc=dsterba@suse.com \
--cc=jbacik@fb.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=viro@ZenIV.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).