XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568

XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568

[Brian Foster]

Hi all,

I hit an assert on a debug kernel while beating on some finobt work and
eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I
hit it through a couple different paths, first while running fsstress on
a CRC enabled filesystem (with otherwise default mkfs options):

(These tests are running on a 4p, 4GB VM against a 100GB virtio disk,
hosted on a single spindle desktop box).

crc=1
fsstress -z -fsymlink=1 -n99999999 -p4 -d /mnt/test

XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length),
file: fs/xfs/xfs_trans_buf.c, line: 568
------------[ cut here ]------------
kernel BUG at fs/xfs/xfs_message.c:108!
invalid opcode: 0000 [#1] SMP
Modules linked in: xfs libcrc32c fuse ebtable_nat
nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE
ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6
nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle bnep
nf_conntrack_ipv4 nf_defrag_ipv4 bluetooth xt_conntrack nf_conntrack
rfkill ebtable_filter ebtables ip6table_filter ip6_tables snd_hda_intel
snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm snd_page_alloc
snd_timer snd joydev soundcore i2c_piix4 pcspkr mperf virtio_balloon
floppy uinput qxl drm_kms_helper ttm drm virtio_blk virtio_net i2c_core
CPU: 0 PID: 1419 Comm: fsstress Not tainted 3.11.0-rc1+ #10
Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
task: ffff8800d65b5dc0 ti: ffff8800d10ba000 task.ti: ffff8800d10ba000
RIP: 0010:[<ffffffffa02b8812>]  [<ffffffffa02b8812>] assfail+0x22/0x30 [xfs]
RSP: 0018:ffff8800d10bb998  EFLAGS: 00010292
RAX: 000000000000006b RBX: ffff8800d67be3a0 RCX: 0000000000000000
RDX: ffff88011fc0ee48 RSI: ffff88011fc0d038 RDI: ffff88011fc0d038
RBP: ffff8800d10bb998 R08: 0000000000000000 R09: 000000000000020a
R10: ffffffff81858260 R11: 0000000000000209 R12: ffff8800d165d500
R13: ffff8800d1158980 R14: 0000000000001007 R15: ffff8800d1cb8300
FS:  00007f1efd2ce740(0000) GS:ffff88011fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1ef80fb018 CR3: 0000000036edb000 CR4: 00000000000006f0
Stack:
 ffff8800d10bb9e8 ffffffffa031d549 000000fc24a6f000 00000e20000000d3
 ffff8800d10bb9f8 ffff8800d67c3040 ffff8800d1cb8208 ffff8800d1cb81e8
 ffff8800d67c3000 ffff8800d1cb8300 ffff8800d10bba48 ffffffffa02e7c1c
Call Trace:
 [<ffffffffa031d549>] xfs_trans_log_buf+0x89/0x1b0 [xfs]
 [<ffffffffa02e7c1c>] xfs_da3_node_add+0x11c/0x210 [xfs]
 [<ffffffffa02ea703>] xfs_da3_node_split+0xc3/0x230 [xfs]
 [<ffffffffa02eaa18>] xfs_da3_split+0x1a8/0x410 [xfs]
 [<ffffffffa02f743f>] xfs_dir2_node_addname+0x47f/0xde0 [xfs]
 [<ffffffffa02ec965>] xfs_dir_createname+0x1d5/0x1e0 [xfs]
 [<ffffffffa02c1607>] ? kmem_alloc+0x67/0xf0 [xfs]
 [<ffffffffa02becb9>] xfs_symlink+0x619/0xa20 [xfs]
 [<ffffffff811abad6>] ? _d_rehash+0x36/0x40
 [<ffffffff8119f498>] ? __lookup_hash+0x38/0x50
 [<ffffffff8119f4c9>] ? lookup_hash+0x19/0x20
 [<ffffffff811a21ee>] ? kern_path_create+0x8e/0x170
 [<ffffffffa02b5e5c>] xfs_vn_symlink+0x5c/0xe0 [xfs]
 [<ffffffff811a3939>] vfs_symlink+0x99/0x100
 [<ffffffff811a59d6>] SyS_symlinkat+0x66/0xd0
 [<ffffffff811a5a56>] SyS_symlink+0x16/0x20
 [<ffffffff81645442>] system_call_fastpath+0x16/0x1b
Code: 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 f1 41 89 d0 48
c7 c6 70 50 33 a0 48 89 fa 31 c0 48 89 e5 31 ff e8 de fb ff ff <0f> 0b
66 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48
RIP  [<ffffffffa02b8812>] assfail+0x22/0x30 [xfs]
 RSP <ffff8800d10bb998>
---[ end trace 9578edaae955ff56 ]---

I repeated the test on a crc=0 fs (with -isize=512) and could not
reproduce during fsstress. I let it populate to about 10GB and
ultimately hit the same assert on unlink during a post-test cleanup:

crc=0
rm -rf /mnt/test

XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length),
file: fs/xfs/xfs_trans_buf.c, line: 568
------------[ cut here ]------------
kernel BUG at fs/xfs/xfs_message.c:108!
invalid opcode: 0000 [#1] SMP
Modules linked in: xfs libcrc32c fuse ebtable_nat
nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE
ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6
nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle
nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack
ebtable_filter ebtables bnep bluetooth rfkill ip6table_filter ip6_tables
snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm
snd_page_alloc snd_timer snd soundcore joydev pcspkr virtio_balloon
i2c_piix4 floppy mperf uinput qxl drm_kms_helper ttm drm virtio_net
virtio_blk i2c_core
CPU: 1 PID: 2198 Comm: rm Not tainted 3.11.0-rc1+ #10
Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
task: ffff8801161ec650 ti: ffff8800c803e000 task.ti: ffff8800c803e000
RIP: 0010:[<ffffffffa02c6812>]  [<ffffffffa02c6812>] assfail+0x22/0x30 [xfs]
RSP: 0018:ffff8800c803fa98  EFLAGS: 00010292
RAX: 000000000000006b RBX: ffff8801029a6e80 RCX: 0000000000000000
RDX: ffff88011fc8ee48 RSI: ffff88011fc8d038 RDI: ffff88011fc8d038
RBP: ffff8800c803fa98 R08: 0000000000000000 R09: 0000000000000209
R10: ffffffff81858260 R11: 0000000000000208 R12: ffff8800302bd200
R13: ffff8800d25e0850 R14: 000000000000122f R15: ffff8800d271f010
FS:  00007f28ef9bf740(0000) GS:ffff88011fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 000000000153a000 CR3: 00000000b1fd3000 CR4: 00000000000006e0
Stack:
 ffff8800c803fae8 ffffffffa032b549 00800201008006cc 000000100185febe
 ffffffffa033fcb0 ffff8800ade0c010 ffff8800ade0c000 ffff8800d3c2b9e0
 ffff8800d25e0850 ffff8800d271f010 ffff8800c803fb58 ffffffffa02f61ff
Call Trace:
 [<ffffffffa032b549>] xfs_trans_log_buf+0x89/0x1b0 [xfs]
 [<ffffffffa02f61ff>] xfs_da3_node_unbalance+0xef/0x1d0 [xfs]
 [<ffffffffa02f98b0>] xfs_da3_join+0x240/0x290 [xfs]
 [<ffffffffa030659b>] xfs_dir2_node_removename+0x69b/0x8b0 [xfs]
 [<ffffffffa02e16ce>] ? xfs_bmap_last_extent+0x6e/0xb0 [xfs]
 [<ffffffffa02fa5b5>] xfs_dir_removename+0x195/0x1a0 [xfs]
 [<ffffffffa0310b69>] xfs_remove+0x2a9/0x410 [xfs]
 [<ffffffffa02c3ca2>] xfs_vn_unlink+0x52/0xa0 [xfs]
 [<ffffffff811a260e>] vfs_unlink+0x9e/0x110
 [<ffffffff811a2821>] do_unlinkat+0x1a1/0x230
 [<ffffffff811a592b>] SyS_unlinkat+0x1b/0x40
 [<ffffffff81645442>] system_call_fastpath+0x16/0x1b
Code: 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 f1 41 89 d0 48
c7 c6 70 30 34 a0 48 89 fa 31 c0 48 89 e5 31 ff e8 de fb ff ff <0f> 0b
66 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48
RIP  [<ffffffffa02c6812>] assfail+0x22/0x30 [xfs]
 RSP <ffff8800c803fa98>
---[ end trace 3ef54f36db3ba0c5 ]---

Info on the crc=0 fs is as follows:

meta-data=/dev/vdb               isize=512    agcount=4, agsize=6553600 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=0
data     =                       bsize=4096   blocks=26214400, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal               bsize=4096   blocks=12800, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0


Brian

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568

[Mark Tinguely]

On 08/22/13 13:28, Brian Foster wrote:
> Hi all,
>
> I hit an assert on a debug kernel while beating on some finobt work and
> eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I
> hit it through a couple different paths, first while running fsstress on
> a CRC enabled filesystem (with otherwise default mkfs options):
>
> (These tests are running on a 4p, 4GB VM against a 100GB virtio disk,
> hosted on a single spindle desktop box).

Eeek.

So both crashes are directory related. What is the top XFS kernel commit 
for these tests?

Have you seen this on earlier versions of the kernel?

Thanks,

--Mark.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568

[Brian Foster]

On 08/23/2013 09:18 AM, Mark Tinguely wrote:
> On 08/22/13 13:28, Brian Foster wrote:
>> Hi all,
>>
>> I hit an assert on a debug kernel while beating on some finobt work and
>> eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I
>> hit it through a couple different paths, first while running fsstress on
>> a CRC enabled filesystem (with otherwise default mkfs options):
>>
>> (These tests are running on a 4p, 4GB VM against a 100GB virtio disk,
>> hosted on a single spindle desktop box).
> 
> Eeek.
> 
> So both crashes are directory related. What is the top XFS kernel commit
> for these tests?
> 

Hi Mark,

It was the latest from at some point yesterday. I don't think I've
pulled since, so I'm at:

3e3c51ce xfs: add xfs sb v4 support for dirent filetype field

> Have you seen this on earlier versions of the kernel?
> 

Well I hit an issue first on my dev. branch for finobt hacking, which is
currently based on a slightly older commit:

2c2bcc07 xfs: call roundup_64() to calculate the min_logblks

Unfortunately I don't have the precise error/stack from that failure on
hand to double check whether it's the exact same failure. I can try to
regenerate it a bit later today.

Brian

> Thanks,
> 
> --Mark.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568

[Mark Tinguely]

On 08/23/13 08:30, Brian Foster wrote:
> On 08/23/2013 09:18 AM, Mark Tinguely wrote:
>> On 08/22/13 13:28, Brian Foster wrote:
>>> Hi all,
>>>
>>> I hit an assert on a debug kernel while beating on some finobt work and
>>> eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I
>>> hit it through a couple different paths, first while running fsstress on
>>> a CRC enabled filesystem (with otherwise default mkfs options):
>>>
>>> (These tests are running on a 4p, 4GB VM against a 100GB virtio disk,
>>> hosted on a single spindle desktop box).
>>
>> Eeek.
>>
>> So both crashes are directory related. What is the top XFS kernel commit
>> for these tests?
>>
>
> Hi Mark,
>
> It was the latest from at some point yesterday. I don't think I've
> pulled since, so I'm at:
>
> 3e3c51ce xfs: add xfs sb v4 support for dirent filetype field
>
>> Have you seen this on earlier versions of the kernel?
>>
>
> Well I hit an issue first on my dev. branch for finobt hacking, which is
> currently based on a slightly older commit:
>
> 2c2bcc07 xfs: call roundup_64() to calculate the min_logblks
>
> Unfortunately I don't have the precise error/stack from that failure on
> hand to double check whether it's the exact same failure. I can try to
> regenerate it a bit later today.
>
> Brian
>
>> Thanks,
>>
>> --Mark.
>
Good, I just want to make sure the common directory mods for the file 
type patch was not the cause. I will try to recreate it too.

Thanks.

--Mark.


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568

[Brian Foster]

On 08/23/2013 09:57 AM, Mark Tinguely wrote:
> On 08/23/13 08:30, Brian Foster wrote:
>> On 08/23/2013 09:18 AM, Mark Tinguely wrote:
>>> On 08/22/13 13:28, Brian Foster wrote:
>>>> Hi all,
>>>>
>>>> I hit an assert on a debug kernel while beating on some finobt work and
>>>> eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I
>>>> hit it through a couple different paths, first while running
>>>> fsstress on
>>>> a CRC enabled filesystem (with otherwise default mkfs options):
>>>>
>>>> (These tests are running on a 4p, 4GB VM against a 100GB virtio disk,
>>>> hosted on a single spindle desktop box).
>>>
>>> Eeek.
>>>
>>> So both crashes are directory related. What is the top XFS kernel commit
>>> for these tests?
>>>
>>
>> Hi Mark,
>>
>> It was the latest from at some point yesterday. I don't think I've
>> pulled since, so I'm at:
>>
>> 3e3c51ce xfs: add xfs sb v4 support for dirent filetype field
>>
>>> Have you seen this on earlier versions of the kernel?
>>>
>>
>> Well I hit an issue first on my dev. branch for finobt hacking, which is
>> currently based on a slightly older commit:
>>
>> 2c2bcc07 xfs: call roundup_64() to calculate the min_logblks
>>
>> Unfortunately I don't have the precise error/stack from that failure on
>> hand to double check whether it's the exact same failure. I can try to
>> regenerate it a bit later today.
>>
>> Brian
>>
>>> Thanks,
>>>
>>> --Mark.
>>
> Good, I just want to make sure the common directory mods for the file
> type patch was not the cause. I will try to recreate it too.
> 

And just to confirm, I can hit the assert from both tests on a 2c2bcc07
kernel (pre-dirent ftype change).

Brian

> Thanks.
> 
> --Mark.
> 
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568

[Mark Tinguely]

On 08/23/13 11:49, Brian Foster wrote:
> On 08/23/2013 09:57 AM, Mark Tinguely wrote:
>> On 08/23/13 08:30, Brian Foster wrote:
>>> On 08/23/2013 09:18 AM, Mark Tinguely wrote:
>>>> On 08/22/13 13:28, Brian Foster wrote:
>>>>> Hi all,
>>>>>
>>>>> I hit an assert on a debug kernel while beating on some finobt work and
>>>>> eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I
>>>>> hit it through a couple different paths, first while running
>>>>> fsstress on
>>>>> a CRC enabled filesystem (with otherwise default mkfs options):
>>>>>
>>>>> (These tests are running on a 4p, 4GB VM against a 100GB virtio disk,
>>>>> hosted on a single spindle desktop box).
>>>>
>>>> Eeek.
>>>>
>>>> So both crashes are directory related. What is the top XFS kernel commit
>>>> for these tests?
>>>>
>>>
>>> Hi Mark,
>>>
>>> It was the latest from at some point yesterday. I don't think I've
>>> pulled since, so I'm at:
>>>
>>> 3e3c51ce xfs: add xfs sb v4 support for dirent filetype field
>>>
>>>> Have you seen this on earlier versions of the kernel?
>>>>
>>>
>>> Well I hit an issue first on my dev. branch for finobt hacking, which is
>>> currently based on a slightly older commit:
>>>
>>> 2c2bcc07 xfs: call roundup_64() to calculate the min_logblks
>>>
>>> Unfortunately I don't have the precise error/stack from that failure on
>>> hand to double check whether it's the exact same failure. I can try to
>>> regenerate it a bit later today.
>>>
>>> Brian
>>>
>>>> Thanks,
>>>>
>>>> --Mark.
>>>
>> Good, I just want to make sure the common directory mods for the file
>> type patch was not the cause. I will try to recreate it too.
>>
>
> And just to confirm, I can hit the assert from both tests on a 2c2bcc07
> kernel (pre-dirent ftype change).
>
> Brian

Suspected as much, but thank-you for the confirmation.

I guess the next bi-section would be Linux 3.11 to make sure the 
user/kernel sync did not introduce anything. I thought those moves were 
clean.

I will shut up and try to help bisect.

--Mark.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH] Re: XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568

[Dave Chinner]

On Thu, Aug 22, 2013 at 02:28:00PM -0400, Brian Foster wrote:
> Hi all,
> 
> I hit an assert on a debug kernel while beating on some finobt work and
> eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I
> hit it through a couple different paths, first while running fsstress on
> a CRC enabled filesystem (with otherwise default mkfs options):
> 
> (These tests are running on a 4p, 4GB VM against a 100GB virtio disk,
> hosted on a single spindle desktop box).
> 
> crc=1
> fsstress -z -fsymlink=1 -n99999999 -p4 -d /mnt/test
> 
> XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length),

Directory buffer overrun.

>  [<ffffffffa031d549>] xfs_trans_log_buf+0x89/0x1b0 [xfs]
>  [<ffffffffa02e7c1c>] xfs_da3_node_add+0x11c/0x210 [xfs]
>  [<ffffffffa02ea703>] xfs_da3_node_split+0xc3/0x230 [xfs]
>  [<ffffffffa02eaa18>] xfs_da3_split+0x1a8/0x410 [xfs]
>  [<ffffffffa02f743f>] xfs_dir2_node_addname+0x47f/0xde0 [xfs]

During a split.

Easily reproduced with "seq 200000 | xargs touch" as Michael Semon
reported last week.

The fix demonstrates my concerns about modifying directory code -
the CRC changes missed a *fundamental* directory format definition,
and we've only just tripped over it....

> rm -rf /mnt/test
> 
> XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length),

Directory buffer overrun.

>  [<ffffffffa032b549>] xfs_trans_log_buf+0x89/0x1b0 [xfs]
>  [<ffffffffa02f61ff>] xfs_da3_node_unbalance+0xef/0x1d0 [xfs]
>  [<ffffffffa02f98b0>] xfs_da3_join+0x240/0x290 [xfs]
>  [<ffffffffa030659b>] xfs_dir2_node_removename+0x69b/0x8b0 [xfs]

During a merge. Not sure why that is happening on a v4 filesystem.
V5 filesystem, yes, due to the above bug but v4 should not be
affected.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

xfs: fix calculation of the number of node entries in a dir3 node

From: Dave Chinner <dchinner@redhat.com>

The calculation doesn't take into account the size of the dir v3
header, so overestimates the hash entries in a node. This causes
directory buffer overruns when splitting and merging nodes.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 fs/xfs/xfs_da_btree.h | 11 +++++++++--
 fs/xfs/xfs_dir2.c     | 16 ++++++++++------
 2 files changed, 19 insertions(+), 8 deletions(-)

diff --git a/fs/xfs/xfs_da_btree.h b/fs/xfs/xfs_da_btree.h
index 8cdc77b..b1f2679 100644
--- a/fs/xfs/xfs_da_btree.h
+++ b/fs/xfs/xfs_da_btree.h
@@ -133,12 +133,19 @@ extern void xfs_da3_node_hdr_to_disk(struct xfs_da_intnode *to,
 				     struct xfs_da3_icnode_hdr *from);
 
 static inline int
-xfs_da3_node_hdr_size(struct xfs_da_intnode *dap)
+__xfs_da3_node_hdr_size(bool v3)
 {
-	if (dap->hdr.info.magic == cpu_to_be16(XFS_DA3_NODE_MAGIC))
+	if (v3)
 		return sizeof(struct xfs_da3_node_hdr);
 	return sizeof(struct xfs_da_node_hdr);
 }
+static inline int
+xfs_da3_node_hdr_size(struct xfs_da_intnode *dap)
+{
+	bool	v3 = dap->hdr.info.magic == cpu_to_be16(XFS_DA3_NODE_MAGIC);
+
+	return __xfs_da3_node_hdr_size(v3);
+}
 
 static inline struct xfs_da_node_entry *
 xfs_da3_node_tree_p(struct xfs_da_intnode *dap)
diff --git a/fs/xfs/xfs_dir2.c b/fs/xfs/xfs_dir2.c
index d3ff96c..edf203a 100644
--- a/fs/xfs/xfs_dir2.c
+++ b/fs/xfs/xfs_dir2.c
@@ -90,6 +90,9 @@ void
 xfs_dir_mount(
 	xfs_mount_t	*mp)
 {
+	int	nodehdr_size;
+
+
 	ASSERT(xfs_sb_version_hasdirv2(&mp->m_sb));
 	ASSERT((1 << (mp->m_sb.sb_blocklog + mp->m_sb.sb_dirblklog)) <=
 	       XFS_MAX_BLOCKSIZE);
@@ -98,12 +101,13 @@ xfs_dir_mount(
 	mp->m_dirdatablk = xfs_dir2_db_to_da(mp, XFS_DIR2_DATA_FIRSTDB(mp));
 	mp->m_dirleafblk = xfs_dir2_db_to_da(mp, XFS_DIR2_LEAF_FIRSTDB(mp));
 	mp->m_dirfreeblk = xfs_dir2_db_to_da(mp, XFS_DIR2_FREE_FIRSTDB(mp));
-	mp->m_attr_node_ents =
-		(mp->m_sb.sb_blocksize - (uint)sizeof(xfs_da_node_hdr_t)) /
-		(uint)sizeof(xfs_da_node_entry_t);
-	mp->m_dir_node_ents =
-		(mp->m_dirblksize - (uint)sizeof(xfs_da_node_hdr_t)) /
-		(uint)sizeof(xfs_da_node_entry_t);
+
+	nodehdr_size = __xfs_da3_node_hdr_size(xfs_sb_version_hascrc(&mp->m_sb));
+	mp->m_attr_node_ents = (mp->m_sb.sb_blocksize - nodehdr_size) /
+				(uint)sizeof(xfs_da_node_entry_t);
+	mp->m_dir_node_ents = (mp->m_dirblksize - nodehdr_size) /
+				(uint)sizeof(xfs_da_node_entry_t);
+
 	mp->m_dir_magicpct = (mp->m_dirblksize * 37) / 100;
 	if (xfs_sb_version_hasasciici(&mp->m_sb))
 		mp->m_dirnameops = &xfs_ascii_ci_nameops;

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH] Re: XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568

[Brian Foster]

On 08/26/2013 12:13 AM, Dave Chinner wrote:
> On Thu, Aug 22, 2013 at 02:28:00PM -0400, Brian Foster wrote:
>> Hi all,
>>
>> I hit an assert on a debug kernel while beating on some finobt work and
>> eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I
>> hit it through a couple different paths, first while running fsstress on
>> a CRC enabled filesystem (with otherwise default mkfs options):
>>
>> (These tests are running on a 4p, 4GB VM against a 100GB virtio disk,
>> hosted on a single spindle desktop box).
>>
>> crc=1
>> fsstress -z -fsymlink=1 -n99999999 -p4 -d /mnt/test
>>
>> XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length),
> 
> Directory buffer overrun.
> 
>>  [<ffffffffa031d549>] xfs_trans_log_buf+0x89/0x1b0 [xfs]
>>  [<ffffffffa02e7c1c>] xfs_da3_node_add+0x11c/0x210 [xfs]
>>  [<ffffffffa02ea703>] xfs_da3_node_split+0xc3/0x230 [xfs]
>>  [<ffffffffa02eaa18>] xfs_da3_split+0x1a8/0x410 [xfs]
>>  [<ffffffffa02f743f>] xfs_dir2_node_addname+0x47f/0xde0 [xfs]
> 
> During a split.
> 
> Easily reproduced with "seq 200000 | xargs touch" as Michael Semon
> reported last week.
> 
> The fix demonstrates my concerns about modifying directory code -
> the CRC changes missed a *fundamental* directory format definition,
> and we've only just tripped over it....
> 
>> rm -rf /mnt/test
>>
>> XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length),
> 
> Directory buffer overrun.
> 
>>  [<ffffffffa032b549>] xfs_trans_log_buf+0x89/0x1b0 [xfs]
>>  [<ffffffffa02f61ff>] xfs_da3_node_unbalance+0xef/0x1d0 [xfs]
>>  [<ffffffffa02f98b0>] xfs_da3_join+0x240/0x290 [xfs]
>>  [<ffffffffa030659b>] xfs_dir2_node_removename+0x69b/0x8b0 [xfs]
> 
> During a merge. Not sure why that is happening on a v4 filesystem.
> V5 filesystem, yes, due to the above bug but v4 should not be
> affected.
> 

Interesting, thanks Dave. FWIW, I no longer reproduce the assert in
either scenario with this patch applied. I also don't see how it would
make a difference for a v4 superblock filesystem. Perhaps that
particular test was bogus. I haven't heard if Mark happened to reproduce
that one. Regardless, consider it:

Tested-by: Brian Foster <bfoster@redhat.com>

(xfs: fix calculation of the number of node entries in a dir3 node)

Brian

> Cheers,
> 
> Dave.
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH] Re: XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568

[Mark Tinguely]

On 08/26/13 08:36, Brian Foster wrote:
> On 08/26/2013 12:13 AM, Dave Chinner wrote:
>> On Thu, Aug 22, 2013 at 02:28:00PM -0400, Brian Foster wrote:
>>> Hi all,
>>>
>>> I hit an assert on a debug kernel while beating on some finobt work and
>>> eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I
>>> hit it through a couple different paths, first while running fsstress on
>>> a CRC enabled filesystem (with otherwise default mkfs options):
>>>
>>> (These tests are running on a 4p, 4GB VM against a 100GB virtio disk,
>>> hosted on a single spindle desktop box).
>>>
>>> crc=1
>>> fsstress -z -fsymlink=1 -n99999999 -p4 -d /mnt/test
>>>
>>> XFS: Assertion failed: first<= last&&  last<  BBTOB(bp->b_length),
>>
>> Directory buffer overrun.
>>
>>>   [<ffffffffa031d549>] xfs_trans_log_buf+0x89/0x1b0 [xfs]
>>>   [<ffffffffa02e7c1c>] xfs_da3_node_add+0x11c/0x210 [xfs]
>>>   [<ffffffffa02ea703>] xfs_da3_node_split+0xc3/0x230 [xfs]
>>>   [<ffffffffa02eaa18>] xfs_da3_split+0x1a8/0x410 [xfs]
>>>   [<ffffffffa02f743f>] xfs_dir2_node_addname+0x47f/0xde0 [xfs]
>>
>> During a split.
>>
>> Easily reproduced with "seq 200000 | xargs touch" as Michael Semon
>> reported last week.
>>
>> The fix demonstrates my concerns about modifying directory code -
>> the CRC changes missed a *fundamental* directory format definition,
>> and we've only just tripped over it....

I agree. As we see here, bugs in common directory code effect all 
filesystems. It may not matter if the feature the code was written for 
is enabled or not.

>>> rm -rf /mnt/test
>>>
>>> XFS: Assertion failed: first<= last&&  last<  BBTOB(bp->b_length),
>>
>> Directory buffer overrun.
>>
>>>   [<ffffffffa032b549>] xfs_trans_log_buf+0x89/0x1b0 [xfs]
>>>   [<ffffffffa02f61ff>] xfs_da3_node_unbalance+0xef/0x1d0 [xfs]
>>>   [<ffffffffa02f98b0>] xfs_da3_join+0x240/0x290 [xfs]
>>>   [<ffffffffa030659b>] xfs_dir2_node_removename+0x69b/0x8b0 [xfs]
>>
>> During a merge. Not sure why that is happening on a v4 filesystem.
>> V5 filesystem, yes, due to the above bug but v4 should not be
>> affected.
>>
>
> Interesting, thanks Dave. FWIW, I no longer reproduce the assert in
> either scenario with this patch applied. I also don't see how it would
> make a difference for a v4 superblock filesystem. Perhaps that
> particular test was bogus. I haven't heard if Mark happened to reproduce
> that one. Regardless, consider it:
>
> Tested-by: Brian Foster<bfoster@redhat.com>
>
> (xfs: fix calculation of the number of node entries in a dir3 node)

I got the XFS v4 to assert on the remove in Linux 3.10 and 3.11.

With the patch, a shorter test on Linux 3.10 did not assert. I will do 
the full test on Linux 3.10/3.11, review and report back.

>
> Brian
>
>> Cheers,
>>
>> Dave.

--Mark.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH] Re: XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568

[Michael L. Semon]

On 08/26/2013 12:13 AM, Dave Chinner wrote:
> On Thu, Aug 22, 2013 at 02:28:00PM -0400, Brian Foster wrote:
>> Hi all,
>>
>> I hit an assert on a debug kernel while beating on some finobt work and
>> eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I
>> hit it through a couple different paths, first while running fsstress on
>> a CRC enabled filesystem (with otherwise default mkfs options):
>>
>> (These tests are running on a 4p, 4GB VM against a 100GB virtio disk,
>> hosted on a single spindle desktop box).
>>
>> crc=1
>> fsstress -z -fsymlink=1 -n99999999 -p4 -d /mnt/test
>>
>> XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length),
> 
> Directory buffer overrun.
> 
>>  [<ffffffffa031d549>] xfs_trans_log_buf+0x89/0x1b0 [xfs]
>>  [<ffffffffa02e7c1c>] xfs_da3_node_add+0x11c/0x210 [xfs]
>>  [<ffffffffa02ea703>] xfs_da3_node_split+0xc3/0x230 [xfs]
>>  [<ffffffffa02eaa18>] xfs_da3_split+0x1a8/0x410 [xfs]
>>  [<ffffffffa02f743f>] xfs_dir2_node_addname+0x47f/0xde0 [xfs]
> 
> During a split.
> 
> Easily reproduced with "seq 200000 | xargs touch" as Michael Semon
> reported last week.
> 
> The fix demonstrates my concerns about modifying directory code -
> the CRC changes missed a *fundamental* directory format definition,
> and we've only just tripped over it....

Don't fret too much over it.  This test was part of coreutils, which 
is something that I rebuild after a glibc upgrade.  Had glibc-2.18 
been released six weeks ago, then I would have stumbled onto this 
XFS issue six weeks ago.

>> rm -rf /mnt/test
>>
>> XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length),
> 
> Directory buffer overrun.
> 
>>  [<ffffffffa032b549>] xfs_trans_log_buf+0x89/0x1b0 [xfs]
>>  [<ffffffffa02f61ff>] xfs_da3_node_unbalance+0xef/0x1d0 [xfs]
>>  [<ffffffffa02f98b0>] xfs_da3_join+0x240/0x290 [xfs]
>>  [<ffffffffa030659b>] xfs_dir2_node_removename+0x69b/0x8b0 [xfs]
> 
> During a merge. Not sure why that is happening on a v4 filesystem.
> V5 filesystem, yes, due to the above bug but v4 should not be
> affected.
> 
> Cheers,
> 
> Dave.

Your patch looks good, and I even applied it to vanilla 3.10.9, 
along with Jeff Liu's MAX_LFS_FILESIZE patch.  [Murphy's Law states 
that if I didn't use Jeff's patch, then I would run xfstests 
generic/308 on accident, leading to a hung umount.  Happens every 
single time.]  Both patches applied cleanly to kernels on a 2.8 GHz 
i686 Pentium 4 PC that was running Slackware 14.0 Linux.

Naturally, `seq 200000 | xargs touch` was run for v5 and v4 XFS 
file systems.  All was okay.  The removal of the populated directory 
went fine as well.

The v5 file systems were tested using a 3.11-rc7+ git kernel.  
xfstests was run from the start of generic/ through generic/127; 
and that went fine.  Some of the xfs/* series was run but merely 
scanned because the v5-output-cleanup patches were not readily 
available.

The v4 file systems were tested with a patched vanilla 3.10.9 kernel, 
and some of generic was run, with patched and unpatched kernels showing 
the same good results, very little difference in timing overall.

Thanks!

Michael

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH] Re: XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568

[Dave Chinner]

On Mon, Aug 26, 2013 at 10:00:24AM -0500, Mark Tinguely wrote:
> On 08/26/13 08:36, Brian Foster wrote:
> >On 08/26/2013 12:13 AM, Dave Chinner wrote:
> >>On Thu, Aug 22, 2013 at 02:28:00PM -0400, Brian Foster wrote:
> >>>Hi all,
> >>>
> >>>I hit an assert on a debug kernel while beating on some finobt work and
> >>>eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I
> >>>hit it through a couple different paths, first while running fsstress on
> >>>a CRC enabled filesystem (with otherwise default mkfs options):
> >>>
> >>>(These tests are running on a 4p, 4GB VM against a 100GB virtio disk,
> >>>hosted on a single spindle desktop box).
> >>>
> >>>crc=1
> >>>fsstress -z -fsymlink=1 -n99999999 -p4 -d /mnt/test
> >>>
> >>>XFS: Assertion failed: first<= last&&  last<  BBTOB(bp->b_length),
> >>
> >>Directory buffer overrun.
> >>
> >>>  [<ffffffffa031d549>] xfs_trans_log_buf+0x89/0x1b0 [xfs]
> >>>  [<ffffffffa02e7c1c>] xfs_da3_node_add+0x11c/0x210 [xfs]
> >>>  [<ffffffffa02ea703>] xfs_da3_node_split+0xc3/0x230 [xfs]
> >>>  [<ffffffffa02eaa18>] xfs_da3_split+0x1a8/0x410 [xfs]
> >>>  [<ffffffffa02f743f>] xfs_dir2_node_addname+0x47f/0xde0 [xfs]
> >>
> >>During a split.
> >>
> >>Easily reproduced with "seq 200000 | xargs touch" as Michael Semon
> >>reported last week.
> >>
> >>The fix demonstrates my concerns about modifying directory code -
> >>the CRC changes missed a *fundamental* directory format definition,
> >>and we've only just tripped over it....
> 
> I agree. As we see here, bugs in common directory code effect all
> filesystems. It may not matter if the feature the code was written
> for is enabled or not.

Well, this is *only* a v5 bug. The fact is, the only difference the
change I made makes to v4 filesystems is that it removed the typedef
from the sizeof calculation. On my test systems, the value
mp->m_dir_node_ents is identical for v4 filesystems with or without
the patch applied.....

> >>During a merge. Not sure why that is happening on a v4 filesystem.
> >>V5 filesystem, yes, due to the above bug but v4 should not be
> >>affected.
> >>
> >
> >Interesting, thanks Dave. FWIW, I no longer reproduce the assert in
> >either scenario with this patch applied. I also don't see how it would
> >make a difference for a v4 superblock filesystem. Perhaps that
> >particular test was bogus. I haven't heard if Mark happened to reproduce
> >that one. Regardless, consider it:
> >
> >Tested-by: Brian Foster<bfoster@redhat.com>
> >
> >(xfs: fix calculation of the number of node entries in a dir3 node)
> 
> I got the XFS v4 to assert on the remove in Linux 3.10 and 3.11.

Did you test 3.9 - before the crc changes were made to the
filesystem?  i.e. if an invalid mp->m_dir_node_ents value is the
real cause of the v4 filesystem problem, then it should reproduce on
just about every kernel we chose to test.

> With the patch, a shorter test on Linux 3.10 did not assert. I will
> do the full test on Linux 3.10/3.11, review and report back.

Because nobody can explain why this patch would fix a problem on a v4
filesystem, we need more triage of the v4 problem needs to be done. I
haven't been able to reproduce the unlink issue (and don't have time
to do everything), so could you triage the problem further, Mark?
We really need to understand the root cause of the problem on v4
filesystems so we can determine what the impact of it is...

Cheers,

Dave.

> 
> >
> >Brian
> >
> >>Cheers,
> >>
> >>Dave.
> 
> --Mark.
> 

-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH] Re: XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568

[Mark Tinguely]

On 08/26/13 16:04, Dave Chinner wrote:
> On Mon, Aug 26, 2013 at 10:00:24AM -0500, Mark Tinguely wrote:
>> On 08/26/13 08:36, Brian Foster wrote:
>>> On 08/26/2013 12:13 AM, Dave Chinner wrote:
>>>> On Thu, Aug 22, 2013 at 02:28:00PM -0400, Brian Foster wrote:
>>>>> Hi all,
>>>>>
>>>>> I hit an assert on a debug kernel while beating on some finobt work and
>>>>> eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I
>>>>> hit it through a couple different paths, first while running fsstress on
>>>>> a CRC enabled filesystem (with otherwise default mkfs options):
>>>>>
>>>>> (These tests are running on a 4p, 4GB VM against a 100GB virtio disk,
>>>>> hosted on a single spindle desktop box).
>>>>>
>>>>> crc=1
>>>>> fsstress -z -fsymlink=1 -n99999999 -p4 -d /mnt/test
>>>>>
>>>>> XFS: Assertion failed: first<= last&&   last<   BBTOB(bp->b_length),
>>>>
>>>> Directory buffer overrun.
>>>>
>>>>>   [<ffffffffa031d549>] xfs_trans_log_buf+0x89/0x1b0 [xfs]
>>>>>   [<ffffffffa02e7c1c>] xfs_da3_node_add+0x11c/0x210 [xfs]
>>>>>   [<ffffffffa02ea703>] xfs_da3_node_split+0xc3/0x230 [xfs]
>>>>>   [<ffffffffa02eaa18>] xfs_da3_split+0x1a8/0x410 [xfs]
>>>>>   [<ffffffffa02f743f>] xfs_dir2_node_addname+0x47f/0xde0 [xfs]
>>>>
>>>> During a split.
>>>>
>>>> Easily reproduced with "seq 200000 | xargs touch" as Michael Semon
>>>> reported last week.
>>>>
>>>> The fix demonstrates my concerns about modifying directory code -
>>>> the CRC changes missed a *fundamental* directory format definition,
>>>> and we've only just tripped over it....
>>
>> I agree. As we see here, bugs in common directory code effect all
>> filesystems. It may not matter if the feature the code was written
>> for is enabled or not.
>
> Well, this is *only* a v5 bug. The fact is, the only difference the
> change I made makes to v4 filesystems is that it removed the typedef
> from the sizeof calculation. On my test systems, the value
> mp->m_dir_node_ents is identical for v4 filesystems with or without
> the patch applied.....
>
>>>> During a merge. Not sure why that is happening on a v4 filesystem.
>>>> V5 filesystem, yes, due to the above bug but v4 should not be
>>>> affected.
>>>>
>>>
>>> Interesting, thanks Dave. FWIW, I no longer reproduce the assert in
>>> either scenario with this patch applied. I also don't see how it would
>>> make a difference for a v4 superblock filesystem. Perhaps that
>>> particular test was bogus. I haven't heard if Mark happened to reproduce
>>> that one. Regardless, consider it:
>>>
>>> Tested-by: Brian Foster<bfoster@redhat.com>
>>>
>>> (xfs: fix calculation of the number of node entries in a dir3 node)
>>
>> I got the XFS v4 to assert on the remove in Linux 3.10 and 3.11.
>
> Did you test 3.9 - before the crc changes were made to the
> filesystem?  i.e. if an invalid mp->m_dir_node_ents value is the
> real cause of the v4 filesystem problem, then it should reproduce on
> just about every kernel we chose to test.
>
>> With the patch, a shorter test on Linux 3.10 did not assert. I will
>> do the full test on Linux 3.10/3.11, review and report back.
>
> Because nobody can explain why this patch would fix a problem on a v4
> filesystem, we need more triage of the v4 problem needs to be done. I
> haven't been able to reproduce the unlink issue (and don't have time
> to do everything), so could you triage the problem further, Mark?
> We really need to understand the root cause of the problem on v4
> filesystems so we can determine what the impact of it is...
>
> Cheers,
>
> Dave.

A full test still asserts on the remove with the patched Linux 3.10 - I 
am about 50% into the retest of Linux 3.10 and then I was planning to 
move back to Linux 3.9.

kdump did not work, so I have no vmcore and therefore no productive 
information.

--Mark.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH] Re: XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568

[Mark Tinguely]

On 08/26/13 16:19, Mark Tinguely wrote:
> On 08/26/13 16:04, Dave Chinner wrote:
>> On Mon, Aug 26, 2013 at 10:00:24AM -0500, Mark Tinguely wrote:
>>> On 08/26/13 08:36, Brian Foster wrote:
>>>> On 08/26/2013 12:13 AM, Dave Chinner wrote:
>>>>> On Thu, Aug 22, 2013 at 02:28:00PM -0400, Brian Foster wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> I hit an assert on a debug kernel while beating on some finobt
>>>>>> work and
>>>>>> eventually reproduced it on unmodified/TOT xfs/xfsprogs as of
>>>>>> today. I
>>>>>> hit it through a couple different paths, first while running
>>>>>> fsstress on
>>>>>> a CRC enabled filesystem (with otherwise default mkfs options):
>>>>>>
>>>>>> (These tests are running on a 4p, 4GB VM against a 100GB virtio disk,
>>>>>> hosted on a single spindle desktop box).
>>>>>>
>>>>>> crc=1
>>>>>> fsstress -z -fsymlink=1 -n99999999 -p4 -d /mnt/test
>>>>>>
>>>>>> XFS: Assertion failed: first<= last&& last< BBTOB(bp->b_length),
>>>>>
>>>>> Directory buffer overrun.

> A full test still asserts on the remove with the patched Linux 3.10 - I
> am about 50% into the retest of Linux 3.10 and then I was planning to
> move back to Linux 3.9.
>
> kdump did not work, so I have no vmcore and therefore no productive
> information.
>

Confirmed the Linux 3.10 asserts. Linux 3.9 does not assert.

I will fix the kdump and try to catch a Linux 3.10 assert.

--Mark.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH] Re: XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568

[Mark Tinguely]

On 08/25/13 23:13, Dave Chinner wrote:
> xfs: fix calculation of the number of node entries in a dir3 node
>
> From: Dave Chinner<dchinner@redhat.com>
>
> The calculation doesn't take into account the size of the dir v3
> header, so overestimates the hash entries in a node. This causes
> directory buffer overruns when splitting and merging nodes.
>
> Signed-off-by: Dave Chinner<dchinner@redhat.com>
> ---

Makes sense to me for the inode v3 assert.

Reviewed-by: Mark Tinguely <tinguely@sgi.com>

The remove assert with the same test must be in common code; it can be 
triggered in both versions of the inode (Linux 3.10 - TOT). The assert 
on the remove may require the test to be run until the filesystem is 
full before doing the remove.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH] Re: XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568

[Ben Myers]

On Thu, Aug 29, 2013 at 07:37:42AM -0500, Mark Tinguely wrote:
> On 08/25/13 23:13, Dave Chinner wrote:
> >xfs: fix calculation of the number of node entries in a dir3 node
> >
> >From: Dave Chinner<dchinner@redhat.com>
> >
> >The calculation doesn't take into account the size of the dir v3
> >header, so overestimates the hash entries in a node. This causes
> >directory buffer overruns when splitting and merging nodes.
> >
> >Signed-off-by: Dave Chinner<dchinner@redhat.com>
> >---
> 
> Makes sense to me for the inode v3 assert.
> 
> Reviewed-by: Mark Tinguely <tinguely@sgi.com>
> 
> The remove assert with the same test must be in common code; it can
> be triggered in both versions of the inode (Linux 3.10 - TOT). The
> assert on the remove may require the test to be run until the
> filesystem is full before doing the remove.

Applied.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568

[Mark Tinguely]

On 08/22/13 13:28, Brian Foster wrote:
> Hi all,
>
> I hit an assert on a debug kernel while beating on some finobt work and
> eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I
> hit it through a couple different paths, first while running fsstress on
> a CRC enabled filesystem (with otherwise default mkfs options):
>
> (These tests are running on a 4p, 4GB VM against a 100GB virtio disk,
> hosted on a single spindle desktop box).
>
> crc=1
> fsstress -z -fsymlink=1 -n99999999 -p4 -d /mnt/test
>
> XFS: Assertion failed: first<= last&&  last<  BBTOB(bp->b_length),
> file: fs/xfs/xfs_trans_buf.c, line: 568
> ------------[ cut here ]------------
> kernel BUG at fs/xfs/xfs_message.c:108!
> invalid opcode: 0000 [#1] SMP
> Modules linked in: xfs libcrc32c fuse ebtable_nat
> nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE
> ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6
> nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle bnep
> nf_conntrack_ipv4 nf_defrag_ipv4 bluetooth xt_conntrack nf_conntrack
> rfkill ebtable_filter ebtables ip6table_filter ip6_tables snd_hda_intel
> snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm snd_page_alloc
> snd_timer snd joydev soundcore i2c_piix4 pcspkr mperf virtio_balloon
> floppy uinput qxl drm_kms_helper ttm drm virtio_blk virtio_net i2c_core
> CPU: 0 PID: 1419 Comm: fsstress Not tainted 3.11.0-rc1+ #10
> Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
> task: ffff8800d65b5dc0 ti: ffff8800d10ba000 task.ti: ffff8800d10ba000
> RIP: 0010:[<ffffffffa02b8812>]  [<ffffffffa02b8812>] assfail+0x22/0x30 [xfs]
> RSP: 0018:ffff8800d10bb998  EFLAGS: 00010292
> RAX: 000000000000006b RBX: ffff8800d67be3a0 RCX: 0000000000000000
> RDX: ffff88011fc0ee48 RSI: ffff88011fc0d038 RDI: ffff88011fc0d038
> RBP: ffff8800d10bb998 R08: 0000000000000000 R09: 000000000000020a
> R10: ffffffff81858260 R11: 0000000000000209 R12: ffff8800d165d500
> R13: ffff8800d1158980 R14: 0000000000001007 R15: ffff8800d1cb8300
> FS:  00007f1efd2ce740(0000) GS:ffff88011fc00000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00007f1ef80fb018 CR3: 0000000036edb000 CR4: 00000000000006f0
> Stack:
>   ffff8800d10bb9e8 ffffffffa031d549 000000fc24a6f000 00000e20000000d3
>   ffff8800d10bb9f8 ffff8800d67c3040 ffff8800d1cb8208 ffff8800d1cb81e8
>   ffff8800d67c3000 ffff8800d1cb8300 ffff8800d10bba48 ffffffffa02e7c1c
> Call Trace:
>   [<ffffffffa031d549>] xfs_trans_log_buf+0x89/0x1b0 [xfs]
>   [<ffffffffa02e7c1c>] xfs_da3_node_add+0x11c/0x210 [xfs]
>   [<ffffffffa02ea703>] xfs_da3_node_split+0xc3/0x230 [xfs]
>   [<ffffffffa02eaa18>] xfs_da3_split+0x1a8/0x410 [xfs]
>   [<ffffffffa02f743f>] xfs_dir2_node_addname+0x47f/0xde0 [xfs]
>   [<ffffffffa02ec965>] xfs_dir_createname+0x1d5/0x1e0 [xfs]
>   [<ffffffffa02c1607>] ? kmem_alloc+0x67/0xf0 [xfs]
>   [<ffffffffa02becb9>] xfs_symlink+0x619/0xa20 [xfs]
>   [<ffffffff811abad6>] ? _d_rehash+0x36/0x40
>   [<ffffffff8119f498>] ? __lookup_hash+0x38/0x50
>   [<ffffffff8119f4c9>] ? lookup_hash+0x19/0x20
>   [<ffffffff811a21ee>] ? kern_path_create+0x8e/0x170
>   [<ffffffffa02b5e5c>] xfs_vn_symlink+0x5c/0xe0 [xfs]
>   [<ffffffff811a3939>] vfs_symlink+0x99/0x100
>   [<ffffffff811a59d6>] SyS_symlinkat+0x66/0xd0
>   [<ffffffff811a5a56>] SyS_symlink+0x16/0x20
>   [<ffffffff81645442>] system_call_fastpath+0x16/0x1b
> Code: 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 f1 41 89 d0 48
> c7 c6 70 50 33 a0 48 89 fa 31 c0 48 89 e5 31 ff e8 de fb ff ff<0f>  0b
> 66 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48
> RIP  [<ffffffffa02b8812>] assfail+0x22/0x30 [xfs]
>   RSP<ffff8800d10bb998>
> ---[ end trace 9578edaae955ff56 ]---
>
> I repeated the test on a crc=0 fs (with -isize=512) and could not
> reproduce during fsstress. I let it populate to about 10GB and
> ultimately hit the same assert on unlink during a post-test cleanup:
>
> crc=0
> rm -rf /mnt/test
>
> XFS: Assertion failed: first<= last&&  last<  BBTOB(bp->b_length),
> file: fs/xfs/xfs_trans_buf.c, line: 568
> ------------[ cut here ]------------
> kernel BUG at fs/xfs/xfs_message.c:108!
> invalid opcode: 0000 [#1] SMP
> Modules linked in: xfs libcrc32c fuse ebtable_nat
> nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE
> ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6
> nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle
> nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack
> ebtable_filter ebtables bnep bluetooth rfkill ip6table_filter ip6_tables
> snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm
> snd_page_alloc snd_timer snd soundcore joydev pcspkr virtio_balloon
> i2c_piix4 floppy mperf uinput qxl drm_kms_helper ttm drm virtio_net
> virtio_blk i2c_core
> CPU: 1 PID: 2198 Comm: rm Not tainted 3.11.0-rc1+ #10
> Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
> task: ffff8801161ec650 ti: ffff8800c803e000 task.ti: ffff8800c803e000
> RIP: 0010:[<ffffffffa02c6812>]  [<ffffffffa02c6812>] assfail+0x22/0x30 [xfs]
> RSP: 0018:ffff8800c803fa98  EFLAGS: 00010292
> RAX: 000000000000006b RBX: ffff8801029a6e80 RCX: 0000000000000000
> RDX: ffff88011fc8ee48 RSI: ffff88011fc8d038 RDI: ffff88011fc8d038
> RBP: ffff8800c803fa98 R08: 0000000000000000 R09: 0000000000000209
> R10: ffffffff81858260 R11: 0000000000000208 R12: ffff8800302bd200
> R13: ffff8800d25e0850 R14: 000000000000122f R15: ffff8800d271f010
> FS:  00007f28ef9bf740(0000) GS:ffff88011fc80000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: 000000000153a000 CR3: 00000000b1fd3000 CR4: 00000000000006e0
> Stack:
>   ffff8800c803fae8 ffffffffa032b549 00800201008006cc 000000100185febe
>   ffffffffa033fcb0 ffff8800ade0c010 ffff8800ade0c000 ffff8800d3c2b9e0
>   ffff8800d25e0850 ffff8800d271f010 ffff8800c803fb58 ffffffffa02f61ff
> Call Trace:
>   [<ffffffffa032b549>] xfs_trans_log_buf+0x89/0x1b0 [xfs]
>   [<ffffffffa02f61ff>] xfs_da3_node_unbalance+0xef/0x1d0 [xfs]
>   [<ffffffffa02f98b0>] xfs_da3_join+0x240/0x290 [xfs]
>   [<ffffffffa030659b>] xfs_dir2_node_removename+0x69b/0x8b0 [xfs]
>   [<ffffffffa02e16ce>] ? xfs_bmap_last_extent+0x6e/0xb0 [xfs]
>   [<ffffffffa02fa5b5>] xfs_dir_removename+0x195/0x1a0 [xfs]
>   [<ffffffffa0310b69>] xfs_remove+0x2a9/0x410 [xfs]
>   [<ffffffffa02c3ca2>] xfs_vn_unlink+0x52/0xa0 [xfs]
>   [<ffffffff811a260e>] vfs_unlink+0x9e/0x110
>   [<ffffffff811a2821>] do_unlinkat+0x1a1/0x230
>   [<ffffffff811a592b>] SyS_unlinkat+0x1b/0x40
>   [<ffffffff81645442>] system_call_fastpath+0x16/0x1b
> Code: 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 f1 41 89 d0 48
> c7 c6 70 30 34 a0 48 89 fa 31 c0 48 89 e5 31 ff e8 de fb ff ff<0f>  0b
> 66 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48
> RIP  [<ffffffffa02c6812>] assfail+0x22/0x30 [xfs]
>   RSP<ffff8800c803fa98>
> ---[ end trace 3ef54f36db3ba0c5 ]---
>
> Info on the crc=0 fs is as follows:
>
> meta-data=/dev/vdb               isize=512    agcount=4, agsize=6553600 blks
>           =                       sectsz=512   attr=2, projid32bit=1
>           =                       crc=0
> data     =                       bsize=4096   blocks=26214400, imaxpct=25
>           =                       sunit=0      swidth=0 blks
> naming   =version 2              bsize=4096   ascii-ci=0
> log      =internal               bsize=4096   blocks=12800, version=2
>           =                       sectsz=512   sunit=0 blks, lazy-count=1
> realtime =none                   extsz=4096   blocks=0, rtextents=0
>
>
> Brian

FYI:

The second (rm version) of the test bisects to the patch:

commit f5ea110044fa858925a880b4fa9f551bfa2dfc38

     xfs: add CRCs to dir2/da node blocks

     ---

The secret to tripping over the bug is run the test until fsstress fills 
the filesystem before removing the files. So an error handling?

I use the test:

#!/bin/sh

ltp/fsstress -z -s 1378390208 -fsymlink=1 -n9999999 -p4 -d /test2
cd /test2
sync
rm -rf *

If your filesystem is smaller, decrease the -n to make the test faster.

I have still not gotten a core, though Michael Semon sent one.

--Mark.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568

[Christoph Hellwig]

On Thu, Sep 12, 2013 at 06:51:05PM -0500, Mark Tinguely wrote:
> The secret to tripping over the bug is run the test until fsstress
> fills the filesystem before removing the files. So an error
> handling?
> 
> I use the test:
> 
> #!/bin/sh
> 
> ltp/fsstress -z -s 1378390208 -fsymlink=1 -n9999999 -p4 -d /test2
> cd /test2
> sync
> rm -rf *
> 
> If your filesystem is smaller, decrease the -n to make the test faster.
> 
> I have still not gotten a core, though Michael Semon sent one.

It would be useful if we could wire this up for xfstests

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568

[Mark Tinguely]

On 09/16/13 10:44, Christoph Hellwig wrote:
> On Thu, Sep 12, 2013 at 06:51:05PM -0500, Mark Tinguely wrote:
>> The secret to tripping over the bug is run the test until fsstress
>> fills the filesystem before removing the files. So an error
>> handling?
>>
>> I use the test:
>>
>> #!/bin/sh
>>
>> ltp/fsstress -z -s 1378390208 -fsymlink=1 -n9999999 -p4 -d /test2
>> cd /test2
>> sync
>> rm -rf *
>>
>> If your filesystem is smaller, decrease the -n to make the test faster.
>>
>> I have still not gotten a core, though Michael Semon sent one.
>
> It would be useful if we could wire this up for xfstests
>

Nod

--Mark.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568

[Michael L. Semon]

On Mon, Sep 16, 2013 at 11:44 AM, Christoph Hellwig <hch@infradead.org> wrote:
> On Thu, Sep 12, 2013 at 06:51:05PM -0500, Mark Tinguely wrote:
>> The secret to tripping over the bug is run the test until fsstress
>> fills the filesystem before removing the files. So an error
>> handling?
>>
>> I use the test:
>>
>> #!/bin/sh
>>
>> ltp/fsstress -z -s 1378390208 -fsymlink=1 -n9999999 -p4 -d /test2
>> cd /test2
>> sync
>> rm -rf *
>>
>> If your filesystem is smaller, decrease the -n to make the test faster.
>>
>> I have still not gotten a core, though Michael Semon sent one.
>
> It would be useful if we could wire this up for xfstests

Just set it up accurately because it takes a long time.   It takes a
while to create the links.  Additionally, fsstress will keep iterating
after the FS is full, and all those extra iterations take time.

In testing on x86, Brian's test will succeed in 10 GB but fail in 11
GB.  I don't know if that is the case on x86_64 as well.  After the
failure case is met and the rm operation hits the assert, continued
mount-and-rm operations will also cause the assert to fire, without
needing to fill the FS with links again.  I don't know when that
"bonus condition" stops happening.  xfs_repair makes matters
worse, IIRC.

That's about all I can add.  Thanks!

Michael

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs