* [PATCH] xfs: add agf freeblocks verify in xfs_agf_verify
@ 2020-02-13 9:53 Zheng Bin
2020-02-13 13:37 ` Eric Sandeen
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Zheng Bin @ 2020-02-13 9:53 UTC (permalink / raw)
To: david, darrick.wong, sandeen, linux-xfs; +Cc: renxudong1
We recently used fuzz(hydra) to test XFS and automatically generate
tmp.img(XFS v5 format, but some metadata is wrong)
Test as follows:
mount tmp.img tmpdir
cp file1M tmpdir
sync
tmpdir/file1M size is 1M, but its data can not sync to disk.
This is because tmp.img has some problems, using xfs_repair detect
information as follows:
agf_freeblks 0, counted 3224 in ag 0
agf_longest 536874136, counted 3224 in ag 0
sb_fdblocks 613, counted 3228
Add these agf freeblocks checks:
1. agf_longest < agf_freeblks
2. agf_freeblks < sb_fdblocks
Signed-off-by: Zheng Bin <zhengbin13@huawei.com>
Signed-off-by: Ren Xudong <renxudong1@huawei.com>
---
fs/xfs/libxfs/xfs_alloc.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/fs/xfs/libxfs/xfs_alloc.c b/fs/xfs/libxfs/xfs_alloc.c
index d8053bc..0f4b4d1 100644
--- a/fs/xfs/libxfs/xfs_alloc.c
+++ b/fs/xfs/libxfs/xfs_alloc.c
@@ -2858,6 +2858,10 @@ xfs_agf_verify(
be32_to_cpu(agf->agf_flcount) <= xfs_agfl_size(mp)))
return __this_address;
+ if (be32_to_cpu(agf->agf_freeblks) < be32_to_cpu(agf->agf_longest) ||
+ be32_to_cpu(agf->agf_freeblks) >= mp->m_sb.sb_fdblocks)
+ return __this_address;
+
if (be32_to_cpu(agf->agf_levels[XFS_BTNUM_BNO]) < 1 ||
be32_to_cpu(agf->agf_levels[XFS_BTNUM_CNT]) < 1 ||
be32_to_cpu(agf->agf_levels[XFS_BTNUM_BNO]) > XFS_BTREE_MAXLEVELS ||
--
2.7.4
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] xfs: add agf freeblocks verify in xfs_agf_verify
2020-02-13 9:53 [PATCH] xfs: add agf freeblocks verify in xfs_agf_verify Zheng Bin
@ 2020-02-13 13:37 ` Eric Sandeen
2020-02-13 17:44 ` Darrick J. Wong
2020-02-13 20:55 ` Dave Chinner
2 siblings, 0 replies; 6+ messages in thread
From: Eric Sandeen @ 2020-02-13 13:37 UTC (permalink / raw)
To: Zheng Bin, david, darrick.wong, sandeen, linux-xfs; +Cc: renxudong1
On 2/13/20 3:53 AM, Zheng Bin wrote:
> We recently used fuzz(hydra) to test XFS and automatically generate
> tmp.img(XFS v5 format, but some metadata is wrong)
>
> Test as follows:
> mount tmp.img tmpdir
> cp file1M tmpdir
> sync
I would leave this part out of the change log, because it is not a
useful testcase for anyone who does not have your test.img.
> tmpdir/file1M size is 1M, but its data can not sync to disk.
>
> This is because tmp.img has some problems, using xfs_repair detect
> information as follows:
>
> agf_freeblks 0, counted 3224 in ag 0
> agf_longest 536874136, counted 3224 in ag 0
> sb_fdblocks 613, counted 3228
>
> Add these agf freeblocks checks:
> 1. agf_longest < agf_freeblks
> 2. agf_freeblks < sb_fdblocks
This left out the 2nd check Dave suggested, that freeblocks is
less than the number of blocks in the AG, i.e.
b) agf_freeblks < sb_dblocks / sb_agcount
(and yes, it must special-case the last AG which may be smaller - see
xfs_ag_block_count())
-Eric
> Signed-off-by: Zheng Bin <zhengbin13@huawei.com>
> Signed-off-by: Ren Xudong <renxudong1@huawei.com>
> ---
> fs/xfs/libxfs/xfs_alloc.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/fs/xfs/libxfs/xfs_alloc.c b/fs/xfs/libxfs/xfs_alloc.c
> index d8053bc..0f4b4d1 100644
> --- a/fs/xfs/libxfs/xfs_alloc.c
> +++ b/fs/xfs/libxfs/xfs_alloc.c
> @@ -2858,6 +2858,10 @@ xfs_agf_verify(
> be32_to_cpu(agf->agf_flcount) <= xfs_agfl_size(mp)))
> return __this_address;
>
> + if (be32_to_cpu(agf->agf_freeblks) < be32_to_cpu(agf->agf_longest) ||
> + be32_to_cpu(agf->agf_freeblks) >= mp->m_sb.sb_fdblocks)
> + return __this_address;
> +
> if (be32_to_cpu(agf->agf_levels[XFS_BTNUM_BNO]) < 1 ||
> be32_to_cpu(agf->agf_levels[XFS_BTNUM_CNT]) < 1 ||
> be32_to_cpu(agf->agf_levels[XFS_BTNUM_BNO]) > XFS_BTREE_MAXLEVELS ||
> --
> 2.7.4
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] xfs: add agf freeblocks verify in xfs_agf_verify
2020-02-13 9:53 [PATCH] xfs: add agf freeblocks verify in xfs_agf_verify Zheng Bin
2020-02-13 13:37 ` Eric Sandeen
@ 2020-02-13 17:44 ` Darrick J. Wong
2020-02-13 20:55 ` Dave Chinner
2 siblings, 0 replies; 6+ messages in thread
From: Darrick J. Wong @ 2020-02-13 17:44 UTC (permalink / raw)
To: Zheng Bin; +Cc: david, sandeen, linux-xfs, renxudong1
On Thu, Feb 13, 2020 at 05:53:59PM +0800, Zheng Bin wrote:
> We recently used fuzz(hydra) to test XFS and automatically generate
> tmp.img(XFS v5 format, but some metadata is wrong)
>
> Test as follows:
> mount tmp.img tmpdir
> cp file1M tmpdir
> sync
>
> tmpdir/file1M size is 1M, but its data can not sync to disk.
>
> This is because tmp.img has some problems, using xfs_repair detect
> information as follows:
>
> agf_freeblks 0, counted 3224 in ag 0
> agf_longest 536874136, counted 3224 in ag 0
> sb_fdblocks 613, counted 3228
>
> Add these agf freeblocks checks:
> 1. agf_longest < agf_freeblks
> 2. agf_freeblks < sb_fdblocks
Uh... what problem did you encounter? Did block allocation loop
forever? Did errors come pouring out of dmesg? Did other strange
behaviors erupt? What is the smallest number of steps needed to go from
a fresh format to ... whatever went wrong here?
That's what this commit message ought to capture. :)
--D
> Signed-off-by: Zheng Bin <zhengbin13@huawei.com>
> Signed-off-by: Ren Xudong <renxudong1@huawei.com>
> ---
> fs/xfs/libxfs/xfs_alloc.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/fs/xfs/libxfs/xfs_alloc.c b/fs/xfs/libxfs/xfs_alloc.c
> index d8053bc..0f4b4d1 100644
> --- a/fs/xfs/libxfs/xfs_alloc.c
> +++ b/fs/xfs/libxfs/xfs_alloc.c
> @@ -2858,6 +2858,10 @@ xfs_agf_verify(
> be32_to_cpu(agf->agf_flcount) <= xfs_agfl_size(mp)))
> return __this_address;
>
> + if (be32_to_cpu(agf->agf_freeblks) < be32_to_cpu(agf->agf_longest) ||
> + be32_to_cpu(agf->agf_freeblks) >= mp->m_sb.sb_fdblocks)
> + return __this_address;
> +
> if (be32_to_cpu(agf->agf_levels[XFS_BTNUM_BNO]) < 1 ||
> be32_to_cpu(agf->agf_levels[XFS_BTNUM_CNT]) < 1 ||
> be32_to_cpu(agf->agf_levels[XFS_BTNUM_BNO]) > XFS_BTREE_MAXLEVELS ||
> --
> 2.7.4
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] xfs: add agf freeblocks verify in xfs_agf_verify
2020-02-13 9:53 [PATCH] xfs: add agf freeblocks verify in xfs_agf_verify Zheng Bin
2020-02-13 13:37 ` Eric Sandeen
2020-02-13 17:44 ` Darrick J. Wong
@ 2020-02-13 20:55 ` Dave Chinner
2 siblings, 0 replies; 6+ messages in thread
From: Dave Chinner @ 2020-02-13 20:55 UTC (permalink / raw)
To: Zheng Bin; +Cc: darrick.wong, sandeen, linux-xfs, renxudong1
On Thu, Feb 13, 2020 at 05:53:59PM +0800, Zheng Bin wrote:
> We recently used fuzz(hydra) to test XFS and automatically generate
> tmp.img(XFS v5 format, but some metadata is wrong)
>
> Test as follows:
> mount tmp.img tmpdir
> cp file1M tmpdir
> sync
>
> tmpdir/file1M size is 1M, but its data can not sync to disk.
>
> This is because tmp.img has some problems, using xfs_repair detect
> information as follows:
>
> agf_freeblks 0, counted 3224 in ag 0
> agf_longest 536874136, counted 3224 in ag 0
> sb_fdblocks 613, counted 3228
>
> Add these agf freeblocks checks:
> 1. agf_longest < agf_freeblks
> 2. agf_freeblks < sb_fdblocks
Did you audit the other fields in the AGF to see if they were
adequately bounds checked by xfs_agf_verify()?
A quick look at struct xfs_agf and xfs_agf_verify() indicates that
agf_length, agf_rmap_blocks and agf_refcount_blocks are not bounds
checked, either. And agf_spare64 and agf_spare2 are not checked for
being zero....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH] xfs: add agf freeblocks verify in xfs_agf_verify
2020-06-25 22:59 Request for inclusion on linux-4.14.y and linux-5.4.y yishache
@ 2020-06-25 22:59 ` yishache
0 siblings, 0 replies; 6+ messages in thread
From: yishache @ 2020-06-25 22:59 UTC (permalink / raw)
To: gregkh; +Cc: stable, Zheng Bin, Ren Xudong, Darrick J . Wong
From: Zheng Bin <zhengbin13@huawei.com>
We recently used fuzz(hydra) to test XFS and automatically generate
tmp.img(XFS v5 format, but some metadata is wrong)
xfs_repair information(just one AG):
agf_freeblks 0, counted 3224 in ag 0
agf_longest 536874136, counted 3224 in ag 0
sb_fdblocks 613, counted 3228
Test as follows:
mount tmp.img tmpdir
cp file1M tmpdir
sync
In 4.19-stable, sync will stuck, the reason is:
xfs_mountfs
xfs_check_summary_counts
if ((!xfs_sb_version_haslazysbcount(&mp->m_sb) ||
XFS_LAST_UNMOUNT_WAS_CLEAN(mp)) &&
!xfs_fs_has_sickness(mp, XFS_SICK_FS_COUNTERS))
return 0; -->just return, incore sb_fdblocks still be 613
xfs_initialize_perag_data
cp file1M tmpdir -->ok(write file to pagecache)
sync -->stuck(write pagecache to disk)
xfs_map_blocks
xfs_iomap_write_allocate
while (count_fsb != 0) {
nimaps = 0;
while (nimaps == 0) { --> endless loop
nimaps = 1;
xfs_bmapi_write(..., &nimaps) --> nimaps becomes 0 again
xfs_bmapi_write
xfs_bmap_alloc
xfs_bmap_btalloc
xfs_alloc_vextent
xfs_alloc_fix_freelist
xfs_alloc_space_available -->fail(agf_freeblks is 0)
In linux-next, sync not stuck, cause commit c2b3164320b5 ("xfs:
use the latest extent at writeback delalloc conversion time") remove
the above while, dmesg is as follows:
[ 55.250114] XFS (loop0): page discard on page ffffea0008bc7380, inode 0x1b0c, offset 0.
Users do not know why this page is discard, the better soultion is:
1. Like xfs_repair, make sure sb_fdblocks is equal to counted
(xfs_initialize_perag_data did this, who is not called at this mount)
2. Add agf verify, if fail, will tell users to repair
This patch use the second soultion.
Signed-off-by: Zheng Bin <zhengbin13@huawei.com>
Signed-off-by: Ren Xudong <renxudong1@huawei.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
(cherry picked from commit d0c7feaf87678371c2c09b3709400be416b2dc62)
---
fs/xfs/libxfs/xfs_alloc.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/fs/xfs/libxfs/xfs_alloc.c b/fs/xfs/libxfs/xfs_alloc.c
index 516e0c57cf9c..43c87cb5a747 100644
--- a/fs/xfs/libxfs/xfs_alloc.c
+++ b/fs/xfs/libxfs/xfs_alloc.c
@@ -2529,6 +2529,13 @@ xfs_agf_verify(
be32_to_cpu(agf->agf_flcount) <= xfs_agfl_size(mp)))
return false;
+ if (be32_to_cpu(agf->agf_length) > mp->m_sb.sb_dblocks)
+ return __this_address;
+
+ if (be32_to_cpu(agf->agf_freeblks) < be32_to_cpu(agf->agf_longest) ||
+ be32_to_cpu(agf->agf_freeblks) > be32_to_cpu(agf->agf_length))
+ return __this_address;
+
if (be32_to_cpu(agf->agf_levels[XFS_BTNUM_BNO]) < 1 ||
be32_to_cpu(agf->agf_levels[XFS_BTNUM_CNT]) < 1 ||
be32_to_cpu(agf->agf_levels[XFS_BTNUM_BNO]) > XFS_BTREE_MAXLEVELS ||
@@ -2540,6 +2547,10 @@ xfs_agf_verify(
be32_to_cpu(agf->agf_levels[XFS_BTNUM_RMAP]) > XFS_BTREE_MAXLEVELS))
return false;
+ if (xfs_sb_version_hasrmapbt(&mp->m_sb) &&
+ be32_to_cpu(agf->agf_rmap_blocks) > be32_to_cpu(agf->agf_length))
+ return __this_address;
+
/*
* during growfs operations, the perag is not fully initialised,
* so we can't use it for any useful checking. growfs ensures we can't
@@ -2553,6 +2564,11 @@ xfs_agf_verify(
be32_to_cpu(agf->agf_btreeblks) > be32_to_cpu(agf->agf_length))
return false;
+ if (xfs_sb_version_hasreflink(&mp->m_sb) &&
+ be32_to_cpu(agf->agf_refcount_blocks) >
+ be32_to_cpu(agf->agf_length))
+ return __this_address;
+
if (xfs_sb_version_hasreflink(&mp->m_sb) &&
(be32_to_cpu(agf->agf_refcount_level) < 1 ||
be32_to_cpu(agf->agf_refcount_level) > XFS_BTREE_MAXLEVELS))
--
2.16.6
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH] xfs: add agf freeblocks verify in xfs_agf_verify
2020-06-25 22:47 Request for inclusion on linux-4.14.y and linux-5.4.y yishache
@ 2020-06-25 22:47 ` yishache
0 siblings, 0 replies; 6+ messages in thread
From: yishache @ 2020-06-25 22:47 UTC (permalink / raw)
To: gregkh; +Cc: stable, Zheng Bin, Ren Xudong, Darrick J . Wong
From: Zheng Bin <zhengbin13@huawei.com>
We recently used fuzz(hydra) to test XFS and automatically generate
tmp.img(XFS v5 format, but some metadata is wrong)
xfs_repair information(just one AG):
agf_freeblks 0, counted 3224 in ag 0
agf_longest 536874136, counted 3224 in ag 0
sb_fdblocks 613, counted 3228
Test as follows:
mount tmp.img tmpdir
cp file1M tmpdir
sync
In 4.19-stable, sync will stuck, the reason is:
xfs_mountfs
xfs_check_summary_counts
if ((!xfs_sb_version_haslazysbcount(&mp->m_sb) ||
XFS_LAST_UNMOUNT_WAS_CLEAN(mp)) &&
!xfs_fs_has_sickness(mp, XFS_SICK_FS_COUNTERS))
return 0; -->just return, incore sb_fdblocks still be 613
xfs_initialize_perag_data
cp file1M tmpdir -->ok(write file to pagecache)
sync -->stuck(write pagecache to disk)
xfs_map_blocks
xfs_iomap_write_allocate
while (count_fsb != 0) {
nimaps = 0;
while (nimaps == 0) { --> endless loop
nimaps = 1;
xfs_bmapi_write(..., &nimaps) --> nimaps becomes 0 again
xfs_bmapi_write
xfs_bmap_alloc
xfs_bmap_btalloc
xfs_alloc_vextent
xfs_alloc_fix_freelist
xfs_alloc_space_available -->fail(agf_freeblks is 0)
In linux-next, sync not stuck, cause commit c2b3164320b5 ("xfs:
use the latest extent at writeback delalloc conversion time") remove
the above while, dmesg is as follows:
[ 55.250114] XFS (loop0): page discard on page ffffea0008bc7380, inode 0x1b0c, offset 0.
Users do not know why this page is discard, the better soultion is:
1. Like xfs_repair, make sure sb_fdblocks is equal to counted
(xfs_initialize_perag_data did this, who is not called at this mount)
2. Add agf verify, if fail, will tell users to repair
This patch use the second soultion.
Signed-off-by: Zheng Bin <zhengbin13@huawei.com>
Signed-off-by: Ren Xudong <renxudong1@huawei.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
(cherry picked from commit d0c7feaf87678371c2c09b3709400be416b2dc62)
---
fs/xfs/libxfs/xfs_alloc.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/fs/xfs/libxfs/xfs_alloc.c b/fs/xfs/libxfs/xfs_alloc.c
index 516e0c57cf9c..43c87cb5a747 100644
--- a/fs/xfs/libxfs/xfs_alloc.c
+++ b/fs/xfs/libxfs/xfs_alloc.c
@@ -2529,6 +2529,13 @@ xfs_agf_verify(
be32_to_cpu(agf->agf_flcount) <= xfs_agfl_size(mp)))
return false;
+ if (be32_to_cpu(agf->agf_length) > mp->m_sb.sb_dblocks)
+ return __this_address;
+
+ if (be32_to_cpu(agf->agf_freeblks) < be32_to_cpu(agf->agf_longest) ||
+ be32_to_cpu(agf->agf_freeblks) > be32_to_cpu(agf->agf_length))
+ return __this_address;
+
if (be32_to_cpu(agf->agf_levels[XFS_BTNUM_BNO]) < 1 ||
be32_to_cpu(agf->agf_levels[XFS_BTNUM_CNT]) < 1 ||
be32_to_cpu(agf->agf_levels[XFS_BTNUM_BNO]) > XFS_BTREE_MAXLEVELS ||
@@ -2540,6 +2547,10 @@ xfs_agf_verify(
be32_to_cpu(agf->agf_levels[XFS_BTNUM_RMAP]) > XFS_BTREE_MAXLEVELS))
return false;
+ if (xfs_sb_version_hasrmapbt(&mp->m_sb) &&
+ be32_to_cpu(agf->agf_rmap_blocks) > be32_to_cpu(agf->agf_length))
+ return __this_address;
+
/*
* during growfs operations, the perag is not fully initialised,
* so we can't use it for any useful checking. growfs ensures we can't
@@ -2553,6 +2564,11 @@ xfs_agf_verify(
be32_to_cpu(agf->agf_btreeblks) > be32_to_cpu(agf->agf_length))
return false;
+ if (xfs_sb_version_hasreflink(&mp->m_sb) &&
+ be32_to_cpu(agf->agf_refcount_blocks) >
+ be32_to_cpu(agf->agf_length))
+ return __this_address;
+
if (xfs_sb_version_hasreflink(&mp->m_sb) &&
(be32_to_cpu(agf->agf_refcount_level) < 1 ||
be32_to_cpu(agf->agf_refcount_level) > XFS_BTREE_MAXLEVELS))
--
2.16.6
^ permalink raw reply related [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-06-25 23:00 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-13 9:53 [PATCH] xfs: add agf freeblocks verify in xfs_agf_verify Zheng Bin
2020-02-13 13:37 ` Eric Sandeen
2020-02-13 17:44 ` Darrick J. Wong
2020-02-13 20:55 ` Dave Chinner
2020-06-25 22:47 Request for inclusion on linux-4.14.y and linux-5.4.y yishache
2020-06-25 22:47 ` [PATCH] xfs: add agf freeblocks verify in xfs_agf_verify yishache
2020-06-25 22:59 Request for inclusion on linux-4.14.y and linux-5.4.y yishache
2020-06-25 22:59 ` [PATCH] xfs: add agf freeblocks verify in xfs_agf_verify yishache
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.