* [PATCH 01/47] tests: fix test scripts that don't work on non-Linux systems.
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
@ 2014-11-07 21:50 ` Darrick J. Wong
2014-11-08 1:55 ` Theodore Ts'o
2014-11-07 21:50 ` [PATCH 02/47] misc: fix compiler warnings and minor build errors Darrick J. Wong
` (47 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:50 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Fix various tests that break on non-Linux systems; in particular,
sed -i doesn't work the same on all platforms; and try to keep the
GNU getopt-isms out.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
tests/d_xattr_edits/expect | 4 +--
tests/d_xattr_edits/script | 32 ++++++++++----------
tests/f_detect_junk/script | 3 +-
tests/f_detect_xfs/script | 3 +-
tests/f_itable_collision/script | 5 +--
tests/j_corrupt_commit_csum/expect | 2 +
tests/j_corrupt_commit_csum/script | 1 +
tests/j_corrupt_commit_tid/expect | 2 +
tests/j_corrupt_commit_tid/script | 1 +
tests/j_corrupt_descr_csum/expect | 2 +
tests/j_corrupt_descr_csum/script | 1 +
tests/j_corrupt_descr_tid/expect | 2 +
tests/j_corrupt_descr_tid/script | 1 +
tests/j_corrupt_journal_block/expect | 3 +-
tests/j_corrupt_journal_block/script | 1 +
tests/j_corrupt_revoke_block/expect | 2 +
tests/j_corrupt_revoke_block/script | 1 +
tests/j_corrupt_revoke_csum/expect | 2 +
tests/j_corrupt_revoke_csum/script | 1 +
tests/j_corrupt_sb_csum/expect | 2 +
tests/j_corrupt_sb_csum/script | 1 +
tests/j_corrupt_sb_magic/expect | 2 +
tests/j_corrupt_sb_magic/script | 1 +
tests/j_ext_long_revoke_trans/script | 6 ++--
tests/j_ext_long_trans/script | 4 +--
tests/j_long_revoke_trans/script | 4 +--
tests/j_long_revoke_trans_mcsum_32bit/script | 4 +--
tests/j_long_revoke_trans_mcsum_64bit/script | 4 +--
tests/j_long_trans/script | 4 +--
tests/j_long_trans_mcsum_32bit/script | 4 +--
tests/j_long_trans_mcsum_64bit/script | 4 +--
tests/j_short_revoke_trans/script | 4 +--
tests/j_short_revoke_trans_mcsum_64bit/script | 4 +--
tests/j_short_trans/script | 4 +--
tests/j_short_trans_64bit/script | 4 +--
tests/j_short_trans_mcsum_64bit/script | 4 +--
tests/j_short_trans_old_csum/script | 4 +--
tests/j_short_trans_open_recover/script | 6 ++--
tests/j_short_trans_recover/script | 6 ++--
tests/j_short_trans_recover_mcsum_64bit/script | 6 ++--
tests/j_short_uncommitted_trans/script | 4 +--
tests/j_short_uncommitted_trans_mcsum_64bit/script | 4 +--
tests/m_mmp_bad_csum/script | 2 +
tests/m_mmp_bad_magic/script | 2 +
tests/t_mke2fs_errors/script | 8 +++--
45 files changed, 91 insertions(+), 80 deletions(-)
diff --git a/tests/d_xattr_edits/expect b/tests/d_xattr_edits/expect
index 44ce5e4..4db5414 100644
--- a/tests/d_xattr_edits/expect
+++ b/tests/d_xattr_edits/expect
@@ -31,13 +31,13 @@ ea_rm / user.joe
Exit status is 0
ea_list /
Exit status is 0
-ea_set / user.file_based_xattr -f d_xattr_edits.tmp
+ea_set -f d_xattr_edits.tmp / user.file_based_xattr
Exit status is 0
ea_list /
Extended attributes:
user.file_based_xattr = "12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567\012" (108)
Exit status is 0
-ea_get / user.file_based_xattr -f d_xattr_edits.ver.tmp
+ea_get -f d_xattr_edits.ver.tmp / user.file_based_xattr
Exit status is 0
Compare big attribute
e2fsck -yf -N test_filesys
diff --git a/tests/d_xattr_edits/script b/tests/d_xattr_edits/script
index 1e33716..2e356e8 100644
--- a/tests/d_xattr_edits/script
+++ b/tests/d_xattr_edits/script
@@ -18,86 +18,86 @@ status=$?
echo Exit status is $status >> $OUT
echo "ea_set / user.joe smith" > $OUT.new
-$DEBUGFS -w $TMPFILE -R "ea_set / user.joe smith" >> $OUT.new 2>&1
+$DEBUGFS -w -R "ea_set / user.joe smith" $TMPFILE >> $OUT.new 2>&1
status=$?
echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed $OUT.new >> $OUT
echo "ea_set / user.moo FEE_FIE_FOE_FUMMMMMM" > $OUT.new
-$DEBUGFS -w $TMPFILE -R "ea_set / user.moo FEE_FIE_FOE_FUMMMMMM" >> $OUT.new 2>&1
+$DEBUGFS -w -R "ea_set / user.moo FEE_FIE_FOE_FUMMMMMM" $TMPFILE >> $OUT.new 2>&1
status=$?
echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed $OUT.new >> $OUT
echo "ea_list /" > $OUT.new
-$DEBUGFS -w $TMPFILE -R "ea_list /" >> $OUT.new 2>&1
+$DEBUGFS -w -R "ea_list /" $TMPFILE >> $OUT.new 2>&1
status=$?
echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed $OUT.new >> $OUT
echo "ea_get / user.moo" > $OUT.new
-$DEBUGFS -w $TMPFILE -R "ea_get / user.moo" >> $OUT.new 2>&1
+$DEBUGFS -w -R "ea_get / user.moo" $TMPFILE >> $OUT.new 2>&1
status=$?
echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed $OUT.new >> $OUT
echo "ea_get / nosuchea" > $OUT.new
-$DEBUGFS -w $TMPFILE -R "ea_get / nosuchea" >> $OUT.new 2>&1
+$DEBUGFS -w -R "ea_get / nosuchea" $TMPFILE >> $OUT.new 2>&1
status=$?
echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed $OUT.new >> $OUT
echo "ea_rm / user.moo" > $OUT.new
-$DEBUGFS -w $TMPFILE -R "ea_rm / user.moo" >> $OUT.new 2>&1
+$DEBUGFS -w -R "ea_rm / user.moo" $TMPFILE >> $OUT.new 2>&1
status=$?
echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed $OUT.new >> $OUT
echo "ea_rm / nosuchea" > $OUT.new
-$DEBUGFS -w $TMPFILE -R "ea_rm / nosuchea" >> $OUT.new 2>&1
+$DEBUGFS -w -R "ea_rm / nosuchea" $TMPFILE >> $OUT.new 2>&1
status=$?
echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed $OUT.new >> $OUT
echo "ea_list /" > $OUT.new
-$DEBUGFS -w $TMPFILE -R "ea_list /" >> $OUT.new 2>&1
+$DEBUGFS -w -R "ea_list /" $TMPFILE >> $OUT.new 2>&1
status=$?
echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed $OUT.new >> $OUT
echo "ea_get / user.moo" > $OUT.new
-$DEBUGFS -w $TMPFILE -R "ea_get / user.moo" >> $OUT.new 2>&1
+$DEBUGFS -w -R "ea_get / user.moo" $TMPFILE >> $OUT.new 2>&1
status=$?
echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed $OUT.new >> $OUT
echo "ea_rm / user.joe" > $OUT.new
-$DEBUGFS -w $TMPFILE -R "ea_rm / user.joe" >> $OUT.new 2>&1
+$DEBUGFS -w -R "ea_rm / user.joe" $TMPFILE >> $OUT.new 2>&1
status=$?
echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed $OUT.new >> $OUT
echo "ea_list /" > $OUT.new
-$DEBUGFS -w $TMPFILE -R "ea_list /" >> $OUT.new 2>&1
+$DEBUGFS -w -R "ea_list /" $TMPFILE >> $OUT.new 2>&1
status=$?
echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed $OUT.new >> $OUT
echo "12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567" > $TEST_DATA
-echo "ea_set / user.file_based_xattr -f $TEST_DATA" > $OUT.new
-$DEBUGFS -w $TMPFILE -R "ea_set / user.file_based_xattr -f $TEST_DATA" >> $OUT.new 2>&1
+echo "ea_set -f $TEST_DATA / user.file_based_xattr" > $OUT.new
+$DEBUGFS -w -R "ea_set -f $TEST_DATA / user.file_based_xattr" $TMPFILE >> $OUT.new 2>&1
status=$?
echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed $OUT.new >> $OUT
echo "ea_list /" > $OUT.new
-$DEBUGFS -w $TMPFILE -R "ea_list /" >> $OUT.new 2>&1
+$DEBUGFS -w -R "ea_list /" $TMPFILE >> $OUT.new 2>&1
status=$?
echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed $OUT.new >> $OUT
-echo "ea_get / user.file_based_xattr -f $VERIFY_DATA" > $OUT.new
-$DEBUGFS -w $TMPFILE -R "ea_get / user.file_based_xattr -f $VERIFY_DATA" >> $OUT.new 2>&1
+echo "ea_get -f $VERIFY_DATA / user.file_based_xattr" > $OUT.new
+$DEBUGFS -w -R "ea_get -f $VERIFY_DATA / user.file_based_xattr" $TMPFILE >> $OUT.new 2>&1
status=$?
echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed $OUT.new >> $OUT
diff --git a/tests/f_detect_junk/script b/tests/f_detect_junk/script
index 3aebcd0..2721c79 100644
--- a/tests/f_detect_junk/script
+++ b/tests/f_detect_junk/script
@@ -26,7 +26,8 @@ $TUNE2FS -i 0 $TMPFILE >> $OUT 2>&1
echo "*** mke2fs" >> $OUT
$MKE2FS -n $TMPFILE >> $OUT 2>&1
-sed -f $cmd_dir/filter.sed -e "s|$TMPFILE|test.img|g" -i $OUT
+sed -f $cmd_dir/filter.sed -e "s|$TMPFILE|test.img|g" < $OUT > $OUT.new
+mv $OUT.new $OUT
# Figure out what happened
if cmp -s $EXP $OUT; then
diff --git a/tests/f_detect_xfs/script b/tests/f_detect_xfs/script
index 2531c5e..b32ba85 100644
--- a/tests/f_detect_xfs/script
+++ b/tests/f_detect_xfs/script
@@ -23,7 +23,8 @@ $TUNE2FS -i 0 $TMPFILE >> $OUT 2>&1
echo "*** mke2fs" >> $OUT
$MKE2FS -n $TMPFILE >> $OUT 2>&1
-sed -f $cmd_dir/filter.sed -e "s|$TMPFILE|test.img|g" -i $OUT
+sed -f $cmd_dir/filter.sed -e "s|$TMPFILE|test.img|g" < $OUT > $OUT.new
+mv $OUT.new $OUT
# Figure out what happened
if cmp -s $EXP $OUT; then
diff --git a/tests/f_itable_collision/script b/tests/f_itable_collision/script
index 7eedfd3..52b69a2 100755
--- a/tests/f_itable_collision/script
+++ b/tests/f_itable_collision/script
@@ -10,21 +10,20 @@ E2FSCK_TIME=4294967294
export E2FSCK_TIME
gzip -d < $IMAGE > $TMPFILE
-e2label $TMPFILE test_filesys
# Run fsck to fix things?
EXP1=$test_dir/expect.1
OUT1=$test_name.1.log
rm -rf $test_name.failed $test_name.ok
-$FSCK $FSCK_OPT $TMPFILE 2>&1 | tail -n +2 > $OUT1
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE 2>&1 | tail -n +2 > $OUT1
echo "Exit status is $?" >> $OUT1
# Run a second time
EXP2=$test_dir/expect.2
OUT2=$test_name.2.log
-$FSCK $FSCK_OPT $TMPFILE 2>&1 | tail -n +2 > $OUT2
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE 2>&1 | tail -n +2 > $OUT2
echo "Exit status is $?" >> $OUT2
# Figure out what happened
diff --git a/tests/j_corrupt_commit_csum/expect b/tests/j_corrupt_commit_csum/expect
index 5fc82b8..d799772 100644
--- a/tests/j_corrupt_commit_csum/expect
+++ b/tests/j_corrupt_commit_csum/expect
@@ -15,4 +15,4 @@ Pass 5: Checking group summary information
test_filesys: 12/128 files (8.3% non-contiguous), 1092/2048 blocks
Exit status is 0
debugfs: cat /a
-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaa
\ No newline at end of file
+aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaadebugfs:
diff --git a/tests/j_corrupt_commit_csum/script b/tests/j_corrupt_commit_csum/script
index 102feba..9c16975 100644
--- a/tests/j_corrupt_commit_csum/script
+++ b/tests/j_corrupt_commit_csum/script
@@ -27,6 +27,7 @@ sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
rm -f $OUT.new
echo "cat /a" > $TMPFILE.cmd
+echo >> $TMPFILE.cmd
$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -f $TMPFILE.cmd
diff --git a/tests/j_corrupt_commit_tid/expect b/tests/j_corrupt_commit_tid/expect
index dfbb3f5..0a7df89 100644
--- a/tests/j_corrupt_commit_tid/expect
+++ b/tests/j_corrupt_commit_tid/expect
@@ -14,4 +14,4 @@ Pass 5: Checking group summary information
test_filesys: 12/128 files (8.3% non-contiguous), 1092/2048 blocks
Exit status is 0
debugfs: cat /a
-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaa
\ No newline at end of file
+aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaadebugfs:
diff --git a/tests/j_corrupt_commit_tid/script b/tests/j_corrupt_commit_tid/script
index 102feba..9c16975 100644
--- a/tests/j_corrupt_commit_tid/script
+++ b/tests/j_corrupt_commit_tid/script
@@ -27,6 +27,7 @@ sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
rm -f $OUT.new
echo "cat /a" > $TMPFILE.cmd
+echo >> $TMPFILE.cmd
$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -f $TMPFILE.cmd
diff --git a/tests/j_corrupt_descr_csum/expect b/tests/j_corrupt_descr_csum/expect
index 419e775..4e5ec6a 100644
--- a/tests/j_corrupt_descr_csum/expect
+++ b/tests/j_corrupt_descr_csum/expect
@@ -15,4 +15,4 @@ Pass 5: Checking group summary information
test_filesys: 12/128 files (8.3% non-contiguous), 1093/2048 blocks
Exit status is 0
debugfs: cat /a
-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
\ No newline at end of file
+aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaadebugfs:
diff --git a/tests/j_corrupt_descr_csum/script b/tests/j_corrupt_descr_csum/script
index 102feba..9c16975 100644
--- a/tests/j_corrupt_descr_csum/script
+++ b/tests/j_corrupt_descr_csum/script
@@ -27,6 +27,7 @@ sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
rm -f $OUT.new
echo "cat /a" > $TMPFILE.cmd
+echo >> $TMPFILE.cmd
$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -f $TMPFILE.cmd
diff --git a/tests/j_corrupt_descr_tid/expect b/tests/j_corrupt_descr_tid/expect
index 0444203..00bd2d5 100644
--- a/tests/j_corrupt_descr_tid/expect
+++ b/tests/j_corrupt_descr_tid/expect
@@ -14,4 +14,4 @@ Pass 5: Checking group summary information
test_filesys: 12/128 files (8.3% non-contiguous), 1093/2048 blocks
Exit status is 0
debugfs: cat /a
-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
\ No newline at end of file
+aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaadebugfs:
diff --git a/tests/j_corrupt_descr_tid/script b/tests/j_corrupt_descr_tid/script
index 102feba..9c16975 100644
--- a/tests/j_corrupt_descr_tid/script
+++ b/tests/j_corrupt_descr_tid/script
@@ -27,6 +27,7 @@ sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
rm -f $OUT.new
echo "cat /a" > $TMPFILE.cmd
+echo >> $TMPFILE.cmd
$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -f $TMPFILE.cmd
diff --git a/tests/j_corrupt_journal_block/expect b/tests/j_corrupt_journal_block/expect
index 443a2b3..bc75707 100644
--- a/tests/j_corrupt_journal_block/expect
+++ b/tests/j_corrupt_journal_block/expect
@@ -16,4 +16,5 @@ Pass 5: Checking group summary information
test_filesys: 12/128 files (8.3% non-contiguous), 1093/2048 blocks
Exit status is 0
debugfs: cat /a
-bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb!
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc
ccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc!
cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc!
ccccccccccddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd
ddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd!
dddddddddddddddddddddddddddddddddddddddddddddd
+bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb!
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc
ccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc!
cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc!
ccccccccccddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd
ddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd!
dddddddddddddddddddddddddddddddddddddddddddddddebugfs:
+
diff --git a/tests/j_corrupt_journal_block/script b/tests/j_corrupt_journal_block/script
index c17c70e..2bce973 100644
--- a/tests/j_corrupt_journal_block/script
+++ b/tests/j_corrupt_journal_block/script
@@ -27,6 +27,7 @@ sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
rm -f $OUT.new
echo "cat /a" > $TMPFILE.cmd
+echo >> $TMPFILE.cmd
$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
echo >> $OUT
rm -f $TMPFILE.cmd
diff --git a/tests/j_corrupt_revoke_block/expect b/tests/j_corrupt_revoke_block/expect
index 52cda2c..c357f53 100644
--- a/tests/j_corrupt_revoke_block/expect
+++ b/tests/j_corrupt_revoke_block/expect
@@ -14,4 +14,4 @@ Pass 5: Checking group summary information
test_filesys: 12/128 files (8.3% non-contiguous), 1092/2048 blocks
Exit status is 0
debugfs: cat /a
-bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb!
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb!
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb!
bbbbbbbbbb
\ No newline at end of file
+bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb!
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb!
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb!
bbbbbbbbbbdebugfs:
diff --git a/tests/j_corrupt_revoke_block/script b/tests/j_corrupt_revoke_block/script
index 102feba..9c16975 100644
--- a/tests/j_corrupt_revoke_block/script
+++ b/tests/j_corrupt_revoke_block/script
@@ -27,6 +27,7 @@ sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
rm -f $OUT.new
echo "cat /a" > $TMPFILE.cmd
+echo >> $TMPFILE.cmd
$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -f $TMPFILE.cmd
diff --git a/tests/j_corrupt_revoke_csum/expect b/tests/j_corrupt_revoke_csum/expect
index 52cda2c..c357f53 100644
--- a/tests/j_corrupt_revoke_csum/expect
+++ b/tests/j_corrupt_revoke_csum/expect
@@ -14,4 +14,4 @@ Pass 5: Checking group summary information
test_filesys: 12/128 files (8.3% non-contiguous), 1092/2048 blocks
Exit status is 0
debugfs: cat /a
-bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb!
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb!
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb!
bbbbbbbbbb
\ No newline at end of file
+bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb!
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb!
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb!
bbbbbbbbbbdebugfs:
diff --git a/tests/j_corrupt_revoke_csum/script b/tests/j_corrupt_revoke_csum/script
index 102feba..9c16975 100644
--- a/tests/j_corrupt_revoke_csum/script
+++ b/tests/j_corrupt_revoke_csum/script
@@ -27,6 +27,7 @@ sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
rm -f $OUT.new
echo "cat /a" > $TMPFILE.cmd
+echo >> $TMPFILE.cmd
$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -f $TMPFILE.cmd
diff --git a/tests/j_corrupt_sb_csum/expect b/tests/j_corrupt_sb_csum/expect
index 0493500..5c88a09 100644
--- a/tests/j_corrupt_sb_csum/expect
+++ b/tests/j_corrupt_sb_csum/expect
@@ -18,4 +18,4 @@ Pass 5: Checking group summary information
test_filesys: 12/128 files (8.3% non-contiguous), 1092/2048 blocks
Exit status is 0
debugfs: cat /a
-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaa
\ No newline at end of file
+aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaadebugfs:
diff --git a/tests/j_corrupt_sb_csum/script b/tests/j_corrupt_sb_csum/script
index 102feba..9c16975 100644
--- a/tests/j_corrupt_sb_csum/script
+++ b/tests/j_corrupt_sb_csum/script
@@ -27,6 +27,7 @@ sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
rm -f $OUT.new
echo "cat /a" > $TMPFILE.cmd
+echo >> $TMPFILE.cmd
$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -f $TMPFILE.cmd
diff --git a/tests/j_corrupt_sb_magic/expect b/tests/j_corrupt_sb_magic/expect
index a233033..2169a15 100644
--- a/tests/j_corrupt_sb_magic/expect
+++ b/tests/j_corrupt_sb_magic/expect
@@ -39,4 +39,4 @@ Pass 5: Checking group summary information
test_filesys: 12/128 files (8.3% non-contiguous), 1092/2048 blocks
Exit status is 0
debugfs: cat /a
-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaa
\ No newline at end of file
+aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa!
aaaaaaaaaadebugfs:
diff --git a/tests/j_corrupt_sb_magic/script b/tests/j_corrupt_sb_magic/script
index 102feba..9c16975 100644
--- a/tests/j_corrupt_sb_magic/script
+++ b/tests/j_corrupt_sb_magic/script
@@ -27,6 +27,7 @@ sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
rm -f $OUT.new
echo "cat /a" > $TMPFILE.cmd
+echo >> $TMPFILE.cmd
$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -f $TMPFILE.cmd
diff --git a/tests/j_ext_long_revoke_trans/script b/tests/j_ext_long_revoke_trans/script
index 4a60850..fe45c72 100644
--- a/tests/j_ext_long_revoke_trans/script
+++ b/tests/j_ext_long_revoke_trans/script
@@ -18,7 +18,7 @@ $MKE2FS -F -o Linux -b 1024 -O journal_dev -T ext4 -U 1db3f677-6832-4adb-bafc-8e
$MKE2FS -F -o Linux -b 1024 -O ^has_journal -T ext4 $TMPFILE 262144 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT 2>&1
echo "debugfs add journal device/UUID" >> $OUT
-$DEBUGFS -w -f - $TMPFILE <<- EOF >> $OUT.new 2>&1
+$DEBUGFS -w -f - $TMPFILE <<-EOF >> $OUT.new 2>&1
feature has_journal
ssv journal_dev 0x9999
ssv journal_uuid 1db3f677-6832-4adb-bafc-8e4059c30a34
@@ -39,14 +39,14 @@ echo "jc" >> $TMPFILE.cmd
echo "jo -f $JNLFILE" >> $TMPFILE.cmd
echo "jw -r 259-4356 /dev/zero" >> $TMPFILE.cmd
echo "jc" >> $TMPFILE.cmd
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd 2>> $OUT.new > /dev/null
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE 2>> $OUT.new > /dev/null
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
test -d "$JOURNAL_DUMP_DIR" -a -w "$JOURNAL_DUMP_DIR" && cp "$TMPFILE" "$JOURNAL_DUMP_DIR/$test_name.img"
test -d "$JOURNAL_DUMP_DIR" -a -w "$JOURNAL_DUMP_DIR" && cp "$JNLFILE" "$JOURNAL_DUMP_DIR/$test_name.img.jnl"
echo "logdump -c -f $JNLFILE" > $TMPFILE.cmd
-$DEBUGFS_EXE $TMPFILE -f $TMPFILE.cmd 2>&1 | sed -f $cmd_dir/filter.sed -e 's/logdump -c -f.*/logdump -c/g' >> $OUT
+$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed -e 's/logdump -c -f.*/logdump -c/g' >> $OUT
rm -rf $TMPFILE.cmd
echo "debugfs fsck" >> $OUT
diff --git a/tests/j_ext_long_trans/script b/tests/j_ext_long_trans/script
index c02f791..f71e5a4 100644
--- a/tests/j_ext_long_trans/script
+++ b/tests/j_ext_long_trans/script
@@ -36,14 +36,14 @@ echo "debugfs write journal" >> $OUT
echo "jo -f $JNLFILE" > $TMPFILE.cmd
echo "jw -b 259-4356 /dev/zero" >> $TMPFILE.cmd
echo "jc" >> $TMPFILE.cmd
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd 2>> $OUT.new > /dev/null
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE 2>> $OUT.new > /dev/null
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
test -d "$JOURNAL_DUMP_DIR" -a -w "$JOURNAL_DUMP_DIR" && cp "$TMPFILE" "$JOURNAL_DUMP_DIR/$test_name.img"
test -d "$JOURNAL_DUMP_DIR" -a -w "$JOURNAL_DUMP_DIR" && cp "$JNLFILE" "$JOURNAL_DUMP_DIR/$test_name.img.jnl"
echo "logdump -c -f $JNLFILE" > $TMPFILE.cmd
-$DEBUGFS_EXE $TMPFILE -f $TMPFILE.cmd 2>&1 | sed -f $cmd_dir/filter.sed -e 's/logdump -c -f.*/logdump -c/g' >> $OUT
+$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed -e 's/logdump -c -f.*/logdump -c/g' >> $OUT
rm -rf $TMPFILE.cmd
echo "debugfs fsck" >> $OUT
diff --git a/tests/j_long_revoke_trans/script b/tests/j_long_revoke_trans/script
index 4289c2e..4a0c5d1 100644
--- a/tests/j_long_revoke_trans/script
+++ b/tests/j_long_revoke_trans/script
@@ -26,13 +26,13 @@ echo "jc" >> $TMPFILE.cmd
echo "jo" >> $TMPFILE.cmd
echo "jw -r 259-4356" >> $TMPFILE.cmd
echo "jc" >> $TMPFILE.cmd
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd 2>> $OUT.new > /dev/null
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE 2>> $OUT.new > /dev/null
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
test -d "$JOURNAL_DUMP_DIR" -a -w "$JOURNAL_DUMP_DIR" && cp "$TMPFILE" "$JOURNAL_DUMP_DIR/$test_name.img"
echo "logdump -c" > $TMPFILE.cmd
-$DEBUGFS_EXE $TMPFILE -f $TMPFILE.cmd 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -rf $TMPFILE.cmd
$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
diff --git a/tests/j_long_revoke_trans_mcsum_32bit/script b/tests/j_long_revoke_trans_mcsum_32bit/script
index 4354fe5..fc0d0ad 100644
--- a/tests/j_long_revoke_trans_mcsum_32bit/script
+++ b/tests/j_long_revoke_trans_mcsum_32bit/script
@@ -29,7 +29,7 @@ echo "jo" >> $TMPFILE.cmd
echo "jw -r 260-4356" >> $TMPFILE.cmd
echo "jc" >> $TMPFILE.cmd
#$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd >> $OUT 2>&1
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd 2>> $OUT.new > /dev/null
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE 2>> $OUT.new > /dev/null
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
@@ -37,7 +37,7 @@ $DUMPE2FS $TMPFILE 2>&1 | grep '^Journal features:' >> $OUT
test -d "$JOURNAL_DUMP_DIR" -a -w "$JOURNAL_DUMP_DIR" && cp "$TMPFILE" "$JOURNAL_DUMP_DIR/$test_name.img"
echo "logdump -c" > $TMPFILE.cmd
-$DEBUGFS_EXE $TMPFILE -f $TMPFILE.cmd 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -rf $TMPFILE.cmd
$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
diff --git a/tests/j_long_revoke_trans_mcsum_64bit/script b/tests/j_long_revoke_trans_mcsum_64bit/script
index 8133a66..e206f88 100644
--- a/tests/j_long_revoke_trans_mcsum_64bit/script
+++ b/tests/j_long_revoke_trans_mcsum_64bit/script
@@ -29,7 +29,7 @@ echo "jo" >> $TMPFILE.cmd
echo "jw -r 262-4358" >> $TMPFILE.cmd
echo "jc" >> $TMPFILE.cmd
#$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd >> $OUT 2>&1
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd 2>> $OUT.new > /dev/null
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE 2>> $OUT.new > /dev/null
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
@@ -37,7 +37,7 @@ $DUMPE2FS $TMPFILE 2>&1 | grep '^Journal features:' >> $OUT
test -d "$JOURNAL_DUMP_DIR" -a -w "$JOURNAL_DUMP_DIR" && cp "$TMPFILE" "$JOURNAL_DUMP_DIR/$test_name.img"
echo "logdump -c" > $TMPFILE.cmd
-$DEBUGFS_EXE $TMPFILE -f $TMPFILE.cmd 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -rf $TMPFILE.cmd
$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
diff --git a/tests/j_long_trans/script b/tests/j_long_trans/script
index 9a4f131..68172c4 100644
--- a/tests/j_long_trans/script
+++ b/tests/j_long_trans/script
@@ -23,13 +23,13 @@ echo "debugfs write journal" >> $OUT
echo "jo" > $TMPFILE.cmd
echo "jw -b 259-4356 /dev/zero" >> $TMPFILE.cmd
echo "jc" >> $TMPFILE.cmd
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd 2>> $OUT.new > /dev/null
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE 2>> $OUT.new > /dev/null
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
test -d "$JOURNAL_DUMP_DIR" -a -w "$JOURNAL_DUMP_DIR" && cp "$TMPFILE" "$JOURNAL_DUMP_DIR/$test_name.img"
echo "logdump -c" > $TMPFILE.cmd
-$DEBUGFS_EXE $TMPFILE -f $TMPFILE.cmd 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -rf $TMPFILE.cmd
$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
diff --git a/tests/j_long_trans_mcsum_32bit/script b/tests/j_long_trans_mcsum_32bit/script
index 3255001..4722242 100644
--- a/tests/j_long_trans_mcsum_32bit/script
+++ b/tests/j_long_trans_mcsum_32bit/script
@@ -25,7 +25,7 @@ echo "debugfs write journal" >> $OUT
echo "jo -c" > $TMPFILE.cmd
echo "jw -b 260-4356 /dev/zero" >> $TMPFILE.cmd
echo "jc" >> $TMPFILE.cmd
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd 2>> $OUT.new > /dev/null
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE 2>> $OUT.new > /dev/null
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
@@ -33,7 +33,7 @@ $DUMPE2FS $TMPFILE 2>&1 | grep '^Journal features:' >> $OUT
test -d "$JOURNAL_DUMP_DIR" -a -w "$JOURNAL_DUMP_DIR" && cp "$TMPFILE" "$JOURNAL_DUMP_DIR/$test_name.img"
echo "logdump -c" > $TMPFILE.cmd
-$DEBUGFS_EXE $TMPFILE -f $TMPFILE.cmd 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -rf $TMPFILE.cmd
$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
diff --git a/tests/j_long_trans_mcsum_64bit/script b/tests/j_long_trans_mcsum_64bit/script
index 8ded550..65ca1b7 100644
--- a/tests/j_long_trans_mcsum_64bit/script
+++ b/tests/j_long_trans_mcsum_64bit/script
@@ -25,7 +25,7 @@ echo "debugfs write journal" >> $OUT
echo "jo -c" > $TMPFILE.cmd
echo "jw -b 262-4358 /dev/zero" >> $TMPFILE.cmd
echo "jc" >> $TMPFILE.cmd
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd 2>> $OUT.new > /dev/null
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE 2>> $OUT.new > /dev/null
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
@@ -33,7 +33,7 @@ $DUMPE2FS $TMPFILE 2>&1 | grep '^Journal features:' >> $OUT
test -d "$JOURNAL_DUMP_DIR" -a -w "$JOURNAL_DUMP_DIR" && cp "$TMPFILE" "$JOURNAL_DUMP_DIR/$test_name.img"
echo "logdump -c" > $TMPFILE.cmd
-$DEBUGFS_EXE $TMPFILE -f $TMPFILE.cmd 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -rf $TMPFILE.cmd
$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
diff --git a/tests/j_short_revoke_trans/script b/tests/j_short_revoke_trans/script
index edf28f9..94f20b9 100644
--- a/tests/j_short_revoke_trans/script
+++ b/tests/j_short_revoke_trans/script
@@ -28,13 +28,13 @@ echo "jc" >> $TMPFILE.cmd
echo "jo" >> $TMPFILE.cmd
echo "jw -r $bitmaps" >> $TMPFILE.cmd
echo "jc" >> $TMPFILE.cmd
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd 2>> $OUT.new > /dev/null
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE 2>> $OUT.new > /dev/null
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
test -d "$JOURNAL_DUMP_DIR" -a -w "$JOURNAL_DUMP_DIR" && cp "$TMPFILE" "$JOURNAL_DUMP_DIR/$test_name.img"
echo "logdump -c" > $TMPFILE.cmd
-$DEBUGFS_EXE $TMPFILE -f $TMPFILE.cmd 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -rf $TMPFILE.cmd
$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
diff --git a/tests/j_short_revoke_trans_mcsum_64bit/script b/tests/j_short_revoke_trans_mcsum_64bit/script
index 8bd6e08..8358012 100644
--- a/tests/j_short_revoke_trans_mcsum_64bit/script
+++ b/tests/j_short_revoke_trans_mcsum_64bit/script
@@ -30,7 +30,7 @@ echo "jc" >> $TMPFILE.cmd
echo "jo" >> $TMPFILE.cmd
echo "jw -r $bitmaps" >> $TMPFILE.cmd
echo "jc" >> $TMPFILE.cmd
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd 2>> $OUT.new > /dev/null
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE 2>> $OUT.new > /dev/null
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
@@ -38,7 +38,7 @@ $DUMPE2FS $TMPFILE 2>&1 | grep '^Journal features:' >> $OUT
test -d "$JOURNAL_DUMP_DIR" -a -w "$JOURNAL_DUMP_DIR" && cp "$TMPFILE" "$JOURNAL_DUMP_DIR/$test_name.img"
echo "logdump -c" > $TMPFILE.cmd
-$DEBUGFS_EXE $TMPFILE -f $TMPFILE.cmd 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -rf $TMPFILE.cmd
$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
diff --git a/tests/j_short_trans/script b/tests/j_short_trans/script
index 00f4d6e..c83daba 100644
--- a/tests/j_short_trans/script
+++ b/tests/j_short_trans/script
@@ -25,13 +25,13 @@ echo "debugfs write journal" >> $OUT
echo "jo" > $TMPFILE.cmd
echo "jw -b $bitmaps /dev/zero" >> $TMPFILE.cmd
echo "jc" >> $TMPFILE.cmd
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd 2>> $OUT.new > /dev/null
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE 2>> $OUT.new > /dev/null
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
test -d "$JOURNAL_DUMP_DIR" -a -w "$JOURNAL_DUMP_DIR" && cp "$TMPFILE" "$JOURNAL_DUMP_DIR/$test_name.img"
echo "logdump -c" > $TMPFILE.cmd
-$DEBUGFS_EXE $TMPFILE -f $TMPFILE.cmd 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -rf $TMPFILE.cmd
$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
diff --git a/tests/j_short_trans_64bit/script b/tests/j_short_trans_64bit/script
index daf0866..7a60284 100644
--- a/tests/j_short_trans_64bit/script
+++ b/tests/j_short_trans_64bit/script
@@ -27,7 +27,7 @@ echo "debugfs write journal" >> $OUT
echo "jo" > $TMPFILE.cmd
echo "jw -b $bitmaps /dev/zero" >> $TMPFILE.cmd
echo "jc" >> $TMPFILE.cmd
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd 2>> $OUT.new > /dev/null
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE 2>> $OUT.new > /dev/null
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
@@ -35,7 +35,7 @@ $DUMPE2FS $TMPFILE 2>&1 | grep '^Journal features:' >> $OUT
test -d "$JOURNAL_DUMP_DIR" -a -w "$JOURNAL_DUMP_DIR" && cp "$TMPFILE" "$JOURNAL_DUMP_DIR/$test_name.img"
echo "logdump -c" > $TMPFILE.cmd
-$DEBUGFS_EXE $TMPFILE -f $TMPFILE.cmd 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -rf $TMPFILE.cmd
$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
diff --git a/tests/j_short_trans_mcsum_64bit/script b/tests/j_short_trans_mcsum_64bit/script
index 0534941..bc9f09b 100644
--- a/tests/j_short_trans_mcsum_64bit/script
+++ b/tests/j_short_trans_mcsum_64bit/script
@@ -27,7 +27,7 @@ echo "debugfs write journal" >> $OUT
echo "jo -c" > $TMPFILE.cmd
echo "jw -b $bitmaps /dev/zero" >> $TMPFILE.cmd
echo "jc" >> $TMPFILE.cmd
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd 2>> $OUT.new > /dev/null
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE 2>> $OUT.new > /dev/null
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
@@ -35,7 +35,7 @@ $DUMPE2FS $TMPFILE 2>&1 | grep '^Journal features:' >> $OUT
test -d "$JOURNAL_DUMP_DIR" -a -w "$JOURNAL_DUMP_DIR" && cp "$TMPFILE" "$JOURNAL_DUMP_DIR/$test_name.img"
echo "logdump -c" > $TMPFILE.cmd
-$DEBUGFS_EXE $TMPFILE -f $TMPFILE.cmd 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -rf $TMPFILE.cmd
$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
diff --git a/tests/j_short_trans_old_csum/script b/tests/j_short_trans_old_csum/script
index b968bc1..52fc58f 100644
--- a/tests/j_short_trans_old_csum/script
+++ b/tests/j_short_trans_old_csum/script
@@ -27,7 +27,7 @@ echo "debugfs write journal" >> $OUT
echo "jo -c" > $TMPFILE.cmd
echo "jw -b $bitmaps /dev/zero" >> $TMPFILE.cmd
echo "jc" >> $TMPFILE.cmd
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd 2>> $OUT.new > /dev/null
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE 2>> $OUT.new > /dev/null
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
@@ -35,7 +35,7 @@ $DUMPE2FS $TMPFILE 2>&1 | grep '^Journal features:' >> $OUT
test -d "$JOURNAL_DUMP_DIR" -a -w "$JOURNAL_DUMP_DIR" && cp "$TMPFILE" "$JOURNAL_DUMP_DIR/$test_name.img"
echo "logdump -c" > $TMPFILE.cmd
-$DEBUGFS_EXE $TMPFILE -f $TMPFILE.cmd 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -rf $TMPFILE.cmd
$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
diff --git a/tests/j_short_trans_open_recover/script b/tests/j_short_trans_open_recover/script
index fbbb714..e5b772d 100644
--- a/tests/j_short_trans_open_recover/script
+++ b/tests/j_short_trans_open_recover/script
@@ -25,20 +25,20 @@ echo "debugfs write journal" >> $OUT
echo "jo" > $TMPFILE.cmd
echo "jw -b $bitmaps /dev/zero" >> $TMPFILE.cmd
echo "jc" >> $TMPFILE.cmd
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd 2>> $OUT.new > /dev/null
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE 2>> $OUT.new > /dev/null
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
test -d "$JOURNAL_DUMP_DIR" -a -w "$JOURNAL_DUMP_DIR" && cp "$TMPFILE" "$JOURNAL_DUMP_DIR/$test_name.img"
echo "logdump -c" > $TMPFILE.cmd
-$DEBUGFS_EXE $TMPFILE -f $TMPFILE.cmd 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -rf $TMPFILE.cmd
echo "debugfs can't recover open journal" >> $OUT
echo "jo" > $TMPFILE.cmd
echo "jr" >> $TMPFILE.cmd
echo "jc" >> $TMPFILE.cmd
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd > $OUT.new 2>&1
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE > $OUT.new 2>&1
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
diff --git a/tests/j_short_trans_recover/script b/tests/j_short_trans_recover/script
index 7fa6158..2494a2e 100644
--- a/tests/j_short_trans_recover/script
+++ b/tests/j_short_trans_recover/script
@@ -28,18 +28,18 @@ echo "jc" >> $TMPFILE.cmd
echo "jo" >> $TMPFILE.cmd
echo "jw -r 333" >> $TMPFILE.cmd
echo "jc" >> $TMPFILE.cmd
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd 2>> $OUT.new > /dev/null
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE 2>> $OUT.new > /dev/null
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
test -d "$JOURNAL_DUMP_DIR" -a -w "$JOURNAL_DUMP_DIR" && cp "$TMPFILE" "$JOURNAL_DUMP_DIR/$test_name.img"
echo "logdump -c" > $TMPFILE.cmd
-$DEBUGFS_EXE $TMPFILE -f $TMPFILE.cmd 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -rf $TMPFILE.cmd
echo "debugfs recover journal" >> $OUT
echo "jr" > $TMPFILE.cmd
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd 2>> $OUT.new > /dev/null
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE 2>> $OUT.new > /dev/null
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
diff --git a/tests/j_short_trans_recover_mcsum_64bit/script b/tests/j_short_trans_recover_mcsum_64bit/script
index c283fe2..6076e03 100644
--- a/tests/j_short_trans_recover_mcsum_64bit/script
+++ b/tests/j_short_trans_recover_mcsum_64bit/script
@@ -30,7 +30,7 @@ echo "jc" >> $TMPFILE.cmd
echo "jo" >> $TMPFILE.cmd
echo "jw -r 333" >> $TMPFILE.cmd
echo "jc" >> $TMPFILE.cmd
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd 2>> $OUT.new > /dev/null
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE 2>> $OUT.new > /dev/null
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
@@ -38,12 +38,12 @@ $DUMPE2FS $TMPFILE 2>&1 | grep '^Journal features:' >> $OUT
test -d "$JOURNAL_DUMP_DIR" -a -w "$JOURNAL_DUMP_DIR" && cp "$TMPFILE" "$JOURNAL_DUMP_DIR/$test_name.img"
echo "logdump -c" > $TMPFILE.cmd
-$DEBUGFS_EXE $TMPFILE -f $TMPFILE.cmd 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -rf $TMPFILE.cmd
echo "debugfs recover journal" >> $OUT
echo "jr" > $TMPFILE.cmd
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd 2>> $OUT.new > /dev/null
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE 2>> $OUT.new > /dev/null
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
diff --git a/tests/j_short_uncommitted_trans/script b/tests/j_short_uncommitted_trans/script
index 207d746..a988948 100644
--- a/tests/j_short_uncommitted_trans/script
+++ b/tests/j_short_uncommitted_trans/script
@@ -25,13 +25,13 @@ echo "debugfs write journal" >> $OUT
echo "jo" > $TMPFILE.cmd
echo "jw -b $bitmaps -c /dev/zero" >> $TMPFILE.cmd
echo "jc" >> $TMPFILE.cmd
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd 2>> $OUT.new > /dev/null
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE 2>> $OUT.new > /dev/null
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
test -d "$JOURNAL_DUMP_DIR" -a -w "$JOURNAL_DUMP_DIR" && cp "$TMPFILE" "$JOURNAL_DUMP_DIR/$test_name.img"
echo "logdump -c" > $TMPFILE.cmd
-$DEBUGFS_EXE $TMPFILE -f $TMPFILE.cmd 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -rf $TMPFILE.cmd
$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
diff --git a/tests/j_short_uncommitted_trans_mcsum_64bit/script b/tests/j_short_uncommitted_trans_mcsum_64bit/script
index de506be..43313bb 100644
--- a/tests/j_short_uncommitted_trans_mcsum_64bit/script
+++ b/tests/j_short_uncommitted_trans_mcsum_64bit/script
@@ -27,7 +27,7 @@ echo "debugfs write journal" >> $OUT
echo "jo -c" > $TMPFILE.cmd
echo "jw -b $bitmaps /dev/zero -c" >> $TMPFILE.cmd
echo "jc" >> $TMPFILE.cmd
-$DEBUGFS_EXE -w $TMPFILE -f $TMPFILE.cmd 2>> $OUT.new > /dev/null
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE 2>> $OUT.new > /dev/null
sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
rm -rf $OUT.new
@@ -35,7 +35,7 @@ $DUMPE2FS $TMPFILE 2>&1 | grep '^Journal features:' >> $OUT
test -d "$JOURNAL_DUMP_DIR" -a -w "$JOURNAL_DUMP_DIR" && cp "$TMPFILE" "$JOURNAL_DUMP_DIR/$test_name.img"
echo "logdump -c" > $TMPFILE.cmd
-$DEBUGFS_EXE $TMPFILE -f $TMPFILE.cmd 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DEBUGFS_EXE -f $TMPFILE.cmd $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
rm -rf $TMPFILE.cmd
$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
diff --git a/tests/m_mmp_bad_csum/script b/tests/m_mmp_bad_csum/script
index d101294..09e870c 100644
--- a/tests/m_mmp_bad_csum/script
+++ b/tests/m_mmp_bad_csum/script
@@ -10,7 +10,7 @@ if [ $? = 0 ]; then
fi
gzip -dc < $test_dir/image.gz > $TMPFILE
-OUT=$test_dir.log
+OUT=$test_name.log
EXP=$test_dir/expect
$FSCK -fy $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed > $OUT
echo Exit status is $? >> $OUT
diff --git a/tests/m_mmp_bad_magic/script b/tests/m_mmp_bad_magic/script
index d101294..09e870c 100644
--- a/tests/m_mmp_bad_magic/script
+++ b/tests/m_mmp_bad_magic/script
@@ -10,7 +10,7 @@ if [ $? = 0 ]; then
fi
gzip -dc < $test_dir/image.gz > $TMPFILE
-OUT=$test_dir.log
+OUT=$test_name.log
EXP=$test_dir/expect
$FSCK -fy $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed > $OUT
echo Exit status is $? >> $OUT
diff --git a/tests/t_mke2fs_errors/script b/tests/t_mke2fs_errors/script
index d09e926..a5c553b 100755
--- a/tests/t_mke2fs_errors/script
+++ b/tests/t_mke2fs_errors/script
@@ -77,23 +77,23 @@ $DUMPE2FS $TMPFILE 2>&1 | grep 'Errors behavior' >> $OUT
# Test errors= in a fs type
echo "error fs_types profile continue" >> $OUT
write_section_conf continue
-MKE2FS_CONFIG=$conf $MKE2FS -F $TMPFILE -T test_suite > /dev/null 2>&1
+MKE2FS_CONFIG=$conf $MKE2FS -T test_suite -F $TMPFILE > /dev/null 2>&1
$DUMPE2FS $TMPFILE 2>&1 | grep 'Errors behavior' >> $OUT
echo "error fs_types profile panic" >> $OUT
write_section_conf panic
-MKE2FS_CONFIG=$conf $MKE2FS -F $TMPFILE -T test_suite > /dev/null 2>&1
+MKE2FS_CONFIG=$conf $MKE2FS -T test_suite -F $TMPFILE > /dev/null 2>&1
$DUMPE2FS $TMPFILE 2>&1 | grep 'Errors behavior' >> $OUT
echo "error fs_types profile remount-ro" >> $OUT
write_section_conf remount-ro
-MKE2FS_CONFIG=$conf $MKE2FS -F $TMPFILE -T test_suite > /dev/null 2>&1
+MKE2FS_CONFIG=$conf $MKE2FS -T test_suite -F $TMPFILE > /dev/null 2>&1
$DUMPE2FS $TMPFILE 2>&1 | grep 'Errors behavior' >> $OUT
# Test command line override
echo "error fs_types profile remount-ro" >> $OUT
write_section_conf remount-ro
-MKE2FS_CONFIG=$conf $MKE2FS -F $TMPFILE -T test_suite -e panic > /dev/null 2>&1
+MKE2FS_CONFIG=$conf $MKE2FS -T test_suite -e panic -F $TMPFILE > /dev/null 2>&1
$DUMPE2FS $TMPFILE 2>&1 | grep 'Errors behavior' >> $OUT
cmp -s $OUT $EXP
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 02/47] misc: fix compiler warnings and minor build errors
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
2014-11-07 21:50 ` [PATCH 01/47] tests: fix test scripts that don't work on non-Linux systems Darrick J. Wong
@ 2014-11-07 21:50 ` Darrick J. Wong
2014-11-08 2:25 ` Theodore Ts'o
2014-11-07 21:51 ` [PATCH 03/47] dumpe2fs: don't crash when the user provides no block device argument Darrick J. Wong
` (46 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:50 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Fix some gcc-4.8 warnings and other problems that broke the build.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
e2fsck/region.c | 8 ++++----
lib/ext2fs/Makefile.in | 2 +-
lib/ext2fs/inline_data.c | 3 ---
resize/resize2fs.c | 2 +-
4 files changed, 6 insertions(+), 9 deletions(-)
diff --git a/e2fsck/region.c b/e2fsck/region.c
index aaaaa19..e32f89d 100644
--- a/e2fsck/region.c
+++ b/e2fsck/region.c
@@ -159,10 +159,10 @@ void region_print(region_t region, FILE *f)
struct region_el *r;
int i = 0;
- fprintf(f, "Printing region (min=%d. max=%d)\n\t", region->min,
+ fprintf(f, "Printing region (min=%llu. max=%llu)\n\t", region->min,
region->max);
for (r = region->allocated; r; r = r->next) {
- fprintf(f, "(%d, %d) ", r->start, r->end);
+ fprintf(f, "(%llu, %llu) ", r->start, r->end);
if (++i >= 8)
fprintf(f, "\n\t");
}
@@ -183,7 +183,7 @@ int main(int argc, char **argv)
case BCODE_CREATE:
start = bcode_program[pc++];
end = bcode_program[pc++];
- printf("Creating region with args(%d, %d)\n",
+ printf("Creating region with args(%llu, %llu)\n",
start, end);
r = region_create(start, end);
if (!r) {
@@ -195,7 +195,7 @@ int main(int argc, char **argv)
start = bcode_program[pc++];
end = bcode_program[pc++];
ret = region_allocate(r, start, end);
- printf("Region_allocate(%d, %d) returns %d\n",
+ printf("Region_allocate(%llu, %llu) returns %d\n",
start, end, ret);
break;
case BCODE_PRINT:
diff --git a/lib/ext2fs/Makefile.in b/lib/ext2fs/Makefile.in
index dc3e144..2706bfa 100644
--- a/lib/ext2fs/Makefile.in
+++ b/lib/ext2fs/Makefile.in
@@ -415,7 +415,7 @@ recovery.o: $(top_srcdir)/e2fsck/recovery.c
plausible.o: $(top_srcdir)/misc/plausible.c
$(E) " CC $<"
- $(Q) $(CC) $(ALL_CFLAGS) -c $< -o $@
+ $(Q) $(CC) $(DEBUGFS_CFLAGS) -c $< -o $@
do_journal.o: $(top_srcdir)/debugfs/do_journal.c
$(E) " CC $<"
diff --git a/lib/ext2fs/inline_data.c b/lib/ext2fs/inline_data.c
index 8167b76..88e08e7 100644
--- a/lib/ext2fs/inline_data.c
+++ b/lib/ext2fs/inline_data.c
@@ -613,7 +613,6 @@ static errcode_t file_test(ext2_filsys fs)
errcode_t retval;
size_t size;
char *buf = 0, *cmpbuf = 0;
- int i;
/* create a new file */
retval = ext2fs_new_inode(fs, 2, 010755, 0, &newfile);
@@ -793,7 +792,6 @@ int main(int argc, char *argv[])
ext2_filsys fs;
struct ext2_super_block param;
errcode_t retval;
- int i;
/* setup */
initialize_ext2_error_table();
@@ -823,7 +821,6 @@ int main(int argc, char *argv[])
/* initialize inode cache */
if (!fs->icache) {
- struct ext2_inode inode;
ext2_ino_t first_ino = EXT2_FIRST_INO(fs->super);
int i;
diff --git a/resize/resize2fs.c b/resize/resize2fs.c
index 57fe485..8c5db86 100644
--- a/resize/resize2fs.c
+++ b/resize/resize2fs.c
@@ -1544,7 +1544,7 @@ static errcode_t progress_callback(ext2_filsys fs,
static errcode_t migrate_ea_block(ext2_resize_t rfs, ext2_ino_t ino,
struct ext2_inode *inode, int *changed)
{
- char *buf;
+ char *buf = NULL;
blk64_t new_block;
errcode_t err = 0;
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 03/47] dumpe2fs: don't crash when the user provides no block device argument
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
2014-11-07 21:50 ` [PATCH 01/47] tests: fix test scripts that don't work on non-Linux systems Darrick J. Wong
2014-11-07 21:50 ` [PATCH 02/47] misc: fix compiler warnings and minor build errors Darrick J. Wong
@ 2014-11-07 21:51 ` Darrick J. Wong
2014-11-08 2:26 ` Theodore Ts'o
2014-11-07 21:51 ` [PATCH 04/47] libext2fs: fix endian handling error; reduce fragmentation some Darrick J. Wong
` (45 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:51 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
If the user doesn't provide any arguments, the guard fails to run and
the whole thing segfaults on ext2fs_open2(). Don't do that.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
misc/dumpe2fs.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/misc/dumpe2fs.c b/misc/dumpe2fs.c
index e565f2d..26f2e37 100644
--- a/misc/dumpe2fs.c
+++ b/misc/dumpe2fs.c
@@ -656,7 +656,7 @@ int main (int argc, char ** argv)
usage();
}
}
- if (argc - 1 > optind) {
+ if (optind != argc - 1) {
usage();
exit(1);
}
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 04/47] libext2fs: fix endian handling error; reduce fragmentation some
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (2 preceding siblings ...)
2014-11-07 21:51 ` [PATCH 03/47] dumpe2fs: don't crash when the user provides no block device argument Darrick J. Wong
@ 2014-11-07 21:51 ` Darrick J. Wong
2014-11-17 21:26 ` Theodore Ts'o
2014-11-07 21:51 ` [PATCH 05/47] libext2fs: set BLOCK_UNINIT for non-last blockgroups if all blocks are free Darrick J. Wong
` (44 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:51 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
If we're going to read the "nr - 1" entry in an indirect block for use
as a "goal" input to the block allocator, we need to byteswap the
entry. While we're at it, if we're allocating blocks for the zeroth
entry in the indirect block, we might as well use the indirect block
as the starting point to try to reduce fragmentation.
(d_fallocate_blkmap will test this...)
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
lib/ext2fs/bmap.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/ext2fs/bmap.c b/lib/ext2fs/bmap.c
index c1d0e6f..f819e9c 100644
--- a/lib/ext2fs/bmap.c
+++ b/lib/ext2fs/bmap.c
@@ -67,7 +67,7 @@ static _BMAP_INLINE_ errcode_t block_ind_bmap(ext2_filsys fs, int flags,
#endif
if (!b && (flags & BMAP_ALLOC)) {
- b = nr ? ((blk_t *) block_buf)[nr-1] : 0;
+ b = nr ? ext2fs_le32_to_cpu(((blk_t *)block_buf)[nr - 1]) : ind;
retval = ext2fs_alloc_block(fs, b,
block_buf + fs->blocksize, &b);
if (retval)
^ permalink raw reply related [flat|nested] 104+ messages in thread* Re: [PATCH 04/47] libext2fs: fix endian handling error; reduce fragmentation some
2014-11-07 21:51 ` [PATCH 04/47] libext2fs: fix endian handling error; reduce fragmentation some Darrick J. Wong
@ 2014-11-17 21:26 ` Theodore Ts'o
0 siblings, 0 replies; 104+ messages in thread
From: Theodore Ts'o @ 2014-11-17 21:26 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Fri, Nov 07, 2014 at 01:51:08PM -0800, Darrick J. Wong wrote:
> If we're going to read the "nr - 1" entry in an indirect block for use
> as a "goal" input to the block allocator, we need to byteswap the
> entry. While we're at it, if we're allocating blocks for the zeroth
> entry in the indirect block, we might as well use the indirect block
> as the starting point to try to reduce fragmentation.
>
> (d_fallocate_blkmap will test this...)
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Applied, thanks.
- Ted
^ permalink raw reply [flat|nested] 104+ messages in thread
* [PATCH 05/47] libext2fs: set BLOCK_UNINIT for non-last blockgroups if all blocks are free
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (3 preceding siblings ...)
2014-11-07 21:51 ` [PATCH 04/47] libext2fs: fix endian handling error; reduce fragmentation some Darrick J. Wong
@ 2014-11-07 21:51 ` Darrick J. Wong
2014-11-17 22:47 ` Theodore Ts'o
2014-11-07 21:51 ` [PATCH 06/47] libext2fs: don't allow alloc_stats on bad inode/block numbers Darrick J. Wong
` (43 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:51 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Set BLOCK_UNINIT in any group whose blocks are all unused, so long as
it isn't the last group. This helps us speed up future e2fsck runs
and mounts because we don't need to read or checksum block bitmaps for
these groups.
v2: Take care of this in the library, since e2fsck always updates these
summary counts.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
lib/ext2fs/csum.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/lib/ext2fs/csum.c b/lib/ext2fs/csum.c
index 12f6fe9..6dcefb9 100644
--- a/lib/ext2fs/csum.c
+++ b/lib/ext2fs/csum.c
@@ -854,6 +854,11 @@ errcode_t ext2fs_set_gdt_csum(ext2_filsys fs)
__u32 old_unused = ext2fs_bg_itable_unused(fs, i);
__u32 old_flags = ext2fs_bg_flags(fs, i);
__u32 old_free_inodes_count = ext2fs_bg_free_inodes_count(fs, i);
+ __u32 old_free_blocks_count = ext2fs_bg_free_blocks_count(fs, i);
+
+ if (old_free_blocks_count == sb->s_blocks_per_group &&
+ i != fs->group_desc_count - 1)
+ ext2fs_bg_flags_set(fs, i, EXT2_BG_BLOCK_UNINIT);
if (old_free_inodes_count == sb->s_inodes_per_group) {
ext2fs_bg_flags_set(fs, i, EXT2_BG_INODE_UNINIT);
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 06/47] libext2fs: don't allow alloc_stats on bad inode/block numbers
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (4 preceding siblings ...)
2014-11-07 21:51 ` [PATCH 05/47] libext2fs: set BLOCK_UNINIT for non-last blockgroups if all blocks are free Darrick J. Wong
@ 2014-11-07 21:51 ` Darrick J. Wong
2014-11-17 23:00 ` Theodore Ts'o
2014-11-07 21:51 ` [PATCH 07/47] libext2fs: refactor extent head creation Darrick J. Wong
` (42 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:51 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Don't allow callers to feed bad block/inode numbers to
ext2fs_*_alloc_stats2, because evil callers (<cough>resize2fs<cough>)
can corrupt library state this way, leading to a crash.
(There will be a subsequent patch to resize2fs to fix its bad
behavior.)
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
lib/ext2fs/alloc_stats.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/lib/ext2fs/alloc_stats.c b/lib/ext2fs/alloc_stats.c
index 3d3697c..aca5004 100644
--- a/lib/ext2fs/alloc_stats.c
+++ b/lib/ext2fs/alloc_stats.c
@@ -20,13 +20,13 @@ void ext2fs_inode_alloc_stats2(ext2_filsys fs, ext2_ino_t ino,
{
int group = ext2fs_group_of_ino(fs, ino);
-#ifndef OMIT_COM_ERR
if (ino > fs->super->s_inodes_count) {
+#ifndef OMIT_COM_ERR
com_err("ext2fs_inode_alloc_stats2", 0,
"Illegal inode number: %lu", (unsigned long) ino);
+#endif
return;
}
-#endif
if (inuse > 0)
ext2fs_mark_inode_bitmap2(fs->inode_map, ino);
else
@@ -62,13 +62,13 @@ void ext2fs_block_alloc_stats2(ext2_filsys fs, blk64_t blk, int inuse)
{
int group = ext2fs_group_of_blk2(fs, blk);
-#ifndef OMIT_COM_ERR
if (blk >= ext2fs_blocks_count(fs->super)) {
+#ifndef OMIT_COM_ERR
com_err("ext2fs_block_alloc_stats", 0,
"Illegal block number: %lu", (unsigned long) blk);
+#endif
return;
}
-#endif
if (inuse > 0)
ext2fs_mark_block_bitmap2(fs->block_map, blk);
else
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 07/47] libext2fs: refactor extent head creation
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (5 preceding siblings ...)
2014-11-07 21:51 ` [PATCH 06/47] libext2fs: don't allow alloc_stats on bad inode/block numbers Darrick J. Wong
@ 2014-11-07 21:51 ` Darrick J. Wong
2014-12-03 3:55 ` Theodore Ts'o
2014-11-07 21:51 ` [PATCH 08/47] libext2fs: file IO routines should handle uninit blocks Darrick J. Wong
` (41 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:51 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Don't open-code the creation of the extent tree header, since
ext2fs_extent_open2() knows how to take care of this.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
lib/ext2fs/inline_data.c | 19 ++++++++-----------
misc/create_inode.c | 18 +++++++-----------
2 files changed, 15 insertions(+), 22 deletions(-)
diff --git a/lib/ext2fs/inline_data.c b/lib/ext2fs/inline_data.c
index 88e08e7..6260c5f 100644
--- a/lib/ext2fs/inline_data.c
+++ b/lib/ext2fs/inline_data.c
@@ -412,19 +412,16 @@ ext2fs_inline_data_file_expand(ext2_filsys fs, ext2_ino_t ino,
errcode_t retval;
/* Update inode */
+ memset(inode->i_block, 0, sizeof(inode->i_block));
if (EXT2_HAS_INCOMPAT_FEATURE(fs->super,
EXT3_FEATURE_INCOMPAT_EXTENTS)) {
- int i;
- struct ext3_extent_header *eh;
-
- eh = (struct ext3_extent_header *) &inode->i_block[0];
- eh->eh_depth = 0;
- eh->eh_entries = 0;
- eh->eh_magic = ext2fs_cpu_to_le16(EXT3_EXT_MAGIC);
- i = (sizeof(inode->i_block) - sizeof(*eh)) /
- sizeof(struct ext3_extent);
- eh->eh_max = ext2fs_cpu_to_le16(i);
- inode->i_flags |= EXT4_EXTENTS_FL;
+ ext2_extent_handle_t handle;
+
+ inode->i_flags &= ~EXT4_EXTENTS_FL;
+ retval = ext2fs_extent_open2(fs, ino, inode, &handle);
+ if (retval)
+ return retval;
+ ext2fs_extent_free(handle);
}
inode->i_flags &= ~EXT4_INLINE_DATA_FL;
inode->i_size = 0;
diff --git a/misc/create_inode.c b/misc/create_inode.c
index cf7c097..a024d1c 100644
--- a/misc/create_inode.c
+++ b/misc/create_inode.c
@@ -518,17 +518,13 @@ errcode_t do_write_internal(ext2_filsys fs, ext2_ino_t cwd, const char *src,
inode.i_flags |= EXT4_INLINE_DATA_FL;
} else if (fs->super->s_feature_incompat &
EXT3_FEATURE_INCOMPAT_EXTENTS) {
- int i;
- struct ext3_extent_header *eh;
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 08/47] libext2fs: file IO routines should handle uninit blocks
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (6 preceding siblings ...)
2014-11-07 21:51 ` [PATCH 07/47] libext2fs: refactor extent head creation Darrick J. Wong
@ 2014-11-07 21:51 ` Darrick J. Wong
2014-12-03 3:57 ` Theodore Ts'o
2014-11-07 21:51 ` [PATCH 09/47] libext2fs: use a dynamically sized (or caller-provided) block zeroing buffer Darrick J. Wong
` (40 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:51 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
The file IO routines do not handle uninit blocks at all. The read
method should check for the uninit flag and return a buffer of zeroes,
and the write routine should convert unwritten extents.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
lib/ext2fs/fileio.c | 24 ++++++++++++++++++++++--
tests/f_uninit_cat/expect | Bin
tests/f_uninit_cat/image.gz | Bin
tests/f_uninit_cat/name | 1 +
tests/f_uninit_cat/script | 37 +++++++++++++++++++++++++++++++++++++
5 files changed, 60 insertions(+), 2 deletions(-)
create mode 100644 tests/f_uninit_cat/expect
create mode 100644 tests/f_uninit_cat/image.gz
create mode 100644 tests/f_uninit_cat/name
create mode 100755 tests/f_uninit_cat/script
diff --git a/lib/ext2fs/fileio.c b/lib/ext2fs/fileio.c
index 1d5032a..d0a05d6 100644
--- a/lib/ext2fs/fileio.c
+++ b/lib/ext2fs/fileio.c
@@ -123,6 +123,8 @@ errcode_t ext2fs_file_flush(ext2_file_t file)
{
errcode_t retval;
ext2_filsys fs;
+ int ret_flags;
+ blk64_t dontcare;
EXT2_CHECK_MAGIC(file, EXT2_ET_MAGIC_EXT2_FILE);
fs = file->fs;
@@ -131,6 +133,22 @@ errcode_t ext2fs_file_flush(ext2_file_t file)
!(file->flags & EXT2_FILE_BUF_DIRTY))
return 0;
+ /* Is this an uninit block? */
+ if (file->physblock && file->inode.i_flags & EXT4_EXTENTS_FL) {
+ retval = ext2fs_bmap2(fs, file->ino, &file->inode, BMAP_BUFFER,
+ 0, file->blockno, &ret_flags, &dontcare);
+ if (retval)
+ return retval;
+ if (ret_flags & BMAP_RET_UNINIT) {
+ retval = ext2fs_bmap2(fs, file->ino, &file->inode,
+ BMAP_BUFFER, BMAP_SET,
+ file->blockno, 0,
+ &file->physblock);
+ if (retval)
+ return retval;
+ }
+ }
+
/*
* OK, the physical block hasn't been allocated yet.
* Allocate it.
@@ -185,15 +203,17 @@ static errcode_t load_buffer(ext2_file_t file, int dontfill)
{
ext2_filsys fs = file->fs;
errcode_t retval;
+ int ret_flags;
if (!(file->flags & EXT2_FILE_BUF_VALID)) {
retval = ext2fs_bmap2(fs, file->ino, &file->inode,
- BMAP_BUFFER, 0, file->blockno, 0,
+ BMAP_BUFFER, 0, file->blockno, &ret_flags,
&file->physblock);
if (retval)
return retval;
if (!dontfill) {
- if (file->physblock) {
+ if (file->physblock &&
+ !(ret_flags & BMAP_RET_UNINIT)) {
retval = io_channel_read_blk64(fs->io,
file->physblock,
1, file->buf);
diff --git a/tests/f_uninit_cat/expect b/tests/f_uninit_cat/expect
new file mode 100644
index 0000000000000000000000000000000000000000..0c0a5cf84cc58483ca33e257259607e4ca54ebbe
GIT binary patch
literal 3623
zcmeH{v2KGf5Qa1J6t`4rrKVs4QNvoLQ-&;^E5{tf3b8An9fdx94LB%6-ypu_bol%I
zbN}EyCjwT%#}UOzsurZuHPR~_IxSAVb5#S$U!-I|p!pqIOM}8{(*s%KgmnfdX!S27
zv{Igz7is&6EABXh4H{GeL1?FJuq*F~)@b(w<j!aAEv0I-IddzuN-UE7Ze)klQw1zf
z*9D9tJZEp&6DX~g-rdU9X-6-wP?Ral@**smY_HP#9k_J_k|0ZJJh-+Y5Zr=OQu*WI
zzT5W-@CqqUc6h-Kw#pib1XJyFD+TYSVSnstoOY;MdxX!9x0FDZLgoRM0<1<bgJtNx
zmG9H!`&bOV#$T9q`K)6>#|E(61l{JQn-!~Bkq1RSFzev!`&hG6*uNSB@QW^D#ROsk
a51YU>R!kr!@URIy?86rqj|s#C7RevINPdd|
literal 0
HcmV?d00001
diff --git a/tests/f_uninit_cat/image.gz b/tests/f_uninit_cat/image.gz
new file mode 100644
index 0000000000000000000000000000000000000000..d2ae66cb35f93dd4022e85f8667365487a7b24ea
GIT binary patch
literal 1553
zcmb2|=3wBo77AfvetUOgws4>X!-w+e5!ulJUCtMqn*6x7X0!`+M-&JY2D}M!S)+8+
zDVJ6IMapY?1&f!@9@sy4qUHCltf(-cFi@sM<Z<;*mRk{C3zur%doRAP`s7{J>E%1$
zf6YFjs>qO9YUaC2aZcCS-(DQb8_$NVE7;Mo<=)q*|3~ghM6S5L{?7@i*S^PYuMLg<
z{yO)bMDN+#HZAQNWmni(e$ISz=k4?0&B2S~=hf!a%c;LRqnoqOeE0R(Td}i${(5!v
zN9*zV=lkXUet4HAR(*B4`0223$*Xyed2c)RXUFctq3v?k4*osNhJB`=>Q$cfF5YY)
zq&Ig<I|BqHJYRiXzO>c;>t5Mgf0ATh%=q$g`@RWJoj2;Pd;ar6z(>h_e;2CHpSyp1
zK~2@ae|Oey(P??=HSMlJ&Y}8++jalfhj#s8_ga)C-tdS0iM_&q#-IEr>K*<uf0Cc@
zA1uVcut9k1xj*G<CR6-R{q_2++YoEG%ju)G)m_7H5^{`CS3sx*%4*qvh1Y5X@893w
ze(dD=mG_O$YD^1}+TT9sT+6vfTb`$HdzF3Z(w=*|H~uB`<f~6#9=s)v-CcQutl0EN
zDLbP6mj5|_XrHg$w}ZTW^>6Fj_h@afy|?67dd1pb`uxmif2pqt{Qt!?S^L#|zi0nX
zFI}X(^4;uQx9mT!qeB0OhF!1kR=hH+G@<KquWi^F`Rv=r-e*s4zxMk7(bMbaFY_=>
zsFGV7C4lT;jqS<~nYIjGr#U#DKDAdsmKtS`hQMeD44Duxd!&D1J4-kNg8%~n1^y`^
literal 0
HcmV?d00001
diff --git a/tests/f_uninit_cat/name b/tests/f_uninit_cat/name
new file mode 100644
index 0000000..f6b5674
--- /dev/null
+++ b/tests/f_uninit_cat/name
@@ -0,0 +1 @@
+cat a file with uninit blocks
diff --git a/tests/f_uninit_cat/script b/tests/f_uninit_cat/script
new file mode 100755
index 0000000..1655bb0
--- /dev/null
+++ b/tests/f_uninit_cat/script
@@ -0,0 +1,37 @@
+#!/bin/bash
+
+if test -x $DEBUGFS_EXE; then
+FSCK_OPT=-fy
+IMAGE=$test_dir/image.gz
+
+gzip -d < $IMAGE > $TMPFILE
+#e2label $TMPFILE test_filesys
+
+# Run fsck to fix things?
+EXP=$test_dir/expect
+OUT=$test_name.log
+rm -rf $test_name.failed $test_name.ok
+
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed > $OUT
+echo "Exit status is $?" >> $OUT
+
+echo "debugfs cat uninit file" >> $OUT
+echo "ex /a" > $TMPFILE.cmd
+echo "cat /a" >> $TMPFILE.cmd
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE >> $OUT.new 2>&1
+echo >> $OUT.new
+sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
+rm -rf $OUT.new $TMPFILE
+
+# Figure out what happened
+if cmp -s $EXP $OUT; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff -u $EXP $OUT >> $test_name.failed
+fi
+unset EXP OUT FSCK_OPT IMAGE
+else #if test -a -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 09/47] libext2fs: use a dynamically sized (or caller-provided) block zeroing buffer
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (7 preceding siblings ...)
2014-11-07 21:51 ` [PATCH 08/47] libext2fs: file IO routines should handle uninit blocks Darrick J. Wong
@ 2014-11-07 21:51 ` Darrick J. Wong
2014-12-12 2:37 ` Theodore Ts'o
2014-11-07 21:51 ` [PATCH 10/47] libext2fs: support BLKZEROOUT/FALLOC_FL_ZERO_RANGE in ext2fs_zero_blocks Darrick J. Wong
` (39 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:51 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Dynamically grow the block zeroing buffer to a maximum of 4MB, and
allow callers to provide their own zeroed buffer.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
lib/ext2fs/alloc.c | 2 +-
lib/ext2fs/ext2fs.h | 3 ++
lib/ext2fs/mkjournal.c | 62 ++++++++++++++++++++++++++++++++++++++----------
3 files changed, 53 insertions(+), 14 deletions(-)
diff --git a/lib/ext2fs/alloc.c b/lib/ext2fs/alloc.c
index 54535c4..cb778f7 100644
--- a/lib/ext2fs/alloc.c
+++ b/lib/ext2fs/alloc.c
@@ -203,7 +203,7 @@ errcode_t ext2fs_alloc_block2(ext2_filsys fs, blk64_t goal,
goto fail;
}
- retval = ext2fs_zero_blocks2(fs, block, 1, NULL, NULL);
+ retval = ext2fs_zero_blocks3(fs, block, 1, NULL, NULL, block_buf);
if (retval)
goto fail;
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 506d43b..f9599a3 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -1491,6 +1491,9 @@ extern errcode_t ext2fs_zero_blocks(ext2_filsys fs, blk_t blk, int num,
blk_t *ret_blk, int *ret_count);
extern errcode_t ext2fs_zero_blocks2(ext2_filsys fs, blk64_t blk, int num,
blk64_t *ret_blk, int *ret_count);
+extern errcode_t ext2fs_zero_blocks3(ext2_filsys fs, blk64_t blk, int num,
+ blk64_t *ret_blk, int *ret_count,
+ const void *block_buf);
extern errcode_t ext2fs_create_journal_superblock(ext2_filsys fs,
__u32 num_blocks, int flags,
char **ret_jsb);
diff --git a/lib/ext2fs/mkjournal.c b/lib/ext2fs/mkjournal.c
index e8f8b30..fdd788b 100644
--- a/lib/ext2fs/mkjournal.c
+++ b/lib/ext2fs/mkjournal.c
@@ -148,12 +148,14 @@ errfree:
* attempt to free the static zeroizing buffer. (This is to keep
* programs that check for memory leaks happy.)
*/
-#define STRIDE_LENGTH (4194304 / fs->blocksize)
-errcode_t ext2fs_zero_blocks2(ext2_filsys fs, blk64_t blk, int num,
- blk64_t *ret_blk, int *ret_count)
+#define MAX_STRIDE_LENGTH (4194304 / fs->blocksize)
+errcode_t ext2fs_zero_blocks3(ext2_filsys fs, blk64_t blk, int num,
+ blk64_t *ret_blk, int *ret_count,
+ const void *block_buf)
{
int j, count;
- static char *buf;
+ static void *buf;
+ static unsigned stride_length;
errcode_t retval;
/* If fs is null, clean up the static buffer and return */
@@ -164,24 +166,52 @@ errcode_t ext2fs_zero_blocks2(ext2_filsys fs, blk64_t blk, int num,
}
return 0;
}
+
+ /* Deal with zeroing less than 1 block */
+ if (num <= 0)
+ return 0;
+
+ /* If the user gave us a buffer, write that out */
+ if (block_buf) {
+ retval = io_channel_write_blk64(fs->io, blk, num, block_buf);
+ if (retval) {
+ if (ret_count)
+ *ret_count = num;
+ if (ret_blk)
+ *ret_blk = blk;
+ }
+
+ return retval;
+ }
+
/* Allocate the zeroizing buffer if necessary */
- if (!buf) {
- buf = malloc(fs->blocksize * STRIDE_LENGTH);
- if (!buf)
- return ENOMEM;
- memset(buf, 0, fs->blocksize * STRIDE_LENGTH);
+ if (num > stride_length) {
+ void *p;
+ unsigned new_stride = num;
+
+ if (new_stride > MAX_STRIDE_LENGTH)
+ new_stride = MAX_STRIDE_LENGTH;
+ if (new_stride == stride_length)
+ goto skip_alloc;
+ p = realloc(buf, fs->blocksize * new_stride);
+ if (!p)
+ return EXT2_ET_NO_MEMORY;
+ buf = p;
+ stride_length = new_stride;
+ memset(buf, 0, fs->blocksize * stride_length);
}
+skip_alloc:
/* OK, do the write loop */
j=0;
while (j < num) {
- if (blk % STRIDE_LENGTH) {
- count = STRIDE_LENGTH - (blk % STRIDE_LENGTH);
+ if (blk % stride_length) {
+ count = stride_length - (blk % stride_length);
if (count > (num - j))
count = num - j;
} else {
count = num - j;
- if (count > STRIDE_LENGTH)
- count = STRIDE_LENGTH;
+ if (count > stride_length)
+ count = stride_length;
}
retval = io_channel_write_blk64(fs->io, blk, count, buf);
if (retval) {
@@ -196,6 +226,12 @@ errcode_t ext2fs_zero_blocks2(ext2_filsys fs, blk64_t blk, int num,
return 0;
}
+errcode_t ext2fs_zero_blocks2(ext2_filsys fs, blk64_t blk, int num,
+ blk64_t *ret_blk, int *ret_count)
+{
+ return ext2fs_zero_blocks3(fs, blk, num, ret_blk, ret_count, NULL);
+}
+
errcode_t ext2fs_zero_blocks(ext2_filsys fs, blk_t blk, int num,
blk_t *ret_blk, int *ret_count)
{
^ permalink raw reply related [flat|nested] 104+ messages in thread* Re: [PATCH 09/47] libext2fs: use a dynamically sized (or caller-provided) block zeroing buffer
2014-11-07 21:51 ` [PATCH 09/47] libext2fs: use a dynamically sized (or caller-provided) block zeroing buffer Darrick J. Wong
@ 2014-12-12 2:37 ` Theodore Ts'o
2014-12-12 5:02 ` Darrick J. Wong
0 siblings, 1 reply; 104+ messages in thread
From: Theodore Ts'o @ 2014-12-12 2:37 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Fri, Nov 07, 2014 at 01:51:46PM -0800, Darrick J. Wong wrote:
> Dynamically grow the block zeroing buffer to a maximum of 4MB, and
> allow callers to provide their own zeroed buffer.
>
> +
> + /* If the user gave us a buffer, write that out */
> + if (block_buf) {
> + retval = io_channel_write_blk64(fs->io, blk, num, block_buf);
> + if (retval) {
> + if (ret_count)
> + *ret_count = num;
> + if (ret_blk)
> + *ret_blk = blk;
> + }
> +
> + return retval;
> + }
I've looked through the whole patch series,and there's only a single
caller of the new ext2fs_zero_blocks3 function --- and the caller is
going to be giving us a pre-zero'ed buffer, it might as well call
io_channel_write_blk64() directly instead of the new
ext2fs_zero_block3(). This is what we do in 1.42.x, and it makes a
lot more sense than changing the code to call ext2fs_zero_blocks3(),
and requiring it to depend on the caller having zero'ed the block
buffer, and then adding absolutely no value other than calling
i_channel_zeroout(), having it return UNIMPLEMENTED because it makes
no sense to use BLKZEROOUT, and then falling back to
io_channel_write_blk64().
So what I think makes sense is to drop the new ext2fs_zero_blocks3()
function, and to revert the change in ext2fs_alloc_block2() so that it
calls io_channel_write_blk64() instead of ext2fs_zero_block2().
I'll take care of making this change in the patch.
- Ted
^ permalink raw reply [flat|nested] 104+ messages in thread* Re: [PATCH 09/47] libext2fs: use a dynamically sized (or caller-provided) block zeroing buffer
2014-12-12 2:37 ` Theodore Ts'o
@ 2014-12-12 5:02 ` Darrick J. Wong
2014-12-13 16:24 ` Theodore Ts'o
0 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-12-12 5:02 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: linux-ext4
On Thu, Dec 11, 2014 at 09:37:56PM -0500, Theodore Ts'o wrote:
> On Fri, Nov 07, 2014 at 01:51:46PM -0800, Darrick J. Wong wrote:
> > Dynamically grow the block zeroing buffer to a maximum of 4MB, and
> > allow callers to provide their own zeroed buffer.
> >
> > +
> > + /* If the user gave us a buffer, write that out */
> > + if (block_buf) {
> > + retval = io_channel_write_blk64(fs->io, blk, num, block_buf);
> > + if (retval) {
> > + if (ret_count)
> > + *ret_count = num;
> > + if (ret_blk)
> > + *ret_blk = blk;
> > + }
> > +
> > + return retval;
> > + }
>
> I've looked through the whole patch series,and there's only a single
> caller of the new ext2fs_zero_blocks3 function --- and the caller is
> going to be giving us a pre-zero'ed buffer, it might as well call
> io_channel_write_blk64() directly instead of the new
> ext2fs_zero_block3(). This is what we do in 1.42.x, and it makes a
> lot more sense than changing the code to call ext2fs_zero_blocks3(),
> and requiring it to depend on the caller having zero'ed the block
> buffer, and then adding absolutely no value other than calling
> i_channel_zeroout(), having it return UNIMPLEMENTED because it makes
> no sense to use BLKZEROOUT, and then falling back to
> io_channel_write_blk64().
>
> So what I think makes sense is to drop the new ext2fs_zero_blocks3()
> function, and to revert the change in ext2fs_alloc_block2() so that it
> calls io_channel_write_blk64() instead of ext2fs_zero_block2().
>
> I'll take care of making this change in the patch.
Ok, thank you!
(Esp. since the power may or may not stay up out here in the West.)
--D
>
> - Ted
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 104+ messages in thread* Re: [PATCH 09/47] libext2fs: use a dynamically sized (or caller-provided) block zeroing buffer
2014-12-12 5:02 ` Darrick J. Wong
@ 2014-12-13 16:24 ` Theodore Ts'o
2014-12-13 16:25 ` Theodore Ts'o
2014-12-15 16:09 ` Andreas Dilger
0 siblings, 2 replies; 104+ messages in thread
From: Theodore Ts'o @ 2014-12-13 16:24 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
This is my replacement patch.
- Ted
commit 0a92af260da381e2581074871707e487728373ad
Author: Darrick J. Wong <darrick.wong@oracle.com>
Date: Fri Dec 12 18:27:12 2014 -0500
libext2fs: use a dynamically sized block zeroing buffer
Dynamically grow the block zeroing buffer to a maximum of 4MB, and
allow callers to provide their own zeroed buffer in
ext2fs_zero_blocks2().
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
diff --git a/lib/ext2fs/mkjournal.c b/lib/ext2fs/mkjournal.c
index e8f8b30..fcc6741 100644
--- a/lib/ext2fs/mkjournal.c
+++ b/lib/ext2fs/mkjournal.c
@@ -148,12 +148,13 @@ errfree:
* attempt to free the static zeroizing buffer. (This is to keep
* programs that check for memory leaks happy.)
*/
-#define STRIDE_LENGTH (4194304 / fs->blocksize)
+#define MAX_STRIDE_LENGTH (4194304 / (int) fs->blocksize)
errcode_t ext2fs_zero_blocks2(ext2_filsys fs, blk64_t blk, int num,
blk64_t *ret_blk, int *ret_count)
{
int j, count;
- static char *buf;
+ static void *buf;
+ static int stride_length;
errcode_t retval;
/* If fs is null, clean up the static buffer and return */
@@ -164,24 +165,36 @@ errcode_t ext2fs_zero_blocks2(ext2_filsys fs, blk64_t blk, int num,
}
return 0;
}
+
+ /* Deal with zeroing less than 1 block */
+ if (num <= 0)
+ return 0;
+
/* Allocate the zeroizing buffer if necessary */
- if (!buf) {
- buf = malloc(fs->blocksize * STRIDE_LENGTH);
- if (!buf)
- return ENOMEM;
- memset(buf, 0, fs->blocksize * STRIDE_LENGTH);
+ if (num > stride_length && stride_length < MAX_STRIDE_LENGTH) {
+ void *p;
+ int new_stride = num;
+
+ if (new_stride > MAX_STRIDE_LENGTH)
+ new_stride = MAX_STRIDE_LENGTH;
+ p = realloc(buf, fs->blocksize * new_stride);
+ if (!p)
+ return EXT2_ET_NO_MEMORY;
+ buf = p;
+ stride_length = new_stride;
+ memset(buf, 0, fs->blocksize * stride_length);
}
/* OK, do the write loop */
j=0;
while (j < num) {
- if (blk % STRIDE_LENGTH) {
- count = STRIDE_LENGTH - (blk % STRIDE_LENGTH);
+ if (blk % stride_length) {
+ count = stride_length - (blk % stride_length);
if (count > (num - j))
count = num - j;
} else {
count = num - j;
- if (count > STRIDE_LENGTH)
- count = STRIDE_LENGTH;
+ if (count > stride_length)
+ count = stride_length;
}
retval = io_channel_write_blk64(fs->io, blk, count, buf);
if (retval) {
^ permalink raw reply related [flat|nested] 104+ messages in thread* Re: [PATCH 09/47] libext2fs: use a dynamically sized (or caller-provided) block zeroing buffer
2014-12-13 16:24 ` Theodore Ts'o
@ 2014-12-13 16:25 ` Theodore Ts'o
2014-12-15 16:09 ` Andreas Dilger
1 sibling, 0 replies; 104+ messages in thread
From: Theodore Ts'o @ 2014-12-13 16:25 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Sat, Dec 13, 2014 at 11:24:03AM -0500, Theodore Ts'o wrote:
> This is my replacement patch.
Followed by...
commit bc57b123d61f49cddcc3fb594fda2dcf589f9b38
Author: Theodore Ts'o <tytso@mit.edu>
Date: Fri Dec 12 22:12:45 2014 -0500
libext2fs: use block_buf in ext2fs_alloc_block2() if it is provided
If the caller supplies a buffer to ext2fs_alloc_block2(), use it
instead of calling ext2fs_zero_blocks2().
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
diff --git a/lib/ext2fs/alloc.c b/lib/ext2fs/alloc.c
index 54535c4..1bb4d47 100644
--- a/lib/ext2fs/alloc.c
+++ b/lib/ext2fs/alloc.c
@@ -184,9 +184,6 @@ errcode_t ext2fs_alloc_block2(ext2_filsys fs, blk64_t goal,
errcode_t retval;
blk64_t block;
- if (block_buf)
- memset(block_buf, 0, fs->blocksize);
^ permalink raw reply related [flat|nested] 104+ messages in thread* Re: [PATCH 09/47] libext2fs: use a dynamically sized (or caller-provided) block zeroing buffer
2014-12-13 16:24 ` Theodore Ts'o
2014-12-13 16:25 ` Theodore Ts'o
@ 2014-12-15 16:09 ` Andreas Dilger
1 sibling, 0 replies; 104+ messages in thread
From: Andreas Dilger @ 2014-12-15 16:09 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: Darrick J. Wong, linux-ext4
On Dec 13, 2014, at 09:24, Theodore Ts'o <tytso@mit.edu> wrote:
>
> This is my replacement patch.
>
> - Ted
>
> commit 0a92af260da381e2581074871707e487728373ad
> Author: Darrick J. Wong <darrick.wong@oracle.com>
> Date: Fri Dec 12 18:27:12 2014 -0500
>
> libext2fs: use a dynamically sized block zeroing buffer
>
> Dynamically grow the block zeroing buffer to a maximum of 4MB, and
> allow callers to provide their own zeroed buffer in
> ext2fs_zero_blocks2().
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
>
> diff --git a/lib/ext2fs/mkjournal.c b/lib/ext2fs/mkjournal.c
> index e8f8b30..fcc6741 100644
> --- a/lib/ext2fs/mkjournal.c
> +++ b/lib/ext2fs/mkjournal.c
> @@ -148,12 +148,13 @@ errfree:
> * attempt to free the static zeroizing buffer. (This is to keep
> * programs that check for memory leaks happy.)
> */
> -#define STRIDE_LENGTH (4194304 / fs->blocksize)
> +#define MAX_STRIDE_LENGTH (4194304 / (int) fs->blocksize)
> errcode_t ext2fs_zero_blocks2(ext2_filsys fs, blk64_t blk, int num,
> blk64_t *ret_blk, int *ret_count)
> {
> int j, count;
> - static char *buf;
> + static void *buf;
> + static int stride_length;
> errcode_t retval;
>
> /* If fs is null, clean up the static buffer and return */
> @@ -164,24 +165,36 @@ errcode_t ext2fs_zero_blocks2(ext2_filsys fs, blk64_t blk, int num,
> }
> return 0;
> }
> +
> + /* Deal with zeroing less than 1 block */
> + if (num <= 0)
> + return 0;
> +
> /* Allocate the zeroizing buffer if necessary */
> - if (!buf) {
> - buf = malloc(fs->blocksize * STRIDE_LENGTH);
> - if (!buf)
> - return ENOMEM;
> - memset(buf, 0, fs->blocksize * STRIDE_LENGTH);
> + if (num > stride_length && stride_length < MAX_STRIDE_LENGTH) {
> + void *p;
> + int new_stride = num;
> +
> + if (new_stride > MAX_STRIDE_LENGTH)
> + new_stride = MAX_STRIDE_LENGTH;
> + p = realloc(buf, fs->blocksize * new_stride);
> + if (!p)
> + return EXT2_ET_NO_MEMORY;
Shouldn't this continue using the original buf/num if this realloc fails?
Cheers, Andreas
> + buf = p;
> + stride_length = new_stride;
> + memset(buf, 0, fs->blocksize * stride_length);
> }
> /* OK, do the write loop */
> j=0;
> while (j < num) {
> - if (blk % STRIDE_LENGTH) {
> - count = STRIDE_LENGTH - (blk % STRIDE_LENGTH);
> + if (blk % stride_length) {
> + count = stride_length - (blk % stride_length);
> if (count > (num - j))
> count = num - j;
> } else {
> count = num - j;
> - if (count > STRIDE_LENGTH)
> - count = STRIDE_LENGTH;
> + if (count > stride_length)
> + count = stride_length;
> }
> retval = io_channel_write_blk64(fs->io, blk, count, buf);
> if (retval) {
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 104+ messages in thread
* [PATCH 10/47] libext2fs: support BLKZEROOUT/FALLOC_FL_ZERO_RANGE in ext2fs_zero_blocks
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (8 preceding siblings ...)
2014-11-07 21:51 ` [PATCH 09/47] libext2fs: use a dynamically sized (or caller-provided) block zeroing buffer Darrick J. Wong
@ 2014-11-07 21:51 ` Darrick J. Wong
2014-12-13 16:29 ` Theodore Ts'o
2014-11-07 21:51 ` [PATCH 11/47] libext2fs: find inode goal when allocating blocks Darrick J. Wong
` (38 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:51 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Plumb a new call into the IO manager to support translating
ext2fs_zero_blocks calls into the equivalent kernel-level BLKZEROOUT
ioctl or FALLOC_FL_ZERO_RANGE fallocate flag primitives when possible.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
lib/ext2fs/ext2_io.h | 7 ++-
lib/ext2fs/io_manager.c | 11 ++++
lib/ext2fs/mkjournal.c | 5 ++
lib/ext2fs/test_io.c | 21 ++++++++
lib/ext2fs/unix_io.c | 125 +++++++++++++++++++++++++++++++++++++++++++++++
5 files changed, 168 insertions(+), 1 deletion(-)
diff --git a/lib/ext2fs/ext2_io.h b/lib/ext2fs/ext2_io.h
index 4c5a5c5..1faa720 100644
--- a/lib/ext2fs/ext2_io.h
+++ b/lib/ext2fs/ext2_io.h
@@ -93,7 +93,9 @@ struct struct_io_manager {
errcode_t (*cache_readahead)(io_channel channel,
unsigned long long block,
unsigned long long count);
- long reserved[15];
+ errcode_t (*zeroout)(io_channel channel, unsigned long long block,
+ unsigned long long count);
+ long reserved[14];
};
#define IO_FLAG_RW 0x0001
@@ -125,6 +127,9 @@ extern errcode_t io_channel_write_blk64(io_channel channel,
extern errcode_t io_channel_discard(io_channel channel,
unsigned long long block,
unsigned long long count);
+extern errcode_t io_channel_zeroout(io_channel channel,
+ unsigned long long block,
+ unsigned long long count);
extern errcode_t io_channel_alloc_buf(io_channel channel,
int count, void *ptr);
extern errcode_t io_channel_cache_readahead(io_channel io,
diff --git a/lib/ext2fs/io_manager.c b/lib/ext2fs/io_manager.c
index dc5888d..c395d61 100644
--- a/lib/ext2fs/io_manager.c
+++ b/lib/ext2fs/io_manager.c
@@ -112,6 +112,17 @@ errcode_t io_channel_discard(io_channel channel, unsigned long long block,
return EXT2_ET_UNIMPLEMENTED;
}
+errcode_t io_channel_zeroout(io_channel channel, unsigned long long block,
+ unsigned long long count)
+{
+ EXT2_CHECK_MAGIC(channel, EXT2_ET_MAGIC_IO_CHANNEL);
+
+ if (channel->manager->zeroout)
+ return (channel->manager->zeroout)(channel, block, count);
+
+ return EXT2_ET_UNIMPLEMENTED;
+}
+
errcode_t io_channel_alloc_buf(io_channel io, int count, void *ptr)
{
size_t size;
diff --git a/lib/ext2fs/mkjournal.c b/lib/ext2fs/mkjournal.c
index fdd788b..2cd7ca5 100644
--- a/lib/ext2fs/mkjournal.c
+++ b/lib/ext2fs/mkjournal.c
@@ -171,6 +171,11 @@ errcode_t ext2fs_zero_blocks3(ext2_filsys fs, blk64_t blk, int num,
if (num <= 0)
return 0;
+ /* Try a zero out command, if supported */
+ retval = io_channel_zeroout(fs->io, blk, num);
+ if (retval == 0)
+ return 0;
+
/* If the user gave us a buffer, write that out */
if (block_buf) {
retval = io_channel_write_blk64(fs->io, blk, num, block_buf);
diff --git a/lib/ext2fs/test_io.c b/lib/ext2fs/test_io.c
index b03a939..f7c50d1 100644
--- a/lib/ext2fs/test_io.c
+++ b/lib/ext2fs/test_io.c
@@ -86,6 +86,7 @@ void (*test_io_cb_write_byte)
#define TEST_FLAG_SET_OPTION 0x20
#define TEST_FLAG_DISCARD 0x40
#define TEST_FLAG_READAHEAD 0x80
+#define TEST_FLAG_ZEROOUT 0x100
static void test_dump_block(io_channel channel,
struct test_private_data *data,
@@ -507,6 +508,25 @@ static errcode_t test_cache_readahead(io_channel channel,
return retval;
}
+static errcode_t test_zeroout(io_channel channel, unsigned long long block,
+ unsigned long long count)
+{
+ struct test_private_data *data;
+ errcode_t retval = 0;
+
+ EXT2_CHECK_MAGIC(channel, EXT2_ET_MAGIC_IO_CHANNEL);
+ data = (struct test_private_data *) channel->private_data;
+ EXT2_CHECK_MAGIC(data, EXT2_ET_MAGIC_TEST_IO_CHANNEL);
+
+ if (data->real)
+ retval = io_channel_zeroout(data->real, block, count);
+ if (data->flags & TEST_FLAG_ZEROOUT)
+ fprintf(data->outfile,
+ "Test_io: zeroout(%llu, %llu) returned %s\n",
+ block, count, retval ? error_message(retval) : "OK");
+ return retval;
+}
+
static struct struct_io_manager struct_test_manager = {
.magic = EXT2_ET_MAGIC_IO_MANAGER,
.name = "Test I/O Manager",
@@ -523,6 +543,7 @@ static struct struct_io_manager struct_test_manager = {
.write_blk64 = test_write_blk64,
.discard = test_discard,
.cache_readahead = test_cache_readahead,
+ .zeroout = test_zeroout,
};
io_manager test_io_manager = &struct_test_manager;
diff --git a/lib/ext2fs/unix_io.c b/lib/ext2fs/unix_io.c
index c3a8ea5..6bc543b 100644
--- a/lib/ext2fs/unix_io.c
+++ b/lib/ext2fs/unix_io.c
@@ -987,6 +987,130 @@ unimplemented:
return EXT2_ET_UNIMPLEMENTED;
}
+#if defined(__linux__) && !defined(BLKZEROOUT)
+#define BLKZEROOUT _IO(0x12, 127)
+#endif
+
+#if defined(__linux__) && !defined(BLKFLSBUF)
+#define BLKFLSBUF _IO(0x12, 97)
+#endif
+
+#if defined(__linux__) && !defined(FALLOC_FL_ZERO_RANGE)
+#define FALLOC_FL_ZERO_RANGE 0x10
+#endif
+
+static errcode_t unix_zeroout(io_channel channel, unsigned long long block,
+ unsigned long long count)
+{
+ struct unix_private_data *data;
+ int ret;
+
+ EXT2_CHECK_MAGIC(channel, EXT2_ET_MAGIC_IO_CHANNEL);
+ data = (struct unix_private_data *) channel->private_data;
+ EXT2_CHECK_MAGIC(data, EXT2_ET_MAGIC_UNIX_IO_CHANNEL);
+
+ if (getenv("UNIX_IO_NOZEROOUT"))
+ goto unimplemented;
+
+ if (channel->flags & CHANNEL_FLAGS_BLOCK_DEVICE) {
+ __u64 range[2];
+
+ if (count == 0)
+ return 0;
+ /*
+ * This is a heavyweight operation. Refuse to
+ * zero-out a single block, and hope that the
+ * caller will fall back to io_channel_write().
+ */
+ else if (count == 1)
+ goto unimplemented;
+
+ range[0] = (__u64)(block) * channel->block_size;
+ range[1] = (__u64)(count) * channel->block_size;
+
+ errno = EOPNOTSUPP;
+ ret = -1;
+#if defined(BLKZEROOUT) && (defined(POSIX_FADV_DONTNEED) || defined(BLKFLSBUF))
+ /*
+ * BLKZEROOUT doesn't invalidate the page cache. Therefore,
+ * we need to flush dirty pages to disk prior to calling the
+ * ioctl and invalidate at least those same pages from the page
+ * cache afterwards. If we're using direct IO we can skip all
+ * that wrangling.
+ */
+ if (!(data->flags & IO_FLAG_DIRECT_IO)) {
+ ret = fsync(data->dev);
+ if (ret)
+ goto err;
+ }
+ ret = ioctl(data->dev, BLKZEROOUT, range);
+ if (ret)
+ goto err;
+ if (!(data->flags & IO_FLAG_DIRECT_IO)) {
+#ifdef POSIX_FADV_DONTNEED
+ ret = posix_fadvise(data->dev, range[0], range[1],
+ POSIX_FADV_DONTNEED);
+ if (ret == 0)
+ goto err;
+#endif
+#ifdef BLKFLSBUF
+ ret = ioctl(data->dev, BLKFLSBUF, 0);
+#endif
+ }
+#elif defined(BLKZEROOUT)
+ if (data->flags & IO_FLAG_DIRECT_IO)
+ ret = ioctl(data->dev, BLKZEROOUT, range);
+#endif
+ } else {
+ /* Regular file, try to use truncate/punch/zero. */
+#if defined(HAVE_FALLOCATE) && (defined(FALLOC_FL_ZERO_RANGE) || \
+ (defined(FALLOC_FL_PUNCH_HOLE) && defined(FALLOC_FL_KEEP_SIZE)))
+ struct stat statbuf;
+
+ if (count == 0)
+ return 0;
+ /*
+ * If we're trying to zero a range past the end of the file,
+ * extend the file size, then punch (or zero_range) everything.
+ */
+ ret = fstat(data->dev, &statbuf);
+ if (ret)
+ goto err;
+ if (statbuf.st_size < (block + count) * channel->block_size) {
+ ret = ftruncate(data->dev,
+ (block + count) * channel->block_size);
+ if (ret)
+ goto err;
+ }
+#if defined(FALLOC_FL_PUNCH_HOLE) && defined(FALLOC_FL_KEEP_SIZE)
+ ret = fallocate(data->dev,
+ FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE,
+ (off_t)(block) * channel->block_size,
+ (off_t)(count) * channel->block_size);
+ if (ret == 0)
+ goto err;
+#endif
+#ifdef FALLOC_FL_ZERO_RANGE
+ ret = fallocate(data->dev,
+ FALLOC_FL_ZERO_RANGE,
+ (off_t)(block) * channel->block_size,
+ (off_t)(count) * channel->block_size);
+#endif
+#else
+ goto unimplemented;
+#endif /* HAVE_FALLOCATE && (ZERO_RANGE || (PUNCH_HOLE && KEEP_SIZE)) */
+ }
+err:
+ if (ret < 0) {
+ if (errno == EOPNOTSUPP)
+ goto unimplemented;
+ return errno;
+ }
+ return 0;
+unimplemented:
+ return EXT2_ET_UNIMPLEMENTED;
+}
+
static struct struct_io_manager struct_unix_manager = {
.magic = EXT2_ET_MAGIC_IO_MANAGER,
.name = "Unix I/O Manager",
@@ -1003,6 +1127,7 @@ static struct struct_io_manager struct_unix_manager = {
.write_blk64 = unix_write_blk64,
.discard = unix_discard,
.cache_readahead = unix_cache_readahead,
+ .zeroout = unix_zeroout,
};
io_manager unix_io_manager = &struct_unix_manager;
^ permalink raw reply related [flat|nested] 104+ messages in thread* Re: [PATCH 10/47] libext2fs: support BLKZEROOUT/FALLOC_FL_ZERO_RANGE in ext2fs_zero_blocks
2014-11-07 21:51 ` [PATCH 10/47] libext2fs: support BLKZEROOUT/FALLOC_FL_ZERO_RANGE in ext2fs_zero_blocks Darrick J. Wong
@ 2014-12-13 16:29 ` Theodore Ts'o
2014-12-14 3:08 ` Darrick J. Wong
0 siblings, 1 reply; 104+ messages in thread
From: Theodore Ts'o @ 2014-12-13 16:29 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Fri, Nov 07, 2014 at 01:51:52PM -0800, Darrick J. Wong wrote:
> + ret = ioctl(data->dev, BLKZEROOUT, range);
> + if (ret)
> + goto err;
> + if (!(data->flags & IO_FLAG_DIRECT_IO)) {
> +#ifdef POSIX_FADV_DONTNEED
> + ret = posix_fadvise(data->dev, range[0], range[1],
> + POSIX_FADV_DONTNEED);
> + if (ret == 0)
> + goto err;
> +#endif
> +#ifdef BLKFLSBUF
> + ret = ioctl(data->dev, BLKFLSBUF, 0);
> +#endif
Why do you need BLKFLSBUF after calling POSIX_FADV_DONTNEED? The
problem with BLKFLSBUF is that it has a special meaning for ramdisks,
dating back to when the ramdisk was stored only in the buffer cache, namely:
http://www.memegen.com/meme/hotw4i
We could add yet another special case for the ramdisk, but I'm
seriously wondering whether the use of BLKZEROOUT is worth it. Sure,
it will speed up mke2fs, but with the advent of metadata checksums, we
can skip clearing the journal and the inode table.
- Ted
^ permalink raw reply [flat|nested] 104+ messages in thread* Re: [PATCH 10/47] libext2fs: support BLKZEROOUT/FALLOC_FL_ZERO_RANGE in ext2fs_zero_blocks
2014-12-13 16:29 ` Theodore Ts'o
@ 2014-12-14 3:08 ` Darrick J. Wong
0 siblings, 0 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-12-14 3:08 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: linux-ext4
On Sat, Dec 13, 2014 at 11:29:17AM -0500, Theodore Ts'o wrote:
> On Fri, Nov 07, 2014 at 01:51:52PM -0800, Darrick J. Wong wrote:
> > + ret = ioctl(data->dev, BLKZEROOUT, range);
> > + if (ret)
> > + goto err;
> > + if (!(data->flags & IO_FLAG_DIRECT_IO)) {
> > +#ifdef POSIX_FADV_DONTNEED
> > + ret = posix_fadvise(data->dev, range[0], range[1],
> > + POSIX_FADV_DONTNEED);
> > + if (ret == 0)
> > + goto err;
> > +#endif
> > +#ifdef BLKFLSBUF
> > + ret = ioctl(data->dev, BLKFLSBUF, 0);
> > +#endif
>
> Why do you need BLKFLSBUF after calling POSIX_FADV_DONTNEED? The
I don't; if the DONTNEED call returns 0, we exit.
(The 'goto err' could be 'return 0' to be more clear, I suppose.)
--D
> problem with BLKFLSBUF is that it has a special meaning for ramdisks,
> dating back to when the ramdisk was stored only in the buffer cache, namely:
>
> http://www.memegen.com/meme/hotw4i
>
> We could add yet another special case for the ramdisk, but I'm
> seriously wondering whether the use of BLKZEROOUT is worth it. Sure,
> it will speed up mke2fs, but with the advent of metadata checksums, we
> can skip clearing the journal and the inode table.
>
> - Ted
^ permalink raw reply [flat|nested] 104+ messages in thread
* [PATCH 11/47] libext2fs: find inode goal when allocating blocks
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (9 preceding siblings ...)
2014-11-07 21:51 ` [PATCH 10/47] libext2fs: support BLKZEROOUT/FALLOC_FL_ZERO_RANGE in ext2fs_zero_blocks Darrick J. Wong
@ 2014-11-07 21:51 ` Darrick J. Wong
2014-12-14 1:13 ` Theodore Ts'o
2014-11-07 21:52 ` [PATCH 12/47] libext2fs: set interior tree block goal more intelligently Darrick J. Wong
` (37 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:51 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Try to be a little smarter about where we go to allocate blocks for a
inode. For a given inode and logical offset, set the goal as if the
file were physically continuous. If it's bmapped, just start looking
at wherever lblk 0 is. If that's not possible (the file has no
lblk>pblk mappings, inline data, etc.) then start looking in the
inode's block group.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
e2fsck/pass2.c | 3 ++-
lib/ext2fs/alloc.c | 43 ++++++++++++++++++++++++++++++++++++++++
lib/ext2fs/bmap.c | 5 +++--
lib/ext2fs/expanddir.c | 6 +++++-
lib/ext2fs/ext2fs.h | 2 ++
lib/ext2fs/ext_attr.c | 4 +---
lib/ext2fs/extent.c | 11 +++-------
lib/ext2fs/mkdir.c | 7 +++++--
lib/ext2fs/symlink.c | 7 +++++--
tests/f_extent_oobounds/script | 6 +++---
tests/f_extents2/expect.1 | 2 +-
11 files changed, 73 insertions(+), 23 deletions(-)
diff --git a/e2fsck/pass2.c b/e2fsck/pass2.c
index c354e4a..cd0762a 100644
--- a/e2fsck/pass2.c
+++ b/e2fsck/pass2.c
@@ -1772,7 +1772,8 @@ static int allocate_dir_block(e2fsck_t ctx,
pctx->errcode = ext2fs_map_cluster_block(fs, db->ino, &inode,
db->blockcnt, &blk);
if (pctx->errcode || blk == 0) {
- pctx->errcode = ext2fs_new_block2(fs, 0,
+ blk = ext2fs_find_inode_goal(fs, db->ino, &inode, db->blockcnt);
+ pctx->errcode = ext2fs_new_block2(fs, blk,
ctx->block_found_map, &blk);
if (pctx->errcode) {
pctx->str = "ext2fs_new_block";
diff --git a/lib/ext2fs/alloc.c b/lib/ext2fs/alloc.c
index cb778f7..fda45f9 100644
--- a/lib/ext2fs/alloc.c
+++ b/lib/ext2fs/alloc.c
@@ -288,3 +288,46 @@ void ext2fs_set_alloc_block_callback(ext2_filsys fs,
fs->get_alloc_block = func;
}
+
+blk64_t ext2fs_find_inode_goal(ext2_filsys fs, ext2_ino_t ino,
+ struct ext2_inode *inode, blk64_t lblk)
+{
+ dgrp_t group;
+ __u8 log_flex;
+ struct ext2fs_extent extent;
+ ext2_extent_handle_t handle;
+ errcode_t err;
+
+ if (inode == NULL || ext2fs_inode_data_blocks2(fs, inode) == 0)
+ goto no_blocks;
+
+ if (inode->i_flags & EXT4_INLINE_DATA_FL)
+ goto no_blocks;
+
+ if (inode->i_flags & EXT4_EXTENTS_FL) {
+ err = ext2fs_extent_open2(fs, ino, inode, &handle);
+ if (err)
+ goto no_blocks;
+ err = ext2fs_extent_goto2(handle, 0, lblk);
+ if (err)
+ goto extent_err;
+ err = ext2fs_extent_get(handle, EXT2_EXTENT_CURRENT, &extent);
+ if (err)
+ goto extent_err;
+ return extent.e_pblk + (lblk - extent.e_lblk);
+extent_err:
+ ext2fs_extent_free(handle);
+ goto no_blocks;
+ }
+
+ /* block mapped file; see if block zero is mapped? */
+ if (inode->i_block[0])
+ return inode->i_block[0];
+
+no_blocks:
+ log_flex = fs->super->s_log_groups_per_flex;
+ group = ext2fs_group_of_ino(fs, ino);
+ if (log_flex)
+ group = group & ~((1 << (log_flex)) - 1);
+ return ext2fs_group_first_block2(fs, group);
+}
diff --git a/lib/ext2fs/bmap.c b/lib/ext2fs/bmap.c
index f819e9c..cb3f5a1 100644
--- a/lib/ext2fs/bmap.c
+++ b/lib/ext2fs/bmap.c
@@ -244,7 +244,7 @@ got_block:
retval = extent_bmap(fs, ino, inode, handle, block_buf,
0, block-1, 0, blocks_alloc, &blk64);
if (retval)
- blk64 = 0;
+ blk64 = ext2fs_find_inode_goal(fs, ino, inode, block);
retval = ext2fs_alloc_block2(fs, blk64, block_buf,
&blk64);
if (retval)
@@ -354,7 +354,8 @@ errcode_t ext2fs_bmap2(ext2_filsys fs, ext2_ino_t ino, struct ext2_inode *inode,
}
*phys_blk = inode_bmap(inode, block);
- b = block ? inode_bmap(inode, block-1) : 0;
+ b = block ? inode_bmap(inode, block - 1) :
+ ext2fs_find_inode_goal(fs, ino, inode, block);
if ((*phys_blk == 0) && (bmap_flags & BMAP_ALLOC)) {
retval = ext2fs_alloc_block(fs, b, block_buf, &b);
diff --git a/lib/ext2fs/expanddir.c b/lib/ext2fs/expanddir.c
index ecc13ae..9f02312 100644
--- a/lib/ext2fs/expanddir.c
+++ b/lib/ext2fs/expanddir.c
@@ -102,9 +102,13 @@ errcode_t ext2fs_expand_dir(ext2_filsys fs, ext2_ino_t dir)
if (retval)
return retval;
+ retval = ext2fs_read_inode(fs, dir, &inode);
+ if (retval)
+ return retval;
+
es.done = 0;
es.err = 0;
- es.goal = 0;
+ es.goal = ext2fs_find_inode_goal(fs, dir, &inode, 0);
es.newblocks = 0;
es.dir = dir;
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index f9599a3..afe4fd5 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -689,6 +689,8 @@ extern void ext2fs_set_alloc_block_callback(ext2_filsys fs,
errcode_t (**old)(ext2_filsys fs,
blk64_t goal,
blk64_t *ret));
+blk64_t ext2fs_find_inode_goal(ext2_filsys fs, ext2_ino_t ino,
+ struct ext2_inode *inode, blk64_t lblk);
/* alloc_sb.c */
extern int ext2fs_reserve_super_and_bgd(ext2_filsys fs,
diff --git a/lib/ext2fs/ext_attr.c b/lib/ext2fs/ext_attr.c
index c397e00..70bc3f9 100644
--- a/lib/ext2fs/ext_attr.c
+++ b/lib/ext2fs/ext_attr.c
@@ -374,7 +374,6 @@ static errcode_t prep_ea_block_for_write(ext2_filsys fs, ext2_ino_t ino,
{
struct ext2_ext_attr_header *header;
void *block_buf = NULL;
- dgrp_t grp;
blk64_t blk, goal;
errcode_t err;
@@ -420,8 +419,7 @@ static errcode_t prep_ea_block_for_write(ext2_filsys fs, ext2_ino_t ino,
}
/* Allocate a block */
- grp = ext2fs_group_of_ino(fs, ino);
- goal = ext2fs_inode_table_loc(fs, grp);
+ goal = ext2fs_find_inode_goal(fs, ino, (struct ext2_inode *)inode, 0);
err = ext2fs_alloc_block2(fs, goal, NULL, &blk);
if (err)
goto out2;
diff --git a/lib/ext2fs/extent.c b/lib/ext2fs/extent.c
index ca5b78b..b6e4fbd 100644
--- a/lib/ext2fs/extent.c
+++ b/lib/ext2fs/extent.c
@@ -1012,14 +1012,9 @@ static errcode_t extent_node_split(ext2_extent_handle_t handle,
goto done;
}
- if (!goal_blk) {
- dgrp_t group = ext2fs_group_of_ino(handle->fs, handle->ino);
- __u8 log_flex = handle->fs->super->s_log_groups_per_flex;
-
- if (log_flex)
- group = group & ~((1 << (log_flex)) - 1);
- goal_blk = ext2fs_group_first_block2(handle->fs, group);
- }
+ if (!goal_blk)
+ goal_blk = ext2fs_find_inode_goal(handle->fs, handle->ino,
+ handle->inode, 0);
retval = ext2fs_alloc_block2(handle->fs, goal_blk, block_buf,
&new_node_pblk);
if (retval)
diff --git a/lib/ext2fs/mkdir.c b/lib/ext2fs/mkdir.c
index c4c7967..433f3b4 100644
--- a/lib/ext2fs/mkdir.c
+++ b/lib/ext2fs/mkdir.c
@@ -68,8 +68,12 @@ errcode_t ext2fs_mkdir(ext2_filsys fs, ext2_ino_t parent, ext2_ino_t inum,
/*
* Allocate a data block for the directory
*/
+ memset(&inode, 0, sizeof(struct ext2_inode));
if (!inline_data) {
- retval = ext2fs_new_block2(fs, 0, 0, &blk);
+ retval = ext2fs_new_block2(fs, ext2fs_find_inode_goal(fs, ino,
+ &inode,
+ 0),
+ NULL, &blk);
if (retval)
goto cleanup;
}
@@ -77,7 +81,6 @@ errcode_t ext2fs_mkdir(ext2_filsys fs, ext2_ino_t parent, ext2_ino_t inum,
/*
* Create a scratch template for the directory
*/
- memset(&inode, 0, sizeof(struct ext2_inode));
if (inline_data)
retval = ext2fs_new_dir_inline_data(fs, ino, parent,
inode.i_block);
diff --git a/lib/ext2fs/symlink.c b/lib/ext2fs/symlink.c
index f6eb6b6..0732afe 100644
--- a/lib/ext2fs/symlink.c
+++ b/lib/ext2fs/symlink.c
@@ -51,9 +51,13 @@ errcode_t ext2fs_symlink(ext2_filsys fs, ext2_ino_t parent, ext2_ino_t ino,
/*
* Allocate a data block for slow links
*/
+ memset(&inode, 0, sizeof(struct ext2_inode));
fastlink = (target_len < sizeof(inode.i_block));
if (!fastlink) {
- retval = ext2fs_new_block2(fs, 0, 0, &blk);
+ retval = ext2fs_new_block2(fs, ext2fs_find_inode_goal(fs, ino,
+ &inode,
+ 0),
+ NULL, &blk);
if (retval)
goto cleanup;
retval = ext2fs_get_mem(fs->blocksize, &block_buf);
@@ -74,7 +78,6 @@ errcode_t ext2fs_symlink(ext2_filsys fs, ext2_ino_t parent, ext2_ino_t ino,
/*
* Create the inode structure....
*/
- memset(&inode, 0, sizeof(struct ext2_inode));
inode.i_mode = LINUX_S_IFLNK | 0777;
inode.i_uid = inode.i_gid = 0;
inode.i_links_count = 1;
diff --git a/tests/f_extent_oobounds/script b/tests/f_extent_oobounds/script
index b00b031..54674ca 100644
--- a/tests/f_extent_oobounds/script
+++ b/tests/f_extent_oobounds/script
@@ -7,6 +7,9 @@ dd if=/dev/zero of=$TMPFILE bs=1k count=256 > /dev/null 2>&1
$MKE2FS -Ft ext4 $TMPFILE > /dev/null 2>&1
$DEBUGFS -w $TMPFILE << EOF > /dev/null 2>&1
write /dev/null testfile
+setb 100 15
+setb 130 30
+setb 200 30
extent_open testfile
insert_node 0 15 100
insert_node --after 15 15 115
@@ -22,9 +25,6 @@ extent_open testfile
extent_close
set_inode_field testfile i_size 61400
set_inode_field testfile i_blocks 154
-setb 100 15
-setb 130 30
-setb 200 30
set_bg 0 free_blocks_count 156
set_bg 0 bg_checksum calc
set_super_value free_blocks_count 156
diff --git a/tests/f_extents2/expect.1 b/tests/f_extents2/expect.1
index fa7f6eb..51e36ff 100644
--- a/tests/f_extents2/expect.1
+++ b/tests/f_extents2/expect.1
@@ -55,7 +55,7 @@ Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
-Block bitmap differences: -(26--34) -(154--199)
+Block bitmap differences: -(25--34) -(155--199)
Fix? yes
Free blocks count wrong for group #0 (65535, counted=55).
^ permalink raw reply related [flat|nested] 104+ messages in thread* Re: [PATCH 11/47] libext2fs: find inode goal when allocating blocks
2014-11-07 21:51 ` [PATCH 11/47] libext2fs: find inode goal when allocating blocks Darrick J. Wong
@ 2014-12-14 1:13 ` Theodore Ts'o
2014-12-14 21:02 ` Darrick J. Wong
0 siblings, 1 reply; 104+ messages in thread
From: Theodore Ts'o @ 2014-12-14 1:13 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Fri, Nov 07, 2014 at 01:51:59PM -0800, Darrick J. Wong wrote:
> + if (inode->i_flags & EXT4_EXTENTS_FL) {
> + err = ext2fs_extent_open2(fs, ino, inode, &handle);
> + if (err)
> + goto no_blocks;
> + err = ext2fs_extent_goto2(handle, 0, lblk);
> + if (err)
> + goto extent_err;
> + err = ext2fs_extent_get(handle, EXT2_EXTENT_CURRENT, &extent);
> + if (err)
> + goto extent_err;
> + return extent.e_pblk + (lblk - extent.e_lblk);
There's a memory leak here; the extent handle never gets freed. I've
checked this in with the attached fix up patched.
BTW, a really useful way of checking for memory leaks and which
allowed me to confirm the problem after noticing the problem from
reviewing the patch (and showing that with the attached patch below
the leak went away):
cd tests; ./test_script --valgrind-leakcheck f_extents f_extents2
more /tmp/valgrind-*
Cheers,
- Ted
diff --git a/lib/ext2fs/alloc.c b/lib/ext2fs/alloc.c
index 57bf5f0..62b36fe 100644
--- a/lib/ext2fs/alloc.c
+++ b/lib/ext2fs/alloc.c
@@ -296,7 +296,7 @@ blk64_t ext2fs_find_inode_goal(ext2_filsys fs, ext2_ino_t ino,
dgrp_t group;
__u8 log_flex;
struct ext2fs_extent extent;
- ext2_extent_handle_t handle;
+ ext2_extent_handle_t handle = NULL;
errcode_t err;
if (inode == NULL || ext2fs_inode_data_blocks2(fs, inode) == 0)
@@ -311,14 +311,12 @@ blk64_t ext2fs_find_inode_goal(ext2_filsys fs, ext2_ino_t ino,
goto no_blocks;
err = ext2fs_extent_goto2(handle, 0, lblk);
if (err)
- goto extent_err;
+ goto no_blocks;
err = ext2fs_extent_get(handle, EXT2_EXTENT_CURRENT, &extent);
if (err)
- goto extent_err;
- return extent.e_pblk + (lblk - extent.e_lblk);
-extent_err:
+ goto no_blocks;
ext2fs_extent_free(handle);
- goto no_blocks;
+ return extent.e_pblk + (lblk - extent.e_lblk);
}
/* block mapped file; see if block zero is mapped? */
@@ -326,6 +324,7 @@ extent_err:
return inode->i_block[0];
no_blocks:
+ ext2fs_extent_free(handle);
log_flex = fs->super->s_log_groups_per_flex;
group = ext2fs_group_of_ino(fs, ino);
if (log_flex)
^ permalink raw reply related [flat|nested] 104+ messages in thread* Re: [PATCH 11/47] libext2fs: find inode goal when allocating blocks
2014-12-14 1:13 ` Theodore Ts'o
@ 2014-12-14 21:02 ` Darrick J. Wong
0 siblings, 0 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-12-14 21:02 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: linux-ext4
On Sat, Dec 13, 2014 at 08:13:27PM -0500, Theodore Ts'o wrote:
> On Fri, Nov 07, 2014 at 01:51:59PM -0800, Darrick J. Wong wrote:
> > + if (inode->i_flags & EXT4_EXTENTS_FL) {
> > + err = ext2fs_extent_open2(fs, ino, inode, &handle);
> > + if (err)
> > + goto no_blocks;
> > + err = ext2fs_extent_goto2(handle, 0, lblk);
> > + if (err)
> > + goto extent_err;
> > + err = ext2fs_extent_get(handle, EXT2_EXTENT_CURRENT, &extent);
> > + if (err)
> > + goto extent_err;
> > + return extent.e_pblk + (lblk - extent.e_lblk);
>
> There's a memory leak here; the extent handle never gets freed. I've
> checked this in with the attached fix up patched.
>
> BTW, a really useful way of checking for memory leaks and which
> allowed me to confirm the problem after noticing the problem from
> reviewing the patch (and showing that with the attached patch below
> the leak went away):
>
> cd tests; ./test_script --valgrind-leakcheck f_extents f_extents2
> more /tmp/valgrind-*
Oops, thank you for catching this one. I periodically run make check with
USE_VALGRIND, forgot to do so. :/
--D
>
> Cheers,
>
> - Ted
>
> diff --git a/lib/ext2fs/alloc.c b/lib/ext2fs/alloc.c
> index 57bf5f0..62b36fe 100644
> --- a/lib/ext2fs/alloc.c
> +++ b/lib/ext2fs/alloc.c
> @@ -296,7 +296,7 @@ blk64_t ext2fs_find_inode_goal(ext2_filsys fs, ext2_ino_t ino,
> dgrp_t group;
> __u8 log_flex;
> struct ext2fs_extent extent;
> - ext2_extent_handle_t handle;
> + ext2_extent_handle_t handle = NULL;
> errcode_t err;
>
> if (inode == NULL || ext2fs_inode_data_blocks2(fs, inode) == 0)
> @@ -311,14 +311,12 @@ blk64_t ext2fs_find_inode_goal(ext2_filsys fs, ext2_ino_t ino,
> goto no_blocks;
> err = ext2fs_extent_goto2(handle, 0, lblk);
> if (err)
> - goto extent_err;
> + goto no_blocks;
> err = ext2fs_extent_get(handle, EXT2_EXTENT_CURRENT, &extent);
> if (err)
> - goto extent_err;
> - return extent.e_pblk + (lblk - extent.e_lblk);
> -extent_err:
> + goto no_blocks;
> ext2fs_extent_free(handle);
> - goto no_blocks;
> + return extent.e_pblk + (lblk - extent.e_lblk);
> }
>
> /* block mapped file; see if block zero is mapped? */
> @@ -326,6 +324,7 @@ extent_err:
> return inode->i_block[0];
>
> no_blocks:
> + ext2fs_extent_free(handle);
> log_flex = fs->super->s_log_groups_per_flex;
> group = ext2fs_group_of_ino(fs, ino);
> if (log_flex)
^ permalink raw reply [flat|nested] 104+ messages in thread
* [PATCH 12/47] libext2fs: set interior tree block goal more intelligently
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (10 preceding siblings ...)
2014-11-07 21:51 ` [PATCH 11/47] libext2fs: find inode goal when allocating blocks Darrick J. Wong
@ 2014-11-07 21:52 ` Darrick J. Wong
2014-12-14 1:17 ` Theodore Ts'o
2014-11-07 21:52 ` [PATCH 13/47] libext2fs: add a way to check the theoretical maximum extent tree depth Darrick J. Wong
` (36 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:52 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
When we're splitting an extent node, try to allocate the new interior
tree block just prior to the first extent in the block we're trying to
split. The previous logic only set a goal block if we had to split
both the current node and its parent, which is somewhat infrequent.
When that would happen, the goal would start at zero, leading to poor
locality.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
lib/ext2fs/extent.c | 20 +++++++++++++++++++-
tests/f_extents2/expect.1 | 2 +-
tests/m_bigjournal/expect.1 | 10 +++++-----
3 files changed, 25 insertions(+), 7 deletions(-)
diff --git a/lib/ext2fs/extent.c b/lib/ext2fs/extent.c
index b6e4fbd..afb79c8 100644
--- a/lib/ext2fs/extent.c
+++ b/lib/ext2fs/extent.c
@@ -934,6 +934,25 @@ static errcode_t extent_node_split(ext2_extent_handle_t handle,
orig_height = info.max_depth - info.curr_level;
orig_lblk = extent.e_lblk;
+ /* Try to put the index block before the first extent */
+ path = handle->path + handle->level;
+ eh = (struct ext3_extent_header *) path->buf;
+ if (handle->level == handle->max_depth) {
+ struct ext3_extent *ex;
+
+ ex = EXT_FIRST_EXTENT(eh);
+ goal_blk = ext2fs_le32_to_cpu(ex->ee_start) +
+ ((__u64) ext2fs_le16_to_cpu(ex->ee_start_hi) << 32);
+ } else {
+ struct ext3_extent_idx *ix;
+
+ ix = EXT_FIRST_INDEX(eh);
+ goal_blk = ext2fs_le32_to_cpu(ix->ei_leaf) +
+ ((__u64) ext2fs_le16_to_cpu(ix->ei_leaf_hi) << 32);
+ }
+ goal_blk -= EXT2FS_CLUSTER_RATIO(handle->fs);
+ goal_blk &= ~EXT2FS_CLUSTER_MASK(handle->fs);
+
/* Is there room in the parent for a new entry? */
if (handle->level &&
(handle->path[handle->level - 1].entries >=
@@ -947,7 +966,6 @@ static errcode_t extent_node_split(ext2_extent_handle_t handle,
retval = ext2fs_extent_get(handle, EXT2_EXTENT_UP, &extent);
if (retval)
goto done;
- goal_blk = extent.e_pblk;
retval = extent_node_split(handle, expand_allowed);
if (retval)
diff --git a/tests/f_extents2/expect.1 b/tests/f_extents2/expect.1
index 51e36ff..180568b 100644
--- a/tests/f_extents2/expect.1
+++ b/tests/f_extents2/expect.1
@@ -55,7 +55,7 @@ Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
-Block bitmap differences: -(25--34) -(155--199)
+Block bitmap differences: -(25--33) -(154--199)
Fix? yes
Free blocks count wrong for group #0 (65535, counted=55).
diff --git a/tests/m_bigjournal/expect.1 b/tests/m_bigjournal/expect.1
index 7246739..61d85f9 100644
--- a/tests/m_bigjournal/expect.1
+++ b/tests/m_bigjournal/expect.1
@@ -52,8 +52,8 @@ Group 0: (Blocks 0-32767)
Reserved GDT blocks at 2-672
Block bitmap at 673 (+673), Inode bitmap at 757 (+757)
Inode table at 841-841 (+841)
- 31836 free blocks, 5 free inodes, 2 directories, 5 unused inodes
- Free blocks: 932-32767
+ 31837 free blocks, 5 free inodes, 2 directories, 5 unused inodes
+ Free blocks: 931-32767
Free inodes: 12-16
Group 1: (Blocks 32768-65535) [INODE_UNINIT, BLOCK_UNINIT]
Backup superblock at 32768, Group descriptors at 32769-32769
@@ -297,11 +297,11 @@ Group 38: (Blocks 1245184-1277951) [INODE_UNINIT, BLOCK_UNINIT]
32768 free blocks, 16 free inodes, 0 directories, 16 unused inodes
Free blocks: 1245184-1277951
Free inodes: 609-624
-Group 39: (Blocks 1277952-1310719) [INODE_UNINIT, BLOCK_UNINIT]
+Group 39: (Blocks 1277952-1310719) [INODE_UNINIT]
Block bitmap at 712 (bg #0 + 712), Inode bitmap at 796 (bg #0 + 796)
Inode table at 880-880 (bg #0 + 880)
- 32768 free blocks, 16 free inodes, 0 directories, 16 unused inodes
- Free blocks: 1277952-1310719
+ 32767 free blocks, 16 free inodes, 0 directories, 16 unused inodes
+ Free blocks: 1277952-1310718
Free inodes: 625-640
Group 40: (Blocks 1310720-1343487) [INODE_UNINIT]
Block bitmap at 713 (bg #0 + 713), Inode bitmap at 797 (bg #0 + 797)
^ permalink raw reply related [flat|nested] 104+ messages in thread* Re: [PATCH 12/47] libext2fs: set interior tree block goal more intelligently
2014-11-07 21:52 ` [PATCH 12/47] libext2fs: set interior tree block goal more intelligently Darrick J. Wong
@ 2014-12-14 1:17 ` Theodore Ts'o
0 siblings, 0 replies; 104+ messages in thread
From: Theodore Ts'o @ 2014-12-14 1:17 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Fri, Nov 07, 2014 at 01:52:05PM -0800, Darrick J. Wong wrote:
> When we're splitting an extent node, try to allocate the new interior
> tree block just prior to the first extent in the block we're trying to
> split. The previous logic only set a goal block if we had to split
> both the current node and its parent, which is somewhat infrequent.
> When that would happen, the goal would start at zero, leading to poor
> locality.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Thanks, applied.
- Ted
^ permalink raw reply [flat|nested] 104+ messages in thread
* [PATCH 13/47] libext2fs: add a way to check the theoretical maximum extent tree depth
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (11 preceding siblings ...)
2014-11-07 21:52 ` [PATCH 12/47] libext2fs: set interior tree block goal more intelligently Darrick J. Wong
@ 2014-11-07 21:52 ` Darrick J. Wong
2014-12-14 2:23 ` Theodore Ts'o
2014-11-07 21:52 ` [PATCH 14/47] libext2fs: ext2fs_new_block2() should call alloc_block hook Darrick J. Wong
` (35 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:52 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Add an API so that client programs can discover a reasonable maximum
extent tree depth. This will eventually be used by e2fsck as one of
the criteria to decide if an extent-based file should have its extent
tree rebuilt.
Turn some related magic numbers into constants while we're at it.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
lib/ext2fs/ext2fs.h | 1 +
lib/ext2fs/ext3_extents.h | 2 ++
lib/ext2fs/extent.c | 33 +++++++++++++++++++++++++++++----
3 files changed, 32 insertions(+), 4 deletions(-)
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index afe4fd5..c3142f7 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -1218,6 +1218,7 @@ extern errcode_t ext2fs_extent_goto(ext2_extent_handle_t handle,
extern errcode_t ext2fs_extent_goto2(ext2_extent_handle_t handle,
int leaf_level, blk64_t blk);
extern errcode_t ext2fs_extent_fix_parents(ext2_extent_handle_t handle);
+size_t ext2fs_max_extent_depth(ext2_extent_handle_t handle);
/* fileio.c */
extern errcode_t ext2fs_file_open2(ext2_filsys fs, ext2_ino_t ino,
diff --git a/lib/ext2fs/ext3_extents.h b/lib/ext2fs/ext3_extents.h
index 66c23fd..f73a332 100644
--- a/lib/ext2fs/ext3_extents.h
+++ b/lib/ext2fs/ext3_extents.h
@@ -98,6 +98,8 @@ struct ext3_ext_path {
*/
#define EXT_INIT_MAX_LEN (1UL << 15)
#define EXT_UNINIT_MAX_LEN (EXT_INIT_MAX_LEN - 1)
+#define EXT_MAX_EXTENT_LBLK (((__u64) 1 << 32) - 1)
+#define EXT_MAX_EXTENT_PBLK (((__u64) 1 << 48) - 1)
#define EXT_FIRST_EXTENT(__hdr__) \
((struct ext3_extent *) (((char *) (__hdr__)) + \
diff --git a/lib/ext2fs/extent.c b/lib/ext2fs/extent.c
index afb79c8..b1130f6 100644
--- a/lib/ext2fs/extent.c
+++ b/lib/ext2fs/extent.c
@@ -1697,14 +1697,39 @@ errcode_t ext2fs_extent_get_info(ext2_extent_handle_t handle,
info->curr_level = handle->level;
info->max_depth = handle->max_depth;
- info->max_lblk = ((__u64) 1 << 32) - 1;
- info->max_pblk = ((__u64) 1 << 48) - 1;
- info->max_len = (1UL << 15);
- info->max_uninit_len = (1UL << 15) - 1;
+ info->max_lblk = EXT_MAX_EXTENT_LBLK;
+ info->max_pblk = EXT_MAX_EXTENT_PBLK;
+ info->max_len = EXT_INIT_MAX_LEN;
+ info->max_uninit_len = EXT_UNINIT_MAX_LEN;
return 0;
}
+static int ul_log2(unsigned long arg)
+{
+ int l = 0;
+
+ arg >>= 1;
+ while (arg) {
+ l++;
+ arg >>= 1;
+ }
+ return l;
+}
+
+size_t ext2fs_max_extent_depth(ext2_extent_handle_t handle)
+{
+ size_t iblock_sz = sizeof(((struct ext2_inode *)NULL)->i_block);
+ size_t iblock_extents = (iblock_sz - sizeof(struct ext3_extent_header)) /
+ sizeof(struct ext3_extent);
+ size_t extents_per_block = (handle->fs->blocksize -
+ sizeof(struct ext3_extent_header)) /
+ sizeof(struct ext3_extent);
+
+ return 1 + ((ul_log2(EXT_MAX_EXTENT_LBLK) - ul_log2(iblock_extents)) /
+ ul_log2(extents_per_block));
+}
+
#ifdef DEBUG
/*
* Override debugfs's prompt
^ permalink raw reply related [flat|nested] 104+ messages in thread* Re: [PATCH 13/47] libext2fs: add a way to check the theoretical maximum extent tree depth
2014-11-07 21:52 ` [PATCH 13/47] libext2fs: add a way to check the theoretical maximum extent tree depth Darrick J. Wong
@ 2014-12-14 2:23 ` Theodore Ts'o
2014-12-14 3:11 ` Darrick J. Wong
0 siblings, 1 reply; 104+ messages in thread
From: Theodore Ts'o @ 2014-12-14 2:23 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Fri, Nov 07, 2014 at 01:52:12PM -0800, Darrick J. Wong wrote:
> Add an API so that client programs can discover a reasonable maximum
> extent tree depth. This will eventually be used by e2fsck as one of
> the criteria to decide if an extent-based file should have its extent
> tree rebuilt.
>
> +size_t ext2fs_max_extent_depth(ext2_extent_handle_t handle)
> +{
> + size_t iblock_sz = sizeof(((struct ext2_inode *)NULL)->i_block);
> + size_t iblock_extents = (iblock_sz - sizeof(struct ext3_extent_header)) /
> + sizeof(struct ext3_extent);
> + size_t extents_per_block = (handle->fs->blocksize -
> + sizeof(struct ext3_extent_header)) /
> + sizeof(struct ext3_extent);
> +
> + return 1 + ((ul_log2(EXT_MAX_EXTENT_LBLK) - ul_log2(iblock_extents)) /
> + ul_log2(extents_per_block));
> +}
> +
This patch is only used by patch #33, and it gets called for every
single inode which is extent-mapped.
But if you look at the above code, there is nothing which is inode or
handle specific. The value is only dependent on fs->blocksize. But
to calculate this value the function calls ul_log2 three times, which
is implemented using a looping construct. So we will be recalculating
what is effectively a constant value every single inode in an ext4
file system, which seems very unfortunate.
BTW, there are some ways we could accelerate the log2 function, and
I've considered implementing an ext2fs_log2_u32() and
ext2fs_log2_u64() using something like this:
int log2i(unsigned int x)
{
if (!x)
return 0;
return sizeof(unsigned int) * 8 - __builtin_clz(x) - 1;
}
or
int log2u64(unsigned long long x)
{
if (!x)
return 0;
return sizeof(unsigned long long) * 8 - __builtin_clzll(x) - 1;
}
or
int generic_log2(unsigned int x)
{
int r = 0;
if (x >= ((unsigned int)(1) << 16)) {
r += 16;
x >>= 16;
}
if (x >= ((unsigned int)(1) << 8)) {
r += 8;
x >>= 8;
}
if (x >= ((unsigned int)(1) << 4)) {
r += 4;
x >>= 4;
}
if (x >= ((unsigned int)(1) << 2)) {
r += 2;
x >>= 2;
}
if (x >= ((unsigned int)(1) << 1)) {
r += 1;
x >>= 1;
}
return r;
}
But it's not at all clear it's worth doing this because I haven't seen
any place in e2fsprogs where we really should be calculating an
integer log2 in a tight loop where performance should make a
difference. If we are, we're probably doing something wrong....
- Ted
^ permalink raw reply [flat|nested] 104+ messages in thread* Re: [PATCH 13/47] libext2fs: add a way to check the theoretical maximum extent tree depth
2014-12-14 2:23 ` Theodore Ts'o
@ 2014-12-14 3:11 ` Darrick J. Wong
2014-12-15 3:48 ` Theodore Ts'o
0 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-12-14 3:11 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: linux-ext4
On Sat, Dec 13, 2014 at 09:23:57PM -0500, Theodore Ts'o wrote:
> On Fri, Nov 07, 2014 at 01:52:12PM -0800, Darrick J. Wong wrote:
> > Add an API so that client programs can discover a reasonable maximum
> > extent tree depth. This will eventually be used by e2fsck as one of
> > the criteria to decide if an extent-based file should have its extent
> > tree rebuilt.
> >
> > +size_t ext2fs_max_extent_depth(ext2_extent_handle_t handle)
> > +{
> > + size_t iblock_sz = sizeof(((struct ext2_inode *)NULL)->i_block);
> > + size_t iblock_extents = (iblock_sz - sizeof(struct ext3_extent_header)) /
> > + sizeof(struct ext3_extent);
> > + size_t extents_per_block = (handle->fs->blocksize -
> > + sizeof(struct ext3_extent_header)) /
> > + sizeof(struct ext3_extent);
> > +
> > + return 1 + ((ul_log2(EXT_MAX_EXTENT_LBLK) - ul_log2(iblock_extents)) /
> > + ul_log2(extents_per_block));
> > +}
> > +
>
> This patch is only used by patch #33, and it gets called for every
> single inode which is extent-mapped.
>
> But if you look at the above code, there is nothing which is inode or
> handle specific. The value is only dependent on fs->blocksize. But
> to calculate this value the function calls ul_log2 three times, which
> is implemented using a looping construct. So we will be recalculating
> what is effectively a constant value every single inode in an ext4
> file system, which seems very unfortunate.
How about I save the last fs->blocksize and the last result, and return the
cached last result if fs->blocksize == lastblocksize?
--D
>
> BTW, there are some ways we could accelerate the log2 function, and
> I've considered implementing an ext2fs_log2_u32() and
> ext2fs_log2_u64() using something like this:
>
> int log2i(unsigned int x)
> {
> if (!x)
> return 0;
>
> return sizeof(unsigned int) * 8 - __builtin_clz(x) - 1;
> }
>
> or
>
> int log2u64(unsigned long long x)
> {
> if (!x)
> return 0;
>
> return sizeof(unsigned long long) * 8 - __builtin_clzll(x) - 1;
> }
>
> or
>
> int generic_log2(unsigned int x)
> {
> int r = 0;
>
> if (x >= ((unsigned int)(1) << 16)) {
> r += 16;
> x >>= 16;
> }
> if (x >= ((unsigned int)(1) << 8)) {
> r += 8;
> x >>= 8;
> }
> if (x >= ((unsigned int)(1) << 4)) {
> r += 4;
> x >>= 4;
> }
> if (x >= ((unsigned int)(1) << 2)) {
> r += 2;
> x >>= 2;
> }
> if (x >= ((unsigned int)(1) << 1)) {
> r += 1;
> x >>= 1;
> }
> return r;
> }
>
> But it's not at all clear it's worth doing this because I haven't seen
> any place in e2fsprogs where we really should be calculating an
> integer log2 in a tight loop where performance should make a
> difference. If we are, we're probably doing something wrong....
>
> - Ted
^ permalink raw reply [flat|nested] 104+ messages in thread* Re: [PATCH 13/47] libext2fs: add a way to check the theoretical maximum extent tree depth
2014-12-14 3:11 ` Darrick J. Wong
@ 2014-12-15 3:48 ` Theodore Ts'o
2014-12-15 6:38 ` Darrick J. Wong
0 siblings, 1 reply; 104+ messages in thread
From: Theodore Ts'o @ 2014-12-15 3:48 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Sat, Dec 13, 2014 at 07:11:11PM -0800, Darrick J. Wong wrote:
> > But if you look at the above code, there is nothing which is inode or
> > handle specific. The value is only dependent on fs->blocksize. But
> > to calculate this value the function calls ul_log2 three times, which
> > is implemented using a looping construct. So we will be recalculating
> > what is effectively a constant value every single inode in an ext4
> > file system, which seems very unfortunate.
>
> How about I save the last fs->blocksize and the last result, and return the
> cached last result if fs->blocksize == lastblocksize?
That sounds like a plan. BTW, it looks like I had accidentally
included this in the next branch, but adding a cache won't disturb the
API, so that's fine.
- Ted
^ permalink raw reply [flat|nested] 104+ messages in thread
* Re: [PATCH 13/47] libext2fs: add a way to check the theoretical maximum extent tree depth
2014-12-15 3:48 ` Theodore Ts'o
@ 2014-12-15 6:38 ` Darrick J. Wong
0 siblings, 0 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-12-15 6:38 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: linux-ext4
On Sun, Dec 14, 2014 at 10:48:22PM -0500, Theodore Ts'o wrote:
> On Sat, Dec 13, 2014 at 07:11:11PM -0800, Darrick J. Wong wrote:
> > > But if you look at the above code, there is nothing which is inode or
> > > handle specific. The value is only dependent on fs->blocksize. But
> > > to calculate this value the function calls ul_log2 three times, which
> > > is implemented using a looping construct. So we will be recalculating
> > > what is effectively a constant value every single inode in an ext4
> > > file system, which seems very unfortunate.
> >
> > How about I save the last fs->blocksize and the last result, and return the
> > cached last result if fs->blocksize == lastblocksize?
>
> That sounds like a plan. BTW, it looks like I had accidentally
> included this in the next branch, but adding a cache won't disturb the
> API, so that's fine.
Ok, I'll send the patch that adds the cache.
--D
>
> - Ted
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 104+ messages in thread
* [PATCH 14/47] libext2fs: ext2fs_new_block2() should call alloc_block hook
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (12 preceding siblings ...)
2014-11-07 21:52 ` [PATCH 13/47] libext2fs: add a way to check the theoretical maximum extent tree depth Darrick J. Wong
@ 2014-11-07 21:52 ` Darrick J. Wong
2014-11-07 21:52 ` [PATCH 15/47] misc: don't allow mk_hugefiles unless the fs supports extents Darrick J. Wong
` (34 subsequent siblings)
48 siblings, 0 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:52 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
If ext2fs_new_block2() is called without a specific block map, we
should call the alloc_block hook before checking fs->block_map. This
helps us to avoid a bug in e2fsck where we need to allocate a block
but instead of consulting block_found_map, we use the FS bitmaps,
which (prior to pass 5) could be wrong.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
e2fsck/pass1.c | 2 +-
lib/ext2fs/alloc.c | 15 +++++++++++++++
2 files changed, 16 insertions(+), 1 deletion(-)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index af64fd1..4cc58c4 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -3728,7 +3728,7 @@ static errcode_t e2fsck_get_alloc_block(ext2_filsys fs, blk64_t goal,
return retval;
}
- retval = ext2fs_new_block2(fs, goal, 0, &new_block);
+ retval = ext2fs_new_block2(fs, goal, fs->block_map, &new_block);
if (retval)
return retval;
}
diff --git a/lib/ext2fs/alloc.c b/lib/ext2fs/alloc.c
index fda45f9..d3c8e50 100644
--- a/lib/ext2fs/alloc.c
+++ b/lib/ext2fs/alloc.c
@@ -137,9 +137,23 @@ errcode_t ext2fs_new_block2(ext2_filsys fs, blk64_t goal,
{
errcode_t retval;
blk64_t b = 0;
+ errcode_t (*gab)(ext2_filsys fs, blk64_t goal, blk64_t *ret);
EXT2_CHECK_MAGIC(fs, EXT2_ET_MAGIC_EXT2FS_FILSYS);
+ if (!map && fs->get_alloc_block) {
+ /*
+ * In case there are clients out there whose get_alloc_block
+ * handlers call ext2fs_new_block2 with a NULL block map,
+ * temporarily swap out the function pointer so that we don't
+ * end up in an infinite loop.
+ */
+ gab = fs->get_alloc_block;
+ fs->get_alloc_block = NULL;
+ retval = gab(fs, goal, &b);
+ fs->get_alloc_block = gab;
+ goto allocated;
+ }
if (!map)
map = fs->block_map;
if (!map)
@@ -153,6 +167,7 @@ errcode_t ext2fs_new_block2(ext2_filsys fs, blk64_t goal,
if ((retval == ENOENT) && (goal != fs->super->s_first_data_block))
retval = ext2fs_find_first_zero_block_bitmap2(map,
fs->super->s_first_data_block, goal - 1, &b);
+allocated:
if (retval == ENOENT)
return EXT2_ET_BLOCK_ALLOC_FAIL;
if (retval)
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 15/47] misc: don't allow mk_hugefiles unless the fs supports extents
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (13 preceding siblings ...)
2014-11-07 21:52 ` [PATCH 14/47] libext2fs: ext2fs_new_block2() should call alloc_block hook Darrick J. Wong
@ 2014-11-07 21:52 ` Darrick J. Wong
2014-12-14 2:51 ` Theodore Ts'o
2014-11-07 21:52 ` [PATCH 16/47] dumpe2fs: 80 column outputs, please Darrick J. Wong
` (33 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:52 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
The current mk_hugefile code in mke2fs doesn't support creating
non-extent files, so disable the functionality when we're mkfs'ing
without extent support.
The fallocate patches further on will eliminate the need for this.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
misc/mk_hugefiles.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/misc/mk_hugefiles.c b/misc/mk_hugefiles.c
index 3e4274c..9097b7c 100644
--- a/misc/mk_hugefiles.c
+++ b/misc/mk_hugefiles.c
@@ -477,6 +477,10 @@ errcode_t mk_hugefiles(ext2_filsys fs, const char *device_name)
if (!get_bool_from_profile(fs_types, "make_hugefiles", 0))
return 0;
+ if (!EXT2_HAS_INCOMPAT_FEATURE(fs->super,
+ EXT3_FEATURE_INCOMPAT_EXTENTS))
+ return EXT2_ET_EXTENT_NOT_SUPPORTED;
+
uid = get_int_from_profile(fs_types, "hugefiles_uid", 0);
gid = get_int_from_profile(fs_types, "hugefiles_gid", 0);
fs->umask = get_int_from_profile(fs_types, "hugefiles_umask", 077);
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 16/47] dumpe2fs: 80 column outputs, please
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (14 preceding siblings ...)
2014-11-07 21:52 ` [PATCH 15/47] misc: don't allow mk_hugefiles unless the fs supports extents Darrick J. Wong
@ 2014-11-07 21:52 ` Darrick J. Wong
2014-12-14 2:52 ` Theodore Ts'o
2014-11-07 21:52 ` [PATCH 17/47] dumpe2fs: output cleanup Darrick J. Wong
` (32 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:52 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Try to reduce dumpe2fs output to 80 columns or less.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
misc/dumpe2fs.c | 6 +++++-
tests/f_jnl_errno/expect.0 | 3 ++-
tests/j_short_revoke_trans/script | 2 +-
tests/j_short_revoke_trans_mcsum_64bit/script | 2 +-
tests/j_short_trans/script | 2 +-
tests/j_short_trans_64bit/script | 2 +-
tests/j_short_trans_mcsum_64bit/script | 2 +-
tests/j_short_trans_old_csum/script | 2 +-
tests/j_short_trans_open_recover/script | 2 +-
tests/j_short_trans_recover/script | 2 +-
tests/j_short_trans_recover_mcsum_64bit/script | 2 +-
tests/j_short_uncommitted_trans/script | 2 +-
tests/j_short_uncommitted_trans_mcsum_64bit/script | 2 +-
tests/m_64bit_flexbg/expect.1 | 3 ++-
tests/m_bigjournal/script | 3 +++
tests/m_dasd_bs/expect.1 | 6 ++++--
tests/m_desc_size_128/script | 3 +++
tests/m_extent_journal/script | 3 +++
tests/m_large_file/expect.1 | 3 ++-
tests/m_meta_bg/script | 3 +++
tests/m_mmp/expect.1 | 6 ++++--
tests/m_no_opt/script | 3 +++
tests/m_quota/script | 3 +++
tests/m_raid_opt/script | 3 +++
tests/m_root_owner/expect.1 | 3 ++-
tests/m_std/script | 3 +++
tests/m_uninit/script | 3 +++
tests/r_move_itable/script | 8 ++++----
tests/r_resize_inode/script | 6 +++---
29 files changed, 66 insertions(+), 27 deletions(-)
diff --git a/misc/dumpe2fs.c b/misc/dumpe2fs.c
index 26f2e37..40aa242 100644
--- a/misc/dumpe2fs.c
+++ b/misc/dumpe2fs.c
@@ -264,7 +264,11 @@ static void list_desc(ext2_filsys fs, int grp_only)
EXT4_FEATURE_RO_COMPAT_METADATA_CSUM)
printf(_(", csum 0x%08x"),
ext2fs_block_bitmap_checksum(fs, i));
- fputs(_(", Inode bitmap at "), stdout);
+ if (getenv("DUMPE2FS_IGNORE_80COL"))
+ fputs(_(","), stdout);
+ else
+ fputs(_("\n "), stdout);
+ fputs(_(" Inode bitmap at "), stdout);
print_number(ext2fs_inode_bitmap_loc(fs, i));
print_bg_rel_offset(fs, ext2fs_inode_bitmap_loc(fs, i), 0,
first_block, last_block);
diff --git a/tests/f_jnl_errno/expect.0 b/tests/f_jnl_errno/expect.0
index 6dad72a..2a9426d 100644
--- a/tests/f_jnl_errno/expect.0
+++ b/tests/f_jnl_errno/expect.0
@@ -40,7 +40,8 @@ Journal start: 0
Group 0: (Blocks 1-8191) [ITABLE_ZEROED]
Primary superblock at 1, Group descriptors at 2-2
Reserved GDT blocks at 3-33
- Block bitmap at 34 (+33), Inode bitmap at 50 (+49)
+ Block bitmap at 34 (+33)
+ Inode bitmap at 50 (+49)
Inode table at 66-321 (+65)
6862 free blocks, 2037 free inodes, 2 directories, 2037 unused inodes
Free blocks: 1330-8191
diff --git a/tests/j_short_revoke_trans/script b/tests/j_short_revoke_trans/script
index 94f20b9..4eec436 100644
--- a/tests/j_short_revoke_trans/script
+++ b/tests/j_short_revoke_trans/script
@@ -19,7 +19,7 @@ echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
rm -f $OUT.new
-bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*Block bitmap at \([0-9]*\) .*Inode bitmap at \([0-9]*\).*$/\1,\2/g' | tr '\n' ',')"
+bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*bitmap at \([0-9]*\).*$/\1/g' | tr '\n' ',')"
echo "debugfs write journal" >> $OUT
echo "jo" > $TMPFILE.cmd
diff --git a/tests/j_short_revoke_trans_mcsum_64bit/script b/tests/j_short_revoke_trans_mcsum_64bit/script
index 8358012..c943efa 100644
--- a/tests/j_short_revoke_trans_mcsum_64bit/script
+++ b/tests/j_short_revoke_trans_mcsum_64bit/script
@@ -19,7 +19,7 @@ echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
rm -f $OUT.new
-bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*Block bitmap at \([0-9]*\) .*Inode bitmap at \([0-9]*\).*$/\1,\2/g' | tr '\n' ',')"
+bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*bitmap at \([0-9]*\).*$/\1/g' | tr '\n' ',')"
$DUMPE2FS $TMPFILE 2>&1 | grep '^Journal features:' >> $OUT
diff --git a/tests/j_short_trans/script b/tests/j_short_trans/script
index c83daba..852e7a5 100644
--- a/tests/j_short_trans/script
+++ b/tests/j_short_trans/script
@@ -19,7 +19,7 @@ echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
rm -f $OUT.new
-bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*Block bitmap at \([0-9]*\) .*Inode bitmap at \([0-9]*\).*$/\1,\2/g' | tr '\n' ',')"
+bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*bitmap at \([0-9]*\).*$/\1/g' | tr '\n' ',')"
echo "debugfs write journal" >> $OUT
echo "jo" > $TMPFILE.cmd
diff --git a/tests/j_short_trans_64bit/script b/tests/j_short_trans_64bit/script
index 7a60284..994fa21 100644
--- a/tests/j_short_trans_64bit/script
+++ b/tests/j_short_trans_64bit/script
@@ -19,7 +19,7 @@ echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
rm -f $OUT.new
-bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*Block bitmap at \([0-9]*\) .*Inode bitmap at \([0-9]*\).*$/\1,\2/g' | tr '\n' ',')"
+bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*bitmap at \([0-9]*\).*$/\1/g' | tr '\n' ',')"
$DUMPE2FS $TMPFILE 2>&1 | grep '^Journal features:' >> $OUT
diff --git a/tests/j_short_trans_mcsum_64bit/script b/tests/j_short_trans_mcsum_64bit/script
index bc9f09b..024346a 100644
--- a/tests/j_short_trans_mcsum_64bit/script
+++ b/tests/j_short_trans_mcsum_64bit/script
@@ -19,7 +19,7 @@ echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
rm -f $OUT.new
-bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*Block bitmap at \([0-9]*\) .*Inode bitmap at \([0-9]*\).*$/\1,\2/g' | tr '\n' ',')"
+bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*bitmap at \([0-9]*\).*$/\1/g' | tr '\n' ',')"
$DUMPE2FS $TMPFILE 2>&1 | grep '^Journal features:' >> $OUT
diff --git a/tests/j_short_trans_old_csum/script b/tests/j_short_trans_old_csum/script
index 52fc58f..56f8bcb 100644
--- a/tests/j_short_trans_old_csum/script
+++ b/tests/j_short_trans_old_csum/script
@@ -19,7 +19,7 @@ echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
rm -f $OUT.new
-bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*Block bitmap at \([0-9]*\) .*Inode bitmap at \([0-9]*\).*$/\1,\2/g' | tr '\n' ',')"
+bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*bitmap at \([0-9]*\).*$/\1/g' | tr '\n' ',')"
$DUMPE2FS $TMPFILE 2>&1 | grep '^Journal features:' >> $OUT
diff --git a/tests/j_short_trans_open_recover/script b/tests/j_short_trans_open_recover/script
index e5b772d..2a016df 100644
--- a/tests/j_short_trans_open_recover/script
+++ b/tests/j_short_trans_open_recover/script
@@ -19,7 +19,7 @@ echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
rm -f $OUT.new
-bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*Block bitmap at \([0-9]*\) .*Inode bitmap at \([0-9]*\).*$/\1,\2/g' | tr '\n' ',')"
+bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*bitmap at \([0-9]*\).*$/\1/g' | tr '\n' ',')"
echo "debugfs write journal" >> $OUT
echo "jo" > $TMPFILE.cmd
diff --git a/tests/j_short_trans_recover/script b/tests/j_short_trans_recover/script
index 2494a2e..33d2345 100644
--- a/tests/j_short_trans_recover/script
+++ b/tests/j_short_trans_recover/script
@@ -19,7 +19,7 @@ echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
rm -f $OUT.new
-bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*Block bitmap at \([0-9]*\) .*Inode bitmap at \([0-9]*\).*$/\1,\2/g' | tr '\n' ',')"
+bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*bitmap at \([0-9]*\).*$/\1/g' | tr '\n' ',')"
echo "debugfs write journal" >> $OUT
echo "jo" > $TMPFILE.cmd
diff --git a/tests/j_short_trans_recover_mcsum_64bit/script b/tests/j_short_trans_recover_mcsum_64bit/script
index 6076e03..b6c8c49 100644
--- a/tests/j_short_trans_recover_mcsum_64bit/script
+++ b/tests/j_short_trans_recover_mcsum_64bit/script
@@ -19,7 +19,7 @@ echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
rm -f $OUT.new
-bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*Block bitmap at \([0-9]*\) .*Inode bitmap at \([0-9]*\).*$/\1,\2/g' | tr '\n' ',')"
+bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*bitmap at \([0-9]*\).*$/\1/g' | tr '\n' ',')"
$DUMPE2FS $TMPFILE 2>&1 | grep '^Journal features:' >> $OUT
diff --git a/tests/j_short_uncommitted_trans/script b/tests/j_short_uncommitted_trans/script
index a988948..a1fc3b4 100644
--- a/tests/j_short_uncommitted_trans/script
+++ b/tests/j_short_uncommitted_trans/script
@@ -19,7 +19,7 @@ echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
rm -f $OUT.new
-bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*Block bitmap at \([0-9]*\) .*Inode bitmap at \([0-9]*\).*$/\1,\2/g' | tr '\n' ',')"
+bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*bitmap at \([0-9]*\).*$/\1/g' | tr '\n' ',')"
echo "debugfs write journal" >> $OUT
echo "jo" > $TMPFILE.cmd
diff --git a/tests/j_short_uncommitted_trans_mcsum_64bit/script b/tests/j_short_uncommitted_trans_mcsum_64bit/script
index 43313bb..a0b8c7f 100644
--- a/tests/j_short_uncommitted_trans_mcsum_64bit/script
+++ b/tests/j_short_uncommitted_trans_mcsum_64bit/script
@@ -19,7 +19,7 @@ echo Exit status is $status >> $OUT.new
sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
rm -f $OUT.new
-bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*Block bitmap at \([0-9]*\) .*Inode bitmap at \([0-9]*\).*$/\1,\2/g' | tr '\n' ',')"
+bitmaps="$($DUMPE2FS $TMPFILE 2>&1 | grep 'bitmap at' | sed -e 's/^.*bitmap at \([0-9]*\).*$/\1/g' | tr '\n' ',')"
$DUMPE2FS $TMPFILE 2>&1 | grep '^Journal features:' >> $OUT
diff --git a/tests/m_64bit_flexbg/expect.1 b/tests/m_64bit_flexbg/expect.1
index 3635318..cfa3bc9 100644
--- a/tests/m_64bit_flexbg/expect.1
+++ b/tests/m_64bit_flexbg/expect.1
@@ -48,7 +48,8 @@ Default directory hash: half_md4
Group 0: (Blocks 1-1023)
Primary superblock at 1, Group descriptors at 2-2
Reserved GDT blocks at 3-9
- Block bitmap at 10 (+9), Inode bitmap at 26 (+25)
+ Block bitmap at 10 (+9)
+ Inode bitmap at 26 (+25)
Inode table at 42-57 (+41)
982 free blocks, 117 free inodes, 2 directories
Free blocks: 24-25, 28-41, 58-1023
diff --git a/tests/m_bigjournal/script b/tests/m_bigjournal/script
index 4c1ed9a..576d99e 100644
--- a/tests/m_bigjournal/script
+++ b/tests/m_bigjournal/script
@@ -1,8 +1,11 @@
DESCRIPTION="journal over 4GB in size"
FS_SIZE=11000000
+DUMPE2FS_IGNORE_80COL=1
+export DUMPE2FS_IGNORE_80COL
MKE2FS_OPTS="-t ext4 -G 512 -N 1280 -J size=5000 -q -E lazy_journal_init,lazy_itable_init,nodiscard"
if [ $(uname -s) = "Darwin" ]; then
echo "$test_name: $DESCRIPTION: skipped for HFS+ (no sparse files)"
return 0
fi
. $cmd_dir/run_mke2fs
+unset DUMPE2FS_IGNORE_80COL
diff --git a/tests/m_dasd_bs/expect.1 b/tests/m_dasd_bs/expect.1
index 75a401d..0e55e8f 100644
--- a/tests/m_dasd_bs/expect.1
+++ b/tests/m_dasd_bs/expect.1
@@ -48,7 +48,8 @@ Default directory hash: half_md4
Group 0: (Blocks 0-16383)
Primary superblock at 0, Group descriptors at 1-1
Reserved GDT blocks at 2-32
- Block bitmap at 33 (+33), Inode bitmap at 34 (+34)
+ Block bitmap at 33 (+33)
+ Inode bitmap at 34 (+34)
Inode table at 35-546 (+35)
15827 free blocks, 8181 free inodes, 2 directories
Free blocks: 557-16383
@@ -56,7 +57,8 @@ Group 0: (Blocks 0-16383)
Group 1: (Blocks 16384-32767)
Backup superblock at 16384, Group descriptors at 16385-16385
Reserved GDT blocks at 16386-16416
- Block bitmap at 16417 (+33), Inode bitmap at 16418 (+34)
+ Block bitmap at 16417 (+33)
+ Inode bitmap at 16418 (+34)
Inode table at 16419-16930 (+35)
15837 free blocks, 8192 free inodes, 0 directories
Free blocks: 16931-32767
diff --git a/tests/m_desc_size_128/script b/tests/m_desc_size_128/script
index de3def9..a4def21 100644
--- a/tests/m_desc_size_128/script
+++ b/tests/m_desc_size_128/script
@@ -1,4 +1,7 @@
DESCRIPTION="enable 128-byte group descriptor on mkfs"
FS_SIZE=131072
+DUMPE2FS_IGNORE_80COL=1
+export DUMPE2FS_IGNORE_80COL
MKE2FS_OPTS="-b 1024 -O 64bit,extents -g 1024 -N 8192 -E desc_size=128"
. $cmd_dir/run_mke2fs
+unset DUMPE2FS_IGNORE_80COL
diff --git a/tests/m_extent_journal/script b/tests/m_extent_journal/script
index 5e0cac2..efade21 100644
--- a/tests/m_extent_journal/script
+++ b/tests/m_extent_journal/script
@@ -1,4 +1,7 @@
DESCRIPTION="extent-mapped journal"
FS_SIZE=65536
+DUMPE2FS_IGNORE_80COL=1
+export DUMPE2FS_IGNORE_80COL
MKE2FS_OPTS="-O extents -j"
. $cmd_dir/run_mke2fs
+unset DUMPE2FS_IGNORE_80COL
diff --git a/tests/m_large_file/expect.1 b/tests/m_large_file/expect.1
index 8ab2711..4acca41 100644
--- a/tests/m_large_file/expect.1
+++ b/tests/m_large_file/expect.1
@@ -48,7 +48,8 @@ Default directory hash: half_md4
Group 0: (Blocks 0-16383)
Primary superblock at 0, Group descriptors at 1-1
Reserved GDT blocks at 2-4
- Block bitmap at 5 (+5), Inode bitmap at 6 (+6)
+ Block bitmap at 5 (+5)
+ Inode bitmap at 6 (+6)
Inode table at 7-10 (+7)
16367 free blocks, 53 free inodes, 2 directories
Free blocks: 17-16383
diff --git a/tests/m_meta_bg/script b/tests/m_meta_bg/script
index 7ca2bea..5e285b4 100644
--- a/tests/m_meta_bg/script
+++ b/tests/m_meta_bg/script
@@ -1,4 +1,7 @@
DESCRIPTION="meta blockgroup feature"
FS_SIZE=131072
+DUMPE2FS_IGNORE_80COL=1
+export DUMPE2FS_IGNORE_80COL
MKE2FS_OPTS="-O meta_bg,sparse_super,^resize_inode -g 1024"
. $cmd_dir/run_mke2fs
+unset DUMPE2FS_IGNORE_80COL
diff --git a/tests/m_mmp/expect.1 b/tests/m_mmp/expect.1
index aec0a84..a1452e6 100644
--- a/tests/m_mmp/expect.1
+++ b/tests/m_mmp/expect.1
@@ -51,7 +51,8 @@ MMP update interval: 5
Group 0: (Blocks 0-32767)
Primary superblock at 0, Group descriptors at 1-1
Reserved GDT blocks at 2-16
- Block bitmap at 17 (+17), Inode bitmap at 18 (+18)
+ Block bitmap at 17 (+17)
+ Inode bitmap at 18 (+18)
Inode table at 19-1042 (+19)
31718 free blocks, 32757 free inodes, 2 directories
Free blocks: 1050-32767
@@ -59,7 +60,8 @@ Group 0: (Blocks 0-32767)
Group 1: (Blocks 32768-65535)
Backup superblock at 32768, Group descriptors at 32769-32769
Reserved GDT blocks at 32770-32784
- Block bitmap at 32785 (+17), Inode bitmap at 32786 (+18)
+ Block bitmap at 32785 (+17)
+ Inode bitmap at 32786 (+18)
Inode table at 32787-33810 (+19)
31725 free blocks, 32768 free inodes, 0 directories
Free blocks: 33811-65535
diff --git a/tests/m_no_opt/script b/tests/m_no_opt/script
index 6d1d0eb..223f117 100644
--- a/tests/m_no_opt/script
+++ b/tests/m_no_opt/script
@@ -1,4 +1,7 @@
DESCRIPTION="no filesystem extensions"
FS_SIZE=65536
+DUMPE2FS_IGNORE_80COL=1
+export DUMPE2FS_IGNORE_80COL
MKE2FS_OPTS="-O ^sparse_super,^filetype,^resize_inode,^dir_index,^ext_attr"
. $cmd_dir/run_mke2fs
+unset DUMPE2FS_IGNORE_80COL
diff --git a/tests/m_quota/script b/tests/m_quota/script
index 36ab630..fe63939 100644
--- a/tests/m_quota/script
+++ b/tests/m_quota/script
@@ -5,4 +5,7 @@ if [ "$QUOTA" != "y" ]; then
echo "$test_name: $DESCRIPTION: skipped"
return 0
fi
+DUMPE2FS_IGNORE_80COL=1
+export DUMPE2FS_IGNORE_80COL
. $cmd_dir/run_mke2fs
+unset DUMPE2FS_IGNORE_80COL
diff --git a/tests/m_raid_opt/script b/tests/m_raid_opt/script
index 296fe94..8c859e0 100644
--- a/tests/m_raid_opt/script
+++ b/tests/m_raid_opt/script
@@ -1,4 +1,7 @@
DESCRIPTION="raid options"
FS_SIZE=131072
MKE2FS_OPTS="-E stride=13 -O sparse_super -g 1024"
+DUMPE2FS_IGNORE_80COL=1
+export DUMPE2FS_IGNORE_80COL
. $cmd_dir/run_mke2fs
+unset DUMPE2FS_IGNORE_80COL
diff --git a/tests/m_root_owner/expect.1 b/tests/m_root_owner/expect.1
index cd94993..30d119e 100644
--- a/tests/m_root_owner/expect.1
+++ b/tests/m_root_owner/expect.1
@@ -46,7 +46,8 @@ Default directory hash: half_md4
Group 0: (Blocks 1-1023)
Primary superblock at 1, Group descriptors at 2-2
Reserved GDT blocks at 3-5
- Block bitmap at 6 (+5), Inode bitmap at 7 (+6)
+ Block bitmap at 6 (+5)
+ Inode bitmap at 7 (+6)
Inode table at 8-23 (+7)
986 free blocks, 117 free inodes, 2 directories
Free blocks: 38-1023
diff --git a/tests/m_std/script b/tests/m_std/script
index a2f2cda..61c50c7 100644
--- a/tests/m_std/script
+++ b/tests/m_std/script
@@ -1,3 +1,6 @@
DESCRIPTION="standard filesystem options"
+DUMPE2FS_IGNORE_80COL=1
+export DUMPE2FS_IGNORE_80COL
FS_SIZE=65536
. $cmd_dir/run_mke2fs
+unset DUMPE2FS_IGNORE_80COL
diff --git a/tests/m_uninit/script b/tests/m_uninit/script
index 0b565ac..d4d373a 100644
--- a/tests/m_uninit/script
+++ b/tests/m_uninit/script
@@ -1,4 +1,7 @@
DESCRIPTION="uninitialized group feature"
+DUMPE2FS_IGNORE_80COL=1
+export DUMPE2FS_IGNORE_80COL
FS_SIZE=131072
MKE2FS_OPTS="-O uninit_bg"
. $cmd_dir/run_mke2fs
+unset DUMPE2FS_IGNORE_80COL
diff --git a/tests/r_move_itable/script b/tests/r_move_itable/script
index e567888..3f02a79 100644
--- a/tests/r_move_itable/script
+++ b/tests/r_move_itable/script
@@ -33,7 +33,7 @@ $DEBUGFS -R "set_super_value mkfs_time 0" -w $TMPFILE >/dev/null 2>&1
$TUNE2FS -c 20 -U clear $TMPFILE >/dev/null 2>&1
echo dumpe2fs test.img >> $OUT
-$DUMPE2FS $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DUMPE2FS $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed -e '/Block bitmap.*$/N;s/\n Inode bitmap/, Inode bitmap/g' >> $OUT
echo "--------------------------------" >> $OUT
@@ -53,7 +53,7 @@ $DEBUGFS -R "set_super_value mkfs_time 0" -w $TMPFILE >/dev/null 2>&1
$TUNE2FS -c 20 -U clear $TMPFILE >/dev/null 2>&1
echo dumpe2fs test.img >> $OUT
-$DUMPE2FS $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DUMPE2FS $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed -e '/Block bitmap.*$/N;s/\n Inode bitmap/, Inode bitmap/g' >> $OUT
echo "--------------------------------" >> $OUT
@@ -73,7 +73,7 @@ $DEBUGFS -R "set_super_value mkfs_time 0" -w $TMPFILE >/dev/null 2>&1
$TUNE2FS -c 20 -U clear $TMPFILE >/dev/null 2>&1
echo dumpe2fs test.img >> $OUT
-$DUMPE2FS $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DUMPE2FS $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed -e '/Block bitmap.*$/N;s/\n Inode bitmap/, Inode bitmap/g' >> $OUT
echo "--------------------------------" >> $OUT
@@ -93,7 +93,7 @@ $DEBUGFS -R "set_super_value mkfs_time 0" -w $TMPFILE >/dev/null 2>&1
$TUNE2FS -c 20 -U clear $TMPFILE >/dev/null 2>&1
echo dumpe2fs test.img >> $OUT
-$DUMPE2FS $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DUMPE2FS $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed -e '/Block bitmap.*$/N;s/\n Inode bitmap/, Inode bitmap/g' >> $OUT
rm -f $TMPFILE
diff --git a/tests/r_resize_inode/script b/tests/r_resize_inode/script
index 0f12138..4e3eb19 100644
--- a/tests/r_resize_inode/script
+++ b/tests/r_resize_inode/script
@@ -33,7 +33,7 @@ $DEBUGFS -R "set_super_value mkfs_time 0" -w $TMPFILE >/dev/null 2>&1
$TUNE2FS -c 20 -U clear $TMPFILE >/dev/null 2>&1
echo dumpe2fs test.img >> $OUT
-$DUMPE2FS $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DUMPE2FS $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed -e '/Block bitmap.*$/N;s/\n Inode bitmap/, Inode bitmap/g' >> $OUT
echo "--------------------------------" >> $OUT
@@ -59,7 +59,7 @@ $DEBUGFS -R "set_super_value mkfs_time 0" -w $TMPFILE >/dev/null 2>&1
$TUNE2FS -c 20 -U clear $TMPFILE >/dev/null 2>&1
echo dumpe2fs test.img >> $OUT
-$DUMPE2FS $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DUMPE2FS $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed -e '/Block bitmap.*$/N;s/\n Inode bitmap/, Inode bitmap/g' >> $OUT
echo "--------------------------------" >> $OUT
@@ -79,7 +79,7 @@ $DEBUGFS -R "set_super_value mkfs_time 0" -w $TMPFILE >/dev/null 2>&1
$TUNE2FS -c 20 -U clear $TMPFILE >/dev/null 2>&1
echo dumpe2fs test.img >> $OUT
-$DUMPE2FS $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT
+$DUMPE2FS $TMPFILE 2>&1 | sed -f $cmd_dir/filter.sed -e '/Block bitmap.*$/N;s/\n Inode bitmap/, Inode bitmap/g' >> $OUT
rm -f $TMPFILE
cmp -s $OUT $EXP
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 17/47] dumpe2fs: output cleanup
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (15 preceding siblings ...)
2014-11-07 21:52 ` [PATCH 16/47] dumpe2fs: 80 column outputs, please Darrick J. Wong
@ 2014-11-07 21:52 ` Darrick J. Wong
2014-12-14 2:53 ` Theodore Ts'o
2014-11-07 21:52 ` [PATCH 18/47] e2fsck: only complain about no-checksum directory blocks once Darrick J. Wong
` (31 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:52 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4, TR Reardon
Don't display unused inodes twice, and make it clear that we're
printing a descriptor checksum.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Cc: TR Reardon <thomas_reardon@hotmail.com>
---
misc/dumpe2fs.c | 8 +++-----
tests/filter.sed | 2 ++
2 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/misc/dumpe2fs.c b/misc/dumpe2fs.c
index 40aa242..b409650 100644
--- a/misc/dumpe2fs.c
+++ b/misc/dumpe2fs.c
@@ -218,20 +218,18 @@ static void list_desc(ext2_filsys fs, int grp_only)
continue;
}
- printf (_("Group %lu: (Blocks "), i);
+ printf(_("Group %lu: (Blocks "), i);
print_range(first_block, last_block);
fputs(")", stdout);
- print_bg_opts(fs, i);
if (ext2fs_has_group_desc_csum(fs)) {
unsigned csum = ext2fs_bg_checksum(fs, i);
unsigned exp_csum = ext2fs_group_desc_csum(fs, i);
- printf(_(" Checksum 0x%04x"), csum);
+ printf(_(" csum 0x%04x"), csum);
if (csum != exp_csum)
printf(_(" (EXPECTED 0x%04x)"), exp_csum);
- printf(_(", unused inodes %u\n"),
- ext2fs_bg_itable_unused(fs, i));
}
+ print_bg_opts(fs, i);
has_super = ((i==0) || super_blk);
if (has_super) {
printf (_(" %s superblock at "),
diff --git a/tests/filter.sed b/tests/filter.sed
index 6ca6bbc..a31b00f 100644
--- a/tests/filter.sed
+++ b/tests/filter.sed
@@ -23,3 +23,5 @@ s/\\015//g
/Reserved blocks gid:/s/ (group .*)//
/whichever comes first/d
/^ Checksum /d
+s/, csum 0x\([0-9a-f]*\)//g
+s/ csum 0x\([0-9a-f]*\)//g
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 18/47] e2fsck: only complain about no-checksum directory blocks once
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (16 preceding siblings ...)
2014-11-07 21:52 ` [PATCH 17/47] dumpe2fs: output cleanup Darrick J. Wong
@ 2014-11-07 21:52 ` Darrick J. Wong
2014-12-14 2:55 ` Theodore Ts'o
2014-11-07 21:52 ` [PATCH 19/47] e2fsck: don't complain about root dir csum failures when getting lnf Darrick J. Wong
` (30 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:52 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
If a directory block lacks space for a checksum and the user directs
e2fsck to fix the directory block (by rehashing it), don't complain a
second time about the checksum verification failure when we get to the
end of the directory block.
Also, don't complain about broken HTREE directories if we're already
planning to rebuild the HTREE directory.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
e2fsck/pass2.c | 11 ++++++++---
tests/f_dir_bad_csum/expect.1 | 6 ------
2 files changed, 8 insertions(+), 9 deletions(-)
diff --git a/e2fsck/pass2.c b/e2fsck/pass2.c
index cd0762a..7aaebce 100644
--- a/e2fsck/pass2.c
+++ b/e2fsck/pass2.c
@@ -166,7 +166,8 @@ void e2fsck_pass2(e2fsck_t ctx)
for (i=0; (dx_dir = e2fsck_dx_dir_info_iter(ctx, &i)) != 0;) {
if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
return;
- if (dx_dir->numblocks == 0)
+ if (e2fsck_dir_will_be_rehashed(ctx, dx_dir->ino) ||
+ dx_dir->numblocks == 0)
continue;
clear_problem_context(&pctx);
bad_dir = 0;
@@ -1022,11 +1023,15 @@ out_htree:
if (is_leaf && !inline_data_size && failed_csum &&
!ext2fs_dirent_has_tail(fs, (struct ext2_dir_entry *)buf)) {
de_csum_size = 0;
- if (e2fsck_dir_will_be_rehashed(ctx, ino) ||
- !fix_problem(cd->ctx, PR_2_LEAF_NODE_MISSING_CSUM,
+ if (e2fsck_dir_will_be_rehashed(ctx, ino)) {
+ failed_csum = 0;
+ goto skip_checksum;
+ }
+ if (!fix_problem(cd->ctx, PR_2_LEAF_NODE_MISSING_CSUM,
&cd->pctx))
goto skip_checksum;
e2fsck_rehash_dir_later(ctx, ino);
+ failed_csum = 0;
goto skip_checksum;
}
/* htree nodes don't use fake dirents to store checksums */
diff --git a/tests/f_dir_bad_csum/expect.1 b/tests/f_dir_bad_csum/expect.1
index bbcbfcb..b2b2fae 100644
--- a/tests/f_dir_bad_csum/expect.1
+++ b/tests/f_dir_bad_csum/expect.1
@@ -6,15 +6,9 @@ Fix? yes
Directory inode 13, block #0, offset 0: directory has no checksum.
Fix? yes
-Directory inode 13, block #0, offset 1012: directory passes checks but fails checksum.
-Fix? yes
-
Directory inode 14, block #0, offset 0: directory has no checksum.
Fix? yes
-Directory inode 14, block #0, offset 12: directory passes checks but fails checksum.
-Fix? yes
^ permalink raw reply related [flat|nested] 104+ messages in thread* Re: [PATCH 18/47] e2fsck: only complain about no-checksum directory blocks once
2014-11-07 21:52 ` [PATCH 18/47] e2fsck: only complain about no-checksum directory blocks once Darrick J. Wong
@ 2014-12-14 2:55 ` Theodore Ts'o
0 siblings, 0 replies; 104+ messages in thread
From: Theodore Ts'o @ 2014-12-14 2:55 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Fri, Nov 07, 2014 at 01:52:45PM -0800, Darrick J. Wong wrote:
> If a directory block lacks space for a checksum and the user directs
> e2fsck to fix the directory block (by rehashing it), don't complain a
> second time about the checksum verification failure when we get to the
> end of the directory block.
>
> Also, don't complain about broken HTREE directories if we're already
> planning to rebuild the HTREE directory.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Applied, thanks.
- Ted
^ permalink raw reply [flat|nested] 104+ messages in thread
* [PATCH 19/47] e2fsck: don't complain about root dir csum failures when getting lnf
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (17 preceding siblings ...)
2014-11-07 21:52 ` [PATCH 18/47] e2fsck: only complain about no-checksum directory blocks once Darrick J. Wong
@ 2014-11-07 21:52 ` Darrick J. Wong
2014-12-14 2:57 ` Theodore Ts'o
2014-11-07 21:52 ` [PATCH 20/47] tune2fs: warn if extents are not enabled when turning on metadata_csum Darrick J. Wong
` (29 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:52 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Don't complain about checksum failures on the root dir when we're
trying to find l+f if the root dir is going to be rehashed anyway.
The test case for this is t_enable_mcsum in the next patch.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
e2fsck/pass3.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/e2fsck/pass3.c b/e2fsck/pass3.c
index 2d94ece..1d5255f 100644
--- a/e2fsck/pass3.c
+++ b/e2fsck/pass3.c
@@ -384,14 +384,23 @@ ext2_ino_t e2fsck_get_lost_and_found(e2fsck_t ctx, int fix)
char * block;
static const char name[] = "lost+found";
struct problem_context pctx;
+ int will_rehash, flags;
if (ctx->lost_and_found)
return ctx->lost_and_found;
clear_problem_context(&pctx);
+ will_rehash = e2fsck_dir_will_be_rehashed(ctx, EXT2_ROOT_INO);
+ if (will_rehash) {
+ flags = ctx->fs->flags;
+ ctx->fs->flags |= EXT2_FLAG_IGNORE_CSUM_ERRORS;
+ }
retval = ext2fs_lookup(fs, EXT2_ROOT_INO, name,
sizeof(name)-1, 0, &ino);
+ if (will_rehash)
+ ctx->fs->flags = (flags & EXT2_FLAG_IGNORE_CSUM_ERRORS) |
+ (ctx->fs->flags & ~EXT2_FLAG_IGNORE_CSUM_ERRORS);
if (retval && !fix)
return 0;
if (!retval) {
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 20/47] tune2fs: warn if extents are not enabled when turning on metadata_csum
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (18 preceding siblings ...)
2014-11-07 21:52 ` [PATCH 19/47] e2fsck: don't complain about root dir csum failures when getting lnf Darrick J. Wong
@ 2014-11-07 21:52 ` Darrick J. Wong
2014-12-14 2:58 ` Theodore Ts'o
2014-11-07 21:53 ` [PATCH 21/47] tune2fs: enable uninit_bg when disabling metadata_csum Darrick J. Wong
` (28 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:52 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Warn the user if we're trying to enable metadata_csum on a FS that
doesn't support extents (since block maps cannot contain checksums).
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
misc/tune2fs.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/misc/tune2fs.c b/misc/tune2fs.c
index 7fee870..6feaab1 100644
--- a/misc/tune2fs.c
+++ b/misc/tune2fs.c
@@ -1113,6 +1113,14 @@ mmp_error:
if (mount_flags & EXT2_MF_MOUNTED)
fputs(_("Cannot enable metadata_csum on a mounted "
"filesystem!\n"), stderr);
+ if (!EXT2_HAS_INCOMPAT_FEATURE(fs->super,
+ EXT3_FEATURE_INCOMPAT_EXTENTS))
+ printf("%s",
+ _("Extents are not enabled. The file extent "
+ "tree can be checksummed, whereas block maps "
+ "cannot. Not enabling extents reduces the "
+ "coverage of metadata checksumming. "
+ "Re-run with -O extent to rectify.\n"));
rewrite_checksums = 1;
/* metadata_csum supersedes uninit_bg */
fs->super->s_feature_ro_compat &=
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 21/47] tune2fs: enable uninit_bg when disabling metadata_csum
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (19 preceding siblings ...)
2014-11-07 21:52 ` [PATCH 20/47] tune2fs: warn if extents are not enabled when turning on metadata_csum Darrick J. Wong
@ 2014-11-07 21:53 ` Darrick J. Wong
2014-12-14 2:58 ` Theodore Ts'o
2014-11-07 21:53 ` [PATCH 22/47] tests: testcases for enabling/disabling metadata_csum via tune2fs Darrick J. Wong
` (27 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:53 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
If we're disabling metadata_csum and the user doesn't provide explicit
instructions to enable or disable uninit_bg, assume that they want
uninit_bg to be turned on by default. Otherwise, we lose all block
group flags and unused inode count, which is a big hit to performance.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
misc/tune2fs.c | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/misc/tune2fs.c b/misc/tune2fs.c
index 6feaab1..1c83d09 100644
--- a/misc/tune2fs.c
+++ b/misc/tune2fs.c
@@ -1141,12 +1141,26 @@ mmp_error:
if (FEATURE_OFF(E2P_FEATURE_RO_INCOMPAT,
EXT4_FEATURE_RO_COMPAT_METADATA_CSUM)) {
+ __u32 test_features[3];
+
if (check_fsck_needed(fs))
exit(1);
if (mount_flags & EXT2_MF_MOUNTED)
fputs(_("Cannot disable metadata_csum on a mounted "
"filesystem!\n"), stderr);
rewrite_checksums = 1;
+
+ /* Enable uninit_bg unless the user expressly turned it off */
+ memcpy(test_features, old_features, sizeof(test_features));
+ test_features[E2P_FEATURE_RO_INCOMPAT] |=
+ EXT4_FEATURE_RO_COMPAT_GDT_CSUM;
+ e2p_edit_feature2(features, test_features, ok_features,
+ clear_ok_features, NULL, NULL);
+ if (test_features[E2P_FEATURE_RO_INCOMPAT] &
+ EXT4_FEATURE_RO_COMPAT_GDT_CSUM)
+ fs->super->s_feature_ro_compat |=
+ EXT4_FEATURE_RO_COMPAT_GDT_CSUM;
+
/*
* If we're turning off metadata_csum and not turning on
* uninit_bg, rewrite group desc.
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 22/47] tests: testcases for enabling/disabling metadata_csum via tune2fs
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (20 preceding siblings ...)
2014-11-07 21:53 ` [PATCH 21/47] tune2fs: enable uninit_bg when disabling metadata_csum Darrick J. Wong
@ 2014-11-07 21:53 ` Darrick J. Wong
2014-12-14 3:00 ` Theodore Ts'o
2014-11-07 21:53 ` [PATCH 23/47] mke2fs: don't zero inode table blocks that are already zeroed Darrick J. Wong
` (26 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:53 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
tests/filter.sed | 1
tests/j_ext_dumpe2fs/expect | 1
tests/t_disable_mcsum/expect | 45 +++++++++++++++
tests/t_disable_mcsum/name | 1
tests/t_disable_mcsum/script | 67 ++++++++++++++++++++++
tests/t_disable_mcsum_noinitbg/expect | 68 ++++++++++++++++++++++
tests/t_disable_mcsum_noinitbg/name | 1
tests/t_disable_mcsum_noinitbg/script | 67 ++++++++++++++++++++++
tests/t_disable_mcsum_yesinitbg/expect | 45 +++++++++++++++
tests/t_disable_mcsum_yesinitbg/name | 1
tests/t_disable_mcsum_yesinitbg/script | 67 ++++++++++++++++++++++
tests/t_enable_mcsum/expect | 78 +++++++++++++++++++++++++
tests/t_enable_mcsum/name | 1
tests/t_enable_mcsum/script | 70 +++++++++++++++++++++++
tests/t_enable_mcsum_ext3/expect | 73 ++++++++++++++++++++++++
tests/t_enable_mcsum_ext3/name | 1
tests/t_enable_mcsum_ext3/script | 70 +++++++++++++++++++++++
tests/t_enable_mcsum_initbg/expect | 98 ++++++++++++++++++++++++++++++++
tests/t_enable_mcsum_initbg/name | 1
tests/t_enable_mcsum_initbg/script | 70 +++++++++++++++++++++++
20 files changed, 825 insertions(+), 1 deletion(-)
create mode 100644 tests/t_disable_mcsum/expect
create mode 100644 tests/t_disable_mcsum/name
create mode 100644 tests/t_disable_mcsum/script
create mode 100644 tests/t_disable_mcsum_noinitbg/expect
create mode 100644 tests/t_disable_mcsum_noinitbg/name
create mode 100644 tests/t_disable_mcsum_noinitbg/script
create mode 100644 tests/t_disable_mcsum_yesinitbg/expect
create mode 100644 tests/t_disable_mcsum_yesinitbg/name
create mode 100644 tests/t_disable_mcsum_yesinitbg/script
create mode 100644 tests/t_enable_mcsum/expect
create mode 100644 tests/t_enable_mcsum/name
create mode 100644 tests/t_enable_mcsum/script
create mode 100644 tests/t_enable_mcsum_ext3/expect
create mode 100644 tests/t_enable_mcsum_ext3/name
create mode 100644 tests/t_enable_mcsum_ext3/script
create mode 100644 tests/t_enable_mcsum_initbg/expect
create mode 100644 tests/t_enable_mcsum_initbg/name
create mode 100644 tests/t_enable_mcsum_initbg/script
diff --git a/tests/filter.sed b/tests/filter.sed
index a31b00f..4a630bf 100644
--- a/tests/filter.sed
+++ b/tests/filter.sed
@@ -25,3 +25,4 @@ s/\\015//g
/^ Checksum /d
s/, csum 0x\([0-9a-f]*\)//g
s/ csum 0x\([0-9a-f]*\)//g
+/^Checksum:/d
diff --git a/tests/j_ext_dumpe2fs/expect b/tests/j_ext_dumpe2fs/expect
index c75f74c..8dc95cd 100644
--- a/tests/j_ext_dumpe2fs/expect
+++ b/tests/j_ext_dumpe2fs/expect
@@ -42,7 +42,6 @@ Required extra isize: 28
Desired extra isize: 28
Default directory hash: half_md4
Checksum type: crc32c
-Checksum: 0x3a9935ab
Journal checksum type: crc32c
Journal checksum: 0x661e816f
Journal features: journal_64bit journal_checksum_v3
diff --git a/tests/t_disable_mcsum/expect b/tests/t_disable_mcsum/expect
new file mode 100644
index 0000000..e04f26a
--- /dev/null
+++ b/tests/t_disable_mcsum/expect
@@ -0,0 +1,45 @@
+tune2fs ^metadata_csum test
+Creating filesystem with 524288 1k blocks and 65536 inodes
+Superblock backups stored on blocks:
+ 8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409
+
+Allocating group tables: \b\b\b\b\bdone
+Writing inode tables: \b\b\b\b\bdone
+Creating journal (16384 blocks): done
+Creating 477 huge file(s) with 1024 blocks each: done
+Writing superblocks and filesystem accounting information: \b\b\b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
+tune2fs -O ^metadata_csum test.img
+Exit status is 0
+Change in FS metadata:
+@@ -2,7 +2,7 @@
+ Last mounted on: <not available>
+ Filesystem magic number: 0xEF53
+ Filesystem revision #: 1 (dynamic)
+-Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum
++Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
+ Default mount options: user_xattr acl
+ Filesystem state: clean
+ Errors behavior: Continue
+@@ -33,7 +33,6 @@
+ Journal inode: 8
+ Default directory hash: half_md4
+ Journal backup: inode blocks
+-Checksum type: crc32c
+ Journal features: (none)
+ Journal size: 16M
+ Journal length: 16384
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
diff --git a/tests/t_disable_mcsum/name b/tests/t_disable_mcsum/name
new file mode 100644
index 0000000..cf9dc8d
--- /dev/null
+++ b/tests/t_disable_mcsum/name
@@ -0,0 +1 @@
+disable metadata_csum
diff --git a/tests/t_disable_mcsum/script b/tests/t_disable_mcsum/script
new file mode 100644
index 0000000..7f1b3b4
--- /dev/null
+++ b/tests/t_disable_mcsum/script
@@ -0,0 +1,67 @@
+FSCK_OPT=-fn
+OUT=$test_name.log
+EXP=$test_dir/expect
+CONF=$TMPFILE.conf
+
+cat > $CONF << ENDL
+[fs_types]
+ ext4h = {
+ features = has_journal,extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,sparse_super,filetype,dir_index,ext_attr,resize_inode,64bit,metadata_csum
+ blocksize = 1024
+ inode_size = 256
+ make_hugefiles = true
+ hugefiles_dir = /
+ hugefiles_slack = 0
+ hugefiles_name = aaaaa
+ hugefiles_digits = 4
+ hugefiles_size = 1M
+ zero_hugefiles = false
+ }
+ENDL
+
+echo "tune2fs ^metadata_csum test" > $OUT
+
+MKE2FS_CONFIG=$CONF $MKE2FS -F -T ext4h $TMPFILE 524288 >> $OUT 2>&1
+rm -rf $CONF
+
+# dump and check
+$DUMPE2FS $TMPFILE 2> /dev/null | grep '^Group 0:' -B99 -A20 | sed -f $cmd_dir/filter.sed > $OUT.before
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# remove mcsum
+echo "tune2fs -O ^metadata_csum test.img" >> $OUT
+$TUNE2FS -O ^metadata_csum $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# dump and check
+$DUMPE2FS $TMPFILE 2> /dev/null | grep '^Group 0:' -B99 -A20 | sed -f $cmd_dir/filter.sed > $OUT.after
+echo "Change in FS metadata:" >> $OUT
+diff -u $OUT.before $OUT.after | tail -n +3 >> $OUT
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+rm $TMPFILE $OUT.before $OUT.after
+
+#
+# Do the verification
+#
+
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" -e 's/test_filesys:.*//g' < $OUT > $OUT.new
+mv $OUT.new $OUT
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+fi
+
+unset IMAGE FSCK_OPT OUT EXP CONF
diff --git a/tests/t_disable_mcsum_noinitbg/expect b/tests/t_disable_mcsum_noinitbg/expect
new file mode 100644
index 0000000..a022631
--- /dev/null
+++ b/tests/t_disable_mcsum_noinitbg/expect
@@ -0,0 +1,68 @@
+tune2fs ^metadata_csum test
+Creating filesystem with 524288 1k blocks and 65536 inodes
+Superblock backups stored on blocks:
+ 8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409
+
+Allocating group tables: \b\b\b\b\bdone
+Writing inode tables: \b\b\b\b\bdone
+Creating journal (16384 blocks): done
+Creating 477 huge file(s) with 1024 blocks each: done
+Writing superblocks and filesystem accounting information: \b\b\b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
+tune2fs -O ^metadata_csum,^uninit_bg test.img
+Exit status is 0
+Change in FS metadata:
+@@ -2,7 +2,7 @@
+ Last mounted on: <not available>
+ Filesystem magic number: 0xEF53
+ Filesystem revision #: 1 (dynamic)
+-Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum
++Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize
+ Default mount options: user_xattr acl
+ Filesystem state: clean
+ Errors behavior: Continue
+@@ -33,7 +33,6 @@
+ Journal inode: 8
+ Default directory hash: half_md4
+ Journal backup: inode blocks
+-Checksum type: crc32c
+ Journal features: (none)
+ Journal size: 16M
+ Journal length: 16384
+@@ -47,18 +46,18 @@
+ Block bitmap at 262 (+261)
+ Inode bitmap at 278 (+277)
+ Inode table at 294-549 (+293)
+- 21 free blocks, 536 free inodes, 2 directories, 536 unused inodes
++ 21 free blocks, 536 free inodes, 2 directories
+ Free blocks: 4413-4433
+ Free inodes: 489-1024
+-Group 1: (Blocks 8193-16384) [INODE_UNINIT]
++Group 1: (Blocks 8193-16384)
+ Backup superblock at 8193, Group descriptors at 8194-8197
+ Reserved GDT blocks at 8198-8453
+ Block bitmap at 263 (bg #0 + 262)
+ Inode bitmap at 279 (bg #0 + 278)
+ Inode table at 550-805 (bg #0 + 549)
+- 0 free blocks, 1024 free inodes, 0 directories, 1024 unused inodes
++ 0 free blocks, 1024 free inodes, 0 directories
+ Free blocks:
+ Free inodes: 1025-2048
+-Group 2: (Blocks 16385-24576) [INODE_UNINIT]
++Group 2: (Blocks 16385-24576)
+ Block bitmap at 264 (bg #0 + 263)
+ Inode bitmap at 280 (bg #0 + 279)
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
diff --git a/tests/t_disable_mcsum_noinitbg/name b/tests/t_disable_mcsum_noinitbg/name
new file mode 100644
index 0000000..4b906be
--- /dev/null
+++ b/tests/t_disable_mcsum_noinitbg/name
@@ -0,0 +1 @@
+disable metadata_csum and uninit_bg
diff --git a/tests/t_disable_mcsum_noinitbg/script b/tests/t_disable_mcsum_noinitbg/script
new file mode 100644
index 0000000..db49864
--- /dev/null
+++ b/tests/t_disable_mcsum_noinitbg/script
@@ -0,0 +1,67 @@
+FSCK_OPT=-fn
+OUT=$test_name.log
+EXP=$test_dir/expect
+CONF=$TMPFILE.conf
+
+cat > $CONF << ENDL
+[fs_types]
+ ext4h = {
+ features = has_journal,extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,sparse_super,filetype,dir_index,ext_attr,resize_inode,64bit,metadata_csum
+ blocksize = 1024
+ inode_size = 256
+ make_hugefiles = true
+ hugefiles_dir = /
+ hugefiles_slack = 0
+ hugefiles_name = aaaaa
+ hugefiles_digits = 4
+ hugefiles_size = 1M
+ zero_hugefiles = false
+ }
+ENDL
+
+echo "tune2fs ^metadata_csum test" > $OUT
+
+MKE2FS_CONFIG=$CONF $MKE2FS -F -T ext4h $TMPFILE 524288 >> $OUT 2>&1
+rm -rf $CONF
+
+# dump and check
+$DUMPE2FS $TMPFILE 2> /dev/null | grep '^Group 0:' -B99 -A20 | sed -f $cmd_dir/filter.sed > $OUT.before
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# remove mcsum
+echo "tune2fs -O ^metadata_csum,^uninit_bg test.img" >> $OUT
+$TUNE2FS -O ^metadata_csum,^uninit_bg $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# dump and check
+$DUMPE2FS $TMPFILE 2> /dev/null | grep '^Group 0:' -B99 -A20 | sed -f $cmd_dir/filter.sed > $OUT.after
+echo "Change in FS metadata:" >> $OUT
+diff -u $OUT.before $OUT.after | tail -n +3 >> $OUT
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+rm $TMPFILE $OUT.before $OUT.after
+
+#
+# Do the verification
+#
+
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" -e 's/test_filesys:.*//g' < $OUT > $OUT.new
+mv $OUT.new $OUT
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+fi
+
+unset IMAGE FSCK_OPT OUT EXP CONF
diff --git a/tests/t_disable_mcsum_yesinitbg/expect b/tests/t_disable_mcsum_yesinitbg/expect
new file mode 100644
index 0000000..df3d6c0
--- /dev/null
+++ b/tests/t_disable_mcsum_yesinitbg/expect
@@ -0,0 +1,45 @@
+tune2fs ^metadata_csum test
+Creating filesystem with 524288 1k blocks and 65536 inodes
+Superblock backups stored on blocks:
+ 8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409
+
+Allocating group tables: \b\b\b\b\bdone
+Writing inode tables: \b\b\b\b\bdone
+Creating journal (16384 blocks): done
+Creating 477 huge file(s) with 1024 blocks each: done
+Writing superblocks and filesystem accounting information: \b\b\b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
+tune2fs -O ^metadata_csum,uninit_bg test.img
+Exit status is 0
+Change in FS metadata:
+@@ -2,7 +2,7 @@
+ Last mounted on: <not available>
+ Filesystem magic number: 0xEF53
+ Filesystem revision #: 1 (dynamic)
+-Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum
++Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
+ Default mount options: user_xattr acl
+ Filesystem state: clean
+ Errors behavior: Continue
+@@ -33,7 +33,6 @@
+ Journal inode: 8
+ Default directory hash: half_md4
+ Journal backup: inode blocks
+-Checksum type: crc32c
+ Journal features: (none)
+ Journal size: 16M
+ Journal length: 16384
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
diff --git a/tests/t_disable_mcsum_yesinitbg/name b/tests/t_disable_mcsum_yesinitbg/name
new file mode 100644
index 0000000..880de1c
--- /dev/null
+++ b/tests/t_disable_mcsum_yesinitbg/name
@@ -0,0 +1 @@
+disable metadata_csum and enable uninit_bg
diff --git a/tests/t_disable_mcsum_yesinitbg/script b/tests/t_disable_mcsum_yesinitbg/script
new file mode 100644
index 0000000..2d04041
--- /dev/null
+++ b/tests/t_disable_mcsum_yesinitbg/script
@@ -0,0 +1,67 @@
+FSCK_OPT=-fn
+OUT=$test_name.log
+EXP=$test_dir/expect
+CONF=$TMPFILE.conf
+
+cat > $CONF << ENDL
+[fs_types]
+ ext4h = {
+ features = has_journal,extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,sparse_super,filetype,dir_index,ext_attr,resize_inode,64bit,metadata_csum
+ blocksize = 1024
+ inode_size = 256
+ make_hugefiles = true
+ hugefiles_dir = /
+ hugefiles_slack = 0
+ hugefiles_name = aaaaa
+ hugefiles_digits = 4
+ hugefiles_size = 1M
+ zero_hugefiles = false
+ }
+ENDL
+
+echo "tune2fs ^metadata_csum test" > $OUT
+
+MKE2FS_CONFIG=$CONF $MKE2FS -F -T ext4h $TMPFILE 524288 >> $OUT 2>&1
+rm -rf $CONF
+
+# dump and check
+$DUMPE2FS $TMPFILE 2> /dev/null | grep '^Group 0:' -B99 -A20 | sed -f $cmd_dir/filter.sed > $OUT.before
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# remove mcsum
+echo "tune2fs -O ^metadata_csum,uninit_bg test.img" >> $OUT
+$TUNE2FS -O ^metadata_csum,uninit_bg $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# dump and check
+$DUMPE2FS $TMPFILE 2> /dev/null | grep '^Group 0:' -B99 -A20 | sed -f $cmd_dir/filter.sed > $OUT.after
+echo "Change in FS metadata:" >> $OUT
+diff -u $OUT.before $OUT.after | tail -n +3 >> $OUT
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+rm $TMPFILE $OUT.before $OUT.after
+
+#
+# Do the verification
+#
+
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" -e 's/test_filesys:.*//g' < $OUT > $OUT.new
+mv $OUT.new $OUT
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+fi
+
+unset IMAGE FSCK_OPT OUT EXP CONF
diff --git a/tests/t_enable_mcsum/expect b/tests/t_enable_mcsum/expect
new file mode 100644
index 0000000..2ee3c27
--- /dev/null
+++ b/tests/t_enable_mcsum/expect
@@ -0,0 +1,78 @@
+tune2fs metadata_csum test
+Creating filesystem with 524288 1k blocks and 65536 inodes
+Superblock backups stored on blocks:
+ 8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409
+
+Allocating group tables: \b\b\b\b\bdone
+Writing inode tables: \b\b\b\b\bdone
+Creating journal (16384 blocks): done
+Creating 477 huge file(s) with 1024 blocks each: done
+Writing superblocks and filesystem accounting information: \b\b\b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
+tune2fs -O metadata_csum test.img
+
+Please run e2fsck -D on the filesystem.
+
+Exit status is 0
+test_filesys was not cleanly unmounted, check forced.
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 3A: Optimizing directories
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+
+
+Change in FS metadata:
+@@ -2,7 +2,7 @@
+ Last mounted on: <not available>
+ Filesystem magic number: 0xEF53
+ Filesystem revision #: 1 (dynamic)
+-Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
++Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum
+ Default mount options: user_xattr acl
+ Filesystem state: clean
+ Errors behavior: Continue
+@@ -10,7 +10,7 @@
+ Inode count: 65536
+ Block count: 524288
+ Reserved block count: 26214
+-Free blocks: 571
++Free blocks: 568
+ Free inodes: 65048
+ First block: 1
+ Block size: 1024
+@@ -33,6 +33,7 @@
+ Journal inode: 8
+ Default directory hash: half_md4
+ Journal backup: inode blocks
++Checksum type: crc32c
+ Journal features: (none)
+ Journal size: 16M
+ Journal length: 16384
+@@ -46,8 +47,8 @@
+ Block bitmap at 262 (+261)
+ Inode bitmap at 278 (+277)
+ Inode table at 294-549 (+293)
+- 21 free blocks, 536 free inodes, 2 directories, 536 unused inodes
+- Free blocks: 4413-4433
++ 18 free blocks, 536 free inodes, 2 directories, 536 unused inodes
++ Free blocks: 4413, 4417-4433
+ Free inodes: 489-1024
+ Group 1: (Blocks 8193-16384) [INODE_UNINIT]
+ Backup superblock at 8193, Group descriptors at 8194-8197
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
diff --git a/tests/t_enable_mcsum/name b/tests/t_enable_mcsum/name
new file mode 100644
index 0000000..22f295a
--- /dev/null
+++ b/tests/t_enable_mcsum/name
@@ -0,0 +1 @@
+enable metadata_csum
diff --git a/tests/t_enable_mcsum/script b/tests/t_enable_mcsum/script
new file mode 100644
index 0000000..5239eb3
--- /dev/null
+++ b/tests/t_enable_mcsum/script
@@ -0,0 +1,70 @@
+FSCK_OPT=-fn
+OUT=$test_name.log
+EXP=$test_dir/expect
+CONF=$TMPFILE.conf
+
+cat > $CONF << ENDL
+[fs_types]
+ ext4h = {
+ features = has_journal,extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,sparse_super,filetype,dir_index,ext_attr,resize_inode,64bit
+ blocksize = 1024
+ inode_size = 256
+ make_hugefiles = true
+ hugefiles_dir = /
+ hugefiles_slack = 0
+ hugefiles_name = aaaaa
+ hugefiles_digits = 4
+ hugefiles_size = 1M
+ zero_hugefiles = false
+ }
+ENDL
+
+echo "tune2fs metadata_csum test" > $OUT
+
+MKE2FS_CONFIG=$CONF $MKE2FS -F -T ext4h $TMPFILE 524288 >> $OUT 2>&1
+rm -rf $CONF
+
+# dump and check
+$DUMPE2FS $TMPFILE 2> /dev/null | grep '^Group 0:' -B99 -A20 | sed -f $cmd_dir/filter.sed > $OUT.before
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# add mcsum
+echo "tune2fs -O metadata_csum test.img" >> $OUT
+$TUNE2FS -O metadata_csum $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# check
+$FSCK -yD -N test_filesys $TMPFILE >> $OUT 2>&1
+
+# dump and check
+$DUMPE2FS $TMPFILE 2> /dev/null | grep '^Group 0:' -B99 -A20 | sed -f $cmd_dir/filter.sed > $OUT.after
+echo "Change in FS metadata:" >> $OUT
+diff -u $OUT.before $OUT.after | tail -n +3 >> $OUT
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+rm $TMPFILE $OUT.before $OUT.after
+
+#
+# Do the verification
+#
+
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" -e 's/test_filesys:.*//g' < $OUT > $OUT.new
+mv $OUT.new $OUT
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+fi
+
+unset IMAGE FSCK_OPT OUT EXP CONF
diff --git a/tests/t_enable_mcsum_ext3/expect b/tests/t_enable_mcsum_ext3/expect
new file mode 100644
index 0000000..1e005bd
--- /dev/null
+++ b/tests/t_enable_mcsum_ext3/expect
@@ -0,0 +1,73 @@
+tune2fs metadata_csum test
+Creating filesystem with 524288 1k blocks and 65536 inodes
+Superblock backups stored on blocks:
+ 8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409
+
+Allocating group tables: \b\b\b\b\bdone
+Writing inode tables: \b\b\b\b\bdone
+Creating journal (16384 blocks): done
+mke2fs: Operation not supported for inodes containing extents while creating huge files
+Writing superblocks and filesystem accounting information: \b\b\b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
+tune2fs -O metadata_csum test.img
+Extents are not enabled. The file extent tree can be checksummed, whereas block maps cannot. Not enabling extents reduces the coverage of metadata checksumming. Re-run with -O extent to rectify.
+Exit status is 0
+
+Change in FS metadata:
+@@ -2,7 +2,7 @@
+ Last mounted on: <not available>
+ Filesystem magic number: 0xEF53
+ Filesystem revision #: 1 (dynamic)
+-Filesystem features: has_journal ext_attr resize_inode dir_index filetype sparse_super large_file
++Filesystem features: has_journal ext_attr resize_inode dir_index filetype sparse_super large_file metadata_csum
+ Default mount options: user_xattr acl
+ Filesystem state: clean
+ Errors behavior: Continue
+@@ -29,6 +29,7 @@
+ Journal inode: 8
+ Default directory hash: half_md4
+ Journal backup: inode blocks
++Checksum type: crc32c
+ Journal features: (none)
+ Journal size: 16M
+ Journal length: 16384
+@@ -36,7 +37,7 @@
+ Journal start: 0
+
+
+-Group 0: (Blocks 1-8192)
++Group 0: (Blocks 1-8192) [ITABLE_ZEROED]
+ Primary superblock at 1, Group descriptors at 2-3
+ Reserved GDT blocks at 4-259
+ Block bitmap at 260 (+259)
+@@ -45,7 +46,7 @@
+ 7789 free blocks, 1013 free inodes, 2 directories
+ Free blocks: 404-8192
+ Free inodes: 12-1024
+-Group 1: (Blocks 8193-16384)
++Group 1: (Blocks 8193-16384) [ITABLE_ZEROED]
+ Backup superblock at 8193, Group descriptors at 8194-8195
+ Reserved GDT blocks at 8196-8451
+ Block bitmap at 8452 (+259)
+@@ -54,6 +55,6 @@
+ 7803 free blocks, 1024 free inodes, 0 directories
+ Free blocks: 8582-16384
+ Free inodes: 1025-2048
+-Group 2: (Blocks 16385-24576)
++Group 2: (Blocks 16385-24576) [ITABLE_ZEROED]
+ Block bitmap at 16385 (+0)
+ Inode bitmap at 16386 (+1)
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
diff --git a/tests/t_enable_mcsum_ext3/name b/tests/t_enable_mcsum_ext3/name
new file mode 100644
index 0000000..d9a29c8
--- /dev/null
+++ b/tests/t_enable_mcsum_ext3/name
@@ -0,0 +1 @@
+enable metadata_csum on ext3 fs
diff --git a/tests/t_enable_mcsum_ext3/script b/tests/t_enable_mcsum_ext3/script
new file mode 100644
index 0000000..ac63866
--- /dev/null
+++ b/tests/t_enable_mcsum_ext3/script
@@ -0,0 +1,70 @@
+FSCK_OPT=-fn
+OUT=$test_name.log
+EXP=$test_dir/expect
+CONF=$TMPFILE.conf
+
+cat > $CONF << ENDL
+[fs_types]
+ ext4h = {
+ features = has_journal,^extent,^huge_file,^flex_bg,^uninit_bg,^dir_nlink,^extra_isize,sparse_super,filetype,dir_index,ext_attr,resize_inode,^64bit
+ blocksize = 1024
+ inode_size = 128
+ make_hugefiles = true
+ hugefiles_dir = /
+ num_hugefiles = 10
+ hugefiles_name = aaaaa
+ hugefiles_digits = 4
+ hugefiles_size = 1M
+ zero_hugefiles = false
+ }
+ENDL
+
+echo "tune2fs metadata_csum test" > $OUT
+
+MKE2FS_CONFIG=$CONF $MKE2FS -F -T ext4h $TMPFILE 524288 >> $OUT 2>&1
+rm -rf $CONF
+
+# dump and check
+$DUMPE2FS $TMPFILE 2> /dev/null | grep '^Group 0:' -B99 -A20 | sed -f $cmd_dir/filter.sed > $OUT.before
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# add mcsum
+echo "tune2fs -O metadata_csum test.img" >> $OUT
+$TUNE2FS -O metadata_csum $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# check
+$FSCK -yD -N test_filesys $TMPFILE >> $OUT 2>&1
+
+# dump and check
+$DUMPE2FS $TMPFILE 2> /dev/null | grep '^Group 0:' -B99 -A20 | sed -f $cmd_dir/filter.sed > $OUT.after
+echo "Change in FS metadata:" >> $OUT
+diff -u $OUT.before $OUT.after | tail -n +3 >> $OUT
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+rm $TMPFILE $OUT.before $OUT.after
+
+#
+# Do the verification
+#
+
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" -e 's/test_filesys:.*//g' < $OUT > $OUT.new
+mv $OUT.new $OUT
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+fi
+
+unset IMAGE FSCK_OPT OUT EXP CONF
diff --git a/tests/t_enable_mcsum_initbg/expect b/tests/t_enable_mcsum_initbg/expect
new file mode 100644
index 0000000..d3b4444
--- /dev/null
+++ b/tests/t_enable_mcsum_initbg/expect
@@ -0,0 +1,98 @@
+tune2fs metadata_csum test
+Creating filesystem with 524288 1k blocks and 65536 inodes
+Superblock backups stored on blocks:
+ 8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409
+
+Allocating group tables: \b\b\b\b\bdone
+Writing inode tables: \b\b\b\b\bdone
+Creating journal (16384 blocks): done
+Creating 477 huge file(s) with 1024 blocks each: done
+Writing superblocks and filesystem accounting information: \b\b\b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
+tune2fs -O metadata_csum test.img
+
+Please run e2fsck -D on the filesystem.
+
+Exit status is 0
+test_filesys was not cleanly unmounted, check forced.
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 3A: Optimizing directories
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+
+
+Change in FS metadata:
+@@ -2,7 +2,7 @@
+ Last mounted on: <not available>
+ Filesystem magic number: 0xEF53
+ Filesystem revision #: 1 (dynamic)
+-Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize
++Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum
+ Default mount options: user_xattr acl
+ Filesystem state: clean
+ Errors behavior: Continue
+@@ -10,7 +10,7 @@
+ Inode count: 65536
+ Block count: 524288
+ Reserved block count: 26214
+-Free blocks: 571
++Free blocks: 568
+ Free inodes: 65048
+ First block: 1
+ Block size: 1024
+@@ -33,6 +33,7 @@
+ Journal inode: 8
+ Default directory hash: half_md4
+ Journal backup: inode blocks
++Checksum type: crc32c
+ Journal features: (none)
+ Journal size: 16M
+ Journal length: 16384
+@@ -40,24 +41,24 @@
+ Journal start: 0
+
+
+-Group 0: (Blocks 1-8192)
++Group 0: (Blocks 1-8192) [ITABLE_ZEROED]
+ Primary superblock at 1, Group descriptors at 2-5
+ Reserved GDT blocks at 6-261
+ Block bitmap at 262 (+261)
+ Inode bitmap at 278 (+277)
+ Inode table at 294-549 (+293)
+- 21 free blocks, 536 free inodes, 2 directories
+- Free blocks: 4413-4433
++ 18 free blocks, 536 free inodes, 2 directories, 536 unused inodes
++ Free blocks: 4413, 4417-4433
+ Free inodes: 489-1024
+-Group 1: (Blocks 8193-16384)
++Group 1: (Blocks 8193-16384) [INODE_UNINIT, ITABLE_ZEROED]
+ Backup superblock at 8193, Group descriptors at 8194-8197
+ Reserved GDT blocks at 8198-8453
+ Block bitmap at 263 (bg #0 + 262)
+ Inode bitmap at 279 (bg #0 + 278)
+ Inode table at 550-805 (bg #0 + 549)
+- 0 free blocks, 1024 free inodes, 0 directories
++ 0 free blocks, 1024 free inodes, 0 directories, 1024 unused inodes
+ Free blocks:
+ Free inodes: 1025-2048
+-Group 2: (Blocks 16385-24576)
++Group 2: (Blocks 16385-24576) [INODE_UNINIT, ITABLE_ZEROED]
+ Block bitmap at 264 (bg #0 + 263)
+ Inode bitmap at 280 (bg #0 + 279)
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
diff --git a/tests/t_enable_mcsum_initbg/name b/tests/t_enable_mcsum_initbg/name
new file mode 100644
index 0000000..291eb6b
--- /dev/null
+++ b/tests/t_enable_mcsum_initbg/name
@@ -0,0 +1 @@
+enable metadata_csum when ^uninit_bg
diff --git a/tests/t_enable_mcsum_initbg/script b/tests/t_enable_mcsum_initbg/script
new file mode 100644
index 0000000..35c4541
--- /dev/null
+++ b/tests/t_enable_mcsum_initbg/script
@@ -0,0 +1,70 @@
+FSCK_OPT=-fn
+OUT=$test_name.log
+EXP=$test_dir/expect
+CONF=$TMPFILE.conf
+
+cat > $CONF << ENDL
+[fs_types]
+ ext4h = {
+ features = has_journal,extent,huge_file,flex_bg,^uninit_bg,dir_nlink,extra_isize,sparse_super,filetype,dir_index,ext_attr,resize_inode,64bit
+ blocksize = 1024
+ inode_size = 256
+ make_hugefiles = true
+ hugefiles_dir = /
+ hugefiles_slack = 0
+ hugefiles_name = aaaaa
+ hugefiles_digits = 4
+ hugefiles_size = 1M
+ zero_hugefiles = false
+ }
+ENDL
+
+echo "tune2fs metadata_csum test" > $OUT
+
+MKE2FS_CONFIG=$CONF $MKE2FS -F -T ext4h $TMPFILE 524288 >> $OUT 2>&1
+rm -rf $CONF
+
+# dump and check
+$DUMPE2FS $TMPFILE 2> /dev/null | grep '^Group 0:' -B99 -A20 | sed -f $cmd_dir/filter.sed > $OUT.before
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# add mcsum
+echo "tune2fs -O metadata_csum test.img" >> $OUT
+$TUNE2FS -O metadata_csum $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# check
+$FSCK -yD -N test_filesys $TMPFILE >> $OUT 2>&1
+
+# dump and check
+$DUMPE2FS $TMPFILE 2> /dev/null | grep '^Group 0:' -B99 -A20 | sed -f $cmd_dir/filter.sed > $OUT.after
+echo "Change in FS metadata:" >> $OUT
+diff -u $OUT.before $OUT.after | tail -n +3 >> $OUT
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+rm $TMPFILE $OUT.before $OUT.after
+
+#
+# Do the verification
+#
+
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" -e 's/test_filesys:.*//g' < $OUT > $OUT.new
+mv $OUT.new $OUT
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+fi
+
+unset IMAGE FSCK_OPT OUT EXP CONF
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 23/47] mke2fs: don't zero inode table blocks that are already zeroed
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (21 preceding siblings ...)
2014-11-07 21:53 ` [PATCH 22/47] tests: testcases for enabling/disabling metadata_csum via tune2fs Darrick J. Wong
@ 2014-11-07 21:53 ` Darrick J. Wong
2014-12-14 3:01 ` Theodore Ts'o
2014-11-07 21:53 ` [PATCH 24/47] resize2fs: don't exit if shrinking sparse_super2 fs to one bg Darrick J. Wong
` (25 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:53 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
At mke2fs time, if we discard the device and discard zeroes data,
don't bother zeroing the inode table blocks a second time.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
misc/mke2fs.c | 14 ++++++++------
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git a/misc/mke2fs.c b/misc/mke2fs.c
index 0a0d2b3..aeb852f 100644
--- a/misc/mke2fs.c
+++ b/misc/mke2fs.c
@@ -421,12 +421,14 @@ static void write_inode_tables(ext2_filsys fs, int lazy_flag, int itable_zeroed)
ext2fs_bg_flags_set(fs, i, EXT2_BG_INODE_ZEROED);
ext2fs_group_desc_csum_set(fs, i);
}
- retval = ext2fs_zero_blocks2(fs, blk, num, &blk, &num);
- if (retval) {
- fprintf(stderr, _("\nCould not write %d "
- "blocks in inode table starting at %llu: %s\n"),
- num, blk, error_message(retval));
- exit(1);
+ if (!itable_zeroed) {
+ retval = ext2fs_zero_blocks2(fs, blk, num, &blk, &num);
+ if (retval) {
+ fprintf(stderr, _("\nCould not write %d "
+ "blocks in inode table starting at %llu: %s\n"),
+ num, blk, error_message(retval));
+ exit(1);
+ }
}
if (sync_kludge) {
if (sync_kludge == 1)
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 24/47] resize2fs: don't exit if shrinking sparse_super2 fs to one bg
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (22 preceding siblings ...)
2014-11-07 21:53 ` [PATCH 23/47] mke2fs: don't zero inode table blocks that are already zeroed Darrick J. Wong
@ 2014-11-07 21:53 ` Darrick J. Wong
2014-12-14 3:06 ` Theodore Ts'o
2014-11-07 21:53 ` [PATCH 25/47] resize2fs: quickly rewrite extent blocks when moving an inode w/ metadata_csum Darrick J. Wong
` (24 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:53 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
If we're shrinking a sparse_super2 filesystem to a single block group,
the superblock will be in block 0. This is perfectly valid (for block
group 0 with a blocksize > 1024) so don't exit.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
resize/resize2fs.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/resize/resize2fs.c b/resize/resize2fs.c
index 8c5db86..4861719 100644
--- a/resize/resize2fs.c
+++ b/resize/resize2fs.c
@@ -2126,7 +2126,7 @@ static errcode_t reserve_sparse_super2_last_group(ext2_resize_t rfs,
if (retval)
return retval;
- if (!sb) {
+ if (last_bg && !sb) {
fputs(_("Should never happen! No sb in last super_sparse bg?\n"),
stderr);
exit(1);
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 25/47] resize2fs: quickly rewrite extent blocks when moving an inode w/ metadata_csum
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (23 preceding siblings ...)
2014-11-07 21:53 ` [PATCH 24/47] resize2fs: don't exit if shrinking sparse_super2 fs to one bg Darrick J. Wong
@ 2014-11-07 21:53 ` Darrick J. Wong
2014-12-14 3:09 ` Theodore Ts'o
2014-11-07 21:53 ` [PATCH 26/47] resize2fs: use old_fs to detect per-bg metadata blocks to free Darrick J. Wong
` (23 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:53 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
When we're moving an inode on a metadata_csum filesystem, we need to
rewrite the checksum of all interior nodes of the extent tree. The
current code does this inefficiently via set_bmap, but we can do this
more efficiently through direct iteration of the extent tree.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
resize/resize2fs.c | 74 +++++++++++++++++++++++++++++++++++++++++++++-------
1 file changed, 64 insertions(+), 10 deletions(-)
diff --git a/resize/resize2fs.c b/resize/resize2fs.c
index 4861719..1415244 100644
--- a/resize/resize2fs.c
+++ b/resize/resize2fs.c
@@ -1492,16 +1492,6 @@ static int process_block(ext2_filsys fs, blk64_t *block_nr,
}
}
- /*
- * If we moved inodes and metadata_csum is enabled, we must force the
- * extent block to be rewritten with new checksum.
- */
- if (EXT2_HAS_RO_COMPAT_FEATURE(fs->super,
- EXT4_FEATURE_RO_COMPAT_METADATA_CSUM) &&
- pb->has_extents &&
- pb->old_ino != pb->ino)
- ret |= BLOCK_CHANGED;
-
if (pb->is_dir) {
retval = ext2fs_add_dir_block2(fs->dblist, pb->ino,
block, (int) blockcnt);
@@ -1581,6 +1571,59 @@ out:
return err;
}
+/* Rewrite extents */
+static errcode_t rewrite_extents(ext2_filsys fs, ext2_ino_t ino)
+{
+ ext2_extent_handle_t handle;
+ struct ext2fs_extent extent;
+ errcode_t errcode;
+ struct ext2_extent_info info;
+
+ errcode = ext2fs_extent_open(fs, ino, &handle);
+ if (errcode)
+ return errcode;
+
+ errcode = ext2fs_extent_get(handle, EXT2_EXTENT_ROOT, &extent);
+ if (errcode)
+ goto out;
+
+ do {
+ errcode = ext2fs_extent_get_info(handle, &info);
+ if (errcode)
+ break;
+
+ /*
+ * If this is the first extent in an extent block that we
+ * haven't visited, rewrite the extent to force the ETB
+ * checksum to be rewritten.
+ */
+ if (info.curr_entry == 1 && info.curr_level != 0 &&
+ !(extent.e_flags & EXT2_EXTENT_FLAGS_SECOND_VISIT)) {
+ errcode = ext2fs_extent_replace(handle, 0, &extent);
+ if (errcode)
+ break;
+ }
+
+ /* Skip to the end of a block of leaf nodes */
+ if (extent.e_flags & EXT2_EXTENT_FLAGS_LEAF) {
+ errcode = ext2fs_extent_get(handle,
+ EXT2_EXTENT_LAST_SIB,
+ &extent);
+ if (errcode)
+ break;
+ }
+
+ errcode = ext2fs_extent_get(handle, EXT2_EXTENT_NEXT, &extent);
+ } while (errcode == 0);
+
+out:
+ /* Ok if we run off the end */
+ if (errcode == EXT2_ET_EXTENT_NO_NEXT)
+ errcode = 0;
+ ext2fs_extent_free(handle);
+ return errcode;
+}
+
static errcode_t inode_scan_and_fix(ext2_resize_t rfs)
{
struct process_block_struct pb;
@@ -1699,6 +1742,17 @@ remap_blocks:
if (retval)
goto errout;
+ /* Rewrite extent block checksums with new inode number */
+ if (EXT2_HAS_RO_COMPAT_FEATURE(rfs->old_fs->super,
+ EXT4_FEATURE_RO_COMPAT_METADATA_CSUM) &&
+ (inode->i_flags & EXT4_EXTENTS_FL)) {
+ rfs->old_fs->flags |= EXT2_FLAG_IGNORE_CSUM_ERRORS;
+ retval = rewrite_extents(rfs->old_fs, new_inode);
+ rfs->old_fs->flags &= ~EXT2_FLAG_IGNORE_CSUM_ERRORS;
+ if (retval)
+ goto errout;
+ }
+
/*
* Update inodes to point to new blocks; schedule directory
* blocks for inode remapping. Need to write out dir blocks
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 26/47] resize2fs: use old_fs to detect per-bg metadata blocks to free
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (24 preceding siblings ...)
2014-11-07 21:53 ` [PATCH 25/47] resize2fs: quickly rewrite extent blocks when moving an inode w/ metadata_csum Darrick J. Wong
@ 2014-11-07 21:53 ` Darrick J. Wong
2014-12-15 2:09 ` Theodore Ts'o
2014-11-07 21:53 ` [PATCH 27/47] resize2fs: don't interpret bitmap shift while crossing flexbg as raid stride Darrick J. Wong
` (22 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:53 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
When shrinking a filesystem, resize2fs wants to free per-bg metadata
blocks that are no longer needed. This behavior is gated on whether
there's a superblock in the group as told by new_fs. The check really
should be against old_fs, since we're effectively freeing blocks out
of old_fs in the transition to new_fs, but prior to sparse_super2 this
didn't matter since superblocks didn't move, so it didn't matter.
Under sparse_super2, however, there's a superblock in the last group,
so now we need to change the test to use old_fs as it should.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
resize/resize2fs.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/resize/resize2fs.c b/resize/resize2fs.c
index 1415244..da1ec9c 100644
--- a/resize/resize2fs.c
+++ b/resize/resize2fs.c
@@ -1042,7 +1042,7 @@ static errcode_t blocks_to_move(ext2_resize_t rfs)
}
for (i = 0; i < max_groups; i++) {
- if (!ext2fs_bg_has_super(fs, i)) {
+ if (!ext2fs_bg_has_super(old_fs, i)) {
group_blk += fs->super->s_blocks_per_group;
continue;
}
^ permalink raw reply related [flat|nested] 104+ messages in thread* Re: [PATCH 26/47] resize2fs: use old_fs to detect per-bg metadata blocks to free
2014-11-07 21:53 ` [PATCH 26/47] resize2fs: use old_fs to detect per-bg metadata blocks to free Darrick J. Wong
@ 2014-12-15 2:09 ` Theodore Ts'o
0 siblings, 0 replies; 104+ messages in thread
From: Theodore Ts'o @ 2014-12-15 2:09 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Fri, Nov 07, 2014 at 01:53:35PM -0800, Darrick J. Wong wrote:
> When shrinking a filesystem, resize2fs wants to free per-bg metadata
> blocks that are no longer needed. This behavior is gated on whether
> there's a superblock in the group as told by new_fs. The check really
> should be against old_fs, since we're effectively freeing blocks out
> of old_fs in the transition to new_fs, but prior to sparse_super2 this
> didn't matter since superblocks didn't move, so it didn't matter.
>
> Under sparse_super2, however, there's a superblock in the last group,
> so now we need to change the test to use old_fs as it should.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Applied, thanks.
- Ted
^ permalink raw reply [flat|nested] 104+ messages in thread
* [PATCH 27/47] resize2fs: don't interpret bitmap shift while crossing flexbg as raid stride
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (25 preceding siblings ...)
2014-11-07 21:53 ` [PATCH 26/47] resize2fs: use old_fs to detect per-bg metadata blocks to free Darrick J. Wong
@ 2014-11-07 21:53 ` Darrick J. Wong
2014-12-15 2:11 ` Theodore Ts'o
2014-11-07 21:53 ` [PATCH 28/47] resize2fs: set bg flags and unused inode count when resizing Darrick J. Wong
` (21 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:53 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4, TR Reardon
resize2fs tries to infer the RAID stride by observing differences between the
locations of adjacent block groups' block and inode bitmaps within the block
group. If the two block groups being compared belong to different flexbgs,
however, it'll be fooled by the large offset into thinking that the FS has an
abnormally large RAID stride.
Therefore, teach it not to get confused by crossing a flexbg.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: TR Reardon <thomas_reardon@hotmail.com>
---
resize/main.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/resize/main.c b/resize/main.c
index 983d8c2..e017792 100644
--- a/resize/main.c
+++ b/resize/main.c
@@ -105,6 +105,7 @@ static void determine_fs_stride(ext2_filsys fs)
unsigned long long sum;
unsigned int has_sb, prev_has_sb = 0, num;
int i_stride, b_stride;
+ int flexbg_size = 1 << fs->super->s_log_groups_per_flex;
if (fs->stride)
return;
@@ -120,7 +121,8 @@ static void determine_fs_stride(ext2_filsys fs)
ext2fs_inode_bitmap_loc(fs, group - 1) -
fs->super->s_blocks_per_group;
if (b_stride != i_stride ||
- b_stride < 0)
+ b_stride < 0 ||
+ (flexbg_size > 1 && (group % flexbg_size == 0)))
goto next;
/* printf("group %d has stride %d\n", group, b_stride); */
^ permalink raw reply related [flat|nested] 104+ messages in thread* Re: [PATCH 27/47] resize2fs: don't interpret bitmap shift while crossing flexbg as raid stride
2014-11-07 21:53 ` [PATCH 27/47] resize2fs: don't interpret bitmap shift while crossing flexbg as raid stride Darrick J. Wong
@ 2014-12-15 2:11 ` Theodore Ts'o
0 siblings, 0 replies; 104+ messages in thread
From: Theodore Ts'o @ 2014-12-15 2:11 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4, TR Reardon
On Fri, Nov 07, 2014 at 01:53:41PM -0800, Darrick J. Wong wrote:
> resize2fs tries to infer the RAID stride by observing differences between the
> locations of adjacent block groups' block and inode bitmaps within the block
> group. If the two block groups being compared belong to different flexbgs,
> however, it'll be fooled by the large offset into thinking that the FS has an
> abnormally large RAID stride.
>
> Therefore, teach it not to get confused by crossing a flexbg.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> Reported-by: TR Reardon <thomas_reardon@hotmail.com>
Applied, thanks.
- Ted
^ permalink raw reply [flat|nested] 104+ messages in thread
* [PATCH 28/47] resize2fs: set bg flags and unused inode count when resizing
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (26 preceding siblings ...)
2014-11-07 21:53 ` [PATCH 27/47] resize2fs: don't interpret bitmap shift while crossing flexbg as raid stride Darrick J. Wong
@ 2014-11-07 21:53 ` Darrick J. Wong
2014-12-15 2:12 ` Theodore Ts'o
2014-11-07 21:53 ` [PATCH 29/47] resize2fs: don't mark unallocated bg metadata blocks when fixing bg flags Darrick J. Wong
` (20 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:53 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Recalculate the unused inode count and the block/inode uninit flags
when resizing a filesystem. This can speed up future e2fsck runs
considerably and will reduce mount times.
v2: Use a library function to take care of this.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
resize/resize2fs.c | 4 ++++
tests/r_fixup_lastbg/expect | 39 +++++++++++++++++++++++++++++++++++
tests/r_fixup_lastbg/script | 37 ++++++++++++++++++++++++++++++++++
tests/r_fixup_lastbg_big/expect | 43 +++++++++++++++++++++++++++++++++++++++
tests/r_fixup_lastbg_big/script | 37 ++++++++++++++++++++++++++++++++++
5 files changed, 160 insertions(+)
create mode 100644 tests/r_fixup_lastbg/expect
create mode 100755 tests/r_fixup_lastbg/script
create mode 100644 tests/r_fixup_lastbg_big/expect
create mode 100755 tests/r_fixup_lastbg_big/script
diff --git a/resize/resize2fs.c b/resize/resize2fs.c
index da1ec9c..94c0643 100644
--- a/resize/resize2fs.c
+++ b/resize/resize2fs.c
@@ -198,6 +198,10 @@ errcode_t resize_fs(ext2_filsys fs, blk64_t *new_size, int flags,
if (retval)
goto errout;
+ retval = ext2fs_set_gdt_csum(rfs->new_fs);
+ if (retval)
+ goto errout;
+
rfs->new_fs->super->s_state &= ~EXT2_ERROR_FS;
rfs->new_fs->flags &= ~EXT2_FLAG_MASTER_SB_ONLY;
diff --git a/tests/r_fixup_lastbg/expect b/tests/r_fixup_lastbg/expect
new file mode 100644
index 0000000..96b154a
--- /dev/null
+++ b/tests/r_fixup_lastbg/expect
@@ -0,0 +1,39 @@
+Creating filesystem with 20000 1k blocks and 1248 inodes
+Superblock backups stored on blocks:
+ 8193
+
+Allocating group tables: \b\b\bdone
+Writing inode tables: \b\b\bdone
+Creating journal (1024 blocks): done
+Writing superblocks and filesystem accounting information: \b\b\bdone
+
+Group 2: (Blocks 16385-19999) [INODE_UNINIT, ITABLE_ZEROED]
+ Block bitmap at 83 (bg #0 + 82)
+ Inode bitmap at 86 (bg #0 + 85)
+ Inode table at 295-398 (bg #0 + 294)
+ 3615 free blocks, 416 free inodes, 0 directories, 416 unused inodes
+ Free blocks: 16385-19999
+ Free inodes: 833-1248
+Group 2: (Blocks 16385-19999)
+ Block bitmap at 83 (bg #0 + 82)
+ Inode bitmap at 86 (bg #0 + 85)
+ Inode table at 295-398 (bg #0 + 294)
+ 3615 free blocks, 416 free inodes, 0 directories
+ Free blocks: 16385-19999
+ Free inodes: 833-1248
+Resizing the filesystem on test.img to 20004 (1k) blocks.
+The filesystem on test.img is now 20004 (1k) blocks long.
+
+Group 2: (Blocks 16385-20003) [INODE_UNINIT]
+ Block bitmap at 83 (bg #0 + 82)
+ Inode bitmap at 86 (bg #0 + 85)
+ Inode table at 295-398 (bg #0 + 294)
+ 3619 free blocks, 416 free inodes, 0 directories, 416 unused inodes
+ Free blocks: 16385-20003
+ Free inodes: 833-1248
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+test.img: 11/1248 files (0.0% non-contiguous), 1517/20004 blocks
diff --git a/tests/r_fixup_lastbg/script b/tests/r_fixup_lastbg/script
new file mode 100755
index 0000000..6a5c5af
--- /dev/null
+++ b/tests/r_fixup_lastbg/script
@@ -0,0 +1,37 @@
+if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+test_description="fix up last bg when expanding within the last bg"
+
+EXP=$test_dir/expect
+OUT=$test_name.out
+LOG=$test_name.log
+E2FSCK=../e2fsck/e2fsck
+
+$MKE2FS -T ext4 -b 1024 -F -U 56d3ee50-8532-4f29-8181-d7c6ea4a94a6 $TMPFILE 20000 > $OUT 2>&1
+$DUMPE2FS $TMPFILE 2>&1 | grep -A10 '^Group 2:' >> $OUT
+$DEBUGFS -R "set_bg 2 itable_unused 0" -w $TMPFILE > /dev/null 2>&1
+$DEBUGFS -R "set_bg 2 flags 0" -w $TMPFILE > /dev/null 2>&1
+$DEBUGFS -R "set_bg 2 checksum 0xd318" -w $TMPFILE > /dev/null 2>&1
+$DUMPE2FS $TMPFILE 2>&1 | grep -A10 '^Group 2:' >> $OUT
+#dd if=/dev/zero of=$TMPFILE bs=1 count=1 seek=$((1024 * 20004)) conv=notrunc >> $OUT 2> /dev/null
+$RESIZE2FS_EXE -f -p $TMPFILE 20004 >> $OUT 2>&1
+$DUMPE2FS $TMPFILE 2>&1 | grep -A10 '^Group 2:' >> $OUT
+$E2FSCK -fy $TMPFILE >> $OUT 2>&1
+
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" < $OUT > $LOG
+rm -rf $OUT
+
+cmp -s $LOG $EXP
+RC=$?
+if [ $RC -eq 0 ]; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff -u $EXP $LOG > $test_name.failed
+fi
+
+unset EXP LOG OUT E2FSCK
+
+else #if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
diff --git a/tests/r_fixup_lastbg_big/expect b/tests/r_fixup_lastbg_big/expect
new file mode 100644
index 0000000..8f302a3
--- /dev/null
+++ b/tests/r_fixup_lastbg_big/expect
@@ -0,0 +1,43 @@
+Creating filesystem with 20000 1k blocks and 1248 inodes
+Superblock backups stored on blocks:
+ 8193
+
+Allocating group tables: \b\b\bdone
+Writing inode tables: \b\b\bdone
+Creating journal (1024 blocks): done
+Writing superblocks and filesystem accounting information: \b\b\bdone
+
+Group 2: (Blocks 16385-19999) [INODE_UNINIT, ITABLE_ZEROED]
+ Block bitmap at 83 (bg #0 + 82)
+ Inode bitmap at 86 (bg #0 + 85)
+ Inode table at 295-398 (bg #0 + 294)
+ 3615 free blocks, 416 free inodes, 0 directories, 416 unused inodes
+ Free blocks: 16385-19999
+ Free inodes: 833-1248
+Group 2: (Blocks 16385-19999)
+ Block bitmap at 83 (bg #0 + 82)
+ Inode bitmap at 86 (bg #0 + 85)
+ Inode table at 295-398 (bg #0 + 294)
+ 3615 free blocks, 416 free inodes, 0 directories
+ Free blocks: 16385-19999
+ Free inodes: 833-1248
+Resizing the filesystem on test.img to 40000 (1k) blocks.
+The filesystem on test.img is now 40000 (1k) blocks long.
+
+Group 2: (Blocks 16385-24576) [INODE_UNINIT, BLOCK_UNINIT]
+ Block bitmap at 83 (bg #0 + 82)
+ Inode bitmap at 86 (bg #0 + 85)
+ Inode table at 295-398 (bg #0 + 294)
+ 8192 free blocks, 416 free inodes, 0 directories, 416 unused inodes
+ Free blocks: 16385-24576
+ Free inodes: 833-1248
+Group 3: (Blocks 24577-32768) [INODE_UNINIT]
+ Backup superblock at 24577, Group descriptors at 24578-24578
+ Reserved GDT blocks at 24579-24656
+ Block bitmap at 413 (bg #0 + 412)
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+test.img: 11/2080 files (0.0% non-contiguous), 1809/40000 blocks
diff --git a/tests/r_fixup_lastbg_big/script b/tests/r_fixup_lastbg_big/script
new file mode 100755
index 0000000..4c4a351
--- /dev/null
+++ b/tests/r_fixup_lastbg_big/script
@@ -0,0 +1,37 @@
+if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+test_description="fix up last bg when expanding beyond the last bg"
+
+EXP=$test_dir/expect
+OUT=$test_name.out
+LOG=$test_name.log
+E2FSCK=../e2fsck/e2fsck
+
+$MKE2FS -T ext4 -b 1024 -F -U 56d3ee50-8532-4f29-8181-d7c6ea4a94a6 $TMPFILE 20000 > $OUT 2>&1
+$DUMPE2FS $TMPFILE 2>&1 | grep -A10 '^Group 2:' >> $OUT
+$DEBUGFS -R "set_bg 2 itable_unused 0" -w $TMPFILE > /dev/null 2>&1
+$DEBUGFS -R "set_bg 2 flags 0" -w $TMPFILE > /dev/null 2>&1
+$DEBUGFS -R "set_bg 2 checksum 0xd318" -w $TMPFILE > /dev/null 2>&1
+$DUMPE2FS $TMPFILE 2>&1 | grep -A10 '^Group 2:' >> $OUT
+dd if=/dev/zero of=$TMPFILE bs=1 count=1 seek=$((1024 * 40000)) conv=notrunc >> $OUT 2> /dev/null
+$RESIZE2FS_EXE -f -p $TMPFILE >> $OUT 2>&1
+$DUMPE2FS $TMPFILE 2>&1 | grep -A10 '^Group 2:' >> $OUT
+$E2FSCK -fy $TMPFILE >> $OUT 2>&1
+
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" < $OUT > $LOG
+rm -rf $OUT
+
+cmp -s $LOG $EXP
+RC=$?
+if [ $RC -eq 0 ]; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff -u $EXP $LOG > $test_name.failed
+fi
+
+unset EXP LOG OUT E2FSCK
+
+else #if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 29/47] resize2fs: don't mark unallocated bg metadata blocks when fixing bg flags
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (27 preceding siblings ...)
2014-11-07 21:53 ` [PATCH 28/47] resize2fs: set bg flags and unused inode count when resizing Darrick J. Wong
@ 2014-11-07 21:53 ` Darrick J. Wong
2014-12-15 2:17 ` Theodore Ts'o
2014-11-07 21:54 ` [PATCH 30/47] resize2fs: don't play stupid games with the block count Darrick J. Wong
` (19 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:53 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
When fixing up block group bitmaps on the new fs, don't try to mark
unallocated group metadata blocks; these will be allocated (and
marked) later.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
resize/resize2fs.c | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/resize/resize2fs.c b/resize/resize2fs.c
index 94c0643..a760593 100644
--- a/resize/resize2fs.c
+++ b/resize/resize2fs.c
@@ -242,7 +242,6 @@ static void fix_uninit_block_bitmaps(ext2_filsys fs)
{
blk64_t blk, lblk;
dgrp_t g;
- int i;
if (!ext2fs_has_group_desc_csum(fs))
return;
@@ -257,14 +256,16 @@ static void fix_uninit_block_bitmaps(ext2_filsys fs)
lblk - blk + 1);
ext2fs_reserve_super_and_bgd(fs, g, fs->block_map);
- ext2fs_mark_block_bitmap2(fs->block_map,
- ext2fs_block_bitmap_loc(fs, g));
- ext2fs_mark_block_bitmap2(fs->block_map,
- ext2fs_inode_bitmap_loc(fs, g));
- for (i = 0, blk = ext2fs_inode_table_loc(fs, g);
- i < (unsigned int) fs->inode_blocks_per_group;
- i++, blk++)
+ blk = ext2fs_block_bitmap_loc(fs, g);
+ if (blk)
+ ext2fs_mark_block_bitmap2(fs->block_map, blk);
+ blk = ext2fs_inode_bitmap_loc(fs, g);
+ if (blk)
ext2fs_mark_block_bitmap2(fs->block_map, blk);
+ blk = ext2fs_inode_table_loc(fs, g);
+ if (blk)
+ ext2fs_mark_block_bitmap_range2(fs->block_map, blk,
+ fs->inode_blocks_per_group);
}
}
^ permalink raw reply related [flat|nested] 104+ messages in thread* Re: [PATCH 29/47] resize2fs: don't mark unallocated bg metadata blocks when fixing bg flags
2014-11-07 21:53 ` [PATCH 29/47] resize2fs: don't mark unallocated bg metadata blocks when fixing bg flags Darrick J. Wong
@ 2014-12-15 2:17 ` Theodore Ts'o
2014-12-15 7:09 ` Darrick J. Wong
0 siblings, 1 reply; 104+ messages in thread
From: Theodore Ts'o @ 2014-12-15 2:17 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Fri, Nov 07, 2014 at 01:53:54PM -0800, Darrick J. Wong wrote:
> When fixing up block group bitmaps on the new fs, don't try to mark
> unallocated group metadata blocks; these will be allocated (and
> marked) later.
The commit description doesn't match the diff. It looks like what you
are doing is adding a check in case ext2fs_{block,inode}_bitmap_loc is
zero (which is not bad, but when can that happen; and if it's
something that _can_ happen, should we be triggering an assert
failure), and using ext2fs_mark_block_bitmap_range2() instead of a for
loop.
Is there a back story to this patch?
Thanks,
- Ted
> diff --git a/resize/resize2fs.c b/resize/resize2fs.c
> index 94c0643..a760593 100644
> --- a/resize/resize2fs.c
> +++ b/resize/resize2fs.c
> @@ -242,7 +242,6 @@ static void fix_uninit_block_bitmaps(ext2_filsys fs)
> {
> blk64_t blk, lblk;
> dgrp_t g;
> - int i;
>
> if (!ext2fs_has_group_desc_csum(fs))
> return;
> @@ -257,14 +256,16 @@ static void fix_uninit_block_bitmaps(ext2_filsys fs)
> lblk - blk + 1);
>
> ext2fs_reserve_super_and_bgd(fs, g, fs->block_map);
> - ext2fs_mark_block_bitmap2(fs->block_map,
> - ext2fs_block_bitmap_loc(fs, g));
> - ext2fs_mark_block_bitmap2(fs->block_map,
> - ext2fs_inode_bitmap_loc(fs, g));
> - for (i = 0, blk = ext2fs_inode_table_loc(fs, g);
> - i < (unsigned int) fs->inode_blocks_per_group;
> - i++, blk++)
> + blk = ext2fs_block_bitmap_loc(fs, g);
> + if (blk)
> + ext2fs_mark_block_bitmap2(fs->block_map, blk);
> + blk = ext2fs_inode_bitmap_loc(fs, g);
> + if (blk)
> ext2fs_mark_block_bitmap2(fs->block_map, blk);
> + blk = ext2fs_inode_table_loc(fs, g);
> + if (blk)
> + ext2fs_mark_block_bitmap_range2(fs->block_map, blk,
> + fs->inode_blocks_per_group);
> }
> }
>
>
^ permalink raw reply [flat|nested] 104+ messages in thread* Re: [PATCH 29/47] resize2fs: don't mark unallocated bg metadata blocks when fixing bg flags
2014-12-15 2:17 ` Theodore Ts'o
@ 2014-12-15 7:09 ` Darrick J. Wong
2014-12-19 19:15 ` Darrick J. Wong
0 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-12-15 7:09 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: linux-ext4
On Sun, Dec 14, 2014 at 09:17:48PM -0500, Theodore Ts'o wrote:
> On Fri, Nov 07, 2014 at 01:53:54PM -0800, Darrick J. Wong wrote:
> > When fixing up block group bitmaps on the new fs, don't try to mark
> > unallocated group metadata blocks; these will be allocated (and
> > marked) later.
>
> The commit description doesn't match the diff. It looks like what you
> are doing is adding a check in case ext2fs_{block,inode}_bitmap_loc is
> zero (which is not bad, but when can that happen; and if it's
> something that _can_ happen, should we be triggering an assert
> failure), and using ext2fs_mark_block_bitmap_range2() instead of a for
> loop.
>
> Is there a back story to this patch?
Looking through my notes, I think the reason I wrote this patch was that if
we're running resize2fs to enlarge inodes on a nearly full filesystem, we can
end up setting those bg fields to zero to force a later reallocation because
the only free space big enough to hold the larger inode table is somewhere else
in the FS. In the meantime, trying to mark block zero results in a lot of
unnecessary stderr spew about the invalid block number. This issue doesn't
affect resize2fs functionality, it just looks scary.
Yeah, probably we should use mark...range().
--D
>
> Thanks,
>
> - Ted
>
> > diff --git a/resize/resize2fs.c b/resize/resize2fs.c
> > index 94c0643..a760593 100644
> > --- a/resize/resize2fs.c
> > +++ b/resize/resize2fs.c
> > @@ -242,7 +242,6 @@ static void fix_uninit_block_bitmaps(ext2_filsys fs)
> > {
> > blk64_t blk, lblk;
> > dgrp_t g;
> > - int i;
> >
> > if (!ext2fs_has_group_desc_csum(fs))
> > return;
> > @@ -257,14 +256,16 @@ static void fix_uninit_block_bitmaps(ext2_filsys fs)
> > lblk - blk + 1);
> >
> > ext2fs_reserve_super_and_bgd(fs, g, fs->block_map);
> > - ext2fs_mark_block_bitmap2(fs->block_map,
> > - ext2fs_block_bitmap_loc(fs, g));
> > - ext2fs_mark_block_bitmap2(fs->block_map,
> > - ext2fs_inode_bitmap_loc(fs, g));
> > - for (i = 0, blk = ext2fs_inode_table_loc(fs, g);
> > - i < (unsigned int) fs->inode_blocks_per_group;
> > - i++, blk++)
> > + blk = ext2fs_block_bitmap_loc(fs, g);
> > + if (blk)
> > + ext2fs_mark_block_bitmap2(fs->block_map, blk);
> > + blk = ext2fs_inode_bitmap_loc(fs, g);
> > + if (blk)
> > ext2fs_mark_block_bitmap2(fs->block_map, blk);
> > + blk = ext2fs_inode_table_loc(fs, g);
> > + if (blk)
> > + ext2fs_mark_block_bitmap_range2(fs->block_map, blk,
> > + fs->inode_blocks_per_group);
> > }
> > }
> >
> >
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 104+ messages in thread* Re: [PATCH 29/47] resize2fs: don't mark unallocated bg metadata blocks when fixing bg flags
2014-12-15 7:09 ` Darrick J. Wong
@ 2014-12-19 19:15 ` Darrick J. Wong
0 siblings, 0 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-12-19 19:15 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: linux-ext4
On Sun, Dec 14, 2014 at 11:09:00PM -0800, Darrick J. Wong wrote:
> On Sun, Dec 14, 2014 at 09:17:48PM -0500, Theodore Ts'o wrote:
> > On Fri, Nov 07, 2014 at 01:53:54PM -0800, Darrick J. Wong wrote:
> > > When fixing up block group bitmaps on the new fs, don't try to mark
> > > unallocated group metadata blocks; these will be allocated (and
> > > marked) later.
> >
> > The commit description doesn't match the diff. It looks like what you
> > are doing is adding a check in case ext2fs_{block,inode}_bitmap_loc is
> > zero (which is not bad, but when can that happen; and if it's
> > something that _can_ happen, should we be triggering an assert
> > failure), and using ext2fs_mark_block_bitmap_range2() instead of a for
> > loop.
Aha, I have recaptured when this happens:
0) I have a debug patch that complains if anyone other than mke2fs tries
to allocate, deallocate, mark, or unmark block 0.
1) If we tell resize2fs to convert a FS to 64bit mode, it's possible that
move_bg_metadata() has to move the bitmaps or the inode table out of the way
because the group descriptors overflow the end of the regular and the reserved
gdt blocks. This can happen if you format a small FS, fill it with files,
expand it to use most of the reserved GDT blocks, then try to upgrade it to
64bit.
2) To force a reallocation of the bitmaps/inode table, move_bg_metadata() sets
the fields to zero in new_fs.
3) fix_uninit_block_bitmaps() is called on new_fs, and fails when we try to
mark block 0 in the block bitmap. Due to (0), a complaint happens.
4) Later, blocks_to_move() will also notice group descriptors pointing to block
zero and fix this via ext2fs_allocate_group_table(). Then it moves the blocks,
and we're done.
I conclude that this patch isn't really needed. It's a little funny to be
marking block zero (which should already be marked) but doing so doesn't
negatively affect the FS in this particular case.
--D
> >
> > Is there a back story to this patch?
>
> Looking through my notes, I think the reason I wrote this patch was that if
> we're running resize2fs to enlarge inodes on a nearly full filesystem, we can
> end up setting those bg fields to zero to force a later reallocation because
> the only free space big enough to hold the larger inode table is somewhere else
> in the FS. In the meantime, trying to mark block zero results in a lot of
> unnecessary stderr spew about the invalid block number. This issue doesn't
> affect resize2fs functionality, it just looks scary.
>
> Yeah, probably we should use mark...range().
>
> --D
>
> >
> > Thanks,
> >
> > - Ted
> >
> > > diff --git a/resize/resize2fs.c b/resize/resize2fs.c
> > > index 94c0643..a760593 100644
> > > --- a/resize/resize2fs.c
> > > +++ b/resize/resize2fs.c
> > > @@ -242,7 +242,6 @@ static void fix_uninit_block_bitmaps(ext2_filsys fs)
> > > {
> > > blk64_t blk, lblk;
> > > dgrp_t g;
> > > - int i;
> > >
> > > if (!ext2fs_has_group_desc_csum(fs))
> > > return;
> > > @@ -257,14 +256,16 @@ static void fix_uninit_block_bitmaps(ext2_filsys fs)
> > > lblk - blk + 1);
> > >
> > > ext2fs_reserve_super_and_bgd(fs, g, fs->block_map);
> > > - ext2fs_mark_block_bitmap2(fs->block_map,
> > > - ext2fs_block_bitmap_loc(fs, g));
> > > - ext2fs_mark_block_bitmap2(fs->block_map,
> > > - ext2fs_inode_bitmap_loc(fs, g));
> > > - for (i = 0, blk = ext2fs_inode_table_loc(fs, g);
> > > - i < (unsigned int) fs->inode_blocks_per_group;
> > > - i++, blk++)
> > > + blk = ext2fs_block_bitmap_loc(fs, g);
> > > + if (blk)
> > > + ext2fs_mark_block_bitmap2(fs->block_map, blk);
> > > + blk = ext2fs_inode_bitmap_loc(fs, g);
> > > + if (blk)
> > > ext2fs_mark_block_bitmap2(fs->block_map, blk);
> > > + blk = ext2fs_inode_table_loc(fs, g);
> > > + if (blk)
> > > + ext2fs_mark_block_bitmap_range2(fs->block_map, blk,
> > > + fs->inode_blocks_per_group);
> > > }
> > > }
> > >
> > >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 104+ messages in thread
* [PATCH 30/47] resize2fs: don't play stupid games with the block count
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (28 preceding siblings ...)
2014-11-07 21:53 ` [PATCH 29/47] resize2fs: don't mark unallocated bg metadata blocks when fixing bg flags Darrick J. Wong
@ 2014-11-07 21:54 ` Darrick J. Wong
2014-12-15 3:13 ` Theodore Ts'o
2014-11-07 21:54 ` [PATCH 31/47] libext2fs/e2fsck: provide routines to read-ahead metadata Darrick J. Wong
` (18 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:54 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
While it may be true that playing games with old_fs' block count
during a grow operation shuts up a bunch of warnings, resize2fs
doesn't actually expand the group descriptor array to match the size
we're artificially stuffing into old_fs, which means that if we
actually need to allocate a block out of the larger fs (i.e. we're in
desperation mode), ext2fs_block_alloc_stats2() scribbles on the heap,
leading to crashes if you're lucky and FS corruption if not.
So, rip that piece out and turn off com_err warnings properly and add
a test case to deal with growing a nearly full filesystem.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
resize/resize2fs.c | 19 +-
tests/r_expand_full/expect | 375 ++++++++++++++++++++++++++++++++++++++++++++
tests/r_expand_full/name | 1
tests/r_expand_full/script | 79 +++++++++
4 files changed, 462 insertions(+), 12 deletions(-)
create mode 100644 tests/r_expand_full/expect
create mode 100644 tests/r_expand_full/name
create mode 100644 tests/r_expand_full/script
diff --git a/resize/resize2fs.c b/resize/resize2fs.c
index a760593..0d1f7fa 100644
--- a/resize/resize2fs.c
+++ b/resize/resize2fs.c
@@ -1629,6 +1629,11 @@ out:
return errcode;
}
+static void quiet_com_err_proc(const char *whoami, errcode_t code,
+ const char *fmt, va_list args)
+{
+}
+
static errcode_t inode_scan_and_fix(ext2_resize_t rfs)
{
struct process_block_struct pb;
@@ -1638,7 +1643,6 @@ static errcode_t inode_scan_and_fix(ext2_resize_t rfs)
errcode_t retval;
char *block_buf = 0;
ext2_ino_t start_to_move;
- blk64_t orig_size;
int inode_size;
if ((rfs->old_fs->group_desc_count <=
@@ -1646,16 +1650,7 @@ static errcode_t inode_scan_and_fix(ext2_resize_t rfs)
!rfs->bmap)
return 0;
- /*
- * Save the original size of the old filesystem, and
- * temporarily set the size to be the new size if the new size
- * is larger. We need to do this to avoid catching an error
- * by the block iterator routines
- */
- orig_size = ext2fs_blocks_count(rfs->old_fs->super);
- if (orig_size < ext2fs_blocks_count(rfs->new_fs->super))
- ext2fs_blocks_count_set(rfs->old_fs->super,
- ext2fs_blocks_count(rfs->new_fs->super));
+ set_com_err_hook(quiet_com_err_proc);
retval = ext2fs_open_inode_scan(rfs->old_fs, 0, &scan);
if (retval) goto errout;
@@ -1791,7 +1786,7 @@ remap_blocks:
io_channel_flush(rfs->old_fs->io);
errout:
- ext2fs_blocks_count_set(rfs->old_fs->super, orig_size);
+ reset_com_err_hook();
if (rfs->bmap) {
ext2fs_free_extent_table(rfs->bmap);
rfs->bmap = 0;
diff --git a/tests/r_expand_full/expect b/tests/r_expand_full/expect
new file mode 100644
index 0000000..c80baa2
--- /dev/null
+++ b/tests/r_expand_full/expect
@@ -0,0 +1,375 @@
+resize2fs test
+Creating filesystem with 786432 1k blocks and 98304 inodes
+Superblock backups stored on blocks:
+ 8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409, 663553
+
+Allocating group tables: \b\b\b\b\bdone
+Writing inode tables: \b\b\b\b\bdone
+Creating journal (16384 blocks): done
+Creating 727 huge file(s) with 1024 blocks each: done
+Writing superblocks and filesystem accounting information: \b\b\b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
+resize2fs test.img
+Resizing the filesystem on test.img to 3145728 (1k) blocks.
+The filesystem on test.img is now 3145728 (1k) blocks long.
+
+Exit status is 0
+Change in FS metadata:
+@@ -1,15 +1,15 @@
+
+ group:block:super:gdt:bbitmap:ibitmap:itable
+-0:1:1:2-4:5:6:7
+-1:8193:8193:8194-8196:8197:8198:8199
++0:1:1:2-13:263:264:265
++1:8193:8193:8194-8205:8455:8456:8457
+ 2:16385:-1:-1:16385:16386:16387
+-3:24577:24577:24578-24580:24581:24582:24583
++3:24577:24577:24578-24589:24839:24840:24841
+ 4:32769:-1:-1:32769:32770:32771
+-5:40961:40961:40962-40964:40965:40966:40967
++5:40961:40961:40962-40973:41223:41224:41225
+ 6:49153:-1:-1:49153:49154:49155
+-7:57345:57345:57346-57348:57349:57350:57351
++7:57345:57345:57346-57357:57607:57608:57609
+ 8:65537:-1:-1:65537:65538:65539
+-9:73729:73729:73730-73732:73733:73734:73735
++9:73729:73729:73730-73741:73991:73992:73993
+ 10:81921:-1:-1:81921:81922:81923
+ 11:90113:-1:-1:90113:90114:90115
+ 12:98305:-1:-1:98305:98306:98307
+@@ -25,9 +25,9 @@
+ 22:180225:-1:-1:180225:180226:180227
+ 23:188417:-1:-1:188417:188418:188419
+ 24:196609:-1:-1:196609:196610:196611
+-25:204801:204801:204802-204804:204805:204806:204807
++25:204801:204801:204802-204813:205063:205064:205065
+ 26:212993:-1:-1:212993:212994:212995
+-27:221185:221185:221186-221188:221189:221190:221191
++27:221185:221185:221186-221197:221447:221448:221449
+ 28:229377:-1:-1:229377:229378:229379
+ 29:237569:-1:-1:237569:237570:237571
+ 30:245761:-1:-1:245761:245762:245763
+@@ -49,7 +49,7 @@
+ 46:376833:-1:-1:376833:376834:376835
+ 47:385025:-1:-1:385025:385026:385027
+ 48:393217:-1:-1:393217:393218:393219
+-49:401409:401409:401410-401412:401413:401414:401415
++49:401409:401409:401410-401421:401671:401672:401673
+ 50:409601:-1:-1:409601:409602:409603
+ 51:417793:-1:-1:417793:417794:417795
+ 52:425985:-1:-1:425985:425986:425987
+@@ -81,7 +81,7 @@
+ 78:638977:-1:-1:638977:638978:638979
+ 79:647169:-1:-1:647169:647170:647171
+ 80:655361:-1:-1:655361:655362:655363
+-81:663553:663553:663554-663556:663557:663558:663559
++81:663553:663553:663554-663565:663815:663816:663817
+ 82:671745:-1:-1:671745:671746:671747
+ 83:679937:-1:-1:679937:679938:679939
+ 84:688129:-1:-1:688129:688130:688131
+@@ -96,3 +96,291 @@
+ 93:761857:-1:-1:761857:761858:761859
+ 94:770049:-1:-1:770049:770050:770051
+ 95:778241:-1:-1:778241:778242:778243
++96:786433:-1:-1:786433:786434:786435
++97:794625:-1:-1:794625:794626:794627
++98:802817:-1:-1:802817:802818:802819
++99:811009:-1:-1:811009:811010:811011
++100:819201:-1:-1:819201:819202:819203
++101:827393:-1:-1:827393:827394:827395
++102:835585:-1:-1:835585:835586:835587
++103:843777:-1:-1:843777:843778:843779
++104:851969:-1:-1:851969:851970:851971
++105:860161:-1:-1:860161:860162:860163
++106:868353:-1:-1:868353:868354:868355
++107:876545:-1:-1:876545:876546:876547
++108:884737:-1:-1:884737:884738:884739
++109:892929:-1:-1:892929:892930:892931
++110:901121:-1:-1:901121:901122:901123
++111:909313:-1:-1:909313:909314:909315
++112:917505:-1:-1:917505:917506:917507
++113:925697:-1:-1:925697:925698:925699
++114:933889:-1:-1:933889:933890:933891
++115:942081:-1:-1:942081:942082:942083
++116:950273:-1:-1:950273:950274:950275
++117:958465:-1:-1:958465:958466:958467
++118:966657:-1:-1:966657:966658:966659
++119:974849:-1:-1:974849:974850:974851
++120:983041:-1:-1:983041:983042:983043
++121:991233:-1:-1:991233:991234:991235
++122:999425:-1:-1:999425:999426:999427
++123:1007617:-1:-1:1007617:1007618:1007619
++124:1015809:-1:-1:1015809:1015810:1015811
++125:1024001:1024001:1024002-1024013:1024014:1024015:1024016
++126:1032193:-1:-1:1032193:1032194:1032195
++127:1040385:-1:-1:1040385:1040386:1040387
++128:1048577:-1:-1:1048577:1048578:1048579
++129:1056769:-1:-1:1056769:1056770:1056771
++130:1064961:-1:-1:1064961:1064962:1064963
++131:1073153:-1:-1:1073153:1073154:1073155
++132:1081345:-1:-1:1081345:1081346:1081347
++133:1089537:-1:-1:1089537:1089538:1089539
++134:1097729:-1:-1:1097729:1097730:1097731
++135:1105921:-1:-1:1105921:1105922:1105923
++136:1114113:-1:-1:1114113:1114114:1114115
++137:1122305:-1:-1:1122305:1122306:1122307
++138:1130497:-1:-1:1130497:1130498:1130499
++139:1138689:-1:-1:1138689:1138690:1138691
++140:1146881:-1:-1:1146881:1146882:1146883
++141:1155073:-1:-1:1155073:1155074:1155075
++142:1163265:-1:-1:1163265:1163266:1163267
++143:1171457:-1:-1:1171457:1171458:1171459
++144:1179649:-1:-1:1179649:1179650:1179651
++145:1187841:-1:-1:1187841:1187842:1187843
++146:1196033:-1:-1:1196033:1196034:1196035
++147:1204225:-1:-1:1204225:1204226:1204227
++148:1212417:-1:-1:1212417:1212418:1212419
++149:1220609:-1:-1:1220609:1220610:1220611
++150:1228801:-1:-1:1228801:1228802:1228803
++151:1236993:-1:-1:1236993:1236994:1236995
++152:1245185:-1:-1:1245185:1245186:1245187
++153:1253377:-1:-1:1253377:1253378:1253379
++154:1261569:-1:-1:1261569:1261570:1261571
++155:1269761:-1:-1:1269761:1269762:1269763
++156:1277953:-1:-1:1277953:1277954:1277955
++157:1286145:-1:-1:1286145:1286146:1286147
++158:1294337:-1:-1:1294337:1294338:1294339
++159:1302529:-1:-1:1302529:1302530:1302531
++160:1310721:-1:-1:1310721:1310722:1310723
++161:1318913:-1:-1:1318913:1318914:1318915
++162:1327105:-1:-1:1327105:1327106:1327107
++163:1335297:-1:-1:1335297:1335298:1335299
++164:1343489:-1:-1:1343489:1343490:1343491
++165:1351681:-1:-1:1351681:1351682:1351683
++166:1359873:-1:-1:1359873:1359874:1359875
++167:1368065:-1:-1:1368065:1368066:1368067
++168:1376257:-1:-1:1376257:1376258:1376259
++169:1384449:-1:-1:1384449:1384450:1384451
++170:1392641:-1:-1:1392641:1392642:1392643
++171:1400833:-1:-1:1400833:1400834:1400835
++172:1409025:-1:-1:1409025:1409026:1409027
++173:1417217:-1:-1:1417217:1417218:1417219
++174:1425409:-1:-1:1425409:1425410:1425411
++175:1433601:-1:-1:1433601:1433602:1433603
++176:1441793:-1:-1:1441793:1441794:1441795
++177:1449985:-1:-1:1449985:1449986:1449987
++178:1458177:-1:-1:1458177:1458178:1458179
++179:1466369:-1:-1:1466369:1466370:1466371
++180:1474561:-1:-1:1474561:1474562:1474563
++181:1482753:-1:-1:1482753:1482754:1482755
++182:1490945:-1:-1:1490945:1490946:1490947
++183:1499137:-1:-1:1499137:1499138:1499139
++184:1507329:-1:-1:1507329:1507330:1507331
++185:1515521:-1:-1:1515521:1515522:1515523
++186:1523713:-1:-1:1523713:1523714:1523715
++187:1531905:-1:-1:1531905:1531906:1531907
++188:1540097:-1:-1:1540097:1540098:1540099
++189:1548289:-1:-1:1548289:1548290:1548291
++190:1556481:-1:-1:1556481:1556482:1556483
++191:1564673:-1:-1:1564673:1564674:1564675
++192:1572865:-1:-1:1572865:1572866:1572867
++193:1581057:-1:-1:1581057:1581058:1581059
++194:1589249:-1:-1:1589249:1589250:1589251
++195:1597441:-1:-1:1597441:1597442:1597443
++196:1605633:-1:-1:1605633:1605634:1605635
++197:1613825:-1:-1:1613825:1613826:1613827
++198:1622017:-1:-1:1622017:1622018:1622019
++199:1630209:-1:-1:1630209:1630210:1630211
++200:1638401:-1:-1:1638401:1638402:1638403
++201:1646593:-1:-1:1646593:1646594:1646595
++202:1654785:-1:-1:1654785:1654786:1654787
++203:1662977:-1:-1:1662977:1662978:1662979
++204:1671169:-1:-1:1671169:1671170:1671171
++205:1679361:-1:-1:1679361:1679362:1679363
++206:1687553:-1:-1:1687553:1687554:1687555
++207:1695745:-1:-1:1695745:1695746:1695747
++208:1703937:-1:-1:1703937:1703938:1703939
++209:1712129:-1:-1:1712129:1712130:1712131
++210:1720321:-1:-1:1720321:1720322:1720323
++211:1728513:-1:-1:1728513:1728514:1728515
++212:1736705:-1:-1:1736705:1736706:1736707
++213:1744897:-1:-1:1744897:1744898:1744899
++214:1753089:-1:-1:1753089:1753090:1753091
++215:1761281:-1:-1:1761281:1761282:1761283
++216:1769473:-1:-1:1769473:1769474:1769475
++217:1777665:-1:-1:1777665:1777666:1777667
++218:1785857:-1:-1:1785857:1785858:1785859
++219:1794049:-1:-1:1794049:1794050:1794051
++220:1802241:-1:-1:1802241:1802242:1802243
++221:1810433:-1:-1:1810433:1810434:1810435
++222:1818625:-1:-1:1818625:1818626:1818627
++223:1826817:-1:-1:1826817:1826818:1826819
++224:1835009:-1:-1:1835009:1835010:1835011
++225:1843201:-1:-1:1843201:1843202:1843203
++226:1851393:-1:-1:1851393:1851394:1851395
++227:1859585:-1:-1:1859585:1859586:1859587
++228:1867777:-1:-1:1867777:1867778:1867779
++229:1875969:-1:-1:1875969:1875970:1875971
++230:1884161:-1:-1:1884161:1884162:1884163
++231:1892353:-1:-1:1892353:1892354:1892355
++232:1900545:-1:-1:1900545:1900546:1900547
++233:1908737:-1:-1:1908737:1908738:1908739
++234:1916929:-1:-1:1916929:1916930:1916931
++235:1925121:-1:-1:1925121:1925122:1925123
++236:1933313:-1:-1:1933313:1933314:1933315
++237:1941505:-1:-1:1941505:1941506:1941507
++238:1949697:-1:-1:1949697:1949698:1949699
++239:1957889:-1:-1:1957889:1957890:1957891
++240:1966081:-1:-1:1966081:1966082:1966083
++241:1974273:-1:-1:1974273:1974274:1974275
++242:1982465:-1:-1:1982465:1982466:1982467
++243:1990657:1990657:1990658-1990669:1990670:1990671:1990672
++244:1998849:-1:-1:1998849:1998850:1998851
++245:2007041:-1:-1:2007041:2007042:2007043
++246:2015233:-1:-1:2015233:2015234:2015235
++247:2023425:-1:-1:2023425:2023426:2023427
++248:2031617:-1:-1:2031617:2031618:2031619
++249:2039809:-1:-1:2039809:2039810:2039811
++250:2048001:-1:-1:2048001:2048002:2048003
++251:2056193:-1:-1:2056193:2056194:2056195
++252:2064385:-1:-1:2064385:2064386:2064387
++253:2072577:-1:-1:2072577:2072578:2072579
++254:2080769:-1:-1:2080769:2080770:2080771
++255:2088961:-1:-1:2088961:2088962:2088963
++256:2097153:-1:-1:2097153:2097154:2097155
++257:2105345:-1:-1:2105345:2105346:2105347
++258:2113537:-1:-1:2113537:2113538:2113539
++259:2121729:-1:-1:2121729:2121730:2121731
++260:2129921:-1:-1:2129921:2129922:2129923
++261:2138113:-1:-1:2138113:2138114:2138115
++262:2146305:-1:-1:2146305:2146306:2146307
++263:2154497:-1:-1:2154497:2154498:2154499
++264:2162689:-1:-1:2162689:2162690:2162691
++265:2170881:-1:-1:2170881:2170882:2170883
++266:2179073:-1:-1:2179073:2179074:2179075
++267:2187265:-1:-1:2187265:2187266:2187267
++268:2195457:-1:-1:2195457:2195458:2195459
++269:2203649:-1:-1:2203649:2203650:2203651
++270:2211841:-1:-1:2211841:2211842:2211843
++271:2220033:-1:-1:2220033:2220034:2220035
++272:2228225:-1:-1:2228225:2228226:2228227
++273:2236417:-1:-1:2236417:2236418:2236419
++274:2244609:-1:-1:2244609:2244610:2244611
++275:2252801:-1:-1:2252801:2252802:2252803
++276:2260993:-1:-1:2260993:2260994:2260995
++277:2269185:-1:-1:2269185:2269186:2269187
++278:2277377:-1:-1:2277377:2277378:2277379
++279:2285569:-1:-1:2285569:2285570:2285571
++280:2293761:-1:-1:2293761:2293762:2293763
++281:2301953:-1:-1:2301953:2301954:2301955
++282:2310145:-1:-1:2310145:2310146:2310147
++283:2318337:-1:-1:2318337:2318338:2318339
++284:2326529:-1:-1:2326529:2326530:2326531
++285:2334721:-1:-1:2334721:2334722:2334723
++286:2342913:-1:-1:2342913:2342914:2342915
++287:2351105:-1:-1:2351105:2351106:2351107
++288:2359297:-1:-1:2359297:2359298:2359299
++289:2367489:-1:-1:2367489:2367490:2367491
++290:2375681:-1:-1:2375681:2375682:2375683
++291:2383873:-1:-1:2383873:2383874:2383875
++292:2392065:-1:-1:2392065:2392066:2392067
++293:2400257:-1:-1:2400257:2400258:2400259
++294:2408449:-1:-1:2408449:2408450:2408451
++295:2416641:-1:-1:2416641:2416642:2416643
++296:2424833:-1:-1:2424833:2424834:2424835
++297:2433025:-1:-1:2433025:2433026:2433027
++298:2441217:-1:-1:2441217:2441218:2441219
++299:2449409:-1:-1:2449409:2449410:2449411
++300:2457601:-1:-1:2457601:2457602:2457603
++301:2465793:-1:-1:2465793:2465794:2465795
++302:2473985:-1:-1:2473985:2473986:2473987
++303:2482177:-1:-1:2482177:2482178:2482179
++304:2490369:-1:-1:2490369:2490370:2490371
++305:2498561:-1:-1:2498561:2498562:2498563
++306:2506753:-1:-1:2506753:2506754:2506755
++307:2514945:-1:-1:2514945:2514946:2514947
++308:2523137:-1:-1:2523137:2523138:2523139
++309:2531329:-1:-1:2531329:2531330:2531331
++310:2539521:-1:-1:2539521:2539522:2539523
++311:2547713:-1:-1:2547713:2547714:2547715
++312:2555905:-1:-1:2555905:2555906:2555907
++313:2564097:-1:-1:2564097:2564098:2564099
++314:2572289:-1:-1:2572289:2572290:2572291
++315:2580481:-1:-1:2580481:2580482:2580483
++316:2588673:-1:-1:2588673:2588674:2588675
++317:2596865:-1:-1:2596865:2596866:2596867
++318:2605057:-1:-1:2605057:2605058:2605059
++319:2613249:-1:-1:2613249:2613250:2613251
++320:2621441:-1:-1:2621441:2621442:2621443
++321:2629633:-1:-1:2629633:2629634:2629635
++322:2637825:-1:-1:2637825:2637826:2637827
++323:2646017:-1:-1:2646017:2646018:2646019
++324:2654209:-1:-1:2654209:2654210:2654211
++325:2662401:-1:-1:2662401:2662402:2662403
++326:2670593:-1:-1:2670593:2670594:2670595
++327:2678785:-1:-1:2678785:2678786:2678787
++328:2686977:-1:-1:2686977:2686978:2686979
++329:2695169:-1:-1:2695169:2695170:2695171
++330:2703361:-1:-1:2703361:2703362:2703363
++331:2711553:-1:-1:2711553:2711554:2711555
++332:2719745:-1:-1:2719745:2719746:2719747
++333:2727937:-1:-1:2727937:2727938:2727939
++334:2736129:-1:-1:2736129:2736130:2736131
++335:2744321:-1:-1:2744321:2744322:2744323
++336:2752513:-1:-1:2752513:2752514:2752515
++337:2760705:-1:-1:2760705:2760706:2760707
++338:2768897:-1:-1:2768897:2768898:2768899
++339:2777089:-1:-1:2777089:2777090:2777091
++340:2785281:-1:-1:2785281:2785282:2785283
++341:2793473:-1:-1:2793473:2793474:2793475
++342:2801665:-1:-1:2801665:2801666:2801667
++343:2809857:2809857:2809858-2809869:2809870:2809871:2809872
++344:2818049:-1:-1:2818049:2818050:2818051
++345:2826241:-1:-1:2826241:2826242:2826243
++346:2834433:-1:-1:2834433:2834434:2834435
++347:2842625:-1:-1:2842625:2842626:2842627
++348:2850817:-1:-1:2850817:2850818:2850819
++349:2859009:-1:-1:2859009:2859010:2859011
++350:2867201:-1:-1:2867201:2867202:2867203
++351:2875393:-1:-1:2875393:2875394:2875395
++352:2883585:-1:-1:2883585:2883586:2883587
++353:2891777:-1:-1:2891777:2891778:2891779
++354:2899969:-1:-1:2899969:2899970:2899971
++355:2908161:-1:-1:2908161:2908162:2908163
++356:2916353:-1:-1:2916353:2916354:2916355
++357:2924545:-1:-1:2924545:2924546:2924547
++358:2932737:-1:-1:2932737:2932738:2932739
++359:2940929:-1:-1:2940929:2940930:2940931
++360:2949121:-1:-1:2949121:2949122:2949123
++361:2957313:-1:-1:2957313:2957314:2957315
++362:2965505:-1:-1:2965505:2965506:2965507
++363:2973697:-1:-1:2973697:2973698:2973699
++364:2981889:-1:-1:2981889:2981890:2981891
++365:2990081:-1:-1:2990081:2990082:2990083
++366:2998273:-1:-1:2998273:2998274:2998275
++367:3006465:-1:-1:3006465:3006466:3006467
++368:3014657:-1:-1:3014657:3014658:3014659
++369:3022849:-1:-1:3022849:3022850:3022851
++370:3031041:-1:-1:3031041:3031042:3031043
++371:3039233:-1:-1:3039233:3039234:3039235
++372:3047425:-1:-1:3047425:3047426:3047427
++373:3055617:-1:-1:3055617:3055618:3055619
++374:3063809:-1:-1:3063809:3063810:3063811
++375:3072001:-1:-1:3072001:3072002:3072003
++376:3080193:-1:-1:3080193:3080194:3080195
++377:3088385:-1:-1:3088385:3088386:3088387
++378:3096577:-1:-1:3096577:3096578:3096579
++379:3104769:-1:-1:3104769:3104770:3104771
++380:3112961:-1:-1:3112961:3112962:3112963
++381:3121153:-1:-1:3121153:3121154:3121155
++382:3129345:-1:-1:3129345:3129346:3129347
++383:3137537:-1:-1:3137537:3137538:3137539
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
diff --git a/tests/r_expand_full/name b/tests/r_expand_full/name
new file mode 100644
index 0000000..af40cba
--- /dev/null
+++ b/tests/r_expand_full/name
@@ -0,0 +1 @@
+expand a totally full filesystem
diff --git a/tests/r_expand_full/script b/tests/r_expand_full/script
new file mode 100644
index 0000000..a2c09db
--- /dev/null
+++ b/tests/r_expand_full/script
@@ -0,0 +1,79 @@
+if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+
+FSCK_OPT=-fn
+OUT=$test_name.log
+EXP=$test_dir/expect
+CONF=$TMPFILE.conf
+
+#gzip -d < $EXP.gz > $EXP
+
+cat > $CONF << ENDL
+[fs_types]
+ ext4h = {
+ features = has_journal,extent,huge_file,uninit_bg,dir_nlink,extra_isize,sparse_super,filetype,dir_index,ext_attr,^resize_inode,^meta_bg,^flex_bg
+ blocksize = 1024
+ inode_size = 256
+ make_hugefiles = true
+ hugefiles_dir = /
+ hugefiles_slack = 0
+ hugefiles_name = aaaaa
+ hugefiles_digits = 4
+ hugefiles_size = 1M
+ zero_hugefiles = false
+ }
+ENDL
+
+echo "resize2fs test" > $OUT
+
+MKE2FS_CONFIG=$CONF $MKE2FS -F -T ext4h $TMPFILE 786432 >> $OUT 2>&1
+rm -rf $CONF
+
+# dump and check
+$DUMPE2FS -g $TMPFILE 2>&1 >> $OUT.before 2> /dev/null
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# convert it
+echo "resize2fs test.img" >> $OUT
+dd if=/dev/zero of=$TMPFILE conv=notrunc bs=1 count=1 seek=3221225471 2> /dev/null
+$RESIZE2FS -f $TMPFILE 2>&1 >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# dump and check
+$DUMPE2FS -g $TMPFILE 2>&1 >> $OUT.after 2> /dev/null
+echo "Change in FS metadata:" >> $OUT
+diff -u $OUT.before $OUT.after | tail -n +3 >> $OUT
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+rm $TMPFILE
+
+#
+# Do the verification
+#
+
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" -e 's/test_filesys:.*//g' < $OUT > $OUT.new
+mv $OUT.new $OUT
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+fi
+
+rm $OUT.before $OUT.after
+
+unset IMAGE FSCK_OPT OUT EXP CONF
+
+else #if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
+
^ permalink raw reply related [flat|nested] 104+ messages in thread* Re: [PATCH 30/47] resize2fs: don't play stupid games with the block count
2014-11-07 21:54 ` [PATCH 30/47] resize2fs: don't play stupid games with the block count Darrick J. Wong
@ 2014-12-15 3:13 ` Theodore Ts'o
0 siblings, 0 replies; 104+ messages in thread
From: Theodore Ts'o @ 2014-12-15 3:13 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Fri, Nov 07, 2014 at 01:54:01PM -0800, Darrick J. Wong wrote:
> While it may be true that playing games with old_fs' block count
> during a grow operation shuts up a bunch of warnings, resize2fs
> doesn't actually expand the group descriptor array to match the size
> we're artificially stuffing into old_fs, which means that if we
> actually need to allocate a block out of the larger fs (i.e. we're in
> desperation mode), ext2fs_block_alloc_stats2() scribbles on the heap,
> leading to crashes if you're lucky and FS corruption if not.
>
> So, rip that piece out and turn off com_err warnings properly and add
> a test case to deal with growing a nearly full filesystem.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Applied, thanks.
- Ted
^ permalink raw reply [flat|nested] 104+ messages in thread
* [PATCH 31/47] libext2fs/e2fsck: provide routines to read-ahead metadata
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (29 preceding siblings ...)
2014-11-07 21:54 ` [PATCH 30/47] resize2fs: don't play stupid games with the block count Darrick J. Wong
@ 2014-11-07 21:54 ` Darrick J. Wong
2014-12-15 17:36 ` Theodore Ts'o
2014-11-07 21:54 ` [PATCH 32/47] e2fsck: read-ahead metadata during passes 1, 2, and 4 Darrick J. Wong
` (17 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:54 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
This patch adds to e2fsck the ability to pre-fetch metadata into the
page cache in the hopes of speeding up fsck runs. There are two new
functions -- the first allows a caller to readahead a list of blocks,
and the second is a helper function that uses that first mechanism to
load group data (bitmaps, inode tables).
These new e2fsck routines require the addition of a dblist API to
allow us to iterate a subset of a dblist. This will enable
incremental directory block readahead in e2fsck pass 2.
There's also a function to estimate the readahead given a FS.
v2: Add an API to create a dblist with a given number of list elements
pre-allocated. This enables us to save ~2ms per call to
e2fsck_readahead() (assuming a 2MB RA buffer) by not having to
repeatedly call ext2_resize_mem as we add blocks to the list.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
configure | 2
configure.in | 1
e2fsck/Makefile.in | 9 +-
e2fsck/e2fsck.h | 18 ++++
e2fsck/readahead.c | 252 +++++++++++++++++++++++++++++++++++++++++++++++++++
e2fsck/util.c | 51 ++++++++++
lib/config.h.in | 3 +
lib/ext2fs/dblist.c | 21 ++++
lib/ext2fs/ext2fs.h | 10 ++
9 files changed, 359 insertions(+), 8 deletions(-)
create mode 100644 e2fsck/readahead.c
diff --git a/configure b/configure
index f59d232..fdc93c0 100755
--- a/configure
+++ b/configure
@@ -12414,7 +12414,7 @@ fi
done
fi
-for ac_header in dirent.h errno.h execinfo.h getopt.h malloc.h mntent.h paths.h semaphore.h setjmp.h signal.h stdarg.h stdint.h stdlib.h termios.h termio.h unistd.h utime.h attr/xattr.h linux/falloc.h linux/fd.h linux/major.h linux/loop.h net/if_dl.h netinet/in.h sys/disklabel.h sys/disk.h sys/file.h sys/ioctl.h sys/mkdev.h sys/mman.h sys/mount.h sys/prctl.h sys/resource.h sys/select.h sys/socket.h sys/sockio.h sys/stat.h sys/syscall.h sys/sysmacros.h sys/time.h sys/types.h sys/un.h sys/wait.h
+for ac_header in dirent.h errno.h execinfo.h getopt.h malloc.h mntent.h paths.h semaphore.h setjmp.h signal.h stdarg.h stdint.h stdlib.h termios.h termio.h unistd.h utime.h attr/xattr.h linux/falloc.h linux/fd.h linux/major.h linux/loop.h net/if_dl.h netinet/in.h sys/disklabel.h sys/disk.h sys/file.h sys/ioctl.h sys/mkdev.h sys/mman.h sys/mount.h sys/prctl.h sys/resource.h sys/select.h sys/socket.h sys/sockio.h sys/stat.h sys/syscall.h sys/sysctl.h sys/sysmacros.h sys/time.h sys/types.h sys/un.h sys/wait.h
do :
as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh`
ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default"
diff --git a/configure.in b/configure.in
index 9069234..73cfeb4 100644
--- a/configure.in
+++ b/configure.in
@@ -949,6 +949,7 @@ AC_CHECK_HEADERS(m4_flatten([
sys/sockio.h
sys/stat.h
sys/syscall.h
+ sys/sysctl.h
sys/sysmacros.h
sys/time.h
sys/types.h
diff --git a/e2fsck/Makefile.in b/e2fsck/Makefile.in
index d0e64eb..e40e51b 100644
--- a/e2fsck/Makefile.in
+++ b/e2fsck/Makefile.in
@@ -62,7 +62,7 @@ OBJS= dict.o unix.o e2fsck.o super.o pass1.o pass1b.o pass2.o \
pass3.o pass4.o pass5.o journal.o badblocks.o util.o dirinfo.o \
dx_dirinfo.o ehandler.o problem.o message.o quota.o recovery.o \
region.o revoke.o ea_refcount.o rehash.o profile.o prof_err.o \
- logfile.o sigcatcher.o $(MTRACE_OBJ) plausible.o
+ logfile.o sigcatcher.o $(MTRACE_OBJ) plausible.o readahead.o
PROFILED_OBJS= profiled/dict.o profiled/unix.o profiled/e2fsck.o \
profiled/super.o profiled/pass1.o profiled/pass1b.o \
@@ -73,7 +73,8 @@ PROFILED_OBJS= profiled/dict.o profiled/unix.o profiled/e2fsck.o \
profiled/recovery.o profiled/region.o profiled/revoke.o \
profiled/ea_refcount.o profiled/rehash.o profiled/profile.o \
profiled/prof_err.o profiled/logfile.o \
- profiled/sigcatcher.o profiled/plausible.o
+ profiled/sigcatcher.o profiled/plausible.o \
+ profiled/sigcatcher.o profiled/readahead.o
SRCS= $(srcdir)/e2fsck.c \
$(srcdir)/dict.c \
@@ -97,6 +98,7 @@ SRCS= $(srcdir)/e2fsck.c \
$(srcdir)/message.c \
$(srcdir)/ea_refcount.c \
$(srcdir)/rehash.c \
+ $(srcdir)/readahead.c \
$(srcdir)/region.c \
$(srcdir)/profile.c \
$(srcdir)/sigcatcher.c \
@@ -541,3 +543,6 @@ plausible.o: $(srcdir)/../misc/plausible.c $(top_builddir)/lib/config.h \
$(top_builddir)/lib/ext2fs/ext2_err.h \
$(top_srcdir)/lib/ext2fs/ext2_ext_attr.h $(top_srcdir)/lib/ext2fs/bitops.h \
$(srcdir)/../misc/nls-enable.h $(srcdir)/../misc/plausible.h
+readahead.o: $(srcdir)/readahead.c $(top_builddir)/lib/config.h \
+ $(top_srcdir)/lib/ext2fs/ext2fs.h $(top_srcdir)/lib/ext2fs/ext2_fs.h \
+ $(top_builddir)/lib/ext2fs/ext2_err.h $(srcdir)/e2fsck.h prof_err.h
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index 615ad75..0252824 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -490,6 +490,23 @@ extern ext2_ino_t e2fsck_get_lost_and_found(e2fsck_t ctx, int fix);
extern errcode_t e2fsck_adjust_inode_count(e2fsck_t ctx, ext2_ino_t ino,
int adj);
+/* readahead.c */
+#define E2FSCK_READA_SUPER (0x01)
+#define E2FSCK_READA_GDT (0x02)
+#define E2FSCK_READA_BBITMAP (0x04)
+#define E2FSCK_READA_IBITMAP (0x08)
+#define E2FSCK_READA_ITABLE (0x10)
+#define E2FSCK_READA_ALL_FLAGS (0x1F)
+errcode_t e2fsck_readahead(ext2_filsys fs, int flags, dgrp_t start,
+ dgrp_t ngroups);
+#define E2FSCK_RA_DBLIST_IGNORE_BLOCKCNT (0x01)
+#define E2FSCK_RA_DBLIST_ALL_FLAGS (0x01)
+errcode_t e2fsck_readahead_dblist(ext2_filsys fs, int flags,
+ ext2_dblist dblist,
+ unsigned long long start,
+ unsigned long long count);
+int e2fsck_can_readahead(ext2_filsys fs);
+unsigned long long e2fsck_guess_readahead(ext2_filsys fs);
/* region.c */
extern region_t region_create(region_addr_t min, region_addr_t max);
@@ -577,6 +594,7 @@ extern errcode_t e2fsck_allocate_subcluster_bitmap(ext2_filsys fs,
int default_type,
const char *profile_name,
ext2fs_block_bitmap *ret);
+unsigned long long get_memory_size(void);
/* unix.c */
extern void e2fsck_clear_progbar(e2fsck_t ctx);
diff --git a/e2fsck/readahead.c b/e2fsck/readahead.c
new file mode 100644
index 0000000..a35f9f8
--- /dev/null
+++ b/e2fsck/readahead.c
@@ -0,0 +1,252 @@
+/*
+ * readahead.c -- Prefetch filesystem metadata to speed up fsck.
+ *
+ * Copyright (C) 2014 Oracle.
+ *
+ * %Begin-Header%
+ * This file may be redistributed under the terms of the GNU Library
+ * General Public License, version 2.
+ * %End-Header%
+ */
+
+#include "config.h"
+#include <string.h>
+
+#include "e2fsck.h"
+
+#undef DEBUG
+
+#ifdef DEBUG
+# define dbg_printf(f, a...) do {printf(f, ## a); fflush(stdout); } while (0)
+#else
+# define dbg_printf(f, a...)
+#endif
+
+struct read_dblist {
+ errcode_t err;
+ blk64_t run_start;
+ blk64_t run_len;
+ int flags;
+};
+
+static int readahead_dir_block(ext2_filsys fs, struct ext2_db_entry2 *db,
+ void *priv_data)
+{
+ struct read_dblist *pr = priv_data;
+ e2_blkcnt_t count = (pr->flags & E2FSCK_RA_DBLIST_IGNORE_BLOCKCNT ?
+ 1 : db->blockcnt);
+
+ if (!pr->run_len || db->blk != pr->run_start + pr->run_len) {
+ if (pr->run_len) {
+ pr->err = io_channel_cache_readahead(fs->io,
+ pr->run_start,
+ pr->run_len);
+ dbg_printf("readahead start=%llu len=%llu err=%d\n",
+ pr->run_start, pr->run_len,
+ (int)pr->err);
+ }
+ pr->run_start = db->blk;
+ pr->run_len = 0;
+ }
+ pr->run_len += count;
+
+ return pr->err ? DBLIST_ABORT : 0;
+}
+
+errcode_t e2fsck_readahead_dblist(ext2_filsys fs, int flags,
+ ext2_dblist dblist,
+ unsigned long long start,
+ unsigned long long count)
+{
+ errcode_t err;
+ struct read_dblist pr;
+
+ dbg_printf("%s: flags=0x%x\n", __func__, flags);
+ if (flags & ~E2FSCK_RA_DBLIST_ALL_FLAGS)
+ return EXT2_ET_INVALID_ARGUMENT;
+
+ memset(&pr, 0, sizeof(pr));
+ pr.flags = flags;
+ err = ext2fs_dblist_iterate3(dblist, readahead_dir_block, start,
+ count, &pr);
+ if (pr.err)
+ return pr.err;
+ if (err)
+ return err;
+
+ if (pr.run_len)
+ err = io_channel_cache_readahead(fs->io, pr.run_start,
+ pr.run_len);
+
+ return err;
+}
+
+static errcode_t e2fsck_readahead_bitmap(ext2_filsys fs,
+ ext2fs_block_bitmap ra_map)
+{
+ blk64_t start, end, out;
+ errcode_t err;
+
+ start = 1;
+ end = ext2fs_blocks_count(fs->super) - 1;
+
+ err = ext2fs_find_first_set_block_bitmap2(ra_map, start, end, &out);
+ while (err == 0) {
+ start = out;
+ err = ext2fs_find_first_zero_block_bitmap2(ra_map, start, end,
+ &out);
+ if (err == ENOENT) {
+ out = end;
+ err = 0;
+ } else if (err)
+ break;
+
+ err = io_channel_cache_readahead(fs->io, start, out - start);
+ if (err)
+ break;
+ start = out;
+ err = ext2fs_find_first_set_block_bitmap2(ra_map, start, end,
+ &out);
+ }
+
+ if (err == ENOENT)
+ err = 0;
+
+ return err;
+}
+
+/* Try not to spew bitmap range errors for readahead */
+static errcode_t mark_bmap_range(ext2_filsys fs, ext2fs_block_bitmap map,
+ blk64_t blk, unsigned int num)
+{
+ if (blk >= ext2fs_get_generic_bmap_start(map) &&
+ blk + num <= ext2fs_get_generic_bmap_end(map))
+ ext2fs_mark_block_bitmap_range2(map, blk, num);
+ else
+ return EXT2_ET_INVALID_ARGUMENT;
+ return 0;
+}
+
+static errcode_t mark_bmap(ext2_filsys fs, ext2fs_block_bitmap map, blk64_t blk)
+{
+ if (blk >= ext2fs_get_generic_bmap_start(map) &&
+ blk <= ext2fs_get_generic_bmap_end(map))
+ ext2fs_mark_block_bitmap2(map, blk);
+ else
+ return EXT2_ET_INVALID_ARGUMENT;
+ return 0;
+}
+
+errcode_t e2fsck_readahead(ext2_filsys fs, int flags, dgrp_t start,
+ dgrp_t ngroups)
+{
+ blk64_t super, old_gdt, new_gdt;
+ blk_t blocks;
+ dgrp_t i;
+ ext2fs_block_bitmap ra_map = NULL;
+ dgrp_t end = start + ngroups;
+ errcode_t err = 0;
+
+ dbg_printf("%s: flags=0x%x start=%d groups=%d\n", __func__, flags,
+ start, ngroups);
+ if (flags & ~E2FSCK_READA_ALL_FLAGS)
+ return EXT2_ET_INVALID_ARGUMENT;
+
+ if (end > fs->group_desc_count)
+ end = fs->group_desc_count;
+
+ if (flags == 0)
+ return 0;
+
+ err = ext2fs_allocate_block_bitmap(fs, "readahead bitmap",
+ &ra_map);
+ if (err)
+ return err;
+
+ for (i = start; i < end; i++) {
+ err = ext2fs_super_and_bgd_loc2(fs, i, &super, &old_gdt,
+ &new_gdt, &blocks);
+ if (err)
+ break;
+
+ if (flags & E2FSCK_READA_SUPER) {
+ err = mark_bmap(fs, ra_map, super);
+ if (err)
+ break;
+ }
+
+ if (flags & E2FSCK_READA_GDT) {
+ err = mark_bmap_range(fs, ra_map,
+ old_gdt ? old_gdt : new_gdt,
+ blocks);
+ if (err)
+ break;
+ }
+
+ if ((flags & E2FSCK_READA_BBITMAP) &&
+ !ext2fs_bg_flags_test(fs, i, EXT2_BG_BLOCK_UNINIT) &&
+ ext2fs_bg_free_blocks_count(fs, i) <
+ fs->super->s_blocks_per_group) {
+ super = ext2fs_block_bitmap_loc(fs, i);
+ err = mark_bmap(fs, ra_map, super);
+ if (err)
+ break;
+ }
+
+ if ((flags & E2FSCK_READA_IBITMAP) &&
+ !ext2fs_bg_flags_test(fs, i, EXT2_BG_INODE_UNINIT) &&
+ ext2fs_bg_free_inodes_count(fs, i) <
+ fs->super->s_inodes_per_group) {
+ super = ext2fs_inode_bitmap_loc(fs, i);
+ err = mark_bmap(fs, ra_map, super);
+ if (err)
+ break;
+ }
+
+ if ((flags & E2FSCK_READA_ITABLE) &&
+ ext2fs_bg_free_inodes_count(fs, i) <
+ fs->super->s_inodes_per_group) {
+ super = ext2fs_inode_table_loc(fs, i);
+ blocks = fs->inode_blocks_per_group -
+ (ext2fs_bg_itable_unused(fs, i) *
+ EXT2_INODE_SIZE(fs->super) / fs->blocksize);
+ err = mark_bmap_range(fs, ra_map, super, blocks);
+ if (err)
+ break;
+ }
+ }
+
+ if (!err)
+ err = e2fsck_readahead_bitmap(fs, ra_map);
+
+ ext2fs_free_block_bitmap(ra_map);
+ return err;
+}
+
+int e2fsck_can_readahead(ext2_filsys fs)
+{
+ errcode_t err;
+
+ err = io_channel_cache_readahead(fs->io, 0, 1);
+ dbg_printf("%s: supp=%d\n", __func__, err != EXT2_ET_OP_NOT_SUPPORTED);
+ return err != EXT2_ET_OP_NOT_SUPPORTED;
+}
+
+unsigned long long e2fsck_guess_readahead(ext2_filsys fs)
+{
+ unsigned long long guess;
+
+ /*
+ * The optimal readahead sizes were experimentally determined by
+ * djwong in August 2014. Setting the RA size to one block group's
+ * worth of inode table blocks seems to yield the largest reductions
+ * in e2fsck runtime.
+ */
+ guess = fs->blocksize * fs->inode_blocks_per_group;
+
+ /* Disable RA if it'd use more 1/100th of RAM. */
+ if (get_memory_size() > (guess * 100))
+ return guess / 1024;
+
+ return 0;
+}
diff --git a/e2fsck/util.c b/e2fsck/util.c
index 2de45f8..723dafb 100644
--- a/e2fsck/util.c
+++ b/e2fsck/util.c
@@ -37,6 +37,10 @@
#include <errno.h>
#endif
+#ifdef HAVE_SYS_SYSCTL_H
+#include <sys/sysctl.h>
+#endif
+
#include "e2fsck.h"
extern e2fsck_t e2fsck_global_ctx; /* Try your very best not to use this! */
@@ -795,3 +799,50 @@ errcode_t e2fsck_allocate_subcluster_bitmap(ext2_filsys fs, const char *descr,
fs->default_bitmap_type = save_type;
return retval;
}
+
+/* Return memory size in bytes */
+unsigned long long get_memory_size(void)
+{
+#if defined(_SC_PHYS_PAGES)
+# if defined(_SC_PAGESIZE)
+ return (unsigned long long)sysconf(_SC_PHYS_PAGES) *
+ (unsigned long long)sysconf(_SC_PAGESIZE);
+# elif defined(_SC_PAGE_SIZE)
+ return (unsigned long long)sysconf(_SC_PHYS_PAGES) *
+ (unsigned long long)sysconf(_SC_PAGE_SIZE);
+# endif
+#elif defined(CTL_HW)
+# if (defined(HW_MEMSIZE) || defined(HW_PHYSMEM64))
+# define CTL_HW_INT64
+# elif (defined(HW_PHYSMEM) || defined(HW_REALMEM))
+# define CTL_HW_UINT
+# endif
+ int mib[2];
+
+ mib[0] = CTL_HW;
+# if defined(HW_MEMSIZE)
+ mib[1] = HW_MEMSIZE;
+# elif defined(HW_PHYSMEM64)
+ mib[1] = HW_PHYSMEM64;
+# elif defined(HW_REALMEM)
+ mib[1] = HW_REALMEM;
+# elif defined(HW_PYSMEM)
+ mib[1] = HW_PHYSMEM;
+# endif
+# if defined(CTL_HW_INT64)
+ unsigned long long size = 0;
+# elif defined(CTL_HW_UINT)
+ unsigned int size = 0;
+# endif
+# if defined(CTL_HW_INT64) || defined(CTL_HW_UINT)
+ size_t len = sizeof(size);
+
+ if (sysctl(mib, 2, &size, &len, NULL, 0) == 0)
+ return (unsigned long long)size;
+# endif
+ return 0;
+#else
+# warning "Don't know how to detect memory on your platform?"
+ return 0;
+#endif
+}
diff --git a/lib/config.h.in b/lib/config.h.in
index 0db010f..cd7ec90 100644
--- a/lib/config.h.in
+++ b/lib/config.h.in
@@ -509,6 +509,9 @@
/* Define to 1 if you have the <sys/syscall.h> header file. */
#undef HAVE_SYS_SYSCALL_H
+/* Define to 1 if you have the <sys/sysctl.h> header file. */
+#undef HAVE_SYS_SYSCTL_H
+
/* Define to 1 if you have the <sys/sysmacros.h> header file. */
#undef HAVE_SYS_SYSMACROS_H
diff --git a/lib/ext2fs/dblist.c b/lib/ext2fs/dblist.c
index 942c4f0..bbdb221 100644
--- a/lib/ext2fs/dblist.c
+++ b/lib/ext2fs/dblist.c
@@ -194,20 +194,25 @@ void ext2fs_dblist_sort2(ext2_dblist dblist,
/*
* This function iterates over the directory block list
*/
-errcode_t ext2fs_dblist_iterate2(ext2_dblist dblist,
+errcode_t ext2fs_dblist_iterate3(ext2_dblist dblist,
int (*func)(ext2_filsys fs,
struct ext2_db_entry2 *db_info,
void *priv_data),
+ unsigned long long start,
+ unsigned long long count,
void *priv_data)
{
- unsigned long long i;
+ unsigned long long i, end;
int ret;
EXT2_CHECK_MAGIC(dblist, EXT2_ET_MAGIC_DBLIST);
+ end = start + count;
if (!dblist->sorted)
ext2fs_dblist_sort2(dblist, 0);
- for (i=0; i < dblist->count; i++) {
+ if (end > dblist->count)
+ end = dblist->count;
+ for (i = start; i < end; i++) {
ret = (*func)(dblist->fs, &dblist->list[i], priv_data);
if (ret & DBLIST_ABORT)
return 0;
@@ -215,6 +220,16 @@ errcode_t ext2fs_dblist_iterate2(ext2_dblist dblist,
return 0;
}
+errcode_t ext2fs_dblist_iterate2(ext2_dblist dblist,
+ int (*func)(ext2_filsys fs,
+ struct ext2_db_entry2 *db_info,
+ void *priv_data),
+ void *priv_data)
+{
+ return ext2fs_dblist_iterate3(dblist, func, 0, dblist->count,
+ priv_data);
+}
+
static EXT2_QSORT_TYPE dir_block_cmp2(const void *a, const void *b)
{
const struct ext2_db_entry2 *db_a =
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index c3142f7..34b9132 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -1051,11 +1051,17 @@ extern void ext2fs_dblist_sort2(ext2_dblist dblist,
extern errcode_t ext2fs_dblist_iterate(ext2_dblist dblist,
int (*func)(ext2_filsys fs, struct ext2_db_entry *db_info,
void *priv_data),
- void *priv_data);
+ void *priv_data);
extern errcode_t ext2fs_dblist_iterate2(ext2_dblist dblist,
int (*func)(ext2_filsys fs, struct ext2_db_entry2 *db_info,
void *priv_data),
- void *priv_data);
+ void *priv_data);
+extern errcode_t ext2fs_dblist_iterate3(ext2_dblist dblist,
+ int (*func)(ext2_filsys fs, struct ext2_db_entry2 *db_info,
+ void *priv_data),
+ unsigned long long start,
+ unsigned long long count,
+ void *priv_data);
extern errcode_t ext2fs_set_dir_block(ext2_dblist dblist, ext2_ino_t ino,
blk_t blk, int blockcnt);
extern errcode_t ext2fs_set_dir_block2(ext2_dblist dblist, ext2_ino_t ino,
^ permalink raw reply related [flat|nested] 104+ messages in thread* Re: [PATCH 31/47] libext2fs/e2fsck: provide routines to read-ahead metadata
2014-11-07 21:54 ` [PATCH 31/47] libext2fs/e2fsck: provide routines to read-ahead metadata Darrick J. Wong
@ 2014-12-15 17:36 ` Theodore Ts'o
2014-12-15 19:19 ` Darrick J. Wong
0 siblings, 1 reply; 104+ messages in thread
From: Theodore Ts'o @ 2014-12-15 17:36 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Fri, Nov 07, 2014 at 01:54:07PM -0800, Darrick J. Wong wrote:
>
> v2: Add an API to create a dblist with a given number of list elements
> pre-allocated. This enables us to save ~2ms per call to
> e2fsck_readahead() (assuming a 2MB RA buffer) by not having to
> repeatedly call ext2_resize_mem as we add blocks to the list.
I don't see the API to create a dblist with a specified number of list
elements? It should be pretty trivial, given make_dblist() but I
don't see it in this patch?
- Ted
^ permalink raw reply [flat|nested] 104+ messages in thread
* Re: [PATCH 31/47] libext2fs/e2fsck: provide routines to read-ahead metadata
2014-12-15 17:36 ` Theodore Ts'o
@ 2014-12-15 19:19 ` Darrick J. Wong
0 siblings, 0 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-12-15 19:19 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: linux-ext4
On Mon, Dec 15, 2014 at 12:36:28PM -0500, Theodore Ts'o wrote:
> On Fri, Nov 07, 2014 at 01:54:07PM -0800, Darrick J. Wong wrote:
> >
> > v2: Add an API to create a dblist with a given number of list elements
> > pre-allocated. This enables us to save ~2ms per call to
> > e2fsck_readahead() (assuming a 2MB RA buffer) by not having to
> > repeatedly call ext2_resize_mem as we add blocks to the list.
>
> I don't see the API to create a dblist with a specified number of list
> elements? It should be pretty trivial, given make_dblist() but I
> don't see it in this patch?
Ugh, part of the commit message came off. There should be after that:
v3: Instead of creating dblists of arbitrary size, change the dblist
iterator to allow iterating a sub-range. This eliminates a lot of
unnecessary list copying during e2fsck part2.
Sorry about that. :/
--D
>
> - Ted
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 104+ messages in thread
* [PATCH 32/47] e2fsck: read-ahead metadata during passes 1, 2, and 4
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (30 preceding siblings ...)
2014-11-07 21:54 ` [PATCH 31/47] libext2fs/e2fsck: provide routines to read-ahead metadata Darrick J. Wong
@ 2014-11-07 21:54 ` Darrick J. Wong
2014-12-10 20:27 ` Darrick J. Wong
2014-11-07 21:54 ` [PATCH 33/47] e2fsck: rebuild sparse extent trees/convert non-extent ext3 files Darrick J. Wong
` (16 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:54 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
e2fsck pass1 is modified to use the block group data prefetch function
to try to fetch the inode tables into the pagecache before it is
needed. We iterate through the blockgroups until we have enough inode
tables that need reading such that we can issue readahead; then we sit
and wait until the last inode table block read of the last group to
start fetching the next bunch.
pass2 is modified to use the dirblock prefetching function to prefetch
the list of directory blocks that are assembled in pass1. We use the
"iterate a subset of a dblist" and avoid copying the dblist. Directory
blocks are fetched incrementally as we walk through the directory
block list. In previous iterations of this patch we would free the
directory blocks after processing, but the performance hit to e2fsck
itself wasn't worth it. Furthermore, it is anticipated that most
users will then mount the FS and start using the directories, so they
may as well remain in the page cache.
pass4 is modified to prefetch the block and inode bitmaps in
anticipation of pass 5, because pass4 is entirely CPU bound.
In general, these mechanisms can decrease fsck time by 10-40%, if the
host system has sufficient memory and the storage system can provide a
lot of IOPs. Pretty much any storage system capable of handling
multiple IOs in-flight at any time will see a fairly large performance
boost. (Single-issue USB mass storage disks seem to suffer badly.)
By default, the readahead buffer size will be set to the size of a block
group's inode table (which is 2MiB for a regular ext4 FS). The -E
readahead_kb= option can be given to specify the amount of memory to
use for readahead or zero to disable it entirely; or an option can be
given in e2fsck.conf.
v2: Fix an off-by-one error in the pass1 readahead which made the
readahead trigger one inode too late if the block groups are full.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
e2fsck/e2fsck.8.in | 7 +++++
e2fsck/e2fsck.conf.5.in | 15 +++++++++++
e2fsck/e2fsck.h | 3 ++
e2fsck/pass1.c | 65 +++++++++++++++++++++++++++++++++++++++++++++++
e2fsck/pass2.c | 38 +++++++++++++++++++++++++++
e2fsck/pass4.c | 9 +++++++
e2fsck/unix.c | 28 ++++++++++++++++++++
lib/ext2fs/ext2fs.h | 1 +
lib/ext2fs/inode.c | 3 +-
9 files changed, 167 insertions(+), 2 deletions(-)
diff --git a/e2fsck/e2fsck.8.in b/e2fsck/e2fsck.8.in
index f5ed758..84ae50f 100644
--- a/e2fsck/e2fsck.8.in
+++ b/e2fsck/e2fsck.8.in
@@ -207,6 +207,13 @@ option may prevent you from further manual data recovery.
.BI nodiscard
Do not attempt to discard free blocks and unused inode blocks. This option is
exactly the opposite of discard option. This is set as default.
+.TP
+.BI readahead_kb
+Use this many KiB of memory to pre-fetch metadata in the hopes of reducing
+e2fsck runtime. By default, this is set to the size of a block group's inode
+table (typically 2MiB on a regular ext4 filesystem); if this amount is more
+than 1/100 of total physical memory, readahead is disabled. Set this to zero
+to disable readahead entirely.
.RE
.TP
.B \-f
diff --git a/e2fsck/e2fsck.conf.5.in b/e2fsck/e2fsck.conf.5.in
index 9ebfbbf..e1d0518 100644
--- a/e2fsck/e2fsck.conf.5.in
+++ b/e2fsck/e2fsck.conf.5.in
@@ -205,6 +205,21 @@ of that type are squelched. This can be useful if the console is slow
(i.e., connected to a serial port) and so a large amount of output could
end up delaying the boot process for a long time (potentially hours).
.TP
+.I readahead_mem_pct
+Use this percentage of memory to try to read in metadata blocks ahead of the
+main e2fsck thread. This should reduce run times, depending on the speed of
+the underlying storage and the amount of free memory. There is no default, but
+see
+.B readahead_mem_pct
+for more details.
+.TP
+.I readahead_kb
+Use this amount of memory to read in metadata blocks ahead of the main checking
+thread. Setting this value to zero disables readahead entirely. By default,
+this is set the size of one block group's inode table (typically 2MiB on a
+regular ext4 filesystem); if this amount is more than 1/100th of total physical
+memory, readahead is disabled.
+.TP
.I report_features
If this boolean relation is true, e2fsck will print the file system
features as part of its verbose reporting (i.e., if the
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index 0252824..e359515 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -378,6 +378,9 @@ struct e2fsck_struct {
*/
void *priv_data;
ext2fs_block_bitmap block_metadata_map; /* Metadata blocks */
+
+ /* How much are we allowed to readahead? */
+ unsigned long long readahead_kb;
};
/* Used by the region allocation code */
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 4cc58c4..a963849 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -868,6 +868,60 @@ out:
return 0;
}
+static void pass1_readahead(e2fsck_t ctx, dgrp_t *group, ext2_ino_t *next_ino)
+{
+ ext2_ino_t inodes_in_group = 0, inodes_per_block, inodes_per_buffer;
+ dgrp_t start = *group, grp;
+ blk64_t blocks_to_read = 0;
+ errcode_t err = EXT2_ET_INVALID_ARGUMENT;
+
+ if (ctx->readahead_kb == 0)
+ goto out;
+
+ /* Keep iterating groups until we have enough to readahead */
+ inodes_per_block = EXT2_INODES_PER_BLOCK(ctx->fs->super);
+ for (grp = start; grp < ctx->fs->group_desc_count; grp++) {
+ if (ext2fs_bg_flags_test(ctx->fs, grp, EXT2_BG_INODE_UNINIT))
+ continue;
+ inodes_in_group = ctx->fs->super->s_inodes_per_group -
+ ext2fs_bg_itable_unused(ctx->fs, grp);
+ blocks_to_read += (inodes_in_group + inodes_per_block - 1) /
+ inodes_per_block;
+ if (blocks_to_read * ctx->fs->blocksize >
+ ctx->readahead_kb * 1024)
+ break;
+ }
+
+ err = e2fsck_readahead(ctx->fs, E2FSCK_READA_ITABLE, start,
+ grp - start + 1);
+ if (err == EAGAIN) {
+ ctx->readahead_kb /= 2;
+ err = 0;
+ }
+
+out:
+ if (err) {
+ /* Error; disable itable readahead */
+ *group = ctx->fs->group_desc_count;
+ *next_ino = ctx->fs->super->s_inodes_count;
+ } else {
+ /*
+ * Don't do more readahead until we've reached the first inode
+ * of the last inode scan buffer block for the last group.
+ */
+ *group = grp + 1;
+ inodes_per_buffer = (ctx->inode_buffer_blocks ?
+ ctx->inode_buffer_blocks :
+ EXT2_INODE_SCAN_DEFAULT_BUFFER_BLOCKS) *
+ ctx->fs->blocksize /
+ EXT2_INODE_SIZE(ctx->fs->super);
+ inodes_in_group--;
+ *next_ino = inodes_in_group -
+ (inodes_in_group % inodes_per_buffer) + 1 +
+ (grp * ctx->fs->super->s_inodes_per_group);
+ }
+}
+
void e2fsck_pass1(e2fsck_t ctx)
{
int i;
@@ -890,10 +944,19 @@ void e2fsck_pass1(e2fsck_t ctx)
int low_dtime_check = 1;
int inode_size;
int failed_csum = 0;
+ ext2_ino_t ino_threshold = 0;
+ dgrp_t ra_group = 0;
init_resource_track(&rtrack, ctx->fs->io);
clear_problem_context(&pctx);
+ /* If we can do readahead, figure out how many groups to pull in. */
+ if (!e2fsck_can_readahead(ctx->fs))
+ ctx->readahead_kb = 0;
+ else if (ctx->readahead_kb == ~0ULL)
+ ctx->readahead_kb = e2fsck_guess_readahead(ctx->fs);
+ pass1_readahead(ctx, &ra_group, &ino_threshold);
+
if (!(ctx->options & E2F_OPT_PREEN))
fix_problem(ctx, PR_1_PASS_HEADER, &pctx);
@@ -1073,6 +1136,8 @@ void e2fsck_pass1(e2fsck_t ctx)
old_op = ehandler_operation(_("getting next inode from scan"));
pctx.errcode = ext2fs_get_next_inode_full(scan, &ino,
inode, inode_size);
+ if (ino > ino_threshold)
+ pass1_readahead(ctx, &ra_group, &ino_threshold);
ehandler_operation(old_op);
if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
return;
diff --git a/e2fsck/pass2.c b/e2fsck/pass2.c
index 7aaebce..cffaac4 100644
--- a/e2fsck/pass2.c
+++ b/e2fsck/pass2.c
@@ -61,6 +61,9 @@
* Keeps track of how many times an inode is referenced.
*/
static void deallocate_inode(e2fsck_t ctx, ext2_ino_t ino, char* block_buf);
+static int check_dir_block2(ext2_filsys fs,
+ struct ext2_db_entry2 *dir_blocks_info,
+ void *priv_data);
static int check_dir_block(ext2_filsys fs,
struct ext2_db_entry2 *dir_blocks_info,
void *priv_data);
@@ -77,6 +80,9 @@ struct check_dir_struct {
struct problem_context pctx;
int count, max;
e2fsck_t ctx;
+ unsigned long long list_offset;
+ unsigned long long ra_entries;
+ unsigned long long next_ra_off;
};
void e2fsck_pass2(e2fsck_t ctx)
@@ -96,6 +102,9 @@ void e2fsck_pass2(e2fsck_t ctx)
int i, depth;
problem_t code;
int bad_dir;
+ int (*check_dir_func)(ext2_filsys fs,
+ struct ext2_db_entry2 *dir_blocks_info,
+ void *priv_data);
init_resource_track(&rtrack, ctx->fs->io);
clear_problem_context(&cd.pctx);
@@ -139,6 +148,9 @@ void e2fsck_pass2(e2fsck_t ctx)
cd.ctx = ctx;
cd.count = 1;
cd.max = ext2fs_dblist_count2(fs->dblist);
+ cd.list_offset = 0;
+ cd.ra_entries = ctx->readahead_kb * 1024 / ctx->fs->blocksize;
+ cd.next_ra_off = 0;
if (ctx->progress)
(void) (ctx->progress)(ctx, 2, 0, cd.max);
@@ -146,7 +158,8 @@ void e2fsck_pass2(e2fsck_t ctx)
if (fs->super->s_feature_compat & EXT2_FEATURE_COMPAT_DIR_INDEX)
ext2fs_dblist_sort2(fs->dblist, special_dir_block_cmp);
- cd.pctx.errcode = ext2fs_dblist_iterate2(fs->dblist, check_dir_block,
+ check_dir_func = cd.ra_entries ? check_dir_block2 : check_dir_block;
+ cd.pctx.errcode = ext2fs_dblist_iterate2(fs->dblist, check_dir_func,
&cd);
if (ctx->flags & E2F_FLAG_SIGNAL_MASK || ctx->flags & E2F_FLAG_RESTART)
return;
@@ -825,6 +838,29 @@ err:
return retval;
}
+static int check_dir_block2(ext2_filsys fs,
+ struct ext2_db_entry2 *db,
+ void *priv_data)
+{
+ int err;
+ struct check_dir_struct *cd = priv_data;
+
+ if (cd->ra_entries && cd->list_offset >= cd->next_ra_off) {
+ err = e2fsck_readahead_dblist(fs,
+ E2FSCK_RA_DBLIST_IGNORE_BLOCKCNT,
+ fs->dblist,
+ cd->list_offset + cd->ra_entries / 8,
+ cd->ra_entries);
+ if (err)
+ cd->ra_entries = 0;
+ cd->next_ra_off = cd->list_offset + (cd->ra_entries * 7 / 8);
+ }
+
+ err = check_dir_block(fs, db, priv_data);
+ cd->list_offset++;
+ return err;
+}
+
static int check_dir_block(ext2_filsys fs,
struct ext2_db_entry2 *db,
void *priv_data)
diff --git a/e2fsck/pass4.c b/e2fsck/pass4.c
index 21d93f0..bc9a2c4 100644
--- a/e2fsck/pass4.c
+++ b/e2fsck/pass4.c
@@ -106,6 +106,15 @@ void e2fsck_pass4(e2fsck_t ctx)
#ifdef MTRACE
mtrace_print("Pass 4");
#endif
+ /*
+ * Since pass4 is mostly CPU bound, start readahead of bitmaps
+ * ahead of pass 5 if we haven't already loaded them.
+ */
+ if (ctx->readahead_kb &&
+ (fs->block_map == NULL || fs->inode_map == NULL))
+ e2fsck_readahead(fs, E2FSCK_READA_BBITMAP |
+ E2FSCK_READA_IBITMAP,
+ 0, fs->group_desc_count);
clear_problem_context(&pctx);
diff --git a/e2fsck/unix.c b/e2fsck/unix.c
index 615d690..f3672c0 100644
--- a/e2fsck/unix.c
+++ b/e2fsck/unix.c
@@ -650,6 +650,7 @@ static void parse_extended_opts(e2fsck_t ctx, const char *opts)
char *buf, *token, *next, *p, *arg;
int ea_ver;
int extended_usage = 0;
+ unsigned long long reada_kb;
buf = string_copy(ctx, opts, 0);
for (token = buf; token && *token; token = next) {
@@ -678,6 +679,15 @@ static void parse_extended_opts(e2fsck_t ctx, const char *opts)
continue;
}
ctx->ext_attr_ver = ea_ver;
+ } else if (strcmp(token, "readahead_kb") == 0) {
+ reada_kb = strtoull(arg, &p, 0);
+ if (*p) {
+ fprintf(stderr, "%s",
+ _("Invalid readahead buffer size.\n"));
+ extended_usage++;
+ continue;
+ }
+ ctx->readahead_kb = reada_kb;
} else if (strcmp(token, "fragcheck") == 0) {
ctx->options |= E2F_OPT_FRAGCHECK;
continue;
@@ -717,6 +727,7 @@ static void parse_extended_opts(e2fsck_t ctx, const char *opts)
fputs(("\tjournal_only\n"), stderr);
fputs(("\tdiscard\n"), stderr);
fputs(("\tnodiscard\n"), stderr);
+ fputs(("\treadahead_kb=<buffer size>\n"), stderr);
fputc('\n', stderr);
exit(1);
}
@@ -750,6 +761,7 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
#ifdef CONFIG_JBD_DEBUG
char *jbd_debug;
#endif
+ unsigned long long phys_mem_kb;
retval = e2fsck_allocate_context(&ctx);
if (retval)
@@ -777,6 +789,8 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
else
ctx->program_name = "e2fsck";
+ phys_mem_kb = get_memory_size() / 1024;
+ ctx->readahead_kb = ~0ULL;
while ((c = getopt (argc, argv, "panyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDk")) != EOF)
switch (c) {
case 'C':
@@ -961,6 +975,20 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
if (c)
verbose = 1;
+ if (ctx->readahead_kb == ~0ULL) {
+ profile_get_integer(ctx->profile, "options",
+ "readahead_mem_pct", 0, -1, &c);
+ if (c >= 0 && c <= 100)
+ ctx->readahead_kb = phys_mem_kb * c / 100;
+ profile_get_integer(ctx->profile, "options",
+ "readahead_kb", 0, -1, &c);
+ if (c >= 0)
+ ctx->readahead_kb = c;
+ if (ctx->readahead_kb != ~0ULL &&
+ ctx->readahead_kb > phys_mem_kb)
+ ctx->readahead_kb = phys_mem_kb;
+ }
+
/* Turn off discard in read-only mode */
if ((ctx->options & E2F_OPT_NO) &&
(ctx->options & E2F_OPT_DISCARD))
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 34b9132..84e4e1f 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -1418,6 +1418,7 @@ extern errcode_t ext2fs_get_next_inode_full(ext2_inode_scan scan,
ext2_ino_t *ino,
struct ext2_inode *inode,
int bufsize);
+#define EXT2_INODE_SCAN_DEFAULT_BUFFER_BLOCKS 8
extern errcode_t ext2fs_open_inode_scan(ext2_filsys fs, int buffer_blocks,
ext2_inode_scan *ret_scan);
extern void ext2fs_close_inode_scan(ext2_inode_scan scan);
diff --git a/lib/ext2fs/inode.c b/lib/ext2fs/inode.c
index 4310b82..4b3e14e 100644
--- a/lib/ext2fs/inode.c
+++ b/lib/ext2fs/inode.c
@@ -175,7 +175,8 @@ errcode_t ext2fs_open_inode_scan(ext2_filsys fs, int buffer_blocks,
scan->bytes_left = 0;
scan->current_group = 0;
scan->groups_left = fs->group_desc_count - 1;
- scan->inode_buffer_blocks = buffer_blocks ? buffer_blocks : 8;
+ scan->inode_buffer_blocks = buffer_blocks ? buffer_blocks :
+ EXT2_INODE_SCAN_DEFAULT_BUFFER_BLOCKS;
scan->current_block = ext2fs_inode_table_loc(scan->fs,
scan->current_group);
scan->inodes_left = EXT2_INODES_PER_GROUP(scan->fs->super);
^ permalink raw reply related [flat|nested] 104+ messages in thread* Re: [PATCH 32/47] e2fsck: read-ahead metadata during passes 1, 2, and 4
2014-11-07 21:54 ` [PATCH 32/47] e2fsck: read-ahead metadata during passes 1, 2, and 4 Darrick J. Wong
@ 2014-12-10 20:27 ` Darrick J. Wong
2014-12-10 22:00 ` Theodore Ts'o
0 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-12-10 20:27 UTC (permalink / raw)
To: tytso; +Cc: linux-ext4
On Fri, Nov 07, 2014 at 01:54:14PM -0800, Darrick J. Wong wrote:
> e2fsck pass1 is modified to use the block group data prefetch function
> to try to fetch the inode tables into the pagecache before it is
> needed. We iterate through the blockgroups until we have enough inode
> tables that need reading such that we can issue readahead; then we sit
> and wait until the last inode table block read of the last group to
> start fetching the next bunch.
>
> pass2 is modified to use the dirblock prefetching function to prefetch
> the list of directory blocks that are assembled in pass1. We use the
> "iterate a subset of a dblist" and avoid copying the dblist. Directory
> blocks are fetched incrementally as we walk through the directory
> block list. In previous iterations of this patch we would free the
> directory blocks after processing, but the performance hit to e2fsck
> itself wasn't worth it. Furthermore, it is anticipated that most
> users will then mount the FS and start using the directories, so they
> may as well remain in the page cache.
>
> pass4 is modified to prefetch the block and inode bitmaps in
> anticipation of pass 5, because pass4 is entirely CPU bound.
>
> In general, these mechanisms can decrease fsck time by 10-40%, if the
> host system has sufficient memory and the storage system can provide a
> lot of IOPs. Pretty much any storage system capable of handling
> multiple IOs in-flight at any time will see a fairly large performance
> boost. (Single-issue USB mass storage disks seem to suffer badly.)
FWIW I finally got UAS working in Linux; with that hardware and the default
readahead size, I see about a 5% reduction in fsck runtime. The same dock
plugged into a USB2 port (i.e. BOT) shows about a 2% increase in runtime.
--D
>
> By default, the readahead buffer size will be set to the size of a block
> group's inode table (which is 2MiB for a regular ext4 FS). The -E
> readahead_kb= option can be given to specify the amount of memory to
> use for readahead or zero to disable it entirely; or an option can be
> given in e2fsck.conf.
>
> v2: Fix an off-by-one error in the pass1 readahead which made the
> readahead trigger one inode too late if the block groups are full.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
> e2fsck/e2fsck.8.in | 7 +++++
> e2fsck/e2fsck.conf.5.in | 15 +++++++++++
> e2fsck/e2fsck.h | 3 ++
> e2fsck/pass1.c | 65 +++++++++++++++++++++++++++++++++++++++++++++++
> e2fsck/pass2.c | 38 +++++++++++++++++++++++++++
> e2fsck/pass4.c | 9 +++++++
> e2fsck/unix.c | 28 ++++++++++++++++++++
> lib/ext2fs/ext2fs.h | 1 +
> lib/ext2fs/inode.c | 3 +-
> 9 files changed, 167 insertions(+), 2 deletions(-)
>
>
> diff --git a/e2fsck/e2fsck.8.in b/e2fsck/e2fsck.8.in
> index f5ed758..84ae50f 100644
> --- a/e2fsck/e2fsck.8.in
> +++ b/e2fsck/e2fsck.8.in
> @@ -207,6 +207,13 @@ option may prevent you from further manual data recovery.
> .BI nodiscard
> Do not attempt to discard free blocks and unused inode blocks. This option is
> exactly the opposite of discard option. This is set as default.
> +.TP
> +.BI readahead_kb
> +Use this many KiB of memory to pre-fetch metadata in the hopes of reducing
> +e2fsck runtime. By default, this is set to the size of a block group's inode
> +table (typically 2MiB on a regular ext4 filesystem); if this amount is more
> +than 1/100 of total physical memory, readahead is disabled. Set this to zero
> +to disable readahead entirely.
> .RE
> .TP
> .B \-f
> diff --git a/e2fsck/e2fsck.conf.5.in b/e2fsck/e2fsck.conf.5.in
> index 9ebfbbf..e1d0518 100644
> --- a/e2fsck/e2fsck.conf.5.in
> +++ b/e2fsck/e2fsck.conf.5.in
> @@ -205,6 +205,21 @@ of that type are squelched. This can be useful if the console is slow
> (i.e., connected to a serial port) and so a large amount of output could
> end up delaying the boot process for a long time (potentially hours).
> .TP
> +.I readahead_mem_pct
> +Use this percentage of memory to try to read in metadata blocks ahead of the
> +main e2fsck thread. This should reduce run times, depending on the speed of
> +the underlying storage and the amount of free memory. There is no default, but
> +see
> +.B readahead_mem_pct
> +for more details.
> +.TP
> +.I readahead_kb
> +Use this amount of memory to read in metadata blocks ahead of the main checking
> +thread. Setting this value to zero disables readahead entirely. By default,
> +this is set the size of one block group's inode table (typically 2MiB on a
> +regular ext4 filesystem); if this amount is more than 1/100th of total physical
> +memory, readahead is disabled.
> +.TP
> .I report_features
> If this boolean relation is true, e2fsck will print the file system
> features as part of its verbose reporting (i.e., if the
> diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
> index 0252824..e359515 100644
> --- a/e2fsck/e2fsck.h
> +++ b/e2fsck/e2fsck.h
> @@ -378,6 +378,9 @@ struct e2fsck_struct {
> */
> void *priv_data;
> ext2fs_block_bitmap block_metadata_map; /* Metadata blocks */
> +
> + /* How much are we allowed to readahead? */
> + unsigned long long readahead_kb;
> };
>
> /* Used by the region allocation code */
> diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
> index 4cc58c4..a963849 100644
> --- a/e2fsck/pass1.c
> +++ b/e2fsck/pass1.c
> @@ -868,6 +868,60 @@ out:
> return 0;
> }
>
> +static void pass1_readahead(e2fsck_t ctx, dgrp_t *group, ext2_ino_t *next_ino)
> +{
> + ext2_ino_t inodes_in_group = 0, inodes_per_block, inodes_per_buffer;
> + dgrp_t start = *group, grp;
> + blk64_t blocks_to_read = 0;
> + errcode_t err = EXT2_ET_INVALID_ARGUMENT;
> +
> + if (ctx->readahead_kb == 0)
> + goto out;
> +
> + /* Keep iterating groups until we have enough to readahead */
> + inodes_per_block = EXT2_INODES_PER_BLOCK(ctx->fs->super);
> + for (grp = start; grp < ctx->fs->group_desc_count; grp++) {
> + if (ext2fs_bg_flags_test(ctx->fs, grp, EXT2_BG_INODE_UNINIT))
> + continue;
> + inodes_in_group = ctx->fs->super->s_inodes_per_group -
> + ext2fs_bg_itable_unused(ctx->fs, grp);
> + blocks_to_read += (inodes_in_group + inodes_per_block - 1) /
> + inodes_per_block;
> + if (blocks_to_read * ctx->fs->blocksize >
> + ctx->readahead_kb * 1024)
> + break;
> + }
> +
> + err = e2fsck_readahead(ctx->fs, E2FSCK_READA_ITABLE, start,
> + grp - start + 1);
> + if (err == EAGAIN) {
> + ctx->readahead_kb /= 2;
> + err = 0;
> + }
> +
> +out:
> + if (err) {
> + /* Error; disable itable readahead */
> + *group = ctx->fs->group_desc_count;
> + *next_ino = ctx->fs->super->s_inodes_count;
> + } else {
> + /*
> + * Don't do more readahead until we've reached the first inode
> + * of the last inode scan buffer block for the last group.
> + */
> + *group = grp + 1;
> + inodes_per_buffer = (ctx->inode_buffer_blocks ?
> + ctx->inode_buffer_blocks :
> + EXT2_INODE_SCAN_DEFAULT_BUFFER_BLOCKS) *
> + ctx->fs->blocksize /
> + EXT2_INODE_SIZE(ctx->fs->super);
> + inodes_in_group--;
> + *next_ino = inodes_in_group -
> + (inodes_in_group % inodes_per_buffer) + 1 +
> + (grp * ctx->fs->super->s_inodes_per_group);
> + }
> +}
> +
> void e2fsck_pass1(e2fsck_t ctx)
> {
> int i;
> @@ -890,10 +944,19 @@ void e2fsck_pass1(e2fsck_t ctx)
> int low_dtime_check = 1;
> int inode_size;
> int failed_csum = 0;
> + ext2_ino_t ino_threshold = 0;
> + dgrp_t ra_group = 0;
>
> init_resource_track(&rtrack, ctx->fs->io);
> clear_problem_context(&pctx);
>
> + /* If we can do readahead, figure out how many groups to pull in. */
> + if (!e2fsck_can_readahead(ctx->fs))
> + ctx->readahead_kb = 0;
> + else if (ctx->readahead_kb == ~0ULL)
> + ctx->readahead_kb = e2fsck_guess_readahead(ctx->fs);
> + pass1_readahead(ctx, &ra_group, &ino_threshold);
> +
> if (!(ctx->options & E2F_OPT_PREEN))
> fix_problem(ctx, PR_1_PASS_HEADER, &pctx);
>
> @@ -1073,6 +1136,8 @@ void e2fsck_pass1(e2fsck_t ctx)
> old_op = ehandler_operation(_("getting next inode from scan"));
> pctx.errcode = ext2fs_get_next_inode_full(scan, &ino,
> inode, inode_size);
> + if (ino > ino_threshold)
> + pass1_readahead(ctx, &ra_group, &ino_threshold);
> ehandler_operation(old_op);
> if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
> return;
> diff --git a/e2fsck/pass2.c b/e2fsck/pass2.c
> index 7aaebce..cffaac4 100644
> --- a/e2fsck/pass2.c
> +++ b/e2fsck/pass2.c
> @@ -61,6 +61,9 @@
> * Keeps track of how many times an inode is referenced.
> */
> static void deallocate_inode(e2fsck_t ctx, ext2_ino_t ino, char* block_buf);
> +static int check_dir_block2(ext2_filsys fs,
> + struct ext2_db_entry2 *dir_blocks_info,
> + void *priv_data);
> static int check_dir_block(ext2_filsys fs,
> struct ext2_db_entry2 *dir_blocks_info,
> void *priv_data);
> @@ -77,6 +80,9 @@ struct check_dir_struct {
> struct problem_context pctx;
> int count, max;
> e2fsck_t ctx;
> + unsigned long long list_offset;
> + unsigned long long ra_entries;
> + unsigned long long next_ra_off;
> };
>
> void e2fsck_pass2(e2fsck_t ctx)
> @@ -96,6 +102,9 @@ void e2fsck_pass2(e2fsck_t ctx)
> int i, depth;
> problem_t code;
> int bad_dir;
> + int (*check_dir_func)(ext2_filsys fs,
> + struct ext2_db_entry2 *dir_blocks_info,
> + void *priv_data);
>
> init_resource_track(&rtrack, ctx->fs->io);
> clear_problem_context(&cd.pctx);
> @@ -139,6 +148,9 @@ void e2fsck_pass2(e2fsck_t ctx)
> cd.ctx = ctx;
> cd.count = 1;
> cd.max = ext2fs_dblist_count2(fs->dblist);
> + cd.list_offset = 0;
> + cd.ra_entries = ctx->readahead_kb * 1024 / ctx->fs->blocksize;
> + cd.next_ra_off = 0;
>
> if (ctx->progress)
> (void) (ctx->progress)(ctx, 2, 0, cd.max);
> @@ -146,7 +158,8 @@ void e2fsck_pass2(e2fsck_t ctx)
> if (fs->super->s_feature_compat & EXT2_FEATURE_COMPAT_DIR_INDEX)
> ext2fs_dblist_sort2(fs->dblist, special_dir_block_cmp);
>
> - cd.pctx.errcode = ext2fs_dblist_iterate2(fs->dblist, check_dir_block,
> + check_dir_func = cd.ra_entries ? check_dir_block2 : check_dir_block;
> + cd.pctx.errcode = ext2fs_dblist_iterate2(fs->dblist, check_dir_func,
> &cd);
> if (ctx->flags & E2F_FLAG_SIGNAL_MASK || ctx->flags & E2F_FLAG_RESTART)
> return;
> @@ -825,6 +838,29 @@ err:
> return retval;
> }
>
> +static int check_dir_block2(ext2_filsys fs,
> + struct ext2_db_entry2 *db,
> + void *priv_data)
> +{
> + int err;
> + struct check_dir_struct *cd = priv_data;
> +
> + if (cd->ra_entries && cd->list_offset >= cd->next_ra_off) {
> + err = e2fsck_readahead_dblist(fs,
> + E2FSCK_RA_DBLIST_IGNORE_BLOCKCNT,
> + fs->dblist,
> + cd->list_offset + cd->ra_entries / 8,
> + cd->ra_entries);
> + if (err)
> + cd->ra_entries = 0;
> + cd->next_ra_off = cd->list_offset + (cd->ra_entries * 7 / 8);
> + }
> +
> + err = check_dir_block(fs, db, priv_data);
> + cd->list_offset++;
> + return err;
> +}
> +
> static int check_dir_block(ext2_filsys fs,
> struct ext2_db_entry2 *db,
> void *priv_data)
> diff --git a/e2fsck/pass4.c b/e2fsck/pass4.c
> index 21d93f0..bc9a2c4 100644
> --- a/e2fsck/pass4.c
> +++ b/e2fsck/pass4.c
> @@ -106,6 +106,15 @@ void e2fsck_pass4(e2fsck_t ctx)
> #ifdef MTRACE
> mtrace_print("Pass 4");
> #endif
> + /*
> + * Since pass4 is mostly CPU bound, start readahead of bitmaps
> + * ahead of pass 5 if we haven't already loaded them.
> + */
> + if (ctx->readahead_kb &&
> + (fs->block_map == NULL || fs->inode_map == NULL))
> + e2fsck_readahead(fs, E2FSCK_READA_BBITMAP |
> + E2FSCK_READA_IBITMAP,
> + 0, fs->group_desc_count);
>
> clear_problem_context(&pctx);
>
> diff --git a/e2fsck/unix.c b/e2fsck/unix.c
> index 615d690..f3672c0 100644
> --- a/e2fsck/unix.c
> +++ b/e2fsck/unix.c
> @@ -650,6 +650,7 @@ static void parse_extended_opts(e2fsck_t ctx, const char *opts)
> char *buf, *token, *next, *p, *arg;
> int ea_ver;
> int extended_usage = 0;
> + unsigned long long reada_kb;
>
> buf = string_copy(ctx, opts, 0);
> for (token = buf; token && *token; token = next) {
> @@ -678,6 +679,15 @@ static void parse_extended_opts(e2fsck_t ctx, const char *opts)
> continue;
> }
> ctx->ext_attr_ver = ea_ver;
> + } else if (strcmp(token, "readahead_kb") == 0) {
> + reada_kb = strtoull(arg, &p, 0);
> + if (*p) {
> + fprintf(stderr, "%s",
> + _("Invalid readahead buffer size.\n"));
> + extended_usage++;
> + continue;
> + }
> + ctx->readahead_kb = reada_kb;
> } else if (strcmp(token, "fragcheck") == 0) {
> ctx->options |= E2F_OPT_FRAGCHECK;
> continue;
> @@ -717,6 +727,7 @@ static void parse_extended_opts(e2fsck_t ctx, const char *opts)
> fputs(("\tjournal_only\n"), stderr);
> fputs(("\tdiscard\n"), stderr);
> fputs(("\tnodiscard\n"), stderr);
> + fputs(("\treadahead_kb=<buffer size>\n"), stderr);
> fputc('\n', stderr);
> exit(1);
> }
> @@ -750,6 +761,7 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
> #ifdef CONFIG_JBD_DEBUG
> char *jbd_debug;
> #endif
> + unsigned long long phys_mem_kb;
>
> retval = e2fsck_allocate_context(&ctx);
> if (retval)
> @@ -777,6 +789,8 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
> else
> ctx->program_name = "e2fsck";
>
> + phys_mem_kb = get_memory_size() / 1024;
> + ctx->readahead_kb = ~0ULL;
> while ((c = getopt (argc, argv, "panyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDk")) != EOF)
> switch (c) {
> case 'C':
> @@ -961,6 +975,20 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
> if (c)
> verbose = 1;
>
> + if (ctx->readahead_kb == ~0ULL) {
> + profile_get_integer(ctx->profile, "options",
> + "readahead_mem_pct", 0, -1, &c);
> + if (c >= 0 && c <= 100)
> + ctx->readahead_kb = phys_mem_kb * c / 100;
> + profile_get_integer(ctx->profile, "options",
> + "readahead_kb", 0, -1, &c);
> + if (c >= 0)
> + ctx->readahead_kb = c;
> + if (ctx->readahead_kb != ~0ULL &&
> + ctx->readahead_kb > phys_mem_kb)
> + ctx->readahead_kb = phys_mem_kb;
> + }
> +
> /* Turn off discard in read-only mode */
> if ((ctx->options & E2F_OPT_NO) &&
> (ctx->options & E2F_OPT_DISCARD))
> diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
> index 34b9132..84e4e1f 100644
> --- a/lib/ext2fs/ext2fs.h
> +++ b/lib/ext2fs/ext2fs.h
> @@ -1418,6 +1418,7 @@ extern errcode_t ext2fs_get_next_inode_full(ext2_inode_scan scan,
> ext2_ino_t *ino,
> struct ext2_inode *inode,
> int bufsize);
> +#define EXT2_INODE_SCAN_DEFAULT_BUFFER_BLOCKS 8
> extern errcode_t ext2fs_open_inode_scan(ext2_filsys fs, int buffer_blocks,
> ext2_inode_scan *ret_scan);
> extern void ext2fs_close_inode_scan(ext2_inode_scan scan);
> diff --git a/lib/ext2fs/inode.c b/lib/ext2fs/inode.c
> index 4310b82..4b3e14e 100644
> --- a/lib/ext2fs/inode.c
> +++ b/lib/ext2fs/inode.c
> @@ -175,7 +175,8 @@ errcode_t ext2fs_open_inode_scan(ext2_filsys fs, int buffer_blocks,
> scan->bytes_left = 0;
> scan->current_group = 0;
> scan->groups_left = fs->group_desc_count - 1;
> - scan->inode_buffer_blocks = buffer_blocks ? buffer_blocks : 8;
> + scan->inode_buffer_blocks = buffer_blocks ? buffer_blocks :
> + EXT2_INODE_SCAN_DEFAULT_BUFFER_BLOCKS;
> scan->current_block = ext2fs_inode_table_loc(scan->fs,
> scan->current_group);
> scan->inodes_left = EXT2_INODES_PER_GROUP(scan->fs->super);
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 104+ messages in thread* Re: [PATCH 32/47] e2fsck: read-ahead metadata during passes 1, 2, and 4
2014-12-10 20:27 ` Darrick J. Wong
@ 2014-12-10 22:00 ` Theodore Ts'o
0 siblings, 0 replies; 104+ messages in thread
From: Theodore Ts'o @ 2014-12-10 22:00 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4
On Wed, Dec 10, 2014 at 12:27:56PM -0800, Darrick J. Wong wrote:
> FWIW I finally got UAS working in Linux; with that hardware and the default
> readahead size, I see about a 5% reduction in fsck runtime. The same dock
> plugged into a USB2 port (i.e. BOT) shows about a 2% increase in runtime.
Interesting; what UAS drive were you using? I've been thinking about
trying to get one to play with.
- Ted
^ permalink raw reply [flat|nested] 104+ messages in thread
* [PATCH 33/47] e2fsck: rebuild sparse extent trees/convert non-extent ext3 files
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (31 preceding siblings ...)
2014-11-07 21:54 ` [PATCH 32/47] e2fsck: read-ahead metadata during passes 1, 2, and 4 Darrick J. Wong
@ 2014-11-07 21:54 ` Darrick J. Wong
2014-12-04 23:20 ` Andreas Dilger
2014-12-11 22:05 ` [PATCH v2 " Darrick J. Wong
2014-11-07 21:54 ` [PATCH 34/47] tests: verify proper rebuilding of sparse extent trees and block map file conversion Darrick J. Wong
` (15 subsequent siblings)
48 siblings, 2 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:54 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Teach e2fsck to construct extent trees. This enables us to do either
of the following: compress a highly sparse extent tree into fewer ETB
blocks; or convert a ext3-style block mapped file to an extent file.
For files that are already extent based, this algorithm will only run
if pass1 determines either (1) that a whole level of extent tree will
fit into a higher level of the tree; (2) that the size of any level
can be reduced by at least one ETB block; or (3) the extent tree is
unnecessarily deep. It will not run at all if errors are found and
the user declines to fix the errors.
For block-mapped files, conversion only happens if the extent feature
is enabled and "-E bmap2extent" is passed to e2fsck. It will not run
at all if errors are left unfixed. After conversion, files larger
than 12 blocks should be defragmented to eliminate empty holes where a
block lives.
The extent tree constructor is pretty dumb -- it creates a list of
leaf extents (adjacent extents are collapsed), marks all indirect
blocks / ETB blocks free, installs a new extent tree root in the
inode, then loads the leaf extents into the tree.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
e2fsck/Makefile.in | 16 +
e2fsck/e2fsck.8.in | 3
e2fsck/e2fsck.c | 2
e2fsck/e2fsck.h | 11 +
e2fsck/extents.c | 352 ++++++++++++++++++++++++++++++++
e2fsck/pass1.c | 95 ++++++++-
e2fsck/problem.c | 43 ++++
e2fsck/problem.h | 28 +++
e2fsck/super.c | 7 +
e2fsck/unix.c | 4
tests/f_extent_bad_node/expect.1 | 9 +
tests/f_extent_bad_node/expect.2 | 2
tests/f_extent_int_bad_magic/expect.1 | 3
tests/f_extent_leaf_bad_magic/expect.1 | 3
tests/f_extent_oobounds/expect.1 | 9 +
tests/f_extent_oobounds/expect.2 | 2
tests/f_extents/expect.1 | 5
17 files changed, 577 insertions(+), 17 deletions(-)
create mode 100644 e2fsck/extents.c
diff --git a/e2fsck/Makefile.in b/e2fsck/Makefile.in
index e40e51b..a4413d9 100644
--- a/e2fsck/Makefile.in
+++ b/e2fsck/Makefile.in
@@ -62,7 +62,8 @@ OBJS= dict.o unix.o e2fsck.o super.o pass1.o pass1b.o pass2.o \
pass3.o pass4.o pass5.o journal.o badblocks.o util.o dirinfo.o \
dx_dirinfo.o ehandler.o problem.o message.o quota.o recovery.o \
region.o revoke.o ea_refcount.o rehash.o profile.o prof_err.o \
- logfile.o sigcatcher.o $(MTRACE_OBJ) plausible.o readahead.o
+ logfile.o sigcatcher.o $(MTRACE_OBJ) plausible.o readahead.o \
+ extents.o
PROFILED_OBJS= profiled/dict.o profiled/unix.o profiled/e2fsck.o \
profiled/super.o profiled/pass1.o profiled/pass1b.o \
@@ -74,7 +75,7 @@ PROFILED_OBJS= profiled/dict.o profiled/unix.o profiled/e2fsck.o \
profiled/ea_refcount.o profiled/rehash.o profiled/profile.o \
profiled/prof_err.o profiled/logfile.o \
profiled/sigcatcher.o profiled/plausible.o \
- profiled/sigcatcher.o profiled/readahead.o
+ profiled/sigcatcher.o profiled/readahead.o profiled/extents.o
SRCS= $(srcdir)/e2fsck.c \
$(srcdir)/dict.c \
@@ -106,6 +107,7 @@ SRCS= $(srcdir)/e2fsck.c \
prof_err.c \
$(srcdir)/quota.c \
$(srcdir)/../misc/plausible.c \
+ $(srcdir)/extents.c \
$(MTRACE_SRC)
all:: profiled $(PROGS) e2fsck $(MANPAGES) $(FMANPAGES)
@@ -308,6 +310,16 @@ pass1.o: $(srcdir)/pass1.c $(top_builddir)/lib/config.h \
$(srcdir)/profile.h prof_err.h $(top_srcdir)/lib/quota/quotaio.h \
$(top_srcdir)/lib/quota/dqblk_v2.h $(top_srcdir)/lib/quota/quotaio_tree.h \
$(top_srcdir)/lib/../e2fsck/dict.h $(srcdir)/problem.h
+extents.o: $(srcdir)/extents.c $(top_builddir)/lib/config.h \
+ $(top_builddir)/lib/dirpaths.h $(top_srcdir)/lib/et/com_err.h \
+ $(srcdir)/e2fsck.h $(top_srcdir)/lib/ext2fs/ext2_fs.h \
+ $(top_builddir)/lib/ext2fs/ext2_types.h $(top_srcdir)/lib/ext2fs/ext2fs.h \
+ $(top_srcdir)/lib/ext2fs/ext3_extents.h $(top_srcdir)/lib/ext2fs/ext2_io.h \
+ $(top_builddir)/lib/ext2fs/ext2_err.h \
+ $(top_srcdir)/lib/ext2fs/ext2_ext_attr.h $(top_srcdir)/lib/ext2fs/bitops.h \
+ $(srcdir)/profile.h prof_err.h $(top_srcdir)/lib/quota/quotaio.h \
+ $(top_srcdir)/lib/quota/dqblk_v2.h $(top_srcdir)/lib/quota/quotaio_tree.h \
+ $(top_srcdir)/lib/../e2fsck/dict.h $(srcdir)/problem.h $(srcdir)/dict.h
pass1b.o: $(srcdir)/pass1b.c $(top_builddir)/lib/config.h \
$(top_builddir)/lib/dirpaths.h $(top_srcdir)/lib/et/com_err.h \
$(srcdir)/e2fsck.h $(top_srcdir)/lib/ext2fs/ext2_fs.h \
diff --git a/e2fsck/e2fsck.8.in b/e2fsck/e2fsck.8.in
index 84ae50f..0c2725e 100644
--- a/e2fsck/e2fsck.8.in
+++ b/e2fsck/e2fsck.8.in
@@ -214,6 +214,9 @@ e2fsck runtime. By default, this is set to the size of a block group's inode
table (typically 2MiB on a regular ext4 filesystem); if this amount is more
than 1/100 of total physical memory, readahead is disabled. Set this to zero
to disable readahead entirely.
+.TP
+.BI bmap2extent
+Convert block-mapped files to extent-mapped files.
.RE
.TP
.B \-f
diff --git a/e2fsck/e2fsck.c b/e2fsck/e2fsck.c
index fcda7d7..83506cb 100644
--- a/e2fsck/e2fsck.c
+++ b/e2fsck/e2fsck.c
@@ -204,7 +204,7 @@ void e2fsck_free_context(e2fsck_t ctx)
typedef void (*pass_t)(e2fsck_t ctx);
static pass_t e2fsck_passes[] = {
- e2fsck_pass1, e2fsck_pass2, e2fsck_pass3, e2fsck_pass4,
+ e2fsck_pass1, e2fsck_pass1e, e2fsck_pass2, e2fsck_pass3, e2fsck_pass4,
e2fsck_pass5, 0 };
#define E2F_FLAG_RUN_RETURN (E2F_FLAG_SIGNAL_MASK|E2F_FLAG_RESTART)
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index e359515..66d71ee 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -167,6 +167,7 @@ struct resource_track {
#define E2F_OPT_FRAGCHECK 0x0800
#define E2F_OPT_JOURNAL_ONLY 0x1000 /* only replay the journal */
#define E2F_OPT_DISCARD 0x2000
+#define E2F_OPT_CONVERT_BMAP 0x4000 /* convert blockmap to extent */
/*
* E2fsck flags
@@ -381,6 +382,11 @@ struct e2fsck_struct {
/* How much are we allowed to readahead? */
unsigned long long readahead_kb;
+
+ /*
+ * Inodes to rebuild extent trees
+ */
+ ext2fs_inode_bitmap inodes_to_rebuild;
};
/* Used by the region allocation code */
@@ -456,6 +462,11 @@ extern blk64_t ea_refcount_intr_next(ext2_refcount_t refcount, int *ret);
extern const char *ehandler_operation(const char *op);
extern void ehandler_init(io_channel channel);
+/* extents.c */
+void e2fsck_rebuild_extents_later(e2fsck_t ctx, ext2_ino_t ino);
+int e2fsck_ino_will_be_rebuilt(e2fsck_t ctx, ext2_ino_t ino);
+void e2fsck_pass1e(e2fsck_t ctx);
+
/* journal.c */
extern errcode_t e2fsck_check_ext3_journal(e2fsck_t ctx);
extern errcode_t e2fsck_run_ext3_journal(e2fsck_t ctx);
diff --git a/e2fsck/extents.c b/e2fsck/extents.c
new file mode 100644
index 0000000..91d36d2
--- /dev/null
+++ b/e2fsck/extents.c
@@ -0,0 +1,352 @@
+/*
+ * extents.c --- rebuild extent tree
+ *
+ * Copyright (C) 2014 Oracle.
+ *
+ * %Begin-Header%
+ * This file may be redistributed under the terms of the GNU Public
+ * License.
+ * %End-Header%
+ */
+
+#include "config.h"
+#include <string.h>
+#include <ctype.h>
+#include <errno.h>
+#include "e2fsck.h"
+#include "problem.h"
+
+#undef DEBUG
+#undef DEBUG_SUMMARY
+#undef DEBUG_FREE
+
+#define NUM_EXTENTS 341 /* about one ETB' worth of extents */
+
+/* Schedule an inode to have its extent tree rebuilt during pass 1E. */
+void e2fsck_rebuild_extents_later(e2fsck_t ctx, ext2_ino_t ino)
+{
+ if (!EXT2_HAS_INCOMPAT_FEATURE(ctx->fs->super,
+ EXT3_FEATURE_INCOMPAT_EXTENTS) ||
+ (ctx->options & E2F_OPT_NO) ||
+ (ino != EXT2_ROOT_INO && ino < ctx->fs->super->s_first_ino))
+ return;
+ if (!ctx->inodes_to_rebuild)
+ e2fsck_allocate_inode_bitmap(ctx->fs,
+ _("extent rebuild inode map"),
+ EXT2FS_BMAP64_AUTODIR,
+ "inodes_to_rebuild",
+ &ctx->inodes_to_rebuild);
+ if (ctx->inodes_to_rebuild)
+ ext2fs_mark_inode_bitmap2(ctx->inodes_to_rebuild, ino);
+}
+
+/* Ask if an inode will have its extents rebuilt during pass 1E. */
+int e2fsck_ino_will_be_rebuilt(e2fsck_t ctx, ext2_ino_t ino)
+{
+ if (!ctx->inodes_to_rebuild)
+ return 0;
+ return ext2fs_test_inode_bitmap2(ctx->inodes_to_rebuild, ino);
+}
+
+struct extent_list {
+ blk64_t blocks_freed;
+ struct ext2fs_extent *extents;
+ unsigned int count;
+ unsigned int size;
+ unsigned int ext_read;
+ errcode_t retval;
+ ext2_ino_t ino;
+};
+
+static errcode_t load_extents(e2fsck_t ctx, struct extent_list *list)
+{
+ ext2_filsys fs = ctx->fs;
+ ext2_extent_handle_t handle;
+ struct ext2fs_extent extent;
+ errcode_t retval;
+
+ retval = ext2fs_extent_open(fs, list->ino, &handle);
+ if (retval)
+ return retval;
+
+ retval = ext2fs_extent_get(handle, EXT2_EXTENT_ROOT, &extent);
+ if (retval)
+ goto out;
+
+ do {
+ if (extent.e_flags & EXT2_EXTENT_FLAGS_SECOND_VISIT)
+ goto next;
+
+ /* Internal node; free it and we'll re-allocate it later */
+ if (!(extent.e_flags & EXT2_EXTENT_FLAGS_LEAF)) {
+#if defined(DEBUG) || defined(DEBUG_FREE)
+ printf("ino=%d free=%llu bf=%llu\n", list->ino,
+ extent.e_pblk, list->blocks_freed + 1);
+#endif
+ list->blocks_freed++;
+ ext2fs_block_alloc_stats2(fs, extent.e_pblk, -1);
+ goto next;
+ }
+
+ list->ext_read++;
+ /* Can we attach it to the previous extent? */
+ if (list->count) {
+ struct ext2fs_extent *last = list->extents +
+ list->count - 1;
+ blk64_t end = last->e_len + extent.e_len;
+
+ if (last->e_pblk + last->e_len == extent.e_pblk &&
+ last->e_lblk + last->e_len == extent.e_lblk &&
+ (last->e_flags & EXT2_EXTENT_FLAGS_UNINIT) ==
+ (extent.e_flags & EXT2_EXTENT_FLAGS_UNINIT) &&
+ end < (1ULL << 32)) {
+ last->e_len += extent.e_len;
+#ifdef DEBUG
+ printf("R: ino=%d len=%u\n", list->ino,
+ last->e_len);
+#endif
+ goto next;
+ }
+ }
+
+ /* Do we need to expand? */
+ if (list->count == list->size) {
+ unsigned int new_size = (list->size + NUM_EXTENTS) *
+ sizeof(struct ext2fs_extent);
+ retval = ext2fs_resize_mem(0, new_size, &list->extents);
+ if (retval)
+ goto out;
+ list->size += NUM_EXTENTS;
+ }
+
+ /* Add a new extent */
+ memcpy(list->extents + list->count, &extent, sizeof(extent));
+#ifdef DEBUG
+ printf("R: ino=%d pblk=%llu lblk=%llu len=%u\n", list->ino,
+ extent.e_pblk, extent.e_lblk, extent.e_len);
+#endif
+ list->count++;
+next:
+ retval = ext2fs_extent_get(handle, EXT2_EXTENT_NEXT, &extent);
+ } while (retval == 0);
+
+out:
+ /* Ok if we run off the end */
+ if (retval == EXT2_ET_EXTENT_NO_NEXT)
+ retval = 0;
+ ext2fs_extent_free(handle);
+ return retval;
+}
+
+static int find_blocks(ext2_filsys fs, blk64_t *blocknr, e2_blkcnt_t blockcnt,
+ blk64_t ref_blk, int ref_offset, void *priv_data)
+{
+ struct extent_list *list = priv_data;
+
+ /* Internal node? */
+ if (blockcnt < 0) {
+#if defined(DEBUG) || defined(DEBUG_FREE)
+ printf("ino=%d free=%llu bf=%llu\n", list->ino, *blocknr,
+ list->blocks_freed + 1);
+#endif
+ list->blocks_freed++;
+ ext2fs_block_alloc_stats2(fs, *blocknr, -1);
+ return 0;
+ }
+
+ /* Can we attach it to the previous extent? */
+ if (list->count) {
+ struct ext2fs_extent *last = list->extents +
+ list->count - 1;
+ blk64_t end = last->e_len + 1;
+
+ if (last->e_pblk + last->e_len == *blocknr &&
+ end < (1ULL << 32)) {
+ last->e_len++;
+#ifdef DEBUG
+ printf("R: ino=%d len=%u\n", list->ino, last->e_len);
+#endif
+ return 0;
+ }
+ }
+
+ /* Do we need to expand? */
+ if (list->count == list->size) {
+ unsigned int new_size = (list->size + NUM_EXTENTS) *
+ sizeof(struct ext2fs_extent);
+ list->retval = ext2fs_resize_mem(0, new_size, &list->extents);
+ if (list->retval)
+ return BLOCK_ABORT;
+ list->size += NUM_EXTENTS;
+ }
+
+ /* Add a new extent */
+ list->extents[list->count].e_pblk = *blocknr;
+ list->extents[list->count].e_lblk = blockcnt;
+ list->extents[list->count].e_len = 1;
+ list->extents[list->count].e_flags = 0;
+#ifdef DEBUG
+ printf("R: ino=%d pblk=%llu lblk=%llu len=%u\n", list->ino, *blocknr,
+ blockcnt, 1);
+#endif
+ list->count++;
+
+ return 0;
+}
+
+static errcode_t rebuild_extent_tree(e2fsck_t ctx, struct extent_list *list,
+ ext2_ino_t ino)
+{
+ struct ext2_inode inode;
+ errcode_t retval;
+ ext2_extent_handle_t handle;
+ unsigned int i, ext_written;
+ struct ext2fs_extent *ex, extent;
+
+ list->count = 0;
+ list->blocks_freed = 0;
+ list->ino = ino;
+ list->ext_read = 0;
+ e2fsck_read_inode(ctx, ino, &inode, "rebuild_extents");
+
+ /* Collect lblk->pblk mappings */
+ if (inode.i_flags & EXT4_EXTENTS_FL) {
+ retval = load_extents(ctx, list);
+ goto extents_loaded;
+ }
+
+ retval = ext2fs_block_iterate3(ctx->fs, ino, BLOCK_FLAG_READ_ONLY, 0,
+ find_blocks, list);
+ if (retval)
+ goto err;
+ if (list->retval) {
+ retval = list->retval;
+ goto err;
+ }
+
+extents_loaded:
+ /* Reset extent tree */
+ inode.i_flags &= ~EXT4_EXTENTS_FL;
+ memset(inode.i_block, 0, sizeof(inode.i_block));
+
+ /* Make a note of freed blocks */
+ retval = ext2fs_iblk_sub_blocks(ctx->fs, &inode, list->blocks_freed);
+ if (retval)
+ goto err;
+
+ /* Now stuff extents into the file */
+ retval = ext2fs_extent_open2(ctx->fs, ino, &inode, &handle);
+ if (retval)
+ goto err;
+
+ ext_written = 0;
+ for (i = 0, ex = list->extents; i < list->count; i++, ex++) {
+ memcpy(&extent, ex, sizeof(struct ext2fs_extent));
+ extent.e_flags &= EXT2_EXTENT_FLAGS_UNINIT;
+ if (extent.e_flags & EXT2_EXTENT_FLAGS_UNINIT) {
+ if (extent.e_len > EXT_UNINIT_MAX_LEN) {
+ extent.e_len = EXT_UNINIT_MAX_LEN;
+ ex->e_pblk += EXT_UNINIT_MAX_LEN;
+ ex->e_lblk += EXT_UNINIT_MAX_LEN;
+ ex->e_len -= EXT_UNINIT_MAX_LEN;
+ ex--;
+ i--;
+ }
+ } else {
+ if (extent.e_len > EXT_INIT_MAX_LEN) {
+ extent.e_len = EXT_INIT_MAX_LEN;
+ ex->e_pblk += EXT_INIT_MAX_LEN;
+ ex->e_lblk += EXT_INIT_MAX_LEN;
+ ex->e_len -= EXT_INIT_MAX_LEN;
+ ex--;
+ i--;
+ }
+ }
+
+#ifdef DEBUG
+ printf("W: ino=%d pblk=%llu lblk=%llu len=%u\n", ino,
+ extent.e_pblk, extent.e_lblk, extent.e_len);
+#endif
+ retval = ext2fs_extent_insert(handle, EXT2_EXTENT_INSERT_AFTER,
+ &extent);
+ if (retval)
+ goto err2;
+ retval = ext2fs_extent_fix_parents(handle);
+ if (retval)
+ goto err2;
+ ext_written++;
+ }
+
+#if defined(DEBUG) || defined(DEBUG_SUMMARY)
+ printf("rebuild: ino=%d extents=%d->%d\n", ino, list->ext_read,
+ ext_written);
+#endif
+ e2fsck_write_inode(ctx, ino, &inode, "rebuild_extents");
+
+err2:
+ ext2fs_extent_free(handle);
+err:
+ return retval;
+}
+
+void e2fsck_pass1e(e2fsck_t ctx)
+{
+ struct problem_context pctx;
+#ifdef RESOURCE_TRACK
+ struct resource_track rtrack;
+#endif
+ struct extent_list list;
+ int first = 1;
+ ext2_ino_t ino = 0;
+ errcode_t retval;
+
+ if (!EXT2_HAS_INCOMPAT_FEATURE(ctx->fs->super,
+ EXT3_FEATURE_INCOMPAT_EXTENTS) ||
+ !ext2fs_test_valid(ctx->fs) ||
+ ctx->invalid_bitmaps) {
+ if (ctx->inodes_to_rebuild)
+ ext2fs_free_inode_bitmap(ctx->inodes_to_rebuild);
+ ctx->inodes_to_rebuild = NULL;
+ }
+
+ if (ctx->inodes_to_rebuild == NULL)
+ return;
+
+ init_resource_track(&rtrack, ctx->fs->io);
+ clear_problem_context(&pctx);
+ e2fsck_read_bitmaps(ctx);
+
+ memset(&list, 0, sizeof(list));
+ retval = ext2fs_get_mem(sizeof(struct ext2fs_extent) * NUM_EXTENTS,
+ &list.extents);
+ list.size = NUM_EXTENTS;
+ while (1) {
+ retval = ext2fs_find_first_set_inode_bitmap2(
+ ctx->inodes_to_rebuild, ino + 1,
+ ctx->fs->super->s_inodes_count, &ino);
+ if (retval)
+ break;
+ pctx.ino = ino;
+ if (first) {
+ fix_problem(ctx, PR_1E_PASS_HEADER, &pctx);
+ first = 0;
+ }
+ pctx.errcode = rebuild_extent_tree(ctx, &list, ino);
+ if (pctx.errcode) {
+ end_problem_latch(ctx, PR_LATCH_OPTIMIZE_EXT);
+ fix_problem(ctx, PR_1E_OPTIMIZE_EXT_ERR, &pctx);
+ }
+ if (ctx->progress && !ctx->progress_fd)
+ e2fsck_simple_progress(ctx, "Rebuilding extents",
+ 100.0 * (float) ino /
+ (float) ctx->fs->super->s_inodes_count,
+ ino);
+ }
+ end_problem_latch(ctx, PR_LATCH_OPTIMIZE_EXT);
+
+ ext2fs_free_inode_bitmap(ctx->inodes_to_rebuild);
+ ctx->inodes_to_rebuild = NULL;
+ ext2fs_free_mem(&list.extents);
+
+ print_resource_track(ctx, "Pass 1E", &rtrack, ctx->fs->io);
+}
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index a963849..10008d9 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -56,6 +56,8 @@
#define _INLINE_ inline
#endif
+#undef DEBUG
+
static int process_block(ext2_filsys fs, blk64_t *blocknr,
e2_blkcnt_t blockcnt, blk64_t ref_blk,
int ref_offset, void *priv_data);
@@ -77,11 +79,16 @@ static void adjust_extattr_refcount(e2fsck_t ctx, ext2_refcount_t refcount,
char *block_buf, int adjust_sign);
/* static char *describe_illegal_block(ext2_filsys fs, blk64_t block); */
+struct extent_info {
+ unsigned int num_extents;
+ unsigned int max_extents;
+};
+
struct process_block_struct {
ext2_ino_t ino;
unsigned is_dir:1, is_reg:1, clear:1, suppress:1,
fragmented:1, compressed:1, bbcheck:1,
- inode_modified:1;
+ inode_modified:1, extent_rebuild:1;
blk64_t num_blocks;
blk64_t max_blocks;
e2_blkcnt_t last_block;
@@ -95,6 +102,7 @@ struct process_block_struct {
e2fsck_t ctx;
blk64_t bad_ref;
region_t region;
+ struct extent_info ext_info[MAX_EXTENT_DEPTH_COUNT];
};
struct process_inode_block {
@@ -2402,6 +2410,53 @@ static int has_unaligned_cluster_map(e2fsck_t ctx,
return 0;
}
+static void should_rebuild_extents(e2fsck_t ctx,
+ struct problem_context *pctx,
+ struct process_block_struct *pb,
+ struct ext2_extent_info *info)
+{
+ struct extent_info *ei;
+ int i, j;
+ unsigned int extents_per_block;
+
+ if (pb->extent_rebuild)
+ goto rebuild;
+
+ extents_per_block = (ctx->fs->blocksize -
+ sizeof(struct ext3_extent_header)) /
+ sizeof(struct ext3_extent);
+ /*
+ * If we can consolidate a level or shorten the tree, schedule the
+ * extent tree to be rebuilt.
+ */
+ for (i = 0, ei = pb->ext_info; i < info->max_depth + 1; i++, ei++) {
+ if (ei->max_extents - ei->num_extents > extents_per_block) {
+#ifdef DEBUG
+ printf("rebuild extents, ino=%d level=%d slack=%d epb=%d\n",
+ pb->ino, i,
+ ei->max_extents - ei->num_extents,
+ extents_per_block);
+#endif
+ goto rebuild;
+ }
+ for (j = 0; j < i; j++) {
+ if (ei->num_extents < pb->ext_info[j].max_extents) {
+#ifdef DEBUG
+ printf("rebuild extents, ino=%d level=%d num=%d level=%d\n",
+ pb->ino, i, ei->num_extents, j);
+#endif
+ goto rebuild;
+ }
+ }
+ }
+ return;
+
+rebuild:
+ if (pb->extent_rebuild ||
+ fix_problem(ctx, PR_1E_CAN_COMPRESS_EXTENT_TREE, pctx))
+ e2fsck_rebuild_extents_later(ctx, pb->ino);
+}
+
static void scan_extent_node(e2fsck_t ctx, struct problem_context *pctx,
struct process_block_struct *pb,
blk64_t start_block, blk64_t end_block,
@@ -2424,6 +2479,10 @@ static void scan_extent_node(e2fsck_t ctx, struct problem_context *pctx,
pctx->errcode = ext2fs_extent_get_info(ehandle, &info);
if (pctx->errcode)
return;
+ if (!pb->extent_rebuild) {
+ pb->ext_info[info.curr_level].num_extents += info.num_entries;
+ pb->ext_info[info.curr_level].max_extents += info.max_entries;
+ }
pctx->errcode = ext2fs_extent_get(ehandle, EXT2_EXTENT_FIRST_SIB,
&extent);
@@ -2760,17 +2819,31 @@ static void check_blocks_extents(e2fsck_t ctx, struct problem_context *pctx,
retval = ext2fs_extent_get_info(ehandle, &info);
if (retval == 0) {
- if (info.max_depth >= MAX_EXTENT_DEPTH_COUNT)
- info.max_depth = MAX_EXTENT_DEPTH_COUNT-1;
- ctx->extent_depth_count[info.max_depth]++;
+ int max_depth = info.max_depth;
+
+ if (max_depth >= MAX_EXTENT_DEPTH_COUNT)
+ max_depth = MAX_EXTENT_DEPTH_COUNT-1;
+ ctx->extent_depth_count[max_depth]++;
}
+ /* Check maximum extent depth */
+ pctx->blk = info.max_depth;
+ pctx->blk2 = ext2fs_max_extent_depth(ehandle);
+ if (pctx->blk2 < pctx->blk &&
+ fix_problem(ctx, PR_1_EXTENT_BAD_MAX_DEPTH, pctx))
+ pb->extent_rebuild = 1;
+
+ /* Can we collect extent tree level stats? */
+ pctx->blk = MAX_EXTENT_DEPTH_COUNT;
+ if (pctx->blk2 > pctx->blk)
+ fix_problem(ctx, PR_1E_MAX_EXTENT_TREE_DEPTH, pctx);
+ memset(pb->ext_info, 0, sizeof(pb->ext_info));
+
pb->region = region_create(0, info.max_lblk);
if (!pb->region) {
- ext2fs_extent_free(ehandle);
fix_problem(ctx, PR_1_EXTENT_ALLOC_REGION_ABORT, pctx);
ctx->flags |= E2F_FLAG_ABORT;
- return;
+ goto out;
}
eof_lblk = ((EXT2_I_SIZE(inode) + fs->blocksize - 1) >>
@@ -2786,7 +2859,9 @@ static void check_blocks_extents(e2fsck_t ctx, struct problem_context *pctx,
}
region_free(pb->region);
pb->region = NULL;
+out:
ext2fs_extent_free(ehandle);
+ should_rebuild_extents(ctx, pctx, pb, &info);
}
/*
@@ -2846,6 +2921,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
pb.ctx = ctx;
pb.inode_modified = 0;
pb.bad_ref = 0;
+ pb.extent_rebuild = 0;
pctx->ino = ino;
pctx->errcode = 0;
@@ -2909,6 +2985,13 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
"check_blocks");
fs->flags = (flags & EXT2_FLAG_IGNORE_CSUM_ERRORS) |
(fs->flags & ~EXT2_FLAG_IGNORE_CSUM_ERRORS);
+
+ if (ctx->options & E2F_OPT_CONVERT_BMAP) {
+#ifdef DEBUG
+ printf("bmap rebuild ino=%d\n", ino);
+#endif
+ e2fsck_rebuild_extents_later(ctx, ino);
+ }
}
}
end_problem_latch(ctx, PR_LATCH_BLOCK);
diff --git a/e2fsck/problem.c b/e2fsck/problem.c
index a4da64b..75e0305 100644
--- a/e2fsck/problem.c
+++ b/e2fsck/problem.c
@@ -1101,6 +1101,11 @@ static struct e2fsck_problem problem_table[] = {
N_("@i %i has a duplicate @x mapping\n\t(logical @b %c, @n physical @b %b, len %N)\n"),
PROMPT_CLEAR, 0 },
+ /* Inode extent tree could be more shallow */
+ { PR_1_EXTENT_BAD_MAX_DEPTH,
+ N_("@i %i @x tree could be more shallow (%b; could be <= %c)\n"),
+ PROMPT_FIX, PR_NO_OK | PR_PREEN_NO | PR_PREEN_OK },
+
/* Pass 1b errors */
/* Pass 1B: Rescan for duplicate/bad blocks */
@@ -1198,6 +1203,43 @@ static struct e2fsck_problem problem_table[] = {
{ PR_1D_CLONE_ERROR,
N_("Couldn't clone file: %m\n"), PROMPT_NONE, 0 },
+ /* Pass 1E Extent tree Optimization */
+
+ /* Pass 1E: Optimizing extent trees */
+ { PR_1E_PASS_HEADER,
+ N_("Pass 1E: Optimizing @x trees\n"),
+ PROMPT_NONE, PR_PREEN_NOMSG },
+
+ /* Failed to optimize extent tree */
+ { PR_1E_OPTIMIZE_EXT_ERR,
+ N_("Failed to optimize @x tree %p (%i): %m\n"),
+ PROMPT_NONE, 0 },
+
+ /* Rebuilding extent trees */
+ { PR_1E_OPTIMIZE_EXT_HEADER,
+ N_("Optimizing @x trees: "),
+ PROMPT_NONE, PR_MSG_ONLY },
+
+ /* Rebuilding extent tree %d */
+ { PR_1E_OPTIMIZE_EXT,
+ " %i",
+ PROMPT_NONE, PR_LATCH_OPTIMIZE_EXT | PR_PREEN_NOHDR},
+
+ /* Rebuilding extent tree end */
+ { PR_1E_OPTIMIZE_EXT_END,
+ "\n",
+ PROMPT_NONE, PR_PREEN_NOHDR },
+
+ /* Internal error: extent tree depth too large */
+ { PR_1E_MAX_EXTENT_TREE_DEPTH,
+ N_("Internal error: max extent tree depth too large (%b; expected=%c).\n"),
+ PROMPT_NONE, PR_FATAL },
+
+ /* Inode extent tree could be more compact */
+ { PR_1E_CAN_COMPRESS_EXTENT_TREE,
+ N_("@i %i @x tree could be more compact. "),
+ PROMPT_FIX, PR_NO_OK | PR_PREEN_NO | PR_PREEN_OK },
+
/* Pass 2 errors */
/* Pass 2: Checking directory structure */
@@ -1946,6 +1988,7 @@ static struct latch_descr pr_latch_info[] = {
{ PR_LATCH_TOOBIG, PR_1_INODE_TOOBIG, 0 },
{ PR_LATCH_OPTIMIZE_DIR, PR_3A_OPTIMIZE_DIR_HEADER, PR_3A_OPTIMIZE_DIR_END },
{ PR_LATCH_BG_CHECKSUM, PR_0_GDT_CSUM_LATCH, 0 },
+ { PR_LATCH_OPTIMIZE_EXT, PR_1E_OPTIMIZE_EXT_HEADER, PR_1E_OPTIMIZE_EXT_END },
{ -1, 0, 0 },
};
diff --git a/e2fsck/problem.h b/e2fsck/problem.h
index 3c28166..d3dcc9e 100644
--- a/e2fsck/problem.h
+++ b/e2fsck/problem.h
@@ -40,6 +40,7 @@ struct problem_context {
#define PR_LATCH_TOOBIG 0x0080 /* Latch for file to big errors */
#define PR_LATCH_OPTIMIZE_DIR 0x0090 /* Latch for optimize directories */
#define PR_LATCH_BG_CHECKSUM 0x00A0 /* Latch for block group checksums */
+#define PR_LATCH_OPTIMIZE_EXT 0x00B0 /* Latch for rebuild extents */
#define PR_LATCH(x) ((((x) & PR_LATCH_MASK) >> 4) - 1)
@@ -641,6 +642,9 @@ struct problem_context {
/* leaf extent collision */
#define PR_1_EXTENT_COLLISION 0x01007D
+/* extent tree max depth too big */
+#define PR_1_EXTENT_BAD_MAX_DEPTH 0x01007E
+
/*
* Pass 1b errors
*/
@@ -704,6 +708,30 @@ struct problem_context {
#define PR_1D_CLONE_ERROR 0x013008
/*
+ * Pass 1e --- rebuilding extent trees
+ */
+/* Pass 1e: Rebuilding extent trees */
+#define PR_1E_PASS_HEADER 0x014000
+
+/* Error rehash directory */
+#define PR_1E_OPTIMIZE_EXT_ERR 0x014001
+
+/* Rebuilding extent trees */
+#define PR_1E_OPTIMIZE_EXT_HEADER 0x014002
+
+/* Rebuilding extent %d */
+#define PR_1E_OPTIMIZE_EXT 0x014003
+
+/* Rebuilding extent tree end */
+#define PR_1E_OPTIMIZE_EXT_END 0x014004
+
+/* Internal error: extent tree depth too large */
+#define PR_1E_MAX_EXTENT_TREE_DEPTH 0x014005
+
+/* Inode extent tree could be more compact */
+#define PR_1E_CAN_COMPRESS_EXTENT_TREE 0x014006
+
+/*
* Pass 2 errors
*/
diff --git a/e2fsck/super.c b/e2fsck/super.c
index 1e7e749..e64262a 100644
--- a/e2fsck/super.c
+++ b/e2fsck/super.c
@@ -606,6 +606,13 @@ void check_super_block(e2fsck_t ctx)
ext2fs_mark_super_dirty(fs);
}
+ /* Did user ask us to convert files to extents? */
+ if (ctx->options & E2F_OPT_CONVERT_BMAP) {
+ fs->super->s_feature_incompat |=
+ EXT3_FEATURE_INCOMPAT_EXTENTS;
+ ext2fs_mark_super_dirty(fs);
+ }
+
if ((fs->super->s_feature_incompat & EXT2_FEATURE_INCOMPAT_META_BG) &&
(fs->super->s_first_meta_bg > fs->desc_blocks)) {
pctx.group = fs->desc_blocks;
diff --git a/e2fsck/unix.c b/e2fsck/unix.c
index f3672c0..fe5127a 100644
--- a/e2fsck/unix.c
+++ b/e2fsck/unix.c
@@ -709,6 +709,9 @@ static void parse_extended_opts(e2fsck_t ctx, const char *opts)
else
ctx->log_fn = string_copy(ctx, arg, 0);
continue;
+ } else if (strcmp(token, "bmap2extent") == 0) {
+ ctx->options |= E2F_OPT_CONVERT_BMAP;
+ continue;
} else {
fprintf(stderr, _("Unknown extended option: %s\n"),
token);
@@ -728,6 +731,7 @@ static void parse_extended_opts(e2fsck_t ctx, const char *opts)
fputs(("\tdiscard\n"), stderr);
fputs(("\tnodiscard\n"), stderr);
fputs(("\treadahead_kb=<buffer size>\n"), stderr);
+ fputs(("\tbmap2extent\n"), stderr);
fputc('\n', stderr);
exit(1);
}
diff --git a/tests/f_extent_bad_node/expect.1 b/tests/f_extent_bad_node/expect.1
index 0c0bc28..c13ad39 100644
--- a/tests/f_extent_bad_node/expect.1
+++ b/tests/f_extent_bad_node/expect.1
@@ -2,8 +2,11 @@ Pass 1: Checking inodes, blocks, and sizes
Inode 12 has an invalid extent node (blk 22, lblk 0)
Clear? yes
+Inode 12 extent tree could be more compact. Fix? yes
+
Inode 12, i_blocks is 16, should be 8. Fix? yes
+Pass 1E: Optimizing extent trees
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
@@ -11,13 +14,13 @@ Pass 5: Checking group summary information
Block bitmap differences: -(21--23) -25
Fix? yes
-Free blocks count wrong for group #0 (71, counted=75).
+Free blocks count wrong for group #0 (73, counted=77).
Fix? yes
-Free blocks count wrong (71, counted=75).
+Free blocks count wrong (73, counted=77).
Fix? yes
test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
-test_filesys: 12/16 files (0.0% non-contiguous), 25/100 blocks
+test_filesys: 12/16 files (0.0% non-contiguous), 23/100 blocks
Exit status is 1
diff --git a/tests/f_extent_bad_node/expect.2 b/tests/f_extent_bad_node/expect.2
index 568c792..b78b193 100644
--- a/tests/f_extent_bad_node/expect.2
+++ b/tests/f_extent_bad_node/expect.2
@@ -3,5 +3,5 @@ Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
-test_filesys: 12/16 files (0.0% non-contiguous), 25/100 blocks
+test_filesys: 12/16 files (0.0% non-contiguous), 23/100 blocks
Exit status is 0
diff --git a/tests/f_extent_int_bad_magic/expect.1 b/tests/f_extent_int_bad_magic/expect.1
index 0e82e2b..0bd163f 100644
--- a/tests/f_extent_int_bad_magic/expect.1
+++ b/tests/f_extent_int_bad_magic/expect.1
@@ -2,8 +2,11 @@ Pass 1: Checking inodes, blocks, and sizes
Inode 12 has an invalid extent node (blk 1295, lblk 0)
Clear? yes
+Inode 12 extent tree could be more compact. Fix? yes
+
Inode 12, i_blocks is 712, should be 0. Fix? yes
+Pass 1E: Optimizing extent trees
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
diff --git a/tests/f_extent_leaf_bad_magic/expect.1 b/tests/f_extent_leaf_bad_magic/expect.1
index 7b6dbf1..c31a309 100644
--- a/tests/f_extent_leaf_bad_magic/expect.1
+++ b/tests/f_extent_leaf_bad_magic/expect.1
@@ -2,8 +2,11 @@ Pass 1: Checking inodes, blocks, and sizes
Inode 12 has an invalid extent node (blk 1604, lblk 0)
Clear? yes
+Inode 12 extent tree could be more compact. Fix? yes
+
Inode 12, i_blocks is 18, should be 0. Fix? yes
+Pass 1E: Optimizing extent trees
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
diff --git a/tests/f_extent_oobounds/expect.1 b/tests/f_extent_oobounds/expect.1
index 3164ea0..237829a 100644
--- a/tests/f_extent_oobounds/expect.1
+++ b/tests/f_extent_oobounds/expect.1
@@ -3,8 +3,11 @@ Inode 12, end of extent exceeds allowed value
(logical block 15, physical block 200, len 30)
Clear? yes
+Inode 12 extent tree could be more compact. Fix? yes
+
Inode 12, i_blocks is 154, should be 94. Fix? yes
+Pass 1E: Optimizing extent trees
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
@@ -12,13 +15,13 @@ Pass 5: Checking group summary information
Block bitmap differences: -(200--229)
Fix? yes
-Free blocks count wrong for group #0 (156, counted=186).
+Free blocks count wrong for group #0 (158, counted=188).
Fix? yes
-Free blocks count wrong (156, counted=186).
+Free blocks count wrong (158, counted=188).
Fix? yes
test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
-test_filesys: 12/32 files (8.3% non-contiguous), 70/256 blocks
+test_filesys: 12/32 files (8.3% non-contiguous), 68/256 blocks
Exit status is 1
diff --git a/tests/f_extent_oobounds/expect.2 b/tests/f_extent_oobounds/expect.2
index 22c4f2c..0729283 100644
--- a/tests/f_extent_oobounds/expect.2
+++ b/tests/f_extent_oobounds/expect.2
@@ -3,5 +3,5 @@ Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
-test_filesys: 12/32 files (8.3% non-contiguous), 70/256 blocks
+test_filesys: 12/32 files (8.3% non-contiguous), 68/256 blocks
Exit status is 0
diff --git a/tests/f_extents/expect.1 b/tests/f_extents/expect.1
index aeebc7b..d682929 100644
--- a/tests/f_extents/expect.1
+++ b/tests/f_extents/expect.1
@@ -6,6 +6,8 @@ Inode 12 has an invalid extent
(logical block 0, invalid physical block 21994527527949, len 17)
Clear? yes
+Inode 12 extent tree could be more compact. Fix? yes
+
Inode 12, i_blocks is 34, should be 0. Fix? yes
Inode 13 missing EXTENT_FL, but is in extents format
@@ -21,6 +23,8 @@ Inode 17 has an invalid extent
(logical block 0, invalid physical block 22011707397135, len 15)
Clear? yes
+Inode 17 extent tree could be more compact. Fix? yes
+
Inode 17, i_blocks is 32, should be 0. Fix? yes
Error while reading over extent tree in inode 18: Corrupt extent header
@@ -31,6 +35,7 @@ Inode 18, i_blocks is 2, should be 0. Fix? yes
Special (device/socket/fifo) file (inode 19) has extents
or inline-data flag set. Clear? yes
+Pass 1E: Optimizing extent trees
Pass 2: Checking directory structure
Entry 'fbad-flag' in / (2) has deleted/unused inode 18. Clear? yes
^ permalink raw reply related [flat|nested] 104+ messages in thread* Re: [PATCH 33/47] e2fsck: rebuild sparse extent trees/convert non-extent ext3 files
2014-11-07 21:54 ` [PATCH 33/47] e2fsck: rebuild sparse extent trees/convert non-extent ext3 files Darrick J. Wong
@ 2014-12-04 23:20 ` Andreas Dilger
2014-12-04 23:45 ` Darrick J. Wong
2014-12-11 22:05 ` [PATCH v2 " Darrick J. Wong
1 sibling, 1 reply; 104+ messages in thread
From: Andreas Dilger @ 2014-12-04 23:20 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: tytso, linux-ext4
On Nov 7, 2014, at 2:54 PM, Darrick J. Wong <darrick.wong@oracle.com> wrote:
>
> Teach e2fsck to construct extent trees. This enables us to do either
> of the following: compress a highly sparse extent tree into fewer ETB
> blocks; or convert a ext3-style block mapped file to an extent file.
>
> For files that are already extent based, this algorithm will only run
> if pass1 determines either (1) that a whole level of extent tree will
> fit into a higher level of the tree; (2) that the size of any level
> can be reduced by at least one ETB block; or (3) the extent tree is
> unnecessarily deep. It will not run at all if errors are found and
> the user declines to fix the errors.
>
> For block-mapped files, conversion only happens if the extent feature
> is enabled and "-E bmap2extent" is passed to e2fsck. It will not run
> at all if errors are left unfixed. After conversion, files larger
> than 12 blocks should be defragmented to eliminate empty holes where a
> block lives.
>
> The extent tree constructor is pretty dumb -- it creates a list of
> leaf extents (adjacent extents are collapsed), marks all indirect
> blocks / ETB blocks free, installs a new extent tree root in the
> inode, then loads the leaf extents into the tree.
Out of curiosity, when doing block-to-extent conversion, does it use
the freed indirect blocks as index blocks to avoid holes in the file
(and single unallocated blocks that would cause fragmentation), or
does it allocate the index blocks at the start of the group according
to keep index blocks together for fast access during e2fsck?
Cheers, Andreas
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
> e2fsck/Makefile.in | 16 +
> e2fsck/e2fsck.8.in | 3
> e2fsck/e2fsck.c | 2
> e2fsck/e2fsck.h | 11 +
> e2fsck/extents.c | 352 ++++++++++++++++++++++++++++++++
> e2fsck/pass1.c | 95 ++++++++-
> e2fsck/problem.c | 43 ++++
> e2fsck/problem.h | 28 +++
> e2fsck/super.c | 7 +
> e2fsck/unix.c | 4
> tests/f_extent_bad_node/expect.1 | 9 +
> tests/f_extent_bad_node/expect.2 | 2
> tests/f_extent_int_bad_magic/expect.1 | 3
> tests/f_extent_leaf_bad_magic/expect.1 | 3
> tests/f_extent_oobounds/expect.1 | 9 +
> tests/f_extent_oobounds/expect.2 | 2
> tests/f_extents/expect.1 | 5
> 17 files changed, 577 insertions(+), 17 deletions(-)
> create mode 100644 e2fsck/extents.c
>
>
> diff --git a/e2fsck/Makefile.in b/e2fsck/Makefile.in
> index e40e51b..a4413d9 100644
> --- a/e2fsck/Makefile.in
> +++ b/e2fsck/Makefile.in
> @@ -62,7 +62,8 @@ OBJS= dict.o unix.o e2fsck.o super.o pass1.o pass1b.o pass2.o \
> pass3.o pass4.o pass5.o journal.o badblocks.o util.o dirinfo.o \
> dx_dirinfo.o ehandler.o problem.o message.o quota.o recovery.o \
> region.o revoke.o ea_refcount.o rehash.o profile.o prof_err.o \
> - logfile.o sigcatcher.o $(MTRACE_OBJ) plausible.o readahead.o
> + logfile.o sigcatcher.o $(MTRACE_OBJ) plausible.o readahead.o \
> + extents.o
>
> PROFILED_OBJS= profiled/dict.o profiled/unix.o profiled/e2fsck.o \
> profiled/super.o profiled/pass1.o profiled/pass1b.o \
> @@ -74,7 +75,7 @@ PROFILED_OBJS= profiled/dict.o profiled/unix.o profiled/e2fsck.o \
> profiled/ea_refcount.o profiled/rehash.o profiled/profile.o \
> profiled/prof_err.o profiled/logfile.o \
> profiled/sigcatcher.o profiled/plausible.o \
> - profiled/sigcatcher.o profiled/readahead.o
> + profiled/sigcatcher.o profiled/readahead.o profiled/extents.o
>
> SRCS= $(srcdir)/e2fsck.c \
> $(srcdir)/dict.c \
> @@ -106,6 +107,7 @@ SRCS= $(srcdir)/e2fsck.c \
> prof_err.c \
> $(srcdir)/quota.c \
> $(srcdir)/../misc/plausible.c \
> + $(srcdir)/extents.c \
> $(MTRACE_SRC)
>
> all:: profiled $(PROGS) e2fsck $(MANPAGES) $(FMANPAGES)
> @@ -308,6 +310,16 @@ pass1.o: $(srcdir)/pass1.c $(top_builddir)/lib/config.h \
> $(srcdir)/profile.h prof_err.h $(top_srcdir)/lib/quota/quotaio.h \
> $(top_srcdir)/lib/quota/dqblk_v2.h $(top_srcdir)/lib/quota/quotaio_tree.h \
> $(top_srcdir)/lib/../e2fsck/dict.h $(srcdir)/problem.h
> +extents.o: $(srcdir)/extents.c $(top_builddir)/lib/config.h \
> + $(top_builddir)/lib/dirpaths.h $(top_srcdir)/lib/et/com_err.h \
> + $(srcdir)/e2fsck.h $(top_srcdir)/lib/ext2fs/ext2_fs.h \
> + $(top_builddir)/lib/ext2fs/ext2_types.h $(top_srcdir)/lib/ext2fs/ext2fs.h \
> + $(top_srcdir)/lib/ext2fs/ext3_extents.h $(top_srcdir)/lib/ext2fs/ext2_io.h \
> + $(top_builddir)/lib/ext2fs/ext2_err.h \
> + $(top_srcdir)/lib/ext2fs/ext2_ext_attr.h $(top_srcdir)/lib/ext2fs/bitops.h \
> + $(srcdir)/profile.h prof_err.h $(top_srcdir)/lib/quota/quotaio.h \
> + $(top_srcdir)/lib/quota/dqblk_v2.h $(top_srcdir)/lib/quota/quotaio_tree.h \
> + $(top_srcdir)/lib/../e2fsck/dict.h $(srcdir)/problem.h $(srcdir)/dict.h
> pass1b.o: $(srcdir)/pass1b.c $(top_builddir)/lib/config.h \
> $(top_builddir)/lib/dirpaths.h $(top_srcdir)/lib/et/com_err.h \
> $(srcdir)/e2fsck.h $(top_srcdir)/lib/ext2fs/ext2_fs.h \
> diff --git a/e2fsck/e2fsck.8.in b/e2fsck/e2fsck.8.in
> index 84ae50f..0c2725e 100644
> --- a/e2fsck/e2fsck.8.in
> +++ b/e2fsck/e2fsck.8.in
> @@ -214,6 +214,9 @@ e2fsck runtime. By default, this is set to the size of a block group's inode
> table (typically 2MiB on a regular ext4 filesystem); if this amount is more
> than 1/100 of total physical memory, readahead is disabled. Set this to zero
> to disable readahead entirely.
> +.TP
> +.BI bmap2extent
> +Convert block-mapped files to extent-mapped files.
> .RE
> .TP
> .B \-f
> diff --git a/e2fsck/e2fsck.c b/e2fsck/e2fsck.c
> index fcda7d7..83506cb 100644
> --- a/e2fsck/e2fsck.c
> +++ b/e2fsck/e2fsck.c
> @@ -204,7 +204,7 @@ void e2fsck_free_context(e2fsck_t ctx)
> typedef void (*pass_t)(e2fsck_t ctx);
>
> static pass_t e2fsck_passes[] = {
> - e2fsck_pass1, e2fsck_pass2, e2fsck_pass3, e2fsck_pass4,
> + e2fsck_pass1, e2fsck_pass1e, e2fsck_pass2, e2fsck_pass3, e2fsck_pass4,
> e2fsck_pass5, 0 };
>
> #define E2F_FLAG_RUN_RETURN (E2F_FLAG_SIGNAL_MASK|E2F_FLAG_RESTART)
> diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
> index e359515..66d71ee 100644
> --- a/e2fsck/e2fsck.h
> +++ b/e2fsck/e2fsck.h
> @@ -167,6 +167,7 @@ struct resource_track {
> #define E2F_OPT_FRAGCHECK 0x0800
> #define E2F_OPT_JOURNAL_ONLY 0x1000 /* only replay the journal */
> #define E2F_OPT_DISCARD 0x2000
> +#define E2F_OPT_CONVERT_BMAP 0x4000 /* convert blockmap to extent */
>
> /*
> * E2fsck flags
> @@ -381,6 +382,11 @@ struct e2fsck_struct {
>
> /* How much are we allowed to readahead? */
> unsigned long long readahead_kb;
> +
> + /*
> + * Inodes to rebuild extent trees
> + */
> + ext2fs_inode_bitmap inodes_to_rebuild;
> };
>
> /* Used by the region allocation code */
> @@ -456,6 +462,11 @@ extern blk64_t ea_refcount_intr_next(ext2_refcount_t refcount, int *ret);
> extern const char *ehandler_operation(const char *op);
> extern void ehandler_init(io_channel channel);
>
> +/* extents.c */
> +void e2fsck_rebuild_extents_later(e2fsck_t ctx, ext2_ino_t ino);
> +int e2fsck_ino_will_be_rebuilt(e2fsck_t ctx, ext2_ino_t ino);
> +void e2fsck_pass1e(e2fsck_t ctx);
> +
> /* journal.c */
> extern errcode_t e2fsck_check_ext3_journal(e2fsck_t ctx);
> extern errcode_t e2fsck_run_ext3_journal(e2fsck_t ctx);
> diff --git a/e2fsck/extents.c b/e2fsck/extents.c
> new file mode 100644
> index 0000000..91d36d2
> --- /dev/null
> +++ b/e2fsck/extents.c
> @@ -0,0 +1,352 @@
> +/*
> + * extents.c --- rebuild extent tree
> + *
> + * Copyright (C) 2014 Oracle.
> + *
> + * %Begin-Header%
> + * This file may be redistributed under the terms of the GNU Public
> + * License.
> + * %End-Header%
> + */
> +
> +#include "config.h"
> +#include <string.h>
> +#include <ctype.h>
> +#include <errno.h>
> +#include "e2fsck.h"
> +#include "problem.h"
> +
> +#undef DEBUG
> +#undef DEBUG_SUMMARY
> +#undef DEBUG_FREE
> +
> +#define NUM_EXTENTS 341 /* about one ETB' worth of extents */
> +
> +/* Schedule an inode to have its extent tree rebuilt during pass 1E. */
> +void e2fsck_rebuild_extents_later(e2fsck_t ctx, ext2_ino_t ino)
> +{
> + if (!EXT2_HAS_INCOMPAT_FEATURE(ctx->fs->super,
> + EXT3_FEATURE_INCOMPAT_EXTENTS) ||
> + (ctx->options & E2F_OPT_NO) ||
> + (ino != EXT2_ROOT_INO && ino < ctx->fs->super->s_first_ino))
> + return;
> + if (!ctx->inodes_to_rebuild)
> + e2fsck_allocate_inode_bitmap(ctx->fs,
> + _("extent rebuild inode map"),
> + EXT2FS_BMAP64_AUTODIR,
> + "inodes_to_rebuild",
> + &ctx->inodes_to_rebuild);
> + if (ctx->inodes_to_rebuild)
> + ext2fs_mark_inode_bitmap2(ctx->inodes_to_rebuild, ino);
> +}
> +
> +/* Ask if an inode will have its extents rebuilt during pass 1E. */
> +int e2fsck_ino_will_be_rebuilt(e2fsck_t ctx, ext2_ino_t ino)
> +{
> + if (!ctx->inodes_to_rebuild)
> + return 0;
> + return ext2fs_test_inode_bitmap2(ctx->inodes_to_rebuild, ino);
> +}
> +
> +struct extent_list {
> + blk64_t blocks_freed;
> + struct ext2fs_extent *extents;
> + unsigned int count;
> + unsigned int size;
> + unsigned int ext_read;
> + errcode_t retval;
> + ext2_ino_t ino;
> +};
> +
> +static errcode_t load_extents(e2fsck_t ctx, struct extent_list *list)
> +{
> + ext2_filsys fs = ctx->fs;
> + ext2_extent_handle_t handle;
> + struct ext2fs_extent extent;
> + errcode_t retval;
> +
> + retval = ext2fs_extent_open(fs, list->ino, &handle);
> + if (retval)
> + return retval;
> +
> + retval = ext2fs_extent_get(handle, EXT2_EXTENT_ROOT, &extent);
> + if (retval)
> + goto out;
> +
> + do {
> + if (extent.e_flags & EXT2_EXTENT_FLAGS_SECOND_VISIT)
> + goto next;
> +
> + /* Internal node; free it and we'll re-allocate it later */
> + if (!(extent.e_flags & EXT2_EXTENT_FLAGS_LEAF)) {
> +#if defined(DEBUG) || defined(DEBUG_FREE)
> + printf("ino=%d free=%llu bf=%llu\n", list->ino,
> + extent.e_pblk, list->blocks_freed + 1);
> +#endif
> + list->blocks_freed++;
> + ext2fs_block_alloc_stats2(fs, extent.e_pblk, -1);
> + goto next;
> + }
> +
> + list->ext_read++;
> + /* Can we attach it to the previous extent? */
> + if (list->count) {
> + struct ext2fs_extent *last = list->extents +
> + list->count - 1;
> + blk64_t end = last->e_len + extent.e_len;
> +
> + if (last->e_pblk + last->e_len == extent.e_pblk &&
> + last->e_lblk + last->e_len == extent.e_lblk &&
> + (last->e_flags & EXT2_EXTENT_FLAGS_UNINIT) ==
> + (extent.e_flags & EXT2_EXTENT_FLAGS_UNINIT) &&
> + end < (1ULL << 32)) {
> + last->e_len += extent.e_len;
> +#ifdef DEBUG
> + printf("R: ino=%d len=%u\n", list->ino,
> + last->e_len);
> +#endif
> + goto next;
> + }
> + }
> +
> + /* Do we need to expand? */
> + if (list->count == list->size) {
> + unsigned int new_size = (list->size + NUM_EXTENTS) *
> + sizeof(struct ext2fs_extent);
> + retval = ext2fs_resize_mem(0, new_size, &list->extents);
> + if (retval)
> + goto out;
> + list->size += NUM_EXTENTS;
> + }
> +
> + /* Add a new extent */
> + memcpy(list->extents + list->count, &extent, sizeof(extent));
> +#ifdef DEBUG
> + printf("R: ino=%d pblk=%llu lblk=%llu len=%u\n", list->ino,
> + extent.e_pblk, extent.e_lblk, extent.e_len);
> +#endif
> + list->count++;
> +next:
> + retval = ext2fs_extent_get(handle, EXT2_EXTENT_NEXT, &extent);
> + } while (retval == 0);
> +
> +out:
> + /* Ok if we run off the end */
> + if (retval == EXT2_ET_EXTENT_NO_NEXT)
> + retval = 0;
> + ext2fs_extent_free(handle);
> + return retval;
> +}
> +
> +static int find_blocks(ext2_filsys fs, blk64_t *blocknr, e2_blkcnt_t blockcnt,
> + blk64_t ref_blk, int ref_offset, void *priv_data)
> +{
> + struct extent_list *list = priv_data;
> +
> + /* Internal node? */
> + if (blockcnt < 0) {
> +#if defined(DEBUG) || defined(DEBUG_FREE)
> + printf("ino=%d free=%llu bf=%llu\n", list->ino, *blocknr,
> + list->blocks_freed + 1);
> +#endif
> + list->blocks_freed++;
> + ext2fs_block_alloc_stats2(fs, *blocknr, -1);
> + return 0;
> + }
> +
> + /* Can we attach it to the previous extent? */
> + if (list->count) {
> + struct ext2fs_extent *last = list->extents +
> + list->count - 1;
> + blk64_t end = last->e_len + 1;
> +
> + if (last->e_pblk + last->e_len == *blocknr &&
> + end < (1ULL << 32)) {
> + last->e_len++;
> +#ifdef DEBUG
> + printf("R: ino=%d len=%u\n", list->ino, last->e_len);
> +#endif
> + return 0;
> + }
> + }
> +
> + /* Do we need to expand? */
> + if (list->count == list->size) {
> + unsigned int new_size = (list->size + NUM_EXTENTS) *
> + sizeof(struct ext2fs_extent);
> + list->retval = ext2fs_resize_mem(0, new_size, &list->extents);
> + if (list->retval)
> + return BLOCK_ABORT;
> + list->size += NUM_EXTENTS;
> + }
> +
> + /* Add a new extent */
> + list->extents[list->count].e_pblk = *blocknr;
> + list->extents[list->count].e_lblk = blockcnt;
> + list->extents[list->count].e_len = 1;
> + list->extents[list->count].e_flags = 0;
> +#ifdef DEBUG
> + printf("R: ino=%d pblk=%llu lblk=%llu len=%u\n", list->ino, *blocknr,
> + blockcnt, 1);
> +#endif
> + list->count++;
> +
> + return 0;
> +}
> +
> +static errcode_t rebuild_extent_tree(e2fsck_t ctx, struct extent_list *list,
> + ext2_ino_t ino)
> +{
> + struct ext2_inode inode;
> + errcode_t retval;
> + ext2_extent_handle_t handle;
> + unsigned int i, ext_written;
> + struct ext2fs_extent *ex, extent;
> +
> + list->count = 0;
> + list->blocks_freed = 0;
> + list->ino = ino;
> + list->ext_read = 0;
> + e2fsck_read_inode(ctx, ino, &inode, "rebuild_extents");
> +
> + /* Collect lblk->pblk mappings */
> + if (inode.i_flags & EXT4_EXTENTS_FL) {
> + retval = load_extents(ctx, list);
> + goto extents_loaded;
> + }
> +
> + retval = ext2fs_block_iterate3(ctx->fs, ino, BLOCK_FLAG_READ_ONLY, 0,
> + find_blocks, list);
> + if (retval)
> + goto err;
> + if (list->retval) {
> + retval = list->retval;
> + goto err;
> + }
> +
> +extents_loaded:
> + /* Reset extent tree */
> + inode.i_flags &= ~EXT4_EXTENTS_FL;
> + memset(inode.i_block, 0, sizeof(inode.i_block));
> +
> + /* Make a note of freed blocks */
> + retval = ext2fs_iblk_sub_blocks(ctx->fs, &inode, list->blocks_freed);
> + if (retval)
> + goto err;
> +
> + /* Now stuff extents into the file */
> + retval = ext2fs_extent_open2(ctx->fs, ino, &inode, &handle);
> + if (retval)
> + goto err;
> +
> + ext_written = 0;
> + for (i = 0, ex = list->extents; i < list->count; i++, ex++) {
> + memcpy(&extent, ex, sizeof(struct ext2fs_extent));
> + extent.e_flags &= EXT2_EXTENT_FLAGS_UNINIT;
> + if (extent.e_flags & EXT2_EXTENT_FLAGS_UNINIT) {
> + if (extent.e_len > EXT_UNINIT_MAX_LEN) {
> + extent.e_len = EXT_UNINIT_MAX_LEN;
> + ex->e_pblk += EXT_UNINIT_MAX_LEN;
> + ex->e_lblk += EXT_UNINIT_MAX_LEN;
> + ex->e_len -= EXT_UNINIT_MAX_LEN;
> + ex--;
> + i--;
> + }
> + } else {
> + if (extent.e_len > EXT_INIT_MAX_LEN) {
> + extent.e_len = EXT_INIT_MAX_LEN;
> + ex->e_pblk += EXT_INIT_MAX_LEN;
> + ex->e_lblk += EXT_INIT_MAX_LEN;
> + ex->e_len -= EXT_INIT_MAX_LEN;
> + ex--;
> + i--;
> + }
> + }
> +
> +#ifdef DEBUG
> + printf("W: ino=%d pblk=%llu lblk=%llu len=%u\n", ino,
> + extent.e_pblk, extent.e_lblk, extent.e_len);
> +#endif
> + retval = ext2fs_extent_insert(handle, EXT2_EXTENT_INSERT_AFTER,
> + &extent);
> + if (retval)
> + goto err2;
> + retval = ext2fs_extent_fix_parents(handle);
> + if (retval)
> + goto err2;
> + ext_written++;
> + }
> +
> +#if defined(DEBUG) || defined(DEBUG_SUMMARY)
> + printf("rebuild: ino=%d extents=%d->%d\n", ino, list->ext_read,
> + ext_written);
> +#endif
> + e2fsck_write_inode(ctx, ino, &inode, "rebuild_extents");
> +
> +err2:
> + ext2fs_extent_free(handle);
> +err:
> + return retval;
> +}
> +
> +void e2fsck_pass1e(e2fsck_t ctx)
> +{
> + struct problem_context pctx;
> +#ifdef RESOURCE_TRACK
> + struct resource_track rtrack;
> +#endif
> + struct extent_list list;
> + int first = 1;
> + ext2_ino_t ino = 0;
> + errcode_t retval;
> +
> + if (!EXT2_HAS_INCOMPAT_FEATURE(ctx->fs->super,
> + EXT3_FEATURE_INCOMPAT_EXTENTS) ||
> + !ext2fs_test_valid(ctx->fs) ||
> + ctx->invalid_bitmaps) {
> + if (ctx->inodes_to_rebuild)
> + ext2fs_free_inode_bitmap(ctx->inodes_to_rebuild);
> + ctx->inodes_to_rebuild = NULL;
> + }
> +
> + if (ctx->inodes_to_rebuild == NULL)
> + return;
> +
> + init_resource_track(&rtrack, ctx->fs->io);
> + clear_problem_context(&pctx);
> + e2fsck_read_bitmaps(ctx);
> +
> + memset(&list, 0, sizeof(list));
> + retval = ext2fs_get_mem(sizeof(struct ext2fs_extent) * NUM_EXTENTS,
> + &list.extents);
> + list.size = NUM_EXTENTS;
> + while (1) {
> + retval = ext2fs_find_first_set_inode_bitmap2(
> + ctx->inodes_to_rebuild, ino + 1,
> + ctx->fs->super->s_inodes_count, &ino);
> + if (retval)
> + break;
> + pctx.ino = ino;
> + if (first) {
> + fix_problem(ctx, PR_1E_PASS_HEADER, &pctx);
> + first = 0;
> + }
> + pctx.errcode = rebuild_extent_tree(ctx, &list, ino);
> + if (pctx.errcode) {
> + end_problem_latch(ctx, PR_LATCH_OPTIMIZE_EXT);
> + fix_problem(ctx, PR_1E_OPTIMIZE_EXT_ERR, &pctx);
> + }
> + if (ctx->progress && !ctx->progress_fd)
> + e2fsck_simple_progress(ctx, "Rebuilding extents",
> + 100.0 * (float) ino /
> + (float) ctx->fs->super->s_inodes_count,
> + ino);
> + }
> + end_problem_latch(ctx, PR_LATCH_OPTIMIZE_EXT);
> +
> + ext2fs_free_inode_bitmap(ctx->inodes_to_rebuild);
> + ctx->inodes_to_rebuild = NULL;
> + ext2fs_free_mem(&list.extents);
> +
> + print_resource_track(ctx, "Pass 1E", &rtrack, ctx->fs->io);
> +}
> diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
> index a963849..10008d9 100644
> --- a/e2fsck/pass1.c
> +++ b/e2fsck/pass1.c
> @@ -56,6 +56,8 @@
> #define _INLINE_ inline
> #endif
>
> +#undef DEBUG
> +
> static int process_block(ext2_filsys fs, blk64_t *blocknr,
> e2_blkcnt_t blockcnt, blk64_t ref_blk,
> int ref_offset, void *priv_data);
> @@ -77,11 +79,16 @@ static void adjust_extattr_refcount(e2fsck_t ctx, ext2_refcount_t refcount,
> char *block_buf, int adjust_sign);
> /* static char *describe_illegal_block(ext2_filsys fs, blk64_t block); */
>
> +struct extent_info {
> + unsigned int num_extents;
> + unsigned int max_extents;
> +};
> +
> struct process_block_struct {
> ext2_ino_t ino;
> unsigned is_dir:1, is_reg:1, clear:1, suppress:1,
> fragmented:1, compressed:1, bbcheck:1,
> - inode_modified:1;
> + inode_modified:1, extent_rebuild:1;
> blk64_t num_blocks;
> blk64_t max_blocks;
> e2_blkcnt_t last_block;
> @@ -95,6 +102,7 @@ struct process_block_struct {
> e2fsck_t ctx;
> blk64_t bad_ref;
> region_t region;
> + struct extent_info ext_info[MAX_EXTENT_DEPTH_COUNT];
> };
>
> struct process_inode_block {
> @@ -2402,6 +2410,53 @@ static int has_unaligned_cluster_map(e2fsck_t ctx,
> return 0;
> }
>
> +static void should_rebuild_extents(e2fsck_t ctx,
> + struct problem_context *pctx,
> + struct process_block_struct *pb,
> + struct ext2_extent_info *info)
> +{
> + struct extent_info *ei;
> + int i, j;
> + unsigned int extents_per_block;
> +
> + if (pb->extent_rebuild)
> + goto rebuild;
> +
> + extents_per_block = (ctx->fs->blocksize -
> + sizeof(struct ext3_extent_header)) /
> + sizeof(struct ext3_extent);
> + /*
> + * If we can consolidate a level or shorten the tree, schedule the
> + * extent tree to be rebuilt.
> + */
> + for (i = 0, ei = pb->ext_info; i < info->max_depth + 1; i++, ei++) {
> + if (ei->max_extents - ei->num_extents > extents_per_block) {
> +#ifdef DEBUG
> + printf("rebuild extents, ino=%d level=%d slack=%d epb=%d\n",
> + pb->ino, i,
> + ei->max_extents - ei->num_extents,
> + extents_per_block);
> +#endif
> + goto rebuild;
> + }
> + for (j = 0; j < i; j++) {
> + if (ei->num_extents < pb->ext_info[j].max_extents) {
> +#ifdef DEBUG
> + printf("rebuild extents, ino=%d level=%d num=%d level=%d\n",
> + pb->ino, i, ei->num_extents, j);
> +#endif
> + goto rebuild;
> + }
> + }
> + }
> + return;
> +
> +rebuild:
> + if (pb->extent_rebuild ||
> + fix_problem(ctx, PR_1E_CAN_COMPRESS_EXTENT_TREE, pctx))
> + e2fsck_rebuild_extents_later(ctx, pb->ino);
> +}
> +
> static void scan_extent_node(e2fsck_t ctx, struct problem_context *pctx,
> struct process_block_struct *pb,
> blk64_t start_block, blk64_t end_block,
> @@ -2424,6 +2479,10 @@ static void scan_extent_node(e2fsck_t ctx, struct problem_context *pctx,
> pctx->errcode = ext2fs_extent_get_info(ehandle, &info);
> if (pctx->errcode)
> return;
> + if (!pb->extent_rebuild) {
> + pb->ext_info[info.curr_level].num_extents += info.num_entries;
> + pb->ext_info[info.curr_level].max_extents += info.max_entries;
> + }
>
> pctx->errcode = ext2fs_extent_get(ehandle, EXT2_EXTENT_FIRST_SIB,
> &extent);
> @@ -2760,17 +2819,31 @@ static void check_blocks_extents(e2fsck_t ctx, struct problem_context *pctx,
>
> retval = ext2fs_extent_get_info(ehandle, &info);
> if (retval == 0) {
> - if (info.max_depth >= MAX_EXTENT_DEPTH_COUNT)
> - info.max_depth = MAX_EXTENT_DEPTH_COUNT-1;
> - ctx->extent_depth_count[info.max_depth]++;
> + int max_depth = info.max_depth;
> +
> + if (max_depth >= MAX_EXTENT_DEPTH_COUNT)
> + max_depth = MAX_EXTENT_DEPTH_COUNT-1;
> + ctx->extent_depth_count[max_depth]++;
> }
>
> + /* Check maximum extent depth */
> + pctx->blk = info.max_depth;
> + pctx->blk2 = ext2fs_max_extent_depth(ehandle);
> + if (pctx->blk2 < pctx->blk &&
> + fix_problem(ctx, PR_1_EXTENT_BAD_MAX_DEPTH, pctx))
> + pb->extent_rebuild = 1;
> +
> + /* Can we collect extent tree level stats? */
> + pctx->blk = MAX_EXTENT_DEPTH_COUNT;
> + if (pctx->blk2 > pctx->blk)
> + fix_problem(ctx, PR_1E_MAX_EXTENT_TREE_DEPTH, pctx);
> + memset(pb->ext_info, 0, sizeof(pb->ext_info));
> +
> pb->region = region_create(0, info.max_lblk);
> if (!pb->region) {
> - ext2fs_extent_free(ehandle);
> fix_problem(ctx, PR_1_EXTENT_ALLOC_REGION_ABORT, pctx);
> ctx->flags |= E2F_FLAG_ABORT;
> - return;
> + goto out;
> }
>
> eof_lblk = ((EXT2_I_SIZE(inode) + fs->blocksize - 1) >>
> @@ -2786,7 +2859,9 @@ static void check_blocks_extents(e2fsck_t ctx, struct problem_context *pctx,
> }
> region_free(pb->region);
> pb->region = NULL;
> +out:
> ext2fs_extent_free(ehandle);
> + should_rebuild_extents(ctx, pctx, pb, &info);
> }
>
> /*
> @@ -2846,6 +2921,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
> pb.ctx = ctx;
> pb.inode_modified = 0;
> pb.bad_ref = 0;
> + pb.extent_rebuild = 0;
> pctx->ino = ino;
> pctx->errcode = 0;
>
> @@ -2909,6 +2985,13 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
> "check_blocks");
> fs->flags = (flags & EXT2_FLAG_IGNORE_CSUM_ERRORS) |
> (fs->flags & ~EXT2_FLAG_IGNORE_CSUM_ERRORS);
> +
> + if (ctx->options & E2F_OPT_CONVERT_BMAP) {
> +#ifdef DEBUG
> + printf("bmap rebuild ino=%d\n", ino);
> +#endif
> + e2fsck_rebuild_extents_later(ctx, ino);
> + }
> }
> }
> end_problem_latch(ctx, PR_LATCH_BLOCK);
> diff --git a/e2fsck/problem.c b/e2fsck/problem.c
> index a4da64b..75e0305 100644
> --- a/e2fsck/problem.c
> +++ b/e2fsck/problem.c
> @@ -1101,6 +1101,11 @@ static struct e2fsck_problem problem_table[] = {
> N_("@i %i has a duplicate @x mapping\n\t(logical @b %c, @n physical @b %b, len %N)\n"),
> PROMPT_CLEAR, 0 },
>
> + /* Inode extent tree could be more shallow */
> + { PR_1_EXTENT_BAD_MAX_DEPTH,
> + N_("@i %i @x tree could be more shallow (%b; could be <= %c)\n"),
> + PROMPT_FIX, PR_NO_OK | PR_PREEN_NO | PR_PREEN_OK },
> +
> /* Pass 1b errors */
>
> /* Pass 1B: Rescan for duplicate/bad blocks */
> @@ -1198,6 +1203,43 @@ static struct e2fsck_problem problem_table[] = {
> { PR_1D_CLONE_ERROR,
> N_("Couldn't clone file: %m\n"), PROMPT_NONE, 0 },
>
> + /* Pass 1E Extent tree Optimization */
> +
> + /* Pass 1E: Optimizing extent trees */
> + { PR_1E_PASS_HEADER,
> + N_("Pass 1E: Optimizing @x trees\n"),
> + PROMPT_NONE, PR_PREEN_NOMSG },
> +
> + /* Failed to optimize extent tree */
> + { PR_1E_OPTIMIZE_EXT_ERR,
> + N_("Failed to optimize @x tree %p (%i): %m\n"),
> + PROMPT_NONE, 0 },
> +
> + /* Rebuilding extent trees */
> + { PR_1E_OPTIMIZE_EXT_HEADER,
> + N_("Optimizing @x trees: "),
> + PROMPT_NONE, PR_MSG_ONLY },
> +
> + /* Rebuilding extent tree %d */
> + { PR_1E_OPTIMIZE_EXT,
> + " %i",
> + PROMPT_NONE, PR_LATCH_OPTIMIZE_EXT | PR_PREEN_NOHDR},
> +
> + /* Rebuilding extent tree end */
> + { PR_1E_OPTIMIZE_EXT_END,
> + "\n",
> + PROMPT_NONE, PR_PREEN_NOHDR },
> +
> + /* Internal error: extent tree depth too large */
> + { PR_1E_MAX_EXTENT_TREE_DEPTH,
> + N_("Internal error: max extent tree depth too large (%b; expected=%c).\n"),
> + PROMPT_NONE, PR_FATAL },
> +
> + /* Inode extent tree could be more compact */
> + { PR_1E_CAN_COMPRESS_EXTENT_TREE,
> + N_("@i %i @x tree could be more compact. "),
> + PROMPT_FIX, PR_NO_OK | PR_PREEN_NO | PR_PREEN_OK },
> +
> /* Pass 2 errors */
>
> /* Pass 2: Checking directory structure */
> @@ -1946,6 +1988,7 @@ static struct latch_descr pr_latch_info[] = {
> { PR_LATCH_TOOBIG, PR_1_INODE_TOOBIG, 0 },
> { PR_LATCH_OPTIMIZE_DIR, PR_3A_OPTIMIZE_DIR_HEADER, PR_3A_OPTIMIZE_DIR_END },
> { PR_LATCH_BG_CHECKSUM, PR_0_GDT_CSUM_LATCH, 0 },
> + { PR_LATCH_OPTIMIZE_EXT, PR_1E_OPTIMIZE_EXT_HEADER, PR_1E_OPTIMIZE_EXT_END },
> { -1, 0, 0 },
> };
>
> diff --git a/e2fsck/problem.h b/e2fsck/problem.h
> index 3c28166..d3dcc9e 100644
> --- a/e2fsck/problem.h
> +++ b/e2fsck/problem.h
> @@ -40,6 +40,7 @@ struct problem_context {
> #define PR_LATCH_TOOBIG 0x0080 /* Latch for file to big errors */
> #define PR_LATCH_OPTIMIZE_DIR 0x0090 /* Latch for optimize directories */
> #define PR_LATCH_BG_CHECKSUM 0x00A0 /* Latch for block group checksums */
> +#define PR_LATCH_OPTIMIZE_EXT 0x00B0 /* Latch for rebuild extents */
>
> #define PR_LATCH(x) ((((x) & PR_LATCH_MASK) >> 4) - 1)
>
> @@ -641,6 +642,9 @@ struct problem_context {
> /* leaf extent collision */
> #define PR_1_EXTENT_COLLISION 0x01007D
>
> +/* extent tree max depth too big */
> +#define PR_1_EXTENT_BAD_MAX_DEPTH 0x01007E
> +
> /*
> * Pass 1b errors
> */
> @@ -704,6 +708,30 @@ struct problem_context {
> #define PR_1D_CLONE_ERROR 0x013008
>
> /*
> + * Pass 1e --- rebuilding extent trees
> + */
> +/* Pass 1e: Rebuilding extent trees */
> +#define PR_1E_PASS_HEADER 0x014000
> +
> +/* Error rehash directory */
> +#define PR_1E_OPTIMIZE_EXT_ERR 0x014001
> +
> +/* Rebuilding extent trees */
> +#define PR_1E_OPTIMIZE_EXT_HEADER 0x014002
> +
> +/* Rebuilding extent %d */
> +#define PR_1E_OPTIMIZE_EXT 0x014003
> +
> +/* Rebuilding extent tree end */
> +#define PR_1E_OPTIMIZE_EXT_END 0x014004
> +
> +/* Internal error: extent tree depth too large */
> +#define PR_1E_MAX_EXTENT_TREE_DEPTH 0x014005
> +
> +/* Inode extent tree could be more compact */
> +#define PR_1E_CAN_COMPRESS_EXTENT_TREE 0x014006
> +
> +/*
> * Pass 2 errors
> */
>
> diff --git a/e2fsck/super.c b/e2fsck/super.c
> index 1e7e749..e64262a 100644
> --- a/e2fsck/super.c
> +++ b/e2fsck/super.c
> @@ -606,6 +606,13 @@ void check_super_block(e2fsck_t ctx)
> ext2fs_mark_super_dirty(fs);
> }
>
> + /* Did user ask us to convert files to extents? */
> + if (ctx->options & E2F_OPT_CONVERT_BMAP) {
> + fs->super->s_feature_incompat |=
> + EXT3_FEATURE_INCOMPAT_EXTENTS;
> + ext2fs_mark_super_dirty(fs);
> + }
> +
> if ((fs->super->s_feature_incompat & EXT2_FEATURE_INCOMPAT_META_BG) &&
> (fs->super->s_first_meta_bg > fs->desc_blocks)) {
> pctx.group = fs->desc_blocks;
> diff --git a/e2fsck/unix.c b/e2fsck/unix.c
> index f3672c0..fe5127a 100644
> --- a/e2fsck/unix.c
> +++ b/e2fsck/unix.c
> @@ -709,6 +709,9 @@ static void parse_extended_opts(e2fsck_t ctx, const char *opts)
> else
> ctx->log_fn = string_copy(ctx, arg, 0);
> continue;
> + } else if (strcmp(token, "bmap2extent") == 0) {
> + ctx->options |= E2F_OPT_CONVERT_BMAP;
> + continue;
> } else {
> fprintf(stderr, _("Unknown extended option: %s\n"),
> token);
> @@ -728,6 +731,7 @@ static void parse_extended_opts(e2fsck_t ctx, const char *opts)
> fputs(("\tdiscard\n"), stderr);
> fputs(("\tnodiscard\n"), stderr);
> fputs(("\treadahead_kb=<buffer size>\n"), stderr);
> + fputs(("\tbmap2extent\n"), stderr);
> fputc('\n', stderr);
> exit(1);
> }
> diff --git a/tests/f_extent_bad_node/expect.1 b/tests/f_extent_bad_node/expect.1
> index 0c0bc28..c13ad39 100644
> --- a/tests/f_extent_bad_node/expect.1
> +++ b/tests/f_extent_bad_node/expect.1
> @@ -2,8 +2,11 @@ Pass 1: Checking inodes, blocks, and sizes
> Inode 12 has an invalid extent node (blk 22, lblk 0)
> Clear? yes
>
> +Inode 12 extent tree could be more compact. Fix? yes
> +
> Inode 12, i_blocks is 16, should be 8. Fix? yes
>
> +Pass 1E: Optimizing extent trees
> Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> @@ -11,13 +14,13 @@ Pass 5: Checking group summary information
> Block bitmap differences: -(21--23) -25
> Fix? yes
>
> -Free blocks count wrong for group #0 (71, counted=75).
> +Free blocks count wrong for group #0 (73, counted=77).
> Fix? yes
>
> -Free blocks count wrong (71, counted=75).
> +Free blocks count wrong (73, counted=77).
> Fix? yes
>
>
> test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
> -test_filesys: 12/16 files (0.0% non-contiguous), 25/100 blocks
> +test_filesys: 12/16 files (0.0% non-contiguous), 23/100 blocks
> Exit status is 1
> diff --git a/tests/f_extent_bad_node/expect.2 b/tests/f_extent_bad_node/expect.2
> index 568c792..b78b193 100644
> --- a/tests/f_extent_bad_node/expect.2
> +++ b/tests/f_extent_bad_node/expect.2
> @@ -3,5 +3,5 @@ Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
> -test_filesys: 12/16 files (0.0% non-contiguous), 25/100 blocks
> +test_filesys: 12/16 files (0.0% non-contiguous), 23/100 blocks
> Exit status is 0
> diff --git a/tests/f_extent_int_bad_magic/expect.1 b/tests/f_extent_int_bad_magic/expect.1
> index 0e82e2b..0bd163f 100644
> --- a/tests/f_extent_int_bad_magic/expect.1
> +++ b/tests/f_extent_int_bad_magic/expect.1
> @@ -2,8 +2,11 @@ Pass 1: Checking inodes, blocks, and sizes
> Inode 12 has an invalid extent node (blk 1295, lblk 0)
> Clear? yes
>
> +Inode 12 extent tree could be more compact. Fix? yes
> +
> Inode 12, i_blocks is 712, should be 0. Fix? yes
>
> +Pass 1E: Optimizing extent trees
> Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> diff --git a/tests/f_extent_leaf_bad_magic/expect.1 b/tests/f_extent_leaf_bad_magic/expect.1
> index 7b6dbf1..c31a309 100644
> --- a/tests/f_extent_leaf_bad_magic/expect.1
> +++ b/tests/f_extent_leaf_bad_magic/expect.1
> @@ -2,8 +2,11 @@ Pass 1: Checking inodes, blocks, and sizes
> Inode 12 has an invalid extent node (blk 1604, lblk 0)
> Clear? yes
>
> +Inode 12 extent tree could be more compact. Fix? yes
> +
> Inode 12, i_blocks is 18, should be 0. Fix? yes
>
> +Pass 1E: Optimizing extent trees
> Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> diff --git a/tests/f_extent_oobounds/expect.1 b/tests/f_extent_oobounds/expect.1
> index 3164ea0..237829a 100644
> --- a/tests/f_extent_oobounds/expect.1
> +++ b/tests/f_extent_oobounds/expect.1
> @@ -3,8 +3,11 @@ Inode 12, end of extent exceeds allowed value
> (logical block 15, physical block 200, len 30)
> Clear? yes
>
> +Inode 12 extent tree could be more compact. Fix? yes
> +
> Inode 12, i_blocks is 154, should be 94. Fix? yes
>
> +Pass 1E: Optimizing extent trees
> Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> @@ -12,13 +15,13 @@ Pass 5: Checking group summary information
> Block bitmap differences: -(200--229)
> Fix? yes
>
> -Free blocks count wrong for group #0 (156, counted=186).
> +Free blocks count wrong for group #0 (158, counted=188).
> Fix? yes
>
> -Free blocks count wrong (156, counted=186).
> +Free blocks count wrong (158, counted=188).
> Fix? yes
>
>
> test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
> -test_filesys: 12/32 files (8.3% non-contiguous), 70/256 blocks
> +test_filesys: 12/32 files (8.3% non-contiguous), 68/256 blocks
> Exit status is 1
> diff --git a/tests/f_extent_oobounds/expect.2 b/tests/f_extent_oobounds/expect.2
> index 22c4f2c..0729283 100644
> --- a/tests/f_extent_oobounds/expect.2
> +++ b/tests/f_extent_oobounds/expect.2
> @@ -3,5 +3,5 @@ Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
> -test_filesys: 12/32 files (8.3% non-contiguous), 70/256 blocks
> +test_filesys: 12/32 files (8.3% non-contiguous), 68/256 blocks
> Exit status is 0
> diff --git a/tests/f_extents/expect.1 b/tests/f_extents/expect.1
> index aeebc7b..d682929 100644
> --- a/tests/f_extents/expect.1
> +++ b/tests/f_extents/expect.1
> @@ -6,6 +6,8 @@ Inode 12 has an invalid extent
> (logical block 0, invalid physical block 21994527527949, len 17)
> Clear? yes
>
> +Inode 12 extent tree could be more compact. Fix? yes
> +
> Inode 12, i_blocks is 34, should be 0. Fix? yes
>
> Inode 13 missing EXTENT_FL, but is in extents format
> @@ -21,6 +23,8 @@ Inode 17 has an invalid extent
> (logical block 0, invalid physical block 22011707397135, len 15)
> Clear? yes
>
> +Inode 17 extent tree could be more compact. Fix? yes
> +
> Inode 17, i_blocks is 32, should be 0. Fix? yes
>
> Error while reading over extent tree in inode 18: Corrupt extent header
> @@ -31,6 +35,7 @@ Inode 18, i_blocks is 2, should be 0. Fix? yes
> Special (device/socket/fifo) file (inode 19) has extents
> or inline-data flag set. Clear? yes
>
> +Pass 1E: Optimizing extent trees
> Pass 2: Checking directory structure
> Entry 'fbad-flag' in / (2) has deleted/unused inode 18. Clear? yes
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
Cheers, Andreas
^ permalink raw reply [flat|nested] 104+ messages in thread* Re: [PATCH 33/47] e2fsck: rebuild sparse extent trees/convert non-extent ext3 files
2014-12-04 23:20 ` Andreas Dilger
@ 2014-12-04 23:45 ` Darrick J. Wong
0 siblings, 0 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-12-04 23:45 UTC (permalink / raw)
To: Andreas Dilger; +Cc: tytso, linux-ext4
On Thu, Dec 04, 2014 at 04:20:12PM -0700, Andreas Dilger wrote:
> On Nov 7, 2014, at 2:54 PM, Darrick J. Wong <darrick.wong@oracle.com> wrote:
> >
> > Teach e2fsck to construct extent trees. This enables us to do either
> > of the following: compress a highly sparse extent tree into fewer ETB
> > blocks; or convert a ext3-style block mapped file to an extent file.
> >
> > For files that are already extent based, this algorithm will only run
> > if pass1 determines either (1) that a whole level of extent tree will
> > fit into a higher level of the tree; (2) that the size of any level
> > can be reduced by at least one ETB block; or (3) the extent tree is
> > unnecessarily deep. It will not run at all if errors are found and
> > the user declines to fix the errors.
> >
> > For block-mapped files, conversion only happens if the extent feature
> > is enabled and "-E bmap2extent" is passed to e2fsck. It will not run
> > at all if errors are left unfixed. After conversion, files larger
> > than 12 blocks should be defragmented to eliminate empty holes where a
> > block lives.
> >
> > The extent tree constructor is pretty dumb -- it creates a list of
> > leaf extents (adjacent extents are collapsed), marks all indirect
> > blocks / ETB blocks free, installs a new extent tree root in the
> > inode, then loads the leaf extents into the tree.
>
> Out of curiosity, when doing block-to-extent conversion, does it use
> the freed indirect blocks as index blocks to avoid holes in the file
> (and single unallocated blocks that would cause fragmentation), or
> does it allocate the index blocks at the start of the group according
> to keep index blocks together for fast access during e2fsck?
If you apply the two patches "libext2fs: find inode goal when allocating
blocks" and "libext2fs: set interior tree block goal more intelligently" then
it should pick up the freed indirect blocks for the extent blocks for faster
access during regular operation. Specifically, the second patch optimizes ETB
splits to try to find a block just prior to the first pblock of the first
extent, and the first patch makes the block allocator start looking for blocks
at the start of the group (instead of block 0) when it can't find any lblocks.
(On its own, the patch doesn't care and you'll get blocks starting at zero,
which is rather stupid.)
--D
>
> Cheers, Andreas
>
> >
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > ---
> > e2fsck/Makefile.in | 16 +
> > e2fsck/e2fsck.8.in | 3
> > e2fsck/e2fsck.c | 2
> > e2fsck/e2fsck.h | 11 +
> > e2fsck/extents.c | 352 ++++++++++++++++++++++++++++++++
> > e2fsck/pass1.c | 95 ++++++++-
> > e2fsck/problem.c | 43 ++++
> > e2fsck/problem.h | 28 +++
> > e2fsck/super.c | 7 +
> > e2fsck/unix.c | 4
> > tests/f_extent_bad_node/expect.1 | 9 +
> > tests/f_extent_bad_node/expect.2 | 2
> > tests/f_extent_int_bad_magic/expect.1 | 3
> > tests/f_extent_leaf_bad_magic/expect.1 | 3
> > tests/f_extent_oobounds/expect.1 | 9 +
> > tests/f_extent_oobounds/expect.2 | 2
> > tests/f_extents/expect.1 | 5
> > 17 files changed, 577 insertions(+), 17 deletions(-)
> > create mode 100644 e2fsck/extents.c
> >
> >
> > diff --git a/e2fsck/Makefile.in b/e2fsck/Makefile.in
> > index e40e51b..a4413d9 100644
> > --- a/e2fsck/Makefile.in
> > +++ b/e2fsck/Makefile.in
> > @@ -62,7 +62,8 @@ OBJS= dict.o unix.o e2fsck.o super.o pass1.o pass1b.o pass2.o \
> > pass3.o pass4.o pass5.o journal.o badblocks.o util.o dirinfo.o \
> > dx_dirinfo.o ehandler.o problem.o message.o quota.o recovery.o \
> > region.o revoke.o ea_refcount.o rehash.o profile.o prof_err.o \
> > - logfile.o sigcatcher.o $(MTRACE_OBJ) plausible.o readahead.o
> > + logfile.o sigcatcher.o $(MTRACE_OBJ) plausible.o readahead.o \
> > + extents.o
> >
> > PROFILED_OBJS= profiled/dict.o profiled/unix.o profiled/e2fsck.o \
> > profiled/super.o profiled/pass1.o profiled/pass1b.o \
> > @@ -74,7 +75,7 @@ PROFILED_OBJS= profiled/dict.o profiled/unix.o profiled/e2fsck.o \
> > profiled/ea_refcount.o profiled/rehash.o profiled/profile.o \
> > profiled/prof_err.o profiled/logfile.o \
> > profiled/sigcatcher.o profiled/plausible.o \
> > - profiled/sigcatcher.o profiled/readahead.o
> > + profiled/sigcatcher.o profiled/readahead.o profiled/extents.o
> >
> > SRCS= $(srcdir)/e2fsck.c \
> > $(srcdir)/dict.c \
> > @@ -106,6 +107,7 @@ SRCS= $(srcdir)/e2fsck.c \
> > prof_err.c \
> > $(srcdir)/quota.c \
> > $(srcdir)/../misc/plausible.c \
> > + $(srcdir)/extents.c \
> > $(MTRACE_SRC)
> >
> > all:: profiled $(PROGS) e2fsck $(MANPAGES) $(FMANPAGES)
> > @@ -308,6 +310,16 @@ pass1.o: $(srcdir)/pass1.c $(top_builddir)/lib/config.h \
> > $(srcdir)/profile.h prof_err.h $(top_srcdir)/lib/quota/quotaio.h \
> > $(top_srcdir)/lib/quota/dqblk_v2.h $(top_srcdir)/lib/quota/quotaio_tree.h \
> > $(top_srcdir)/lib/../e2fsck/dict.h $(srcdir)/problem.h
> > +extents.o: $(srcdir)/extents.c $(top_builddir)/lib/config.h \
> > + $(top_builddir)/lib/dirpaths.h $(top_srcdir)/lib/et/com_err.h \
> > + $(srcdir)/e2fsck.h $(top_srcdir)/lib/ext2fs/ext2_fs.h \
> > + $(top_builddir)/lib/ext2fs/ext2_types.h $(top_srcdir)/lib/ext2fs/ext2fs.h \
> > + $(top_srcdir)/lib/ext2fs/ext3_extents.h $(top_srcdir)/lib/ext2fs/ext2_io.h \
> > + $(top_builddir)/lib/ext2fs/ext2_err.h \
> > + $(top_srcdir)/lib/ext2fs/ext2_ext_attr.h $(top_srcdir)/lib/ext2fs/bitops.h \
> > + $(srcdir)/profile.h prof_err.h $(top_srcdir)/lib/quota/quotaio.h \
> > + $(top_srcdir)/lib/quota/dqblk_v2.h $(top_srcdir)/lib/quota/quotaio_tree.h \
> > + $(top_srcdir)/lib/../e2fsck/dict.h $(srcdir)/problem.h $(srcdir)/dict.h
> > pass1b.o: $(srcdir)/pass1b.c $(top_builddir)/lib/config.h \
> > $(top_builddir)/lib/dirpaths.h $(top_srcdir)/lib/et/com_err.h \
> > $(srcdir)/e2fsck.h $(top_srcdir)/lib/ext2fs/ext2_fs.h \
> > diff --git a/e2fsck/e2fsck.8.in b/e2fsck/e2fsck.8.in
> > index 84ae50f..0c2725e 100644
> > --- a/e2fsck/e2fsck.8.in
> > +++ b/e2fsck/e2fsck.8.in
> > @@ -214,6 +214,9 @@ e2fsck runtime. By default, this is set to the size of a block group's inode
> > table (typically 2MiB on a regular ext4 filesystem); if this amount is more
> > than 1/100 of total physical memory, readahead is disabled. Set this to zero
> > to disable readahead entirely.
> > +.TP
> > +.BI bmap2extent
> > +Convert block-mapped files to extent-mapped files.
> > .RE
> > .TP
> > .B \-f
> > diff --git a/e2fsck/e2fsck.c b/e2fsck/e2fsck.c
> > index fcda7d7..83506cb 100644
> > --- a/e2fsck/e2fsck.c
> > +++ b/e2fsck/e2fsck.c
> > @@ -204,7 +204,7 @@ void e2fsck_free_context(e2fsck_t ctx)
> > typedef void (*pass_t)(e2fsck_t ctx);
> >
> > static pass_t e2fsck_passes[] = {
> > - e2fsck_pass1, e2fsck_pass2, e2fsck_pass3, e2fsck_pass4,
> > + e2fsck_pass1, e2fsck_pass1e, e2fsck_pass2, e2fsck_pass3, e2fsck_pass4,
> > e2fsck_pass5, 0 };
> >
> > #define E2F_FLAG_RUN_RETURN (E2F_FLAG_SIGNAL_MASK|E2F_FLAG_RESTART)
> > diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
> > index e359515..66d71ee 100644
> > --- a/e2fsck/e2fsck.h
> > +++ b/e2fsck/e2fsck.h
> > @@ -167,6 +167,7 @@ struct resource_track {
> > #define E2F_OPT_FRAGCHECK 0x0800
> > #define E2F_OPT_JOURNAL_ONLY 0x1000 /* only replay the journal */
> > #define E2F_OPT_DISCARD 0x2000
> > +#define E2F_OPT_CONVERT_BMAP 0x4000 /* convert blockmap to extent */
> >
> > /*
> > * E2fsck flags
> > @@ -381,6 +382,11 @@ struct e2fsck_struct {
> >
> > /* How much are we allowed to readahead? */
> > unsigned long long readahead_kb;
> > +
> > + /*
> > + * Inodes to rebuild extent trees
> > + */
> > + ext2fs_inode_bitmap inodes_to_rebuild;
> > };
> >
> > /* Used by the region allocation code */
> > @@ -456,6 +462,11 @@ extern blk64_t ea_refcount_intr_next(ext2_refcount_t refcount, int *ret);
> > extern const char *ehandler_operation(const char *op);
> > extern void ehandler_init(io_channel channel);
> >
> > +/* extents.c */
> > +void e2fsck_rebuild_extents_later(e2fsck_t ctx, ext2_ino_t ino);
> > +int e2fsck_ino_will_be_rebuilt(e2fsck_t ctx, ext2_ino_t ino);
> > +void e2fsck_pass1e(e2fsck_t ctx);
> > +
> > /* journal.c */
> > extern errcode_t e2fsck_check_ext3_journal(e2fsck_t ctx);
> > extern errcode_t e2fsck_run_ext3_journal(e2fsck_t ctx);
> > diff --git a/e2fsck/extents.c b/e2fsck/extents.c
> > new file mode 100644
> > index 0000000..91d36d2
> > --- /dev/null
> > +++ b/e2fsck/extents.c
> > @@ -0,0 +1,352 @@
> > +/*
> > + * extents.c --- rebuild extent tree
> > + *
> > + * Copyright (C) 2014 Oracle.
> > + *
> > + * %Begin-Header%
> > + * This file may be redistributed under the terms of the GNU Public
> > + * License.
> > + * %End-Header%
> > + */
> > +
> > +#include "config.h"
> > +#include <string.h>
> > +#include <ctype.h>
> > +#include <errno.h>
> > +#include "e2fsck.h"
> > +#include "problem.h"
> > +
> > +#undef DEBUG
> > +#undef DEBUG_SUMMARY
> > +#undef DEBUG_FREE
> > +
> > +#define NUM_EXTENTS 341 /* about one ETB' worth of extents */
> > +
> > +/* Schedule an inode to have its extent tree rebuilt during pass 1E. */
> > +void e2fsck_rebuild_extents_later(e2fsck_t ctx, ext2_ino_t ino)
> > +{
> > + if (!EXT2_HAS_INCOMPAT_FEATURE(ctx->fs->super,
> > + EXT3_FEATURE_INCOMPAT_EXTENTS) ||
> > + (ctx->options & E2F_OPT_NO) ||
> > + (ino != EXT2_ROOT_INO && ino < ctx->fs->super->s_first_ino))
> > + return;
> > + if (!ctx->inodes_to_rebuild)
> > + e2fsck_allocate_inode_bitmap(ctx->fs,
> > + _("extent rebuild inode map"),
> > + EXT2FS_BMAP64_AUTODIR,
> > + "inodes_to_rebuild",
> > + &ctx->inodes_to_rebuild);
> > + if (ctx->inodes_to_rebuild)
> > + ext2fs_mark_inode_bitmap2(ctx->inodes_to_rebuild, ino);
> > +}
> > +
> > +/* Ask if an inode will have its extents rebuilt during pass 1E. */
> > +int e2fsck_ino_will_be_rebuilt(e2fsck_t ctx, ext2_ino_t ino)
> > +{
> > + if (!ctx->inodes_to_rebuild)
> > + return 0;
> > + return ext2fs_test_inode_bitmap2(ctx->inodes_to_rebuild, ino);
> > +}
> > +
> > +struct extent_list {
> > + blk64_t blocks_freed;
> > + struct ext2fs_extent *extents;
> > + unsigned int count;
> > + unsigned int size;
> > + unsigned int ext_read;
> > + errcode_t retval;
> > + ext2_ino_t ino;
> > +};
> > +
> > +static errcode_t load_extents(e2fsck_t ctx, struct extent_list *list)
> > +{
> > + ext2_filsys fs = ctx->fs;
> > + ext2_extent_handle_t handle;
> > + struct ext2fs_extent extent;
> > + errcode_t retval;
> > +
> > + retval = ext2fs_extent_open(fs, list->ino, &handle);
> > + if (retval)
> > + return retval;
> > +
> > + retval = ext2fs_extent_get(handle, EXT2_EXTENT_ROOT, &extent);
> > + if (retval)
> > + goto out;
> > +
> > + do {
> > + if (extent.e_flags & EXT2_EXTENT_FLAGS_SECOND_VISIT)
> > + goto next;
> > +
> > + /* Internal node; free it and we'll re-allocate it later */
> > + if (!(extent.e_flags & EXT2_EXTENT_FLAGS_LEAF)) {
> > +#if defined(DEBUG) || defined(DEBUG_FREE)
> > + printf("ino=%d free=%llu bf=%llu\n", list->ino,
> > + extent.e_pblk, list->blocks_freed + 1);
> > +#endif
> > + list->blocks_freed++;
> > + ext2fs_block_alloc_stats2(fs, extent.e_pblk, -1);
> > + goto next;
> > + }
> > +
> > + list->ext_read++;
> > + /* Can we attach it to the previous extent? */
> > + if (list->count) {
> > + struct ext2fs_extent *last = list->extents +
> > + list->count - 1;
> > + blk64_t end = last->e_len + extent.e_len;
> > +
> > + if (last->e_pblk + last->e_len == extent.e_pblk &&
> > + last->e_lblk + last->e_len == extent.e_lblk &&
> > + (last->e_flags & EXT2_EXTENT_FLAGS_UNINIT) ==
> > + (extent.e_flags & EXT2_EXTENT_FLAGS_UNINIT) &&
> > + end < (1ULL << 32)) {
> > + last->e_len += extent.e_len;
> > +#ifdef DEBUG
> > + printf("R: ino=%d len=%u\n", list->ino,
> > + last->e_len);
> > +#endif
> > + goto next;
> > + }
> > + }
> > +
> > + /* Do we need to expand? */
> > + if (list->count == list->size) {
> > + unsigned int new_size = (list->size + NUM_EXTENTS) *
> > + sizeof(struct ext2fs_extent);
> > + retval = ext2fs_resize_mem(0, new_size, &list->extents);
> > + if (retval)
> > + goto out;
> > + list->size += NUM_EXTENTS;
> > + }
> > +
> > + /* Add a new extent */
> > + memcpy(list->extents + list->count, &extent, sizeof(extent));
> > +#ifdef DEBUG
> > + printf("R: ino=%d pblk=%llu lblk=%llu len=%u\n", list->ino,
> > + extent.e_pblk, extent.e_lblk, extent.e_len);
> > +#endif
> > + list->count++;
> > +next:
> > + retval = ext2fs_extent_get(handle, EXT2_EXTENT_NEXT, &extent);
> > + } while (retval == 0);
> > +
> > +out:
> > + /* Ok if we run off the end */
> > + if (retval == EXT2_ET_EXTENT_NO_NEXT)
> > + retval = 0;
> > + ext2fs_extent_free(handle);
> > + return retval;
> > +}
> > +
> > +static int find_blocks(ext2_filsys fs, blk64_t *blocknr, e2_blkcnt_t blockcnt,
> > + blk64_t ref_blk, int ref_offset, void *priv_data)
> > +{
> > + struct extent_list *list = priv_data;
> > +
> > + /* Internal node? */
> > + if (blockcnt < 0) {
> > +#if defined(DEBUG) || defined(DEBUG_FREE)
> > + printf("ino=%d free=%llu bf=%llu\n", list->ino, *blocknr,
> > + list->blocks_freed + 1);
> > +#endif
> > + list->blocks_freed++;
> > + ext2fs_block_alloc_stats2(fs, *blocknr, -1);
> > + return 0;
> > + }
> > +
> > + /* Can we attach it to the previous extent? */
> > + if (list->count) {
> > + struct ext2fs_extent *last = list->extents +
> > + list->count - 1;
> > + blk64_t end = last->e_len + 1;
> > +
> > + if (last->e_pblk + last->e_len == *blocknr &&
> > + end < (1ULL << 32)) {
> > + last->e_len++;
> > +#ifdef DEBUG
> > + printf("R: ino=%d len=%u\n", list->ino, last->e_len);
> > +#endif
> > + return 0;
> > + }
> > + }
> > +
> > + /* Do we need to expand? */
> > + if (list->count == list->size) {
> > + unsigned int new_size = (list->size + NUM_EXTENTS) *
> > + sizeof(struct ext2fs_extent);
> > + list->retval = ext2fs_resize_mem(0, new_size, &list->extents);
> > + if (list->retval)
> > + return BLOCK_ABORT;
> > + list->size += NUM_EXTENTS;
> > + }
> > +
> > + /* Add a new extent */
> > + list->extents[list->count].e_pblk = *blocknr;
> > + list->extents[list->count].e_lblk = blockcnt;
> > + list->extents[list->count].e_len = 1;
> > + list->extents[list->count].e_flags = 0;
> > +#ifdef DEBUG
> > + printf("R: ino=%d pblk=%llu lblk=%llu len=%u\n", list->ino, *blocknr,
> > + blockcnt, 1);
> > +#endif
> > + list->count++;
> > +
> > + return 0;
> > +}
> > +
> > +static errcode_t rebuild_extent_tree(e2fsck_t ctx, struct extent_list *list,
> > + ext2_ino_t ino)
> > +{
> > + struct ext2_inode inode;
> > + errcode_t retval;
> > + ext2_extent_handle_t handle;
> > + unsigned int i, ext_written;
> > + struct ext2fs_extent *ex, extent;
> > +
> > + list->count = 0;
> > + list->blocks_freed = 0;
> > + list->ino = ino;
> > + list->ext_read = 0;
> > + e2fsck_read_inode(ctx, ino, &inode, "rebuild_extents");
> > +
> > + /* Collect lblk->pblk mappings */
> > + if (inode.i_flags & EXT4_EXTENTS_FL) {
> > + retval = load_extents(ctx, list);
> > + goto extents_loaded;
> > + }
> > +
> > + retval = ext2fs_block_iterate3(ctx->fs, ino, BLOCK_FLAG_READ_ONLY, 0,
> > + find_blocks, list);
> > + if (retval)
> > + goto err;
> > + if (list->retval) {
> > + retval = list->retval;
> > + goto err;
> > + }
> > +
> > +extents_loaded:
> > + /* Reset extent tree */
> > + inode.i_flags &= ~EXT4_EXTENTS_FL;
> > + memset(inode.i_block, 0, sizeof(inode.i_block));
> > +
> > + /* Make a note of freed blocks */
> > + retval = ext2fs_iblk_sub_blocks(ctx->fs, &inode, list->blocks_freed);
> > + if (retval)
> > + goto err;
> > +
> > + /* Now stuff extents into the file */
> > + retval = ext2fs_extent_open2(ctx->fs, ino, &inode, &handle);
> > + if (retval)
> > + goto err;
> > +
> > + ext_written = 0;
> > + for (i = 0, ex = list->extents; i < list->count; i++, ex++) {
> > + memcpy(&extent, ex, sizeof(struct ext2fs_extent));
> > + extent.e_flags &= EXT2_EXTENT_FLAGS_UNINIT;
> > + if (extent.e_flags & EXT2_EXTENT_FLAGS_UNINIT) {
> > + if (extent.e_len > EXT_UNINIT_MAX_LEN) {
> > + extent.e_len = EXT_UNINIT_MAX_LEN;
> > + ex->e_pblk += EXT_UNINIT_MAX_LEN;
> > + ex->e_lblk += EXT_UNINIT_MAX_LEN;
> > + ex->e_len -= EXT_UNINIT_MAX_LEN;
> > + ex--;
> > + i--;
> > + }
> > + } else {
> > + if (extent.e_len > EXT_INIT_MAX_LEN) {
> > + extent.e_len = EXT_INIT_MAX_LEN;
> > + ex->e_pblk += EXT_INIT_MAX_LEN;
> > + ex->e_lblk += EXT_INIT_MAX_LEN;
> > + ex->e_len -= EXT_INIT_MAX_LEN;
> > + ex--;
> > + i--;
> > + }
> > + }
> > +
> > +#ifdef DEBUG
> > + printf("W: ino=%d pblk=%llu lblk=%llu len=%u\n", ino,
> > + extent.e_pblk, extent.e_lblk, extent.e_len);
> > +#endif
> > + retval = ext2fs_extent_insert(handle, EXT2_EXTENT_INSERT_AFTER,
> > + &extent);
> > + if (retval)
> > + goto err2;
> > + retval = ext2fs_extent_fix_parents(handle);
> > + if (retval)
> > + goto err2;
> > + ext_written++;
> > + }
> > +
> > +#if defined(DEBUG) || defined(DEBUG_SUMMARY)
> > + printf("rebuild: ino=%d extents=%d->%d\n", ino, list->ext_read,
> > + ext_written);
> > +#endif
> > + e2fsck_write_inode(ctx, ino, &inode, "rebuild_extents");
> > +
> > +err2:
> > + ext2fs_extent_free(handle);
> > +err:
> > + return retval;
> > +}
> > +
> > +void e2fsck_pass1e(e2fsck_t ctx)
> > +{
> > + struct problem_context pctx;
> > +#ifdef RESOURCE_TRACK
> > + struct resource_track rtrack;
> > +#endif
> > + struct extent_list list;
> > + int first = 1;
> > + ext2_ino_t ino = 0;
> > + errcode_t retval;
> > +
> > + if (!EXT2_HAS_INCOMPAT_FEATURE(ctx->fs->super,
> > + EXT3_FEATURE_INCOMPAT_EXTENTS) ||
> > + !ext2fs_test_valid(ctx->fs) ||
> > + ctx->invalid_bitmaps) {
> > + if (ctx->inodes_to_rebuild)
> > + ext2fs_free_inode_bitmap(ctx->inodes_to_rebuild);
> > + ctx->inodes_to_rebuild = NULL;
> > + }
> > +
> > + if (ctx->inodes_to_rebuild == NULL)
> > + return;
> > +
> > + init_resource_track(&rtrack, ctx->fs->io);
> > + clear_problem_context(&pctx);
> > + e2fsck_read_bitmaps(ctx);
> > +
> > + memset(&list, 0, sizeof(list));
> > + retval = ext2fs_get_mem(sizeof(struct ext2fs_extent) * NUM_EXTENTS,
> > + &list.extents);
> > + list.size = NUM_EXTENTS;
> > + while (1) {
> > + retval = ext2fs_find_first_set_inode_bitmap2(
> > + ctx->inodes_to_rebuild, ino + 1,
> > + ctx->fs->super->s_inodes_count, &ino);
> > + if (retval)
> > + break;
> > + pctx.ino = ino;
> > + if (first) {
> > + fix_problem(ctx, PR_1E_PASS_HEADER, &pctx);
> > + first = 0;
> > + }
> > + pctx.errcode = rebuild_extent_tree(ctx, &list, ino);
> > + if (pctx.errcode) {
> > + end_problem_latch(ctx, PR_LATCH_OPTIMIZE_EXT);
> > + fix_problem(ctx, PR_1E_OPTIMIZE_EXT_ERR, &pctx);
> > + }
> > + if (ctx->progress && !ctx->progress_fd)
> > + e2fsck_simple_progress(ctx, "Rebuilding extents",
> > + 100.0 * (float) ino /
> > + (float) ctx->fs->super->s_inodes_count,
> > + ino);
> > + }
> > + end_problem_latch(ctx, PR_LATCH_OPTIMIZE_EXT);
> > +
> > + ext2fs_free_inode_bitmap(ctx->inodes_to_rebuild);
> > + ctx->inodes_to_rebuild = NULL;
> > + ext2fs_free_mem(&list.extents);
> > +
> > + print_resource_track(ctx, "Pass 1E", &rtrack, ctx->fs->io);
> > +}
> > diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
> > index a963849..10008d9 100644
> > --- a/e2fsck/pass1.c
> > +++ b/e2fsck/pass1.c
> > @@ -56,6 +56,8 @@
> > #define _INLINE_ inline
> > #endif
> >
> > +#undef DEBUG
> > +
> > static int process_block(ext2_filsys fs, blk64_t *blocknr,
> > e2_blkcnt_t blockcnt, blk64_t ref_blk,
> > int ref_offset, void *priv_data);
> > @@ -77,11 +79,16 @@ static void adjust_extattr_refcount(e2fsck_t ctx, ext2_refcount_t refcount,
> > char *block_buf, int adjust_sign);
> > /* static char *describe_illegal_block(ext2_filsys fs, blk64_t block); */
> >
> > +struct extent_info {
> > + unsigned int num_extents;
> > + unsigned int max_extents;
> > +};
> > +
> > struct process_block_struct {
> > ext2_ino_t ino;
> > unsigned is_dir:1, is_reg:1, clear:1, suppress:1,
> > fragmented:1, compressed:1, bbcheck:1,
> > - inode_modified:1;
> > + inode_modified:1, extent_rebuild:1;
> > blk64_t num_blocks;
> > blk64_t max_blocks;
> > e2_blkcnt_t last_block;
> > @@ -95,6 +102,7 @@ struct process_block_struct {
> > e2fsck_t ctx;
> > blk64_t bad_ref;
> > region_t region;
> > + struct extent_info ext_info[MAX_EXTENT_DEPTH_COUNT];
> > };
> >
> > struct process_inode_block {
> > @@ -2402,6 +2410,53 @@ static int has_unaligned_cluster_map(e2fsck_t ctx,
> > return 0;
> > }
> >
> > +static void should_rebuild_extents(e2fsck_t ctx,
> > + struct problem_context *pctx,
> > + struct process_block_struct *pb,
> > + struct ext2_extent_info *info)
> > +{
> > + struct extent_info *ei;
> > + int i, j;
> > + unsigned int extents_per_block;
> > +
> > + if (pb->extent_rebuild)
> > + goto rebuild;
> > +
> > + extents_per_block = (ctx->fs->blocksize -
> > + sizeof(struct ext3_extent_header)) /
> > + sizeof(struct ext3_extent);
> > + /*
> > + * If we can consolidate a level or shorten the tree, schedule the
> > + * extent tree to be rebuilt.
> > + */
> > + for (i = 0, ei = pb->ext_info; i < info->max_depth + 1; i++, ei++) {
> > + if (ei->max_extents - ei->num_extents > extents_per_block) {
> > +#ifdef DEBUG
> > + printf("rebuild extents, ino=%d level=%d slack=%d epb=%d\n",
> > + pb->ino, i,
> > + ei->max_extents - ei->num_extents,
> > + extents_per_block);
> > +#endif
> > + goto rebuild;
> > + }
> > + for (j = 0; j < i; j++) {
> > + if (ei->num_extents < pb->ext_info[j].max_extents) {
> > +#ifdef DEBUG
> > + printf("rebuild extents, ino=%d level=%d num=%d level=%d\n",
> > + pb->ino, i, ei->num_extents, j);
> > +#endif
> > + goto rebuild;
> > + }
> > + }
> > + }
> > + return;
> > +
> > +rebuild:
> > + if (pb->extent_rebuild ||
> > + fix_problem(ctx, PR_1E_CAN_COMPRESS_EXTENT_TREE, pctx))
> > + e2fsck_rebuild_extents_later(ctx, pb->ino);
> > +}
> > +
> > static void scan_extent_node(e2fsck_t ctx, struct problem_context *pctx,
> > struct process_block_struct *pb,
> > blk64_t start_block, blk64_t end_block,
> > @@ -2424,6 +2479,10 @@ static void scan_extent_node(e2fsck_t ctx, struct problem_context *pctx,
> > pctx->errcode = ext2fs_extent_get_info(ehandle, &info);
> > if (pctx->errcode)
> > return;
> > + if (!pb->extent_rebuild) {
> > + pb->ext_info[info.curr_level].num_extents += info.num_entries;
> > + pb->ext_info[info.curr_level].max_extents += info.max_entries;
> > + }
> >
> > pctx->errcode = ext2fs_extent_get(ehandle, EXT2_EXTENT_FIRST_SIB,
> > &extent);
> > @@ -2760,17 +2819,31 @@ static void check_blocks_extents(e2fsck_t ctx, struct problem_context *pctx,
> >
> > retval = ext2fs_extent_get_info(ehandle, &info);
> > if (retval == 0) {
> > - if (info.max_depth >= MAX_EXTENT_DEPTH_COUNT)
> > - info.max_depth = MAX_EXTENT_DEPTH_COUNT-1;
> > - ctx->extent_depth_count[info.max_depth]++;
> > + int max_depth = info.max_depth;
> > +
> > + if (max_depth >= MAX_EXTENT_DEPTH_COUNT)
> > + max_depth = MAX_EXTENT_DEPTH_COUNT-1;
> > + ctx->extent_depth_count[max_depth]++;
> > }
> >
> > + /* Check maximum extent depth */
> > + pctx->blk = info.max_depth;
> > + pctx->blk2 = ext2fs_max_extent_depth(ehandle);
> > + if (pctx->blk2 < pctx->blk &&
> > + fix_problem(ctx, PR_1_EXTENT_BAD_MAX_DEPTH, pctx))
> > + pb->extent_rebuild = 1;
> > +
> > + /* Can we collect extent tree level stats? */
> > + pctx->blk = MAX_EXTENT_DEPTH_COUNT;
> > + if (pctx->blk2 > pctx->blk)
> > + fix_problem(ctx, PR_1E_MAX_EXTENT_TREE_DEPTH, pctx);
> > + memset(pb->ext_info, 0, sizeof(pb->ext_info));
> > +
> > pb->region = region_create(0, info.max_lblk);
> > if (!pb->region) {
> > - ext2fs_extent_free(ehandle);
> > fix_problem(ctx, PR_1_EXTENT_ALLOC_REGION_ABORT, pctx);
> > ctx->flags |= E2F_FLAG_ABORT;
> > - return;
> > + goto out;
> > }
> >
> > eof_lblk = ((EXT2_I_SIZE(inode) + fs->blocksize - 1) >>
> > @@ -2786,7 +2859,9 @@ static void check_blocks_extents(e2fsck_t ctx, struct problem_context *pctx,
> > }
> > region_free(pb->region);
> > pb->region = NULL;
> > +out:
> > ext2fs_extent_free(ehandle);
> > + should_rebuild_extents(ctx, pctx, pb, &info);
> > }
> >
> > /*
> > @@ -2846,6 +2921,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
> > pb.ctx = ctx;
> > pb.inode_modified = 0;
> > pb.bad_ref = 0;
> > + pb.extent_rebuild = 0;
> > pctx->ino = ino;
> > pctx->errcode = 0;
> >
> > @@ -2909,6 +2985,13 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
> > "check_blocks");
> > fs->flags = (flags & EXT2_FLAG_IGNORE_CSUM_ERRORS) |
> > (fs->flags & ~EXT2_FLAG_IGNORE_CSUM_ERRORS);
> > +
> > + if (ctx->options & E2F_OPT_CONVERT_BMAP) {
> > +#ifdef DEBUG
> > + printf("bmap rebuild ino=%d\n", ino);
> > +#endif
> > + e2fsck_rebuild_extents_later(ctx, ino);
> > + }
> > }
> > }
> > end_problem_latch(ctx, PR_LATCH_BLOCK);
> > diff --git a/e2fsck/problem.c b/e2fsck/problem.c
> > index a4da64b..75e0305 100644
> > --- a/e2fsck/problem.c
> > +++ b/e2fsck/problem.c
> > @@ -1101,6 +1101,11 @@ static struct e2fsck_problem problem_table[] = {
> > N_("@i %i has a duplicate @x mapping\n\t(logical @b %c, @n physical @b %b, len %N)\n"),
> > PROMPT_CLEAR, 0 },
> >
> > + /* Inode extent tree could be more shallow */
> > + { PR_1_EXTENT_BAD_MAX_DEPTH,
> > + N_("@i %i @x tree could be more shallow (%b; could be <= %c)\n"),
> > + PROMPT_FIX, PR_NO_OK | PR_PREEN_NO | PR_PREEN_OK },
> > +
> > /* Pass 1b errors */
> >
> > /* Pass 1B: Rescan for duplicate/bad blocks */
> > @@ -1198,6 +1203,43 @@ static struct e2fsck_problem problem_table[] = {
> > { PR_1D_CLONE_ERROR,
> > N_("Couldn't clone file: %m\n"), PROMPT_NONE, 0 },
> >
> > + /* Pass 1E Extent tree Optimization */
> > +
> > + /* Pass 1E: Optimizing extent trees */
> > + { PR_1E_PASS_HEADER,
> > + N_("Pass 1E: Optimizing @x trees\n"),
> > + PROMPT_NONE, PR_PREEN_NOMSG },
> > +
> > + /* Failed to optimize extent tree */
> > + { PR_1E_OPTIMIZE_EXT_ERR,
> > + N_("Failed to optimize @x tree %p (%i): %m\n"),
> > + PROMPT_NONE, 0 },
> > +
> > + /* Rebuilding extent trees */
> > + { PR_1E_OPTIMIZE_EXT_HEADER,
> > + N_("Optimizing @x trees: "),
> > + PROMPT_NONE, PR_MSG_ONLY },
> > +
> > + /* Rebuilding extent tree %d */
> > + { PR_1E_OPTIMIZE_EXT,
> > + " %i",
> > + PROMPT_NONE, PR_LATCH_OPTIMIZE_EXT | PR_PREEN_NOHDR},
> > +
> > + /* Rebuilding extent tree end */
> > + { PR_1E_OPTIMIZE_EXT_END,
> > + "\n",
> > + PROMPT_NONE, PR_PREEN_NOHDR },
> > +
> > + /* Internal error: extent tree depth too large */
> > + { PR_1E_MAX_EXTENT_TREE_DEPTH,
> > + N_("Internal error: max extent tree depth too large (%b; expected=%c).\n"),
> > + PROMPT_NONE, PR_FATAL },
> > +
> > + /* Inode extent tree could be more compact */
> > + { PR_1E_CAN_COMPRESS_EXTENT_TREE,
> > + N_("@i %i @x tree could be more compact. "),
> > + PROMPT_FIX, PR_NO_OK | PR_PREEN_NO | PR_PREEN_OK },
> > +
> > /* Pass 2 errors */
> >
> > /* Pass 2: Checking directory structure */
> > @@ -1946,6 +1988,7 @@ static struct latch_descr pr_latch_info[] = {
> > { PR_LATCH_TOOBIG, PR_1_INODE_TOOBIG, 0 },
> > { PR_LATCH_OPTIMIZE_DIR, PR_3A_OPTIMIZE_DIR_HEADER, PR_3A_OPTIMIZE_DIR_END },
> > { PR_LATCH_BG_CHECKSUM, PR_0_GDT_CSUM_LATCH, 0 },
> > + { PR_LATCH_OPTIMIZE_EXT, PR_1E_OPTIMIZE_EXT_HEADER, PR_1E_OPTIMIZE_EXT_END },
> > { -1, 0, 0 },
> > };
> >
> > diff --git a/e2fsck/problem.h b/e2fsck/problem.h
> > index 3c28166..d3dcc9e 100644
> > --- a/e2fsck/problem.h
> > +++ b/e2fsck/problem.h
> > @@ -40,6 +40,7 @@ struct problem_context {
> > #define PR_LATCH_TOOBIG 0x0080 /* Latch for file to big errors */
> > #define PR_LATCH_OPTIMIZE_DIR 0x0090 /* Latch for optimize directories */
> > #define PR_LATCH_BG_CHECKSUM 0x00A0 /* Latch for block group checksums */
> > +#define PR_LATCH_OPTIMIZE_EXT 0x00B0 /* Latch for rebuild extents */
> >
> > #define PR_LATCH(x) ((((x) & PR_LATCH_MASK) >> 4) - 1)
> >
> > @@ -641,6 +642,9 @@ struct problem_context {
> > /* leaf extent collision */
> > #define PR_1_EXTENT_COLLISION 0x01007D
> >
> > +/* extent tree max depth too big */
> > +#define PR_1_EXTENT_BAD_MAX_DEPTH 0x01007E
> > +
> > /*
> > * Pass 1b errors
> > */
> > @@ -704,6 +708,30 @@ struct problem_context {
> > #define PR_1D_CLONE_ERROR 0x013008
> >
> > /*
> > + * Pass 1e --- rebuilding extent trees
> > + */
> > +/* Pass 1e: Rebuilding extent trees */
> > +#define PR_1E_PASS_HEADER 0x014000
> > +
> > +/* Error rehash directory */
> > +#define PR_1E_OPTIMIZE_EXT_ERR 0x014001
> > +
> > +/* Rebuilding extent trees */
> > +#define PR_1E_OPTIMIZE_EXT_HEADER 0x014002
> > +
> > +/* Rebuilding extent %d */
> > +#define PR_1E_OPTIMIZE_EXT 0x014003
> > +
> > +/* Rebuilding extent tree end */
> > +#define PR_1E_OPTIMIZE_EXT_END 0x014004
> > +
> > +/* Internal error: extent tree depth too large */
> > +#define PR_1E_MAX_EXTENT_TREE_DEPTH 0x014005
> > +
> > +/* Inode extent tree could be more compact */
> > +#define PR_1E_CAN_COMPRESS_EXTENT_TREE 0x014006
> > +
> > +/*
> > * Pass 2 errors
> > */
> >
> > diff --git a/e2fsck/super.c b/e2fsck/super.c
> > index 1e7e749..e64262a 100644
> > --- a/e2fsck/super.c
> > +++ b/e2fsck/super.c
> > @@ -606,6 +606,13 @@ void check_super_block(e2fsck_t ctx)
> > ext2fs_mark_super_dirty(fs);
> > }
> >
> > + /* Did user ask us to convert files to extents? */
> > + if (ctx->options & E2F_OPT_CONVERT_BMAP) {
> > + fs->super->s_feature_incompat |=
> > + EXT3_FEATURE_INCOMPAT_EXTENTS;
> > + ext2fs_mark_super_dirty(fs);
> > + }
> > +
> > if ((fs->super->s_feature_incompat & EXT2_FEATURE_INCOMPAT_META_BG) &&
> > (fs->super->s_first_meta_bg > fs->desc_blocks)) {
> > pctx.group = fs->desc_blocks;
> > diff --git a/e2fsck/unix.c b/e2fsck/unix.c
> > index f3672c0..fe5127a 100644
> > --- a/e2fsck/unix.c
> > +++ b/e2fsck/unix.c
> > @@ -709,6 +709,9 @@ static void parse_extended_opts(e2fsck_t ctx, const char *opts)
> > else
> > ctx->log_fn = string_copy(ctx, arg, 0);
> > continue;
> > + } else if (strcmp(token, "bmap2extent") == 0) {
> > + ctx->options |= E2F_OPT_CONVERT_BMAP;
> > + continue;
> > } else {
> > fprintf(stderr, _("Unknown extended option: %s\n"),
> > token);
> > @@ -728,6 +731,7 @@ static void parse_extended_opts(e2fsck_t ctx, const char *opts)
> > fputs(("\tdiscard\n"), stderr);
> > fputs(("\tnodiscard\n"), stderr);
> > fputs(("\treadahead_kb=<buffer size>\n"), stderr);
> > + fputs(("\tbmap2extent\n"), stderr);
> > fputc('\n', stderr);
> > exit(1);
> > }
> > diff --git a/tests/f_extent_bad_node/expect.1 b/tests/f_extent_bad_node/expect.1
> > index 0c0bc28..c13ad39 100644
> > --- a/tests/f_extent_bad_node/expect.1
> > +++ b/tests/f_extent_bad_node/expect.1
> > @@ -2,8 +2,11 @@ Pass 1: Checking inodes, blocks, and sizes
> > Inode 12 has an invalid extent node (blk 22, lblk 0)
> > Clear? yes
> >
> > +Inode 12 extent tree could be more compact. Fix? yes
> > +
> > Inode 12, i_blocks is 16, should be 8. Fix? yes
> >
> > +Pass 1E: Optimizing extent trees
> > Pass 2: Checking directory structure
> > Pass 3: Checking directory connectivity
> > Pass 4: Checking reference counts
> > @@ -11,13 +14,13 @@ Pass 5: Checking group summary information
> > Block bitmap differences: -(21--23) -25
> > Fix? yes
> >
> > -Free blocks count wrong for group #0 (71, counted=75).
> > +Free blocks count wrong for group #0 (73, counted=77).
> > Fix? yes
> >
> > -Free blocks count wrong (71, counted=75).
> > +Free blocks count wrong (73, counted=77).
> > Fix? yes
> >
> >
> > test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
> > -test_filesys: 12/16 files (0.0% non-contiguous), 25/100 blocks
> > +test_filesys: 12/16 files (0.0% non-contiguous), 23/100 blocks
> > Exit status is 1
> > diff --git a/tests/f_extent_bad_node/expect.2 b/tests/f_extent_bad_node/expect.2
> > index 568c792..b78b193 100644
> > --- a/tests/f_extent_bad_node/expect.2
> > +++ b/tests/f_extent_bad_node/expect.2
> > @@ -3,5 +3,5 @@ Pass 2: Checking directory structure
> > Pass 3: Checking directory connectivity
> > Pass 4: Checking reference counts
> > Pass 5: Checking group summary information
> > -test_filesys: 12/16 files (0.0% non-contiguous), 25/100 blocks
> > +test_filesys: 12/16 files (0.0% non-contiguous), 23/100 blocks
> > Exit status is 0
> > diff --git a/tests/f_extent_int_bad_magic/expect.1 b/tests/f_extent_int_bad_magic/expect.1
> > index 0e82e2b..0bd163f 100644
> > --- a/tests/f_extent_int_bad_magic/expect.1
> > +++ b/tests/f_extent_int_bad_magic/expect.1
> > @@ -2,8 +2,11 @@ Pass 1: Checking inodes, blocks, and sizes
> > Inode 12 has an invalid extent node (blk 1295, lblk 0)
> > Clear? yes
> >
> > +Inode 12 extent tree could be more compact. Fix? yes
> > +
> > Inode 12, i_blocks is 712, should be 0. Fix? yes
> >
> > +Pass 1E: Optimizing extent trees
> > Pass 2: Checking directory structure
> > Pass 3: Checking directory connectivity
> > Pass 4: Checking reference counts
> > diff --git a/tests/f_extent_leaf_bad_magic/expect.1 b/tests/f_extent_leaf_bad_magic/expect.1
> > index 7b6dbf1..c31a309 100644
> > --- a/tests/f_extent_leaf_bad_magic/expect.1
> > +++ b/tests/f_extent_leaf_bad_magic/expect.1
> > @@ -2,8 +2,11 @@ Pass 1: Checking inodes, blocks, and sizes
> > Inode 12 has an invalid extent node (blk 1604, lblk 0)
> > Clear? yes
> >
> > +Inode 12 extent tree could be more compact. Fix? yes
> > +
> > Inode 12, i_blocks is 18, should be 0. Fix? yes
> >
> > +Pass 1E: Optimizing extent trees
> > Pass 2: Checking directory structure
> > Pass 3: Checking directory connectivity
> > Pass 4: Checking reference counts
> > diff --git a/tests/f_extent_oobounds/expect.1 b/tests/f_extent_oobounds/expect.1
> > index 3164ea0..237829a 100644
> > --- a/tests/f_extent_oobounds/expect.1
> > +++ b/tests/f_extent_oobounds/expect.1
> > @@ -3,8 +3,11 @@ Inode 12, end of extent exceeds allowed value
> > (logical block 15, physical block 200, len 30)
> > Clear? yes
> >
> > +Inode 12 extent tree could be more compact. Fix? yes
> > +
> > Inode 12, i_blocks is 154, should be 94. Fix? yes
> >
> > +Pass 1E: Optimizing extent trees
> > Pass 2: Checking directory structure
> > Pass 3: Checking directory connectivity
> > Pass 4: Checking reference counts
> > @@ -12,13 +15,13 @@ Pass 5: Checking group summary information
> > Block bitmap differences: -(200--229)
> > Fix? yes
> >
> > -Free blocks count wrong for group #0 (156, counted=186).
> > +Free blocks count wrong for group #0 (158, counted=188).
> > Fix? yes
> >
> > -Free blocks count wrong (156, counted=186).
> > +Free blocks count wrong (158, counted=188).
> > Fix? yes
> >
> >
> > test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
> > -test_filesys: 12/32 files (8.3% non-contiguous), 70/256 blocks
> > +test_filesys: 12/32 files (8.3% non-contiguous), 68/256 blocks
> > Exit status is 1
> > diff --git a/tests/f_extent_oobounds/expect.2 b/tests/f_extent_oobounds/expect.2
> > index 22c4f2c..0729283 100644
> > --- a/tests/f_extent_oobounds/expect.2
> > +++ b/tests/f_extent_oobounds/expect.2
> > @@ -3,5 +3,5 @@ Pass 2: Checking directory structure
> > Pass 3: Checking directory connectivity
> > Pass 4: Checking reference counts
> > Pass 5: Checking group summary information
> > -test_filesys: 12/32 files (8.3% non-contiguous), 70/256 blocks
> > +test_filesys: 12/32 files (8.3% non-contiguous), 68/256 blocks
> > Exit status is 0
> > diff --git a/tests/f_extents/expect.1 b/tests/f_extents/expect.1
> > index aeebc7b..d682929 100644
> > --- a/tests/f_extents/expect.1
> > +++ b/tests/f_extents/expect.1
> > @@ -6,6 +6,8 @@ Inode 12 has an invalid extent
> > (logical block 0, invalid physical block 21994527527949, len 17)
> > Clear? yes
> >
> > +Inode 12 extent tree could be more compact. Fix? yes
> > +
> > Inode 12, i_blocks is 34, should be 0. Fix? yes
> >
> > Inode 13 missing EXTENT_FL, but is in extents format
> > @@ -21,6 +23,8 @@ Inode 17 has an invalid extent
> > (logical block 0, invalid physical block 22011707397135, len 15)
> > Clear? yes
> >
> > +Inode 17 extent tree could be more compact. Fix? yes
> > +
> > Inode 17, i_blocks is 32, should be 0. Fix? yes
> >
> > Error while reading over extent tree in inode 18: Corrupt extent header
> > @@ -31,6 +35,7 @@ Inode 18, i_blocks is 2, should be 0. Fix? yes
> > Special (device/socket/fifo) file (inode 19) has extents
> > or inline-data flag set. Clear? yes
> >
> > +Pass 1E: Optimizing extent trees
> > Pass 2: Checking directory structure
> > Entry 'fbad-flag' in / (2) has deleted/unused inode 18. Clear? yes
> >
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
> Cheers, Andreas
>
>
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 104+ messages in thread
* [PATCH v2 33/47] e2fsck: rebuild sparse extent trees/convert non-extent ext3 files
2014-11-07 21:54 ` [PATCH 33/47] e2fsck: rebuild sparse extent trees/convert non-extent ext3 files Darrick J. Wong
2014-12-04 23:20 ` Andreas Dilger
@ 2014-12-11 22:05 ` Darrick J. Wong
1 sibling, 0 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-12-11 22:05 UTC (permalink / raw)
To: tytso, Andreas Dilger; +Cc: linux-ext4
Teach e2fsck to construct extent trees. This enables us to do either
of the following: compress a highly sparse extent tree into fewer ETB
blocks; or convert a ext3-style block mapped file to an extent file.
For files that are already extent based, this algorithm will only run
if pass1 determines either (1) that a whole level of extent tree will
fit into a higher level of the tree; (2) that the size of any level
can be reduced by at least one ETB block; or (3) the extent tree is
unnecessarily deep. It will not run at all if errors are found and
the user declines to fix the errors.
For block-mapped files, conversion only happens if the extent feature
is enabled and "-E bmap2extent" is passed to e2fsck. It will not run
at all if errors are left unfixed. After conversion, files larger
than 12 blocks should be defragmented to eliminate empty holes where a
block lives.
The extent tree constructor is pretty dumb -- it creates a list of
leaf extents (adjacent extents are collapsed), marks all indirect
blocks / ETB blocks free, installs a new extent tree root in the
inode, then loads the leaf extents into the tree.
v2: Account for extent tree block slack that we create when splitting
a block, so that we don't repeatedly annoy the user to rebuild a tree
that we can't optimize further.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
e2fsck/Makefile.in | 16 +
e2fsck/e2fsck.8.in | 3
e2fsck/e2fsck.c | 2
e2fsck/e2fsck.h | 11 +
e2fsck/extents.c | 352 ++++++++++++++++++++++++++++++++
e2fsck/pass1.c | 104 +++++++++
e2fsck/problem.c | 43 ++++
e2fsck/problem.h | 28 +++
e2fsck/super.c | 7 +
e2fsck/unix.c | 4
tests/f_extent_bad_node/expect.1 | 9 +
tests/f_extent_bad_node/expect.2 | 2
tests/f_extent_int_bad_magic/expect.1 | 3
tests/f_extent_leaf_bad_magic/expect.1 | 3
tests/f_extent_oobounds/expect.1 | 9 +
tests/f_extent_oobounds/expect.2 | 2
tests/f_extents/expect.1 | 5
17 files changed, 586 insertions(+), 17 deletions(-)
create mode 100644 e2fsck/extents.c
diff --git a/e2fsck/Makefile.in b/e2fsck/Makefile.in
index e40e51b..a4413d9 100644
--- a/e2fsck/Makefile.in
+++ b/e2fsck/Makefile.in
@@ -62,7 +62,8 @@ OBJS= dict.o unix.o e2fsck.o super.o pass1.o pass1b.o pass2.o \
pass3.o pass4.o pass5.o journal.o badblocks.o util.o dirinfo.o \
dx_dirinfo.o ehandler.o problem.o message.o quota.o recovery.o \
region.o revoke.o ea_refcount.o rehash.o profile.o prof_err.o \
- logfile.o sigcatcher.o $(MTRACE_OBJ) plausible.o readahead.o
+ logfile.o sigcatcher.o $(MTRACE_OBJ) plausible.o readahead.o \
+ extents.o
PROFILED_OBJS= profiled/dict.o profiled/unix.o profiled/e2fsck.o \
profiled/super.o profiled/pass1.o profiled/pass1b.o \
@@ -74,7 +75,7 @@ PROFILED_OBJS= profiled/dict.o profiled/unix.o profiled/e2fsck.o \
profiled/ea_refcount.o profiled/rehash.o profiled/profile.o \
profiled/prof_err.o profiled/logfile.o \
profiled/sigcatcher.o profiled/plausible.o \
- profiled/sigcatcher.o profiled/readahead.o
+ profiled/sigcatcher.o profiled/readahead.o profiled/extents.o
SRCS= $(srcdir)/e2fsck.c \
$(srcdir)/dict.c \
@@ -106,6 +107,7 @@ SRCS= $(srcdir)/e2fsck.c \
prof_err.c \
$(srcdir)/quota.c \
$(srcdir)/../misc/plausible.c \
+ $(srcdir)/extents.c \
$(MTRACE_SRC)
all:: profiled $(PROGS) e2fsck $(MANPAGES) $(FMANPAGES)
@@ -308,6 +310,16 @@ pass1.o: $(srcdir)/pass1.c $(top_builddir)/lib/config.h \
$(srcdir)/profile.h prof_err.h $(top_srcdir)/lib/quota/quotaio.h \
$(top_srcdir)/lib/quota/dqblk_v2.h $(top_srcdir)/lib/quota/quotaio_tree.h \
$(top_srcdir)/lib/../e2fsck/dict.h $(srcdir)/problem.h
+extents.o: $(srcdir)/extents.c $(top_builddir)/lib/config.h \
+ $(top_builddir)/lib/dirpaths.h $(top_srcdir)/lib/et/com_err.h \
+ $(srcdir)/e2fsck.h $(top_srcdir)/lib/ext2fs/ext2_fs.h \
+ $(top_builddir)/lib/ext2fs/ext2_types.h $(top_srcdir)/lib/ext2fs/ext2fs.h \
+ $(top_srcdir)/lib/ext2fs/ext3_extents.h $(top_srcdir)/lib/ext2fs/ext2_io.h \
+ $(top_builddir)/lib/ext2fs/ext2_err.h \
+ $(top_srcdir)/lib/ext2fs/ext2_ext_attr.h $(top_srcdir)/lib/ext2fs/bitops.h \
+ $(srcdir)/profile.h prof_err.h $(top_srcdir)/lib/quota/quotaio.h \
+ $(top_srcdir)/lib/quota/dqblk_v2.h $(top_srcdir)/lib/quota/quotaio_tree.h \
+ $(top_srcdir)/lib/../e2fsck/dict.h $(srcdir)/problem.h $(srcdir)/dict.h
pass1b.o: $(srcdir)/pass1b.c $(top_builddir)/lib/config.h \
$(top_builddir)/lib/dirpaths.h $(top_srcdir)/lib/et/com_err.h \
$(srcdir)/e2fsck.h $(top_srcdir)/lib/ext2fs/ext2_fs.h \
diff --git a/e2fsck/e2fsck.8.in b/e2fsck/e2fsck.8.in
index 84ae50f..0c2725e 100644
--- a/e2fsck/e2fsck.8.in
+++ b/e2fsck/e2fsck.8.in
@@ -214,6 +214,9 @@ e2fsck runtime. By default, this is set to the size of a block group's inode
table (typically 2MiB on a regular ext4 filesystem); if this amount is more
than 1/100 of total physical memory, readahead is disabled. Set this to zero
to disable readahead entirely.
+.TP
+.BI bmap2extent
+Convert block-mapped files to extent-mapped files.
.RE
.TP
.B \-f
diff --git a/e2fsck/e2fsck.c b/e2fsck/e2fsck.c
index fcda7d7..83506cb 100644
--- a/e2fsck/e2fsck.c
+++ b/e2fsck/e2fsck.c
@@ -204,7 +204,7 @@ void e2fsck_free_context(e2fsck_t ctx)
typedef void (*pass_t)(e2fsck_t ctx);
static pass_t e2fsck_passes[] = {
- e2fsck_pass1, e2fsck_pass2, e2fsck_pass3, e2fsck_pass4,
+ e2fsck_pass1, e2fsck_pass1e, e2fsck_pass2, e2fsck_pass3, e2fsck_pass4,
e2fsck_pass5, 0 };
#define E2F_FLAG_RUN_RETURN (E2F_FLAG_SIGNAL_MASK|E2F_FLAG_RESTART)
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index e359515..66d71ee 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -167,6 +167,7 @@ struct resource_track {
#define E2F_OPT_FRAGCHECK 0x0800
#define E2F_OPT_JOURNAL_ONLY 0x1000 /* only replay the journal */
#define E2F_OPT_DISCARD 0x2000
+#define E2F_OPT_CONVERT_BMAP 0x4000 /* convert blockmap to extent */
/*
* E2fsck flags
@@ -381,6 +382,11 @@ struct e2fsck_struct {
/* How much are we allowed to readahead? */
unsigned long long readahead_kb;
+
+ /*
+ * Inodes to rebuild extent trees
+ */
+ ext2fs_inode_bitmap inodes_to_rebuild;
};
/* Used by the region allocation code */
@@ -456,6 +462,11 @@ extern blk64_t ea_refcount_intr_next(ext2_refcount_t refcount, int *ret);
extern const char *ehandler_operation(const char *op);
extern void ehandler_init(io_channel channel);
+/* extents.c */
+void e2fsck_rebuild_extents_later(e2fsck_t ctx, ext2_ino_t ino);
+int e2fsck_ino_will_be_rebuilt(e2fsck_t ctx, ext2_ino_t ino);
+void e2fsck_pass1e(e2fsck_t ctx);
+
/* journal.c */
extern errcode_t e2fsck_check_ext3_journal(e2fsck_t ctx);
extern errcode_t e2fsck_run_ext3_journal(e2fsck_t ctx);
diff --git a/e2fsck/extents.c b/e2fsck/extents.c
new file mode 100644
index 0000000..a9d8e3c
--- /dev/null
+++ b/e2fsck/extents.c
@@ -0,0 +1,352 @@
+/*
+ * extents.c --- rebuild extent tree
+ *
+ * Copyright (C) 2014 Oracle.
+ *
+ * %Begin-Header%
+ * This file may be redistributed under the terms of the GNU Public
+ * License.
+ * %End-Header%
+ */
+
+#include "config.h"
+#include <string.h>
+#include <ctype.h>
+#include <errno.h>
+#include "e2fsck.h"
+#include "problem.h"
+
+#undef DEBUG
+#undef DEBUG_SUMMARY
+#undef DEBUG_FREE
+
+#define NUM_EXTENTS 341 /* about one ETB' worth of extents */
+
+/* Schedule an inode to have its extent tree rebuilt during pass 1E. */
+void e2fsck_rebuild_extents_later(e2fsck_t ctx, ext2_ino_t ino)
+{
+ if (!EXT2_HAS_INCOMPAT_FEATURE(ctx->fs->super,
+ EXT3_FEATURE_INCOMPAT_EXTENTS) ||
+ (ctx->options & E2F_OPT_NO) ||
+ (ino != EXT2_ROOT_INO && ino < ctx->fs->super->s_first_ino))
+ return;
+ if (!ctx->inodes_to_rebuild)
+ e2fsck_allocate_inode_bitmap(ctx->fs,
+ _("extent rebuild inode map"),
+ EXT2FS_BMAP64_AUTODIR,
+ "inodes_to_rebuild",
+ &ctx->inodes_to_rebuild);
+ if (ctx->inodes_to_rebuild)
+ ext2fs_mark_inode_bitmap2(ctx->inodes_to_rebuild, ino);
+}
+
+/* Ask if an inode will have its extents rebuilt during pass 1E. */
+int e2fsck_ino_will_be_rebuilt(e2fsck_t ctx, ext2_ino_t ino)
+{
+ if (!ctx->inodes_to_rebuild)
+ return 0;
+ return ext2fs_test_inode_bitmap2(ctx->inodes_to_rebuild, ino);
+}
+
+struct extent_list {
+ blk64_t blocks_freed;
+ struct ext2fs_extent *extents;
+ unsigned int count;
+ unsigned int size;
+ unsigned int ext_read;
+ errcode_t retval;
+ ext2_ino_t ino;
+};
+
+static errcode_t load_extents(e2fsck_t ctx, struct extent_list *list)
+{
+ ext2_filsys fs = ctx->fs;
+ ext2_extent_handle_t handle;
+ struct ext2fs_extent extent;
+ errcode_t retval;
+
+ retval = ext2fs_extent_open(fs, list->ino, &handle);
+ if (retval)
+ return retval;
+
+ retval = ext2fs_extent_get(handle, EXT2_EXTENT_ROOT, &extent);
+ if (retval)
+ goto out;
+
+ do {
+ if (extent.e_flags & EXT2_EXTENT_FLAGS_SECOND_VISIT)
+ goto next;
+
+ /* Internal node; free it and we'll re-allocate it later */
+ if (!(extent.e_flags & EXT2_EXTENT_FLAGS_LEAF)) {
+#if defined(DEBUG) || defined(DEBUG_FREE)
+ printf("ino=%d free=%llu bf=%llu\n", list->ino,
+ extent.e_pblk, list->blocks_freed + 1);
+#endif
+ list->blocks_freed++;
+ ext2fs_block_alloc_stats2(fs, extent.e_pblk, -1);
+ goto next;
+ }
+
+ list->ext_read++;
+ /* Can we attach it to the previous extent? */
+ if (list->count) {
+ struct ext2fs_extent *last = list->extents +
+ list->count - 1;
+ blk64_t end = last->e_len + extent.e_len;
+
+ if (last->e_pblk + last->e_len == extent.e_pblk &&
+ last->e_lblk + last->e_len == extent.e_lblk &&
+ (last->e_flags & EXT2_EXTENT_FLAGS_UNINIT) ==
+ (extent.e_flags & EXT2_EXTENT_FLAGS_UNINIT) &&
+ end < (1ULL << 32)) {
+ last->e_len += extent.e_len;
+#ifdef DEBUG
+ printf("R: ino=%d len=%u\n", list->ino,
+ last->e_len);
+#endif
+ goto next;
+ }
+ }
+
+ /* Do we need to expand? */
+ if (list->count == list->size) {
+ unsigned int new_size = (list->size + NUM_EXTENTS) *
+ sizeof(struct ext2fs_extent);
+ retval = ext2fs_resize_mem(0, new_size, &list->extents);
+ if (retval)
+ goto out;
+ list->size += NUM_EXTENTS;
+ }
+
+ /* Add a new extent */
+ memcpy(list->extents + list->count, &extent, sizeof(extent));
+#ifdef DEBUG
+ printf("R: ino=%d pblk=%llu lblk=%llu len=%u\n", list->ino,
+ extent.e_pblk, extent.e_lblk, extent.e_len);
+#endif
+ list->count++;
+next:
+ retval = ext2fs_extent_get(handle, EXT2_EXTENT_NEXT, &extent);
+ } while (retval == 0);
+
+out:
+ /* Ok if we run off the end */
+ if (retval == EXT2_ET_EXTENT_NO_NEXT)
+ retval = 0;
+ ext2fs_extent_free(handle);
+ return retval;
+}
+
+static int find_blocks(ext2_filsys fs, blk64_t *blocknr, e2_blkcnt_t blockcnt,
+ blk64_t ref_blk, int ref_offset, void *priv_data)
+{
+ struct extent_list *list = priv_data;
+
+ /* Internal node? */
+ if (blockcnt < 0) {
+#if defined(DEBUG) || defined(DEBUG_FREE)
+ printf("ino=%d free=%llu bf=%llu\n", list->ino, *blocknr,
+ list->blocks_freed + 1);
+#endif
+ list->blocks_freed++;
+ ext2fs_block_alloc_stats2(fs, *blocknr, -1);
+ return 0;
+ }
+
+ /* Can we attach it to the previous extent? */
+ if (list->count) {
+ struct ext2fs_extent *last = list->extents +
+ list->count - 1;
+ blk64_t end = last->e_len + 1;
+
+ if (last->e_pblk + last->e_len == *blocknr &&
+ end < (1ULL << 32)) {
+ last->e_len++;
+#ifdef DEBUG
+ printf("R: ino=%d len=%u\n", list->ino, last->e_len);
+#endif
+ return 0;
+ }
+ }
+
+ /* Do we need to expand? */
+ if (list->count == list->size) {
+ unsigned int new_size = (list->size + NUM_EXTENTS) *
+ sizeof(struct ext2fs_extent);
+ list->retval = ext2fs_resize_mem(0, new_size, &list->extents);
+ if (list->retval)
+ return BLOCK_ABORT;
+ list->size += NUM_EXTENTS;
+ }
+
+ /* Add a new extent */
+ list->extents[list->count].e_pblk = *blocknr;
+ list->extents[list->count].e_lblk = blockcnt;
+ list->extents[list->count].e_len = 1;
+ list->extents[list->count].e_flags = 0;
+#ifdef DEBUG
+ printf("R: ino=%d pblk=%llu lblk=%llu len=%u\n", list->ino, *blocknr,
+ blockcnt, 1);
+#endif
+ list->count++;
+
+ return 0;
+}
+
+static errcode_t rebuild_extent_tree(e2fsck_t ctx, struct extent_list *list,
+ ext2_ino_t ino)
+{
+ struct ext2_inode inode;
+ errcode_t retval;
+ ext2_extent_handle_t handle;
+ unsigned int i, ext_written;
+ struct ext2fs_extent *ex, extent;
+
+ list->count = 0;
+ list->blocks_freed = 0;
+ list->ino = ino;
+ list->ext_read = 0;
+ e2fsck_read_inode(ctx, ino, &inode, "rebuild_extents");
+
+ /* Collect lblk->pblk mappings */
+ if (inode.i_flags & EXT4_EXTENTS_FL) {
+ retval = load_extents(ctx, list);
+ goto extents_loaded;
+ }
+
+ retval = ext2fs_block_iterate3(ctx->fs, ino, BLOCK_FLAG_READ_ONLY, 0,
+ find_blocks, list);
+ if (retval)
+ goto err;
+ if (list->retval) {
+ retval = list->retval;
+ goto err;
+ }
+
+extents_loaded:
+ /* Reset extent tree */
+ inode.i_flags &= ~EXT4_EXTENTS_FL;
+ memset(inode.i_block, 0, sizeof(inode.i_block));
+
+ /* Make a note of freed blocks */
+ retval = ext2fs_iblk_sub_blocks(ctx->fs, &inode, list->blocks_freed);
+ if (retval)
+ goto err;
+
+ /* Now stuff extents into the file */
+ retval = ext2fs_extent_open2(ctx->fs, ino, &inode, &handle);
+ if (retval)
+ goto err;
+
+ ext_written = 0;
+ for (i = 0, ex = list->extents; i < list->count; i++, ex++) {
+ memcpy(&extent, ex, sizeof(struct ext2fs_extent));
+ extent.e_flags &= EXT2_EXTENT_FLAGS_UNINIT;
+ if (extent.e_flags & EXT2_EXTENT_FLAGS_UNINIT) {
+ if (extent.e_len > EXT_UNINIT_MAX_LEN) {
+ extent.e_len = EXT_UNINIT_MAX_LEN;
+ ex->e_pblk += EXT_UNINIT_MAX_LEN;
+ ex->e_lblk += EXT_UNINIT_MAX_LEN;
+ ex->e_len -= EXT_UNINIT_MAX_LEN;
+ ex--;
+ i--;
+ }
+ } else {
+ if (extent.e_len > EXT_INIT_MAX_LEN) {
+ extent.e_len = EXT_INIT_MAX_LEN;
+ ex->e_pblk += EXT_INIT_MAX_LEN;
+ ex->e_lblk += EXT_INIT_MAX_LEN;
+ ex->e_len -= EXT_INIT_MAX_LEN;
+ ex--;
+ i--;
+ }
+ }
+
+#ifdef DEBUG
+ printf("W: ino=%d pblk=%llu lblk=%llu len=%u\n", ino,
+ extent.e_pblk, extent.e_lblk, extent.e_len);
+#endif
+ retval = ext2fs_extent_insert(handle, EXT2_EXTENT_INSERT_AFTER,
+ &extent);
+ if (retval)
+ goto err2;
+ retval = ext2fs_extent_fix_parents(handle);
+ if (retval)
+ goto err2;
+ ext_written++;
+ }
+
+#if defined(DEBUG) || defined(DEBUG_SUMMARY)
+ printf("rebuild: ino=%d extents=%d->%d\n", ino, list->ext_read,
+ ext_written);
+#endif
+ e2fsck_write_inode(ctx, ino, &inode, "rebuild_extents");
+
+err2:
+ ext2fs_extent_free(handle);
+err:
+ return retval;
+}
+
+void e2fsck_pass1e(e2fsck_t ctx)
+{
+ struct problem_context pctx;
+#ifdef RESOURCE_TRACK
+ struct resource_track rtrack;
+#endif
+ struct extent_list list;
+ int first = 1;
+ ext2_ino_t ino = 0;
+ errcode_t retval;
+
+ if (!EXT2_HAS_INCOMPAT_FEATURE(ctx->fs->super,
+ EXT3_FEATURE_INCOMPAT_EXTENTS) ||
+ !ext2fs_test_valid(ctx->fs) ||
+ ctx->invalid_bitmaps) {
+ if (ctx->inodes_to_rebuild)
+ ext2fs_free_inode_bitmap(ctx->inodes_to_rebuild);
+ ctx->inodes_to_rebuild = NULL;
+ }
+
+ if (ctx->inodes_to_rebuild == NULL)
+ return;
+
+ init_resource_track(&rtrack, ctx->fs->io);
+ clear_problem_context(&pctx);
+ e2fsck_read_bitmaps(ctx);
+
+ memset(&list, 0, sizeof(list));
+ retval = ext2fs_get_mem(sizeof(struct ext2fs_extent) * NUM_EXTENTS,
+ &list.extents);
+ list.size = NUM_EXTENTS;
+ while (1) {
+ retval = ext2fs_find_first_set_inode_bitmap2(
+ ctx->inodes_to_rebuild, ino + 1,
+ ctx->fs->super->s_inodes_count, &ino);
+ if (retval)
+ break;
+ pctx.ino = ino;
+ if (first) {
+ fix_problem(ctx, PR_1E_PASS_HEADER, &pctx);
+ first = 0;
+ }
+ pctx.errcode = rebuild_extent_tree(ctx, &list, ino);
+ if (pctx.errcode) {
+ end_problem_latch(ctx, PR_LATCH_OPTIMIZE_EXT);
+ fix_problem(ctx, PR_1E_OPTIMIZE_EXT_ERR, &pctx);
+ }
+ if (ctx->progress && !ctx->progress_fd)
+ e2fsck_simple_progress(ctx, "Rebuilding extents",
+ 100.0 * (float) ino /
+ (float) ctx->fs->super->s_inodes_count,
+ ino);
+ }
+ end_problem_latch(ctx, PR_LATCH_OPTIMIZE_EXT);
+
+ ext2fs_free_inode_bitmap(ctx->inodes_to_rebuild);
+ ctx->inodes_to_rebuild = NULL;
+ ext2fs_free_mem(&list.extents);
+
+ print_resource_track(ctx, "Pass 1E", &rtrack, ctx->fs->io);
+}
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index a963849..8567419 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -56,6 +56,8 @@
#define _INLINE_ inline
#endif
+#undef DEBUG
+
static int process_block(ext2_filsys fs, blk64_t *blocknr,
e2_blkcnt_t blockcnt, blk64_t ref_blk,
int ref_offset, void *priv_data);
@@ -77,11 +79,16 @@ static void adjust_extattr_refcount(e2fsck_t ctx, ext2_refcount_t refcount,
char *block_buf, int adjust_sign);
/* static char *describe_illegal_block(ext2_filsys fs, blk64_t block); */
+struct extent_info {
+ unsigned int num_extents;
+ unsigned int max_extents;
+};
+
struct process_block_struct {
ext2_ino_t ino;
unsigned is_dir:1, is_reg:1, clear:1, suppress:1,
fragmented:1, compressed:1, bbcheck:1,
- inode_modified:1;
+ inode_modified:1, extent_rebuild:1;
blk64_t num_blocks;
blk64_t max_blocks;
e2_blkcnt_t last_block;
@@ -95,6 +102,7 @@ struct process_block_struct {
e2fsck_t ctx;
blk64_t bad_ref;
region_t region;
+ struct extent_info ext_info[MAX_EXTENT_DEPTH_COUNT];
};
struct process_inode_block {
@@ -2402,6 +2410,53 @@ static int has_unaligned_cluster_map(e2fsck_t ctx,
return 0;
}
+static void should_rebuild_extents(e2fsck_t ctx,
+ struct problem_context *pctx,
+ struct process_block_struct *pb,
+ struct ext2_extent_info *info)
+{
+ struct extent_info *ei;
+ int i, j;
+ unsigned int extents_per_block;
+
+ if (pb->extent_rebuild)
+ goto rebuild;
+
+ extents_per_block = (ctx->fs->blocksize -
+ sizeof(struct ext3_extent_header)) /
+ sizeof(struct ext3_extent);
+ /*
+ * If we can consolidate a level or shorten the tree, schedule the
+ * extent tree to be rebuilt.
+ */
+ for (i = 0, ei = pb->ext_info; i < info->max_depth + 1; i++, ei++) {
+ if (ei->max_extents - ei->num_extents > extents_per_block) {
+#ifdef DEBUG
+ printf("rebuild extents, ino=%d level=%d slack=%d epb=%d\n",
+ pb->ino, i,
+ ei->max_extents - ei->num_extents,
+ extents_per_block);
+#endif
+ goto rebuild;
+ }
+ for (j = 0; j < i; j++) {
+ if (ei->num_extents < pb->ext_info[j].max_extents) {
+#ifdef DEBUG
+ printf("rebuild extents, ino=%d level=%d num=%d level=%d\n",
+ pb->ino, i, ei->num_extents, j);
+#endif
+ goto rebuild;
+ }
+ }
+ }
+ return;
+
+rebuild:
+ if (pb->extent_rebuild ||
+ fix_problem(ctx, PR_1E_CAN_COMPRESS_EXTENT_TREE, pctx))
+ e2fsck_rebuild_extents_later(ctx, pb->ino);
+}
+
static void scan_extent_node(e2fsck_t ctx, struct problem_context *pctx,
struct process_block_struct *pb,
blk64_t start_block, blk64_t end_block,
@@ -2424,6 +2479,19 @@ static void scan_extent_node(e2fsck_t ctx, struct problem_context *pctx,
pctx->errcode = ext2fs_extent_get_info(ehandle, &info);
if (pctx->errcode)
return;
+ if (!pb->extent_rebuild) {
+ pb->ext_info[info.curr_level].num_extents += info.num_entries;
+ pb->ext_info[info.curr_level].max_extents += info.max_entries;
+ /*
+ * Implementation wart: Splitting extent blocks when appending
+ * will leave the old block with one free entry. Therefore,
+ * pretend that a non-root extent block can hold one fewer
+ * entry than it actually does, so that we don't repeatedly
+ * rebuild the extent tree.
+ */
+ if (info.curr_level)
+ pb->ext_info[info.curr_level].max_extents--;
+ }
pctx->errcode = ext2fs_extent_get(ehandle, EXT2_EXTENT_FIRST_SIB,
&extent);
@@ -2760,17 +2828,31 @@ static void check_blocks_extents(e2fsck_t ctx, struct problem_context *pctx,
retval = ext2fs_extent_get_info(ehandle, &info);
if (retval == 0) {
- if (info.max_depth >= MAX_EXTENT_DEPTH_COUNT)
- info.max_depth = MAX_EXTENT_DEPTH_COUNT-1;
- ctx->extent_depth_count[info.max_depth]++;
+ int max_depth = info.max_depth;
+
+ if (max_depth >= MAX_EXTENT_DEPTH_COUNT)
+ max_depth = MAX_EXTENT_DEPTH_COUNT-1;
+ ctx->extent_depth_count[max_depth]++;
}
+ /* Check maximum extent depth */
+ pctx->blk = info.max_depth;
+ pctx->blk2 = ext2fs_max_extent_depth(ehandle);
+ if (pctx->blk2 < pctx->blk &&
+ fix_problem(ctx, PR_1_EXTENT_BAD_MAX_DEPTH, pctx))
+ pb->extent_rebuild = 1;
+
+ /* Can we collect extent tree level stats? */
+ pctx->blk = MAX_EXTENT_DEPTH_COUNT;
+ if (pctx->blk2 > pctx->blk)
+ fix_problem(ctx, PR_1E_MAX_EXTENT_TREE_DEPTH, pctx);
+ memset(pb->ext_info, 0, sizeof(pb->ext_info));
+
pb->region = region_create(0, info.max_lblk);
if (!pb->region) {
- ext2fs_extent_free(ehandle);
fix_problem(ctx, PR_1_EXTENT_ALLOC_REGION_ABORT, pctx);
ctx->flags |= E2F_FLAG_ABORT;
- return;
+ goto out;
}
eof_lblk = ((EXT2_I_SIZE(inode) + fs->blocksize - 1) >>
@@ -2786,7 +2868,9 @@ static void check_blocks_extents(e2fsck_t ctx, struct problem_context *pctx,
}
region_free(pb->region);
pb->region = NULL;
+out:
ext2fs_extent_free(ehandle);
+ should_rebuild_extents(ctx, pctx, pb, &info);
}
/*
@@ -2846,6 +2930,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
pb.ctx = ctx;
pb.inode_modified = 0;
pb.bad_ref = 0;
+ pb.extent_rebuild = 0;
pctx->ino = ino;
pctx->errcode = 0;
@@ -2909,6 +2994,13 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
"check_blocks");
fs->flags = (flags & EXT2_FLAG_IGNORE_CSUM_ERRORS) |
(fs->flags & ~EXT2_FLAG_IGNORE_CSUM_ERRORS);
+
+ if (ctx->options & E2F_OPT_CONVERT_BMAP) {
+#ifdef DEBUG
+ printf("bmap rebuild ino=%d\n", ino);
+#endif
+ e2fsck_rebuild_extents_later(ctx, ino);
+ }
}
}
end_problem_latch(ctx, PR_LATCH_BLOCK);
diff --git a/e2fsck/problem.c b/e2fsck/problem.c
index a63e61c..b1bcc0d 100644
--- a/e2fsck/problem.c
+++ b/e2fsck/problem.c
@@ -1101,6 +1101,11 @@ static struct e2fsck_problem problem_table[] = {
N_("@i %i has a duplicate @x mapping\n\t(logical @b %c, @n physical @b %b, len %N)\n"),
PROMPT_CLEAR, 0 },
+ /* Inode extent tree could be more shallow */
+ { PR_1_EXTENT_BAD_MAX_DEPTH,
+ N_("@i %i @x tree could be more shallow (%b; could be <= %c)\n"),
+ PROMPT_FIX, PR_NO_OK | PR_PREEN_NO | PR_PREEN_OK },
+
/* Pass 1b errors */
/* Pass 1B: Rescan for duplicate/bad blocks */
@@ -1198,6 +1203,43 @@ static struct e2fsck_problem problem_table[] = {
{ PR_1D_CLONE_ERROR,
N_("Couldn't clone file: %m\n"), PROMPT_NONE, 0 },
+ /* Pass 1E Extent tree Optimization */
+
+ /* Pass 1E: Optimizing extent trees */
+ { PR_1E_PASS_HEADER,
+ N_("Pass 1E: Optimizing @x trees\n"),
+ PROMPT_NONE, PR_PREEN_NOMSG },
+
+ /* Failed to optimize extent tree */
+ { PR_1E_OPTIMIZE_EXT_ERR,
+ N_("Failed to optimize @x tree %p (%i): %m\n"),
+ PROMPT_NONE, 0 },
+
+ /* Rebuilding extent trees */
+ { PR_1E_OPTIMIZE_EXT_HEADER,
+ N_("Optimizing @x trees: "),
+ PROMPT_NONE, PR_MSG_ONLY },
+
+ /* Rebuilding extent tree %d */
+ { PR_1E_OPTIMIZE_EXT,
+ " %i",
+ PROMPT_NONE, PR_LATCH_OPTIMIZE_EXT | PR_PREEN_NOHDR},
+
+ /* Rebuilding extent tree end */
+ { PR_1E_OPTIMIZE_EXT_END,
+ "\n",
+ PROMPT_NONE, PR_PREEN_NOHDR },
+
+ /* Internal error: extent tree depth too large */
+ { PR_1E_MAX_EXTENT_TREE_DEPTH,
+ N_("Internal error: max extent tree depth too large (%b; expected=%c).\n"),
+ PROMPT_NONE, PR_FATAL },
+
+ /* Inode extent tree could be more compact */
+ { PR_1E_CAN_COMPRESS_EXTENT_TREE,
+ N_("@i %i @x tree could be more compact. "),
+ PROMPT_FIX, PR_NO_OK | PR_PREEN_NO | PR_PREEN_OK },
+
/* Pass 2 errors */
/* Pass 2: Checking directory structure */
@@ -1946,6 +1988,7 @@ static struct latch_descr pr_latch_info[] = {
{ PR_LATCH_TOOBIG, PR_1_INODE_TOOBIG, 0 },
{ PR_LATCH_OPTIMIZE_DIR, PR_3A_OPTIMIZE_DIR_HEADER, PR_3A_OPTIMIZE_DIR_END },
{ PR_LATCH_BG_CHECKSUM, PR_0_GDT_CSUM_LATCH, 0 },
+ { PR_LATCH_OPTIMIZE_EXT, PR_1E_OPTIMIZE_EXT_HEADER, PR_1E_OPTIMIZE_EXT_END },
{ -1, 0, 0 },
};
diff --git a/e2fsck/problem.h b/e2fsck/problem.h
index 3c28166..d3dcc9e 100644
--- a/e2fsck/problem.h
+++ b/e2fsck/problem.h
@@ -40,6 +40,7 @@ struct problem_context {
#define PR_LATCH_TOOBIG 0x0080 /* Latch for file to big errors */
#define PR_LATCH_OPTIMIZE_DIR 0x0090 /* Latch for optimize directories */
#define PR_LATCH_BG_CHECKSUM 0x00A0 /* Latch for block group checksums */
+#define PR_LATCH_OPTIMIZE_EXT 0x00B0 /* Latch for rebuild extents */
#define PR_LATCH(x) ((((x) & PR_LATCH_MASK) >> 4) - 1)
@@ -641,6 +642,9 @@ struct problem_context {
/* leaf extent collision */
#define PR_1_EXTENT_COLLISION 0x01007D
+/* extent tree max depth too big */
+#define PR_1_EXTENT_BAD_MAX_DEPTH 0x01007E
+
/*
* Pass 1b errors
*/
@@ -704,6 +708,30 @@ struct problem_context {
#define PR_1D_CLONE_ERROR 0x013008
/*
+ * Pass 1e --- rebuilding extent trees
+ */
+/* Pass 1e: Rebuilding extent trees */
+#define PR_1E_PASS_HEADER 0x014000
+
+/* Error rehash directory */
+#define PR_1E_OPTIMIZE_EXT_ERR 0x014001
+
+/* Rebuilding extent trees */
+#define PR_1E_OPTIMIZE_EXT_HEADER 0x014002
+
+/* Rebuilding extent %d */
+#define PR_1E_OPTIMIZE_EXT 0x014003
+
+/* Rebuilding extent tree end */
+#define PR_1E_OPTIMIZE_EXT_END 0x014004
+
+/* Internal error: extent tree depth too large */
+#define PR_1E_MAX_EXTENT_TREE_DEPTH 0x014005
+
+/* Inode extent tree could be more compact */
+#define PR_1E_CAN_COMPRESS_EXTENT_TREE 0x014006
+
+/*
* Pass 2 errors
*/
diff --git a/e2fsck/super.c b/e2fsck/super.c
index 1e7e749..e64262a 100644
--- a/e2fsck/super.c
+++ b/e2fsck/super.c
@@ -606,6 +606,13 @@ void check_super_block(e2fsck_t ctx)
ext2fs_mark_super_dirty(fs);
}
+ /* Did user ask us to convert files to extents? */
+ if (ctx->options & E2F_OPT_CONVERT_BMAP) {
+ fs->super->s_feature_incompat |=
+ EXT3_FEATURE_INCOMPAT_EXTENTS;
+ ext2fs_mark_super_dirty(fs);
+ }
+
if ((fs->super->s_feature_incompat & EXT2_FEATURE_INCOMPAT_META_BG) &&
(fs->super->s_first_meta_bg > fs->desc_blocks)) {
pctx.group = fs->desc_blocks;
diff --git a/e2fsck/unix.c b/e2fsck/unix.c
index f3672c0..fe5127a 100644
--- a/e2fsck/unix.c
+++ b/e2fsck/unix.c
@@ -709,6 +709,9 @@ static void parse_extended_opts(e2fsck_t ctx, const char *opts)
else
ctx->log_fn = string_copy(ctx, arg, 0);
continue;
+ } else if (strcmp(token, "bmap2extent") == 0) {
+ ctx->options |= E2F_OPT_CONVERT_BMAP;
+ continue;
} else {
fprintf(stderr, _("Unknown extended option: %s\n"),
token);
@@ -728,6 +731,7 @@ static void parse_extended_opts(e2fsck_t ctx, const char *opts)
fputs(("\tdiscard\n"), stderr);
fputs(("\tnodiscard\n"), stderr);
fputs(("\treadahead_kb=<buffer size>\n"), stderr);
+ fputs(("\tbmap2extent\n"), stderr);
fputc('\n', stderr);
exit(1);
}
diff --git a/tests/f_extent_bad_node/expect.1 b/tests/f_extent_bad_node/expect.1
index 0c0bc28..c13ad39 100644
--- a/tests/f_extent_bad_node/expect.1
+++ b/tests/f_extent_bad_node/expect.1
@@ -2,8 +2,11 @@ Pass 1: Checking inodes, blocks, and sizes
Inode 12 has an invalid extent node (blk 22, lblk 0)
Clear? yes
+Inode 12 extent tree could be more compact. Fix? yes
+
Inode 12, i_blocks is 16, should be 8. Fix? yes
+Pass 1E: Optimizing extent trees
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
@@ -11,13 +14,13 @@ Pass 5: Checking group summary information
Block bitmap differences: -(21--23) -25
Fix? yes
-Free blocks count wrong for group #0 (71, counted=75).
+Free blocks count wrong for group #0 (73, counted=77).
Fix? yes
-Free blocks count wrong (71, counted=75).
+Free blocks count wrong (73, counted=77).
Fix? yes
test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
-test_filesys: 12/16 files (0.0% non-contiguous), 25/100 blocks
+test_filesys: 12/16 files (0.0% non-contiguous), 23/100 blocks
Exit status is 1
diff --git a/tests/f_extent_bad_node/expect.2 b/tests/f_extent_bad_node/expect.2
index 568c792..b78b193 100644
--- a/tests/f_extent_bad_node/expect.2
+++ b/tests/f_extent_bad_node/expect.2
@@ -3,5 +3,5 @@ Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
-test_filesys: 12/16 files (0.0% non-contiguous), 25/100 blocks
+test_filesys: 12/16 files (0.0% non-contiguous), 23/100 blocks
Exit status is 0
diff --git a/tests/f_extent_int_bad_magic/expect.1 b/tests/f_extent_int_bad_magic/expect.1
index 0e82e2b..0bd163f 100644
--- a/tests/f_extent_int_bad_magic/expect.1
+++ b/tests/f_extent_int_bad_magic/expect.1
@@ -2,8 +2,11 @@ Pass 1: Checking inodes, blocks, and sizes
Inode 12 has an invalid extent node (blk 1295, lblk 0)
Clear? yes
+Inode 12 extent tree could be more compact. Fix? yes
+
Inode 12, i_blocks is 712, should be 0. Fix? yes
+Pass 1E: Optimizing extent trees
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
diff --git a/tests/f_extent_leaf_bad_magic/expect.1 b/tests/f_extent_leaf_bad_magic/expect.1
index 7b6dbf1..c31a309 100644
--- a/tests/f_extent_leaf_bad_magic/expect.1
+++ b/tests/f_extent_leaf_bad_magic/expect.1
@@ -2,8 +2,11 @@ Pass 1: Checking inodes, blocks, and sizes
Inode 12 has an invalid extent node (blk 1604, lblk 0)
Clear? yes
+Inode 12 extent tree could be more compact. Fix? yes
+
Inode 12, i_blocks is 18, should be 0. Fix? yes
+Pass 1E: Optimizing extent trees
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
diff --git a/tests/f_extent_oobounds/expect.1 b/tests/f_extent_oobounds/expect.1
index 3164ea0..237829a 100644
--- a/tests/f_extent_oobounds/expect.1
+++ b/tests/f_extent_oobounds/expect.1
@@ -3,8 +3,11 @@ Inode 12, end of extent exceeds allowed value
(logical block 15, physical block 200, len 30)
Clear? yes
+Inode 12 extent tree could be more compact. Fix? yes
+
Inode 12, i_blocks is 154, should be 94. Fix? yes
+Pass 1E: Optimizing extent trees
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
@@ -12,13 +15,13 @@ Pass 5: Checking group summary information
Block bitmap differences: -(200--229)
Fix? yes
-Free blocks count wrong for group #0 (156, counted=186).
+Free blocks count wrong for group #0 (158, counted=188).
Fix? yes
-Free blocks count wrong (156, counted=186).
+Free blocks count wrong (158, counted=188).
Fix? yes
test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
-test_filesys: 12/32 files (8.3% non-contiguous), 70/256 blocks
+test_filesys: 12/32 files (8.3% non-contiguous), 68/256 blocks
Exit status is 1
diff --git a/tests/f_extent_oobounds/expect.2 b/tests/f_extent_oobounds/expect.2
index 22c4f2c..0729283 100644
--- a/tests/f_extent_oobounds/expect.2
+++ b/tests/f_extent_oobounds/expect.2
@@ -3,5 +3,5 @@ Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
-test_filesys: 12/32 files (8.3% non-contiguous), 70/256 blocks
+test_filesys: 12/32 files (8.3% non-contiguous), 68/256 blocks
Exit status is 0
diff --git a/tests/f_extents/expect.1 b/tests/f_extents/expect.1
index aeebc7b..d682929 100644
--- a/tests/f_extents/expect.1
+++ b/tests/f_extents/expect.1
@@ -6,6 +6,8 @@ Inode 12 has an invalid extent
(logical block 0, invalid physical block 21994527527949, len 17)
Clear? yes
+Inode 12 extent tree could be more compact. Fix? yes
+
Inode 12, i_blocks is 34, should be 0. Fix? yes
Inode 13 missing EXTENT_FL, but is in extents format
@@ -21,6 +23,8 @@ Inode 17 has an invalid extent
(logical block 0, invalid physical block 22011707397135, len 15)
Clear? yes
+Inode 17 extent tree could be more compact. Fix? yes
+
Inode 17, i_blocks is 32, should be 0. Fix? yes
Error while reading over extent tree in inode 18: Corrupt extent header
@@ -31,6 +35,7 @@ Inode 18, i_blocks is 2, should be 0. Fix? yes
Special (device/socket/fifo) file (inode 19) has extents
or inline-data flag set. Clear? yes
+Pass 1E: Optimizing extent trees
Pass 2: Checking directory structure
Entry 'fbad-flag' in / (2) has deleted/unused inode 18. Clear? yes
^ permalink raw reply related [flat|nested] 104+ messages in thread
* [PATCH 34/47] tests: verify proper rebuilding of sparse extent trees and block map file conversion
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (32 preceding siblings ...)
2014-11-07 21:54 ` [PATCH 33/47] e2fsck: rebuild sparse extent trees/convert non-extent ext3 files Darrick J. Wong
@ 2014-11-07 21:54 ` Darrick J. Wong
2014-11-07 21:54 ` [PATCH 35/47] resize2fs: convert fs to and from 64bit mode Darrick J. Wong
` (14 subsequent siblings)
48 siblings, 0 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:54 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
tests/f_collapse_extent_tree/expect.1 | 16 ++++
tests/f_collapse_extent_tree/expect.2 | 10 ++
tests/f_collapse_extent_tree/image.gz | Bin
tests/f_collapse_extent_tree/name | 1
tests/f_collapse_extent_tree/script | 118 +++++++++++++++++++++++++++
tests/f_compress_extent_tree_level/expect.1 | 23 +++++
tests/f_compress_extent_tree_level/expect.2 | 17 ++++
tests/f_compress_extent_tree_level/image.gz | Bin
tests/f_compress_extent_tree_level/name | 1
tests/f_compress_extent_tree_level/script | 118 +++++++++++++++++++++++++++
tests/f_convert_bmap/expect.1 | 26 ++++++
tests/f_convert_bmap/expect.2 | 10 ++
tests/f_convert_bmap/image.gz | Bin
tests/f_convert_bmap/name | 1
tests/f_convert_bmap/script | 119 +++++++++++++++++++++++++++
tests/f_extent_too_deep/expect.1 | 23 +++++
tests/f_extent_too_deep/expect.2 | 10 ++
tests/f_extent_too_deep/image.gz | Bin
tests/f_extent_too_deep/name | 1
tests/f_extent_too_deep/script | 118 +++++++++++++++++++++++++++
tests/f_opt_extent/expect | 55 ++++++++++++
tests/f_opt_extent/name | 1
tests/f_opt_extent/script | 64 +++++++++++++++
tests/f_opt_extent_ext3/expect | 45 ++++++++++
tests/f_opt_extent_ext3/name | 1
tests/f_opt_extent_ext3/script | 65 +++++++++++++++
26 files changed, 843 insertions(+)
create mode 100644 tests/f_collapse_extent_tree/expect.1
create mode 100644 tests/f_collapse_extent_tree/expect.2
create mode 100644 tests/f_collapse_extent_tree/image.gz
create mode 100644 tests/f_collapse_extent_tree/name
create mode 100644 tests/f_collapse_extent_tree/script
create mode 100644 tests/f_compress_extent_tree_level/expect.1
create mode 100644 tests/f_compress_extent_tree_level/expect.2
create mode 100644 tests/f_compress_extent_tree_level/image.gz
create mode 100644 tests/f_compress_extent_tree_level/name
create mode 100644 tests/f_compress_extent_tree_level/script
create mode 100644 tests/f_convert_bmap/expect.1
create mode 100644 tests/f_convert_bmap/expect.2
create mode 100644 tests/f_convert_bmap/image.gz
create mode 100644 tests/f_convert_bmap/name
create mode 100644 tests/f_convert_bmap/script
create mode 100644 tests/f_extent_too_deep/expect.1
create mode 100644 tests/f_extent_too_deep/expect.2
create mode 100644 tests/f_extent_too_deep/image.gz
create mode 100644 tests/f_extent_too_deep/name
create mode 100644 tests/f_extent_too_deep/script
create mode 100644 tests/f_opt_extent/expect
create mode 100644 tests/f_opt_extent/name
create mode 100644 tests/f_opt_extent/script
create mode 100644 tests/f_opt_extent_ext3/expect
create mode 100644 tests/f_opt_extent_ext3/name
create mode 100644 tests/f_opt_extent_ext3/script
diff --git a/tests/f_collapse_extent_tree/expect.1 b/tests/f_collapse_extent_tree/expect.1
new file mode 100644
index 0000000..0d4b465
--- /dev/null
+++ b/tests/f_collapse_extent_tree/expect.1
@@ -0,0 +1,16 @@
+debugfs: ex /a
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 0 9 1
+ 1/ 1 1/ 1 0 - 0 10 - 10 1
+Pass 1: Checking inodes, blocks, and sizes
+Inode 12 extent tree could be more compact. Fix? yes
+
+Pass 1E: Optimizing extent trees
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 12/128 files (0.0% non-contiguous), 19/512 blocks
+Exit status is 1
diff --git a/tests/f_collapse_extent_tree/expect.2 b/tests/f_collapse_extent_tree/expect.2
new file mode 100644
index 0000000..a1d28b1
--- /dev/null
+++ b/tests/f_collapse_extent_tree/expect.2
@@ -0,0 +1,10 @@
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+test_filesys: 12/128 files (0.0% non-contiguous), 19/512 blocks
+Exit status is 0
+debugfs: ex /a
+Level Entries Logical Physical Length Flags
+ 0/ 0 1/ 1 0 - 0 10 - 10 1
diff --git a/tests/f_collapse_extent_tree/image.gz b/tests/f_collapse_extent_tree/image.gz
new file mode 100644
index 0000000000000000000000000000000000000000..97036cc597c5d2bcf6da6ffa68039b743b46aa20
GIT binary patch
literal 2537
zcmb2|=3r2~7aqdI{Pxc7?BGBdh6lxY##=IfImEQ^-3nbX`9VZ0->#??lNCe@FW4FQ
z#YVL}TII*Z*HrHjAfO&wvUze}my@ja$2)hVrIT2)v*+IzulqiySl;uUviknXEiDW(
zp{IYUa6fd>vO6yD>gk^~J=eHSyKdZJX8c#hmFMiUr|S)uTs`cRwf(*Jw5JI-Ev9#g
zzx10H^z+x-+1&PezM<j%wN-x#UY^?f?s?&}Z!v%C-)BF5eDl?n_HOO-adCV1e|+^f
zi)Zq}X`-jCz9%p5SsMMuaQ~#F154Q%4#=Ijd_P(9SoF_&+pXIBr(Hc<aafRvfuW&!
zy=4ribnWdMatsU%H@<&*|36zXO?crCT?gHin@3zeO5c0DaMs;f`-AOklmETD^R*_D
z<Gowbo@dFzK=l>@Y`c&Bzwh4l(H_VvIMHLp0;CiE@B>MS|I9!V1gcl=k~>^-C+GRq
zERBES2J6qaJ*xkI`|HBKUj^>6K=~WTAJ?~oRpKF>ikGbB1<K1<*)cNWQAa>3L0jxv
z+Ope)Q@;H^{nJEJ_gnqOo+Qus7x`2FDlW?`wQch{Ubb=Tv2VRgcl<QbdwS81f1O)`
zxz73}p2?fs|I45L_x?U_+1AJMzxKcTu9(hR=B|9cUak6ReWXR?Gy8?^|AIA>)j!SO
zTl3dGUh~(|qxV1UpRv3(_UHXHezOY~k3TI=558jc%31d7spkCte`^0vE7$)i)8ioE
m=uzp>5Eu=C(GVC7fzc2c4FPgP;J~%NEDm#Conv57U;qF<XAwLA
literal 0
HcmV?d00001
diff --git a/tests/f_collapse_extent_tree/name b/tests/f_collapse_extent_tree/name
new file mode 100644
index 0000000..83e506f
--- /dev/null
+++ b/tests/f_collapse_extent_tree/name
@@ -0,0 +1 @@
+extent tree can be collapsed one level
diff --git a/tests/f_collapse_extent_tree/script b/tests/f_collapse_extent_tree/script
new file mode 100644
index 0000000..ee18438
--- /dev/null
+++ b/tests/f_collapse_extent_tree/script
@@ -0,0 +1,118 @@
+if [ "$DESCRIPTION"x != x ]; then
+ test_description="$DESCRIPTION"
+fi
+if [ "$IMAGE"x = x ]; then
+ IMAGE=$test_dir/image.gz
+fi
+
+if [ "$FSCK_OPT"x = x ]; then
+ FSCK_OPT=-yf
+fi
+
+if [ "$SECOND_FSCK_OPT"x = x ]; then
+ SECOND_FSCK_OPT=-yf
+fi
+
+if [ "$OUT1"x = x ]; then
+ OUT1=$test_name.1.log
+fi
+
+if [ "$OUT2"x = x ]; then
+ OUT2=$test_name.2.log
+fi
+
+if [ "$EXP1"x = x ]; then
+ if [ -f $test_dir/expect.1.gz ]; then
+ EXP1=$test_name.1.tmp
+ gunzip < $test_dir/expect.1.gz > $EXP1
+ else
+ EXP1=$test_dir/expect.1
+ fi
+fi
+
+if [ "$EXP2"x = x ]; then
+ if [ -f $test_dir/expect.2.gz ]; then
+ EXP2=$test_name.2.tmp
+ gunzip < $test_dir/expect.2.gz > $EXP2
+ else
+ EXP2=$test_dir/expect.2
+ fi
+fi
+
+if [ "$SKIP_GUNZIP" != "true" ] ; then
+ gunzip < $IMAGE > $TMPFILE
+fi
+
+cp /dev/null $OUT1
+
+eval $PREP_CMD
+
+echo 'ex /a' > $TMPFILE.cmd
+$DEBUGFS -f $TMPFILE.cmd $TMPFILE > $OUT1.new 2>&1
+rm -rf $TMPFILE.cmd
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT1.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT1.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT1.new >> $OUT1
+rm -f $OUT1.new
+
+if [ "$ONE_PASS_ONLY" != "true" ]; then
+ $FSCK $SECOND_FSCK_OPT -N test_filesys $TMPFILE > $OUT2.new 2>&1
+ status=$?
+ echo Exit status is $status >> $OUT2.new
+ echo 'ex /a' > $TMPFILE.cmd
+ $DEBUGFS -f $TMPFILE.cmd $TMPFILE >> $OUT2.new 2>&1
+ rm -rf $TMPFILE.cmd
+ sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT2.new > $OUT2
+ rm -f $OUT2.new
+fi
+
+eval $AFTER_CMD
+
+if [ "$SKIP_VERIFY" != "true" ] ; then
+ rm -f $test_name.ok $test_name.failed
+ cmp -s $OUT1 $EXP1
+ status1=$?
+ if [ "$ONE_PASS_ONLY" != "true" ]; then
+ cmp -s $OUT2 $EXP2
+ status2=$?
+ else
+ status2=0
+ fi
+ if [ "$PASS_ZERO" = "true" ]; then
+ cmp -s $test_name.0.log $test_dir/expect.0
+ status3=$?
+ else
+ status3=0
+ fi
+
+ if [ -z "$test_description" ] ; then
+ description="$test_name"
+ else
+ description="$test_name: $test_description"
+ fi
+
+ if [ "$status1" -eq 0 -a "$status2" -eq 0 -a "$status3" -eq 0 ] ; then
+ echo "$description: ok"
+ touch $test_name.ok
+ else
+ echo "$description: failed"
+ rm -f $test_name.failed
+ if [ "$PASS_ZERO" = "true" ]; then
+ diff $DIFF_OPTS $test_dir/expect.0 \
+ $test_name.0.log >> $test_name.failed
+ fi
+ diff $DIFF_OPTS $EXP1 $OUT1 >> $test_name.failed
+ if [ "$ONE_PASS_ONLY" != "true" ]; then
+ diff $DIFF_OPTS $EXP2 $OUT2 >> $test_name.failed
+ fi
+ fi
+ rm -f tmp_expect
+fi
+
+if [ "$SKIP_CLEANUP" != "true" ] ; then
+ unset IMAGE FSCK_OPT SECOND_FSCK_OPT OUT1 OUT2 EXP1 EXP2
+ unset SKIP_VERIFY SKIP_CLEANUP SKIP_GUNZIP ONE_PASS_ONLY PREP_CMD
+ unset DESCRIPTION SKIP_UNLINK AFTER_CMD PASS_ZERO
+fi
+
diff --git a/tests/f_compress_extent_tree_level/expect.1 b/tests/f_compress_extent_tree_level/expect.1
new file mode 100644
index 0000000..9db4131
--- /dev/null
+++ b/tests/f_compress_extent_tree_level/expect.1
@@ -0,0 +1,23 @@
+debugfs: ex /a
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 2 0 - 16 9 17
+ 1/ 1 1/ 4 0 - 0 10 - 10 1
+ 1/ 1 2/ 4 11 - 11 100 - 100 1
+ 1/ 1 3/ 4 13 - 13 101 - 101 1
+ 1/ 1 4/ 4 15 - 15 102 - 102 1
+ 0/ 1 2/ 2 17 - 21 12 5
+ 1/ 1 1/ 3 17 - 17 103 - 103 1
+ 1/ 1 2/ 3 19 - 19 104 - 104 1
+ 1/ 1 3/ 3 21 - 21 105 - 105 1
+Pass 1: Checking inodes, blocks, and sizes
+Inode 12 extent tree could be more compact. Fix? yes
+
+Pass 1E: Optimizing extent trees
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 12/128 files (8.3% non-contiguous), 26/512 blocks
+Exit status is 1
diff --git a/tests/f_compress_extent_tree_level/expect.2 b/tests/f_compress_extent_tree_level/expect.2
new file mode 100644
index 0000000..07d1082
--- /dev/null
+++ b/tests/f_compress_extent_tree_level/expect.2
@@ -0,0 +1,17 @@
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+test_filesys: 12/128 files (8.3% non-contiguous), 26/512 blocks
+Exit status is 0
+debugfs: ex /a
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 21 9 22
+ 1/ 1 1/ 7 0 - 0 10 - 10 1
+ 1/ 1 2/ 7 11 - 11 100 - 100 1
+ 1/ 1 3/ 7 13 - 13 101 - 101 1
+ 1/ 1 4/ 7 15 - 15 102 - 102 1
+ 1/ 1 5/ 7 17 - 17 103 - 103 1
+ 1/ 1 6/ 7 19 - 19 104 - 104 1
+ 1/ 1 7/ 7 21 - 21 105 - 105 1
diff --git a/tests/f_compress_extent_tree_level/image.gz b/tests/f_compress_extent_tree_level/image.gz
new file mode 100644
index 0000000000000000000000000000000000000000..a552a586ce20e4fbc01d6f332ec800c5c5b6011f
GIT binary patch
literal 2581
zcmb2|=3rR!C_IFT`RyHR9}!0hh6gkEYCmX@`O&3zM1osdIA^Jlw(gQiEytLI<J9c9
z&L3%E^Y!W3JGZlUZ`ds<>4n?Hq`jLtg`|b&q_WGat95BG-FN>tx4!H<U)lS-AJ6lO
z6&ZG4&Dp8pWHDvdcEgsixcDa#cN4c|Jrj3xXS?6icVxq+qB_I2E!8u;UtfEt{cM|d
zx!<y9uDd4Byy90XBb_r>eqZ_jr)RIH>*?If{BP|Sv-8%k+sAkPd2;YL_u=Paa{ab8
z7R65wT$X5aDi6&4{msJ6yZ%?_>ox0_zxZ;6nIVCD%G3ArMBKw){||e0eSc1N>&EnR
z0t^fcIrZA@uO9}lJ$3)OSwS^W;KBazx_|d&lMDq`Wi#$_4&>npx&GlDckiQ5TVJnz
zJo$QFjI93l)(@*BAG_wy2C6@>?b21V|Nk$ndG;T~<GcJ?1V}ghv<H$2|M-C<2;?k1
z7G&FVWc^>G+?H3>jHhdCT>a0f2bmPx&0M7aJXrJFvl^?T|K`LORWSmc@PEdCYgULx
zh7E5`r)9;j`(JJgG7%fdIpei?Ey$ub7WWu|+Q|hE%FpQb^sPCXXZ5ap`Q=61t^WO#
zxjHTCMMSQ(lh5g6K|Ejc^Tfly|0w>exlI4=-q>Te9zVFr@rQf!_Dhd`Ua)#xf9CQ1
zW2IZaKK4Jh|J#3OInf{H^QM12f1&DYykOsxU+Qau|6eK2ar?V@>iYlgPk)$;9sa}r
zHJ&GZ|LXr;C41uCH=MikZRv}1mHu*%)xaV1=-A3zpVnWgPo1*Aeytb_x&9i}HyQ$?
fAut*OqaiRF0;3^7AOs$KuVp*dzh0JsL4g4PCSfgc
literal 0
HcmV?d00001
diff --git a/tests/f_compress_extent_tree_level/name b/tests/f_compress_extent_tree_level/name
new file mode 100644
index 0000000..fde4f4a
--- /dev/null
+++ b/tests/f_compress_extent_tree_level/name
@@ -0,0 +1 @@
+compress an extent tree level
diff --git a/tests/f_compress_extent_tree_level/script b/tests/f_compress_extent_tree_level/script
new file mode 100644
index 0000000..ee18438
--- /dev/null
+++ b/tests/f_compress_extent_tree_level/script
@@ -0,0 +1,118 @@
+if [ "$DESCRIPTION"x != x ]; then
+ test_description="$DESCRIPTION"
+fi
+if [ "$IMAGE"x = x ]; then
+ IMAGE=$test_dir/image.gz
+fi
+
+if [ "$FSCK_OPT"x = x ]; then
+ FSCK_OPT=-yf
+fi
+
+if [ "$SECOND_FSCK_OPT"x = x ]; then
+ SECOND_FSCK_OPT=-yf
+fi
+
+if [ "$OUT1"x = x ]; then
+ OUT1=$test_name.1.log
+fi
+
+if [ "$OUT2"x = x ]; then
+ OUT2=$test_name.2.log
+fi
+
+if [ "$EXP1"x = x ]; then
+ if [ -f $test_dir/expect.1.gz ]; then
+ EXP1=$test_name.1.tmp
+ gunzip < $test_dir/expect.1.gz > $EXP1
+ else
+ EXP1=$test_dir/expect.1
+ fi
+fi
+
+if [ "$EXP2"x = x ]; then
+ if [ -f $test_dir/expect.2.gz ]; then
+ EXP2=$test_name.2.tmp
+ gunzip < $test_dir/expect.2.gz > $EXP2
+ else
+ EXP2=$test_dir/expect.2
+ fi
+fi
+
+if [ "$SKIP_GUNZIP" != "true" ] ; then
+ gunzip < $IMAGE > $TMPFILE
+fi
+
+cp /dev/null $OUT1
+
+eval $PREP_CMD
+
+echo 'ex /a' > $TMPFILE.cmd
+$DEBUGFS -f $TMPFILE.cmd $TMPFILE > $OUT1.new 2>&1
+rm -rf $TMPFILE.cmd
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT1.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT1.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT1.new >> $OUT1
+rm -f $OUT1.new
+
+if [ "$ONE_PASS_ONLY" != "true" ]; then
+ $FSCK $SECOND_FSCK_OPT -N test_filesys $TMPFILE > $OUT2.new 2>&1
+ status=$?
+ echo Exit status is $status >> $OUT2.new
+ echo 'ex /a' > $TMPFILE.cmd
+ $DEBUGFS -f $TMPFILE.cmd $TMPFILE >> $OUT2.new 2>&1
+ rm -rf $TMPFILE.cmd
+ sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT2.new > $OUT2
+ rm -f $OUT2.new
+fi
+
+eval $AFTER_CMD
+
+if [ "$SKIP_VERIFY" != "true" ] ; then
+ rm -f $test_name.ok $test_name.failed
+ cmp -s $OUT1 $EXP1
+ status1=$?
+ if [ "$ONE_PASS_ONLY" != "true" ]; then
+ cmp -s $OUT2 $EXP2
+ status2=$?
+ else
+ status2=0
+ fi
+ if [ "$PASS_ZERO" = "true" ]; then
+ cmp -s $test_name.0.log $test_dir/expect.0
+ status3=$?
+ else
+ status3=0
+ fi
+
+ if [ -z "$test_description" ] ; then
+ description="$test_name"
+ else
+ description="$test_name: $test_description"
+ fi
+
+ if [ "$status1" -eq 0 -a "$status2" -eq 0 -a "$status3" -eq 0 ] ; then
+ echo "$description: ok"
+ touch $test_name.ok
+ else
+ echo "$description: failed"
+ rm -f $test_name.failed
+ if [ "$PASS_ZERO" = "true" ]; then
+ diff $DIFF_OPTS $test_dir/expect.0 \
+ $test_name.0.log >> $test_name.failed
+ fi
+ diff $DIFF_OPTS $EXP1 $OUT1 >> $test_name.failed
+ if [ "$ONE_PASS_ONLY" != "true" ]; then
+ diff $DIFF_OPTS $EXP2 $OUT2 >> $test_name.failed
+ fi
+ fi
+ rm -f tmp_expect
+fi
+
+if [ "$SKIP_CLEANUP" != "true" ] ; then
+ unset IMAGE FSCK_OPT SECOND_FSCK_OPT OUT1 OUT2 EXP1 EXP2
+ unset SKIP_VERIFY SKIP_CLEANUP SKIP_GUNZIP ONE_PASS_ONLY PREP_CMD
+ unset DESCRIPTION SKIP_UNLINK AFTER_CMD PASS_ZERO
+fi
+
diff --git a/tests/f_convert_bmap/expect.1 b/tests/f_convert_bmap/expect.1
new file mode 100644
index 0000000..7d2ca86
--- /dev/null
+++ b/tests/f_convert_bmap/expect.1
@@ -0,0 +1,26 @@
+debugfs: stat /a
+Inode: 12 Type: regular Mode: 0644 Flags: 0x0
+Generation: 1573716129 Version: 0x00000000:00000001
+User: 0 Group: 0 Size: 524288
+File ACL: 0 Directory ACL: 0
+Links: 1 Blockcount: 1030
+Fragment: Address: 0 Number: 0 Size: 0
+ ctime: 0x5457f87a:62ae2980 -- Mon Nov 3 21:49:46 2014
+ atime: 0x5457f87a:61ba0598 -- Mon Nov 3 21:49:46 2014
+ mtime: 0x5457f87a:62ae2980 -- Mon Nov 3 21:49:46 2014
+crtime: 0x5457f87a:61ba0598 -- Mon Nov 3 21:49:46 2014
+Size of extra inode fields: 28
+BLOCKS:
+(0-11):1025-1036, (IND):24, (12-267):1037-1292, (DIND):25, (IND):41, (268-511):1293-1536
+TOTAL: 515
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 1E: Optimizing extent trees
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 12/128 files (8.3% non-contiguous), 570/2048 blocks
+Exit status is 1
diff --git a/tests/f_convert_bmap/expect.2 b/tests/f_convert_bmap/expect.2
new file mode 100644
index 0000000..632d411
--- /dev/null
+++ b/tests/f_convert_bmap/expect.2
@@ -0,0 +1,10 @@
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+test_filesys: 12/128 files (0.0% non-contiguous), 570/2048 blocks
+Exit status is 0
+debugfs: ex /a
+Level Entries Logical Physical Length Flags
+ 0/ 0 1/ 1 0 - 511 1025 - 1536 512
diff --git a/tests/f_convert_bmap/image.gz b/tests/f_convert_bmap/image.gz
new file mode 100644
index 0000000000000000000000000000000000000000..7c22532397ba8d42e928f75190dc545cae2140cf
GIT binary patch
literal 3548
zcmeH{TTD|27{|LAMF+@(0Tox+B<ivR0l73*3mPNpq{>AKJy;kpC>$)zXsxs>&77h{
z5d{R1Tf|{+oO3i%(L%x52w{Q@ZBMl=7l%+hhteTCTuLv?;vP1$_wAwI!}sN%d{4js
zKY#y=D@kV;l8$`5%sjNJl`~FyG$XVKkeS>DH}6f}NR0H7XZeLMHZ~n7S%IBv`1#=N
zvthAe6*sSJFDc(vQMvN!_C4jFe<j65!*^OUm={)9=G$@69F1o=F+fw#*9&#jyvB6W
zL}KW^E$$xb{J^bs+dN*RTpr^8kjSR!^SeC_oE_k?>(^gS=<hNObc@YMH>Kvk-p-E5
z35ksKaBDTb(PT!@CVwq0qb~h&i*n<*7OXbTS(15(N<89733W!X1W$_EjVBkiwdO+O
zgdm1$#XPIm+f@lYY^0qqFBCAP$X(HIv%QKlVn5zT?)`;ZVOpN;uzxq3w!?I(<aK$X
zaS!SJe5<2`goNateEzncYj1mM?y%OZ3HI8cvU9NDh)8IB%F@##rq|m2I-85nb<Z2w
zU2pNJ`vVSvg$htSs_DKP485(ZoNIkL`xm<MpHIzr^#yEhcYn0tc1iSaYjk?luhhO5
z90Xc?kWFAOAghA2z*hiVg?NKc04W0&pkFFLh%qk{=rBd}kQi-QLIK4oC}Cc{q-zD*
zP`R6C8NQns<?n~mydY0dt(W*QypVmG06dPE)IW%j((~@|V>qd0P#|_(kwxolh)vP1
z!4D|7w1*Ax2F-m!!v{D?hY_}Bj_Bhv%&`NoNpo3|U_RQefeE7?I)#|DeuRy`+sqVb
zhZToacT$fmp+(`UIb^NwAlA&?)i0^me$S019}PlkxcgO2tDz)Rj%kS-d=8m`$kjMO
z6!SC5aRssfTtZb|mQr*n*h?xr3>3)6@Uzsrhh!CaB~>w;YLUFa>Is<7Q;82Dp_q#3
z<csQ={t_?rJf<eDT62Ug&tzq~S40U_9La2yoxnS+C+pK1rS8~c>oFl5nU*FALaVW-
z#5F6(I+7;8h~wM?EMP?P6ssj5Wk)A#L~23336-^o#f*~pqh(kRQDM!sx4^PST@Y4H
z$d$?>mQGWE>8%_)T$Cn~;8+<UR(9q~qI9L)1bcHnoC0S2?i-o4t{d#wtbKDMepYK!
z><UbF*FFwcpxc%|-kFy%XsC0WSf9P?uV}qL5AcUQmFdYp`an;>8K9}Er5St++^&Mg
z!A0OEqvaYnu4F^;aunnuN*<J&rxAkvO3UQwWiHGAd&`@O7ag;OUt9kyt+v8kM|<{I
zIyy5D-6rU+%SLL#@YyT-x{F^mR@l1kJP-CwGEIen(_^EFs~!aO?fhmP`84O<$bd6b
zozXVSD_9JACTbIs-$@>k$83X+0lqWGS@2!*G{ZnA+mYmpn#n71w;rbdEhKizylA>_
zMx@ViZ17o^Z5wioybK<B(`9y+686({S#NlAG;TtcgFgWhA6R-TGwW2%xanw*RK0hH
z?Vcg(1Qg2i3)j1S%5oZoyZ;f(55(v*iZt4vOKqNzFP<OV(zIHAX4?0IAD46ya1n43
aa1n43a1r?55SY$(e6qN4MIedv8R-w=i-Ai3
literal 0
HcmV?d00001
diff --git a/tests/f_convert_bmap/name b/tests/f_convert_bmap/name
new file mode 100644
index 0000000..67e0d47
--- /dev/null
+++ b/tests/f_convert_bmap/name
@@ -0,0 +1 @@
+convert blockmap file to extents file
diff --git a/tests/f_convert_bmap/script b/tests/f_convert_bmap/script
new file mode 100644
index 0000000..c463e3d
--- /dev/null
+++ b/tests/f_convert_bmap/script
@@ -0,0 +1,119 @@
+if [ "$DESCRIPTION"x != x ]; then
+ test_description="$DESCRIPTION"
+fi
+if [ "$IMAGE"x = x ]; then
+ IMAGE=$test_dir/image.gz
+fi
+
+if [ "$FSCK_OPT"x = x ]; then
+ FSCK_OPT=-yf
+fi
+
+if [ "$SECOND_FSCK_OPT"x = x ]; then
+ SECOND_FSCK_OPT=-yf
+fi
+
+if [ "$OUT1"x = x ]; then
+ OUT1=$test_name.1.log
+fi
+
+if [ "$OUT2"x = x ]; then
+ OUT2=$test_name.2.log
+fi
+
+if [ "$EXP1"x = x ]; then
+ if [ -f $test_dir/expect.1.gz ]; then
+ EXP1=$test_name.1.tmp
+ gunzip < $test_dir/expect.1.gz > $EXP1
+ else
+ EXP1=$test_dir/expect.1
+ fi
+fi
+
+if [ "$EXP2"x = x ]; then
+ if [ -f $test_dir/expect.2.gz ]; then
+ EXP2=$test_name.2.tmp
+ gunzip < $test_dir/expect.2.gz > $EXP2
+ else
+ EXP2=$test_dir/expect.2
+ fi
+fi
+
+if [ "$SKIP_GUNZIP" != "true" ] ; then
+ gunzip < $IMAGE > $TMPFILE
+fi
+
+cp /dev/null $OUT1
+
+eval $PREP_CMD
+
+echo 'stat /a' > $TMPFILE.cmd
+$DEBUGFS -f $TMPFILE.cmd $TMPFILE > $OUT1.new 2>&1
+rm -rf $TMPFILE.cmd
+$TUNE2FS -O extent $TMPFILE >> $OUT1.new 2>&1
+$FSCK $FSCK_OPT -E bmap2extent -N test_filesys $TMPFILE >> $OUT1.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT1.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT1.new >> $OUT1
+rm -f $OUT1.new
+
+if [ "$ONE_PASS_ONLY" != "true" ]; then
+ $FSCK $SECOND_FSCK_OPT -N test_filesys $TMPFILE > $OUT2.new 2>&1
+ status=$?
+ echo Exit status is $status >> $OUT2.new
+ echo 'ex /a' > $TMPFILE.cmd
+ $DEBUGFS -f $TMPFILE.cmd $TMPFILE >> $OUT2.new 2>&1
+ rm -rf $TMPFILE.cmd
+ sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT2.new > $OUT2
+ rm -f $OUT2.new
+fi
+
+eval $AFTER_CMD
+
+if [ "$SKIP_VERIFY" != "true" ] ; then
+ rm -f $test_name.ok $test_name.failed
+ cmp -s $OUT1 $EXP1
+ status1=$?
+ if [ "$ONE_PASS_ONLY" != "true" ]; then
+ cmp -s $OUT2 $EXP2
+ status2=$?
+ else
+ status2=0
+ fi
+ if [ "$PASS_ZERO" = "true" ]; then
+ cmp -s $test_name.0.log $test_dir/expect.0
+ status3=$?
+ else
+ status3=0
+ fi
+
+ if [ -z "$test_description" ] ; then
+ description="$test_name"
+ else
+ description="$test_name: $test_description"
+ fi
+
+ if [ "$status1" -eq 0 -a "$status2" -eq 0 -a "$status3" -eq 0 ] ; then
+ echo "$description: ok"
+ touch $test_name.ok
+ else
+ echo "$description: failed"
+ rm -f $test_name.failed
+ if [ "$PASS_ZERO" = "true" ]; then
+ diff $DIFF_OPTS $test_dir/expect.0 \
+ $test_name.0.log >> $test_name.failed
+ fi
+ diff $DIFF_OPTS $EXP1 $OUT1 >> $test_name.failed
+ if [ "$ONE_PASS_ONLY" != "true" ]; then
+ diff $DIFF_OPTS $EXP2 $OUT2 >> $test_name.failed
+ fi
+ fi
+ rm -f tmp_expect
+fi
+
+if [ "$SKIP_CLEANUP" != "true" ] ; then
+ unset IMAGE FSCK_OPT SECOND_FSCK_OPT OUT1 OUT2 EXP1 EXP2
+ unset SKIP_VERIFY SKIP_CLEANUP SKIP_GUNZIP ONE_PASS_ONLY PREP_CMD
+ unset DESCRIPTION SKIP_UNLINK AFTER_CMD PASS_ZERO
+fi
+
diff --git a/tests/f_extent_too_deep/expect.1 b/tests/f_extent_too_deep/expect.1
new file mode 100644
index 0000000..a595482
--- /dev/null
+++ b/tests/f_extent_too_deep/expect.1
@@ -0,0 +1,23 @@
+debugfs: ex /a
+Level Entries Logical Physical Length Flags
+ 0/ 7 1/ 1 0 - 0 12 1
+ 1/ 7 1/ 1 0 - 0 13 1
+ 2/ 7 1/ 1 0 - 0 14 1
+ 3/ 7 1/ 1 0 - 0 15 1
+ 4/ 7 1/ 1 0 - 0 16 1
+ 5/ 7 1/ 1 0 - 0 17 1
+ 6/ 7 1/ 1 0 - 0 9 1
+ 7/ 7 1/ 1 0 - 0 10 - 10 1
+Pass 1: Checking inodes, blocks, and sizes
+Inode 12 extent tree could be more shallow (7; could be <= 4)
+Fix? yes
+
+Pass 1E: Optimizing extent trees
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 12/128 files (0.0% non-contiguous), 19/512 blocks
+Exit status is 1
diff --git a/tests/f_extent_too_deep/expect.2 b/tests/f_extent_too_deep/expect.2
new file mode 100644
index 0000000..a1d28b1
--- /dev/null
+++ b/tests/f_extent_too_deep/expect.2
@@ -0,0 +1,10 @@
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+test_filesys: 12/128 files (0.0% non-contiguous), 19/512 blocks
+Exit status is 0
+debugfs: ex /a
+Level Entries Logical Physical Length Flags
+ 0/ 0 1/ 1 0 - 0 10 - 10 1
diff --git a/tests/f_extent_too_deep/image.gz b/tests/f_extent_too_deep/image.gz
new file mode 100644
index 0000000000000000000000000000000000000000..0f5adff562c7f45f275a4401344e784b6c61ef0b
GIT binary patch
literal 2592
zcmb2|=3wx?6duCF{PymCf8jtGh7aG@g)R_rT-?mre9>1}CPu>N$ctAhoXss6N_=iF
z5@Jgn)7j0{g?;~U$MjrWSXgN2BJ`haZVz)`*522$TUX9#P_RGs-1*+#-}m|t)ke$R
z5APFT+7Par(yh|6IP<$^TY&yHb=Pd!4HL>YcLz^?KYf{6a)z0HeXpXe&6y>)!{oHZ
z{w4_aEWdmEi{rbce>>ZI-7<e&n#^wgT>k9*+V6{hzqYV7kN<!F`meNSFQ+Q6*V|Y3
z^T*9EtryRSC(q!xs&fB&rR?lflfNCSbM%+z9`NR8Xvm!~b$|P%vOS^yuV3DA=UzC!
zdV2~70|P_Z{%7a+eb~FD@iQX>LxZgT@4xk?M^bd2d}j)Ap48L(^!o=nzNt^b-j@Bn
z(tB&}&(}Bi3E1%+uQ+>K4XAm;YM!#Q|M&Ag|6C8`EjZre0W^eR#(!oY`RqTC0D%K#
zt8|VVNY)>}nWdV0b3e-(v5!&LK$gwTUKeJq&J9t;uwnPp@BiQDA9oI70*csL0e#oU
z@4*RVe|hm||DC?}r7S>pRms2jtmDcsHU76gI%<N<lAHVgc$tlYDp1LV;#slvAMS%y
z_mw*UMaTsa{WFiQJUMmS^mV^a|12?_6#qZx;wRgs7w*6M?d&ss>QjkRvd@oR6O=Ze
zwT(YrEiNVZY2}A3mTiX){VlmTBO>)r|Ir`c-?Qz0^ke$R^}nA_NN3&Vro6gdEo}Aw
zNd;?G)F0CN>c2B%cHsY<=imQ-{{3@@$#3_+;+waWT&=Ht`+1dYa@5urll#5~9k0rL
z`R+&5tNm+I=hrV419@x|jE2By2#kinXb6mkz-S22A_Nlt)-hg8%bdf&puhkC@B%oQ
literal 0
HcmV?d00001
diff --git a/tests/f_extent_too_deep/name b/tests/f_extent_too_deep/name
new file mode 100644
index 0000000..7e8654a
--- /dev/null
+++ b/tests/f_extent_too_deep/name
@@ -0,0 +1 @@
+extent tree is deeper than it needs to be
diff --git a/tests/f_extent_too_deep/script b/tests/f_extent_too_deep/script
new file mode 100644
index 0000000..ee18438
--- /dev/null
+++ b/tests/f_extent_too_deep/script
@@ -0,0 +1,118 @@
+if [ "$DESCRIPTION"x != x ]; then
+ test_description="$DESCRIPTION"
+fi
+if [ "$IMAGE"x = x ]; then
+ IMAGE=$test_dir/image.gz
+fi
+
+if [ "$FSCK_OPT"x = x ]; then
+ FSCK_OPT=-yf
+fi
+
+if [ "$SECOND_FSCK_OPT"x = x ]; then
+ SECOND_FSCK_OPT=-yf
+fi
+
+if [ "$OUT1"x = x ]; then
+ OUT1=$test_name.1.log
+fi
+
+if [ "$OUT2"x = x ]; then
+ OUT2=$test_name.2.log
+fi
+
+if [ "$EXP1"x = x ]; then
+ if [ -f $test_dir/expect.1.gz ]; then
+ EXP1=$test_name.1.tmp
+ gunzip < $test_dir/expect.1.gz > $EXP1
+ else
+ EXP1=$test_dir/expect.1
+ fi
+fi
+
+if [ "$EXP2"x = x ]; then
+ if [ -f $test_dir/expect.2.gz ]; then
+ EXP2=$test_name.2.tmp
+ gunzip < $test_dir/expect.2.gz > $EXP2
+ else
+ EXP2=$test_dir/expect.2
+ fi
+fi
+
+if [ "$SKIP_GUNZIP" != "true" ] ; then
+ gunzip < $IMAGE > $TMPFILE
+fi
+
+cp /dev/null $OUT1
+
+eval $PREP_CMD
+
+echo 'ex /a' > $TMPFILE.cmd
+$DEBUGFS -f $TMPFILE.cmd $TMPFILE > $OUT1.new 2>&1
+rm -rf $TMPFILE.cmd
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT1.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT1.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT1.new >> $OUT1
+rm -f $OUT1.new
+
+if [ "$ONE_PASS_ONLY" != "true" ]; then
+ $FSCK $SECOND_FSCK_OPT -N test_filesys $TMPFILE > $OUT2.new 2>&1
+ status=$?
+ echo Exit status is $status >> $OUT2.new
+ echo 'ex /a' > $TMPFILE.cmd
+ $DEBUGFS -f $TMPFILE.cmd $TMPFILE >> $OUT2.new 2>&1
+ rm -rf $TMPFILE.cmd
+ sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT2.new > $OUT2
+ rm -f $OUT2.new
+fi
+
+eval $AFTER_CMD
+
+if [ "$SKIP_VERIFY" != "true" ] ; then
+ rm -f $test_name.ok $test_name.failed
+ cmp -s $OUT1 $EXP1
+ status1=$?
+ if [ "$ONE_PASS_ONLY" != "true" ]; then
+ cmp -s $OUT2 $EXP2
+ status2=$?
+ else
+ status2=0
+ fi
+ if [ "$PASS_ZERO" = "true" ]; then
+ cmp -s $test_name.0.log $test_dir/expect.0
+ status3=$?
+ else
+ status3=0
+ fi
+
+ if [ -z "$test_description" ] ; then
+ description="$test_name"
+ else
+ description="$test_name: $test_description"
+ fi
+
+ if [ "$status1" -eq 0 -a "$status2" -eq 0 -a "$status3" -eq 0 ] ; then
+ echo "$description: ok"
+ touch $test_name.ok
+ else
+ echo "$description: failed"
+ rm -f $test_name.failed
+ if [ "$PASS_ZERO" = "true" ]; then
+ diff $DIFF_OPTS $test_dir/expect.0 \
+ $test_name.0.log >> $test_name.failed
+ fi
+ diff $DIFF_OPTS $EXP1 $OUT1 >> $test_name.failed
+ if [ "$ONE_PASS_ONLY" != "true" ]; then
+ diff $DIFF_OPTS $EXP2 $OUT2 >> $test_name.failed
+ fi
+ fi
+ rm -f tmp_expect
+fi
+
+if [ "$SKIP_CLEANUP" != "true" ] ; then
+ unset IMAGE FSCK_OPT SECOND_FSCK_OPT OUT1 OUT2 EXP1 EXP2
+ unset SKIP_VERIFY SKIP_CLEANUP SKIP_GUNZIP ONE_PASS_ONLY PREP_CMD
+ unset DESCRIPTION SKIP_UNLINK AFTER_CMD PASS_ZERO
+fi
+
diff --git a/tests/f_opt_extent/expect b/tests/f_opt_extent/expect
new file mode 100644
index 0000000..6d4863b
--- /dev/null
+++ b/tests/f_opt_extent/expect
@@ -0,0 +1,55 @@
+tune2fs metadata_csum test
+Creating filesystem with 524288 1k blocks and 65536 inodes
+Superblock backups stored on blocks:
+ 8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409
+
+Allocating group tables: \b\b\b\b\bdone
+Writing inode tables: \b\b\b\b\bdone
+Creating journal (16384 blocks): done
+Creating 477 huge file(s) with 1024 blocks each: done
+Writing superblocks and filesystem accounting information: \b\b\b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 3A: Optimizing directories
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+
+
+Change in FS metadata:
+@@ -10,7 +10,7 @@
+ Inode count: 65536
+ Block count: 524288
+ Reserved block count: 26214
+-Free blocks: 570
++Free blocks: 567
+ Free inodes: 65047
+ First block: 1
+ Block size: 1024
+@@ -47,8 +47,8 @@
+ Block bitmap at 262 (+261)
+ Inode bitmap at 278 (+277)
+ Inode table at 294-549 (+293)
+- 21 free blocks, 535 free inodes, 3 directories, 535 unused inodes
+- Free blocks: 4414-4434
++ 18 free blocks, 535 free inodes, 3 directories, 535 unused inodes
++ Free blocks: 4417-4434
+ Free inodes: 490-1024
+ Group 1: (Blocks 8193-16384) [INODE_UNINIT]
+ Backup superblock at 8193, Group descriptors at 8194-8197
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
diff --git a/tests/f_opt_extent/name b/tests/f_opt_extent/name
new file mode 100644
index 0000000..7d4389c
--- /dev/null
+++ b/tests/f_opt_extent/name
@@ -0,0 +1 @@
+optimize extent tree
diff --git a/tests/f_opt_extent/script b/tests/f_opt_extent/script
new file mode 100644
index 0000000..2da5e91
--- /dev/null
+++ b/tests/f_opt_extent/script
@@ -0,0 +1,64 @@
+FSCK_OPT=-fn
+OUT=$test_name.log
+EXP=$test_dir/expect
+CONF=$TMPFILE.conf
+
+cat > $CONF << ENDL
+[fs_types]
+ ext4h = {
+ features = has_journal,extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,sparse_super,filetype,dir_index,ext_attr,resize_inode,64bit,metadata_csum
+ blocksize = 1024
+ inode_size = 256
+ make_hugefiles = true
+ hugefiles_dir = /xyz
+ hugefiles_slack = 0
+ hugefiles_name = aaaaa
+ hugefiles_digits = 4
+ hugefiles_size = 1M
+ zero_hugefiles = false
+ }
+ENDL
+
+echo "tune2fs metadata_csum test" > $OUT
+
+MKE2FS_CONFIG=$CONF $MKE2FS -F -T ext4h $TMPFILE 524288 >> $OUT 2>&1
+rm -rf $CONF
+
+# dump and check
+$DUMPE2FS $TMPFILE 2> /dev/null | grep '^Group 0:' -B99 -A20 | sed -f $cmd_dir/filter.sed > $OUT.before
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# check
+$FSCK -fyD -N test_filesys $TMPFILE >> $OUT 2>&1
+
+# dump and check
+$DUMPE2FS $TMPFILE 2> /dev/null | grep '^Group 0:' -B99 -A20 | sed -f $cmd_dir/filter.sed > $OUT.after
+echo "Change in FS metadata:" >> $OUT
+diff -u $OUT.before $OUT.after | tail -n +3 >> $OUT
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+rm $TMPFILE $OUT.before $OUT.after
+
+#
+# Do the verification
+#
+
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" -e 's/test_filesys:.*//g' < $OUT > $OUT.new
+mv $OUT.new $OUT
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+fi
+
+unset IMAGE FSCK_OPT OUT EXP CONF
diff --git a/tests/f_opt_extent_ext3/expect b/tests/f_opt_extent_ext3/expect
new file mode 100644
index 0000000..de1fdf6
--- /dev/null
+++ b/tests/f_opt_extent_ext3/expect
@@ -0,0 +1,45 @@
+rebuild extent metadata_csum test
+Creating filesystem with 524288 1k blocks and 65536 inodes
+Superblock backups stored on blocks:
+ 8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409
+
+Allocating group tables: \b\b\b\b\bdone
+Writing inode tables: \b\b\b\b\bdone
+Creating journal (16384 blocks): done
+mke2fs: Operation not supported for inodes containing extents while creating huge files
+Writing superblocks and filesystem accounting information: \b\b\b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
+Pass 1: Checking inodes, blocks, and sizes
+Pass 1E: Optimizing extent trees
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 3A: Optimizing directories
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+
+
+Change in FS metadata:
+@@ -2,7 +2,7 @@
+ Last mounted on: <not available>
+ Filesystem magic number: 0xEF53
+ Filesystem revision #: 1 (dynamic)
+-Filesystem features: has_journal ext_attr resize_inode dir_index filetype sparse_super large_file huge_file dir_nlink
++Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent sparse_super large_file huge_file dir_nlink
+ Default mount options: user_xattr acl
+ Filesystem state: clean
+ Errors behavior: Continue
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
diff --git a/tests/f_opt_extent_ext3/name b/tests/f_opt_extent_ext3/name
new file mode 100644
index 0000000..b369685
--- /dev/null
+++ b/tests/f_opt_extent_ext3/name
@@ -0,0 +1 @@
+convert ext3 to extent tree
diff --git a/tests/f_opt_extent_ext3/script b/tests/f_opt_extent_ext3/script
new file mode 100644
index 0000000..931eae7
--- /dev/null
+++ b/tests/f_opt_extent_ext3/script
@@ -0,0 +1,65 @@
+FSCK_OPT=-fn
+OUT=$test_name.log
+EXP=$test_dir/expect
+CONF=$TMPFILE.conf
+
+cat > $CONF << ENDL
+[fs_types]
+ ext4h = {
+ features = has_journal,^extent,huge_file,^flex_bg,^uninit_bg,dir_nlink,^extra_isize,sparse_super,filetype,dir_index,ext_attr,resize_inode,^64bit,^metadata_csum
+ blocksize = 1024
+ inode_size = 256
+ make_hugefiles = true
+ hugefiles_dir = /
+ num_hugefiles = 100
+ hugefiles_slack = 0
+ hugefiles_name = aaaaa
+ hugefiles_digits = 4
+ hugefiles_size = 1M
+ zero_hugefiles = false
+ }
+ENDL
+
+echo "rebuild extent metadata_csum test" > $OUT
+
+MKE2FS_CONFIG=$CONF $MKE2FS -F -T ext4h $TMPFILE 524288 >> $OUT 2>&1
+rm -rf $CONF
+
+# dump and check
+$DUMPE2FS $TMPFILE 2> /dev/null | grep '^Group 0:' -B99 -A20 | sed -f $cmd_dir/filter.sed > $OUT.before
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# check
+$FSCK -fyD -N test_filesys -E bmap2extent $TMPFILE >> $OUT 2>&1
+
+# dump and check
+$DUMPE2FS $TMPFILE 2> /dev/null | grep '^Group 0:' -B99 -A20 | sed -f $cmd_dir/filter.sed > $OUT.after
+echo "Change in FS metadata:" >> $OUT
+diff -u $OUT.before $OUT.after | tail -n +3 >> $OUT
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+rm $TMPFILE $OUT.before $OUT.after
+
+#
+# Do the verification
+#
+
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" -e 's/test_filesys:.*//g' < $OUT > $OUT.new
+mv $OUT.new $OUT
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+fi
+
+unset IMAGE FSCK_OPT OUT EXP CONF
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 35/47] resize2fs: convert fs to and from 64bit mode
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (33 preceding siblings ...)
2014-11-07 21:54 ` [PATCH 34/47] tests: verify proper rebuilding of sparse extent trees and block map file conversion Darrick J. Wong
@ 2014-11-07 21:54 ` Darrick J. Wong
2014-12-15 3:32 ` Theodore Ts'o
2014-12-15 17:46 ` Theodore Ts'o
2014-11-07 21:54 ` [PATCH 36/47] tests: test resize2fs 32->64 and 64->32bit conversion code Darrick J. Wong
` (13 subsequent siblings)
48 siblings, 2 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:54 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4, TR Reardon
resize2fs does its magic by loading a filesystem, duplicating the
in-memory image of that fs, moving relevant blocks out of the way of
whatever new metadata get created, and finally writing everything back
out to disk. Enabling 64bit mode enlarges the group descriptors,
which makes resize2fs a reasonable vehicle for taking care of the rest
of the bookkeeping requirements, so add to resize2fs the ability to
convert a filesystem to 64bit mode and back.
v2: Fix a couple of minor resource leaks; make it more explicit to the
user what we're doing to the fs.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Cc: TR Reardon <thomas_reardon@hotmail.com>
---
misc/tune2fs.c | 7 +
resize/main.c | 60 ++++++
resize/resize2fs.8.in | 18 ++
resize/resize2fs.c | 362 ++++++++++++++++++++++++++++++++++++--
resize/resize2fs.h | 3
tests/t_enable_mcsum_ext3/expect | 1
6 files changed, 428 insertions(+), 23 deletions(-)
diff --git a/misc/tune2fs.c b/misc/tune2fs.c
index 1c83d09..da5030b 100644
--- a/misc/tune2fs.c
+++ b/misc/tune2fs.c
@@ -1121,6 +1121,13 @@ mmp_error:
"cannot. Not enabling extents reduces the "
"coverage of metadata checksumming. "
"Re-run with -O extent to rectify.\n"));
+ if (!EXT2_HAS_INCOMPAT_FEATURE(fs->super,
+ EXT4_FEATURE_INCOMPAT_64BIT))
+ printf("%s",
+ _("64-bit filesystem support is not enabled. "
+ "The larger fields afforded by this feature "
+ "enable full-strength checksumming. "
+ "Run resize2fs -b to rectify.\n"));
rewrite_checksums = 1;
/* metadata_csum supersedes uninit_bg */
fs->super->s_feature_ro_compat &=
diff --git a/resize/main.c b/resize/main.c
index e017792..f5cdab9 100644
--- a/resize/main.c
+++ b/resize/main.c
@@ -42,7 +42,7 @@ static char *device_name, *io_options;
static void usage (char *prog)
{
fprintf (stderr, _("Usage: %s [-d debug_flags] [-f] [-F] [-M] [-P] "
- "[-p] device [new_size]\n\n"), prog);
+ "[-p] device [-b|-s|new_size]\n\n"), prog);
exit (1);
}
@@ -203,7 +203,7 @@ int main (int argc, char ** argv)
if (argc && *argv)
program_name = *argv;
- while ((c = getopt (argc, argv, "d:fFhMPpS:")) != EOF) {
+ while ((c = getopt(argc, argv, "d:fFhMPpS:bs")) != EOF) {
switch (c) {
case 'h':
usage(program_name);
@@ -229,6 +229,12 @@ int main (int argc, char ** argv)
case 'S':
use_stride = atoi(optarg);
break;
+ case 'b':
+ flags |= RESIZE_ENABLE_64BIT;
+ break;
+ case 's':
+ flags |= RESIZE_DISABLE_64BIT;
+ break;
default:
usage(program_name);
}
@@ -393,6 +399,10 @@ int main (int argc, char ** argv)
if (sys_page_size > blocksize)
new_size &= ~((sys_page_size / blocksize)-1);
}
+ /* If changing 64bit, don't change the filesystem size. */
+ if (flags & (RESIZE_DISABLE_64BIT | RESIZE_ENABLE_64BIT)) {
+ new_size = ext2fs_blocks_count(fs->super);
+ }
if (!EXT2_HAS_INCOMPAT_FEATURE(fs->super,
EXT4_FEATURE_INCOMPAT_64BIT)) {
/* Take 16T down to 2^32-1 blocks */
@@ -444,20 +454,58 @@ int main (int argc, char ** argv)
blocksize / 1024, new_size);
exit(1);
}
- if (new_size == ext2fs_blocks_count(fs->super)) {
+ if ((flags & RESIZE_DISABLE_64BIT) && (flags & RESIZE_ENABLE_64BIT)) {
+ fprintf(stderr, _("Cannot set and unset 64bit feature.\n"));
+ exit(1);
+ } else if (flags & (RESIZE_DISABLE_64BIT | RESIZE_ENABLE_64BIT)) {
+ if (new_size >= (1ULL << 32)) {
+ fprintf(stderr, _("Cannot change the 64bit feature "
+ "on a filesystem that is larger than "
+ "2^32 blocks.\n"));
+ exit(1);
+ }
+ if (mount_flags & EXT2_MF_MOUNTED) {
+ fprintf(stderr, _("Cannot change the 64bit feature "
+ "while the filesystem is mounted.\n"));
+ exit(1);
+ }
+ if (flags & RESIZE_ENABLE_64BIT &&
+ !EXT2_HAS_INCOMPAT_FEATURE(fs->super,
+ EXT3_FEATURE_INCOMPAT_EXTENTS)) {
+ fprintf(stderr, _("Please enable the extents feature "
+ "with tune2fs before enabling the 64bit "
+ "feature.\n"));
+ exit(1);
+ }
+ } else if (new_size == ext2fs_blocks_count(fs->super)) {
fprintf(stderr, _("The filesystem is already %llu (%dk) "
"blocks long. Nothing to do!\n\n"), new_size,
blocksize / 1024);
exit(0);
}
+ if ((flags & RESIZE_ENABLE_64BIT) &&
+ EXT2_HAS_INCOMPAT_FEATURE(fs->super, EXT4_FEATURE_INCOMPAT_64BIT)) {
+ fprintf(stderr, _("The filesystem is already 64-bit.\n"));
+ exit(0);
+ }
+ if ((flags & RESIZE_DISABLE_64BIT) &&
+ !EXT2_HAS_INCOMPAT_FEATURE(fs->super, EXT4_FEATURE_INCOMPAT_64BIT)) {
+ fprintf(stderr, _("The filesystem is already 32-bit.\n"));
+ exit(0);
+ }
if (mount_flags & EXT2_MF_MOUNTED) {
bigalloc_check(fs, force);
retval = online_resize_fs(fs, mtpt, &new_size, flags);
} else {
bigalloc_check(fs, force);
- printf(_("Resizing the filesystem on "
- "%s to %llu (%dk) blocks.\n"),
- device_name, new_size, blocksize / 1024);
+ if (flags & RESIZE_ENABLE_64BIT)
+ printf(_("Converting the filesystem to 64-bit.\n"));
+ else if (flags & RESIZE_DISABLE_64BIT)
+ printf(_("Converting the filesystem to 32-bit.\n"));
+ else
+ printf(_("Resizing the filesystem on "
+ "%s to %llu (%dk) blocks.\n"),
+ device_name, new_size, blocksize / 1024);
retval = resize_fs(fs, &new_size, flags,
((flags & RESIZE_PERCENT_COMPLETE) ?
resize_progress_func : 0));
diff --git a/resize/resize2fs.8.in b/resize/resize2fs.8.in
index 86495c6..0129bfc 100644
--- a/resize/resize2fs.8.in
+++ b/resize/resize2fs.8.in
@@ -8,7 +8,7 @@ resize2fs \- ext2/ext3/ext4 file system resizer
.SH SYNOPSIS
.B resize2fs
[
-.B \-fFpPM
+.B \-fFpPMbs
]
[
.B \-d
@@ -86,8 +86,21 @@ to shrink the size of filesystem. Then you may use
to shrink the size of the partition. When shrinking the size of
the partition, make sure you do not make it smaller than the new size
of the ext2 filesystem!
+.PP
+The
+.B \-b
+and
+.B \-s
+options enable and disable the 64bit feature, respectively. The resize2fs
+program will, of course, take care of resizing the block group descriptors
+and moving other data blocks out of the way, as needed. It is not possible
+to resize the filesystem concurrent with changing the 64bit status.
.SH OPTIONS
.TP
+.B \-b
+Turns on the 64bit feature, resizes the group descriptors as necessary, and
+moves other metadata out of the way.
+.TP
.B \-d \fIdebug-flags
Turns on various resize2fs debugging features, if they have been compiled
into the binary.
@@ -127,6 +140,9 @@ of what the program is doing.
.B \-P
Print the minimum size of the filesystem and exit.
.TP
+.B \-s
+Turns off the 64bit feature and frees blocks that are no longer in use.
+.TP
.B \-S \fIRAID-stride
The
.B resize2fs
diff --git a/resize/resize2fs.c b/resize/resize2fs.c
index 0d1f7fa..0787c8a 100644
--- a/resize/resize2fs.c
+++ b/resize/resize2fs.c
@@ -56,6 +56,9 @@ static errcode_t mark_table_blocks(ext2_filsys fs,
static errcode_t clear_sparse_super2_last_group(ext2_resize_t rfs);
static errcode_t reserve_sparse_super2_last_group(ext2_resize_t rfs,
ext2fs_block_bitmap meta_bmap);
+static errcode_t resize_group_descriptors(ext2_resize_t rfs, blk64_t new_size);
+static errcode_t move_bg_metadata(ext2_resize_t rfs);
+static errcode_t zero_high_bits_in_inodes(ext2_resize_t rfs);
/*
* Some helper CPP macros
@@ -122,13 +125,30 @@ errcode_t resize_fs(ext2_filsys fs, blk64_t *new_size, int flags,
if (retval)
goto errout;
+ init_resource_track(&rtrack, "resize_group_descriptors", fs->io);
+ retval = resize_group_descriptors(rfs, *new_size);
+ if (retval)
+ goto errout;
+ print_resource_track(rfs, &rtrack, fs->io);
+
+ init_resource_track(&rtrack, "move_bg_metadata", fs->io);
+ retval = move_bg_metadata(rfs);
+ if (retval)
+ goto errout;
+ print_resource_track(rfs, &rtrack, fs->io);
+
+ init_resource_track(&rtrack, "zero_high_bits_in_metadata", fs->io);
+ retval = zero_high_bits_in_inodes(rfs);
+ if (retval)
+ goto errout;
+ print_resource_track(rfs, &rtrack, fs->io);
+
init_resource_track(&rtrack, "adjust_superblock", fs->io);
retval = adjust_superblock(rfs, *new_size);
if (retval)
goto errout;
print_resource_track(rfs, &rtrack, fs->io);
-
init_resource_track(&rtrack, "fix_uninit_block_bitmaps 2", fs->io);
fix_uninit_block_bitmaps(rfs->new_fs);
print_resource_track(rfs, &rtrack, fs->io);
@@ -235,6 +255,322 @@ errout:
return retval;
}
+/* Keep the size of the group descriptor region constant */
+static void adjust_reserved_gdt_blocks(ext2_filsys old_fs, ext2_filsys fs)
+{
+ if ((fs->super->s_feature_compat &
+ EXT2_FEATURE_COMPAT_RESIZE_INODE) &&
+ (old_fs->desc_blocks != fs->desc_blocks)) {
+ int new;
+
+ new = ((int) fs->super->s_reserved_gdt_blocks) +
+ (old_fs->desc_blocks - fs->desc_blocks);
+ if (new < 0)
+ new = 0;
+ if (new > (int) fs->blocksize/4)
+ new = fs->blocksize/4;
+ fs->super->s_reserved_gdt_blocks = new;
+ }
+}
+
+/* Toggle 64bit mode */
+static errcode_t resize_group_descriptors(ext2_resize_t rfs, blk64_t new_size)
+{
+ void *o, *n, *new_group_desc;
+ dgrp_t i;
+ int copy_size;
+ errcode_t retval;
+
+ if (!(rfs->flags & (RESIZE_DISABLE_64BIT | RESIZE_ENABLE_64BIT)))
+ return 0;
+
+ if (new_size != ext2fs_blocks_count(rfs->new_fs->super) ||
+ ext2fs_blocks_count(rfs->new_fs->super) >= (1ULL << 32) ||
+ (rfs->flags & RESIZE_DISABLE_64BIT &&
+ rfs->flags & RESIZE_ENABLE_64BIT))
+ return EXT2_ET_INVALID_ARGUMENT;
+
+ if (rfs->flags & RESIZE_DISABLE_64BIT) {
+ rfs->new_fs->super->s_feature_incompat &=
+ ~EXT4_FEATURE_INCOMPAT_64BIT;
+ rfs->new_fs->super->s_desc_size = EXT2_MIN_DESC_SIZE;
+ } else if (rfs->flags & RESIZE_ENABLE_64BIT) {
+ rfs->new_fs->super->s_feature_incompat |=
+ EXT4_FEATURE_INCOMPAT_64BIT;
+ rfs->new_fs->super->s_desc_size = EXT2_MIN_DESC_SIZE_64BIT;
+ }
+
+ if (EXT2_DESC_SIZE(rfs->old_fs->super) ==
+ EXT2_DESC_SIZE(rfs->new_fs->super))
+ return 0;
+
+ o = rfs->new_fs->group_desc;
+ rfs->new_fs->desc_blocks = ext2fs_div_ceil(
+ rfs->old_fs->group_desc_count,
+ EXT2_DESC_PER_BLOCK(rfs->new_fs->super));
+ retval = ext2fs_get_arrayzero(rfs->new_fs->desc_blocks,
+ rfs->old_fs->blocksize, &new_group_desc);
+ if (retval)
+ return retval;
+
+ n = new_group_desc;
+
+ if (EXT2_DESC_SIZE(rfs->old_fs->super) <=
+ EXT2_DESC_SIZE(rfs->new_fs->super))
+ copy_size = EXT2_DESC_SIZE(rfs->old_fs->super);
+ else
+ copy_size = EXT2_DESC_SIZE(rfs->new_fs->super);
+ for (i = 0; i < rfs->old_fs->group_desc_count; i++) {
+ memcpy(n, o, copy_size);
+ n += EXT2_DESC_SIZE(rfs->new_fs->super);
+ o += EXT2_DESC_SIZE(rfs->old_fs->super);
+ }
+
+ ext2fs_free_mem(&rfs->new_fs->group_desc);
+ rfs->new_fs->group_desc = new_group_desc;
+
+ for (i = 0; i < rfs->old_fs->group_desc_count; i++)
+ ext2fs_group_desc_csum_set(rfs->new_fs, i);
+
+ adjust_reserved_gdt_blocks(rfs->old_fs, rfs->new_fs);
+
+ return 0;
+}
+
+/* Move bitmaps/inode tables out of the way. */
+static errcode_t move_bg_metadata(ext2_resize_t rfs)
+{
+ dgrp_t i;
+ blk64_t b, c, d, old_desc_blocks, new_desc_blocks, j;
+ ext2fs_block_bitmap old_map, new_map;
+ int old, new;
+ errcode_t retval;
+ int cluster_ratio;
+
+ if (!(rfs->flags & (RESIZE_DISABLE_64BIT | RESIZE_ENABLE_64BIT)))
+ return 0;
+
+ retval = ext2fs_allocate_block_bitmap(rfs->old_fs, "oldfs", &old_map);
+ if (retval)
+ return retval;
+
+ retval = ext2fs_allocate_block_bitmap(rfs->new_fs, "newfs", &new_map);
+ if (retval)
+ goto out;
+
+ if (EXT2_HAS_INCOMPAT_FEATURE(rfs->old_fs->super,
+ EXT2_FEATURE_INCOMPAT_META_BG)) {
+ old_desc_blocks = rfs->old_fs->super->s_first_meta_bg;
+ new_desc_blocks = rfs->new_fs->super->s_first_meta_bg;
+ } else {
+ old_desc_blocks = rfs->old_fs->desc_blocks +
+ rfs->old_fs->super->s_reserved_gdt_blocks;
+ new_desc_blocks = rfs->new_fs->desc_blocks +
+ rfs->new_fs->super->s_reserved_gdt_blocks;
+ }
+
+ /* Construct bitmaps of super/descriptor blocks in old and new fs */
+ for (i = 0; i < rfs->old_fs->group_desc_count; i++) {
+ retval = ext2fs_super_and_bgd_loc2(rfs->old_fs, i, &b, &c, &d,
+ NULL);
+ if (retval)
+ goto out;
+ if (b)
+ ext2fs_mark_block_bitmap2(old_map, b);
+ for (j = 0; c != 0 && j < old_desc_blocks; j++)
+ ext2fs_mark_block_bitmap2(old_map, c + j);
+ if (d)
+ ext2fs_mark_block_bitmap2(old_map, d);
+
+ retval = ext2fs_super_and_bgd_loc2(rfs->new_fs, i, &b, &c, &d,
+ NULL);
+ if (retval)
+ goto out;
+ if (b)
+ ext2fs_mark_block_bitmap2(new_map, b);
+ for (j = 0; c != 0 && j < new_desc_blocks; j++)
+ ext2fs_mark_block_bitmap2(new_map, c + j);
+ if (d)
+ ext2fs_mark_block_bitmap2(new_map, d);
+ }
+
+ cluster_ratio = EXT2FS_CLUSTER_RATIO(rfs->new_fs);
+
+ /* Find changes in block allocations for bg metadata */
+ for (b = EXT2FS_B2C(rfs->old_fs,
+ rfs->old_fs->super->s_first_data_block);
+ b < ext2fs_blocks_count(rfs->new_fs->super);
+ b += cluster_ratio) {
+ old = ext2fs_test_block_bitmap2(old_map, b);
+ new = ext2fs_test_block_bitmap2(new_map, b);
+
+ if (old && !new) {
+ /* mark old_map, unmark new_map */
+ if (cluster_ratio == 1)
+ ext2fs_unmark_block_bitmap2(
+ rfs->new_fs->block_map, b);
+ } else if (!old && new)
+ ; /* unmark old_map, mark new_map */
+ else {
+ ext2fs_unmark_block_bitmap2(old_map, b);
+ ext2fs_unmark_block_bitmap2(new_map, b);
+ }
+ }
+
+ /*
+ * new_map now shows blocks that have been newly allocated.
+ * old_map now shows blocks that have been newly freed.
+ */
+
+ /*
+ * Move any conflicting bitmaps and inode tables. Ensure that we
+ * don't try to free clusters associated with bitmaps or tables.
+ */
+ for (i = 0; i < rfs->old_fs->group_desc_count; i++) {
+ b = ext2fs_block_bitmap_loc(rfs->new_fs, i);
+ if (ext2fs_test_block_bitmap2(new_map, b))
+ ext2fs_block_bitmap_loc_set(rfs->new_fs, i, 0);
+ else if (ext2fs_test_block_bitmap2(old_map, b))
+ ext2fs_unmark_block_bitmap2(old_map, b);
+
+ b = ext2fs_inode_bitmap_loc(rfs->new_fs, i);
+ if (ext2fs_test_block_bitmap2(new_map, b))
+ ext2fs_inode_bitmap_loc_set(rfs->new_fs, i, 0);
+ else if (ext2fs_test_block_bitmap2(old_map, b))
+ ext2fs_unmark_block_bitmap2(old_map, b);
+
+ c = ext2fs_inode_table_loc(rfs->new_fs, i);
+ for (b = 0;
+ b < rfs->new_fs->inode_blocks_per_group;
+ b++) {
+ if (ext2fs_test_block_bitmap2(new_map, b + c))
+ ext2fs_inode_table_loc_set(rfs->new_fs, i, 0);
+ else if (ext2fs_test_block_bitmap2(old_map, b + c))
+ ext2fs_unmark_block_bitmap2(old_map, b + c);
+ }
+ }
+
+ /* Free unused clusters */
+ for (b = 0;
+ cluster_ratio > 1 && b < ext2fs_blocks_count(rfs->new_fs->super);
+ b += cluster_ratio)
+ if (ext2fs_test_block_bitmap2(old_map, b))
+ ext2fs_unmark_block_bitmap2(rfs->new_fs->block_map, b);
+out:
+ if (old_map)
+ ext2fs_free_block_bitmap(old_map);
+ if (new_map)
+ ext2fs_free_block_bitmap(new_map);
+ return retval;
+}
+
+/* Zero out the high bits of extent fields */
+static errcode_t zero_high_bits_in_extents(ext2_filsys fs, ext2_ino_t ino,
+ struct ext2_inode *inode)
+{
+ ext2_extent_handle_t handle;
+ struct ext2fs_extent extent;
+ int op = EXT2_EXTENT_ROOT;
+ errcode_t errcode;
+
+ if (!(inode->i_flags & EXT4_EXTENTS_FL))
+ return 0;
+
+ errcode = ext2fs_extent_open(fs, ino, &handle);
+ if (errcode)
+ return errcode;
+
+ while (1) {
+ errcode = ext2fs_extent_get(handle, op, &extent);
+ if (errcode)
+ break;
+
+ op = EXT2_EXTENT_NEXT_SIB;
+
+ if (extent.e_pblk > (1ULL << 32)) {
+ extent.e_pblk &= (1ULL << 32) - 1;
+ errcode = ext2fs_extent_replace(handle, 0, &extent);
+ if (errcode)
+ break;
+ }
+ }
+
+ /* Ok if we run off the end */
+ if (errcode == EXT2_ET_EXTENT_NO_NEXT)
+ errcode = 0;
+ ext2fs_extent_free(handle);
+ return errcode;
+}
+
+/* Zero out the high bits of inodes. */
+static errcode_t zero_high_bits_in_inodes(ext2_resize_t rfs)
+{
+ ext2_filsys fs = rfs->old_fs;
+ int length = EXT2_INODE_SIZE(fs->super);
+ struct ext2_inode *inode = NULL;
+ ext2_inode_scan scan = NULL;
+ errcode_t retval;
+ ext2_ino_t ino;
+
+ if (!(rfs->flags & (RESIZE_DISABLE_64BIT | RESIZE_ENABLE_64BIT)))
+ return 0;
+
+ if (fs->super->s_creator_os != EXT2_OS_LINUX)
+ return 0;
+
+ retval = ext2fs_open_inode_scan(fs, 0, &scan);
+ if (retval)
+ return retval;
+
+ retval = ext2fs_get_mem(length, &inode);
+ if (retval)
+ goto out;
+
+ do {
+ retval = ext2fs_get_next_inode_full(scan, &ino, inode, length);
+ if (retval)
+ goto out;
+ if (!ino)
+ break;
+ if (!ext2fs_test_inode_bitmap2(fs->inode_map, ino))
+ continue;
+
+ /*
+ * Here's how we deal with high block number fields:
+ *
+ * - i_size_high has been been written out with i_size_lo
+ * since the ext2 days, so no conversion is needed.
+ *
+ * - i_blocks_hi is guarded by both the huge_file feature and
+ * inode flags and has always been written out with
+ * i_blocks_lo if the feature is set. The field is only
+ * ever read if both feature and inode flag are set, so
+ * we don't need to zero it now.
+ *
+ * - i_file_acl_high can be uninitialized, so zero it if
+ * it isn't already.
+ */
+ if (inode->osd2.linux2.l_i_file_acl_high) {
+ inode->osd2.linux2.l_i_file_acl_high = 0;
+ retval = ext2fs_write_inode_full(fs, ino, inode,
+ length);
+ if (retval)
+ goto out;
+ }
+
+ retval = zero_high_bits_in_extents(fs, ino, inode);
+ if (retval)
+ goto out;
+ } while (ino);
+
+out:
+ if (inode)
+ ext2fs_free_mem(&inode);
+ if (scan)
+ ext2fs_close_inode_scan(scan);
+ return retval;
+}
+
/*
* Clean up the bitmaps for unitialized bitmaps
*/
@@ -459,7 +795,8 @@ retry:
/*
* Reallocate the group descriptors as necessary.
*/
- if (old_fs->desc_blocks != fs->desc_blocks) {
+ if (EXT2_DESC_SIZE(old_fs->super) == EXT2_DESC_SIZE(fs->super) &&
+ old_fs->desc_blocks != fs->desc_blocks) {
retval = ext2fs_resize_mem(old_fs->desc_blocks *
fs->blocksize,
fs->desc_blocks * fs->blocksize,
@@ -478,20 +815,11 @@ retry:
* number of descriptor blocks, then adjust
* s_reserved_gdt_blocks if possible to avoid needing to move
* the inode table either now or in the future.
+ *
+ * Note: If we're converting to 64bit mode, we did this earlier.
*/
- if ((fs->super->s_feature_compat &
- EXT2_FEATURE_COMPAT_RESIZE_INODE) &&
- (old_fs->desc_blocks != fs->desc_blocks)) {
- int new;
-
- new = ((int) fs->super->s_reserved_gdt_blocks) +
- (old_fs->desc_blocks - fs->desc_blocks);
- if (new < 0)
- new = 0;
- if (new > (int) fs->blocksize/4)
- new = fs->blocksize/4;
- fs->super->s_reserved_gdt_blocks = new;
- }
+ if (EXT2_DESC_SIZE(old_fs->super) == EXT2_DESC_SIZE(fs->super))
+ adjust_reserved_gdt_blocks(old_fs, fs);
if ((fs->super->s_feature_incompat & EXT2_FEATURE_INCOMPAT_META_BG) &&
(fs->super->s_first_meta_bg > fs->desc_blocks)) {
@@ -1019,7 +1347,9 @@ static errcode_t blocks_to_move(ext2_resize_t rfs)
if (retval)
goto errout;
- if (old_blocks == new_blocks) {
+ if (EXT2_DESC_SIZE(rfs->old_fs->super) ==
+ EXT2_DESC_SIZE(rfs->new_fs->super) &&
+ old_blocks == new_blocks) {
retval = 0;
goto errout;
}
diff --git a/resize/resize2fs.h b/resize/resize2fs.h
index 7aeab91..829fcd8 100644
--- a/resize/resize2fs.h
+++ b/resize/resize2fs.h
@@ -82,6 +82,9 @@ typedef struct ext2_sim_progress *ext2_sim_progmeter;
#define RESIZE_PERCENT_COMPLETE 0x0100
#define RESIZE_VERBOSE 0x0200
+#define RESIZE_ENABLE_64BIT 0x0400
+#define RESIZE_DISABLE_64BIT 0x0800
+
/*
* This structure is used for keeping track of how much resources have
* been used for a particular resize2fs pass.
diff --git a/tests/t_enable_mcsum_ext3/expect b/tests/t_enable_mcsum_ext3/expect
index 1e005bd..5460482 100644
--- a/tests/t_enable_mcsum_ext3/expect
+++ b/tests/t_enable_mcsum_ext3/expect
@@ -18,6 +18,7 @@ Pass 5: Checking group summary information
Exit status is 0
tune2fs -O metadata_csum test.img
Extents are not enabled. The file extent tree can be checksummed, whereas block maps cannot. Not enabling extents reduces the coverage of metadata checksumming. Re-run with -O extent to rectify.
+64-bit filesystem support is not enabled. The larger fields afforded by this feature enable full-strength checksumming. Run resize2fs -b to rectify.
Exit status is 0
Change in FS metadata:
^ permalink raw reply related [flat|nested] 104+ messages in thread* Re: [PATCH 35/47] resize2fs: convert fs to and from 64bit mode
2014-11-07 21:54 ` [PATCH 35/47] resize2fs: convert fs to and from 64bit mode Darrick J. Wong
@ 2014-12-15 3:32 ` Theodore Ts'o
2014-12-15 6:41 ` Darrick J. Wong
2014-12-15 17:46 ` Theodore Ts'o
1 sibling, 1 reply; 104+ messages in thread
From: Theodore Ts'o @ 2014-12-15 3:32 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4, TR Reardon
On Fri, Nov 07, 2014 at 01:54:33PM -0800, Darrick J. Wong wrote:
> + case 'b':
> + flags |= RESIZE_ENABLE_64BIT;
> + break;
> + case 's':
> + flags |= RESIZE_DISABLE_64BIT;
> + break;
Out of curiosity, how did you come up with the options -b and -s?
"big" and "small"?
It's not necessarily the most mnemonic, but I can't come up with
anything better, myself.
- Ted
^ permalink raw reply [flat|nested] 104+ messages in thread
* Re: [PATCH 35/47] resize2fs: convert fs to and from 64bit mode
2014-12-15 3:32 ` Theodore Ts'o
@ 2014-12-15 6:41 ` Darrick J. Wong
0 siblings, 0 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-12-15 6:41 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: linux-ext4, TR Reardon
On Sun, Dec 14, 2014 at 10:32:12PM -0500, Theodore Ts'o wrote:
> On Fri, Nov 07, 2014 at 01:54:33PM -0800, Darrick J. Wong wrote:
> > + case 'b':
> > + flags |= RESIZE_ENABLE_64BIT;
> > + break;
> > + case 's':
> > + flags |= RESIZE_DISABLE_64BIT;
> > + break;
>
> Out of curiosity, how did you come up with the options -b and -s?
> "big" and "small"?
>
> It's not necessarily the most mnemonic, but I can't come up with
> anything better, myself.
Yep. It's also tempting to fix up tune2fs to detect "-O 64bit" and
call out to resize2fs, but that's another discussion.
--D
>
> - Ted
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 104+ messages in thread
* Re: [PATCH 35/47] resize2fs: convert fs to and from 64bit mode
2014-11-07 21:54 ` [PATCH 35/47] resize2fs: convert fs to and from 64bit mode Darrick J. Wong
2014-12-15 3:32 ` Theodore Ts'o
@ 2014-12-15 17:46 ` Theodore Ts'o
1 sibling, 0 replies; 104+ messages in thread
From: Theodore Ts'o @ 2014-12-15 17:46 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-ext4, TR Reardon
On Fri, Nov 07, 2014 at 01:54:33PM -0800, Darrick J. Wong wrote:
> resize2fs does its magic by loading a filesystem, duplicating the
> in-memory image of that fs, moving relevant blocks out of the way of
> whatever new metadata get created, and finally writing everything back
> out to disk. Enabling 64bit mode enlarges the group descriptors,
> which makes resize2fs a reasonable vehicle for taking care of the rest
> of the bookkeeping requirements, so add to resize2fs the ability to
> convert a filesystem to 64bit mode and back.
>
> v2: Fix a couple of minor resource leaks; make it more explicit to the
> user what we're doing to the fs.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> Cc: TR Reardon <thomas_reardon@hotmail.com>
Thanks, applied.
- Ted
^ permalink raw reply [flat|nested] 104+ messages in thread
* [PATCH 36/47] tests: test resize2fs 32->64 and 64->32bit conversion code
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (34 preceding siblings ...)
2014-11-07 21:54 ` [PATCH 35/47] resize2fs: convert fs to and from 64bit mode Darrick J. Wong
@ 2014-11-07 21:54 ` Darrick J. Wong
2014-12-15 17:46 ` Theodore Ts'o
2014-11-07 21:54 ` [PATCH 37/47] libext2fs: support allocating uninit blocks in bmap2() Darrick J. Wong
` (12 subsequent siblings)
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:54 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Add some simple tests to check that flex_bg and meta_bg filesystems
can be converted between 32 and 64bit layouts.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
tests/r_32to64bit/expect | 94 +++++++++++++++++++++++
tests/r_32to64bit/name | 1
tests/r_32to64bit/script | 74 ++++++++++++++++++
tests/r_32to64bit_expand_full/expect | 139 ++++++++++++++++++++++++++++++++++
tests/r_32to64bit_expand_full/name | 1
tests/r_32to64bit_expand_full/script | 83 ++++++++++++++++++++
tests/r_32to64bit_meta/expect | 80 ++++++++++++++++++++
tests/r_32to64bit_meta/name | 1
tests/r_32to64bit_meta/script | 74 ++++++++++++++++++
tests/r_32to64bit_move_itable/expect | 107 ++++++++++++++++++++++++++
tests/r_32to64bit_move_itable/name | 1
tests/r_32to64bit_move_itable/script | 74 ++++++++++++++++++
tests/r_64to32bit/expect | 98 ++++++++++++++++++++++++
tests/r_64to32bit/name | 1
tests/r_64to32bit/script | 76 +++++++++++++++++++
tests/r_64to32bit_meta/expect | 80 ++++++++++++++++++++
tests/r_64to32bit_meta/name | 1
tests/r_64to32bit_meta/script | 76 +++++++++++++++++++
18 files changed, 1061 insertions(+)
create mode 100644 tests/r_32to64bit/expect
create mode 100644 tests/r_32to64bit/name
create mode 100644 tests/r_32to64bit/script
create mode 100644 tests/r_32to64bit_expand_full/expect
create mode 100644 tests/r_32to64bit_expand_full/name
create mode 100644 tests/r_32to64bit_expand_full/script
create mode 100644 tests/r_32to64bit_meta/expect
create mode 100644 tests/r_32to64bit_meta/name
create mode 100644 tests/r_32to64bit_meta/script
create mode 100644 tests/r_32to64bit_move_itable/expect
create mode 100644 tests/r_32to64bit_move_itable/name
create mode 100644 tests/r_32to64bit_move_itable/script
create mode 100644 tests/r_64to32bit/expect
create mode 100644 tests/r_64to32bit/name
create mode 100644 tests/r_64to32bit/script
create mode 100644 tests/r_64to32bit_meta/expect
create mode 100644 tests/r_64to32bit_meta/name
create mode 100644 tests/r_64to32bit_meta/script
diff --git a/tests/r_32to64bit/expect b/tests/r_32to64bit/expect
new file mode 100644
index 0000000..f5fa56b
--- /dev/null
+++ b/tests/r_32to64bit/expect
@@ -0,0 +1,94 @@
+resize2fs test
+Creating filesystem with 524288 1k blocks and 65536 inodes
+Superblock backups stored on blocks:
+ 8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409
+
+Allocating group tables: \b\b\b\b\bdone
+Writing inode tables: \b\b\b\b\bdone
+Creating journal (16384 blocks): done
+Creating 477 huge file(s) with 1024 blocks each: done
+Writing superblocks and filesystem accounting information: \b\b\b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
+resize2fs test.img -b
+Converting the filesystem to 64-bit.
+The filesystem on test.img is now 524288 (1k) blocks long.
+
+Exit status is 0
+Change in FS metadata:
+@@ -2,7 +2,7 @@
+ Last mounted on: <not available>
+ Filesystem magic number: 0xEF53
+ Filesystem revision #: 1 (dynamic)
+-Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
++Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
+ Default mount options: user_xattr acl
+ Filesystem state: clean
+ Errors behavior: Continue
+@@ -15,7 +15,8 @@
+ First block: 1
+ Block size: 1024
+ Fragment size: 1024
+-Reserved GDT blocks: 256
++Group descriptor size: 64
++Reserved GDT blocks: 254
+ Blocks per group: 8192
+ Fragments per group: 8192
+ Inodes per group: 1024
+@@ -40,16 +41,16 @@
+
+
+ group:block:super:gdt:bbitmap:ibitmap:itable
+-0:1:1:2-3:260:276:292
+-1:8193:8193:8194-8195:261:277:548
++0:1:1:2-5:260:276:292
++1:8193:8193:8194-8197:261:277:548
+ 2:16385:-1:-1:262:278:804
+-3:24577:24577:24578-24579:263:279:1060
++3:24577:24577:24578-24581:263:279:1060
+ 4:32769:-1:-1:264:280:1316
+-5:40961:40961:40962-40963:265:281:1572
++5:40961:40961:40962-40965:265:281:1572
+ 6:49153:-1:-1:266:282:1828
+-7:57345:57345:57346-57347:267:283:2084
++7:57345:57345:57346-57349:267:283:2084
+ 8:65537:-1:-1:268:284:2340
+-9:73729:73729:73730-73731:269:285:2596
++9:73729:73729:73730-73733:269:285:2596
+ 10:81921:-1:-1:270:286:2852
+ 11:90113:-1:-1:271:287:3108
+ 12:98305:-1:-1:272:288:3364
+@@ -65,9 +66,9 @@
+ 22:180225:-1:-1:131079:131095:132641
+ 23:188417:-1:-1:131080:131096:132897
+ 24:196609:-1:-1:131081:131097:133153
+-25:204801:204801:204802-204803:131082:131098:133409
++25:204801:204801:204802-204805:131082:131098:133409
+ 26:212993:-1:-1:131083:131099:133665
+-27:221185:221185:221186-221187:131084:131100:133921
++27:221185:221185:221186-221189:131084:131100:133921
+ 28:229377:-1:-1:131085:131101:134177
+ 29:237569:-1:-1:131086:131102:134433
+ 30:245761:-1:-1:131087:131103:134689
+@@ -89,7 +90,7 @@
+ 46:376833:-1:-1:262159:262175:265761
+ 47:385025:-1:-1:262160:262176:266017
+ 48:393217:-1:-1:393217:393233:393249
+-49:401409:401409:401410-401411:393218:393234:393505
++49:401409:401409:401410-401413:393218:393234:393505
+ 50:409601:-1:-1:393219:393235:393761
+ 51:417793:-1:-1:393220:393236:394017
+ 52:425985:-1:-1:393221:393237:394273
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
diff --git a/tests/r_32to64bit/name b/tests/r_32to64bit/name
new file mode 100644
index 0000000..fb45fab
--- /dev/null
+++ b/tests/r_32to64bit/name
@@ -0,0 +1 @@
+convert flex_bg 32bit fs to 64bit fs
diff --git a/tests/r_32to64bit/script b/tests/r_32to64bit/script
new file mode 100644
index 0000000..de08bfb
--- /dev/null
+++ b/tests/r_32to64bit/script
@@ -0,0 +1,74 @@
+if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+
+FSCK_OPT=-fn
+OUT=$test_name.log
+EXP=$test_dir/expect
+CONF=$TMPFILE.conf
+
+cat > $CONF << ENDL
+[fs_types]
+ ext4h = {
+ features = has_journal,extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,sparse_super,filetype,dir_index,ext_attr,resize_inode
+ blocksize = 1024
+ inode_size = 256
+ make_hugefiles = true
+ hugefiles_dir = /
+ hugefiles_slack = 0
+ hugefiles_name = aaaaa
+ hugefiles_digits = 4
+ hugefiles_size = 1M
+ zero_hugefiles = false
+ }
+ENDL
+
+echo "resize2fs test" > $OUT
+
+MKE2FS_CONFIG=$CONF $MKE2FS -F -T ext4h $TMPFILE 524288 >> $OUT 2>&1
+rm -rf $CONF
+
+# dump and check
+($DUMPE2FS -h $TMPFILE; $DUMPE2FS -g $TMPFILE) 2> /dev/null | sed -f $cmd_dir/filter.sed > $OUT.before
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# resize it
+echo "resize2fs test.img -b" >> $OUT
+$RESIZE2FS -b -f $TMPFILE 2>&1 >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# dump and check
+($DUMPE2FS -h $TMPFILE; $DUMPE2FS -g $TMPFILE) 2> /dev/null | sed -f $cmd_dir/filter.sed > $OUT.after
+echo "Change in FS metadata:" >> $OUT
+diff -u $OUT.before $OUT.after | tail -n +3 >> $OUT
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+rm $TMPFILE $OUT.before $OUT.after
+
+#
+# Do the verification
+#
+
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" -e 's/test_filesys:.*//g' < $OUT > $OUT.new
+mv $OUT.new $OUT
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+fi
+
+unset IMAGE FSCK_OPT OUT EXP CONF
+
+else #if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
+
diff --git a/tests/r_32to64bit_expand_full/expect b/tests/r_32to64bit_expand_full/expect
new file mode 100644
index 0000000..f9e30f3
--- /dev/null
+++ b/tests/r_32to64bit_expand_full/expect
@@ -0,0 +1,139 @@
+resize2fs test
+Creating filesystem with 786432 1k blocks and 98304 inodes
+Superblock backups stored on blocks:
+ 8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409, 663553
+
+Allocating group tables: \b\b\b\b\bdone
+Writing inode tables: \b\b\b\b\bdone
+Creating journal (16384 blocks): done
+Creating 727 huge file(s) with 1024 blocks each: done
+Writing superblocks and filesystem accounting information: \b\b\b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
+resize2fs -b test.img
+Converting the filesystem to 64-bit.
+The filesystem on test.img is now 786432 (1k) blocks long.
+
+resize2fs test.img
+Resizing the filesystem on test.img to 1179648 (1k) blocks.
+The filesystem on test.img is now 1179648 (1k) blocks long.
+
+Exit status is 0
+Change in FS metadata:
+@@ -1,15 +1,15 @@
+
+ group:block:super:gdt:bbitmap:ibitmap:itable
+-0:1:1:2-7:8:9:10
+-1:8193:8193:8194-8199:8200:8201:8202
++0:1:1:2-10:266:267:268
++1:8193:8193:8194-8202:8458:8459:8460
+ 2:16385:-1:-1:16385:16386:16387
+-3:24577:24577:24578-24583:24584:24585:24586
++3:24577:24577:24578-24586:24842:24843:24844
+ 4:32769:-1:-1:32769:32770:32771
+-5:40961:40961:40962-40967:40968:40969:40970
++5:40961:40961:40962-40970:41226:41227:41228
+ 6:49153:-1:-1:49153:49154:49155
+-7:57345:57345:57346-57351:57352:57353:57354
++7:57345:57345:57346-57354:57610:57611:57612
+ 8:65537:-1:-1:65537:65538:65539
+-9:73729:73729:73730-73735:73736:73737:73738
++9:73729:73729:73730-73738:73994:73995:73996
+ 10:81921:-1:-1:81921:81922:81923
+ 11:90113:-1:-1:90113:90114:90115
+ 12:98305:-1:-1:98305:98306:98307
+@@ -25,9 +25,9 @@
+ 22:180225:-1:-1:180225:180226:180227
+ 23:188417:-1:-1:188417:188418:188419
+ 24:196609:-1:-1:196609:196610:196611
+-25:204801:204801:204802-204807:204808:204809:204810
++25:204801:204801:204802-204810:205066:205067:205068
+ 26:212993:-1:-1:212993:212994:212995
+-27:221185:221185:221186-221191:221192:221193:221194
++27:221185:221185:221186-221194:221450:221451:221452
+ 28:229377:-1:-1:229377:229378:229379
+ 29:237569:-1:-1:237569:237570:237571
+ 30:245761:-1:-1:245761:245762:245763
+@@ -49,7 +49,7 @@
+ 46:376833:-1:-1:376833:376834:376835
+ 47:385025:-1:-1:385025:385026:385027
+ 48:393217:-1:-1:393217:393218:393219
+-49:401409:401409:401410-401415:401416:401417:401418
++49:401409:401409:401410-401418:401674:401675:401676
+ 50:409601:-1:-1:409601:409602:409603
+ 51:417793:-1:-1:417793:417794:417795
+ 52:425985:-1:-1:425985:425986:425987
+@@ -81,7 +81,7 @@
+ 78:638977:-1:-1:638977:638978:638979
+ 79:647169:-1:-1:647169:647170:647171
+ 80:655361:-1:-1:655361:655362:655363
+-81:663553:663553:663554-663559:663560:663561:663562
++81:663553:663553:663554-663562:663818:663819:663820
+ 82:671745:-1:-1:671745:671746:671747
+ 83:679937:-1:-1:679937:679938:679939
+ 84:688129:-1:-1:688129:688130:688131
+@@ -96,3 +96,51 @@
+ 93:761857:-1:-1:761857:761858:761859
+ 94:770049:-1:-1:770049:770050:770051
+ 95:778241:-1:-1:778241:778242:778243
++96:786433:-1:-1:786433:786434:786435
++97:794625:-1:-1:794625:794626:794627
++98:802817:-1:-1:802817:802818:802819
++99:811009:-1:-1:811009:811010:811011
++100:819201:-1:-1:819201:819202:819203
++101:827393:-1:-1:827393:827394:827395
++102:835585:-1:-1:835585:835586:835587
++103:843777:-1:-1:843777:843778:843779
++104:851969:-1:-1:851969:851970:851971
++105:860161:-1:-1:860161:860162:860163
++106:868353:-1:-1:868353:868354:868355
++107:876545:-1:-1:876545:876546:876547
++108:884737:-1:-1:884737:884738:884739
++109:892929:-1:-1:892929:892930:892931
++110:901121:-1:-1:901121:901122:901123
++111:909313:-1:-1:909313:909314:909315
++112:917505:-1:-1:917505:917506:917507
++113:925697:-1:-1:925697:925698:925699
++114:933889:-1:-1:933889:933890:933891
++115:942081:-1:-1:942081:942082:942083
++116:950273:-1:-1:950273:950274:950275
++117:958465:-1:-1:958465:958466:958467
++118:966657:-1:-1:966657:966658:966659
++119:974849:-1:-1:974849:974850:974851
++120:983041:-1:-1:983041:983042:983043
++121:991233:-1:-1:991233:991234:991235
++122:999425:-1:-1:999425:999426:999427
++123:1007617:-1:-1:1007617:1007618:1007619
++124:1015809:-1:-1:1015809:1015810:1015811
++125:1024001:1024001:1024002-1024010:1024011:1024012:1024013
++126:1032193:-1:-1:1032193:1032194:1032195
++127:1040385:-1:-1:1040385:1040386:1040387
++128:1048577:-1:-1:1048577:1048578:1048579
++129:1056769:-1:-1:1056769:1056770:1056771
++130:1064961:-1:-1:1064961:1064962:1064963
++131:1073153:-1:-1:1073153:1073154:1073155
++132:1081345:-1:-1:1081345:1081346:1081347
++133:1089537:-1:-1:1089537:1089538:1089539
++134:1097729:-1:-1:1097729:1097730:1097731
++135:1105921:-1:-1:1105921:1105922:1105923
++136:1114113:-1:-1:1114113:1114114:1114115
++137:1122305:-1:-1:1122305:1122306:1122307
++138:1130497:-1:-1:1130497:1130498:1130499
++139:1138689:-1:-1:1138689:1138690:1138691
++140:1146881:-1:-1:1146881:1146882:1146883
++141:1155073:-1:-1:1155073:1155074:1155075
++142:1163265:-1:-1:1163265:1163266:1163267
++143:1171457:-1:-1:1171457:1171458:1171459
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
diff --git a/tests/r_32to64bit_expand_full/name b/tests/r_32to64bit_expand_full/name
new file mode 100644
index 0000000..2f04601
--- /dev/null
+++ b/tests/r_32to64bit_expand_full/name
@@ -0,0 +1 @@
+convert a totally full filesystem to 64bit, then expand
diff --git a/tests/r_32to64bit_expand_full/script b/tests/r_32to64bit_expand_full/script
new file mode 100644
index 0000000..7a57084
--- /dev/null
+++ b/tests/r_32to64bit_expand_full/script
@@ -0,0 +1,83 @@
+if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+
+FSCK_OPT=-fn
+OUT=$test_name.log
+EXP=$test_dir/expect
+CONF=$TMPFILE.conf
+
+#gzip -d < $EXP.gz > $EXP
+
+cat > $CONF << ENDL
+[fs_types]
+ ext4h = {
+ features = has_journal,extent,huge_file,uninit_bg,dir_nlink,extra_isize,sparse_super,filetype,dir_index,ext_attr,^resize_inode,^meta_bg,^flex_bg
+ blocksize = 1024
+ inode_size = 256
+ make_hugefiles = true
+ hugefiles_dir = /
+ hugefiles_slack = 0
+ hugefiles_name = aaaaa
+ hugefiles_digits = 4
+ hugefiles_size = 1M
+ zero_hugefiles = false
+ }
+ENDL
+
+echo "resize2fs test" > $OUT
+
+MKE2FS_CONFIG=$CONF $MKE2FS -F -T ext4h $TMPFILE 786432 >> $OUT 2>&1
+rm -rf $CONF
+
+# check
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# convert it
+echo "resize2fs -b test.img" >> $OUT
+$RESIZE2FS -b -f $TMPFILE 2>&1 >> $OUT 2>&1
+$DUMPE2FS -g $TMPFILE 2>&1 >> $OUT.before 2> /dev/null
+
+# grow it
+echo "resize2fs test.img" >> $OUT
+dd if=/dev/zero of=$TMPFILE conv=notrunc bs=1 count=1 seek=1207959552 2> /dev/null
+$RESIZE2FS -f $TMPFILE 2>&1 >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# dump and check
+$DUMPE2FS -g $TMPFILE 2>&1 >> $OUT.after 2> /dev/null
+echo "Change in FS metadata:" >> $OUT
+diff -u $OUT.before $OUT.after | tail -n +3 >> $OUT
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+rm $TMPFILE
+
+#
+# Do the verification
+#
+
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" -e 's/test_filesys:.*//g' < $OUT > $OUT.new
+mv $OUT.new $OUT
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+fi
+
+rm $OUT.before $OUT.after
+
+unset IMAGE FSCK_OPT OUT EXP CONF
+
+else #if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
+
diff --git a/tests/r_32to64bit_meta/expect b/tests/r_32to64bit_meta/expect
new file mode 100644
index 0000000..0eacd45
--- /dev/null
+++ b/tests/r_32to64bit_meta/expect
@@ -0,0 +1,80 @@
+resize2fs test
+Creating filesystem with 524288 1k blocks and 65536 inodes
+Superblock backups stored on blocks:
+ 8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409
+
+Allocating group tables: \b\b\b\b\bdone
+Writing inode tables: \b\b\b\b\bdone
+Creating journal (16384 blocks): done
+Creating 479 huge file(s) with 1024 blocks each: done
+Writing superblocks and filesystem accounting information: \b\b\b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
+resize2fs test.img -b
+Converting the filesystem to 64-bit.
+The filesystem on test.img is now 524288 (1k) blocks long.
+
+Exit status is 0
+Change in FS metadata:
+@@ -2,7 +2,7 @@
+ Last mounted on: <not available>
+ Filesystem magic number: 0xEF53
+ Filesystem revision #: 1 (dynamic)
+-Filesystem features: has_journal ext_attr dir_index filetype meta_bg extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
++Filesystem features: has_journal ext_attr dir_index filetype meta_bg extent 64bit flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
+ Default mount options: user_xattr acl
+ Filesystem state: clean
+ Errors behavior: Continue
+@@ -10,11 +10,12 @@
+ Inode count: 65536
+ Block count: 524288
+ Reserved block count: 26214
+-Free blocks: 858
++Free blocks: 852
+ Free inodes: 65046
+ First block: 1
+ Block size: 1024
+ Fragment size: 1024
++Group descriptor size: 64
+ Blocks per group: 8192
+ Fragments per group: 8192
+ Inodes per group: 1024
+@@ -54,9 +55,9 @@
+ 12:98305:-1:-1:15:31:3107
+ 13:106497:-1:-1:16:32:3363
+ 14:114689:-1:-1:17:33:3619
+-15:122881:-1:-1:18:34:3875
+-16:131073:-1:-1:131073:131089:131105
+-17:139265:-1:-1:131074:131090:131361
++15:122881:-1:122881:18:34:3875
++16:131073:-1:131073:135201:131089:131105
++17:139265:-1:139265:131074:131090:131361
+ 18:147457:-1:-1:131075:131091:131617
+ 19:155649:-1:-1:131076:131092:131873
+ 20:163841:-1:-1:131077:131093:132129
+@@ -86,9 +87,9 @@
+ 44:360449:-1:-1:262158:262174:265250
+ 45:368641:-1:-1:262159:262175:265506
+ 46:376833:-1:-1:262160:262176:265762
+-47:385025:-1:-1:262161:262177:266018
+-48:393217:-1:-1:393217:393233:393249
+-49:401409:401409:-1:393218:393234:393505
++47:385025:-1:385025:262161:262177:266018
++48:393217:-1:393217:397345:393233:393249
++49:401409:401409:401410:393218:393234:393505
+ 50:409601:-1:-1:393219:393235:393761
+ 51:417793:-1:-1:393220:393236:394017
+ 52:425985:-1:-1:393221:393237:394273
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
diff --git a/tests/r_32to64bit_meta/name b/tests/r_32to64bit_meta/name
new file mode 100644
index 0000000..d83492e
--- /dev/null
+++ b/tests/r_32to64bit_meta/name
@@ -0,0 +1 @@
+convert meta_bg 32bit fs to 64bit fs
diff --git a/tests/r_32to64bit_meta/script b/tests/r_32to64bit_meta/script
new file mode 100644
index 0000000..25872c4
--- /dev/null
+++ b/tests/r_32to64bit_meta/script
@@ -0,0 +1,74 @@
+if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+
+FSCK_OPT=-fn
+OUT=$test_name.log
+EXP=$test_dir/expect
+CONF=$TMPFILE.conf
+
+cat > $CONF << ENDL
+[fs_types]
+ ext4h = {
+ features = has_journal,extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,sparse_super,filetype,dir_index,ext_attr,meta_bg,^resize_inode
+ blocksize = 1024
+ inode_size = 256
+ make_hugefiles = true
+ hugefiles_dir = /
+ hugefiles_slack = 0
+ hugefiles_name = aaaaa
+ hugefiles_digits = 4
+ hugefiles_size = 1M
+ zero_hugefiles = false
+ }
+ENDL
+
+echo "resize2fs test" > $OUT
+
+MKE2FS_CONFIG=$CONF $MKE2FS -F -T ext4h $TMPFILE 524288 >> $OUT 2>&1
+rm -rf $CONF
+
+# dump and check
+($DUMPE2FS -h $TMPFILE; $DUMPE2FS -g $TMPFILE) 2> /dev/null | sed -f $cmd_dir/filter.sed > $OUT.before
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# resize it
+echo "resize2fs test.img -b" >> $OUT
+$RESIZE2FS -b -f $TMPFILE 2>&1 >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# dump and check
+($DUMPE2FS -h $TMPFILE; $DUMPE2FS -g $TMPFILE) 2> /dev/null | sed -f $cmd_dir/filter.sed > $OUT.after
+echo "Change in FS metadata:" >> $OUT
+diff -u $OUT.before $OUT.after | tail -n +3 >> $OUT
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+rm $TMPFILE $OUT.before $OUT.after
+
+#
+# Do the verification
+#
+
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" -e 's/test_filesys:.*//g' < $OUT > $OUT.new
+mv $OUT.new $OUT
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+fi
+
+unset IMAGE FSCK_OPT OUT EXP CONF
+
+else #if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
+
diff --git a/tests/r_32to64bit_move_itable/expect b/tests/r_32to64bit_move_itable/expect
new file mode 100644
index 0000000..b51663d
--- /dev/null
+++ b/tests/r_32to64bit_move_itable/expect
@@ -0,0 +1,107 @@
+resize2fs test
+Creating filesystem with 786432 1k blocks and 98304 inodes
+Superblock backups stored on blocks:
+ 8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409, 663553
+
+Allocating group tables: \b\b\b\b\bdone
+Writing inode tables: \b\b\b\b\bdone
+Creating journal (16384 blocks): done
+Creating 727 huge file(s) with 1024 blocks each: done
+Writing superblocks and filesystem accounting information: \b\b\b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
+resize2fs test.img -b
+Converting the filesystem to 64-bit.
+The filesystem on test.img is now 786432 (1k) blocks long.
+
+Exit status is 0
+Change in FS metadata:
+@@ -2,7 +2,7 @@
+ Last mounted on: <not available>
+ Filesystem magic number: 0xEF53
+ Filesystem revision #: 1 (dynamic)
+-Filesystem features: has_journal ext_attr dir_index filetype extent sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
++Filesystem features: has_journal ext_attr dir_index filetype extent 64bit sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
+ Default mount options: user_xattr acl
+ Filesystem state: clean
+ Errors behavior: Continue
+@@ -10,11 +10,12 @@
+ Inode count: 98304
+ Block count: 786432
+ Reserved block count: 39321
+-Free blocks: 764
++Free blocks: 734
+ Free inodes: 97566
+ First block: 1
+ Block size: 1024
+ Fragment size: 1024
++Group descriptor size: 64
+ Blocks per group: 8192
+ Fragments per group: 8192
+ Inodes per group: 1024
+@@ -38,16 +39,16 @@
+
+
+ group:block:super:gdt:bbitmap:ibitmap:itable
+-0:1:1:2-4:5:6:7
+-1:8193:8193:8194-8196:8197:8198:8199
++0:1:1:2-7:8:9:10
++1:8193:8193:8194-8199:8200:8201:8202
+ 2:16385:-1:-1:16385:16386:16387
+-3:24577:24577:24578-24580:24581:24582:24583
++3:24577:24577:24578-24583:24584:24585:24586
+ 4:32769:-1:-1:32769:32770:32771
+-5:40961:40961:40962-40964:40965:40966:40967
++5:40961:40961:40962-40967:40968:40969:40970
+ 6:49153:-1:-1:49153:49154:49155
+-7:57345:57345:57346-57348:57349:57350:57351
++7:57345:57345:57346-57351:57352:57353:57354
+ 8:65537:-1:-1:65537:65538:65539
+-9:73729:73729:73730-73732:73733:73734:73735
++9:73729:73729:73730-73735:73736:73737:73738
+ 10:81921:-1:-1:81921:81922:81923
+ 11:90113:-1:-1:90113:90114:90115
+ 12:98305:-1:-1:98305:98306:98307
+@@ -63,9 +64,9 @@
+ 22:180225:-1:-1:180225:180226:180227
+ 23:188417:-1:-1:188417:188418:188419
+ 24:196609:-1:-1:196609:196610:196611
+-25:204801:204801:204802-204804:204805:204806:204807
++25:204801:204801:204802-204807:204808:204809:204810
+ 26:212993:-1:-1:212993:212994:212995
+-27:221185:221185:221186-221188:221189:221190:221191
++27:221185:221185:221186-221191:221192:221193:221194
+ 28:229377:-1:-1:229377:229378:229379
+ 29:237569:-1:-1:237569:237570:237571
+ 30:245761:-1:-1:245761:245762:245763
+@@ -87,7 +88,7 @@
+ 46:376833:-1:-1:376833:376834:376835
+ 47:385025:-1:-1:385025:385026:385027
+ 48:393217:-1:-1:393217:393218:393219
+-49:401409:401409:401410-401412:401413:401414:401415
++49:401409:401409:401410-401415:401416:401417:401418
+ 50:409601:-1:-1:409601:409602:409603
+ 51:417793:-1:-1:417793:417794:417795
+ 52:425985:-1:-1:425985:425986:425987
+@@ -119,7 +120,7 @@
+ 78:638977:-1:-1:638977:638978:638979
+ 79:647169:-1:-1:647169:647170:647171
+ 80:655361:-1:-1:655361:655362:655363
+-81:663553:663553:663554-663556:663557:663558:663559
++81:663553:663553:663554-663559:663560:663561:663562
+ 82:671745:-1:-1:671745:671746:671747
+ 83:679937:-1:-1:679937:679938:679939
+ 84:688129:-1:-1:688129:688130:688131
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
diff --git a/tests/r_32to64bit_move_itable/name b/tests/r_32to64bit_move_itable/name
new file mode 100644
index 0000000..7e6bb25
--- /dev/null
+++ b/tests/r_32to64bit_move_itable/name
@@ -0,0 +1 @@
+convert 32bit fs to 64bit fs, forcing inode table move
diff --git a/tests/r_32to64bit_move_itable/script b/tests/r_32to64bit_move_itable/script
new file mode 100644
index 0000000..c188acd
--- /dev/null
+++ b/tests/r_32to64bit_move_itable/script
@@ -0,0 +1,74 @@
+if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+
+FSCK_OPT=-fn
+OUT=$test_name.log
+EXP=$test_dir/expect
+CONF=$TMPFILE.conf
+
+cat > $CONF << ENDL
+[fs_types]
+ ext4h = {
+ features = has_journal,extent,huge_file,uninit_bg,dir_nlink,extra_isize,sparse_super,filetype,dir_index,ext_attr,^resize_inode,^meta_bg,^flex_bg
+ blocksize = 1024
+ inode_size = 256
+ make_hugefiles = true
+ hugefiles_dir = /
+ hugefiles_slack = 0
+ hugefiles_name = aaaaa
+ hugefiles_digits = 4
+ hugefiles_size = 1M
+ zero_hugefiles = false
+ }
+ENDL
+
+echo "resize2fs test" > $OUT
+
+MKE2FS_CONFIG=$CONF $MKE2FS -F -T ext4h $TMPFILE 786432 >> $OUT 2>&1
+rm -rf $CONF
+
+# dump and check
+($DUMPE2FS -h $TMPFILE; $DUMPE2FS -g $TMPFILE) 2> /dev/null | sed -f $cmd_dir/filter.sed > $OUT.before
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# convert it
+echo "resize2fs test.img -b" >> $OUT
+$RESIZE2FS -b -f $TMPFILE 2>&1 >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# dump and check
+($DUMPE2FS -h $TMPFILE; $DUMPE2FS -g $TMPFILE) 2> /dev/null | sed -f $cmd_dir/filter.sed > $OUT.after
+echo "Change in FS metadata:" >> $OUT
+diff -u $OUT.before $OUT.after | tail -n +3 >> $OUT
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+rm $TMPFILE $OUT.before $OUT.after
+
+#
+# Do the verification
+#
+
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" -e 's/test_filesys:.*//g' < $OUT > $OUT.new
+mv $OUT.new $OUT
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+fi
+
+unset IMAGE FSCK_OPT OUT EXP CONF
+
+else #if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
+
diff --git a/tests/r_64to32bit/expect b/tests/r_64to32bit/expect
new file mode 100644
index 0000000..13e94a2
--- /dev/null
+++ b/tests/r_64to32bit/expect
@@ -0,0 +1,98 @@
+resize2fs test
+Creating filesystem with 524288 1k blocks and 65536 inodes
+Superblock backups stored on blocks:
+ 8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409
+
+Allocating group tables: \b\b\b\b\bdone
+Writing inode tables: \b\b\b\b\bdone
+Creating journal (16384 blocks): done
+Creating 477 huge file(s) with 1024 blocks each: done
+Writing superblocks and filesystem accounting information: \b\b\b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
+resize2fs test.img -s
+Converting the filesystem to 32-bit.
+The filesystem on test.img is now 524288 (1k) blocks long.
+
+Exit status is 0
+Change in FS metadata:
+@@ -2,7 +2,7 @@
+ Last mounted on: <not available>
+ Filesystem magic number: 0xEF53
+ Filesystem revision #: 1 (dynamic)
+-Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
++Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
+ Default mount options: user_xattr acl
+ Filesystem state: clean
+ Errors behavior: Continue
+@@ -10,12 +10,11 @@
+ Inode count: 65536
+ Block count: 524288
+ Reserved block count: 26214
+-Free blocks: 571
++Free blocks: 589
+ Free inodes: 65048
+ First block: 1
+ Block size: 1024
+ Fragment size: 1024
+-Group descriptor size: 64
+ Reserved GDT blocks: 256
+ Blocks per group: 8192
+ Fragments per group: 8192
+@@ -41,16 +40,16 @@
+
+
+ group:block:super:gdt:bbitmap:ibitmap:itable
+-0:1:1:2-5:262:278:294
+-1:8193:8193:8194-8197:263:279:550
++0:1:1:2-3:262:278:294
++1:8193:8193:8194-8195:263:279:550
+ 2:16385:-1:-1:264:280:806
+-3:24577:24577:24578-24581:265:281:1062
++3:24577:24577:24578-24579:265:281:1062
+ 4:32769:-1:-1:266:282:1318
+-5:40961:40961:40962-40965:267:283:1574
++5:40961:40961:40962-40963:267:283:1574
+ 6:49153:-1:-1:268:284:1830
+-7:57345:57345:57346-57349:269:285:2086
++7:57345:57345:57346-57347:269:285:2086
+ 8:65537:-1:-1:270:286:2342
+-9:73729:73729:73730-73733:271:287:2598
++9:73729:73729:73730-73731:271:287:2598
+ 10:81921:-1:-1:272:288:2854
+ 11:90113:-1:-1:273:289:3110
+ 12:98305:-1:-1:274:290:3366
+@@ -66,9 +65,9 @@
+ 22:180225:-1:-1:131079:131095:132641
+ 23:188417:-1:-1:131080:131096:132897
+ 24:196609:-1:-1:131081:131097:133153
+-25:204801:204801:204802-204805:131082:131098:133409
++25:204801:204801:204802-204803:131082:131098:133409
+ 26:212993:-1:-1:131083:131099:133665
+-27:221185:221185:221186-221189:131084:131100:133921
++27:221185:221185:221186-221187:131084:131100:133921
+ 28:229377:-1:-1:131085:131101:134177
+ 29:237569:-1:-1:131086:131102:134433
+ 30:245761:-1:-1:131087:131103:134689
+@@ -90,7 +89,7 @@
+ 46:376833:-1:-1:262159:262175:265761
+ 47:385025:-1:-1:262160:262176:266017
+ 48:393217:-1:-1:393217:393233:393249
+-49:401409:401409:401410-401413:393218:393234:393505
++49:401409:401409:401410-401411:393218:393234:393505
+ 50:409601:-1:-1:393219:393235:393761
+ 51:417793:-1:-1:393220:393236:394017
+ 52:425985:-1:-1:393221:393237:394273
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
diff --git a/tests/r_64to32bit/name b/tests/r_64to32bit/name
new file mode 100644
index 0000000..4c82371
--- /dev/null
+++ b/tests/r_64to32bit/name
@@ -0,0 +1 @@
+convert flex_bg 64bit fs to 32bit fs
diff --git a/tests/r_64to32bit/script b/tests/r_64to32bit/script
new file mode 100644
index 0000000..5d959f0
--- /dev/null
+++ b/tests/r_64to32bit/script
@@ -0,0 +1,76 @@
+if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+
+FSCK_OPT=-fn
+OUT=$test_name.log
+EXP=$test_dir/expect
+CONF=$TMPFILE.conf
+
+cat > $CONF << ENDL
+[fs_types]
+ ext4h = {
+ features = has_journal,extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,sparse_super,filetype,dir_index,ext_attr,resize_inode,64bit
+ blocksize = 1024
+ inode_size = 256
+ make_hugefiles = true
+ hugefiles_dir = /
+ hugefiles_slack = 0
+ hugefiles_name = aaaaa
+ hugefiles_digits = 4
+ hugefiles_size = 1M
+ zero_hugefiles = false
+ }
+ENDL
+
+echo "resize2fs test" > $OUT
+
+MKE2FS_CONFIG=$CONF $MKE2FS -F -T ext4h $TMPFILE 524288 >> $OUT 2>&1
+rm -rf $CONF
+
+# dump and check
+($DUMPE2FS -h $TMPFILE; $DUMPE2FS -g $TMPFILE) 2> /dev/null | sed -f $cmd_dir/filter.sed > $OUT.before
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# resize it
+echo "resize2fs test.img -s" >> $OUT
+$RESIZE2FS -s -f $TMPFILE 2>&1 >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# dump and check
+($DUMPE2FS -h $TMPFILE; $DUMPE2FS -g $TMPFILE) 2> /dev/null | sed -f $cmd_dir/filter.sed > $OUT.after
+echo "Change in FS metadata:" >> $OUT
+diff -u $OUT.before $OUT.after | tail -n +3 >> $OUT
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+rm $TMPFILE
+
+#
+# Do the verification
+#
+
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" -e 's/test_filesys:.*//g' < $OUT > $OUT.new
+mv $OUT.new $OUT
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+fi
+
+rm $OUT.before $OUT.after
+
+unset IMAGE FSCK_OPT OUT EXP CONF
+
+else #if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
+
diff --git a/tests/r_64to32bit_meta/expect b/tests/r_64to32bit_meta/expect
new file mode 100644
index 0000000..d6e2dcc
--- /dev/null
+++ b/tests/r_64to32bit_meta/expect
@@ -0,0 +1,80 @@
+resize2fs test
+Creating filesystem with 524288 1k blocks and 65536 inodes
+Superblock backups stored on blocks:
+ 8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409
+
+Allocating group tables: \b\b\b\b\bdone
+Writing inode tables: \b\b\b\b\bdone
+Creating journal (16384 blocks): done
+Creating 479 huge file(s) with 1024 blocks each: done
+Writing superblocks and filesystem accounting information: \b\b\b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
+resize2fs test.img -s
+Converting the filesystem to 32-bit.
+The filesystem on test.img is now 524288 (1k) blocks long.
+
+Exit status is 0
+Change in FS metadata:
+@@ -2,7 +2,7 @@
+ Last mounted on: <not available>
+ Filesystem magic number: 0xEF53
+ Filesystem revision #: 1 (dynamic)
+-Filesystem features: has_journal ext_attr dir_index filetype meta_bg extent 64bit flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
++Filesystem features: has_journal ext_attr dir_index filetype meta_bg extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
+ Default mount options: user_xattr acl
+ Filesystem state: clean
+ Errors behavior: Continue
+@@ -10,12 +10,11 @@
+ Inode count: 65536
+ Block count: 524288
+ Reserved block count: 26214
+-Free blocks: 852
++Free blocks: 858
+ Free inodes: 65046
+ First block: 1
+ Block size: 1024
+ Fragment size: 1024
+-Group descriptor size: 64
+ Blocks per group: 8192
+ Fragments per group: 8192
+ Inodes per group: 1024
+@@ -55,9 +54,9 @@
+ 12:98305:-1:-1:15:31:3107
+ 13:106497:-1:-1:16:32:3363
+ 14:114689:-1:-1:17:33:3619
+-15:122881:-1:122881:18:34:3875
+-16:131073:-1:131073:131074:131090:131106
+-17:139265:-1:139265:131075:131091:131362
++15:122881:-1:-1:18:34:3875
++16:131073:-1:-1:131074:131090:131106
++17:139265:-1:-1:131075:131091:131362
+ 18:147457:-1:-1:131076:131092:131618
+ 19:155649:-1:-1:131077:131093:131874
+ 20:163841:-1:-1:131078:131094:132130
+@@ -87,9 +86,9 @@
+ 44:360449:-1:-1:262158:262174:265250
+ 45:368641:-1:-1:262159:262175:265506
+ 46:376833:-1:-1:262160:262176:265762
+-47:385025:-1:385025:262161:262177:266018
+-48:393217:-1:393217:393218:393234:393250
+-49:401409:401409:401410:393219:393235:393506
++47:385025:-1:-1:262161:262177:266018
++48:393217:-1:-1:393218:393234:393250
++49:401409:401409:-1:393219:393235:393506
+ 50:409601:-1:-1:393220:393236:393762
+ 51:417793:-1:-1:393221:393237:394018
+ 52:425985:-1:-1:393222:393238:394274
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
diff --git a/tests/r_64to32bit_meta/name b/tests/r_64to32bit_meta/name
new file mode 100644
index 0000000..e99ed8b
--- /dev/null
+++ b/tests/r_64to32bit_meta/name
@@ -0,0 +1 @@
+convert meta_bg 64bit fs to 32bit fs
diff --git a/tests/r_64to32bit_meta/script b/tests/r_64to32bit_meta/script
new file mode 100644
index 0000000..5a02e26
--- /dev/null
+++ b/tests/r_64to32bit_meta/script
@@ -0,0 +1,76 @@
+if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+
+FSCK_OPT=-fn
+OUT=$test_name.log
+EXP=$test_dir/expect
+CONF=$TMPFILE.conf
+
+cat > $CONF << ENDL
+[fs_types]
+ ext4h = {
+ features = has_journal,extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,sparse_super,filetype,dir_index,ext_attr,meta_bg,^resize_inode,64bit
+ blocksize = 1024
+ inode_size = 256
+ make_hugefiles = true
+ hugefiles_dir = /
+ hugefiles_slack = 0
+ hugefiles_name = aaaaa
+ hugefiles_digits = 4
+ hugefiles_size = 1M
+ zero_hugefiles = false
+ }
+ENDL
+
+echo "resize2fs test" > $OUT
+
+MKE2FS_CONFIG=$CONF $MKE2FS -F -T ext4h $TMPFILE 524288 >> $OUT 2>&1
+rm -rf $CONF
+
+# dump and check
+($DUMPE2FS -h $TMPFILE; $DUMPE2FS -g $TMPFILE) 2> /dev/null | sed -f $cmd_dir/filter.sed > $OUT.before
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# resize it
+echo "resize2fs test.img -s" >> $OUT
+$RESIZE2FS -s -f $TMPFILE 2>&1 >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+# dump and check
+($DUMPE2FS -h $TMPFILE; $DUMPE2FS -g $TMPFILE) 2> /dev/null | sed -f $cmd_dir/filter.sed > $OUT.after
+echo "Change in FS metadata:" >> $OUT
+diff -u $OUT.before $OUT.after | tail -n +3 >> $OUT
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+rm $TMPFILE
+
+#
+# Do the verification
+#
+
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" -e 's/test_filesys:.*//g' < $OUT > $OUT.new
+mv $OUT.new $OUT
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+fi
+
+rm $OUT.before $OUT.after
+
+unset IMAGE FSCK_OPT OUT EXP CONF
+
+else #if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
+
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 37/47] libext2fs: support allocating uninit blocks in bmap2()
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (35 preceding siblings ...)
2014-11-07 21:54 ` [PATCH 36/47] tests: test resize2fs 32->64 and 64->32bit conversion code Darrick J. Wong
@ 2014-11-07 21:54 ` Darrick J. Wong
2014-11-07 21:54 ` [PATCH 38/47] libext2fs: find/alloc a range of empty blocks Darrick J. Wong
` (11 subsequent siblings)
48 siblings, 0 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:54 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
As part of supporting fallocate-like functionality, extend
ext2fs_bmap() with two flags -- BMAP_UNINIT and BMAP_ZERO. The first
will cause it to mark/set a block uninitialized, if it's part of an
extent based file. For a block mapped file, the mapping is put in,
but there is no way to remember the uninitialized status. The second
flag causes the block to be zeroed to support the use case of
emulating uninitialized blocks on a block-map file by zeroing them.
Eventually fallocate or fuse2fs or somebody will use these.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
lib/ext2fs/bmap.c | 9 +++++++--
lib/ext2fs/ext2fs.h | 2 ++
2 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/lib/ext2fs/bmap.c b/lib/ext2fs/bmap.c
index cb3f5a1..c18f742 100644
--- a/lib/ext2fs/bmap.c
+++ b/lib/ext2fs/bmap.c
@@ -214,10 +214,13 @@ static errcode_t extent_bmap(ext2_filsys fs, ext2_ino_t ino,
errcode_t retval = 0;
blk64_t blk64 = 0;
int alloc = 0;
+ int set_flags;
+
+ set_flags = bmap_flags & BMAP_UNINIT ? EXT2_EXTENT_SET_BMAP_UNINIT : 0;
if (bmap_flags & BMAP_SET) {
retval = ext2fs_extent_set_bmap(handle, block,
- *phys_blk, 0);
+ *phys_blk, set_flags);
return retval;
}
retval = ext2fs_extent_goto(handle, block);
@@ -254,7 +257,7 @@ got_block:
alloc++;
set_extent:
retval = ext2fs_extent_set_bmap(handle, block,
- blk64, 0);
+ blk64, set_flags);
if (retval) {
ext2fs_block_alloc_stats2(fs, blk64, -1);
return retval;
@@ -441,6 +444,8 @@ errcode_t ext2fs_bmap2(ext2_filsys fs, ext2_ino_t ino, struct ext2_inode *inode,
if (retval == 0)
*phys_blk = blk32;
done:
+ if (*phys_blk && retval == 0 && (bmap_flags & BMAP_ZERO))
+ retval = ext2fs_zero_blocks2(fs, *phys_blk, 1, NULL, NULL);
if (buf)
ext2fs_free_mem(&buf);
if (handle)
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 84e4e1f..6a97a1f 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -527,6 +527,8 @@ typedef struct ext2_icount *ext2_icount_t;
*/
#define BMAP_ALLOC 0x0001
#define BMAP_SET 0x0002
+#define BMAP_UNINIT 0x0004
+#define BMAP_ZERO 0x0008
/*
* Returned flags from ext2fs_bmap
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 38/47] libext2fs: find/alloc a range of empty blocks
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (36 preceding siblings ...)
2014-11-07 21:54 ` [PATCH 37/47] libext2fs: support allocating uninit blocks in bmap2() Darrick J. Wong
@ 2014-11-07 21:54 ` Darrick J. Wong
2014-11-07 21:54 ` [PATCH 39/47] libext2fs: add new hooks to support large allocations Darrick J. Wong
` (10 subsequent siblings)
48 siblings, 0 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:54 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Provide a function that, given a goal pblk and a range, will try to
find a run of free blocks to satisfy the allocation. By default the
function will look anywhere in the filesystem for the run, though this
can be constrained with optional flags. One flag indicates that the
range must start at the goal block; the other flag indicates that we
should not return a range shorter than len.
v2: Add a second function to allocate a range of blocks.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
lib/ext2fs/alloc.c | 141 +++++++++++++++++++++++++++++++++++++++++++++++++++
lib/ext2fs/ext2fs.h | 11 ++++
2 files changed, 152 insertions(+)
diff --git a/lib/ext2fs/alloc.c b/lib/ext2fs/alloc.c
index d3c8e50..51f7a0c 100644
--- a/lib/ext2fs/alloc.c
+++ b/lib/ext2fs/alloc.c
@@ -26,6 +26,16 @@
#include "ext2_fs.h"
#include "ext2fs.h"
+#define min(a, b) ((a) < (b) ? (a) : (b))
+
+#undef DEBUG
+
+#ifdef DEBUG
+# define dbg_printf(f, a...) do {printf(f, ## a); fflush(stdout); } while (0)
+#else
+# define dbg_printf(f, a...)
+#endif
+
/*
* Clear the uninit block bitmap flag if necessary
*/
@@ -346,3 +356,134 @@ no_blocks:
group = group & ~((1 << (log_flex)) - 1);
return ext2fs_group_first_block2(fs, group);
}
+
+/*
+ * Starting at _goal_, scan around the filesystem to find a run of free blocks
+ * that's at least _len_ blocks long. Possible flags:
+ * - EXT2_NEWRANGE_EXACT_GOAL: The range of blocks must start at _goal_.
+ * - EXT2_NEWRANGE_MIN_LENGTH: do not return a allocation shorter than _len_.
+ * - EXT2_NEWRANGE_ZERO_BLOCKS: Zero blocks pblk to pblk+plen before returning.
+ *
+ * The starting block is returned in _pblk_ and the length is returned via
+ * _plen_. The blocks are not marked in the bitmap; the caller must mark
+ * however much of the returned run they actually use, hopefully via
+ * ext2fs_block_alloc_stats_range().
+ *
+ * This function can return a range that is longer than what was requested.
+ */
+errcode_t ext2fs_new_range(ext2_filsys fs, int flags, blk64_t goal,
+ blk64_t len, ext2fs_block_bitmap map, blk64_t *pblk,
+ blk64_t *plen)
+{
+ errcode_t retval;
+ blk64_t start, end, b;
+ int looped = 0;
+ blk64_t max_blocks = ext2fs_blocks_count(fs->super);
+
+ dbg_printf("%s: flags=0x%x goal=%llu len=%llu\n", __func__, flags,
+ goal, len);
+ EXT2_CHECK_MAGIC(fs, EXT2_ET_MAGIC_EXT2FS_FILSYS);
+ if (len == 0 || (flags & ~EXT2_NEWRANGE_ALL_FLAGS))
+ return EXT2_ET_INVALID_ARGUMENT;
+ if (!map)
+ map = fs->block_map;
+ if (!map)
+ return EXT2_ET_NO_BLOCK_BITMAP;
+ if (!goal || goal >= ext2fs_blocks_count(fs->super))
+ goal = fs->super->s_first_data_block;
+
+ start = goal;
+ while (!looped || start <= goal) {
+ retval = ext2fs_find_first_zero_block_bitmap2(map, start,
+ max_blocks - 1,
+ &start);
+ if (retval == ENOENT) {
+ /*
+ * If there are no free blocks beyond the starting
+ * point, try scanning the whole filesystem, unless the
+ * user told us only to allocate from _goal_, or if
+ * we're already scanning the whole filesystem.
+ */
+ if (flags & EXT2_NEWRANGE_FIXED_GOAL ||
+ start == fs->super->s_first_data_block)
+ goto fail;
+ start = fs->super->s_first_data_block;
+ continue;
+ } else if (retval)
+ goto errout;
+
+ if (flags & EXT2_NEWRANGE_FIXED_GOAL && start != goal)
+ goto fail;
+
+ b = min(start + len - 1, max_blocks - 1);
+ retval = ext2fs_find_first_set_block_bitmap2(map, start, b,
+ &end);
+ if (retval == ENOENT)
+ end = b + 1;
+ else if (retval)
+ goto errout;
+
+ if (!(flags & EXT2_NEWRANGE_MIN_LENGTH) ||
+ (end - start) >= len) {
+ /* Success! */
+ *pblk = start;
+ *plen = end - start;
+ dbg_printf("%s: new_range goal=%llu--%llu "
+ "blk=%llu--%llu %llu\n",
+ __func__, goal, goal + len - 1,
+ *pblk, *pblk + *plen - 1, *plen);
+
+ for (b = start; b < end;
+ b += fs->super->s_blocks_per_group)
+ clear_block_uninit(fs,
+ ext2fs_group_of_blk2(fs, b));
+ return 0;
+ }
+
+ if (flags & EXT2_NEWRANGE_FIXED_GOAL)
+ goto fail;
+ start = end;
+ if (start >= max_blocks) {
+ if (looped)
+ goto fail;
+ looped = 1;
+ start = fs->super->s_first_data_block;
+ }
+ }
+
+fail:
+ retval = EXT2_ET_BLOCK_ALLOC_FAIL;
+errout:
+ return retval;
+}
+
+errcode_t ext2fs_alloc_range(ext2_filsys fs, int flags, blk64_t goal,
+ blk_t len, blk64_t *ret)
+{
+ int newr_flags = EXT2_NEWRANGE_MIN_LENGTH;
+ errcode_t retval;
+ blk64_t plen;
+
+ EXT2_CHECK_MAGIC(fs, EXT2_ET_MAGIC_EXT2FS_FILSYS);
+ if (len == 0 || (flags & ~EXT2_ALLOCRANGE_ALL_FLAGS))
+ return EXT2_ET_INVALID_ARGUMENT;
+
+ if (flags & EXT2_ALLOCRANGE_FIXED_GOAL)
+ newr_flags |= EXT2_NEWRANGE_FIXED_GOAL;
+
+ retval = ext2fs_new_range(fs, newr_flags, goal, len, NULL, ret, &plen);
+ if (retval)
+ return retval;
+
+ if (plen < len)
+ return EXT2_ET_BLOCK_ALLOC_FAIL;
+
+ if (flags & EXT2_ALLOCRANGE_ZERO_BLOCKS) {
+ retval = ext2fs_zero_blocks2(fs, *ret, len, NULL, NULL);
+ if (retval)
+ return retval;
+ }
+
+ ext2fs_block_alloc_stats_range(fs, *ret, len, +1);
+ return retval;
+}
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 6a97a1f..b577cb0 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -693,6 +693,17 @@ extern void ext2fs_set_alloc_block_callback(ext2_filsys fs,
blk64_t *ret));
blk64_t ext2fs_find_inode_goal(ext2_filsys fs, ext2_ino_t ino,
struct ext2_inode *inode, blk64_t lblk);
+#define EXT2_NEWRANGE_FIXED_GOAL (0x1)
+#define EXT2_NEWRANGE_MIN_LENGTH (0x2)
+#define EXT2_NEWRANGE_ALL_FLAGS (0x3)
+errcode_t ext2fs_new_range(ext2_filsys fs, int flags, blk64_t goal,
+ blk64_t len, ext2fs_block_bitmap map, blk64_t *pblk,
+ blk64_t *plen);
+#define EXT2_ALLOCRANGE_FIXED_GOAL (0x1)
+#define EXT2_ALLOCRANGE_ZERO_BLOCKS (0x2)
+#define EXT2_ALLOCRANGE_ALL_FLAGS (0x3)
+errcode_t ext2fs_alloc_range(ext2_filsys fs, int flags, blk64_t goal,
+ blk_t len, blk64_t *ret);
/* alloc_sb.c */
extern int ext2fs_reserve_super_and_bgd(ext2_filsys fs,
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 39/47] libext2fs: add new hooks to support large allocations
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (37 preceding siblings ...)
2014-11-07 21:54 ` [PATCH 38/47] libext2fs: find/alloc a range of empty blocks Darrick J. Wong
@ 2014-11-07 21:54 ` Darrick J. Wong
2014-11-07 21:55 ` [PATCH 40/47] libext2fs: implement fallocate Darrick J. Wong
` (9 subsequent siblings)
48 siblings, 0 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:54 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Add a new get_alloc_blocks hook and a block_alloc_stats_range hook so
that e2fsck can capture allocation requests spanning more than a
block to its block_found_map.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
e2fsck/pass1.c | 45 +++++++++++++++++++++++++++++++++++++++++++++
lib/ext2fs/alloc.c | 37 ++++++++++++++++++++++++++++++++++++-
lib/ext2fs/alloc_stats.c | 16 ++++++++++++++++
lib/ext2fs/ext2fs.h | 16 ++++++++++++++++
4 files changed, 113 insertions(+), 1 deletion(-)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 10008d9..82792e4 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -3885,6 +3885,26 @@ static errcode_t e2fsck_get_alloc_block(ext2_filsys fs, blk64_t goal,
return (0);
}
+static errcode_t e2fsck_new_range(ext2_filsys fs, int flags, blk64_t goal,
+ blk64_t len, blk64_t *pblk, blk64_t *plen)
+{
+ e2fsck_t ctx = (e2fsck_t) fs->priv_data;
+ errcode_t retval;
+
+ if (ctx->block_found_map)
+ return ext2fs_new_range(fs, flags, goal, len,
+ ctx->block_found_map, pblk, plen);
+
+ if (!fs->block_map) {
+ retval = ext2fs_read_block_bitmap(fs);
+ if (retval)
+ return retval;
+ }
+
+ return ext2fs_new_range(fs, flags, goal, len, fs->block_map,
+ pblk, plen);
+}
+
static void e2fsck_block_alloc_stats(ext2_filsys fs, blk64_t blk, int inuse)
{
e2fsck_t ctx = (e2fsck_t) fs->priv_data;
@@ -3904,6 +3924,28 @@ static void e2fsck_block_alloc_stats(ext2_filsys fs, blk64_t blk, int inuse)
}
}
+static void e2fsck_block_alloc_stats_range(ext2_filsys fs, blk64_t blk,
+ blk_t num, int inuse)
+{
+ e2fsck_t ctx = (e2fsck_t) fs->priv_data;
+
+ /* Never free a critical metadata block */
+ if (ctx->block_found_map &&
+ ctx->block_metadata_map &&
+ inuse < 0 &&
+ ext2fs_test_block_bitmap_range2(ctx->block_metadata_map, blk, num))
+ return;
+
+ if (ctx->block_found_map) {
+ if (inuse > 0)
+ ext2fs_mark_block_bitmap_range2(ctx->block_found_map,
+ blk, num);
+ else
+ ext2fs_unmark_block_bitmap_range2(ctx->block_found_map,
+ blk, num);
+ }
+}
+
void e2fsck_use_inode_shortcuts(e2fsck_t ctx, int use_shortcuts)
{
ext2_filsys fs = ctx->fs;
@@ -3927,4 +3969,7 @@ void e2fsck_intercept_block_allocations(e2fsck_t ctx)
ext2fs_set_alloc_block_callback(ctx->fs, e2fsck_get_alloc_block, 0);
ext2fs_set_block_alloc_stats_callback(ctx->fs,
e2fsck_block_alloc_stats, 0);
+ ext2fs_set_new_range_callback(ctx->fs, e2fsck_new_range, NULL);
+ ext2fs_set_block_alloc_stats_range_callback(ctx->fs,
+ e2fsck_block_alloc_stats_range, NULL);
}
diff --git a/lib/ext2fs/alloc.c b/lib/ext2fs/alloc.c
index 51f7a0c..6f07069 100644
--- a/lib/ext2fs/alloc.c
+++ b/lib/ext2fs/alloc.c
@@ -379,12 +379,32 @@ errcode_t ext2fs_new_range(ext2_filsys fs, int flags, blk64_t goal,
blk64_t start, end, b;
int looped = 0;
blk64_t max_blocks = ext2fs_blocks_count(fs->super);
+ errcode_t (*nrf)(ext2_filsys fs, int flags, blk64_t goal,
+ blk64_t len, blk64_t *pblk, blk64_t *plen);
dbg_printf("%s: flags=0x%x goal=%llu len=%llu\n", __func__, flags,
goal, len);
EXT2_CHECK_MAGIC(fs, EXT2_ET_MAGIC_EXT2FS_FILSYS);
if (len == 0 || (flags & ~EXT2_NEWRANGE_ALL_FLAGS))
return EXT2_ET_INVALID_ARGUMENT;
+
+ if (!map && fs->new_range) {
+ /*
+ * In case there are clients out there whose new_range
+ * handlers call ext2fs_new_range with a NULL block map,
+ * temporarily swap out the function pointer so that we don't
+ * end up in an infinite loop.
+ */
+ nrf = fs->new_range;
+ fs->new_range = NULL;
+ retval = nrf(fs, flags, goal, len, pblk, plen);
+ fs->new_range = nrf;
+ if (retval)
+ return retval;
+ start = *pblk;
+ end = *pblk + *plen;
+ goto allocated;
+ }
if (!map)
map = fs->block_map;
if (!map)
@@ -432,7 +452,7 @@ errcode_t ext2fs_new_range(ext2_filsys fs, int flags, blk64_t goal,
"blk=%llu--%llu %llu\n",
__func__, goal, goal + len - 1,
*pblk, *pblk + *plen - 1, *plen);
-
+allocated:
for (b = start; b < end;
b += fs->super->s_blocks_per_group)
clear_block_uninit(fs,
@@ -457,6 +477,21 @@ errout:
return retval;
}
+void ext2fs_set_new_range_callback(ext2_filsys fs,
+ errcode_t (*func)(ext2_filsys fs, int flags, blk64_t goal,
+ blk64_t len, blk64_t *pblk, blk64_t *plen),
+ errcode_t (**old)(ext2_filsys fs, int flags, blk64_t goal,
+ blk64_t len, blk64_t *pblk, blk64_t *plen))
+{
+ if (!fs || fs->magic != EXT2_ET_MAGIC_EXT2FS_FILSYS)
+ return;
+
+ if (old)
+ *old = fs->new_range;
+
+ fs->new_range = func;
+}
+
errcode_t ext2fs_alloc_range(ext2_filsys fs, int flags, blk64_t goal,
blk_t len, blk64_t *ret)
{
diff --git a/lib/ext2fs/alloc_stats.c b/lib/ext2fs/alloc_stats.c
index aca5004..3949f61 100644
--- a/lib/ext2fs/alloc_stats.c
+++ b/lib/ext2fs/alloc_stats.c
@@ -145,4 +145,20 @@ void ext2fs_block_alloc_stats_range(ext2_filsys fs, blk64_t blk,
}
ext2fs_mark_super_dirty(fs);
ext2fs_mark_bb_dirty(fs);
+ if (fs->block_alloc_stats_range)
+ (fs->block_alloc_stats_range)(fs, blk, num, inuse);
+}
+
+void ext2fs_set_block_alloc_stats_range_callback(ext2_filsys fs,
+ void (*func)(ext2_filsys fs, blk64_t blk,
+ blk_t num, int inuse),
+ void (**old)(ext2_filsys fs, blk64_t blk,
+ blk_t num, int inuse))
+{
+ if (!fs || fs->magic != EXT2_ET_MAGIC_EXT2FS_FILSYS)
+ return;
+ if (old)
+ *old = fs->block_alloc_stats_range;
+
+ fs->block_alloc_stats_range = func;
}
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index b577cb0..df74965 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -279,6 +279,12 @@ struct struct_ext2_filsys {
io_channel journal_io;
char *journal_name;
+
+ /* New block range allocation hooks */
+ errcode_t (*new_range)(ext2_filsys fs, int flags, blk64_t goal,
+ blk64_t len, blk64_t *pblk, blk64_t *plen);
+ void (*block_alloc_stats_range)(ext2_filsys fs, blk64_t blk, blk_t num,
+ int inuse);
};
#if EXT2_FLAT_INCLUDES
@@ -693,6 +699,16 @@ extern void ext2fs_set_alloc_block_callback(ext2_filsys fs,
blk64_t *ret));
blk64_t ext2fs_find_inode_goal(ext2_filsys fs, ext2_ino_t ino,
struct ext2_inode *inode, blk64_t lblk);
+extern void ext2fs_set_new_range_callback(ext2_filsys fs,
+ errcode_t (*func)(ext2_filsys fs, int flags, blk64_t goal,
+ blk64_t len, blk64_t *pblk, blk64_t *plen),
+ errcode_t (**old)(ext2_filsys fs, int flags, blk64_t goal,
+ blk64_t len, blk64_t *pblk, blk64_t *plen));
+extern void ext2fs_set_block_alloc_stats_range_callback(ext2_filsys fs,
+ void (*func)(ext2_filsys fs, blk64_t blk,
+ blk_t num, int inuse),
+ void (**old)(ext2_filsys fs, blk64_t blk,
+ blk_t num, int inuse));
#define EXT2_NEWRANGE_FIXED_GOAL (0x1)
#define EXT2_NEWRANGE_MIN_LENGTH (0x2)
#define EXT2_NEWRANGE_ALL_FLAGS (0x3)
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 40/47] libext2fs: implement fallocate
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (38 preceding siblings ...)
2014-11-07 21:54 ` [PATCH 39/47] libext2fs: add new hooks to support large allocations Darrick J. Wong
@ 2014-11-07 21:55 ` Darrick J. Wong
2014-11-07 21:55 ` [PATCH 41/47] libext2fs: use fallocate for creating journals and hugefiles Darrick J. Wong
` (8 subsequent siblings)
48 siblings, 0 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:55 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Create a library function to perform fallocation on arbitrary files,
and wire up a few users for this function. This is a bit more intense
than Ted's original mk_hugefiles implementation since we have to honor
any blocks that may already be allocated to the file.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
lib/ext2fs/Makefile.in | 8
lib/ext2fs/ext2fs.h | 10 +
lib/ext2fs/fallocate.c | 853 ++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 871 insertions(+)
create mode 100644 lib/ext2fs/fallocate.c
diff --git a/lib/ext2fs/Makefile.in b/lib/ext2fs/Makefile.in
index 2706bfa..d1350b5 100644
--- a/lib/ext2fs/Makefile.in
+++ b/lib/ext2fs/Makefile.in
@@ -79,6 +79,7 @@ OBJS= $(DEBUGFS_LIB_OBJS) $(RESIZE_LIB_OBJS) $(E2IMAGE_LIB_OBJS) \
expanddir.o \
ext_attr.o \
extent.o \
+ fallocate.o \
fileio.o \
finddev.o \
flushb.o \
@@ -763,6 +764,13 @@ extent.o: $(srcdir)/extent.c $(top_builddir)/lib/config.h \
$(top_srcdir)/lib/et/com_err.h $(srcdir)/ext2_io.h \
$(top_builddir)/lib/ext2fs/ext2_err.h $(srcdir)/ext2_ext_attr.h \
$(srcdir)/bitops.h $(srcdir)/e2image.h
+fallocate.o: $(srcdir)/fallocate.c $(top_builddir)/lib/config.h \
+ $(top_builddir)/lib/dirpaths.h $(srcdir)/ext2_fs.h \
+ $(top_builddir)/lib/ext2fs/ext2_types.h $(srcdir)/ext2fsP.h \
+ $(srcdir)/ext2fs.h $(srcdir)/ext2_fs.h $(srcdir)/ext3_extents.h \
+ $(top_srcdir)/lib/et/com_err.h $(srcdir)/ext2_io.h \
+ $(top_builddir)/lib/ext2fs/ext2_err.h $(srcdir)/ext2_ext_attr.h \
+ $(srcdir)/bitops.h $(srcdir)/e2image.h
fileio.o: $(srcdir)/fileio.c $(top_builddir)/lib/config.h \
$(top_builddir)/lib/dirpaths.h $(srcdir)/ext2_fs.h \
$(top_builddir)/lib/ext2fs/ext2_types.h $(srcdir)/ext2fs.h \
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index df74965..5738294 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -1255,6 +1255,16 @@ extern errcode_t ext2fs_extent_goto2(ext2_extent_handle_t handle,
extern errcode_t ext2fs_extent_fix_parents(ext2_extent_handle_t handle);
size_t ext2fs_max_extent_depth(ext2_extent_handle_t handle);
+/* fallocate.c */
+#define EXT2_FALLOCATE_ZERO_BLOCKS (0x1)
+#define EXT2_FALLOCATE_FORCE_INIT (0x2)
+#define EXT2_FALLOCATE_FORCE_UNINIT (0x4)
+#define EXT2_FALLOCATE_INIT_BEYOND_EOF (0x8)
+#define EXT2_FALLOCATE_ALL_FLAGS (0xF)
+errcode_t ext2fs_fallocate(ext2_filsys fs, int flags, ext2_ino_t ino,
+ struct ext2_inode *inode, blk64_t goal,
+ blk64_t start, blk64_t len);
+
/* fileio.c */
extern errcode_t ext2fs_file_open2(ext2_filsys fs, ext2_ino_t ino,
struct ext2_inode *inode,
diff --git a/lib/ext2fs/fallocate.c b/lib/ext2fs/fallocate.c
new file mode 100644
index 0000000..8b502d5
--- /dev/null
+++ b/lib/ext2fs/fallocate.c
@@ -0,0 +1,853 @@
+/*
+ * fallocate.c -- Allocate large chunks of file.
+ *
+ * Copyright (C) 2014 Oracle.
+ *
+ * %Begin-Header%
+ * This file may be redistributed under the terms of the GNU Library
+ * General Public License, version 2.
+ * %End-Header%
+ */
+
+#include "config.h"
+
+#include "ext2_fs.h"
+#include "ext2fs.h"
+#define min(a, b) ((a) < (b) ? (a) : (b))
+
+#undef DEBUG
+
+#ifdef DEBUG
+# define dbg_printf(f, a...) do {printf(f, ## a); fflush(stdout); } while (0)
+#else
+# define dbg_printf(f, a...)
+#endif
+
+/*
+ * Extent-based fallocate code.
+ *
+ * Find runs of unmapped logical blocks by starting at start and walking the
+ * extents until we reach the end of the range we want.
+ *
+ * For each run of unmapped blocks, try to find the extents on either side of
+ * the range. If there's a left extent that can grow by at least a cluster and
+ * there are lblocks between start and the next lcluster after start, see if
+ * there's an implied cluster allocation; if so, zero the blocks (if the left
+ * extent is initialized) and adjust the extent. Ditto for the blocks between
+ * the end of the last full lcluster and end, if there's a right extent.
+ *
+ * Try to attach as much as we can to the left extent, then try to attach as
+ * much as we can to the right extent. For the remainder, try to allocate the
+ * whole range; map in whatever we get; and repeat until we're done.
+ *
+ * To attach to a left extent, figure out the maximum amount we can add to the
+ * extent and try to allocate that much, and append if successful. To attach
+ * to a right extent, figure out the max we can add to the extent, try to
+ * allocate that much, and prepend if successful.
+ *
+ * We need an alloc_range function that tells us how much we can allocate given
+ * a maximum length and one of a suggested start, a fixed start, or a fixed end
+ * point.
+ *
+ * Every time we modify the extent tree we also need to update the block stats.
+ *
+ * At the end, update i_blocks and i_size appropriately.
+ */
+
+static void dbg_print_extent(char *desc, struct ext2fs_extent *extent)
+{
+#ifdef DEBUG
+ if (desc)
+ printf("%s: ", desc);
+ printf("extent: lblk %llu--%llu, len %u, pblk %llu, flags: ",
+ extent->e_lblk, extent->e_lblk + extent->e_len - 1,
+ extent->e_len, extent->e_pblk);
+ if (extent->e_flags & EXT2_EXTENT_FLAGS_LEAF)
+ fputs("LEAF ", stdout);
+ if (extent->e_flags & EXT2_EXTENT_FLAGS_UNINIT)
+ fputs("UNINIT ", stdout);
+ if (extent->e_flags & EXT2_EXTENT_FLAGS_SECOND_VISIT)
+ fputs("2ND_VISIT ", stdout);
+ if (!extent->e_flags)
+ fputs("(none)", stdout);
+ fputc('\n', stdout);
+ fflush(stdout);
+#endif
+}
+
+static errcode_t claim_range(ext2_filsys fs, struct ext2_inode *inode,
+ blk64_t blk, blk64_t len)
+{
+ blk64_t clusters;
+
+ clusters = (len + EXT2FS_CLUSTER_RATIO(fs) - 1) /
+ EXT2FS_CLUSTER_RATIO(fs);
+ ext2fs_block_alloc_stats_range(fs, blk,
+ clusters * EXT2FS_CLUSTER_RATIO(fs), +1);
+ return ext2fs_iblk_add_blocks(fs, inode, clusters);
+}
+
+static errcode_t ext_falloc_helper(ext2_filsys fs,
+ int flags,
+ ext2_ino_t ino,
+ struct ext2_inode *inode,
+ ext2_extent_handle_t handle,
+ struct ext2fs_extent *left_ext,
+ struct ext2fs_extent *right_ext,
+ blk64_t range_start, blk64_t range_len,
+ blk64_t alloc_goal)
+{
+ struct ext2fs_extent newex, ex;
+ int op;
+ blk64_t fillable, pblk, plen, x, y;
+ blk64_t eof_blk = 0, cluster_fill = 0;
+ errcode_t err;
+ blk_t max_extent_len, max_uninit_len, max_init_len;
+
+#ifdef DEBUG
+ printf("%s: ", __func__);
+ if (left_ext)
+ printf("left_ext=%llu--%llu, ", left_ext->e_lblk,
+ left_ext->e_lblk + left_ext->e_len - 1);
+ if (right_ext)
+ printf("right_ext=%llu--%llu, ", right_ext->e_lblk,
+ right_ext->e_lblk + right_ext->e_len - 1);
+ printf("start=%llu len=%llu, goal=%llu\n", range_start, range_len,
+ alloc_goal);
+ fflush(stdout);
+#endif
+ /* Can't create initialized extents past EOF? */
+ if (!(flags & EXT2_FALLOCATE_INIT_BEYOND_EOF))
+ eof_blk = EXT2_I_SIZE(inode) / fs->blocksize;
+
+ /* The allocation goal must be as far into a cluster as range_start. */
+ alloc_goal = (alloc_goal & ~EXT2FS_CLUSTER_MASK(fs)) |
+ (range_start & EXT2FS_CLUSTER_MASK(fs));
+
+ max_uninit_len = EXT_UNINIT_MAX_LEN & ~EXT2FS_CLUSTER_MASK(fs);
+ max_init_len = EXT_INIT_MAX_LEN & ~EXT2FS_CLUSTER_MASK(fs);
+
+ /* We must lengthen the left extent to the end of the cluster */
+ if (left_ext && EXT2FS_CLUSTER_RATIO(fs) > 1) {
+ /* How many more blocks can be attached to left_ext? */
+ if (left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)
+ fillable = max_uninit_len - left_ext->e_len;
+ else
+ fillable = max_init_len - left_ext->e_len;
+
+ if (fillable > range_len)
+ fillable = range_len;
+ if (fillable == 0)
+ goto expand_right;
+
+ /*
+ * If range_start isn't on a cluster boundary, try an
+ * implied cluster allocation for left_ext.
+ */
+ cluster_fill = EXT2FS_CLUSTER_RATIO(fs) -
+ (range_start & EXT2FS_CLUSTER_MASK(fs));
+ cluster_fill &= EXT2FS_CLUSTER_MASK(fs);
+ if (cluster_fill == 0)
+ goto expand_right;
+
+ if (cluster_fill > fillable)
+ cluster_fill = fillable;
+
+ /* Don't expand an initialized left_ext beyond EOF */
+ if (!(flags & EXT2_FALLOCATE_INIT_BEYOND_EOF)) {
+ x = left_ext->e_lblk + left_ext->e_len - 1;
+ dbg_printf("%s: lend=%llu newlend=%llu eofblk=%llu\n",
+ __func__, x, x + cluster_fill, eof_blk);
+ if (eof_blk >= x && eof_blk <= x + cluster_fill)
+ cluster_fill = eof_blk - x;
+ if (cluster_fill == 0)
+ goto expand_right;
+ }
+
+ err = ext2fs_extent_goto(handle, left_ext->e_lblk);
+ if (err)
+ goto expand_right;
+ left_ext->e_len += cluster_fill;
+ range_start += cluster_fill;
+ range_len -= cluster_fill;
+ alloc_goal += cluster_fill;
+
+ dbg_print_extent("ext_falloc clus left+", left_ext);
+ err = ext2fs_extent_replace(handle, 0, left_ext);
+ if (err)
+ goto out;
+ err = ext2fs_extent_fix_parents(handle);
+ if (err)
+ goto out;
+
+ /* Zero blocks */
+ if (!(left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)) {
+ err = ext2fs_zero_blocks2(fs, left_ext->e_pblk +
+ left_ext->e_len -
+ cluster_fill, cluster_fill,
+ NULL, NULL);
+ if (err)
+ goto out;
+ }
+ }
+
+expand_right:
+ /* We must lengthen the right extent to the beginning of the cluster */
+ if (right_ext && EXT2FS_CLUSTER_RATIO(fs) > 1) {
+ /* How much can we attach to right_ext? */
+ if (right_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)
+ fillable = max_uninit_len - right_ext->e_len;
+ else
+ fillable = max_init_len - right_ext->e_len;
+
+ if (fillable > range_len)
+ fillable = range_len;
+ if (fillable == 0)
+ goto try_merge;
+
+ /*
+ * If range_end isn't on a cluster boundary, try an implied
+ * cluster allocation for right_ext.
+ */
+ cluster_fill = right_ext->e_lblk & EXT2FS_CLUSTER_MASK(fs);
+ if (cluster_fill == 0)
+ goto try_merge;
+
+ err = ext2fs_extent_goto(handle, right_ext->e_lblk);
+ if (err)
+ goto out;
+
+ if (cluster_fill > fillable)
+ cluster_fill = fillable;
+ right_ext->e_lblk -= cluster_fill;
+ right_ext->e_pblk -= cluster_fill;
+ right_ext->e_len += cluster_fill;
+ range_len -= cluster_fill;
+
+ dbg_print_extent("ext_falloc clus right+", right_ext);
+ err = ext2fs_extent_replace(handle, 0, right_ext);
+ if (err)
+ goto out;
+ err = ext2fs_extent_fix_parents(handle);
+ if (err)
+ goto out;
+
+ /* Zero blocks if necessary */
+ if (!(right_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)) {
+ err = ext2fs_zero_blocks2(fs, right_ext->e_pblk,
+ cluster_fill, NULL, NULL);
+ if (err)
+ goto out;
+ }
+ }
+
+try_merge:
+ /* Merge both extents together, perhaps? */
+ if (left_ext && right_ext) {
+ /* Are the two extents mergeable? */
+ if ((left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT) !=
+ (right_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT))
+ goto try_left;
+
+ /* User requires init/uninit but extent is uninit/init. */
+ if (((flags & EXT2_FALLOCATE_FORCE_INIT) &&
+ (left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)) ||
+ ((flags & EXT2_FALLOCATE_FORCE_UNINIT) &&
+ !(left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)))
+ goto try_left;
+
+ /*
+ * Skip initialized extent unless user wants to zero blocks
+ * or requires init extent.
+ */
+ if (!(left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT) &&
+ (!(flags & EXT2_FALLOCATE_ZERO_BLOCKS) ||
+ !(flags & EXT2_FALLOCATE_FORCE_INIT)))
+ goto try_left;
+
+ /* Will it even fit? */
+ x = left_ext->e_len + range_len + right_ext->e_len;
+ if (x > (left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT ?
+ max_uninit_len : max_init_len))
+ goto try_left;
+
+ err = ext2fs_extent_goto(handle, left_ext->e_lblk);
+ if (err)
+ goto try_left;
+
+ /* Allocate blocks */
+ y = left_ext->e_pblk + left_ext->e_len;
+ err = ext2fs_new_range(fs, EXT2_NEWRANGE_FIXED_GOAL |
+ EXT2_NEWRANGE_MIN_LENGTH, y,
+ right_ext->e_pblk - y + 1, NULL,
+ &pblk, &plen);
+ if (err)
+ goto try_left;
+ if (pblk + plen != right_ext->e_pblk)
+ goto try_left;
+ err = claim_range(fs, inode, pblk, plen);
+ if (err)
+ goto out;
+
+ /* Modify extents */
+ left_ext->e_len = x;
+ dbg_print_extent("ext_falloc merge", left_ext);
+ err = ext2fs_extent_replace(handle, 0, left_ext);
+ if (err)
+ goto out;
+ err = ext2fs_extent_fix_parents(handle);
+ if (err)
+ goto out;
+ err = ext2fs_extent_get(handle, EXT2_EXTENT_NEXT_LEAF, &newex);
+ if (err)
+ goto out;
+ err = ext2fs_extent_delete(handle, 0);
+ if (err)
+ goto out;
+ err = ext2fs_extent_fix_parents(handle);
+ if (err)
+ goto out;
+ *right_ext = *left_ext;
+
+ /* Zero blocks */
+ if (!(left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT) &&
+ (flags & EXT2_FALLOCATE_ZERO_BLOCKS)) {
+ err = ext2fs_zero_blocks2(fs, range_start, range_len,
+ NULL, NULL);
+ if (err)
+ goto out;
+ }
+
+ return 0;
+ }
+
+try_left:
+ /* Extend the left extent */
+ if (left_ext) {
+ /* How many more blocks can be attached to left_ext? */
+ if (left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)
+ fillable = max_uninit_len - left_ext->e_len;
+ else if (flags & EXT2_FALLOCATE_ZERO_BLOCKS)
+ fillable = max_init_len - left_ext->e_len;
+ else
+ fillable = 0;
+
+ /* User requires init/uninit but extent is uninit/init. */
+ if (((flags & EXT2_FALLOCATE_FORCE_INIT) &&
+ (left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)) ||
+ ((flags & EXT2_FALLOCATE_FORCE_UNINIT) &&
+ !(left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)))
+ goto try_right;
+
+ if (fillable > range_len)
+ fillable = range_len;
+
+ /* Don't expand an initialized left_ext beyond EOF */
+ x = left_ext->e_lblk + left_ext->e_len - 1;
+ if (!(flags & EXT2_FALLOCATE_INIT_BEYOND_EOF)) {
+ dbg_printf("%s: lend=%llu newlend=%llu eofblk=%llu\n",
+ __func__, x, x + fillable, eof_blk);
+ if (eof_blk >= x && eof_blk <= x + fillable)
+ fillable = eof_blk - x;
+ }
+
+ if (fillable == 0)
+ goto try_right;
+
+ /* Test if the right edge of the range is already mapped? */
+ if (EXT2FS_CLUSTER_RATIO(fs) > 1) {
+ err = ext2fs_map_cluster_block(fs, ino, inode,
+ x + fillable, &pblk);
+ if (err)
+ goto out;
+ if (pblk)
+ fillable -= 1 + ((x + fillable)
+ & EXT2FS_CLUSTER_MASK(fs));
+ if (fillable == 0)
+ goto try_right;
+ }
+
+ /* Allocate range of blocks */
+ x = left_ext->e_pblk + left_ext->e_len;
+ err = ext2fs_new_range(fs, EXT2_NEWRANGE_FIXED_GOAL |
+ EXT2_NEWRANGE_MIN_LENGTH,
+ x, fillable, NULL, &pblk, &plen);
+ if (err)
+ goto try_right;
+ err = claim_range(fs, inode, pblk, plen);
+ if (err)
+ goto out;
+
+ /* Modify left_ext */
+ err = ext2fs_extent_goto(handle, left_ext->e_lblk);
+ if (err)
+ goto out;
+ range_start += plen;
+ range_len -= plen;
+ left_ext->e_len += plen;
+ dbg_print_extent("ext_falloc left+", left_ext);
+ err = ext2fs_extent_replace(handle, 0, left_ext);
+ if (err)
+ goto out;
+ err = ext2fs_extent_fix_parents(handle);
+ if (err)
+ goto out;
+
+ /* Zero blocks if necessary */
+ if (!(left_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT) &&
+ (flags & EXT2_FALLOCATE_ZERO_BLOCKS)) {
+ err = ext2fs_zero_blocks2(fs, pblk, plen, NULL, NULL);
+ if (err)
+ goto out;
+ }
+ }
+
+try_right:
+ /* Extend the right extent */
+ if (right_ext) {
+ /* How much can we attach to right_ext? */
+ if (right_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)
+ fillable = max_uninit_len - right_ext->e_len;
+ else if (flags & EXT2_FALLOCATE_ZERO_BLOCKS)
+ fillable = max_init_len - right_ext->e_len;
+ else
+ fillable = 0;
+
+ /* User requires init/uninit but extent is uninit/init. */
+ if (((flags & EXT2_FALLOCATE_FORCE_INIT) &&
+ (right_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)) ||
+ ((flags & EXT2_FALLOCATE_FORCE_UNINIT) &&
+ !(right_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT)))
+ goto try_anywhere;
+
+ if (fillable > range_len)
+ fillable = range_len;
+ if (fillable == 0)
+ goto try_anywhere;
+
+ /* Test if the left edge of the range is already mapped? */
+ if (EXT2FS_CLUSTER_RATIO(fs) > 1) {
+ err = ext2fs_map_cluster_block(fs, ino, inode,
+ right_ext->e_lblk - fillable, &pblk);
+ if (err)
+ goto out;
+ if (pblk)
+ fillable -= EXT2FS_CLUSTER_RATIO(fs) -
+ ((right_ext->e_lblk - fillable)
+ & EXT2FS_CLUSTER_MASK(fs));
+ if (fillable == 0)
+ goto try_anywhere;
+ }
+
+ /*
+ * FIXME: It would be nice if we could handle allocating a
+ * variable range from a fixed end point instead of just
+ * skipping to the general allocator if the whole range is
+ * unavailable.
+ */
+ err = ext2fs_new_range(fs, EXT2_NEWRANGE_FIXED_GOAL |
+ EXT2_NEWRANGE_MIN_LENGTH,
+ right_ext->e_pblk - fillable,
+ fillable, NULL, &pblk, &plen);
+ if (err)
+ goto try_anywhere;
+ err = claim_range(fs, inode,
+ pblk & ~EXT2FS_CLUSTER_MASK(fs),
+ plen + (pblk & EXT2FS_CLUSTER_MASK(fs)));
+ if (err)
+ goto out;
+
+ /* Modify right_ext */
+ err = ext2fs_extent_goto(handle, right_ext->e_lblk);
+ if (err)
+ goto out;
+ range_len -= plen;
+ right_ext->e_lblk -= plen;
+ right_ext->e_pblk -= plen;
+ right_ext->e_len += plen;
+ dbg_print_extent("ext_falloc right+", right_ext);
+ err = ext2fs_extent_replace(handle, 0, right_ext);
+ if (err)
+ goto out;
+ err = ext2fs_extent_fix_parents(handle);
+ if (err)
+ goto out;
+
+ /* Zero blocks if necessary */
+ if (!(right_ext->e_flags & EXT2_EXTENT_FLAGS_UNINIT) &&
+ (flags & EXT2_FALLOCATE_ZERO_BLOCKS)) {
+ err = ext2fs_zero_blocks2(fs, pblk,
+ plen + cluster_fill, NULL, NULL);
+ if (err)
+ goto out;
+ }
+ }
+
+try_anywhere:
+ /* Try implied cluster alloc on the left and right ends */
+ if (range_len > 0 && (range_start & EXT2FS_CLUSTER_MASK(fs))) {
+ cluster_fill = EXT2FS_CLUSTER_RATIO(fs) -
+ (range_start & EXT2FS_CLUSTER_MASK(fs));
+ cluster_fill &= EXT2FS_CLUSTER_MASK(fs);
+ if (cluster_fill > range_len)
+ cluster_fill = range_len;
+ newex.e_lblk = range_start;
+ err = ext2fs_map_cluster_block(fs, ino, inode, newex.e_lblk,
+ &pblk);
+ if (err)
+ goto out;
+ if (pblk == 0)
+ goto try_right_implied;
+ newex.e_pblk = pblk;
+ newex.e_len = cluster_fill;
+ newex.e_flags = (flags & EXT2_FALLOCATE_FORCE_INIT ? 0 :
+ EXT2_EXTENT_FLAGS_UNINIT);
+ dbg_print_extent("ext_falloc iclus left+", &newex);
+ ext2fs_extent_goto(handle, newex.e_lblk);
+ err = ext2fs_extent_get(handle, EXT2_EXTENT_CURRENT,
+ &ex);
+ if (err == EXT2_ET_NO_CURRENT_NODE)
+ ex.e_lblk = 0;
+ else if (err)
+ goto out;
+
+ if (ex.e_lblk > newex.e_lblk)
+ op = 0; /* insert before */
+ else
+ op = EXT2_EXTENT_INSERT_AFTER;
+ dbg_printf("%s: inserting %s lblk %llu newex=%llu\n",
+ __func__, op ? "after" : "before", ex.e_lblk,
+ newex.e_lblk);
+ err = ext2fs_extent_insert(handle, op, &newex);
+ if (err)
+ goto out;
+ err = ext2fs_extent_fix_parents(handle);
+ if (err)
+ goto out;
+
+ if (!(newex.e_flags & EXT2_EXTENT_FLAGS_UNINIT) &&
+ (flags & EXT2_FALLOCATE_ZERO_BLOCKS)) {
+ err = ext2fs_zero_blocks2(fs, newex.e_pblk,
+ newex.e_len, NULL, NULL);
+ if (err)
+ goto out;
+ }
+
+ range_start += cluster_fill;
+ range_len -= cluster_fill;
+ }
+
+try_right_implied:
+ y = range_start + range_len;
+ if (range_len > 0 && (y & EXT2FS_CLUSTER_MASK(fs))) {
+ cluster_fill = y & EXT2FS_CLUSTER_MASK(fs);
+ if (cluster_fill > range_len)
+ cluster_fill = range_len;
+ newex.e_lblk = y & ~EXT2FS_CLUSTER_MASK(fs);
+ err = ext2fs_map_cluster_block(fs, ino, inode, newex.e_lblk,
+ &pblk);
+ if (err)
+ goto out;
+ if (pblk == 0)
+ goto no_implied;
+ newex.e_pblk = pblk;
+ newex.e_len = cluster_fill;
+ newex.e_flags = (flags & EXT2_FALLOCATE_FORCE_INIT ? 0 :
+ EXT2_EXTENT_FLAGS_UNINIT);
+ dbg_print_extent("ext_falloc iclus right+", &newex);
+ ext2fs_extent_goto(handle, newex.e_lblk);
+ err = ext2fs_extent_get(handle, EXT2_EXTENT_CURRENT,
+ &ex);
+ if (err == EXT2_ET_NO_CURRENT_NODE)
+ ex.e_lblk = 0;
+ else if (err)
+ goto out;
+
+ if (ex.e_lblk > newex.e_lblk)
+ op = 0; /* insert before */
+ else
+ op = EXT2_EXTENT_INSERT_AFTER;
+ dbg_printf("%s: inserting %s lblk %llu newex=%llu\n",
+ __func__, op ? "after" : "before", ex.e_lblk,
+ newex.e_lblk);
+ err = ext2fs_extent_insert(handle, op, &newex);
+ if (err)
+ goto out;
+ err = ext2fs_extent_fix_parents(handle);
+ if (err)
+ goto out;
+
+ if (!(newex.e_flags & EXT2_EXTENT_FLAGS_UNINIT) &&
+ (flags & EXT2_FALLOCATE_ZERO_BLOCKS)) {
+ err = ext2fs_zero_blocks2(fs, newex.e_pblk,
+ newex.e_len, NULL, NULL);
+ if (err)
+ goto out;
+ }
+
+ range_len -= cluster_fill;
+ }
+
+no_implied:
+ if (range_len == 0)
+ return 0;
+
+ newex.e_lblk = range_start;
+ if (flags & EXT2_FALLOCATE_FORCE_INIT) {
+ max_extent_len = max_init_len;
+ newex.e_flags = 0;
+ } else {
+ max_extent_len = max_uninit_len;
+ newex.e_flags = EXT2_EXTENT_FLAGS_UNINIT;
+ }
+ pblk = alloc_goal;
+ y = range_len;
+ for (x = 0; x < y;) {
+ cluster_fill = newex.e_lblk & EXT2FS_CLUSTER_MASK(fs);
+ fillable = min(range_len + cluster_fill, max_extent_len);
+ err = ext2fs_new_range(fs, 0, pblk & ~EXT2FS_CLUSTER_MASK(fs),
+ fillable,
+ NULL, &pblk, &plen);
+ if (err)
+ goto out;
+ err = claim_range(fs, inode, pblk, plen);
+ if (err)
+ goto out;
+
+ /* Create extent */
+ newex.e_pblk = pblk + cluster_fill;
+ newex.e_len = plen - cluster_fill;
+ dbg_print_extent("ext_falloc create", &newex);
+ ext2fs_extent_goto(handle, newex.e_lblk);
+ err = ext2fs_extent_get(handle, EXT2_EXTENT_CURRENT,
+ &ex);
+ if (err == EXT2_ET_NO_CURRENT_NODE)
+ ex.e_lblk = 0;
+ else if (err)
+ goto out;
+
+ if (ex.e_lblk > newex.e_lblk)
+ op = 0; /* insert before */
+ else
+ op = EXT2_EXTENT_INSERT_AFTER;
+ dbg_printf("%s: inserting %s lblk %llu newex=%llu\n",
+ __func__, op ? "after" : "before", ex.e_lblk,
+ newex.e_lblk);
+ err = ext2fs_extent_insert(handle, op, &newex);
+ if (err)
+ goto out;
+ err = ext2fs_extent_fix_parents(handle);
+ if (err)
+ goto out;
+
+ if (!(newex.e_flags & EXT2_EXTENT_FLAGS_UNINIT) &&
+ (flags & EXT2_FALLOCATE_ZERO_BLOCKS)) {
+ err = ext2fs_zero_blocks2(fs, pblk, plen, NULL, NULL);
+ if (err)
+ goto out;
+ }
+
+ /* Update variables at end of loop */
+ x += plen - cluster_fill;
+ range_len -= plen - cluster_fill;
+ newex.e_lblk += plen - cluster_fill;
+ pblk += plen - cluster_fill;
+ if (pblk >= ext2fs_blocks_count(fs->super))
+ pblk = fs->super->s_first_data_block;
+ }
+
+out:
+ return err;
+}
+
+static errcode_t extent_fallocate(ext2_filsys fs, int flags, ext2_ino_t ino,
+ struct ext2_inode *inode, blk64_t goal,
+ blk64_t start, blk64_t len)
+{
+ ext2_extent_handle_t handle;
+ struct ext2fs_extent left_extent, right_extent;
+ struct ext2fs_extent *left_adjacent, *right_adjacent;
+ errcode_t err;
+ blk64_t range_start, range_end = 0, end, next;
+ blk64_t count, goal_distance;
+
+ end = start + len - 1;
+ err = ext2fs_extent_open2(fs, ino, inode, &handle);
+ if (err)
+ return err;
+
+ /*
+ * Find the extent closest to the start of the alloc range. We don't
+ * check the return value because _goto() sets the current node to the
+ * next-lowest extent if 'start' is in a hole; or the next-highest
+ * extent if there aren't any lower ones; or doesn't set a current node
+ * if there was a real error reading the extent tree. In that case,
+ * _get() will error out.
+ */
+start_again:
+ ext2fs_extent_goto(handle, start);
+ err = ext2fs_extent_get(handle, EXT2_EXTENT_CURRENT, &left_extent);
+ if (err == EXT2_ET_NO_CURRENT_NODE) {
+ blk64_t max_blocks = ext2fs_blocks_count(fs->super);
+
+ if (goal == ~0ULL)
+ goal = ext2fs_find_inode_goal(fs, ino, inode, start);
+ err = ext2fs_find_first_zero_block_bitmap2(fs->block_map,
+ goal, max_blocks - 1, &goal);
+ goal += start;
+ err = ext_falloc_helper(fs, flags, ino, inode, handle, NULL,
+ NULL, start, len, goal);
+ goto errout;
+ } else if (err)
+ goto errout;
+
+ dbg_print_extent("ext_falloc initial", &left_extent);
+ next = left_extent.e_lblk + left_extent.e_len;
+ if (left_extent.e_lblk > start) {
+ /* The nearest extent we found was beyond start??? */
+ goal = left_extent.e_pblk - (left_extent.e_lblk - start);
+ err = ext_falloc_helper(fs, flags, ino, inode, handle, NULL,
+ &left_extent, start,
+ left_extent.e_lblk - start, goal);
+ if (err)
+ goto errout;
+
+ goto start_again;
+ } else if (next >= start) {
+ range_start = next;
+ left_adjacent = &left_extent;
+ } else {
+ range_start = start;
+ left_adjacent = NULL;
+ }
+ goal = left_extent.e_pblk + (range_start - left_extent.e_lblk);
+ goal_distance = range_start - next;
+
+ do {
+ err = ext2fs_extent_get(handle, EXT2_EXTENT_NEXT_LEAF,
+ &right_extent);
+ dbg_printf("%s: ino=%d get next =%d\n", __func__, ino,
+ (int)err);
+ dbg_print_extent("ext_falloc next", &right_extent);
+ /* Stop if we've seen this extent before */
+ if (!err && right_extent.e_lblk <= left_extent.e_lblk)
+ err = EXT2_ET_EXTENT_NO_NEXT;
+
+ if (err && err != EXT2_ET_EXTENT_NO_NEXT)
+ goto errout;
+ if (err == EXT2_ET_EXTENT_NO_NEXT ||
+ right_extent.e_lblk > end + 1) {
+ range_end = end;
+ right_adjacent = NULL;
+ } else {
+ /* Handle right_extent.e_lblk <= end */
+ range_end = right_extent.e_lblk - 1;
+ right_adjacent = &right_extent;
+ }
+ if (err != EXT2_ET_EXTENT_NO_NEXT &&
+ goal_distance > (range_end - right_extent.e_lblk)) {
+ goal = right_extent.e_pblk -
+ (right_extent.e_lblk - range_start);
+ goal_distance = range_end - right_extent.e_lblk;
+ }
+
+ dbg_printf("%s: ino=%d rstart=%llu rend=%llu\n", __func__, ino,
+ range_start, range_end);
+ err = 0;
+ if (range_start <= range_end) {
+ count = range_end - range_start + 1;
+ err = ext_falloc_helper(fs, flags, ino, inode, handle,
+ left_adjacent, right_adjacent,
+ range_start, count, goal);
+ if (err)
+ goto errout;
+ }
+
+ if (range_end == end)
+ break;
+
+ err = ext2fs_extent_goto(handle, right_extent.e_lblk);
+ if (err)
+ goto errout;
+ next = right_extent.e_lblk + right_extent.e_len;
+ left_extent = right_extent;
+ left_adjacent = &left_extent;
+ range_start = next;
+ goal = left_extent.e_pblk + (range_start - left_extent.e_lblk);
+ goal_distance = range_start - next;
+ } while (range_end < end);
+
+errout:
+ ext2fs_extent_free(handle);
+ return err;
+}
+
+/*
+ * Map physical blocks to a range of logical blocks within a file. The range
+ * of logical blocks are (start, start + len). If there are already extents,
+ * the mappings will try to extend the mappings; otherwise, it will try to map
+ * start as if logical block 0 points to goal. If goal is ~0ULL, then the goal
+ * is calculated based on the inode group.
+ *
+ * Flags:
+ * - EXT2_FALLOCATE_ZERO_BLOCKS: Zero the blocks that are allocated.
+ * - EXT2_FALLOCATE_FORCE_INIT: Create only initialized extents.
+ * - EXT2_FALLOCATE_FORCE_UNINIT: Create only uninitialized extents.
+ * - EXT2_FALLOCATE_INIT_BEYOND_EOF: Create extents beyond EOF.
+ *
+ * If neither FORCE_INIT nor FORCE_UNINIT are specified, this function will
+ * try to expand any extents it finds, zeroing blocks as necessary.
+ */
+errcode_t ext2fs_fallocate(ext2_filsys fs, int flags, ext2_ino_t ino,
+ struct ext2_inode *inode, blk64_t goal,
+ blk64_t start, blk64_t len)
+{
+ struct ext2_inode inode_buf;
+ blk64_t blk, x;
+ errcode_t err;
+
+ if (((flags & EXT2_FALLOCATE_FORCE_INIT) &&
+ (flags & EXT2_FALLOCATE_FORCE_UNINIT)) ||
+ (flags & ~EXT2_FALLOCATE_ALL_FLAGS))
+ return EXT2_ET_INVALID_ARGUMENT;
+
+ if (len > ext2fs_blocks_count(fs->super))
+ return EXT2_ET_BLOCK_ALLOC_FAIL;
+ else if (len == 0)
+ return 0;
+
+ /* Read inode structure if necessary */
+ if (!inode) {
+ err = ext2fs_read_inode(fs, ino, &inode_buf);
+ if (err)
+ return err;
+ inode = &inode_buf;
+ }
+ dbg_printf("%s: ino=%d start=%llu len=%llu goal=%llu\n", __func__, ino,
+ start, len, goal);
+
+ if (inode->i_flags & EXT4_EXTENTS_FL) {
+ err = extent_fallocate(fs, flags, ino, inode, goal, start, len);
+ goto out;
+ }
+
+ /* XXX: Allocate a bunch of blocks the slow way */
+ for (blk = start; blk < start + len; blk++) {
+ err = ext2fs_bmap2(fs, ino, inode, NULL, 0, blk, 0, &x);
+ if (err)
+ return err;
+ if (x)
+ continue;
+
+ err = ext2fs_bmap2(fs, ino, inode, NULL,
+ BMAP_ALLOC | BMAP_UNINIT | BMAP_ZERO, blk,
+ 0, &x);
+ if (err)
+ return err;
+ }
+
+out:
+ if (inode == &inode_buf)
+ ext2fs_write_inode(fs, ino, inode);
+ return err;
+}
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 41/47] libext2fs: use fallocate for creating journals and hugefiles
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (39 preceding siblings ...)
2014-11-07 21:55 ` [PATCH 40/47] libext2fs: implement fallocate Darrick J. Wong
@ 2014-11-07 21:55 ` Darrick J. Wong
2014-11-07 21:55 ` [PATCH 42/47] debugfs: implement fallocate Darrick J. Wong
` (7 subsequent siblings)
48 siblings, 0 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:55 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Use the new fallocate API for creating the journal and the mk_hugefile
feature.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
lib/ext2fs/mkjournal.c | 134 +++++----------------------------
misc/mk_hugefiles.c | 96 ++----------------------
tests/f_opt_extent/expect | 15 ----
tests/r_32to64bit_meta/expect | 4 -
tests/r_32to64bit_move_itable/expect | 4 -
tests/r_64to32bit/expect | 4 -
tests/r_64to32bit_meta/expect | 4 -
tests/t_disable_mcsum_noinitbg/expect | 6 +
tests/t_enable_mcsum/expect | 15 ----
tests/t_enable_mcsum_ext3/expect | 8 +-
tests/t_enable_mcsum_initbg/expect | 11 +--
11 files changed, 54 insertions(+), 247 deletions(-)
diff --git a/lib/ext2fs/mkjournal.c b/lib/ext2fs/mkjournal.c
index 2cd7ca5..9378314 100644
--- a/lib/ext2fs/mkjournal.c
+++ b/lib/ext2fs/mkjournal.c
@@ -250,89 +250,6 @@ errcode_t ext2fs_zero_blocks(ext2_filsys fs, blk_t blk, int num,
}
/*
- * Helper function for creating the journal using direct I/O routines
- */
-struct mkjournal_struct {
- int num_blocks;
- int newblocks;
- blk64_t goal;
- blk64_t blk_to_zero;
- int zero_count;
- int flags;
- char *buf;
- errcode_t err;
-};
-
-static int mkjournal_proc(ext2_filsys fs,
- blk64_t *blocknr,
- e2_blkcnt_t blockcnt,
- blk64_t ref_block EXT2FS_ATTR((unused)),
- int ref_offset EXT2FS_ATTR((unused)),
- void *priv_data)
-{
- struct mkjournal_struct *es = (struct mkjournal_struct *) priv_data;
- blk64_t new_blk;
- errcode_t retval;
-
- if (*blocknr) {
- es->goal = *blocknr;
- return 0;
- }
- if (blockcnt &&
- (EXT2FS_B2C(fs, es->goal) == EXT2FS_B2C(fs, es->goal+1)))
- new_blk = es->goal+1;
- else {
- es->goal &= ~EXT2FS_CLUSTER_MASK(fs);
- retval = ext2fs_new_block2(fs, es->goal, 0, &new_blk);
- if (retval) {
- es->err = retval;
- return BLOCK_ABORT;
- }
- ext2fs_block_alloc_stats2(fs, new_blk, +1);
- es->newblocks++;
- }
- if (blockcnt >= 0)
- es->num_blocks--;
-
- retval = 0;
- if (blockcnt <= 0)
- retval = io_channel_write_blk64(fs->io, new_blk, 1, es->buf);
- else if (!(es->flags & EXT2_MKJOURNAL_LAZYINIT)) {
- if (es->zero_count) {
- if ((es->blk_to_zero + es->zero_count == new_blk) &&
- (es->zero_count < 1024))
- es->zero_count++;
- else {
- retval = ext2fs_zero_blocks2(fs,
- es->blk_to_zero,
- es->zero_count,
- 0, 0);
- es->zero_count = 0;
- }
- }
- if (es->zero_count == 0) {
- es->blk_to_zero = new_blk;
- es->zero_count = 1;
- }
- }
-
- if (blockcnt == 0)
- memset(es->buf, 0, fs->blocksize);
-
- if (retval) {
- es->err = retval;
- return BLOCK_ABORT;
- }
- *blocknr = es->goal = new_blk;
-
- if (es->num_blocks == 0)
- return (BLOCK_CHANGED | BLOCK_ABORT);
- else
- return BLOCK_CHANGED;
-
-}
-
-/*
* Calculate the initial goal block to be roughly at the middle of the
* filesystem. Pick a group that has the largest number of free
* blocks.
@@ -373,7 +290,8 @@ static errcode_t write_journal_inode(ext2_filsys fs, ext2_ino_t journal_ino,
errcode_t retval;
struct ext2_inode inode;
unsigned long long inode_size;
- struct mkjournal_struct es;
+ int falloc_flags = EXT2_FALLOCATE_FORCE_INIT;
+ blk64_t zblk;
if ((retval = ext2fs_create_journal_superblock(fs, num_blocks, flags,
&buf)))
@@ -390,40 +308,16 @@ static errcode_t write_journal_inode(ext2_filsys fs, ext2_ino_t journal_ino,
goto out2;
}
- es.num_blocks = num_blocks;
- es.newblocks = 0;
- es.buf = buf;
- es.err = 0;
- es.flags = flags;
- es.zero_count = 0;
- es.goal = (goal != ~0ULL) ? goal : get_midpoint_journal_block(fs);
+ if (goal == ~0ULL)
+ goal = get_midpoint_journal_block(fs);
- if (fs->super->s_feature_incompat & EXT3_FEATURE_INCOMPAT_EXTENTS) {
+ if (fs->super->s_feature_incompat & EXT3_FEATURE_INCOMPAT_EXTENTS)
inode.i_flags |= EXT4_EXTENTS_FL;
- if ((retval = ext2fs_write_inode(fs, journal_ino, &inode)))
- goto out2;
- }
- retval = ext2fs_block_iterate3(fs, journal_ino, BLOCK_FLAG_APPEND,
- 0, mkjournal_proc, &es);
- if (retval)
- goto out2;
- if (es.err) {
- retval = es.err;
- goto out2;
- }
- if (es.zero_count) {
- retval = ext2fs_zero_blocks2(fs, es.blk_to_zero,
- es.zero_count, 0, 0);
- if (retval)
- goto out2;
- }
-
- if ((retval = ext2fs_read_inode(fs, journal_ino, &inode)))
- goto out2;
+ if (!(flags & EXT2_MKJOURNAL_LAZYINIT))
+ falloc_flags |= EXT2_FALLOCATE_ZERO_BLOCKS;
inode_size = (unsigned long long)fs->blocksize * num_blocks;
- ext2fs_iblk_add_blocks(fs, &inode, es.newblocks);
inode.i_mtime = inode.i_ctime = fs->now ? fs->now : time(0);
inode.i_links_count = 1;
inode.i_mode = LINUX_S_IFREG | 0600;
@@ -431,9 +325,21 @@ static errcode_t write_journal_inode(ext2_filsys fs, ext2_ino_t journal_ino,
if (retval)
goto out2;
+ retval = ext2fs_fallocate(fs, falloc_flags, journal_ino,
+ &inode, goal, 0, num_blocks);
+ if (retval)
+ goto out2;
+
if ((retval = ext2fs_write_new_inode(fs, journal_ino, &inode)))
goto out2;
- retval = 0;
+
+ retval = ext2fs_bmap2(fs, journal_ino, &inode, NULL, 0, 0, NULL, &zblk);
+ if (retval)
+ goto out2;
+
+ retval = io_channel_write_blk64(fs->io, zblk, 1, buf);
+ if (retval)
+ goto out2;
memcpy(fs->super->s_jnl_blocks, inode.i_block, EXT2_N_BLOCKS*4);
fs->super->s_jnl_blocks[15] = inode.i_size_high;
diff --git a/misc/mk_hugefiles.c b/misc/mk_hugefiles.c
index 9097b7c..0a0254d 100644
--- a/misc/mk_hugefiles.c
+++ b/misc/mk_hugefiles.c
@@ -258,12 +258,7 @@ static errcode_t mk_hugefile(ext2_filsys fs, blk64_t num,
{
errcode_t retval;
- blk64_t lblk, bend = 0;
- __u64 size;
- blk64_t left;
- blk64_t count = 0;
struct ext2_inode inode;
- ext2_extent_handle_t handle;
retval = ext2fs_new_inode(fs, 0, LINUX_S_IFREG, NULL, ino);
if (retval)
@@ -283,85 +278,20 @@ static errcode_t mk_hugefile(ext2_filsys fs, blk64_t num,
ext2fs_inode_alloc_stats2(fs, *ino, +1, 0);
- retval = ext2fs_extent_open2(fs, *ino, &inode, &handle);
+ if (EXT2_HAS_INCOMPAT_FEATURE(fs->super,
+ EXT3_FEATURE_INCOMPAT_EXTENTS))
+ inode.i_flags |= EXT4_EXTENTS_FL;
+ retval = ext2fs_fallocate(fs,
+ EXT2_FALLOCATE_FORCE_INIT |
+ EXT2_FALLOCATE_ZERO_BLOCKS,
+ *ino, &inode, ~0ULL, 0, num);
if (retval)
return retval;
-
- lblk = 0;
- left = num ? num : 1;
- while (left) {
- blk64_t pblk, end;
- blk64_t n = left;
-
- retval = ext2fs_find_first_zero_block_bitmap2(fs->block_map,
- goal, ext2fs_blocks_count(fs->super) - 1, &end);
- if (retval)
- goto errout;
- goal = end;
-
- retval = ext2fs_find_first_set_block_bitmap2(fs->block_map, goal,
- ext2fs_blocks_count(fs->super) - 1, &bend);
- if (retval == ENOENT) {
- bend = ext2fs_blocks_count(fs->super);
- if (num == 0)
- left = 0;
- }
- if (!num || bend - goal < left)
- n = bend - goal;
- pblk = goal;
- if (num)
- left -= n;
- goal += n;
- count += n;
- ext2fs_block_alloc_stats_range(fs, pblk, n, +1);
-
- if (zero_hugefile) {
- blk64_t ret_blk;
- retval = ext2fs_zero_blocks2(fs, pblk, n,
- &ret_blk, NULL);
-
- if (retval)
- com_err(program_name, retval,
- _("while zeroing block %llu "
- "for hugefile"), ret_blk);
- }
-
- while (n) {
- blk64_t l = n;
- struct ext2fs_extent newextent;
-
- if (l > EXT_INIT_MAX_LEN)
- l = EXT_INIT_MAX_LEN;
-
- newextent.e_len = l;
- newextent.e_pblk = pblk;
- newextent.e_lblk = lblk;
- newextent.e_flags = 0;
-
- retval = ext2fs_extent_insert(handle,
- EXT2_EXTENT_INSERT_AFTER, &newextent);
- if (retval)
- return retval;
- pblk += l;
- lblk += l;
- n -= l;
- }
- }
-
- retval = ext2fs_read_inode(fs, *ino, &inode);
- if (retval)
- goto errout;
-
- retval = ext2fs_iblk_add_blocks(fs, &inode,
- count / EXT2FS_CLUSTER_RATIO(fs));
- if (retval)
- goto errout;
- size = (__u64) count * fs->blocksize;
- retval = ext2fs_inode_size_set(fs, &inode, size);
+ retval = ext2fs_inode_size_set(fs, &inode, num * fs->blocksize);
if (retval)
- goto errout;
+ return retval;
- retval = ext2fs_write_new_inode(fs, *ino, &inode);
+ retval = ext2fs_write_inode(fs, *ino, &inode);
if (retval)
goto errout;
@@ -379,13 +309,7 @@ retry:
goto retry;
}
- if (retval)
- goto errout;
-
errout:
- if (handle)
- ext2fs_extent_free(handle);
-
return retval;
}
diff --git a/tests/f_opt_extent/expect b/tests/f_opt_extent/expect
index 6d4863b..f4ed7ff 100644
--- a/tests/f_opt_extent/expect
+++ b/tests/f_opt_extent/expect
@@ -30,22 +30,11 @@ Change in FS metadata:
Inode count: 65536
Block count: 524288
Reserved block count: 26214
--Free blocks: 570
-+Free blocks: 567
+-Free blocks: 569
++Free blocks: 566
Free inodes: 65047
First block: 1
Block size: 1024
-@@ -47,8 +47,8 @@
- Block bitmap at 262 (+261)
- Inode bitmap at 278 (+277)
- Inode table at 294-549 (+293)
-- 21 free blocks, 535 free inodes, 3 directories, 535 unused inodes
-- Free blocks: 4414-4434
-+ 18 free blocks, 535 free inodes, 3 directories, 535 unused inodes
-+ Free blocks: 4417-4434
- Free inodes: 490-1024
- Group 1: (Blocks 8193-16384) [INODE_UNINIT]
- Backup superblock at 8193, Group descriptors at 8194-8197
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
diff --git a/tests/r_32to64bit_meta/expect b/tests/r_32to64bit_meta/expect
index 0eacd45..8796503 100644
--- a/tests/r_32to64bit_meta/expect
+++ b/tests/r_32to64bit_meta/expect
@@ -35,8 +35,8 @@ Change in FS metadata:
Inode count: 65536
Block count: 524288
Reserved block count: 26214
--Free blocks: 858
-+Free blocks: 852
+-Free blocks: 857
++Free blocks: 851
Free inodes: 65046
First block: 1
Block size: 1024
diff --git a/tests/r_32to64bit_move_itable/expect b/tests/r_32to64bit_move_itable/expect
index b51663d..999bb8d 100644
--- a/tests/r_32to64bit_move_itable/expect
+++ b/tests/r_32to64bit_move_itable/expect
@@ -35,8 +35,8 @@ Change in FS metadata:
Inode count: 98304
Block count: 786432
Reserved block count: 39321
--Free blocks: 764
-+Free blocks: 734
+-Free blocks: 763
++Free blocks: 733
Free inodes: 97566
First block: 1
Block size: 1024
diff --git a/tests/r_64to32bit/expect b/tests/r_64to32bit/expect
index 13e94a2..5d2ea4b 100644
--- a/tests/r_64to32bit/expect
+++ b/tests/r_64to32bit/expect
@@ -35,8 +35,8 @@ Change in FS metadata:
Inode count: 65536
Block count: 524288
Reserved block count: 26214
--Free blocks: 571
-+Free blocks: 589
+-Free blocks: 570
++Free blocks: 588
Free inodes: 65048
First block: 1
Block size: 1024
diff --git a/tests/r_64to32bit_meta/expect b/tests/r_64to32bit_meta/expect
index d6e2dcc..1400c6b 100644
--- a/tests/r_64to32bit_meta/expect
+++ b/tests/r_64to32bit_meta/expect
@@ -35,8 +35,8 @@ Change in FS metadata:
Inode count: 65536
Block count: 524288
Reserved block count: 26214
--Free blocks: 852
-+Free blocks: 858
+-Free blocks: 851
++Free blocks: 857
Free inodes: 65046
First block: 1
Block size: 1024
diff --git a/tests/t_disable_mcsum_noinitbg/expect b/tests/t_disable_mcsum_noinitbg/expect
index a022631..09e4ff1 100644
--- a/tests/t_disable_mcsum_noinitbg/expect
+++ b/tests/t_disable_mcsum_noinitbg/expect
@@ -40,9 +40,9 @@ Change in FS metadata:
Block bitmap at 262 (+261)
Inode bitmap at 278 (+277)
Inode table at 294-549 (+293)
-- 21 free blocks, 536 free inodes, 2 directories, 536 unused inodes
-+ 21 free blocks, 536 free inodes, 2 directories
- Free blocks: 4413-4433
+- 0 free blocks, 536 free inodes, 2 directories, 536 unused inodes
++ 0 free blocks, 536 free inodes, 2 directories
+ Free blocks:
Free inodes: 489-1024
-Group 1: (Blocks 8193-16384) [INODE_UNINIT]
+Group 1: (Blocks 8193-16384)
diff --git a/tests/t_enable_mcsum/expect b/tests/t_enable_mcsum/expect
index 2ee3c27..81e1125 100644
--- a/tests/t_enable_mcsum/expect
+++ b/tests/t_enable_mcsum/expect
@@ -45,8 +45,8 @@ Change in FS metadata:
Inode count: 65536
Block count: 524288
Reserved block count: 26214
--Free blocks: 571
-+Free blocks: 568
+-Free blocks: 570
++Free blocks: 567
Free inodes: 65048
First block: 1
Block size: 1024
@@ -58,17 +58,6 @@ Change in FS metadata:
Journal features: (none)
Journal size: 16M
Journal length: 16384
-@@ -46,8 +47,8 @@
- Block bitmap at 262 (+261)
- Inode bitmap at 278 (+277)
- Inode table at 294-549 (+293)
-- 21 free blocks, 536 free inodes, 2 directories, 536 unused inodes
-- Free blocks: 4413-4433
-+ 18 free blocks, 536 free inodes, 2 directories, 536 unused inodes
-+ Free blocks: 4413, 4417-4433
- Free inodes: 489-1024
- Group 1: (Blocks 8193-16384) [INODE_UNINIT]
- Backup superblock at 8193, Group descriptors at 8194-8197
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
diff --git a/tests/t_enable_mcsum_ext3/expect b/tests/t_enable_mcsum_ext3/expect
index 5460482..0f761a9 100644
--- a/tests/t_enable_mcsum_ext3/expect
+++ b/tests/t_enable_mcsum_ext3/expect
@@ -49,8 +49,8 @@ Change in FS metadata:
Reserved GDT blocks at 4-259
Block bitmap at 260 (+259)
@@ -45,7 +46,7 @@
- 7789 free blocks, 1013 free inodes, 2 directories
- Free blocks: 404-8192
+ 0 free blocks, 1013 free inodes, 2 directories
+ Free blocks:
Free inodes: 12-1024
-Group 1: (Blocks 8193-16384)
+Group 1: (Blocks 8193-16384) [ITABLE_ZEROED]
@@ -58,8 +58,8 @@ Change in FS metadata:
Reserved GDT blocks at 8196-8451
Block bitmap at 8452 (+259)
@@ -54,6 +55,6 @@
- 7803 free blocks, 1024 free inodes, 0 directories
- Free blocks: 8582-16384
+ 0 free blocks, 1024 free inodes, 0 directories
+ Free blocks:
Free inodes: 1025-2048
-Group 2: (Blocks 16385-24576)
+Group 2: (Blocks 16385-24576) [ITABLE_ZEROED]
diff --git a/tests/t_enable_mcsum_initbg/expect b/tests/t_enable_mcsum_initbg/expect
index d3b4444..3cbb98f 100644
--- a/tests/t_enable_mcsum_initbg/expect
+++ b/tests/t_enable_mcsum_initbg/expect
@@ -45,8 +45,8 @@ Change in FS metadata:
Inode count: 65536
Block count: 524288
Reserved block count: 26214
--Free blocks: 571
-+Free blocks: 568
+-Free blocks: 570
++Free blocks: 567
Free inodes: 65048
First block: 1
Block size: 1024
@@ -69,10 +69,9 @@ Change in FS metadata:
Block bitmap at 262 (+261)
Inode bitmap at 278 (+277)
Inode table at 294-549 (+293)
-- 21 free blocks, 536 free inodes, 2 directories
-- Free blocks: 4413-4433
-+ 18 free blocks, 536 free inodes, 2 directories, 536 unused inodes
-+ Free blocks: 4413, 4417-4433
+- 0 free blocks, 536 free inodes, 2 directories
++ 0 free blocks, 536 free inodes, 2 directories, 536 unused inodes
+ Free blocks:
Free inodes: 489-1024
-Group 1: (Blocks 8193-16384)
+Group 1: (Blocks 8193-16384) [INODE_UNINIT, ITABLE_ZEROED]
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 42/47] debugfs: implement fallocate
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (40 preceding siblings ...)
2014-11-07 21:55 ` [PATCH 41/47] libext2fs: use fallocate for creating journals and hugefiles Darrick J. Wong
@ 2014-11-07 21:55 ` Darrick J. Wong
2014-11-07 21:55 ` [PATCH 43/47] tests: test debugfs punch command Darrick J. Wong
` (6 subsequent siblings)
48 siblings, 0 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:55 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Implement a fallocate function for debugfs, and add some tests to
demonstrate that it works (more or less).
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
debugfs/debug_cmds.ct | 3 +
debugfs/debugfs.c | 36 +++++++
debugfs/debugfs.h | 1
tests/d_fallocate/expect.gz | Bin
tests/d_fallocate/name | 1
tests/d_fallocate/script | 175 ++++++++++++++++++++++++++++++++++
tests/d_fallocate_bigalloc/expect.gz | Bin
tests/d_fallocate_bigalloc/name | 1
tests/d_fallocate_bigalloc/script | 176 ++++++++++++++++++++++++++++++++++
tests/d_fallocate_blkmap/expect | 58 +++++++++++
tests/d_fallocate_blkmap/name | 1
tests/d_fallocate_blkmap/script | 85 ++++++++++++++++
12 files changed, 537 insertions(+)
create mode 100644 tests/d_fallocate/expect.gz
create mode 100644 tests/d_fallocate/name
create mode 100644 tests/d_fallocate/script
create mode 100644 tests/d_fallocate_bigalloc/expect.gz
create mode 100644 tests/d_fallocate_bigalloc/name
create mode 100644 tests/d_fallocate_bigalloc/script
create mode 100644 tests/d_fallocate_blkmap/expect
create mode 100644 tests/d_fallocate_blkmap/name
create mode 100644 tests/d_fallocate_blkmap/script
diff --git a/debugfs/debug_cmds.ct b/debugfs/debug_cmds.ct
index c6f6d6c..34dad9e 100644
--- a/debugfs/debug_cmds.ct
+++ b/debugfs/debug_cmds.ct
@@ -157,6 +157,9 @@ request do_dirsearch, "Search a directory for a particular filename",
request do_bmap, "Calculate the logical->physical block mapping for an inode",
bmap;
+request do_fallocate, "Allocate uninitialized blocks to an inode",
+ fallocate;
+
request do_punch, "Punch (or truncate) blocks from an inode by deallocating them",
punch, truncate;
diff --git a/debugfs/debugfs.c b/debugfs/debugfs.c
index fe57366..6bcc044 100644
--- a/debugfs/debugfs.c
+++ b/debugfs/debugfs.c
@@ -2109,6 +2109,42 @@ void do_punch(int argc, char *argv[])
return;
}
}
+
+void do_fallocate(int argc, char *argv[])
+{
+ ext2_ino_t ino;
+ blk64_t start, end;
+ int err;
+ errcode_t errcode;
+
+ if (common_args_process(argc, argv, 3, 4, argv[0],
+ "<file> start_blk [end_blk]",
+ CHECK_FS_RW | CHECK_FS_BITMAPS))
+ return;
+
+ ino = string_to_inode(argv[1]);
+ if (!ino)
+ return;
+ err = strtoblk(argv[0], argv[2], "logical block", &start);
+ if (err)
+ return;
+ if (argc == 4) {
+ err = strtoblk(argv[0], argv[3], "logical block", &end);
+ if (err)
+ return;
+ } else
+ end = ~0;
+
+ errcode = ext2fs_fallocate(current_fs, EXT2_FALLOCATE_INIT_BEYOND_EOF,
+ ino, NULL, ~0ULL, start, end - start + 1);
+
+ if (errcode) {
+ com_err(argv[0], errcode,
+ "while fallocating inode %u from %llu to %llu\n", ino,
+ (unsigned long long) start, (unsigned long long) end);
+ return;
+ }
+}
#endif /* READ_ONLY */
void do_symlink(int argc, char *argv[])
diff --git a/debugfs/debugfs.h b/debugfs/debugfs.h
index e163d0a..76bb22c 100644
--- a/debugfs/debugfs.h
+++ b/debugfs/debugfs.h
@@ -166,6 +166,7 @@ extern void do_imap(int argc, char **argv);
extern void do_set_current_time(int argc, char **argv);
extern void do_supported_features(int argc, char **argv);
extern void do_punch(int argc, char **argv);
+extern void do_fallocate(int argc, char **argv);
extern void do_symlink(int argc, char **argv);
extern void do_dump_mmp(int argc, char **argv);
diff --git a/tests/d_fallocate/expect.gz b/tests/d_fallocate/expect.gz
new file mode 100644
index 0000000000000000000000000000000000000000..3e6ffc38595b7c451e3de0df3a16f795cf33421b
GIT binary patch
literal 3770
zcmZ9Nc{~$-{KqA}=E^ZmjxiFcMP=1u5+X-hxpK>W6gKA^k(*pYteD9aQf!j@-ZF9@
zIa|z%<Svs4zxDn8{`ft9|GXaW_xt(29`DEJ@%Tt50odT_7N+b+X*TO=5_LlRIp3=+
zUYwp2x!ot`yF<Qymm`s%12x3vEwp8VZp|Eg<<)~-o3D>}6x37KoE6w$b-7eoz5PW1
zRlzvTCtl@Ca_ekW)!OBmnT6E_^6ti0RNcX3*ym3z4_+tK^0)YW4x6-X(}3NZ$?Yn;
zpW7c8^!D0oyV(e<b;fMYm3tHG>l)x3GR?cK)g#dQKH*A*28L(XgRF}W72n*d5h_i2
zZ9e<RJ+!35=U3AnvY+-bH}KeC?)G9uRcqsv*F=Bc;CI}<d=w+*;NFHzMDmsp7tF@i
zetoU(#QduFAt`U!TTRm8-R|%2DEDu{KlhuM%ss?&vdQ@kSnHz0&-`rSWEPjL&*eR4
zfZ-G_yMBM4N!t(4Q(fx0J<+sUyT86L(NDvT4f5@-PWVjhF}Fi#6n!VS*p{B8^qNL^
zv*)bWMxuGpGMU!4(UE`U#^Agv)gFc%tNV`Z53JjN2|o1p3vFxC2%{;vH$@F*+!#@?
zZQ1ein+V5NH0^1>y$bUTcpL5)v9~=G=Kb8RG-x^ESF`2eIHJj_#p<x}V5oWW^Xiq=
zJ4=fn*8=?>F=bkIe$#N;VNB1K7B8LOKgb_f9X^)1rFb413{FRo@;zIZ#KF_{?jy9b
zBb}Q*x|a?oy=!xu7{dbhq%|H{Xmp9(nn*bxTkCbB<Q^$SV+>h9MR$43`L9$(j&kDH
z&<}x?UNZ+vW)Dlz@(gyA%(g&#Ebx<%na`i1;W12@jrVioSD_zkpPIi)e+wf~4piyW
z_jVYTIbkb4kv}~UkFsBy-?cr&qEv@(aOQL}hDG9FVtfLcb+*~VaxZ$!N1Qb?-xT*a
z61)lW#%YIRI)o?#2A5J_Q@e>CBK;L3@hN9zs)BK%LZJ#`*aah-hmp0V0zEdI&jeqb
z>oNVoLCeI`6O|p_&!E8=-^Dj@fnZ-eN(%8j77@Jh<!tbeIYm5O-#$*^B?YKFyDE$1
zQyIu|#XEF0j3le{!SgRLoRuh<j}^+la0lYguENQwekEYHN%S>0>Q^m9M{3Zj?|S*N
z0HM=zJ<++_1e-8QWH1&3auoR0Di>FGE@==5IR)SeJZ6N*sA*5loz&PYHVPTs?gwF>
z+eYx(q~NBQTu9gP-e%VZK_m##@D-uIB0%`_V^Qe`huY_NX8+EBBDXIh#Ii6KO{tlY
z;43!&N5kS2f@BDMm<$8LeX3C`IEbWysa7(|5&gIPE#f!t=9dnWGS>PwXZF#N8cMzU
zo86i1ti_>7`b7lyO8qxg0>^(4ENapNT&c&C7*@6k3X|ya8V0+XQftOvbAxKe+AHKR
zmiO<StBH)-kyPtRwDnTjU!|v+tmI>=b8e$<B4`LHL)Y$%3m#hsrl+9J+~%LmLl_{I
zVi;(I1$ub|lalMa6qM&YvT;6ZteyI<ML8;edulCYn@#NMI%uE?thU~c?&8l2;w|!_
z4yd7286ibay9~GB0lP#de3X%j7l`jKd{9cwOqLMWEZyC7O_`gyX`Om7ye2oV4Yx~;
zi|Z|5DniQKd5x`Yj?JBx%02tlF~Y#jgI2NCyXor>H;AvNZAPcKWx=DDe`TL1Npjy*
zqR;!D-r?a1IeCfYQriRSOr8*fgmI`^s>j?dii5I@b!UT=TL?ZWQy$NKHxikTp3_m4
zB+I`&lOh^|->N^61f)NVE&XcriuxWYlAlniF&=5;fGCt&eC#JOD1M&sw2P6<i1{fL
z&t#13-W^X`M=QEgF$FNY@uVy5&nIWu++I#ZR@*hRi8`r9TS#9UJ%M8i2OU-hklKz8
zh%P*5&XdAr=1~hpB^1MP!*6Is5Czi;v#Sy{K*9q`KX^qvvj_7q&{d+ca|w>nuYxAW
zyPQ7M*Mw7q;@>(K88Vn5GHd7ymx)rO#C$uhaWP!w&&JX3Ycdo}C94b0--=c%zncm#
z5;;Q;X~7=3EqiQPp@hk>Q<iJDd?KLS!NoTkhz)JQaw;o%N2}(6t&5O@u8)_QT1x5y
zv~u(!xV`DI-9L8RC`;wfM?_nM`$n?~8hK3Nmg!o4kEXa^B7c@Xf?4p$Prhs75}n(M
zoHpZzdW?YVx6A8~&wexc6+(w=O`lOK0!<8-J{xKfl&@R1KE&+38(cF{3n+B@l^DNw
z>7go{5eJ1<^FX^Jk1llu)zlbi+NqB35D4;w002+a#Q;hx&hiZSUcYU`QM{ts?RIrI
zCfRUij4&|d8q>4*)E23qa80TpQC*d8Q62x88PwkJLyG@Xv|)F>)a}(JK5gClu#4nP
z+20a1qfLn);Q3CGwQXIVOHuc9Ex|_@GtokM)0tw%<FV%2M?4JxMYmTbNC#xbk_KNq
z4!A!Wb3ndx_)k8{zYC>%$gb-`I^f*J8SL|JDQE49ezONH+rNmgE|1{dpE20C{x+*L
z-WGe{RdN6}?=|~As|5}u9e}si7nz59zQP8ZDvOwxf*N}y?9noACmBE8p*z?k(xjSL
zYWGwcrx>LBJgB$_pdw<fH!_$gj5w~cmrgV;yy2-(ED$3Bh<SRnkVp#};qpF>eQK3H
zJ(?bt1{Vf6FpATiFtUP;$=7$=14!2ap<GKP>@q7yuFg6YT+iUjn|5KB;g5V~5P1SX
zfr1huuiq-UPHe+kLIqc#a&8=*qQVJpjsudHAzm(85c0Wmr(PG=hHYP#5#)-jf6tB<
zOsEn`h`;{FF%koy+~c8WL&8bd?~@eqLJ6&`kta){u7A>l@MThA=LD}<xyP*!09%^|
z66jhgg)#UUm0;+yE^q%vM{p+>F8PEKj6AbA(xW4iAaM>x#x0PP!6kGSCpNzxgw`|+
zY2{AG#w|HoHV>hUEkVk#Pjo!K#=n66u-cXtdf=xzIuaW<+8wy5MUJXgdpqP?5WD@3
zmmf%RU`2j2&esi*%Vb^j(Gu0!h3QMN<EVogtHvSmfqe4--Z%6=9`F1ME<p<jK?k2n
z!CT9CSnIW8Y5kPXgepJT#~0}H0bIyk4)K?U%RL)A6p!M`%*bclX$oQ7{G!AMy}vx2
z?6t^KV^+AJouUnk2Y`{j7%H$eChl#y0^Uet3Fr<_iRP#^iR_N$n0MtA?qVldxT`5(
z&w~KB2!W&Q+)k109%J6OAr4T%`p34`AiWd390M=^BX9s^kXtT!01309R-Pl|vjxdP
z;pNjvmW5_nxzWjOCJs}*fZjVLL}TCk6XV>y?G$mVGtlU05AA@?i>RyoA6cP>OEsnz
z|8R}QEyuXby%MhkW=EPl#FP{H@H3iEzLXO`#m1cS>x=@V#<)NPRZhN+Q3HYS)+X>x
zGbaZ+WulL;^NK{Ao6ng7+FxQ3kN;1^bN`p1{*oe~eKgQsC3L=`D=G$fty@R^cySvR
zLVgYLfa(c2{JyF=Vu3|<8a)HarTqK674*b;g%^qPecTzWicz1=n!lOw{lWzf{gawJ
zHkiA9QN`&H&W@-u56uxhY!vS(eoIYJPux}$uzjka;VeL$+fvY(FC!KiE2(tSc{oc8
z#KGGc@K+0QJeGpcgi2sF#6r+k!C8|C2`R_2T0Jw6Vc;7cRKz4hwEG-oBmh&UJYst<
z^h&ToV|o|LhQDiR5%@jlm|$)(&6GaQOF&#_)C=8DBZdx|cTQHW8CwXBB~&ztfH*GA
zXWQ}cMu~3*{F@O*D<MM4gxa=@Ef6s)%Sy4Fd!;drtSZ4%;XNg&F;4GWr-p2ft*aT~
zg4v*E*VsTEII{qO$M5CcMBO+oIqYjMp6!<1sm`+0WD#5Zac7~hsSmFqNprpCu77zT
z+8nE6j(yX|Q#~2v<-&4^ZlW%n@6EbWh+rB(TAeMIn%O4y;Si?SeY<;>@2d}_GQ6Ui
z_iNglVWrzV)c_$c7h|kWg8SLbD5%zrCb(j^?2Vc%eNB)|;hS53`RaTzBuQ}CT%OZ%
zWxVzxi`Ud7Wkkt}>hXNfa7|<S%N<78F9eoW7suT-CX+LArX0m}FYpT_dARnvpKyWM
zSn85b7ttNW`X3(QFM1po&k3W}v_*U|y5ho~|8&5g^TN#%l`vyf?bCS$4oiJ^hxzhF
z`URi3?%W#YlRn;0X+ATm;4z=LX`-W&ESqZz1FF=A<$p5o)K7B~L4JF-9@&nqH!XGw
zmCX3oBzEzMYh!JUpZdUyZgTM-VL3=Ys9#R93;5(YfBm>=$);8sFj+y-HWFKT{vIq7
zHTT}P`92j>GJmR2%IyO3!8{n=m3uBnA1+c<4@cr*B;^zwGs-#Q>xdcjKk)7Sd$l0E
zM87Dj?Gu7cGge2?Mk%}}1bswDMJRs1&gM@Bb3W9j5wuwkQe1^tDr<f%6w}@0#U<I7
zP4sFH-$FD*lU)stNmL}C6zRVqDOu~gY54c|E+k!=T>78`ZLw>V&5cP)Kju#pB_$^r
zgtQdPbyOPN<Y?FDcDr~?M@qgc9G>5&0}X%WE<b-q?xwIho8;gZSy0ZIoJ6=tQ-HoV
zEu2rfMA?yEUidsgd)UhD?g<^Jjgq57KKc>KXv=E@A(CVPUw+TjQ4y(A>I9jeaqJ1v
zBuNB=_tfSLbfA{X<&#E@{(-tehQGi$H2I2_0M_j#_cTP>YRsYFkl@7t|7*AGejO%N
zy$NfuuiEobNcpz#GkoGv=90I-n6%kXN=OmmWm}ccfn!UpeMD18!|&Cqe%$uxxnQ8@
zY|Ii}ow74sH@A>zvA+S5FVpeaq@1XYUXmEKBAoovg9|!n*z>SE*!iyWb2q>0@L;}l
y0k>{<MPq+8UfTX(XFh&w5ttwuF<UycB^qw5mO16~r}$&Uk<ipHmv5P~vHcHq|7W-W
literal 0
HcmV?d00001
diff --git a/tests/d_fallocate/name b/tests/d_fallocate/name
new file mode 100644
index 0000000..72d0ed3
--- /dev/null
+++ b/tests/d_fallocate/name
@@ -0,0 +1 @@
+fallocate sparse files and big files
diff --git a/tests/d_fallocate/script b/tests/d_fallocate/script
new file mode 100644
index 0000000..ae8956e
--- /dev/null
+++ b/tests/d_fallocate/script
@@ -0,0 +1,175 @@
+if test -x $DEBUGFS_EXE; then
+
+FSCK_OPT=-fy
+OUT=$test_name.log
+if [ -f $test_dir/expect.gz ]; then
+ EXP=$test_name.tmp
+ gunzip < $test_dir/expect.gz > $EXP
+else
+ EXP=$test_dir/expect
+fi
+
+cp /dev/null $OUT
+
+cat > $TMPFILE.conf << ENDL
+[fs_types]
+ext4 = {
+ base_features = sparse_super,filetype,resize_inode,dir_index,ext_attr,^has_journal,extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,64bit
+ blocksize = 1024
+ inode_size = 256
+ inode_ratio = 16384
+}
+ENDL
+MKE2FS_CONFIG=$TMPFILE.conf $MKE2FS -F -o Linux -b 1024 -O ^bigalloc -T ext4 $TMPFILE 65536 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT 2>&1
+rm -rf $TMPFILE.conf
+
+$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
+rm -f $OUT.new
+
+echo "debugfs write files" >> $OUT
+make_file() {
+ name="$1"
+ start="$2"
+ flag="$3"
+
+ cat << ENDL
+write /dev/null $name
+sif /$name size 40960
+eo /$name
+set_bmap $flag 10 $((start + 10))
+set_bmap $flag 13 $((start + 13))
+set_bmap $flag 26 $((start + 26))
+set_bmap $flag 29 $((start + 29))
+ec
+sif /$name blocks 8
+setb $((start + 10))
+setb $((start + 13))
+setb $((start + 26))
+setb $((start + 29))
+ENDL
+}
+
+#Files we create:
+# a: fallocate a 40k file
+# b*: falloc sparse file starting at b*
+# c*: falloc spare file ending at c*
+# d: midcluster to midcluster, surrounding sparse
+# e: partial middle cluster alloc
+# f: one big file
+# g*: falloc sparse init file starting at g*
+# h*: falloc sparse init file ending at h*
+# i: midcluster to midcluster, surrounding sparse init
+# j: partial middle cluster alloc
+# k: one big init file
+base=5000
+cat > $TMPFILE.cmd << ENDL
+write /dev/null a
+sif /a size 40960
+fallocate /a 0 39
+ENDL
+echo "ex /a" >> $TMPFILE.cmd2
+
+make_file sample $base --uninit >> $TMPFILE.cmd
+echo "ex /sample" >> $TMPFILE.cmd2
+base=10000
+
+for i in 8 9 10 11 12 13 14 15; do
+ make_file b$i $(($base + (40 * ($i - 8)))) --uninit >> $TMPFILE.cmd
+ echo "fallocate /b$i $i 39" >> $TMPFILE.cmd
+ echo "ex /b$i" >> $TMPFILE.cmd2
+done
+
+for i in 24 25 26 27 28 29 30 31; do
+ make_file c$i $(($base + 320 + (40 * ($i - 24)))) --uninit >> $TMPFILE.cmd
+ echo "fallocate /c$i 0 $i" >> $TMPFILE.cmd
+ echo "ex /c$i" >> $TMPFILE.cmd2
+done
+
+make_file d $(($base + 640)) --uninit >> $TMPFILE.cmd
+echo "fallocate /d 4 35" >> $TMPFILE.cmd
+echo "ex /d" >> $TMPFILE.cmd2
+
+make_file e $(($base + 680)) --uninit >> $TMPFILE.cmd
+echo "fallocate /e 19 20" >> $TMPFILE.cmd
+echo "ex /e" >> $TMPFILE.cmd2
+
+cat >> $TMPFILE.cmd << ENDL
+write /dev/null f
+sif /f size 1024
+eo /f
+set_bmap --uninit 0 9000
+ec
+sif /f blocks 2
+setb 9000
+fallocate /f 0 8999
+ENDL
+echo "ex /f" >> $TMPFILE.cmd2
+
+# Now do it again, but with initialized blocks
+base=20000
+for i in 8 9 10 11 12 13 14 15; do
+ make_file g$i $(($base + (40 * ($i - 8)))) >> $TMPFILE.cmd
+ echo "fallocate /g$i $i 39" >> $TMPFILE.cmd
+ echo "ex /g$i" >> $TMPFILE.cmd2
+done
+
+for i in 24 25 26 27 28 29 30 31; do
+ make_file h$i $(($base + 320 + (40 * ($i - 24)))) >> $TMPFILE.cmd
+ echo "fallocate /h$i 0 $i" >> $TMPFILE.cmd
+ echo "ex /h$i" >> $TMPFILE.cmd2
+done
+
+make_file i $(($base + 640)) >> $TMPFILE.cmd
+echo "fallocate /i 4 35" >> $TMPFILE.cmd
+echo "ex /i" >> $TMPFILE.cmd2
+
+make_file j $(($base + 680)) >> $TMPFILE.cmd
+echo "fallocate /j 19 20" >> $TMPFILE.cmd
+echo "ex /j" >> $TMPFILE.cmd2
+
+cat >> $TMPFILE.cmd << ENDL
+write /dev/null k
+sif /k size 1024
+eo /k
+set_bmap 0 19000
+ec
+sif /k blocks 2
+setb 19000
+fallocate /k 0 8999
+sif /k size 9216000
+ENDL
+echo "ex /k" >> $TMPFILE.cmd2
+
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE > /dev/null 2>&1
+$DEBUGFS_EXE -f $TMPFILE.cmd2 $TMPFILE >> $OUT.new 2>&1
+sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
+rm -rf $OUT.new $TMPFILE.cmd $TMPFILE.cmd2
+
+$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
+rm -f $OUT.new
+
+rm -f $TMPFILE
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+ rm -f $test_name.tmp
+fi
+
+unset IMAGE FSCK_OPT OUT EXP
+
+else #if test -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
diff --git a/tests/d_fallocate_bigalloc/expect.gz b/tests/d_fallocate_bigalloc/expect.gz
new file mode 100644
index 0000000000000000000000000000000000000000..8640bc29dc71b09810f110d07d84d883b1109b88
GIT binary patch
literal 2673
zcmV-%3Xb(3iwFQXkyum$1KnF)Z``;QeGTSU5GV@l7Fk=o<fTLdMX}vYfB;!^fwTzv
zR7`BeR&CE1GviItUtftl8dDm5kloQs2MB0octq(O>GE7&&g$cOx2ZO_SK@G2ici(;
z`r%=FEk2j)!}G2b)n4om)x(2$-rSTsaZ~<QK5U;J%gtf+p)AGS;qbJ-y1XpE9`wbR
z>b|<TFL#^r;bObHy{u};*NdmSr^~+{PrKcT$8B@oP5tNN`cQ2*tJP1tdbIi)#OLav
z+<)62%17~~I@}4vtc;NN;!_R2-;4F;M(E6%8os?L_p8;9%~?J6cDH?g5{LDt`V?29
z{=9ni>Snts#l&azaaTF8JKz5b-akK;yWk3gyI5aeZ=W|9;&{j52&>hv>-}EHEAi7^
zd3}GpnEh+q^lNv{ef6LEeqC1zy53Z~^7^pdeXCX4JzqDH`A%xE)Ae?<ssF5gs}A3M
zKOOYjm7mL9xw$TdyJ!r@K@ZRI{`v87U7sz~@KEj#|H6%52`Mj~OzuC#clJX5QEaxG
zAL{WB)$Q~4dH?<Ex|7zIUSWTyt2bY(LwzIllecdkyliz-etN$Bycb_;rb~ae_W!O#
z`Bhx5SMQrAQoPw5c2zx;|GeMcR@dtXfAH(wx4k=hUv6&eXX@?4`gXq(*`>(pZ{=nE
z&-Kg14^5-2a97qGNm(fOozBER>X)QCbVuK>AD<q|b6kRMx=CC@-3re9k!#&YwwKW2
z=!jFmDc4^Cr$%t9#gXy2x~=>BBi9b&^bu!)v#!4d&h~H}sy-EGqe&FFU8<SNvgW96
z>m&E!S(-?>{>tOX(<CQo(!dq2zXq-t8SjLq`iW}#TaQz=H(r+}dp??MNR!6!44QN#
zO$?e$B25M~nM9flt}~yeNoMDN>l{xu^W$6aq#}8;o+p{5@gy@1TpCX@^E^p8!;@Tw
zC#gJ7Ql;@E6?oE#JZbQx6M4dOFL{P1S|W<q<L2NgpO4}c@|2UuVFIeZjN_>QPeme6
z1$eSEJgH@PBI$Xe_}R|!L<r-Q7{&=<oD#!0A&k>Z7?*k>o)|oFB2S3plo-beahww4
zI3bSHTpZ`$!;=gRoXo=Ui>%|-Rkmt+wg5gQ7J<KMd&B}qN8F3uYWjm%t446@h-2W`
z^=IIC1gDQU6AEID!zoM{QuT4%;yQvTOc-i<!nh`VHbF!vh-!@oK6*e95za7*ydVlQ
z%;}Au2;!b}g&9Vj&M<vJj!(LF{WWlUPLRMiH~k@qHY2zbLC6t;nW<?feDgK&`5a+~
zUju&g6MiQniORum5g$n$PnelhVMZLoP=3+$2T#QePi7gObmn=|GL0vVnMqR`PZ~3m
zrZk>3W+qKDJmm}Vq`?y>@`Ra5C(KN_>kpnxK9bRl@PwJEX(xFq7UHP@Pd1S!Of;OB
zVK6cbj?XY0PmBqpX(xHI%kadA|E$t@V#I$=%rF@7pLHTnh~qpH$5plvPl)537{?iL
zoQp)B5XU(o!^m8J@YEVC8sSMU!;?WAH;HlFAdZ{F48tIfn}iHQy8fOgqhv4scX+xv
zA4@u932%XE?V62b2~pf6<`)K0+$7`|?D~VH9QHXMWT{*xyD+KQg+X><60-|~pl>3w
zi#90v@jN|w)_>Wi&1{|-;L^-&p0(pm2Eor~&n=oqq_DP|TL>A-Ax#HRy9JxMSkSca
z4#j{Rz`4a{>3Z7ZVza|F?SLE&PWqm1;mNJOVpUO(ji+0jPxhJx26qzRiZL9wIR92P
z`CW4o%$<_PaHhp60?w@hVJ?wo441b!cSlXzcrik3rBObMBM!W<OM<}*jq+k!9D^6*
zauDlR#}rnpPH-}7aR$8P4KG@GoK|Bv+2V5Wk~`0c)i|Ax;i$zG;H7AIVd-%!$8g8z
zr2sFM9XY`981DGISkFtd?%d3+z>V_KJ}-XVxo)MKbHIxk<)wXIq+g}1Upe>M`}-^W
zczID2gRlK<alZGyIELZ+q04$<m^xwD^om7<ViASKqD}xdodC?Prvif8*|-yWO=p2d
zxt_)&;1>qpb&m^XLPBAQ!opG~=$f8WAe5Bq_QffL@3%)N3JG&ZbSy%N5xwmB^s?db
zx~G>M(94E;Y24vU)oM;JhF%=47eg<TsF#6WCR#6pyUC;V%Ar?2t=I3XhK>0ga`VrG
z&oRBS5TSfVVfk!MFblzKv|!c?)|T301@j?~!cyCeV7|y!w=z~RA0FvgmKzmJE+beo
z@h1u9>v$B_@n!_`6}+a66-+7!rV|Cz5KKo4hIk=4S}=xSJR=w_B$!DMENcKc1bZnc
z4hTk}U<e|zO_lVlU^xUU5(O(DSP?B)0l{pvU>0{{X9QEr2u4|=V7{_RVP$jX4I_9C
ziuN1?&q0ZvgWx$R+H(**2Ss}hg6E(a&mr|fg6UMj7=m%MU@#pNZ8`|1gAz>#!E{ix
z=^&U6iZ&gDctLZfgBKEP+H~Zng28kIrQ42Rd2d;TP_(76XxkAi@0Du_<yx94*XE}}
z!>p)Lv%)Q|Bea59Q3Gbh91)?=3T8zOn3cRmbc9wgD{9)T+$~cQuE$koswABEmTd?n
z;ijEfwgE0o`xN=KuPr5qWU7+~gk4Vs#OWJ7EVP1=P-7|kfY7?>50f)qc7BojWu=Z#
zcBZiGd@6UaMn@<(Q&@2B3(X4&olgpFPK7oQ+Dr+3n%^q}q4Qy(PeZq_W;gAy(5InW
z!GMgeJq3YShl0Tz1@-UKK-?7!<|wFq_XRV{2&QGCV7^G+v}uBAEXZgXE0~5k(lS;s
z4FfW|=r<ymhB*of@iT&Hn4_Qs-xn-jNH9(nOhYi8D42#|94#2kk>+T@U_eG!TL%Pl
z_XKkk><E|>4CW}<3(yy=SV%CNDp&!*ibTN*2xg-NgE`VRRxpN<7+vNb5Uepd43onn
zCg)TzhRF#wCG-We%Lv9QQ80#CVM-H>VOChh3dS%itfB>jSz%5T3}%IOv|unDJZd^r
zwvb?{rh{QRIMH-4Ob17s4u<LAXw$(k9c&W?gX!QRLNJxNo)GM1n?_$Sxr|^Y)pQt`
z4wGm)3`~cKHXR10!$g}71JhwtnqUT|!%&Q1()F|$=_T*$2v(g>E*;9%vmD9{WN$i@
z3&F$alMQ>5*_(moFdUn`8CVX(vDuq}<uJKTO4AK2hcU5fx`E{|c}$w_?glfzX;)+Q
zV4D8#WmaQyNhw=h2wPn!-0Cv9$%U}Vg~ClPliOPe+gm7cdkbNE3x(TTCO5VaHnvc>
zv87A<{W<nwsa|Z8*r6iB9V(q2SSol;TrIxcn_{h>BAPuh13R!3wym@^{QAzkPbqR|
z-ilyP%m2milHcyivi<eALyIrFZSza>?GC6vWa2x{GkWd6FK_<LhRpXDtGCtHzld+m
zugu3z@?D*GN%I2KT&vJQ_I?-rr{*KxzI*>heE8>wzrXoKeEji)_~mauzkB=c&CmYp
f9)!C;rDnRY1HYwqzH#!$)N=J-JwLQM>OTMg)z2kH
literal 0
HcmV?d00001
diff --git a/tests/d_fallocate_bigalloc/name b/tests/d_fallocate_bigalloc/name
new file mode 100644
index 0000000..915645c
--- /dev/null
+++ b/tests/d_fallocate_bigalloc/name
@@ -0,0 +1 @@
+fallocate sparse files and big files with bigalloc
diff --git a/tests/d_fallocate_bigalloc/script b/tests/d_fallocate_bigalloc/script
new file mode 100644
index 0000000..6b6bf97
--- /dev/null
+++ b/tests/d_fallocate_bigalloc/script
@@ -0,0 +1,176 @@
+if test -x $DEBUGFS_EXE; then
+
+FSCK_OPT=-fy
+OUT=$test_name.log
+if [ -f $test_dir/expect.gz ]; then
+ EXP=$test_name.tmp
+ gunzip < $test_dir/expect.gz > $EXP
+else
+ EXP=$test_dir/expect
+fi
+
+cp /dev/null $OUT
+
+cat > $TMPFILE.conf << ENDL
+[fs_types]
+ext4 = {
+ cluster_size = 8192
+ base_features = sparse_super,filetype,resize_inode,dir_index,ext_attr,^has_journal,extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,64bit
+ blocksize = 1024
+ inode_size = 256
+ inode_ratio = 16384
+}
+ENDL
+MKE2FS_CONFIG=$TMPFILE.conf $MKE2FS -F -o Linux -b 1024 -O bigalloc -T ext4 $TMPFILE 65536 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT 2>&1
+rm -rf $TMPFILE.conf
+
+$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
+rm -f $OUT.new
+
+echo "debugfs write files" >> $OUT
+make_file() {
+ name="$1"
+ start="$2"
+ flag="$3"
+
+ cat << ENDL
+write /dev/null $name
+sif /$name size 40960
+eo /$name
+set_bmap $flag 10 $((start + 10))
+set_bmap $flag 13 $((start + 13))
+set_bmap $flag 26 $((start + 26))
+set_bmap $flag 29 $((start + 29))
+ec
+sif /$name blocks 32
+setb $((start + 10))
+setb $((start + 13))
+setb $((start + 26))
+setb $((start + 29))
+ENDL
+}
+
+#Files we create:
+# a: fallocate a 40k file
+# b*: falloc sparse file starting at b*
+# c*: falloc spare file ending at c*
+# d: midcluster to midcluster, surrounding sparse
+# e: partial middle cluster alloc
+# f: one big file
+# g*: falloc sparse init file starting at g*
+# h*: falloc sparse init file ending at h*
+# i: midcluster to midcluster, surrounding sparse init
+# j: partial middle cluster alloc
+# k: one big init file
+base=5000
+cat > $TMPFILE.cmd << ENDL
+write /dev/null a
+sif /a size 40960
+fallocate /a 0 39
+ENDL
+echo "ex /a" >> $TMPFILE.cmd2
+
+make_file sample $base --uninit >> $TMPFILE.cmd
+echo "ex /sample" >> $TMPFILE.cmd2
+base=10000
+
+for i in 8 9 10 11 12 13 14 15; do
+ make_file b$i $(($base + (40 * ($i - 8)))) --uninit >> $TMPFILE.cmd
+ echo "fallocate /b$i $i 39" >> $TMPFILE.cmd
+ echo "ex /b$i" >> $TMPFILE.cmd2
+done
+
+for i in 24 25 26 27 28 29 30 31; do
+ make_file c$i $(($base + 320 + (40 * ($i - 24)))) --uninit >> $TMPFILE.cmd
+ echo "fallocate /c$i 0 $i" >> $TMPFILE.cmd
+ echo "ex /c$i" >> $TMPFILE.cmd2
+done
+
+make_file d $(($base + 640)) --uninit >> $TMPFILE.cmd
+echo "fallocate /d 4 35" >> $TMPFILE.cmd
+echo "ex /d" >> $TMPFILE.cmd2
+
+make_file e $(($base + 680)) --uninit >> $TMPFILE.cmd
+echo "fallocate /e 19 20" >> $TMPFILE.cmd
+echo "ex /e" >> $TMPFILE.cmd2
+
+cat >> $TMPFILE.cmd << ENDL
+write /dev/null f
+sif /f size 1024
+eo /f
+set_bmap --uninit 0 9000
+ec
+sif /f blocks 16
+setb 9000
+fallocate /f 0 8999
+ENDL
+echo "ex /f" >> $TMPFILE.cmd2
+
+# Now do it again, but with initialized blocks
+base=20000
+for i in 8 9 10 11 12 13 14 15; do
+ make_file g$i $(($base + (40 * ($i - 8)))) >> $TMPFILE.cmd
+ echo "fallocate /g$i $i 39" >> $TMPFILE.cmd
+ echo "ex /g$i" >> $TMPFILE.cmd2
+done
+
+for i in 24 25 26 27 28 29 30 31; do
+ make_file h$i $(($base + 320 + (40 * ($i - 24)))) >> $TMPFILE.cmd
+ echo "fallocate /h$i 0 $i" >> $TMPFILE.cmd
+ echo "ex /h$i" >> $TMPFILE.cmd2
+done
+
+make_file i $(($base + 640)) >> $TMPFILE.cmd
+echo "fallocate /i 4 35" >> $TMPFILE.cmd
+echo "ex /i" >> $TMPFILE.cmd2
+
+make_file j $(($base + 680)) >> $TMPFILE.cmd
+echo "fallocate /j 19 20" >> $TMPFILE.cmd
+echo "ex /j" >> $TMPFILE.cmd2
+
+cat >> $TMPFILE.cmd << ENDL
+write /dev/null k
+sif /k size 1024
+eo /k
+set_bmap 0 19000
+ec
+sif /k blocks 16
+setb 19000
+fallocate /k 0 8999
+sif /k size 9216000
+ENDL
+echo "ex /k" >> $TMPFILE.cmd2
+
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE > /dev/null 2>&1
+$DEBUGFS_EXE -f $TMPFILE.cmd2 $TMPFILE >> $OUT.new 2>&1
+sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
+rm -rf $OUT.new $TMPFILE.cmd $TMPFILE.cmd2
+
+$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
+rm -f $OUT.new
+
+rm -f $TMPFILE
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+ rm -f $test_name.tmp
+fi
+
+unset IMAGE FSCK_OPT OUT EXP
+
+else #if test -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
diff --git a/tests/d_fallocate_blkmap/expect b/tests/d_fallocate_blkmap/expect
new file mode 100644
index 0000000..f7ae606
--- /dev/null
+++ b/tests/d_fallocate_blkmap/expect
@@ -0,0 +1,58 @@
+Creating filesystem with 65536 1k blocks and 4096 inodes
+Superblock backups stored on blocks:
+ 8193, 24577, 40961, 57345
+
+Allocating group tables: \b\b\bdone
+Writing inode tables: \b\b\bdone
+Writing superblocks and filesystem accounting information: \b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+test_filesys: 11/4096 files (0.0% non-contiguous), 2340/65536 blocks
+Exit status is 0
+debugfs write files
+debugfs: stat /a
+Inode: 12 Type: regular Mode: 0666 Flags: 0x0
+Generation: 0 Version: 0x00000000:00000000
+User: 0 Group: 0 Size: 40960
+File ACL: 0 Directory ACL: 0
+Links: 1 Blockcount: 82
+Fragment: Address: 0 Number: 0 Size: 0
+Size of extra inode fields: 28
+BLOCKS:
+(0-1):1312-1313, (2-11):8000-8009, (IND):8010, (12-39):8011-8038
+TOTAL: 41
+
+debugfs: stat /b
+Inode: 13 Type: regular Mode: 0666 Flags: 0x0
+Generation: 0 Version: 0x00000000:00000000
+User: 0 Group: 0 Size: 10240000
+File ACL: 0 Directory ACL: 0
+Links: 1 Blockcount: 20082
+Fragment: Address: 0 Number: 0 Size: 0
+Size of extra inode fields: 28
+BLOCKS:
+(0-11):10000-10011, (IND):10012, (12-267):10013-10268, (DIND):10269, (IND):10270, (268-523):10271-10526, (IND):10527, (524-779):10528-10783, (IND):10784, (780-1035):10785-11040, (IND):11041, (1036-1291):11042-11297, (IND):11298, (1292-1547):11299-11554, (IND):11555, (1548-1803):11556-11811, (IND):11812, (1804-2059):11813-12068, (IND):12069, (2060-2315):12070-12325, (IND):12326, (2316-2571):12327-12582, (IND):12583, (2572-2827):12584-12839, (IND):12840, (2828-3083):12841-13096, (IND):13097, (3084-3339):13098-13353, (IND):13354, (3340-3595):13355-13610, (IND):13611, (3596-3851):13612-13867, (IND):13868, (3852-4107):13869-14124, (IND):14125, (4108-4363):14126-14381, (IND):14382, (4364-4619):14383-14638, (IND):14639, (4620-4875):14640-14895, (IND):14896, (4876-5131):14897-15152, (IND):15153,
(5132-5387):15154-15409, (IND):15410, (5388-5643):15411-15666, (IND):15667, (5644-5899):15668-15923, (IND):15924, (5900-6155):15925-16180, (IND):16181, (6156-6411):16182-16437, (IND):16438,!
(6412-6667):16439-16694, (IND):16695, (6668-6923):16696-16951, (IND):16952, (6924-7179):16953-17208, (IND):17209, (7180-7435):17210-17465, (IND):17466, (7436-7691):17467-17722, (IND):17723, (7692-7947):17724-17979, (IND):17980, (7948-8203):17981-18236, (IND):18237, (8204-8459):18238-18493, (IND):18494, (8460-8715):18495-18750, (IND):18751, (8716-8971):18752-19007, (IND):19008, (8972-9227):19009-19264, (IND):19265, (9228-9483):19266-19521, (IND):19522, (9484-9739):19523-19778, (IND):19779, (9740-9995):19780-20035, (IND):20036, (9996-9999):20037-20040
+TOTAL: 10041
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (6841, counted=6840).
+Fix? yes
+
+Free blocks count wrong for group #1 (1551, counted=1550).
+Fix? yes
+
+Free blocks count wrong (53116, counted=53114).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 13/4096 files (7.7% non-contiguous), 12422/65536 blocks
+Exit status is 1
diff --git a/tests/d_fallocate_blkmap/name b/tests/d_fallocate_blkmap/name
new file mode 100644
index 0000000..ba2b61d
--- /dev/null
+++ b/tests/d_fallocate_blkmap/name
@@ -0,0 +1 @@
+fallocate sparse files and big files on a blockmap fs
diff --git a/tests/d_fallocate_blkmap/script b/tests/d_fallocate_blkmap/script
new file mode 100644
index 0000000..9c48cbc
--- /dev/null
+++ b/tests/d_fallocate_blkmap/script
@@ -0,0 +1,85 @@
+if test -x $DEBUGFS_EXE; then
+
+FSCK_OPT=-fy
+OUT=$test_name.log
+if [ -f $test_dir/expect.gz ]; then
+ EXP=$test_name.tmp
+ gunzip < $test_dir/expect.gz > $EXP1
+else
+ EXP=$test_dir/expect
+fi
+
+cp /dev/null $OUT
+
+cat > $TMPFILE.conf << ENDL
+[fs_types]
+ext4 = {
+ base_features = sparse_super,filetype,resize_inode,dir_index,ext_attr,^has_journal,^extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,^64bit
+ blocksize = 1024
+ inode_size = 256
+ inode_ratio = 16384
+}
+ENDL
+MKE2FS_CONFIG=$TMPFILE.conf $MKE2FS -F -o Linux -b 1024 -O ^bigalloc -T ext4 $TMPFILE 65536 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT 2>&1
+rm -rf $TMPFILE.conf
+
+$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
+rm -f $OUT.new
+
+echo "debugfs write files" >> $OUT
+
+#Files we create:
+# a: fallocate a 40k file
+# k: one big file
+base=5000
+cat > $TMPFILE.cmd << ENDL
+write /dev/null a
+sif /a bmap[2] 8000
+sif /a size 40960
+sif /a i_blocks 2
+setb 8000
+fallocate /a 0 39
+
+write /dev/null b
+sif /b size 10240000
+sif /b bmap[0] 10000
+sif /b i_blocks 2
+setb 10000
+fallocate /b 0 9999
+ENDL
+echo "stat /a" >> $TMPFILE.cmd2
+echo "stat /b" >> $TMPFILE.cmd2
+
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE > /dev/null 2>&1
+$DEBUGFS_EXE -f $TMPFILE.cmd2 $TMPFILE >> $OUT.new 2>&1
+sed -f $cmd_dir/filter.sed -e '/^.*time:.*$/d' < $OUT.new >> $OUT
+rm -rf $OUT.new $TMPFILE.cmd $TMPFILE.cmd2
+
+$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
+rm -f $OUT.new
+
+rm -f $TMPFILE
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+ rm -f $test_name.tmp
+fi
+
+unset IMAGE FSCK_OPT OUT EXP
+
+else #if test -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 43/47] tests: test debugfs punch command
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (41 preceding siblings ...)
2014-11-07 21:55 ` [PATCH 42/47] debugfs: implement fallocate Darrick J. Wong
@ 2014-11-07 21:55 ` Darrick J. Wong
2014-11-07 21:55 ` [PATCH 45/47] fuse2fs: translate ACL structures Darrick J. Wong
` (5 subsequent siblings)
48 siblings, 0 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:55 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Test punching out various parts of sparse files.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
tests/d_punch/expect | 176 +++++++++++++++++++++++++++++++++++++++++
tests/d_punch/name | 1
tests/d_punch/script | 129 ++++++++++++++++++++++++++++++
tests/d_punch_bigalloc/expect | 175 +++++++++++++++++++++++++++++++++++++++++
tests/d_punch_bigalloc/name | 1
tests/d_punch_bigalloc/script | 130 ++++++++++++++++++++++++++++++
6 files changed, 612 insertions(+)
create mode 100644 tests/d_punch/expect
create mode 100644 tests/d_punch/name
create mode 100644 tests/d_punch/script
create mode 100644 tests/d_punch_bigalloc/expect
create mode 100644 tests/d_punch_bigalloc/name
create mode 100644 tests/d_punch_bigalloc/script
diff --git a/tests/d_punch/expect b/tests/d_punch/expect
new file mode 100644
index 0000000..7de0208
--- /dev/null
+++ b/tests/d_punch/expect
@@ -0,0 +1,176 @@
+Creating filesystem with 65536 1k blocks and 4096 inodes
+Superblock backups stored on blocks:
+ 8193, 24577, 40961, 57345
+
+Allocating group tables: \b\b\bdone
+Writing inode tables: \b\b\bdone
+Writing superblocks and filesystem accounting information: \b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+test_filesys: 11/4096 files (0.0% non-contiguous), 2345/65536 blocks
+Exit status is 0
+debugfs write files
+debugfs: ex /a
+Level Entries Logical Physical Length Flags
+debugfs: ex /sample
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1323 0
+ 1/ 1 1/ 5 0 - 9 1313 - 1322 10 Uninit
+ 1/ 1 2/ 5 11 - 12 1324 - 1325 2 Uninit
+ 1/ 1 3/ 5 14 - 25 1327 - 1338 12 Uninit
+ 1/ 1 4/ 5 27 - 28 1340 - 1341 2 Uninit
+ 1/ 1 5/ 5 30 - 39 1343 - 1352 10 Uninit
+debugfs: ex /b8
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1390 0
+ 1/ 1 1/ 4 0 - 0 1326 - 1326 1 Uninit
+ 1/ 1 2/ 4 1 - 1 1339 - 1339 1 Uninit
+ 1/ 1 3/ 4 2 - 2 1342 - 1342 1 Uninit
+ 1/ 1 4/ 4 3 - 7 1353 - 1357 5 Uninit
+debugfs: ex /b9
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1368 0
+ 1/ 1 1/ 1 0 - 8 1358 - 1366 9 Uninit
+debugfs: ex /b10
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1378 0
+ 1/ 1 1/ 2 0 - 0 1367 - 1367 1 Uninit
+ 1/ 1 2/ 2 1 - 9 1369 - 1377 9 Uninit
+debugfs: ex /b11
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1389 0
+ 1/ 1 1/ 1 0 - 9 1379 - 1388 10 Uninit
+debugfs: ex /b12
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1401 0
+ 1/ 1 1/ 2 0 - 9 1391 - 1400 10 Uninit
+ 1/ 1 2/ 2 11 - 11 1402 - 1402 1 Uninit
+debugfs: ex /b13
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1413 0
+ 1/ 1 1/ 2 0 - 9 1403 - 1412 10 Uninit
+ 1/ 1 2/ 2 11 - 12 1414 - 1415 2 Uninit
+debugfs: ex /b14
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1426 0
+ 1/ 1 1/ 2 0 - 9 1416 - 1425 10 Uninit
+ 1/ 1 2/ 2 11 - 12 1427 - 1428 2 Uninit
+debugfs: ex /b15
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1439 0
+ 1/ 1 1/ 3 0 - 9 1429 - 1438 10 Uninit
+ 1/ 1 2/ 3 11 - 12 1440 - 1441 2 Uninit
+ 1/ 1 3/ 3 14 - 14 1443 - 1443 1 Uninit
+debugfs: ex /c24
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 25 - 4294967295 1453 4294967271
+ 1/ 1 1/ 3 25 - 25 1468 - 1468 1 Uninit
+ 1/ 1 2/ 3 27 - 28 1470 - 1471 2 Uninit
+ 1/ 1 3/ 3 30 - 39 1473 - 1482 10 Uninit
+debugfs: ex /c25
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 27 - 4294967295 1483 4294967269
+ 1/ 1 1/ 2 27 - 28 1485 - 1486 2 Uninit
+ 1/ 1 2/ 2 30 - 39 1488 - 1497 10 Uninit
+debugfs: ex /c26
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 27 - 4294967295 1484 4294967269
+ 1/ 1 1/ 2 27 - 28 1498 - 1499 2 Uninit
+ 1/ 1 2/ 2 30 - 39 1501 - 1510 10 Uninit
+debugfs: ex /c27
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 28 - 4294967295 1487 4294967268
+ 1/ 1 1/ 2 28 - 28 1512 - 1512 1 Uninit
+ 1/ 1 2/ 2 30 - 39 1514 - 1523 10 Uninit
+debugfs: ex /c28
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 30 - 4294967295 1500 4294967266
+ 1/ 1 1/ 1 30 - 39 1526 - 1535 10 Uninit
+debugfs: ex /c29
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 30 - 4294967295 1511 4294967266
+ 1/ 1 1/ 1 30 - 39 1537 - 1546 10 Uninit
+debugfs: ex /c30
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 31 - 4294967295 1513 4294967265
+ 1/ 1 1/ 1 31 - 39 1549 - 1557 9 Uninit
+debugfs: ex /c31
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 32 - 4294967295 1524 4294967264
+ 1/ 1 1/ 1 32 - 39 1560 - 1567 8 Uninit
+debugfs: ex /d
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1525 0
+ 1/ 1 1/ 3 0 - 0 1442 - 1442 1 Uninit
+ 1/ 1 2/ 3 1 - 3 1444 - 1446 3 Uninit
+ 1/ 1 3/ 3 36 - 39 1573 - 1576 4 Uninit
+debugfs: ex /e
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1547 0
+ 1/ 1 1/ 11 0 - 5 1447 - 1452 6 Uninit
+ 1/ 1 2/ 11 6 - 9 1454 - 1457 4 Uninit
+ 1/ 1 3/ 11 11 - 12 1459 - 1460 2 Uninit
+ 1/ 1 4/ 11 14 - 18 1462 - 1466 5 Uninit
+ 1/ 1 5/ 11 21 - 21 1472 - 1472 1 Uninit
+ 1/ 1 6/ 11 22 - 22 1536 - 1536 1 Uninit
+ 1/ 1 7/ 11 23 - 23 1548 - 1548 1 Uninit
+ 1/ 1 8/ 11 24 - 25 1558 - 1559 2 Uninit
+ 1/ 1 9/ 11 27 - 28 1569 - 1570 2 Uninit
+ 1/ 1 10/ 11 30 - 30 1572 - 1572 1 Uninit
+ 1/ 1 11/ 11 31 - 39 1577 - 1585 9 Uninit
+debugfs: ex /f
+Level Entries Logical Physical Length Flags
+ 0/ 0 1/ 2 0 - 0 9000 - 9000 1 Uninit
+ 0/ 0 2/ 2 8999 - 8999 17999 - 17999 1 Uninit
+Pass 1: Checking inodes, blocks, and sizes
+Inode 15 extent tree could be more compact. Fix? yes
+
+Inode 16 extent tree could be more compact. Fix? yes
+
+Inode 17 extent tree could be more compact. Fix? yes
+
+Inode 18 extent tree could be more compact. Fix? yes
+
+Inode 19 extent tree could be more compact. Fix? yes
+
+Inode 20 extent tree could be more compact. Fix? yes
+
+Inode 21 extent tree could be more compact. Fix? yes
+
+Inode 22 extent tree could be more compact. Fix? yes
+
+Inode 23 extent tree could be more compact. Fix? yes
+
+Inode 24 extent tree could be more compact. Fix? yes
+
+Inode 25 extent tree could be more compact. Fix? yes
+
+Inode 26 extent tree could be more compact. Fix? yes
+
+Inode 27 extent tree could be more compact. Fix? yes
+
+Inode 28 extent tree could be more compact. Fix? yes
+
+Inode 29 extent tree could be more compact. Fix? yes
+
+Inode 30 extent tree could be more compact. Fix? yes
+
+Pass 1E: Optimizing extent trees
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #1 (7934, counted=7933).
+Fix? yes
+
+Free blocks count wrong (62939, counted=62938).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 32/4096 files (43.8% non-contiguous), 2598/65536 blocks
+Exit status is 1
diff --git a/tests/d_punch/name b/tests/d_punch/name
new file mode 100644
index 0000000..724639f
--- /dev/null
+++ b/tests/d_punch/name
@@ -0,0 +1 @@
+punch sparse files and big files
diff --git a/tests/d_punch/script b/tests/d_punch/script
new file mode 100644
index 0000000..7a77c69
--- /dev/null
+++ b/tests/d_punch/script
@@ -0,0 +1,129 @@
+if test -x $DEBUGFS_EXE; then
+
+FSCK_OPT=-fy
+OUT=$test_name.log
+if [ -f $test_dir/expect.gz ]; then
+ EXP=$test_name.tmp
+ gunzip < $test_dir/expect.gz > $EXP1
+else
+ EXP=$test_dir/expect
+fi
+
+cp /dev/null $OUT
+
+cat > $TMPFILE.conf << ENDL
+[fs_types]
+ext4 = {
+ base_features = sparse_super,filetype,resize_inode,dir_index,ext_attr,^has_journal,extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,64bit
+ blocksize = 1024
+ inode_size = 256
+ inode_ratio = 16384
+}
+ENDL
+MKE2FS_CONFIG=$TMPFILE.conf $MKE2FS -F -o Linux -b 1024 -O ^bigalloc -T ext4 $TMPFILE 65536 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT 2>&1
+rm -rf $TMPFILE.conf
+
+$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
+rm -f $OUT.new
+
+echo "debugfs write files" >> $OUT
+make_file() {
+ name="$1"
+ start="$2"
+ flag="$3"
+
+ cat << ENDL
+write /dev/null $name
+fallocate /$name 0 39
+punch /$name 10 10
+punch /$name 13 13
+punch /$name 26 26
+punch /$name 29 29
+ENDL
+}
+
+#Files we create:
+# a: punch a 40k file
+# b*: punch sparse file starting at b*
+# c*: punch spare file ending at c*
+# d: midcluster to midcluster, surrounding sparse
+# e: partial middle cluster alloc
+# f: one big file
+base=5000
+cat > $TMPFILE.cmd << ENDL
+write /dev/null a
+fallocate /a 0 39
+punch /a 0 39
+ENDL
+echo "ex /a" >> $TMPFILE.cmd2
+
+make_file sample $base --uninit >> $TMPFILE.cmd
+echo "ex /sample" >> $TMPFILE.cmd2
+base=10000
+
+for i in 8 9 10 11 12 13 14 15; do
+ make_file b$i $(($base + (40 * ($i - 8)))) --uninit >> $TMPFILE.cmd
+ echo "punch /b$i $i 39" >> $TMPFILE.cmd
+ echo "ex /b$i" >> $TMPFILE.cmd2
+done
+
+for i in 24 25 26 27 28 29 30 31; do
+ make_file c$i $(($base + 320 + (40 * ($i - 24)))) --uninit >> $TMPFILE.cmd
+ echo "punch /c$i 0 $i" >> $TMPFILE.cmd
+ echo "ex /c$i" >> $TMPFILE.cmd2
+done
+
+make_file d $(($base + 640)) --uninit >> $TMPFILE.cmd
+echo "punch /d 4 35" >> $TMPFILE.cmd
+echo "ex /d" >> $TMPFILE.cmd2
+
+make_file e $(($base + 680)) --uninit >> $TMPFILE.cmd
+echo "punch /e 19 20" >> $TMPFILE.cmd
+echo "ex /e" >> $TMPFILE.cmd2
+
+cat >> $TMPFILE.cmd << ENDL
+write /dev/null f
+sif /f size 1024
+eo /f
+set_bmap --uninit 0 9000
+ec
+sif /f blocks 2
+setb 9000
+fallocate /f 0 8999
+punch /f 1 8998
+ENDL
+echo "ex /f" >> $TMPFILE.cmd2
+
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE > /dev/null 2>&1
+$DEBUGFS_EXE -f $TMPFILE.cmd2 $TMPFILE >> $OUT.new 2>&1
+sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
+rm -rf $OUT.new $TMPFILE.cmd $TMPFILE.cmd2
+
+$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
+rm -f $OUT.new
+
+rm -f $TMPFILE
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+ rm -f $test_name.tmp
+fi
+
+unset IMAGE FSCK_OPT OUT EXP
+
+else #if test -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
diff --git a/tests/d_punch_bigalloc/expect b/tests/d_punch_bigalloc/expect
new file mode 100644
index 0000000..977d5c8
--- /dev/null
+++ b/tests/d_punch_bigalloc/expect
@@ -0,0 +1,175 @@
+
+Warning: the bigalloc feature is still under development
+See https://ext4.wiki.kernel.org/index.php/Bigalloc for more information
+
+Creating filesystem with 65536 1k blocks and 4096 inodes
+
+Allocating group tables: \b\b\bdone
+Writing inode tables: \b\b\bdone
+Writing superblocks and filesystem accounting information: \b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+test_filesys: 11/4096 files (9.1% non-contiguous), 1144/65536 blocks
+Exit status is 0
+debugfs write files
+debugfs: ex /a
+Level Entries Logical Physical Length Flags
+debugfs: ex /sample
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1184 0
+ 1/ 1 1/ 5 0 - 9 1144 - 1153 10 Uninit
+ 1/ 1 2/ 5 11 - 12 1155 - 1156 2 Uninit
+ 1/ 1 3/ 5 14 - 25 1158 - 1169 12 Uninit
+ 1/ 1 4/ 5 27 - 28 1171 - 1172 2 Uninit
+ 1/ 1 5/ 5 30 - 39 1174 - 1183 10 Uninit
+debugfs: ex /b8
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1232 0
+ 1/ 1 1/ 1 0 - 7 1192 - 1199 8 Uninit
+debugfs: ex /b9
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1248 0
+ 1/ 1 1/ 1 0 - 8 1200 - 1208 9 Uninit
+debugfs: ex /b10
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1272 0
+ 1/ 1 1/ 1 0 - 9 1216 - 1225 10 Uninit
+debugfs: ex /b11
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1296 0
+ 1/ 1 1/ 2 0 - 7 1240 - 1247 8 Uninit
+ 1/ 1 2/ 2 8 - 9 1256 - 1257 2 Uninit
+debugfs: ex /b12
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1320 0
+ 1/ 1 1/ 3 0 - 7 1264 - 1271 8 Uninit
+ 1/ 1 2/ 3 8 - 9 1280 - 1281 2 Uninit
+ 1/ 1 3/ 3 11 - 11 1283 - 1283 1 Uninit
+debugfs: ex /b13
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1344 0
+ 1/ 1 1/ 3 0 - 7 1288 - 1295 8 Uninit
+ 1/ 1 2/ 3 8 - 9 1304 - 1305 2 Uninit
+ 1/ 1 3/ 3 11 - 12 1307 - 1308 2 Uninit
+debugfs: ex /b14
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1368 0
+ 1/ 1 1/ 3 0 - 7 1312 - 1319 8 Uninit
+ 1/ 1 2/ 3 8 - 9 1328 - 1329 2 Uninit
+ 1/ 1 3/ 3 11 - 12 1331 - 1332 2 Uninit
+debugfs: ex /b15
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1392 0
+ 1/ 1 1/ 4 0 - 7 1336 - 1343 8 Uninit
+ 1/ 1 2/ 4 8 - 9 1352 - 1353 2 Uninit
+ 1/ 1 3/ 4 11 - 12 1355 - 1356 2 Uninit
+ 1/ 1 4/ 4 14 - 14 1358 - 1358 1 Uninit
+debugfs: ex /c24
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 25 - 4294967295 1416 4294967271
+ 1/ 1 1/ 3 25 - 25 1401 - 1401 1 Uninit
+ 1/ 1 2/ 3 27 - 28 1403 - 1404 2 Uninit
+ 1/ 1 3/ 3 30 - 39 1406 - 1415 10 Uninit
+debugfs: ex /c25
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 27 - 4294967295 1440 4294967269
+ 1/ 1 1/ 2 27 - 28 1427 - 1428 2 Uninit
+ 1/ 1 2/ 2 30 - 39 1430 - 1439 10 Uninit
+debugfs: ex /c26
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 27 - 4294967295 1464 4294967269
+ 1/ 1 1/ 2 27 - 28 1451 - 1452 2 Uninit
+ 1/ 1 2/ 2 30 - 39 1454 - 1463 10 Uninit
+debugfs: ex /c27
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 28 - 4294967295 1488 4294967268
+ 1/ 1 1/ 2 28 - 28 1476 - 1476 1 Uninit
+ 1/ 1 2/ 2 30 - 39 1478 - 1487 10 Uninit
+debugfs: ex /c28
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 30 - 4294967295 1512 4294967266
+ 1/ 1 1/ 1 30 - 39 1502 - 1511 10 Uninit
+debugfs: ex /c29
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 30 - 4294967295 1536 4294967266
+ 1/ 1 1/ 1 30 - 39 1526 - 1535 10 Uninit
+debugfs: ex /c30
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 31 - 4294967295 1560 4294967265
+ 1/ 1 1/ 1 31 - 39 1551 - 1559 9 Uninit
+debugfs: ex /c31
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 32 - 4294967295 1584 4294967264
+ 1/ 1 1/ 1 32 - 39 1576 - 1583 8 Uninit
+debugfs: ex /d
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1600 0
+ 1/ 1 1/ 2 0 - 3 1360 - 1363 4 Uninit
+ 1/ 1 2/ 2 36 - 39 1596 - 1599 4 Uninit
+debugfs: ex /e
+Level Entries Logical Physical Length Flags
+ 0/ 1 1/ 1 0 - 4294967295 1624 0
+ 1/ 1 1/ 8 0 - 9 1376 - 1385 10 Uninit
+ 1/ 1 2/ 8 11 - 12 1387 - 1388 2 Uninit
+ 1/ 1 3/ 8 14 - 15 1390 - 1391 2 Uninit
+ 1/ 1 4/ 8 16 - 18 1568 - 1570 3 Uninit
+ 1/ 1 5/ 8 21 - 23 1573 - 1575 3 Uninit
+ 1/ 1 6/ 8 24 - 25 1608 - 1609 2 Uninit
+ 1/ 1 7/ 8 27 - 28 1611 - 1612 2 Uninit
+ 1/ 1 8/ 8 30 - 39 1614 - 1623 10 Uninit
+debugfs: ex /f
+Level Entries Logical Physical Length Flags
+ 0/ 0 1/ 2 0 - 0 9000 - 9000 1 Uninit
+ 0/ 0 2/ 2 8999 - 8999 17999 - 17999 1 Uninit
+Pass 1: Checking inodes, blocks, and sizes
+Inode 14 extent tree could be more compact. Fix? yes
+
+Inode 15 extent tree could be more compact. Fix? yes
+
+Inode 16 extent tree could be more compact. Fix? yes
+
+Inode 17 extent tree could be more compact. Fix? yes
+
+Inode 18 extent tree could be more compact. Fix? yes
+
+Inode 19 extent tree could be more compact. Fix? yes
+
+Inode 20 extent tree could be more compact. Fix? yes
+
+Inode 22 extent tree could be more compact. Fix? yes
+
+Inode 23 extent tree could be more compact. Fix? yes
+
+Inode 24 extent tree could be more compact. Fix? yes
+
+Inode 25 extent tree could be more compact. Fix? yes
+
+Inode 26 extent tree could be more compact. Fix? yes
+
+Inode 27 extent tree could be more compact. Fix? yes
+
+Inode 28 extent tree could be more compact. Fix? yes
+
+Inode 29 extent tree could be more compact. Fix? yes
+
+Inode 30 extent tree could be more compact. Fix? yes
+
+Pass 1E: Optimizing extent trees
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (8003, counted=8002).
+Fix? yes
+
+Free blocks count wrong (64024, counted=64016).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 32/4096 files (43.8% non-contiguous), 1520/65536 blocks
+Exit status is 1
diff --git a/tests/d_punch_bigalloc/name b/tests/d_punch_bigalloc/name
new file mode 100644
index 0000000..6d61ebe
--- /dev/null
+++ b/tests/d_punch_bigalloc/name
@@ -0,0 +1 @@
+punch sparse files and big files with bigalloc
diff --git a/tests/d_punch_bigalloc/script b/tests/d_punch_bigalloc/script
new file mode 100644
index 0000000..6eb0571
--- /dev/null
+++ b/tests/d_punch_bigalloc/script
@@ -0,0 +1,130 @@
+if test -x $DEBUGFS_EXE; then
+
+FSCK_OPT=-fy
+OUT=$test_name.log
+if [ -f $test_dir/expect.gz ]; then
+ EXP=$test_name.tmp
+ gunzip < $test_dir/expect.gz > $EXP1
+else
+ EXP=$test_dir/expect
+fi
+
+cp /dev/null $OUT
+
+cat > $TMPFILE.conf << ENDL
+[fs_types]
+ext4 = {
+ cluster_size = 8192
+ base_features = sparse_super,filetype,resize_inode,dir_index,ext_attr,^has_journal,extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,64bit
+ blocksize = 1024
+ inode_size = 256
+ inode_ratio = 16384
+}
+ENDL
+MKE2FS_CONFIG=$TMPFILE.conf $MKE2FS -F -o Linux -b 1024 -O bigalloc -T ext4 $TMPFILE 65536 2>&1 | sed -f $cmd_dir/filter.sed >> $OUT 2>&1
+rm -rf $TMPFILE.conf
+
+$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
+rm -f $OUT.new
+
+echo "debugfs write files" >> $OUT
+make_file() {
+ name="$1"
+ start="$2"
+ flag="$3"
+
+ cat << ENDL
+write /dev/null $name
+fallocate /$name 0 39
+punch /$name 10 10
+punch /$name 13 13
+punch /$name 26 26
+punch /$name 29 29
+ENDL
+}
+
+#Files we create:
+# a: punch a 40k file
+# b*: punch sparse file starting at b*
+# c*: punch spare file ending at c*
+# d: midcluster to midcluster, surrounding sparse
+# e: partial middle cluster alloc
+# f: one big file
+base=5000
+cat > $TMPFILE.cmd << ENDL
+write /dev/null a
+fallocate /a 0 39
+punch /a 0 39
+ENDL
+echo "ex /a" >> $TMPFILE.cmd2
+
+make_file sample $base --uninit >> $TMPFILE.cmd
+echo "ex /sample" >> $TMPFILE.cmd2
+base=10000
+
+for i in 8 9 10 11 12 13 14 15; do
+ make_file b$i $(($base + (40 * ($i - 8)))) --uninit >> $TMPFILE.cmd
+ echo "punch /b$i $i 39" >> $TMPFILE.cmd
+ echo "ex /b$i" >> $TMPFILE.cmd2
+done
+
+for i in 24 25 26 27 28 29 30 31; do
+ make_file c$i $(($base + 320 + (40 * ($i - 24)))) --uninit >> $TMPFILE.cmd
+ echo "punch /c$i 0 $i" >> $TMPFILE.cmd
+ echo "ex /c$i" >> $TMPFILE.cmd2
+done
+
+make_file d $(($base + 640)) --uninit >> $TMPFILE.cmd
+echo "punch /d 4 35" >> $TMPFILE.cmd
+echo "ex /d" >> $TMPFILE.cmd2
+
+make_file e $(($base + 680)) --uninit >> $TMPFILE.cmd
+echo "punch /e 19 20" >> $TMPFILE.cmd
+echo "ex /e" >> $TMPFILE.cmd2
+
+cat >> $TMPFILE.cmd << ENDL
+write /dev/null f
+sif /f size 1024
+eo /f
+set_bmap --uninit 0 9000
+ec
+sif /f blocks 16
+setb 9000
+fallocate /f 0 8999
+punch /f 1 8998
+ENDL
+echo "ex /f" >> $TMPFILE.cmd2
+
+$DEBUGFS_EXE -w -f $TMPFILE.cmd $TMPFILE > /dev/null 2>&1
+$DEBUGFS_EXE -f $TMPFILE.cmd2 $TMPFILE >> $OUT.new 2>&1
+sed -f $cmd_dir/filter.sed < $OUT.new >> $OUT
+rm -rf $OUT.new $TMPFILE.cmd $TMPFILE.cmd2
+
+$FSCK -fy -N test_filesys $TMPFILE > $OUT.new 2>&1
+status=$?
+echo Exit status is $status >> $OUT.new
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new >> $OUT
+rm -f $OUT.new
+
+rm -f $TMPFILE
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+ rm -f $test_name.tmp
+fi
+
+unset IMAGE FSCK_OPT OUT EXP
+
+else #if test -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 45/47] fuse2fs: translate ACL structures
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (42 preceding siblings ...)
2014-11-07 21:55 ` [PATCH 43/47] tests: test debugfs punch command Darrick J. Wong
@ 2014-11-07 21:55 ` Darrick J. Wong
2014-11-07 21:55 ` [PATCH 46/47] fuse2fs: handle 64-bit dates correctly Darrick J. Wong
` (4 subsequent siblings)
48 siblings, 0 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:55 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Translate "native" ACL structures into ext4 ACL structures when
reading or writing the ACL EAs.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
configure | 5 +
configure.in | 8 +-
lib/config.h.in | 3 +
misc/fuse2fs.c | 263 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-
4 files changed, 271 insertions(+), 8 deletions(-)
diff --git a/configure b/configure
index 106cc50..b0dfa9a 100755
--- a/configure
+++ b/configure
@@ -12418,7 +12418,7 @@ fi
done
fi
-for ac_header in dirent.h errno.h execinfo.h getopt.h malloc.h mntent.h paths.h semaphore.h setjmp.h signal.h stdarg.h stdint.h stdlib.h termios.h termio.h unistd.h utime.h attr/xattr.h linux/falloc.h linux/fd.h linux/major.h linux/loop.h net/if_dl.h netinet/in.h sys/disklabel.h sys/disk.h sys/file.h sys/ioctl.h sys/mkdev.h sys/mman.h sys/mount.h sys/prctl.h sys/resource.h sys/select.h sys/socket.h sys/sockio.h sys/stat.h sys/syscall.h sys/sysctl.h sys/sysmacros.h sys/time.h sys/types.h sys/un.h sys/wait.h
+for ac_header in dirent.h errno.h execinfo.h getopt.h malloc.h mntent.h paths.h semaphore.h setjmp.h signal.h stdarg.h stdint.h stdlib.h termios.h termio.h unistd.h utime.h attr/xattr.h linux/falloc.h linux/fd.h linux/major.h linux/loop.h net/if_dl.h netinet/in.h sys/acl.h sys/disklabel.h sys/disk.h sys/file.h sys/ioctl.h sys/mkdev.h sys/mman.h sys/mount.h sys/prctl.h sys/resource.h sys/select.h sys/socket.h sys/sockio.h sys/stat.h sys/syscall.h sys/sysctl.h sys/sysmacros.h sys/time.h sys/types.h sys/un.h sys/wait.h
do :
as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh`
ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default"
@@ -13209,6 +13209,7 @@ else
do :
as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh`
ac_fn_c_check_header_compile "$LINENO" "$ac_header" "$as_ac_Header" "#define _FILE_OFFSET_BITS 64
+#define FUSE_USE_VERSION 29
"
if eval test \"x\$"$as_ac_Header"\" = x"yes"; then :
cat >>confdefs.h <<_ACEOF
@@ -13227,6 +13228,7 @@ done
cat confdefs.h - <<_ACEOF >conftest.$ac_ext
/* end confdefs.h. */
+#define FUSE_USE_VERSION 29
#ifdef __linux__
#include <linux/fs.h>
#include <linux/falloc.h>
@@ -13346,6 +13348,7 @@ else
do :
as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh`
ac_fn_c_check_header_compile "$LINENO" "$ac_header" "$as_ac_Header" "#define _FILE_OFFSET_BITS 64
+#define FUSE_USE_VERSION 29
#ifdef __linux__
# include <linux/fs.h>
# include <linux/falloc.h>
diff --git a/configure.in b/configure.in
index 719ce06..98f490f 100644
--- a/configure.in
+++ b/configure.in
@@ -935,6 +935,7 @@ AC_CHECK_HEADERS(m4_flatten([
linux/loop.h
net/if_dl.h
netinet/in.h
+ sys/acl.h
sys/disklabel.h
sys/disk.h
sys/file.h
@@ -1178,10 +1179,12 @@ then
else
AC_CHECK_HEADERS([pthread.h fuse.h], [],
[AC_MSG_FAILURE([Cannot find fuse2fs headers.])],
-[#define _FILE_OFFSET_BITS 64])
+[#define _FILE_OFFSET_BITS 64
+#define FUSE_USE_VERSION 29])
AC_PREPROC_IFELSE(
-[AC_LANG_PROGRAM([[#ifdef __linux__
+[AC_LANG_PROGRAM([[#define FUSE_USE_VERSION 29
+#ifdef __linux__
#include <linux/fs.h>
#include <linux/falloc.h>
#include <linux/xattr.h>
@@ -1196,6 +1199,7 @@ fi
,
AC_CHECK_HEADERS([pthread.h fuse.h], [], [FUSE_CMT="#"],
[#define _FILE_OFFSET_BITS 64
+#define FUSE_USE_VERSION 29
#ifdef __linux__
# include <linux/fs.h>
# include <linux/falloc.h>
diff --git a/lib/config.h.in b/lib/config.h.in
index 25658d9..61b5f38 100644
--- a/lib/config.h.in
+++ b/lib/config.h.in
@@ -467,6 +467,9 @@
/* Define to 1 if you have the `sysconf' function. */
#undef HAVE_SYSCONF
+/* Define to 1 if you have the <sys/acl.h> header file. */
+#undef HAVE_SYS_ACL_H
+
/* Define to 1 if you have the <sys/disklabel.h> header file. */
#undef HAVE_SYS_DISKLABEL_H
diff --git a/misc/fuse2fs.c b/misc/fuse2fs.c
index fff9312..9d0af5b 100644
--- a/misc/fuse2fs.c
+++ b/misc/fuse2fs.c
@@ -18,9 +18,15 @@
# include <linux/falloc.h>
# include <linux/xattr.h>
# define FUSE_PLATFORM_OPTS ",nonempty,big_writes"
+# ifdef HAVE_SYS_ACL_H
+# define TRANSLATE_LINUX_ACLS
+# endif
#else
# define FUSE_PLATFORM_OPTS ""
#endif
+#ifdef TRANSLATE_LINUX_ACLS
+# include <sys/acl.h>
+#endif
#include <sys/ioctl.h>
#include <unistd.h>
#include <fuse.h>
@@ -85,6 +91,200 @@ static ext2_filsys global_fs; /* Try not to use this directly */
errcode_t ext2fs_run_ext3_journal(ext2_filsys *fs);
+/* ACL translation stuff */
+#ifdef TRANSLATE_LINUX_ACLS
+/*
+ * Copied from acl_ea.h in libacl source; ACLs have to be sent to and from fuse
+ * in this format... at least on Linux.
+ */
+#define ACL_EA_ACCESS "system.posix_acl_access"
+#define ACL_EA_DEFAULT "system.posix_acl_default"
+
+#define ACL_EA_VERSION 0x0002
+
+typedef struct {
+ u_int16_t e_tag;
+ u_int16_t e_perm;
+ u_int32_t e_id;
+} acl_ea_entry;
+
+typedef struct {
+ u_int32_t a_version;
+ acl_ea_entry a_entries[0];
+} acl_ea_header;
+
+static inline size_t acl_ea_size(int count)
+{
+ return sizeof(acl_ea_header) + count * sizeof(acl_ea_entry);
+}
+
+static inline int acl_ea_count(size_t size)
+{
+ if (size < sizeof(acl_ea_header))
+ return -1;
+ size -= sizeof(acl_ea_header);
+ if (size % sizeof(acl_ea_entry))
+ return -1;
+ return size / sizeof(acl_ea_entry);
+}
+
+/*
+ * ext4 ACL structures, copied from fs/ext4/acl.h.
+ */
+#define EXT4_ACL_VERSION 0x0001
+
+typedef struct {
+ __u16 e_tag;
+ __u16 e_perm;
+ __u32 e_id;
+} ext4_acl_entry;
+
+typedef struct {
+ __u16 e_tag;
+ __u16 e_perm;
+} ext4_acl_entry_short;
+
+typedef struct {
+ __u32 a_version;
+} ext4_acl_header;
+
+static inline size_t ext4_acl_size(int count)
+{
+ if (count <= 4) {
+ return sizeof(ext4_acl_header) +
+ count * sizeof(ext4_acl_entry_short);
+ } else {
+ return sizeof(ext4_acl_header) +
+ 4 * sizeof(ext4_acl_entry_short) +
+ (count - 4) * sizeof(ext4_acl_entry);
+ }
+}
+
+static inline int ext4_acl_count(size_t size)
+{
+ ssize_t s;
+
+ size -= sizeof(ext4_acl_header);
+ s = size - 4 * sizeof(ext4_acl_entry_short);
+ if (s < 0) {
+ if (size % sizeof(ext4_acl_entry_short))
+ return -1;
+ return size / sizeof(ext4_acl_entry_short);
+ } else {
+ if (s % sizeof(ext4_acl_entry))
+ return -1;
+ return s / sizeof(ext4_acl_entry) + 4;
+ }
+}
+
+static errcode_t fuse_to_ext4_acl(acl_ea_header *facl, size_t facl_sz,
+ ext4_acl_header **eacl, size_t *eacl_sz)
+{
+ int i, facl_count;
+ ext4_acl_header *h;
+ size_t h_sz;
+ ext4_acl_entry *e;
+ acl_ea_entry *a;
+ void *hptr;
+ errcode_t err;
+
+ facl_count = acl_ea_count(facl_sz);
+ h_sz = ext4_acl_size(facl_count);
+ if (facl_count < 0 || facl->a_version != ACL_EA_VERSION)
+ return EXT2_ET_INVALID_ARGUMENT;
+
+ err = ext2fs_get_mem(h_sz, &h);
+ if (err)
+ return err;
+
+ h->a_version = ext2fs_cpu_to_le32(EXT4_ACL_VERSION);
+ hptr = h + 1;
+ for (i = 0, a = facl->a_entries; i < facl_count; i++, a++) {
+ e = hptr;
+ e->e_tag = ext2fs_cpu_to_le16(a->e_tag);
+ e->e_perm = ext2fs_cpu_to_le16(a->e_perm);
+
+ switch (a->e_tag) {
+ case ACL_USER:
+ case ACL_GROUP:
+ e->e_id = ext2fs_cpu_to_le32(a->e_id);
+ hptr += sizeof(ext4_acl_entry);
+ break;
+ case ACL_USER_OBJ:
+ case ACL_GROUP_OBJ:
+ case ACL_MASK:
+ case ACL_OTHER:
+ hptr += sizeof(ext4_acl_entry_short);
+ break;
+ default:
+ err = EXT2_ET_INVALID_ARGUMENT;
+ goto out;
+ }
+ }
+
+ *eacl = h;
+ *eacl_sz = h_sz;
+ return err;
+out:
+ ext2fs_free_mem(&h);
+ return err;
+}
+
+static errcode_t ext4_to_fuse_acl(acl_ea_header **facl, size_t *facl_sz,
+ ext4_acl_header *eacl, size_t eacl_sz)
+{
+ int i, eacl_count;
+ acl_ea_header *f;
+ ext4_acl_entry *e;
+ acl_ea_entry *a;
+ size_t f_sz;
+ void *hptr;
+ errcode_t err;
+
+ eacl_count = ext4_acl_count(eacl_sz);
+ f_sz = acl_ea_size(eacl_count);
+ if (eacl_count < 0 ||
+ eacl->a_version != ext2fs_cpu_to_le32(EXT4_ACL_VERSION))
+ return EXT2_ET_INVALID_ARGUMENT;
+
+ err = ext2fs_get_mem(f_sz, &f);
+ if (err)
+ return err;
+
+ f->a_version = ACL_EA_VERSION;
+ hptr = eacl + 1;
+ for (i = 0, a = f->a_entries; i < eacl_count; i++, a++) {
+ e = hptr;
+ a->e_tag = ext2fs_le16_to_cpu(e->e_tag);
+ a->e_perm = ext2fs_le16_to_cpu(e->e_perm);
+
+ switch (a->e_tag) {
+ case ACL_USER:
+ case ACL_GROUP:
+ a->e_id = ext2fs_le32_to_cpu(e->e_id);
+ hptr += sizeof(ext4_acl_entry);
+ break;
+ case ACL_USER_OBJ:
+ case ACL_GROUP_OBJ:
+ case ACL_MASK:
+ case ACL_OTHER:
+ hptr += sizeof(ext4_acl_entry_short);
+ break;
+ default:
+ err = EXT2_ET_INVALID_ARGUMENT;
+ goto out;
+ }
+ }
+
+ *facl = f;
+ *facl_sz = f_sz;
+ return err;
+out:
+ ext2fs_free_mem(&f);
+ return err;
+}
+#endif /* TRANSLATE_LINUX_ACLS */
+
/*
* ext2_file_t contains a struct inode, so we can't leave files open.
* Use this as a proxy instead.
@@ -2143,6 +2343,30 @@ static int op_statfs(const char *path, struct statvfs *buf)
return 0;
}
+typedef errcode_t (*xattr_xlate_get)(void **cooked_buf, size_t *cooked_sz,
+ const void *raw_buf, size_t raw_sz);
+typedef errcode_t (*xattr_xlate_set)(const void *cooked_buf, size_t cooked_sz,
+ void **raw_buf, size_t *raw_sz);
+struct xattr_translate {
+ const char *prefix;
+ xattr_xlate_get get;
+ xattr_xlate_set set;
+};
+
+#define XATTR_TRANSLATOR(p, g, s) \
+ {.prefix = (p), \
+ .get = (xattr_xlate_get)(g), \
+ .set = (xattr_xlate_set)(s)}
+
+static struct xattr_translate xattr_translators[] = {
+#ifdef TRANSLATE_LINUX_ACLS
+ XATTR_TRANSLATOR(ACL_EA_ACCESS, ext4_to_fuse_acl, fuse_to_ext4_acl),
+ XATTR_TRANSLATOR(ACL_EA_DEFAULT, ext4_to_fuse_acl, fuse_to_ext4_acl),
+#endif
+ XATTR_TRANSLATOR(NULL, NULL, NULL),
+};
+#undef XATTR_TRANSLATOR
+
static int op_getxattr(const char *path, const char *key, char *value,
size_t len)
{
@@ -2150,8 +2374,9 @@ static int op_getxattr(const char *path, const char *key, char *value,
struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
ext2_filsys fs;
struct ext2_xattr_handle *h;
- void *ptr;
- size_t plen;
+ struct xattr_translate *xt;
+ void *ptr, *cptr;
+ size_t plen, clen;
ext2_ino_t ino;
errcode_t err;
int ret = 0;
@@ -2194,6 +2419,17 @@ static int op_getxattr(const char *path, const char *key, char *value,
goto out2;
}
+ for (xt = xattr_translators; xt->prefix != NULL; xt++) {
+ if (strncmp(key, xt->prefix, strlen(xt->prefix)) == 0) {
+ err = xt->get(&cptr, &clen, ptr, plen);
+ if (err)
+ goto out3;
+ ext2fs_free_mem(&ptr);
+ ptr = cptr;
+ plen = clen;
+ }
+ }
+
if (!len) {
ret = plen;
} else if (len < plen) {
@@ -2203,6 +2439,7 @@ static int op_getxattr(const char *path, const char *key, char *value,
ret = plen;
}
+out3:
ext2fs_free_mem(&ptr);
out2:
err = ext2fs_xattrs_close(&h);
@@ -2317,6 +2554,9 @@ static int op_setxattr(const char *path, const char *key, const char *value,
struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
ext2_filsys fs;
struct ext2_xattr_handle *h;
+ struct xattr_translate *xt;
+ void *cvalue;
+ size_t clen;
ext2_ino_t ino;
errcode_t err;
int ret = 0;
@@ -2356,19 +2596,32 @@ static int op_setxattr(const char *path, const char *key, const char *value,
goto out2;
}
- err = ext2fs_xattr_set(h, key, value, len);
+ cvalue = (void *)value;
+ clen = len;
+ for (xt = xattr_translators; xt->prefix != NULL; xt++) {
+ if (strncmp(key, xt->prefix, strlen(xt->prefix)) == 0) {
+ err = xt->set(value, len, &cvalue, &clen);
+ if (err)
+ goto out3;
+ }
+ }
+
+ err = ext2fs_xattr_set(h, key, cvalue, clen);
if (err) {
ret = translate_error(fs, ino, err);
- goto out2;
+ goto out3;
}
err = ext2fs_xattrs_write(h);
if (err) {
ret = translate_error(fs, ino, err);
- goto out2;
+ goto out3;
}
ret = update_ctime(fs, ino, NULL);
+out3:
+ if (cvalue != value)
+ ext2fs_free_mem(&cvalue);
out2:
err = ext2fs_xattrs_close(&h);
if (!ret && err)
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 46/47] fuse2fs: handle 64-bit dates correctly
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (43 preceding siblings ...)
2014-11-07 21:55 ` [PATCH 45/47] fuse2fs: translate ACL structures Darrick J. Wong
@ 2014-11-07 21:55 ` Darrick J. Wong
2014-11-07 21:55 ` [PATCH 47/47] fuse2fs: implement fallocate Darrick J. Wong
` (3 subsequent siblings)
48 siblings, 0 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:55 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Fix fuse2fs' interpretation of 64-bit date quantities to match the
kernel.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
misc/fuse2fs.c | 31 ++++++++++++++++++++++---------
1 file changed, 22 insertions(+), 9 deletions(-)
diff --git a/misc/fuse2fs.c b/misc/fuse2fs.c
index 9d0af5b..022b0ef 100644
--- a/misc/fuse2fs.c
+++ b/misc/fuse2fs.c
@@ -353,15 +353,24 @@ static int __translate_error(ext2_filsys fs, errcode_t err, ext2_ino_t ino,
static inline __u32 ext4_encode_extra_time(const struct timespec *time)
{
- return (sizeof(time->tv_sec) > 4 ?
- (time->tv_sec >> 32) & EXT4_EPOCH_MASK : 0) |
- ((time->tv_nsec << EXT4_EPOCH_BITS) & EXT4_NSEC_MASK);
+ __u32 extra = sizeof(time->tv_sec) > 4 ?
+ ((time->tv_sec - (__s32)time->tv_sec) >> 32) &
+ EXT4_EPOCH_MASK : 0;
+ return extra | (time->tv_nsec << EXT4_EPOCH_BITS);
}
static inline void ext4_decode_extra_time(struct timespec *time, __u32 extra)
{
- if (sizeof(time->tv_sec) > 4)
- time->tv_sec |= (__u64)((extra) & EXT4_EPOCH_MASK) << 32;
+ if (sizeof(time->tv_sec) > 4 && (extra & EXT4_EPOCH_MASK)) {
+ __u64 extra_bits = extra & EXT4_EPOCH_MASK;
+ /*
+ * Prior to kernel 3.14?, we had a broken decode function,
+ * wherein we effectively did this:
+ * if (extra_bits == 3)
+ * extra_bits = 0;
+ */
+ time->tv_sec += extra_bits << 32;
+ }
time->tv_nsec = ((extra) & EXT4_NSEC_MASK) >> EXT4_EPOCH_BITS;
}
@@ -387,7 +396,7 @@ do { \
(timespec)->tv_sec = (signed)((raw_inode)->xtime); \
if (EXT4_FITS_IN_INODE(raw_inode, xtime ## _extra)) \
ext4_decode_extra_time((timespec), \
- raw_inode->xtime ## _extra); \
+ (raw_inode)->xtime ## _extra); \
else \
(timespec)->tv_nsec = 0; \
} while (0)
@@ -749,6 +758,7 @@ static int stat_inode(ext2_filsys fs, ext2_ino_t ino, struct stat *statbuf)
dev_t fakedev = 0;
errcode_t err;
int ret = 0;
+ struct timespec tv;
memset(&inode, 0, sizeof(inode));
err = ext2fs_read_inode_full(fs, ino, (struct ext2_inode *)&inode,
@@ -766,9 +776,12 @@ static int stat_inode(ext2_filsys fs, ext2_ino_t ino, struct stat *statbuf)
statbuf->st_size = EXT2_I_SIZE(&inode);
statbuf->st_blksize = fs->blocksize;
statbuf->st_blocks = blocks_from_inode(fs, &inode);
- statbuf->st_atime = inode.i_atime;
- statbuf->st_mtime = inode.i_mtime;
- statbuf->st_ctime = inode.i_ctime;
+ EXT4_INODE_GET_XTIME(i_atime, &tv, &inode);
+ statbuf->st_atime = tv.tv_sec;
+ EXT4_INODE_GET_XTIME(i_mtime, &tv, &inode);
+ statbuf->st_mtime = tv.tv_sec;
+ EXT4_INODE_GET_XTIME(i_ctime, &tv, &inode);
+ statbuf->st_ctime = tv.tv_sec;
if (LINUX_S_ISCHR(inode.i_mode) ||
LINUX_S_ISBLK(inode.i_mode)) {
if (inode.i_block[0])
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 47/47] fuse2fs: implement fallocate
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (44 preceding siblings ...)
2014-11-07 21:55 ` [PATCH 46/47] fuse2fs: handle 64-bit dates correctly Darrick J. Wong
@ 2014-11-07 21:55 ` Darrick J. Wong
2014-11-12 22:43 ` [PATCH 48/47] misc: fix infinite loop when finding the start of the hugefile start range Darrick J. Wong
` (2 subsequent siblings)
48 siblings, 0 replies; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-07 21:55 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
Use the (new) ext2fs_fallocate() to fallocate file space.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
misc/fuse2fs.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 57 insertions(+), 1 deletion(-)
diff --git a/misc/fuse2fs.c b/misc/fuse2fs.c
index 022b0ef..679c6e9 100644
--- a/misc/fuse2fs.c
+++ b/misc/fuse2fs.c
@@ -3297,7 +3297,63 @@ out:
static int fallocate_helper(struct fuse_file_info *fp, int mode, off_t offset,
off_t len)
{
- return -EOPNOTSUPP;
+ struct fuse_context *ctxt = fuse_get_context();
+ struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+ struct fuse2fs_file_handle *fh = (struct fuse2fs_file_handle *)fp->fh;
+ ext2_filsys fs;
+ struct ext2_inode_large inode;
+ blk64_t start, end;
+ __u64 fsize;
+ errcode_t err;
+ int flags;
+
+ FUSE2FS_CHECK_CONTEXT(ff);
+ fs = ff->fs;
+ FUSE2FS_CHECK_MAGIC(fs, fh, FUSE2FS_FILE_MAGIC);
+ start = offset / fs->blocksize;
+ end = (offset + len - 1) / fs->blocksize;
+ dbg_printf("%s: ino=%d mode=0x%x start=%jd end=%llu\n", __func__,
+ fh->ino, mode, offset / fs->blocksize, end);
+ if (!fs_can_allocate(ff, len / fs->blocksize))
+ return -ENOSPC;
+
+ memset(&inode, 0, sizeof(inode));
+ err = ext2fs_read_inode_full(fs, fh->ino, (struct ext2_inode *)&inode,
+ sizeof(inode));
+ if (err)
+ return err;
+ fsize = EXT2_I_SIZE(&inode);
+
+ /* Allocate a bunch of blocks */
+ flags = (mode & FL_KEEP_SIZE_FLAG ? 0 :
+ EXT2_FALLOCATE_INIT_BEYOND_EOF);
+ err = ext2fs_fallocate(fs, flags, fh->ino,
+ (struct ext2_inode *)&inode,
+ ~0ULL, start, end - start + 1);
+ if (err && err != EXT2_ET_BLOCK_ALLOC_FAIL)
+ return translate_error(fs, fh->ino, err);
+
+ /* Update i_size */
+ if (!(mode & FL_KEEP_SIZE_FLAG)) {
+ if (offset + len > fsize) {
+ err = ext2fs_inode_size_set(fs,
+ (struct ext2_inode *)&inode,
+ offset + len);
+ if (err)
+ return translate_error(fs, fh->ino, err);
+ }
+ }
+
+ err = update_mtime(fs, fh->ino, &inode);
+ if (err)
+ return err;
+
+ err = ext2fs_write_inode_full(fs, fh->ino, (struct ext2_inode *)&inode,
+ sizeof(inode));
+ if (err)
+ return translate_error(fs, fh->ino, err);
+
+ return err;
}
static errcode_t clean_block_middle(ext2_filsys fs, ext2_ino_t ino,
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 48/47] misc: fix infinite loop when finding the start of the hugefile start range
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (45 preceding siblings ...)
2014-11-07 21:55 ` [PATCH 47/47] fuse2fs: implement fallocate Darrick J. Wong
@ 2014-11-12 22:43 ` Darrick J. Wong
2014-12-03 3:06 ` Theodore Ts'o
2014-11-27 0:01 ` [PATCH 49/47] libext2fs: don't report garbage inodes with really large inodes Darrick J. Wong
2014-12-04 20:39 ` [PATCH 51/47] e2fsck: force-reread of inode from disk when re-checking a checksum error Darrick J. Wong
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-12 22:43 UTC (permalink / raw)
To: tytso, darrick.wong; +Cc: linux-ext4
When looking for the start of the hugefile range, the 'next' variable
is incorrectly decremented. If we happened to find a single free
block, the effect of this decrement is that blk == next, which means
that we never modify the loop control variable, so get_start_block
never returns.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
misc/mk_hugefiles.c | 1 -
tests/m_hugefile_slack/expect | 18 ++++++++++++
tests/m_hugefile_slack/name | 1 +
tests/m_hugefile_slack/script | 61 +++++++++++++++++++++++++++++++++++++++++
4 files changed, 80 insertions(+), 1 deletion(-)
create mode 100644 tests/m_hugefile_slack/expect
create mode 100644 tests/m_hugefile_slack/name
create mode 100644 tests/m_hugefile_slack/script
diff --git a/misc/mk_hugefiles.c b/misc/mk_hugefiles.c
index 3e4274c..230844f 100644
--- a/misc/mk_hugefiles.c
+++ b/misc/mk_hugefiles.c
@@ -437,7 +437,6 @@ static blk64_t get_start_block(ext2_filsys fs, blk64_t slack)
blk, last_blk, &next);
if (retval)
next = last_blk;
- next--;
if (next - blk > slack) {
blk += slack;
diff --git a/tests/m_hugefile_slack/expect b/tests/m_hugefile_slack/expect
new file mode 100644
index 0000000..96a628a
--- /dev/null
+++ b/tests/m_hugefile_slack/expect
@@ -0,0 +1,18 @@
+tune2fs test
+Creating filesystem with 786432 1k blocks and 98304 inodes
+Superblock backups stored on blocks:
+ 8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409, 663553
+
+Allocating group tables: \b\b\b\b\bdone
+Writing inode tables: \b\b\b\b\bdone
+Creating journal (16384 blocks): done
+Creating 6368 huge file(s) with 117 blocks each: done
+Writing superblocks and filesystem accounting information: \b\b\b\b\bdone
+
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+
+Exit status is 0
diff --git a/tests/m_hugefile_slack/name b/tests/m_hugefile_slack/name
new file mode 100644
index 0000000..51cef19
--- /dev/null
+++ b/tests/m_hugefile_slack/name
@@ -0,0 +1 @@
+trigger infinite loop when creating hugefiles with a lot of slack
diff --git a/tests/m_hugefile_slack/script b/tests/m_hugefile_slack/script
new file mode 100644
index 0000000..96f5ec5
--- /dev/null
+++ b/tests/m_hugefile_slack/script
@@ -0,0 +1,61 @@
+if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+
+FSCK_OPT=-fn
+OUT=$test_name.log
+EXP=$test_dir/expect
+CONF=$TMPFILE.conf
+
+#gzip -d < $EXP.gz > $EXP
+
+cat > $CONF << ENDL
+[fs_types]
+ ext4h = {
+ features = has_journal,extent,huge_file,uninit_bg,dir_nlink,extra_isize,sparse_super,filetype,dir_index,ext_attr,^resize_inode,^meta_bg,^flex_bg,metadata_csum,64bit
+ blocksize = 1024
+ inode_size = 256
+ make_hugefiles = true
+ hugefiles_dir = /
+ hugefiles_slack = 12000K
+ hugefiles_name = aaaaa
+ hugefiles_digits = 4
+ hugefiles_size = 117K
+ zero_hugefiles = false
+ }
+ENDL
+
+echo "tune2fs test" > $OUT
+
+MKE2FS_CONFIG=$CONF $MKE2FS -F -T ext4h -I 128 $TMPFILE 786432 >> $OUT 2>&1
+rm -rf $CONF
+
+# dump and check. if we get this far, we succeeded...
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT 2>&1
+status=$?
+echo Exit status is $status >> $OUT
+
+rm $TMPFILE
+
+#
+# Do the verification
+#
+
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" -e 's/test_filesys:.*//g' < $OUT > $OUT.new
+mv $OUT.new $OUT
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+ echo "$test_name: $test_description: ok"
+ touch $test_name.ok
+else
+ echo "$test_name: $test_description: failed"
+ diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+fi
+
+unset IMAGE FSCK_OPT OUT EXP CONF
+
+else #if test -x $RESIZE2FS_EXE -a -x $DEBUGFS_EXE; then
+ echo "$test_name: $test_description: skipped"
+fi
+
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 49/47] libext2fs: don't report garbage inodes with really large inodes
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (46 preceding siblings ...)
2014-11-12 22:43 ` [PATCH 48/47] misc: fix infinite loop when finding the start of the hugefile start range Darrick J. Wong
@ 2014-11-27 0:01 ` Darrick J. Wong
2014-12-03 3:18 ` Theodore Ts'o
2014-12-04 20:39 ` [PATCH 51/47] e2fsck: force-reread of inode from disk when re-checking a checksum error Darrick J. Wong
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-11-27 0:01 UTC (permalink / raw)
To: tytso; +Cc: linux-ext4
If the inode size is large enough that there are fewer than two inodes
per block, don't report an inode checksum failure as a garbage inode
during the scan because the "more than half are broken" criteria that
we use to decide if a block of inodes is garbage doesn't really apply.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
lib/ext2fs/inode.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/lib/ext2fs/inode.c b/lib/ext2fs/inode.c
index 4b3e14e..17e49d8 100644
--- a/lib/ext2fs/inode.c
+++ b/lib/ext2fs/inode.c
@@ -424,6 +424,9 @@ static void check_inode_block_sanity(ext2_inode_scan scan, blk64_t num_blocks)
memset(block_status, 0, scan->inode_buffer_blocks);
inodes_per_block = EXT2_INODES_PER_BLOCK(scan->fs->super);
+ if (inodes_per_block < 2)
+ return;
+
#ifdef WORDS_BIGENDIAN
if (ext2fs_get_mem(EXT2_INODE_SIZE(scan->fs->super), &inode))
return;
^ permalink raw reply related [flat|nested] 104+ messages in thread* [PATCH 51/47] e2fsck: force-reread of inode from disk when re-checking a checksum error
2014-11-07 21:50 [PATCH 00/47] e2fsprogs November 2014 patchbomb Darrick J. Wong
` (47 preceding siblings ...)
2014-11-27 0:01 ` [PATCH 49/47] libext2fs: don't report garbage inodes with really large inodes Darrick J. Wong
@ 2014-12-04 20:39 ` Darrick J. Wong
2014-12-11 22:49 ` Theodore Ts'o
48 siblings, 1 reply; 104+ messages in thread
From: Darrick J. Wong @ 2014-12-04 20:39 UTC (permalink / raw)
To: tytso; +Cc: linux-ext4
When we're rechecking an inode checksum failure, we need to force the
inode to be re-read from disk so that the verification routine runs,
so drop the stashed inode.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
e2fsck/pass1.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 82792e4..0073bed 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -671,6 +671,7 @@ static errcode_t recheck_bad_inode_checksum(ext2_filsys fs, ext2_ino_t ino,
* Reread inode. If we don't see checksum error, then this inode
* has been fixed elsewhere.
*/
+ ctx->stashed_ino = 0;
retval = ext2fs_read_inode_full(fs, ino, (struct ext2_inode *)&inode,
sizeof(inode));
if (retval && retval != EXT2_ET_INODE_CSUM_INVALID)
^ permalink raw reply related [flat|nested] 104+ messages in thread