* [PATCH 1/4] xfs/{319,323}: don't checksum files after log recovery @ 2017-08-30 4:40 Darrick J. Wong 2017-08-30 4:40 ` [PATCH 2/4] xfs/013: exit cleaner thread if fsstress dies Darrick J. Wong ` (5 more replies) 0 siblings, 6 replies; 13+ messages in thread From: Darrick J. Wong @ 2017-08-30 4:40 UTC (permalink / raw) To: eguan; +Cc: linux-xfs, fstests From: Darrick J. Wong <darrick.wong@oracle.com> These two tests simulate log failure during a reflink operation. However, the contents of the target of the reflink operation depend on the block size, so we cannot hardcode md5 hashes in this test. Since the whole point of the test is to ensure that the the complex chain of transactions actually finishes no matter where the interruption, it is sufficient simply to run the usual end-of-test fsck to look for corrupt metadata. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> --- tests/xfs/319 | 5 ----- tests/xfs/319.out | 4 ---- tests/xfs/323 | 4 ---- tests/xfs/323.out | 3 --- 4 files changed, 16 deletions(-) diff --git a/tests/xfs/319 b/tests/xfs/319 index bbe21df..457f181 100755 --- a/tests/xfs/319 +++ b/tests/xfs/319 @@ -82,11 +82,6 @@ touch $SCRATCH_MNT/badfs 2>&1 | _filter_scratch echo "Remount to replay log" _scratch_inject_logprint >> $seqres.full -echo "Check files" -md5sum $SCRATCH_MNT/file1 | _filter_scratch -md5sum $SCRATCH_MNT/file2 | _filter_scratch -md5sum $SCRATCH_MNT/file3 | _filter_scratch - echo "FS should be online, touch should succeed" touch $SCRATCH_MNT/goodfs diff --git a/tests/xfs/319.out b/tests/xfs/319.out index bccde84..160f5fd 100644 --- a/tests/xfs/319.out +++ b/tests/xfs/319.out @@ -11,9 +11,5 @@ XFS_IOC_CLONE_RANGE: Input/output error FS should be shut down, touch will fail touch: cannot touch 'SCRATCH_MNT/badfs': Input/output error Remount to replay log -Check files -2a4f043bf9730a9e8882c9264b9797b3 SCRATCH_MNT/file1 -2a4f043bf9730a9e8882c9264b9797b3 SCRATCH_MNT/file2 -2a4f043bf9730a9e8882c9264b9797b3 SCRATCH_MNT/file3 FS should be online, touch should succeed Done diff --git a/tests/xfs/323 b/tests/xfs/323 index c124f25..18893f2 100755 --- a/tests/xfs/323 +++ b/tests/xfs/323 @@ -80,10 +80,6 @@ touch $SCRATCH_MNT/badfs 2>&1 | _filter_scratch echo "Remount to replay log" _scratch_inject_logprint >> $seqres.full -echo "Check files" -md5sum $SCRATCH_MNT/file1 | _filter_scratch -md5sum $SCRATCH_MNT/file3 | _filter_scratch - echo "FS should be online, touch should succeed" touch $SCRATCH_MNT/goodfs diff --git a/tests/xfs/323.out b/tests/xfs/323.out index be41b7e..99b9688 100644 --- a/tests/xfs/323.out +++ b/tests/xfs/323.out @@ -10,8 +10,5 @@ XFS_IOC_CLONE_RANGE: Input/output error FS should be shut down, touch will fail touch: cannot touch 'SCRATCH_MNT/badfs': Input/output error Remount to replay log -Check files -2a4f043bf9730a9e8882c9264b9797b3 SCRATCH_MNT/file1 -2a4f043bf9730a9e8882c9264b9797b3 SCRATCH_MNT/file3 FS should be online, touch should succeed Done ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 2/4] xfs/013: exit cleaner thread if fsstress dies 2017-08-30 4:40 [PATCH 1/4] xfs/{319,323}: don't checksum files after log recovery Darrick J. Wong @ 2017-08-30 4:40 ` Darrick J. Wong 2017-08-30 7:51 ` Christoph Hellwig 2017-08-30 4:41 ` [PATCH 3/4] xfs: test rmapbt updates are correct with insert/collapse range Darrick J. Wong ` (4 subsequent siblings) 5 siblings, 1 reply; 13+ messages in thread From: Darrick J. Wong @ 2017-08-30 4:40 UTC (permalink / raw) To: eguan; +Cc: linux-xfs, fstests From: Darrick J. Wong <darrick.wong@oracle.com> In this test, the cleaner thread deletes the directory trees created by fsstress in order to exercise the free inode btree code. However, if fsstress dies, the cleaner can end up waiting forever for a directory that will never be created, which hangs up the test run. Therefore, abort if fsstress has ended. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> --- tests/xfs/013 | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tests/xfs/013 b/tests/xfs/013 index 817558f..4e49e4b 100755 --- a/tests/xfs/013 +++ b/tests/xfs/013 @@ -88,6 +88,10 @@ _cleaner() while [ ! -e $need ] do sleep 3 + if ! pgrep fsstress > /dev/null 2>&1; then + echo "fsstress died?" + return + fi done rm -rf $dir/dir$i ^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH 2/4] xfs/013: exit cleaner thread if fsstress dies 2017-08-30 4:40 ` [PATCH 2/4] xfs/013: exit cleaner thread if fsstress dies Darrick J. Wong @ 2017-08-30 7:51 ` Christoph Hellwig 0 siblings, 0 replies; 13+ messages in thread From: Christoph Hellwig @ 2017-08-30 7:51 UTC (permalink / raw) To: Darrick J. Wong; +Cc: eguan, linux-xfs, fstests Looks good, Reviewed-by: Christoph Hellwig <hch@lst.de> ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH 3/4] xfs: test rmapbt updates are correct with insert/collapse range 2017-08-30 4:40 [PATCH 1/4] xfs/{319,323}: don't checksum files after log recovery Darrick J. Wong 2017-08-30 4:40 ` [PATCH 2/4] xfs/013: exit cleaner thread if fsstress dies Darrick J. Wong @ 2017-08-30 4:41 ` Darrick J. Wong 2017-08-30 7:53 ` Christoph Hellwig ` (2 more replies) 2017-08-30 4:41 ` [PATCH 4/4] generic: try various unicode normalization games Darrick J. Wong ` (3 subsequent siblings) 5 siblings, 3 replies; 13+ messages in thread From: Darrick J. Wong @ 2017-08-30 4:41 UTC (permalink / raw) To: eguan; +Cc: linux-xfs, fstests From: Darrick J. Wong <darrick.wong@oracle.com> Make sure that we update the rmapbt correctly when we collapse-range a file and the extents on both sides of the hole can be merged. We can construct this pretty trivially with insert-range and write, so test that too. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> --- src/punch-alternating.c | 15 +++++- tests/xfs/706 | 127 +++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/706.out | 9 +++ tests/xfs/group | 1 4 files changed, 150 insertions(+), 2 deletions(-) create mode 100755 tests/xfs/706 create mode 100644 tests/xfs/706.out diff --git a/src/punch-alternating.c b/src/punch-alternating.c index 25228cd..281e283 100644 --- a/src/punch-alternating.c +++ b/src/punch-alternating.c @@ -24,6 +24,7 @@ int main(int argc, char *argv[]) struct stat s; struct statfs sf; off_t offset; + off_t start_offset = 0; int fd; blksize_t blksz; off_t sz; @@ -33,11 +34,19 @@ int main(int argc, char *argv[]) int size = 1; /* punch $SIZE blocks ... */ int interval = 2; /* every $INTERVAL blocks */ - while ((c = getopt(argc, argv, "i:s:")) != EOF) { + while ((c = getopt(argc, argv, "i:o:s:")) != EOF) { switch (c) { case 'i': interval = atoi(optarg); break; + case 'o': + errno = 0; + start_offset = strtoull(optarg, NULL, 0); + if (errno) { + fprintf(stderr, "invalid offset '%s'\n", optarg); + return 1; + } + break; case 's': size = atoi(optarg); break; @@ -75,7 +84,9 @@ int main(int argc, char *argv[]) blksz = sf.f_bsize; mode = FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE; - for (offset = 0; offset < sz; offset += blksz * interval) { + for (offset = start_offset * blksz; + offset < sz; + offset += blksz * interval) { error = fallocate(fd, mode, offset, blksz * size); if (error) goto err; diff --git a/tests/xfs/706 b/tests/xfs/706 new file mode 100755 index 0000000..88d789b --- /dev/null +++ b/tests/xfs/706 @@ -0,0 +1,127 @@ +#! /bin/bash +# FS QA Test No. 706 +# +# Make sure that we can handle insert-range followed by collapse-range. +# XFS had a bug where the rmap wasn't getting updated for that case. +# +#----------------------------------------------------------------------- +# Copyright (c) 2017, Oracle and/or its affiliates. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- + +seq=`basename "$0"` +seqres="$RESULT_DIR/$seq" +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -rf "$tmp".* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/reflink + +# real QA test starts here +_supported_os Linux +_supported_fs xfs +_require_test_program "punch-alternating" +_require_xfs_scratch_rmapbt +_require_xfs_io_command "fcollapse" +_require_xfs_io_command "finsert" + +rm -f "$seqres.full" + +echo "Format and mount" +_scratch_mkfs > "$seqres.full" 2>&1 +_scratch_mount +blksz=65536 +blocks1=150 # enough extents to force btree format +blocks2=16 # local format +len1=$((blocks1 * blksz)) +len2=$((blocks2 * blksz)) +file_blksz=$(_get_file_block_size $SCRATCH_MNT) + +echo "Create some files" +$XFS_IO_PROG -f \ + -c "falloc 0 $len1" \ + -c "pwrite -S 0x68 -b 1048576 0 $len1" \ + $SCRATCH_MNT/f1 >> $seqres.full +$XFS_IO_PROG -f \ + -c "falloc 0 $len2" \ + -c "pwrite -S 0x68 -b 1048576 0 $len2" \ + $SCRATCH_MNT/f2 >> $seqres.full + +./src/punch-alternating -o $((16 * blksz / file_blksz)) \ + -s $((blksz / file_blksz)) \ + -i $((blksz * 2 / file_blksz)) \ + $SCRATCH_MNT/f1 >> "$seqres.full" + +echo "Insert and write file range" +$XFS_IO_PROG \ + -c "finsert $((8 * blksz)) $blksz" \ + -c "pwrite -S 0x69 $((8 * blksz)) $blksz" \ + -c "fsync" \ + $SCRATCH_MNT/f1 >> $seqres.full +$XFS_IO_PROG \ + -c "finsert $((8 * blksz)) $blksz" \ + -c "pwrite -S 0x69 $((8 * blksz)) $blksz" \ + -c "fsync" \ + $SCRATCH_MNT/f2 >> $seqres.full + +# Make reflink copies to force it to use the fancy rmap operations. +# The test's functionality doesn't depend on the fs supporting reflink. +_cp_reflink $SCRATCH_MNT/f1 $SCRATCH_MNT/e1 >> $seqres.full 2>&1 +_cp_reflink $SCRATCH_MNT/f2 $SCRATCH_MNT/e2 >> $seqres.full 2>&1 + +echo "f1 bmap" >> $seqres.full +$XFS_IO_PROG -c 'bmap -elpv' $SCRATCH_MNT/f1 >> $seqres.full +echo "f2 bmap" >> $seqres.full +$XFS_IO_PROG -c 'bmap -elpv' $SCRATCH_MNT/f2 >> $seqres.full +echo "fsmap" >> $seqres.full +$XFS_IO_PROG -c 'fsmap -v' $SCRATCH_MNT >> $seqres.full + +echo "Remount" +_scratch_unmount +_scratch_xfs_check +_scratch_mount + +echo "Collapse file" +$XFS_IO_PROG -c "fcollapse $((8 * blksz)) $blksz" $SCRATCH_MNT/f1 >> $seqres.full +$XFS_IO_PROG -c "fcollapse $((8 * blksz)) $blksz" $SCRATCH_MNT/f2 >> $seqres.full + +echo "f1 bmap" >> $seqres.full +$XFS_IO_PROG -c 'bmap -elpv' $SCRATCH_MNT/f1 >> $seqres.full +echo "f2 bmap" >> $seqres.full +$XFS_IO_PROG -c 'bmap -elpv' $SCRATCH_MNT/f2 >> $seqres.full +echo "fsmap" >> $seqres.full +$XFS_IO_PROG -c 'fsmap -v' $SCRATCH_MNT >> $seqres.full + +echo "Check file" +md5sum $SCRATCH_MNT/f1 | _filter_scratch +md5sum $SCRATCH_MNT/f2 | _filter_scratch +od -tx1 -Ad -c $SCRATCH_MNT/f1 >> $seqres.full +od -tx1 -Ad -c $SCRATCH_MNT/f2 >> $seqres.full + +# success, all done +status=0 +exit diff --git a/tests/xfs/706.out b/tests/xfs/706.out new file mode 100644 index 0000000..064d33c --- /dev/null +++ b/tests/xfs/706.out @@ -0,0 +1,9 @@ +QA output created by 706 +Format and mount +Create some files +Insert and write file range +Remount +Collapse file +Check file +f45e899d0ed7c055f7baa4e2039eb152 SCRATCH_MNT/f1 +e57a7dd63e41d5ba02a754dd72386c3d SCRATCH_MNT/f2 diff --git a/tests/xfs/group b/tests/xfs/group index 5977330..bfed828 100644 --- a/tests/xfs/group +++ b/tests/xfs/group @@ -426,3 +426,4 @@ 428 dangerous_fuzzers dangerous_scrub dangerous_online_repair 429 dangerous_fuzzers dangerous_scrub dangerous_repair 430 dangerous_fuzzers dangerous_scrub dangerous_online_repair +706 auto quick clone rmap ^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH 3/4] xfs: test rmapbt updates are correct with insert/collapse range 2017-08-30 4:41 ` [PATCH 3/4] xfs: test rmapbt updates are correct with insert/collapse range Darrick J. Wong @ 2017-08-30 7:53 ` Christoph Hellwig 2017-08-30 22:58 ` Darrick J. Wong 2017-08-31 10:28 ` Eryu Guan 2017-08-31 14:47 ` [PATCH v2 " Darrick J. Wong 2 siblings, 1 reply; 13+ messages in thread From: Christoph Hellwig @ 2017-08-30 7:53 UTC (permalink / raw) To: Darrick J. Wong; +Cc: eguan, linux-xfs, fstests On Tue, Aug 29, 2017 at 09:41:02PM -0700, Darrick J. Wong wrote: > From: Darrick J. Wong <darrick.wong@oracle.com> > > Make sure that we update the rmapbt correctly when we collapse-range a > file and the extents on both sides of the hole can be merged. We can > construct this pretty trivially with insert-range and write, so test > that too. Does this break on the current tree? From code inspection I suspect the current code is doing the wrong thing there. > +./src/punch-alternating -o $((16 * blksz / file_blksz)) \ Shouldn't we always use $here/src/progname ? ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 3/4] xfs: test rmapbt updates are correct with insert/collapse range 2017-08-30 7:53 ` Christoph Hellwig @ 2017-08-30 22:58 ` Darrick J. Wong 0 siblings, 0 replies; 13+ messages in thread From: Darrick J. Wong @ 2017-08-30 22:58 UTC (permalink / raw) To: Christoph Hellwig; +Cc: eguan, linux-xfs, fstests On Wed, Aug 30, 2017 at 12:53:21AM -0700, Christoph Hellwig wrote: > On Tue, Aug 29, 2017 at 09:41:02PM -0700, Darrick J. Wong wrote: > > From: Darrick J. Wong <darrick.wong@oracle.com> > > > > Make sure that we update the rmapbt correctly when we collapse-range a > > file and the extents on both sides of the hole can be merged. We can > > construct this pretty trivially with insert-range and write, so test > > that too. > > Does this break on the current tree? From code inspection I suspect > the current code is doing the wrong thing there. AFAICR the existing rmap code in bmse_shift_one /does/ work, though only due to the subtlety that xfs_rmap_{,un}map_extent is smart enough to automatically merge and unmerge rmap extents for you. Hence if you have the following: AAAAABBBAAAAAAAA Where both A's could be merged after BBB goes away: AAAAAAAAAAAAA--- The deferred rmap code will merge the two A regions into a single rmap extent record automatically. Hmm, so I guess that means that in the patch "xfs: use xfs_iext_*_extent helpers in xfs_bmap_shift_extents" you can replace the three xfs_rmap calls in xfs_bmse_merge with: /* update reverse mapping */ error = xfs_rmap_unmap_extent(mp, dfops, ip, whichfork, got); if (error) return error; memcpy(&new, got, sizeof(new)); new.br_startoff = left->br_startoff + left->br_blockcount; return xfs_rmap_map_extent(mp, dfops, ip, whichfork, &new); ...which saves us a deferred op. Ok, will propose that in the actual thread. > > +./src/punch-alternating -o $((16 * blksz / file_blksz)) \ > > Shouldn't we always use $here/src/progname ? Ok, I didn't realize that. There are a lot of programs that still use ./src.... --D > -- > To unsubscribe from this list: send the line "unsubscribe fstests" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 3/4] xfs: test rmapbt updates are correct with insert/collapse range 2017-08-30 4:41 ` [PATCH 3/4] xfs: test rmapbt updates are correct with insert/collapse range Darrick J. Wong 2017-08-30 7:53 ` Christoph Hellwig @ 2017-08-31 10:28 ` Eryu Guan 2017-08-31 14:47 ` [PATCH v2 " Darrick J. Wong 2 siblings, 0 replies; 13+ messages in thread From: Eryu Guan @ 2017-08-31 10:28 UTC (permalink / raw) To: Darrick J. Wong; +Cc: linux-xfs, fstests On Tue, Aug 29, 2017 at 09:41:02PM -0700, Darrick J. Wong wrote: > From: Darrick J. Wong <darrick.wong@oracle.com> > > Make sure that we update the rmapbt correctly when we collapse-range a > file and the extents on both sides of the hole can be merged. We can > construct this pretty trivially with insert-range and write, so test > that too. > > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> > --- > src/punch-alternating.c | 15 +++++- > tests/xfs/706 | 127 +++++++++++++++++++++++++++++++++++++++++++++++ > tests/xfs/706.out | 9 +++ > tests/xfs/group | 1 > 4 files changed, 150 insertions(+), 2 deletions(-) > create mode 100755 tests/xfs/706 > create mode 100644 tests/xfs/706.out > > > diff --git a/src/punch-alternating.c b/src/punch-alternating.c > index 25228cd..281e283 100644 > --- a/src/punch-alternating.c > +++ b/src/punch-alternating.c > @@ -24,6 +24,7 @@ int main(int argc, char *argv[]) > struct stat s; > struct statfs sf; > off_t offset; > + off_t start_offset = 0; > int fd; > blksize_t blksz; > off_t sz; > @@ -33,11 +34,19 @@ int main(int argc, char *argv[]) > int size = 1; /* punch $SIZE blocks ... */ > int interval = 2; /* every $INTERVAL blocks */ > > - while ((c = getopt(argc, argv, "i:s:")) != EOF) { > + while ((c = getopt(argc, argv, "i:o:s:")) != EOF) { Update usage info too? Otherwise looks fine to me. Thanks, Eryu ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH v2 3/4] xfs: test rmapbt updates are correct with insert/collapse range 2017-08-30 4:41 ` [PATCH 3/4] xfs: test rmapbt updates are correct with insert/collapse range Darrick J. Wong 2017-08-30 7:53 ` Christoph Hellwig 2017-08-31 10:28 ` Eryu Guan @ 2017-08-31 14:47 ` Darrick J. Wong 2 siblings, 0 replies; 13+ messages in thread From: Darrick J. Wong @ 2017-08-31 14:47 UTC (permalink / raw) To: eguan; +Cc: linux-xfs, fstests Make sure that we update the rmapbt correctly when we collapse-range a file and the extents on both sides of the hole can be merged. We can construct this pretty trivially with insert-range and write, so test that too. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> --- v2: update usage info and expand the test description --- src/punch-alternating.c | 20 ++++++- tests/xfs/706 | 128 +++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/706.out | 9 +++ tests/xfs/group | 1 4 files changed, 154 insertions(+), 4 deletions(-) create mode 100755 tests/xfs/706 create mode 100644 tests/xfs/706.out diff --git a/src/punch-alternating.c b/src/punch-alternating.c index 25228cd..18dd215 100644 --- a/src/punch-alternating.c +++ b/src/punch-alternating.c @@ -13,9 +13,10 @@ void usage(char *cmd) { - printf("Usage: %s [-i interval] [-s size] file\n", cmd); + printf("Usage: %s [-o offset] [-i interval] [-s size] file\n", cmd); printf("Punches every other block in the file by default,\n"); - printf("or 'size' blocks every 'interval' blocks with options.\n"); + printf("or 'size' blocks every 'interval' blocks starting at\n"); + printf("'offset'. Units are in fstatfs blocks.\n"); exit(1); } @@ -24,6 +25,7 @@ int main(int argc, char *argv[]) struct stat s; struct statfs sf; off_t offset; + off_t start_offset = 0; int fd; blksize_t blksz; off_t sz; @@ -33,11 +35,19 @@ int main(int argc, char *argv[]) int size = 1; /* punch $SIZE blocks ... */ int interval = 2; /* every $INTERVAL blocks */ - while ((c = getopt(argc, argv, "i:s:")) != EOF) { + while ((c = getopt(argc, argv, "i:o:s:")) != EOF) { switch (c) { case 'i': interval = atoi(optarg); break; + case 'o': + errno = 0; + start_offset = strtoull(optarg, NULL, 0); + if (errno) { + fprintf(stderr, "invalid offset '%s'\n", optarg); + return 1; + } + break; case 's': size = atoi(optarg); break; @@ -75,7 +85,9 @@ int main(int argc, char *argv[]) blksz = sf.f_bsize; mode = FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE; - for (offset = 0; offset < sz; offset += blksz * interval) { + for (offset = start_offset * blksz; + offset < sz; + offset += blksz * interval) { error = fallocate(fd, mode, offset, blksz * size); if (error) goto err; diff --git a/tests/xfs/706 b/tests/xfs/706 new file mode 100755 index 0000000..c2aefa2 --- /dev/null +++ b/tests/xfs/706 @@ -0,0 +1,128 @@ +#! /bin/bash +# FS QA Test No. 706 +# +# Make sure that we can handle insert-range followed by collapse-range. +# In particular, make sure that fcollapse works for rmap when the +# extents on either side of the collapse area are mergeable. +# +#----------------------------------------------------------------------- +# Copyright (c) 2017, Oracle and/or its affiliates. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- + +seq=`basename "$0"` +seqres="$RESULT_DIR/$seq" +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -rf "$tmp".* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/reflink + +# real QA test starts here +_supported_os Linux +_supported_fs xfs +_require_test_program "punch-alternating" +_require_xfs_scratch_rmapbt +_require_xfs_io_command "fcollapse" +_require_xfs_io_command "finsert" + +rm -f "$seqres.full" + +echo "Format and mount" +_scratch_mkfs > "$seqres.full" 2>&1 +_scratch_mount +blksz=65536 +blocks1=150 # enough extents to force btree format +blocks2=16 # local format +len1=$((blocks1 * blksz)) +len2=$((blocks2 * blksz)) +file_blksz=$(_get_file_block_size $SCRATCH_MNT) + +echo "Create some files" +$XFS_IO_PROG -f \ + -c "falloc 0 $len1" \ + -c "pwrite -S 0x68 -b 1048576 0 $len1" \ + $SCRATCH_MNT/f1 >> $seqres.full +$XFS_IO_PROG -f \ + -c "falloc 0 $len2" \ + -c "pwrite -S 0x68 -b 1048576 0 $len2" \ + $SCRATCH_MNT/f2 >> $seqres.full + +./src/punch-alternating -o $((16 * blksz / file_blksz)) \ + -s $((blksz / file_blksz)) \ + -i $((blksz * 2 / file_blksz)) \ + $SCRATCH_MNT/f1 >> "$seqres.full" + +echo "Insert and write file range" +$XFS_IO_PROG \ + -c "finsert $((8 * blksz)) $blksz" \ + -c "pwrite -S 0x69 $((8 * blksz)) $blksz" \ + -c "fsync" \ + $SCRATCH_MNT/f1 >> $seqres.full +$XFS_IO_PROG \ + -c "finsert $((8 * blksz)) $blksz" \ + -c "pwrite -S 0x69 $((8 * blksz)) $blksz" \ + -c "fsync" \ + $SCRATCH_MNT/f2 >> $seqres.full + +# Make reflink copies to force it to use the fancy rmap operations. +# The test's functionality doesn't depend on the fs supporting reflink. +_cp_reflink $SCRATCH_MNT/f1 $SCRATCH_MNT/e1 >> $seqres.full 2>&1 +_cp_reflink $SCRATCH_MNT/f2 $SCRATCH_MNT/e2 >> $seqres.full 2>&1 + +echo "f1 bmap" >> $seqres.full +$XFS_IO_PROG -c 'bmap -elpv' $SCRATCH_MNT/f1 >> $seqres.full +echo "f2 bmap" >> $seqres.full +$XFS_IO_PROG -c 'bmap -elpv' $SCRATCH_MNT/f2 >> $seqres.full +echo "fsmap" >> $seqres.full +$XFS_IO_PROG -c 'fsmap -v' $SCRATCH_MNT >> $seqres.full + +echo "Remount" +_scratch_unmount +_scratch_xfs_check +_scratch_mount + +echo "Collapse file" +$XFS_IO_PROG -c "fcollapse $((8 * blksz)) $blksz" $SCRATCH_MNT/f1 >> $seqres.full +$XFS_IO_PROG -c "fcollapse $((8 * blksz)) $blksz" $SCRATCH_MNT/f2 >> $seqres.full + +echo "f1 bmap" >> $seqres.full +$XFS_IO_PROG -c 'bmap -elpv' $SCRATCH_MNT/f1 >> $seqres.full +echo "f2 bmap" >> $seqres.full +$XFS_IO_PROG -c 'bmap -elpv' $SCRATCH_MNT/f2 >> $seqres.full +echo "fsmap" >> $seqres.full +$XFS_IO_PROG -c 'fsmap -v' $SCRATCH_MNT >> $seqres.full + +echo "Check file" +md5sum $SCRATCH_MNT/f1 | _filter_scratch +md5sum $SCRATCH_MNT/f2 | _filter_scratch +od -tx1 -Ad -c $SCRATCH_MNT/f1 >> $seqres.full +od -tx1 -Ad -c $SCRATCH_MNT/f2 >> $seqres.full + +# success, all done +status=0 +exit diff --git a/tests/xfs/706.out b/tests/xfs/706.out new file mode 100644 index 0000000..064d33c --- /dev/null +++ b/tests/xfs/706.out @@ -0,0 +1,9 @@ +QA output created by 706 +Format and mount +Create some files +Insert and write file range +Remount +Collapse file +Check file +f45e899d0ed7c055f7baa4e2039eb152 SCRATCH_MNT/f1 +e57a7dd63e41d5ba02a754dd72386c3d SCRATCH_MNT/f2 diff --git a/tests/xfs/group b/tests/xfs/group index 5977330..bfed828 100644 --- a/tests/xfs/group +++ b/tests/xfs/group @@ -426,3 +426,4 @@ 428 dangerous_fuzzers dangerous_scrub dangerous_online_repair 429 dangerous_fuzzers dangerous_scrub dangerous_repair 430 dangerous_fuzzers dangerous_scrub dangerous_online_repair +706 auto quick clone rmap ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 4/4] generic: try various unicode normalization games 2017-08-30 4:40 [PATCH 1/4] xfs/{319,323}: don't checksum files after log recovery Darrick J. Wong 2017-08-30 4:40 ` [PATCH 2/4] xfs/013: exit cleaner thread if fsstress dies Darrick J. Wong 2017-08-30 4:41 ` [PATCH 3/4] xfs: test rmapbt updates are correct with insert/collapse range Darrick J. Wong @ 2017-08-30 4:41 ` Darrick J. Wong 2017-08-30 22:52 ` [PATCH v2 " Darrick J. Wong 2017-08-30 22:31 ` [PATCH 5/4] generic/173: don't dump core when mwrite fails Darrick J. Wong ` (2 subsequent siblings) 5 siblings, 1 reply; 13+ messages in thread From: Darrick J. Wong @ 2017-08-30 4:41 UTC (permalink / raw) To: eguan; +Cc: linux-xfs, fstests From: Darrick J. Wong <darrick.wong@oracle.com> Linux filesystems generally treat filenames as a bag of bytes, which means that there can be unique sequences of bytes that render the same on most modern GUIs. So, let's rig up a test to see if it's really true that we can create filenames that look the same but point to different files. xfs_scrub will warn about these kinds of situations, though they're not technically fs "corruption". Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> --- tests/generic/703 | 151 +++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/703.out | 6 ++ tests/generic/group | 1 3 files changed, 158 insertions(+) create mode 100755 tests/generic/703 create mode 100644 tests/generic/703.out diff --git a/tests/generic/703 b/tests/generic/703 new file mode 100755 index 0000000..1ae70ed --- /dev/null +++ b/tests/generic/703 @@ -0,0 +1,151 @@ +#! /bin/bash +# FS QA Test No. 703 +# +# Create a directory with multiple filenames that all appear the same +# (in unicode, anyway) but point to different inodes. In theory all +# Linux filesystems should allow this (filenames are a sequence of +# arbitrary bytes) even if the user implications are horrifying. +# +#----------------------------------------------------------------------- +# Copyright (c) 2017, Oracle and/or its affiliates. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- + +seq=`basename "$0"` +seqres="$RESULT_DIR/$seq" +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc + +_supported_os Linux +_require_scratch + +echo "Format and mount" +_scratch_mkfs > $seqres.full 2>&1 +_scratch_mount >> $seqres.full 2>&1 + +testdir="${SCRATCH_MNT}/test-${seq}" +mkdir $testdir + +hexbytes() { + echo -n "$1" | od -tx1 -w99999 | head -n1 | sed -e 's/^0* //g' +} + +setf() { + key="$(echo -e "$1")" + value="$2" + + echo "${value}" > "${testdir}/${key}" + echo "Storing ${key} ($(hexbytes "${key}")) -> ${value}" >> $seqres.full +} + +testf() { + key="$(echo -e "$1")" + value="$2" + fname="${testdir}/${key}" + + echo "Testing ${key} ($(hexbytes "${key}")) -> ${value}" >> $seqres.full + + if [ ! -e "${fname}" ]; then + echo "Key ${key} does not exist for ${value} test??" + return + fi + + actual_value="$(cat "${fname}")" + if [ "${actual_value}" != "${value}" ]; then + echo "Key ${key} has value ${value}, expected ${actual_value}." + fi +} + +filter_scrub() { + grep 'Unicode' | sed -e 's/^.*Duplicate/Duplicate/g' +} + +echo "Create files" +# These two render the same +setf "french_caf\xc3\xa9.txt" "NFC" +setf "french_cafe\xcc\x81.txt" "NFD" + +# These two may have different widths +setf "chinese_\xef\xbd\xb6.txt" "NFKC1" +setf "chinese_\xe3\x82\xab.txt" "NFKC2" + +# Same point, different byte representations in NFC/NFD/NFKC/NFKD +setf "greek_\xcf\x93.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFC" +setf "greek_\xcf\x92\xcc\x81.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFD" +setf "greek_\xce\x8e.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFKC" +setf "greek_\xce\xa5\xcc\x81.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFKD" + +# Arabic code point can expand into a muuuch longer series +setf "arabic_\xef\xb7\xba.txt" "ARABIC LIGATURE SALLALLAHOU ALAYHE WASALLAM, NFC" +setf "arabic_\xd8\xb5\xd9\x84\xd9\x89\x20\xd8\xa7\xd9\x84\xd9\x84\xd9\x87\x20\xd8\xb9\xd9\x84\xd9\x8a\xd9\x87\x20\xd9\x88\xd8\xb3\xd9\x84\xd9\x85.txt" "ARABIC LIGATURE SALLALLAHOU ALAYHE WASALLAM, NFKC" + +# Fake slash? +setf "urk\xc0\xafmoo" "FAKESLASH" + +ls -la $testdir >> $seqres.full + +echo "Test files" +testf "french_caf\xc3\xa9.txt" "NFC" +testf "french_cafe\xcc\x81.txt" "NFD" + +testf "chinese_\xef\xbd\xb6.txt" "NFKC1" +testf "chinese_\xe3\x82\xab.txt" "NFKC2" + +testf "greek_\xcf\x93.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFC" +testf "greek_\xcf\x92\xcc\x81.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFD" +testf "greek_\xce\x8e.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFKC" +testf "greek_\xce\xa5\xcc\x81.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFKD" + +testf "arabic_\xef\xb7\xba.txt" "ARABIC LIGATURE SALLALLAHOU ALAYHE WASALLAM, NFC" +testf "arabic_\xd8\xb5\xd9\x84\xd9\x89\x20\xd8\xa7\xd9\x84\xd9\x84\xd9\x87\x20\xd8\xb9\xd9\x84\xd9\x8a\xd9\x87\x20\xd9\x88\xd8\xb3\xd9\x84\xd9\x85.txt" "ARABIC LIGATURE SALLALLAHOU ALAYHE WASALLAM, NFKC" + +testf "urk\xc0\xafmoo" "FAKESLASH" + +echo "Uniqueness of inodes?" +stat -c '%i' "${testdir}/"* | sort | uniq -c | while read nr inum; do + if [ "${nr}" -gt 1 ]; then + echo "${nr} ${inum}" + fi +done + +echo "Test XFS online scrub, if applicable" +# Only run this on xfs if xfs_scrub is available and has the unicode checker +if [ "${FSTYP}" = "xfs" ] && \ + [ -x "${XFS_SCRUB_PROG}" ] && \ + type ldd > /dev/null 2>&1 && ldd "${XFS_SCRUB_PROG}" | grep -q libunistring; then + output="$(${XFS_SCRUB_PROG} -n "${SCRATCH_MNT}" 2>&1 | filter_scrub)" + echo "${output}" | grep -q "french_" || echo "No complaints about french e accent?" + echo "${output}" | grep -q "chinese_" || echo "No complaints about chinese width-different?" + echo "${output}" | grep -q "greek_" || echo "No complaints about greek letter mess?" + echo "${output}" | grep -q "arabic_" || echo "No complaints about arabic expanded string?" + echo "Actual xfs_scrub output:" >> $seqres.full + echo "${output}" >> $seqres.full +fi + +# success, all done +status=0 +exit diff --git a/tests/generic/703.out b/tests/generic/703.out new file mode 100644 index 0000000..f46b1c6 --- /dev/null +++ b/tests/generic/703.out @@ -0,0 +1,6 @@ +QA output created by 703 +Format and mount +Create files +Test files +Uniqueness of inodes? +Test XFS online scrub, if applicable diff --git a/tests/generic/group b/tests/generic/group index 044ec3f..2255865 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -453,3 +453,4 @@ 448 auto quick rw 449 auto quick acl enospc 450 auto quick rw +703 auto quick dir ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v2 4/4] generic: try various unicode normalization games 2017-08-30 4:41 ` [PATCH 4/4] generic: try various unicode normalization games Darrick J. Wong @ 2017-08-30 22:52 ` Darrick J. Wong 0 siblings, 0 replies; 13+ messages in thread From: Darrick J. Wong @ 2017-08-30 22:52 UTC (permalink / raw) To: eguan; +Cc: linux-xfs, fstests Linux filesystems generally treat filenames and extended attribute keys as a bag of bytes, which means that there can be unique sequences of bytes that render the same on most modern GUIs. So, let's rig up a test to see if it's really true that we can create filenames and xattrs that look the same but point to different files. xfs_scrub will warn about these kinds of situations, though they're not technically fs "corruption". Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> --- v2: might as well test xattrs too --- tests/generic/703 | 171 +++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/703.out | 6 ++ tests/generic/704 | 167 ++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/704.out | 6 ++ tests/generic/group | 2 + 5 files changed, 352 insertions(+) create mode 100755 tests/generic/703 create mode 100644 tests/generic/703.out create mode 100755 tests/generic/704 create mode 100644 tests/generic/704.out diff --git a/tests/generic/703 b/tests/generic/703 new file mode 100755 index 0000000..a9cd245 --- /dev/null +++ b/tests/generic/703 @@ -0,0 +1,171 @@ +#! /bin/bash +# FS QA Test No. 703 +# +# Create a directory with multiple filenames that all appear the same +# (in unicode, anyway) but point to different inodes. In theory all +# Linux filesystems should allow this (filenames are a sequence of +# arbitrary bytes) even if the user implications are horrifying. +# +#----------------------------------------------------------------------- +# Copyright (c) 2017, Oracle and/or its affiliates. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- + +seq=`basename "$0"` +seqres="$RESULT_DIR/$seq" +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc + +_supported_os Linux +_require_scratch + +echo "Format and mount" +_scratch_mkfs > $seqres.full 2>&1 +_scratch_mount >> $seqres.full 2>&1 + +testdir="${SCRATCH_MNT}/test-${seq}" +mkdir $testdir + +hexbytes() { + echo -n "$1" | od -tx1 -w99999 | head -n1 | sed -e 's/^0* //g' +} + +setf() { + key="$(echo -e "$1")" + value="$2" + + echo "${value}" > "${testdir}/${key}" + echo "Storing ${key} ($(hexbytes "${key}")) -> ${value}" >> $seqres.full +} + +testf() { + key="$(echo -e "$1")" + value="$2" + fname="${testdir}/${key}" + + echo "Testing ${key} ($(hexbytes "${key}")) -> ${value}" >> $seqres.full + + if [ ! -e "${fname}" ]; then + echo "Key ${key} does not exist for ${value} test??" + return + fi + + actual_value="$(cat "${fname}")" + if [ "${actual_value}" != "${value}" ]; then + echo "Key ${key} has value ${value}, expected ${actual_value}." + fi +} + +filter_scrub() { + grep 'Unicode' | sed -e 's/^.*Duplicate/Duplicate/g' +} + +echo "Create files" +# These two render the same +setf "french_caf\xc3\xa9.txt" "NFC" +setf "french_cafe\xcc\x81.txt" "NFD" + +# These two may have different widths +setf "chinese_\xef\xbd\xb6.txt" "NFKC1" +setf "chinese_\xe3\x82\xab.txt" "NFKC2" + +# Same point, different byte representations in NFC/NFD/NFKC/NFKD +setf "greek_\xcf\x93.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFC" +setf "greek_\xcf\x92\xcc\x81.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFD" +setf "greek_\xce\x8e.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFKC" +setf "greek_\xce\xa5\xcc\x81.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFKD" + +# Arabic code point can expand into a muuuch longer series +setf "arabic_\xef\xb7\xba.txt" "ARABIC LIGATURE SALLALLAHOU ALAYHE WASALLAM, NFC" +setf "arabic_\xd8\xb5\xd9\x84\xd9\x89\x20\xd8\xa7\xd9\x84\xd9\x84\xd9\x87\x20\xd8\xb9\xd9\x84\xd9\x8a\xd9\x87\x20\xd9\x88\xd8\xb3\xd9\x84\xd9\x85.txt" "ARABIC LIGATURE SALLALLAHOU ALAYHE WASALLAM, NFKC" + +# Fake slash? +setf "urk\xc0\xafmoo" "FAKESLASH" + +ls -la $testdir >> $seqres.full + +echo "Test files" +testf "french_caf\xc3\xa9.txt" "NFC" +testf "french_cafe\xcc\x81.txt" "NFD" + +testf "chinese_\xef\xbd\xb6.txt" "NFKC1" +testf "chinese_\xe3\x82\xab.txt" "NFKC2" + +testf "greek_\xcf\x93.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFC" +testf "greek_\xcf\x92\xcc\x81.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFD" +testf "greek_\xce\x8e.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFKC" +testf "greek_\xce\xa5\xcc\x81.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFKD" + +testf "arabic_\xef\xb7\xba.txt" "ARABIC LIGATURE SALLALLAHOU ALAYHE WASALLAM, NFC" +testf "arabic_\xd8\xb5\xd9\x84\xd9\x89\x20\xd8\xa7\xd9\x84\xd9\x84\xd9\x87\x20\xd8\xb9\xd9\x84\xd9\x8a\xd9\x87\x20\xd9\x88\xd8\xb3\xd9\x84\xd9\x85.txt" "ARABIC LIGATURE SALLALLAHOU ALAYHE WASALLAM, NFKC" + +testf "urk\xc0\xafmoo" "FAKESLASH" + +echo "Uniqueness of inodes?" +stat -c '%i' "${testdir}/"* | sort | uniq -c | while read nr inum; do + if [ "${nr}" -gt 1 ]; then + echo "${nr} ${inum}" + fi +done + +echo "Test XFS online scrub, if applicable" + +# Only run this on xfs if xfs_scrub is available and has the unicode checker +check_xfs_scrub() { + # Ignore non-XFS fs or no scrub program... + if [ "${FSTYP}" != "xfs" ] || [ ! -x "${XFS_SCRUB_PROG}" ]; then + return 1 + fi + + # We only care if xfs_scrub has unicode string support... + if ! type ldd > /dev/null 2>&1 || \ + ! ldd "${XFS_SCRUB_PROG}" | grep -q libunistring; then + return 1 + fi + + # Does the ioctl work? + if $XFS_IO_PROG -x -c "scrub test 0" $SCRATCH_MNT 2>&1 | \ + grep -q "Inappropriate ioctl"; then + return 1 + fi + + return 0 +} + +if check_xfs_scrub; then + output="$(${XFS_SCRUB_PROG} -n "${SCRATCH_MNT}" 2>&1 | filter_scrub)" + echo "${output}" | grep -q "french_" || echo "No complaints about french e accent?" + echo "${output}" | grep -q "chinese_" || echo "No complaints about chinese width-different?" + echo "${output}" | grep -q "greek_" || echo "No complaints about greek letter mess?" + echo "${output}" | grep -q "arabic_" || echo "No complaints about arabic expanded string?" + echo "Actual xfs_scrub output:" >> $seqres.full + echo "${output}" >> $seqres.full +fi + +# success, all done +status=0 +exit diff --git a/tests/generic/703.out b/tests/generic/703.out new file mode 100644 index 0000000..f46b1c6 --- /dev/null +++ b/tests/generic/703.out @@ -0,0 +1,6 @@ +QA output created by 703 +Format and mount +Create files +Test files +Uniqueness of inodes? +Test XFS online scrub, if applicable diff --git a/tests/generic/704 b/tests/generic/704 new file mode 100755 index 0000000..6431848 --- /dev/null +++ b/tests/generic/704 @@ -0,0 +1,167 @@ +#! /bin/bash +# FS QA Test No. 704 +# +# Create xattrs with multiple keys that all appear the same +# (in unicode, anyway) but point to different values. In theory all +# Linux filesystems should allow this (filenames are a sequence of +# arbitrary bytes) even if the user implications are horrifying. +# +#----------------------------------------------------------------------- +# Copyright (c) 2017, Oracle and/or its affiliates. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- + +seq=`basename "$0"` +seqres="$RESULT_DIR/$seq" +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/attr + +_supported_os Linux +_require_scratch +_require_attrs + +echo "Format and mount" +_scratch_mkfs > $seqres.full 2>&1 +_scratch_mount >> $seqres.full 2>&1 + +testdir="${SCRATCH_MNT}/test-${seq}" +mkdir $testdir +testfile="${testdir}/attrfile" +touch "${testfile}" + +hexbytes() { + echo -n "$1" | od -tx1 -w99999 | head -n1 | sed -e 's/^0* //g' +} + +setf() { + key="$(echo -e "$1")" + value="$2" + + $SETFATTR_PROG -n "user.${key}" -v "${value}" "${testfile}" + echo "Storing ${key} ($(hexbytes "${key}")) -> ${value}" >> $seqres.full +} + +testf() { + key="$(echo -e "$1")" + value="$2" + + echo "Testing ${key} ($(hexbytes "${key}")) -> ${value}" >> $seqres.full + + actual_value="$($GETFATTR_PROG --absolute-names --only-values -n "user.${key}" "${testfile}")" + if [ "${actual_value}" != "${value}" ]; then + echo "Key ${key} has value ${actual_value}, expected ${value}." + fi +} + +filter_scrub() { + grep 'Unicode' | sed -e 's/^.*Duplicate/Duplicate/g' +} + +echo "Create files" +# These two render the same +setf "french_caf\xc3\xa9.txt" "NFC" +setf "french_cafe\xcc\x81.txt" "NFD" + +# These two may have different widths +setf "chinese_\xef\xbd\xb6.txt" "NFKC1" +setf "chinese_\xe3\x82\xab.txt" "NFKC2" + +# Same point, different byte representations in NFC/NFD/NFKC/NFKD +setf "greek_\xcf\x93.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFC" +setf "greek_\xcf\x92\xcc\x81.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFD" +setf "greek_\xce\x8e.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFKC" +setf "greek_\xce\xa5\xcc\x81.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFKD" + +# Arabic code point can expand into a muuuch longer series +setf "arabic_\xef\xb7\xba.txt" "ARABIC LIGATURE SALLALLAHOU ALAYHE WASALLAM, NFC" +setf "arabic_\xd8\xb5\xd9\x84\xd9\x89\x20\xd8\xa7\xd9\x84\xd9\x84\xd9\x87\x20\xd8\xb9\xd9\x84\xd9\x8a\xd9\x87\x20\xd9\x88\xd8\xb3\xd9\x84\xd9\x85.txt" "ARABIC LIGATURE SALLALLAHOU ALAYHE WASALLAM, NFKC" + +# Fake slash? +setf "urk\xc0\xafmoo" "FAKESLASH" + +$GETFATTR_PROG --absolute-names -d "${testfile}" >> $seqres.full + +echo "Test files" +testf "french_caf\xc3\xa9.txt" "NFC" +testf "french_cafe\xcc\x81.txt" "NFD" + +testf "chinese_\xef\xbd\xb6.txt" "NFKC1" +testf "chinese_\xe3\x82\xab.txt" "NFKC2" + +testf "greek_\xcf\x93.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFC" +testf "greek_\xcf\x92\xcc\x81.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFD" +testf "greek_\xce\x8e.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFKC" +testf "greek_\xce\xa5\xcc\x81.txt" "GREEK UPSILON WITH ACUTE AND HOOK SYMBOL, NFKD" + +testf "arabic_\xef\xb7\xba.txt" "ARABIC LIGATURE SALLALLAHOU ALAYHE WASALLAM, NFC" +testf "arabic_\xd8\xb5\xd9\x84\xd9\x89\x20\xd8\xa7\xd9\x84\xd9\x84\xd9\x87\x20\xd8\xb9\xd9\x84\xd9\x8a\xd9\x87\x20\xd9\x88\xd8\xb3\xd9\x84\xd9\x85.txt" "ARABIC LIGATURE SALLALLAHOU ALAYHE WASALLAM, NFKC" + +testf "urk\xc0\xafmoo" "FAKESLASH" + +echo "Uniqueness of keys?" +crazy_keys="$($GETFATTR_PROG --absolute-names -d "${testfile}" | egrep -c '(french_|chinese_|greek_|arabic_|urk)')" +expected_keys=11 +test "${crazy_keys}" -ne "${expected_keys}" && echo "Expected ${expected_keys} keys, saw ${crazy_keys}." + +echo "Test XFS online scrub, if applicable" + +# Only run this on xfs if xfs_scrub is available and has the unicode checker +check_xfs_scrub() { + # Ignore non-XFS fs or no scrub program... + if [ "${FSTYP}" != "xfs" ] || [ ! -x "${XFS_SCRUB_PROG}" ]; then + return 1 + fi + + # We only care if xfs_scrub has unicode string support... + if ! type ldd > /dev/null 2>&1 || \ + ! ldd "${XFS_SCRUB_PROG}" | grep -q libunistring; then + return 1 + fi + + # Does the ioctl work? + if $XFS_IO_PROG -x -c "scrub test 0" $SCRATCH_MNT 2>&1 | \ + grep -q "Inappropriate ioctl"; then + return 1 + fi + + return 0 +} + +if check_xfs_scrub; then + output="$(${XFS_SCRUB_PROG} -n "${SCRATCH_MNT}" 2>&1 | filter_scrub)" + echo "${output}" | grep -q "french_" || echo "No complaints about french e accent?" + echo "${output}" | grep -q "chinese_" || echo "No complaints about chinese width-different?" + echo "${output}" | grep -q "greek_" || echo "No complaints about greek letter mess?" + echo "${output}" | grep -q "arabic_" || echo "No complaints about arabic expanded string?" + echo "Actual xfs_scrub output:" >> $seqres.full + echo "${output}" >> $seqres.full +fi + +# success, all done +status=0 +exit diff --git a/tests/generic/704.out b/tests/generic/704.out new file mode 100644 index 0000000..6990019 --- /dev/null +++ b/tests/generic/704.out @@ -0,0 +1,6 @@ +QA output created by 704 +Format and mount +Create files +Test files +Uniqueness of keys? +Test XFS online scrub, if applicable diff --git a/tests/generic/group b/tests/generic/group index 044ec3f..d91e083 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -453,3 +453,5 @@ 448 auto quick rw 449 auto quick acl enospc 450 auto quick rw +703 auto quick dir +704 auto quick attr ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 5/4] generic/173: don't dump core when mwrite fails 2017-08-30 4:40 [PATCH 1/4] xfs/{319,323}: don't checksum files after log recovery Darrick J. Wong ` (2 preceding siblings ...) 2017-08-30 4:41 ` [PATCH 4/4] generic: try various unicode normalization games Darrick J. Wong @ 2017-08-30 22:31 ` Darrick J. Wong 2017-08-30 22:31 ` [PATCH 6/4] common/rc: fix xfs_io scrub command existence test Darrick J. Wong 2017-08-30 22:32 ` [PATCH 7/4] xfs/122: fix the size of fsop_ag_resblks structure Darrick J. Wong 5 siblings, 0 replies; 13+ messages in thread From: Darrick J. Wong @ 2017-08-30 22:31 UTC (permalink / raw) To: eguan; +Cc: linux-xfs, fstests In generic/173, we try to force a CoW to a mmap'd region to fail if there's no space to actually stage the CoW operation. That failure comes in the form of a SIGBUS to xfs_io. If the tester just happens to have a nonzero coresize ulimit set, a core dump is generated and the test is marked as having failed, even though the dump generation is exactly the correct behavior. Therefore, set the coresize ulimit to zero while calling _mwrite_byte. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> --- tests/generic/173 | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/generic/173 b/tests/generic/173 index ad4e5da..526590b 100755 --- a/tests/generic/173 +++ b/tests/generic/173 @@ -81,6 +81,8 @@ _fill_fs $((blksz * nr_free)) $testdir/space $blksz 0 >> $seqres.full 2>&1 sync echo "mmap CoW the big file" +core_ulimit="$(ulimit -c)" +ulimit -c 0 out="$(_mwrite_byte 0x62 0 $((blksz * nr_blks)) $((blksz * nr_blks)) $testdir/bigfile 2>&1)" err="$?" if [ $err -lt 128 ]; then @@ -95,6 +97,7 @@ err="$?" if [ $err -lt 128 ]; then echo "mmap CoW should have failed with SIGBUS, got SIG$(kill -l $err)" fi +ulimit -c "${core_ulimit}" # success, all done status=0 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 6/4] common/rc: fix xfs_io scrub command existence test 2017-08-30 4:40 [PATCH 1/4] xfs/{319,323}: don't checksum files after log recovery Darrick J. Wong ` (3 preceding siblings ...) 2017-08-30 22:31 ` [PATCH 5/4] generic/173: don't dump core when mwrite fails Darrick J. Wong @ 2017-08-30 22:31 ` Darrick J. Wong 2017-08-30 22:32 ` [PATCH 7/4] xfs/122: fix the size of fsop_ag_resblks structure Darrick J. Wong 5 siblings, 0 replies; 13+ messages in thread From: Darrick J. Wong @ 2017-08-30 22:31 UTC (permalink / raw) To: eguan; +Cc: linux-xfs, fstests The name of the xfs_io scrub subcommand to test for the existence of the ioctl has been changed to 'test' from 'dummy', so fix xfstests. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> --- common/rc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/common/rc b/common/rc index b4f0b32..ea58736 100644 --- a/common/rc +++ b/common/rc @@ -2203,7 +2203,7 @@ _require_xfs_io_command() _notrun "xfs_io $command support is missing" ;; "scrub"|"repair") - testio=`$XFS_IO_PROG -x -c "$command dummy 0" $TEST_DIR 2>&1` + testio=`$XFS_IO_PROG -x -c "$command test 0" $TEST_DIR 2>&1` echo $testio | egrep -q "Inappropriate ioctl" && \ _notrun "xfs_io $command support is missing" ;; ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 7/4] xfs/122: fix the size of fsop_ag_resblks structure 2017-08-30 4:40 [PATCH 1/4] xfs/{319,323}: don't checksum files after log recovery Darrick J. Wong ` (4 preceding siblings ...) 2017-08-30 22:31 ` [PATCH 6/4] common/rc: fix xfs_io scrub command existence test Darrick J. Wong @ 2017-08-30 22:32 ` Darrick J. Wong 5 siblings, 0 replies; 13+ messages in thread From: Darrick J. Wong @ 2017-08-30 22:32 UTC (permalink / raw) To: eguan; +Cc: linux-xfs, fstests The size of the structure used to retrieve per-AG reserved blocks status has changed (it's not in a released upstream), so update xfs/122. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> --- tests/xfs/122.out | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/xfs/122.out b/tests/xfs/122.out index 168cfd5..9ca30b7 100644 --- a/tests/xfs/122.out +++ b/tests/xfs/122.out @@ -81,7 +81,7 @@ sizeof(struct xfs_dsymlink_hdr) = 56 sizeof(struct xfs_extent_data) = 24 sizeof(struct xfs_extent_data_info) = 32 sizeof(struct xfs_fs_eofblocks) = 128 -sizeof(struct xfs_fsop_ag_resblks) = 32 +sizeof(struct xfs_fsop_ag_resblks) = 64 sizeof(struct xfs_icreate_log) = 28 sizeof(struct xfs_log_dinode) = 176 sizeof(struct xfs_map_extent) = 32 ^ permalink raw reply related [flat|nested] 13+ messages in thread
end of thread, other threads:[~2017-08-31 14:47 UTC | newest] Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-08-30 4:40 [PATCH 1/4] xfs/{319,323}: don't checksum files after log recovery Darrick J. Wong 2017-08-30 4:40 ` [PATCH 2/4] xfs/013: exit cleaner thread if fsstress dies Darrick J. Wong 2017-08-30 7:51 ` Christoph Hellwig 2017-08-30 4:41 ` [PATCH 3/4] xfs: test rmapbt updates are correct with insert/collapse range Darrick J. Wong 2017-08-30 7:53 ` Christoph Hellwig 2017-08-30 22:58 ` Darrick J. Wong 2017-08-31 10:28 ` Eryu Guan 2017-08-31 14:47 ` [PATCH v2 " Darrick J. Wong 2017-08-30 4:41 ` [PATCH 4/4] generic: try various unicode normalization games Darrick J. Wong 2017-08-30 22:52 ` [PATCH v2 " Darrick J. Wong 2017-08-30 22:31 ` [PATCH 5/4] generic/173: don't dump core when mwrite fails Darrick J. Wong 2017-08-30 22:31 ` [PATCH 6/4] common/rc: fix xfs_io scrub command existence test Darrick J. Wong 2017-08-30 22:32 ` [PATCH 7/4] xfs/122: fix the size of fsop_ag_resblks structure Darrick J. Wong
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.