running xfstests on tmpfs

running xfstests on tmpfs

[Hugh Dickins]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1978 bytes --]

On Thu, 26 Nov 2020, Matthew Wilcox wrote:
> 
> The good news is that I've sorted out the SCRATCH_DEV issue with
> running xfstests.  The bad news is that (even on an unmodified kernel),
> generic/027 takes 19 hours to run.  On xfs, it's 4 minutes.  Any idea
> why it's so slow on tmpfs?

It's to slow you down, to give me a better chance of keeping up.

But seriously, I suppose I ought to hand over the four xfstests
patches I add: let me change the Subject and attach a tarfile of them.

The problem with generic/027, if I'm recalling aright, is that it
(not unreasonably) expects the inodes to be limited by filesystem size,
whereas tmpfs has independent size (or nr_blocks) and nr_inodes limits.

I keep a six-year-old patch to make generic/027 more reasonable; but
a month ago switched over to limiting my nr_inodes better.  Maybe the
recent patch makes the six-year-old one unnecessary, but I've had
bigger fish to worry about than check into that.

The script I use to run xfstests on huge tmpfs says:

export FSTYP=tmpfs
export DISABLE_UDF_TEST=1
export TEST_DEV=tmpfs1:
export TEST_DIR=/xft
export SCRATCH_DEV=tmpfs2:
export SCRATCH_MNT=/mnt
export TMPFS_MOUNT_OPTIONS="-o size=1088M,nr_inodes=272K,huge=always"

# mkfs -t $FSTYP $TEST_DEV || exit $?
mount -t $FSTYP $TMPFS_MOUNT_OPTIONS $TEST_DEV $TEST_DIR || exit $?
# mkfs -t $FSTYP $SCRATCH_DEV || exit $?
# mount -t $FSTYP $SCRATCH_DEV $SCRATCH_DIR || exit $?
./check -b "$@" # typically "-g auto"
umount /xft /mnt 2>/dev/null

And the one for non-huge tmpfs is the same except for "huge=never".
There is then some repetition in mount options shown when running
the tests, but I've not spent time to clean that up.

I'm very pleased for you to run them differently,
so we may find different things :)

generic/080 does fail on tmpfs (no page_mkwrite), and about three
more also fail on huge tmpfs (perhaps hole/data reasons); some more
fail on other test machines, I can't give a full account at the moment.

Hugh

[-- Attachment #2: Type: APPLICATION/x-tar, Size: 20480 bytes --]

Re: running xfstests on tmpfs

[Hugh Dickins]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3208 bytes --]

On Thu, 26 Nov 2020, Hugh Dickins wrote:
> On Thu, 26 Nov 2020, Matthew Wilcox wrote:
> > 
> > The good news is that I've sorted out the SCRATCH_DEV issue with
> > running xfstests.  The bad news is that (even on an unmodified kernel),
> > generic/027 takes 19 hours to run.  On xfs, it's 4 minutes.  Any idea
> > why it's so slow on tmpfs?
> 
> It's to slow you down, to give me a better chance of keeping up.
> 
> But seriously, I suppose I ought to hand over the four xfstests
> patches I add: let me change the Subject and attach a tarfile of them.
> 
> The problem with generic/027, if I'm recalling aright, is that it
> (not unreasonably) expects the inodes to be limited by filesystem size,
> whereas tmpfs has independent size (or nr_blocks) and nr_inodes limits.
> 
> I keep a six-year-old patch to make generic/027 more reasonable; but
> a month ago switched over to limiting my nr_inodes better.  Maybe the
> recent patch makes the six-year-old one unnecessary, but I've had
> bigger fish to worry about than check into that.
> 
> The script I use to run xfstests on huge tmpfs says:
> 
> export FSTYP=tmpfs
> export DISABLE_UDF_TEST=1
> export TEST_DEV=tmpfs1:
> export TEST_DIR=/xft
> export SCRATCH_DEV=tmpfs2:
> export SCRATCH_MNT=/mnt
> export TMPFS_MOUNT_OPTIONS="-o size=1088M,nr_inodes=272K,huge=always"
> 
> # mkfs -t $FSTYP $TEST_DEV || exit $?
> mount -t $FSTYP $TMPFS_MOUNT_OPTIONS $TEST_DEV $TEST_DIR || exit $?
> # mkfs -t $FSTYP $SCRATCH_DEV || exit $?
> # mount -t $FSTYP $SCRATCH_DEV $SCRATCH_DIR || exit $?
> ./check -b "$@" # typically "-g auto"
> umount /xft /mnt 2>/dev/null
> 
> And the one for non-huge tmpfs is the same except for "huge=never".
> There is then some repetition in mount options shown when running
> the tests, but I've not spent time to clean that up.
> 
> I'm very pleased for you to run them differently,
> so we may find different things :)
> 
> generic/080 does fail on tmpfs (no page_mkwrite), and about three
> more also fail on huge tmpfs (perhaps hole/data reasons); some more
> fail on other test machines, I can't give a full account at the moment.

Attached is my latest xfstests-tmpfs.tar, with three further patches:

[PATCH 1/7] xfstests: fix generic/027 on tmpfs, stop so many empties
[PATCH 2/7] xfstests: increase tmpfs memory size to 1088M
[PATCH 3/7] xfstests: generic/449 not run on tmpfs earlier
[PATCH 4/7] xfstests: limit tmpfs nr_inodes to one per 4kB of size
[PATCH 5/7] xfstests: fsx: delete prealloc's platform_test_xfs_fd()
[PATCH 6/7] xfstests: generic/591 _require_odirect
[PATCH 7/7] xfstests: generic/100 cleanup /tmp/populated_root

I wouldn't bother you with an update, except for 5/7 the fsx one:
generic/075 and generic/112 have been very useful for stressing
your changes and my mods to them, but there's some kind of difference
in the build environment on my test machines, such that only one was
running those tests far enough to find the problems, failing early
on the others.  You may or may not suffer from the same problem.

Yes, of course I should mail that fix and others uncontroversial to
the xfstests guys, but I want to review 1-4 (decide whether hack or
not), and our testing is my priority right now.

Hugh

[-- Attachment #2: Type: APPLICATION/x-tar, Size: 30720 bytes --]