[PATCH] common: add support for the "local" file system type

[PATCH] common: add support for the "local" file system type

[Theodore Y. Ts'o]

About a year and half ago, I sent the patch attached below to add
support for a "local" file system type where the file system could not
be mounted or dismounted, but where writes to TEST_DIR and SCRATCH_MNT
would be testing the file system in question.  At the time, I couldn't
describe the use case in any greater detail than what I had in the
commit description, and Dave Chinner at the time rejected the patch
since he didn't like xfstests being used to test a proprietary file
system for which I was not able to explain the details of what we were
trying to do.

Not a big deal, I just kept the patch in my private fork[1] of
xfstests on github.

[1] https://github.com/tytso/xfstests

However, happily, we can now talk a lot more about what the "local"
file system type in xfstests was used to test.  Earlier today, Google
announced gVisor[2], and published it as open source on github[3].
gVisor works much like User Mode Linux, and I suspect much like the
Windows Subsystem for Linux in that it uses the x86_64's hardware
virtualization extensions (so it has the security fencing much like a
VM) but instead of emulating hardware, instead the emulation layer is
done at the system call level (so it's more efficient than a VM, since
there is no guest kernel).

[2] https://cloudplatform.googleblog.com/2018/05/Open-sourcing-gVisor-a-sandboxed-container-runtime.html
[3] https://github.com/google/gvisor

File systems can be implemented using Gophers[4] in a separate
process, where the communication between the gVisor sandbox and the
Gopher is via the 9P2000.L protocol.  This means that if you want to
try to exploit a buffer overflow in the userspace file system, first
you have to get past the system call validation checks done by the
gVisor Sentry process (which is written in Go which makes this rather
more difficult, especially since the Sentry process itself is
protected using seccomp[5]).  Then the buffer overflow attack has to
make it past the 9P protocol encoding/decoding, and then survive the
9P protocol validation checks in the Gopher.  The Gopher process
provides an emulated file system service using the Cloud Provider's
internal cluster storage services, much like in GCE, the Persistent
Disk service provides an emulated block device service.

[4] https://news.ycombinator.com/item?id=16979126
[5] https://news.ycombinator.com/item?id=16976557

This whole system was designed with security first and foremost.  Of
course, we also want it to pass the file system checks, which is why
were using xfstests.

The way we actually tested the gVisor file system was via a Docker
container (the Dockerfile[6] is in the xfstests-bld git repo) which
would then get consumed by gVisor's Docker/Kubernetes integration
layer.

[6] https://github.com/tytso/xfstests-bld/blob/master/Dockerfile

Anyway, back in September, 2016, Dave Chinner was peeved that I
couldn't give this full description, and I'm glad to say, now we can
finally rectify that gap.  :-)

Could this commit therefore please be considered for inclusion in
xfstests upstream?

Many thanks!

						- Ted


>From 8b40b28866dca119d0c807c31ae48f153ec2dc53 Mon Sep 17 00:00:00 2001
From: Theodore Ts'o <tytso@mit.edu>
Date: Sat, 17 Sep 2016 19:08:18 -0400
Subject: [PATCH] common: add support for the "local" file system type

It is sometimes useful to be able to test the local file system
provided in a restricted execution environment (such as that which is
provided by Docker, for example) where it is not possible to mount and
unmount the file system under test.

To support this test case, add support for a new file system type
called "local".  The TEST_DEV and SCRATCH_DEV should be have a
non-block device format (e.g., local:/test or local:/scratch), and the
TEST_DIR and SCRATCH_MNT directories should be pre-existing
directories provided by the execution environment.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
 common/rc | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 70 insertions(+), 3 deletions(-)

diff --git a/common/rc b/common/rc
index 9ffab7fd..d5cb0fe4 100644
--- a/common/rc
+++ b/common/rc
@@ -351,6 +351,18 @@ _supports_filetype()
 	esac
 }
 
+_local_validate_mount_opt()
+{
+    case "$*" in
+	ro|ro,*|*,ro) _notrun "ro mount option not supported" ;;
+	*nosuid*) _notrun "nosuid mount option not supported" ;;
+	*noatime*) _notrun "noatime mount option not supported" ;;
+	*relatime*) _notrun "relatime mount option not supported" ;;
+	*diratime*) _notrun "diratime mount option not supported" ;;
+	*strictatime*) _notrun "strictatime mount option not supported" ;;
+    esac
+}
+
 # mount scratch device with given options but don't check mount status
 _try_scratch_mount()
 {
@@ -376,6 +388,9 @@ _scratch_unmount()
 	btrfs)
 		$UMOUNT_PROG $SCRATCH_MNT
 		;;
+	local)
+                rm -rf $SCRATCH_MNT/*
+                ;;
 	*)
 		$UMOUNT_PROG $SCRATCH_DEV
 		;;
@@ -386,6 +401,10 @@ _scratch_remount()
 {
     local opts="$1"
 
+    if [ "$FSTYP" = "local" ]; then
+	_local_validate_mount_opt "$*"
+	return 0;
+    fi
     if test -n "$opts"; then
 	mount -o "remount,$opts" $SCRATCH_MNT
     fi
@@ -395,7 +414,7 @@ _scratch_cycle_mount()
 {
     local opts="$1"
 
-    if [ "$FSTYP" = tmpfs ]; then
+    if [ "$FSTYP" = "tmpfs" -o "$FSTYP" = "local" ] ; then
 	_scratch_remount "$opts"
 	return
     fi
@@ -429,6 +448,10 @@ _test_mount()
         _overlay_test_mount $*
         return $?
     fi
+    if [ "$FSTYP" == "local" ]; then
+        mkdir -p $TEST_DIR
+	return $?
+    fi
     _test_options mount
     _mount -t $FSTYP $TEST_OPTIONS $TEST_FS_MOUNT_OPTS $SELINUX_MOUNT_OPTIONS $* $TEST_DEV $TEST_DIR
 }
@@ -437,7 +460,7 @@ _test_unmount()
 {
 	if [ "$FSTYP" == "overlay" ]; then
 		_overlay_test_unmount
-	else
+	elif [ "$FSTYP" != "local" ]; then
 		$UMOUNT_PROG $TEST_DEV
 	fi
 }
@@ -723,7 +746,7 @@ _scratch_mkfs()
 	local mkfs_status
 
 	case $FSTYP in
-	nfs*|cifs|ceph|overlay|glusterfs|pvfs2|9p)
+	nfs*|cifs|ceph|overlay|glusterfs|pvfs2|9p|local)
 		# unable to re-create this fstyp, just remove all files in
 		# $SCRATCH_MNT to avoid EEXIST caused by the leftover files
 		# created in previous runs
@@ -1465,6 +1488,10 @@ _check_mounted_on()
 	local mnt=$4
 	local type=$5
 
+	if [ "$FSTYP" == "local" ]; then
+		return 0
+	fi
+
 	# find $dev as the source, and print result in "$dev $mnt" format
 	local mount_rec=`findmnt -rncv -S $dev -o SOURCE,TARGET`
 	[ -n "$mount_rec" ] || return 1 # 1 = not mounted
@@ -1562,6 +1589,13 @@ _require_scratch_nocheck()
 			_notrun "this test requires a valid \$SCRATCH_MNT"
 		fi
 		;;
+	local)
+		if [ -z "$SCRATCH_DEV" -o ! -d "$SCRATCH_MNT" ];
+		then
+		    _notrun "this test requires a valid \$SCRATCH_MNT and unique $SCRATCH_DEV"
+		fi
+		return 0
+		;;
 	*)
 		 if [ -z "$SCRATCH_DEV" -o "`_is_block_dev "$SCRATCH_DEV"`" = "" ]
 		 then
@@ -1683,6 +1717,13 @@ _require_test()
 			_notrun "this test requires a valid \$TEST_DIR"
 		fi
 		;;
+	local)
+		if [ -z "$TEST_DEV" -o ! -d "$TEST_DIR" ];
+		then
+		    _notrun "this test requires a valid \$TEST_DIR and unique $TEST_DEV"
+		fi
+		return 0
+		;;
 	*)
 		 if [ -z "$TEST_DEV" ] || [ "`_is_block_dev "$TEST_DEV"`" = "" ]
 		 then
@@ -2438,6 +2479,10 @@ _remount()
     local device=$1
     local mode=$2
 
+    if [ "$FSTYP" == "local" ] ; then
+       return 0
+    fi
+
     if ! mount -o remount,$mode $device
     then
         echo "_remount: failed to remount filesystem on $device as $mode"
@@ -2483,6 +2528,10 @@ _mount_or_remount_rw()
 	local device=$2
 	local mountpoint=$3
 
+	if [ "$FSTYP" == "local" ] ; then
+	    return 0
+	fi
+
 	if [ $USE_REMOUNT -eq 0 ]; then
 		if [ "$FSTYP" != "overlay" ]; then
 			_mount -t $FSTYP $mount_opts $device $mountpoint
@@ -2636,6 +2685,9 @@ _check_test_fs()
     ubifs)
 	# there is no fsck program for ubifs yet
 	;;
+    local)
+	# no way to check consistency for local
+	;;
     *)
 	_check_generic_filesystem $TEST_DEV
 	;;
@@ -2691,6 +2743,9 @@ _check_scratch_fs()
     ubifs)
 	# there is no fsck program for ubifs yet
 	;;
+    local)
+	# no way to check consistency for local
+	;;
     *)
 	_check_generic_filesystem $device
 	;;
@@ -3003,6 +3058,9 @@ _require_fio()
 # Does freeze work on this fs?
 _require_freeze()
 {
+	if [ "$FSTYP" == "local" ]; then
+		_notrun "local does not support freeze"
+	fi
 	xfs_freeze -f "$TEST_DIR" >/dev/null 2>&1
 	local result=$?
 	xfs_freeze -u "$TEST_DIR" >/dev/null 2>&1
@@ -3024,6 +3082,9 @@ _require_scratch_shutdown()
 {
 	[ -x src/godown ] || _notrun "src/godown executable not found"
 
+	if [ "$FSTYP" == "local" ]; then
+		_notrun "local does not support shutdown"
+	fi
 	_scratch_mkfs > /dev/null 2>&1 || _notrun "_scratch_mkfs failed on $SCRATCH_DEV"
 	_scratch_mount
 
@@ -3049,6 +3110,9 @@ _require_scratch_shutdown()
 # Does dax mount option work on this dev/fs?
 _require_scratch_dax()
 {
+	if [ "$FSTYP" == "local" ]; then
+		_notrun "local does not support dax"
+	fi
 	_require_scratch
 	_scratch_mkfs > /dev/null 2>&1
 	_try_scratch_mount -o dax || \
@@ -3063,6 +3127,9 @@ _require_scratch_dax()
 # Does norecovery support by this fs?
 _require_norecovery()
 {
+	if [ "$FSTYP" == "local" ]; then
+		_notrun "local does not support norecovery"
+	fi
 	_try_scratch_mount -o ro,norecovery || \
 		_notrun "$FSTYP does not support norecovery"
 	_scratch_unmount
-- 
2.16.1.72.g5be1f00a9a

Re: [PATCH] common: add support for the "local" file system type

[Amir Goldstein]

On Thu, May 3, 2018 at 6:43 AM, Theodore Y. Ts'o <tytso@mit.edu> wrote:
[...]
> From 8b40b28866dca119d0c807c31ae48f153ec2dc53 Mon Sep 17 00:00:00 2001
> From: Theodore Ts'o <tytso@mit.edu>
> Date: Sat, 17 Sep 2016 19:08:18 -0400
> Subject: [PATCH] common: add support for the "local" file system type
>
> It is sometimes useful to be able to test the local file system
> provided in a restricted execution environment (such as that which is
> provided by Docker, for example) where it is not possible to mount and
> unmount the file system under test.
>
> To support this test case, add support for a new file system type
> called "local".  The TEST_DEV and SCRATCH_DEV should be have a
> non-block device format (e.g., local:/test or local:/scratch), and the
> TEST_DIR and SCRATCH_MNT directories should be pre-existing
> directories provided by the execution environment.

Ted,

This looks like a very useful feature, but I suspect that some bits of the
patch may be a bit too specific to your use case (see below) - I may be wrong.
I bet there is a large variety of out of tree xfstests patches named
"Add support for XXX fs" that could be avoided with this feature.

The ultimate proof would be to demonstrate the usefulness of the feature
to more than a single use case - how about FUSE passthrough example?
Perhaps FSTYP=user would be more descriptive in general to the use
case at hand, because 'local' is usually the counter of 'remote',
but I'm fine with FSTYP=local.

Another way to "market" the feature is FSTYP=generic, which is the
prototype of all other filesystems. Naturally, it only runs tests under
tests/generic (as FSTYP=local does) and any test that requires an
operation that has no generic implementation is notrun.


>
> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
> ---
>  common/rc | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
>  1 file changed, 70 insertions(+), 3 deletions(-)
>
> diff --git a/common/rc b/common/rc
> index 9ffab7fd..d5cb0fe4 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -351,6 +351,18 @@ _supports_filetype()
>         esac
>  }
>
> +_local_validate_mount_opt()
> +{
> +    case "$*" in
> +       ro|ro,*|*,ro) _notrun "ro mount option not supported" ;;
> +       *nosuid*) _notrun "nosuid mount option not supported" ;;
> +       *noatime*) _notrun "noatime mount option not supported" ;;
> +       *relatime*) _notrun "relatime mount option not supported" ;;
> +       *diratime*) _notrun "diratime mount option not supported" ;;
> +       *strictatime*) _notrun "strictatime mount option not supported" ;;
> +    esac
> +}

Why specifically these mount options? Is this really generic?

> +
>  # mount scratch device with given options but don't check mount status
>  _try_scratch_mount()
>  {
> @@ -376,6 +388,9 @@ _scratch_unmount()
>         btrfs)
>                 $UMOUNT_PROG $SCRATCH_MNT
>                 ;;
> +       local)
> +                rm -rf $SCRATCH_MNT/*
> +                ;;

_scratch_mkfs already does that. Why does it make sense in _scratch_unmount?

>         *)
>                 $UMOUNT_PROG $SCRATCH_DEV
>                 ;;
> @@ -386,6 +401,10 @@ _scratch_remount()
>  {
>      local opts="$1"
>
> +    if [ "$FSTYP" = "local" ]; then
> +       _local_validate_mount_opt "$*"
> +       return 0;
> +    fi

It makes more sense to me to _require_scratch_remount
for tests that need to remount in the beginning of tests - yeh
that's a bit more work.

>      if test -n "$opts"; then
>         mount -o "remount,$opts" $SCRATCH_MNT
>      fi
> @@ -395,7 +414,7 @@ _scratch_cycle_mount()
>  {
>      local opts="$1"
>
> -    if [ "$FSTYP" = tmpfs ]; then
> +    if [ "$FSTYP" = "tmpfs" -o "$FSTYP" = "local" ] ; then
>         _scratch_remount "$opts"

That seems like cheating - seems better to implement
and use _require_scratch_cycle_mount

>         return
>      fi
> @@ -429,6 +448,10 @@ _test_mount()
>          _overlay_test_mount $*
>          return $?
>      fi
> +    if [ "$FSTYP" == "local" ]; then
> +        mkdir -p $TEST_DIR
> +       return $?
> +    fi

What is the desired logic here? can you add a comment?
Seems to me that we need to verify there is something mounted
at $TEST_DIR. no?

>      _test_options mount
>      _mount -t $FSTYP $TEST_OPTIONS $TEST_FS_MOUNT_OPTS $SELINUX_MOUNT_OPTIONS $* $TEST_DEV $TEST_DIR
>  }
> @@ -437,7 +460,7 @@ _test_unmount()
>  {
>         if [ "$FSTYP" == "overlay" ]; then
>                 _overlay_test_unmount
> -       else
> +       elif [ "$FSTYP" != "local" ]; then
>                 $UMOUNT_PROG $TEST_DEV
>         fi
>  }
> @@ -723,7 +746,7 @@ _scratch_mkfs()
>         local mkfs_status
>
>         case $FSTYP in
> -       nfs*|cifs|ceph|overlay|glusterfs|pvfs2|9p)
> +       nfs*|cifs|ceph|overlay|glusterfs|pvfs2|9p|local)
>                 # unable to re-create this fstyp, just remove all files in
>                 # $SCRATCH_MNT to avoid EEXIST caused by the leftover files
>                 # created in previous runs
> @@ -1465,6 +1488,10 @@ _check_mounted_on()
>         local mnt=$4
>         local type=$5
>
> +       if [ "$FSTYP" == "local" ]; then
> +               return 0
> +       fi
> +

Shouldn't we check that "something" is mounted on $mnt?

>         # find $dev as the source, and print result in "$dev $mnt" format
>         local mount_rec=`findmnt -rncv -S $dev -o SOURCE,TARGET`
>         [ -n "$mount_rec" ] || return 1 # 1 = not mounted
> @@ -1562,6 +1589,13 @@ _require_scratch_nocheck()
>                         _notrun "this test requires a valid \$SCRATCH_MNT"
>                 fi
>                 ;;
> +       local)
> +               if [ -z "$SCRATCH_DEV" -o ! -d "$SCRATCH_MNT" ];
> +               then
> +                   _notrun "this test requires a valid \$SCRATCH_MNT and unique $SCRATCH_DEV"
> +               fi
> +               return 0
> +               ;;
>         *)
>                  if [ -z "$SCRATCH_DEV" -o "`_is_block_dev "$SCRATCH_DEV"`" = "" ]
>                  then
> @@ -1683,6 +1717,13 @@ _require_test()
>                         _notrun "this test requires a valid \$TEST_DIR"
>                 fi
>                 ;;
> +       local)
> +               if [ -z "$TEST_DEV" -o ! -d "$TEST_DIR" ];
> +               then
> +                   _notrun "this test requires a valid \$TEST_DIR and unique $TEST_DEV"
> +               fi
> +               return 0
> +               ;;
>         *)
>                  if [ -z "$TEST_DEV" ] || [ "`_is_block_dev "$TEST_DEV"`" = "" ]
>                  then
> @@ -2438,6 +2479,10 @@ _remount()
>      local device=$1
>      local mode=$2
>
> +    if [ "$FSTYP" == "local" ] ; then
> +       return 0
> +    fi
> +
>      if ! mount -o remount,$mode $device
>      then
>          echo "_remount: failed to remount filesystem on $device as $mode"
> @@ -2483,6 +2528,10 @@ _mount_or_remount_rw()
>         local device=$2
>         local mountpoint=$3
>
> +       if [ "$FSTYP" == "local" ] ; then
> +           return 0
> +       fi
> +
>         if [ $USE_REMOUNT -eq 0 ]; then
>                 if [ "$FSTYP" != "overlay" ]; then
>                         _mount -t $FSTYP $mount_opts $device $mountpoint
> @@ -2636,6 +2685,9 @@ _check_test_fs()
>      ubifs)
>         # there is no fsck program for ubifs yet
>         ;;
> +    local)
> +       # no way to check consistency for local
> +       ;;
>      *)
>         _check_generic_filesystem $TEST_DEV
>         ;;
> @@ -2691,6 +2743,9 @@ _check_scratch_fs()
>      ubifs)
>         # there is no fsck program for ubifs yet
>         ;;
> +    local)
> +       # no way to check consistency for local
> +       ;;
>      *)
>         _check_generic_filesystem $device
>         ;;
> @@ -3003,6 +3058,9 @@ _require_fio()
>  # Does freeze work on this fs?
>  _require_freeze()
>  {
> +       if [ "$FSTYP" == "local" ]; then
> +               _notrun "local does not support freeze"

When users read this warning they may be confused,
same for shutdown/dax/morecovery.
Something like "user defined fs does not support freeze"

Thanks,
Amir.

Re: [PATCH] common: add support for the "local" file system type

[Theodore Y. Ts'o]

On Thu, May 03, 2018 at 12:22:42PM +0300, Amir Goldstein wrote:
> 
> This looks like a very useful feature, but I suspect that some bits of the
> patch may be a bit too specific to your use case (see below) - I may be wrong.

The primary characteristic of "local" is there nothing to mount or
unmount.  So for example, it should be possible to use this
Microsoft's Subsystem for Linux.

> The ultimate proof would be to demonstrate the usefulness of the feature
> to more than a single use case - how about FUSE passthrough example?
> Perhaps FSTYP=user would be more descriptive in general to the use
> case at hand, because 'local' is usually the counter of 'remote',
> but I'm fine with FSTYP=local.

It certainly wouldn't be problem to demo using the local file system
type for FUSE --- but it would not be _ideal_ for fuse, since fuse has
the concept of mounting and unmounting.

> > +{
> > +    case "$*" in
> > +       ro|ro,*|*,ro) _notrun "ro mount option not supported" ;;
> > +       *nosuid*) _notrun "nosuid mount option not supported" ;;
> > +       *noatime*) _notrun "noatime mount option not supported" ;;
> > +       *relatime*) _notrun "relatime mount option not supported" ;;
> > +       *diratime*) _notrun "diratime mount option not supported" ;;
> > +       *strictatime*) _notrun "strictatime mount option not supported" ;;
> > +    esac
> > +}
> 
> Why specifically these mount options? Is this really generic?

The local file system type does not support mount, umount, or remount
command.  So there is no way to modulate noatime, relatime, etc.  So
this would be useful if someone wanted to test Windows System for
Linux, for example.

> > +
> >  # mount scratch device with given options but don't check mount status
> >  _try_scratch_mount()
> >  {
> > @@ -376,6 +388,9 @@ _scratch_unmount()
> >         btrfs)
> >                 $UMOUNT_PROG $SCRATCH_MNT
> >                 ;;
> > +       local)
> > +                rm -rf $SCRATCH_MNT/*
> > +                ;;
> 
> _scratch_mkfs already does that. Why does it make sense in _scratch_unmount?

Just for cleanup purposes.

> > @@ -386,6 +401,10 @@ _scratch_remount()
> >  {
> >      local opts="$1"
> >
> > +    if [ "$FSTYP" = "local" ]; then
> > +       _local_validate_mount_opt "$*"
> > +       return 0;
> > +    fi
> 
> It makes more sense to me to _require_scratch_remount
> for tests that need to remount in the beginning of tests - yeh
> that's a bit more work.

There are some tests where the remount operation is just to reset the
file system.  So the test is just to unmount and remount the file
system, and making sure the file status are the same.  So I didn't
want to block all remounts; instead, to let some remounts be no-op's
so the rest of the test could still be used to provide test coverage,
and only block those remounts which were modulating things like ro/rw,
noatime, etc.

> > @@ -395,7 +414,7 @@ _scratch_cycle_mount()
> >  {
> >      local opts="$1"
> >
> > -    if [ "$FSTYP" = tmpfs ]; then
> > +    if [ "$FSTYP" = "tmpfs" -o "$FSTYP" = "local" ] ; then
> >         _scratch_remount "$opts"
> 
> That seems like cheating - seems better to implement
> and use _require_scratch_cycle_mount

Again, I didn't want to end up skipping a huge number of tests.  I
didn't consider this "cheating"; I considered it a case of "let's
maximize test coverage".

> > +    if [ "$FSTYP" == "local" ]; then
> > +        mkdir -p $TEST_DIR
> > +       return $?
> > +    fi
> 
> What is the desired logic here? can you add a comment?
> Seems to me that we need to verify there is something mounted
> at $TEST_DIR. no?

Again -- the big thing about the "local" file system is there is no
way to mount, umount, or remount.  So no block device, no mount point,
nothing to shutdown, etc.

The mkdir -p isn't strictly necessary here.  It could be done by the
kvm-xfstests or whatever is running the xfstests.  It was more of a
safety thing, but if people don't like it, we can drop it.

> > @@ -1465,6 +1488,10 @@ _check_mounted_on()
> >         local mnt=$4
> >         local type=$5
> >
> > +       if [ "$FSTYP" == "local" ]; then
> > +               return 0
> > +       fi
> > +
> 
> Shouldn't we check that "something" is mounted on $mnt?

Nope.  See above.  "local" means never having to say you need to mount
something.

> > @@ -3003,6 +3058,9 @@ _require_fio()
> >  # Does freeze work on this fs?
> >  _require_freeze()
> >  {
> > +       if [ "$FSTYP" == "local" ]; then
> > +               _notrun "local does not support freeze"
> 
> When users read this warning they may be confused,
> same for shutdown/dax/morecovery.
> Something like "user defined fs does not support freeze"

I think you have a very different idea of what "local" means.  The
basic model was that this is something where the file system is
provided for use by the docker infrastructure (or by Windows 10 in the
WSL case), and so the docker job can't actually mount or unmount the
file system.  However, it is still desirable to check to see how POSIX
compliant the underlying file system might be, and it can also be used
to exercise the underlying file system.

I'm not tied to the name "local", but for me what it means is "the
local file system".  If people want to nominate a different name, I'm
certainly open to suggestions.

					- Ted

Re: [PATCH] common: add support for the "local" file system type

[Amir Goldstein]

On Thu, May 3, 2018 at 9:35 PM, Theodore Y. Ts'o <tytso@mit.edu> wrote:
> On Thu, May 03, 2018 at 12:22:42PM +0300, Amir Goldstein wrote:
>>
>> This looks like a very useful feature, but I suspect that some bits of the
>> patch may be a bit too specific to your use case (see below) - I may be wrong.
>
> The primary characteristic of "local" is there nothing to mount or
> unmount.  So for example, it should be possible to use this
> Microsoft's Subsystem for Linux.
>
>> The ultimate proof would be to demonstrate the usefulness of the feature
>> to more than a single use case - how about FUSE passthrough example?
>> Perhaps FSTYP=user would be more descriptive in general to the use
>> case at hand, because 'local' is usually the counter of 'remote',
>> but I'm fine with FSTYP=local.
>
> It certainly wouldn't be problem to demo using the local file system
> type for FUSE --- but it would not be _ideal_ for fuse, since fuse has
> the concept of mounting and unmounting.
>
>> > +{
>> > +    case "$*" in
>> > +       ro|ro,*|*,ro) _notrun "ro mount option not supported" ;;
>> > +       *nosuid*) _notrun "nosuid mount option not supported" ;;
>> > +       *noatime*) _notrun "noatime mount option not supported" ;;
>> > +       *relatime*) _notrun "relatime mount option not supported" ;;
>> > +       *diratime*) _notrun "diratime mount option not supported" ;;
>> > +       *strictatime*) _notrun "strictatime mount option not supported" ;;
>> > +    esac
>> > +}
>>
>> Why specifically these mount options? Is this really generic?
>
> The local file system type does not support mount, umount, or remount
> command.  So there is no way to modulate noatime, relatime, etc.  So
> this would be useful if someone wanted to test Windows System for
> Linux, for example.
>
>> > +
>> >  # mount scratch device with given options but don't check mount status
>> >  _try_scratch_mount()
>> >  {
>> > @@ -376,6 +388,9 @@ _scratch_unmount()
>> >         btrfs)
>> >                 $UMOUNT_PROG $SCRATCH_MNT
>> >                 ;;
>> > +       local)
>> > +                rm -rf $SCRATCH_MNT/*
>> > +                ;;
>>
>> _scratch_mkfs already does that. Why does it make sense in _scratch_unmount?
>
> Just for cleanup purposes.
>
>> > @@ -386,6 +401,10 @@ _scratch_remount()
>> >  {
>> >      local opts="$1"
>> >
>> > +    if [ "$FSTYP" = "local" ]; then
>> > +       _local_validate_mount_opt "$*"
>> > +       return 0;
>> > +    fi
>>
>> It makes more sense to me to _require_scratch_remount
>> for tests that need to remount in the beginning of tests - yeh
>> that's a bit more work.
>
> There are some tests where the remount operation is just to reset the
> file system.  So the test is just to unmount and remount the file
> system, and making sure the file status are the same.  So I didn't
> want to block all remounts; instead, to let some remounts be no-op's
> so the rest of the test could still be used to provide test coverage,
> and only block those remounts which were modulating things like ro/rw,
> noatime, etc.
>
>> > @@ -395,7 +414,7 @@ _scratch_cycle_mount()
>> >  {
>> >      local opts="$1"
>> >
>> > -    if [ "$FSTYP" = tmpfs ]; then
>> > +    if [ "$FSTYP" = "tmpfs" -o "$FSTYP" = "local" ] ; then
>> >         _scratch_remount "$opts"
>>
>> That seems like cheating - seems better to implement
>> and use _require_scratch_cycle_mount
>
> Again, I didn't want to end up skipping a huge number of tests.  I
> didn't consider this "cheating"; I considered it a case of "let's
> maximize test coverage".
>

OK. I've used _scratch_cycle_mount as well in exportfs tests
to make sure caches are evicted, so perhaps the "not cheating"
edition of _scratch_cycle_mount for local should at least drop
caches?

Thanks,
Amir.

Re: [PATCH] common: add support for the "local" file system type

[Eryu Guan]

On Wed, May 02, 2018 at 11:43:40PM -0400, Theodore Y. Ts'o wrote:
> About a year and half ago, I sent the patch attached below to add
> support for a "local" file system type where the file system could not
> be mounted or dismounted, but where writes to TEST_DIR and SCRATCH_MNT
> would be testing the file system in question.  At the time, I couldn't
> describe the use case in any greater detail than what I had in the
> commit description, and Dave Chinner at the time rejected the patch
> since he didn't like xfstests being used to test a proprietary file
> system for which I was not able to explain the details of what we were
> trying to do.
> 
> Not a big deal, I just kept the patch in my private fork[1] of
> xfstests on github.
> 
> [1] https://github.com/tytso/xfstests
> 
> However, happily, we can now talk a lot more about what the "local"
> file system type in xfstests was used to test.  Earlier today, Google
> announced gVisor[2], and published it as open source on github[3].
> gVisor works much like User Mode Linux, and I suspect much like the
> Windows Subsystem for Linux in that it uses the x86_64's hardware
> virtualization extensions (so it has the security fencing much like a
> VM) but instead of emulating hardware, instead the emulation layer is
> done at the system call level (so it's more efficient than a VM, since
> there is no guest kernel).
> 
> [2] https://cloudplatform.googleblog.com/2018/05/Open-sourcing-gVisor-a-sandboxed-container-runtime.html
> [3] https://github.com/google/gvisor
> 
> File systems can be implemented using Gophers[4] in a separate
> process, where the communication between the gVisor sandbox and the
> Gopher is via the 9P2000.L protocol.  This means that if you want to
> try to exploit a buffer overflow in the userspace file system, first
> you have to get past the system call validation checks done by the
> gVisor Sentry process (which is written in Go which makes this rather
> more difficult, especially since the Sentry process itself is
> protected using seccomp[5]).  Then the buffer overflow attack has to
> make it past the 9P protocol encoding/decoding, and then survive the
> 9P protocol validation checks in the Gopher.  The Gopher process
> provides an emulated file system service using the Cloud Provider's
> internal cluster storage services, much like in GCE, the Persistent
> Disk service provides an emulated block device service.
> 
> [4] https://news.ycombinator.com/item?id=16979126
> [5] https://news.ycombinator.com/item?id=16976557
> 
> This whole system was designed with security first and foremost.  Of
> course, we also want it to pass the file system checks, which is why
> were using xfstests.
> 
> The way we actually tested the gVisor file system was via a Docker
> container (the Dockerfile[6] is in the xfstests-bld git repo) which
> would then get consumed by gVisor's Docker/Kubernetes integration
> layer.
> 
> [6] https://github.com/tytso/xfstests-bld/blob/master/Dockerfile
> 
> Anyway, back in September, 2016, Dave Chinner was peeved that I
> couldn't give this full description, and I'm glad to say, now we can
> finally rectify that gap.  :-)
> 
> Could this commit therefore please be considered for inclusion in
> xfstests upstream?

I still think it's useful, and thanks for resending with such detailed
information! And I'm fine with the 'local' file system type. But I may
need some time to do careful testing and go into the details. Just some
random comments inline.

> 
> Many thanks!
> 
> 						- Ted
> 
> 
> From 8b40b28866dca119d0c807c31ae48f153ec2dc53 Mon Sep 17 00:00:00 2001
> From: Theodore Ts'o <tytso@mit.edu>
> Date: Sat, 17 Sep 2016 19:08:18 -0400
> Subject: [PATCH] common: add support for the "local" file system type
> 
> It is sometimes useful to be able to test the local file system
> provided in a restricted execution environment (such as that which is
> provided by Docker, for example) where it is not possible to mount and
> unmount the file system under test.
> 
> To support this test case, add support for a new file system type
> called "local".  The TEST_DEV and SCRATCH_DEV should be have a
> non-block device format (e.g., local:/test or local:/scratch), and the

Does it require a non-block device format in this version of the patch?
I don't think so, as we have the new 'local' FSTYP introduced now.

> TEST_DIR and SCRATCH_MNT directories should be pre-existing
> directories provided by the execution environment.
> 
> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
> ---
>  common/rc | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---

We really need some documentation added in README :)

>  1 file changed, 70 insertions(+), 3 deletions(-)
> 
> diff --git a/common/rc b/common/rc
> index 9ffab7fd..d5cb0fe4 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -351,6 +351,18 @@ _supports_filetype()
>  	esac
>  }
>  
> +_local_validate_mount_opt()
> +{
> +    case "$*" in
> +	ro|ro,*|*,ro) _notrun "ro mount option not supported" ;;
> +	*nosuid*) _notrun "nosuid mount option not supported" ;;
> +	*noatime*) _notrun "noatime mount option not supported" ;;
> +	*relatime*) _notrun "relatime mount option not supported" ;;
> +	*diratime*) _notrun "diratime mount option not supported" ;;
> +	*strictatime*) _notrun "strictatime mount option not supported" ;;
> +    esac
> +}

Would be good to have some comments on why these mount options are not
supported for 'local'.

> +
>  # mount scratch device with given options but don't check mount status
>  _try_scratch_mount()
>  {
> @@ -376,6 +388,9 @@ _scratch_unmount()
>  	btrfs)
>  		$UMOUNT_PROG $SCRATCH_MNT
>  		;;
> +	local)
> +                rm -rf $SCRATCH_MNT/*
> +                ;;

We do this in _scratch_mkfs by calling _scratch_cleanup_files. I noticed
that you already did this in _scratch_mkfs, just return here for
'local'?

>  	*)
>  		$UMOUNT_PROG $SCRATCH_DEV
>  		;;
> @@ -386,6 +401,10 @@ _scratch_remount()
>  {
>      local opts="$1"
>  
> +    if [ "$FSTYP" = "local" ]; then
> +	_local_validate_mount_opt "$*"
> +	return 0;
> +    fi
>      if test -n "$opts"; then
>  	mount -o "remount,$opts" $SCRATCH_MNT
>      fi
> @@ -395,7 +414,7 @@ _scratch_cycle_mount()
>  {
>      local opts="$1"
>  
> -    if [ "$FSTYP" = tmpfs ]; then
> +    if [ "$FSTYP" = "tmpfs" -o "$FSTYP" = "local" ] ; then
>  	_scratch_remount "$opts"
>  	return
>      fi
> @@ -429,6 +448,10 @@ _test_mount()
>          _overlay_test_mount $*
>          return $?
>      fi
> +    if [ "$FSTYP" == "local" ]; then
> +        mkdir -p $TEST_DIR
> +	return $?
> +    fi

$TEST_DIR is guaranteed to be there (in common/config at initialization
time and in _require_test ). Perhaps we can just return in the 'local'
case?

>      _test_options mount
>      _mount -t $FSTYP $TEST_OPTIONS $TEST_FS_MOUNT_OPTS $SELINUX_MOUNT_OPTIONS $* $TEST_DEV $TEST_DIR
>  }
> @@ -437,7 +460,7 @@ _test_unmount()
>  {
>  	if [ "$FSTYP" == "overlay" ]; then
>  		_overlay_test_unmount
> -	else
> +	elif [ "$FSTYP" != "local" ]; then
>  		$UMOUNT_PROG $TEST_DEV
>  	fi
>  }
> @@ -723,7 +746,7 @@ _scratch_mkfs()
>  	local mkfs_status
>  
>  	case $FSTYP in
> -	nfs*|cifs|ceph|overlay|glusterfs|pvfs2|9p)
> +	nfs*|cifs|ceph|overlay|glusterfs|pvfs2|9p|local)
>  		# unable to re-create this fstyp, just remove all files in
>  		# $SCRATCH_MNT to avoid EEXIST caused by the leftover files
>  		# created in previous runs
> @@ -1465,6 +1488,10 @@ _check_mounted_on()
>  	local mnt=$4
>  	local type=$5
>  
> +	if [ "$FSTYP" == "local" ]; then
> +		return 0
> +	fi
> +
>  	# find $dev as the source, and print result in "$dev $mnt" format
>  	local mount_rec=`findmnt -rncv -S $dev -o SOURCE,TARGET`
>  	[ -n "$mount_rec" ] || return 1 # 1 = not mounted
> @@ -1562,6 +1589,13 @@ _require_scratch_nocheck()
>  			_notrun "this test requires a valid \$SCRATCH_MNT"
>  		fi
>  		;;
> +	local)
> +		if [ -z "$SCRATCH_DEV" -o ! -d "$SCRATCH_MNT" ];
> +		then
> +		    _notrun "this test requires a valid \$SCRATCH_MNT and unique $SCRATCH_DEV"
> +		fi
> +		return 0
> +		;;
>  	*)
>  		 if [ -z "$SCRATCH_DEV" -o "`_is_block_dev "$SCRATCH_DEV"`" = "" ]
>  		 then
> @@ -1683,6 +1717,13 @@ _require_test()
>  			_notrun "this test requires a valid \$TEST_DIR"
>  		fi
>  		;;
> +	local)
> +		if [ -z "$TEST_DEV" -o ! -d "$TEST_DIR" ];
> +		then
> +		    _notrun "this test requires a valid \$TEST_DIR and unique $TEST_DEV"
> +		fi
> +		return 0
> +		;;

It seems $SCRATCH_DEV and $TEST_DEV are not important for 'local' type,
as long as we have SCRATCH_MNT and/or TEST_DIR defined as directories.

Thanks,
Eryu

>  	*)
>  		 if [ -z "$TEST_DEV" ] || [ "`_is_block_dev "$TEST_DEV"`" = "" ]
>  		 then
> @@ -2438,6 +2479,10 @@ _remount()
>      local device=$1
>      local mode=$2
>  
> +    if [ "$FSTYP" == "local" ] ; then
> +       return 0
> +    fi
> +
>      if ! mount -o remount,$mode $device
>      then
>          echo "_remount: failed to remount filesystem on $device as $mode"
> @@ -2483,6 +2528,10 @@ _mount_or_remount_rw()
>  	local device=$2
>  	local mountpoint=$3
>  
> +	if [ "$FSTYP" == "local" ] ; then
> +	    return 0
> +	fi
> +
>  	if [ $USE_REMOUNT -eq 0 ]; then
>  		if [ "$FSTYP" != "overlay" ]; then
>  			_mount -t $FSTYP $mount_opts $device $mountpoint
> @@ -2636,6 +2685,9 @@ _check_test_fs()
>      ubifs)
>  	# there is no fsck program for ubifs yet
>  	;;
> +    local)
> +	# no way to check consistency for local
> +	;;
>      *)
>  	_check_generic_filesystem $TEST_DEV
>  	;;
> @@ -2691,6 +2743,9 @@ _check_scratch_fs()
>      ubifs)
>  	# there is no fsck program for ubifs yet
>  	;;
> +    local)
> +	# no way to check consistency for local
> +	;;
>      *)
>  	_check_generic_filesystem $device
>  	;;
> @@ -3003,6 +3058,9 @@ _require_fio()
>  # Does freeze work on this fs?
>  _require_freeze()
>  {
> +	if [ "$FSTYP" == "local" ]; then
> +		_notrun "local does not support freeze"
> +	fi
>  	xfs_freeze -f "$TEST_DIR" >/dev/null 2>&1
>  	local result=$?
>  	xfs_freeze -u "$TEST_DIR" >/dev/null 2>&1
> @@ -3024,6 +3082,9 @@ _require_scratch_shutdown()
>  {
>  	[ -x src/godown ] || _notrun "src/godown executable not found"
>  
> +	if [ "$FSTYP" == "local" ]; then
> +		_notrun "local does not support shutdown"
> +	fi
>  	_scratch_mkfs > /dev/null 2>&1 || _notrun "_scratch_mkfs failed on $SCRATCH_DEV"
>  	_scratch_mount
>  
> @@ -3049,6 +3110,9 @@ _require_scratch_shutdown()
>  # Does dax mount option work on this dev/fs?
>  _require_scratch_dax()
>  {
> +	if [ "$FSTYP" == "local" ]; then
> +		_notrun "local does not support dax"
> +	fi
>  	_require_scratch
>  	_scratch_mkfs > /dev/null 2>&1
>  	_try_scratch_mount -o dax || \
> @@ -3063,6 +3127,9 @@ _require_scratch_dax()
>  # Does norecovery support by this fs?
>  _require_norecovery()
>  {
> +	if [ "$FSTYP" == "local" ]; then
> +		_notrun "local does not support norecovery"
> +	fi
>  	_try_scratch_mount -o ro,norecovery || \
>  		_notrun "$FSTYP does not support norecovery"
>  	_scratch_unmount
> -- 
> 2.16.1.72.g5be1f00a9a
> 
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] common: add support for the "local" file system type

[Eryu Guan]

Hi Ted,

On Wed, May 02, 2018 at 11:43:40PM -0400, Theodore Y. Ts'o wrote:

[snip the background details]

> From 8b40b28866dca119d0c807c31ae48f153ec2dc53 Mon Sep 17 00:00:00 2001
> From: Theodore Ts'o <tytso@mit.edu>
> Date: Sat, 17 Sep 2016 19:08:18 -0400
> Subject: [PATCH] common: add support for the "local" file system type
> 
> It is sometimes useful to be able to test the local file system
> provided in a restricted execution environment (such as that which is
> provided by Docker, for example) where it is not possible to mount and
> unmount the file system under test.
> 
> To support this test case, add support for a new file system type
> called "local".  The TEST_DEV and SCRATCH_DEV should be have a
> non-block device format (e.g., local:/test or local:/scratch), and the
> TEST_DIR and SCRATCH_MNT directories should be pre-existing
> directories provided by the execution environment.
> 
> Signed-off-by: Theodore Ts'o <tytso@mit.edu>

I tested the patch with XFS and ext4 as the mounted underlying
filesystem for multiple rounds and found some more issues (in addition
to previous review comments) that need to be addressed. I'll reply
inline.

> ---
>  common/rc | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
>  1 file changed, 70 insertions(+), 3 deletions(-)

I think we need a new "-local" option in './check' to do some initial
setups, e.g. FSTYP=local, as other fs types that can't be retrieved by
running blkid on $TEST_DEV.

Also need some documentations in README to describe "local"'s use case
and mention that TEST_DEV and SCRATCH_DEV should be have a non-block
device format.

> 
> diff --git a/common/rc b/common/rc
> index 9ffab7fd..d5cb0fe4 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -351,6 +351,18 @@ _supports_filetype()
>  	esac
>  }
>  
> +_local_validate_mount_opt()
> +{
> +    case "$*" in
> +	ro|ro,*|*,ro) _notrun "ro mount option not supported" ;;
> +	*nosuid*) _notrun "nosuid mount option not supported" ;;
> +	*noatime*) _notrun "noatime mount option not supported" ;;
> +	*relatime*) _notrun "relatime mount option not supported" ;;
> +	*diratime*) _notrun "diratime mount option not supported" ;;
> +	*strictatime*) _notrun "strictatime mount option not supported" ;;
> +    esac
> +}
> +
>  # mount scratch device with given options but don't check mount status
>  _try_scratch_mount()
>  {

We need to add a check for "local" in _try_scratch_mount() to let it do
nothing for "local" too, otherwise 'FSTYP=local ./check' won't run for
me, as _scratch_mount() failed and exit the test in 'check' line 629.

(Note that the exit on _scratch_mount() failure behavior was introduced
by commit 69eb6281a9d3 ("fstests: _fail test by default when
_scratch_mount fails"))

> @@ -376,6 +388,9 @@ _scratch_unmount()
>  	btrfs)
>  		$UMOUNT_PROG $SCRATCH_MNT
>  		;;
> +	local)
> +                rm -rf $SCRATCH_MNT/*
> +                ;;

As noted in previous review, we could just return for 'local' in
_scratch_unmount(), _scratch_mkfs() already removed all the files there.

>  	*)
>  		$UMOUNT_PROG $SCRATCH_DEV
>  		;;
> @@ -386,6 +401,10 @@ _scratch_remount()
>  {
>      local opts="$1"
>  
> +    if [ "$FSTYP" = "local" ]; then
> +	_local_validate_mount_opt "$*"
> +	return 0;
> +    fi

I found that there're more places that need to do
_local_validate_mount_opt, e.g. in _try_scratch_mount() and _test_mount,
otherwise tests that use _try_scratch_mount to do the remount would
fail, like generic/294.

>      if test -n "$opts"; then
>  	mount -o "remount,$opts" $SCRATCH_MNT
>      fi
> @@ -395,7 +414,7 @@ _scratch_cycle_mount()
>  {
>      local opts="$1"
>  
> -    if [ "$FSTYP" = tmpfs ]; then
> +    if [ "$FSTYP" = "tmpfs" -o "$FSTYP" = "local" ] ; then
>  	_scratch_remount "$opts"
>  	return
>      fi
> @@ -429,6 +448,10 @@ _test_mount()
>          _overlay_test_mount $*
>          return $?
>      fi
> +    if [ "$FSTYP" == "local" ]; then
> +        mkdir -p $TEST_DIR

Probably need _local_validate_mount_opt here as mentioned above.

And there're some tests or functions that call umount on $SCRATCH_MNT
directly, which would cause the underlying filesystem to be umounted if
it can be umounted (thouth this is not an intended use case for "local",
I think it's better to fix them too). e.g.

generic/034 generic/330 and generic/332 need to use _scratch_unmount
instead of a bare umount.

All the "$UMOUNT_PROG $SCRATCH_MNT"s in dm device helper functions, we
need to change them to umount the actual device, e.g. $FLAKEY_DEV

generic/108 needs to umount $SCSI_DEBUG_DEV in _cleanup()

Similarly, generic/459 needs to umount the lvm snapshot device
/dev/mapper/$vgname-$snapname

Thanks,
Eryu

> +	return $?
> +    fi
>      _test_options mount
>      _mount -t $FSTYP $TEST_OPTIONS $TEST_FS_MOUNT_OPTS $SELINUX_MOUNT_OPTIONS $* $TEST_DEV $TEST_DIR
>  }
> @@ -437,7 +460,7 @@ _test_unmount()
>  {
>  	if [ "$FSTYP" == "overlay" ]; then
>  		_overlay_test_unmount
> -	else
> +	elif [ "$FSTYP" != "local" ]; then
>  		$UMOUNT_PROG $TEST_DEV
>  	fi
>  }
> @@ -723,7 +746,7 @@ _scratch_mkfs()
>  	local mkfs_status
>  
>  	case $FSTYP in
> -	nfs*|cifs|ceph|overlay|glusterfs|pvfs2|9p)
> +	nfs*|cifs|ceph|overlay|glusterfs|pvfs2|9p|local)
>  		# unable to re-create this fstyp, just remove all files in
>  		# $SCRATCH_MNT to avoid EEXIST caused by the leftover files
>  		# created in previous runs
> @@ -1465,6 +1488,10 @@ _check_mounted_on()
>  	local mnt=$4
>  	local type=$5
>  
> +	if [ "$FSTYP" == "local" ]; then
> +		return 0
> +	fi
> +
>  	# find $dev as the source, and print result in "$dev $mnt" format
>  	local mount_rec=`findmnt -rncv -S $dev -o SOURCE,TARGET`
>  	[ -n "$mount_rec" ] || return 1 # 1 = not mounted
> @@ -1562,6 +1589,13 @@ _require_scratch_nocheck()
>  			_notrun "this test requires a valid \$SCRATCH_MNT"
>  		fi
>  		;;
> +	local)
> +		if [ -z "$SCRATCH_DEV" -o ! -d "$SCRATCH_MNT" ];
> +		then
> +		    _notrun "this test requires a valid \$SCRATCH_MNT and unique $SCRATCH_DEV"
> +		fi
> +		return 0
> +		;;
>  	*)
>  		 if [ -z "$SCRATCH_DEV" -o "`_is_block_dev "$SCRATCH_DEV"`" = "" ]
>  		 then
> @@ -1683,6 +1717,13 @@ _require_test()
>  			_notrun "this test requires a valid \$TEST_DIR"
>  		fi
>  		;;
> +	local)
> +		if [ -z "$TEST_DEV" -o ! -d "$TEST_DIR" ];
> +		then
> +		    _notrun "this test requires a valid \$TEST_DIR and unique $TEST_DEV"
> +		fi
> +		return 0
> +		;;
>  	*)
>  		 if [ -z "$TEST_DEV" ] || [ "`_is_block_dev "$TEST_DEV"`" = "" ]
>  		 then
> @@ -2438,6 +2479,10 @@ _remount()
>      local device=$1
>      local mode=$2
>  
> +    if [ "$FSTYP" == "local" ] ; then
> +       return 0
> +    fi
> +
>      if ! mount -o remount,$mode $device
>      then
>          echo "_remount: failed to remount filesystem on $device as $mode"
> @@ -2483,6 +2528,10 @@ _mount_or_remount_rw()
>  	local device=$2
>  	local mountpoint=$3
>  
> +	if [ "$FSTYP" == "local" ] ; then
> +	    return 0
> +	fi
> +
>  	if [ $USE_REMOUNT -eq 0 ]; then
>  		if [ "$FSTYP" != "overlay" ]; then
>  			_mount -t $FSTYP $mount_opts $device $mountpoint
> @@ -2636,6 +2685,9 @@ _check_test_fs()
>      ubifs)
>  	# there is no fsck program for ubifs yet
>  	;;
> +    local)
> +	# no way to check consistency for local
> +	;;
>      *)
>  	_check_generic_filesystem $TEST_DEV
>  	;;
> @@ -2691,6 +2743,9 @@ _check_scratch_fs()
>      ubifs)
>  	# there is no fsck program for ubifs yet
>  	;;
> +    local)
> +	# no way to check consistency for local
> +	;;
>      *)
>  	_check_generic_filesystem $device
>  	;;
> @@ -3003,6 +3058,9 @@ _require_fio()
>  # Does freeze work on this fs?
>  _require_freeze()
>  {
> +	if [ "$FSTYP" == "local" ]; then
> +		_notrun "local does not support freeze"
> +	fi
>  	xfs_freeze -f "$TEST_DIR" >/dev/null 2>&1
>  	local result=$?
>  	xfs_freeze -u "$TEST_DIR" >/dev/null 2>&1
> @@ -3024,6 +3082,9 @@ _require_scratch_shutdown()
>  {
>  	[ -x src/godown ] || _notrun "src/godown executable not found"
>  
> +	if [ "$FSTYP" == "local" ]; then
> +		_notrun "local does not support shutdown"
> +	fi
>  	_scratch_mkfs > /dev/null 2>&1 || _notrun "_scratch_mkfs failed on $SCRATCH_DEV"
>  	_scratch_mount
>  
> @@ -3049,6 +3110,9 @@ _require_scratch_shutdown()
>  # Does dax mount option work on this dev/fs?
>  _require_scratch_dax()
>  {
> +	if [ "$FSTYP" == "local" ]; then
> +		_notrun "local does not support dax"
> +	fi
>  	_require_scratch
>  	_scratch_mkfs > /dev/null 2>&1
>  	_try_scratch_mount -o dax || \
> @@ -3063,6 +3127,9 @@ _require_scratch_dax()
>  # Does norecovery support by this fs?
>  _require_norecovery()
>  {
> +	if [ "$FSTYP" == "local" ]; then
> +		_notrun "local does not support norecovery"
> +	fi
>  	_try_scratch_mount -o ro,norecovery || \
>  		_notrun "$FSTYP does not support norecovery"
>  	_scratch_unmount
> -- 
> 2.16.1.72.g5be1f00a9a
> 
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] common: add support for the "local" file system type

[Theodore Ts'o]

It is sometimes useful to be able to test the local file system
provided in a restricted execution environment (such as that which is
provided by Docker, for example) where it is not possible to mount and
unmount the file system under test.

To support this test case, add support for a new file system type
called "local".  The TEST_DEV and SCRATCH_DEV should be have a
non-block device format (e.g., local:/test or local:/scratch), and the
TEST_DIR and SCRATCH_MNT directories should be pre-existing
directories provided by the execution environment.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
 common/rc         | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++---
 tests/generic/027 |  2 +-
 2 files changed, 70 insertions(+), 4 deletions(-)

diff --git a/common/rc b/common/rc
index 70d79c9..c03b132 100644
--- a/common/rc
+++ b/common/rc
@@ -330,12 +330,29 @@ _overlay_scratch_unmount()
 	$UMOUNT_PROG $SCRATCH_MNT
 }
 
+_local_validate_mount_opt()
+{
+    case "$*" in
+	ro|ro,*|*,ro) _notrun "ro mount option not supported" ;;
+	*nosuid*) _notrun "nosuid mount option not supported" ;;
+	*noatime*) _notrun "noatime mount option not supported" ;;
+	*relatime*) _notrun "relatime mount option not supported" ;;
+	*diratime*) _notrun "diratime mount option not supported" ;;
+	*strictatime*) _notrun "strictatime mount option not supported" ;;
+    esac
+}
+
 _scratch_mount()
 {
     if [ "$FSTYP" == "overlay" ]; then
         _overlay_scratch_mount $*
         return $?
     fi
+    if [ "$FSTYP" == "local" ]; then
+	_local_validate_mount_opt "$*"
+        mkdir -p $SCRATCH_MNT
+	return $?
+    fi
     _mount -t $FSTYP `_scratch_mount_options $*`
 }
 
@@ -348,6 +365,9 @@ _scratch_unmount()
 	btrfs)
 		$UMOUNT_PROG $SCRATCH_MNT
 		;;
+	local)
+                rm -rf $SCRATCH_MNT/*
+                ;;
 	*)
 		$UMOUNT_PROG $SCRATCH_DEV
 		;;
@@ -358,6 +378,10 @@ _scratch_remount()
 {
     local opts="$1"
 
+    if [ "$FSTYP" = "local" ]; then
+	_local_validate_mount_opt "$*"
+	return 0;
+    fi
     if test -n "$opts"; then
 	mount -o "remount,$opts" $SCRATCH_MNT
     fi
@@ -367,7 +391,7 @@ _scratch_cycle_mount()
 {
     local opts="$1"
 
-    if [ "$FSTYP" = tmpfs ]; then
+    if [ "$FSTYP" = "tmpfs" -o "$FSTYP" = "local" ] ; then
 	_scratch_remount "$opts"
 	return
     fi
@@ -384,6 +408,10 @@ _test_mount()
         _overlay_test_mount $*
         return $?
     fi
+    if [ "$FSTYP" == "local" ]; then
+        mkdir -p $TEST_DIR
+	return $?
+    fi
     _test_options mount
     _mount -t $FSTYP $TEST_OPTIONS $TEST_FS_MOUNT_OPTS $SELINUX_MOUNT_OPTIONS $* $TEST_DEV $TEST_DIR
 }
@@ -392,7 +420,7 @@ _test_unmount()
 {
 	if [ "$FSTYP" == "overlay" ]; then
 		_overlay_test_unmount
-	else
+	elif [ "$FSTYP" != "local" ]; then
 		$UMOUNT_PROG $TEST_DEV
 	fi
 }
@@ -811,6 +839,9 @@ _scratch_mkfs()
     tmpfs)
 	# do nothing for tmpfs
 	;;
+    local)
+        _scratch_cleanup_files
+	;;
     f2fs)
         $MKFS_F2FS_PROG $MKFS_OPTIONS $* $SCRATCH_DEV > /dev/null
 	;;
@@ -1031,6 +1062,13 @@ _scratch_mkfs_sized()
 	fi
 	export MOUNT_OPTIONS="-o size=$fssize $TMPFS_MOUNT_OPTIONS"
 	;;
+    local)
+        _scratch_cleanup_files
+	free_space=$(_df_dir $SCRATCH_MNT | $AWK_PROG '{ print $5 }')
+	if [ "$(expr $free_space * 1024)" -lt "$fssize" ] ; then
+	   _notrun "Not enough space ($free_space) for local with $fssize bytes"
+	fi
+	;;
     *)
 	_notrun "Filesystem $FSTYP not supported in _scratch_mkfs_sized"
 	;;
@@ -1517,6 +1555,13 @@ _require_scratch_nocheck()
 		    _notrun "this test requires a valid \$SCRATCH_MNT and unique $SCRATCH_DEV"
 		fi
 		;;
+	local)
+		if [ -z "$SCRATCH_DEV" -o ! -d "$SCRATCH_MNT" ];
+		then
+		    _notrun "this test requires a valid \$SCRATCH_MNT and unique $SCRATCH_DEV"
+		fi
+		return 0
+		;;
 	*)
 		 if [ -z "$SCRATCH_DEV" -o "`_is_block_dev "$SCRATCH_DEV"`" = "" ]
 		 then
@@ -1602,6 +1647,13 @@ _require_test()
 		    _notrun "this test requires a valid \$TEST_DIR and unique $TEST_DEV"
 		fi
 		;;
+	local)
+		if [ -z "$TEST_DEV" -o ! -d "$TEST_DIR" ];
+		then
+		    _notrun "this test requires a valid \$TEST_DIR and unique $TEST_DEV"
+		fi
+		return 0
+		;;
 	*)
 		 if [ -z "$TEST_DEV" ] || [ "`_is_block_dev "$TEST_DEV"`" = "" ]
 		 then
@@ -2264,6 +2316,10 @@ _remount()
     device=$1
     mode=$2
 
+    if [ "$FSTYP" == "local" ] ; then
+       return 0
+    fi
+
     if ! mount -o remount,$mode $device
     then
         echo "_remount: failed to remount filesystem on $device as $mode"
@@ -2309,6 +2365,10 @@ _mount_or_remount_rw()
 	device=$2
 	mountpoint=$3
 
+	if [ "$FSTYP" == "local" ] ; then
+	    return 0
+	fi
+
 	if [ $USE_REMOUNT -eq 0 ]; then
 		if [ "$FSTYP" != "overlay" ]; then
 			_mount -t $FSTYP $mount_opts $device $mountpoint
@@ -2666,6 +2726,9 @@ _check_test_fs()
     tmpfs)
 	# no way to check consistency for tmpfs
 	;;
+    local)
+	# no way to check consistency for local
+	;;
     *)
 	_check_generic_filesystem $TEST_DEV
 	;;
@@ -2707,6 +2770,9 @@ _check_scratch_fs()
     tmpfs)
 	# no way to check consistency for tmpfs
 	;;
+    local)
+	# no way to check consistency for local
+	;;
     *)
 	_check_generic_filesystem $device
 	;;
@@ -3784,7 +3850,7 @@ init_rc()
 		fi
 	fi
 
-	if [ "`_fs_type $TEST_DEV`" != "$FSTYP" ]
+	if [ "$FSTYP" != "local" -a "`_fs_type $TEST_DEV`" != "$FSTYP" ]
 	then
 		echo "common/rc: Error: \$TEST_DEV ($TEST_DEV) is not a MOUNTED $FSTYP filesystem"
 		# raw $DF_PROG cannot handle NFS/CIFS/overlay correctly
diff --git a/tests/generic/027 b/tests/generic/027
index d2e59d6..73565fc 100755
--- a/tests/generic/027
+++ b/tests/generic/027
@@ -77,7 +77,7 @@ rm -f $SCRATCH_MNT/testfile
 
 loop=100
 # btrfs takes much longer time, reduce the loop count
-if [ "$FSTYP" == "btrfs" ]; then
+if [ "$FSTYP" == "btrfs" -o "$FSTYP" == "local" ]; then
 	loop=10
 fi
 
-- 
2.9.0.243.g5c589a7.dirty

Re: [PATCH] common: add support for the "local" file system type

[Eryu Guan]

On Fri, Sep 23, 2016 at 04:05:26PM -0400, Theodore Ts'o wrote:
> It is sometimes useful to be able to test the local file system
> provided in a restricted execution environment (such as that which is
> provided by Docker, for example) where it is not possible to mount and
> unmount the file system under test.

This looks useful to me. But I'm not sure what other people think.

I tested this patch a bit (ran auto group), noticed some isuses.

- Tests call _require_scratch_shutdown would shutdown your root fs, if
  $SCRATCH_MNT is on root fs and root fs is xfs. e.g. generic/044
- Tests do freeze/unfreeze would freeze your root fs, e.g. generic/068
- Tests fulfill $SCRATCH_DEV would eat all free space on root fs,
  because _scratch_mkfs_sized for "local" only checks for lower boundary
  but not upper boundary, some tests rely on the upper boundary too,
  e.g. generic/027

There might be other issues I didn't notice, since I didn't manage to
finish a "FSTYP=local ./check -g auto" run because of above issues.

> 
> To support this test case, add support for a new file system type
> called "local".  The TEST_DEV and SCRATCH_DEV should be have a
> non-block device format (e.g., local:/test or local:/scratch), and the

It's probably good to have a new fstype, as how we test NFS and overlay.
i.e. ./check -local, and do all the necessary checks as how we check NFS
and overlay setups.

> TEST_DIR and SCRATCH_MNT directories should be pre-existing
> directories provided by the execution environment.

Better to have these setups and requirements documented in README.

> 
> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
> ---
>  common/rc         | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++---
>  tests/generic/027 |  2 +-
>  2 files changed, 70 insertions(+), 4 deletions(-)
> 
> diff --git a/common/rc b/common/rc
> index 70d79c9..c03b132 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -330,12 +330,29 @@ _overlay_scratch_unmount()
>  	$UMOUNT_PROG $SCRATCH_MNT
>  }
>  
> +_local_validate_mount_opt()
> +{
> +    case "$*" in
> +	ro|ro,*|*,ro) _notrun "ro mount option not supported" ;;
> +	*nosuid*) _notrun "nosuid mount option not supported" ;;
> +	*noatime*) _notrun "noatime mount option not supported" ;;
> +	*relatime*) _notrun "relatime mount option not supported" ;;
> +	*diratime*) _notrun "diratime mount option not supported" ;;
> +	*strictatime*) _notrun "strictatime mount option not supported" ;;
> +    esac
> +}
> +

It's good to have some comments on this function.

>  _scratch_mount()
>  {
>      if [ "$FSTYP" == "overlay" ]; then
>          _overlay_scratch_mount $*
>          return $?
>      fi
> +    if [ "$FSTYP" == "local" ]; then
> +	_local_validate_mount_opt "$*"
> +        mkdir -p $SCRATCH_MNT
> +	return $?
> +    fi

Perhaps need a new helper like "_local_scratch_mount" here, and it's a
good time to covert "if" to "case" switch, I think :)

>      _mount -t $FSTYP `_scratch_mount_options $*`
>  }
>  
> @@ -348,6 +365,9 @@ _scratch_unmount()
>  	btrfs)
>  		$UMOUNT_PROG $SCRATCH_MNT
>  		;;
> +	local)
> +                rm -rf $SCRATCH_MNT/*
> +                ;;
>  	*)
>  		$UMOUNT_PROG $SCRATCH_DEV
>  		;;
> @@ -358,6 +378,10 @@ _scratch_remount()
>  {
>      local opts="$1"
>  
> +    if [ "$FSTYP" = "local" ]; then
> +	_local_validate_mount_opt "$*"
> +	return 0;
> +    fi
>      if test -n "$opts"; then
>  	mount -o "remount,$opts" $SCRATCH_MNT
>      fi
> @@ -367,7 +391,7 @@ _scratch_cycle_mount()
>  {
>      local opts="$1"
>  
> -    if [ "$FSTYP" = tmpfs ]; then
> +    if [ "$FSTYP" = "tmpfs" -o "$FSTYP" = "local" ] ; then
>  	_scratch_remount "$opts"
>  	return
>      fi
> @@ -384,6 +408,10 @@ _test_mount()
>          _overlay_test_mount $*
>          return $?
>      fi
> +    if [ "$FSTYP" == "local" ]; then
> +        mkdir -p $TEST_DIR
> +	return $?
> +    fi
>      _test_options mount
>      _mount -t $FSTYP $TEST_OPTIONS $TEST_FS_MOUNT_OPTS $SELINUX_MOUNT_OPTIONS $* $TEST_DEV $TEST_DIR
>  }
> @@ -392,7 +420,7 @@ _test_unmount()
>  {
>  	if [ "$FSTYP" == "overlay" ]; then
>  		_overlay_test_unmount
> -	else
> +	elif [ "$FSTYP" != "local" ]; then
>  		$UMOUNT_PROG $TEST_DEV
>  	fi
>  }
> @@ -811,6 +839,9 @@ _scratch_mkfs()
>      tmpfs)
>  	# do nothing for tmpfs
>  	;;
> +    local)
> +        _scratch_cleanup_files
> +	;;
>      f2fs)
>          $MKFS_F2FS_PROG $MKFS_OPTIONS $* $SCRATCH_DEV > /dev/null
>  	;;
> @@ -1031,6 +1062,13 @@ _scratch_mkfs_sized()
>  	fi
>  	export MOUNT_OPTIONS="-o size=$fssize $TMPFS_MOUNT_OPTIONS"
>  	;;
> +    local)
> +        _scratch_cleanup_files
> +	free_space=$(_df_dir $SCRATCH_MNT | $AWK_PROG '{ print $5 }')
> +	if [ "$(expr $free_space * 1024)" -lt "$fssize" ] ; then
> +	   _notrun "Not enough space ($free_space) for local with $fssize bytes"
> +	fi
> +	;;

I'd prefer dropping _scratch_mkfs_sized support for "local", because of
the generic/027 problem I met above.

Thanks,
Eryu

>      *)
>  	_notrun "Filesystem $FSTYP not supported in _scratch_mkfs_sized"
>  	;;
> @@ -1517,6 +1555,13 @@ _require_scratch_nocheck()
>  		    _notrun "this test requires a valid \$SCRATCH_MNT and unique $SCRATCH_DEV"
>  		fi
>  		;;
> +	local)
> +		if [ -z "$SCRATCH_DEV" -o ! -d "$SCRATCH_MNT" ];
> +		then
> +		    _notrun "this test requires a valid \$SCRATCH_MNT and unique $SCRATCH_DEV"
> +		fi
> +		return 0
> +		;;
>  	*)
>  		 if [ -z "$SCRATCH_DEV" -o "`_is_block_dev "$SCRATCH_DEV"`" = "" ]
>  		 then
> @@ -1602,6 +1647,13 @@ _require_test()
>  		    _notrun "this test requires a valid \$TEST_DIR and unique $TEST_DEV"
>  		fi
>  		;;
> +	local)
> +		if [ -z "$TEST_DEV" -o ! -d "$TEST_DIR" ];
> +		then
> +		    _notrun "this test requires a valid \$TEST_DIR and unique $TEST_DEV"
> +		fi
> +		return 0
> +		;;
>  	*)
>  		 if [ -z "$TEST_DEV" ] || [ "`_is_block_dev "$TEST_DEV"`" = "" ]
>  		 then
> @@ -2264,6 +2316,10 @@ _remount()
>      device=$1
>      mode=$2
>  
> +    if [ "$FSTYP" == "local" ] ; then
> +       return 0
> +    fi
> +
>      if ! mount -o remount,$mode $device
>      then
>          echo "_remount: failed to remount filesystem on $device as $mode"
> @@ -2309,6 +2365,10 @@ _mount_or_remount_rw()
>  	device=$2
>  	mountpoint=$3
>  
> +	if [ "$FSTYP" == "local" ] ; then
> +	    return 0
> +	fi
> +
>  	if [ $USE_REMOUNT -eq 0 ]; then
>  		if [ "$FSTYP" != "overlay" ]; then
>  			_mount -t $FSTYP $mount_opts $device $mountpoint
> @@ -2666,6 +2726,9 @@ _check_test_fs()
>      tmpfs)
>  	# no way to check consistency for tmpfs
>  	;;
> +    local)
> +	# no way to check consistency for local
> +	;;
>      *)
>  	_check_generic_filesystem $TEST_DEV
>  	;;
> @@ -2707,6 +2770,9 @@ _check_scratch_fs()
>      tmpfs)
>  	# no way to check consistency for tmpfs
>  	;;
> +    local)
> +	# no way to check consistency for local
> +	;;
>      *)
>  	_check_generic_filesystem $device
>  	;;
> @@ -3784,7 +3850,7 @@ init_rc()
>  		fi
>  	fi
>  
> -	if [ "`_fs_type $TEST_DEV`" != "$FSTYP" ]
> +	if [ "$FSTYP" != "local" -a "`_fs_type $TEST_DEV`" != "$FSTYP" ]
>  	then
>  		echo "common/rc: Error: \$TEST_DEV ($TEST_DEV) is not a MOUNTED $FSTYP filesystem"
>  		# raw $DF_PROG cannot handle NFS/CIFS/overlay correctly
> diff --git a/tests/generic/027 b/tests/generic/027
> index d2e59d6..73565fc 100755
> --- a/tests/generic/027
> +++ b/tests/generic/027
> @@ -77,7 +77,7 @@ rm -f $SCRATCH_MNT/testfile
>  
>  loop=100
>  # btrfs takes much longer time, reduce the loop count
> -if [ "$FSTYP" == "btrfs" ]; then
> +if [ "$FSTYP" == "btrfs" -o "$FSTYP" == "local" ]; then
>  	loop=10
>  fi
>  
> -- 
> 2.9.0.243.g5c589a7.dirty
> 
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] common: add support for the "local" file system type

[Theodore Ts'o]

On Mon, Sep 26, 2016 at 09:25:03PM +0800, Eryu Guan wrote:
> On Fri, Sep 23, 2016 at 04:05:26PM -0400, Theodore Ts'o wrote:
> > It is sometimes useful to be able to test the local file system
> > provided in a restricted execution environment (such as that which is
> > provided by Docker, for example) where it is not possible to mount and
> > unmount the file system under test.
> 
> This looks useful to me. But I'm not sure what other people think.
> 
> I tested this patch a bit (ran auto group), noticed some isuses.
> 
> - Tests call _require_scratch_shutdown would shutdown your root fs, if
>   $SCRATCH_MNT is on root fs and root fs is xfs. e.g. generic/044
> - Tests do freeze/unfreeze would freeze your root fs, e.g. generic/068
> - Tests fulfill $SCRATCH_DEV would eat all free space on root fs,
>   because _scratch_mkfs_sized for "local" only checks for lower boundary
>   but not upper boundary, some tests rely on the upper boundary too,
>   e.g. generic/027
> 
> There might be other issues I didn't notice, since I didn't manage to
> finish a "FSTYP=local ./check -g auto" run because of above issues.

Oops, I was testing on small test devices using ext4, so I didn't
notice these issues.  I'll fix them up.

> > To support this test case, add support for a new file system type
> > called "local".  The TEST_DEV and SCRATCH_DEV should be have a
> > non-block device format (e.g., local:/test or local:/scratch), and the
> 
> It's probably good to have a new fstype, as how we test NFS and overlay.
> i.e. ./check -local, and do all the necessary checks as how we check NFS
> and overlay setups.

Even for NFS and overlayfs there are some tests we do where mounting
and remounting the file system (e.g., with the ro mount option)
probably does make sense.  Although I do agree there are a large
number of the NFS mounts and umounts which are largely pointless.

I was implementing the local file systme type for situations where it
is simply *not* *possible* at all to mount and unmount the underlying
file system because it was operating inside a docker container where
even root didn't have access to modify the supplied file system.
(Yes, in some cases we could test the underlying file system, but not
all, and it is useful to have end-to-end tests.)

> > TEST_DIR and SCRATCH_MNT directories should be pre-existing
> > directories provided by the execution environment.
> 
> Better to have these setups and requirements documented in README.

Agreed, I'll fix this in the next spin of the patch.

	     	      	     	       	      - Ted

Re: [PATCH] common: add support for the "local" file system type

[Eryu Guan]

On Mon, Sep 26, 2016 at 11:14:31AM -0400, Theodore Ts'o wrote:
> On Mon, Sep 26, 2016 at 09:25:03PM +0800, Eryu Guan wrote:
> > On Fri, Sep 23, 2016 at 04:05:26PM -0400, Theodore Ts'o wrote:
> > > It is sometimes useful to be able to test the local file system
> > > provided in a restricted execution environment (such as that which is
> > > provided by Docker, for example) where it is not possible to mount and
> > > unmount the file system under test.
> > 
> > This looks useful to me. But I'm not sure what other people think.
> > 
> > I tested this patch a bit (ran auto group), noticed some isuses.
> > 
> > - Tests call _require_scratch_shutdown would shutdown your root fs, if
> >   $SCRATCH_MNT is on root fs and root fs is xfs. e.g. generic/044
> > - Tests do freeze/unfreeze would freeze your root fs, e.g. generic/068
> > - Tests fulfill $SCRATCH_DEV would eat all free space on root fs,
> >   because _scratch_mkfs_sized for "local" only checks for lower boundary
> >   but not upper boundary, some tests rely on the upper boundary too,
> >   e.g. generic/027
> > 
> > There might be other issues I didn't notice, since I didn't manage to
> > finish a "FSTYP=local ./check -g auto" run because of above issues.
> 
> Oops, I was testing on small test devices using ext4, so I didn't
> notice these issues.  I'll fix them up.
> 
> > > To support this test case, add support for a new file system type
> > > called "local".  The TEST_DEV and SCRATCH_DEV should be have a
> > > non-block device format (e.g., local:/test or local:/scratch), and the
> > 
> > It's probably good to have a new fstype, as how we test NFS and overlay.
> > i.e. ./check -local, and do all the necessary checks as how we check NFS
> > and overlay setups.
> 
> Even for NFS and overlayfs there are some tests we do where mounting
> and remounting the file system (e.g., with the ro mount option)
> probably does make sense.  Although I do agree there are a large
> number of the NFS mounts and umounts which are largely pointless.
> 
> I was implementing the local file systme type for situations where it
> is simply *not* *possible* at all to mount and unmount the underlying
> file system because it was operating inside a docker container where
> even root didn't have access to modify the supplied file system.
> (Yes, in some cases we could test the underlying file system, but not
> all, and it is useful to have end-to-end tests.)

Sorry, I should have been more clear.

It has nothing to do with mount & umount, it's about adding a new
"-local" option to "check", like "-nfs" and "-overlay". And do
TEST_DEV/TEST_DIR and SCRATCH_DEV/SCRATCH_MNT validations in
_require_test and _require_scratch_nocheck, as how we do the check for
NFS and overlayfs. So we don't have to work around the validation by
specifying TEST_DEV/SCRATCH_DEV in a non-block device format (e.g.
local:/test to mimic an NFS export).

Thanks,
Eryu

Re: [PATCH] common: add support for the "local" file system type

[Theodore Ts'o]

On Tue, Sep 27, 2016 at 05:55:29PM +0800, Eryu Guan wrote:
> 
> It has nothing to do with mount & umount, it's about adding a new
> "-local" option to "check", like "-nfs" and "-overlay". And do
> TEST_DEV/TEST_DIR and SCRATCH_DEV/SCRATCH_MNT validations in
> _require_test and _require_scratch_nocheck, as how we do the check for
> NFS and overlayfs. So we don't have to work around the validation by
> specifying TEST_DEV/SCRATCH_DEV in a non-block device format (e.g.
> local:/test to mimic an NFS export).

I'd have to dig and check, but as I recall, there were assumptions
that foo:/bar means "no block device" scattered around, and it's why
we have to use a similar naming scheme for tmpfs.  Is that really a
problem?  I had thought it was a standard convention for xfstests.

If we want to avoid forcing testers to use a foo:/bar naming scheme
for TEST_DEV and SCRATCH_DEV, instead of a new command-line variable,
I'd much rather have a function in rc/common that could be keyed off
the file system type.  You have to specify whether you are using tmpfs
or nfs or local in FSTYP anyway, so why not just use that?

Cheers,

						- Ted

Re: [PATCH] common: add support for the "local" file system type

[Dave Chinner]

On Fri, Sep 23, 2016 at 04:05:26PM -0400, Theodore Ts'o wrote:
> It is sometimes useful to be able to test the local file system
> provided in a restricted execution environment (such as that which is
> provided by Docker, for example) where it is not possible to mount and
> unmount the file system under test.

Ignoring the code changes for the moment - what's the reason/use
case for running /kernel/ test suites inside a user container with
such restricted access to filesystems and devices? Especially
considering test failures can take the kernel down, which will kill
/everything/ on the machine, not just the container.

This doesn't make a whole lot of sense to me - who is going to use
this and what extra test coverage does it bring to the table
compared to just running inside a guest VM with full privileges?

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH] common: add support for the "local" file system type

[Theodore Ts'o]

On Tue, Sep 27, 2016 at 08:18:14AM +1000, Dave Chinner wrote:
> 
> Ignoring the code changes for the moment - what's the reason/use
> case for running /kernel/ test suites inside a user container with
> such restricted access to filesystems and devices? Especially
> considering test failures can take the kernel down, which will kill
> /everything/ on the machine, not just the container.
> 
> This doesn't make a whole lot of sense to me - who is going to use
> this and what extra test coverage does it bring to the table
> compared to just running inside a guest VM with full privileges?

So I'm doing this work on behalf of a team who is working on an
to-be-announced project, so the details aren't mine to give.  However,
I can say that obviously, if they could do that they would.

Let me give an other example where this might be useful.  Suppose you
are using the Windows Subsystem for Linux which emulates system calls
for Linux, so you can run most Linux binaries, but for which certain
system calls simply aren't implemented, either for security reasons or
because they just don't make sense (e.g., reboot, kexec, mount,
unmount, etc.).  To the extent that such a subsystem purports to offer
Linux compatibility, and has a Linux VFS emulation, it would be useful
to have a "local" file system type which doesn't try to do any mounts
and unmounts, but which could otherwise stress test the file system
layer and find bugs in the system call emulation for file systems,
that would be a good and useful thing to do.

As I recall, AIX also had a Linux system emulator (the better to run
the huge numbers of commercial third-party Linux binaries :-), and if
I recall correctly, it didn't support mount and unmount either.
NetBSD also has a Linux emulation layer (and in the past, an Irix
emulation layer), and again, it's not clear that it would extend to
things like mount/unmount --- nor would it really need to in order to
be useful for most of their desired use cases.

Cheers,

						- Ted

Re: [PATCH] common: add support for the "local" file system type

[Dave Chinner]

On Wed, Sep 28, 2016 at 07:57:13PM -0400, Theodore Ts'o wrote:
> On Tue, Sep 27, 2016 at 08:18:14AM +1000, Dave Chinner wrote:
> > 
> > Ignoring the code changes for the moment - what's the reason/use
> > case for running /kernel/ test suites inside a user container with
> > such restricted access to filesystems and devices? Especially
> > considering test failures can take the kernel down, which will kill
> > /everything/ on the machine, not just the container.
> > 
> > This doesn't make a whole lot of sense to me - who is going to use
> > this and what extra test coverage does it bring to the table
> > compared to just running inside a guest VM with full privileges?
> 
> So I'm doing this work on behalf of a team who is working on an
> to-be-announced project, so the details aren't mine to give.  However,
> I can say that obviously, if they could do that they would.

So, sooper-s3kr3t internal google stuff that you can't talk about
and may never see the light of day. When it is announced and
released, then let's talk about integration....

> Let me give an other example where this might be useful.  Suppose you
> are using the Windows Subsystem for Linux which emulates system calls
> for Linux, so you can run most Linux binaries, but for which certain
> system calls simply aren't implemented, either for security reasons or
> because they just don't make sense (e.g., reboot, kexec, mount,
> unmount, etc.).  To the extent that such a subsystem purports to offer
> Linux compatibility, and has a Linux VFS emulation, it would be useful
> to have a "local" file system type which doesn't try to do any mounts
> and unmounts, but which could otherwise stress test the file system
> layer and find bugs in the system call emulation for file systems,
> that would be a good and useful thing to do.

xfstests is not designed for validating system call API compliance -
it's for exercising /filesystem implementations/.  xfstests assumes
the syscall API is valid and working, and tries to break the
underlying storage implementation.  As such, your example really
isn't something you should be using xfstests for - it doesn't test
anything like what is needed to verify that the "VFS emulation" is
valid and complete and working exactly as documented in the linux
man pages.

As it is, we have a test harness that is supposed to validate
Linux syscall behaviour exactly as you are describing: LTP.

> As I recall, AIX also had a Linux system emulator (the better to run
> the huge numbers of commercial third-party Linux binaries :-), and if
> I recall correctly, it didn't support mount and unmount either.
> NetBSD also has a Linux emulation layer (and in the past, an Irix
> emulation layer), and again, it's not clear that it would extend to
> things like mount/unmount --- nor would it really need to in order to
> be useful for most of their desired use cases.

xfstests would be implemented on such platforms using the native
mount/unmount commands, just like it is for Irix. Indeed, in this
situation one should be testing the native filesystem implementation
with xfstests, and then using LTP to test the linux emulation layer
behaves as expected.

Use the right tool for the job....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH] common: add support for the "local" file system type

[Theodore Ts'o]

On Thu, Sep 29, 2016 at 12:16:29PM +1000, Dave Chinner wrote:
> 
> So, sooper-s3kr3t internal google stuff that you can't talk about
> and may never see the light of day. When it is announced and
> released, then let's talk about integration....

Fair enough, I'll carry it as an out-of-tree patch at

https://github.com/tytso/xfstests

... for anyone else who might find it useful for their purposes.

> xfstests is not designed for validating system call API compliance -
> it's for exercising /filesystem implementations/.  xfstests assumes
> the syscall API is valid and working, and tries to break the
> underlying storage implementation.  As such, your example really
> isn't something you should be using xfstests for - it doesn't test
> anything like what is needed to verify that the "VFS emulation" is
> valid and complete and working exactly as documented in the linux
> man pages.

Who says there isn't an underlying file system implementation which we
want to test?  In fact we *are* using the right tool for the job.
(I'm quite aware of the other testing tools that might be available
including LTP and others, such as the LSB tests.)

Cheers,

							- Ted

Re: [PATCH] common: add support for the "local" file system type

[Dave Chinner]

On Wed, Sep 28, 2016 at 11:56:36PM -0400, Theodore Ts'o wrote:
> On Thu, Sep 29, 2016 at 12:16:29PM +1000, Dave Chinner wrote:
> > 
> > So, sooper-s3kr3t internal google stuff that you can't talk
> > about and may never see the light of day. When it is announced
> > and released, then let's talk about integration....
> 
> Fair enough, I'll carry it as an out-of-tree patch at
> 
> https://github.com/tytso/xfstests
> 
> ... for anyone else who might find it useful for their purposes.
> 
> > xfstests is not designed for validating system call API
> > compliance - it's for exercising /filesystem implementations/.
> > xfstests assumes the syscall API is valid and working, and tries
> > to break the underlying storage implementation.  As such, your
> > example really isn't something you should be using xfstests for
> > - it doesn't test anything like what is needed to verify that
> > the "VFS emulation" is valid and complete and working exactly as
> > documented in the linux man pages.
> 
> Who says there isn't an underlying file system implementation
> which we want to test? In fact we *are* using the right tool for the job.

Nobody outside the google echo chamber can say, Ted, because you
haven't told anyone what it is you want this for. binary emulation
layers (your example, not mine) don't implement persistent
filesystems - they translate syscalls.

So let's revist this when everyone has the same information
available and we can have an informed discussion....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH] common: add support for the "local" file system type

[Theodore Ts'o]

On Thu, Sep 29, 2016 at 03:37:52PM +1000, Dave Chinner wrote:
> Nobody outside the google echo chamber can say, Ted, because you
> haven't told anyone what it is you want this for. binary emulation
> layers (your example, not mine) don't implement persistent
> filesystems - they translate syscalls.

Is your only objection that you don't get to can't see what it's going
to test?

That's generally not a reason to object to patches in various open
source projects --- there are plenty of closed source programs for
which we accept kernel patches, for example.

But sure, if that's the way you want to roll, as I said, I'll carry it
as an out of tree patch.

						- Ted

Re: [PATCH] common: add support for the "local" file system type

[Dave Chinner]

On Thu, Sep 29, 2016 at 09:05:01AM -0400, Theodore Ts'o wrote:
> On Thu, Sep 29, 2016 at 03:37:52PM +1000, Dave Chinner wrote:
> > Nobody outside the google echo chamber can say, Ted, because you
> > haven't told anyone what it is you want this for. binary emulation
> > layers (your example, not mine) don't implement persistent
> > filesystems - they translate syscalls.
> 
> Is your only objection that you don't get to can't see what it's going
> to test?

No, it isn't. But it's the most important one on my list right now
because it affects fundamental architecture assumptions the test
harness is based upon. I haven't looked at the implementation,
because if we can't validate the basic architecture there's no point
even looking at the code.

As it is, Eryu and Eric have already outlined a number of serious
issues with the implementation (and your testing process). They've
highlighted a number of magic snowflake conditions we'd have to
sprinkle throughout random tests and maintain forever more. What's
missing from your patch request is the information that allows us to
determine if the benefits of this patchset outweigh the obvious,
significant, ongoing cost of having to maintain these snowflakes.

Ted, as a kernel subsystem maintainer, you should know that
understanding the full picture is a neccessary aspect of reviewing
new proposals. Do you really merge code into ext4 that you think is
going to be a significant future burden on the maintainer without
knowing why it is needed, how it will be used, or even how you'll
maintain it in a working condition over the long term?

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH] common: add support for the "local" file system type

[Theodore Ts'o]

On Fri, Sep 30, 2016 at 08:49:13AM +1000, Dave Chinner wrote:
> As it is, Eryu and Eric have already outlined a number of serious
> issues with the implementation (and your testing process). They've
> highlighted a number of magic snowflake conditions we'd have to
> sprinkle throughout random tests and maintain forever more. What's
> missing from your patch request is the information that allows us to
> determine if the benefits of this patchset outweigh the obvious,
> significant, ongoing cost of having to maintain these snowflakes.

The issues Eryu and Eric pointed out (and many thanks for their
review) were ones that are easily fixed, and I can restrict the
changes to common/* (right now only common/rc), and I no longer need
any changes in the tests/* hierarchy.

My first patch had only a single "snowflake" to generic/027 (and it
was one that already had a snowflake exemption for btrfs).  It was
done to work around the fact that _scratch_mkfs_sized wouldn't
restrict the size of the file system.  I now will _notrun any test
which uses _scratch_mkfs_sized, so I don't need that change any
longer.  This leaves only a single enospc test which will be run,
which is a bit unfortunate, but it solves the "I could fill the root
file system if TEST_DEV or SCRATCH_DEV is pointed on my root fs"
problem".

So I believe the maintainance issue is something that can be be
mitigated.

Testing is something that can be done using a standard Linux systems.
(Indeed I did most of my testing using my existing gce-xfstests and
kvm-xfstests frameworks, just because it was easier.)  It's a fair
point that I should make sure it doesn't cause problems when other
file systems are used as the base file system, and if it is used on a
root file system.  Again, these are issues which I can fix in the
patch.  Indeed, all of the the specific issues pointed out by Eric and
Eryu are already fixed in my tree.

In any case, since the commit are only adding some tests in common/rc
(where we have lots of per-file system conditionals anyway), I can
easily maintain this patch out-of-tree, since you seem to have
fundamental philosophical problems with it.

Regards,

						- Ted

Re: [PATCH] common: add support for the "local" file system type

[Eric Sandeen]

On 9/28/16 10:56 PM, Theodore Ts'o wrote:
>> xfstests is not designed for validating system call API compliance -
>> > it's for exercising /filesystem implementations/.  xfstests assumes
>> > the syscall API is valid and working, and tries to break the
>> > underlying storage implementation.

To play devil's advocate, I think there are a few tests that find problems
pretty high up the callchain, possibly in the vfs or syscall layer...

>>  As such, your example really
>> > isn't something you should be using xfstests for - it doesn't test
>> > anything like what is needed to verify that the "VFS emulation" is
>> > valid and complete and working exactly as documented in the linux
>> > man pages.
> Who says there isn't an underlying file system implementation which we
> want to test?  In fact we *are* using the right tool for the job.
> (I'm quite aware of the other testing tools that might be available
> including LTP and others, such as the LSB tests.)

So you want to test a filesystem without being able to mount/unmount...
Is this because you want to test how things behave in a restricted
privs environment of some sort, or because the system under test has
absolutely no way to mount & unmount a filesystem for any user?

So many of the current tests expect to check fs consistency, check
data integrity after a shutdown or remount, test recovery, examine
on-disk structures of a quiesced, consistent block device, test mount
options, etc, I fear that there will be so many special snowflakes scattered
around individual tests that this might be very hard to maintain.

-Eric

Re: [PATCH] common: add support for the "local" file system type

[Eric Sandeen]

On 9/23/16 3:05 PM, Theodore Ts'o wrote:
> It is sometimes useful to be able to test the local file system
> provided in a restricted execution environment (such as that which is
> provided by Docker, for example) where it is not possible to mount and
> unmount the file system under test.

Big picture concerns logged elsewhere, but other things below :)
 
> To support this test case, add support for a new file system type
> called "local".  The TEST_DEV and SCRATCH_DEV should be have a
> non-block device format (e.g., local:/test or local:/scratch),

That convention should be documented in the README...

As Eryu found, running this on xfs could be disastrous as it stands.
You checked ext4, but it probably needs a thorough workout for multiple
filesystem types with various  block sizes, mount options etc to
see if this really covers the bases of running on an essentially
uncontrolled, unknown filesystem environment...

> and the
> TEST_DIR and SCRATCH_MNT directories should be pre-existing
> directories provided by the execution environment.

no different than we have today, right?

> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
> ---
>  common/rc         | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++---
>  tests/generic/027 |  2 +-
>  2 files changed, 70 insertions(+), 4 deletions(-)
> 
> diff --git a/common/rc b/common/rc
> index 70d79c9..c03b132 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -330,12 +330,29 @@ _overlay_scratch_unmount()
>  	$UMOUNT_PROG $SCRATCH_MNT
>  }
>  
> +_local_validate_mount_opt()
> +{
> +    case "$*" in
> +	ro|ro,*|*,ro) _notrun "ro mount option not supported" ;;
> +	*nosuid*) _notrun "nosuid mount option not supported" ;;
> +	*noatime*) _notrun "noatime mount option not supported" ;;
> +	*relatime*) _notrun "relatime mount option not supported" ;;
> +	*diratime*) _notrun "diratime mount option not supported" ;;
> +	*strictatime*) _notrun "strictatime mount option not supported" ;;
> +    esac
> +}

Does this really cover them all?  The number of mount options that might
be tested across all filesystems is huge.  I'd almost say that if /any/ mount
option is specified, fail, because you have /no/ control over an already-mounted
fs's behavior in your (unspecified) environment.

> +
>  _scratch_mount()
>  {
>      if [ "$FSTYP" == "overlay" ]; then
>          _overlay_scratch_mount $*
>          return $?
>      fi
> +    if [ "$FSTYP" == "local" ]; then
> +	_local_validate_mount_opt "$*"
> +        mkdir -p $SCRATCH_MNT

surely $SCRATCH_MNT should already exist?

> +	return $?
> +    fi
>      _mount -t $FSTYP `_scratch_mount_options $*`
>  }
>  
> @@ -348,6 +365,9 @@ _scratch_unmount()
>  	btrfs)
>  		$UMOUNT_PROG $SCRATCH_MNT
>  		;;
> +	local)
> +                rm -rf $SCRATCH_MNT/*
> +                ;;

unmounting scratch doesn't run mkfs in the normal xfstests environment,
so removing everything doesn't make sense to me.  I'd expect this in
_scratch_mkfs, not _scratch_unmount.

>  	*)
>  		$UMOUNT_PROG $SCRATCH_DEV
>  		;;
> @@ -358,6 +378,10 @@ _scratch_remount()
>  {
>      local opts="$1"
>  
> +    if [ "$FSTYP" = "local" ]; then
> +	_local_validate_mount_opt "$*"
> +	return 0;
> +    fi
>      if test -n "$opts"; then
>  	mount -o "remount,$opts" $SCRATCH_MNT
>      fi
> @@ -367,7 +391,7 @@ _scratch_cycle_mount()
>  {
>      local opts="$1"
>  
> -    if [ "$FSTYP" = tmpfs ]; then
> +    if [ "$FSTYP" = "tmpfs" -o "$FSTYP" = "local" ] ; then
>  	_scratch_remount "$opts"
>  	return

tmpfs is special because an unmount loses all data.  Not clear why
a local fs would need this treatment.  I'd expect an explicit no-op
(even if that's what _scratch_remount does for this type)

>      fi
> @@ -384,6 +408,10 @@ _test_mount()
>          _overlay_test_mount $*
>          return $?
>      fi
> +    if [ "$FSTYP" == "local" ]; then
> +        mkdir -p $TEST_DIR
> +	return $?
> +    fi

Again, surely this should already exist?

>      _test_options mount
>      _mount -t $FSTYP $TEST_OPTIONS $TEST_FS_MOUNT_OPTS $SELINUX_MOUNT_OPTIONS $* $TEST_DEV $TEST_DIR
>  }
> @@ -392,7 +420,7 @@ _test_unmount()
>  {
>  	if [ "$FSTYP" == "overlay" ]; then
>  		_overlay_test_unmount
> -	else
> +	elif [ "$FSTYP" != "local" ]; then
>  		$UMOUNT_PROG $TEST_DEV
>  	fi
>  }
> @@ -811,6 +839,9 @@ _scratch_mkfs()
>      tmpfs)
>  	# do nothing for tmpfs
>  	;;
> +    local)
> +        _scratch_cleanup_files
> +	;;

ok, so you do what I suggested earlier, here ;)

>      f2fs)
>          $MKFS_F2FS_PROG $MKFS_OPTIONS $* $SCRATCH_DEV > /dev/null
>  	;;
> @@ -1031,6 +1062,13 @@ _scratch_mkfs_sized()
>  	fi
>  	export MOUNT_OPTIONS="-o size=$fssize $TMPFS_MOUNT_OPTIONS"
>  	;;
> +    local)
> +        _scratch_cleanup_files
> +	free_space=$(_df_dir $SCRATCH_MNT | $AWK_PROG '{ print $5 }')
> +	if [ "$(expr $free_space * 1024)" -lt "$fssize" ] ; then
> +	   _notrun "Not enough space ($free_space) for local with $fssize bytes"
> +	fi
> +	;;

icky indentation here, at least ...

Wouldn't _require_fs_space be more straightforward here?

>      *)
>  	_notrun "Filesystem $FSTYP not supported in _scratch_mkfs_sized"
>  	;;
> @@ -1517,6 +1555,13 @@ _require_scratch_nocheck()
>  		    _notrun "this test requires a valid \$SCRATCH_MNT and unique $SCRATCH_DEV"
>  		fi
>  		;;
> +	local)
> +		if [ -z "$SCRATCH_DEV" -o ! -d "$SCRATCH_MNT" ];
> +		then
> +		    _notrun "this test requires a valid \$SCRATCH_MNT and unique $SCRATCH_DEV"
> +		fi
> +		return 0
> +		;;

Hm, I guess I'd need to read an updated README to know if this test is correct.

>  	*)
>  		 if [ -z "$SCRATCH_DEV" -o "`_is_block_dev "$SCRATCH_DEV"`" = "" ]
>  		 then
> @@ -1602,6 +1647,13 @@ _require_test()
>  		    _notrun "this test requires a valid \$TEST_DIR and unique $TEST_DEV"
>  		fi
>  		;;
> +	local)
> +		if [ -z "$TEST_DEV" -o ! -d "$TEST_DIR" ];
> +		then
> +		    _notrun "this test requires a valid \$TEST_DIR and unique $TEST_DEV"
> +		fi
> +		return 0
> +		;;

same here.

>  	*)
>  		 if [ -z "$TEST_DEV" ] || [ "`_is_block_dev "$TEST_DEV"`" = "" ]
>  		 then
> @@ -2264,6 +2316,10 @@ _remount()
>      device=$1
>      mode=$2
>  
> +    if [ "$FSTYP" == "local" ] ; then
> +       return 0
> +    fi
> +
>      if ! mount -o remount,$mode $device
>      then
>          echo "_remount: failed to remount filesystem on $device as $mode"
> @@ -2309,6 +2365,10 @@ _mount_or_remount_rw()
>  	device=$2
>  	mountpoint=$3
>  
> +	if [ "$FSTYP" == "local" ] ; then
> +	    return 0
> +	fi
> +
>  	if [ $USE_REMOUNT -eq 0 ]; then
>  		if [ "$FSTYP" != "overlay" ]; then
>  			_mount -t $FSTYP $mount_opts $device $mountpoint
> @@ -2666,6 +2726,9 @@ _check_test_fs()
>      tmpfs)
>  	# no way to check consistency for tmpfs
>  	;;
> +    local)
> +	# no way to check consistency for local
> +	;;
>      *)
>  	_check_generic_filesystem $TEST_DEV
>  	;;
> @@ -2707,6 +2770,9 @@ _check_scratch_fs()
>      tmpfs)
>  	# no way to check consistency for tmpfs
>  	;;
> +    local)
> +	# no way to check consistency for local
> +	;;
>      *)
>  	_check_generic_filesystem $device
>  	;;
> @@ -3784,7 +3850,7 @@ init_rc()
>  		fi
>  	fi
>  
> -	if [ "`_fs_type $TEST_DEV`" != "$FSTYP" ]
> +	if [ "$FSTYP" != "local" -a "`_fs_type $TEST_DEV`" != "$FSTYP" ]
>  	then
>  		echo "common/rc: Error: \$TEST_DEV ($TEST_DEV) is not a MOUNTED $FSTYP filesystem"
>  		# raw $DF_PROG cannot handle NFS/CIFS/overlay correctly

I suppose for "local" I'd check that $TEST_DIR exists...

> diff --git a/tests/generic/027 b/tests/generic/027
> index d2e59d6..73565fc 100755
> --- a/tests/generic/027
> +++ b/tests/generic/027
> @@ -77,7 +77,7 @@ rm -f $SCRATCH_MNT/testfile
>  
>  loop=100
>  # btrfs takes much longer time, reduce the loop count
> -if [ "$FSTYP" == "btrfs" ]; then
> +if [ "$FSTYP" == "btrfs" -o "$FSTYP" == "local" ]; then
>  	loop=10
>  fi

Ok, why?

This is the kind of special snowflake I worry about, I would not expect
it to be the last... ;)

-Eric