[PATCH v3 0/5] Overlayfs exportfs tests

[PATCH v3 0/5] Overlayfs exportfs tests

[Amir Goldstein]

Eryu,

Assuming you did not get to reviewing the overlay/exportfs
v2 patches yet, here is a better version for you to test/review,
which I had tested with various OVERLAY_MOUNT_OPTIONS variants.

NFS export support patches are now on overlayfs-next and all
overlay/exportfs tests are expected to pass with overlayfs-next and
default kernel configuration.

Changes since v2:
- Fix the index feature requirement
- Implement _require_scratch_features (plural) to check dependent
  overlay features together (nfs_export and index)

Thanks,
Amir.

Amir Goldstein (5):
  fstests: implement require of multiple overlayfs features
  overlay: test encode/decode overlay file handles
  overlay: test encode/decode of non-samefs overlay file handles
  overlay: test encode/decode overlay file handles with renames
  overlay: test encode/decode of non-samefs overlay file handles with
    renames

 common/overlay        |  23 ++++-
 common/rc             |  15 +--
 tests/overlay/050     | 237 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/overlay/050.out |  31 ++++++
 tests/overlay/051     | 254 ++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/overlay/051.out |  31 ++++++
 tests/overlay/052     | 182 ++++++++++++++++++++++++++++++++++++
 tests/overlay/052.out |  22 +++++
 tests/overlay/053     | 198 +++++++++++++++++++++++++++++++++++++++
 tests/overlay/053.out |  22 +++++
 tests/overlay/group   |   4 +
 11 files changed, 1010 insertions(+), 9 deletions(-)
 create mode 100755 tests/overlay/050
 create mode 100644 tests/overlay/050.out
 create mode 100755 tests/overlay/051
 create mode 100644 tests/overlay/051.out
 create mode 100755 tests/overlay/052
 create mode 100644 tests/overlay/052.out
 create mode 100755 tests/overlay/053
 create mode 100644 tests/overlay/053.out

-- 
2.7.4

[PATCH v3 1/5] fstests: implement require of multiple overlayfs features

[Amir Goldstein]

Some overlayfs features must be checked together, because they cannot
be enabled without a dependent feature (e.g. nfs_export and index).

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 common/overlay | 23 ++++++++++++++++++++---
 common/rc      | 15 +++++++++------
 2 files changed, 29 insertions(+), 9 deletions(-)

diff --git a/common/overlay b/common/overlay
index 1da4ab1..38fe994 100644
--- a/common/overlay
+++ b/common/overlay
@@ -138,9 +138,6 @@ _require_scratch_overlay_feature()
 	[ "$default" = Y ] || [ "$default" = N ] || \
 		_notrun "feature '${feature}' not supported by ${FSTYP}"
 
-	_scratch_mkfs > /dev/null 2>&1
-	_scratch_mount -o ${feature}=on || \
-		_notrun "${FSTYP} feature '${feature}' cannot be enabled on ${SCRATCH_DEV}"
 	# Check options to be sure. For example, Overlayfs will fallback to
 	# index=off if underlying fs does not support file handles.
 	# Overlayfs only displays mount option if it differs from the default.
@@ -149,5 +146,25 @@ _require_scratch_overlay_feature()
 	  ( [ "$default" = Y ] && ! _fs_options $SCRATCH_DEV | grep -q "${feature}=off" )) && \
 	    touch $SCRATCH_MNT/foo 2>/dev/null ) || \
 	        _notrun "${FSTYP} feature '${feature}' cannot be enabled on ${SCRATCH_DEV}"
+}
+
+# Require a set of overlayfs features
+_require_scratch_overlay_features()
+{
+	local features=( $* )
+	local opts="rw"
+
+	for feature in ${features[*]}; do
+		opts+=",${feature}=on"
+	done
+
+	_scratch_mkfs > /dev/null 2>&1
+	_scratch_mount -o $opts || \
+		_notrun "overlay options '$opts' cannot be enabled on ${SCRATCH_DEV}"
+
+	for feature in ${features[*]}; do
+		_require_scratch_overlay_feature ${feature}
+	done
+
 	_scratch_unmount
 }
diff --git a/common/rc b/common/rc
index 77a4eb4..3b81061 100644
--- a/common/rc
+++ b/common/rc
@@ -3658,22 +3658,25 @@ _get_fs_sysfs_attr()
 	cat /sys/fs/${FSTYP}/${dname}/${attr}
 }
 
-# Generic test for specific filesystem feature.
+# Generic test for specific filesystem features.
 # Currently only implemented to test overlayfs features.
-_require_scratch_feature()
+_require_scratch_features()
 {
-	local feature=$1
-
 	case "$FSTYP" in
 	overlay)
-		_require_scratch_overlay_feature ${feature}
+		_require_scratch_overlay_features $*
 		;;
 	*)
-		_fail "Test for feature '${feature}' of ${FSTYP} is not implemented"
+		_fail "Test for features '$*' of ${FSTYP} is not implemented"
 		;;
 	esac
 }
 
+_require_scratch_feature()
+{
+	_require_scratch_features $1
+}
+
 init_rc
 
 ################################################################################
-- 
2.7.4

[PATCH v3 2/5] overlay: test encode/decode overlay file handles

[Amir Goldstein]

- Check encode/write/decode/read content of lower/upper file handles
- Check encode/decode/write/read content of lower/upper file handles
- Check decode/read of unlinked lower/upper files and directories
- Check decode/read of lower file handles after copy up, link and unlink

This test requires and enables overlayfs NFS export support.
NFS export support depends on and enables overlayfs index feature.

This test covers only encode/decode of file handles for overlayfs
configuration of all layers on the same base fs.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 tests/overlay/050     | 237 ++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/overlay/050.out |  31 +++++++
 tests/overlay/group   |   1 +
 3 files changed, 269 insertions(+)
 create mode 100755 tests/overlay/050
 create mode 100644 tests/overlay/050.out

diff --git a/tests/overlay/050 b/tests/overlay/050
new file mode 100755
index 0000000..ab20b1f
--- /dev/null
+++ b/tests/overlay/050
@@ -0,0 +1,237 @@
+#! /bin/bash
+# FS QA Test No. 050
+#
+# Test encode/decode overlay file handles
+#
+# - Check encode/write/decode/read content of lower/upper file handles
+# - Check encode/decode/write/read content of lower/upper file handles
+# - Check decode/read of unlinked lower/upper files and directories
+# - Check decode/read of lower file handles after copy up, link and unlink
+#
+# This test requires and enables overlayfs NFS export support.
+# NFS export support depends on and enables overlayfs index feature.
+#
+#-----------------------------------------------------------------------
+# Copyright (C) 2018 CTERA Networks. All Rights Reserved.
+# Author: Amir Goldstein <amir73il@gmail.com>
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+
+_supported_fs overlay
+_supported_os Linux
+_require_scratch
+_require_test_program "open_by_handle"
+_require_scratch_features index nfs_export
+
+# All overlay dirs are on scratch partition
+lower=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
+upper=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
+work=$OVL_BASE_SCRATCH_MNT/$OVL_WORK
+
+NUMFILES=1
+
+# Create test dir and empty test files
+create_test_files()
+{
+	local dir=$1
+
+	mkdir -p $dir
+	$here/src/open_by_handle -cp $dir $NUMFILES
+}
+
+# Create hard links to test files
+link_test_files()
+{
+	local dir=$1
+
+	$here/src/open_by_handle -l $dir $NUMFILES
+}
+
+# Test encode/decode file handles on overlay mount
+test_file_handles()
+{
+	local dir=$1
+	local opt=$2
+
+	echo test_file_handles $dir $opt | _filter_scratch
+	$here/src/open_by_handle $opt $dir $NUMFILES
+}
+
+# Re-create lower/upper/work dirs
+create_dirs()
+{
+	_scratch_mkfs
+}
+
+# Mount an overlay on $SCRATCH_MNT with all layers on scratch partition
+mount_dirs()
+{
+	_scratch_mount -o "index=on,nfs_export=on"
+}
+
+# Unmount the overlay without unmounting base fs
+unmount_dirs()
+{
+	$UMOUNT_PROG $SCRATCH_MNT
+}
+
+# Check non-stale file handles of lower/upper files and verify
+# that handle encoded before copy up is decoded to upper after
+# copy up. Verify reading data from file open by file handle
+# and verify access_at() with dirfd open by file handle.
+create_dirs
+create_test_files $upper/uppertestdir
+create_test_files $lower/lowertestdir
+mount_dirs
+# Check encode/decode of upper regular file handles
+test_file_handles $SCRATCH_MNT/uppertestdir
+# Check encode/decode of upper dir file handle
+test_file_handles $SCRATCH_MNT/uppertestdir -p
+# Check encode/write/decode/read/write of upper file handles
+test_file_handles $SCRATCH_MNT/uppertestdir -wrap
+# Check encode/decode of lower regular file handles before copy up
+test_file_handles $SCRATCH_MNT/lowertestdir
+# Check encode/decode of lower dir file handles before copy up
+test_file_handles $SCRATCH_MNT/lowertestdir -p
+# Check encode/write/decode/read/write of lower file handles across copy up
+test_file_handles $SCRATCH_MNT/lowertestdir -wrap
+unmount_dirs
+
+# Check copy up after encode/decode of lower/upper files
+# (copy up of disconnected dentry to index dir)
+create_dirs
+create_test_files $upper/uppertestdir
+create_test_files $lower/lowertestdir
+mount_dirs
+# Check encode/decode/write/read of upper regular file handles
+test_file_handles $SCRATCH_MNT/uppertestdir -a
+test_file_handles $SCRATCH_MNT/uppertestdir -r
+# Check encode/decode/write/read of lower regular file handles
+test_file_handles $SCRATCH_MNT/lowertestdir -a
+test_file_handles $SCRATCH_MNT/lowertestdir -r
+unmount_dirs
+
+# Check non-stale handles to unlinked but open lower/upper files
+create_dirs
+create_test_files $upper/uppertestdir
+create_test_files $upper/uppertestdir.rw
+create_test_files $lower/lowertestdir
+create_test_files $lower/lowertestdir.rw
+mount_dirs
+test_file_handles $SCRATCH_MNT/uppertestdir -dk
+# Check encode/write/unlink/decode/read of upper regular file handles
+test_file_handles $SCRATCH_MNT/uppertestdir.rw -rwdk
+test_file_handles $SCRATCH_MNT/lowertestdir -dk
+# Check encode/write/unlink/decode/read of lower file handles across copy up
+test_file_handles $SCRATCH_MNT/lowertestdir.rw -rwdk
+unmount_dirs
+
+# Check stale handles of unlinked lower/upper files (nlink = 1,0)
+create_dirs
+create_test_files $upper/uppertestdir
+create_test_files $lower/lowertestdir
+mount_dirs
+# Check decode of upper file handles after unlink/rmdir (nlink == 0)
+test_file_handles $SCRATCH_MNT/uppertestdir -dp
+# Check decode of lower file handles after unlink/rmdir (nlink == 0)
+test_file_handles $SCRATCH_MNT/lowertestdir -dp
+unmount_dirs
+
+# Check non-stale file handles of linked lower/upper files (nlink = 1,2,1)
+create_dirs
+create_test_files $upper/uppertestdir
+create_test_files $lower/lowertestdir
+mount_dirs
+# Check decode/read of upper file handles after link (nlink == 2)
+test_file_handles $SCRATCH_MNT/uppertestdir -wlr
+# Check decode/read of upper file handles after link + unlink (nlink == 1)
+test_file_handles $SCRATCH_MNT/uppertestdir -ur
+# Check decode/read of lower file handles after copy up + link (nlink == 2)
+test_file_handles $SCRATCH_MNT/lowertestdir -wlr
+# Check decode/read of lower file handles after copy up + link + unlink (nlink == 1)
+test_file_handles $SCRATCH_MNT/lowertestdir -ur
+unmount_dirs
+
+# Check non-stale file handles of linked lower/upper hardlinks (nlink = 2,1)
+create_dirs
+create_test_files $upper/uppertestdir
+create_test_files $lower/lowertestdir
+# Create lower/upper hardlinks
+link_test_files $lower/lowertestdir
+link_test_files $upper/uppertestdir
+mount_dirs
+# Check encode/decode of upper hardlink file handles (nlink == 2)
+test_file_handles $SCRATCH_MNT/uppertestdir
+# Check decode/read of upper hardlink file handles after unlink (nlink == 1)
+test_file_handles $SCRATCH_MNT/uppertestdir -wur
+# Check encode/decode of lower hardlink file handles before copy up (nlink == 2)
+test_file_handles $SCRATCH_MNT/lowertestdir
+# Check decode/read of lower hardlink file handles after copy up + unlink (nlink == 1)
+test_file_handles $SCRATCH_MNT/lowertestdir -wur
+unmount_dirs
+
+# Check stale file handles of unlinked lower/upper hardlinks (nlink = 2,0)
+create_dirs
+create_test_files $upper/uppertestdir
+create_test_files $lower/lowertestdir
+# Create lower/upper hardlinks
+link_test_files $lower/lowertestdir
+link_test_files $upper/uppertestdir
+mount_dirs
+# Check encode/decode of upper hardlink file handles (nlink == 2)
+test_file_handles $SCRATCH_MNT/uppertestdir
+# Check decode of upper hardlink file handles after 2*unlink (nlink == 0)
+test_file_handles $SCRATCH_MNT/uppertestdir -d
+# Check encode/decode of lower hardlink file handles before copy up (nlink == 2)
+test_file_handles $SCRATCH_MNT/lowertestdir
+# Check decode of lower hardlink file handles after copy up + 2*unlink (nlink == 0)
+test_file_handles $SCRATCH_MNT/lowertestdir -d
+unmount_dirs
+
+# Check non-stale file handles of lower/upper renamed files
+create_dirs
+create_test_files $upper/uppertestdir
+create_test_files $lower/lowertestdir
+mount_dirs
+# Check decode/read of upper file handles after rename in same upper parent
+test_file_handles $SCRATCH_MNT/uppertestdir -wmr
+# Check decode/read of lower file handles after copy up + rename in same merge parent
+test_file_handles $SCRATCH_MNT/lowertestdir -wmr
+unmount_dirs
+
+status=0
+exit
diff --git a/tests/overlay/050.out b/tests/overlay/050.out
new file mode 100644
index 0000000..6e3d0be
--- /dev/null
+++ b/tests/overlay/050.out
@@ -0,0 +1,31 @@
+QA output created by 050
+test_file_handles SCRATCH_MNT/uppertestdir
+test_file_handles SCRATCH_MNT/uppertestdir -p
+test_file_handles SCRATCH_MNT/uppertestdir -wrap
+test_file_handles SCRATCH_MNT/lowertestdir
+test_file_handles SCRATCH_MNT/lowertestdir -p
+test_file_handles SCRATCH_MNT/lowertestdir -wrap
+test_file_handles SCRATCH_MNT/uppertestdir -a
+test_file_handles SCRATCH_MNT/uppertestdir -r
+test_file_handles SCRATCH_MNT/lowertestdir -a
+test_file_handles SCRATCH_MNT/lowertestdir -r
+test_file_handles SCRATCH_MNT/uppertestdir -dk
+test_file_handles SCRATCH_MNT/uppertestdir.rw -rwdk
+test_file_handles SCRATCH_MNT/lowertestdir -dk
+test_file_handles SCRATCH_MNT/lowertestdir.rw -rwdk
+test_file_handles SCRATCH_MNT/uppertestdir -dp
+test_file_handles SCRATCH_MNT/lowertestdir -dp
+test_file_handles SCRATCH_MNT/uppertestdir -wlr
+test_file_handles SCRATCH_MNT/uppertestdir -ur
+test_file_handles SCRATCH_MNT/lowertestdir -wlr
+test_file_handles SCRATCH_MNT/lowertestdir -ur
+test_file_handles SCRATCH_MNT/uppertestdir
+test_file_handles SCRATCH_MNT/uppertestdir -wur
+test_file_handles SCRATCH_MNT/lowertestdir
+test_file_handles SCRATCH_MNT/lowertestdir -wur
+test_file_handles SCRATCH_MNT/uppertestdir
+test_file_handles SCRATCH_MNT/uppertestdir -d
+test_file_handles SCRATCH_MNT/lowertestdir
+test_file_handles SCRATCH_MNT/lowertestdir -d
+test_file_handles SCRATCH_MNT/uppertestdir -wmr
+test_file_handles SCRATCH_MNT/lowertestdir -wmr
diff --git a/tests/overlay/group b/tests/overlay/group
index 4e138b9..18c26b7 100644
--- a/tests/overlay/group
+++ b/tests/overlay/group
@@ -50,3 +50,4 @@
 047 auto quick copyup hardlink
 048 auto quick copyup hardlink
 049 auto quick copyup redirect
+050 auto quick copyup hardlink exportfs
-- 
2.7.4

[PATCH v3 3/5] overlay: test encode/decode of non-samefs overlay file handles

[Amir Goldstein]

This is a variant of overlay file handles test for an overlayfs that
is composed of multiple lower layers not on the same underlying fs.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 tests/overlay/051     | 254 ++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/overlay/051.out |  31 ++++++
 tests/overlay/group   |   1 +
 3 files changed, 286 insertions(+)
 create mode 100755 tests/overlay/051
 create mode 100644 tests/overlay/051.out

diff --git a/tests/overlay/051 b/tests/overlay/051
new file mode 100755
index 0000000..ce4c222
--- /dev/null
+++ b/tests/overlay/051
@@ -0,0 +1,254 @@
+#! /bin/bash
+# FS QA Test No. 051
+#
+# Test encode/decode overlay file handles for non-samefs.
+#
+# This is a variant of overlay file handles test for an overlayfs that is
+# composed of multiple lower layers not on the same underlying fs.
+#
+# - Check encode/write/decode/read content of lower/upper file handles
+# - Check encode/decode/write/read content of lower/upper file handles
+# - Check decode/read of unlinked lower/upper files and directories
+# - Check decode/read of lower file handles after copy up, link and unlink
+#
+# This test requires and enables overlayfs NFS export support.
+# NFS export support depends on and enables overlayfs index feature.
+#
+#-----------------------------------------------------------------------
+# Copyright (C) 2018 CTERA Networks. All Rights Reserved.
+# Author: Amir Goldstein <amir73il@gmail.com>
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+	# Cleanup overlay scratch mount that is holding base test mount
+	# to prevent _check_test_fs and _test_umount from failing before
+	# _check_scratch_fs _scratch_umount
+	$UMOUNT_PROG $SCRATCH_MNT 2>/dev/null
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+
+_supported_fs overlay
+_supported_os Linux
+_require_test
+_require_scratch
+_require_test_program "open_by_handle"
+_require_scratch_features index nfs_export
+
+# Lower is on test partition
+lower=$OVL_BASE_TEST_DIR/$OVL_LOWER-$seq
+# Upper/work are on scratch partition
+middle=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
+upper=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
+work=$OVL_BASE_SCRATCH_MNT/$OVL_WORK
+
+NUMFILES=1
+
+# Create test dir and empty test files
+create_test_files()
+{
+	local dir=$1
+
+	mkdir -p $dir
+	$here/src/open_by_handle -cp $dir $NUMFILES
+}
+
+# Create hard links to test files
+link_test_files()
+{
+	local dir=$1
+
+	$here/src/open_by_handle -l $dir $NUMFILES
+}
+
+# Test encode/decode file handles on overlay mount
+test_file_handles()
+{
+	local dir=$1
+	local opt=$2
+
+	echo test_file_handles $dir $opt | _filter_scratch
+	$here/src/open_by_handle $opt $dir $NUMFILES
+}
+
+# Re-create lower/middle/upper/work dirs
+create_dirs()
+{
+	# Create lower dir on test partition
+	rm -rf $lower
+	mkdir $lower
+
+	# Create middle/upper/work dirs on scratch partition
+	_scratch_mkfs
+}
+
+# Mount an overlay on $SCRATCH_MNT with lower layer on test partition
+# and middle and upper layer on scratch partition
+mount_dirs()
+{
+	_overlay_scratch_mount_dirs $middle:$lower $upper $work \
+				-o "index=on,nfs_export=on"
+}
+
+# Unmount the overlay without unmounting base fs
+unmount_dirs()
+{
+	$UMOUNT_PROG $SCRATCH_MNT
+}
+
+# Check non-stale file handles of lower/upper files and verify
+# that handle encoded before copy up is decoded to upper after
+# copy up. Verify reading data from file open by file handle
+# and verify access_at() with dirfd open by file handle.
+create_dirs
+create_test_files $upper/uppertestdir
+create_test_files $lower/lowertestdir
+mount_dirs
+# Check encode/decode of upper regular file handles
+test_file_handles $SCRATCH_MNT/uppertestdir
+# Check encode/decode of upper dir file handle
+test_file_handles $SCRATCH_MNT/uppertestdir -p
+# Check encode/write/decode/read/write of upper file handles
+test_file_handles $SCRATCH_MNT/uppertestdir -wrap
+# Check encode/decode of lower regular file handles before copy up
+test_file_handles $SCRATCH_MNT/lowertestdir
+# Check encode/decode of lower dir file handles before copy up
+test_file_handles $SCRATCH_MNT/lowertestdir -p
+# Check encode/write/decode/read/write of lower file handles across copy up
+test_file_handles $SCRATCH_MNT/lowertestdir -wrap
+unmount_dirs
+
+# Check copy up after encode/decode of lower/upper files
+# (copy up of disconnected dentry to index dir)
+create_dirs
+create_test_files $upper/uppertestdir
+create_test_files $lower/lowertestdir
+mount_dirs
+# Check encode/decode/write/read of upper regular file handles
+test_file_handles $SCRATCH_MNT/uppertestdir -a
+test_file_handles $SCRATCH_MNT/uppertestdir -r
+# Check encode/decode/write/read of lower regular file handles
+test_file_handles $SCRATCH_MNT/lowertestdir -a
+test_file_handles $SCRATCH_MNT/lowertestdir -r
+unmount_dirs
+
+# Check non-stale handles to unlinked but open lower/upper files
+create_dirs
+create_test_files $upper/uppertestdir
+create_test_files $upper/uppertestdir.rw
+create_test_files $lower/lowertestdir
+create_test_files $lower/lowertestdir.rw
+mount_dirs
+test_file_handles $SCRATCH_MNT/uppertestdir -dk
+# Check encode/write/unlink/decode/read of upper regular file handles
+test_file_handles $SCRATCH_MNT/uppertestdir.rw -rwdk
+test_file_handles $SCRATCH_MNT/lowertestdir -dk
+# Check encode/write/unlink/decode/read of lower file handles across copy up
+test_file_handles $SCRATCH_MNT/lowertestdir.rw -rwdk
+unmount_dirs
+
+# Check stale handles of unlinked lower/upper files (nlink = 1,0)
+create_dirs
+create_test_files $upper/uppertestdir
+create_test_files $lower/lowertestdir
+mount_dirs
+# Check decode of upper file handles after unlink/rmdir (nlink == 0)
+test_file_handles $SCRATCH_MNT/uppertestdir -dp
+# Check decode of lower file handles after unlink/rmdir (nlink == 0)
+test_file_handles $SCRATCH_MNT/lowertestdir -dp
+unmount_dirs
+
+# Check non-stale file handles of linked lower/upper files (nlink = 1,2,1)
+create_dirs
+create_test_files $upper/uppertestdir
+create_test_files $lower/lowertestdir
+mount_dirs
+# Check decode/read of upper file handles after link (nlink == 2)
+test_file_handles $SCRATCH_MNT/uppertestdir -wlr
+# Check decode/read of upper file handles after link + unlink (nlink == 1)
+test_file_handles $SCRATCH_MNT/uppertestdir -ur
+# Check decode/read of lower file handles after copy up + link (nlink == 2)
+test_file_handles $SCRATCH_MNT/lowertestdir -wlr
+# Check decode/read of lower file handles after copy up + link + unlink (nlink == 1)
+test_file_handles $SCRATCH_MNT/lowertestdir -ur
+unmount_dirs
+
+# Check non-stale file handles of linked lower/upper hardlinks (nlink = 2,1)
+create_dirs
+create_test_files $upper/uppertestdir
+create_test_files $lower/lowertestdir
+# Create lower/upper hardlinks
+link_test_files $lower/lowertestdir
+link_test_files $upper/uppertestdir
+mount_dirs
+# Check encode/decode of upper hardlink file handles (nlink == 2)
+test_file_handles $SCRATCH_MNT/uppertestdir
+# Check decode/read of upper hardlink file handles after unlink (nlink == 1)
+test_file_handles $SCRATCH_MNT/uppertestdir -wur
+# Check encode/decode of lower hardlink file handles before copy up (nlink == 2)
+test_file_handles $SCRATCH_MNT/lowertestdir
+# Check decode/read of lower hardlink file handles after copy up + unlink (nlink == 1)
+test_file_handles $SCRATCH_MNT/lowertestdir -wur
+unmount_dirs
+
+# Check stale file handles of unlinked lower/upper hardlinks (nlink = 2,0)
+create_dirs
+create_test_files $upper/uppertestdir
+create_test_files $lower/lowertestdir
+# Create lower/upper hardlinks
+link_test_files $lower/lowertestdir
+link_test_files $upper/uppertestdir
+mount_dirs
+# Check encode/decode of upper hardlink file handles (nlink == 2)
+test_file_handles $SCRATCH_MNT/uppertestdir
+# Check decode of upper hardlink file handles after 2*unlink (nlink == 0)
+test_file_handles $SCRATCH_MNT/uppertestdir -d
+# Check encode/decode of lower hardlink file handles before copy up (nlink == 2)
+test_file_handles $SCRATCH_MNT/lowertestdir
+# Check decode of lower hardlink file handles after copy up + 2*unlink (nlink == 0)
+test_file_handles $SCRATCH_MNT/lowertestdir -d
+unmount_dirs
+
+# Check non-stale file handles of lower/upper renamed files
+create_dirs
+create_test_files $upper/uppertestdir
+create_test_files $lower/lowertestdir
+mount_dirs
+# Check decode/read of upper file handles after rename in same upper parent
+test_file_handles $SCRATCH_MNT/uppertestdir -wmr
+# Check decode/read of lower file handles after copy up + rename in same merge parent
+test_file_handles $SCRATCH_MNT/lowertestdir -wmr
+unmount_dirs
+
+status=0
+exit
diff --git a/tests/overlay/051.out b/tests/overlay/051.out
new file mode 100644
index 0000000..f9255de
--- /dev/null
+++ b/tests/overlay/051.out
@@ -0,0 +1,31 @@
+QA output created by 051
+test_file_handles SCRATCH_MNT/uppertestdir
+test_file_handles SCRATCH_MNT/uppertestdir -p
+test_file_handles SCRATCH_MNT/uppertestdir -wrap
+test_file_handles SCRATCH_MNT/lowertestdir
+test_file_handles SCRATCH_MNT/lowertestdir -p
+test_file_handles SCRATCH_MNT/lowertestdir -wrap
+test_file_handles SCRATCH_MNT/uppertestdir -a
+test_file_handles SCRATCH_MNT/uppertestdir -r
+test_file_handles SCRATCH_MNT/lowertestdir -a
+test_file_handles SCRATCH_MNT/lowertestdir -r
+test_file_handles SCRATCH_MNT/uppertestdir -dk
+test_file_handles SCRATCH_MNT/uppertestdir.rw -rwdk
+test_file_handles SCRATCH_MNT/lowertestdir -dk
+test_file_handles SCRATCH_MNT/lowertestdir.rw -rwdk
+test_file_handles SCRATCH_MNT/uppertestdir -dp
+test_file_handles SCRATCH_MNT/lowertestdir -dp
+test_file_handles SCRATCH_MNT/uppertestdir -wlr
+test_file_handles SCRATCH_MNT/uppertestdir -ur
+test_file_handles SCRATCH_MNT/lowertestdir -wlr
+test_file_handles SCRATCH_MNT/lowertestdir -ur
+test_file_handles SCRATCH_MNT/uppertestdir
+test_file_handles SCRATCH_MNT/uppertestdir -wur
+test_file_handles SCRATCH_MNT/lowertestdir
+test_file_handles SCRATCH_MNT/lowertestdir -wur
+test_file_handles SCRATCH_MNT/uppertestdir
+test_file_handles SCRATCH_MNT/uppertestdir -d
+test_file_handles SCRATCH_MNT/lowertestdir
+test_file_handles SCRATCH_MNT/lowertestdir -d
+test_file_handles SCRATCH_MNT/uppertestdir -wmr
+test_file_handles SCRATCH_MNT/lowertestdir -wmr
diff --git a/tests/overlay/group b/tests/overlay/group
index 18c26b7..9b00e30 100644
--- a/tests/overlay/group
+++ b/tests/overlay/group
@@ -51,3 +51,4 @@
 048 auto quick copyup hardlink
 049 auto quick copyup redirect
 050 auto quick copyup hardlink exportfs
+051 auto quick copyup hardlink exportfs nonsamefs
-- 
2.7.4

[PATCH v3 4/5] overlay: test encode/decode overlay file handles with renames

[Amir Goldstein]

- Check decode/read of file handles after rename of parent
- Check decode/read of file handles after rename of grandparent
- Check decode/read of file handles after move to new parent
- Check encode/decode/read of file handles in non-upper overlay

This test requires and enables overlayfs NFS export support and merge
dir rename support (redirect_dir).
NFS export support depends on and enables overlayfs index feature.

This test covers only encode/decode of file handles for overlayfs
configuration of all layers on the same base fs.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 tests/overlay/052     | 182 ++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/overlay/052.out |  22 ++++++
 tests/overlay/group   |   1 +
 3 files changed, 205 insertions(+)
 create mode 100755 tests/overlay/052
 create mode 100644 tests/overlay/052.out

diff --git a/tests/overlay/052 b/tests/overlay/052
new file mode 100755
index 0000000..148d35b
--- /dev/null
+++ b/tests/overlay/052
@@ -0,0 +1,182 @@
+#! /bin/bash
+# FS QA Test No. 052
+#
+# Test encode/decode overlay file handles with renames
+#
+# - Check decode/read of file handles after rename of parent
+# - Check decode/read of file handles after rename of grandparent
+# - Check decode/read of file handles after move to new parent
+# - Check encode/decode/read of file handles in non-upper overlay
+#
+# This test requires and enables overlayfs NFS export support and merge
+# dir rename support (redirect_dir).
+# NFS export support depends on and enables overlayfs index feature.
+#
+#-----------------------------------------------------------------------
+# Copyright (C) 2018 CTERA Networks. All Rights Reserved.
+# Author: Amir Goldstein <amir73il@gmail.com>
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+
+_supported_fs overlay
+_supported_os Linux
+_require_scratch
+_require_test_program "open_by_handle"
+_require_scratch_features index nfs_export redirect_dir
+
+# All overlay dirs are on scratch partition
+lower=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
+middle=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER.2
+upper=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
+work=$OVL_BASE_SCRATCH_MNT/$OVL_WORK
+
+NUMFILES=1
+
+# Create test dir and empty test files
+create_test_files()
+{
+	local dir=$1
+	local opt=$2
+
+	mkdir -p $dir
+	$here/src/open_by_handle -cp $opt $dir $NUMFILES
+}
+
+# Test encode/decode file handles on overlay mount
+test_file_handles()
+{
+	local dir=$1
+	shift
+
+	echo test_file_handles $dir $* | _filter_scratch | \
+				sed -e "s,$tmp\.,,g"
+	$here/src/open_by_handle $* $dir $NUMFILES
+}
+
+# Re-create lower/middle/upper/work dirs
+create_dirs()
+{
+	_scratch_mkfs
+	mkdir -p $middle
+}
+
+# Mount an overlay on $SCRATCH_MNT with all layers on scratch partition
+mount_dirs()
+{
+	_scratch_mount -o "index=on,nfs_export=on,redirect_dir=on"
+}
+
+# Unmount the overlay without unmounting base fs
+unmount_dirs()
+{
+	$UMOUNT_PROG $SCRATCH_MNT
+}
+
+# Check non-stale file handles of lower/upper moved files
+create_dirs
+create_test_files $upper/uppertestdir -w
+create_test_files $lower/lowertestdir -w
+mkdir -p $lower/lowertestdir.lo $lower/lowertestdir.up $upper/uppertestdir.lo $upper/uppertestdir.up
+mount_dirs
+# Check encode/decode/read of lower/upper file handles after move to new upper testdir
+test_file_handles $SCRATCH_MNT/uppertestdir -o $tmp.upper_file_handles
+test_file_handles $SCRATCH_MNT/lowertestdir -o $tmp.lower_file_handles
+mv $SCRATCH_MNT/uppertestdir/* $SCRATCH_MNT/uppertestdir.up/
+mv $SCRATCH_MNT/lowertestdir/* $SCRATCH_MNT/uppertestdir.lo/
+# Check open and read from stored file handles
+test_file_handles $SCRATCH_MNT -r -i $tmp.upper_file_handles
+test_file_handles $SCRATCH_MNT -r -i $tmp.lower_file_handles
+# Check encode/decode/read of lower/upper file handles after move to new merge testdir
+test_file_handles $SCRATCH_MNT/uppertestdir.up -o $tmp.upper_file_handles
+test_file_handles $SCRATCH_MNT/uppertestdir.lo -o $tmp.lower_file_handles
+mv $SCRATCH_MNT/uppertestdir.up/* $SCRATCH_MNT/lowertestdir.up/
+mv $SCRATCH_MNT/uppertestdir.lo/* $SCRATCH_MNT/lowertestdir.lo/
+# Check open and read from stored file handles
+test_file_handles $SCRATCH_MNT -r -i $tmp.upper_file_handles
+test_file_handles $SCRATCH_MNT -r -i $tmp.lower_file_handles
+unmount_dirs
+
+# Check non-stale file handles of lower/upper renamed dirs
+create_dirs
+create_test_files $upper/uppertestdir -w
+create_test_files $lower/lowertestdir -w
+create_test_files $upper/uppertestdir/subdir -w
+create_test_files $lower/lowertestdir/subdir -w
+mount_dirs
+# Check encode/decode/read of lower/upper file handles after rename of testdir
+test_file_handles $SCRATCH_MNT/uppertestdir -p -o $tmp.upper_file_handles
+test_file_handles $SCRATCH_MNT/lowertestdir -p -o $tmp.lower_file_handles
+# Check encode/decode/read of lower/upper file handles after rename of testdir's parent
+test_file_handles $SCRATCH_MNT/uppertestdir/subdir -p -o $tmp.upper_subdir_file_handles
+test_file_handles $SCRATCH_MNT/lowertestdir/subdir -p -o $tmp.lower_subdir_file_handles
+# Rename pure upper dir
+mv $SCRATCH_MNT/uppertestdir $SCRATCH_MNT/uppertestdir.new/
+# Copy up lower dir, index and rename
+mv $SCRATCH_MNT/lowertestdir $SCRATCH_MNT/lowertestdir.new/
+# Check open, read and readdir from stored file handles
+# (testdir argument is the mount point and NOT the dir
+#  we are trying to open by stored file handle)
+test_file_handles $SCRATCH_MNT -rp -i $tmp.upper_file_handles
+test_file_handles $SCRATCH_MNT -rp -i $tmp.lower_file_handles
+test_file_handles $SCRATCH_MNT -rp -i $tmp.upper_subdir_file_handles
+test_file_handles $SCRATCH_MNT -rp -i $tmp.lower_subdir_file_handles
+# Retry decoding lower subdir file handle when indexed testdir is in dcache
+# (providing renamed testdir argument pins the indexed testdir to dcache)
+test_file_handles $SCRATCH_MNT/lowertestdir.new -rp -i $tmp.lower_subdir_file_handles
+unmount_dirs
+
+# Check encode/decode/read of lower file handles on lower layers only r/o overlay.
+# For non-upper overlay mount, nfs_export requires disabling redirect_dir.
+$MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \
+			-o ro,redirect_dir=nofollow,nfs_export=on,lowerdir=$middle:$lower
+test_file_handles $SCRATCH_MNT/lowertestdir -rp
+test_file_handles $SCRATCH_MNT/lowertestdir/subdir -rp
+unmount_dirs
+
+# Check encode/decode/read of lower file handles on lower layers only r/o overlay
+# with non-indexed redirects from top lower layer to bottom lower layer.
+# Overlay lookup cannot follow the redirect from $upper/lowertestdir.new to
+# $lower/lowertestdir. Instead, we mount an overlay subtree rooted at these
+# directories.
+$MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \
+		-o ro,redirect_dir=nofollow,nfs_export=on,lowerdir=$upper/lowertestdir.new:$lower/lowertestdir
+test_file_handles $SCRATCH_MNT -r
+test_file_handles $SCRATCH_MNT/subdir -rp
+unmount_dirs
+
+status=0
+exit
diff --git a/tests/overlay/052.out b/tests/overlay/052.out
new file mode 100644
index 0000000..f9f4ff7
--- /dev/null
+++ b/tests/overlay/052.out
@@ -0,0 +1,22 @@
+QA output created by 052
+test_file_handles SCRATCH_MNT/uppertestdir -o upper_file_handles
+test_file_handles SCRATCH_MNT/lowertestdir -o lower_file_handles
+test_file_handles SCRATCH_MNT -r -i upper_file_handles
+test_file_handles SCRATCH_MNT -r -i lower_file_handles
+test_file_handles SCRATCH_MNT/uppertestdir.up -o upper_file_handles
+test_file_handles SCRATCH_MNT/uppertestdir.lo -o lower_file_handles
+test_file_handles SCRATCH_MNT -r -i upper_file_handles
+test_file_handles SCRATCH_MNT -r -i lower_file_handles
+test_file_handles SCRATCH_MNT/uppertestdir -p -o upper_file_handles
+test_file_handles SCRATCH_MNT/lowertestdir -p -o lower_file_handles
+test_file_handles SCRATCH_MNT/uppertestdir/subdir -p -o upper_subdir_file_handles
+test_file_handles SCRATCH_MNT/lowertestdir/subdir -p -o lower_subdir_file_handles
+test_file_handles SCRATCH_MNT -rp -i upper_file_handles
+test_file_handles SCRATCH_MNT -rp -i lower_file_handles
+test_file_handles SCRATCH_MNT -rp -i upper_subdir_file_handles
+test_file_handles SCRATCH_MNT -rp -i lower_subdir_file_handles
+test_file_handles SCRATCH_MNT/lowertestdir.new -rp -i lower_subdir_file_handles
+test_file_handles SCRATCH_MNT/lowertestdir -rp
+test_file_handles SCRATCH_MNT/lowertestdir/subdir -rp
+test_file_handles SCRATCH_MNT -r
+test_file_handles SCRATCH_MNT/subdir -rp
diff --git a/tests/overlay/group b/tests/overlay/group
index 9b00e30..07dcf52 100644
--- a/tests/overlay/group
+++ b/tests/overlay/group
@@ -52,3 +52,4 @@
 049 auto quick copyup redirect
 050 auto quick copyup hardlink exportfs
 051 auto quick copyup hardlink exportfs nonsamefs
+052 auto quick copyup redirect exportfs
-- 
2.7.4

[PATCH v3 5/5] overlay: test encode/decode of non-samefs overlay file handles with renames

[Amir Goldstein]

This is a variant of overlay file handles with renames test for an
overlayfs that is composed of multiple lower layers not on the same
underlying fs.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 tests/overlay/053     | 198 ++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/overlay/053.out |  22 ++++++
 tests/overlay/group   |   1 +
 3 files changed, 221 insertions(+)
 create mode 100755 tests/overlay/053
 create mode 100644 tests/overlay/053.out

diff --git a/tests/overlay/053 b/tests/overlay/053
new file mode 100755
index 0000000..97ea0d5
--- /dev/null
+++ b/tests/overlay/053
@@ -0,0 +1,198 @@
+#! /bin/bash
+# FS QA Test No. 053
+#
+# Test encode/decode of non-samefs overlay file handles with renames
+#
+# This is a variant of overlay file handles with renames test for an
+# overlayfs that is composed of multiple lower layers not on the same
+# underlying fs.
+#
+# - Check decode/read of file handles after rename of parent
+# - Check decode/read of file handles after rename of grandparent
+# - Check decode/read of file handles after move to new parent
+# - Check encode/decode/read of file handles in non-upper overlay
+#
+# This test requires and enables overlayfs NFS export support and merge
+# dir rename support (redirect_dir).
+# NFS export support depends on and enables overlayfs index feature.
+#
+#-----------------------------------------------------------------------
+# Copyright (C) 2018 CTERA Networks. All Rights Reserved.
+# Author: Amir Goldstein <amir73il@gmail.com>
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+	# Cleanup overlay scratch mount that is holding base test mount
+	# to prevent _check_test_fs and _test_umount from failing before
+	# _check_scratch_fs _scratch_umount
+	$UMOUNT_PROG $SCRATCH_MNT 2>/dev/null
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+
+_supported_fs overlay
+_supported_os Linux
+_require_test
+_require_scratch
+_require_test_program "open_by_handle"
+_require_scratch_features index nfs_export redirect_dir
+
+# Lower is on test partition
+lower=$OVL_BASE_TEST_DIR/$OVL_LOWER-$seq
+# Upper/work are on scratch partition
+middle=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
+upper=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
+work=$OVL_BASE_SCRATCH_MNT/$OVL_WORK
+
+NUMFILES=1
+
+# Create test dir and empty test files
+create_test_files()
+{
+	local dir=$1
+	local opt=$2
+
+	mkdir -p $dir
+	$here/src/open_by_handle -cp $opt $dir $NUMFILES
+}
+
+# Test encode/decode file handles on overlay mount
+test_file_handles()
+{
+	local dir=$1
+	shift
+
+	echo test_file_handles $dir $* | _filter_scratch | \
+				sed -e "s,$tmp\.,,g"
+	$here/src/open_by_handle $* $dir $NUMFILES
+}
+
+# Re-create lower/middle/upper/work dirs
+create_dirs()
+{
+	# Create lower dir on test partition
+	rm -rf $lower
+	mkdir $lower
+
+	# Create middle/upper/work dirs on scratch partition
+	_scratch_mkfs
+}
+
+# Mount an overlay on $SCRATCH_MNT with lower layer on test partition
+# and middle and upper layer on scratch partition
+mount_dirs()
+{
+	_overlay_scratch_mount_dirs $middle:$lower $upper $work \
+				-o "index=on,nfs_export=on,redirect_dir=on"
+}
+
+# Unmount the overlay without unmounting base fs
+unmount_dirs()
+{
+	$UMOUNT_PROG $SCRATCH_MNT
+}
+
+# Check non-stale file handles of lower/upper moved files
+create_dirs
+create_test_files $upper/uppertestdir -w
+create_test_files $lower/lowertestdir -w
+mkdir -p $lower/lowertestdir.lo $lower/lowertestdir.up $upper/uppertestdir.lo $upper/uppertestdir.up
+mount_dirs
+# Check encode/decode/read of lower/upper file handles after move to new upper testdir
+test_file_handles $SCRATCH_MNT/uppertestdir -o $tmp.upper_file_handles
+test_file_handles $SCRATCH_MNT/lowertestdir -o $tmp.lower_file_handles
+mv $SCRATCH_MNT/uppertestdir/* $SCRATCH_MNT/uppertestdir.up/
+mv $SCRATCH_MNT/lowertestdir/* $SCRATCH_MNT/uppertestdir.lo/
+# Check open and read from stored file handles
+test_file_handles $SCRATCH_MNT -r -i $tmp.upper_file_handles
+test_file_handles $SCRATCH_MNT -r -i $tmp.lower_file_handles
+# Check encode/decode/read of lower/upper file handles after move to new merge testdir
+test_file_handles $SCRATCH_MNT/uppertestdir.up -o $tmp.upper_file_handles
+test_file_handles $SCRATCH_MNT/uppertestdir.lo -o $tmp.lower_file_handles
+mv $SCRATCH_MNT/uppertestdir.up/* $SCRATCH_MNT/lowertestdir.up/
+mv $SCRATCH_MNT/uppertestdir.lo/* $SCRATCH_MNT/lowertestdir.lo/
+# Check open and read from stored file handles
+test_file_handles $SCRATCH_MNT -r -i $tmp.upper_file_handles
+test_file_handles $SCRATCH_MNT -r -i $tmp.lower_file_handles
+unmount_dirs
+
+# Check non-stale file handles of lower/upper renamed dirs
+create_dirs
+create_test_files $upper/uppertestdir -w
+create_test_files $lower/lowertestdir -w
+create_test_files $upper/uppertestdir/subdir -w
+create_test_files $lower/lowertestdir/subdir -w
+mount_dirs
+# Check encode/decode/read of lower/upper file handles after rename of testdir
+test_file_handles $SCRATCH_MNT/uppertestdir -p -o $tmp.upper_file_handles
+test_file_handles $SCRATCH_MNT/lowertestdir -p -o $tmp.lower_file_handles
+# Check encode/decode/read of lower/upper file handles after rename of testdir's parent
+test_file_handles $SCRATCH_MNT/uppertestdir/subdir -p -o $tmp.upper_subdir_file_handles
+test_file_handles $SCRATCH_MNT/lowertestdir/subdir -p -o $tmp.lower_subdir_file_handles
+# Rename pure upper dir
+mv $SCRATCH_MNT/uppertestdir $SCRATCH_MNT/uppertestdir.new/
+# Copy up lower dir, index and rename
+mv $SCRATCH_MNT/lowertestdir $SCRATCH_MNT/lowertestdir.new/
+# Check open, read and readdir from stored file handles
+# (testdir argument is the mount point and NOT the dir
+#  we are trying to open by stored file handle)
+test_file_handles $SCRATCH_MNT -rp -i $tmp.upper_file_handles
+test_file_handles $SCRATCH_MNT -rp -i $tmp.lower_file_handles
+test_file_handles $SCRATCH_MNT -rp -i $tmp.upper_subdir_file_handles
+test_file_handles $SCRATCH_MNT -rp -i $tmp.lower_subdir_file_handles
+# Retry decoding lower subdir file handle when indexed testdir is in dcache
+# (providing renamed testdir argument pins the indexed testdir to dcache)
+test_file_handles $SCRATCH_MNT/lowertestdir.new -rp -i $tmp.lower_subdir_file_handles
+unmount_dirs
+
+# Check encode/decode/read of lower file handles on lower layers only r/o overlay.
+# For non-upper overlay mount, nfs_export requires disabling redirect_dir.
+$MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \
+			-o ro,redirect_dir=nofollow,nfs_export=on,lowerdir=$middle:$lower
+test_file_handles $SCRATCH_MNT/lowertestdir -rp
+test_file_handles $SCRATCH_MNT/lowertestdir/subdir -rp
+unmount_dirs
+
+# Check encode/decode/read of lower file handles on lower layers only r/o overlay
+# with non-indexed redirects from top lower layer to bottom lower layer.
+# Overlay lookup cannot follow the redirect from $upper/lowertestdir.new to
+# $lower/lowertestdir. Instead, we mount an overlay subtree rooted at these
+# directories.
+$MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \
+		-o ro,redirect_dir=nofollow,nfs_export=on,lowerdir=$upper/lowertestdir.new:$lower/lowertestdir
+test_file_handles $SCRATCH_MNT -r
+test_file_handles $SCRATCH_MNT/subdir -rp
+unmount_dirs
+
+status=0
+exit
diff --git a/tests/overlay/053.out b/tests/overlay/053.out
new file mode 100644
index 0000000..c6f9de0
--- /dev/null
+++ b/tests/overlay/053.out
@@ -0,0 +1,22 @@
+QA output created by 053
+test_file_handles SCRATCH_MNT/uppertestdir -o upper_file_handles
+test_file_handles SCRATCH_MNT/lowertestdir -o lower_file_handles
+test_file_handles SCRATCH_MNT -r -i upper_file_handles
+test_file_handles SCRATCH_MNT -r -i lower_file_handles
+test_file_handles SCRATCH_MNT/uppertestdir.up -o upper_file_handles
+test_file_handles SCRATCH_MNT/uppertestdir.lo -o lower_file_handles
+test_file_handles SCRATCH_MNT -r -i upper_file_handles
+test_file_handles SCRATCH_MNT -r -i lower_file_handles
+test_file_handles SCRATCH_MNT/uppertestdir -p -o upper_file_handles
+test_file_handles SCRATCH_MNT/lowertestdir -p -o lower_file_handles
+test_file_handles SCRATCH_MNT/uppertestdir/subdir -p -o upper_subdir_file_handles
+test_file_handles SCRATCH_MNT/lowertestdir/subdir -p -o lower_subdir_file_handles
+test_file_handles SCRATCH_MNT -rp -i upper_file_handles
+test_file_handles SCRATCH_MNT -rp -i lower_file_handles
+test_file_handles SCRATCH_MNT -rp -i upper_subdir_file_handles
+test_file_handles SCRATCH_MNT -rp -i lower_subdir_file_handles
+test_file_handles SCRATCH_MNT/lowertestdir.new -rp -i lower_subdir_file_handles
+test_file_handles SCRATCH_MNT/lowertestdir -rp
+test_file_handles SCRATCH_MNT/lowertestdir/subdir -rp
+test_file_handles SCRATCH_MNT -r
+test_file_handles SCRATCH_MNT/subdir -rp
diff --git a/tests/overlay/group b/tests/overlay/group
index 07dcf52..ec0b711 100644
--- a/tests/overlay/group
+++ b/tests/overlay/group
@@ -53,3 +53,4 @@
 050 auto quick copyup hardlink exportfs
 051 auto quick copyup hardlink exportfs nonsamefs
 052 auto quick copyup redirect exportfs
+053 auto quick copyup redirect exportfs nonsamefs
-- 
2.7.4

Re: [PATCH v3 0/5] Overlayfs exportfs tests

[Eryu Guan]

On Sun, Jan 28, 2018 at 11:22:31AM +0200, Amir Goldstein wrote:
> Eryu,
> 
> Assuming you did not get to reviewing the overlay/exportfs
> v2 patches yet, here is a better version for you to test/review,
> which I had tested with various OVERLAY_MOUNT_OPTIONS variants.

I haven't started looking at them yet..

> 
> NFS export support patches are now on overlayfs-next and all
> overlay/exportfs tests are expected to pass with overlayfs-next and
> default kernel configuration.

Thanks for all revisions, testings and information! I'll go through all
your pending overlay patches next week.

Thanks,
Eryu

Re: [PATCH v3 1/5] fstests: implement require of multiple overlayfs features

[Eryu Guan]

On Sun, Jan 28, 2018 at 11:22:32AM +0200, Amir Goldstein wrote:
> Some overlayfs features must be checked together, because they cannot
> be enabled without a dependent feature (e.g. nfs_export and index).
> 
> Signed-off-by: Amir Goldstein <amir73il@gmail.com>
> ---
>  common/overlay | 23 ++++++++++++++++++++---
>  common/rc      | 15 +++++++++------
>  2 files changed, 29 insertions(+), 9 deletions(-)
> 
> diff --git a/common/overlay b/common/overlay
> index 1da4ab1..38fe994 100644
> --- a/common/overlay
> +++ b/common/overlay
> @@ -138,9 +138,6 @@ _require_scratch_overlay_feature()
>  	[ "$default" = Y ] || [ "$default" = N ] || \
>  		_notrun "feature '${feature}' not supported by ${FSTYP}"
>  
> -	_scratch_mkfs > /dev/null 2>&1
> -	_scratch_mount -o ${feature}=on || \
> -		_notrun "${FSTYP} feature '${feature}' cannot be enabled on ${SCRATCH_DEV}"

So _require_scratch_overlay_feature can't be used by any test directly
now and should only be called by _require_scratch_overlay_features.

IMHO, the name _require_scratch_overlay_feature becomes misleading after
this change, it implies it can be called by tests directly. I'd rename
it to a more proper name (which I can't think of any..) with double
underscores at the beginning to imply it's kind of "static", e.g. the
internal helpers like "__populate_create_dir" in common/populate.

Another way to go is just open-code this in
_require_scratch_overlay_features().

>  	# Check options to be sure. For example, Overlayfs will fallback to
>  	# index=off if underlying fs does not support file handles.
>  	# Overlayfs only displays mount option if it differs from the default.
> @@ -149,5 +146,25 @@ _require_scratch_overlay_feature()
>  	  ( [ "$default" = Y ] && ! _fs_options $SCRATCH_DEV | grep -q "${feature}=off" )) && \
>  	    touch $SCRATCH_MNT/foo 2>/dev/null ) || \
>  	        _notrun "${FSTYP} feature '${feature}' cannot be enabled on ${SCRATCH_DEV}"
> +}
> +
> +# Require a set of overlayfs features
> +_require_scratch_overlay_features()
> +{
> +	local features=( $* )
> +	local opts="rw"
> +
> +	for feature in ${features[*]}; do
> +		opts+=",${feature}=on"
> +	done
> +
> +	_scratch_mkfs > /dev/null 2>&1
> +	_scratch_mount -o $opts || \
> +		_notrun "overlay options '$opts' cannot be enabled on ${SCRATCH_DEV}"
> +
> +	for feature in ${features[*]}; do
> +		_require_scratch_overlay_feature ${feature}
> +	done
> +
>  	_scratch_unmount
>  }
> diff --git a/common/rc b/common/rc
> index 77a4eb4..3b81061 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -3658,22 +3658,25 @@ _get_fs_sysfs_attr()
>  	cat /sys/fs/${FSTYP}/${dname}/${attr}
>  }
>  
> -# Generic test for specific filesystem feature.
> +# Generic test for specific filesystem features.
>  # Currently only implemented to test overlayfs features.
> -_require_scratch_feature()
> +_require_scratch_features()
>  {
> -	local feature=$1
> -
>  	case "$FSTYP" in
>  	overlay)
> -		_require_scratch_overlay_feature ${feature}
> +		_require_scratch_overlay_features $*
>  		;;
>  	*)
> -		_fail "Test for feature '${feature}' of ${FSTYP} is not implemented"
> +		_fail "Test for features '$*' of ${FSTYP} is not implemented"
>  		;;
>  	esac
>  }
>  
> +_require_scratch_feature()
> +{
> +	_require_scratch_features $1
> +}
> +

Hmm, I don't think this is necessary. Just drop all the plural variants
and teach _require_scratch_overlay_feature to accept & test multiple
features, and add some good comments.

Thanks,
Eryu

>  init_rc
>  
>  ################################################################################
> -- 
> 2.7.4
> 

Re: [PATCH v3 2/5] overlay: test encode/decode overlay file handles

[Eryu Guan]

On Sun, Jan 28, 2018 at 11:22:33AM +0200, Amir Goldstein wrote:
> - Check encode/write/decode/read content of lower/upper file handles
> - Check encode/decode/write/read content of lower/upper file handles
> - Check decode/read of unlinked lower/upper files and directories
> - Check decode/read of lower file handles after copy up, link and unlink
> 
> This test requires and enables overlayfs NFS export support.
> NFS export support depends on and enables overlayfs index feature.
> 
> This test covers only encode/decode of file handles for overlayfs
> configuration of all layers on the same base fs.
> 
> Signed-off-by: Amir Goldstein <amir73il@gmail.com>

Looks good to me overall, just two minor issues in comments.

> ---
>  tests/overlay/050     | 237 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/overlay/050.out |  31 +++++++
>  tests/overlay/group   |   1 +
>  3 files changed, 269 insertions(+)
>  create mode 100755 tests/overlay/050
>  create mode 100644 tests/overlay/050.out
> 
> diff --git a/tests/overlay/050 b/tests/overlay/050
> new file mode 100755
> index 0000000..ab20b1f
> --- /dev/null
> +++ b/tests/overlay/050
> @@ -0,0 +1,237 @@
> +#! /bin/bash
> +# FS QA Test No. 050
> +#
> +# Test encode/decode overlay file handles
> +#
> +# - Check encode/write/decode/read content of lower/upper file handles
> +# - Check encode/decode/write/read content of lower/upper file handles
> +# - Check decode/read of unlinked lower/upper files and directories
> +# - Check decode/read of lower file handles after copy up, link and unlink
> +#
> +# This test requires and enables overlayfs NFS export support.
> +# NFS export support depends on and enables overlayfs index feature.
> +#
> +#-----------------------------------------------------------------------
> +# Copyright (C) 2018 CTERA Networks. All Rights Reserved.
> +# Author: Amir Goldstein <amir73il@gmail.com>
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> +#-----------------------------------------------------------------------
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# real QA test starts here
> +
> +_supported_fs overlay
> +_supported_os Linux
> +_require_scratch
> +_require_test_program "open_by_handle"
> +_require_scratch_features index nfs_export
> +
> +# All overlay dirs are on scratch partition
> +lower=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
> +upper=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
> +work=$OVL_BASE_SCRATCH_MNT/$OVL_WORK
> +
> +NUMFILES=1
> +
> +# Create test dir and empty test files
> +create_test_files()
> +{
> +	local dir=$1
> +
> +	mkdir -p $dir
> +	$here/src/open_by_handle -cp $dir $NUMFILES
> +}
> +
> +# Create hard links to test files
> +link_test_files()
> +{
> +	local dir=$1
> +
> +	$here/src/open_by_handle -l $dir $NUMFILES
> +}
> +
> +# Test encode/decode file handles on overlay mount
> +test_file_handles()
> +{
> +	local dir=$1
> +	local opt=$2
> +
> +	echo test_file_handles $dir $opt | _filter_scratch
> +	$here/src/open_by_handle $opt $dir $NUMFILES
> +}
> +
> +# Re-create lower/upper/work dirs
> +create_dirs()
> +{
> +	_scratch_mkfs
> +}
> +
> +# Mount an overlay on $SCRATCH_MNT with all layers on scratch partition
> +mount_dirs()
> +{
> +	_scratch_mount -o "index=on,nfs_export=on"
> +}
> +
> +# Unmount the overlay without unmounting base fs
> +unmount_dirs()
> +{
> +	$UMOUNT_PROG $SCRATCH_MNT
> +}
> +
> +# Check non-stale file handles of lower/upper files and verify
> +# that handle encoded before copy up is decoded to upper after
> +# copy up. Verify reading data from file open by file handle
> +# and verify access_at() with dirfd open by file handle.
> +create_dirs
> +create_test_files $upper/uppertestdir
> +create_test_files $lower/lowertestdir
> +mount_dirs
> +# Check encode/decode of upper regular file handles
> +test_file_handles $SCRATCH_MNT/uppertestdir
> +# Check encode/decode of upper dir file handle
> +test_file_handles $SCRATCH_MNT/uppertestdir -p
> +# Check encode/write/decode/read/write of upper file handles
> +test_file_handles $SCRATCH_MNT/uppertestdir -wrap
> +# Check encode/decode of lower regular file handles before copy up
> +test_file_handles $SCRATCH_MNT/lowertestdir
> +# Check encode/decode of lower dir file handles before copy up
> +test_file_handles $SCRATCH_MNT/lowertestdir -p
> +# Check encode/write/decode/read/write of lower file handles across copy up
> +test_file_handles $SCRATCH_MNT/lowertestdir -wrap
> +unmount_dirs
> +
> +# Check copy up after encode/decode of lower/upper files
> +# (copy up of disconnected dentry to index dir)
> +create_dirs
> +create_test_files $upper/uppertestdir
> +create_test_files $lower/lowertestdir
> +mount_dirs
> +# Check encode/decode/write/read of upper regular file handles
> +test_file_handles $SCRATCH_MNT/uppertestdir -a
> +test_file_handles $SCRATCH_MNT/uppertestdir -r
> +# Check encode/decode/write/read of lower regular file handles
> +test_file_handles $SCRATCH_MNT/lowertestdir -a
> +test_file_handles $SCRATCH_MNT/lowertestdir -r
> +unmount_dirs
> +
> +# Check non-stale handles to unlinked but open lower/upper files
> +create_dirs
> +create_test_files $upper/uppertestdir
> +create_test_files $upper/uppertestdir.rw
> +create_test_files $lower/lowertestdir
> +create_test_files $lower/lowertestdir.rw
> +mount_dirs
> +test_file_handles $SCRATCH_MNT/uppertestdir -dk
> +# Check encode/write/unlink/decode/read of upper regular file handles
> +test_file_handles $SCRATCH_MNT/uppertestdir.rw -rwdk
> +test_file_handles $SCRATCH_MNT/lowertestdir -dk
> +# Check encode/write/unlink/decode/read of lower file handles across copy up
> +test_file_handles $SCRATCH_MNT/lowertestdir.rw -rwdk
> +unmount_dirs
> +
> +# Check stale handles of unlinked lower/upper files (nlink = 1,0)
                                                       ^^^^^^^^^^^^^
Just nlink = 0? The nlink = 1 case was tested above not here, it's a
mismatch between comments and the actual code.

> +create_dirs
> +create_test_files $upper/uppertestdir
> +create_test_files $lower/lowertestdir
> +mount_dirs
> +# Check decode of upper file handles after unlink/rmdir (nlink == 0)
> +test_file_handles $SCRATCH_MNT/uppertestdir -dp
> +# Check decode of lower file handles after unlink/rmdir (nlink == 0)
> +test_file_handles $SCRATCH_MNT/lowertestdir -dp
> +unmount_dirs
> +
> +# Check non-stale file handles of linked lower/upper files (nlink = 1,2,1)

Same here, the first nlink = 1 case was tested above not here. I think
we can either add more comments or just drop the first "1" in comment.

Thanks,
Eryu

> +create_dirs
> +create_test_files $upper/uppertestdir
> +create_test_files $lower/lowertestdir
> +mount_dirs
> +# Check decode/read of upper file handles after link (nlink == 2)
> +test_file_handles $SCRATCH_MNT/uppertestdir -wlr
> +# Check decode/read of upper file handles after link + unlink (nlink == 1)
> +test_file_handles $SCRATCH_MNT/uppertestdir -ur
> +# Check decode/read of lower file handles after copy up + link (nlink == 2)
> +test_file_handles $SCRATCH_MNT/lowertestdir -wlr
> +# Check decode/read of lower file handles after copy up + link + unlink (nlink == 1)
> +test_file_handles $SCRATCH_MNT/lowertestdir -ur
> +unmount_dirs
> +
> +# Check non-stale file handles of linked lower/upper hardlinks (nlink = 2,1)
> +create_dirs
> +create_test_files $upper/uppertestdir
> +create_test_files $lower/lowertestdir
> +# Create lower/upper hardlinks
> +link_test_files $lower/lowertestdir
> +link_test_files $upper/uppertestdir
> +mount_dirs
> +# Check encode/decode of upper hardlink file handles (nlink == 2)
> +test_file_handles $SCRATCH_MNT/uppertestdir
> +# Check decode/read of upper hardlink file handles after unlink (nlink == 1)
> +test_file_handles $SCRATCH_MNT/uppertestdir -wur
> +# Check encode/decode of lower hardlink file handles before copy up (nlink == 2)
> +test_file_handles $SCRATCH_MNT/lowertestdir
> +# Check decode/read of lower hardlink file handles after copy up + unlink (nlink == 1)
> +test_file_handles $SCRATCH_MNT/lowertestdir -wur
> +unmount_dirs
> +
> +# Check stale file handles of unlinked lower/upper hardlinks (nlink = 2,0)
> +create_dirs
> +create_test_files $upper/uppertestdir
> +create_test_files $lower/lowertestdir
> +# Create lower/upper hardlinks
> +link_test_files $lower/lowertestdir
> +link_test_files $upper/uppertestdir
> +mount_dirs
> +# Check encode/decode of upper hardlink file handles (nlink == 2)
> +test_file_handles $SCRATCH_MNT/uppertestdir
> +# Check decode of upper hardlink file handles after 2*unlink (nlink == 0)
> +test_file_handles $SCRATCH_MNT/uppertestdir -d
> +# Check encode/decode of lower hardlink file handles before copy up (nlink == 2)
> +test_file_handles $SCRATCH_MNT/lowertestdir
> +# Check decode of lower hardlink file handles after copy up + 2*unlink (nlink == 0)
> +test_file_handles $SCRATCH_MNT/lowertestdir -d
> +unmount_dirs
> +
> +# Check non-stale file handles of lower/upper renamed files
> +create_dirs
> +create_test_files $upper/uppertestdir
> +create_test_files $lower/lowertestdir
> +mount_dirs
> +# Check decode/read of upper file handles after rename in same upper parent
> +test_file_handles $SCRATCH_MNT/uppertestdir -wmr
> +# Check decode/read of lower file handles after copy up + rename in same merge parent
> +test_file_handles $SCRATCH_MNT/lowertestdir -wmr
> +unmount_dirs
> +
> +status=0
> +exit
> diff --git a/tests/overlay/050.out b/tests/overlay/050.out
> new file mode 100644
> index 0000000..6e3d0be
> --- /dev/null
> +++ b/tests/overlay/050.out
> @@ -0,0 +1,31 @@
> +QA output created by 050
> +test_file_handles SCRATCH_MNT/uppertestdir
> +test_file_handles SCRATCH_MNT/uppertestdir -p
> +test_file_handles SCRATCH_MNT/uppertestdir -wrap
> +test_file_handles SCRATCH_MNT/lowertestdir
> +test_file_handles SCRATCH_MNT/lowertestdir -p
> +test_file_handles SCRATCH_MNT/lowertestdir -wrap
> +test_file_handles SCRATCH_MNT/uppertestdir -a
> +test_file_handles SCRATCH_MNT/uppertestdir -r
> +test_file_handles SCRATCH_MNT/lowertestdir -a
> +test_file_handles SCRATCH_MNT/lowertestdir -r
> +test_file_handles SCRATCH_MNT/uppertestdir -dk
> +test_file_handles SCRATCH_MNT/uppertestdir.rw -rwdk
> +test_file_handles SCRATCH_MNT/lowertestdir -dk
> +test_file_handles SCRATCH_MNT/lowertestdir.rw -rwdk
> +test_file_handles SCRATCH_MNT/uppertestdir -dp
> +test_file_handles SCRATCH_MNT/lowertestdir -dp
> +test_file_handles SCRATCH_MNT/uppertestdir -wlr
> +test_file_handles SCRATCH_MNT/uppertestdir -ur
> +test_file_handles SCRATCH_MNT/lowertestdir -wlr
> +test_file_handles SCRATCH_MNT/lowertestdir -ur
> +test_file_handles SCRATCH_MNT/uppertestdir
> +test_file_handles SCRATCH_MNT/uppertestdir -wur
> +test_file_handles SCRATCH_MNT/lowertestdir
> +test_file_handles SCRATCH_MNT/lowertestdir -wur
> +test_file_handles SCRATCH_MNT/uppertestdir
> +test_file_handles SCRATCH_MNT/uppertestdir -d
> +test_file_handles SCRATCH_MNT/lowertestdir
> +test_file_handles SCRATCH_MNT/lowertestdir -d
> +test_file_handles SCRATCH_MNT/uppertestdir -wmr
> +test_file_handles SCRATCH_MNT/lowertestdir -wmr
> diff --git a/tests/overlay/group b/tests/overlay/group
> index 4e138b9..18c26b7 100644
> --- a/tests/overlay/group
> +++ b/tests/overlay/group
> @@ -50,3 +50,4 @@
>  047 auto quick copyup hardlink
>  048 auto quick copyup hardlink
>  049 auto quick copyup redirect
> +050 auto quick copyup hardlink exportfs
> -- 
> 2.7.4
> 

Re: [PATCH v3 1/5] fstests: implement require of multiple overlayfs features

[Amir Goldstein]

On Mon, Jan 29, 2018 at 4:33 PM, Eryu Guan <eguan@redhat.com> wrote:
> On Sun, Jan 28, 2018 at 11:22:32AM +0200, Amir Goldstein wrote:
>> Some overlayfs features must be checked together, because they cannot
>> be enabled without a dependent feature (e.g. nfs_export and index).
>>
>> Signed-off-by: Amir Goldstein <amir73il@gmail.com>
>> ---
>>  common/overlay | 23 ++++++++++++++++++++---
>>  common/rc      | 15 +++++++++------
>>  2 files changed, 29 insertions(+), 9 deletions(-)
>>
>> diff --git a/common/overlay b/common/overlay
>> index 1da4ab1..38fe994 100644
>> --- a/common/overlay
>> +++ b/common/overlay
>> @@ -138,9 +138,6 @@ _require_scratch_overlay_feature()
>>       [ "$default" = Y ] || [ "$default" = N ] || \
>>               _notrun "feature '${feature}' not supported by ${FSTYP}"
>>
>> -     _scratch_mkfs > /dev/null 2>&1
>> -     _scratch_mount -o ${feature}=on || \
>> -             _notrun "${FSTYP} feature '${feature}' cannot be enabled on ${SCRATCH_DEV}"
>
> So _require_scratch_overlay_feature can't be used by any test directly
> now and should only be called by _require_scratch_overlay_features.
>
> IMHO, the name _require_scratch_overlay_feature becomes misleading after
> this change, it implies it can be called by tests directly. I'd rename
> it to a more proper name (which I can't think of any..) with double
> underscores at the beginning to imply it's kind of "static", e.g. the
> internal helpers like "__populate_create_dir" in common/populate.

__check_one_scratch_overlay_feature

>
> Another way to go is just open-code this in
> _require_scratch_overlay_features().
>
>>       # Check options to be sure. For example, Overlayfs will fallback to
>>       # index=off if underlying fs does not support file handles.
>>       # Overlayfs only displays mount option if it differs from the default.
>> @@ -149,5 +146,25 @@ _require_scratch_overlay_feature()
>>         ( [ "$default" = Y ] && ! _fs_options $SCRATCH_DEV | grep -q "${feature}=off" )) && \
>>           touch $SCRATCH_MNT/foo 2>/dev/null ) || \
>>               _notrun "${FSTYP} feature '${feature}' cannot be enabled on ${SCRATCH_DEV}"
>> +}
>> +
>> +# Require a set of overlayfs features
>> +_require_scratch_overlay_features()
>> +{
>> +     local features=( $* )
>> +     local opts="rw"
>> +
>> +     for feature in ${features[*]}; do
>> +             opts+=",${feature}=on"
>> +     done
>> +
>> +     _scratch_mkfs > /dev/null 2>&1
>> +     _scratch_mount -o $opts || \
>> +             _notrun "overlay options '$opts' cannot be enabled on ${SCRATCH_DEV}"
>> +
>> +     for feature in ${features[*]}; do
>> +             _require_scratch_overlay_feature ${feature}
>> +     done
>> +
>>       _scratch_unmount
>>  }
>> diff --git a/common/rc b/common/rc
>> index 77a4eb4..3b81061 100644
>> --- a/common/rc
>> +++ b/common/rc
>> @@ -3658,22 +3658,25 @@ _get_fs_sysfs_attr()
>>       cat /sys/fs/${FSTYP}/${dname}/${attr}
>>  }
>>
>> -# Generic test for specific filesystem feature.
>> +# Generic test for specific filesystem features.
>>  # Currently only implemented to test overlayfs features.
>> -_require_scratch_feature()
>> +_require_scratch_features()
>>  {
>> -     local feature=$1
>> -
>>       case "$FSTYP" in
>>       overlay)
>> -             _require_scratch_overlay_feature ${feature}
>> +             _require_scratch_overlay_features $*
>>               ;;
>>       *)
>> -             _fail "Test for feature '${feature}' of ${FSTYP} is not implemented"
>> +             _fail "Test for features '$*' of ${FSTYP} is not implemented"
>>               ;;
>>       esac
>>  }
>>
>> +_require_scratch_feature()
>> +{
>> +     _require_scratch_features $1
>> +}
>> +
>
> Hmm, I don't think this is necessary. Just drop all the plural variants
> and teach _require_scratch_overlay_feature to accept & test multiple
> features, and add some good comments.
>

I see what you are saying, but I rather call a function called
_require_scratch_overlay_features from the tests that require plural features
I can leave _require_scratch_feature for the singular feature case though,
until another filesystem needs the plural variant.

Thanks,
Amir.

Re: [PATCH v3 2/5] overlay: test encode/decode overlay file handles

[Amir Goldstein]

On Mon, Jan 29, 2018 at 4:39 PM, Eryu Guan <eguan@redhat.com> wrote:
> On Sun, Jan 28, 2018 at 11:22:33AM +0200, Amir Goldstein wrote:
>> - Check encode/write/decode/read content of lower/upper file handles
>> - Check encode/decode/write/read content of lower/upper file handles
>> - Check decode/read of unlinked lower/upper files and directories
>> - Check decode/read of lower file handles after copy up, link and unlink
>>
>> This test requires and enables overlayfs NFS export support.
>> NFS export support depends on and enables overlayfs index feature.
>>
>> This test covers only encode/decode of file handles for overlayfs
>> configuration of all layers on the same base fs.
>>
>> Signed-off-by: Amir Goldstein <amir73il@gmail.com>
>
> Looks good to me overall, just two minor issues in comments.
>

OK. I fixed those comment in both test variants.
Let me know if there is anything else, otherwise, I'll re-post
with fixed to comment on patch 1.

Thanks,
Amir.

Re: [PATCH v3 1/5] fstests: implement require of multiple overlayfs features

[Eryu Guan]

On Mon, Jan 29, 2018 at 07:06:23PM +0200, Amir Goldstein wrote:
> On Mon, Jan 29, 2018 at 4:33 PM, Eryu Guan <eguan@redhat.com> wrote:
> > On Sun, Jan 28, 2018 at 11:22:32AM +0200, Amir Goldstein wrote:
> >> Some overlayfs features must be checked together, because they cannot
> >> be enabled without a dependent feature (e.g. nfs_export and index).
> >>
> >> Signed-off-by: Amir Goldstein <amir73il@gmail.com>
> >> ---
> >>  common/overlay | 23 ++++++++++++++++++++---
> >>  common/rc      | 15 +++++++++------
> >>  2 files changed, 29 insertions(+), 9 deletions(-)
> >>
> >> diff --git a/common/overlay b/common/overlay
> >> index 1da4ab1..38fe994 100644
> >> --- a/common/overlay
> >> +++ b/common/overlay
> >> @@ -138,9 +138,6 @@ _require_scratch_overlay_feature()
> >>       [ "$default" = Y ] || [ "$default" = N ] || \
> >>               _notrun "feature '${feature}' not supported by ${FSTYP}"
> >>
> >> -     _scratch_mkfs > /dev/null 2>&1
> >> -     _scratch_mount -o ${feature}=on || \
> >> -             _notrun "${FSTYP} feature '${feature}' cannot be enabled on ${SCRATCH_DEV}"
> >
> > So _require_scratch_overlay_feature can't be used by any test directly
> > now and should only be called by _require_scratch_overlay_features.
> >
> > IMHO, the name _require_scratch_overlay_feature becomes misleading after
> > this change, it implies it can be called by tests directly. I'd rename
> > it to a more proper name (which I can't think of any..) with double
> > underscores at the beginning to imply it's kind of "static", e.g. the
> > internal helpers like "__populate_create_dir" in common/populate.
> 
> __check_one_scratch_overlay_feature

"one" seems redundant, given the ending "feature".

> 
> >
> > Another way to go is just open-code this in
> > _require_scratch_overlay_features().
> >
> >>       # Check options to be sure. For example, Overlayfs will fallback to
> >>       # index=off if underlying fs does not support file handles.
> >>       # Overlayfs only displays mount option if it differs from the default.
> >> @@ -149,5 +146,25 @@ _require_scratch_overlay_feature()
> >>         ( [ "$default" = Y ] && ! _fs_options $SCRATCH_DEV | grep -q "${feature}=off" )) && \
> >>           touch $SCRATCH_MNT/foo 2>/dev/null ) || \
> >>               _notrun "${FSTYP} feature '${feature}' cannot be enabled on ${SCRATCH_DEV}"
> >> +}
> >> +
> >> +# Require a set of overlayfs features
> >> +_require_scratch_overlay_features()
> >> +{
> >> +     local features=( $* )
> >> +     local opts="rw"
> >> +
> >> +     for feature in ${features[*]}; do
> >> +             opts+=",${feature}=on"
> >> +     done
> >> +
> >> +     _scratch_mkfs > /dev/null 2>&1
> >> +     _scratch_mount -o $opts || \
> >> +             _notrun "overlay options '$opts' cannot be enabled on ${SCRATCH_DEV}"
> >> +
> >> +     for feature in ${features[*]}; do
> >> +             _require_scratch_overlay_feature ${feature}
> >> +     done
> >> +
> >>       _scratch_unmount
> >>  }
> >> diff --git a/common/rc b/common/rc
> >> index 77a4eb4..3b81061 100644
> >> --- a/common/rc
> >> +++ b/common/rc
> >> @@ -3658,22 +3658,25 @@ _get_fs_sysfs_attr()
> >>       cat /sys/fs/${FSTYP}/${dname}/${attr}
> >>  }
> >>
> >> -# Generic test for specific filesystem feature.
> >> +# Generic test for specific filesystem features.
> >>  # Currently only implemented to test overlayfs features.
> >> -_require_scratch_feature()
> >> +_require_scratch_features()
> >>  {
> >> -     local feature=$1
> >> -
> >>       case "$FSTYP" in
> >>       overlay)
> >> -             _require_scratch_overlay_feature ${feature}
> >> +             _require_scratch_overlay_features $*
> >>               ;;
> >>       *)
> >> -             _fail "Test for feature '${feature}' of ${FSTYP} is not implemented"
> >> +             _fail "Test for features '$*' of ${FSTYP} is not implemented"
> >>               ;;
> >>       esac
> >>  }
> >>
> >> +_require_scratch_feature()
> >> +{
> >> +     _require_scratch_features $1
> >> +}
> >> +
> >
> > Hmm, I don't think this is necessary. Just drop all the plural variants
> > and teach _require_scratch_overlay_feature to accept & test multiple
> > features, and add some good comments.
> >
> 
> I see what you are saying, but I rather call a function called
> _require_scratch_overlay_features from the tests that require plural features
> I can leave _require_scratch_feature for the singular feature case though,
> until another filesystem needs the plural variant.

Yeah, I can live with that :)

Thanks,
Eryu

Re: [PATCH v3 2/5] overlay: test encode/decode overlay file handles

[Eryu Guan]

On Mon, Jan 29, 2018 at 07:23:49PM +0200, Amir Goldstein wrote:
> On Mon, Jan 29, 2018 at 4:39 PM, Eryu Guan <eguan@redhat.com> wrote:
> > On Sun, Jan 28, 2018 at 11:22:33AM +0200, Amir Goldstein wrote:
> >> - Check encode/write/decode/read content of lower/upper file handles
> >> - Check encode/decode/write/read content of lower/upper file handles
> >> - Check decode/read of unlinked lower/upper files and directories
> >> - Check decode/read of lower file handles after copy up, link and unlink
> >>
> >> This test requires and enables overlayfs NFS export support.
> >> NFS export support depends on and enables overlayfs index feature.
> >>
> >> This test covers only encode/decode of file handles for overlayfs
> >> configuration of all layers on the same base fs.
> >>
> >> Signed-off-by: Amir Goldstein <amir73il@gmail.com>
> >
> > Looks good to me overall, just two minor issues in comments.
> >
> 
> OK. I fixed those comment in both test variants.
> Let me know if there is anything else, otherwise, I'll re-post
> with fixed to comment on patch 1.

Everything else looks fine to me. Sorry, I should have sent out an
explicit ack..

Thanks,
Eryu