[PATCH RFC v6 0/4] xfstests: widen timestamps to deal with y2038+

[PATCH RFC v6 0/4] xfstests: widen timestamps to deal with y2038+

[Darrick J. Wong]

Hi all,

This series performs some refactoring of our timestamp and inode
encoding functions, then retrofits the timestamp union to handle
timestamps as a 64-bit nanosecond counter.  Next, it adds bit shifting
to the non-root dquot timer fields to boost their effective size to 34
bits.  These two changes enable correct time handling on XFS through the
year 2486.

On a current V5 filesystem, inodes timestamps are a signed 32-bit
seconds counter, with 0 being the Unix epoch.  Quota timers are an
unsigned 32-bit seconds counter, with 0 also being the Unix epoch.

This means that inode timestamps can range from:
-(2^31-1) (13 Dec 1901) through (2^31-1) (19 Jan 2038).

And quota timers can range from:
0 (1 Jan 1970) through (2^32-1) (7 Feb 2106).

With the bigtime encoding turned on, inode timestamps are an unsigned
64-bit nanoseconds counter, with 0 being the 1901 epoch.  Quota timers
are a 34-bit unsigned second counter right shifted two bits, with 0
being the Unix epoch, and capped at the maximum inode timestamp value.

This means that inode timestamps can range from:
0 (13 Dec 1901) through (2^64-1 / 1e9) (2 Jul 2486)

Quota timers could theoretically range from:
0 (1 Jan 1970) through (((2^34-1) + (2^31-1)) & ~3) (16 Jun 2582).

But with the capping in place, the quota timers maximum is:
max((2^64-1 / 1e9) - (2^31-1), (((2^34-1) + (2^31-1)) & ~3) (2 Jul 2486).

v2: rebase to 5.9, having landed the quota refactoring
v3: various suggestions by Amir and Dave
v4: drop the timestamp unions, add "is bigtime?" predicates everywhere
v5: reintroduce timestamp unions as *legacy* timestamp unions
v6: minor stylistic changes

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=bigtime

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=bigtime

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=bigtime
---
 common/rc             |    2 -
 common/xfs            |   30 ++++++++++
 tests/generic/721     |  117 +++++++++++++++++++++++++++++++++++++
 tests/generic/721.out |    1 
 tests/generic/722     |  120 ++++++++++++++++++++++++++++++++++++++
 tests/generic/722.out |    1 
 tests/generic/group   |    2 +
 tests/xfs/122         |    1 
 tests/xfs/122.out     |    1 
 tests/xfs/908         |   87 ++++++++++++++++++++++++++++
 tests/xfs/908.out     |   10 +++
 tests/xfs/909         |  153 +++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/909.out     |    4 +
 tests/xfs/911         |   44 ++++++++++++++
 tests/xfs/911.out     |   15 +++++
 tests/xfs/group       |    3 +
 16 files changed, 590 insertions(+), 1 deletion(-)
 create mode 100755 tests/generic/721
 create mode 100644 tests/generic/721.out
 create mode 100755 tests/generic/722
 create mode 100644 tests/generic/722.out
 create mode 100755 tests/xfs/908
 create mode 100644 tests/xfs/908.out
 create mode 100755 tests/xfs/909
 create mode 100644 tests/xfs/909.out
 create mode 100755 tests/xfs/911
 create mode 100644 tests/xfs/911.out

[PATCH 1/4] generic: check userspace handling of extreme timestamps

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

These two tests ensure we can store and retrieve timestamps on the
extremes of the date ranges supported by userspace, and the common
places where overflows can happen.

They differ from generic/402 in that they don't constrain the dates
tested to the range that the filesystem claims to support; we attempt
various things that /userspace/ can parse, and then check that the vfs
clamps and persists the values correctly.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 tests/generic/721     |  117 ++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/721.out |    1 
 tests/generic/722     |  120 +++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/722.out |    1 
 tests/generic/group   |    2 +
 5 files changed, 241 insertions(+)
 create mode 100755 tests/generic/721
 create mode 100644 tests/generic/721.out
 create mode 100755 tests/generic/722
 create mode 100644 tests/generic/722.out


diff --git a/tests/generic/721 b/tests/generic/721
new file mode 100755
index 00000000..9638fbfc
--- /dev/null
+++ b/tests/generic/721
@@ -0,0 +1,117 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2020, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# FS QA Test No. 721
+#
+# Make sure we can store and retrieve timestamps on the extremes of the
+# date ranges supported by userspace, and the common places where overflows
+# can happen.
+#
+# This differs from generic/402 in that we don't constrain ourselves to the
+# range that the filesystem claims to support; we attempt various things that
+# /userspace/ can parse, and then check that the vfs clamps and persists the
+# values correctly.
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+
+# real QA test starts here
+_supported_fs generic
+_require_scratch
+
+rm -f $seqres.full
+
+_scratch_mkfs > $seqres.full
+_scratch_mount
+
+# Does our userspace even support large dates?
+test_bigdates=1
+touch -d 'May 30 01:53:03 UTC 2514' $SCRATCH_MNT 2>/dev/null || test_bigdates=0
+
+# And can we do statx?
+test_statx=1
+($XFS_IO_PROG -c 'help statx' | grep -q 'Print raw statx' && \
+ $XFS_IO_PROG -c 'statx -r' $SCRATCH_MNT 2>/dev/null | grep -q 'stat.mtime') || \
+	test_statx=0
+
+echo "Userspace support of large timestamps: $test_bigdates" >> $seqres.full
+echo "xfs_io support of statx: $test_statx" >> $seqres.full
+
+touchme() {
+	local arg="$1"
+	local name="$2"
+
+	echo "$arg" > $SCRATCH_MNT/t_$name
+	touch -d "$arg" $SCRATCH_MNT/t_$name
+}
+
+report() {
+	local files=($SCRATCH_MNT/t_*)
+	for file in "${files[@]}"; do
+		echo "${file}: $(cat "${file}")"
+		TZ=UTC stat -c '%y %Y %n' "${file}"
+		test $test_statx -gt 0 && \
+			$XFS_IO_PROG -c 'statx -r' "${file}" | grep 'stat.mtime'
+	done
+}
+
+# -2147483648 (S32_MIN, or classic unix min)
+touchme 'Dec 13 20:45:52 UTC 1901' s32_min
+
+# 2147483647 (S32_MAX, or classic unix max)
+touchme 'Jan 19 03:14:07 UTC 2038' s32_max
+
+# 7956915742, all twos
+touchme 'Feb 22 22:22:22 UTC 2222' all_twos
+
+if [ $test_bigdates -gt 0 ]; then
+	# 16299260424 (u64 nsec counter from s32_min, like xfs does)
+	touchme 'Tue Jul  2 20:20:24 UTC 2486' u64ns_from_s32_min
+
+	# 15032385535 (u34 time if you start from s32_min, like ext4 does)
+	touchme 'May 10 22:38:55 UTC 2446' u34_from_s32_min
+
+	# 17179869183 (u34 time if you start from the unix epoch)
+	touchme 'May 30 01:53:03 UTC 2514' u34_max
+
+	# Latest date we can synthesize(?)
+	touchme 'Dec 31 23:59:59 UTC 2147483647' abs_max_time
+
+	# Earliest date we can synthesize(?)
+	touchme 'Jan 1 00:00:00 UTC 0' abs_min_time
+fi
+
+# Query timestamps from incore
+echo before >> $seqres.full
+report > $tmp.times0
+cat $tmp.times0 >> $seqres.full
+
+_scratch_cycle_mount
+
+# Query timestamps from disk
+echo after >> $seqres.full
+report > $tmp.times1
+cat $tmp.times1 >> $seqres.full
+
+# Did they match?
+cmp -s $tmp.times0 $tmp.times1
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/721.out b/tests/generic/721.out
new file mode 100644
index 00000000..087decb5
--- /dev/null
+++ b/tests/generic/721.out
@@ -0,0 +1 @@
+QA output created by 721
diff --git a/tests/generic/722 b/tests/generic/722
new file mode 100755
index 00000000..3e8c553b
--- /dev/null
+++ b/tests/generic/722
@@ -0,0 +1,120 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2020, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# FS QA Test No. 722
+#
+# Make sure we can store and retrieve timestamps on the extremes of the
+# date ranges supported by userspace, and the common places where overflows
+# can happen.  This test also ensures that the timestamps are persisted
+# correctly after a shutdown.
+#
+# This differs from generic/402 in that we don't constrain ourselves to the
+# range that the filesystem claims to support; we attempt various things that
+# /userspace/ can parse, and then check that the vfs clamps and persists the
+# values correctly.
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+
+# real QA test starts here
+_supported_fs generic
+_require_scratch
+_require_scratch_shutdown
+
+rm -f $seqres.full
+
+_scratch_mkfs > $seqres.full
+_scratch_mount
+
+# Does our userspace even support large dates?
+test_bigdates=1
+touch -d 'May 30 01:53:03 UTC 2514' $SCRATCH_MNT 2>/dev/null || test_bigdates=0
+
+# And can we do statx?
+test_statx=1
+($XFS_IO_PROG -c 'help statx' | grep -q 'Print raw statx' && \
+ $XFS_IO_PROG -c 'statx -r' $SCRATCH_MNT 2>/dev/null | grep -q 'stat.mtime') || \
+	test_statx=0
+
+echo "Userspace support of large timestamps: $test_bigdates" >> $seqres.full
+echo "xfs_io support of statx: $test_statx" >> $seqres.full
+
+touchme() {
+	local arg="$1"
+	local name="$2"
+
+	echo "$arg" > $SCRATCH_MNT/t_$name
+	touch -d "$arg" $SCRATCH_MNT/t_$name
+}
+
+report() {
+	local files=($SCRATCH_MNT/t_*)
+	for file in "${files[@]}"; do
+		echo "${file}: $(cat "${file}")"
+		TZ=UTC stat -c '%y %Y %n' "${file}"
+		test $test_statx -gt 0 && \
+			$XFS_IO_PROG -c 'statx -r' "${file}" | grep 'stat.mtime'
+	done
+}
+
+# -2147483648 (S32_MIN, or classic unix min)
+touchme 'Dec 13 20:45:52 UTC 1901' s32_min
+
+# 2147483647 (S32_MAX, or classic unix max)
+touchme 'Jan 19 03:14:07 UTC 2038' s32_max
+
+# 7956915742, all twos
+touchme 'Feb 22 22:22:22 UTC 2222' all_twos
+
+if [ $test_bigdates -gt 0 ]; then
+	# 16299260424 (u64 nsec counter from s32_min, like xfs does)
+	touchme 'Tue Jul  2 20:20:24 UTC 2486' u64ns_from_s32_min
+
+	# 15032385535 (u34 time if you start from s32_min, like ext4 does)
+	touchme 'May 10 22:38:55 UTC 2446' u34_from_s32_min
+
+	# 17179869183 (u34 time if you start from the unix epoch)
+	touchme 'May 30 01:53:03 UTC 2514' u34_max
+
+	# Latest date we can synthesize(?)
+	touchme 'Dec 31 23:59:59 UTC 2147483647' abs_max_time
+
+	# Earliest date we can synthesize(?)
+	touchme 'Jan 1 00:00:00 UTC 0' abs_min_time
+fi
+
+# Query timestamps from incore
+echo before >> $seqres.full
+report > $tmp.times0
+cat $tmp.times0 >> $seqres.full
+
+_scratch_shutdown -f
+_scratch_cycle_mount
+
+# Query timestamps from disk
+echo after >> $seqres.full
+report > $tmp.times1
+cat $tmp.times1 >> $seqres.full
+
+# Did they match?
+cmp -s $tmp.times0 $tmp.times1
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/722.out b/tests/generic/722.out
new file mode 100644
index 00000000..83acd5cf
--- /dev/null
+++ b/tests/generic/722.out
@@ -0,0 +1 @@
+QA output created by 722
diff --git a/tests/generic/group b/tests/generic/group
index cf4fdc23..b533d6b2 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -615,5 +615,7 @@
 610 auto quick prealloc zero
 611 auto quick attr
 612 auto quick clone
+721 auto quick atime bigtime
+722 auto quick atime bigtime
 947 auto quick rw clone
 948 auto quick rw copy_range

[PATCH 2/4] xfs/122: add legacy timestamps to ondisk checker

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Add these new ondisk structures.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 tests/xfs/122     |    1 +
 tests/xfs/122.out |    1 +
 2 files changed, 2 insertions(+)


diff --git a/tests/xfs/122 b/tests/xfs/122
index a4248031..db21f2d5 100755
--- a/tests/xfs/122
+++ b/tests/xfs/122
@@ -182,6 +182,7 @@ struct xfs_iext_cursor
 struct xfs_ino_geometry
 struct xfs_attrlist
 struct xfs_attrlist_ent
+struct xfs_legacy_ictimestamp
 EOF
 
 echo 'int main(int argc, char *argv[]) {' >>$cprog
diff --git a/tests/xfs/122.out b/tests/xfs/122.out
index b0773756..f229465a 100644
--- a/tests/xfs/122.out
+++ b/tests/xfs/122.out
@@ -97,6 +97,7 @@ sizeof(struct xfs_inode_log_format) = 56
 sizeof(struct xfs_inode_log_format_32) = 52
 sizeof(struct xfs_inumbers) = 24
 sizeof(struct xfs_inumbers_req) = 64
+sizeof(struct xfs_legacy_timestamp) = 8
 sizeof(struct xfs_log_dinode) = 176
 sizeof(struct xfs_map_extent) = 32
 sizeof(struct xfs_phys_extent) = 16

[PATCH 3/4] xfs: detect time limits from filesystem

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Teach fstests to extract timestamp limits of a filesystem using the new
xfs_db timelimit command.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 common/rc         |    2 +-
 common/xfs        |   14 ++++++++++++++
 tests/xfs/911     |   44 ++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/911.out |   15 +++++++++++++++
 tests/xfs/group   |    1 +
 5 files changed, 75 insertions(+), 1 deletion(-)
 create mode 100755 tests/xfs/911
 create mode 100644 tests/xfs/911.out


diff --git a/common/rc b/common/rc
index 41f93047..162d957a 100644
--- a/common/rc
+++ b/common/rc
@@ -2029,7 +2029,7 @@ _filesystem_timestamp_range()
 		echo "0 $u32max"
 		;;
 	xfs)
-		echo "$s32min $s32max"
+		_xfs_timestamp_range "$device"
 		;;
 	btrfs)
 		echo "$s64min $s64max"
diff --git a/common/xfs b/common/xfs
index e548a0a1..19ccee03 100644
--- a/common/xfs
+++ b/common/xfs
@@ -994,3 +994,17 @@ _require_xfs_scratch_inobtcount()
 		_notrun "inobtcount not supported by scratch filesystem type: $FSTYP"
 	_scratch_unmount
 }
+
+_xfs_timestamp_range()
+{
+	local use_db=0
+	local dbprog="$XFS_DB_PROG $device"
+	test "$device" = "$SCRATCH_DEV" && dbprog=_scratch_xfs_db
+
+	$dbprog -f -c 'help timelimit' | grep -v -q 'not found' && use_db=1
+	if [ $use_db -eq 0 ]; then
+		echo "-$((1<<31)) $(((1<<31)-1))"
+	else
+		$dbprog -f -c 'timelimit --compact' | awk '{printf("%s %s", $1, $2);}'
+	fi
+}
diff --git a/tests/xfs/911 b/tests/xfs/911
new file mode 100755
index 00000000..bccd1e8f
--- /dev/null
+++ b/tests/xfs/911
@@ -0,0 +1,44 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2020, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# FS QA Test No. 911
+#
+# Check that the xfs_db timelimit command prints the ranges that we expect.
+# This in combination with an xfs_ondisk.h build time check in the kernel
+# ensures that the kernel agrees with userspace.
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+
+# real QA test starts here
+_supported_fs xfs
+_require_scratch
+_require_xfs_db_command timelimit
+
+rm -f $seqres.full
+
+# Format filesystem without bigtime support and populate it
+_scratch_mkfs > $seqres.full
+echo classic xfs timelimits
+_scratch_xfs_db -c 'timelimit --classic'
+echo bigtime xfs timelimits
+_scratch_xfs_db -c 'timelimit --bigtime'
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/911.out b/tests/xfs/911.out
new file mode 100644
index 00000000..84dc475b
--- /dev/null
+++ b/tests/xfs/911.out
@@ -0,0 +1,15 @@
+QA output created by 911
+classic xfs timelimits
+time.min = -2147483648
+time.max = 2147483647
+dqtimer.min = 1
+dqtimer.max = 4294967295
+dqgrace.min = 0
+dqgrace.min = 4294967295
+bigtime xfs timelimits
+time.min = -2147483648
+time.max = 16299260424
+dqtimer.min = 4
+dqtimer.max = 16299260424
+dqgrace.min = 0
+dqgrace.min = 4294967295
diff --git a/tests/xfs/group b/tests/xfs/group
index 862df3be..f61d46a1 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -525,6 +525,7 @@
 761 auto quick realtime
 763 auto quick rw realtime
 910 auto quick inobtcount
+911 auto quick bigtime
 915 auto quick quota
 917 auto quick db
 918 auto quick db

[PATCH 4/4] xfs: test upgrading filesystem to bigtime

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Test that we can upgrade an existing filesystem to use bigtime.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 common/xfs        |   16 ++++++
 tests/xfs/908     |   87 ++++++++++++++++++++++++++++++
 tests/xfs/908.out |   10 +++
 tests/xfs/909     |  153 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/909.out |    4 +
 tests/xfs/group   |    2 +
 6 files changed, 272 insertions(+)
 create mode 100755 tests/xfs/908
 create mode 100644 tests/xfs/908.out
 create mode 100755 tests/xfs/909
 create mode 100644 tests/xfs/909.out


diff --git a/common/xfs b/common/xfs
index 19ccee03..4274eee7 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1008,3 +1008,19 @@ _xfs_timestamp_range()
 		$dbprog -f -c 'timelimit --compact' | awk '{printf("%s %s", $1, $2);}'
 	fi
 }
+
+_require_xfs_mkfs_bigtime()
+{
+	_scratch_mkfs_xfs_supported -m bigtime=1 >/dev/null 2>&1 \
+	   || _notrun "mkfs.xfs doesn't have bigtime feature"
+}
+
+_require_xfs_scratch_bigtime()
+{
+	_require_scratch
+
+	_scratch_mkfs -m bigtime=1 > /dev/null
+	_try_scratch_mount || \
+		_notrun "bigtime not supported by scratch filesystem type: $FSTYP"
+	_scratch_unmount
+}
diff --git a/tests/xfs/908 b/tests/xfs/908
new file mode 100755
index 00000000..e368d66c
--- /dev/null
+++ b/tests/xfs/908
@@ -0,0 +1,87 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2020, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# FS QA Test No. 908
+#
+# Check that we can upgrade a filesystem to support bigtime and that inode
+# timestamps work properly after the upgrade.
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# We have very specific formatting parameters, so don't let things get complex
+# with realtime devices and external logs.
+unset USE_EXTERNAL
+
+# real QA test starts here
+_supported_fs xfs
+_require_xfs_mkfs_crc
+_require_xfs_mkfs_bigtime
+_require_xfs_scratch_bigtime
+
+date --date='Jan 1 00:00:00 UTC 2040' > /dev/null 2>&1 || \
+	_notrun "Userspace does not support dates past 2038."
+
+rm -f $seqres.full
+
+# Format V5 filesystem without bigtime support and populate it
+_scratch_mkfs -m crc=1,bigtime=0 > $seqres.full
+_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
+_scratch_mount >> $seqres.full
+
+touch -d 'Jan 9 19:19:19 UTC 1999' $SCRATCH_MNT/a
+touch -d 'Jan 9 19:19:19 UTC 1999' $SCRATCH_MNT/b
+ls -la $SCRATCH_MNT/* >> $seqres.full
+
+echo before upgrade:
+TZ=UTC stat -c '%Y' $SCRATCH_MNT/a
+TZ=UTC stat -c '%Y' $SCRATCH_MNT/b
+
+_scratch_unmount
+_check_scratch_fs
+
+# Now upgrade to bigtime support
+_scratch_xfs_admin -O bigtime >> $seqres.full
+_check_scratch_fs
+_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
+
+# Mount again, look at our files
+_scratch_mount >> $seqres.full
+ls -la $SCRATCH_MNT/* >> $seqres.full
+
+# Modify one of the timestamps to stretch beyond 2038
+touch -d 'Feb 22 22:22:22 UTC 2222' $SCRATCH_MNT/b
+
+echo after upgrade:
+TZ=UTC stat -c '%Y' $SCRATCH_MNT/a
+TZ=UTC stat -c '%Y' $SCRATCH_MNT/b
+
+_scratch_cycle_mount
+
+# Did the timestamp survive the remount?
+ls -la $SCRATCH_MNT/* >> $seqres.full
+
+echo after upgrade and remount:
+TZ=UTC stat -c '%Y' $SCRATCH_MNT/a
+TZ=UTC stat -c '%Y' $SCRATCH_MNT/b
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/908.out b/tests/xfs/908.out
new file mode 100644
index 00000000..f0f412be
--- /dev/null
+++ b/tests/xfs/908.out
@@ -0,0 +1,10 @@
+QA output created by 908
+before upgrade:
+915909559
+915909559
+after upgrade:
+915909559
+7956915742
+after upgrade and remount:
+915909559
+7956915742
diff --git a/tests/xfs/909 b/tests/xfs/909
new file mode 100755
index 00000000..7010eb9e
--- /dev/null
+++ b/tests/xfs/909
@@ -0,0 +1,153 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2020, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# FS QA Test No. 909
+#
+# Check that we can upgrade a filesystem to support bigtime and that quota
+# timers work properly after the upgrade.
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/quota
+
+# We have very specific formatting parameters, so don't let things get complex
+# with realtime devices and external logs.
+unset USE_EXTERNAL
+
+# real QA test starts here
+_supported_fs xfs
+_require_quota
+_require_xfs_mkfs_crc
+_require_xfs_mkfs_bigtime
+_require_xfs_scratch_bigtime
+
+date --date='Jan 1 00:00:00 UTC 2040' > /dev/null 2>&1 || \
+	_notrun "Userspace does not support dates past 2038."
+
+rm -f $seqres.full
+
+# Format V5 filesystem without bigtime support and populate it
+_scratch_mkfs -m crc=1,bigtime=0 > $seqres.full
+_qmount_option "usrquota"
+_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
+_scratch_mount >> $seqres.full
+
+# Force the block counters for uid 1 and 2 above zero
+_pwrite_byte 0x61 0 64k $SCRATCH_MNT/a >> $seqres.full
+_pwrite_byte 0x61 0 64k $SCRATCH_MNT/b >> $seqres.full
+sync
+chown 1 $SCRATCH_MNT/a
+chown 2 $SCRATCH_MNT/b
+
+# Set quota limits on uid 1 before upgrading
+$XFS_QUOTA_PROG -x -c 'limit -u bsoft=12k bhard=1m 1' $SCRATCH_MNT
+
+# Make sure the grace period is at /some/ point in the future.  We have to
+# use bc because not all bashes can handle integer comparisons with 64-bit
+# numbers.
+repquota -upn $SCRATCH_MNT > $tmp.repquota
+cat $tmp.repquota >> $seqres.full
+grace="$(cat $tmp.repquota | grep '^#1' | awk '{print $6}')"
+now="$(date +%s)"
+res="$(echo "${grace} > ${now}" | $BC_PROG)"
+test $res -eq 1 || echo "Expected timer expiry (${grace}) to be after now (${now})."
+
+_scratch_unmount
+
+# Now upgrade to bigtime support
+_scratch_xfs_admin -O bigtime >> $seqres.full
+_check_scratch_fs
+_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
+
+# Mount again, see if our quota timer survived
+_scratch_mount
+
+# Set a very generous grace period and quota limits on uid 2 after upgrading
+$XFS_QUOTA_PROG -x -c 'timer -u -b -d 2147483647' $SCRATCH_MNT
+$XFS_QUOTA_PROG -x -c 'limit -u bsoft=10000 bhard=150000 2' $SCRATCH_MNT
+
+# Query the grace periods to see if they got set properly after the upgrade.
+repquota -upn $SCRATCH_MNT > $tmp.repquota
+cat $tmp.repquota >> $seqres.full
+grace1="$(repquota -upn $SCRATCH_MNT | grep '^#1' | awk '{print $6}')"
+grace2="$(repquota -upn $SCRATCH_MNT | grep '^#2' | awk '{print $6}')"
+now="$(date +%s)"
+
+# Make sure that uid 1's expiration is in the future...
+res1="$(echo "${grace} > ${now}" | $BC_PROG)"
+test "${res1}" -eq 1 || echo "Expected uid 1 expiry (${grace1}) to be after now (${now})."
+
+# ...and that uid 2's expiration is after uid 1's...
+res2="$(echo "${grace2} > ${grace1}" | $BC_PROG)"
+test "${res2}" -eq 1 || echo "Expected uid 2 expiry (${grace2}) to be after uid 1 (${grace1})."
+
+# ...and that uid 2's expiration is after 2038 if right now is far enough
+# past 1970 that our generous grace period would provide for that.
+res3="$(echo "(${now} < 100) || (${grace2} > 2147483648)" | $BC_PROG)"
+test "${res3}" -eq 1 || echo "Expected uid 2 expiry (${grace2}) to be after 2038."
+
+_scratch_cycle_mount
+
+# Query the grace periods to see if they survived a remount.
+repquota -upn $SCRATCH_MNT > $tmp.repquota
+cat $tmp.repquota >> $seqres.full
+grace1="$(repquota -upn $SCRATCH_MNT | grep '^#1' | awk '{print $6}')"
+grace2="$(repquota -upn $SCRATCH_MNT | grep '^#2' | awk '{print $6}')"
+now="$(date +%s)"
+
+# Make sure that uid 1's expiration is in the future...
+res1="$(echo "${grace} > ${now}" | $BC_PROG)"
+test "${res1}" -eq 1 || echo "Expected uid 1 expiry (${grace1}) to be after now (${now})."
+
+# ...and that uid 2's expiration is after uid 1's...
+res2="$(echo "${grace2} > ${grace1}" | $BC_PROG)"
+test "${res2}" -eq 1 || echo "Expected uid 2 expiry (${grace2}) to be after uid 1 (${grace1})."
+
+# ...and that uid 2's expiration is after 2038 if right now is far enough
+# past 1970 that our generous grace period would provide for that.
+res3="$(echo "(${now} < 100) || (${grace2} > 2147483648)" | $BC_PROG)"
+test "${res3}" -eq 1 || echo "Expected uid 2 expiry (${grace2}) to be after 2038."
+
+# Now try setting uid 2's expiration to Feb 22 22:22:22 UTC 2222
+new_expiry=$(date -d 'Feb 22 22:22:22 UTC 2222' +%s)
+now=$(date +%s)
+test $now -ge $new_expiry && \
+	echo "Now is after February 2222?  Expect problems."
+expiry_delta=$((new_expiry - now))
+
+echo "setting expiration to $new_expiry - $now = $expiry_delta" >> $seqres.full
+$XFS_QUOTA_PROG -x -c "timer -u $expiry_delta 2" -c 'report' $SCRATCH_MNT >> $seqres.full
+
+# Did we get an expiration within 5s of the target range?
+grace2="$(repquota -upn $SCRATCH_MNT | grep '^#2' | awk '{print $6}')"
+echo "grace2 is $grace2" >> $seqres.full
+_within_tolerance "grace2 expiry" $grace2 $new_expiry 5 -v
+
+_scratch_cycle_mount
+
+# ...and is it still within 5s after a remount?
+grace2="$(repquota -upn $SCRATCH_MNT | grep '^#2' | awk '{print $6}')"
+echo "grace2 is $grace2" >> $seqres.full
+_within_tolerance "grace2 expiry after remount" $grace2 $new_expiry 5 -v
+
+# success, all done
+echo Silence is golden.
+status=0
+exit
diff --git a/tests/xfs/909.out b/tests/xfs/909.out
new file mode 100644
index 00000000..948502b7
--- /dev/null
+++ b/tests/xfs/909.out
@@ -0,0 +1,4 @@
+QA output created by 909
+grace2 expiry is in range
+grace2 expiry after remount is in range
+Silence is golden.
diff --git a/tests/xfs/group b/tests/xfs/group
index f61d46a1..ab98a706 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -524,6 +524,8 @@
 760 auto quick rw collapse punch insert zero prealloc
 761 auto quick realtime
 763 auto quick rw realtime
+908 auto quick bigtime
+909 auto quick bigtime quota
 910 auto quick inobtcount
 911 auto quick bigtime
 915 auto quick quota

Re: [PATCH 1/4] generic: check userspace handling of extreme timestamps

[Amir Goldstein]

On Wed, Oct 28, 2020 at 10:25 PM Darrick J. Wong
<darrick.wong@oracle.com> wrote:
>
> From: Darrick J. Wong <darrick.wong@oracle.com>
>
> These two tests ensure we can store and retrieve timestamps on the
> extremes of the date ranges supported by userspace, and the common
> places where overflows can happen.
>
> They differ from generic/402 in that they don't constrain the dates
> tested to the range that the filesystem claims to support; we attempt
> various things that /userspace/ can parse, and then check that the vfs
> clamps and persists the values correctly.

So this test will fail when run on stable kernels before the vfs
clamping changes
and there is no require_* to mitigate that failure.

At the time, I discussed this with Deepa and the result was the
_check_dmesg_for part of _require_timestamp_range, which is incomplete.
The complete check for kernel clamping support would be to run
_require_timestamp_range (on the second half thereof) on a loop mounted
ext2, so we know for sure that the kernel is going to emit the y2038 warning.

I am going to leave it to you and the maintainer the decide how
critical that is,
but I would suggest to at least factor out _require_timestamp_limits()
which is true if either the filesystem timestamp range is known or the kernel
emits y2038 warning.

>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  tests/generic/721     |  117 ++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/generic/721.out |    1
>  tests/generic/722     |  120 +++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/generic/722.out |    1
>  tests/generic/group   |    2 +
>  5 files changed, 241 insertions(+)
>  create mode 100755 tests/generic/721
>  create mode 100644 tests/generic/721.out
>  create mode 100755 tests/generic/722
>  create mode 100644 tests/generic/722.out
>
>
> diff --git a/tests/generic/721 b/tests/generic/721
> new file mode 100755
> index 00000000..9638fbfc
> --- /dev/null
> +++ b/tests/generic/721
> @@ -0,0 +1,117 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2020, Oracle and/or its affiliates.  All Rights Reserved.
> +#
> +# FS QA Test No. 721
> +#
> +# Make sure we can store and retrieve timestamps on the extremes of the
> +# date ranges supported by userspace, and the common places where overflows
> +# can happen.
> +#
> +# This differs from generic/402 in that we don't constrain ourselves to the
> +# range that the filesystem claims to support; we attempt various things that
> +# /userspace/ can parse, and then check that the vfs clamps and persists the
> +# values correctly.
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1    # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +       cd /
> +       rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +
> +# real QA test starts here
> +_supported_fs generic
> +_require_scratch
> +
> +rm -f $seqres.full
> +
> +_scratch_mkfs > $seqres.full
> +_scratch_mount
> +
> +# Does our userspace even support large dates?
> +test_bigdates=1
> +touch -d 'May 30 01:53:03 UTC 2514' $SCRATCH_MNT 2>/dev/null || test_bigdates=0
> +
> +# And can we do statx?
> +test_statx=1
> +($XFS_IO_PROG -c 'help statx' | grep -q 'Print raw statx' && \
> + $XFS_IO_PROG -c 'statx -r' $SCRATCH_MNT 2>/dev/null | grep -q 'stat.mtime') || \
> +       test_statx=0
> +
> +echo "Userspace support of large timestamps: $test_bigdates" >> $seqres.full
> +echo "xfs_io support of statx: $test_statx" >> $seqres.full
> +
> +touchme() {
> +       local arg="$1"
> +       local name="$2"
> +
> +       echo "$arg" > $SCRATCH_MNT/t_$name
> +       touch -d "$arg" $SCRATCH_MNT/t_$name
> +}
> +
> +report() {
> +       local files=($SCRATCH_MNT/t_*)
> +       for file in "${files[@]}"; do
> +               echo "${file}: $(cat "${file}")"
> +               TZ=UTC stat -c '%y %Y %n' "${file}"
> +               test $test_statx -gt 0 && \
> +                       $XFS_IO_PROG -c 'statx -r' "${file}" | grep 'stat.mtime'
> +       done
> +}
> +
> +# -2147483648 (S32_MIN, or classic unix min)
> +touchme 'Dec 13 20:45:52 UTC 1901' s32_min
> +
> +# 2147483647 (S32_MAX, or classic unix max)
> +touchme 'Jan 19 03:14:07 UTC 2038' s32_max
> +
> +# 7956915742, all twos
> +touchme 'Feb 22 22:22:22 UTC 2222' all_twos
> +
> +if [ $test_bigdates -gt 0 ]; then
> +       # 16299260424 (u64 nsec counter from s32_min, like xfs does)
> +       touchme 'Tue Jul  2 20:20:24 UTC 2486' u64ns_from_s32_min
> +
> +       # 15032385535 (u34 time if you start from s32_min, like ext4 does)
> +       touchme 'May 10 22:38:55 UTC 2446' u34_from_s32_min
> +
> +       # 17179869183 (u34 time if you start from the unix epoch)
> +       touchme 'May 30 01:53:03 UTC 2514' u34_max
> +
> +       # Latest date we can synthesize(?)
> +       touchme 'Dec 31 23:59:59 UTC 2147483647' abs_max_time
> +
> +       # Earliest date we can synthesize(?)
> +       touchme 'Jan 1 00:00:00 UTC 0' abs_min_time
> +fi
> +
> +# Query timestamps from incore
> +echo before >> $seqres.full
> +report > $tmp.times0
> +cat $tmp.times0 >> $seqres.full
> +
> +_scratch_cycle_mount
> +
> +# Query timestamps from disk
> +echo after >> $seqres.full
> +report > $tmp.times1
> +cat $tmp.times1 >> $seqres.full
> +
> +# Did they match?
> +cmp -s $tmp.times0 $tmp.times1
> +

Please use suffix $tmp.{before,after}_cycle_mount
It makes the meaning of the diff in the test failure much clearer
to a bystander.

> +# success, all done
> +status=0
> +exit
> diff --git a/tests/generic/721.out b/tests/generic/721.out
> new file mode 100644
> index 00000000..087decb5
> --- /dev/null
> +++ b/tests/generic/721.out
> @@ -0,0 +1 @@
> +QA output created by 721

What? no "Silence is golden"? :-D

> diff --git a/tests/generic/722 b/tests/generic/722
> new file mode 100755
> index 00000000..3e8c553b
> --- /dev/null
> +++ b/tests/generic/722
> @@ -0,0 +1,120 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2020, Oracle and/or its affiliates.  All Rights Reserved.
> +#
> +# FS QA Test No. 722
> +#
> +# Make sure we can store and retrieve timestamps on the extremes of the
> +# date ranges supported by userspace, and the common places where overflows
> +# can happen.  This test also ensures that the timestamps are persisted
> +# correctly after a shutdown.
> +#
> +# This differs from generic/402 in that we don't constrain ourselves to the
> +# range that the filesystem claims to support; we attempt various things that
> +# /userspace/ can parse, and then check that the vfs clamps and persists the
> +# values correctly.
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1    # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +       cd /
> +       rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +
> +# real QA test starts here
> +_supported_fs generic
> +_require_scratch
> +_require_scratch_shutdown
> +
> +rm -f $seqres.full
> +
> +_scratch_mkfs > $seqres.full
> +_scratch_mount
> +
> +# Does our userspace even support large dates?
> +test_bigdates=1
> +touch -d 'May 30 01:53:03 UTC 2514' $SCRATCH_MNT 2>/dev/null || test_bigdates=0
> +
> +# And can we do statx?
> +test_statx=1
> +($XFS_IO_PROG -c 'help statx' | grep -q 'Print raw statx' && \
> + $XFS_IO_PROG -c 'statx -r' $SCRATCH_MNT 2>/dev/null | grep -q 'stat.mtime') || \
> +       test_statx=0
> +
> +echo "Userspace support of large timestamps: $test_bigdates" >> $seqres.full
> +echo "xfs_io support of statx: $test_statx" >> $seqres.full
> +
> +touchme() {
> +       local arg="$1"
> +       local name="$2"
> +
> +       echo "$arg" > $SCRATCH_MNT/t_$name
> +       touch -d "$arg" $SCRATCH_MNT/t_$name
> +}
> +
> +report() {
> +       local files=($SCRATCH_MNT/t_*)
> +       for file in "${files[@]}"; do
> +               echo "${file}: $(cat "${file}")"
> +               TZ=UTC stat -c '%y %Y %n' "${file}"
> +               test $test_statx -gt 0 && \
> +                       $XFS_IO_PROG -c 'statx -r' "${file}" | grep 'stat.mtime'
> +       done
> +}
> +
> +# -2147483648 (S32_MIN, or classic unix min)
> +touchme 'Dec 13 20:45:52 UTC 1901' s32_min
> +
> +# 2147483647 (S32_MAX, or classic unix max)
> +touchme 'Jan 19 03:14:07 UTC 2038' s32_max
> +
> +# 7956915742, all twos
> +touchme 'Feb 22 22:22:22 UTC 2222' all_twos
> +
> +if [ $test_bigdates -gt 0 ]; then
> +       # 16299260424 (u64 nsec counter from s32_min, like xfs does)
> +       touchme 'Tue Jul  2 20:20:24 UTC 2486' u64ns_from_s32_min
> +
> +       # 15032385535 (u34 time if you start from s32_min, like ext4 does)
> +       touchme 'May 10 22:38:55 UTC 2446' u34_from_s32_min
> +
> +       # 17179869183 (u34 time if you start from the unix epoch)
> +       touchme 'May 30 01:53:03 UTC 2514' u34_max
> +
> +       # Latest date we can synthesize(?)
> +       touchme 'Dec 31 23:59:59 UTC 2147483647' abs_max_time
> +
> +       # Earliest date we can synthesize(?)
> +       touchme 'Jan 1 00:00:00 UTC 0' abs_min_time
> +fi
> +
> +# Query timestamps from incore
> +echo before >> $seqres.full
> +report > $tmp.times0
> +cat $tmp.times0 >> $seqres.full
> +
> +_scratch_shutdown -f
> +_scratch_cycle_mount
> +
> +# Query timestamps from disk
> +echo after >> $seqres.full
> +report > $tmp.times1
> +cat $tmp.times1 >> $seqres.full
> +
> +# Did they match?
> +cmp -s $tmp.times0 $tmp.times1
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/generic/722.out b/tests/generic/722.out
> new file mode 100644
> index 00000000..83acd5cf
> --- /dev/null
> +++ b/tests/generic/722.out
> @@ -0,0 +1 @@
> +QA output created by 722
> diff --git a/tests/generic/group b/tests/generic/group
> index cf4fdc23..b533d6b2 100644
> --- a/tests/generic/group
> +++ b/tests/generic/group
> @@ -615,5 +615,7 @@
>  610 auto quick prealloc zero
>  611 auto quick attr
>  612 auto quick clone
> +721 auto quick atime bigtime
> +722 auto quick atime bigtime

shutdown group please.

If we are going to use "bigtime" for generic tests to describe y2038 tests,
perhaps add it to 258 and 402 as well?

Thanks,
Amir.

Re: [PATCH 3/4] xfs: detect time limits from filesystem

[Amir Goldstein]

On Wed, Oct 28, 2020 at 10:24 PM Darrick J. Wong
<darrick.wong@oracle.com> wrote:
>
> From: Darrick J. Wong <darrick.wong@oracle.com>
>
> Teach fstests to extract timestamp limits of a filesystem using the new
> xfs_db timelimit command.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  common/rc         |    2 +-
>  common/xfs        |   14 ++++++++++++++
>  tests/xfs/911     |   44 ++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/911.out |   15 +++++++++++++++
>  tests/xfs/group   |    1 +
>  5 files changed, 75 insertions(+), 1 deletion(-)
>  create mode 100755 tests/xfs/911
>  create mode 100644 tests/xfs/911.out
>
>
> diff --git a/common/rc b/common/rc
> index 41f93047..162d957a 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -2029,7 +2029,7 @@ _filesystem_timestamp_range()
>                 echo "0 $u32max"
>                 ;;
>         xfs)
> -               echo "$s32min $s32max"
> +               _xfs_timestamp_range "$device"
>                 ;;
>         btrfs)
>                 echo "$s64min $s64max"
> diff --git a/common/xfs b/common/xfs
> index e548a0a1..19ccee03 100644
> --- a/common/xfs
> +++ b/common/xfs
> @@ -994,3 +994,17 @@ _require_xfs_scratch_inobtcount()
>                 _notrun "inobtcount not supported by scratch filesystem type: $FSTYP"
>         _scratch_unmount
>  }
> +
> +_xfs_timestamp_range()
> +{
> +       local use_db=0
> +       local dbprog="$XFS_DB_PROG $device"
> +       test "$device" = "$SCRATCH_DEV" && dbprog=_scratch_xfs_db
> +
> +       $dbprog -f -c 'help timelimit' | grep -v -q 'not found' && use_db=1
> +       if [ $use_db -eq 0 ]; then
> +               echo "-$((1<<31)) $(((1<<31)-1))"

This embodies an assumption that the tested filesystem does not have
bigtime enabled if xfs_db tool is not uptodate.
Maybe it makes sense, but it may be safer to return "-1 -1" and not_run
generic/402 if xfs_db is not uptodate, perhaps with an extra message
hinting the user to upgrade xfs_db.

Thanks,
Amir.

Re: [PATCH 2/4] xfs/122: add legacy timestamps to ondisk checker

[Amir Goldstein]

On Wed, Oct 28, 2020 at 10:25 PM Darrick J. Wong
<darrick.wong@oracle.com> wrote:
>
> From: Darrick J. Wong <darrick.wong@oracle.com>
>
> Add these new ondisk structures.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Reviewed-by: Amir Goldstein <amir73il@gmail.com>

> ---
>  tests/xfs/122     |    1 +
>  tests/xfs/122.out |    1 +
>  2 files changed, 2 insertions(+)
>
>
> diff --git a/tests/xfs/122 b/tests/xfs/122
> index a4248031..db21f2d5 100755
> --- a/tests/xfs/122
> +++ b/tests/xfs/122
> @@ -182,6 +182,7 @@ struct xfs_iext_cursor
>  struct xfs_ino_geometry
>  struct xfs_attrlist
>  struct xfs_attrlist_ent
> +struct xfs_legacy_ictimestamp

<OCD move near xfs_ictimestamp_t similar to other adjacent struct and
typedefs />

Thanks,
Amir.

Re: [PATCH 4/4] xfs: test upgrading filesystem to bigtime

[Amir Goldstein]

On Wed, Oct 28, 2020 at 10:25 PM Darrick J. Wong
<darrick.wong@oracle.com> wrote:
>
> From: Darrick J. Wong <darrick.wong@oracle.com>
>
> Test that we can upgrade an existing filesystem to use bigtime.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Just some nits.
Not adding RVB because I would rather someone with more understanding of quotas
will review this.

> ---
>  common/xfs        |   16 ++++++
>  tests/xfs/908     |   87 ++++++++++++++++++++++++++++++
>  tests/xfs/908.out |   10 +++
>  tests/xfs/909     |  153 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/909.out |    4 +
>  tests/xfs/group   |    2 +
>  6 files changed, 272 insertions(+)
>  create mode 100755 tests/xfs/908
>  create mode 100644 tests/xfs/908.out
>  create mode 100755 tests/xfs/909
>  create mode 100644 tests/xfs/909.out
>
>
> diff --git a/common/xfs b/common/xfs
> index 19ccee03..4274eee7 100644
> --- a/common/xfs
> +++ b/common/xfs
> @@ -1008,3 +1008,19 @@ _xfs_timestamp_range()
>                 $dbprog -f -c 'timelimit --compact' | awk '{printf("%s %s", $1, $2);}'
>         fi
>  }
> +
> +_require_xfs_mkfs_bigtime()
> +{
> +       _scratch_mkfs_xfs_supported -m bigtime=1 >/dev/null 2>&1 \
> +          || _notrun "mkfs.xfs doesn't have bigtime feature"
> +}
> +
> +_require_xfs_scratch_bigtime()
> +{
> +       _require_scratch
> +
> +       _scratch_mkfs -m bigtime=1 > /dev/null
> +       _try_scratch_mount || \
> +               _notrun "bigtime not supported by scratch filesystem type: $FSTYP"
> +       _scratch_unmount
> +}
> diff --git a/tests/xfs/908 b/tests/xfs/908
> new file mode 100755
> index 00000000..e368d66c
> --- /dev/null
> +++ b/tests/xfs/908
> @@ -0,0 +1,87 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2020, Oracle and/or its affiliates.  All Rights Reserved.
> +#
> +# FS QA Test No. 908
> +#
> +# Check that we can upgrade a filesystem to support bigtime and that inode
> +# timestamps work properly after the upgrade.
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1    # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +       cd /
> +       rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# We have very specific formatting parameters, so don't let things get complex
> +# with realtime devices and external logs.
> +unset USE_EXTERNAL
> +
> +# real QA test starts here
> +_supported_fs xfs
> +_require_xfs_mkfs_crc
> +_require_xfs_mkfs_bigtime
> +_require_xfs_scratch_bigtime

Should we also explicitly require support for xfs_admin -O bigtime?

> +
> +date --date='Jan 1 00:00:00 UTC 2040' > /dev/null 2>&1 || \
> +       _notrun "Userspace does not support dates past 2038."
> +
> +rm -f $seqres.full
> +
> +# Format V5 filesystem without bigtime support and populate it
> +_scratch_mkfs -m crc=1,bigtime=0 > $seqres.full
> +_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
> +_scratch_mount >> $seqres.full
> +
> +touch -d 'Jan 9 19:19:19 UTC 1999' $SCRATCH_MNT/a
> +touch -d 'Jan 9 19:19:19 UTC 1999' $SCRATCH_MNT/b
> +ls -la $SCRATCH_MNT/* >> $seqres.full
> +
> +echo before upgrade:
> +TZ=UTC stat -c '%Y' $SCRATCH_MNT/a
> +TZ=UTC stat -c '%Y' $SCRATCH_MNT/b
> +
> +_scratch_unmount
> +_check_scratch_fs
> +
> +# Now upgrade to bigtime support
> +_scratch_xfs_admin -O bigtime >> $seqres.full
> +_check_scratch_fs
> +_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
> +
> +# Mount again, look at our files
> +_scratch_mount >> $seqres.full
> +ls -la $SCRATCH_MNT/* >> $seqres.full
> +
> +# Modify one of the timestamps to stretch beyond 2038
> +touch -d 'Feb 22 22:22:22 UTC 2222' $SCRATCH_MNT/b
> +
> +echo after upgrade:
> +TZ=UTC stat -c '%Y' $SCRATCH_MNT/a
> +TZ=UTC stat -c '%Y' $SCRATCH_MNT/b
> +
> +_scratch_cycle_mount
> +
> +# Did the timestamp survive the remount?
> +ls -la $SCRATCH_MNT/* >> $seqres.full
> +
> +echo after upgrade and remount:
> +TZ=UTC stat -c '%Y' $SCRATCH_MNT/a
> +TZ=UTC stat -c '%Y' $SCRATCH_MNT/b
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/908.out b/tests/xfs/908.out
> new file mode 100644
> index 00000000..f0f412be
> --- /dev/null
> +++ b/tests/xfs/908.out
> @@ -0,0 +1,10 @@
> +QA output created by 908
> +before upgrade:
> +915909559
> +915909559
> +after upgrade:
> +915909559
> +7956915742
> +after upgrade and remount:
> +915909559
> +7956915742
> diff --git a/tests/xfs/909 b/tests/xfs/909
> new file mode 100755
> index 00000000..7010eb9e
> --- /dev/null
> +++ b/tests/xfs/909
> @@ -0,0 +1,153 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2020, Oracle and/or its affiliates.  All Rights Reserved.
> +#
> +# FS QA Test No. 909
> +#
> +# Check that we can upgrade a filesystem to support bigtime and that quota
> +# timers work properly after the upgrade.
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1    # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +       cd /
> +       rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/quota
> +
> +# We have very specific formatting parameters, so don't let things get complex
> +# with realtime devices and external logs.
> +unset USE_EXTERNAL
> +
> +# real QA test starts here
> +_supported_fs xfs
> +_require_quota
> +_require_xfs_mkfs_crc
> +_require_xfs_mkfs_bigtime
> +_require_xfs_scratch_bigtime
> +
> +date --date='Jan 1 00:00:00 UTC 2040' > /dev/null 2>&1 || \
> +       _notrun "Userspace does not support dates past 2038."
> +
> +rm -f $seqres.full
> +
> +# Format V5 filesystem without bigtime support and populate it
> +_scratch_mkfs -m crc=1,bigtime=0 > $seqres.full
> +_qmount_option "usrquota"
> +_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
> +_scratch_mount >> $seqres.full
> +
> +# Force the block counters for uid 1 and 2 above zero
> +_pwrite_byte 0x61 0 64k $SCRATCH_MNT/a >> $seqres.full
> +_pwrite_byte 0x61 0 64k $SCRATCH_MNT/b >> $seqres.full
> +sync
> +chown 1 $SCRATCH_MNT/a
> +chown 2 $SCRATCH_MNT/b
> +
> +# Set quota limits on uid 1 before upgrading
> +$XFS_QUOTA_PROG -x -c 'limit -u bsoft=12k bhard=1m 1' $SCRATCH_MNT
> +
> +# Make sure the grace period is at /some/ point in the future.  We have to
> +# use bc because not all bashes can handle integer comparisons with 64-bit
> +# numbers.
> +repquota -upn $SCRATCH_MNT > $tmp.repquota
> +cat $tmp.repquota >> $seqres.full
> +grace="$(cat $tmp.repquota | grep '^#1' | awk '{print $6}')"
> +now="$(date +%s)"
> +res="$(echo "${grace} > ${now}" | $BC_PROG)"
> +test $res -eq 1 || echo "Expected timer expiry (${grace}) to be after now (${now})."
> +
> +_scratch_unmount
> +
> +# Now upgrade to bigtime support
> +_scratch_xfs_admin -O bigtime >> $seqres.full
> +_check_scratch_fs
> +_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
> +
> +# Mount again, see if our quota timer survived
> +_scratch_mount
> +
> +# Set a very generous grace period and quota limits on uid 2 after upgrading
> +$XFS_QUOTA_PROG -x -c 'timer -u -b -d 2147483647' $SCRATCH_MNT
> +$XFS_QUOTA_PROG -x -c 'limit -u bsoft=10000 bhard=150000 2' $SCRATCH_MNT
> +
> +# Query the grace periods to see if they got set properly after the upgrade.
> +repquota -upn $SCRATCH_MNT > $tmp.repquota
> +cat $tmp.repquota >> $seqres.full
> +grace1="$(repquota -upn $SCRATCH_MNT | grep '^#1' | awk '{print $6}')"
> +grace2="$(repquota -upn $SCRATCH_MNT | grep '^#2' | awk '{print $6}')"
> +now="$(date +%s)"
> +
> +# Make sure that uid 1's expiration is in the future...
> +res1="$(echo "${grace} > ${now}" | $BC_PROG)"
> +test "${res1}" -eq 1 || echo "Expected uid 1 expiry (${grace1}) to be after now (${now})."
> +
> +# ...and that uid 2's expiration is after uid 1's...
> +res2="$(echo "${grace2} > ${grace1}" | $BC_PROG)"
> +test "${res2}" -eq 1 || echo "Expected uid 2 expiry (${grace2}) to be after uid 1 (${grace1})."
> +
> +# ...and that uid 2's expiration is after 2038 if right now is far enough
> +# past 1970 that our generous grace period would provide for that.
> +res3="$(echo "(${now} < 100) || (${grace2} > 2147483648)" | $BC_PROG)"
> +test "${res3}" -eq 1 || echo "Expected uid 2 expiry (${grace2}) to be after 2038."
> +
> +_scratch_cycle_mount
> +
> +# Query the grace periods to see if they survived a remount.
> +repquota -upn $SCRATCH_MNT > $tmp.repquota
> +cat $tmp.repquota >> $seqres.full
> +grace1="$(repquota -upn $SCRATCH_MNT | grep '^#1' | awk '{print $6}')"
> +grace2="$(repquota -upn $SCRATCH_MNT | grep '^#2' | awk '{print $6}')"
> +now="$(date +%s)"
> +
> +# Make sure that uid 1's expiration is in the future...
> +res1="$(echo "${grace} > ${now}" | $BC_PROG)"
> +test "${res1}" -eq 1 || echo "Expected uid 1 expiry (${grace1}) to be after now (${now})."
> +
> +# ...and that uid 2's expiration is after uid 1's...
> +res2="$(echo "${grace2} > ${grace1}" | $BC_PROG)"
> +test "${res2}" -eq 1 || echo "Expected uid 2 expiry (${grace2}) to be after uid 1 (${grace1})."
> +
> +# ...and that uid 2's expiration is after 2038 if right now is far enough
> +# past 1970 that our generous grace period would provide for that.
> +res3="$(echo "(${now} < 100) || (${grace2} > 2147483648)" | $BC_PROG)"
> +test "${res3}" -eq 1 || echo "Expected uid 2 expiry (${grace2}) to be after 2038."
> +
> +# Now try setting uid 2's expiration to Feb 22 22:22:22 UTC 2222
> +new_expiry=$(date -d 'Feb 22 22:22:22 UTC 2222' +%s)
> +now=$(date +%s)
> +test $now -ge $new_expiry && \
> +       echo "Now is after February 2222?  Expect problems."
> +expiry_delta=$((new_expiry - now))
> +
> +echo "setting expiration to $new_expiry - $now = $expiry_delta" >> $seqres.full
> +$XFS_QUOTA_PROG -x -c "timer -u $expiry_delta 2" -c 'report' $SCRATCH_MNT >> $seqres.full
> +
> +# Did we get an expiration within 5s of the target range?
> +grace2="$(repquota -upn $SCRATCH_MNT | grep '^#2' | awk '{print $6}')"
> +echo "grace2 is $grace2" >> $seqres.full
> +_within_tolerance "grace2 expiry" $grace2 $new_expiry 5 -v
> +
> +_scratch_cycle_mount
> +
> +# ...and is it still within 5s after a remount?
> +grace2="$(repquota -upn $SCRATCH_MNT | grep '^#2' | awk '{print $6}')"
> +echo "grace2 is $grace2" >> $seqres.full
> +_within_tolerance "grace2 expiry after remount" $grace2 $new_expiry 5 -v
> +
> +# success, all done
> +echo Silence is golden.
> +status=0
> +exit
> diff --git a/tests/xfs/909.out b/tests/xfs/909.out
> new file mode 100644
> index 00000000..948502b7
> --- /dev/null
> +++ b/tests/xfs/909.out
> @@ -0,0 +1,4 @@
> +QA output created by 909
> +grace2 expiry is in range
> +grace2 expiry after remount is in range
> +Silence is golden.

Output is not silent ;-)

Thanks,
Amir.

Re: [PATCH 4/4] xfs: test upgrading filesystem to bigtime

[Darrick J. Wong]

On Thu, Oct 29, 2020 at 03:06:55PM +0200, Amir Goldstein wrote:
> On Wed, Oct 28, 2020 at 10:25 PM Darrick J. Wong
> <darrick.wong@oracle.com> wrote:
> >
> > From: Darrick J. Wong <darrick.wong@oracle.com>
> >
> > Test that we can upgrade an existing filesystem to use bigtime.
> >
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> 
> Just some nits.
> Not adding RVB because I would rather someone with more understanding of quotas
> will review this.
> 
> > ---
> >  common/xfs        |   16 ++++++
> >  tests/xfs/908     |   87 ++++++++++++++++++++++++++++++
> >  tests/xfs/908.out |   10 +++
> >  tests/xfs/909     |  153 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> >  tests/xfs/909.out |    4 +
> >  tests/xfs/group   |    2 +
> >  6 files changed, 272 insertions(+)
> >  create mode 100755 tests/xfs/908
> >  create mode 100644 tests/xfs/908.out
> >  create mode 100755 tests/xfs/909
> >  create mode 100644 tests/xfs/909.out
> >
> >
> > diff --git a/common/xfs b/common/xfs
> > index 19ccee03..4274eee7 100644
> > --- a/common/xfs
> > +++ b/common/xfs
> > @@ -1008,3 +1008,19 @@ _xfs_timestamp_range()
> >                 $dbprog -f -c 'timelimit --compact' | awk '{printf("%s %s", $1, $2);}'
> >         fi
> >  }
> > +
> > +_require_xfs_mkfs_bigtime()
> > +{
> > +       _scratch_mkfs_xfs_supported -m bigtime=1 >/dev/null 2>&1 \
> > +          || _notrun "mkfs.xfs doesn't have bigtime feature"
> > +}
> > +
> > +_require_xfs_scratch_bigtime()
> > +{
> > +       _require_scratch
> > +
> > +       _scratch_mkfs -m bigtime=1 > /dev/null
> > +       _try_scratch_mount || \
> > +               _notrun "bigtime not supported by scratch filesystem type: $FSTYP"
> > +       _scratch_unmount
> > +}
> > diff --git a/tests/xfs/908 b/tests/xfs/908
> > new file mode 100755
> > index 00000000..e368d66c
> > --- /dev/null
> > +++ b/tests/xfs/908
> > @@ -0,0 +1,87 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0-or-later
> > +# Copyright (c) 2020, Oracle and/or its affiliates.  All Rights Reserved.
> > +#
> > +# FS QA Test No. 908
> > +#
> > +# Check that we can upgrade a filesystem to support bigtime and that inode
> > +# timestamps work properly after the upgrade.
> > +
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1    # failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > +       cd /
> > +       rm -f $tmp.*
> > +}
> > +
> > +# get standard environment, filters and checks
> > +. ./common/rc
> > +. ./common/filter
> > +
> > +# We have very specific formatting parameters, so don't let things get complex
> > +# with realtime devices and external logs.
> > +unset USE_EXTERNAL
> > +
> > +# real QA test starts here
> > +_supported_fs xfs
> > +_require_xfs_mkfs_crc
> > +_require_xfs_mkfs_bigtime
> > +_require_xfs_scratch_bigtime
> 
> Should we also explicitly require support for xfs_admin -O bigtime?

<shrug> I don't know.  Assuming all the xfsprogs support code goes in at
the same time, then the fact that you can format with bigtime means that
xfs_admin will support upgrading to bigtime.

> > diff --git a/tests/xfs/909.out b/tests/xfs/909.out
> > new file mode 100644
> > index 00000000..948502b7
> > --- /dev/null
> > +++ b/tests/xfs/909.out
> > @@ -0,0 +1,4 @@
> > +QA output created by 909
> > +grace2 expiry is in range
> > +grace2 expiry after remount is in range
> > +Silence is golden.
> 
> Output is not silent ;-)

Will fix, thanks.

--D

> Thanks,
> Amir.

Re: [PATCH 3/4] xfs: detect time limits from filesystem

[Darrick J. Wong]

On Thu, Oct 29, 2020 at 12:47:32PM +0200, Amir Goldstein wrote:
> On Wed, Oct 28, 2020 at 10:24 PM Darrick J. Wong
> <darrick.wong@oracle.com> wrote:
> >
> > From: Darrick J. Wong <darrick.wong@oracle.com>
> >
> > Teach fstests to extract timestamp limits of a filesystem using the new
> > xfs_db timelimit command.
> >
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > ---
> >  common/rc         |    2 +-
> >  common/xfs        |   14 ++++++++++++++
> >  tests/xfs/911     |   44 ++++++++++++++++++++++++++++++++++++++++++++
> >  tests/xfs/911.out |   15 +++++++++++++++
> >  tests/xfs/group   |    1 +
> >  5 files changed, 75 insertions(+), 1 deletion(-)
> >  create mode 100755 tests/xfs/911
> >  create mode 100644 tests/xfs/911.out
> >
> >
> > diff --git a/common/rc b/common/rc
> > index 41f93047..162d957a 100644
> > --- a/common/rc
> > +++ b/common/rc
> > @@ -2029,7 +2029,7 @@ _filesystem_timestamp_range()
> >                 echo "0 $u32max"
> >                 ;;
> >         xfs)
> > -               echo "$s32min $s32max"
> > +               _xfs_timestamp_range "$device"
> >                 ;;
> >         btrfs)
> >                 echo "$s64min $s64max"
> > diff --git a/common/xfs b/common/xfs
> > index e548a0a1..19ccee03 100644
> > --- a/common/xfs
> > +++ b/common/xfs
> > @@ -994,3 +994,17 @@ _require_xfs_scratch_inobtcount()
> >                 _notrun "inobtcount not supported by scratch filesystem type: $FSTYP"
> >         _scratch_unmount
> >  }
> > +
> > +_xfs_timestamp_range()
> > +{
> > +       local use_db=0
> > +       local dbprog="$XFS_DB_PROG $device"

Heh, device isn't defined, I'll fix that.

> > +       test "$device" = "$SCRATCH_DEV" && dbprog=_scratch_xfs_db
> > +
> > +       $dbprog -f -c 'help timelimit' | grep -v -q 'not found' && use_db=1
> > +       if [ $use_db -eq 0 ]; then
> > +               echo "-$((1<<31)) $(((1<<31)-1))"
> 
> This embodies an assumption that the tested filesystem does not have
> bigtime enabled if xfs_db tool is not uptodate.

If the xfs_db tool doesn't support the timelimit command then it doesn't
support formatting with bigtime.  I don't think it's reasonable to
expect to be able to run fstests on a test filesystem that xfsprogs
doesn't support.  Hence it's fine to output the old limits if the
timelimit command doesn't exist.

> Maybe it makes sense, but it may be safer to return "-1 -1" and not_run
> generic/402 if xfs_db is not uptodate, perhaps with an extra message
> hinting the user to upgrade xfs_db.

TBH it boggles my mind that there *still* is no way to ask the kernel
for the supported timestamp range of a mounted filesystem.  The
timelimit command and this mess in fstests was supposed to be a
temporary workaround that would (in my ideal world) have become
unnecessary before this landed, but ... ugh.

--D

> Thanks,
> Amir.

Re: [PATCH 2/4] xfs/122: add legacy timestamps to ondisk checker

[Darrick J. Wong]

On Thu, Oct 29, 2020 at 01:28:07PM +0200, Amir Goldstein wrote:
> On Wed, Oct 28, 2020 at 10:25 PM Darrick J. Wong
> <darrick.wong@oracle.com> wrote:
> >
> > From: Darrick J. Wong <darrick.wong@oracle.com>
> >
> > Add these new ondisk structures.
> >
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> 
> Reviewed-by: Amir Goldstein <amir73il@gmail.com>
> 
> > ---
> >  tests/xfs/122     |    1 +
> >  tests/xfs/122.out |    1 +
> >  2 files changed, 2 insertions(+)
> >
> >
> > diff --git a/tests/xfs/122 b/tests/xfs/122
> > index a4248031..db21f2d5 100755
> > --- a/tests/xfs/122
> > +++ b/tests/xfs/122
> > @@ -182,6 +182,7 @@ struct xfs_iext_cursor
> >  struct xfs_ino_geometry
> >  struct xfs_attrlist
> >  struct xfs_attrlist_ent
> > +struct xfs_legacy_ictimestamp
> 
> <OCD move near xfs_ictimestamp_t similar to other adjacent struct and
> typedefs />

Ok will do.

--D

> 
> Thanks,
> Amir.

Re: [PATCH 3/4] xfs: detect time limits from filesystem

[Amir Goldstein]

On Thu, Oct 29, 2020 at 8:27 PM Darrick J. Wong <darrick.wong@oracle.com> wrote:
>
> On Thu, Oct 29, 2020 at 12:47:32PM +0200, Amir Goldstein wrote:
> > On Wed, Oct 28, 2020 at 10:24 PM Darrick J. Wong
> > <darrick.wong@oracle.com> wrote:
> > >
> > > From: Darrick J. Wong <darrick.wong@oracle.com>
> > >
> > > Teach fstests to extract timestamp limits of a filesystem using the new
> > > xfs_db timelimit command.
> > >
> > > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > > ---
> > >  common/rc         |    2 +-
> > >  common/xfs        |   14 ++++++++++++++
> > >  tests/xfs/911     |   44 ++++++++++++++++++++++++++++++++++++++++++++
> > >  tests/xfs/911.out |   15 +++++++++++++++
> > >  tests/xfs/group   |    1 +
> > >  5 files changed, 75 insertions(+), 1 deletion(-)
> > >  create mode 100755 tests/xfs/911
> > >  create mode 100644 tests/xfs/911.out
> > >
> > >
> > > diff --git a/common/rc b/common/rc
> > > index 41f93047..162d957a 100644
> > > --- a/common/rc
> > > +++ b/common/rc
> > > @@ -2029,7 +2029,7 @@ _filesystem_timestamp_range()
> > >                 echo "0 $u32max"
> > >                 ;;
> > >         xfs)
> > > -               echo "$s32min $s32max"
> > > +               _xfs_timestamp_range "$device"
> > >                 ;;
> > >         btrfs)
> > >                 echo "$s64min $s64max"
> > > diff --git a/common/xfs b/common/xfs
> > > index e548a0a1..19ccee03 100644
> > > --- a/common/xfs
> > > +++ b/common/xfs
> > > @@ -994,3 +994,17 @@ _require_xfs_scratch_inobtcount()
> > >                 _notrun "inobtcount not supported by scratch filesystem type: $FSTYP"
> > >         _scratch_unmount
> > >  }
> > > +
> > > +_xfs_timestamp_range()
> > > +{
> > > +       local use_db=0
> > > +       local dbprog="$XFS_DB_PROG $device"
>
> Heh, device isn't defined, I'll fix that.
>
> > > +       test "$device" = "$SCRATCH_DEV" && dbprog=_scratch_xfs_db
> > > +
> > > +       $dbprog -f -c 'help timelimit' | grep -v -q 'not found' && use_db=1
> > > +       if [ $use_db -eq 0 ]; then
> > > +               echo "-$((1<<31)) $(((1<<31)-1))"
> >
> > This embodies an assumption that the tested filesystem does not have
> > bigtime enabled if xfs_db tool is not uptodate.
>
> If the xfs_db tool doesn't support the timelimit command then it doesn't
> support formatting with bigtime.  I don't think it's reasonable to
> expect to be able to run fstests on a test filesystem that xfsprogs
> doesn't support.  Hence it's fine to output the old limits if the
> timelimit command doesn't exist.

ok.

>
> > Maybe it makes sense, but it may be safer to return "-1 -1" and not_run
> > generic/402 if xfs_db is not uptodate, perhaps with an extra message
> > hinting the user to upgrade xfs_db.
>
> TBH it boggles my mind that there *still* is no way to ask the kernel
> for the supported timestamp range of a mounted filesystem.  The
> timelimit command and this mess in fstests was supposed to be a
> temporary workaround that would (in my ideal world) have become
> unnecessary before this landed, but ... ugh.
>

Oh well, consider this

Reviewed-by: Amir Goldstein <amir73il@gmail.com>

Thanks,
Amir.

Re: [PATCH 1/4] generic: check userspace handling of extreme timestamps

[Darrick J. Wong]

On Thu, Oct 29, 2020 at 12:34:57PM +0200, Amir Goldstein wrote:
> On Wed, Oct 28, 2020 at 10:25 PM Darrick J. Wong
> <darrick.wong@oracle.com> wrote:
> >
> > From: Darrick J. Wong <darrick.wong@oracle.com>
> >
> > These two tests ensure we can store and retrieve timestamps on the
> > extremes of the date ranges supported by userspace, and the common
> > places where overflows can happen.
> >
> > They differ from generic/402 in that they don't constrain the dates
> > tested to the range that the filesystem claims to support; we attempt
> > various things that /userspace/ can parse, and then check that the vfs
> > clamps and persists the values correctly.
> 
> So this test will fail when run on stable kernels before the vfs
> clamping changes
> and there is no require_* to mitigate that failure.

Yes, that is the intended outcome.  Those old kernels silently truncate
the high bits from those timestamps when inodes are flushed to disk, and
the only user-visible evidence of this comes much later when the system
reboots and suddenly the timestamps are wrong.  Clamping also seems a
little strange, but at least it's immediately obvious.

It is very surprising that you could set a timestamp of 2 Apr 2500 on
ext2, ls your shiny futuristic timestamp, reboot, and have it become
5 Nov 1955.  Only Marty McFly would be amused.

> At the time, I discussed this with Deepa and the result was the
> _check_dmesg_for part of _require_timestamp_range, which is incomplete.
> The complete check for kernel clamping support would be to run
> _require_timestamp_range (on the second half thereof) on a loop mounted
> ext2, so we know for sure that the kernel is going to emit the y2038 warning.

Uh... I don't think it's a sensible to require ext2 to test a different
filesystem.

> I am going to leave it to you and the maintainer the decide how
> critical that is,
> but I would suggest to at least factor out _require_timestamp_limits()
> which is true if either the filesystem timestamp range is known or the kernel
> emits y2038 warning.

TBH I had rather hoped that the VFS woulud have a way to export the
supported timestamp range by now, but that probably failed when
dhowells' big complicated fsinfo series died.

--D

> >
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > ---
> >  tests/generic/721     |  117 ++++++++++++++++++++++++++++++++++++++++++++++++
> >  tests/generic/721.out |    1
> >  tests/generic/722     |  120 +++++++++++++++++++++++++++++++++++++++++++++++++
> >  tests/generic/722.out |    1
> >  tests/generic/group   |    2 +
> >  5 files changed, 241 insertions(+)
> >  create mode 100755 tests/generic/721
> >  create mode 100644 tests/generic/721.out
> >  create mode 100755 tests/generic/722
> >  create mode 100644 tests/generic/722.out
> >
> >
> > diff --git a/tests/generic/721 b/tests/generic/721
> > new file mode 100755
> > index 00000000..9638fbfc
> > --- /dev/null
> > +++ b/tests/generic/721
> > @@ -0,0 +1,117 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0-or-later
> > +# Copyright (c) 2020, Oracle and/or its affiliates.  All Rights Reserved.
> > +#
> > +# FS QA Test No. 721
> > +#
> > +# Make sure we can store and retrieve timestamps on the extremes of the
> > +# date ranges supported by userspace, and the common places where overflows
> > +# can happen.
> > +#
> > +# This differs from generic/402 in that we don't constrain ourselves to the
> > +# range that the filesystem claims to support; we attempt various things that
> > +# /userspace/ can parse, and then check that the vfs clamps and persists the
> > +# values correctly.
> > +
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1    # failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > +       cd /
> > +       rm -f $tmp.*
> > +}
> > +
> > +# get standard environment, filters and checks
> > +. ./common/rc
> > +
> > +# real QA test starts here
> > +_supported_fs generic
> > +_require_scratch
> > +
> > +rm -f $seqres.full
> > +
> > +_scratch_mkfs > $seqres.full
> > +_scratch_mount
> > +
> > +# Does our userspace even support large dates?
> > +test_bigdates=1
> > +touch -d 'May 30 01:53:03 UTC 2514' $SCRATCH_MNT 2>/dev/null || test_bigdates=0
> > +
> > +# And can we do statx?
> > +test_statx=1
> > +($XFS_IO_PROG -c 'help statx' | grep -q 'Print raw statx' && \
> > + $XFS_IO_PROG -c 'statx -r' $SCRATCH_MNT 2>/dev/null | grep -q 'stat.mtime') || \
> > +       test_statx=0
> > +
> > +echo "Userspace support of large timestamps: $test_bigdates" >> $seqres.full
> > +echo "xfs_io support of statx: $test_statx" >> $seqres.full
> > +
> > +touchme() {
> > +       local arg="$1"
> > +       local name="$2"
> > +
> > +       echo "$arg" > $SCRATCH_MNT/t_$name
> > +       touch -d "$arg" $SCRATCH_MNT/t_$name
> > +}
> > +
> > +report() {
> > +       local files=($SCRATCH_MNT/t_*)
> > +       for file in "${files[@]}"; do
> > +               echo "${file}: $(cat "${file}")"
> > +               TZ=UTC stat -c '%y %Y %n' "${file}"
> > +               test $test_statx -gt 0 && \
> > +                       $XFS_IO_PROG -c 'statx -r' "${file}" | grep 'stat.mtime'
> > +       done
> > +}
> > +
> > +# -2147483648 (S32_MIN, or classic unix min)
> > +touchme 'Dec 13 20:45:52 UTC 1901' s32_min
> > +
> > +# 2147483647 (S32_MAX, or classic unix max)
> > +touchme 'Jan 19 03:14:07 UTC 2038' s32_max
> > +
> > +# 7956915742, all twos
> > +touchme 'Feb 22 22:22:22 UTC 2222' all_twos
> > +
> > +if [ $test_bigdates -gt 0 ]; then
> > +       # 16299260424 (u64 nsec counter from s32_min, like xfs does)
> > +       touchme 'Tue Jul  2 20:20:24 UTC 2486' u64ns_from_s32_min
> > +
> > +       # 15032385535 (u34 time if you start from s32_min, like ext4 does)
> > +       touchme 'May 10 22:38:55 UTC 2446' u34_from_s32_min
> > +
> > +       # 17179869183 (u34 time if you start from the unix epoch)
> > +       touchme 'May 30 01:53:03 UTC 2514' u34_max
> > +
> > +       # Latest date we can synthesize(?)
> > +       touchme 'Dec 31 23:59:59 UTC 2147483647' abs_max_time
> > +
> > +       # Earliest date we can synthesize(?)
> > +       touchme 'Jan 1 00:00:00 UTC 0' abs_min_time
> > +fi
> > +
> > +# Query timestamps from incore
> > +echo before >> $seqres.full
> > +report > $tmp.times0
> > +cat $tmp.times0 >> $seqres.full
> > +
> > +_scratch_cycle_mount
> > +
> > +# Query timestamps from disk
> > +echo after >> $seqres.full
> > +report > $tmp.times1
> > +cat $tmp.times1 >> $seqres.full
> > +
> > +# Did they match?
> > +cmp -s $tmp.times0 $tmp.times1
> > +
> 
> Please use suffix $tmp.{before,after}_cycle_mount
> It makes the meaning of the diff in the test failure much clearer
> to a bystander.
> 
> > +# success, all done
> > +status=0
> > +exit
> > diff --git a/tests/generic/721.out b/tests/generic/721.out
> > new file mode 100644
> > index 00000000..087decb5
> > --- /dev/null
> > +++ b/tests/generic/721.out
> > @@ -0,0 +1 @@
> > +QA output created by 721
> 
> What? no "Silence is golden"? :-D
> 
> > diff --git a/tests/generic/722 b/tests/generic/722
> > new file mode 100755
> > index 00000000..3e8c553b
> > --- /dev/null
> > +++ b/tests/generic/722
> > @@ -0,0 +1,120 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0-or-later
> > +# Copyright (c) 2020, Oracle and/or its affiliates.  All Rights Reserved.
> > +#
> > +# FS QA Test No. 722
> > +#
> > +# Make sure we can store and retrieve timestamps on the extremes of the
> > +# date ranges supported by userspace, and the common places where overflows
> > +# can happen.  This test also ensures that the timestamps are persisted
> > +# correctly after a shutdown.
> > +#
> > +# This differs from generic/402 in that we don't constrain ourselves to the
> > +# range that the filesystem claims to support; we attempt various things that
> > +# /userspace/ can parse, and then check that the vfs clamps and persists the
> > +# values correctly.
> > +
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1    # failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > +       cd /
> > +       rm -f $tmp.*
> > +}
> > +
> > +# get standard environment, filters and checks
> > +. ./common/rc
> > +
> > +# real QA test starts here
> > +_supported_fs generic
> > +_require_scratch
> > +_require_scratch_shutdown
> > +
> > +rm -f $seqres.full
> > +
> > +_scratch_mkfs > $seqres.full
> > +_scratch_mount
> > +
> > +# Does our userspace even support large dates?
> > +test_bigdates=1
> > +touch -d 'May 30 01:53:03 UTC 2514' $SCRATCH_MNT 2>/dev/null || test_bigdates=0
> > +
> > +# And can we do statx?
> > +test_statx=1
> > +($XFS_IO_PROG -c 'help statx' | grep -q 'Print raw statx' && \
> > + $XFS_IO_PROG -c 'statx -r' $SCRATCH_MNT 2>/dev/null | grep -q 'stat.mtime') || \
> > +       test_statx=0
> > +
> > +echo "Userspace support of large timestamps: $test_bigdates" >> $seqres.full
> > +echo "xfs_io support of statx: $test_statx" >> $seqres.full
> > +
> > +touchme() {
> > +       local arg="$1"
> > +       local name="$2"
> > +
> > +       echo "$arg" > $SCRATCH_MNT/t_$name
> > +       touch -d "$arg" $SCRATCH_MNT/t_$name
> > +}
> > +
> > +report() {
> > +       local files=($SCRATCH_MNT/t_*)
> > +       for file in "${files[@]}"; do
> > +               echo "${file}: $(cat "${file}")"
> > +               TZ=UTC stat -c '%y %Y %n' "${file}"
> > +               test $test_statx -gt 0 && \
> > +                       $XFS_IO_PROG -c 'statx -r' "${file}" | grep 'stat.mtime'
> > +       done
> > +}
> > +
> > +# -2147483648 (S32_MIN, or classic unix min)
> > +touchme 'Dec 13 20:45:52 UTC 1901' s32_min
> > +
> > +# 2147483647 (S32_MAX, or classic unix max)
> > +touchme 'Jan 19 03:14:07 UTC 2038' s32_max
> > +
> > +# 7956915742, all twos
> > +touchme 'Feb 22 22:22:22 UTC 2222' all_twos
> > +
> > +if [ $test_bigdates -gt 0 ]; then
> > +       # 16299260424 (u64 nsec counter from s32_min, like xfs does)
> > +       touchme 'Tue Jul  2 20:20:24 UTC 2486' u64ns_from_s32_min
> > +
> > +       # 15032385535 (u34 time if you start from s32_min, like ext4 does)
> > +       touchme 'May 10 22:38:55 UTC 2446' u34_from_s32_min
> > +
> > +       # 17179869183 (u34 time if you start from the unix epoch)
> > +       touchme 'May 30 01:53:03 UTC 2514' u34_max
> > +
> > +       # Latest date we can synthesize(?)
> > +       touchme 'Dec 31 23:59:59 UTC 2147483647' abs_max_time
> > +
> > +       # Earliest date we can synthesize(?)
> > +       touchme 'Jan 1 00:00:00 UTC 0' abs_min_time
> > +fi
> > +
> > +# Query timestamps from incore
> > +echo before >> $seqres.full
> > +report > $tmp.times0
> > +cat $tmp.times0 >> $seqres.full
> > +
> > +_scratch_shutdown -f
> > +_scratch_cycle_mount
> > +
> > +# Query timestamps from disk
> > +echo after >> $seqres.full
> > +report > $tmp.times1
> > +cat $tmp.times1 >> $seqres.full
> > +
> > +# Did they match?
> > +cmp -s $tmp.times0 $tmp.times1
> > +
> > +# success, all done
> > +status=0
> > +exit
> > diff --git a/tests/generic/722.out b/tests/generic/722.out
> > new file mode 100644
> > index 00000000..83acd5cf
> > --- /dev/null
> > +++ b/tests/generic/722.out
> > @@ -0,0 +1 @@
> > +QA output created by 722
> > diff --git a/tests/generic/group b/tests/generic/group
> > index cf4fdc23..b533d6b2 100644
> > --- a/tests/generic/group
> > +++ b/tests/generic/group
> > @@ -615,5 +615,7 @@
> >  610 auto quick prealloc zero
> >  611 auto quick attr
> >  612 auto quick clone
> > +721 auto quick atime bigtime
> > +722 auto quick atime bigtime
> 
> shutdown group please.
> 
> If we are going to use "bigtime" for generic tests to describe y2038 tests,
> perhaps add it to 258 and 402 as well?
> 
> Thanks,
> Amir.

Re: [PATCH 1/4] generic: check userspace handling of extreme timestamps

[Amir Goldstein]

On Thu, Oct 29, 2020 at 11:02 PM Darrick J. Wong
<darrick.wong@oracle.com> wrote:
>
> On Thu, Oct 29, 2020 at 12:34:57PM +0200, Amir Goldstein wrote:
> > On Wed, Oct 28, 2020 at 10:25 PM Darrick J. Wong
> > <darrick.wong@oracle.com> wrote:
> > >
> > > From: Darrick J. Wong <darrick.wong@oracle.com>
> > >
> > > These two tests ensure we can store and retrieve timestamps on the
> > > extremes of the date ranges supported by userspace, and the common
> > > places where overflows can happen.
> > >
> > > They differ from generic/402 in that they don't constrain the dates
> > > tested to the range that the filesystem claims to support; we attempt
> > > various things that /userspace/ can parse, and then check that the vfs
> > > clamps and persists the values correctly.
> >
> > So this test will fail when run on stable kernels before the vfs
> > clamping changes
> > and there is no require_* to mitigate that failure.
>
> Yes, that is the intended outcome.  Those old kernels silently truncate
> the high bits from those timestamps when inodes are flushed to disk, and
> the only user-visible evidence of this comes much later when the system
> reboots and suddenly the timestamps are wrong.  Clamping also seems a
> little strange, but at least it's immediately obvious.
>
> It is very surprising that you could set a timestamp of 2 Apr 2500 on
> ext2, ls your shiny futuristic timestamp, reboot, and have it become
> 5 Nov 1955.  Only Marty McFly would be amused.
>

OK. So we can call it a bug in old kernels that is not going to be fixed
in stable updates. The minimum we can do for stable kernel testers is
provide a decent way to exclude the tests for clamping.

I guess 'check -x bigtime' is decent enough.
I might have named the group 'timelimit' but I can live with 'bigtime'.

So with fix for the rest of my minor nits, you may add:

Reviewed-by: Amir Goldstein <amir73il@gmail.com>

Thanks,
Amir.

Re: [PATCH 1/4] generic: check userspace handling of extreme timestamps

[Darrick J. Wong]

On Thu, Oct 29, 2020 at 11:40:00PM +0200, Amir Goldstein wrote:
> On Thu, Oct 29, 2020 at 11:02 PM Darrick J. Wong
> <darrick.wong@oracle.com> wrote:
> >
> > On Thu, Oct 29, 2020 at 12:34:57PM +0200, Amir Goldstein wrote:
> > > On Wed, Oct 28, 2020 at 10:25 PM Darrick J. Wong
> > > <darrick.wong@oracle.com> wrote:
> > > >
> > > > From: Darrick J. Wong <darrick.wong@oracle.com>
> > > >
> > > > These two tests ensure we can store and retrieve timestamps on the
> > > > extremes of the date ranges supported by userspace, and the common
> > > > places where overflows can happen.
> > > >
> > > > They differ from generic/402 in that they don't constrain the dates
> > > > tested to the range that the filesystem claims to support; we attempt
> > > > various things that /userspace/ can parse, and then check that the vfs
> > > > clamps and persists the values correctly.
> > >
> > > So this test will fail when run on stable kernels before the vfs
> > > clamping changes
> > > and there is no require_* to mitigate that failure.
> >
> > Yes, that is the intended outcome.  Those old kernels silently truncate
> > the high bits from those timestamps when inodes are flushed to disk, and
> > the only user-visible evidence of this comes much later when the system
> > reboots and suddenly the timestamps are wrong.  Clamping also seems a
> > little strange, but at least it's immediately obvious.
> >
> > It is very surprising that you could set a timestamp of 2 Apr 2500 on
> > ext2, ls your shiny futuristic timestamp, reboot, and have it become
> > 5 Nov 1955.  Only Marty McFly would be amused.
> >
> 
> OK. So we can call it a bug in old kernels that is not going to be fixed
> in stable updates. The minimum we can do for stable kernel testers is
> provide a decent way to exclude the tests for clamping.
> 
> I guess 'check -x bigtime' is decent enough.
> I might have named the group 'timelimit' but I can live with 'bigtime'.
> 
> So with fix for the rest of my minor nits, you may add:

Ok, I've fixed them all.  I also added warnings to 721 and 722 that the
test is expected to fail on pre-5.4 kernels.  Thanks for reviewing!

--D

> Reviewed-by: Amir Goldstein <amir73il@gmail.com>
> 
> Thanks,
> Amir.

Re: [PATCH 1/4] generic: check userspace handling of extreme timestamps

[Darrick J. Wong]

On Thu, Apr 22, 2021 at 02:16:28PM -0700, Allison Henderson wrote:
> 
> 
> On 4/20/21 5:23 PM, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > These two tests ensure we can store and retrieve timestamps on the
> > extremes of the date ranges supported by userspace, and the common
> > places where overflows can happen.
> > 
> > They differ from generic/402 in that they don't constrain the dates
> > tested to the range that the filesystem claims to support; we attempt
> > various things that /userspace/ can parse, and then check that the vfs
> > clamps and persists the values correctly.
> > 
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > Reviewed-by: Amir Goldstein <amir73il@gmail.com>
> Looks ok to me.  The tests have quite a bit of overlap, but they're pretty
> small too.

Yeah.  The difference between 721 and 722 is that 722 tests that
timestamp updates work properly if the fs goes down right after an
update.

> I think if we add on any more, probably we should think about a
> common/bigtime file?  But for now, I think it's fine.

Noted.  Thanks for reviewing this series!

--D

> 
> Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
> 
> > ---
> >   tests/generic/721     |  123 ++++++++++++++++++++++++++++++++++++++++++++++++
> >   tests/generic/721.out |    2 +
> >   tests/generic/722     |  125 +++++++++++++++++++++++++++++++++++++++++++++++++
> >   tests/generic/722.out |    1
> >   tests/generic/group   |    6 ++
> >   5 files changed, 255 insertions(+), 2 deletions(-)
> >   create mode 100755 tests/generic/721
> >   create mode 100644 tests/generic/721.out
> >   create mode 100755 tests/generic/722
> >   create mode 100644 tests/generic/722.out
> > 
> > 
> > diff --git a/tests/generic/721 b/tests/generic/721
> > new file mode 100755
> > index 00000000..9198b6b4
> > --- /dev/null
> > +++ b/tests/generic/721
> > @@ -0,0 +1,123 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0-or-later
> > +# Copyright (c) 2021 Oracle.  All Rights Reserved.
> > +#
> > +# FS QA Test No. 721
> > +#
> > +# Make sure we can store and retrieve timestamps on the extremes of the
> > +# date ranges supported by userspace, and the common places where overflows
> > +# can happen.
> > +#
> > +# This differs from generic/402 in that we don't constrain ourselves to the
> > +# range that the filesystem claims to support; we attempt various things that
> > +# /userspace/ can parse, and then check that the vfs clamps and persists the
> > +# values correctly.
> > +#
> > +# NOTE: Old kernels (pre 5.4) allow filesystems to truncate timestamps silently
> > +# when writing timestamps to disk!  This test detects this silent truncation
> > +# and fails.  If you see a failure on such a kernel, contact your distributor
> > +# for an update.
> > +
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1    # failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > +	cd /
> > +	rm -f $tmp.*
> > +}
> > +
> > +# get standard environment, filters and checks
> > +. ./common/rc
> > +
> > +# real QA test starts here
> > +_supported_fs generic
> > +_require_scratch
> > +
> > +rm -f $seqres.full
> > +
> > +_scratch_mkfs > $seqres.full
> > +_scratch_mount
> > +
> > +# Does our userspace even support large dates?
> > +test_bigdates=1
> > +touch -d 'May 30 01:53:03 UTC 2514' $SCRATCH_MNT 2>/dev/null || test_bigdates=0
> > +
> > +# And can we do statx?
> > +test_statx=1
> > +($XFS_IO_PROG -c 'help statx' | grep -q 'Print raw statx' && \
> > + $XFS_IO_PROG -c 'statx -r' $SCRATCH_MNT 2>/dev/null | grep -q 'stat.mtime') || \
> > +	test_statx=0
> > +
> > +echo "Userspace support of large timestamps: $test_bigdates" >> $seqres.full
> > +echo "xfs_io support of statx: $test_statx" >> $seqres.full
> > +
> > +touchme() {
> > +	local arg="$1"
> > +	local name="$2"
> > +
> > +	echo "$arg" > $SCRATCH_MNT/t_$name
> > +	touch -d "$arg" $SCRATCH_MNT/t_$name
> > +}
> > +
> > +report() {
> > +	local files=($SCRATCH_MNT/t_*)
> > +	for file in "${files[@]}"; do
> > +		echo "${file}: $(cat "${file}")"
> > +		TZ=UTC stat -c '%y %Y %n' "${file}"
> > +		test $test_statx -gt 0 && \
> > +			$XFS_IO_PROG -c 'statx -r' "${file}" | grep 'stat.mtime'
> > +	done
> > +}
> > +
> > +# -2147483648 (S32_MIN, or classic unix min)
> > +touchme 'Dec 13 20:45:52 UTC 1901' s32_min
> > +
> > +# 2147483647 (S32_MAX, or classic unix max)
> > +touchme 'Jan 19 03:14:07 UTC 2038' s32_max
> > +
> > +# 7956915742, all twos
> > +touchme 'Feb 22 22:22:22 UTC 2222' all_twos
> > +
> > +if [ $test_bigdates -gt 0 ]; then
> > +	# 16299260424 (u64 nsec counter from s32_min, like xfs does)
> > +	touchme 'Tue Jul  2 20:20:24 UTC 2486' u64ns_from_s32_min
> > +
> > +	# 15032385535 (u34 time if you start from s32_min, like ext4 does)
> > +	touchme 'May 10 22:38:55 UTC 2446' u34_from_s32_min
> > +
> > +	# 17179869183 (u34 time if you start from the unix epoch)
> > +	touchme 'May 30 01:53:03 UTC 2514' u34_max
> > +
> > +	# Latest date we can synthesize(?)
> > +	touchme 'Dec 31 23:59:59 UTC 2147483647' abs_max_time
> > +
> > +	# Earliest date we can synthesize(?)
> > +	touchme 'Jan 1 00:00:00 UTC 0' abs_min_time
> > +fi
> > +
> > +# Query timestamps from incore
> > +echo before >> $seqres.full
> > +report > $tmp.before_remount
> > +cat $tmp.before_remount >> $seqres.full
> > +
> > +_scratch_cycle_mount
> > +
> > +# Query timestamps from disk
> > +echo after >> $seqres.full
> > +report > $tmp.after_remount
> > +cat $tmp.after_remount >> $seqres.full
> > +
> > +# Did they match?
> > +cmp -s $tmp.before_remount $tmp.after_remount
> > +
> > +# success, all done
> > +echo Silence is golden.
> > +status=0
> > +exit
> > diff --git a/tests/generic/721.out b/tests/generic/721.out
> > new file mode 100644
> > index 00000000..b2bc6d58
> > --- /dev/null
> > +++ b/tests/generic/721.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 721
> > +Silence is golden.
> > diff --git a/tests/generic/722 b/tests/generic/722
> > new file mode 100755
> > index 00000000..305c3bd6
> > --- /dev/null
> > +++ b/tests/generic/722
> > @@ -0,0 +1,125 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0-or-later
> > +# Copyright (c) 2021 Oracle.  All Rights Reserved.
> > +#
> > +# FS QA Test No. 722
> > +#
> > +# Make sure we can store and retrieve timestamps on the extremes of the
> > +# date ranges supported by userspace, and the common places where overflows
> > +# can happen.  This test also ensures that the timestamps are persisted
> > +# correctly after a shutdown.
> > +#
> > +# This differs from generic/402 in that we don't constrain ourselves to the
> > +# range that the filesystem claims to support; we attempt various things that
> > +# /userspace/ can parse, and then check that the vfs clamps and persists the
> > +# values correctly.
> > +#
> > +# NOTE: Old kernels (pre 5.4) allow filesystems to truncate timestamps silently
> > +# when writing timestamps to disk!  This test detects this silent truncation
> > +# and fails.  If you see a failure on such a kernel, contact your distributor
> > +# for an update.
> > +
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1    # failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > +	cd /
> > +	rm -f $tmp.*
> > +}
> > +
> > +# get standard environment, filters and checks
> > +. ./common/rc
> > +
> > +# real QA test starts here
> > +_supported_fs generic
> > +_require_scratch
> > +_require_scratch_shutdown
> > +
> > +rm -f $seqres.full
> > +
> > +_scratch_mkfs > $seqres.full
> > +_scratch_mount
> > +
> > +# Does our userspace even support large dates?
> > +test_bigdates=1
> > +touch -d 'May 30 01:53:03 UTC 2514' $SCRATCH_MNT 2>/dev/null || test_bigdates=0
> > +
> > +# And can we do statx?
> > +test_statx=1
> > +($XFS_IO_PROG -c 'help statx' | grep -q 'Print raw statx' && \
> > + $XFS_IO_PROG -c 'statx -r' $SCRATCH_MNT 2>/dev/null | grep -q 'stat.mtime') || \
> > +	test_statx=0
> > +
> > +echo "Userspace support of large timestamps: $test_bigdates" >> $seqres.full
> > +echo "xfs_io support of statx: $test_statx" >> $seqres.full
> > +
> > +touchme() {
> > +	local arg="$1"
> > +	local name="$2"
> > +
> > +	echo "$arg" > $SCRATCH_MNT/t_$name
> > +	touch -d "$arg" $SCRATCH_MNT/t_$name
> > +}
> > +
> > +report() {
> > +	local files=($SCRATCH_MNT/t_*)
> > +	for file in "${files[@]}"; do
> > +		echo "${file}: $(cat "${file}")"
> > +		TZ=UTC stat -c '%y %Y %n' "${file}"
> > +		test $test_statx -gt 0 && \
> > +			$XFS_IO_PROG -c 'statx -r' "${file}" | grep 'stat.mtime'
> > +	done
> > +}
> > +
> > +# -2147483648 (S32_MIN, or classic unix min)
> > +touchme 'Dec 13 20:45:52 UTC 1901' s32_min
> > +
> > +# 2147483647 (S32_MAX, or classic unix max)
> > +touchme 'Jan 19 03:14:07 UTC 2038' s32_max
> > +
> > +# 7956915742, all twos
> > +touchme 'Feb 22 22:22:22 UTC 2222' all_twos
> > +
> > +if [ $test_bigdates -gt 0 ]; then
> > +	# 16299260424 (u64 nsec counter from s32_min, like xfs does)
> > +	touchme 'Tue Jul  2 20:20:24 UTC 2486' u64ns_from_s32_min
> > +
> > +	# 15032385535 (u34 time if you start from s32_min, like ext4 does)
> > +	touchme 'May 10 22:38:55 UTC 2446' u34_from_s32_min
> > +
> > +	# 17179869183 (u34 time if you start from the unix epoch)
> > +	touchme 'May 30 01:53:03 UTC 2514' u34_max
> > +
> > +	# Latest date we can synthesize(?)
> > +	touchme 'Dec 31 23:59:59 UTC 2147483647' abs_max_time
> > +
> > +	# Earliest date we can synthesize(?)
> > +	touchme 'Jan 1 00:00:00 UTC 0' abs_min_time
> > +fi
> > +
> > +# Query timestamps from incore
> > +echo before >> $seqres.full
> > +report > $tmp.before_crash
> > +cat $tmp.before_crash >> $seqres.full
> > +
> > +_scratch_shutdown -f
> > +_scratch_cycle_mount
> > +
> > +# Query timestamps from disk
> > +echo after >> $seqres.full
> > +report > $tmp.after_crash
> > +cat $tmp.after_crash >> $seqres.full
> > +
> > +# Did they match?
> > +cmp -s $tmp.before_crash $tmp.after_crash
> > +
> > +# success, all done
> > +status=0
> > +exit
> > diff --git a/tests/generic/722.out b/tests/generic/722.out
> > new file mode 100644
> > index 00000000..83acd5cf
> > --- /dev/null
> > +++ b/tests/generic/722.out
> > @@ -0,0 +1 @@
> > +QA output created by 722
> > diff --git a/tests/generic/group b/tests/generic/group
> > index 033465f1..21ac0c8f 100644
> > --- a/tests/generic/group
> > +++ b/tests/generic/group
> > @@ -260,7 +260,7 @@
> >   255 auto quick prealloc punch
> >   256 auto quick punch
> >   257 dir auto quick
> > -258 auto quick
> > +258 auto quick bigtime
> >   259 auto quick clone zero
> >   260 auto quick trim
> >   261 auto quick clone collapse
> > @@ -404,7 +404,7 @@
> >   399 auto encrypt
> >   400 auto quick quota
> >   401 auto quick
> > -402 auto quick rw
> > +402 auto quick rw bigtime
> >   403 auto quick attr
> >   404 auto quick insert
> >   405 auto mkfs thin
> > @@ -636,3 +636,5 @@
> >   631 auto rw overlay rename
> >   632 auto quick mount
> >   633 auto quick atime attr cap idmapped io_uring mount perms rw unlink
> > +721 auto quick atime bigtime
> > +722 auto quick atime bigtime shutdown
> > 

Re: [PATCH 1/4] generic: check userspace handling of extreme timestamps

[Allison Henderson]

On 4/20/21 5:23 PM, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> These two tests ensure we can store and retrieve timestamps on the
> extremes of the date ranges supported by userspace, and the common
> places where overflows can happen.
> 
> They differ from generic/402 in that they don't constrain the dates
> tested to the range that the filesystem claims to support; we attempt
> various things that /userspace/ can parse, and then check that the vfs
> clamps and persists the values correctly.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> Reviewed-by: Amir Goldstein <amir73il@gmail.com>
Looks ok to me.  The tests have quite a bit of overlap, but they're 
pretty small too. I think if we add on any more, probably we should 
think about a common/bigtime file?  But for now, I think it's fine.

Reviewed-by: Allison Henderson <allison.henderson@oracle.com>

> ---
>   tests/generic/721     |  123 ++++++++++++++++++++++++++++++++++++++++++++++++
>   tests/generic/721.out |    2 +
>   tests/generic/722     |  125 +++++++++++++++++++++++++++++++++++++++++++++++++
>   tests/generic/722.out |    1
>   tests/generic/group   |    6 ++
>   5 files changed, 255 insertions(+), 2 deletions(-)
>   create mode 100755 tests/generic/721
>   create mode 100644 tests/generic/721.out
>   create mode 100755 tests/generic/722
>   create mode 100644 tests/generic/722.out
> 
> 
> diff --git a/tests/generic/721 b/tests/generic/721
> new file mode 100755
> index 00000000..9198b6b4
> --- /dev/null
> +++ b/tests/generic/721
> @@ -0,0 +1,123 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2021 Oracle.  All Rights Reserved.
> +#
> +# FS QA Test No. 721
> +#
> +# Make sure we can store and retrieve timestamps on the extremes of the
> +# date ranges supported by userspace, and the common places where overflows
> +# can happen.
> +#
> +# This differs from generic/402 in that we don't constrain ourselves to the
> +# range that the filesystem claims to support; we attempt various things that
> +# /userspace/ can parse, and then check that the vfs clamps and persists the
> +# values correctly.
> +#
> +# NOTE: Old kernels (pre 5.4) allow filesystems to truncate timestamps silently
> +# when writing timestamps to disk!  This test detects this silent truncation
> +# and fails.  If you see a failure on such a kernel, contact your distributor
> +# for an update.
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1    # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +
> +# real QA test starts here
> +_supported_fs generic
> +_require_scratch
> +
> +rm -f $seqres.full
> +
> +_scratch_mkfs > $seqres.full
> +_scratch_mount
> +
> +# Does our userspace even support large dates?
> +test_bigdates=1
> +touch -d 'May 30 01:53:03 UTC 2514' $SCRATCH_MNT 2>/dev/null || test_bigdates=0
> +
> +# And can we do statx?
> +test_statx=1
> +($XFS_IO_PROG -c 'help statx' | grep -q 'Print raw statx' && \
> + $XFS_IO_PROG -c 'statx -r' $SCRATCH_MNT 2>/dev/null | grep -q 'stat.mtime') || \
> +	test_statx=0
> +
> +echo "Userspace support of large timestamps: $test_bigdates" >> $seqres.full
> +echo "xfs_io support of statx: $test_statx" >> $seqres.full
> +
> +touchme() {
> +	local arg="$1"
> +	local name="$2"
> +
> +	echo "$arg" > $SCRATCH_MNT/t_$name
> +	touch -d "$arg" $SCRATCH_MNT/t_$name
> +}
> +
> +report() {
> +	local files=($SCRATCH_MNT/t_*)
> +	for file in "${files[@]}"; do
> +		echo "${file}: $(cat "${file}")"
> +		TZ=UTC stat -c '%y %Y %n' "${file}"
> +		test $test_statx -gt 0 && \
> +			$XFS_IO_PROG -c 'statx -r' "${file}" | grep 'stat.mtime'
> +	done
> +}
> +
> +# -2147483648 (S32_MIN, or classic unix min)
> +touchme 'Dec 13 20:45:52 UTC 1901' s32_min
> +
> +# 2147483647 (S32_MAX, or classic unix max)
> +touchme 'Jan 19 03:14:07 UTC 2038' s32_max
> +
> +# 7956915742, all twos
> +touchme 'Feb 22 22:22:22 UTC 2222' all_twos
> +
> +if [ $test_bigdates -gt 0 ]; then
> +	# 16299260424 (u64 nsec counter from s32_min, like xfs does)
> +	touchme 'Tue Jul  2 20:20:24 UTC 2486' u64ns_from_s32_min
> +
> +	# 15032385535 (u34 time if you start from s32_min, like ext4 does)
> +	touchme 'May 10 22:38:55 UTC 2446' u34_from_s32_min
> +
> +	# 17179869183 (u34 time if you start from the unix epoch)
> +	touchme 'May 30 01:53:03 UTC 2514' u34_max
> +
> +	# Latest date we can synthesize(?)
> +	touchme 'Dec 31 23:59:59 UTC 2147483647' abs_max_time
> +
> +	# Earliest date we can synthesize(?)
> +	touchme 'Jan 1 00:00:00 UTC 0' abs_min_time
> +fi
> +
> +# Query timestamps from incore
> +echo before >> $seqres.full
> +report > $tmp.before_remount
> +cat $tmp.before_remount >> $seqres.full
> +
> +_scratch_cycle_mount
> +
> +# Query timestamps from disk
> +echo after >> $seqres.full
> +report > $tmp.after_remount
> +cat $tmp.after_remount >> $seqres.full
> +
> +# Did they match?
> +cmp -s $tmp.before_remount $tmp.after_remount
> +
> +# success, all done
> +echo Silence is golden.
> +status=0
> +exit
> diff --git a/tests/generic/721.out b/tests/generic/721.out
> new file mode 100644
> index 00000000..b2bc6d58
> --- /dev/null
> +++ b/tests/generic/721.out
> @@ -0,0 +1,2 @@
> +QA output created by 721
> +Silence is golden.
> diff --git a/tests/generic/722 b/tests/generic/722
> new file mode 100755
> index 00000000..305c3bd6
> --- /dev/null
> +++ b/tests/generic/722
> @@ -0,0 +1,125 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2021 Oracle.  All Rights Reserved.
> +#
> +# FS QA Test No. 722
> +#
> +# Make sure we can store and retrieve timestamps on the extremes of the
> +# date ranges supported by userspace, and the common places where overflows
> +# can happen.  This test also ensures that the timestamps are persisted
> +# correctly after a shutdown.
> +#
> +# This differs from generic/402 in that we don't constrain ourselves to the
> +# range that the filesystem claims to support; we attempt various things that
> +# /userspace/ can parse, and then check that the vfs clamps and persists the
> +# values correctly.
> +#
> +# NOTE: Old kernels (pre 5.4) allow filesystems to truncate timestamps silently
> +# when writing timestamps to disk!  This test detects this silent truncation
> +# and fails.  If you see a failure on such a kernel, contact your distributor
> +# for an update.
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1    # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +
> +# real QA test starts here
> +_supported_fs generic
> +_require_scratch
> +_require_scratch_shutdown
> +
> +rm -f $seqres.full
> +
> +_scratch_mkfs > $seqres.full
> +_scratch_mount
> +
> +# Does our userspace even support large dates?
> +test_bigdates=1
> +touch -d 'May 30 01:53:03 UTC 2514' $SCRATCH_MNT 2>/dev/null || test_bigdates=0
> +
> +# And can we do statx?
> +test_statx=1
> +($XFS_IO_PROG -c 'help statx' | grep -q 'Print raw statx' && \
> + $XFS_IO_PROG -c 'statx -r' $SCRATCH_MNT 2>/dev/null | grep -q 'stat.mtime') || \
> +	test_statx=0
> +
> +echo "Userspace support of large timestamps: $test_bigdates" >> $seqres.full
> +echo "xfs_io support of statx: $test_statx" >> $seqres.full
> +
> +touchme() {
> +	local arg="$1"
> +	local name="$2"
> +
> +	echo "$arg" > $SCRATCH_MNT/t_$name
> +	touch -d "$arg" $SCRATCH_MNT/t_$name
> +}
> +
> +report() {
> +	local files=($SCRATCH_MNT/t_*)
> +	for file in "${files[@]}"; do
> +		echo "${file}: $(cat "${file}")"
> +		TZ=UTC stat -c '%y %Y %n' "${file}"
> +		test $test_statx -gt 0 && \
> +			$XFS_IO_PROG -c 'statx -r' "${file}" | grep 'stat.mtime'
> +	done
> +}
> +
> +# -2147483648 (S32_MIN, or classic unix min)
> +touchme 'Dec 13 20:45:52 UTC 1901' s32_min
> +
> +# 2147483647 (S32_MAX, or classic unix max)
> +touchme 'Jan 19 03:14:07 UTC 2038' s32_max
> +
> +# 7956915742, all twos
> +touchme 'Feb 22 22:22:22 UTC 2222' all_twos
> +
> +if [ $test_bigdates -gt 0 ]; then
> +	# 16299260424 (u64 nsec counter from s32_min, like xfs does)
> +	touchme 'Tue Jul  2 20:20:24 UTC 2486' u64ns_from_s32_min
> +
> +	# 15032385535 (u34 time if you start from s32_min, like ext4 does)
> +	touchme 'May 10 22:38:55 UTC 2446' u34_from_s32_min
> +
> +	# 17179869183 (u34 time if you start from the unix epoch)
> +	touchme 'May 30 01:53:03 UTC 2514' u34_max
> +
> +	# Latest date we can synthesize(?)
> +	touchme 'Dec 31 23:59:59 UTC 2147483647' abs_max_time
> +
> +	# Earliest date we can synthesize(?)
> +	touchme 'Jan 1 00:00:00 UTC 0' abs_min_time
> +fi
> +
> +# Query timestamps from incore
> +echo before >> $seqres.full
> +report > $tmp.before_crash
> +cat $tmp.before_crash >> $seqres.full
> +
> +_scratch_shutdown -f
> +_scratch_cycle_mount
> +
> +# Query timestamps from disk
> +echo after >> $seqres.full
> +report > $tmp.after_crash
> +cat $tmp.after_crash >> $seqres.full
> +
> +# Did they match?
> +cmp -s $tmp.before_crash $tmp.after_crash
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/generic/722.out b/tests/generic/722.out
> new file mode 100644
> index 00000000..83acd5cf
> --- /dev/null
> +++ b/tests/generic/722.out
> @@ -0,0 +1 @@
> +QA output created by 722
> diff --git a/tests/generic/group b/tests/generic/group
> index 033465f1..21ac0c8f 100644
> --- a/tests/generic/group
> +++ b/tests/generic/group
> @@ -260,7 +260,7 @@
>   255 auto quick prealloc punch
>   256 auto quick punch
>   257 dir auto quick
> -258 auto quick
> +258 auto quick bigtime
>   259 auto quick clone zero
>   260 auto quick trim
>   261 auto quick clone collapse
> @@ -404,7 +404,7 @@
>   399 auto encrypt
>   400 auto quick quota
>   401 auto quick
> -402 auto quick rw
> +402 auto quick rw bigtime
>   403 auto quick attr
>   404 auto quick insert
>   405 auto mkfs thin
> @@ -636,3 +636,5 @@
>   631 auto rw overlay rename
>   632 auto quick mount
>   633 auto quick atime attr cap idmapped io_uring mount perms rw unlink
> +721 auto quick atime bigtime
> +722 auto quick atime bigtime shutdown
> 

[PATCH 1/4] generic: check userspace handling of extreme timestamps

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

These two tests ensure we can store and retrieve timestamps on the
extremes of the date ranges supported by userspace, and the common
places where overflows can happen.

They differ from generic/402 in that they don't constrain the dates
tested to the range that the filesystem claims to support; we attempt
various things that /userspace/ can parse, and then check that the vfs
clamps and persists the values correctly.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
---
 tests/generic/721     |  123 ++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/721.out |    2 +
 tests/generic/722     |  125 +++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/722.out |    1 
 tests/generic/group   |    6 ++
 5 files changed, 255 insertions(+), 2 deletions(-)
 create mode 100755 tests/generic/721
 create mode 100644 tests/generic/721.out
 create mode 100755 tests/generic/722
 create mode 100644 tests/generic/722.out


diff --git a/tests/generic/721 b/tests/generic/721
new file mode 100755
index 00000000..9198b6b4
--- /dev/null
+++ b/tests/generic/721
@@ -0,0 +1,123 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 721
+#
+# Make sure we can store and retrieve timestamps on the extremes of the
+# date ranges supported by userspace, and the common places where overflows
+# can happen.
+#
+# This differs from generic/402 in that we don't constrain ourselves to the
+# range that the filesystem claims to support; we attempt various things that
+# /userspace/ can parse, and then check that the vfs clamps and persists the
+# values correctly.
+#
+# NOTE: Old kernels (pre 5.4) allow filesystems to truncate timestamps silently
+# when writing timestamps to disk!  This test detects this silent truncation
+# and fails.  If you see a failure on such a kernel, contact your distributor
+# for an update.
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+
+# real QA test starts here
+_supported_fs generic
+_require_scratch
+
+rm -f $seqres.full
+
+_scratch_mkfs > $seqres.full
+_scratch_mount
+
+# Does our userspace even support large dates?
+test_bigdates=1
+touch -d 'May 30 01:53:03 UTC 2514' $SCRATCH_MNT 2>/dev/null || test_bigdates=0
+
+# And can we do statx?
+test_statx=1
+($XFS_IO_PROG -c 'help statx' | grep -q 'Print raw statx' && \
+ $XFS_IO_PROG -c 'statx -r' $SCRATCH_MNT 2>/dev/null | grep -q 'stat.mtime') || \
+	test_statx=0
+
+echo "Userspace support of large timestamps: $test_bigdates" >> $seqres.full
+echo "xfs_io support of statx: $test_statx" >> $seqres.full
+
+touchme() {
+	local arg="$1"
+	local name="$2"
+
+	echo "$arg" > $SCRATCH_MNT/t_$name
+	touch -d "$arg" $SCRATCH_MNT/t_$name
+}
+
+report() {
+	local files=($SCRATCH_MNT/t_*)
+	for file in "${files[@]}"; do
+		echo "${file}: $(cat "${file}")"
+		TZ=UTC stat -c '%y %Y %n' "${file}"
+		test $test_statx -gt 0 && \
+			$XFS_IO_PROG -c 'statx -r' "${file}" | grep 'stat.mtime'
+	done
+}
+
+# -2147483648 (S32_MIN, or classic unix min)
+touchme 'Dec 13 20:45:52 UTC 1901' s32_min
+
+# 2147483647 (S32_MAX, or classic unix max)
+touchme 'Jan 19 03:14:07 UTC 2038' s32_max
+
+# 7956915742, all twos
+touchme 'Feb 22 22:22:22 UTC 2222' all_twos
+
+if [ $test_bigdates -gt 0 ]; then
+	# 16299260424 (u64 nsec counter from s32_min, like xfs does)
+	touchme 'Tue Jul  2 20:20:24 UTC 2486' u64ns_from_s32_min
+
+	# 15032385535 (u34 time if you start from s32_min, like ext4 does)
+	touchme 'May 10 22:38:55 UTC 2446' u34_from_s32_min
+
+	# 17179869183 (u34 time if you start from the unix epoch)
+	touchme 'May 30 01:53:03 UTC 2514' u34_max
+
+	# Latest date we can synthesize(?)
+	touchme 'Dec 31 23:59:59 UTC 2147483647' abs_max_time
+
+	# Earliest date we can synthesize(?)
+	touchme 'Jan 1 00:00:00 UTC 0' abs_min_time
+fi
+
+# Query timestamps from incore
+echo before >> $seqres.full
+report > $tmp.before_remount
+cat $tmp.before_remount >> $seqres.full
+
+_scratch_cycle_mount
+
+# Query timestamps from disk
+echo after >> $seqres.full
+report > $tmp.after_remount
+cat $tmp.after_remount >> $seqres.full
+
+# Did they match?
+cmp -s $tmp.before_remount $tmp.after_remount
+
+# success, all done
+echo Silence is golden.
+status=0
+exit
diff --git a/tests/generic/721.out b/tests/generic/721.out
new file mode 100644
index 00000000..b2bc6d58
--- /dev/null
+++ b/tests/generic/721.out
@@ -0,0 +1,2 @@
+QA output created by 721
+Silence is golden.
diff --git a/tests/generic/722 b/tests/generic/722
new file mode 100755
index 00000000..305c3bd6
--- /dev/null
+++ b/tests/generic/722
@@ -0,0 +1,125 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 722
+#
+# Make sure we can store and retrieve timestamps on the extremes of the
+# date ranges supported by userspace, and the common places where overflows
+# can happen.  This test also ensures that the timestamps are persisted
+# correctly after a shutdown.
+#
+# This differs from generic/402 in that we don't constrain ourselves to the
+# range that the filesystem claims to support; we attempt various things that
+# /userspace/ can parse, and then check that the vfs clamps and persists the
+# values correctly.
+#
+# NOTE: Old kernels (pre 5.4) allow filesystems to truncate timestamps silently
+# when writing timestamps to disk!  This test detects this silent truncation
+# and fails.  If you see a failure on such a kernel, contact your distributor
+# for an update.
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+
+# real QA test starts here
+_supported_fs generic
+_require_scratch
+_require_scratch_shutdown
+
+rm -f $seqres.full
+
+_scratch_mkfs > $seqres.full
+_scratch_mount
+
+# Does our userspace even support large dates?
+test_bigdates=1
+touch -d 'May 30 01:53:03 UTC 2514' $SCRATCH_MNT 2>/dev/null || test_bigdates=0
+
+# And can we do statx?
+test_statx=1
+($XFS_IO_PROG -c 'help statx' | grep -q 'Print raw statx' && \
+ $XFS_IO_PROG -c 'statx -r' $SCRATCH_MNT 2>/dev/null | grep -q 'stat.mtime') || \
+	test_statx=0
+
+echo "Userspace support of large timestamps: $test_bigdates" >> $seqres.full
+echo "xfs_io support of statx: $test_statx" >> $seqres.full
+
+touchme() {
+	local arg="$1"
+	local name="$2"
+
+	echo "$arg" > $SCRATCH_MNT/t_$name
+	touch -d "$arg" $SCRATCH_MNT/t_$name
+}
+
+report() {
+	local files=($SCRATCH_MNT/t_*)
+	for file in "${files[@]}"; do
+		echo "${file}: $(cat "${file}")"
+		TZ=UTC stat -c '%y %Y %n' "${file}"
+		test $test_statx -gt 0 && \
+			$XFS_IO_PROG -c 'statx -r' "${file}" | grep 'stat.mtime'
+	done
+}
+
+# -2147483648 (S32_MIN, or classic unix min)
+touchme 'Dec 13 20:45:52 UTC 1901' s32_min
+
+# 2147483647 (S32_MAX, or classic unix max)
+touchme 'Jan 19 03:14:07 UTC 2038' s32_max
+
+# 7956915742, all twos
+touchme 'Feb 22 22:22:22 UTC 2222' all_twos
+
+if [ $test_bigdates -gt 0 ]; then
+	# 16299260424 (u64 nsec counter from s32_min, like xfs does)
+	touchme 'Tue Jul  2 20:20:24 UTC 2486' u64ns_from_s32_min
+
+	# 15032385535 (u34 time if you start from s32_min, like ext4 does)
+	touchme 'May 10 22:38:55 UTC 2446' u34_from_s32_min
+
+	# 17179869183 (u34 time if you start from the unix epoch)
+	touchme 'May 30 01:53:03 UTC 2514' u34_max
+
+	# Latest date we can synthesize(?)
+	touchme 'Dec 31 23:59:59 UTC 2147483647' abs_max_time
+
+	# Earliest date we can synthesize(?)
+	touchme 'Jan 1 00:00:00 UTC 0' abs_min_time
+fi
+
+# Query timestamps from incore
+echo before >> $seqres.full
+report > $tmp.before_crash
+cat $tmp.before_crash >> $seqres.full
+
+_scratch_shutdown -f
+_scratch_cycle_mount
+
+# Query timestamps from disk
+echo after >> $seqres.full
+report > $tmp.after_crash
+cat $tmp.after_crash >> $seqres.full
+
+# Did they match?
+cmp -s $tmp.before_crash $tmp.after_crash
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/722.out b/tests/generic/722.out
new file mode 100644
index 00000000..83acd5cf
--- /dev/null
+++ b/tests/generic/722.out
@@ -0,0 +1 @@
+QA output created by 722
diff --git a/tests/generic/group b/tests/generic/group
index 033465f1..21ac0c8f 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -260,7 +260,7 @@
 255 auto quick prealloc punch
 256 auto quick punch
 257 dir auto quick
-258 auto quick
+258 auto quick bigtime
 259 auto quick clone zero
 260 auto quick trim
 261 auto quick clone collapse
@@ -404,7 +404,7 @@
 399 auto encrypt
 400 auto quick quota
 401 auto quick
-402 auto quick rw
+402 auto quick rw bigtime
 403 auto quick attr
 404 auto quick insert
 405 auto mkfs thin
@@ -636,3 +636,5 @@
 631 auto rw overlay rename
 632 auto quick mount
 633 auto quick atime attr cap idmapped io_uring mount perms rw unlink
+721 auto quick atime bigtime
+722 auto quick atime bigtime shutdown

[PATCH 1/4] generic: check userspace handling of extreme timestamps

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

These two tests ensure we can store and retrieve timestamps on the
extremes of the date ranges supported by userspace, and the common
places where overflows can happen.

They differ from generic/402 in that they don't constrain the dates
tested to the range that the filesystem claims to support; we attempt
various things that /userspace/ can parse, and then check that the vfs
clamps and persists the values correctly.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
---
 tests/generic/721     |  123 ++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/721.out |    2 +
 tests/generic/722     |  125 +++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/722.out |    1 
 tests/generic/group   |    6 ++
 5 files changed, 255 insertions(+), 2 deletions(-)
 create mode 100755 tests/generic/721
 create mode 100644 tests/generic/721.out
 create mode 100755 tests/generic/722
 create mode 100644 tests/generic/722.out


diff --git a/tests/generic/721 b/tests/generic/721
new file mode 100755
index 00000000..9198b6b4
--- /dev/null
+++ b/tests/generic/721
@@ -0,0 +1,123 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 721
+#
+# Make sure we can store and retrieve timestamps on the extremes of the
+# date ranges supported by userspace, and the common places where overflows
+# can happen.
+#
+# This differs from generic/402 in that we don't constrain ourselves to the
+# range that the filesystem claims to support; we attempt various things that
+# /userspace/ can parse, and then check that the vfs clamps and persists the
+# values correctly.
+#
+# NOTE: Old kernels (pre 5.4) allow filesystems to truncate timestamps silently
+# when writing timestamps to disk!  This test detects this silent truncation
+# and fails.  If you see a failure on such a kernel, contact your distributor
+# for an update.
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+
+# real QA test starts here
+_supported_fs generic
+_require_scratch
+
+rm -f $seqres.full
+
+_scratch_mkfs > $seqres.full
+_scratch_mount
+
+# Does our userspace even support large dates?
+test_bigdates=1
+touch -d 'May 30 01:53:03 UTC 2514' $SCRATCH_MNT 2>/dev/null || test_bigdates=0
+
+# And can we do statx?
+test_statx=1
+($XFS_IO_PROG -c 'help statx' | grep -q 'Print raw statx' && \
+ $XFS_IO_PROG -c 'statx -r' $SCRATCH_MNT 2>/dev/null | grep -q 'stat.mtime') || \
+	test_statx=0
+
+echo "Userspace support of large timestamps: $test_bigdates" >> $seqres.full
+echo "xfs_io support of statx: $test_statx" >> $seqres.full
+
+touchme() {
+	local arg="$1"
+	local name="$2"
+
+	echo "$arg" > $SCRATCH_MNT/t_$name
+	touch -d "$arg" $SCRATCH_MNT/t_$name
+}
+
+report() {
+	local files=($SCRATCH_MNT/t_*)
+	for file in "${files[@]}"; do
+		echo "${file}: $(cat "${file}")"
+		TZ=UTC stat -c '%y %Y %n' "${file}"
+		test $test_statx -gt 0 && \
+			$XFS_IO_PROG -c 'statx -r' "${file}" | grep 'stat.mtime'
+	done
+}
+
+# -2147483648 (S32_MIN, or classic unix min)
+touchme 'Dec 13 20:45:52 UTC 1901' s32_min
+
+# 2147483647 (S32_MAX, or classic unix max)
+touchme 'Jan 19 03:14:07 UTC 2038' s32_max
+
+# 7956915742, all twos
+touchme 'Feb 22 22:22:22 UTC 2222' all_twos
+
+if [ $test_bigdates -gt 0 ]; then
+	# 16299260424 (u64 nsec counter from s32_min, like xfs does)
+	touchme 'Tue Jul  2 20:20:24 UTC 2486' u64ns_from_s32_min
+
+	# 15032385535 (u34 time if you start from s32_min, like ext4 does)
+	touchme 'May 10 22:38:55 UTC 2446' u34_from_s32_min
+
+	# 17179869183 (u34 time if you start from the unix epoch)
+	touchme 'May 30 01:53:03 UTC 2514' u34_max
+
+	# Latest date we can synthesize(?)
+	touchme 'Dec 31 23:59:59 UTC 2147483647' abs_max_time
+
+	# Earliest date we can synthesize(?)
+	touchme 'Jan 1 00:00:00 UTC 0' abs_min_time
+fi
+
+# Query timestamps from incore
+echo before >> $seqres.full
+report > $tmp.before_remount
+cat $tmp.before_remount >> $seqres.full
+
+_scratch_cycle_mount
+
+# Query timestamps from disk
+echo after >> $seqres.full
+report > $tmp.after_remount
+cat $tmp.after_remount >> $seqres.full
+
+# Did they match?
+cmp -s $tmp.before_remount $tmp.after_remount
+
+# success, all done
+echo Silence is golden.
+status=0
+exit
diff --git a/tests/generic/721.out b/tests/generic/721.out
new file mode 100644
index 00000000..b2bc6d58
--- /dev/null
+++ b/tests/generic/721.out
@@ -0,0 +1,2 @@
+QA output created by 721
+Silence is golden.
diff --git a/tests/generic/722 b/tests/generic/722
new file mode 100755
index 00000000..305c3bd6
--- /dev/null
+++ b/tests/generic/722
@@ -0,0 +1,125 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2021 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 722
+#
+# Make sure we can store and retrieve timestamps on the extremes of the
+# date ranges supported by userspace, and the common places where overflows
+# can happen.  This test also ensures that the timestamps are persisted
+# correctly after a shutdown.
+#
+# This differs from generic/402 in that we don't constrain ourselves to the
+# range that the filesystem claims to support; we attempt various things that
+# /userspace/ can parse, and then check that the vfs clamps and persists the
+# values correctly.
+#
+# NOTE: Old kernels (pre 5.4) allow filesystems to truncate timestamps silently
+# when writing timestamps to disk!  This test detects this silent truncation
+# and fails.  If you see a failure on such a kernel, contact your distributor
+# for an update.
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+
+# real QA test starts here
+_supported_fs generic
+_require_scratch
+_require_scratch_shutdown
+
+rm -f $seqres.full
+
+_scratch_mkfs > $seqres.full
+_scratch_mount
+
+# Does our userspace even support large dates?
+test_bigdates=1
+touch -d 'May 30 01:53:03 UTC 2514' $SCRATCH_MNT 2>/dev/null || test_bigdates=0
+
+# And can we do statx?
+test_statx=1
+($XFS_IO_PROG -c 'help statx' | grep -q 'Print raw statx' && \
+ $XFS_IO_PROG -c 'statx -r' $SCRATCH_MNT 2>/dev/null | grep -q 'stat.mtime') || \
+	test_statx=0
+
+echo "Userspace support of large timestamps: $test_bigdates" >> $seqres.full
+echo "xfs_io support of statx: $test_statx" >> $seqres.full
+
+touchme() {
+	local arg="$1"
+	local name="$2"
+
+	echo "$arg" > $SCRATCH_MNT/t_$name
+	touch -d "$arg" $SCRATCH_MNT/t_$name
+}
+
+report() {
+	local files=($SCRATCH_MNT/t_*)
+	for file in "${files[@]}"; do
+		echo "${file}: $(cat "${file}")"
+		TZ=UTC stat -c '%y %Y %n' "${file}"
+		test $test_statx -gt 0 && \
+			$XFS_IO_PROG -c 'statx -r' "${file}" | grep 'stat.mtime'
+	done
+}
+
+# -2147483648 (S32_MIN, or classic unix min)
+touchme 'Dec 13 20:45:52 UTC 1901' s32_min
+
+# 2147483647 (S32_MAX, or classic unix max)
+touchme 'Jan 19 03:14:07 UTC 2038' s32_max
+
+# 7956915742, all twos
+touchme 'Feb 22 22:22:22 UTC 2222' all_twos
+
+if [ $test_bigdates -gt 0 ]; then
+	# 16299260424 (u64 nsec counter from s32_min, like xfs does)
+	touchme 'Tue Jul  2 20:20:24 UTC 2486' u64ns_from_s32_min
+
+	# 15032385535 (u34 time if you start from s32_min, like ext4 does)
+	touchme 'May 10 22:38:55 UTC 2446' u34_from_s32_min
+
+	# 17179869183 (u34 time if you start from the unix epoch)
+	touchme 'May 30 01:53:03 UTC 2514' u34_max
+
+	# Latest date we can synthesize(?)
+	touchme 'Dec 31 23:59:59 UTC 2147483647' abs_max_time
+
+	# Earliest date we can synthesize(?)
+	touchme 'Jan 1 00:00:00 UTC 0' abs_min_time
+fi
+
+# Query timestamps from incore
+echo before >> $seqres.full
+report > $tmp.before_crash
+cat $tmp.before_crash >> $seqres.full
+
+_scratch_shutdown -f
+_scratch_cycle_mount
+
+# Query timestamps from disk
+echo after >> $seqres.full
+report > $tmp.after_crash
+cat $tmp.after_crash >> $seqres.full
+
+# Did they match?
+cmp -s $tmp.before_crash $tmp.after_crash
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/722.out b/tests/generic/722.out
new file mode 100644
index 00000000..83acd5cf
--- /dev/null
+++ b/tests/generic/722.out
@@ -0,0 +1 @@
+QA output created by 722
diff --git a/tests/generic/group b/tests/generic/group
index ab00cc04..f199c7ba 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -260,7 +260,7 @@
 255 auto quick prealloc punch
 256 auto quick punch
 257 dir auto quick
-258 auto quick
+258 auto quick bigtime
 259 auto quick clone zero
 260 auto quick trim
 261 auto quick clone collapse
@@ -404,7 +404,7 @@
 399 auto encrypt
 400 auto quick quota
 401 auto quick
-402 auto quick rw
+402 auto quick rw bigtime
 403 auto quick attr
 404 auto quick insert
 405 auto mkfs thin
@@ -634,3 +634,5 @@
 629 auto quick rw copy_range
 630 auto quick rw dedupe clone
 631 auto rw overlay rename
+721 auto quick atime bigtime
+722 auto quick atime bigtime shutdown