[PATCH RFC 0/4] xfstests: widen timestamps to deal with y2038

[PATCH RFC 0/4] xfstests: widen timestamps to deal with y2038

[Darrick J. Wong]

Hi all,

This series performs some refactoring of our timestamp and inode
encoding functions, then retrofits the timestamp union to handle
timestamps as a 64-bit nanosecond counter.  Next, it refactors the quota
grace period expiration timer code a bit before implementing bit
shifting to widen the effective counter size to 34 bits.  This enables
correct time handling on XFS through the year 2486.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=bigtime

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=bigtime

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=bigtime
---
 common/rc         |    2 +
 common/xfs        |   46 ++++++++++++++++++++++++++++++++
 tests/xfs/010     |    3 +-
 tests/xfs/030     |    2 +
 tests/xfs/122.out |    2 +
 tests/xfs/908     |   74 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/908.out |    3 ++
 tests/xfs/909     |   77 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/909.out |   12 ++++++++
 tests/xfs/910     |   60 +++++++++++++++++++++++++++++++++++++++++
 tests/xfs/910.out |    3 ++
 tests/xfs/911     |   45 +++++++++++++++++++++++++++++++
 tests/xfs/911.out |   15 ++++++++++
 tests/xfs/group   |    4 +++
 14 files changed, 345 insertions(+), 3 deletions(-)
 create mode 100755 tests/xfs/908
 create mode 100644 tests/xfs/908.out
 create mode 100755 tests/xfs/909
 create mode 100644 tests/xfs/909.out
 create mode 100755 tests/xfs/910
 create mode 100644 tests/xfs/910.out
 create mode 100755 tests/xfs/911
 create mode 100644 tests/xfs/911.out

[PATCH 1/4] xfs/122: embiggen struct xfs_agi size for inobtcount feature

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Make the expected AGI size larger for the inobtcount feature.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 tests/xfs/010     |    3 ++-
 tests/xfs/030     |    2 ++
 tests/xfs/122.out |    2 +-
 3 files changed, 5 insertions(+), 2 deletions(-)


diff --git a/tests/xfs/010 b/tests/xfs/010
index b65dd3ba..a5090316 100755
--- a/tests/xfs/010
+++ b/tests/xfs/010
@@ -114,7 +114,8 @@ _check_scratch_fs
 _corrupt_finobt_root $SCRATCH_DEV
 
 filter_finobt_repair() {
-	sed -e '/^agi has bad CRC/d' | \
+	sed -e '/^agi has bad CRC/d' \
+	    -e '/^bad finobt block/d' | \
 		_filter_repair_lostblocks
 }
 
diff --git a/tests/xfs/030 b/tests/xfs/030
index ebe4c92a..52a43ebc 100755
--- a/tests/xfs/030
+++ b/tests/xfs/030
@@ -44,6 +44,8 @@ _check_ag()
 			    -e '/^bad agbno AGBNO for refcntbt/d' \
 			    -e '/^agf has bad CRC/d' \
 			    -e '/^agi has bad CRC/d' \
+			    -e '/^bad inobt block count/d' \
+			    -e '/^bad finobt block count/d' \
 			    -e '/^Missing reverse-mapping record.*/d' \
 			    -e '/^unknown block state, ag AGNO, block.*/d'
 	done
diff --git a/tests/xfs/122.out b/tests/xfs/122.out
index 45c42e59..837f4cae 100644
--- a/tests/xfs/122.out
+++ b/tests/xfs/122.out
@@ -112,7 +112,7 @@ sizeof(struct xfs_scrub_metadata) = 64
 sizeof(struct xfs_unmount_log_format) = 8
 sizeof(xfs_agf_t) = 224
 sizeof(xfs_agfl_t) = 36
-sizeof(xfs_agi_t) = 336
+sizeof(xfs_agi_t) = 344
 sizeof(xfs_alloc_key_t) = 8
 sizeof(xfs_alloc_rec_incore_t) = 8
 sizeof(xfs_alloc_rec_t) = 8

[PATCH 2/4] xfs: test inobtcount upgrade

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Make sure we can actually upgrade filesystems to support inobtcounts.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 common/xfs        |   16 ++++++++++++++
 tests/xfs/910     |   60 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/910.out |    3 +++
 tests/xfs/group   |    1 +
 4 files changed, 80 insertions(+)
 create mode 100755 tests/xfs/910
 create mode 100644 tests/xfs/910.out


diff --git a/common/xfs b/common/xfs
index ce279041..05ed768a 100644
--- a/common/xfs
+++ b/common/xfs
@@ -955,3 +955,19 @@ _xfs_get_cowgc_interval() {
 		_fail "Can't find cowgc interval procfs knob?"
 	fi
 }
+
+_require_xfs_mkfs_inobtcount()
+{
+	_scratch_mkfs_xfs_supported -m inobtcount=1 >/dev/null 2>&1 \
+	   || _notrun "mkfs.xfs doesn't have inobtcount feature"
+}
+
+_require_xfs_scratch_inobtcount()
+{
+	_require_scratch
+
+	_scratch_mkfs -m inobtcount=1 > /dev/null
+	_try_scratch_mount || \
+		_notrun "inobtcount not supported by scratch filesystem type: $FSTYP"
+	_scratch_unmount
+}
diff --git a/tests/xfs/910 b/tests/xfs/910
new file mode 100755
index 00000000..3cc5fca2
--- /dev/null
+++ b/tests/xfs/910
@@ -0,0 +1,60 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2020, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# FS QA Test No. 910
+#
+# Check that we can upgrade a filesystem to support inobtcount and that
+# everything works properly after the upgrade.
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs xfs
+_supported_os Linux
+_require_command "$XFS_ADMIN_PROG" "xfs_admin"
+_require_xfs_mkfs_inobtcount
+_require_xfs_scratch_inobtcount
+
+rm -f $seqres.full
+
+# Format V5 filesystem without inode btree counter support and populate it
+_scratch_mkfs -m crc=1,inobtcount=0 > $seqres.full
+_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
+_scratch_mount >> $seqres.full
+
+echo moo > $SCRATCH_MNT/urk
+
+_scratch_unmount
+_check_scratch_fs
+
+# Now upgrade to inobtcount support
+_scratch_xfs_admin -O inobtcount >> $seqres.full
+_check_scratch_fs
+_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' -c 'agi 0' -c 'p' >> $seqres.full
+
+# Mount again, look at our files
+_scratch_mount >> $seqres.full
+cat $SCRATCH_MNT/urk
+
+# success, all done
+echo Silence is golden.
+status=0
+exit
diff --git a/tests/xfs/910.out b/tests/xfs/910.out
new file mode 100644
index 00000000..83992f49
--- /dev/null
+++ b/tests/xfs/910.out
@@ -0,0 +1,3 @@
+QA output created by 910
+moo
+Silence is golden.
diff --git a/tests/xfs/group b/tests/xfs/group
index 8ecdc5ba..b144d391 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -531,4 +531,5 @@
 746 auto quick online_repair
 747 auto quick scrub
 748 auto quick scrub
+910 auto quick inobtcount
 915 auto quick quota

[PATCH 3/4] xfs: detect time limits from filesystem

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Teach fstests to extract timestamp limits of a filesystem using the new
xfs_db timelimit command.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 common/rc         |    2 +-
 common/xfs        |   14 ++++++++++++++
 tests/xfs/911     |   45 +++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/911.out |   15 +++++++++++++++
 tests/xfs/group   |    1 +
 5 files changed, 76 insertions(+), 1 deletion(-)
 create mode 100755 tests/xfs/911
 create mode 100644 tests/xfs/911.out


diff --git a/common/rc b/common/rc
index aa5a7409..4f76f18d 100644
--- a/common/rc
+++ b/common/rc
@@ -2058,7 +2058,7 @@ _filesystem_timestamp_range()
 		echo "0 $u32max"
 		;;
 	xfs)
-		echo "$s32min $s32max"
+		_xfs_timestamp_range "$device"
 		;;
 	btrfs)
 		echo "$s64min $s64max"
diff --git a/common/xfs b/common/xfs
index 05ed768a..252a5c0d 100644
--- a/common/xfs
+++ b/common/xfs
@@ -971,3 +971,17 @@ _require_xfs_scratch_inobtcount()
 		_notrun "inobtcount not supported by scratch filesystem type: $FSTYP"
 	_scratch_unmount
 }
+
+_xfs_timestamp_range()
+{
+	local use_db=0
+	local dbprog="$XFS_DB_PROG $device"
+	test "$device" = "$SCRATCH_DEV" && dbprog=_scratch_xfs_db
+
+	$dbprog -f -c 'help timelimit' | grep -v -q 'not found' && use_db=1
+	if [ $use_db -eq 0 ]; then
+		echo "-$((1<<31)) $(((1<<31)-1))"
+	else
+		$dbprog -f -c 'timelimit --compact' | awk '{printf("%s %s", $1, $2);}'
+	fi
+}
diff --git a/tests/xfs/911 b/tests/xfs/911
new file mode 100755
index 00000000..2e161a74
--- /dev/null
+++ b/tests/xfs/911
@@ -0,0 +1,45 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2020, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# FS QA Test No. 911
+#
+# Check that the xfs_db timelimit command prints the ranges that we expect.
+# This in combination with an xfs_ondisk.h build time check in the kernel
+# ensures that the kernel agrees with userspace.
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+
+# real QA test starts here
+_supported_fs xfs
+_supported_os Linux
+_require_scratch
+_require_xfs_db_command timelimit
+
+rm -f $seqres.full
+
+# Format filesystem without bigtime support and populate it
+_scratch_mkfs > $seqres.full
+echo classic xfs timelimits
+_scratch_xfs_db -c 'timelimit --classic'
+echo bigtime xfs timelimits
+_scratch_xfs_db -c 'timelimit --bigtime'
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/911.out b/tests/xfs/911.out
new file mode 100644
index 00000000..b0408bac
--- /dev/null
+++ b/tests/xfs/911.out
@@ -0,0 +1,15 @@
+QA output created by 911
+classic xfs timelimits
+time.min = -2147483648
+time.max = 2147483647
+dqtimer.min = 1
+dqtimer.max = 4294967295
+dqgrace.min = 0
+dqgrace.min = 4294967295
+bigtime xfs timelimits
+time.min = -2147483648
+time.max = 16299260425
+dqtimer.min = 1
+dqtimer.max = 16299260425
+dqgrace.min = 0
+dqgrace.min = 4294967295
diff --git a/tests/xfs/group b/tests/xfs/group
index b144d391..0063fd19 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -532,4 +532,5 @@
 747 auto quick scrub
 748 auto quick scrub
 910 auto quick inobtcount
+911 auto quick bigtime
 915 auto quick quota

[PATCH 4/4] xfs: test upgrading filesystem to bigtime

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Test that we can upgrade an existing filesystem to use bigtime.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 common/xfs        |   16 +++++++++++
 tests/xfs/908     |   74 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/908.out |    3 ++
 tests/xfs/909     |   77 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/909.out |   12 ++++++++
 tests/xfs/group   |    2 +
 6 files changed, 184 insertions(+)
 create mode 100755 tests/xfs/908
 create mode 100644 tests/xfs/908.out
 create mode 100755 tests/xfs/909
 create mode 100644 tests/xfs/909.out


diff --git a/common/xfs b/common/xfs
index 252a5c0d..c0735a51 100644
--- a/common/xfs
+++ b/common/xfs
@@ -985,3 +985,19 @@ _xfs_timestamp_range()
 		$dbprog -f -c 'timelimit --compact' | awk '{printf("%s %s", $1, $2);}'
 	fi
 }
+
+_require_xfs_mkfs_bigtime()
+{
+	_scratch_mkfs_xfs_supported -m bigtime=1 >/dev/null 2>&1 \
+	   || _notrun "mkfs.xfs doesn't have bigtime feature"
+}
+
+_require_xfs_scratch_bigtime()
+{
+	_require_scratch
+
+	_scratch_mkfs -m bigtime=1 > /dev/null
+	_try_scratch_mount || \
+		_notrun "bigtime not supported by scratch filesystem type: $FSTYP"
+	_scratch_unmount
+}
diff --git a/tests/xfs/908 b/tests/xfs/908
new file mode 100755
index 00000000..e313e14b
--- /dev/null
+++ b/tests/xfs/908
@@ -0,0 +1,74 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2020, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# FS QA Test No. 908
+#
+# Check that we can upgrade a filesystem to support bigtime and that inode
+# timestamps work properly after the upgrade.
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs xfs
+_supported_os Linux
+_require_xfs_mkfs_crc
+_require_xfs_mkfs_bigtime
+_require_xfs_scratch_bigtime
+
+date --date='Jan 1 00:00:00 UTC 2040' > /dev/null 2>&1 || \
+	_notrun "Userspace does not support dates past 2038."
+
+rm -f $seqres.full
+
+# Format V5 filesystem without bigtime support and populate it
+_scratch_mkfs -m crc=1,bigtime=0 > $seqres.full
+_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
+_scratch_mount >> $seqres.full
+
+touch $SCRATCH_MNT/a
+touch $SCRATCH_MNT/b
+ls -la $SCRATCH_MNT/* >> $seqres.full
+
+_scratch_unmount
+_check_scratch_fs
+
+# Now upgrade to bigtime support
+_scratch_xfs_admin -O bigtime >> $seqres.full
+_check_scratch_fs
+_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
+
+# Mount again, look at our files
+_scratch_mount >> $seqres.full
+ls -la $SCRATCH_MNT/* >> $seqres.full
+
+# Modify some timestamps
+touch -d 'Feb 22 22:22:22 UTC 2222' $SCRATCH_MNT/b
+
+_scratch_cycle_mount
+
+# Did the timestamp survive?
+ls -la $SCRATCH_MNT/* >> $seqres.full
+TZ=UTC stat -c '%Y' $SCRATCH_MNT/b
+
+# success, all done
+echo Silence is golden.
+status=0
+exit
diff --git a/tests/xfs/908.out b/tests/xfs/908.out
new file mode 100644
index 00000000..38fdf6b3
--- /dev/null
+++ b/tests/xfs/908.out
@@ -0,0 +1,3 @@
+QA output created by 908
+7956915742
+Silence is golden.
diff --git a/tests/xfs/909 b/tests/xfs/909
new file mode 100755
index 00000000..8d8675da
--- /dev/null
+++ b/tests/xfs/909
@@ -0,0 +1,77 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2020, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# FS QA Test No. 909
+#
+# Check that we can upgrade a filesystem to support bigtime and that quota
+# timers work properly after the upgrade.
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/quota
+
+# real QA test starts here
+_supported_fs xfs
+_supported_os Linux
+_require_quota
+_require_xfs_mkfs_crc
+_require_xfs_mkfs_bigtime
+_require_xfs_scratch_bigtime
+
+date --date='Jan 1 00:00:00 UTC 2040' > /dev/null 2>&1 || \
+	_notrun "Userspace does not support dates past 2038."
+
+rm -f $seqres.full
+
+# Format V5 filesystem without bigtime support and populate it
+_scratch_mkfs -m crc=1,bigtime=0 > $seqres.full
+_qmount_option "usrquota"
+_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
+_scratch_mount >> $seqres.full
+
+touch $SCRATCH_MNT/a
+touch $SCRATCH_MNT/b
+$XFS_QUOTA_PROG -x -c 'timer -u 300m' $SCRATCH_MNT
+$XFS_QUOTA_PROG -x -c 'state' $SCRATCH_MNT | grep 'grace time'
+
+_scratch_unmount
+
+# Now upgrade to bigtime support
+_scratch_xfs_admin -O bigtime >> $seqres.full
+_check_scratch_fs
+_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
+
+# Mount again, see if our quota timer survived
+_scratch_mount
+$XFS_QUOTA_PROG -x -c 'state' $SCRATCH_MNT | grep 'grace time'
+
+# Create a file to force the dirty dquot out to disk
+touch -d 'Feb 22 22:22:22 UTC 2222' $SCRATCH_MNT/b
+
+_scratch_cycle_mount
+
+# Did the timer (and the timestamp) survive?
+TZ=UTC stat -c '%Y' $SCRATCH_MNT/b
+$XFS_QUOTA_PROG -x -c 'state' $SCRATCH_MNT | grep 'grace time'
+
+# success, all done
+echo Silence is golden.
+status=0
+exit
diff --git a/tests/xfs/909.out b/tests/xfs/909.out
new file mode 100644
index 00000000..70e1b082
--- /dev/null
+++ b/tests/xfs/909.out
@@ -0,0 +1,12 @@
+QA output created by 909
+Blocks grace time: [0 days 05:00:00]
+Inodes grace time: [0 days 05:00:00]
+Realtime Blocks grace time: [0 days 05:00:00]
+Blocks grace time: [0 days 05:00:00]
+Inodes grace time: [0 days 05:00:00]
+Realtime Blocks grace time: [0 days 05:00:00]
+7956915742
+Blocks grace time: [0 days 05:00:00]
+Inodes grace time: [0 days 05:00:00]
+Realtime Blocks grace time: [0 days 05:00:00]
+Silence is golden.
diff --git a/tests/xfs/group b/tests/xfs/group
index 0063fd19..93fdaba9 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -531,6 +531,8 @@
 746 auto quick online_repair
 747 auto quick scrub
 748 auto quick scrub
+908 auto quick bigtime
+909 auto quick bigtime quota
 910 auto quick inobtcount
 911 auto quick bigtime
 915 auto quick quota

Re: [PATCH 4/4] xfs: test upgrading filesystem to bigtime

[Amir Goldstein]

On Tue, Aug 18, 2020 at 2:23 AM Darrick J. Wong <darrick.wong@oracle.com> wrote:
>
> From: Darrick J. Wong <darrick.wong@oracle.com>
>
> Test that we can upgrade an existing filesystem to use bigtime.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  common/xfs        |   16 +++++++++++
>  tests/xfs/908     |   74 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/908.out |    3 ++
>  tests/xfs/909     |   77 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/909.out |   12 ++++++++
>  tests/xfs/group   |    2 +
>  6 files changed, 184 insertions(+)
>  create mode 100755 tests/xfs/908
>  create mode 100644 tests/xfs/908.out
>  create mode 100755 tests/xfs/909
>  create mode 100644 tests/xfs/909.out
>
>
> diff --git a/common/xfs b/common/xfs
> index 252a5c0d..c0735a51 100644
> --- a/common/xfs
> +++ b/common/xfs
> @@ -985,3 +985,19 @@ _xfs_timestamp_range()
>                 $dbprog -f -c 'timelimit --compact' | awk '{printf("%s %s", $1, $2);}'
>         fi
>  }
> +
> +_require_xfs_mkfs_bigtime()
> +{
> +       _scratch_mkfs_xfs_supported -m bigtime=1 >/dev/null 2>&1 \
> +          || _notrun "mkfs.xfs doesn't have bigtime feature"
> +}
> +
> +_require_xfs_scratch_bigtime()
> +{
> +       _require_scratch
> +
> +       _scratch_mkfs -m bigtime=1 > /dev/null
> +       _try_scratch_mount || \
> +               _notrun "bigtime not supported by scratch filesystem type: $FSTYP"
> +       _scratch_unmount
> +}
> diff --git a/tests/xfs/908 b/tests/xfs/908
> new file mode 100755
> index 00000000..e313e14b
> --- /dev/null
> +++ b/tests/xfs/908
> @@ -0,0 +1,74 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2020, Oracle and/or its affiliates.  All Rights Reserved.
> +#
> +# FS QA Test No. 908
> +#
> +# Check that we can upgrade a filesystem to support bigtime and that inode
> +# timestamps work properly after the upgrade.
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1    # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +       cd /
> +       rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# real QA test starts here
> +_supported_fs xfs
> +_supported_os Linux
> +_require_xfs_mkfs_crc
> +_require_xfs_mkfs_bigtime
> +_require_xfs_scratch_bigtime
> +
> +date --date='Jan 1 00:00:00 UTC 2040' > /dev/null 2>&1 || \
> +       _notrun "Userspace does not support dates past 2038."
> +
> +rm -f $seqres.full
> +
> +# Format V5 filesystem without bigtime support and populate it
> +_scratch_mkfs -m crc=1,bigtime=0 > $seqres.full
> +_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
> +_scratch_mount >> $seqres.full
> +
> +touch $SCRATCH_MNT/a
> +touch $SCRATCH_MNT/b
> +ls -la $SCRATCH_MNT/* >> $seqres.full
> +
> +_scratch_unmount
> +_check_scratch_fs
> +
> +# Now upgrade to bigtime support
> +_scratch_xfs_admin -O bigtime >> $seqres.full
> +_check_scratch_fs
> +_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
> +
> +# Mount again, look at our files
> +_scratch_mount >> $seqres.full
> +ls -la $SCRATCH_MNT/* >> $seqres.full
> +
> +# Modify some timestamps
> +touch -d 'Feb 22 22:22:22 UTC 2222' $SCRATCH_MNT/b
> +
> +_scratch_cycle_mount
> +
> +# Did the timestamp survive?
> +ls -la $SCRATCH_MNT/* >> $seqres.full
> +TZ=UTC stat -c '%Y' $SCRATCH_MNT/b
> +

Darrick,

These tests look great, but I wonder.
generic/402 has more test coverage than above.
It tests several data points and it tests them with and without mount cycle.

With your current tests, bigtime will enjoy this test coverage only if
the entire
run is configured with custom XFS_MKFS_OPTIONS or when bigtime
becomes default for mkfs.

Do you think we should have a temporary clone of generic/402 for xfs which
enables bigtime for the time being?

Thanks,
Amir.

Re: [PATCH 4/4] xfs: test upgrading filesystem to bigtime

[Darrick J. Wong]

On Tue, Aug 18, 2020 at 09:16:21AM +0300, Amir Goldstein wrote:
> On Tue, Aug 18, 2020 at 2:23 AM Darrick J. Wong <darrick.wong@oracle.com> wrote:
> >
> > From: Darrick J. Wong <darrick.wong@oracle.com>
> >
> > Test that we can upgrade an existing filesystem to use bigtime.
> >
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > ---
> >  common/xfs        |   16 +++++++++++
> >  tests/xfs/908     |   74 +++++++++++++++++++++++++++++++++++++++++++++++++++
> >  tests/xfs/908.out |    3 ++
> >  tests/xfs/909     |   77 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> >  tests/xfs/909.out |   12 ++++++++
> >  tests/xfs/group   |    2 +
> >  6 files changed, 184 insertions(+)
> >  create mode 100755 tests/xfs/908
> >  create mode 100644 tests/xfs/908.out
> >  create mode 100755 tests/xfs/909
> >  create mode 100644 tests/xfs/909.out
> >
> >
> > diff --git a/common/xfs b/common/xfs
> > index 252a5c0d..c0735a51 100644
> > --- a/common/xfs
> > +++ b/common/xfs
> > @@ -985,3 +985,19 @@ _xfs_timestamp_range()
> >                 $dbprog -f -c 'timelimit --compact' | awk '{printf("%s %s", $1, $2);}'
> >         fi
> >  }
> > +
> > +_require_xfs_mkfs_bigtime()
> > +{
> > +       _scratch_mkfs_xfs_supported -m bigtime=1 >/dev/null 2>&1 \
> > +          || _notrun "mkfs.xfs doesn't have bigtime feature"
> > +}
> > +
> > +_require_xfs_scratch_bigtime()
> > +{
> > +       _require_scratch
> > +
> > +       _scratch_mkfs -m bigtime=1 > /dev/null
> > +       _try_scratch_mount || \
> > +               _notrun "bigtime not supported by scratch filesystem type: $FSTYP"
> > +       _scratch_unmount
> > +}
> > diff --git a/tests/xfs/908 b/tests/xfs/908
> > new file mode 100755
> > index 00000000..e313e14b
> > --- /dev/null
> > +++ b/tests/xfs/908
> > @@ -0,0 +1,74 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0-or-later
> > +# Copyright (c) 2020, Oracle and/or its affiliates.  All Rights Reserved.
> > +#
> > +# FS QA Test No. 908
> > +#
> > +# Check that we can upgrade a filesystem to support bigtime and that inode
> > +# timestamps work properly after the upgrade.
> > +
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1    # failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > +       cd /
> > +       rm -f $tmp.*
> > +}
> > +
> > +# get standard environment, filters and checks
> > +. ./common/rc
> > +. ./common/filter
> > +
> > +# real QA test starts here
> > +_supported_fs xfs
> > +_supported_os Linux
> > +_require_xfs_mkfs_crc
> > +_require_xfs_mkfs_bigtime
> > +_require_xfs_scratch_bigtime
> > +
> > +date --date='Jan 1 00:00:00 UTC 2040' > /dev/null 2>&1 || \
> > +       _notrun "Userspace does not support dates past 2038."
> > +
> > +rm -f $seqres.full
> > +
> > +# Format V5 filesystem without bigtime support and populate it
> > +_scratch_mkfs -m crc=1,bigtime=0 > $seqres.full
> > +_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
> > +_scratch_mount >> $seqres.full
> > +
> > +touch $SCRATCH_MNT/a
> > +touch $SCRATCH_MNT/b
> > +ls -la $SCRATCH_MNT/* >> $seqres.full
> > +
> > +_scratch_unmount
> > +_check_scratch_fs
> > +
> > +# Now upgrade to bigtime support
> > +_scratch_xfs_admin -O bigtime >> $seqres.full
> > +_check_scratch_fs
> > +_scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
> > +
> > +# Mount again, look at our files
> > +_scratch_mount >> $seqres.full
> > +ls -la $SCRATCH_MNT/* >> $seqres.full
> > +
> > +# Modify some timestamps
> > +touch -d 'Feb 22 22:22:22 UTC 2222' $SCRATCH_MNT/b
> > +
> > +_scratch_cycle_mount
> > +
> > +# Did the timestamp survive?
> > +ls -la $SCRATCH_MNT/* >> $seqres.full
> > +TZ=UTC stat -c '%Y' $SCRATCH_MNT/b
> > +
> 
> Darrick,
> 
> These tests look great, but I wonder.
> generic/402 has more test coverage than above.
> It tests several data points and it tests them with and without mount cycle.

Um... these two tests exist to make sure that /upgrading/ works, whereas
generic/402 tests whatever it finds on the formatted scratch filesystem.

> With your current tests, bigtime will enjoy this test coverage only if
> the entire
> run is configured with custom XFS_MKFS_OPTIONS or when bigtime
> becomes default for mkfs.

I don't understand the line of reasoning.  Both tests format with
specific mkfs options, or skip the test entirely if mkfs doesn't know
what bigtime is.

> Do you think we should have a temporary clone of generic/402 for xfs which
> enables bigtime for the time being?

<shrug> I pushed most of my testing to the cloud, so I just spawn enough
VMs so that one of them will test bigtime=0 and another does
bigtime=1...

--D

> Thanks,
> Amir.

Re: [PATCH 4/4] xfs: test upgrading filesystem to bigtime

[Amir Goldstein]

> > Darrick,
> >
> > These tests look great, but I wonder.
> > generic/402 has more test coverage than above.
> > It tests several data points and it tests them with and without mount cycle.
>
> Um... these two tests exist to make sure that /upgrading/ works, whereas
> generic/402 tests whatever it finds on the formatted scratch filesystem.
>
> > With your current tests, bigtime will enjoy this test coverage only if
> > the entire
> > run is configured with custom XFS_MKFS_OPTIONS or when bigtime
> > becomes default for mkfs.
>
> I don't understand the line of reasoning.  Both tests format with
> specific mkfs options, or skip the test entirely if mkfs doesn't know
> what bigtime is.
>

I was referring to generic/402.

> > Do you think we should have a temporary clone of generic/402 for xfs which
> > enables bigtime for the time being?
>
> <shrug> I pushed most of my testing to the cloud, so I just spawn enough
> VMs so that one of them will test bigtime=0 and another does
> bigtime=1...
>

Fine by me. As long as there is test coverage for generic/402 with bigtime.

Thanks,
Amir.