Re: XFS LTS backport cabal

[Leah Rumancik <leah.rumancik@gmail.com>]

On Wed, May 25, 2022 at 02:23:10PM -0700, Darrick J. Wong wrote:
> Hi everyone,
> 
> 3. fstesting -- new patches proposed for stable branches shouldn't
> introduce new regressions, and ideally there would also be a regression
> test that would now pass.  As Dave and I have stated in the past,
> fstests is a big umbrella of a test suite, which implies that A/B
> testing is the way to go.  I think at least Zorro and I would like to
> improve the tagging in fstests to make it more obvious which tests
> contain enough randomness that they cannot be expected to behave 100%
> reliably.
It would be nice to find an agreement on testing requirements. I have
attached some ideas on configs/number of tests/etc as well as the status
of my work on 5.15 below.


> a> I've been following the recent fstests threads, and it seems to me
> that there are really two classes of users -- sustaining people who want
> fstests to run reliably so they can tell if their backports have broken
> anything; and developers, who want the randomness to try to poke into
> dusty corners of the filesystem.  Can we make it easier to associate
> random bits of data (reliability rates, etc.) with a given fstests
> configuration?  And create a test group^Wtag for the tests that rely on
> RNGs to shake things up?
This would be great!

> 
> 
> Thoughts? Flames?
> 
> --D
This thread had good timing :) I have been working on setting up 
some automated testing. Currently, 5.15.y is our priority so I have 
started working on this branch.

Patches are being selected by simply searching for the “Fixes” 
tag and applying if the commit-to-be-fixed is in the stable branch, 
but AUTOSEL would be nice, so I’ll start playing around with that. 
Amir, it would be nice to sync up the patch selection process. I can 
help share the load, especially for 5.15.

Selecting just the tagged “Fixes” for 5.15.y for patches through 
5.17.2, 15 patches were found and applied - if there are no 
complaints about the testing setup, I can go ahead and send out this 
batch:

c30a0cbd07ec xfs: use kmem_cache_free() for kmem_cache objects
5ca5916b6bc9 xfs: punch out data fork delalloc blocks on COW writeback failure
a1de97fe296c xfs: Fix the free logic of state in xfs_attr_node_hasname
1090427bf18f xfs: remove xfs_inew_wait
089558bc7ba7 xfs: remove all COW fork extents when remounting readonly
7993f1a431bc xfs: only run COW extent recovery when there are no live extents
09654ed8a18c xfs: check sb_meta_uuid for dabuf buffer recovery
f8d92a66e810 xfs: prevent UAF in xfs_log_item_in_current_chkpt
b97cca3ba909 xfs: only bother with sync_filesystem during readonly remount
eba0549bc7d1 xfs: don't generate selinux audit messages for capability testing
e014f37db1a2 xfs: use setattr_copy to set vfs inode attributes
70447e0ad978 xfs: async CIL flushes need pending pushes to be made stable
c8c568259772 xfs: don't include bnobt blocks when reserving free block pool
cd6f79d1fb32 xfs: run callbacks before waking waiters in xlog_state_shutdown_callbacks
919edbadebe1 xfs: drop async cache flushes from CIL commits.

Tests are being run through gce-xfstests with the 5.15 kernel config 
from xfstests-bld 
(https://github.com/tytso/xfstests-bld/blob/master/kernel-configs/x86_
64-config-5.15). The configs being tested are the following:

xfs defaults
quota
quota 1k
v4
pmem and fsdax
realtime
8k directory blocks
external log
realtime and external log devices
realtime with 28k extents, external log devices
overlayfs atop xfs
overlayfs atop ext4
ext4 defaults

The test set will be run for each batch of backports, running each 
test 3 times, and if no new failures are seen compared to the same 
branch without the backports, the batch of patches will be deemed 
good. No regressions were seen for the first set of patches listed 
above when applied to 5.15.33. If new failures are seen during 
testing, a bisect can be run to find the offending commits, remove 
these from the batch, and confirm there are no remaining new 
failures. A bug report can be sent to indicate which commits would 
cause new test failures. The test results can be posted publicly 
after each run. The easiest option would be to send the test results 
to a mailing list, such as a google groups mailing list, similar to 
what syzkaller does, or directly to linux-xfs if it isn’t too 
spammy.


- Leah