Re: [LSF/MM TOPIC] FS, MM, and stable trees

["Theodore Ts'o" <tytso@mit.edu>]

On Tue, Mar 08, 2022 at 11:08:48AM +0100, Greg KH wrote:
> > When one looks at xfstest bug reports on the list for xfs on kernels > v4.19
> > one has to wonder if using xfs on kernels v5.x.y is a wise choice.
> 
> That's up to the xfs maintainers to discuss.
> 
> > Which makes me wonder: how do the distro kernel maintainers keep up
> > with xfs fixes?
> 
> Who knows, ask the distro maintainers that use xfs.  What do they do?

This is something which is being worked, so I'm not sure we'll need to
discuss the specifics of the xfs stable backports at LSF/MM.  I'm
hopeful that by May, we'll have come to some kind of resolution of
that topic.

One of my team members has been working with Darrick to set up a set
of xfs configs[1] recommended by Darrick, and she's stood up an
automated test spinner using gce-xfstests which can watch a git branch
and automatically kick off a set of tests whenever it is updated.
Sasha has also provided her with a copy of his scripts so we can do
automated cherry picks of commits with Fixes tags.  So the idea is
that we can, hopefully in a mostly completely automated fashion, do
the backports and do a full set of regression tests on those stable
backports of XFS bug fixes.

[1] https://github.com/tytso/xfstests-bld/tree/master/kvm-xfstests/test-appliance/files/root/fs/xfs/cfg

Next steps are to get a first tranche of cherry-picks for 5.10 and
probably 5.15, and use the test spinner to demonstrate that they don't
have any test regressions (if there are, we'll drop those commits).
Once we have a first set of proposed stable backports for XFS, we'll
present them to the XFS development community for their input.  There
are a couple of things that could happen at this point, depending on
what the XFS community is willing to accept.

The first is that we'll send these tested stable patches directly to
Greg and Sasha for inclusion in the LTS releases, with the linux-xfs
list cc'ed so they know what's going into the stable trees.

The second is that we send them only to the linux-xfs list, and they
have to do whatever approval they want before they go into the
upstream stable trees.

And the third option, if they aren't willing to take our work or they
choose to require manual approvals and those approvals are taking too
long, is that we'll feed the commits into Google's Container-Optimized
OS (COS) kernel, so that our customers can get those fixes and so we
can support XFS fully.  This isn't our preferred path; we'd prefer to
take the backports into the COS tree via the stable trees if at all
possible.  (Note: if requested, we could also publish these
backported-and-tested commits on a git tree for other distros to
take.)

There are still some details we'll need to work out; for example, will
the XFS maintainers let us do minor/obvious patch conflict
resolutions, or perhaps those commits which don't cherry-pick cleanly
will need to go through some round of approval by the linux-xfs list,
if the "we've run a full set of tests and there are no test
regressions" isn't good enough for them.

There is also the problem that sometimes commits aren't marked with
Fixes tag, but maybe there are some other signals we could use (for
example, maybe an indication in a comment in an xfstests test that
it's testing regressions for a specified kernel commit id).  Or
perhaps some other would be willing to contribute candidate commit
id's for backport consideration, with the approval of linux-xfs?
TBD...

Note: Darrick has been very helpful in geting this set up; the issue
is not the XFS maintainer, but rather the will of the whole of the XFS
development community, which sometimes can be a bit... fractious.

						- Ted