Re: [PATH 5.10 0/4] xfs stable candidate patches for 5.10.y (part 1)

From: Luis Chamberlain <mcgrof@kernel.org>
To: "Darrick J. Wong" <djwong@kernel.org>, Tso Ted <tytso@mit.edu>
Cc: Amir Goldstein <amir73il@gmail.com>,
	Dave Chinner <david@fromorbit.com>,
	"Darrick J . Wong" <darrick.wong@oracle.com>,
	Theodore Ts'o <tytso@mit.edu>,
	Adam Manzanares <a.manzanares@samsung.com>,
	Tyler Hicks <code@tyhicks.com>,
	linux-xfs@vger.kernel.org, fstests@vger.kernel.org,
	Leah Rumancik <lrumancik@google.com>,
	masahiroy@kernel.org
Subject: Re: [PATH 5.10 0/4] xfs stable candidate patches for 5.10.y (part 1)
Date: Thu, 26 May 2022 11:44:25 -0700	[thread overview]
Message-ID: <Yo/KibX8TOj+rZkV@bombadil.infradead.org> (raw)
In-Reply-To: <Yo+4jW0e4+fYIxX2@magnolia>

On Thu, May 26, 2022 at 10:27:41AM -0700, Darrick J. Wong wrote:
> /me looks and sees a large collection of expunge lists, along with
> comments about how often failures occur and/or reasons.  Neat!
> 
> Leah mentioned on the ext4 call this morning that she would have found
> it helpful to know (before she started working on 5.15 backports) which
> tests were of the flaky variety so that she could better prioritize the
> time she had to look into fstests failures.  (IOWS: saw a test fail a
> small percentage of the time and then burned a lot of machine time only
> to figure out that 5.15.0 also failed a percentage of th time).

See my proposal to try to make this easier to parse:

https://lore.kernel.org/all/YoW0ZC+zM27Pi0Us@bombadil.infradead.org/

> We talked about where it would be most useful for maintainers and QA
> people to store their historical pass/fail data, before settling on
> "somewhere public where everyone can review their colleagues' notes" and
> "somewhere minimizing commit friction".  At the time, we were thinking
> about having people contribute their notes directly to the fstests
> source code, but I guess Luis has been doing that in the kdevops repo
> for a few years now.
> 
> So, maybe there?

For now sure, I'm happy to add others the linux-kdevops org on github
and they get immediate write access to the repo. This is working well
so far. Long term we need to decide if we want to spin off the
expunge list as a separate effort and make it a git subtree (note
it is different than a git sub module). Another example of a use case
for a git subtree, to use it as an example, is the way I forked
kconfig from Linux into a standalone git tree so to allow any project
to bump kconfig code with just one command. So different projects
don't need to fork kconfig as they do today.

The value in doing the git subtree for expunges is any runner can use
it. I did design kdevops though to run on *any* cloud, and support
local virtualization tech like libvirt and virtualbox.

The linux-kdevops git org also has other projects which both fstest
and blktests depend on, so for example dbench which I had forked and
cleaned up a while ago. It may make sense to share keeping oddball
efforts like thse which are no longer maintained in this repo.

There is other tech I'm evaluating for this sort of collaborative test
efforts such as ledgers, but that is in its infancy at this point in
time. I have a sense though it may be a good outlet for collection of
test artifacts in a decentralized way and also allow *any* entity to
participate in bringing confidence to stable kernel branches or dev
branches prior to release.

  Luis