linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: James Bottomley <James.Bottomley@HansenPartnership.com>
To: Sasha Levin <sashal@kernel.org>
Cc: Greg KH <gregkh@linuxfoundation.org>,
	Amir Goldstein <amir73il@gmail.com>,
	Steve French <smfrench@gmail.com>,
	lsf-pc@lists.linux-foundation.org,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	linux-mm <linux-mm@kvack.org>,
	LKML <linux-kernel@vger.kernel.org>,
	"Luis R. Rodriguez" <mcgrof@kernel.org>
Subject: Re: [LSF/MM TOPIC] FS, MM, and stable trees
Date: Thu, 14 Feb 2019 18:48:22 -0800	[thread overview]
Message-ID: <1550198902.2802.12.camel@HansenPartnership.com> (raw)
In-Reply-To: <20190215015020.GJ69686@sasha-vm>

On Thu, 2019-02-14 at 20:50 -0500, Sasha Levin wrote:
> On Wed, Feb 13, 2019 at 12:14:35PM -0800, James Bottomley wrote:
> > On Wed, 2019-02-13 at 20:52 +0100, Greg KH wrote:
> > > On Wed, Feb 13, 2019 at 02:25:12PM -0500, Sasha Levin wrote:
> > > > On Wed, Feb 13, 2019 at 10:18:03AM +0100, Greg KH wrote:
> > > > > On Wed, Feb 13, 2019 at 11:01:25AM +0200, Amir Goldstein
> > > > > wrote:
> > > > > > Best effort testing in timely manner is good, but a good
> > > > > > way to improve confidence in stable kernel releases is a
> > > > > > publicly available list of tests that the release went
> > > > > > through.
> > > > > 
> > > > > We have that, you aren't noticing them...
> > > > 
> > > > This is one of the biggest things I want to address: there is a
> > > > disconnect between the stable kernel testing story and the
> > > > tests the fs/ and mm/ folks expect to see here.
> > > > 
> > > > On one had, the stable kernel folks see these kernels go
> > > > through entire suites of testing by multiple individuals and
> > > > organizations, receiving way more coverage than any of Linus's
> > > > releases.
> > > > 
> > > > On the other hand, things like LTP and selftests tend to barely
> > > > scratch the surface of our mm/ and fs/ code, and the
> > > > maintainers of these subsystems do not see LTP-like suites as
> > > > something that adds significant value and ignore them. Instead,
> > > > they have a (convoluted) set of testing they do with different
> > > > tools and configurations that qualifies their code as being
> > > > "tested".
> > > > 
> > > > So really, it sounds like a low hanging fruit: we don't really
> > > > need to write much more testing code code nor do we have to
> > > > refactor existing test suites. We just need to make sure the
> > > > right tests are running on stable kernels. I really want to
> > > > clarify what each subsystem sees as "sufficient" (and have that
> > > > documented somewhere).
> > > 
> > > kernel.ci and 0-day and Linaro are starting to add the fs and mm
> > > tests to their test suites to address these issues (I think 0-day
> > > already has many of them).  So this is happening, but not quite
> > > obvious.  I know I keep asking Linaro about this :(
> > 
> > 0day has xfstests at least, but it's opt-in only (you have to
> > request that it be run on your trees).  When I did it for the SCSI
> > tree, I had to email Fenguangg directly, there wasn't any other way
> > of getting it.
> 
> It's very tricky to do even if someone would just run it.

It is?  It's a test suite, so you just run it and it exercises standard
and growing set of regression tests.

>  I worked with the xfs folks for quite a while to gather the various
> configs they want to use, and to establish the baseline for a few of
> the stable trees (some tests are know to fail, etc).

The only real config issue is per-fs non-standard tests (features
specific to a given filesystem).  I just want it to exercise the
storage underneath, so the SCSI tree is configured for the default set
on xfs.

> So just running xfstests "blindly" doesn't add much value beyond ltp
> I think.

Well, we differ on the value of running regression tests, then.  The
whole point of a test infrastructure is that it's simple to run 'make
check' in autoconf parlance.  xfstests does provide a useful baseline
set of regression tests.  However, since my goal is primarily to detect
problems in the storage path rather than the filesystem, the utility is
exercising that path, although I fully appreciate that filesystem
regression tests aren't going to catch every SCSI issue, they do
provide some level of assurance against bugs.

Hopefully we can switch over to blktests when it's ready, but in the
meantime xfstests is way better than nothing.

James


  reply	other threads:[~2019-02-15  2:48 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-02-12 17:00 [LSF/MM TOPIC] FS, MM, and stable trees Sasha Levin
2019-02-12 21:32 ` Steve French
2019-02-13  7:20   ` Amir Goldstein
2019-02-13  7:37     ` Greg KH
2019-02-13  9:01       ` Amir Goldstein
2019-02-13  9:18         ` Greg KH
2019-02-13 19:25           ` Sasha Levin
2019-02-13 19:52             ` Greg KH
2019-02-13 20:14               ` James Bottomley
2019-02-15  1:50                 ` Sasha Levin
2019-02-15  2:48                   ` James Bottomley [this message]
2019-02-16 18:28                     ` Theodore Y. Ts'o
2019-02-21 15:34                       ` Luis Chamberlain
2019-02-21 18:52                         ` Theodore Y. Ts'o
2019-03-20  3:46               ` Jon Masters
2019-03-20  5:06                 ` Greg KH
2019-03-20  6:14                   ` Jon Masters
2019-03-20  6:28                     ` Greg KH
2019-03-20  6:32                       ` Jon Masters

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1550198902.2802.12.camel@HansenPartnership.com \
    --to=james.bottomley@hansenpartnership.com \
    --cc=amir73il@gmail.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=lsf-pc@lists.linux-foundation.org \
    --cc=mcgrof@kernel.org \
    --cc=sashal@kernel.org \
    --cc=smfrench@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).