Re: ✅ PASS: Stable queue: queue-5.0

From: Greg KH <gregkh@linuxfoundation.org>
To: Don Zickus <dzickus@redhat.com>
Cc: Veronika Kabatova <vkabatov@redhat.com>,
	CKI Project <cki-project@redhat.com>,
	Linux Stable maillist <stable@vger.kernel.org>
Subject: Re: ✅ PASS: Stable queue: queue-5.0
Date: Wed, 8 May 2019 18:52:02 +0200	[thread overview]
Message-ID: <20190508165202.GB6157@kroah.com> (raw)
In-Reply-To: <20190508162242.tbhuuwd6wrm66ppb@redhat.com>

On Wed, May 08, 2019 at 12:22:42PM -0400, Don Zickus wrote:
> On Wed, May 08, 2019 at 11:24:39AM +0200, Greg KH wrote:
> > > Hi,
> > > 
> > > in some cases we are running multiple recipes in a single test job, to
> > > get out the results faster. Each recipe is started by a "boot test" since
> > > that's responsible for installing and booting the kernel being tested. The
> > > report joins all recipes for given architecture, hence that one test is
> > > shown there multiple times. I agree that we should make this more clear
> > > and separate the report parts per recipes but we didn't have time for it
> > > yet, sorry. I notified people about the problem and we'll prioritize :)
> > > 
> > > > And I see you are running xfstests, which is great, but does it really
> > > > all "pass"?  What type of filesystem image are you running it on.
> > > > 
> > > 
> > > Here you can find the list of subtests that's being run [0] and a list of
> > > excluded ones from them [1]. This is just a reduced test set as some of the
> > > tests were triggering fake failures or taking too long to run as a part of
> > > CI. The lists may change in the future of course.
> > > 
> > > We set up two separate xfs partitions for the testing. The machine should
> > > have at least 50G of space available for this.
> > > 
> > > 
> > > Hope this explains everything and sorry for the recipe confusion. Let us
> > > know if you have anything else!
> > 
> > Thanks a lot for the information.  It's good to see that someone is
> > finally running xfstests on the stable trees, that's much appreciated.
> 
> Hi Greg,
> 
> Thanks for the feedback.  If you have some other suggested tests, we might
> be able to add them to our test harness.  We do have some bandwidth for
> extra testing.  Thanks!

Oh, hey, yes, I do have a list!

It's the list that 0-day does today:
	https://git.kernel.org/pub/scm/linux/kernel/git/wfg/lkp-tests.git/tree/tests
pick some up from there and run with it.

Or look at what Linaro did, they are running a bunch of tests.

Or better yet, work with kernel.ci to integrate runtime tests there so
that everyone can benifit!

Having these "closed" testing silos is not good overall.  I'm happy to
see the kernels get tested, but the duplication of effort is quite sad.

thanks,

greg k-h