Re: Regression testing of 2.4.x before release?

From: Dan Kegel <dank@kegel.com>
To: Luigi Genoni <kernel@Expansa.sns.it>
Cc: Mike Galbraith <mikeg@wen-online.de>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	stp@osdlab.org
Subject: Re: Regression testing of 2.4.x before release?
Date: Sun, 04 Nov 2001 17:51:49 -0800	[thread overview]
Message-ID: <3BE5F0B5.52274D07@kegel.com> (raw)
In-Reply-To: <Pine.LNX.4.33.0111041955290.30596-100000@Expansa.sns.it>

Luigi Genoni wrote:
> Problem is:
> there is a lot of HW out there, and we should ALL do stress tests, to have
> a wide basis for HWs and test cases.  Basically it is very hard to agree
> about a set of stress tests, because we all have different needs, and our
> tests are based on our needs. That is a streght, because they tend to be
> real life tests.

Sure, no argument there.

> In my esperience, if some default set of tests comes out, then software
> tend to be optimized for this set. And that is badly wrong.

My post was motivated by two observations:

1. Alan Cox complains occasionally that Linus' trees are not well tested,
   and can't survive the torture tests that the ac tree goes through before
   release.  (e.g.
"2.4.8-ac12
        I'm trying to make sure I can keep this testable
        as 2.4.9 vanilla isnt being stable on my test sets "

2. The STP at OSDLab seems like a great resource that we might be able
to leverage to solve the problem Alan points out.

I'm not suggesting anyone do any less testing.  Just the opposite;
if we set things up properly with the STP, we might be able to run
many more tests before each final release.

- Dan