Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] & [TECH TOPIC] Improve regression tracking

[James Bottomley <James.Bottomley-d9PhHud1JfjCXq6kfMZ53/egYHeGw8Jk@public.gmane.org>]

[-- Attachment #1: Type: text/plain, Size: 2140 bytes --]

On Thu, 2017-07-06 at 10:28 +0100, Mark Brown wrote:
> On Wed, Jul 05, 2017 at 01:02:00PM -0400, Steven Rostedt wrote:
> > 
> > Guenter Roeck <linux-0h96xk9xTtrk1uMJSBkQmQ@public.gmane.org> wrote:
> 
> > 
> > > 
> > > If a test to reproduce a problem exists, it might be more
> > > beneficial to suggest to the patch submitter that it would be
> > > great if that test would be submitted as unit test instead of
> > > shaming that person for not doing so. Acknowledging and
> > > praising kselftest submissions might help more than shaming for
> > > non-submissions.
> 
> > 
> > > 
> > > My concern would be that once the shaming starts, it won't stop.
> 
> > 
> > I think this is a communication issue. My word for "shaming" was to
> > call out a developer for not submitting a test. It wasn't about
> > making fun of them, or anything like that. I was only making a
> > point about how to teach people that they need to be more aware of
> > the testing infrastructure. Not about actually demeaning people.
> 
> I think before anything like that is viable we need to show a
> concerted and visible interest in actually running the tests we
> already have and paying attention to the results - if people can see
> that they're just checking a checkbox that will often result in low
> quality tests which can do more harm than good.

it depends what you mean by "we".  I used to run a battery of tests
over every SCSI commit.  It was time consuming and slowed down the
process, plus it was me who always got to diagnose failures.  Nowadays
I don't bother: I rely on 0day to run its usual tests plus a couple of
extras I asked for it's a much more streamlined process (meaning less
work for me) and everyone is happy.

The corollary I take away from this is that the less intrusive the test
infrastructure is (at least to my process) the happier I am.  The 0day
quantum leap for me was going from testing my tree and telling me of
problems after I've added the patch to testing patches posted to the
mailing list, which tells me of problems *before* the commit gets added
to the tree.

James

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]