From mboxrd@z Thu Jan 1 00:00:00 1970 From: Daniel Vetter Subject: Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] & [TECH TOPIC] Improve regression tracking Date: Thu, 6 Jul 2017 11:41:39 +0200 Message-ID: References: <20170703123025.7479702e@gandalf.local.home> <20170705084528.67499f8c@gandalf.local.home> <4080ecc7-1aa8-2940-f230-1b79d656cdb4@redhat.com> <20170705092757.63dc2328@gandalf.local.home> <20170705140607.GA30187@kroah.com> <20170705112707.54d7f345@gandalf.local.home> <20170705130200.7c653f61@gandalf.local.home> <20170706092836.ifcnc2qqwufndhdl@sirena.org.uk> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Return-path: In-Reply-To: <20170706092836.ifcnc2qqwufndhdl-GFdadSzt00ze9xe1eoZjHA@public.gmane.org> Sender: linux-api-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Mark Brown Cc: Steven Rostedt , "ksummit-discuss-cunTk1MwBs98uUxBSJOaYoYkZiVZrdSR2LY78lusg7I@public.gmane.org" , Carlos O'Donell , "open list:ABI/API" , Thorsten Leemhuis , Shuah Khan List-Id: linux-api@vger.kernel.org On Thu, Jul 6, 2017 at 11:28 AM, Mark Brown wrote: > On Wed, Jul 05, 2017 at 01:02:00PM -0400, Steven Rostedt wrote: >> Guenter Roeck wrote: > >> > If a test to reproduce a problem exists, it might be more beneficial to suggest >> > to the patch submitter that it would be great if that test would be submitted >> > as unit test instead of shaming that person for not doing so. Acknowledging and >> > praising kselftest submissions might help more than shaming for non-submissions. > >> > My concern would be that once the shaming starts, it won't stop. > >> I think this is a communication issue. My word for "shaming" was to >> call out a developer for not submitting a test. It wasn't about making >> fun of them, or anything like that. I was only making a point >> about how to teach people that they need to be more aware of the >> testing infrastructure. Not about actually demeaning people. > > I think before anything like that is viable we need to show a concerted > and visible interest in actually running the tests we already have and > paying attention to the results - if people can see that they're just > checking a checkbox that will often result in low quality tests which > can do more harm than good. +1. That pretty much means large-scale CI. The i915 test suite has suffered quite a bit over the past years because the CI infrastructure didn't keep up. Result is that running full CI kills pretty much every platform there is eventually, and it's really hard to get back to a state where the testsuite can be used to catch regressions again. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch