From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: ARC-Seal: i=1; a=rsa-sha256; t=1516628091; cv=none; d=google.com; s=arc-20160816; b=voZtZa9B+ig2PpzLcpPWJ+REWSF5dQwoLrldXFSyeqAQSU1JAAnKAua6U3Rw31vA9m Pp5mbyX6qErxtBiFYb3kjvLfYY2VqcqTJZ3S/ooRHn7lo3Z/aNE2SpnP2vsyortUETZd fqZRownCYYs+ojOvKL9iAHQEWCxMqoXvzFLg3bTsWdTae7fP6vrtZLonmCFD9m80WqOW rK8Xx3/PJ2kJ4K9oKFSU6rG4ywHy+r9gd+JKgXVAFFwje9gaBBLCcp3m0RTjruxruAPe 2mzILSmeGGq8BmZTS7NqEqaMKcictIwuqvlscwLY3s+wg9mDI51R2cBXkIG8Cr5ctHuw 6s2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:dkim-signature:arc-authentication-results; bh=0tZNRzWSVRZqCVkW9y4boE9cnRFalZUsVT1WWmWZuQk=; b=TSCQE/+Fd+8+Dm7oSdWCdESxDtpwQngrVu0IbJWuQfHBwMmPY5KjmP/3PQNP0JM8fT 5b4S78ipQOBfvxoFRKDr5jvvZiZirreJ9Qaklf14RfWadk/k7mgxcwB18+qckvszJ1RC 5HcEvpL9yLoMFFy/7nfQsar0mKFp7c6ljXeQsQQjhph6nG84D2++VBX2MKl2gEKrEJWv W+hhH+vm1IOswtVgqOdRhRyw6Dh0NkRourWzp5qShrETKkhREP5gmHJSuD9777Fg3C4c nvBRi8mqE9yEDZhER6nqaFv+R7gXt+h6PEDXEKcZNWV/sdX6KXUXd7NMdVw7A/61DHy0 Q6fA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=O+6xh1KY; spf=pass (google.com: domain of dvyukov@google.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=dvyukov@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=O+6xh1KY; spf=pass (google.com: domain of dvyukov@google.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=dvyukov@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com X-Google-Smtp-Source: AH8x225NLvM033eE21g/UCT+K0zIMcc3KycC0OrS55O8cwQuP649UQu2k/adNrgfP2nAyAkLIxGzUkvLvt5X2YYToFk= MIME-Version: 1.0 In-Reply-To: <20180119014803.n75l5vrxlpifm3sc@wfg-t540p.sh.intel.com> References: <20180119014803.n75l5vrxlpifm3sc@wfg-t540p.sh.intel.com> From: Dmitry Vyukov Date: Mon, 22 Jan 2018 14:34:30 +0100 Message-ID: Subject: Re: what trees/branches to test on syzbot To: Fengguang Wu Cc: Guenter Roeck , LKML , "Theodore Ts'o" , "Eric W. Biederman" , Greg Kroah-Hartman , Andrew Morton , Linus Torvalds , syzkaller , Stephen Rothwell , David Miller Content-Type: text/plain; charset="UTF-8" X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: =?utf-8?q?1589734662777833222?= X-GMAIL-MSGID: =?utf-8?q?1590299817737850223?= X-Mailing-List: linux-kernel@vger.kernel.org List-ID: On Fri, Jan 19, 2018 at 2:48 AM, Fengguang Wu wrote: > Hi Dmitry, > > > On Tue, Jan 16, 2018 at 10:58:51AM +0100, Dmitry Vyukov wrote: >> >> On Tue, Jan 16, 2018 at 10:45 AM, Guenter Roeck wrote: >>> >>> On Mon, Jan 15, 2018 at 11:51 PM, Dmitry Vyukov >>> wrote: >>>> >>>> Hello, >>>> >>>> Several people proposed that linux-next should not be tested on >>>> syzbot. While some people suggested that it needs to test as many >>>> trees as possible. I've initially included linux-next as it is a >>>> staging area before upstream tree, with the intention that patches are >>>> _tested_ there, is they are not tested there, bugs enter upstream >>>> tree. And then it takes much longer to get fix into other trees. >>>> >>>> So the question is: what trees/branches should be tested? Preferably >>>> in priority order as syzbot can't test all of them. >>>> >>> >>> I always thought that -next existed specifically to give people a >>> chance to test the code in it. Maybe the question is where to report >>> the test results ? >> >> >> FTR, from Guenter on another thread: >> >>> Interesting. Assuming that refers to linux-next, not linux-net, that >>> may explain why linux-next tends to deteriorate. I wonder if I should >>> drop it from my testing as well. I'll be happy to follow whatever the >>> result of this exchange is and do the same. >> >> >> If we agree on some list of important branches, and what branches >> specifically should not be tested with automatic reporting, I think it >> will benefit everybody. >> +Fengguang, can you please share your list and rationale behind it? > > > 0-day aims to aggressively test as much tree and branches as possible, > including various developer trees, maintainer, linux-next, mainline and > stable trees. Here are the complete list of 800+ trees we monitored: > > https://git.kernel.org/pub/scm/linux/kernel/git/wfg/lkp-tests.git/tree/repo/linux > > The rationale is obvious. IMHO what really matters here is about > capability rather than rationale: that policy heavily relies on the > fundamental capability of auto bisecting. Once regressions are > bisected, we know the owners of problem to auto send report to, ie. > the first bad commit's author and committer. > > For the bugs that cannot be bisected, they tend to be old ones and > we report more often on mainline tree than linux-next. Thanks for the info, Fengguang. Bisecting is something we need to syzbot in future. However about 50% of syzbot bugs are due to races and are somewhat difficult to bisect reliably.