From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Google-Smtp-Source: ACJfBouXjnSxHr1OXCshRSY5YOftfkDjHNLLN5vNYMQt9kSgCPN6sOMwTDoUJfLiuhYGfQ4sZe6L ARC-Seal: i=1; a=rsa-sha256; t=1516326524; cv=none; d=google.com; s=arc-20160816; b=Oqz5pemEKqAAHWi2OrWWOZGKEi8OkufSWdB3bAq/I4h8upQe5ySi4ufEfdFOqTRrHU 9zpYQPdDOCHOldiMPWvaKUo8hrX9ETm/OoIUj3vdDxd0sIY1fgpbsD5TSp1LsQRgmfJq VguoeI7v8FZ9SnkFWaNWtk8xfN2cTUhvsCMXC7T7L8cg1DB+RIG6SvbWX6LRgRWXrpi6 nqmSGixajdbbQazdpFJk+RG0+EQ5qllMdfl/a5+hWHPJ778JIdfhKSSvHeg+urPQAKh2 jDU4xdhjlQwWarzAvJL1xOX4xqkYTA/Wpje0C4FcVZV0R2/pEcS5mEerx/chQPYhSiHF z/kA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:arc-authentication-results; bh=hIO+ptS2X8b0Qkp61FXb2Q7yB5W/6EnrVl07O7N2RZc=; b=mnbFox0uqZRtVHelSGdLUg1lNYib+YTnOHHfQfR8/NXYp8Os+dF+tR8zLRzYT5nB96 kYH9gcledQVJ5CWbLH5xteBAaZSwNkQfhI8JNMcwIFkjXdpVrUxIiELeSVtyFsJdDkG5 YPOwNgJmpjDq1TZK9ySjmBf0O9dUVIhykCv6DWOuQylf/cqz8xF5OBdqpQ/uYb6yeZIr glJqDemXGIfkrAuq46FQFGSUx4pPFwdCnR+vcSN7HrqfNiIE6OYIIg4SugH8L6Q/JxsG zXwtg4U+SeZcRWwIdTsrqeymuXZdAykmpDlXsYGRulViaimViHHTd/J/fTgjMkoEjIna 8lLg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of fengguang.wu@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=fengguang.wu@intel.com Authentication-Results: mx.google.com; spf=pass (google.com: domain of fengguang.wu@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=fengguang.wu@intel.com X-Amp-Result: UNSCANNABLE X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.46,379,1511856000"; d="scan'208";a="20595417" Date: Fri, 19 Jan 2018 09:48:03 +0800 From: Fengguang Wu To: Dmitry Vyukov Cc: Guenter Roeck , LKML , Theodore Ts'o , "Eric W. Biederman" , Greg Kroah-Hartman , Andrew Morton , Linus Torvalds , syzkaller , Stephen Rothwell , David Miller Subject: Re: what trees/branches to test on syzbot Message-ID: <20180119014803.n75l5vrxlpifm3sc@wfg-t540p.sh.intel.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: User-Agent: NeoMutt/20170609 (1.8.3) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: =?utf-8?q?1589734662777833222?= X-GMAIL-MSGID: =?utf-8?q?1589983601191827093?= X-Mailing-List: linux-kernel@vger.kernel.org List-ID: Hi Dmitry, On Tue, Jan 16, 2018 at 10:58:51AM +0100, Dmitry Vyukov wrote: >On Tue, Jan 16, 2018 at 10:45 AM, Guenter Roeck wrote: >> On Mon, Jan 15, 2018 at 11:51 PM, Dmitry Vyukov wrote: >>> Hello, >>> >>> Several people proposed that linux-next should not be tested on >>> syzbot. While some people suggested that it needs to test as many >>> trees as possible. I've initially included linux-next as it is a >>> staging area before upstream tree, with the intention that patches are >>> _tested_ there, is they are not tested there, bugs enter upstream >>> tree. And then it takes much longer to get fix into other trees. >>> >>> So the question is: what trees/branches should be tested? Preferably >>> in priority order as syzbot can't test all of them. >>> >> >> I always thought that -next existed specifically to give people a >> chance to test the code in it. Maybe the question is where to report >> the test results ? > >FTR, from Guenter on another thread: > >> Interesting. Assuming that refers to linux-next, not linux-net, that >> may explain why linux-next tends to deteriorate. I wonder if I should >> drop it from my testing as well. I'll be happy to follow whatever the >> result of this exchange is and do the same. > >If we agree on some list of important branches, and what branches >specifically should not be tested with automatic reporting, I think it >will benefit everybody. >+Fengguang, can you please share your list and rationale behind it? 0-day aims to aggressively test as much tree and branches as possible, including various developer trees, maintainer, linux-next, mainline and stable trees. Here are the complete list of 800+ trees we monitored: https://git.kernel.org/pub/scm/linux/kernel/git/wfg/lkp-tests.git/tree/repo/linux The rationale is obvious. IMHO what really matters here is about capability rather than rationale: that policy heavily relies on the fundamental capability of auto bisecting. Once regressions are bisected, we know the owners of problem to auto send report to, ie. the first bad commit's author and committer. For the bugs that cannot be bisected, they tend to be old ones and we report more often on mainline tree than linux-next. Thanks, Fengguang