From: Fengguang Wu <fengguang.wu@intel.com>
To: Boris Brezillon <boris.brezillon@bootlin.com>
Cc: Geert Uytterhoeven <geert@linux-m68k.org>,
Sasha Levin <Alexander.Levin@microsoft.com>,
"ksummit-discuss@lists.linuxfoundation.org"
<ksummit-discuss@lists.linuxfoundation.org>,
Greg KH <gregkh@linuxfoundation.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"w@1wt.eu" <w@1wt.eu>
Subject: Re: [Ksummit-discuss] bug-introducing patches
Date: Mon, 14 May 2018 17:25:48 +0800 [thread overview]
Message-ID: <20180514092548.qwv4ycsixduqd6q3@wfg-t540p.sh.intel.com> (raw)
In-Reply-To: <20180514104803.23d2a8a3@bbrezillon>
On Mon, May 14, 2018 at 10:48:03AM +0200, Boris Brezillon wrote:
>+Fengguang
>
>On Mon, 14 May 2018 10:40:10 +0200
>Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>
>> Hi Boris,
>>
>> On Mon, May 14, 2018 at 10:34 AM, Boris Brezillon
>> <boris.brezillon@bootlin.com> wrote:
>> > On Mon, 14 May 2018 10:29:04 +0200
>> > Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>> >> On Mon, May 14, 2018 at 10:12 AM, Boris Brezillon
>> >> <boris.brezillon@bootlin.com> wrote:
>> >> > On Mon, 14 May 2018 10:00:30 +0200
>> >> > Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>> >> >> On Tue, May 1, 2018 at 10:00 PM, Sasha Levin
>> >> >> <Alexander.Levin@microsoft.com> wrote:
>> >> >> > On Tue, May 01, 2018 at 03:44:50PM -0400, Theodore Y. Ts'o wrote:
>> >> >> >>On Tue, May 01, 2018 at 04:38:21PM +0000, Sasha Levin wrote:
>> >> >> > What's worse is that that commit is tagged for stable, which means
>> >> >> > that (given Greg's schedule) it may find it's way to -stable users
>> >> >> > even before some -next users/bots had a chance to test it out.
>> >> >>
>> >> >> I just noticed a case where a commit was picked up for stable, while a
>> >> >> bot had flagged it as a build regression 18 hours earlier (with a CC to
>> >> >> lkml).
>> >> >
>> >> > Also, this patch has been on a tree that I know is tested by Fengguang's
>> >> > robots for more than a week (and in linux-next for 2 days, which, I
>> >> > agree, is probably not enough), and still, I only received the bug
>> >> > report when the patch reached mainline. Are there tests that are only
>> >> > run on Linus' tree?
>> >>
>> >> Have your received a success report from Fengguang's bot, listing all
>> >> configs tested (the broken one should be included; it is included in the
>> >> configs tested on my branches)?
>> >
>> > Yes I did (see below).
>> >
>> > -->8--
>> > From: kbuild test robot <lkp@intel.com>
>> > To: Boris Brezillon <boris.brezillon@bootlin.com>
>> > Subject: [bbrezillon-0day:mtd/fixes] BUILD SUCCESS fc3a9e15b492eef707afd56b7478001fdecfe53f
>> > Date: Mon, 07 May 2018 20:05:52 +0800
>> > User-Agent: Heirloom mailx 12.5 6/20/10
>> >
>> > tree/branch: https://github.com/bbrezillon/linux-0day mtd/fixes
>> > branch HEAD: fc3a9e15b492eef707afd56b7478001fdecfe53f mtd: rawnand: Make sure we wait tWB before polling the STATUS reg
>> >
>> > elapsed time: 49m
>> >
>> > configs tested: 142
>>
>> But the failed config (m68k/allmodconfig) is not listed?
>
>Yes, that's my point. It seems that some configs are only rarely
>(never?) tested on my linux-0day tree (probably because they take longer
>to build), and I should only take kbuild robot results as an indication
>not a guarantee.
Yeah sorry, there is no 100% guarantee. There are 2 main aspects to
this problem.
- Response time vs coverage. Most build errors can be caught within 1
day. The build success notification email is typically sent within
half day (a reasonable feedback time). At this time, it can only be
a rough indication not a guarantee. After sending the 0day build
success notification, the build tests will actually continue for
about 1 week to increase test coverage.
- Merge-test-bisect based workflow. If one branch is hard to merge
with others, especially if it's based on old kernel, it'll receive
much less test coverage. Branches with known build/boot errors will
be excluded from further merges, too.
Thanks,
Fengguang
next prev parent reply other threads:[~2018-05-14 9:25 UTC|newest]
Thread overview: 145+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-05-01 16:38 bug-introducing patches Sasha Levin
2018-05-01 19:44 ` Theodore Y. Ts'o
2018-05-01 20:00 ` Sasha Levin
2018-05-01 20:33 ` Willy Tarreau
2018-05-01 20:42 ` Sasha Levin
2018-05-01 20:54 ` [Ksummit-discuss] " Theodore Y. Ts'o
2018-05-01 21:15 ` Mark Brown
2018-05-02 8:11 ` Daniel Vetter
2018-05-02 19:46 ` Sasha Levin via Ksummit-discuss
2018-05-03 2:05 ` Mark Brown via Ksummit-discuss
2018-05-03 3:10 ` Theodore Y. Ts'o
2018-05-03 3:52 ` Guenter Roeck
2018-05-03 12:03 ` Greg KH
2018-05-03 22:42 ` Mark Brown
2018-05-03 23:09 ` Tony Lindgren
2018-05-04 14:21 ` Ulf Hansson
2018-05-09 8:44 ` Mark Brown
2018-05-09 8:47 ` Daniel Vetter
2018-05-09 8:51 ` Geert Uytterhoeven
2018-05-09 9:03 ` Mark Brown
2018-05-09 10:47 ` Stephen Rothwell
2018-05-09 10:55 ` Vinod Koul
2018-05-09 12:43 ` Stephen Rothwell
2018-05-09 12:47 ` Vinod Koul
2018-05-15 10:42 ` Krzysztof Kozlowski
2018-05-15 11:54 ` Stephen Rothwell
2018-05-09 14:05 ` Mark Brown
2018-05-09 22:09 ` Stephen Rothwell
2018-05-10 13:36 ` Mark Brown
2018-05-10 22:01 ` Stephen Rothwell
2018-05-09 15:57 ` Guenter Roeck
2018-05-09 21:45 ` Stephen Rothwell
2018-05-09 16:04 ` Dan Williams
2018-05-09 21:51 ` Stephen Rothwell
2018-05-09 19:35 ` Boris Brezillon
2018-05-09 21:58 ` Stephen Rothwell
2018-05-10 3:15 ` Sasha Levin via Ksummit-discuss
2018-05-10 15:57 ` Tony Lindgren
2018-05-10 22:05 ` Stephen Rothwell
2018-05-11 8:47 ` David Sterba
2018-05-12 4:03 ` Stephen Rothwell
2018-05-12 4:38 ` Stephen Rothwell
2018-05-12 18:34 ` Guenter Roeck
2018-05-13 13:53 ` Andy Shevchenko
2018-05-14 8:36 ` Ulf Hansson
2018-05-14 21:45 ` Stephen Rothwell
2018-05-17 5:10 ` Mark Brown
2018-05-10 16:03 ` Jiri Kosina
2018-05-10 16:47 ` Sasha Levin via Ksummit-discuss
2018-05-14 7:53 ` Geert Uytterhoeven
2018-05-14 8:00 ` Geert Uytterhoeven
2018-05-14 8:12 ` Boris Brezillon
2018-05-14 8:29 ` Geert Uytterhoeven
2018-05-14 8:34 ` Boris Brezillon
2018-05-14 8:40 ` Geert Uytterhoeven
2018-05-14 8:48 ` Boris Brezillon
2018-05-14 9:25 ` Fengguang Wu [this message]
2018-05-11 2:10 ` Mark Brown
2018-05-08 2:34 ` Sasha Levin
2018-05-08 3:48 ` Theodore Y. Ts'o
2018-05-08 14:49 ` Tony Lindgren
2018-05-09 8:13 ` Mark Brown
2018-05-10 15:36 ` Tony Lindgren
2018-05-08 20:29 ` Sasha Levin via Ksummit-discuss
2018-05-08 20:40 ` Matthew Wilcox
2018-05-08 20:55 ` Sasha Levin
2018-05-08 20:59 ` David Lang
2018-05-08 21:43 ` Sasha Levin via Ksummit-discuss
2018-05-08 21:51 ` Dan Williams
2018-05-08 22:41 ` James Bottomley
2018-05-08 21:26 ` Justin Forbes
2018-05-08 21:00 ` Ken Moffat
2018-05-08 22:15 ` Theodore Y. Ts'o
2018-05-10 16:39 ` Sasha Levin
2018-05-09 4:47 ` Willy Tarreau
2018-05-08 13:58 ` Justin Forbes
2018-05-08 2:39 ` Sasha Levin via Ksummit-discuss
2018-05-01 22:02 ` Sasha Levin
2018-05-02 4:30 ` Willy Tarreau
2018-05-02 19:42 ` Sasha Levin
2018-05-02 20:02 ` Willy Tarreau
2018-07-14 17:38 ` Pavel Machek
2018-07-14 18:37 ` [Ksummit-discuss] " Guenter Roeck
2018-07-14 19:47 ` Pavel Machek
2018-07-14 20:40 ` Guenter Roeck
2018-07-14 21:09 ` Pavel Machek
2018-07-15 5:57 ` Willy Tarreau
2018-07-15 8:54 ` Greg KH
2018-07-15 14:50 ` Theodore Y. Ts'o
2018-07-15 20:15 ` Pavel Machek
2018-05-03 11:08 ` [Ksummit-discuss] " Jani Nikula
2018-05-03 14:33 ` James Bottomley
2018-05-03 14:48 ` Willy Tarreau
2018-05-03 15:06 ` Sasha Levin via Ksummit-discuss
2018-05-03 15:27 ` James Bottomley
2018-05-03 15:43 ` Sasha Levin via Ksummit-discuss
2018-05-03 17:17 ` Randy Dunlap
2018-05-03 17:39 ` Sasha Levin via Ksummit-discuss
2018-05-03 18:10 ` James Bottomley
2018-05-03 15:56 ` Willy Tarreau
2018-05-03 18:58 ` Theodore Y. Ts'o
2018-05-02 15:32 ` Geert Uytterhoeven
2018-05-02 19:51 ` Sasha Levin via Ksummit-discuss
2018-05-02 20:41 ` Geert Uytterhoeven
2018-05-03 0:06 ` [Ksummit-discuss] " Theodore Y. Ts'o
2018-05-03 0:38 ` Guenter Roeck
2018-05-03 2:30 ` Willy Tarreau
2018-05-03 14:55 ` Sasha Levin
2018-05-03 15:49 ` Guenter Roeck
2018-05-03 16:02 ` Sasha Levin via Ksummit-discuss
2018-05-03 16:50 ` Justin Forbes
2018-05-03 17:09 ` Guenter Roeck
2018-05-03 11:48 ` Al Viro
2018-05-03 14:46 ` Sasha Levin via Ksummit-discuss
2018-05-03 14:52 ` Willy Tarreau
2018-05-03 15:01 ` Sasha Levin via Ksummit-discuss
2018-05-03 16:00 ` Willy Tarreau
2018-05-03 16:14 ` Sasha Levin
2018-05-03 16:35 ` Willy Tarreau
2018-05-03 17:29 ` Sasha Levin via Ksummit-discuss
2018-05-03 17:57 ` Willy Tarreau
2018-05-03 18:12 ` Sasha Levin
2018-05-03 18:46 ` Guenter Roeck
2018-05-03 19:03 ` Willy Tarreau
2018-05-03 16:54 ` Al Viro
2018-05-03 17:34 ` Sasha Levin via Ksummit-discuss
2018-05-03 18:20 ` Al Viro
2018-05-03 18:55 ` Greg KH
2018-05-03 19:14 ` Willy Tarreau
2018-05-03 19:17 ` Sasha Levin via Ksummit-discuss
2018-05-03 19:04 ` Sasha Levin
2018-05-04 9:57 ` David Howells
2018-05-04 12:31 ` Jani Nikula
2018-05-04 13:09 ` Theodore Y. Ts'o
2018-05-04 17:40 ` Greg KH
2018-05-04 21:13 ` Theodore Y. Ts'o
2018-05-04 21:38 ` James Bottomley
2018-05-04 21:51 ` Sasha Levin
2018-05-04 23:35 ` Theodore Y. Ts'o
2018-05-05 4:23 ` Willy Tarreau
2018-05-05 5:02 ` Eric W. Biederman
2018-05-05 16:37 ` Greg KH
2018-05-05 5:27 ` Sasha Levin via Ksummit-discuss
2018-05-03 11:43 ` Al Viro
2018-05-02 15:32 ` Geert Uytterhoeven
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180514092548.qwv4ycsixduqd6q3@wfg-t540p.sh.intel.com \
--to=fengguang.wu@intel.com \
--cc=Alexander.Levin@microsoft.com \
--cc=boris.brezillon@bootlin.com \
--cc=geert@linux-m68k.org \
--cc=gregkh@linuxfoundation.org \
--cc=ksummit-discuss@lists.linuxfoundation.org \
--cc=linux-kernel@vger.kernel.org \
--cc=w@1wt.eu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).