All of lore.kernel.org
 help / color / mirror / Atom feed
From: Ozgur <ozgur@goosey.org>
To: Dmitry Vyukov <dvyukov@google.com>, Eric Biggers <ebiggers3@gmail.com>
Cc: LKML <linux-kernel@vger.kernel.org>,
	syzkaller <syzkaller@googlegroups.com>,
	Eric Dumazet <edumazet@google.com>,
	Kostya Serebryany <kcc@google.com>,
	Alexander Potapenko <glider@google.com>,
	andreyknvl <andreyknvl@google.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp>,
	David Miller <davem@davemloft.net>,
	Willem de Bruijn <willemb@google.com>,
	Guenter Roeck <groeck@google.com>,
	Stephan Mueller <smueller@chronox.de>
Subject: Re: [RFC] syzbot process
Date: Thu, 28 Dec 2017 13:51:21 +0300	[thread overview]
Message-ID: <581031514458281@web9g.yandex.ru> (raw)
In-Reply-To: <CACT4Y+aXLGYw2iaocMyf-zhWao96gRw+98caemaebt-9s2DMfw@mail.gmail.com>



28.12.2017, 13:41, "Dmitry Vyukov" <dvyukov@google.com>:
> On Fri, Dec 22, 2017 at 4:32 AM, Eric Biggers <ebiggers3@gmail.com> wrote:
>>  On Thu, Dec 21, 2017 at 01:52:40PM +0100, Dmitry Vyukov wrote:
>>>  However, the cost is that it needs to understand statuses of bugs:
>>>  most importantly, what commit fixes what bug. It also has support for
>>>  marking a bug as "invalid", e.g. happened once but most likely was
>>>  caused by a previous silent memory corruption. And support for marking
>>>  bugs as duplicates of other bugs, i.e. the same root cause and will be
>>>  fixed when the target bug is fixed. These simple rules are outlined in
>>>  the footer of each report and also explained in more detail at the
>>>  referenced link:
>>>
>>>  ----------------------------------
>>>  This bug is generated by a dumb bot. It may contain errors.
>>>  See https://goo.gl/tpsmEJ for details.
>>>  Direct all questions to syzkaller@googlegroups.com.
>>>  Please credit me with: Reported-by: syzbot <syzkaller@googlegroups.com>
>>>  syzbot will keep track of this bug report.
>>>  Once a fix for this bug is merged into any tree, reply to this email with:
>>>  #syz fix: exact-commit-title
>>>  If you want to test a patch for this bug, please reply with:
>>>  #syz test: git://repo/address.git branch
>>>  and provide the patch inline or as an attachment.
>>>  To mark this as a duplicate of another syzbot report, please reply with:
>>>  #syz dup: exact-subject-of-another-report
>>>  If it's a one-off invalid bug report, please reply with:
>>>  #syz invalid
>>>  Note: if the crash happens again, it will cause creation of a new bug report.
>>>  Note: all commands must start from beginning of the line in the email body.
>>>  ----------------------------------
>>>
>>>  Status tracking allows syzbot to (1) keep track of still unfixed bugs
>>>  (more than half actually gets lost in LKML archives if nobody keeps
>>>  track of them), (2) be able to ever report similarly looking crashes
>>>  as new bugs in future, (3) be able to test fixes.
>>>
>>>  The problem is that these rules are mostly not followed.
>>
>>  As others mentioned, allowing a bug ID to be in the fix's commit message,
>>  perhaps in the Reported-by line which syzbot already suggests to include, would
>>  make things a bit easier.
>>
>>  But I think the larger problem is that people in the community don't have any
>>  visibility into the statuses of the bugs, so they don't have any motivation to
>>  manage the statuses.
>>
>>  Are you planning to make a dashboard app publicly available for upstream kernel
>>  bugs being tracked by syzbot? I think it would be very useful for the
>>  community, especially for finding more details about a bug, e.g. when was it
>>  last seen, how often was it seen, has it been seen in multiple trees. Also for
>>  finding duplicates which may not have been sent to the correct mailing list.
>
> Hi Eric,
>
> Good question. I would very much like to open the UI, and I hope to do
> it in near future, but we need to do some additional work to make it
> possible. The good news is that information is already accumulating
> and we can do pings, etc.

Hello Dmitry,

I think not useful to be a GUI, for example it can be console based ui we can conenct and get information and fixed patches.
So syzbot is perfectly, I founded a patc last time :)

https://09738734946362323617.googlegroups.com/attach/3c6ef7059f77c/patch.txt?part=0.2&view=1&vt=ANaJVrFm49WFVkkKiomlnsrdfnv4P-0znjiC4agFB72ibq9_6iqg1rmZtw9-DxS5VvoOoKx8Ikl88sYEQQ45X0vjrwFkKDRaZELV-oU9DVmmrRAMSfStn24

And, I have a my suggestions:

Please keep to short url addresses and I think syzbot use to .txt file attached.
.txt is not good.

Ozgur

>>  syzbot also should be sending out reminders for bugs that are still open if the
>>  crash is still occurring, and even moreso if there is a reproducer.
>
> Agree. The reasons why this hasn't happen yet are:
> 1. syzbot is being built up as it's running, I am overwhelmed with
> hundreds of bugs and also doing lots of work which may be not directly
> visible but important (e.g. improving quality of generated
> reproducers, increasing percent of cases when reproducers are created,
> improving bug title extraction logic, implementing patch testing by
> request, now this new Reported-by-based process, etc).
> 2. Just sending an email for each open bug every week is simple, but I
> afraid it won't be warmly welcomed. The open questions are: how
> frequently syzbot should ping? should repro/no repro affect this? what
> to do if it stopped happening? stopped happenning for how long? and
> what if it happened just few times, so we can't really conclude if it
> still happens or not (but we've seen very bad races manifesting this
> way)? how should it interact with the following point?
>
>>  However, if the crash isn't still occurring, then I expect it will become
>>  necessary to automatically invalidate the bug after some time, lest the list of
>>  bugs grow without bound due to bugs that have already been fixed that no one has
>>  time to debug to figure out exactly when/what the fix was, especially if there
>>  is no reproducer. Or perhaps the bug was only in linux-next and only existed
>>  due to a buggy patch which was dropped or modified before it reached mainline,
>>  so there is no "fix" commit.
>
> Good point. I think we will need to do this in some form in future.
> Again open questions:
>  - what is the precise formula behind "isn't still occurring"?
>  - should we only close "no repro" bugs?
>  - should we re-test bugs with repro? (re-testing is not 100% precise,
> so we will lose some real subtle bugs this way)
>
> Thanks

  reply	other threads:[~2017-12-28 10:51 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-12-21 12:52 [RFC] syzbot process Dmitry Vyukov
2017-12-21 13:22 ` Andrey Ryabinin
2017-12-21 13:36   ` Stephan Mueller
2017-12-28 10:19     ` Dmitry Vyukov
2017-12-21 18:08   ` Linus Torvalds
2017-12-28 10:14     ` Dmitry Vyukov
2017-12-28 10:15   ` Dmitry Vyukov
2017-12-21 17:05 ` Greg Kroah-Hartman
2017-12-28 10:23   ` Dmitry Vyukov
2017-12-22  3:32 ` Eric Biggers
2017-12-28 10:41   ` Dmitry Vyukov
2017-12-28 10:51     ` Ozgur [this message]
2017-12-28 11:45       ` Dmitry Vyukov
2017-12-28 12:26         ` Ozgur
2017-12-28 12:30           ` Dmitry Vyukov
2017-12-28 12:55             ` Ozgur
2018-01-08 12:51               ` Dmitry Vyukov
2017-12-28 12:27         ` Dmitry Vyukov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=581031514458281@web9g.yandex.ru \
    --to=ozgur@goosey.org \
    --cc=akpm@linux-foundation.org \
    --cc=andreyknvl@google.com \
    --cc=davem@davemloft.net \
    --cc=dvyukov@google.com \
    --cc=ebiggers3@gmail.com \
    --cc=edumazet@google.com \
    --cc=glider@google.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=groeck@google.com \
    --cc=kcc@google.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=penguin-kernel@i-love.sakura.ne.jp \
    --cc=smueller@chronox.de \
    --cc=syzkaller@googlegroups.com \
    --cc=torvalds@linux-foundation.org \
    --cc=willemb@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.