[Linux-kernel-mentees] Question regarding marking bugs as "invalid"

[Linux-kernel-mentees] Question regarding marking bugs as "invalid"

[Himadri Pandya]

Hi,

Is it correct to mark bugs as "invalid" if they have reproducers but
the reproducer doesn't trigger any issue on testing current status? If
not, then what should be done about such bugs?

Thanks & Regards,
Himadri
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Re: [Linux-kernel-mentees] Question regarding marking bugs as "invalid"

[Lukas Bulwahn]

On Tue, Sep 15, 2020 at 1:52 PM Himadri Pandya <himadrispandya@gmail.com> wrote:
>
> Hi,
>
> Is it correct to mark bugs as "invalid" if they have reproducers but
> the reproducer doesn't trigger any issue on testing current status? If
> not, then what should be done about such bugs?
>
> Thanks & Regards,
> Himadri
>

Himadri,

if possible try to determine which commit fixed the issue the
reproducer triggered.

You can potentially bisect with the reproducer on the git history or
you can simply look in the git log of the affected files if someone
mentioned fixing something related to the trigger.

That helps to make sure we do not just close reproducers that just
need a lot of time, configuration or luck to hit a certain crash.

Lukas
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Re: [Linux-kernel-mentees] Question regarding marking bugs as "invalid"

[Dmitry Vyukov via Linux-kernel-mentees]

On Tue, Sep 15, 2020 at 2:03 PM Lukas Bulwahn <lukas.bulwahn@gmail.com> wrote:
>
> On Tue, Sep 15, 2020 at 1:52 PM Himadri Pandya <himadrispandya@gmail.com> wrote:
> >
> > Hi,
> >
> > Is it correct to mark bugs as "invalid" if they have reproducers but
> > the reproducer doesn't trigger any issue on testing current status? If
> > not, then what should be done about such bugs?
> >
> > Thanks & Regards,
> > Himadri
> >
>
> Himadri,
>
> if possible try to determine which commit fixed the issue the
> reproducer triggered.
>
> You can potentially bisect with the reproducer on the git history or
> you can simply look in the git log of the affected files if someone
> mentioned fixing something related to the trigger.
>
> That helps to make sure we do not just close reproducers that just
> need a lot of time, configuration or luck to hit a certain crash.


Hi Himadri,

Basically what Lukas said.
Bulk closing all of them as "invalid" would be bad for several
reasons. Either do some reasonable amount of degging, or wait for
syzbot fix bisection, maybe it will shed some light. It should happen
after 30 days since last crash IIRC. Also all testing requests/results
are shown on the dashboard, so this bit of information is not lost.
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Re: [Linux-kernel-mentees] Question regarding marking bugs as "invalid"

[Himadri Pandya]

On Tue, Sep 15, 2020 at 5:33 PM Lukas Bulwahn <lukas.bulwahn@gmail.com> wrote:
>
> On Tue, Sep 15, 2020 at 1:52 PM Himadri Pandya <himadrispandya@gmail.com> wrote:
> >
> > Hi,
> >
> > Is it correct to mark bugs as "invalid" if they have reproducers but
> > the reproducer doesn't trigger any issue on testing current status? If
> > not, then what should be done about such bugs?
> >
> > Thanks & Regards,
> > Himadri
> >
>
> Himadri,
>
> if possible try to determine which commit fixed the issue the
> reproducer triggered.
>
> You can potentially bisect with the reproducer on the git history or
> you can simply look in the git log of the affected files if someone
> mentioned fixing something related to the trigger.
>
> That helps to make sure we do not just close reproducers that just
> need a lot of time, configuration or luck to hit a certain crash.
>
> Lukas

Understood. Thank you.

Himadri
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Re: [Linux-kernel-mentees] Question regarding marking bugs as "invalid"

[Himadri Pandya]

On Tue, Sep 15, 2020 at 6:12 PM Dmitry Vyukov <dvyukov@google.com> wrote:
>
> On Tue, Sep 15, 2020 at 2:03 PM Lukas Bulwahn <lukas.bulwahn@gmail.com> wrote:
> >
> > On Tue, Sep 15, 2020 at 1:52 PM Himadri Pandya <himadrispandya@gmail.com> wrote:
> > >
> > > Hi,
> > >
> > > Is it correct to mark bugs as "invalid" if they have reproducers but
> > > the reproducer doesn't trigger any issue on testing current status? If
> > > not, then what should be done about such bugs?
> > >
> > > Thanks & Regards,
> > > Himadri
> > >
> >
> > Himadri,
> >
> > if possible try to determine which commit fixed the issue the
> > reproducer triggered.
> >
> > You can potentially bisect with the reproducer on the git history or
> > you can simply look in the git log of the affected files if someone
> > mentioned fixing something related to the trigger.
> >
> > That helps to make sure we do not just close reproducers that just
> > need a lot of time, configuration or luck to hit a certain crash.
>
>
> Hi Himadri,
>
> Basically what Lukas said.
> Bulk closing all of them as "invalid" would be bad for several
> reasons. Either do some reasonable amount of degging, or wait for
> syzbot fix bisection, maybe it will shed some light. It should happen
> after 30 days since last crash IIRC. Also all testing requests/results
> are shown on the dashboard, so this bit of information is not lost.

Understood.

I incorrectly assumed(before posting this question) that I should mark
such bugs as invalid and sent the command to syzbot for one such bug.
Now I understand that it was not the right thing. It doesn't show on
the dashboard and I don't know how to undo it :(.

Bug's dashboard link -
https://syzkaller.appspot.com/bug?id=4c7fd5b46451d957a3d8188f393f1982f9753fe7

Himadri
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Re: [Linux-kernel-mentees] Question regarding marking bugs as "invalid"

[Dmitry Vyukov via Linux-kernel-mentees]

On Tue, Sep 15, 2020 at 3:23 PM Himadri Pandya <himadrispandya@gmail.com> wrote:
> > > > Hi,
> > > >
> > > > Is it correct to mark bugs as "invalid" if they have reproducers but
> > > > the reproducer doesn't trigger any issue on testing current status? If
> > > > not, then what should be done about such bugs?
> > > >
> > > > Thanks & Regards,
> > > > Himadri
> > > >
> > >
> > > Himadri,
> > >
> > > if possible try to determine which commit fixed the issue the
> > > reproducer triggered.
> > >
> > > You can potentially bisect with the reproducer on the git history or
> > > you can simply look in the git log of the affected files if someone
> > > mentioned fixing something related to the trigger.
> > >
> > > That helps to make sure we do not just close reproducers that just
> > > need a lot of time, configuration or luck to hit a certain crash.
> >
> >
> > Hi Himadri,
> >
> > Basically what Lukas said.
> > Bulk closing all of them as "invalid" would be bad for several
> > reasons. Either do some reasonable amount of degging, or wait for
> > syzbot fix bisection, maybe it will shed some light. It should happen
> > after 30 days since last crash IIRC. Also all testing requests/results
> > are shown on the dashboard, so this bit of information is not lost.
>
> Understood.
>
> I incorrectly assumed(before posting this question) that I should mark
> such bugs as invalid and sent the command to syzbot for one such bug.
> Now I understand that it was not the right thing. It doesn't show on
> the dashboard and I don't know how to undo it :(.
>
> Bug's dashboard link -
> https://syzkaller.appspot.com/bug?id=4c7fd5b46451d957a3d8188f393f1982f9753fe7

Hi Himadri,

Transitions to terminal states are not undo-able. Consider the same
bug is rediscovered concurrently with one undoing "#syz invalid". Now
we have 2 versions of the same bug and it will be an incomprehensible
mess.

But marking one bug in such a way is not the end of the world. We have
other real bugs marked as invalid. So no worries.
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Re: [Linux-kernel-mentees] Question regarding marking bugs as "invalid"

[Himadri Pandya]

On Tue, Sep 15, 2020 at 6:57 PM Dmitry Vyukov <dvyukov@google.com> wrote:
>
> On Tue, Sep 15, 2020 at 3:23 PM Himadri Pandya <himadrispandya@gmail.com> wrote:
> > > > > Hi,
> > > > >
> > > > > Is it correct to mark bugs as "invalid" if they have reproducers but
> > > > > the reproducer doesn't trigger any issue on testing current status? If
> > > > > not, then what should be done about such bugs?
> > > > >
> > > > > Thanks & Regards,
> > > > > Himadri
> > > > >
> > > >
> > > > Himadri,
> > > >
> > > > if possible try to determine which commit fixed the issue the
> > > > reproducer triggered.
> > > >
> > > > You can potentially bisect with the reproducer on the git history or
> > > > you can simply look in the git log of the affected files if someone
> > > > mentioned fixing something related to the trigger.
> > > >
> > > > That helps to make sure we do not just close reproducers that just
> > > > need a lot of time, configuration or luck to hit a certain crash.
> > >
> > >
> > > Hi Himadri,
> > >
> > > Basically what Lukas said.
> > > Bulk closing all of them as "invalid" would be bad for several
> > > reasons. Either do some reasonable amount of degging, or wait for
> > > syzbot fix bisection, maybe it will shed some light. It should happen
> > > after 30 days since last crash IIRC. Also all testing requests/results
> > > are shown on the dashboard, so this bit of information is not lost.
> >
> > Understood.
> >
> > I incorrectly assumed(before posting this question) that I should mark
> > such bugs as invalid and sent the command to syzbot for one such bug.
> > Now I understand that it was not the right thing. It doesn't show on
> > the dashboard and I don't know how to undo it :(.
> >
> > Bug's dashboard link -
> > https://syzkaller.appspot.com/bug?id=4c7fd5b46451d957a3d8188f393f1982f9753fe7
>
> Hi Himadri,
>
> Transitions to terminal states are not undo-able. Consider the same
> bug is rediscovered concurrently with one undoing "#syz invalid". Now
> we have 2 versions of the same bug and it will be an incomprehensible
> mess.
>

Understood. My sincerest apologies for being naive.

My assumption was that commands like "invalid" are similar to the
action of submitting a patch, it would generate some discussion about
the bug and if it is really invalid, someone with authority(like
maintainers) would actually go and mark it as "invalid". I was clearly
mistaken. But if we don't have any gatekeeping on such commands and
anyone can directly change the status of the bug by merely sending an
email to syzbot, isn't it a security issue?

Himadri

> But marking one bug in such a way is not the end of the world. We have
> other real bugs marked as invalid. So no worries.
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Re: [Linux-kernel-mentees] Question regarding marking bugs as "invalid"

[Dmitry Vyukov via Linux-kernel-mentees]

On Tue, Sep 15, 2020 at 4:15 PM Himadri Pandya <himadrispandya@gmail.com> wrote:
> > On Tue, Sep 15, 2020 at 3:23 PM Himadri Pandya <himadrispandya@gmail.com> wrote:
> > > > > > Hi,
> > > > > >
> > > > > > Is it correct to mark bugs as "invalid" if they have reproducers but
> > > > > > the reproducer doesn't trigger any issue on testing current status? If
> > > > > > not, then what should be done about such bugs?
> > > > > >
> > > > > > Thanks & Regards,
> > > > > > Himadri
> > > > > >
> > > > >
> > > > > Himadri,
> > > > >
> > > > > if possible try to determine which commit fixed the issue the
> > > > > reproducer triggered.
> > > > >
> > > > > You can potentially bisect with the reproducer on the git history or
> > > > > you can simply look in the git log of the affected files if someone
> > > > > mentioned fixing something related to the trigger.
> > > > >
> > > > > That helps to make sure we do not just close reproducers that just
> > > > > need a lot of time, configuration or luck to hit a certain crash.
> > > >
> > > >
> > > > Hi Himadri,
> > > >
> > > > Basically what Lukas said.
> > > > Bulk closing all of them as "invalid" would be bad for several
> > > > reasons. Either do some reasonable amount of degging, or wait for
> > > > syzbot fix bisection, maybe it will shed some light. It should happen
> > > > after 30 days since last crash IIRC. Also all testing requests/results
> > > > are shown on the dashboard, so this bit of information is not lost.
> > >
> > > Understood.
> > >
> > > I incorrectly assumed(before posting this question) that I should mark
> > > such bugs as invalid and sent the command to syzbot for one such bug.
> > > Now I understand that it was not the right thing. It doesn't show on
> > > the dashboard and I don't know how to undo it :(.
> > >
> > > Bug's dashboard link -
> > > https://syzkaller.appspot.com/bug?id=4c7fd5b46451d957a3d8188f393f1982f9753fe7
> >
> > Hi Himadri,
> >
> > Transitions to terminal states are not undo-able. Consider the same
> > bug is rediscovered concurrently with one undoing "#syz invalid". Now
> > we have 2 versions of the same bug and it will be an incomprehensible
> > mess.
> >
>
> Understood. My sincerest apologies for being naive.
>
> My assumption was that commands like "invalid" are similar to the
> action of submitting a patch, it would generate some discussion about
> the bug and if it is really invalid, someone with authority(like
> maintainers) would actually go and mark it as "invalid". I was clearly
> mistaken. But if we don't have any gatekeeping on such commands and
> anyone can directly change the status of the bug by merely sending an
> email to syzbot, isn't it a security issue?

+workflows

What you are saying is all true. There is no authorization and anybody
can close any bug.

That's the process we could combine from parts we had. Implementing
proper support with users/permissions/assignees would require:
1. Implementing support in syzbot
2. Implementing and deploying some form of user identity and
authorization for kernel developers (emails is not a trusted media on
its own)
3. Finding responsible maintainers for all parts of the kernel and
making them do this additional work

All of these are problematic on different fronts. (2) can be replaced
with use of Bugzilla, but it does not seem to make the problem easier
overall. So so far we have the process we have.
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Re: [Linux-kernel-mentees] Question regarding marking bugs as "invalid"

[Lukas Bulwahn]

On Wed, 16 Sep 2020, Dmitry Vyukov wrote:

> On Tue, Sep 15, 2020 at 4:15 PM Himadri Pandya <himadrispandya@gmail.com> wrote:
> > > On Tue, Sep 15, 2020 at 3:23 PM Himadri Pandya <himadrispandya@gmail.com> wrote:
> > > > > > > Hi,
> > > > > > >
> > > > > > > Is it correct to mark bugs as "invalid" if they have reproducers but
> > > > > > > the reproducer doesn't trigger any issue on testing current status? If
> > > > > > > not, then what should be done about such bugs?
> > > > > > >
> > > > > > > Thanks & Regards,
> > > > > > > Himadri
> > > > > > >
> > > > > >
> > > > > > Himadri,
> > > > > >
> > > > > > if possible try to determine which commit fixed the issue the
> > > > > > reproducer triggered.
> > > > > >
> > > > > > You can potentially bisect with the reproducer on the git history or
> > > > > > you can simply look in the git log of the affected files if someone
> > > > > > mentioned fixing something related to the trigger.
> > > > > >
> > > > > > That helps to make sure we do not just close reproducers that just
> > > > > > need a lot of time, configuration or luck to hit a certain crash.
> > > > >
> > > > >
> > > > > Hi Himadri,
> > > > >
> > > > > Basically what Lukas said.
> > > > > Bulk closing all of them as "invalid" would be bad for several
> > > > > reasons. Either do some reasonable amount of degging, or wait for
> > > > > syzbot fix bisection, maybe it will shed some light. It should happen
> > > > > after 30 days since last crash IIRC. Also all testing requests/results
> > > > > are shown on the dashboard, so this bit of information is not lost.
> > > >
> > > > Understood.
> > > >
> > > > I incorrectly assumed(before posting this question) that I should mark
> > > > such bugs as invalid and sent the command to syzbot for one such bug.
> > > > Now I understand that it was not the right thing. It doesn't show on
> > > > the dashboard and I don't know how to undo it :(.
> > > >
> > > > Bug's dashboard link -
> > > > https://syzkaller.appspot.com/bug?id=4c7fd5b46451d957a3d8188f393f1982f9753fe7
> > >
> > > Hi Himadri,
> > >
> > > Transitions to terminal states are not undo-able. Consider the same
> > > bug is rediscovered concurrently with one undoing "#syz invalid". Now
> > > we have 2 versions of the same bug and it will be an incomprehensible
> > > mess.
> > >
> >
> > Understood. My sincerest apologies for being naive.
> >
> > My assumption was that commands like "invalid" are similar to the
> > action of submitting a patch, it would generate some discussion about
> > the bug and if it is really invalid, someone with authority(like
> > maintainers) would actually go and mark it as "invalid". I was clearly
> > mistaken. But if we don't have any gatekeeping on such commands and
> > anyone can directly change the status of the bug by merely sending an
> > email to syzbot, isn't it a security issue?
>

Himandri, it works through collective ignorance.

Basically, we assume that the evil people try to do something else :)

Now in case, somebody evil would try to do what you suggest, close some 
syzbot report although it is not fixed, it would simply be found again by 
syzkaller and hence there is a new report.

If you now close many issues and repeatedly do that, Dmitry would notice 
as he maintaining the system.

Usually, people are not evil, though, but just err.

Also, the public syzbot is only one of many instances of syzkaller/syzbot 
running. So, we would notice the mismatch between instances as well.
 
> +workflows
> 
> What you are saying is all true. There is no authorization and anybody
> can close any bug.
> 
> That's the process we could combine from parts we had. Implementing
> proper support with users/permissions/assignees would require:
> 1. Implementing support in syzbot
> 2. Implementing and deploying some form of user identity and
> authorization for kernel developers (emails is not a trusted media on
> its own)
> 3. Finding responsible maintainers for all parts of the kernel and
> making them do this additional work
> 
> All of these are problematic on different fronts. (2) can be replaced
> with use of Bugzilla, but it does not seem to make the problem easier
> overall. So so far we have the process we have.
> 

Dmitry, I agree. For tracking static analysis findings that are subject 
to even more false positive findings (and hence need more triage) and 
that are not automatically re-opened if someone closes them wrongly, this 
situation is as far as I see a real issue when collaborating.

Currently, I am thinking about just using emails or so as identities and 
then have workflows for every user to state who they trust and to which 
degree (just take the finding of X as true, show the rationale of Y but 
let me recheck, totally ignore Z's findings).

Certainly, such efforts and many others would benefit from some basic user 
identity management for kernel developers, though.


Lukas
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees