Introducing bugbot

Introducing bugbot

[Konstantin Ryabitsev]

Hi, all:

Per our discussion last fall, I've been working on a bridge between bugzilla
and public-inbox, which went live in the past few days. It's still a very
early release, but it can do the following:

1. Create bugs from mailing list discussions, with full history. For example,
   it is sufficient for someone [1] to say "bugbot on" in a mailing list
   discussion to request bugbot to convert that thread into a new bug. It will
   then continue checking that thread for new messages, creating new comments
   for any added communication. This works both ways -- any comments added on
   bugzilla will be sent as follow-up messages to the original recipients.

   New bugs will be created in the Linux/Kernel component of Bugzilla, but
   they can then be reassigned to any other component. As long as they remain
   open, bugbot will continue tracking threads for new messages until either
   the bug is closed, or it has been over 30 days since any activity on the
   bug.

   [1] At this time, the From address of the person saying "bugbot on" has to
   have an account on bugzilla and be part of the "editbugs" group.

   Here's me using it to turn a thread into a bug:
   https://lore.kernel.org/lkml/CAJfuBxwomDagbdNP-Q6WvzcWsNY0Z2Lu2Yy5aZQ1d9W7Ka1_NQ@mail.gmail.com/

2. Start mailing list threads from pre-triaged bugzilla bugs. This works the
   opposite way and creates mailing list threads based on bug reports filed in
   bugzilla. The useful things here are:

   - bugbot only gets triggered on open bugs in Linux/Kernel that have the
     "bugbot" flag set to "+", which allows pre-triaging a bug before bugbot
     sends it to the mailing list
   - there is a special "cf_subsystem" field added to that component; bugbot
     will match it verbatim to the MAINTAINERS file components and will send
     mail to the M: and L: addresses for that subsystem
   - bugbot will then monitor for any updates to that thread, so any mailing
     list replies will be added as comments to the bug (thus sending a
     bugzilla mail notification to the initial author)
   - if the cf_subsystem is incorrect, it can be changed to a different
     subsystem at any time, triggering a different set of recipients on the
     next update
   - any files added in bugzilla will trigger notification sent to the list
     (but won't attach the actual files, so they don't clog everyone's mailbox
     or list archives)

   Artem, I'm not sure if you're still active with bugzilla.kernel.org, but I
   believe the cf_subsystem functionality is pretty much what you wanted so it
   was possible to bring bugs to the attention of actual subsystem
   maintainers.

   Here is a bug that was turned into a mailing list thread via cf_subsystem:
   https://bugzilla.kernel.org/show_bug.cgi?id=217293

Like I said, bugbot is very young and probably full of bugs, so it will still
see a lot of change and will likely explode a couple of times. The code behind
it is here, woefully lacking any documentation as yet:
https://git.kernel.org/pub/scm/utils/peebz/peebz.git/

At this time, I invite anyone with an account on bugzilla and "editbugs" group
membership can start using bugbot in either direction and to report any bugs
you find to the tools list.

Next features on my todo list are:

- documentation
- tracking repositories and recording when bugs are mentioned in commits
- whatever other feedback I receive

Best regards,
Konstantin

Re: Introducing bugbot

[Kernel.org Bugbot]

Hello:

This conversation is now tracked by Kernel.org Bugzilla:
https://bugzilla.kernel.org/show_bug.cgi?id=217297

There is no need to do anything else, just keep talking.
-- 
Deet-doot-dot, I am a bot.
Kernel.org Bugzilla (peebz 0.1)

Re: Introducing bugbot

[Konstantin Ryabitsev]

On Mon, Apr 03, 2023 at 09:47:49PM +0000, Kernel.org Bugbot wrote:
> Hello:
> 
> This conversation is now tracked by Kernel.org Bugzilla:
> https://bugzilla.kernel.org/show_bug.cgi?id=217297
> 
> There is no need to do anything else, just keep talking.

Thank you for illustrating the "may explode" bits. :)

Unfortunately, triggering on key phrases means it gets triggered when you're
discussing key phrases. I'll make sure that this only happens when the key
phrase is on the line by itself.

-K

Re: Introducing bugbot

[Thorsten Leemhuis]

On 03.04.23 23:45, Konstantin Ryabitsev wrote:
> 
> Per our discussion last fall, I've been working on a bridge between bugzilla
> and public-inbox, which went live in the past few days. It's still a very
> early release, but it can do the following:

thx for working on this.

>    New bugs will be created in the Linux/Kernel component of Bugzilla, but
>    they can then be reassigned to any other component. As long as they remain
>    open, bugbot will continue tracking threads for new messages until either
>    the bug is closed, or it has been over 30 days since any activity on the
>    bug.

30 days from my experience sounds too short for me (but I have no hard
numbers to back this up). I would have gone for 90 days, but I guess
that would create too much load?

> [...]
> 2. Start mailing list threads from pre-triaged bugzilla bugs. This works the
>    opposite way and creates mailing list threads based on bug reports filed in
>    bugzilla. The useful things here are:
> 
>    - bugbot only gets triggered on open bugs in Linux/Kernel that have the
>      "bugbot" flag set to "+", which allows pre-triaging a bug before bugbot
>      sends it to the mailing list
>    - there is a special "cf_subsystem" field added to that component; bugbot
>      will match it verbatim to the MAINTAINERS file components and will send
>      mail to the M: and L: addresses for that subsystem
>    - bugbot will then monitor for any updates to that thread, so any mailing
>      list replies will be added as comments to the bug (thus sending a
>      bugzilla mail notification to the initial author)
> [...]

Would be great if bugbot could CC the regressions list for all bugs that
have the regressions flag set to "yes"

How does one make bugbot forward a bug to everyone in the signed-off-by
chain of a commit that caused a regression, which might or might not
have a bugzilla account?  When the reporter performed a bisection these
are the primary contacts that need to be reached out to, but the
maintainers and lists for the subsystem obviously should be CCed.

Ciao, Thorsten

Re: Introducing bugbot

[Geert Uytterhoeven]

On Tue, Apr 4, 2023 at 8:03 AM Thorsten Leemhuis <linux@leemhuis.info> wrote:
> On 03.04.23 23:45, Konstantin Ryabitsev wrote:
> > Per our discussion last fall, I've been working on a bridge between bugzilla
> > and public-inbox, which went live in the past few days. It's still a very
> > early release, but it can do the following:
>
> thx for working on this.

Thx indeed, looks nice!

> >    New bugs will be created in the Linux/Kernel component of Bugzilla, but
> >    they can then be reassigned to any other component. As long as they remain
> >    open, bugbot will continue tracking threads for new messages until either
> >    the bug is closed, or it has been over 30 days since any activity on the
> >    bug.
>
> 30 days from my experience sounds too short for me (but I have no hard
> numbers to back this up). I would have gone for 90 days, but I guess
> that would create too much load?

30 < release cycle (9..10 weeks) < 90, so 90 days sounds better to me.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Introducing bugbot

[Bagas Sanjaya]

[-- Attachment #1: Type: text/plain, Size: 572 bytes --]

On Tue, Apr 04, 2023 at 07:58:40AM +0200, Thorsten Leemhuis wrote:
> How does one make bugbot forward a bug to everyone in the signed-off-by
> chain of a commit that caused a regression, which might or might not
> have a bugzilla account?  When the reporter performed a bisection these
> are the primary contacts that need to be reached out to, but the
> maintainers and lists for the subsystem obviously should be CCed.

The logic should be similar as on finding commit signers in
get_maintainer.pl.

-- 
An old man doll... just what I always wanted! - Clara

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: Introducing bugbot

[Konstantin Ryabitsev]

On Tue, Apr 04, 2023 at 07:58:40AM +0200, Thorsten Leemhuis wrote:
> Would be great if bugbot could CC the regressions list for all bugs that
> have the regressions flag set to "yes"

I'll see if I can add this logic and make it configurable.

> How does one make bugbot forward a bug to everyone in the signed-off-by
> chain of a commit that caused a regression, which might or might not
> have a bugzilla account?  When the reporter performed a bisection these
> are the primary contacts that need to be reached out to, but the
> maintainers and lists for the subsystem obviously should be CCed.

I can add a cf_bisected_commit field that can take a commit-id. When gathering
recipients, we can check if we know that commit in one of the tracked
repositories, and get the author field out of it. Would that be sufficient, or
do we need to run get_maintainer on it for a full picture?

Note, that you can also add addresses to Cc: on the thread -- they will be
collected behind the scenes as each message is processed and any bugbot
notifications originating from the bugzilla side will also include them.

-K

Re: Introducing bugbot

[Thorsten Leemhuis]

On 04.04.23 14:36, Konstantin Ryabitsev wrote:
> On Tue, Apr 04, 2023 at 07:58:40AM +0200, Thorsten Leemhuis wrote:
>> Would be great if bugbot could CC the regressions list for all bugs that
>> have the regressions flag set to "yes"
> I'll see if I can add this logic and make it configurable.

great, thx!

>> How does one make bugbot forward a bug to everyone in the signed-off-by
>> chain of a commit that caused a regression, which might or might not
>> have a bugzilla account?  When the reporter performed a bisection these
>> are the primary contacts that need to be reached out to, but the
>> maintainers and lists for the subsystem obviously should be CCed.
> 
> I can add a cf_bisected_commit field that can take a commit-id. When gathering
> recipients, we can check if we know that commit in one of the tracked
> repositories, and get the author field out of it. Would that be sufficient,

Nearly, as mentioned above: it should be everybody in the signed-off-by
chain (as explained in reporting-issues.rst), as that will catch
co-developers and the people that handled the change on the way.

> or do we need to run get_maintainer on it for a full picture?

The normal magic you described that bugbot applies to match reports to
subsystems is also needed, as otherwise lists and co-maintainer would
not be in the loop.

> Note, that you can also add addresses to Cc: on the thread -- they will be
> collected behind the scenes as each message is processed and any bugbot
> notifications originating from the bugzilla side will also include them.

That's good to know and likely will be handy sometimes, but well, for
regular use of the bugzilla->mailing list use-case it would be slightly
annoying if you'd first have to tell bugzilla to create a ml thread,
which you then need to find to write a reply to add people to the CC.

Ciao, Thorsten

b4 having trouble parsing recipients from cover

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1629 bytes --]

Hi,

I've got a cover letter commit for a single patch with the contents:

    # Lines starting with # will be removed from the cover letter. You can
    # use them to add notes or reminders to yourself. If you want to use
    # markdown headers in your cover letter, start the line with ">#".
    
    # You can add trailers to the cover letter. Any email addresses found in
    # these trailers will be added to the addresses specified/generated
    # during the b4 send stage. You can also run "b4 prep --auto-to-cc" to
    # auto-populate the To: and Cc: trailers based on the code being
    # modified.
    
    To: Liam.Howlett@Oracle.com
    Cc: linux-mm@kvack.org
    Cc: linux-kernel@vger.kernel.org
    Signed-off-by: Mark Brown <broonie@kernel.org>
    
    Changes in v2:
    - Don't set index and last again needlessly.
    - Link to v1: https://lore.kernel.org/r/20230403-regmap-maple-walk-fine-v1-1-09512dd51e50@kernel.org
    
    --- b4-submit-tracking ---
    # This section is used internally by b4 prep for tracking purposes.
    {
      "series": {
        "revision": 2,
        "change-id": "20230403-regmap-maple-walk-fine-982bbd2dcfe5",
        "base-branch": "work",
        "prefixes": [],
        "history": {
          "v1": [
            "20230403-regmap-maple-walk-fine-v1-1-09512dd51e50@kernel.org"
          ]
        }
      }
    }

but when I try to run b4 send it says:

  Converted the branch to 1 messages
  CRITICAL: Could not find any destination addresses (try: b4 prep --auto-to-cc).

which I don't understand.  I'm using current b4 master.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: b4 having trouble parsing recipients from cover

[Konstantin Ryabitsev]

On Tue, Apr 04, 2023 at 02:26:29PM +0100, Mark Brown wrote:
> Hi,
> 
> I've got a cover letter commit for a single patch with the contents:

Does it work if you add a single line title at the top?

-K

Re: b4 having trouble parsing recipients from cover

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 363 bytes --]

On Tue, Apr 04, 2023 at 09:42:37AM -0400, Konstantin Ryabitsev wrote:
> On Tue, Apr 04, 2023 at 02:26:29PM +0100, Mark Brown wrote:

> > I've got a cover letter commit for a single patch with the contents:

> Does it work if you add a single line title at the top?

No, same behaviour (I just added a line with a couple of characters
rather than a full subject).

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: b4 having trouble parsing recipients from cover

[Konstantin Ryabitsev]

On Tue, Apr 04, 2023 at 04:03:54PM +0100, Mark Brown wrote:
> > > I've got a cover letter commit for a single patch with the contents:
> 
> > Does it work if you add a single line title at the top?
> 
> No, same behaviour (I just added a line with a couple of characters
> rather than a full subject).

Okay, you can work around this by adding --- between your trailers and the
changelog:

	Signed-off-by: Mark Brown
	---
	Changes in v2:

I'll work on returning a more sensible error here.

-K

Re: b4 having trouble parsing recipients from cover

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 473 bytes --]

On Tue, Apr 04, 2023 at 12:51:15PM -0400, Konstantin Ryabitsev wrote:
> On Tue, Apr 04, 2023 at 04:03:54PM +0100, Mark Brown wrote:

> > No, same behaviour (I just added a line with a couple of characters
> > rather than a full subject).

> Okay, you can work around this by adding --- between your trailers and the
> changelog:

> 	Signed-off-by: Mark Brown
> 	---
> 	Changes in v2:

> I'll work on returning a more sensible error here.

That seems to be working, thanks.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Introducing bugbot

[Thorsten Leemhuis]

CCing Kalle (JFYI)

On 03.04.23 23:45, Konstantin Ryabitsev wrote:
> 
> 2. Start mailing list threads from pre-triaged bugzilla bugs. This works the
>    opposite way and creates mailing list threads based on bug reports filed in
>    bugzilla. The useful things here are:
> 
>    - bugbot only gets triggered on open bugs in Linux/Kernel that have the
>      "bugbot" flag set to "+", which allows pre-triaging a bug before bugbot
>      sends it to the mailing list
> [...]

Are there any policies or best practices on how people should/are
allowed to use this? From what I can see it seems one needs to change
the Product/Component of the bug to start a thread. I wonder if a few
maintainers that are active in bugzilla might be annoyed by this, as
that might break their workflow.

Which puts me in an awkward position when I see regressions reports in
bugzilla and would like to create threads for them. Using bugbot would
be better then the manual forwards I do now, like this one:

https://lore.kernel.org/all/ed31b6fe-e73d-34af-445b-81c5c644d615@leemhuis.info/


But here I decided to *not* use bugbot, as I know Kalle sometimes is
active in bugzilla -- and thus might hate it, if I re-categorize the bug.

Ciao, Thorsten

Re: Introducing bugbot

[Kalle Valo]

Thorsten Leemhuis <linux@leemhuis.info> writes:

> CCing Kalle (JFYI)
>
> On 03.04.23 23:45, Konstantin Ryabitsev wrote:
>> 
>> 2. Start mailing list threads from pre-triaged bugzilla bugs. This works the
>>    opposite way and creates mailing list threads based on bug reports filed in
>>    bugzilla. The useful things here are:
>> 
>>    - bugbot only gets triggered on open bugs in Linux/Kernel that have the
>>      "bugbot" flag set to "+", which allows pre-triaging a bug before bugbot
>>      sends it to the mailing list
>> [...]
>
> Are there any policies or best practices on how people should/are
> allowed to use this? From what I can see it seems one needs to change
> the Product/Component of the bug to start a thread. I wonder if a few
> maintainers that are active in bugzilla might be annoyed by this, as
> that might break their workflow.
>
> Which puts me in an awkward position when I see regressions reports in
> bugzilla and would like to create threads for them. Using bugbot would
> be better then the manual forwards I do now, like this one:
>
> https://lore.kernel.org/all/ed31b6fe-e73d-34af-445b-81c5c644d615@leemhuis.info/
>
> But here I decided to *not* use bugbot, as I know Kalle sometimes is
> active in bugzilla -- and thus might hate it, if I re-categorize the bug.

Yeah, for me moving ath11k bugs away from Drivers/network-wireless
component is not really helpful. And for ath11k I try to look at all
reported bugs in bugzilla anyway, though just slowly.

While at it, I have some things on my wishlist to make my use of
bugzilla.kernel.org easier:

* A new state named UNCONFIRMED and have it as the default state for
  reported bugs. This would help triaging bugs as some of the reports
  are not valid. In other words only valid bugs would have NEW state.
  IIRC the Mozilla project did this back in the day.

* Use P3 as the default priority for the new bugs. I try to keep ath11k
  bugs in priority order but new reported bugs having P1 always messes
  up the list always.

-- 
https://patchwork.kernel.org/project/linux-wireless/list/

https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches

Re: Introducing bugbot

[Konstantin Ryabitsev]

On Wed, Apr 12, 2023 at 03:23:25PM +0300, Kalle Valo wrote:
> > But here I decided to *not* use bugbot, as I know Kalle sometimes is
> > active in bugzilla -- and thus might hate it, if I re-categorize the bug.
> 
> Yeah, for me moving ath11k bugs away from Drivers/network-wireless
> component is not really helpful. And for ath11k I try to look at all
> reported bugs in bugzilla anyway, though just slowly.

Right, I did mean to say that the bot is only limited to Linux/Kernel during
the initial tryout stage. Once I'm more confident that it's not exploding and
mostly doing the right thing, I can enable it for other components.

> While at it, I have some things on my wishlist to make my use of
> bugzilla.kernel.org easier:
> 
> * A new state named UNCONFIRMED and have it as the default state for
>   reported bugs. This would help triaging bugs as some of the reports
>   are not valid. In other words only valid bugs would have NEW state.
>   IIRC the Mozilla project did this back in the day.

This is more hairy than it looks, but I'll try to figure out what happened to
the UNCONFIRMED state in our bugzilla.

> * Use P3 as the default priority for the new bugs. I try to keep ath11k
>   bugs in priority order but new reported bugs having P1 always messes
>   up the list always.

Okay, I set the default to P3, which is nicely in the middle.

-K

Re: Introducing bugbot

[Kalle Valo]

Konstantin Ryabitsev <konstantin@linuxfoundation.org> writes:

> On Wed, Apr 12, 2023 at 03:23:25PM +0300, Kalle Valo wrote:
>
>> While at it, I have some things on my wishlist to make my use of
>> bugzilla.kernel.org easier:
>> 
>> * A new state named UNCONFIRMED and have it as the default state for
>>   reported bugs. This would help triaging bugs as some of the reports
>>   are not valid. In other words only valid bugs would have NEW state.
>>   IIRC the Mozilla project did this back in the day.
>
> This is more hairy than it looks, but I'll try to figure out what happened to
> the UNCONFIRMED state in our bugzilla.

I can only imagine how difficult bugzilla can to administer. So if it's
too hairy to get UNCONFIRMED working again don't use too much on it.
This is more like a nice to have feature, not a critical thing.

>> * Use P3 as the default priority for the new bugs. I try to keep ath11k
>>   bugs in priority order but new reported bugs having P1 always messes
>>   up the list always.
>
> Okay, I set the default to P3, which is nicely in the middle.

Nice, thanks!

-- 
https://patchwork.kernel.org/project/linux-wireless/list/

https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches

Re: Introducing bugbot

[Jakub Kicinski]

On Mon, 3 Apr 2023 17:45:31 -0400 Konstantin Ryabitsev wrote:
> Hi, all:
> 
> Per our discussion last fall, I've been working on a bridge between bugzilla
> and public-inbox, which went live in the past few days. It's still a very
> early release, but it can do the following:

What about closing the bugs once they had been fixed/triaged/proven
invalid?

Re: Introducing bugbot

[Konstantin Ryabitsev]

On Fri, May 05, 2023 at 08:52:18AM -0700, Jakub Kicinski wrote:
> > Per our discussion last fall, I've been working on a bridge between bugzilla
> > and public-inbox, which went live in the past few days. It's still a very
> > early release, but it can do the following:
> 
> What about closing the bugs once they had been fixed/triaged/proven
> invalid?

For now, bugbot can look at a branch in a git repo and close bugs when it sees
a matching "Closes:" trailer.

-K

Re: Introducing bugbot

[Jakub Kicinski]

On Fri, 5 May 2023 15:12:02 -0400 Konstantin Ryabitsev wrote:
> On Fri, May 05, 2023 at 08:52:18AM -0700, Jakub Kicinski wrote:
> > > Per our discussion last fall, I've been working on a bridge between bugzilla
> > > and public-inbox, which went live in the past few days. It's still a very
> > > early release, but it can do the following:  
> > 
> > What about closing the bugs once they had been fixed/triaged/proven
> > invalid?  
> 
> For now, bugbot can look at a branch in a git repo and close bugs when it sees
> a matching "Closes:" trailer.

Interesting. Obviously that assumes the bug needs a fix, and we're
talking bugzilla so lots of clueless people lobbing non-issues at us.

But also - Closes: is a legit trailer now? I thought we only allow Link:

In any case - I'm guessing I just don't have permissions to close the
issues via web. Is your preference to iterate on the bot or maintainers
to request permissions in BZ (how?)?

Re: Introducing bugbot

[Konstantin Ryabitsev]

On Fri, May 05, 2023 at 12:32:26PM -0700, Jakub Kicinski wrote:
> Interesting. Obviously that assumes the bug needs a fix, and we're
> talking bugzilla so lots of clueless people lobbing non-issues at us.

This is why anyone can create a bug on bugzilla, but only a subset of people
can enable bugbot on it (by setting the bugbot+ flag).

> But also - Closes: is a legit trailer now? I thought we only allow Link:

It's a semi-legit trailer and is going through -next:
https://docs.kernel.org/next/process/submitting-patches.html#using-reported-by-tested-by-reviewed-by-suggested-by-and-fixes

> In any case - I'm guessing I just don't have permissions to close the
> issues via web. Is your preference to iterate on the bot or maintainers
> to request permissions in BZ (how?)?

Yes, it's not a power granted by default, but people just need to ask.
You should have this awesome power now.

-K

Re: Introducing bugbot

[Jakub Kicinski]

On Fri, 5 May 2023 15:39:38 -0400 Konstantin Ryabitsev wrote:
> On Fri, May 05, 2023 at 12:32:26PM -0700, Jakub Kicinski wrote:
> > Interesting. Obviously that assumes the bug needs a fix, and we're
> > talking bugzilla so lots of clueless people lobbing non-issues at us.  
> 
> This is why anyone can create a bug on bugzilla, but only a subset of people
> can enable bugbot on it (by setting the bugbot+ flag).

Makes sense!

> > But also - Closes: is a legit trailer now? I thought we only allow Link:  
> 
> It's a semi-legit trailer and is going through -next:
> https://docs.kernel.org/next/process/submitting-patches.html#using-reported-by-tested-by-reviewed-by-suggested-by-and-fixes

Ah, must have come in this merge window, cause we don't have that text 
in networking trees yet. GTK.

> > In any case - I'm guessing I just don't have permissions to close the
> > issues via web. Is your preference to iterate on the bot or maintainers
> > to request permissions in BZ (how?)?  
> 
> Yes, it's not a power granted by default, but people just need to ask.
> You should have this awesome power now.

Can confirm, thanks!