Bug tracking in the run up from 2.5 to 2.6

Bug tracking in the run up from 2.5 to 2.6

[Martin J. Bligh]

We're proposing to create a bug tracking system to help keep track of
2.5 kernel bugs ... in an attempt to help get 2.5 stabilised as quickly as
possible. 

It would be based on Bugzilla, and open for anyone to log bugs against
2.5, though those would then be filtered by a set of "maintainers" to keep
the quality of the data up to snuff. Ideally those would be the subsystem
maintainers as we know them now, though in the event that certain people
weren't interested, we'd find a "bugzilla maintainer" for the subsystem to
fill that role.

IBM's Linux Technology Centre is willing to provide the machine, admin, 
and people to help maintain the data in the database. We have someone
who's kindly agreed in principle to host the machine for us (feel free to 
speak up if you like, otherwise I'll wait until the proposal is firmed up).

We'll also have a slew of engineers dedicated to stabilise 2.5 after the
freeze, but this is not intended to be solely an IBM thing by any means;
we're volunteering to host the tracking database on behalf of the community,
and do some of the dirty work of administration. The intent is to have this
up and running by Halloween, and for a signification cross-section of the 
community to use it.

So ... are the maintainers interested in working with this kind of system?
We really need to get some feedback before commiting resources to doing
this - if you'd be willing to close out bugs as you find / fix them, please let
me know. This is a web interface system, with handy email triggers, and is
very easy to use.

Feedback saying "well, it'll only be useful if you do XYZ" is welcome too.
We're very unlikely to change tools to use something other than Bugzilla
at this point, so that's not really open for debate.

Martin.

Re: Bug tracking in the run up from 2.5 to 2.6

[Timothy D. Witham]

On Thu, 2002-10-17 at 15:52, Martin J. Bligh wrote:
> We're proposing to create a bug tracking system to help keep track of
> 2.5 kernel bugs ... in an attempt to help get 2.5 stabilised as quickly as
> possible. 
> 
> It would be based on Bugzilla, and open for anyone to log bugs against
> 2.5, though those would then be filtered by a set of "maintainers" to keep
> the quality of the data up to snuff. Ideally those would be the subsystem
> maintainers as we know them now, though in the event that certain people
> weren't interested, we'd find a "bugzilla maintainer" for the subsystem to
> fill that role.
> 
> IBM's Linux Technology Centre is willing to provide the machine, admin, 
> and people to help maintain the data in the database. We have someone
> who's kindly agreed in principle to host the machine for us (feel free to 
> speak up if you like, otherwise I'll wait until the proposal is firmed up).
> 
  Thanks for asking before you used my name. :-)

  But yes OSDL would host it.

  Tim

> We'll also have a slew of engineers dedicated to stabilise 2.5 after the
> freeze, but this is not intended to be solely an IBM thing by any means;
> we're volunteering to host the tracking database on behalf of the community,
> and do some of the dirty work of administration. The intent is to have this
> up and running by Halloween, and for a signification cross-section of the 
> community to use it.
> 
> So ... are the maintainers interested in working with this kind of system?
> We really need to get some feedback before commiting resources to doing
> this - if you'd be willing to close out bugs as you find / fix them, please let
> me know. This is a web interface system, with handy email triggers, and is
> very easy to use.
> 
> Feedback saying "well, it'll only be useful if you do XYZ" is welcome too.
> We're very unlikely to change tools to use something other than Bugzilla
> at this point, so that's not really open for debate.
> 
  I would like to know this also. :-)

Tim

> Martin.
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
-- 
Timothy D. Witham - Lab Director - wookie@osdlab.org
Open Source Development Lab Inc - A non-profit corporation
15275 SW Koll Parkway - Suite H - Beaverton OR, 97006
(503)-626-2455 x11 (office)    (503)-702-2871     (cell)
(503)-626-2436     (fax)

Re: Bug tracking in the run up from 2.5 to 2.6

[Greg KH]

On Thu, Oct 17, 2002 at 03:52:05PM -0700, Martin J. Bligh wrote:
> 
> So ... are the maintainers interested in working with this kind of system?

If there are people to do the categorizing and "cleaning" of the
reported bugs, I would love to use this.  By "cleaning" I mean the
following at the minimum:
	- marking bugs as duplicates of existing bugs
	- throwing away useless bug reports

thanks,

greg k-h

Re: Bug tracking in the run up from 2.5 to 2.6

[Martin J. Bligh]

> If there are people to do the categorizing and "cleaning" of the
> reported bugs, I would love to use this.  By "cleaning" I mean the
> following at the minimum:
> 	- marking bugs as duplicates of existing bugs
> 	- throwing away useless bug reports

Right, that would be covered. The team that would be doing this
actually has people in various locations round the world (ie 
various timezones), and is used to doing this for our internal
Bugzilla database.

M.

Re: Bug tracking in the run up from 2.5 to 2.6

[Zwane Mwaikambo]

On Thu, 17 Oct 2002, Martin J. Bligh wrote:

> > If there are people to do the categorizing and "cleaning" of the
> > reported bugs, I would love to use this.  By "cleaning" I mean the
> > following at the minimum:
> > 	- marking bugs as duplicates of existing bugs
> > 	- throwing away useless bug reports

Sounds great, how do we handle duplicate work? ie will there be bug 
assignments ?

	Zwane
-- 
function.linuxpower.ca

Re: Bug tracking in the run up from 2.5 to 2.6

[Martin J. Bligh]

>> > If there are people to do the categorizing and "cleaning" of the
>> > reported bugs, I would love to use this.  By "cleaning" I mean the
>> > following at the minimum:
>> > 	- marking bugs as duplicates of existing bugs
>> > 	- throwing away useless bug reports
> 
> Sounds great, how do we handle duplicate work? ie will there be bug 
> assignments ?

Obvious duplicates will be filtered out by the general admins.
More subtle type things will probably need to be by the "owners"
of the bugs, ie yes ... we'll have some sort of assingment of bugs.
Ideally I'd like this to just be the subsystem maintainers as we
have them now, but if they don't want to play, we'll find someone
to act as administrative owner for each technical area that has
experience in that field.

Martin.

Re: Bug tracking in the run up from 2.5 to 2.6

[mgross]

On Thursday 17 October 2002 07:02 pm, Timothy D. Witham wrote:
> On Thu, 2002-10-17 at 15:52, Martin J. Bligh wrote:
> > We're proposing to create a bug tracking system to help keep track of
> > 2.5 kernel bugs ... in an attempt to help get 2.5 stabilised as quickly
> > as possible.
> > 


Defect tracking works best when used over a significant portion of the 
product life cycle.  In this case this bug data base will be of more use and 
value to 2.6 than anything else.  You'll need to be patient and persistant. I 
think something good has a chance of coming out of this.

Prioritizing bugs and bug fix activities is a political and user / market 
driven activity.  It'll be interesting to watch this shake out.  I wish you 
all the best of luck.

Defect tracking and fixing is a "push" activity.  The owners of 
the bug data base and "the product" along with input from key stake holders, 
dictate the priority and owners of issues (pushes the responsibility onto an 
owner).  The owners then go and make fixes.  This tends to be effective in 
hierarchical organizations.  

The Linux  development model is more of a pull model.  Someone finds a bug 
takes it upon them selves to fix it (pulls the responsibility onto 
themselves), and then attempts to get the fix accepted.  If bugs in some 
areas sit around too long it becomes an opportunity for someone new to step 
up.

Perhaps if you set up the bug queries as a type of advertisement of the "most 
wanted bugs"  that aren't getting  attention could proved a nice place 
for folks to look if they are interested in getting involved / helping out.  
It could also become a type of "wall of shame".  You'll need to be carefull 
to keep things positive.

--mgross

Re: Bug tracking in the run up from 2.5 to 2.6

[Adrian Bunk]

On Thu, 17 Oct 2002, Martin J. Bligh wrote:

>...
> We really need to get some feedback before commiting resources to doing
> this - if you'd be willing to close out bugs as you find / fix them, please let
> me know. This is a web interface system, with handy email triggers, and is
> very easy to use.
>
> Feedback saying "well, it'll only be useful if you do XYZ" is welcome too.
> We're very unlikely to change tools to use something other than Bugzilla
> at this point, so that's not really open for debate.


First of all thanks for this, it sounds like a pretty good idea. :-)


Some comments/questions:


1. Related projects

1a Some time ago Dave Jones did something similar [1] and currently
   Thomas Molina maintains a bug tracking page [2]. These were/are both
   projects where one person did/does collect bug reports from
   linux-kernel by hand but roughly spoken it's the same thing you want to
   start (modulo the additional manpower you can provide). You should
   at least ask them for their experiences and use Thomas' collection
   as input for your bug tracking system.

1b The Trivial Patch Monkey [3] by Rusty Russell is not very related
   but it would be good if there was enough communication that it's noted
   in the bug tracking system whenever a bug is already fixed by a patch
   submitted to the Trivial Patch Monkey.


2. What is the suggested workflow?

   Let's say I want to report a "xyz doesn't compile in 2.5.44" bug.

   Currently I'm sending it to the people that seem to be responsible
   (looking at MAINTAINERS and the contents of the file that doesn't
   compile) with a Cc to linux-kernel.

   With a Web-based bug tracking system like Bugzilla it's additional work
   to add a bug to the bug tracking system, too, and to keep the
   information that is there updated (e.g. if the maintainer sends a patch
   a few hours later).

   What is the suggested workflow to avoid additional work?


> Martin.

cu
Adrian

[1] http://www.codemonkey.org.uk/Linux-2.5.html
[2] http://members.cox.net/tmolina/kernprobs/status.html
[3] http://www.kernel.org/pub/linux/kernel/people/rusty/trivial/

-- 

"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
                                Pearl S. Buck - Dragon Seed

Re: Bug tracking in the run up from 2.5 to 2.6

[Martin J. Bligh]

> 1. Related projects
> 
> 1a Some time ago Dave Jones did something similar [1] and currently
>    Thomas Molina maintains a bug tracking page [2]. These were/are both
>    projects where one person did/does collect bug reports from
>    linux-kernel by hand but roughly spoken it's the same thing you want to
>    start (modulo the additional manpower you can provide). You should
>    at least ask them for their experiences and use Thomas' collection
>    as input for your bug tracking system.

I talked to Thomas already, hadn't seen Davej's list, thanks for that.
I guess the main difference with an full bug tracking system rather
than a web page or text email is that it's easier to distribute the
workload around. I think this is too much for any one person to maintain,
hence the problem with these things getting out of date.

> 1b The Trivial Patch Monkey [3] by Rusty Russell is not very related
>    but it would be good if there was enough communication that it's noted
>    in the bug tracking system whenever a bug is already fixed by a patch
>    submitted to the Trivial Patch Monkey.

Right, some sort of linkage would be needed with both this and the
main Linus tree for checkins - I think the latter is actually a more
major concern ... bugs should be closed when they're (confirmed) fixed
in the mail tree.

> 2. What is the suggested workflow?
> 
>    Let's say I want to report a "xyz doesn't compile in 2.5.44" bug.
> 
>    Currently I'm sending it to the people that seem to be responsible
>    (looking at MAINTAINERS and the contents of the file that doesn't
>    compile) with a Cc to linux-kernel.
> 
>    With a Web-based bug tracking system like Bugzilla it's additional work
>    to add a bug to the bug tracking system, too, and to keep the
>    information that is there updated (e.g. if the maintainer sends a patch
>    a few hours later).
> 
>    What is the suggested workflow to avoid additional work?

Systems like this will normally involve a little additional overhead,
but I think the payback is worthwhile. Bugzilla has email triggers, 
so it would be easy to set up bugs filed against any particular
category to be automatically emailed to the maintainer(s) for that 
category. The bug could then be sanity checked, and sent to lkml.
Or something like that ;-)

M.

Re: Bug tracking in the run up from 2.5 to 2.6

[Arnaldo Carvalho de Melo]

Em Thu, Oct 17, 2002 at 04:15:01PM -0700, Greg KH escreveu:
> On Thu, Oct 17, 2002 at 03:52:05PM -0700, Martin J. Bligh wrote:
> > 
> > So ... are the maintainers interested in working with this kind of system?
> 
> If there are people to do the categorizing and "cleaning" of the
> reported bugs, I would love to use this.  By "cleaning" I mean the
> following at the minimum:
> 	- marking bugs as duplicates of existing bugs
> 	- throwing away useless bug reports

/me too :-)

- Arnaldo

Yay, bug tracking! (was Re: Bug tracking in the run up from 2.5 to 2.6)

[Jeff Garzik]

Martin J. Bligh wrote:
> We're proposing to create a bug tracking system to help keep track of
> 2.5 kernel bugs ... in an attempt to help get 2.5 stabilised as quickly as
> possible. 

> It would be based on Bugzilla, and open for anyone to log bugs against
> 2.5, though those would then be filtered by a set of "maintainers" to keep
> the quality of the data up to snuff. Ideally those would be the subsystem
> maintainers as we know them now, though in the event that certain people
> weren't interested, we'd find a "bugzilla maintainer" for the subsystem to
> fill that role.

This is fantastic.

This is one of the things I think the Linux kernel sorely lacks, both 
image-wise and practically-speaking.  Ted T'so maintained a bug list for 
a while, and it was (a) valuable and (b) way too much work for one human 
to possibly handle.


> IBM's Linux Technology Centre is willing to provide the machine, admin, 
> and people to help maintain the data in the database. We have someone
> who's kindly agreed in principle to host the machine for us (feel free to 
> speak up if you like, otherwise I'll wait until the proposal is firmed up).

Again, fantastic.  As was discussed at the original Kernel Summit (and 
perhaps the latest one as well?), everyone knows we need a bug tracking 
system, but without humans to help keep the garbage bug reports to a 
minimum, the system won't be used by developers _or_ users with 
legitimate problems.


> We'll also have a slew of engineers dedicated to stabilise 2.5 after the
> freeze, but this is not intended to be solely an IBM thing by any means;
> we're volunteering to host the tracking database on behalf of the community,
> and do some of the dirty work of administration. The intent is to have this
> up and running by Halloween, and for a signification cross-section of the 
> community to use it.
> 
> So ... are the maintainers interested in working with this kind of system?

I certainly am.  Even though I claim to be net driver maintainer, I 
don't begin to claim that I can track and fix _all_ bugs in _all_ net 
drivers, even though I endeavor to do the best I can.

This Bugzilla system, I believe, should help a great deal.

I'm definitely willing to give it an honest shot with net drivers, and I 
hope that we can persuade others to do so as well.

One suggestion - it might be nice for IBM to register "linuxbugs.org" or 
bugzilla.kernel.org or similar, so to provide an additional impression 
of impartiality.  I don't mind "Hosted by IBM!" plastered all over the 
site, but if Bugzilla ever needs to move, that makes things much easier.

	Jeff

Re: Yay, bug tracking! (was Re: Bug tracking in the run up from 2.5 to 2.6)

[Rik van Riel]

On Mon, 21 Oct 2002, Jeff Garzik wrote:
> Martin J. Bligh wrote:
> > We're proposing to create a bug tracking system to help keep track of
> > 2.5 kernel bugs ... in an attempt to help get 2.5 stabilised as quickly as
> > possible.
>
> This is fantastic.

<AOL/>

Count me in as a participant.  One thing that would be nice is
a mailing list where the bugs (and the changes to bugs) get
mailed, so we can get updates by email.

I'll help track down, fix and admin bugs.

regards,

Rik
-- 
Bravely reimplemented by the knights who say "NIH".
http://www.surriel.com/		http://distro.conectiva.com/
Current spamtrap:  <a href=mailto:"october@surriel.com">october@surriel.com</a>

Re: Yay, bug tracking! (was Re: Bug tracking in the run up from 2.5 to 2.6)

[Martin J. Bligh]

> Count me in as a participant.  One thing that would be nice is
> a mailing list where the bugs (and the changes to bugs) get
> mailed, so we can get updates by email.

Bugzilla has email triggers, so this should be dead easy to do.
Not sure if we need a list - bugzilla should be able to keep
each people's "watching" criteria itself. If it turns out lists
are easier, that should be easy to fix.
 
> I'll help track down, fix and admin bugs.

Thanks (to both of you ;-))!

M.

Re: Yay, bug tracking! (was Re: Bug tracking in the run up from 2.5 to 2.6)

[Jeff Garzik]

Alan Cox wrote:
> Bug reporting systems need maintenance or they collapse

Agreed, and IBM said that they would provide people doing this... 
otherwise I would not have been so enthusiastic :)

Re: Yay, bug tracking! (was Re: Bug tracking in the run up from 2.5 to 2.6)

[Alan Cox]

On Tue, 2002-10-22 at 02:43, Martin J. Bligh wrote:
> > Count me in as a participant.  One thing that would be nice is
> > a mailing list where the bugs (and the changes to bugs) get
> > mailed, so we can get updates by email.
> 
> Bugzilla has email triggers, so this should be dead easy to do.
> Not sure if we need a list - bugzilla should be able to keep
> each people's "watching" criteria itself. If it turns out lists
> are easier, that should be easy to fix.
>  
> > I'll help track down, fix and admin bugs.
> 
> Thanks (to both of you ;-))!

>From some good and bad experiences in the desktop universe - you need
people who are willing to collate bugs, sort them, tidy duplicates,
identify the most common bug report etc. Without that it doesn't work.

Watching what Luis did to the gnome bug reporting has been an education
but I don't think he can be cloned trivially and I don't think we could
run off with him 8)

Bug reporting systems need maintenance or they collapse

Re: Yay, bug tracking! (was Re: Bug tracking in the run up from 2.5 to 2.6)

[Cliff White]

> Alan Cox wrote:
> > Bug reporting systems need maintenance or they collapse
> 
> Agreed, and IBM said that they would provide people doing this... 
> otherwise I would not have been so enthusiastic :)
> 
OSDL is very interested in linking this with the PLM - since the 
PLM is doing automated kernel builds with multiple .configs, we can
harvest build bugs from there.  I am also willing to help with some of the
maintenance and sorting work. 

Martin, i left you a message. 
cliffw
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>