Bug reporting and good bug reports

Bug reporting and good bug reports

[Richard Purdie]

I was informed on irc yesterday that bug reports are hard and that
debugging via irc is easier. I think I need to remind people why good
bug reports are important and how they do actually help immensely.

I do actually believe that not everything has to have bug report. If you
mention an issue, someone says "hey, I know how to fix that" and sends a
patch out, a bug report is wasted overhead IMO. I know some programme
managers who disagree strongly with me and would suggest *every* bugfix
commit should have a defect tracking item. We're not going there I
understand the idea, its not practical.

That said, if its not immediately clear what the problem is, it should
become a bug report. Why?

Firstly, the random selection of people on irc at the time probably
aren't the right people to fix it. Telling those people to read 48 hours
of irc log for the details is disrespectful of their time.

It also happens that the first people referred to a bug may not be the
person who actually can fix it. If someone else needs to come to a bug,
having a summary of the issue, the salient facts and the current status
is immensley useful for handover.

As a case in point, I tried to debug a qt4 build failure yesterday for
which there is no bug report. I lost hours building the wrong things,
experimenting to try and find the reproducer steps and generally chasing
my tail, losing the autobuilder log of the failure, the name of the qt4
recipe that was failing (which task was it again?) and so on.

I do now have a set of reproducer steps, its quite simple:

MACHINE=imx53qsb bitbake virtual/kernel
MACHINE=imx53qsb bitbake virtual/kernel -c clean
MACHINE=imx53qsb bitbake qt4-x11-free

I also have a patch. Where should I share them? How do I ensure everyone
with an interest in this defect actually gets the patch? Sure I can
create email and send to the people who I think need to know. The
bugzilla lets interested parties add themselves to bugs though.

I should also note that QA actually go through bugs in the bugzilla,
including closed ones, looking for test cases. We're not great at this
yet but it does happen. If there is a well documented test case like
that above, we might write an automated QA test for it. Having it
documented is therefore a good thing.

I do appreciate writing a bug report is hard, especially if you don't
know where the problem is, or how to reproduce it exactly. It takes time
and effort. You can however document what you know and discussion can
happen in a common place to figure out how to reproduce it. I do except
the submitters to fully understand the bug, if they did they'd probably
write a patch instead.

So fair warning, I am going to start ignoring things on irc and ask for
bug numbers in future, assuming something isn't a 5 minute fix with an
immediate patch. I will back and encourage anyone else doing this too.

Cheers,

Richard

Re: Bug reporting and good bug reports

[Trevor Woerner]

On 01/07/15 04:25, Richard Purdie wrote:
> I also have a patch. Where should I share them? How do I ensure everyone
> with an interest in this defect actually gets the patch? Sure I can
> create email and send to the people who I think need to know.

When I went through that whole "let's add a MAINTAINERS file" thing last
year this is exactly what I had in mind at that time. A developer could
configure git to automatically invoke the script that accompanied that
patch during a "git send-email ...". The purpose of the script was to go
through the MAINTAINERS file to create a list of relevant people. But
the script was also smart enough to look at the lines of code the patch
was modifying and include the people who wrote the lines the patch was
going to modify.

Unfortunately everyone seemed to think a MAINTAINERS file was only good
for assigning responsibility, which is something that had never occurred
to me at the time :-)

> The
> bugzilla lets interested parties add themselves to bugs though.

...and people could add themselves as "R"'s in a MAINTAINERS file! :-)
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/MAINTAINERS#n73

Re: [yocto] Bug reporting and good bug reports

[Trevor Woerner]

On 01/07/15 04:25, Richard Purdie wrote:
> I also have a patch. Where should I share them? How do I ensure everyone
> with an interest in this defect actually gets the patch? Sure I can
> create email and send to the people who I think need to know.

When I went through that whole "let's add a MAINTAINERS file" thing last
year this is exactly what I had in mind at that time. A developer could
configure git to automatically invoke the script that accompanied that
patch during a "git send-email ...". The purpose of the script was to go
through the MAINTAINERS file to create a list of relevant people. But
the script was also smart enough to look at the lines of code the patch
was modifying and include the people who wrote the lines the patch was
going to modify.

Unfortunately everyone seemed to think a MAINTAINERS file was only good
for assigning responsibility, which is something that had never occurred
to me at the time :-)

> The
> bugzilla lets interested parties add themselves to bugs though.

...and people could add themselves as "R"'s in a MAINTAINERS file! :-)
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/MAINTAINERS#n73

Re: Bug reporting and good bug reports

[Richard Purdie]

On Wed, 2015-01-07 at 16:21 -0500, Trevor Woerner wrote:
> On 01/07/15 04:25, Richard Purdie wrote:
> > I also have a patch. Where should I share them? How do I ensure everyone
> > with an interest in this defect actually gets the patch? Sure I can
> > create email and send to the people who I think need to know.
> 
> When I went through that whole "let's add a MAINTAINERS file" thing last
> year this is exactly what I had in mind at that time.

But that isn't what I'm talking about.

I'm talking about people being able to say "I have some interest in a
particular defect and I'd like updates about it". That is completely
different to a MAINTAINERS file. The user and easily opt in to specific
things using the web UI and its self maintaining to a large degree.

Cheers,

Richard

Re: [yocto] Bug reporting and good bug reports

[Richard Purdie]

On Wed, 2015-01-07 at 16:21 -0500, Trevor Woerner wrote:
> On 01/07/15 04:25, Richard Purdie wrote:
> > I also have a patch. Where should I share them? How do I ensure everyone
> > with an interest in this defect actually gets the patch? Sure I can
> > create email and send to the people who I think need to know.
> 
> When I went through that whole "let's add a MAINTAINERS file" thing last
> year this is exactly what I had in mind at that time.

But that isn't what I'm talking about.

I'm talking about people being able to say "I have some interest in a
particular defect and I'd like updates about it". That is completely
different to a MAINTAINERS file. The user and easily opt in to specific
things using the web UI and its self maintaining to a large degree.

Cheers,

Richard

Re: [OE-core] Bug reporting and good bug reports

[Khem Raj]

> On Jan 7, 2015, at 1:25 AM, Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
> 
> I was informed on irc yesterday that bug reports are hard and that
> debugging via irc is easier. I think I need to remind people why good
> bug reports are important and how they do actually help immensely.
> 
> I do actually believe that not everything has to have bug report. If you
> mention an issue, someone says "hey, I know how to fix that" and sends a
> patch out, a bug report is wasted overhead IMO. I know some programme
> managers who disagree strongly with me and would suggest *every* bugfix
> commit should have a defect tracking item. We're not going there I
> understand the idea, its not practical.
> 
> That said, if its not immediately clear what the problem is, it should
> become a bug report. Why?
> 
> Firstly, the random selection of people on irc at the time probably
> aren't the right people to fix it. Telling those people to read 48 hours
> of irc log for the details is disrespectful of their time.
> 
> It also happens that the first people referred to a bug may not be the
> person who actually can fix it. If someone else needs to come to a bug,
> having a summary of the issue, the salient facts and the current status
> is immensley useful for handover.
> 
> As a case in point, I tried to debug a qt4 build failure yesterday for
> which there is no bug report. I lost hours building the wrong things,
> experimenting to try and find the reproducer steps and generally chasing
> my tail, losing the autobuilder log of the failure, the name of the qt4
> recipe that was failing (which task was it again?) and so on.
> 
> I do now have a set of reproducer steps, its quite simple:
> 
> MACHINE=imx53qsb bitbake virtual/kernel
> MACHINE=imx53qsb bitbake virtual/kernel -c clean
> MACHINE=imx53qsb bitbake qt4-x11-free
> 
> I also have a patch. Where should I share them? How do I ensure everyone
> with an interest in this defect actually gets the patch? Sure I can
> create email and send to the people who I think need to know. The
> bugzilla lets interested parties add themselves to bugs though.
> 
> I should also note that QA actually go through bugs in the bugzilla,
> including closed ones, looking for test cases. We're not great at this
> yet but it does happen. If there is a well documented test case like
> that above, we might write an automated QA test for it. Having it
> documented is therefore a good thing.
> 
> I do appreciate writing a bug report is hard, especially if you don't
> know where the problem is, or how to reproduce it exactly. It takes time
> and effort. You can however document what you know and discussion can
> happen in a common place to figure out how to reproduce it. I do except
> the submitters to fully understand the bug, if they did they'd probably
> write a patch instead.
> 
> So fair warning, I am going to start ignoring things on irc and ask for
> bug numbers in future, assuming something isn't a 5 minute fix with an
> immediate patch. I will back and encourage anyone else doing this too.

What about developer mailing lists ?. isn’t it also a way to report problems via emails after all
we use emails for patch work flow. Not all people working with OE-Core e.g. might be following yocto bugzilla

Re: Bug reporting and good bug reports

[Khem Raj]

> On Jan 7, 2015, at 1:25 AM, Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
> 
> I was informed on irc yesterday that bug reports are hard and that
> debugging via irc is easier. I think I need to remind people why good
> bug reports are important and how they do actually help immensely.
> 
> I do actually believe that not everything has to have bug report. If you
> mention an issue, someone says "hey, I know how to fix that" and sends a
> patch out, a bug report is wasted overhead IMO. I know some programme
> managers who disagree strongly with me and would suggest *every* bugfix
> commit should have a defect tracking item. We're not going there I
> understand the idea, its not practical.
> 
> That said, if its not immediately clear what the problem is, it should
> become a bug report. Why?
> 
> Firstly, the random selection of people on irc at the time probably
> aren't the right people to fix it. Telling those people to read 48 hours
> of irc log for the details is disrespectful of their time.
> 
> It also happens that the first people referred to a bug may not be the
> person who actually can fix it. If someone else needs to come to a bug,
> having a summary of the issue, the salient facts and the current status
> is immensley useful for handover.
> 
> As a case in point, I tried to debug a qt4 build failure yesterday for
> which there is no bug report. I lost hours building the wrong things,
> experimenting to try and find the reproducer steps and generally chasing
> my tail, losing the autobuilder log of the failure, the name of the qt4
> recipe that was failing (which task was it again?) and so on.
> 
> I do now have a set of reproducer steps, its quite simple:
> 
> MACHINE=imx53qsb bitbake virtual/kernel
> MACHINE=imx53qsb bitbake virtual/kernel -c clean
> MACHINE=imx53qsb bitbake qt4-x11-free
> 
> I also have a patch. Where should I share them? How do I ensure everyone
> with an interest in this defect actually gets the patch? Sure I can
> create email and send to the people who I think need to know. The
> bugzilla lets interested parties add themselves to bugs though.
> 
> I should also note that QA actually go through bugs in the bugzilla,
> including closed ones, looking for test cases. We're not great at this
> yet but it does happen. If there is a well documented test case like
> that above, we might write an automated QA test for it. Having it
> documented is therefore a good thing.
> 
> I do appreciate writing a bug report is hard, especially if you don't
> know where the problem is, or how to reproduce it exactly. It takes time
> and effort. You can however document what you know and discussion can
> happen in a common place to figure out how to reproduce it. I do except
> the submitters to fully understand the bug, if they did they'd probably
> write a patch instead.
> 
> So fair warning, I am going to start ignoring things on irc and ask for
> bug numbers in future, assuming something isn't a 5 minute fix with an
> immediate patch. I will back and encourage anyone else doing this too.

What about developer mailing lists ?. isn’t it also a way to report problems via emails after all
we use emails for patch work flow. Not all people working with OE-Core e.g. might be following yocto bugzilla

Re: [OE-core] Bug reporting and good bug reports

[Paul Eggleton]

On Wednesday 07 January 2015 16:36:37 Khem Raj wrote:
> > On Jan 7, 2015, at 1:25 AM, Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> > 
> > I was informed on irc yesterday that bug reports are hard and that
> > debugging via irc is easier. I think I need to remind people why good
> > bug reports are important and how they do actually help immensely.
> > 
> > I do actually believe that not everything has to have bug report. If you
> > mention an issue, someone says "hey, I know how to fix that" and sends a
> > patch out, a bug report is wasted overhead IMO. I know some programme
> > managers who disagree strongly with me and would suggest *every* bugfix
> > commit should have a defect tracking item. We're not going there I
> > understand the idea, its not practical.
> > 
> > That said, if its not immediately clear what the problem is, it should
> > become a bug report. Why?
> > 
> > Firstly, the random selection of people on irc at the time probably
> > aren't the right people to fix it. Telling those people to read 48 hours
> > of irc log for the details is disrespectful of their time.
> > 
> > It also happens that the first people referred to a bug may not be the
> > person who actually can fix it. If someone else needs to come to a bug,
> > having a summary of the issue, the salient facts and the current status
> > is immensley useful for handover.
> > 
> > As a case in point, I tried to debug a qt4 build failure yesterday for
> > which there is no bug report. I lost hours building the wrong things,
> > experimenting to try and find the reproducer steps and generally chasing
> > my tail, losing the autobuilder log of the failure, the name of the qt4
> > recipe that was failing (which task was it again?) and so on.
> > 
> > I do now have a set of reproducer steps, its quite simple:
> > 
> > MACHINE=imx53qsb bitbake virtual/kernel
> > MACHINE=imx53qsb bitbake virtual/kernel -c clean
> > MACHINE=imx53qsb bitbake qt4-x11-free
> > 
> > I also have a patch. Where should I share them? How do I ensure everyone
> > with an interest in this defect actually gets the patch? Sure I can
> > create email and send to the people who I think need to know. The
> > bugzilla lets interested parties add themselves to bugs though.
> > 
> > I should also note that QA actually go through bugs in the bugzilla,
> > including closed ones, looking for test cases. We're not great at this
> > yet but it does happen. If there is a well documented test case like
> > that above, we might write an automated QA test for it. Having it
> > documented is therefore a good thing.
> > 
> > I do appreciate writing a bug report is hard, especially if you don't
> > know where the problem is, or how to reproduce it exactly. It takes time
> > and effort. You can however document what you know and discussion can
> > happen in a common place to figure out how to reproduce it. I do except
> > the submitters to fully understand the bug, if they did they'd probably
> > write a patch instead.
> > 
> > So fair warning, I am going to start ignoring things on irc and ask for
> > bug numbers in future, assuming something isn't a 5 minute fix with an
> > immediate patch. I will back and encourage anyone else doing this too.
> 
> What about developer mailing lists ?. isn’t it also a way to report problems
> via emails after all we use emails for patch work flow. Not all people
> working with OE-Core e.g. might be following yocto bugzilla

Mailing lists are great for discussion (e.g. "I have this problem but I'm not 
sure of the right way to solve it") but a terrible way to track actual bugs, 
because mailing lists tend to concentrate on the latest postings; older issues 
that haven't been resolved rapidly disappear with the tide of new postings. Of 
course old issues can build up in a bug tracker, but at least it's trivially 
easy to find open issues where it's much more difficult to find unresolved issues 
on a mailing list.

The point is, the best way to ensure that an issue gets solved at least for 
OE-Core is to file a bug in the YP bugzilla. There is then a triage process and 
in most cases someone to actually assign the issue to so that it can be dealt 
with. There is no such process on the mailing lists.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Bug reporting and good bug reports

[Paul Eggleton]

On Wednesday 07 January 2015 16:36:37 Khem Raj wrote:
> > On Jan 7, 2015, at 1:25 AM, Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> > 
> > I was informed on irc yesterday that bug reports are hard and that
> > debugging via irc is easier. I think I need to remind people why good
> > bug reports are important and how they do actually help immensely.
> > 
> > I do actually believe that not everything has to have bug report. If you
> > mention an issue, someone says "hey, I know how to fix that" and sends a
> > patch out, a bug report is wasted overhead IMO. I know some programme
> > managers who disagree strongly with me and would suggest *every* bugfix
> > commit should have a defect tracking item. We're not going there I
> > understand the idea, its not practical.
> > 
> > That said, if its not immediately clear what the problem is, it should
> > become a bug report. Why?
> > 
> > Firstly, the random selection of people on irc at the time probably
> > aren't the right people to fix it. Telling those people to read 48 hours
> > of irc log for the details is disrespectful of their time.
> > 
> > It also happens that the first people referred to a bug may not be the
> > person who actually can fix it. If someone else needs to come to a bug,
> > having a summary of the issue, the salient facts and the current status
> > is immensley useful for handover.
> > 
> > As a case in point, I tried to debug a qt4 build failure yesterday for
> > which there is no bug report. I lost hours building the wrong things,
> > experimenting to try and find the reproducer steps and generally chasing
> > my tail, losing the autobuilder log of the failure, the name of the qt4
> > recipe that was failing (which task was it again?) and so on.
> > 
> > I do now have a set of reproducer steps, its quite simple:
> > 
> > MACHINE=imx53qsb bitbake virtual/kernel
> > MACHINE=imx53qsb bitbake virtual/kernel -c clean
> > MACHINE=imx53qsb bitbake qt4-x11-free
> > 
> > I also have a patch. Where should I share them? How do I ensure everyone
> > with an interest in this defect actually gets the patch? Sure I can
> > create email and send to the people who I think need to know. The
> > bugzilla lets interested parties add themselves to bugs though.
> > 
> > I should also note that QA actually go through bugs in the bugzilla,
> > including closed ones, looking for test cases. We're not great at this
> > yet but it does happen. If there is a well documented test case like
> > that above, we might write an automated QA test for it. Having it
> > documented is therefore a good thing.
> > 
> > I do appreciate writing a bug report is hard, especially if you don't
> > know where the problem is, or how to reproduce it exactly. It takes time
> > and effort. You can however document what you know and discussion can
> > happen in a common place to figure out how to reproduce it. I do except
> > the submitters to fully understand the bug, if they did they'd probably
> > write a patch instead.
> > 
> > So fair warning, I am going to start ignoring things on irc and ask for
> > bug numbers in future, assuming something isn't a 5 minute fix with an
> > immediate patch. I will back and encourage anyone else doing this too.
> 
> What about developer mailing lists ?. isn’t it also a way to report problems
> via emails after all we use emails for patch work flow. Not all people
> working with OE-Core e.g. might be following yocto bugzilla

Mailing lists are great for discussion (e.g. "I have this problem but I'm not 
sure of the right way to solve it") but a terrible way to track actual bugs, 
because mailing lists tend to concentrate on the latest postings; older issues 
that haven't been resolved rapidly disappear with the tide of new postings. Of 
course old issues can build up in a bug tracker, but at least it's trivially 
easy to find open issues where it's much more difficult to find unresolved issues 
on a mailing list.

The point is, the best way to ensure that an issue gets solved at least for 
OE-Core is to file a bug in the YP bugzilla. There is then a triage process and 
in most cases someone to actually assign the issue to so that it can be dealt 
with. There is no such process on the mailing lists.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [OE-core] Bug reporting and good bug reports

[Khem Raj]

> On Jan 8, 2015, at 2:01 AM, Paul Eggleton <paul.eggleton@linux.intel.com> wrote:
> 
> On Wednesday 07 January 2015 16:36:37 Khem Raj wrote:
>>> On Jan 7, 2015, at 1:25 AM, Richard Purdie
>>> <richard.purdie@linuxfoundation.org> wrote:
>>> 
>>> I was informed on irc yesterday that bug reports are hard and that
>>> debugging via irc is easier. I think I need to remind people why good
>>> bug reports are important and how they do actually help immensely.
>>> 
>>> I do actually believe that not everything has to have bug report. If you
>>> mention an issue, someone says "hey, I know how to fix that" and sends a
>>> patch out, a bug report is wasted overhead IMO. I know some programme
>>> managers who disagree strongly with me and would suggest *every* bugfix
>>> commit should have a defect tracking item. We're not going there I
>>> understand the idea, its not practical.
>>> 
>>> That said, if its not immediately clear what the problem is, it should
>>> become a bug report. Why?
>>> 
>>> Firstly, the random selection of people on irc at the time probably
>>> aren't the right people to fix it. Telling those people to read 48 hours
>>> of irc log for the details is disrespectful of their time.
>>> 
>>> It also happens that the first people referred to a bug may not be the
>>> person who actually can fix it. If someone else needs to come to a bug,
>>> having a summary of the issue, the salient facts and the current status
>>> is immensley useful for handover.
>>> 
>>> As a case in point, I tried to debug a qt4 build failure yesterday for
>>> which there is no bug report. I lost hours building the wrong things,
>>> experimenting to try and find the reproducer steps and generally chasing
>>> my tail, losing the autobuilder log of the failure, the name of the qt4
>>> recipe that was failing (which task was it again?) and so on.
>>> 
>>> I do now have a set of reproducer steps, its quite simple:
>>> 
>>> MACHINE=imx53qsb bitbake virtual/kernel
>>> MACHINE=imx53qsb bitbake virtual/kernel -c clean
>>> MACHINE=imx53qsb bitbake qt4-x11-free
>>> 
>>> I also have a patch. Where should I share them? How do I ensure everyone
>>> with an interest in this defect actually gets the patch? Sure I can
>>> create email and send to the people who I think need to know. The
>>> bugzilla lets interested parties add themselves to bugs though.
>>> 
>>> I should also note that QA actually go through bugs in the bugzilla,
>>> including closed ones, looking for test cases. We're not great at this
>>> yet but it does happen. If there is a well documented test case like
>>> that above, we might write an automated QA test for it. Having it
>>> documented is therefore a good thing.
>>> 
>>> I do appreciate writing a bug report is hard, especially if you don't
>>> know where the problem is, or how to reproduce it exactly. It takes time
>>> and effort. You can however document what you know and discussion can
>>> happen in a common place to figure out how to reproduce it. I do except
>>> the submitters to fully understand the bug, if they did they'd probably
>>> write a patch instead.
>>> 
>>> So fair warning, I am going to start ignoring things on irc and ask for
>>> bug numbers in future, assuming something isn't a 5 minute fix with an
>>> immediate patch. I will back and encourage anyone else doing this too.
>> 
>> What about developer mailing lists ?. isn’t it also a way to report problems
>> via emails after all we use emails for patch work flow. Not all people
>> working with OE-Core e.g. might be following yocto bugzilla
> 
> Mailing lists are great for discussion (e.g. "I have this problem but I'm not 
> sure of the right way to solve it") but a terrible way to track actual bugs, 
> because mailing lists tend to concentrate on the latest postings; older issues 
> that haven't been resolved rapidly disappear with the tide of new postings. Of 
> course old issues can build up in a bug tracker, but at least it's trivially 
> easy to find open issues where it's much more difficult to find unresolved issues 
> on a mailing list.
> 
> The point is, the best way to ensure that an issue gets solved at least for 
> OE-Core is to file a bug in the YP bugzilla. There is then a triage process and 
> in most cases someone to actually assign the issue to so that it can be dealt 
> with. There is no such process on the mailing lists.

for a user perspective
when I try to google for any issue, first hits are not bugzilla, but the ml discussions
I am just saying all different ways for reporting issues should be encouraged. Now if someone
wants to turn it into a bugzilla entry thats good.

> 
> Cheers,
> Paul
> 
> -- 
> 
> Paul Eggleton
> Intel Open Source Technology Centre

Re: Bug reporting and good bug reports

[Khem Raj]

> On Jan 8, 2015, at 2:01 AM, Paul Eggleton <paul.eggleton@linux.intel.com> wrote:
> 
> On Wednesday 07 January 2015 16:36:37 Khem Raj wrote:
>>> On Jan 7, 2015, at 1:25 AM, Richard Purdie
>>> <richard.purdie@linuxfoundation.org> wrote:
>>> 
>>> I was informed on irc yesterday that bug reports are hard and that
>>> debugging via irc is easier. I think I need to remind people why good
>>> bug reports are important and how they do actually help immensely.
>>> 
>>> I do actually believe that not everything has to have bug report. If you
>>> mention an issue, someone says "hey, I know how to fix that" and sends a
>>> patch out, a bug report is wasted overhead IMO. I know some programme
>>> managers who disagree strongly with me and would suggest *every* bugfix
>>> commit should have a defect tracking item. We're not going there I
>>> understand the idea, its not practical.
>>> 
>>> That said, if its not immediately clear what the problem is, it should
>>> become a bug report. Why?
>>> 
>>> Firstly, the random selection of people on irc at the time probably
>>> aren't the right people to fix it. Telling those people to read 48 hours
>>> of irc log for the details is disrespectful of their time.
>>> 
>>> It also happens that the first people referred to a bug may not be the
>>> person who actually can fix it. If someone else needs to come to a bug,
>>> having a summary of the issue, the salient facts and the current status
>>> is immensley useful for handover.
>>> 
>>> As a case in point, I tried to debug a qt4 build failure yesterday for
>>> which there is no bug report. I lost hours building the wrong things,
>>> experimenting to try and find the reproducer steps and generally chasing
>>> my tail, losing the autobuilder log of the failure, the name of the qt4
>>> recipe that was failing (which task was it again?) and so on.
>>> 
>>> I do now have a set of reproducer steps, its quite simple:
>>> 
>>> MACHINE=imx53qsb bitbake virtual/kernel
>>> MACHINE=imx53qsb bitbake virtual/kernel -c clean
>>> MACHINE=imx53qsb bitbake qt4-x11-free
>>> 
>>> I also have a patch. Where should I share them? How do I ensure everyone
>>> with an interest in this defect actually gets the patch? Sure I can
>>> create email and send to the people who I think need to know. The
>>> bugzilla lets interested parties add themselves to bugs though.
>>> 
>>> I should also note that QA actually go through bugs in the bugzilla,
>>> including closed ones, looking for test cases. We're not great at this
>>> yet but it does happen. If there is a well documented test case like
>>> that above, we might write an automated QA test for it. Having it
>>> documented is therefore a good thing.
>>> 
>>> I do appreciate writing a bug report is hard, especially if you don't
>>> know where the problem is, or how to reproduce it exactly. It takes time
>>> and effort. You can however document what you know and discussion can
>>> happen in a common place to figure out how to reproduce it. I do except
>>> the submitters to fully understand the bug, if they did they'd probably
>>> write a patch instead.
>>> 
>>> So fair warning, I am going to start ignoring things on irc and ask for
>>> bug numbers in future, assuming something isn't a 5 minute fix with an
>>> immediate patch. I will back and encourage anyone else doing this too.
>> 
>> What about developer mailing lists ?. isn’t it also a way to report problems
>> via emails after all we use emails for patch work flow. Not all people
>> working with OE-Core e.g. might be following yocto bugzilla
> 
> Mailing lists are great for discussion (e.g. "I have this problem but I'm not 
> sure of the right way to solve it") but a terrible way to track actual bugs, 
> because mailing lists tend to concentrate on the latest postings; older issues 
> that haven't been resolved rapidly disappear with the tide of new postings. Of 
> course old issues can build up in a bug tracker, but at least it's trivially 
> easy to find open issues where it's much more difficult to find unresolved issues 
> on a mailing list.
> 
> The point is, the best way to ensure that an issue gets solved at least for 
> OE-Core is to file a bug in the YP bugzilla. There is then a triage process and 
> in most cases someone to actually assign the issue to so that it can be dealt 
> with. There is no such process on the mailing lists.

for a user perspective
when I try to google for any issue, first hits are not bugzilla, but the ml discussions
I am just saying all different ways for reporting issues should be encouraged. Now if someone
wants to turn it into a bugzilla entry thats good.

> 
> Cheers,
> Paul
> 
> -- 
> 
> Paul Eggleton
> Intel Open Source Technology Centre

Re: [OE-core] Bug reporting and good bug reports

[Bruce Ashfield]

On Thu, Jan 8, 2015 at 11:40 AM, Khem Raj <raj.khem@gmail.com> wrote:
>
>> On Jan 8, 2015, at 2:01 AM, Paul Eggleton <paul.eggleton@linux.intel.com> wrote:
>>
>> On Wednesday 07 January 2015 16:36:37 Khem Raj wrote:
>>>> On Jan 7, 2015, at 1:25 AM, Richard Purdie
>>>> <richard.purdie@linuxfoundation.org> wrote:
>>>>
>>>> I was informed on irc yesterday that bug reports are hard and that
>>>> debugging via irc is easier. I think I need to remind people why good
>>>> bug reports are important and how they do actually help immensely.
>>>>
>>>> I do actually believe that not everything has to have bug report. If you
>>>> mention an issue, someone says "hey, I know how to fix that" and sends a
>>>> patch out, a bug report is wasted overhead IMO. I know some programme
>>>> managers who disagree strongly with me and would suggest *every* bugfix
>>>> commit should have a defect tracking item. We're not going there I
>>>> understand the idea, its not practical.
>>>>
>>>> That said, if its not immediately clear what the problem is, it should
>>>> become a bug report. Why?
>>>>
>>>> Firstly, the random selection of people on irc at the time probably
>>>> aren't the right people to fix it. Telling those people to read 48 hours
>>>> of irc log for the details is disrespectful of their time.
>>>>
>>>> It also happens that the first people referred to a bug may not be the
>>>> person who actually can fix it. If someone else needs to come to a bug,
>>>> having a summary of the issue, the salient facts and the current status
>>>> is immensley useful for handover.
>>>>
>>>> As a case in point, I tried to debug a qt4 build failure yesterday for
>>>> which there is no bug report. I lost hours building the wrong things,
>>>> experimenting to try and find the reproducer steps and generally chasing
>>>> my tail, losing the autobuilder log of the failure, the name of the qt4
>>>> recipe that was failing (which task was it again?) and so on.
>>>>
>>>> I do now have a set of reproducer steps, its quite simple:
>>>>
>>>> MACHINE=imx53qsb bitbake virtual/kernel
>>>> MACHINE=imx53qsb bitbake virtual/kernel -c clean
>>>> MACHINE=imx53qsb bitbake qt4-x11-free
>>>>
>>>> I also have a patch. Where should I share them? How do I ensure everyone
>>>> with an interest in this defect actually gets the patch? Sure I can
>>>> create email and send to the people who I think need to know. The
>>>> bugzilla lets interested parties add themselves to bugs though.
>>>>
>>>> I should also note that QA actually go through bugs in the bugzilla,
>>>> including closed ones, looking for test cases. We're not great at this
>>>> yet but it does happen. If there is a well documented test case like
>>>> that above, we might write an automated QA test for it. Having it
>>>> documented is therefore a good thing.
>>>>
>>>> I do appreciate writing a bug report is hard, especially if you don't
>>>> know where the problem is, or how to reproduce it exactly. It takes time
>>>> and effort. You can however document what you know and discussion can
>>>> happen in a common place to figure out how to reproduce it. I do except
>>>> the submitters to fully understand the bug, if they did they'd probably
>>>> write a patch instead.
>>>>
>>>> So fair warning, I am going to start ignoring things on irc and ask for
>>>> bug numbers in future, assuming something isn't a 5 minute fix with an
>>>> immediate patch. I will back and encourage anyone else doing this too.
>>>
>>> What about developer mailing lists ?. isn’t it also a way to report problems
>>> via emails after all we use emails for patch work flow. Not all people
>>> working with OE-Core e.g. might be following yocto bugzilla
>>
>> Mailing lists are great for discussion (e.g. "I have this problem but I'm not
>> sure of the right way to solve it") but a terrible way to track actual bugs,
>> because mailing lists tend to concentrate on the latest postings; older issues
>> that haven't been resolved rapidly disappear with the tide of new postings. Of
>> course old issues can build up in a bug tracker, but at least it's trivially
>> easy to find open issues where it's much more difficult to find unresolved issues
>> on a mailing list.
>>
>> The point is, the best way to ensure that an issue gets solved at least for
>> OE-Core is to file a bug in the YP bugzilla. There is then a triage process and
>> in most cases someone to actually assign the issue to so that it can be dealt
>> with. There is no such process on the mailing lists.
>
> for a user perspective
> when I try to google for any issue, first hits are not bugzilla, but the ml discussions
> I am just saying all different ways for reporting issues should be encouraged. Now if someone
> wants to turn it into a bugzilla entry thats good.

For reporting an issue and discussing it, I don't disagree that something can
happen on the mailing list.

But if the reporter wants serious time spent on a bug (or I should say that the
fix for the bug is something that will take more than a few minutes),
I think it is
reasonable to expect that the reporter (not the person working on the bug) takes
the responsibility to get it into the bug tracker.

Assigning and scoping the actual work doesn't scale based on email threads
(as someone who has been thrashing to keep information straight, and reproduce
issues .. I feel this pain greatly).

Just my frozen 2 canadian cents! :)

Bruce


>
>>
>> Cheers,
>> Paul
>>
>> --
>>
>> Paul Eggleton
>> Intel Open Source Technology Centre
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"

Re: Bug reporting and good bug reports

[Bruce Ashfield]

On Thu, Jan 8, 2015 at 11:40 AM, Khem Raj <raj.khem@gmail.com> wrote:
>
>> On Jan 8, 2015, at 2:01 AM, Paul Eggleton <paul.eggleton@linux.intel.com> wrote:
>>
>> On Wednesday 07 January 2015 16:36:37 Khem Raj wrote:
>>>> On Jan 7, 2015, at 1:25 AM, Richard Purdie
>>>> <richard.purdie@linuxfoundation.org> wrote:
>>>>
>>>> I was informed on irc yesterday that bug reports are hard and that
>>>> debugging via irc is easier. I think I need to remind people why good
>>>> bug reports are important and how they do actually help immensely.
>>>>
>>>> I do actually believe that not everything has to have bug report. If you
>>>> mention an issue, someone says "hey, I know how to fix that" and sends a
>>>> patch out, a bug report is wasted overhead IMO. I know some programme
>>>> managers who disagree strongly with me and would suggest *every* bugfix
>>>> commit should have a defect tracking item. We're not going there I
>>>> understand the idea, its not practical.
>>>>
>>>> That said, if its not immediately clear what the problem is, it should
>>>> become a bug report. Why?
>>>>
>>>> Firstly, the random selection of people on irc at the time probably
>>>> aren't the right people to fix it. Telling those people to read 48 hours
>>>> of irc log for the details is disrespectful of their time.
>>>>
>>>> It also happens that the first people referred to a bug may not be the
>>>> person who actually can fix it. If someone else needs to come to a bug,
>>>> having a summary of the issue, the salient facts and the current status
>>>> is immensley useful for handover.
>>>>
>>>> As a case in point, I tried to debug a qt4 build failure yesterday for
>>>> which there is no bug report. I lost hours building the wrong things,
>>>> experimenting to try and find the reproducer steps and generally chasing
>>>> my tail, losing the autobuilder log of the failure, the name of the qt4
>>>> recipe that was failing (which task was it again?) and so on.
>>>>
>>>> I do now have a set of reproducer steps, its quite simple:
>>>>
>>>> MACHINE=imx53qsb bitbake virtual/kernel
>>>> MACHINE=imx53qsb bitbake virtual/kernel -c clean
>>>> MACHINE=imx53qsb bitbake qt4-x11-free
>>>>
>>>> I also have a patch. Where should I share them? How do I ensure everyone
>>>> with an interest in this defect actually gets the patch? Sure I can
>>>> create email and send to the people who I think need to know. The
>>>> bugzilla lets interested parties add themselves to bugs though.
>>>>
>>>> I should also note that QA actually go through bugs in the bugzilla,
>>>> including closed ones, looking for test cases. We're not great at this
>>>> yet but it does happen. If there is a well documented test case like
>>>> that above, we might write an automated QA test for it. Having it
>>>> documented is therefore a good thing.
>>>>
>>>> I do appreciate writing a bug report is hard, especially if you don't
>>>> know where the problem is, or how to reproduce it exactly. It takes time
>>>> and effort. You can however document what you know and discussion can
>>>> happen in a common place to figure out how to reproduce it. I do except
>>>> the submitters to fully understand the bug, if they did they'd probably
>>>> write a patch instead.
>>>>
>>>> So fair warning, I am going to start ignoring things on irc and ask for
>>>> bug numbers in future, assuming something isn't a 5 minute fix with an
>>>> immediate patch. I will back and encourage anyone else doing this too.
>>>
>>> What about developer mailing lists ?. isn’t it also a way to report problems
>>> via emails after all we use emails for patch work flow. Not all people
>>> working with OE-Core e.g. might be following yocto bugzilla
>>
>> Mailing lists are great for discussion (e.g. "I have this problem but I'm not
>> sure of the right way to solve it") but a terrible way to track actual bugs,
>> because mailing lists tend to concentrate on the latest postings; older issues
>> that haven't been resolved rapidly disappear with the tide of new postings. Of
>> course old issues can build up in a bug tracker, but at least it's trivially
>> easy to find open issues where it's much more difficult to find unresolved issues
>> on a mailing list.
>>
>> The point is, the best way to ensure that an issue gets solved at least for
>> OE-Core is to file a bug in the YP bugzilla. There is then a triage process and
>> in most cases someone to actually assign the issue to so that it can be dealt
>> with. There is no such process on the mailing lists.
>
> for a user perspective
> when I try to google for any issue, first hits are not bugzilla, but the ml discussions
> I am just saying all different ways for reporting issues should be encouraged. Now if someone
> wants to turn it into a bugzilla entry thats good.

For reporting an issue and discussing it, I don't disagree that something can
happen on the mailing list.

But if the reporter wants serious time spent on a bug (or I should say that the
fix for the bug is something that will take more than a few minutes),
I think it is
reasonable to expect that the reporter (not the person working on the bug) takes
the responsibility to get it into the bug tracker.

Assigning and scoping the actual work doesn't scale based on email threads
(as someone who has been thrashing to keep information straight, and reproduce
issues .. I feel this pain greatly).

Just my frozen 2 canadian cents! :)

Bruce


>
>>
>> Cheers,
>> Paul
>>
>> --
>>
>> Paul Eggleton
>> Intel Open Source Technology Centre
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"

Re: [OE-core] Bug reporting and good bug reports

[Khem Raj]

> On Jan 8, 2015, at 8:59 AM, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> 
> Assigning and scoping the actual work doesn't scale based on email threads
> (as someone who has been thrashing to keep information straight, and reproduce
> issues .. I feel this pain greatly).

if you are a talking about corp development yes I agree for open source projects
thats the nature of the beast.

Re: Bug reporting and good bug reports

[Khem Raj]

> On Jan 8, 2015, at 8:59 AM, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> 
> Assigning and scoping the actual work doesn't scale based on email threads
> (as someone who has been thrashing to keep information straight, and reproduce
> issues .. I feel this pain greatly).

if you are a talking about corp development yes I agree for open source projects
thats the nature of the beast.

Re: [OE-core] Bug reporting and good bug reports

[Bruce Ashfield]

On Thu, Jan 8, 2015 at 12:29 PM, Khem Raj <raj.khem@gmail.com> wrote:
>
>> On Jan 8, 2015, at 8:59 AM, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>>
>> Assigning and scoping the actual work doesn't scale based on email threads
>> (as someone who has been thrashing to keep information straight, and reproduce
>> issues .. I feel this pain greatly).
>
> if you are a talking about corp development yes I agree for open source projects
> thats the nature of the beast.

I work with plenty of open source projects in my own time .. and if I
have an issue,
I open a bug if it requires a fix.

Probably a point of view thing.

We aren't saying that this is mandated .. just that anyone who can't be bothered
to log a bug (and steps to reproduce it) .. should set their
expectations for someone
to fix it accordingly :)

Bruce



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"

Re: Bug reporting and good bug reports

[Bruce Ashfield]

On Thu, Jan 8, 2015 at 12:29 PM, Khem Raj <raj.khem@gmail.com> wrote:
>
>> On Jan 8, 2015, at 8:59 AM, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>>
>> Assigning and scoping the actual work doesn't scale based on email threads
>> (as someone who has been thrashing to keep information straight, and reproduce
>> issues .. I feel this pain greatly).
>
> if you are a talking about corp development yes I agree for open source projects
> thats the nature of the beast.

I work with plenty of open source projects in my own time .. and if I
have an issue,
I open a bug if it requires a fix.

Probably a point of view thing.

We aren't saying that this is mandated .. just that anyone who can't be bothered
to log a bug (and steps to reproduce it) .. should set their
expectations for someone
to fix it accordingly :)

Bruce



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"

Re: Bug reporting and good bug reports

[Trevor Woerner]

[If I reply, people will think that I'm really big on this MAINTAINERS
file thing, which I'm really not. But if I don't reply I'll feel as
though my point was missed :-(]

On 01/07/15 16:30, Richard Purdie wrote:
> On Wed, 2015-01-07 at 16:21 -0500, Trevor Woerner wrote:
>> On 01/07/15 04:25, Richard Purdie wrote:
>>> I also have a patch. Where should I share them? How do I ensure everyone
>>> with an interest in this defect actually gets the patch? Sure I can
>>> create email and send to the people who I think need to know.
>> When I went through that whole "let's add a MAINTAINERS file" thing last
>> year this is exactly what I had in mind at that time.
> But that isn't what I'm talking about.

Okay, sorry for going off-topic.

> I'm talking about people being able to say "I have some interest in a
> particular defect and I'd like updates about it". That is completely
> different to a MAINTAINERS file. The user and easily opt in to specific
> things using the web UI and its self maintaining to a large degree.

While, on the one hand, it would be nice for random people to say "I'm
interested in this defect, I'm going to add myself to the list of people
who are notified when something about this defect changes" (which, as
I'm trying to point out, is a use-case of the MAINTAINERS file), I
believe your more basic question of "I have a patch, who do I email it
to?" needs to be addressed as well.

At this point, we don't have much of a mechanism for people to figure
out who to email their patches to. Emailing one's patches to the right
people is a good thing since it'll increase the chances of it being
reviewed by the most relevant people, and every project could stand to
have more reviewers. What we have is a README file. But this mechanism
requires people to read the README file (which isn't always a given) and
follow its instructions (which is prone to various errors).

The MAINTAINERS file automates the process of figuring out who the
relevant people are to email your patches to. Not only can people add
themselves to the list for a given area of the project, but the
MAINTAINERS mechanism (i.e. the script) will also look at the git
history of the lines your patch is modifying and add those people to the
list as well (the assumption being the last person who modified the code
you're about to modify might be interested in what you're doing to their
code).

Can you email your patch to bugzilla? And have bugzilla figure out all
the people to forward your patch to? Not that I know of.

Do you expect people working on a bug they find in the code one random
day to go searching through bugzilla in case someone else has already
noticed this bug and filed a report so they can look at the bugzilla
issue's CC list to find out who to email the patch to?

If a developer starts their day by looking through bugzilla for an open
issue to fix (or is assigned such an issue from a manager) then your
workflow makes sense. They will prepare a patch, update the issue, and
then email the patch *hopefully* CC'ing the people in the bugzilla CC
list and/or updating the issue with a link to their patch submission.

But if a developer is working on their own thing and stumbles across
some bug, completely unaware that this has been reported in bugzilla,
they will prepare their patch and email it to the mailing list (and most
likely nobody else). At least the MAINTAINERS file would help a little
bit in this case (if for nothing else but to figure out which mailing
list to email!) and if people added themselves to the MAINTAINERS file
as being interested in a certain area, then those people would be
emailed too.

The biggest difference between what you're talking about and what I'm
talking about is the granularity. A bugzilla entry addresses a defect,
which could cross many files and layers. The granularity of the
MAINTAINERS file is more about individual files and directories.

Thanks for listening :-)

Re: [yocto] Bug reporting and good bug reports

[Trevor Woerner]

[If I reply, people will think that I'm really big on this MAINTAINERS
file thing, which I'm really not. But if I don't reply I'll feel as
though my point was missed :-(]

On 01/07/15 16:30, Richard Purdie wrote:
> On Wed, 2015-01-07 at 16:21 -0500, Trevor Woerner wrote:
>> On 01/07/15 04:25, Richard Purdie wrote:
>>> I also have a patch. Where should I share them? How do I ensure everyone
>>> with an interest in this defect actually gets the patch? Sure I can
>>> create email and send to the people who I think need to know.
>> When I went through that whole "let's add a MAINTAINERS file" thing last
>> year this is exactly what I had in mind at that time.
> But that isn't what I'm talking about.

Okay, sorry for going off-topic.

> I'm talking about people being able to say "I have some interest in a
> particular defect and I'd like updates about it". That is completely
> different to a MAINTAINERS file. The user and easily opt in to specific
> things using the web UI and its self maintaining to a large degree.

While, on the one hand, it would be nice for random people to say "I'm
interested in this defect, I'm going to add myself to the list of people
who are notified when something about this defect changes" (which, as
I'm trying to point out, is a use-case of the MAINTAINERS file), I
believe your more basic question of "I have a patch, who do I email it
to?" needs to be addressed as well.

At this point, we don't have much of a mechanism for people to figure
out who to email their patches to. Emailing one's patches to the right
people is a good thing since it'll increase the chances of it being
reviewed by the most relevant people, and every project could stand to
have more reviewers. What we have is a README file. But this mechanism
requires people to read the README file (which isn't always a given) and
follow its instructions (which is prone to various errors).

The MAINTAINERS file automates the process of figuring out who the
relevant people are to email your patches to. Not only can people add
themselves to the list for a given area of the project, but the
MAINTAINERS mechanism (i.e. the script) will also look at the git
history of the lines your patch is modifying and add those people to the
list as well (the assumption being the last person who modified the code
you're about to modify might be interested in what you're doing to their
code).

Can you email your patch to bugzilla? And have bugzilla figure out all
the people to forward your patch to? Not that I know of.

Do you expect people working on a bug they find in the code one random
day to go searching through bugzilla in case someone else has already
noticed this bug and filed a report so they can look at the bugzilla
issue's CC list to find out who to email the patch to?

If a developer starts their day by looking through bugzilla for an open
issue to fix (or is assigned such an issue from a manager) then your
workflow makes sense. They will prepare a patch, update the issue, and
then email the patch *hopefully* CC'ing the people in the bugzilla CC
list and/or updating the issue with a link to their patch submission.

But if a developer is working on their own thing and stumbles across
some bug, completely unaware that this has been reported in bugzilla,
they will prepare their patch and email it to the mailing list (and most
likely nobody else). At least the MAINTAINERS file would help a little
bit in this case (if for nothing else but to figure out which mailing
list to email!) and if people added themselves to the MAINTAINERS file
as being interested in a certain area, then those people would be
emailed too.

The biggest difference between what you're talking about and what I'm
talking about is the granularity. A bugzilla entry addresses a defect,
which could cross many files and layers. The granularity of the
MAINTAINERS file is more about individual files and directories.

Thanks for listening :-)