[RFC] GitLab issue tracker labeling process: arch/target, os, and accel labels

[RFC] GitLab issue tracker labeling process: arch/target, os, and accel labels

[John Snow]

Hi, I'd like to work out our collective preferences for how we triage 
issues that concern the execution environment; namely the arch (now 
"target", os, and accel labels.

If you're CC'd on this mail, you're either listed as a TCG maintainer, a 
target maintainer, or have been heavily involved in the GitLab issue 
tracker migration process. You might care about how we triage and file 
these issues.

I'm sending the mail because I've been (so far) responsible for a lot of 
the labeling as we move over from Launchpad. I'll need to change my 
process to accommodate what our maintainers want to see. (And to 
encourage others to join in on the triage process.)

As of right now, there's no formal process or document for how we use 
any of the labels -- Thomas and I had been only informally collaborating 
in our usage of them as we migrated from Launchpad. Thomas has a script 
he uses to move the bugs, so I usually don't edit these labels until I 
give him a heads up so I don't break his script.

Let's discuss what changes we need to make and collaborate on when and 
how we'll make them.


# Accel

Currently "accel: XXX", for HAX, HVF, KVM, TCG, WHPX and Xen.

https://gitlab.com/qemu-project/qemu/-/labels?subscribed=&search=accel%3A

Multiple accel labels can be applied to an issue, not just one. The 
intent was to allow for issues that may impact multiple accelerators, 
though that case may be rare.

I think these are reasonably straightforward and unambiguous, though for 
some qemu-system reports it's not always evident which accelerator 
applies right away without more info. The accel tag is often omitted 
because of this uncertainty.

I'd like to keep the mapping here 1:1 against ACCEL_CLASS_NAME if I can. 
It makes the mapping from CLI to accel tag fairly straightforward.

We technically lack a "qtest" accel tag for that parity. It could be 
added, but it's likely not actually useful versus the "tests" label. 
It's not really a user-facing accelerator.

I see we also have a new "nvmm" accelerator, too, which should now be 
added here.

RTH raises the issue of the "TCI" subsystem of TCG, which is not a full 
accelerator in its own right, but (I think) a special case of TCG. If I 
keep the 1:1 mapping to ACCEL_CLASS_NAME, "accel: TCI" is inappropriate.

Some suggestions:
- "TCI" by itself, simple enough.
- "TCG-TCI" or "TCG: TCI" or "TCG/TCI" or similar, so that it shows up 
in label search when you search for 'tcg'.
- "accel: TCG:TCI". Similar to above but uses the "accel:" prefix too.

My only concern here is completeness of the label: this one seems like 
it's at particular risk of being forgotten or lost. It works perfectly 
well as an organizational bucket for people working on TCI, but I wonder 
if it will work well as an "issue inbox". Intended use begins to matter 
here. Your thoughts, Stefan?


# OS

Currently "os: XXX" for BSD, Linux, Windows, and macOS.

https://gitlab.com/qemu-project/qemu/-/labels?subscribed=&search=os%3A

Multiple OS labels can be applied to an issue.

Originally, we kept this label somewhat vague and have been using it to 
identify both the host AND guest involved with an issue.

Stefan Weil has requested that we refactor this to separate the concerns 
so that he can identify issues where Windows is the host without wading 
through numerous reports where Windows is merely the guest. Reasonable 
request.

Shall we split it into "host: XXX" and "guest: XXX" for {BSD, Linux, 
Windows, macOS}?

This isn't too hard to do at initial triage time, but we'll need to sift 
through the bugs we've labeled so far and re-label them. Help on this 
would be appreciated. I would prefer we create a *new* set of labels and 
then draw down on the old labels instead of just renaming them. That 
way, the old label can be used as a re-triage queue.


# arch/target

Currently "target: XXX" for alpha, arm, avr, cris, hexagon, hppa, i386, 
m68k, microblaze, mips, nios2, openrisc, ppc, riscv, rx, s390x, sh4, 
sparc, tricore, xtensa.

https://gitlab.com/qemu-project/qemu/-/labels?subscribed=&search=target%3A

The names map 1:1 to the directories in target/.
The names in [square brackets] in the label descriptions correspond 1:1 
with the SysEmuTarget QAPI enum defined in qapi/machine.json.

Multiple target labels can be applied to an issue. Originally, this was 
named "arch", so this was to allow multiple architectures to be 
specified to cover the host/guest environment. If we disentangle this, 
we may still want to allow multiple labels to cover bugs that might 
affect multiple targets, though that case might be rare.

Recently, we renamed this from "arch: XXX" to "target: XXX", though the 
label had been being used for both the host and guest architecture, so 
this will need to be re-audited to remove cases where the label had been 
applied for the host architecture.

We probably want to keep a set of labels that apply to the host 
architecture. These are useful for build failures, environment setup 
issues, or just documenting the exact environment on which an issue was 
observed.

We won't likely require the full set of targets to be duplicated for 
this purpose: possibly just the most common ones. I assume those are:

arm, i386, ppc, s390x

How should we tag those? "host-arch: XXX"?

What I would like to avoid is creating labels like "host: windows-i386" 
where the cross matrix of ({host,guest} X OS x ARCH) starts to require 
ever-increasing specificity of initial triage labels and may increase 
the risk of overly-specified bugs going unnoticed. Maybe my concern is 
unfounded, but I think the over-specificity will hurt more than help at 
this stage.


# Current Plan

- Add "accel: NVVM" label
- Don't add "accel: qtest".
- Add "host: {Windows, BSD, Linux, macOS}" and "guest: {Windows, BSD, 
Linux, macOS}" labels.
- Migrate "os: XXX" labels to the above split. Help wanted.
- Remove the "os: XXX" labels when the above is done.
- Re-audit "target: XXX" labels and remove usages that applied to the 
host instead of the guest. Help wanted. Possibly assign new host-arch 
labels.
- Create a document in /docs/devel/gitlab-process.rst based on the 
outcome of this thread to formalize our decisions and reasonings. Future 
patches to this doc can serve as the discussion point for changing that 
process. There are other topics to discuss beyond these labels as well, 
but these three topics felt like the most pressing to address given 
where we are in our Launchpad migration process.


Less sure:

- Create host-arch: XXX labels (Unsure of name, which platforms are
   important to track? see above.)
- Create "TCG: TCI" label (Unsure of naming)


Thanks,
--js

Re: [RFC] GitLab issue tracker labeling process: arch/target, os, and accel labels

[Stefan Weil]

Am 14.06.21 um 19:32 schrieb John Snow:

> RTH raises the issue of the "TCI" subsystem of TCG, which is not a 
> full accelerator in its own right, but (I think) a special case of 
> TCG. If I keep the 1:1 mapping to ACCEL_CLASS_NAME, "accel: TCI" is 
> inappropriate.
>
> Some suggestions:
> - "TCI" by itself, simple enough.
> - "TCG-TCI" or "TCG: TCI" or "TCG/TCI" or similar, so that it shows up 
> in label search when you search for 'tcg'.
> - "accel: TCG:TCI". Similar to above but uses the "accel:" prefix too.
>
> My only concern here is completeness of the label: this one seems like 
> it's at particular risk of being forgotten or lost. It works perfectly 
> well as an organizational bucket for people working on TCI, but I 
> wonder if it will work well as an "issue inbox". Intended use begins 
> to matter here. Your thoughts, Stefan?


I appreciate your, Richard's and all other efforts to further improve 
the label system.

Regarding the label for TCI I have no special personal preferences. The 
above suggestions are all fine for me, so choose one which fits best to 
other labels.

Thanks,

Stefan

Re: [RFC] GitLab issue tracker labeling process: arch/target, os, and accel labels

[Philippe Mathieu-Daudé]

On 6/14/21 7:32 PM, John Snow wrote:

> # Accel
> 
> Currently "accel: XXX", for HAX, HVF, KVM, TCG, WHPX and Xen.
> 
> https://gitlab.com/qemu-project/qemu/-/labels?subscribed=&search=accel%3A
> 
> Multiple accel labels can be applied to an issue, not just one. The
> intent was to allow for issues that may impact multiple accelerators,
> though that case may be rare.

While we can have QEMU built for multiple accelerators, AFAIK there is
only one at runtime. This is the one we want to know in bug reports.

I don't see multiple accel: labels as a valid user case.

The exception could be nested virtualization, but it is the same accel,
so this could be covered by an extra accel:nested label.

> I think these are reasonably straightforward and unambiguous, though for
> some qemu-system reports it's not always evident which accelerator
> applies right away without more info. The accel tag is often omitted
> because of this uncertainty.
> 
> I'd like to keep the mapping here 1:1 against ACCEL_CLASS_NAME if I can.
> It makes the mapping from CLI to accel tag fairly straightforward.
> 
> We technically lack a "qtest" accel tag for that parity. It could be
> added, but it's likely not actually useful versus the "tests" label.
> It's not really a user-facing accelerator.

All the accel:qtest issues I recall are the Fuzzer ones.
Sufficient so far.

> I see we also have a new "nvmm" accelerator, too, which should now be
> added here.
> 
> RTH raises the issue of the "TCI" subsystem of TCG, which is not a full
> accelerator in its own right, but (I think) a special case of TCG. If I
> keep the 1:1 mapping to ACCEL_CLASS_NAME, "accel: TCI" is inappropriate.
> 
> Some suggestions:
> - "TCI" by itself, simple enough.
> - "TCG-TCI" or "TCG: TCI" or "TCG/TCI" or similar, so that it shows up
> in label search when you search for 'tcg'.
> - "accel: TCG:TCI". Similar to above but uses the "accel:" prefix too.

It is unlikely a reporter add the TCI label. We'll add it upon triage.
Likely from TCG with the sole objective to Cc Stefan, which isn't up to
date with the recent TCI changes, so back to TCG maintainers.
Is it worth the churn?

> My only concern here is completeness of the label: this one seems like
> it's at particular risk of being forgotten or lost. It works perfectly
> well as an organizational bucket for people working on TCI, but I wonder
> if it will work well as an "issue inbox". Intended use begins to matter
> here. Your thoughts, Stefan?
> 
> 
> # OS
> 
> Currently "os: XXX" for BSD, Linux, Windows, and macOS.
> 
> https://gitlab.com/qemu-project/qemu/-/labels?subscribed=&search=os%3A
> 
> Multiple OS labels can be applied to an issue.
> 
> Originally, we kept this label somewhat vague and have been using it to
> identify both the host AND guest involved with an issue.
> 
> Stefan Weil has requested that we refactor this to separate the concerns
> so that he can identify issues where Windows is the host without wading
> through numerous reports where Windows is merely the guest. Reasonable
> request.
> 
> Shall we split it into "host: XXX" and "guest: XXX" for {BSD, Linux,
> Windows, macOS}?

I'm missing the importance of the guest OS. Either it is in pair with
the host accel, or it is accel:TCG and I see the guest irrelevant (do
we want to list all firmwares?).

So I'll let other sort this out.

> This isn't too hard to do at initial triage time, but we'll need to sift
> through the bugs we've labeled so far and re-label them. Help on this
> would be appreciated. I would prefer we create a *new* set of labels and
> then draw down on the old labels instead of just renaming them. That
> way, the old label can be used as a re-triage queue.
> 
> 
> # arch/target
> 
> Currently "target: XXX" for alpha, arm, avr, cris, hexagon, hppa, i386,
> m68k, microblaze, mips, nios2, openrisc, ppc, riscv, rx, s390x, sh4,
> sparc, tricore, xtensa.
> 
> https://gitlab.com/qemu-project/qemu/-/labels?subscribed=&search=target%3A
> 
> The names map 1:1 to the directories in target/.
> The names in [square brackets] in the label descriptions correspond 1:1
> with the SysEmuTarget QAPI enum defined in qapi/machine.json.
> 
> Multiple target labels can be applied to an issue. Originally, this was
> named "arch", so this was to allow multiple architectures to be
> specified to cover the host/guest environment. If we disentangle this,
> we may still want to allow multiple labels to cover bugs that might
> affect multiple targets, though that case might be rare.
> 
> Recently, we renamed this from "arch: XXX" to "target: XXX", though the
> label had been being used for both the host and guest architecture, so
> this will need to be re-audited to remove cases where the label had been
> applied for the host architecture.

This audit list is simply ...

> We probably want to keep a set of labels that apply to the host
> architecture. These are useful for build failures, environment setup
> issues, or just documenting the exact environment on which an issue was
> observed.
> 
> We won't likely require the full set of targets to be duplicated for
> this purpose: possibly just the most common ones. I assume those are:
> 
> arm, i386, ppc, s390x

... this one ^.

Maybe use the list of architecture with accelerators?

The rest of problems likely fits into the "Build System" label
(or is not supported).

Regards,

Phil.

Re: [RFC] GitLab issue tracker labeling process: arch/target, os, and accel labels

[Philippe Mathieu-Daudé]

On 6/14/21 8:53 PM, Philippe Mathieu-Daudé wrote:
> On 6/14/21 7:32 PM, John Snow wrote:
>>
>> # OS
>>
>> Currently "os: XXX" for BSD, Linux, Windows, and macOS.
>>
>> https://gitlab.com/qemu-project/qemu/-/labels?subscribed=&search=os%3A
>>
>> Multiple OS labels can be applied to an issue.
>>
>> Originally, we kept this label somewhat vague and have been using it to
>> identify both the host AND guest involved with an issue.
>>
>> Stefan Weil has requested that we refactor this to separate the concerns
>> so that he can identify issues where Windows is the host without wading
>> through numerous reports where Windows is merely the guest. Reasonable
>> request.
>>
>> Shall we split it into "host: XXX" and "guest: XXX" for {BSD, Linux,
>> Windows, macOS}?
> 
> I'm missing the importance of the guest OS. Either it is in pair with
> the host accel, or it is accel:TCG and I see the guest irrelevant

Err I mentioned the nested case, but generally I tend to have the same
view, guest is not very relevant (except for qemu-guest-agent maybe).

> (do we want to list all firmwares?).
> 
> So I'll let other sort this out.

Re: [RFC] GitLab issue tracker labeling process: arch/target, os, and accel labels

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 8478 bytes --]

On Mon, Jun 14, 2021 at 01:32:11PM -0400, John Snow wrote:
> Hi, I'd like to work out our collective preferences for how we triage issues
> that concern the execution environment; namely the arch (now "target", os,
> and accel labels.
> 
> If you're CC'd on this mail, you're either listed as a TCG maintainer, a
> target maintainer, or have been heavily involved in the GitLab issue tracker
> migration process. You might care about how we triage and file these issues.
> 
> I'm sending the mail because I've been (so far) responsible for a lot of the
> labeling as we move over from Launchpad. I'll need to change my process to
> accommodate what our maintainers want to see. (And to encourage others to
> join in on the triage process.)
> 
> As of right now, there's no formal process or document for how we use any of
> the labels -- Thomas and I had been only informally collaborating in our
> usage of them as we migrated from Launchpad. Thomas has a script he uses to
> move the bugs, so I usually don't edit these labels until I give him a heads
> up so I don't break his script.
> 
> Let's discuss what changes we need to make and collaborate on when and how
> we'll make them.

In general, what's the convention when a bug is independent of (say)
the accel: does it get none of the accel tags, or all of them?
Likewise with OS and the other categories.

> # Accel
> 
> Currently "accel: XXX", for HAX, HVF, KVM, TCG, WHPX and Xen.
> 
> https://gitlab.com/qemu-project/qemu/-/labels?subscribed=&search=accel%3A
> 
> Multiple accel labels can be applied to an issue, not just one. The intent
> was to allow for issues that may impact multiple accelerators, though that
> case may be rare.
> 
> I think these are reasonably straightforward and unambiguous, though for
> some qemu-system reports it's not always evident which accelerator applies
> right away without more info. The accel tag is often omitted because of this
> uncertainty.
> 
> I'd like to keep the mapping here 1:1 against ACCEL_CLASS_NAME if I can. It
> makes the mapping from CLI to accel tag fairly straightforward.
> 
> We technically lack a "qtest" accel tag for that parity. It could be added,
> but it's likely not actually useful versus the "tests" label. It's not
> really a user-facing accelerator.
> 
> I see we also have a new "nvmm" accelerator, too, which should now be added
> here.
> 
> RTH raises the issue of the "TCI" subsystem of TCG, which is not a full
> accelerator in its own right, but (I think) a special case of TCG. If I keep
> the 1:1 mapping to ACCEL_CLASS_NAME, "accel: TCI" is inappropriate.
> 
> Some suggestions:
> - "TCI" by itself, simple enough.
> - "TCG-TCI" or "TCG: TCI" or "TCG/TCI" or similar, so that it shows up in
> label search when you search for 'tcg'.
> - "accel: TCG:TCI". Similar to above but uses the "accel:" prefix too.
> 
> My only concern here is completeness of the label: this one seems like it's
> at particular risk of being forgotten or lost. It works perfectly well as an
> organizational bucket for people working on TCI, but I wonder if it will
> work well as an "issue inbox". Intended use begins to matter here. Your
> thoughts, Stefan?
> 
> 
> # OS
> 
> Currently "os: XXX" for BSD, Linux, Windows, and macOS.
> 
> https://gitlab.com/qemu-project/qemu/-/labels?subscribed=&search=os%3A
> 
> Multiple OS labels can be applied to an issue.
> 
> Originally, we kept this label somewhat vague and have been using it to
> identify both the host AND guest involved with an issue.
> 
> Stefan Weil has requested that we refactor this to separate the concerns so
> that he can identify issues where Windows is the host without wading through
> numerous reports where Windows is merely the guest. Reasonable request.
> 
> Shall we split it into "host: XXX" and "guest: XXX" for {BSD, Linux,
> Windows, macOS}?

I think that would be a good idea.  Except maybe "host-os" and
"guest-os", because "host" in particular is ambiguous with host
architecture. (not relevant that often, but sometimes).

> This isn't too hard to do at initial triage time, but we'll need to sift
> through the bugs we've labeled so far and re-label them. Help on this would
> be appreciated. I would prefer we create a *new* set of labels and then draw
> down on the old labels instead of just renaming them. That way, the old
> label can be used as a re-triage queue.

That seems like the right way to do it to me.

> # arch/target
> 
> Currently "target: XXX" for alpha, arm, avr, cris, hexagon, hppa, i386,
> m68k, microblaze, mips, nios2, openrisc, ppc, riscv, rx, s390x, sh4, sparc,
> tricore, xtensa.
> 
> https://gitlab.com/qemu-project/qemu/-/labels?subscribed=&search=target%3A
> 
> The names map 1:1 to the directories in target/.
> The names in [square brackets] in the label descriptions correspond 1:1 with
> the SysEmuTarget QAPI enum defined in qapi/machine.json.
> 
> Multiple target labels can be applied to an issue. Originally, this was
> named "arch", so this was to allow multiple architectures to be specified to
> cover the host/guest environment. If we disentangle this, we may still want
> to allow multiple labels to cover bugs that might affect multiple targets,
> though that case might be rare.

Agreed.  I think mixing host and guest properties together is a bad
idea.  These are relatively rarely related to each other.
Bugs affecting multiple, but not all targets are uncommon, but not
super rare (mostly due to chunks of code shared across several target
archs, like ACPI and device tree).

> Recently, we renamed this from "arch: XXX" to "target: XXX", though the
> label had been being used for both the host and guest architecture, so this
> will need to be re-audited to remove cases where the label had been applied
> for the host architecture.

Oops.

> We probably want to keep a set of labels that apply to the host
> architecture. These are useful for build failures, environment setup issues,
> or just documenting the exact environment on which an issue was observed.

Ah.. that's another general question.  Are the labels supposed to
document where the problem has been definitely observed, or a best
estimate at where it will appear.  It would be very common for a bug
to be observed initially on only one, but quickly turn out to be
independent of host and/or target arch.

> We won't likely require the full set of targets to be duplicated for this
> purpose: possibly just the most common ones. I assume those are:
> 
> arm, i386, ppc, s390x
> 
> How should we tag those? "host-arch: XXX"?
> 
> What I would like to avoid is creating labels like "host: windows-i386"
> where the cross matrix of ({host,guest} X OS x ARCH) starts to require
> ever-increasing specificity of initial triage labels and may increase the
> risk of overly-specified bugs going unnoticed. Maybe my concern is
> unfounded, but I think the over-specificity will hurt more than help at this
> stage.
> 
> 
> # Current Plan
> 
> - Add "accel: NVVM" label
> - Don't add "accel: qtest".
> - Add "host: {Windows, BSD, Linux, macOS}" and "guest: {Windows, BSD, Linux,
> macOS}" labels.

Again, I suggest "host-os" and "guest-os"

> - Migrate "os: XXX" labels to the above split. Help wanted.
> - Remove the "os: XXX" labels when the above is done.
> - Re-audit "target: XXX" labels and remove usages that applied to the host
> instead of the guest. Help wanted. Possibly assign new host-arch labels.
> - Create a document in /docs/devel/gitlab-process.rst based on the outcome
> of this thread to formalize our decisions and reasonings. Future patches to
> this doc can serve as the discussion point for changing that process. There
> are other topics to discuss beyond these labels as well, but these three
> topics felt like the most pressing to address given where we are in our
> Launchpad migration process.
> 
> 
> Less sure:
> 
> - Create host-arch: XXX labels (Unsure of name, which platforms are
>   important to track? see above.)

Maybe leave and see if looks like it would be useful?

> - Create "TCG: TCI" label (Unsure of naming)
> 
> 
> Thanks,
> --js
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC] GitLab issue tracker labeling process: arch/target, os, and accel labels

[Cornelia Huck]

On Mon, Jun 14 2021, John Snow <jsnow@redhat.com> wrote:

(...)

> # OS
>
> Currently "os: XXX" for BSD, Linux, Windows, and macOS.
>
> https://gitlab.com/qemu-project/qemu/-/labels?subscribed=&search=os%3A
>
> Multiple OS labels can be applied to an issue.
>
> Originally, we kept this label somewhat vague and have been using it to 
> identify both the host AND guest involved with an issue.
>
> Stefan Weil has requested that we refactor this to separate the concerns 
> so that he can identify issues where Windows is the host without wading 
> through numerous reports where Windows is merely the guest. Reasonable 
> request.
>
> Shall we split it into "host: XXX" and "guest: XXX" for {BSD, Linux, 
> Windows, macOS}?

Yes to splitting and using something like "hostOS:" and "guestOS:", as
had already been suggested downthread.

For the guest OS, I think we also want "Other". It can be valuable to
know that the guest OS might be doing something that is not done by the
OSes usually run as a guest, so I think this is useful information.

What about linux-user? We probably can't categorize what is being run
very neatly.

>
> This isn't too hard to do at initial triage time, but we'll need to sift 
> through the bugs we've labeled so far and re-label them. Help on this 
> would be appreciated. I would prefer we create a *new* set of labels and 
> then draw down on the old labels instead of just renaming them. That 
> way, the old label can be used as a re-triage queue.
>
>
> # arch/target
>
> Currently "target: XXX" for alpha, arm, avr, cris, hexagon, hppa, i386, 
> m68k, microblaze, mips, nios2, openrisc, ppc, riscv, rx, s390x, sh4, 
> sparc, tricore, xtensa.
>
> https://gitlab.com/qemu-project/qemu/-/labels?subscribed=&search=target%3A
>
> The names map 1:1 to the directories in target/.
> The names in [square brackets] in the label descriptions correspond 1:1 
> with the SysEmuTarget QAPI enum defined in qapi/machine.json.
>
> Multiple target labels can be applied to an issue. Originally, this was 
> named "arch", so this was to allow multiple architectures to be 
> specified to cover the host/guest environment. If we disentangle this, 
> we may still want to allow multiple labels to cover bugs that might 
> affect multiple targets, though that case might be rare.
>
> Recently, we renamed this from "arch: XXX" to "target: XXX", though the 
> label had been being used for both the host and guest architecture, so 
> this will need to be re-audited to remove cases where the label had been 
> applied for the host architecture.
>
> We probably want to keep a set of labels that apply to the host 
> architecture. These are useful for build failures, environment setup 
> issues, or just documenting the exact environment on which an issue was 
> observed.
>
> We won't likely require the full set of targets to be duplicated for 
> this purpose: possibly just the most common ones. I assume those are:
>
> arm, i386, ppc, s390x
>
> How should we tag those? "host-arch: XXX"?

host-arch sounds good; maybe add a catch-all "host-arch: other" to catch
uncommon host architectures?

>
> What I would like to avoid is creating labels like "host: windows-i386" 
> where the cross matrix of ({host,guest} X OS x ARCH) starts to require 
> ever-increasing specificity of initial triage labels and may increase 
> the risk of overly-specified bugs going unnoticed. Maybe my concern is 
> unfounded, but I think the over-specificity will hurt more than help at 
> this stage.

I think having "host-arch:" and "hostOS:" is enough.

Re: [RFC] GitLab issue tracker labeling process: arch/target, os, and accel labels

[Thomas Huth]

On 14/06/2021 19.32, John Snow wrote:
[...]
> RTH raises the issue of the "TCI" subsystem of TCG, which is not a full 
> accelerator in its own right, but (I think) a special case of TCG. If I keep 
> the 1:1 mapping to ACCEL_CLASS_NAME, "accel: TCI" is inappropriate.
> 
> Some suggestions:
> - "TCI" by itself, simple enough.
> - "TCG-TCI" or "TCG: TCI" or "TCG/TCI" or similar, so that it shows up in 
> label search when you search for 'tcg'.
> - "accel: TCG:TCI". Similar to above but uses the "accel:" prefix too.

I wonder whether we need a label for TCI at all... I can't recall having 
ever seen a bug ticket filed for TCI. It's quite a special use-case with 
some few users only, so it's maybe not worth the effort to create a separate 
label for this... just my 0.02 €.

> We probably want to keep a set of labels that apply to the host 
> architecture. These are useful for build failures, environment setup issues, 
> or just documenting the exact environment on which an issue was observed.
> 
> We won't likely require the full set of targets to be duplicated for this 
> purpose: possibly just the most common ones. I assume those are:
> 
> arm, i386, ppc, s390x
> 
> How should we tag those? "host-arch: XXX"?

"host-arch" sounds fine to me. I think you can limit the selection here to 
the list of TCG backends that we support:

  arm, i386, mips, ppc, riscv, s390x, sparc

... and maybe tci here (i.e. "host-arch: tci")?

  Thomas

Re: [RFC] GitLab issue tracker labeling process: arch/target, os, and accel labels

[Philippe Mathieu-Daudé]

On 6/15/21 4:08 AM, David Gibson wrote:
> On Mon, Jun 14, 2021 at 01:32:11PM -0400, John Snow wrote:
> In general, what's the convention when a bug is independent of (say)
> the accel: does it get none of the accel tags, or all of them?
> Likewise with OS and the other categories.

None: remove the label. Otherwise you'll notify everybody subscribed
to specific labels.

>> We probably want to keep a set of labels that apply to the host
>> architecture. These are useful for build failures, environment setup issues,
>> or just documenting the exact environment on which an issue was observed.
> 
> Ah.. that's another general question.  Are the labels supposed to
> document where the problem has been definitely observed, or a best
> estimate at where it will appear.  It would be very common for a bug
> to be observed initially on only one, but quickly turn out to be
> independent of host and/or target arch.

Similar. If the problem is generic, remove the specific labels.

Regards,

Phil.

Re: [RFC] GitLab issue tracker labeling process: arch/target, os, and accel labels

[Philippe Mathieu-Daudé]

On 6/15/21 9:28 AM, Cornelia Huck wrote:
> On Mon, Jun 14 2021, John Snow <jsnow@redhat.com> wrote:
> 
> (...)
> 
>> # OS
>>
>> Currently "os: XXX" for BSD, Linux, Windows, and macOS.
>>
>> https://gitlab.com/qemu-project/qemu/-/labels?subscribed=&search=os%3A
>>
>> Multiple OS labels can be applied to an issue.

> What about linux-user? We probably can't categorize what is being run
> very neatly.

There is already a 'linux-user' label, usually paired with 'accel:TCG'
and the target arch:
https://gitlab.com/qemu-project/qemu/-/issues?label_name%5B%5D=linux-user

Re: [RFC] GitLab issue tracker labeling process: arch/target, os, and accel labels

[Philippe Mathieu-Daudé]

On 6/15/21 10:56 AM, Thomas Huth wrote:
> On 14/06/2021 19.32, John Snow wrote:
> [...]
>> RTH raises the issue of the "TCI" subsystem of TCG, which is not a
>> full accelerator in its own right, but (I think) a special case of
>> TCG. If I keep the 1:1 mapping to ACCEL_CLASS_NAME, "accel: TCI" is
>> inappropriate.
>>
>> Some suggestions:
>> - "TCI" by itself, simple enough.
>> - "TCG-TCI" or "TCG: TCI" or "TCG/TCI" or similar, so that it shows up
>> in label search when you search for 'tcg'.
>> - "accel: TCG:TCI". Similar to above but uses the "accel:" prefix too.
> 
> I wonder whether we need a label for TCI at all... I can't recall having
> ever seen a bug ticket filed for TCI. It's quite a special use-case with
> some few users only, so it's maybe not worth the effort to create a
> separate label for this... just my 0.02 €.
> 
>> We probably want to keep a set of labels that apply to the host
>> architecture. These are useful for build failures, environment setup
>> issues, or just documenting the exact environment on which an issue
>> was observed.
>>
>> We won't likely require the full set of targets to be duplicated for
>> this purpose: possibly just the most common ones. I assume those are:
>>
>> arm, i386, ppc, s390x
>>
>> How should we tag those? "host-arch: XXX"?
> 
> "host-arch" sounds fine to me. I think you can limit the selection here
> to the list of TCG backends that we support:
> 
>  arm, i386, mips, ppc, riscv, s390x, sparc
> 
> ... and maybe tci here (i.e. "host-arch: tci")?

Great idea!

Re: [RFC] GitLab issue tracker labeling process: arch/target, os, and accel labels

[John Snow]

On 6/14/21 10:08 PM, David Gibson wrote:
> On Mon, Jun 14, 2021 at 01:32:11PM -0400, John Snow wrote:
>> Hi, I'd like to work out our collective preferences for how we triage issues
>> that concern the execution environment; namely the arch (now "target", os,
>> and accel labels.

[...]

> In general, what's the convention when a bug is independent of (say)
> the accel: does it get none of the accel tags, or all of them?
> Likewise with OS and the other categories.

So far, I have been labeling bugs reported against a specific 
accel/guest/host combination with those bugs. It doesn't necessarily 
mean they are bugs *in* those components. They might be, they might not be.

Generally I have been treating these labels as descriptors of the 
problem environment and not necessarily descriptors of the root cause. 
At a glance I often have no clue what the root cause might be. In just a 
few minutes, translating some of the details of the environment into 
labels in the hopes that it floats by someone with more knowledge in one 
or more of those areas is the best I can do.

This *does* mean that for TCG developers, there's a high ambiguity here 
because "accel: TCG" && "target: i386" applies to a pretty broad 
category of reports, not all of them necessarily bugs primarily 
suspected to be *about* TCG. Maybe, maybe not.

Phil sometimes removes these labels once it becomes apparent to him that 
the bug doesn't actually involve the system mentioned. Maybe it was 
filed under i386 but impacts all architectures, so we'd remove that label.

(Open to suggestions...)

>> # Accel
>>
>> Currently "accel: XXX", for HAX, HVF, KVM, TCG, WHPX and Xen.
>>
>> https://gitlab.com/qemu-project/qemu/-/labels?subscribed=&search=accel%3A

[...]

>> # OS
>>
>> Currently "os: XXX" for BSD, Linux, Windows, and macOS.
>>
>> https://gitlab.com/qemu-project/qemu/-/labels?subscribed=&search=os%3A
>>
>> Multiple OS labels can be applied to an issue.
>>
>> Originally, we kept this label somewhat vague and have been using it to
>> identify both the host AND guest involved with an issue.
>>
>> Stefan Weil has requested that we refactor this to separate the concerns so
>> that he can identify issues where Windows is the host without wading through
>> numerous reports where Windows is merely the guest. Reasonable request.
>>
>> Shall we split it into "host: XXX" and "guest: XXX" for {BSD, Linux,
>> Windows, macOS}?
> 
> I think that would be a good idea.  Except maybe "host-os" and
> "guest-os", because "host" in particular is ambiguous with host
> architecture. (not relevant that often, but sometimes).

Easy enough.


>> # arch/target
>>
>> Currently "target: XXX" for alpha, arm, avr, cris, hexagon, hppa, i386,
>> m68k, microblaze, mips, nios2, openrisc, ppc, riscv, rx, s390x, sh4, sparc,
>> tricore, xtensa.
>>
>> https://gitlab.com/qemu-project/qemu/-/labels?subscribed=&search=target%3A
>>
>> The names map 1:1 to the directories in target/.
>> The names in [square brackets] in the label descriptions correspond 1:1 with
>> the SysEmuTarget QAPI enum defined in qapi/machine.json.
>>
>> Multiple target labels can be applied to an issue. Originally, this was
>> named "arch", so this was to allow multiple architectures to be specified to
>> cover the host/guest environment. If we disentangle this, we may still want
>> to allow multiple labels to cover bugs that might affect multiple targets,
>> though that case might be rare.
> 
> Agreed.  I think mixing host and guest properties together is a bad
> idea.  These are relatively rarely related to each other.
> Bugs affecting multiple, but not all targets are uncommon, but not
> super rare (mostly due to chunks of code shared across several target
> archs, like ACPI and device tree).

Right. It's not super common, but I see no real reason to *enforce* that 
a bug only ever has zero-or-one target label.

>> We probably want to keep a set of labels that apply to the host
>> architecture. These are useful for build failures, environment setup issues,
>> or just documenting the exact environment on which an issue was observed.
> 
> Ah.. that's another general question.  Are the labels supposed to
> document where the problem has been definitely observed, or a best
> estimate at where it will appear.  It would be very common for a bug
> to be observed initially on only one, but quickly turn out to be
> independent of host and/or target arch.

This is a triage problem. In an ideal world, as I see it:

1. Maintainers (in general) look at the queue of "open" bugs that have 
not yet been marked as triaged/confirmed/in-progress/closed/assigned/etc

2. They spend no more than a few minutes assessing the issue and 
assigning some fairly broad topic labels. Ideally, at least one of these 
labels will be one that a Maintainer who knows more about that area 
actively receives reports for.

3. Specific subsystem maintainers watching certain labels re-label bugs 
that catch their eye with far more explicit labels as they desire.

e.g. I kick a lot of things into broad labels like "Graphics", "Audio", 
"Storage". At a glance, and quickly, it can be hard to get more specific 
than that.

However, as an example, maybe I would glance at the Storage tag and 
occasionally add a label like 'block:ATA' to pull things into a queue I 
would watch more closely.

So for right now, these labels under question (accel, target, os) don't 
differentiate between "observed on" and "have been definitely identified 
as a problem with". They're more like hints that might wind up being wrong.

Is that the wrong idea? Maybe!

>> # Current Plan
>>
>> - Add "accel: NVVM" label
>> - Don't add "accel: qtest".
>> - Add "host: {Windows, BSD, Linux, macOS}" and "guest: {Windows, BSD, Linux,
>> macOS}" labels. >
> Again, I suggest "host-os" and "guest-os"
>

ACK

[...]

>> Less sure:
>>
>> - Create host-arch: XXX labels (Unsure of name, which platforms are
>>    important to track? see above.)
> 
> Maybe leave and see if looks like it would be useful?

At a glance from other feedback, that looks like the likely route. Just 
kill it off and see if anyone wants it badly enough to bring it back.

--js

Re: [RFC] GitLab issue tracker labeling process: arch/target, os, and accel labels

[Philippe Mathieu-Daudé]

On 6/15/21 9:27 PM, John Snow wrote:
> On 6/14/21 10:08 PM, David Gibson wrote:
>> On Mon, Jun 14, 2021 at 01:32:11PM -0400, John Snow wrote:
>>> Hi, I'd like to work out our collective preferences for how we triage
>>> issues
>>> that concern the execution environment; namely the arch (now
>>> "target", os,
>>> and accel labels.
> 
> [...]
> 
>> In general, what's the convention when a bug is independent of (say)
>> the accel: does it get none of the accel tags, or all of them?
>> Likewise with OS and the other categories.
> 
> So far, I have been labeling bugs reported against a specific
> accel/guest/host combination with those bugs. It doesn't necessarily
> mean they are bugs *in* those components. They might be, they might not be.
> 
> Generally I have been treating these labels as descriptors of the
> problem environment and not necessarily descriptors of the root cause.
> At a glance I often have no clue what the root cause might be. In just a
> few minutes, translating some of the details of the environment into
> labels in the hopes that it floats by someone with more knowledge in one
> or more of those areas is the best I can do.
> 
> This *does* mean that for TCG developers, there's a high ambiguity here
> because "accel: TCG" && "target: i386" applies to a pretty broad
> category of reports, not all of them necessarily bugs primarily
> suspected to be *about* TCG. Maybe, maybe not.
> 
> Phil sometimes removes these labels once it becomes apparent to him that
> the bug doesn't actually involve the system mentioned. Maybe it was
> filed under i386 but impacts all architectures, so we'd remove that label.

I was doing this hoping it would help the triage, so further updates
won't trigger notification to the maintainers subscribed to the labels
that became irrelevant.

But now I understood other maintainers use the labels to sort bugs, so
I think we should first agree on what we expect from gitlab labels and
how to use them, before discussing on what labels to use.

Personally I see labels like a tree of IRQ lines :)
We want to always reach someone, starting broad to eventually get to
the right person able to help. The sooner we disable an IRQ where it
is not required the better, because we release uninterested maintainers
from noise, so they can attending other issues.

Re: [RFC] GitLab issue tracker labeling process: arch/target, os, and accel labels

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1365 bytes --]

On Tue, Jun 15, 2021 at 03:56:46PM +0200, Philippe Mathieu-Daudé wrote:
> On 6/15/21 4:08 AM, David Gibson wrote:
> > On Mon, Jun 14, 2021 at 01:32:11PM -0400, John Snow wrote:
> > In general, what's the convention when a bug is independent of (say)
> > the accel: does it get none of the accel tags, or all of them?
> > Likewise with OS and the other categories.
> 
> None: remove the label. Otherwise you'll notify everybody subscribed
> to specific labels.
> 
> >> We probably want to keep a set of labels that apply to the host
> >> architecture. These are useful for build failures, environment setup issues,
> >> or just documenting the exact environment on which an issue was observed.
> > 
> > Ah.. that's another general question.  Are the labels supposed to
> > document where the problem has been definitely observed, or a best
> > estimate at where it will appear.  It would be very common for a bug
> > to be observed initially on only one, but quickly turn out to be
> > independent of host and/or target arch.
> 
> Similar. If the problem is generic, remove the specific labels.
> 
> Regards,
> 
> Phil.


Understood, thanks.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]