Migrating to the gitlab issue tracker

Migrating to the gitlab issue tracker

[John Snow]

If you're in the CC list, it's because you are listed in MAINTAINERS.

Paolo's QEMU keynote this morning mentioned the possible use of the 
Gitlab issue tracker instead of using Launchpad.

I'm quite fond of the gitlab issue tracker, I think it works quite well 
and it has pretty good and uncomplicated API access to it in order to 
customize your workflow if you'd really like to.

In experimenting with my mirror on gitlab though, I was unable to find a 
way to configure it to send issue tracker notifications to the email 
list. A move to gitlab would likely mean, then:

1. The cessation of (automatic) issue tracker mails to the list
2. The loss of the ability to update the issue tracker by replying to 
said emails
3. Anyone listed in MAINTAINERS would be expected to have a gitlab 
account in order to interact with the issue tracker.

However, once you have a gitlab account, you DO gain the ability to 
receive emails for issues; possibly only those tagged with labels that 
you cared about -- giving a nice filtering mechanism to receive only 
bugs you care about.

Gitlab also does support individual accounts updating issues using a 
generated personalized email address, so if the email workflow is 
crucial to you, it is still available.

I'm for it, or at least for beginning a pilot program where we 
experiment with the idea for interested parties. I wanted to send up a 
trial balloon to see how we were feeling about this.

--js

Re: Migrating to the gitlab issue tracker

[Daniel P. Berrangé]

On Thu, Oct 29, 2020 at 12:01:27PM -0400, John Snow wrote:
> If you're in the CC list, it's because you are listed in MAINTAINERS.
> 
> Paolo's QEMU keynote this morning mentioned the possible use of the Gitlab
> issue tracker instead of using Launchpad.
> 
> I'm quite fond of the gitlab issue tracker, I think it works quite well and
> it has pretty good and uncomplicated API access to it in order to customize
> your workflow if you'd really like to.
> 
> In experimenting with my mirror on gitlab though, I was unable to find a way
> to configure it to send issue tracker notifications to the email list. A
> move to gitlab would likely mean, then:
> 
> 1. The cessation of (automatic) issue tracker mails to the list
> 2. The loss of the ability to update the issue tracker by replying to said
> emails
> 3. Anyone listed in MAINTAINERS would be expected to have a gitlab account
> in order to interact with the issue tracker.
> 
> However, once you have a gitlab account, you DO gain the ability to receive
> emails for issues; possibly only those tagged with labels that you cared
> about -- giving a nice filtering mechanism to receive only bugs you care
> about.

The other thing gained by having a gitlab account is that you can
create a fork of QEMU, and push subsystem trees to the fork. This will
run a load of CI jobs enabling subsystem maintainer to catch many build
problems before sending an email PULL to Peter. This will make it more
likely the PULL will get merged first time without problems, and reduce
the load on Peter letting him do more productive stuff than finding build
failures.

I think we should have an expectation that all PULLs have undergone
GitLab CI testing before being emailed to the list.

NB, GitLab CI doesn't cover every platform combo - there is also
Cirrus CI and Travis. Maintainer can optionally setup accounts on
those too, but I don't think we should seek to require that as it
starts to get a bit more conplex to keep everything sane. Just
focusing on GitLab CI for subsystem maintainers would be a big
enough win on its own I expect.

> Gitlab also does support individual accounts updating issues using a
> generated personalized email address, so if the email workflow is crucial to
> you, it is still available.
> 
> I'm for it, or at least for beginning a pilot program where we experiment
> with the idea for interested parties. I wanted to send up a trial balloon to
> see how we were feeling about this.

The other benefit with using GitLab issues instead of Launchpad is
auto-closing based on comments. A commit message can include a line:

  Closes https://gitlab.com/qemu-project/qemu/-/issues/NNN

When that git commit gets merged to git master in a PULL by Peter, the
GitLab issue will be automatically closed, and it will cross-link to
the commit.

This will eliminate the manual actions that our Launchpad Janitor(s) do
today to close old issues that have been fixed, improving the signal/noise
ratio.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: Migrating to the gitlab issue tracker

[Cornelia Huck]

On Thu, 29 Oct 2020 12:01:27 -0400
John Snow <jsnow@redhat.com> wrote:

> If you're in the CC list, it's because you are listed in MAINTAINERS.

<cleared the cc: list except for qemu-devel :)>

> 
> Paolo's QEMU keynote this morning mentioned the possible use of the 
> Gitlab issue tracker instead of using Launchpad.
> 
> I'm quite fond of the gitlab issue tracker, I think it works quite well 
> and it has pretty good and uncomplicated API access to it in order to 
> customize your workflow if you'd really like to.
> 
> In experimenting with my mirror on gitlab though, I was unable to find a 
> way to configure it to send issue tracker notifications to the email 
> list. A move to gitlab would likely mean, then:
> 
> 1. The cessation of (automatic) issue tracker mails to the list
> 2. The loss of the ability to update the issue tracker by replying to 
> said emails
> 3. Anyone listed in MAINTAINERS would be expected to have a gitlab 
> account in order to interact with the issue tracker.

The gitlab issue tracker is almost certainly is an improvement over
launchpad (and I do have a gitlab account); but not being able to
interact via email is at least annoying. I expect that not only
maintainers will want to interact with bug reports?

> 
> However, once you have a gitlab account, you DO gain the ability to 
> receive emails for issues; possibly only those tagged with labels that 
> you cared about -- giving a nice filtering mechanism to receive only 
> bugs you care about.
> 
> Gitlab also does support individual accounts updating issues using a 
> generated personalized email address, so if the email workflow is 
> crucial to you, it is still available.

You mean that I can update via email, provided it's an address
associated with my account?

> 
> I'm for it, or at least for beginning a pilot program where we 
> experiment with the idea for interested parties. I wanted to send up a 
> trial balloon to see how we were feeling about this.
> 
> --js

Re: Migrating to the gitlab issue tracker

[Alistair Francis]

On Thu, Oct 29, 2020 at 9:41 AM Cornelia Huck <cohuck@redhat.com> wrote:
>
> On Thu, 29 Oct 2020 12:01:27 -0400
> John Snow <jsnow@redhat.com> wrote:
>
> > If you're in the CC list, it's because you are listed in MAINTAINERS.
>
> <cleared the cc: list except for qemu-devel :)>
>
> >
> > Paolo's QEMU keynote this morning mentioned the possible use of the
> > Gitlab issue tracker instead of using Launchpad.
> >
> > I'm quite fond of the gitlab issue tracker, I think it works quite well
> > and it has pretty good and uncomplicated API access to it in order to
> > customize your workflow if you'd really like to.
> >
> > In experimenting with my mirror on gitlab though, I was unable to find a
> > way to configure it to send issue tracker notifications to the email
> > list. A move to gitlab would likely mean, then:
> >
> > 1. The cessation of (automatic) issue tracker mails to the list
> > 2. The loss of the ability to update the issue tracker by replying to
> > said emails
> > 3. Anyone listed in MAINTAINERS would be expected to have a gitlab
> > account in order to interact with the issue tracker.
>
> The gitlab issue tracker is almost certainly is an improvement over
> launchpad (and I do have a gitlab account); but not being able to
> interact via email is at least annoying. I expect that not only
> maintainers will want to interact with bug reports?
>
> >
> > However, once you have a gitlab account, you DO gain the ability to
> > receive emails for issues; possibly only those tagged with labels that
> > you cared about -- giving a nice filtering mechanism to receive only
> > bugs you care about.
> >
> > Gitlab also does support individual accounts updating issues using a
> > generated personalized email address, so if the email workflow is
> > crucial to you, it is still available.
>
> You mean that I can update via email, provided it's an address
> associated with my account?
>
> >
> > I'm for it, or at least for beginning a pilot program where we
> > experiment with the idea for interested parties. I wanted to send up a
> > trial balloon to see how we were feeling about this.

I'm not sure if you want Acks, but it sounds good to me.

Alistair

> >
> > --js
>
>

Re: Migrating to the gitlab issue tracker

[John Snow]

On 10/29/20 12:49 PM, Alistair Francis wrote:
> On Thu, Oct 29, 2020 at 9:41 AM Cornelia Huck <cohuck@redhat.com> wrote:
>>
>> On Thu, 29 Oct 2020 12:01:27 -0400
>> John Snow <jsnow@redhat.com> wrote:
>>
>>> If you're in the CC list, it's because you are listed in MAINTAINERS.
>>
>> <cleared the cc: list except for qemu-devel :)>
>>
>>>
>>> Paolo's QEMU keynote this morning mentioned the possible use of the
>>> Gitlab issue tracker instead of using Launchpad.
>>>
>>> I'm quite fond of the gitlab issue tracker, I think it works quite well
>>> and it has pretty good and uncomplicated API access to it in order to
>>> customize your workflow if you'd really like to.
>>>
>>> In experimenting with my mirror on gitlab though, I was unable to find a
>>> way to configure it to send issue tracker notifications to the email
>>> list. A move to gitlab would likely mean, then:
>>>
>>> 1. The cessation of (automatic) issue tracker mails to the list
>>> 2. The loss of the ability to update the issue tracker by replying to
>>> said emails
>>> 3. Anyone listed in MAINTAINERS would be expected to have a gitlab
>>> account in order to interact with the issue tracker.
>>
>> The gitlab issue tracker is almost certainly is an improvement over
>> launchpad (and I do have a gitlab account); but not being able to
>> interact via email is at least annoying. I expect that not only
>> maintainers will want to interact with bug reports?
>>
>>>
>>> However, once you have a gitlab account, you DO gain the ability to
>>> receive emails for issues; possibly only those tagged with labels that
>>> you cared about -- giving a nice filtering mechanism to receive only
>>> bugs you care about.
>>>
>>> Gitlab also does support individual accounts updating issues using a
>>> generated personalized email address, so if the email workflow is
>>> crucial to you, it is still available.
>>
>> You mean that I can update via email, provided it's an address
>> associated with my account?
>>
>>>
>>> I'm for it, or at least for beginning a pilot program where we
>>> experiment with the idea for interested parties. I wanted to send up a
>>> trial balloon to see how we were feeling about this.
> 
> I'm not sure if you want Acks, but it sounds good to me.
> 
> Alistair
> 

Mostly I was looking for any hard objections over the idea of issues not 
necessarily being sent to the list anymore, if there were any.

I want to hear from Thomas Huth too, but maybe we can work out a pilot 
migration and give it a test-run and find more concrete objections that way.

--js

Re: Migrating to the gitlab issue tracker

[Kashyap Chamarthy]

On Thu, Oct 29, 2020 at 01:12:22PM -0400, John Snow wrote:

Hi,

[...]

> Mostly I was looking for any hard objections over the idea of issues not
> necessarily being sent to the list anymore, if there were any.

Not an objection, but I would miss discovering "interesting" issues by
virtue of bug emails sent to the list.  Several times I've "stumbled
into" bugs on 'qemu-devel' that affect upper layers, or just Damn
Interesting QEMU Bugs™ (especially in the Block Layer) that helped me
gain better understanding, or something is already a "known issue".

That said, the reactions so far seem to be positive for moving to GitLab
Issues.  

Maybe there are some acceptable workarounds.  It looks like there's a
way to subscribe to GitLab issues via email[1]; and I can triage that
maildir as part of my regular email workflow.

https://docs.gitlab.com/ee/administration/incoming_email.html

[...]

-- 
/kashyap

Re: Migrating to the gitlab issue tracker

[Thomas Huth]

On 29/10/2020 18.12, John Snow wrote:
> On 10/29/20 12:49 PM, Alistair Francis wrote:
>> On Thu, Oct 29, 2020 at 9:41 AM Cornelia Huck <cohuck@redhat.com> wrote:
>>>
>>> On Thu, 29 Oct 2020 12:01:27 -0400
>>> John Snow <jsnow@redhat.com> wrote:
>>>
>>>> If you're in the CC list, it's because you are listed in MAINTAINERS.
>>>
>>> <cleared the cc: list except for qemu-devel :)>
>>>
>>>>
>>>> Paolo's QEMU keynote this morning mentioned the possible use of the
>>>> Gitlab issue tracker instead of using Launchpad.
>>>>
>>>> I'm quite fond of the gitlab issue tracker, I think it works quite well
>>>> and it has pretty good and uncomplicated API access to it in order to
>>>> customize your workflow if you'd really like to.
>>>>
>>>> In experimenting with my mirror on gitlab though, I was unable to find a
>>>> way to configure it to send issue tracker notifications to the email
>>>> list. A move to gitlab would likely mean, then:
>>>>
>>>> 1. The cessation of (automatic) issue tracker mails to the list
>>>> 2. The loss of the ability to update the issue tracker by replying to
>>>> said emails
>>>> 3. Anyone listed in MAINTAINERS would be expected to have a gitlab
>>>> account in order to interact with the issue tracker.
>>>
>>> The gitlab issue tracker is almost certainly is an improvement over
>>> launchpad (and I do have a gitlab account); but not being able to
>>> interact via email is at least annoying. I expect that not only
>>> maintainers will want to interact with bug reports?
>>>
>>>>
>>>> However, once you have a gitlab account, you DO gain the ability to
>>>> receive emails for issues; possibly only those tagged with labels that
>>>> you cared about -- giving a nice filtering mechanism to receive only
>>>> bugs you care about.
>>>>
>>>> Gitlab also does support individual accounts updating issues using a
>>>> generated personalized email address, so if the email workflow is
>>>> crucial to you, it is still available.
>>>
>>> You mean that I can update via email, provided it's an address
>>> associated with my account?
>>>
>>>>
>>>> I'm for it, or at least for beginning a pilot program where we
>>>> experiment with the idea for interested parties. I wanted to send up a
>>>> trial balloon to see how we were feeling about this.
>>
>> I'm not sure if you want Acks, but it sounds good to me.
>>
>> Alistair
>>
> 
> Mostly I was looking for any hard objections over the idea of issues not
> necessarily being sent to the list anymore, if there were any.
> 
> I want to hear from Thomas Huth too, but maybe we can work out a pilot
> migration and give it a test-run and find more concrete objections that way.

I'd certainly give it a try! Launchpad is IMHO really a pain (let me know if
I should elaborate on that topic again...), and the email bridge is often
also not working correctly (replies to bug mails sometimes get put into the
bug tracker, sometimes not), so this is not something I would really miss
when we quite Launchpad.

So could somebody please enable the issue tracker there, so we can give it a
try? Phil? Alex? Stefan? ...?

If it works well, I can certainly help to get the links etc. in our website
fixed.

 Thomas

Re: Migrating to the gitlab issue tracker

[John Snow]

On 10/29/20 3:55 PM, Thomas Huth wrote:
> On 29/10/2020 18.12, John Snow wrote:
>> On 10/29/20 12:49 PM, Alistair Francis wrote:
>>> On Thu, Oct 29, 2020 at 9:41 AM Cornelia Huck <cohuck@redhat.com> wrote:
>>>>
>>>> On Thu, 29 Oct 2020 12:01:27 -0400
>>>> John Snow <jsnow@redhat.com> wrote:
>>>>
>>>>> If you're in the CC list, it's because you are listed in MAINTAINERS.
>>>>
>>>> <cleared the cc: list except for qemu-devel :)>
>>>>
>>>>>
>>>>> Paolo's QEMU keynote this morning mentioned the possible use of the
>>>>> Gitlab issue tracker instead of using Launchpad.
>>>>>
>>>>> I'm quite fond of the gitlab issue tracker, I think it works quite well
>>>>> and it has pretty good and uncomplicated API access to it in order to
>>>>> customize your workflow if you'd really like to.
>>>>>
>>>>> In experimenting with my mirror on gitlab though, I was unable to find a
>>>>> way to configure it to send issue tracker notifications to the email
>>>>> list. A move to gitlab would likely mean, then:
>>>>>
>>>>> 1. The cessation of (automatic) issue tracker mails to the list
>>>>> 2. The loss of the ability to update the issue tracker by replying to
>>>>> said emails
>>>>> 3. Anyone listed in MAINTAINERS would be expected to have a gitlab
>>>>> account in order to interact with the issue tracker.
>>>>
>>>> The gitlab issue tracker is almost certainly is an improvement over
>>>> launchpad (and I do have a gitlab account); but not being able to
>>>> interact via email is at least annoying. I expect that not only
>>>> maintainers will want to interact with bug reports?
>>>>
>>>>>
>>>>> However, once you have a gitlab account, you DO gain the ability to
>>>>> receive emails for issues; possibly only those tagged with labels that
>>>>> you cared about -- giving a nice filtering mechanism to receive only
>>>>> bugs you care about.
>>>>>
>>>>> Gitlab also does support individual accounts updating issues using a
>>>>> generated personalized email address, so if the email workflow is
>>>>> crucial to you, it is still available.
>>>>
>>>> You mean that I can update via email, provided it's an address
>>>> associated with my account?
>>>>
>>>>>
>>>>> I'm for it, or at least for beginning a pilot program where we
>>>>> experiment with the idea for interested parties. I wanted to send up a
>>>>> trial balloon to see how we were feeling about this.
>>>
>>> I'm not sure if you want Acks, but it sounds good to me.
>>>
>>> Alistair
>>>
>>
>> Mostly I was looking for any hard objections over the idea of issues not
>> necessarily being sent to the list anymore, if there were any.
>>
>> I want to hear from Thomas Huth too, but maybe we can work out a pilot
>> migration and give it a test-run and find more concrete objections that way.
> 
> I'd certainly give it a try! Launchpad is IMHO really a pain (let me know if
> I should elaborate on that topic again...), and the email bridge is often
> also not working correctly (replies to bug mails sometimes get put into the
> bug tracker, sometimes not), so this is not something I would really miss
> when we quite Launchpad.
> 
> So could somebody please enable the issue tracker there, so we can give it a
> try? Phil? Alex? Stefan? ...?
> 
> If it works well, I can certainly help to get the links etc. in our website
> fixed.
> 

Great! You are the primary bug wrangler, so if you are interested in 
trialing it, I am interested in helping!

If we enable the bug tracker, can we please add Thomas and myself as 
'Reporter' level contributors to QEMU so we can wrangle bugs?

Info on permissions: https://docs.gitlab.com/ee/user/permissions.html

There's also an old bug (2 years) about migrating Launchpad to Gitlab, 
but there's been no movement.

https://gitlab.com/gitlab-org/gitlab/-/issues/22600

If we like the results of the trial, we'll need a convincing migration 
strategy.

Re: Migrating to the gitlab issue tracker

[Daniel P. Berrangé]

On Thu, Oct 29, 2020 at 04:27:44PM -0400, John Snow wrote:
> On 10/29/20 3:55 PM, Thomas Huth wrote:
> > On 29/10/2020 18.12, John Snow wrote:
> > > On 10/29/20 12:49 PM, Alistair Francis wrote:
> > > > On Thu, Oct 29, 2020 at 9:41 AM Cornelia Huck <cohuck@redhat.com> wrote:
> > > > > 
> > > > > On Thu, 29 Oct 2020 12:01:27 -0400
> > > > > John Snow <jsnow@redhat.com> wrote:
> > > > > 
> > > > > > If you're in the CC list, it's because you are listed in MAINTAINERS.
> > > > > 
> > > > > <cleared the cc: list except for qemu-devel :)>
> > > > > 
> > > > > > 
> > > > > > Paolo's QEMU keynote this morning mentioned the possible use of the
> > > > > > Gitlab issue tracker instead of using Launchpad.
> > > > > > 
> > > > > > I'm quite fond of the gitlab issue tracker, I think it works quite well
> > > > > > and it has pretty good and uncomplicated API access to it in order to
> > > > > > customize your workflow if you'd really like to.
> > > > > > 
> > > > > > In experimenting with my mirror on gitlab though, I was unable to find a
> > > > > > way to configure it to send issue tracker notifications to the email
> > > > > > list. A move to gitlab would likely mean, then:
> > > > > > 
> > > > > > 1. The cessation of (automatic) issue tracker mails to the list
> > > > > > 2. The loss of the ability to update the issue tracker by replying to
> > > > > > said emails
> > > > > > 3. Anyone listed in MAINTAINERS would be expected to have a gitlab
> > > > > > account in order to interact with the issue tracker.
> > > > > 
> > > > > The gitlab issue tracker is almost certainly is an improvement over
> > > > > launchpad (and I do have a gitlab account); but not being able to
> > > > > interact via email is at least annoying. I expect that not only
> > > > > maintainers will want to interact with bug reports?
> > > > > 
> > > > > > 
> > > > > > However, once you have a gitlab account, you DO gain the ability to
> > > > > > receive emails for issues; possibly only those tagged with labels that
> > > > > > you cared about -- giving a nice filtering mechanism to receive only
> > > > > > bugs you care about.
> > > > > > 
> > > > > > Gitlab also does support individual accounts updating issues using a
> > > > > > generated personalized email address, so if the email workflow is
> > > > > > crucial to you, it is still available.
> > > > > 
> > > > > You mean that I can update via email, provided it's an address
> > > > > associated with my account?
> > > > > 
> > > > > > 
> > > > > > I'm for it, or at least for beginning a pilot program where we
> > > > > > experiment with the idea for interested parties. I wanted to send up a
> > > > > > trial balloon to see how we were feeling about this.
> > > > 
> > > > I'm not sure if you want Acks, but it sounds good to me.
> > > > 
> > > > Alistair
> > > > 
> > > 
> > > Mostly I was looking for any hard objections over the idea of issues not
> > > necessarily being sent to the list anymore, if there were any.
> > > 
> > > I want to hear from Thomas Huth too, but maybe we can work out a pilot
> > > migration and give it a test-run and find more concrete objections that way.
> > 
> > I'd certainly give it a try! Launchpad is IMHO really a pain (let me know if
> > I should elaborate on that topic again...), and the email bridge is often
> > also not working correctly (replies to bug mails sometimes get put into the
> > bug tracker, sometimes not), so this is not something I would really miss
> > when we quite Launchpad.
> > 
> > So could somebody please enable the issue tracker there, so we can give it a
> > try? Phil? Alex? Stefan? ...?
> > 
> > If it works well, I can certainly help to get the links etc. in our website
> > fixed.
> > 
> 
> Great! You are the primary bug wrangler, so if you are interested in
> trialing it, I am interested in helping!
> 
> If we enable the bug tracker, can we please add Thomas and myself as
> 'Reporter' level contributors to QEMU so we can wrangle bugs?
> 
> Info on permissions: https://docs.gitlab.com/ee/user/permissions.html
> 
> There's also an old bug (2 years) about migrating Launchpad to Gitlab, but
> there's been no movement.
> 
> https://gitlab.com/gitlab-org/gitlab/-/issues/22600
> 
> If we like the results of the trial, we'll need a convincing migration
> strategy.

My convincing strategy is "do nothing" :-)

Most importantly we need to be able to make the existing "QEMU" component
in launch read-only to prevent people filing new bugs there, ideally with
a change in the description to point people to the new bug tracker.

We can leave existing bugs in LP to continue their discussion. If there
are some we explicitly want in gitlab manually re-file them. Aside from
that if we periodically auto-close any stale bugs, after a while we'll
have culled launchpad down to zero.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: Migrating to the gitlab issue tracker

[Peter Maydell]

On Fri, 30 Oct 2020 at 09:23, Daniel P. Berrangé <berrange@redhat.com> wrote:
> My convincing strategy is "do nothing" :-)

I am, er, not convinced :-)

> Most importantly we need to be able to make the existing "QEMU" component
> in launch read-only to prevent people filing new bugs there, ideally with
> a change in the description to point people to the new bug tracker.
>
> We can leave existing bugs in LP to continue their discussion. If there
> are some we explicitly want in gitlab manually re-file them. Aside from
> that if we periodically auto-close any stale bugs, after a while we'll
> have culled launchpad down to zero.

Minimally, we should have an easy way to refile specific bugs
that doesn't involve manual cut-n-paste. Most of the Arm bugs
in launchpad are valid, for instance, I think, and I really
don't want to be spending a day in unnecessary clerical work
copying information into gitlab...

thanks
-- PMM

Re: Migrating to the gitlab issue tracker

[Daniel P. Berrangé]

On Fri, Oct 30, 2020 at 10:03:44AM +0000, Peter Maydell wrote:
> On Fri, 30 Oct 2020 at 09:23, Daniel P. Berrangé <berrange@redhat.com> wrote:
> > My convincing strategy is "do nothing" :-)
> 
> I am, er, not convinced :-)
> 
> > Most importantly we need to be able to make the existing "QEMU" component
> > in launch read-only to prevent people filing new bugs there, ideally with
> > a change in the description to point people to the new bug tracker.
> >
> > We can leave existing bugs in LP to continue their discussion. If there
> > are some we explicitly want in gitlab manually re-file them. Aside from
> > that if we periodically auto-close any stale bugs, after a while we'll
> > have culled launchpad down to zero.
> 
> Minimally, we should have an easy way to refile specific bugs
> that doesn't involve manual cut-n-paste. Most of the Arm bugs
> in launchpad are valid, for instance, I think, and I really
> don't want to be spending a day in unnecessary clerical work
> copying information into gitlab...

Auto-migrating content is easy enough. The challenge is user accounts,
because theres no mapping from launchpad to gitlab. If you just
import issues using a generic account, then you loose the communication
with the original bug report in a large portion of migrated bugs. This
makes it more appealing to leave existing bugs in the LP tracker until
they are resolved, auto-closed, or there is a compelling reason to move
to gitlab.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: Migrating to the gitlab issue tracker

[Peter Maydell]

On Fri, 30 Oct 2020 at 10:10, Daniel P. Berrangé <berrange@redhat.com> wrote:
> This
> makes it more appealing to leave existing bugs in the LP tracker until
> they are resolved, auto-closed, or there is a compelling reason to move
> to gitlab.

The compelling reason is that there is no way that I want to
have to consult two entirely separate bug tracking systems
to see what our reported bugs are. We must have an entry
in the new BTS for every 'live' bug, whether it was originally
reported to LP or to gitlab.

thanks
-- PMM

Re: Migrating to the gitlab issue tracker

[John Snow]

On 10/30/20 6:57 AM, Peter Maydell wrote:
> On Fri, 30 Oct 2020 at 10:10, Daniel P. Berrangé <berrange@redhat.com> wrote:
>> This
>> makes it more appealing to leave existing bugs in the LP tracker until
>> they are resolved, auto-closed, or there is a compelling reason to move
>> to gitlab.
> 
> The compelling reason is that there is no way that I want to
> have to consult two entirely separate bug tracking systems
> to see what our reported bugs are. We must have an entry
> in the new BTS for every 'live' bug, whether it was originally
> reported to LP or to gitlab.
> 

OK. I will try to investigate using the Launchpad API to pull our 
existing information, and then using the Gitlab API to re-create them. 
We will lose things like the list of subscribers and account 
information. Tags, Priority and Status can be preserved as labels. I'm 
not sure what the fate of attachments and other things are yet, I will see.


Current status, as of the start of this email:

New: 751 items
Opinion: 10 items
Invalid: 373 items
Won't Fix: 88 items
Expired: 438 items
Confirmed: 56 items
Triaged: 21 items
In Progress: 21 items
Fix Committed: 10 items
Fix Released: 1034 items
Incomplete (with response): 1 item
Incomplete (without response): 17 items


I think these things we do not have to migrate at all:

- Invalid
- Won't Fix
- Expired
- Fix Committed (Let's just graduate them to Released on LP.)
- Fix Released (Already done)


The Incomplete bugs we can likely avoid migrating; we can just let them 
expire as they are quite likely to do. This might leave us open to a 
situation where we might need to manually migrate or handle a bug or two 
that revives itself from the Incomplete tag, but it should be 
exceedingly few cases.

The Opinion tag is for, by description, "Doesn't fit with the project, 
but can be discussed." They don't have to be migrated, but sometimes 
this status was used "accidentally" by the reporter; so we can switch 
those back to Incomplete, Wontfix, or Released as appropriate.


That leaves these:

- New (755)
- Confirmed (59)
- Triaged (21)
- In Progress (6)


Likely we can migrate everything in the New/Confirmed/Triaged categories 
all as "new" bugs to Gitlab.

We could keep the "Confirmed" stage as a label as a hint for us on which 
ones to re-triage first, it should go quickly. We could also make a 
temporary "formerly-triaged" label for the "Triaged" state to give us a 
hint for a small handful of bugs to re-triage first.

That leaves "In Progress", which is a pretty small list now:

https://bugs.launchpad.net/qemu/+bugs?search=Search&field.status=In+Progress

Perhaps small enough to not worry about migrating these and just 
special-casing working through them for the migration, either:

1. Getting the attention of the contributor and getting them hooked into 
gitlab to own the new bug as a manual migration
2. Marking the bug as Fix Committed if it's done, or
3. Kicking the bug back to New if nobody is working on it.


Thanks,
--js

Re: Migrating to the gitlab issue tracker

[Thomas Huth]

On 05/11/2020 01.06, John Snow wrote:
> On 10/30/20 6:57 AM, Peter Maydell wrote:
>> On Fri, 30 Oct 2020 at 10:10, Daniel P. Berrangé <berrange@redhat.com> wrote:
>>> This
>>> makes it more appealing to leave existing bugs in the LP tracker until
>>> they are resolved, auto-closed, or there is a compelling reason to move
>>> to gitlab.
>>
>> The compelling reason is that there is no way that I want to
>> have to consult two entirely separate bug tracking systems
>> to see what our reported bugs are. We must have an entry
>> in the new BTS for every 'live' bug, whether it was originally
>> reported to LP or to gitlab.
[...]
> OK. I will try to investigate using the Launchpad API to pull our
> existing information, and then using the Gitlab API to re-create them. 

Before we migrate hundreds of bugs around, I think we should first check
which ones are stale, and which are still valid. So for all bugs that are in
"New" state and older than, let's say 2 years, I think we should add a
message a la:

 The QEMU project is currently considering to move its bug tracking to
another system. For this we need to know which bugs are still valid and
which could be closed already. Thus we are setting all older bugs to
"Incomplete" now. If you still think this bug report here is valid, then
please switch the state back to "New" within the next 60 days, otherwise
this report will be marked as "Expired". Thank you and sorry for the
inconvenience.

Then set the state to "Incomplete" and wait and see how many bugs expire in
60 days.

As a start, we could use the bug list from my QEMU bug dashboard here:

 http://people.redhat.com/~thuth/qemu/bugs-dashboard.html

See the "Expired" tab for the list with old bugs.

 Thomas


PS: I think we should also not migrate the bugs marked with "Wishlist" ...
if people are interested in new features, they should either contribute code
or pay for support, but opening feature requests often simply get ignored
completely, so we should likely rather close them now, too, instead of
migrating them.

Re: Migrating to the gitlab issue tracker

[Daniel P. Berrangé]

On Thu, Nov 05, 2020 at 07:14:47AM +0100, Thomas Huth wrote:
> On 05/11/2020 01.06, John Snow wrote:
> > On 10/30/20 6:57 AM, Peter Maydell wrote:
> >> On Fri, 30 Oct 2020 at 10:10, Daniel P. Berrangé <berrange@redhat.com> wrote:
> >>> This
> >>> makes it more appealing to leave existing bugs in the LP tracker until
> >>> they are resolved, auto-closed, or there is a compelling reason to move
> >>> to gitlab.
> >>
> >> The compelling reason is that there is no way that I want to
> >> have to consult two entirely separate bug tracking systems
> >> to see what our reported bugs are. We must have an entry
> >> in the new BTS for every 'live' bug, whether it was originally
> >> reported to LP or to gitlab.
> [...]
> > OK. I will try to investigate using the Launchpad API to pull our
> > existing information, and then using the Gitlab API to re-create them. 
> 
> Before we migrate hundreds of bugs around, I think we should first check
> which ones are stale, and which are still valid. So for all bugs that are in
> "New" state and older than, let's say 2 years, I think we should add a
> message a la:
> 
>  The QEMU project is currently considering to move its bug tracking to
> another system. For this we need to know which bugs are still valid and
> which could be closed already. Thus we are setting all older bugs to
> "Incomplete" now. If you still think this bug report here is valid, then
> please switch the state back to "New" within the next 60 days, otherwise
> this report will be marked as "Expired". Thank you and sorry for the
> inconvenience.
> 
> Then set the state to "Incomplete" and wait and see how many bugs expire in
> 60 days.

This sounds like a good idea.

I would further suggest that for bugs older than 5 years, we just close
them straightaway and skip this message. Users can always re-open or
re-file the bug in the very unlikely case they still care after 5 years.
We have some bugs that date from 2010, and just doesn't seem like we will
ever address those even if the user responded to the message by setting
it back to "New".

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: Migrating to the gitlab issue tracker

[John Snow]

On 11/5/20 1:14 AM, Thomas Huth wrote:
> On 05/11/2020 01.06, John Snow wrote:
>> On 10/30/20 6:57 AM, Peter Maydell wrote:
>>> On Fri, 30 Oct 2020 at 10:10, Daniel P. Berrangé <berrange@redhat.com> wrote:
>>>> This
>>>> makes it more appealing to leave existing bugs in the LP tracker until
>>>> they are resolved, auto-closed, or there is a compelling reason to move
>>>> to gitlab.
>>>
>>> The compelling reason is that there is no way that I want to
>>> have to consult two entirely separate bug tracking systems
>>> to see what our reported bugs are. We must have an entry
>>> in the new BTS for every 'live' bug, whether it was originally
>>> reported to LP or to gitlab.
> [...]
>> OK. I will try to investigate using the Launchpad API to pull our
>> existing information, and then using the Gitlab API to re-create them.
> 
> Before we migrate hundreds of bugs around, I think we should first check
> which ones are stale, and which are still valid. So for all bugs that are in
> "New" state and older than, let's say 2 years, I think we should add a
> message a la:
> 
>   The QEMU project is currently considering to move its bug tracking to
> another system. For this we need to know which bugs are still valid and
> which could be closed already. Thus we are setting all older bugs to
> "Incomplete" now. If you still think this bug report here is valid, then
> please switch the state back to "New" within the next 60 days, otherwise
> this report will be marked as "Expired". Thank you and sorry for the
> inconvenience.
> 

One reason to NOT do this is that if the bug does wind up being 
legitimate -- perhaps it is still a top google hit for searching a 
particular error string -- once we have migrated, there will be no 
recourse for the hapless googler.

We can leave a generic forwarder to the new tracker once we migrate, but 
there's no way to "re-open" the issue. If someone re-files on the new 
tracker, they won't be able to update the bug to leave a new breadcrumb.

However, if we migrate the bug first, we can leave breadcrumbs on the 
old tracker pointing to the new one, and then if the bug winds up being 
legitimate, googlers can follow the breadcrumb to the gitlab issue and 
either update that bug, reopen it, etc.

So I might actually, though it seems strange, recommend migrating all of 
these bugs but labeling any that are over that 2 year mark with "stale", 
then closing them on the new system.

> Then set the state to "Incomplete" and wait and see how many bugs expire in
> 60 days.
> 
> As a start, we could use the bug list from my QEMU bug dashboard here:
> 
>   http://people.redhat.com/~thuth/qemu/bugs-dashboard.html
> 
> See the "Expired" tab for the list with old bugs.
> 
>   Thomas
> 
> 
> PS: I think we should also not migrate the bugs marked with "Wishlist" ...
> if people are interested in new features, they should either contribute code
> or pay for support, but opening feature requests often simply get ignored
> completely, so we should likely rather close them now, too, instead of
> migrating them.
> 

I might recommend a similar thing here: migrate-then-close.



P.S.: I am playing around with the Launchpad API right now. I think what 
I can see as a "non-contributor" is enough for us to migrate, but just 
in case, can I receive elevated privileges?

--js

Re: Migrating to the gitlab issue tracker

[Daniel P. Berrangé]

On Thu, Nov 05, 2020 at 10:44:42AM -0500, John Snow wrote:
> On 11/5/20 1:14 AM, Thomas Huth wrote:
> > On 05/11/2020 01.06, John Snow wrote:
> > > On 10/30/20 6:57 AM, Peter Maydell wrote:
> > > > On Fri, 30 Oct 2020 at 10:10, Daniel P. Berrangé <berrange@redhat.com> wrote:
> > > > > This
> > > > > makes it more appealing to leave existing bugs in the LP tracker until
> > > > > they are resolved, auto-closed, or there is a compelling reason to move
> > > > > to gitlab.
> > > > 
> > > > The compelling reason is that there is no way that I want to
> > > > have to consult two entirely separate bug tracking systems
> > > > to see what our reported bugs are. We must have an entry
> > > > in the new BTS for every 'live' bug, whether it was originally
> > > > reported to LP or to gitlab.
> > [...]
> > > OK. I will try to investigate using the Launchpad API to pull our
> > > existing information, and then using the Gitlab API to re-create them.
> > 
> > Before we migrate hundreds of bugs around, I think we should first check
> > which ones are stale, and which are still valid. So for all bugs that are in
> > "New" state and older than, let's say 2 years, I think we should add a
> > message a la:
> > 
> >   The QEMU project is currently considering to move its bug tracking to
> > another system. For this we need to know which bugs are still valid and
> > which could be closed already. Thus we are setting all older bugs to
> > "Incomplete" now. If you still think this bug report here is valid, then
> > please switch the state back to "New" within the next 60 days, otherwise
> > this report will be marked as "Expired". Thank you and sorry for the
> > inconvenience.
> > 
> 
> One reason to NOT do this is that if the bug does wind up being legitimate
> -- perhaps it is still a top google hit for searching a particular error
> string -- once we have migrated, there will be no recourse for the hapless
> googler.

AFAIK, Google will index closed bugs, so they'll still appear in the
search results.

If we really want to, we could put a comment in the bugs we're about
to close, telling people that we're using gitlab now, and to re-file
their bug there if they care about it. I'm not sure that's needed
though, since it is no different a situation to what we have already
with the 1000's of bugs we've closed over the years.

> We can leave a generic forwarder to the new tracker once we migrate, but
> there's no way to "re-open" the issue. If someone re-files on the new
> tracker, they won't be able to update the bug to leave a new breadcrumb.
> 
> However, if we migrate the bug first, we can leave breadcrumbs on the old
> tracker pointing to the new one, and then if the bug winds up being
> legitimate, googlers can follow the breadcrumb to the gitlab issue and
> either update that bug, reopen it, etc.

IMHO they can just file a fresh bug in GitLab from scratch easily
enough by just copy+pasting the existin bug description. I don't
see a benefit in creating 100's of bugs in GitLab that we will
immediately close as being stale.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: Migrating to the gitlab issue tracker

[Thomas Huth]

On 05/11/2020 16.50, Daniel P. Berrangé wrote:
> On Thu, Nov 05, 2020 at 10:44:42AM -0500, John Snow wrote:
>> On 11/5/20 1:14 AM, Thomas Huth wrote:
>>> On 05/11/2020 01.06, John Snow wrote:
>>>> On 10/30/20 6:57 AM, Peter Maydell wrote:
>>>>> On Fri, 30 Oct 2020 at 10:10, Daniel P. Berrangé <berrange@redhat.com> wrote:
>>>>>> This
>>>>>> makes it more appealing to leave existing bugs in the LP tracker until
>>>>>> they are resolved, auto-closed, or there is a compelling reason to move
>>>>>> to gitlab.
>>>>>
>>>>> The compelling reason is that there is no way that I want to
>>>>> have to consult two entirely separate bug tracking systems
>>>>> to see what our reported bugs are. We must have an entry
>>>>> in the new BTS for every 'live' bug, whether it was originally
>>>>> reported to LP or to gitlab.
>>> [...]
>>>> OK. I will try to investigate using the Launchpad API to pull our
>>>> existing information, and then using the Gitlab API to re-create them.
>>>
>>> Before we migrate hundreds of bugs around, I think we should first check
>>> which ones are stale, and which are still valid. So for all bugs that are in
>>> "New" state and older than, let's say 2 years, I think we should add a
>>> message a la:
>>>
>>>   The QEMU project is currently considering to move its bug tracking to
>>> another system. For this we need to know which bugs are still valid and
>>> which could be closed already. Thus we are setting all older bugs to
>>> "Incomplete" now. If you still think this bug report here is valid, then
>>> please switch the state back to "New" within the next 60 days, otherwise
>>> this report will be marked as "Expired". Thank you and sorry for the
>>> inconvenience.
>>>
>>
>> One reason to NOT do this is that if the bug does wind up being legitimate
>> -- perhaps it is still a top google hit for searching a particular error
>> string -- once we have migrated, there will be no recourse for the hapless
>> googler.
> 
> AFAIK, Google will index closed bugs, so they'll still appear in the
> search results.
> 
> If we really want to, we could put a comment in the bugs we're about
> to close, telling people that we're using gitlab now, and to re-file
> their bug there if they care about it. I'm not sure that's needed
> though, since it is no different a situation to what we have already
> with the 1000's of bugs we've closed over the years.
> 
>> We can leave a generic forwarder to the new tracker once we migrate, but
>> there's no way to "re-open" the issue. If someone re-files on the new
>> tracker, they won't be able to update the bug to leave a new breadcrumb.
>>
>> However, if we migrate the bug first, we can leave breadcrumbs on the old
>> tracker pointing to the new one, and then if the bug winds up being
>> legitimate, googlers can follow the breadcrumb to the gitlab issue and
>> either update that bug, reopen it, etc.
> 
> IMHO they can just file a fresh bug in GitLab from scratch easily
> enough by just copy+pasting the existin bug description. I don't
> see a benefit in creating 100's of bugs in GitLab that we will
> immediately close as being stale.

I agree with Daniel. Please let's not clog the new bug tracker right from
the start with hundreds of bugs - that only makes it harder to focus on the
tickets that are really important. Let's use the migration instead to start
as clean as possible again.

 Thomas

Re: Migrating to the gitlab issue tracker

[Peter Maydell]

On Sun, 8 Nov 2020 at 09:01, Thomas Huth <thuth@redhat.com> wrote:
> I agree with Daniel. Please let's not clog the new bug tracker right from
> the start with hundreds of bugs - that only makes it harder to focus on the
> tickets that are really important. Let's use the migration instead to start
> as clean as possible again.

I really don't like doing this kind of thing. It basically
tells bug reporters "we don't care about your reports".
We ought to at least triage them. Certainly for arm a
lot of the reports in LP are real bug reports which we
shouldn't just drop on the floor.

thanks
-- PMM

Re: Migrating to the gitlab issue tracker

[Thomas Huth]

On 08/11/2020 12.58, Peter Maydell wrote:
> On Sun, 8 Nov 2020 at 09:01, Thomas Huth <thuth@redhat.com> wrote:
>> I agree with Daniel. Please let's not clog the new bug tracker right from
>> the start with hundreds of bugs - that only makes it harder to focus on the
>> tickets that are really important. Let's use the migration instead to start
>> as clean as possible again.
> 
> I really don't like doing this kind of thing. It basically
> tells bug reporters "we don't care about your reports".

But all those bugs that got stale and did not receive an answer within years
certainly give the same impression to the reporters.

> We ought to at least triage them. Certainly for arm a
> lot of the reports in LP are real bug reports which we
> shouldn't just drop on the floor.

Ok, then let's do it this way:

- Let's do *not* automatically close bugs directly

- Let's try to do at least some basic triage. If bugs are
  obviously still present, there is no need to mark them
  as incomplete. For all other bugs, let's ask whether they
  are still important and mark them ask incomplete so that
  they can expire if nobody cares anymore.

- Should I assign opened arm bugs that I find along the way
  to you, Peter, so you can triage them?

- Let's not auto-migrate any bugs within the next weeks and
  wait 'till the LP bug triage has been done.

- I think we could already update the links on the website
  to the new bug tracker at gitlab to get some more experience
  with the new bug tracker. That of course means that we would
  be using two bug tracker during the next weeks or months,
  but I think that's ok if we set a final date for the complete
  switch (e.g. at the end of the LP bug expiration period, so
  something like January 2021)

Sounds like a plan?

 Thomas

Re: Migrating to the gitlab issue tracker

[Daniel P. Berrangé]

On Sun, Nov 08, 2020 at 11:58:28AM +0000, Peter Maydell wrote:
> On Sun, 8 Nov 2020 at 09:01, Thomas Huth <thuth@redhat.com> wrote:
> > I agree with Daniel. Please let's not clog the new bug tracker right from
> > the start with hundreds of bugs - that only makes it harder to focus on the
> > tickets that are really important. Let's use the migration instead to start
> > as clean as possible again.
> 
> I really don't like doing this kind of thing. It basically
> tells bug reporters "we don't care about your reports".
> We ought to at least triage them. Certainly for arm a
> lot of the reports in LP are real bug reports which we
> shouldn't just drop on the floor.

Mostly it is just a reflection of the reality we find ourselves in where
we have more bug reports than we have willing maintainer time to investigate
and resolve. Regardless of whether the bug are open or closed, there are a
large number we clearly don't consider important, otherwise someone would
have already looked at them.

This isn't a comment on whether the bugs are valid or not, just on whether
the bugs look important/critical enough to be worth spending time on, and
unfortunately most come up short.

If we accept this, then I think bulk closing old bugs is a good solution.
It makes it clear to users sooner than the problem they face is unlikely
to be ever resolved, and so they can stop waiting in vain for something
that will never happen and focus on working around the problem they hit.

We will certainly close many valid bugs in doing this. Some of those may
in fact turn out to be things we will fix, or want to fix but that's OK.
We can still fix them even if the bug is closed. Or if we notice a bug
being closed that's important, we can re-open it.

IMHO on balance it is still better if we blindly auto-close the 261
expired bugs in Thomas' dashboard, and then manually re-open perhaps
15-20 that might have been valid, than to manually triage all 261 bugs.

This will leave more time to spend on the masses of other non-stale bugs
which are more likely to be useful to our current users.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: Migrating to the gitlab issue tracker

[Peter Maydell]

On Mon, 9 Nov 2020 at 10:11, Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> On Sun, Nov 08, 2020 at 11:58:28AM +0000, Peter Maydell wrote:
> > On Sun, 8 Nov 2020 at 09:01, Thomas Huth <thuth@redhat.com> wrote:
> > > I agree with Daniel. Please let's not clog the new bug tracker right from
> > > the start with hundreds of bugs - that only makes it harder to focus on the
> > > tickets that are really important. Let's use the migration instead to start
> > > as clean as possible again.
> >
> > I really don't like doing this kind of thing. It basically
> > tells bug reporters "we don't care about your reports".
> > We ought to at least triage them. Certainly for arm a
> > lot of the reports in LP are real bug reports which we
> > shouldn't just drop on the floor.
>
> Mostly it is just a reflection of the reality we find ourselves in where
> we have more bug reports than we have willing maintainer time to investigate
> and resolve. Regardless of whether the bug are open or closed, there are a
> large number we clearly don't consider important, otherwise someone would
> have already looked at them.

Yeah, I agree with this. But there's a corollary: unless we
somehow find more maintainer time in future to investigate
bug reports, the new gitlab bug tracker is just going to quickly
fill up with more unresolved bug reports, so why worry about
whether it's empty at the start or not ?

thanks
-- PMM

Re: Migrating to the gitlab issue tracker

[Thomas Huth]

On 05/11/2020 01.06, John Snow wrote:
> On 10/30/20 6:57 AM, Peter Maydell wrote:
>> On Fri, 30 Oct 2020 at 10:10, Daniel P. Berrangé <berrange@redhat.com> wrote:
>>> This
>>> makes it more appealing to leave existing bugs in the LP tracker until
>>> they are resolved, auto-closed, or there is a compelling reason to move
>>> to gitlab.
>>
>> The compelling reason is that there is no way that I want to
>> have to consult two entirely separate bug tracking systems
>> to see what our reported bugs are. We must have an entry
>> in the new BTS for every 'live' bug, whether it was originally
>> reported to LP or to gitlab.
>>
> 
> OK. I will try to investigate using the Launchpad API to pull our existing 
> information, and then using the Gitlab API to re-create them. We will lose 
> things like the list of subscribers and account information. Tags, Priority 
> and Status can be preserved as labels. I'm not sure what the fate of 
> attachments and other things are yet, I will see.

  Hi John,

since we are switched now the main git repo to gitlab, and many of the 
incomplete bugs on Launchpad already expired, what about switching the bug 
tracker to Gitlab now, too? How far did you get with your migration scripts?

  Thomas

Re: Migrating to the gitlab issue tracker

[John Snow]

On 1/21/21 5:57 AM, Thomas Huth wrote:
> On 05/11/2020 01.06, John Snow wrote:
>> On 10/30/20 6:57 AM, Peter Maydell wrote:
>>> On Fri, 30 Oct 2020 at 10:10, Daniel P. Berrangé 
>>> <berrange@redhat.com> wrote:
>>>> This
>>>> makes it more appealing to leave existing bugs in the LP tracker until
>>>> they are resolved, auto-closed, or there is a compelling reason to move
>>>> to gitlab.
>>>
>>> The compelling reason is that there is no way that I want to
>>> have to consult two entirely separate bug tracking systems
>>> to see what our reported bugs are. We must have an entry
>>> in the new BTS for every 'live' bug, whether it was originally
>>> reported to LP or to gitlab.
>>>
>>
>> OK. I will try to investigate using the Launchpad API to pull our 
>> existing information, and then using the Gitlab API to re-create them. 
>> We will lose things like the list of subscribers and account 
>> information. Tags, Priority and Status can be preserved as labels. I'm 
>> not sure what the fate of attachments and other things are yet, I will 
>> see.
> 
>   Hi John,
> 
> since we are switched now the main git repo to gitlab, and many of the 
> incomplete bugs on Launchpad already expired, what about switching the 
> bug tracker to Gitlab now, too? How far did you get with your migration 
> scripts?
> 
>   Thomas

Not very! Some small doodles exploring the API, but after a long PTO I 
haven't had a lot of free time since returning and haven't resumed 
prototyping with it yet.

If you have more time to look at it, you're more than welcome to.

--js

Re: Migrating to the gitlab issue tracker

[Alex Bennée]

John Snow <jsnow@redhat.com> writes:

> On 10/29/20 3:55 PM, Thomas Huth wrote:
>> On 29/10/2020 18.12, John Snow wrote:
>>> On 10/29/20 12:49 PM, Alistair Francis wrote:
>>>> On Thu, Oct 29, 2020 at 9:41 AM Cornelia Huck <cohuck@redhat.com> wrote:
>>>>>
>>>>> On Thu, 29 Oct 2020 12:01:27 -0400
>>>>> John Snow <jsnow@redhat.com> wrote:
>>>>>
>>>>>> If you're in the CC list, it's because you are listed in MAINTAINERS.
>>>>>
>>>>> <cleared the cc: list except for qemu-devel :)>
>>>>>
>>>>>>
>>>>>> Paolo's QEMU keynote this morning mentioned the possible use of the
>>>>>> Gitlab issue tracker instead of using Launchpad.
>>>>>>
>>>>>> I'm quite fond of the gitlab issue tracker, I think it works quite well
>>>>>> and it has pretty good and uncomplicated API access to it in order to
>>>>>> customize your workflow if you'd really like to.
>>>>>>
>>>>>> In experimenting with my mirror on gitlab though, I was unable to find a
>>>>>> way to configure it to send issue tracker notifications to the email
>>>>>> list. A move to gitlab would likely mean, then:
>>>>>>
>>>>>> 1. The cessation of (automatic) issue tracker mails to the list

It would miss this feature as sometimes I get wind of things I can track
down. On the other hand there is a fair bit of list noise at the end of
a release when stuff gets closed.

>>>>>> 2. The loss of the ability to update the issue tracker by replying to
>>>>>> said emails
>>>>>> 3. Anyone listed in MAINTAINERS would be expected to have a gitlab
>>>>>> account in order to interact with the issue tracker.

Not a problem for me at least - it's just another (2FA) account.

<snip>
>> So could somebody please enable the issue tracker there, so we can give it a
>> try? Phil? Alex? Stefan? ...?
>> 
>> If it works well, I can certainly help to get the links etc. in our website
>> fixed.
>> 
>
> Great! You are the primary bug wrangler, so if you are interested in 
> trialing it, I am interested in helping!
>
> If we enable the bug tracker, can we please add Thomas and myself as 
> 'Reporter' level contributors to QEMU so we can wrangle bugs?

I have enabled the issue tracker and assigned Thomas and you the first
bug.

> Info on permissions: https://docs.gitlab.com/ee/user/permissions.html
>
> There's also an old bug (2 years) about migrating Launchpad to Gitlab, 
> but there's been no movement.
>
> https://gitlab.com/gitlab-org/gitlab/-/issues/22600
>
> If we like the results of the trial, we'll need a convincing migration 
> strategy.

Can we extract data as a CSV from Launchpad?

-- 
Alex Bennée

Re: Migrating to the gitlab issue tracker

[John Snow]

On 10/30/20 6:26 AM, Alex Bennée wrote:
> Can we extract data as a CSV from Launchpad?

Not sure, I don't have maintainer access there. Thomas?

--js

Re: Migrating to the gitlab issue tracker

[Thomas Huth]

On 30/10/2020 13.53, John Snow wrote:
> On 10/30/20 6:26 AM, Alex Bennée wrote:
>> Can we extract data as a CSV from Launchpad?
> 
> Not sure, I don't have maintainer access there. Thomas?

I've never seen such an option. But (as you've already discovered) there is
an API for Launchpad which can be used for retrieving information about
bugs. I'm using it for my QEMU bug dashboard here:

 http://people.redhat.com/~thuth/qemu/bugs-dashboard.html

FWIW, sources can be found here:

 https://github.com/huth/openstack/tree/qemu

 Thomas

Re: Migrating to the gitlab issue tracker

[John Snow]

On 10/29/20 12:41 PM, Cornelia Huck wrote:
> On Thu, 29 Oct 2020 12:01:27 -0400
> John Snow <jsnow@redhat.com> wrote:
> 
>> If you're in the CC list, it's because you are listed in MAINTAINERS.
> 
> <cleared the cc: list except for qemu-devel :)>
> 
>>
>> Paolo's QEMU keynote this morning mentioned the possible use of the
>> Gitlab issue tracker instead of using Launchpad.
>>
>> I'm quite fond of the gitlab issue tracker, I think it works quite well
>> and it has pretty good and uncomplicated API access to it in order to
>> customize your workflow if you'd really like to.
>>
>> In experimenting with my mirror on gitlab though, I was unable to find a
>> way to configure it to send issue tracker notifications to the email
>> list. A move to gitlab would likely mean, then:
>>
>> 1. The cessation of (automatic) issue tracker mails to the list
>> 2. The loss of the ability to update the issue tracker by replying to
>> said emails
>> 3. Anyone listed in MAINTAINERS would be expected to have a gitlab
>> account in order to interact with the issue tracker.
> 
> The gitlab issue tracker is almost certainly is an improvement over
> launchpad (and I do have a gitlab account); but not being able to
> interact via email is at least annoying. I expect that not only
> maintainers will want to interact with bug reports?
> 

Nothing stopping reviewers or contributors from signing up and 
subscribing to labels or issues they care about... It will just be more 
opaque to the ebb and flow of the list.

There are still perhaps things we could do; a bot that generates weekly 
bug report summaries might be a solution.

>>
>> However, once you have a gitlab account, you DO gain the ability to
>> receive emails for issues; possibly only those tagged with labels that
>> you cared about -- giving a nice filtering mechanism to receive only
>> bugs you care about.
>>
>> Gitlab also does support individual accounts updating issues using a
>> generated personalized email address, so if the email workflow is
>> crucial to you, it is still available.
> 
> You mean that I can update via email, provided it's an address
> associated with my account?
> 

https://gitlab.com/qemu-project/qemu

Click the "bell" icon, choose "custom", and you can subscribe to issues 
project-wide if you'd like. (Reopen, New, Closed, Reassigned).

I started experimenting with using the gitlab issue tracker for my 
Python library project, I'll use it as an example here:

https://gitlab.com/jsnow/qemu/-/labels

You can "subscribe" to labels to be notified about tracker activity in 
just an area of your concern. Here I'm using "Python" and "QMP" labels 
for areas of concern for this topic area.

When an issue gets tagged with one of your subscribed labels, you'll 
receive a notification (I get an email) informing you of the new issue.

(Unfortunately, it looks like issues that are triaged to contain your 
tag after their initial creation only show you the tag change event and 
not the bug text. It might be the case that subscribing to *all* new 
bugs, but then subscribing to labels of concern is the way to go.)

You can reply directly to that email, or any update emails, to update 
the tracker.

Also, you can view your notification settings by going to 
https://gitlab.com/-/profile/notifications

and you can check out your notification settings per-group, per-project, 
etc.

Lastly, you can go to the issue tracker for a project e.g. 
https://gitlab.com/jsnow/qemu/-/issues and at the bottom (If you have 
permission, I assume?) you can click "Email a new issue to this 
project." and receive a special email address for you to use to create 
new issues:

  You can create a new issue inside this project by sending an email to 
the following email address:

The subject will be used as the title of the new issue, and the message 
will be the description. Quick actions and styling with Markdown are 
supported.

This is a private email address generated just for you. Anyone who gets 
ahold of it can create issues or merge requests as if they were you. You 
should reset it if that ever happens.

>>
>> I'm for it, or at least for beginning a pilot program where we
>> experiment with the idea for interested parties. I wanted to send up a
>> trial balloon to see how we were feeling about this.
>>
>> --js
> 
> 

Re: Migrating to the gitlab issue tracker

[Cornelia Huck]

On Thu, 29 Oct 2020 14:04:04 -0400
John Snow <jsnow@redhat.com> wrote:

> On 10/29/20 12:41 PM, Cornelia Huck wrote:
> > On Thu, 29 Oct 2020 12:01:27 -0400
> > John Snow <jsnow@redhat.com> wrote:
> >   
> >> If you're in the CC list, it's because you are listed in MAINTAINERS.  
> > 
> > <cleared the cc: list except for qemu-devel :)>
> >   
> >>
> >> Paolo's QEMU keynote this morning mentioned the possible use of the
> >> Gitlab issue tracker instead of using Launchpad.
> >>
> >> I'm quite fond of the gitlab issue tracker, I think it works quite well
> >> and it has pretty good and uncomplicated API access to it in order to
> >> customize your workflow if you'd really like to.
> >>
> >> In experimenting with my mirror on gitlab though, I was unable to find a
> >> way to configure it to send issue tracker notifications to the email
> >> list. A move to gitlab would likely mean, then:
> >>
> >> 1. The cessation of (automatic) issue tracker mails to the list
> >> 2. The loss of the ability to update the issue tracker by replying to
> >> said emails
> >> 3. Anyone listed in MAINTAINERS would be expected to have a gitlab
> >> account in order to interact with the issue tracker.  
> > 
> > The gitlab issue tracker is almost certainly is an improvement over
> > launchpad (and I do have a gitlab account); but not being able to
> > interact via email is at least annoying. I expect that not only
> > maintainers will want to interact with bug reports?
> >   
> 
> Nothing stopping reviewers or contributors from signing up and 
> subscribing to labels or issues they care about... It will just be more 
> opaque to the ebb and flow of the list.
> 
> There are still perhaps things we could do; a bot that generates weekly 
> bug report summaries might be a solution.

That might be useful. TBH, I'm not sure how many random people that are
not either the reporter or a maintainer anyway typically interact with
launchpad bugs, so requiring a gitlab account might not be that bad on
the whole (especially since people can still write an email to the
list).

> 
> >>
> >> However, once you have a gitlab account, you DO gain the ability to
> >> receive emails for issues; possibly only those tagged with labels that
> >> you cared about -- giving a nice filtering mechanism to receive only
> >> bugs you care about.
> >>
> >> Gitlab also does support individual accounts updating issues using a
> >> generated personalized email address, so if the email workflow is
> >> crucial to you, it is still available.  
> > 
> > You mean that I can update via email, provided it's an address
> > associated with my account?
> >   
> 
> https://gitlab.com/qemu-project/qemu
> 
> Click the "bell" icon, choose "custom", and you can subscribe to issues 
> project-wide if you'd like. (Reopen, New, Closed, Reassigned).
> 
> I started experimenting with using the gitlab issue tracker for my 
> Python library project, I'll use it as an example here:

[nice instructions stripped]

Thanks, that is helpful; I'll play with it a bit when I find some time.

Re: Migrating to the gitlab issue tracker

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 749 bytes --]

On Thu, Oct 29, 2020 at 12:01:27PM -0400, John Snow wrote:
> In experimenting with my mirror on gitlab though, I was unable to find a way
> to configure it to send issue tracker notifications to the email list. A
> move to gitlab would likely mean, then:
> 
> 1. The cessation of (automatic) issue tracker mails to the list

A bot could do this.

> 2. The loss of the ability to update the issue tracker by replying to said
> emails

This is too bad. It's something I liked about Launchpad.

> 3. Anyone listed in MAINTAINERS would be expected to have a gitlab account
> in order to interact with the issue tracker.

Maintainers should start running pull requests through the GitLab CI
anyway, so this is probably okay.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Migrating to the gitlab issue tracker

[John Snow]

On 10/30/20 5:16 AM, Stefan Hajnoczi wrote:
> On Thu, Oct 29, 2020 at 12:01:27PM -0400, John Snow wrote:
>> In experimenting with my mirror on gitlab though, I was unable to find a way
>> to configure it to send issue tracker notifications to the email list. A
>> move to gitlab would likely mean, then:
>>
>> 1. The cessation of (automatic) issue tracker mails to the list
> 
> A bot could do this.
> 

Yes, but someone would have to write it, and we'd have to host it 
somewhere. It might not worth be doing.

>> 2. The loss of the ability to update the issue tracker by replying to said
>> emails
> 
> This is too bad. It's something I liked about Launchpad.
> 

However, if you subscribe to the issue tracker using your gitlab 
account, you can reply to the emails you get to update the tracker that way!

https://gitlab.com/qemu-project/qemu and click the little bell near 
un/star and fork; click "custom" and subscribe to issue events.

>> 3. Anyone listed in MAINTAINERS would be expected to have a gitlab account
>> in order to interact with the issue tracker.
> 
> Maintainers should start running pull requests through the GitLab CI
> anyway, so this is probably okay.
> 

Sounds like the consensus so far, yes.

> Stefan
> 

Re: Migrating to the gitlab issue tracker

[Laszlo Ersek]

On 10/30/20 10:16, Stefan Hajnoczi wrote:
> On Thu, Oct 29, 2020 at 12:01:27PM -0400, John Snow wrote:
>> In experimenting with my mirror on gitlab though, I was unable to find a way
>> to configure it to send issue tracker notifications to the email list. A
>> move to gitlab would likely mean, then:
>>
>> 1. The cessation of (automatic) issue tracker mails to the list
> 
> A bot could do this.

I think a "bug traffic" mailing list (possibly but not necessarily
separate from the main qemu development list) is important.

> 
>> 2. The loss of the ability to update the issue tracker by replying to said
>> emails
> 
> This is too bad. It's something I liked about Launchpad.

To me personally: not too important. (I concede that this may have been
the one and only redeeming quality of Launchpad.)

> 
>> 3. Anyone listed in MAINTAINERS would be expected to have a gitlab account
>> in order to interact with the issue tracker.
> 
> Maintainers should start running pull requests through the GitLab CI
> anyway, so this is probably okay.

100% fine with me.

Thanks
Laszlo

Re: Migrating to the gitlab issue tracker

[Daniel P. Berrangé]

On Mon, Nov 02, 2020 at 02:57:49PM +0100, Laszlo Ersek wrote:
> On 10/30/20 10:16, Stefan Hajnoczi wrote:
> > On Thu, Oct 29, 2020 at 12:01:27PM -0400, John Snow wrote:
> >> In experimenting with my mirror on gitlab though, I was unable to find a way
> >> to configure it to send issue tracker notifications to the email list. A
> >> move to gitlab would likely mean, then:
> >>
> >> 1. The cessation of (automatic) issue tracker mails to the list
> > 
> > A bot could do this.
> 
> I think a "bug traffic" mailing list (possibly but not necessarily
> separate from the main qemu development list) is important.

What benefit is there to a bug traffic mailing list, as opposed to people
subscribing to direct bug notifications ? In both case people who are
interested in watching bugs can get the same content in their inboxes.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: Migrating to the gitlab issue tracker

[Eric Blake]

On 11/2/20 8:26 AM, Daniel P. Berrangé wrote:
> On Mon, Nov 02, 2020 at 02:57:49PM +0100, Laszlo Ersek wrote:
>> On 10/30/20 10:16, Stefan Hajnoczi wrote:
>>> On Thu, Oct 29, 2020 at 12:01:27PM -0400, John Snow wrote:
>>>> In experimenting with my mirror on gitlab though, I was unable to find a way
>>>> to configure it to send issue tracker notifications to the email list. A
>>>> move to gitlab would likely mean, then:
>>>>
>>>> 1. The cessation of (automatic) issue tracker mails to the list
>>>
>>> A bot could do this.
>>
>> I think a "bug traffic" mailing list (possibly but not necessarily
>> separate from the main qemu development list) is important.
> 
> What benefit is there to a bug traffic mailing list, as opposed to people
> subscribing to direct bug notifications ? In both case people who are
> interested in watching bugs can get the same content in their inboxes.

A mailing list would have archives (which can be helpful for archaeology
of past bug traffic even when you were not a gitlab subscriber), and
(depending on subscription settings) could permit additional
conversation in response to traffic by non-gitlab contributors.  But you
are right that a mailing list configured as read-only bug traffic (where
replies are not permitted) has no benefit over developers directly
subscribing to bug activity.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: Migrating to the gitlab issue tracker

[Laszlo Ersek]

On 11/02/20 15:42, Eric Blake wrote:
> On 11/2/20 8:26 AM, Daniel P. Berrangé wrote:
>> On Mon, Nov 02, 2020 at 02:57:49PM +0100, Laszlo Ersek wrote:
>>> On 10/30/20 10:16, Stefan Hajnoczi wrote:
>>>> On Thu, Oct 29, 2020 at 12:01:27PM -0400, John Snow wrote:
>>>>> In experimenting with my mirror on gitlab though, I was unable to find a way
>>>>> to configure it to send issue tracker notifications to the email list. A
>>>>> move to gitlab would likely mean, then:
>>>>>
>>>>> 1. The cessation of (automatic) issue tracker mails to the list
>>>>
>>>> A bot could do this.
>>>
>>> I think a "bug traffic" mailing list (possibly but not necessarily
>>> separate from the main qemu development list) is important.
>>
>> What benefit is there to a bug traffic mailing list, as opposed to people
>> subscribing to direct bug notifications ? In both case people who are
>> interested in watching bugs can get the same content in their inboxes.
> 
> A mailing list would have archives (which can be helpful for archaeology
> of past bug traffic even when you were not a gitlab subscriber), and
> (depending on subscription settings) could permit additional
> conversation in response to traffic by non-gitlab contributors.  But you
> are right that a mailing list configured as read-only bug traffic (where
> replies are not permitted) has no benefit over developers directly
> subscribing to bug activity.
> 

BTW I think this could be handled by an "auto-CC" or "default-CC"
feature in the bug tracker. Simply subscribe the list automatically to
every new bug reported.

Laszlo

Re: Migrating to the gitlab issue tracker

[Laszlo Ersek]

On 11/02/20 15:26, Daniel P. Berrangé wrote:
> On Mon, Nov 02, 2020 at 02:57:49PM +0100, Laszlo Ersek wrote:
>> On 10/30/20 10:16, Stefan Hajnoczi wrote:
>>> On Thu, Oct 29, 2020 at 12:01:27PM -0400, John Snow wrote:
>>>> In experimenting with my mirror on gitlab though, I was unable to find a way
>>>> to configure it to send issue tracker notifications to the email list. A
>>>> move to gitlab would likely mean, then:
>>>>
>>>> 1. The cessation of (automatic) issue tracker mails to the list
>>>
>>> A bot could do this.
>>
>> I think a "bug traffic" mailing list (possibly but not necessarily
>> separate from the main qemu development list) is important.
> 
> What benefit is there to a bug traffic mailing list, as opposed to people
> subscribing to direct bug notifications ? In both case people who are
> interested in watching bugs can get the same content in their inboxes.

People never subscribed to a particular bug can retroactively learn --
when it turns into an interest for them -- about actions on the bug,
over time. Bug tracker web UIs do not always present such a view. (IOW
it's not interchangeable with retroactively opening the ticket and
reading through it.)

Additionally, mailing list archives can be imported locally, for
indexing / searching, etc. Nothing beats a local search engine that has
access to years of bug traffic. Google may come somewhat close; bug
trackers' own search functionalities are hopeless in comparison (IME).

Thanks
Laszlo

Re: Migrating to the gitlab issue tracker

[Daniel P. Berrangé]

On Wed, Nov 04, 2020 at 06:03:26PM +0100, Laszlo Ersek wrote:
> On 11/02/20 15:26, Daniel P. Berrangé wrote:
> > On Mon, Nov 02, 2020 at 02:57:49PM +0100, Laszlo Ersek wrote:
> >> On 10/30/20 10:16, Stefan Hajnoczi wrote:
> >>> On Thu, Oct 29, 2020 at 12:01:27PM -0400, John Snow wrote:
> >>>> In experimenting with my mirror on gitlab though, I was unable to find a way
> >>>> to configure it to send issue tracker notifications to the email list. A
> >>>> move to gitlab would likely mean, then:
> >>>>
> >>>> 1. The cessation of (automatic) issue tracker mails to the list
> >>>
> >>> A bot could do this.
> >>
> >> I think a "bug traffic" mailing list (possibly but not necessarily
> >> separate from the main qemu development list) is important.
> > 
> > What benefit is there to a bug traffic mailing list, as opposed to people
> > subscribing to direct bug notifications ? In both case people who are
> > interested in watching bugs can get the same content in their inboxes.
> 
> People never subscribed to a particular bug can retroactively learn --
> when it turns into an interest for them -- about actions on the bug,
> over time. Bug tracker web UIs do not always present such a view. (IOW
> it's not interchangeable with retroactively opening the ticket and
> reading through it.)

I can't say I've ever come aross a situation where I cared about a bug
and found information missing in the web UI that I then found in the
mailing list archives. 

> Additionally, mailing list archives can be imported locally, for
> indexing / searching, etc. Nothing beats a local search engine that has
> access to years of bug traffic. Google may come somewhat close; bug
> trackers' own search functionalities are hopeless in comparison (IME).

If you subscribe to the project directly you get local archives which
can be indexed & searched too if that's important.

This just sounds like fairly niche requirements for which directly
subscribing to the project issue tracker will satisfy 99% of the time.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: Migrating to the gitlab issue tracker

[Laszlo Ersek]

On 11/04/20 18:19, Daniel P. Berrangé wrote:

> This just sounds like fairly niche requirements for which directly
> subscribing to the project issue tracker will satisfy 99% of the time.

OK.

Laszlo