Action: Setup working group for the GNU Toolchain proposal to migrate to LF IT managed services

Action: Setup working group for the GNU Toolchain proposal to migrate to LF IT managed services

[Carlos O'Donell]

TAC,

We discussed this proposal on the 2022-11-09 TAC meeting.

Here is the action with more details. Please review and provide comments.

I'm out for the next 2 weeks, but I wanted to start this ball rolling.

Action:

Setup a working group for the proposal to transition to LF IT managed services.

The WG is tasked by the TAC to initially enumerate in detail the GNU Toolchain
project services.

Details:

- Working group will enumerate in detail the GNU Toolchain project services
  for the projects including gcc, glibc, binutils and gdb.
- Working group will commit documents to the GTI TAC sphinx git repo with
  all of the enumerated service details.
- Working group will nominate a rep to report at GTI TAC meetings on the
  current status of the working group.
- GTI TAC and WG must determine next step after enumeration is complete.

Next step:

- Seek volunteers. Please respond to this email if you want to volunteer
  to be on the "LF IT managed services" working group and help enumerate
  services.

-- 
Cheers,
Carlos.

Re: Action: Setup working group for the GNU Toolchain proposal to migrate to LF IT managed services

[Joseph Myers]

On Fri, 11 Nov 2022, Carlos O'Donell wrote:

> Next step:
> 
> - Seek volunteers. Please respond to this email if you want to volunteer
>   to be on the "LF IT managed services" working group and help enumerate
>   services.

I'll volunteer to be on it, but, as noted at the meeting, I don't expect 
to do much on the enumeration before January.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Action: Setup working group for the GNU Toolchain proposal to migrate to LF IT managed services

[Carlos O'Donell]

On 11/11/22 18:36, Joseph Myers wrote:
> On Fri, 11 Nov 2022, Carlos O'Donell wrote:
> 
>> Next step:
>>
>> - Seek volunteers. Please respond to this email if you want to volunteer
>>   to be on the "LF IT managed services" working group and help enumerate
>>   services.
> 
> I'll volunteer to be on it, but, as noted at the meeting, I don't expect 
> to do much on the enumeration before January.
> 

I will also volunteer to be on it, and I can start documenting right away for all the
services I know glibc needs.

-- 
Cheers,
Carlos.

Re: Action: Setup working group for the GNU Toolchain proposal to migrate to LF IT managed services

[Joel Brobecker]

Hi everyone,

> >> - Seek volunteers. Please respond to this email if you want to volunteer
> >>   to be on the "LF IT managed services" working group and help enumerate
> >>   services.
> > 
> > I'll volunteer to be on it, but, as noted at the meeting, I don't expect 
> > to do much on the enumeration before January.
> > 
> 
> I will also volunteer to be on it, and I can start documenting right
> away for all the services I know glibc needs.

I can help with the GDB side of things. I don't know if I know all
of them, but I know some, at least.

-- 
Joel

Re: Action: Setup working group for the GNU Toolchain proposal to migrate to LF IT managed services

[Simon Marchi]

On 11/15/22 15:25, Joel Brobecker wrote:
> Hi everyone,
> 
>>>> - Seek volunteers. Please respond to this email if you want to volunteer
>>>>   to be on the "LF IT managed services" working group and help enumerate
>>>>   services.
>>>
>>> I'll volunteer to be on it, but, as noted at the meeting, I don't expect 
>>> to do much on the enumeration before January.
>>>
>>
>> I will also volunteer to be on it, and I can start documenting right
>> away for all the services I know glibc needs.
> 
> I can help with the GDB side of things. I don't know if I know all
> of them, but I know some, at least.

Hi Joel,

Let me know when you have started list, I'll review and try to complete
it to the best of my knowledge.

Simon

Re: Action: Setup working group for the GNU Toolchain proposal to migrate to LF IT managed services

[Joel Brobecker]

> Details:
> 
> - Working group will enumerate in detail the GNU Toolchain project services
>   for the projects including gcc, glibc, binutils and gdb.

Here is what Simon and I could think of for GDB:

* Git Repository
* Web-based navigation of the Git repository
  https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git
* git-hooks

* Website
    - Handwritten pages (in a CVS (!) repository)
    - scripts generating website contents (doc, ARI, etc)

* [methodology can be rediscussed, but for now, we have the following
  workflow]

  SSH access to the machine hosting the website, as the scripts above
  generating website contents are run by the release manager, after
  having ssh'ed onto sourceware.org.

* Wiki

* bug tracker (bugzilla)

* Not sure if we want to change this or not, but sourceware.org
  also offers FTP download of releases, pre-releases, and snapshots.

  Releases are made available on ftp.gnu.org as well, and I do not
  think we want to change that part.

* Some scripts spawned from the git-hooks, due to the binutils-gdb's
  git-hooks config:

  - /[...]/binutils-gdb.git/hooks-bin/commit-extra-checker.py

    Verifies that we do not have this issue:

        # With commits created using "git am" of patches sent via the gdb@ or
        # gdb-patches@ mailing list, it's possible that the author information
        # gets changed into "xxx via xxx@sourceware.org". Catch and reject those,
        # so the person doing the push can fix this before the push is allowed.

  - /sourceware/infra/bin/email-to-bugzilla

        Sends a copy of commit messages to bugzilla if commit
        has a PR number in it.

  - /git/binutils-gdb.git/hooks-bin/post-receive

        Calls the irker (IRC notification of new commits)

  We'll need a way to get those installed on the machine hosting
  the git repository, and hopefully also a convenient way for us
  to allow testing before deployment (right now, we SSH to
  sourceware.org).

* patchwork
  (https://patchwork.sourceware.org/project/gdb/list/)

  Simon reports that this instance doesn't get much use at the moment,
  because it gets filled so quickly and is impossible to keep up to date.

  I (Joel) think that the project really needs to have a proper
  review system. It's too time-consuming and ineffective to have each
  maintainer do its own tracking. So having a review system that GDB
  Maintaniers are willing to adopt is very important, IMO. Maybe
  patchwork can fit the bill, or maybe it might be another system.
  Something for Simon and I perhaps to discuss with the other GDB
  maintainers?

-- 
Joel

Re: Action: Setup working group for the GNU Toolchain proposal to migrate to LF IT managed services

[Joel Brobecker]

I forgot one item that Simon pointed out to me (sorry!)

> Here is what Simon and I could think of for GDB:
> 
> * Git Repository
> * Web-based navigation of the Git repository
>   https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git
> * git-hooks
> 
> * Website
>     - Handwritten pages (in a CVS (!) repository)
>     - scripts generating website contents (doc, ARI, etc)
> 
> * [methodology can be rediscussed, but for now, we have the following
>   workflow]
> 
>   SSH access to the machine hosting the website, as the scripts above
>   generating website contents are run by the release manager, after
>   having ssh'ed onto sourceware.org.
> 
> * Wiki
> 
> * bug tracker (bugzilla)
> 
> * Not sure if we want to change this or not, but sourceware.org
>   also offers FTP download of releases, pre-releases, and snapshots.
> 
>   Releases are made available on ftp.gnu.org as well, and I do not
>   think we want to change that part.
> 
> * Some scripts spawned from the git-hooks, due to the binutils-gdb's
>   git-hooks config:
> 
>   - /[...]/binutils-gdb.git/hooks-bin/commit-extra-checker.py
> 
>     Verifies that we do not have this issue:
> 
>         # With commits created using "git am" of patches sent via the gdb@ or
>         # gdb-patches@ mailing list, it's possible that the author information
>         # gets changed into "xxx via xxx@sourceware.org". Catch and reject those,
>         # so the person doing the push can fix this before the push is allowed.
> 
>   - /sourceware/infra/bin/email-to-bugzilla
> 
>         Sends a copy of commit messages to bugzilla if commit
>         has a PR number in it.
> 
>   - /git/binutils-gdb.git/hooks-bin/post-receive
> 
>         Calls the irker (IRC notification of new commits)
> 
>   We'll need a way to get those installed on the machine hosting
>   the git repository, and hopefully also a convenient way for us
>   to allow testing before deployment (right now, we SSH to
>   sourceware.org).
> 
> * patchwork
>   (https://patchwork.sourceware.org/project/gdb/list/)
> 
>   Simon reports that this instance doesn't get much use at the moment,
>   because it gets filled so quickly and is impossible to keep up to date.
> 
>   I (Joel) think that the project really needs to have a proper
>   review system. It's too time-consuming and ineffective to have each
>   maintainer do its own tracking. So having a review system that GDB
>   Maintaniers are willing to adopt is very important, IMO. Maybe
>   patchwork can fit the bill, or maybe it might be another system.
>   Something for Simon and I perhaps to discuss with the other GDB
>   maintainers?

* buildbot
  (https://builder.sourceware.org/buildbot/#/builders?tags=gdb).

-- 
Joel

Re: Action: Setup working group for the GNU Toolchain proposal to migrate to LF IT managed services

[Joseph Myers]

On Mon, 21 Nov 2022, Joel Brobecker wrote:

> * bug tracker (bugzilla)

I think this needs to go into more details.  Details of incoming email 
handling (some Bugzilla installations don't use incoming email, we need to 
be explicit about how it's a key feature used in our installations), 
details of outgoing email handling, details of local changes to the 
Bugzilla installation and how account creation is handled, for example.

>   - /sourceware/infra/bin/email-to-bugzilla
> 
>         Sends a copy of commit messages to bugzilla if commit
>         has a PR number in it.

The fact that this currently seems to use SQL access to the database is a 
really important thing to include in the list of services.  Remember that 
we're trying for more isolation of components with minimal interfaces 
between them, to improve security.  So if this script could be changed or 
rewritten to use the (public) REST interface instead of SQL access to 
check for whether bugs exist, that would be helpful.  (The fact that it 
sends email to add to Bugzilla is also relevant, because it means that any 
system running this script needs to be able to send email - and for any 
system sending email, it will be necessary to avoid losing outgoing email 
if it's a transient system and there's a transient email problem.)

In general, details of exactly what interfaces are used by components to 
interact with others - especially if they make any assumptions about 
direct database or filesystem access, or about different services being 
hosted on the same system - are really important.  (This would then give a 
list of cases where we should *change* the interfaces used to remove such 
dependencies - for example, using the public read-only REST API to extract 
information from Bugzilla instead of SQL access.)

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Action: Setup working group for the GNU Toolchain proposal to migrate to LF IT managed services

[David Edelsohn]

[-- Attachment #1: Type: text/plain, Size: 2072 bytes --]

On Mon, Nov 21, 2022 at 3:56 PM Joseph Myers <joseph@codesourcery.com>
wrote:

> On Mon, 21 Nov 2022, Joel Brobecker wrote:
>
> > * bug tracker (bugzilla)
>
> I think this needs to go into more details.  Details of incoming email
> handling (some Bugzilla installations don't use incoming email, we need to
> be explicit about how it's a key feature used in our installations),
> details of outgoing email handling, details of local changes to the
> Bugzilla installation and how account creation is handled, for example.
>
> >   - /sourceware/infra/bin/email-to-bugzilla
> >
> >         Sends a copy of commit messages to bugzilla if commit
> >         has a PR number in it.
>
> The fact that this currently seems to use SQL access to the database is a
> really important thing to include in the list of services.  Remember that
> we're trying for more isolation of components with minimal interfaces
> between them, to improve security.  So if this script could be changed or
> rewritten to use the (public) REST interface instead of SQL access to
> check for whether bugs exist, that would be helpful.  (The fact that it
> sends email to add to Bugzilla is also relevant, because it means that any
> system running this script needs to be able to send email - and for any
> system sending email, it will be necessary to avoid losing outgoing email
> if it's a transient system and there's a transient email problem.)
>
> In general, details of exactly what interfaces are used by components to
> interact with others - especially if they make any assumptions about
> direct database or filesystem access, or about different services being
> hosted on the same system - are really important.  (This would then give a
> list of cases where we should *change* the interfaces used to remove such
> dependencies - for example, using the public read-only REST API to extract
> information from Bugzilla instead of SQL access.)
>
>
Joel and Simon,

Can the GDB analysis be updated to the level of detail that Carlos provided
for GLIBC in his separate message?

Thanks, David

[-- Attachment #2: Type: text/html, Size: 2610 bytes --]

Re: Action: Setup working group for the GNU Toolchain proposal to migrate to LF IT managed services

[Joel Brobecker]

> Joel and Simon,
> 
> Can the GDB analysis be updated to the level of detail that Carlos provided
> for GLIBC in his separate message?

I was planning on looking at this over the weekend, but unfortunately
it was too full. I will try to do it next weekend.

-- 
Joel

Re: Action: Setup working group for the GNU Toolchain proposal to migrate to LF IT managed services

[Simon Marchi]

On 11/21/22 15:49, Joseph Myers wrote:
> On Mon, 21 Nov 2022, Joel Brobecker wrote:
> 
>> * bug tracker (bugzilla)
> 
> I think this needs to go into more details.  Details of incoming email 
> handling (some Bugzilla installations don't use incoming email, we need to 
> be explicit about how it's a key feature used in our installations), 
> details of outgoing email handling, details of local changes to the 
> Bugzilla installation and how account creation is handled, for example.

I wasn't aware that Bugzilla could accept posts by email (I personally
don't use it).  I am also unaware of any local Bugzilla changes.  Like I
said in the meeting, I never touched at the infrastructure, I just know
it as a "black box" user.

>>   - /sourceware/infra/bin/email-to-bugzilla
>>
>>         Sends a copy of commit messages to bugzilla if commit
>>         has a PR number in it.
> 
> The fact that this currently seems to use SQL access to the database is a 
> really important thing to include in the list of services.  Remember that 
> we're trying for more isolation of components with minimal interfaces 
> between them, to improve security.  So if this script could be changed or 
> rewritten to use the (public) REST interface instead of SQL access to 
> check for whether bugs exist, that would be helpful.  (The fact that it 
> sends email to add to Bugzilla is also relevant, because it means that any 
> system running this script needs to be able to send email - and for any 
> system sending email, it will be necessary to avoid losing outgoing email 
> if it's a transient system and there's a transient email problem.)
> 
> In general, details of exactly what interfaces are used by components to 
> interact with others - especially if they make any assumptions about 
> direct database or filesystem access, or about different services being 
> hosted on the same system - are really important.  (This would then give a 
> list of cases where we should *change* the interfaces used to remove such 
> dependencies - for example, using the public read-only REST API to extract 
> information from Bugzilla instead of SQL access.)

Ok, maybe I misunderstood the point of the list then.  I thought it was
a list of functional requirements, what needs to work from a user point
of view, regardless of how it's implemented.

[Implementation detail: I know about the email-to-bugzilla script,
because Joel sent it to me lately.  I am in the process of rewriting it
in Python (if it's in Perl, I just can't contribute, sorry) to make it
recognize bugzilla URLs in commit messages.  The script accesses the DB
directly just to see if a given bug number exists.  If Bugzilla has a
proper REST interface, it should be easy to change.]

I'll go over the list again, see if I can add some details, but it
certainly won't be in as much details as what Carlos did.

Simon

Re: Action: Setup working group for the GNU Toolchain proposal to migrate to LF IT managed services

[Simon Marchi]

Here's a revised list for GDB, I added some user-facing / functional
requirements I would think of.  It's probably still lacking some of the
implementation details that you are looking for, but I probably don't
know them, and it's hard to know what you don't know.

* Git Repository
  * Need push access
  * Ability for project maintainers to request push access for new
    contributors (or to do it themselves)
  * Ability for users to create user-branches (e.g. users/simark/foo),
    which will be ignored by the notification-sending script.

* git hooks
  * Need to be able to manage git hooks (https://github.com/AdaCore/git-hooks) on
    the server (right now, ssh to sourceware.org)
  * binutils-gdb.git/hooks-bin/commit-extra-checker.py

    Verifies that we do not have this issue:

        # With commits created using "git am" of patches sent via the gdb@ or
        # gdb-patches@ mailing list, it's possible that the author information
        # gets changed into "xxx via xxx@sourceware.org". Catch and reject those,
        # so the person doing the push can fix this before the push is allowed.

  * /sourceware/infra/bin/email-to-bugzilla

        Sends a copy of commit messages to bugzilla if commit
        has a PR number in it.

  * binutils-gdb.git/hooks-bin/post-receive

        Calls the irker (IRC notification of new commits)

  * Something sends emails about commits to the gdb-cvs mailing list

* Web-based navigation of the Git repository
  * Need to be able to browse the git repository online
  * https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git
  * If it exists, a more featureful git web frontends would be nice, one
    that allows searching files by name for instance.

* email-to-bugzilla script
  * The goal is that when a commit is pushed to the git repository, we
    check for any mention of a Bugzilla bugs, and post a message to those
    bugs
  * It accesses the Bugzilla database directly to check if a given bug
    number exists (could easily be changed to use some HTTP request)
  * It posts to bugzilla by sending an email to a local email account

* mailing lists
  * Those I know for sure are used / useful:
    * gdb@
    * gdb-announce@
    * gdb-cvs@ (now misnaed, but can be used to follow commits to the repo)
    * gdb-patches@
    * gdb-prs@
    * gdb-testers@
  * Not sure about these:
    * gdbadmin@: seems useful for whoever is responsible for snapshot
      generation or other GDB-related automated task.  It seems a bit
      spammy to send an email every day when the task succeeds.

      The daily commits in binutils-gdb that bump the date are made using
      this email address as the From, not sure if it matters
  * Pretty sure we can ignore these:
    * gdb-testresults@
    * gdb-patches-prs@:
    * gdb-webpages-cvs@

  * We also have a "global maintainers" list hosted at AdaCore.  It could
    make sense to move it to the main infrastructure.  Maybe Joel knows if
    there's a reason it's separate.

* Mailing list web interface
  * The original one, which I find not so useful now that we have
    public-inbox: https://sourceware.org/pipermail/gdb-patches/
  * However, old links need to keep working
  * public-inbox or something equivalent is necessary:
    * have a way to browse easily message threads that span multiple
      months / years
    * have a way to download raw messages, to apply patches locally

* daily snapshot generation
  * We have a script that generates daily tarballs (I don't know where
    or how)

* daily bump
  * We have a script that does a daily commit in the binutils-gdb repo
    to update a date (I don't know where or how)

* Website
  * Handwritten pages (in a CVS (!) repository)
  * scripts generating website contents (doc, ARI, etc)
  * SSH access to the machine hosting the website is used, as the scripts
    above generating website contents are run by the release manager, after
    having ssh'ed onto sourceware.org.

* Wiki
  * Currently, anyone with write access can give someone else write access.
  * At the very least, we need a way for project maintainers to request
    write access for a new member, or do it themselves

* bug tracker (bugzilla)
  * Sends notifications of new bugs and comments to gdb-prs
  * Sends notifications of comments on a big to users who are watching
    that bug
  * Accepts replies by email, both from users and from the
    email-to-bugzilla script.

* Release hosting
  * Releases, the release manager must be able to upload there
    * ftp://sourceware.org/pub/gdb/releases/
    * http://sourceware.org/pub/gdb/releases/
  * Snapshots, the daily script needs to be able to upload there
    * https://sourceware.org/pub/gdb/snapshots/

* patchwork (https://patchwork.sourceware.org/project/gdb/list/)
  * It receives emails from the gdb-patches mailing list
  * This instance doesn't get much use at the moment, because it gets
    filled so quickly and is impossible to keep up to date.

Simon

Re: Action: Setup working group for the GNU Toolchain proposal to migrate to LF IT managed services

[David Edelsohn]

[-- Attachment #1: Type: text/plain, Size: 5329 bytes --]

On Tue, Dec 6, 2022 at 12:42 PM Simon Marchi <simon.marchi@efficios.com>
wrote:

> Here's a revised list for GDB, I added some user-facing / functional
> requirements I would think of.  It's probably still lacking some of the
> implementation details that you are looking for, but I probably don't
> know them, and it's hard to know what you don't know.
>
> * Git Repository
>   * Need push access
>   * Ability for project maintainers to request push access for new
>     contributors (or to do it themselves)
>   * Ability for users to create user-branches (e.g. users/simark/foo),
>     which will be ignored by the notification-sending script.
>
> * git hooks
>   * Need to be able to manage git hooks (
> https://github.com/AdaCore/git-hooks) on
>     the server (right now, ssh to sourceware.org)
>   * binutils-gdb.git/hooks-bin/commit-extra-checker.py
>
>     Verifies that we do not have this issue:
>
>         # With commits created using "git am" of patches sent via the gdb@
> or
>         # gdb-patches@ mailing list, it's possible that the author
> information
>         # gets changed into "xxx via xxx@sourceware.org". Catch and
> reject those,
>         # so the person doing the push can fix this before the push is
> allowed.
>
>   * /sourceware/infra/bin/email-to-bugzilla
>
>         Sends a copy of commit messages to bugzilla if commit
>         has a PR number in it.
>
>   * binutils-gdb.git/hooks-bin/post-receive
>
>         Calls the irker (IRC notification of new commits)
>
>   * Something sends emails about commits to the gdb-cvs mailing list
>
> * Web-based navigation of the Git repository
>   * Need to be able to browse the git repository online
>   * https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git
>   * If it exists, a more featureful git web frontends would be nice, one
>     that allows searching files by name for instance.
>
> * email-to-bugzilla script
>   * The goal is that when a commit is pushed to the git repository, we
>     check for any mention of a Bugzilla bugs, and post a message to those
>     bugs
>   * It accesses the Bugzilla database directly to check if a given bug
>     number exists (could easily be changed to use some HTTP request)
>   * It posts to bugzilla by sending an email to a local email account
>
> * mailing lists
>   * Those I know for sure are used / useful:
>     * gdb@
>     * gdb-announce@
>     * gdb-cvs@ (now misnaed, but can be used to follow commits to the
> repo)
>     * gdb-patches@
>     * gdb-prs@
>     * gdb-testers@
>   * Not sure about these:
>     * gdbadmin@: seems useful for whoever is responsible for snapshot
>       generation or other GDB-related automated task.  It seems a bit
>       spammy to send an email every day when the task succeeds.
>
>       The daily commits in binutils-gdb that bump the date are made using
>       this email address as the From, not sure if it matters
>   * Pretty sure we can ignore these:
>     * gdb-testresults@
>     * gdb-patches-prs@:
>     * gdb-webpages-cvs@
>
>   * We also have a "global maintainers" list hosted at AdaCore.  It could
>     make sense to move it to the main infrastructure.  Maybe Joel knows if
>     there's a reason it's separate.
>
> * Mailing list web interface
>   * The original one, which I find not so useful now that we have
>     public-inbox: https://sourceware.org/pipermail/gdb-patches/
>   * However, old links need to keep working
>   * public-inbox or something equivalent is necessary:
>     * have a way to browse easily message threads that span multiple
>       months / years
>     * have a way to download raw messages, to apply patches locally
>
> * daily snapshot generation
>   * We have a script that generates daily tarballs (I don't know where
>     or how)
>
> * daily bump
>   * We have a script that does a daily commit in the binutils-gdb repo
>     to update a date (I don't know where or how)
>
> * Website
>   * Handwritten pages (in a CVS (!) repository)
>   * scripts generating website contents (doc, ARI, etc)
>   * SSH access to the machine hosting the website is used, as the scripts
>     above generating website contents are run by the release manager, after
>     having ssh'ed onto sourceware.org.
>
> * Wiki
>   * Currently, anyone with write access can give someone else write access.
>   * At the very least, we need a way for project maintainers to request
>     write access for a new member, or do it themselves
>
> * bug tracker (bugzilla)
>   * Sends notifications of new bugs and comments to gdb-prs
>   * Sends notifications of comments on a big to users who are watching
>     that bug
>   * Accepts replies by email, both from users and from the
>     email-to-bugzilla script.
>
> * Release hosting
>   * Releases, the release manager must be able to upload there
>     * ftp://sourceware.org/pub/gdb/releases/
>     * http://sourceware.org/pub/gdb/releases/
>   * Snapshots, the daily script needs to be able to upload there
>     * https://sourceware.org/pub/gdb/snapshots/
>
> * patchwork (https://patchwork.sourceware.org/project/gdb/list/)
>   * It receives emails from the gdb-patches mailing list
>   * This instance doesn't get much use at the moment, because it gets
>     filled so quickly and is impossible to keep up to date.
>

Thanks for the additional detail!

Thanks, David

[-- Attachment #2: Type: text/html, Size: 7082 bytes --]

Re: Action: Setup working group for the GNU Toolchain proposal to migrate to LF IT managed services

[Joel Brobecker]

Hi Simon and everyone,

> Here's a revised list for GDB, I added some user-facing / functional
> requirements I would think of.  It's probably still lacking some of the
> implementation details that you are looking for, but I probably don't
> know them, and it's hard to know what you don't know.

I think Simon actually did a great job on that second pass, and
I found in doing my own pass that I didn't have very much I could
add, just some details here and there. Here is the updated version
of the document.

* Git Repository
  * Need push access
  * Ability for project maintainers to request push access for new
    contributors (or to do it themselves)
  * Ability for users to create user-branches (e.g. users/simark/foo),
    which will be ignored by the notification-sending script.

* git hooks

  * git-hooks (https://github.com/AdaCore/git-hooks) installed in
    the binutils-gdb.git repository.

  * binutils-gdb.git's git-hoks configuration pointing to the following
    scripts locally installed on the machine hosting the repository,
    to which the binutils & GDB admins need access (currently done
    via SSH access)

      - binutils-gdb.git/hooks-bin/email_to.py
        Small python script which determine which mailing-list(s) to use
        when sending commit email notifications. No special requirements
        other than a Python3 interpreter.

      - binutils-gdb.git/hooks-bin/commit-extra-checker.py

        Verifies that we do not have this issue:

        # With commits created using "git am" of patches sent via the gdb@ or
        # gdb-patches@ mailing list, it's possible that the author information
        # gets changed into "xxx via xxx@sourceware.org". Catch and reject those,
        # so the person doing the push can fix this before the push is allowed.

        No special requirements other than a Python3 interpreter.

      - /git/binutils-gdb.git/hooks-bin/style_checker

        An empty script at the moment (set because the git-hooks require
        it to be set).

      - /sourceware/infra/bin/email-to-bugzilla

        A perl script whose maintainorship is unclear. Comments at
        the beginning of the script mention David Hampton <hampton@cisco.com>
        as contributor and Greg A. Woods <woods@web.net> as having "greatly
        hacked" it. Beyond that, don't know.

          . The goal is that when a commit is pushed to the git
            repository, we check for any mention of a Bugzilla bugs, and
            post a message to those bugs
          . It accesses the Bugzilla database directly to check if a given bug
            number exists (could easily be changed to use some HTTP request)
          . It posts to bugzilla by sending an email to a local email account
            (Joel: I think sourceware-bugzilla@localhost)

        Joel's note: If I understood the script correctly, then there
        must be a handler in the local MTA to catch those emails and
        send them to bugzilla for further processing. Don't know how
        that works, though.

      - binutils-gdb.git/hooks-bin/post-receive

        Bash wrapper scripts which essentially calls the following
        script: /usr/local/bin/irkerhook.py. I suspect the origin
        of this script is this github repository:
        https://gitlab.com/esr/irker

        If not, ISTR that it was Tom Tromey who got it installed,
        so we could ask him.

  * The git-hooks themselves send emails about commits to the gdb-cvs
    mailing list (each repository configures the destination, and for
    binutils-gdb, this this a "dynamic" configuration via the email_to.py
    script mentioned above allowing the destination to vary depending
    on which files got modified, whether they are binutils fils or GDB
    file, or both).

* Web-based navigation of the Git repository
  * Need to be able to browse the git repository online
  * https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git
  * If it exists, a more featureful git web frontends would be nice, one
    that allows searching files by name for instance.
  * URLs to a given commit in the Web UI should be easily computable
    using their SHA1; e.g.
        "https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=%(rev)s"
    We use those in the commit emails being sent by the git-hooks.

* mailing lists
  * Those I know for sure are used / useful:
    * gdb@
    * gdb-announce@
    * gdb-cvs@ (now misnaed, but can be used to follow commits to the repo)
    * gdb-patches@
    * gdb-prs@
    * gdb-testers@
  * Not sure about these:
    * gdbadmin@: seems useful for whoever is responsible for snapshot
      generation or other GDB-related automated task.  It seems a bit
      spammy to send an email every day when the task succeeds.

      The daily commits in binutils-gdb that bump the date are made using
      this email address as the From, not sure if it matters
  * Pretty sure we can ignore these:
    * gdb-testresults@
    * gdb-patches-prs@:
    * gdb-webpages-cvs@

  * We also have a "global maintainers" list hosted at AdaCore.  It could
    make sense to move it to the main infrastructure.  Maybe Joel knows if
    there's a reason it's separate.

    No reason to keep it separate. It just started as an email alias
    on my own home server, then moved to being hosted by AdaCore to
    avoid having me maintaining a working email configuration on my
    server. Agree it might make sense to move, but I would say email
    discussions between Global Maintainers should remain private,
    or at least closed.

* Mailing list web interface
  * The original one, which I find not so useful now that we have
    public-inbox: https://sourceware.org/pipermail/gdb-patches/
  * However, old links need to keep working
  * public-inbox or something equivalent is necessary:
    * have a way to browse easily message threads that span multiple
      months / years
    * have a way to download raw messages, to apply patches locally

* daily snapshot generation

  * We have a script that generates daily tarballs:

    This is currently a collection of shell scripts stored on sourceware
    at /cvs/gdbadmin/ss, and tracked in a (ahem) CVS repository in
    /cvs/gdbadmin/ss.

    These are incredibly over-engineered, with poor error investigation
    support. But the technology used is simple bash scripting.

* daily bump

  * We have a gdbadmin user crontab entry which calls a script in
    the ss repository mentioned above daily, once for the master branch,
    once for the current binutils release branch, and once again for
    the current gdb release branch.

    Simple shell script.

* Website
  * Handwritten pages in a couple of CVS repositories:
    - The gnu.org copy is maintained on https://savannah.gnu.org/;
    - The sourceware.org copy is maintained on sourceware.org itself,
      in /cvs/gdb/htdocs
    - It used to be that both versions were kept in sync by duplicating
      all the commits in both repositories. But this was recently changed
      in favor of installing redirects from the gnu.org website to
      the sourceware.org. This was the first step towards a possible
      transition of the web-pages to Git.
  * scripts generating website contents (doc, ARI, etc), also part
    of the "ss" repository mentioned above.
  * SSH access to the machine hosting the website is used, as the scripts
    above generating website contents are run by the release manager, after
    having ssh'ed onto sourceware.org.

* Wiki
  * Currently, anyone with write access can give someone else write access.
  * At the very least, we need a way for project maintainers to request
    write access for a new member, or do it themselves

* bug tracker (bugzilla)
  * Sends notifications of new bugs and comments to gdb-prs
  * Sends notifications of comments on a bug to users who are watching
    that bug
  * Accepts replies by email, both from users and from the
    email-to-bugzilla script.

* Release hosting
  * Releases, the release manager must be able to upload there
    * ftp://sourceware.org/pub/gdb/releases/
    * http://sourceware.org/pub/gdb/releases/
  * Snapshots, the daily script needs to be able to upload there
    * https://sourceware.org/pub/gdb/snapshots/

* patchwork (https://patchwork.sourceware.org/project/gdb/list/)
  * It receives emails from the gdb-patches mailing list
  * This instance doesn't get much use at the moment, because it gets
    filled so quickly and is impossible to keep up to date.

-- 
Joel