Re: Action: Setup working group for the GNU Toolchain proposal to migrate to LF IT managed services

[Joel Brobecker <brobecker@adacore.com>]

Hi Simon and everyone,

> Here's a revised list for GDB, I added some user-facing / functional
> requirements I would think of.  It's probably still lacking some of the
> implementation details that you are looking for, but I probably don't
> know them, and it's hard to know what you don't know.

I think Simon actually did a great job on that second pass, and
I found in doing my own pass that I didn't have very much I could
add, just some details here and there. Here is the updated version
of the document.

* Git Repository
  * Need push access
  * Ability for project maintainers to request push access for new
    contributors (or to do it themselves)
  * Ability for users to create user-branches (e.g. users/simark/foo),
    which will be ignored by the notification-sending script.

* git hooks

  * git-hooks (https://github.com/AdaCore/git-hooks) installed in
    the binutils-gdb.git repository.

  * binutils-gdb.git's git-hoks configuration pointing to the following
    scripts locally installed on the machine hosting the repository,
    to which the binutils & GDB admins need access (currently done
    via SSH access)

      - binutils-gdb.git/hooks-bin/email_to.py
        Small python script which determine which mailing-list(s) to use
        when sending commit email notifications. No special requirements
        other than a Python3 interpreter.

      - binutils-gdb.git/hooks-bin/commit-extra-checker.py

        Verifies that we do not have this issue:

        # With commits created using "git am" of patches sent via the gdb@ or
        # gdb-patches@ mailing list, it's possible that the author information
        # gets changed into "xxx via xxx@sourceware.org". Catch and reject those,
        # so the person doing the push can fix this before the push is allowed.

        No special requirements other than a Python3 interpreter.

      - /git/binutils-gdb.git/hooks-bin/style_checker

        An empty script at the moment (set because the git-hooks require
        it to be set).

      - /sourceware/infra/bin/email-to-bugzilla

        A perl script whose maintainorship is unclear. Comments at
        the beginning of the script mention David Hampton <hampton@cisco.com>
        as contributor and Greg A. Woods <woods@web.net> as having "greatly
        hacked" it. Beyond that, don't know.

          . The goal is that when a commit is pushed to the git
            repository, we check for any mention of a Bugzilla bugs, and
            post a message to those bugs
          . It accesses the Bugzilla database directly to check if a given bug
            number exists (could easily be changed to use some HTTP request)
          . It posts to bugzilla by sending an email to a local email account
            (Joel: I think sourceware-bugzilla@localhost)

        Joel's note: If I understood the script correctly, then there
        must be a handler in the local MTA to catch those emails and
        send them to bugzilla for further processing. Don't know how
        that works, though.

      - binutils-gdb.git/hooks-bin/post-receive

        Bash wrapper scripts which essentially calls the following
        script: /usr/local/bin/irkerhook.py. I suspect the origin
        of this script is this github repository:
        https://gitlab.com/esr/irker

        If not, ISTR that it was Tom Tromey who got it installed,
        so we could ask him.

  * The git-hooks themselves send emails about commits to the gdb-cvs
    mailing list (each repository configures the destination, and for
    binutils-gdb, this this a "dynamic" configuration via the email_to.py
    script mentioned above allowing the destination to vary depending
    on which files got modified, whether they are binutils fils or GDB
    file, or both).

* Web-based navigation of the Git repository
  * Need to be able to browse the git repository online
  * https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git
  * If it exists, a more featureful git web frontends would be nice, one
    that allows searching files by name for instance.
  * URLs to a given commit in the Web UI should be easily computable
    using their SHA1; e.g.
        "https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=%(rev)s"
    We use those in the commit emails being sent by the git-hooks.

* mailing lists
  * Those I know for sure are used / useful:
    * gdb@
    * gdb-announce@
    * gdb-cvs@ (now misnaed, but can be used to follow commits to the repo)
    * gdb-patches@
    * gdb-prs@
    * gdb-testers@
  * Not sure about these:
    * gdbadmin@: seems useful for whoever is responsible for snapshot
      generation or other GDB-related automated task.  It seems a bit
      spammy to send an email every day when the task succeeds.

      The daily commits in binutils-gdb that bump the date are made using
      this email address as the From, not sure if it matters
  * Pretty sure we can ignore these:
    * gdb-testresults@
    * gdb-patches-prs@:
    * gdb-webpages-cvs@

  * We also have a "global maintainers" list hosted at AdaCore.  It could
    make sense to move it to the main infrastructure.  Maybe Joel knows if
    there's a reason it's separate.

    No reason to keep it separate. It just started as an email alias
    on my own home server, then moved to being hosted by AdaCore to
    avoid having me maintaining a working email configuration on my
    server. Agree it might make sense to move, but I would say email
    discussions between Global Maintainers should remain private,
    or at least closed.

* Mailing list web interface
  * The original one, which I find not so useful now that we have
    public-inbox: https://sourceware.org/pipermail/gdb-patches/
  * However, old links need to keep working
  * public-inbox or something equivalent is necessary:
    * have a way to browse easily message threads that span multiple
      months / years
    * have a way to download raw messages, to apply patches locally

* daily snapshot generation

  * We have a script that generates daily tarballs:

    This is currently a collection of shell scripts stored on sourceware
    at /cvs/gdbadmin/ss, and tracked in a (ahem) CVS repository in
    /cvs/gdbadmin/ss.

    These are incredibly over-engineered, with poor error investigation
    support. But the technology used is simple bash scripting.

* daily bump

  * We have a gdbadmin user crontab entry which calls a script in
    the ss repository mentioned above daily, once for the master branch,
    once for the current binutils release branch, and once again for
    the current gdb release branch.

    Simple shell script.

* Website
  * Handwritten pages in a couple of CVS repositories:
    - The gnu.org copy is maintained on https://savannah.gnu.org/;
    - The sourceware.org copy is maintained on sourceware.org itself,
      in /cvs/gdb/htdocs
    - It used to be that both versions were kept in sync by duplicating
      all the commits in both repositories. But this was recently changed
      in favor of installing redirects from the gnu.org website to
      the sourceware.org. This was the first step towards a possible
      transition of the web-pages to Git.
  * scripts generating website contents (doc, ARI, etc), also part
    of the "ss" repository mentioned above.
  * SSH access to the machine hosting the website is used, as the scripts
    above generating website contents are run by the release manager, after
    having ssh'ed onto sourceware.org.

* Wiki
  * Currently, anyone with write access can give someone else write access.
  * At the very least, we need a way for project maintainers to request
    write access for a new member, or do it themselves

* bug tracker (bugzilla)
  * Sends notifications of new bugs and comments to gdb-prs
  * Sends notifications of comments on a bug to users who are watching
    that bug
  * Accepts replies by email, both from users and from the
    email-to-bugzilla script.

* Release hosting
  * Releases, the release manager must be able to upload there
    * ftp://sourceware.org/pub/gdb/releases/
    * http://sourceware.org/pub/gdb/releases/
  * Snapshots, the daily script needs to be able to upload there
    * https://sourceware.org/pub/gdb/snapshots/

* patchwork (https://patchwork.sourceware.org/project/gdb/list/)
  * It receives emails from the gdb-patches mailing list
  * This instance doesn't get much use at the moment, because it gets
    filled so quickly and is impossible to keep up to date.

-- 
Joel