Re: thoughts on a Merge Request based development workflow

[Daniel Vetter <daniel@ffwll.ch>]

On Tue, Oct 15, 2019 at 3:14 PM Han-Wen Nienhuys <hanwen@google.com> wrote:
> On Tue, Oct 15, 2019 at 2:00 PM Daniel Vetter <daniel@ffwll.ch> wrote:
> > > Today, this can be easily done by cc'ing the patch to multiple mailing
> > > lists.  Exactly how this works may get tricky, especially in the
> > > federated model where (for example) perhaps the btrfs tree might be
> > > administered by Facebook, while the xfs tree might be administrated by
> > > Red Hat.  Given that we *also* have to support people who want to keep
> > > using e-mail during the transition period, it may be that using
> > > unauthenticated e-mail messages where comments are attached quoted
> > > patch hunks, perhaps that can be the interchange format between
> > > different servers that aren't under a common administrative domain.
> >
> > Last time I looked none of the common web ui tools (gerrit, gitlab,
> > github) had any reasonable support for topic branches/patch series
> > that target multiple different branches/repositories. They all assume
> > that a submission gets merged into one branch only and that's it. You
> > can of course submit the same stuff for inclusion into multiple
> > places, but that gives you separate discussion tracking for each one
> > (at least with the merge/pull request model, maybe gerrit is better
> > here), which is real bad.
>
> Can you say a little more about what you expect when working with
> multiple branches/repos?
>
> In gerrit, you can assign freeform tags ("topics") to changes, to
> group them. See eg.
>
>   https://gerrit-review.googlesource.com/q/topic:"rename-reviewdb-package"+(status:open%20OR%20status:merged)
>
> this will let you group changes, that can be in different repos and/or
> different branches. See also
> https://gerrit-review.googlesource.com/Documentation/intro-user.html#topics
>
> Discussions are tied to a single commit, but you can easily navigate
> between different changes in topics, and submission is synchronized
> (submitting one change will submit all of the topic. it's
> unfortunately not atomic).
>
> This is how submissions to Android work, as Android is stitched
> together from ~1000 repos. It is likely that this support will further
> improve, as Android is one of our biggest internal key customers.

I think gitlab is working on this under the heading of "supermerge",
where you tie together a pile of changes for different repos under one
overall label to keep the discussion together.

For the kernel we need something slightly different:
- There's a large pile of forks of the same underlying repo (Linus'
upstream branch). So not a huge pile of independent histories and file
trees, but all the same common kernel history and file layout.
- The _same_ set of patches is submitted to multiple branches in that
fork network. E.g. a refactoring patch series which touches both
driver core and a few subsystems.

Afaiui Android has cross-tree pulls, but the patches heading to each
target repo are distinct, and they're all for disjoint history chains
(i.e. no common ancestor commit, no shared files between all the
different branches/patches). I'm not aware of any project which uses
topic branches and cross-tree submissions as extensively as the linux
kernel in this fashion. Everyone with multiple repos seems to use the
Android approach of splitting up the entire space in disjoint repos
(with disjoint histories and disjoint files). I've done a fairly
lengthy write-up of this problem:

https://blog.ffwll.ch/2017/08/github-why-cant-host-the-kernel.html

Android is a multi-repo, multi-tree approach, the linux kernel is a
monotree but multi-repo approach. Most people think that the only
other approach than multi-tree is the huge monolithic monotree w/
monorepo approach. That one just doesn't scale. If you'd do Android
like the linux kernel you'd throw out the repo tool, instead have
_all_ repos merged into one overall git history (placed at the same
directory like they're now placed by the repo tool). Still each
"project/subsystem" would retain their individual git repo to be able
to scale sufficiently well, through localizing of most
development/review work to their specific area.

Cheers, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch