Re: thoughts on a Merge Request based development workflow

[Han-Wen Nienhuys <hanwen@google.com>]

On Tue, Oct 15, 2019 at 3:45 PM Daniel Vetter <daniel@ffwll.ch> wrote:
> > > Last time I looked none of the common web ui tools (gerrit, gitlab,
> > > github) had any reasonable support for topic branches/patch series
> > > that target multiple different branches/repositories. They all assume
> > > that a submission gets merged into one branch only and that's it. You
> > > can of course submit the same stuff for inclusion into multiple
> > > places, but that gives you separate discussion tracking for each one
> > > (at least with the merge/pull request model, maybe gerrit is better
> > > here), which is real bad.
> >
> > Can you say a little more about what you expect when working with
> > multiple branches/repos?
> >
> > In gerrit, you can assign freeform tags ("topics") to changes, to
> > group them. See eg.
> >
> >   https://gerrit-review.googlesource.com/q/topic:"rename-reviewdb-package"+(status:open%20OR%20status:merged)
> >
> > this will let you group changes, that can be in different repos and/or
> > different branches. See also
> > https://gerrit-review.googlesource.com/Documentation/intro-user.html#topics
> >
> > Discussions are tied to a single commit, but you can easily navigate
> > between different changes in topics, and submission is synchronized
> > (submitting one change will submit all of the topic. it's
> > unfortunately not atomic).
> >
> > This is how submissions to Android work, as Android is stitched
> > together from ~1000 repos. It is likely that this support will further
> > improve, as Android is one of our biggest internal key customers.
>
> I think gitlab is working on this under the heading of "supermerge",
> where you tie together a pile of changes for different repos under one
> overall label to keep the discussion together.
>
> For the kernel we need something slightly different:
> - There's a large pile of forks of the same underlying repo (Linus'
> upstream branch). So not a huge pile of independent histories and file
> trees, but all the same common kernel history and file layout.
> - The _same_ set of patches is submitted to multiple branches in that
> fork network. E.g. a refactoring patch series which touches both
> driver core and a few subsystems.

You're simplifying the android situation, and it's actually more
similar to the linux kernel than you think. There are several hosts
that have versions of Android (one for AOSP, one for Google, and a
couple hosts for different partners).

Then, there are a large number of branches, which represent releases.
Changes (or sets of changes across repositories) that are submitted to
one branch must then be propagated to other release branches, if they
are relevant bug fixes.

With the email workflow, isn't it hard to keep track of which patch
went into which tree? Something that tracks the identity of commit
(like Change-Id) as it travels across trees could help here.

> Afaiui Android has cross-tree pulls, but the patches heading to each
> target repo are distinct, and they're all for disjoint history chains
> (i.e. no common ancestor commit, no shared files between all the
> different branches/patches). I'm not aware of any project which uses
> topic branches and cross-tree submissions as extensively as the linux
> kernel in this fashion. Everyone with multiple repos seems to use the
> Android approach of splitting up the entire space in disjoint repos
> (with disjoint histories and disjoint files). I've done a fairly
> lengthy write-up of this problem:
>
> https://blog.ffwll.ch/2017/08/github-why-cant-host-the-kernel.html
>
> Android is a multi-repo, multi-tree approach, the linux kernel is a
> monotree but multi-repo approach. Most people think that the only
> other approach than multi-tree is the huge monolithic monotree w/
> monorepo approach. That one just doesn't scale. If you'd do Android
> like the linux kernel you'd throw out the repo tool, instead have
> _all_ repos merged into one overall git history (placed at the same
> directory like they're now placed by the repo tool). Still each
> "project/subsystem" would retain their individual git repo to be able
> to scale sufficiently well, through localizing of most
> development/review work to their specific area.

this is not really relevant to the Linux kernel discussion, but it's
more complicated:

* Android exists in several flavors (eg. the vanilla Google flavor,
AOSP, the Samsung flavor, etc.). With different subrepositories,
partners in the ecosystem can swap out components of the Android
platform as needed, while still keeping up with some of the upstream
repositories.

* Android (AOSP) is more than 600k files, ie. 10x larger than the
linux kernel, and about as large as the Chromium repo. Working with
repo that large is painful because the git client itself just doesn't
work that well to trees with that many files.

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--
Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado