Re: thoughts on a Merge Request based development workflow

[Dmitry Vyukov <dvyukov@google.com>]

On Thu, Oct 10, 2019 at 10:58 PM Theodore Y. Ts'o <tytso@mit.edu> wrote:
> On Thu, Oct 10, 2019 at 07:52:50PM +0200, Dmitry Vyukov wrote:
> > I know you all love Gerrit but just to clarify :)
> > Gerrit stores all metadata in a git repo, all users can have a replica
> > and you can always have, say, a "backup" replica on a side done
> > automatically. Patches and versions of the patches are committed into
> > git into special branches (e.g. change/XXX/version/YYY), comments and
> > metadata are in a pretty straightforward json (this is comment text
> > for line X, etc) also committed into git, so one can always read that
> > in and transform into any other format. And you can also run Gerrit
> > locally over your replica.
>
> Konstantin has spoken about some his concerns about git's scalability,
> and it's important to remember that just because Gerrit has shown to
> work well on some very large repositories, it doesn't necessarily mean
> that it will work well on git repositories using the open source C
> implementation of git.
>
> That's because Gerrit as used by Google (and made available in various
> public-facing Gerrit servers) uses a Git-on-Borg implementation[1],
> where the storage is done using Google's internal storage
> infrastructure.  This is implemented on top of Jgit (which is git
> implemented in Java)[2].
>
> [1] https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/xM9THFr55L8
> [2] https://groups.google.com/a/chromium.org/d/msg/blink-dev/GZOeMUPE7Bc/LmxSj_ezcQ8J
>
> That doesn't necessarily mean that git can't be made to work well
> enough as a transport layer.  I'm just pointing out this may be the
> explanation for why some say, "See, Gerrit works really well on
> super-large repos storing huge numbers of review comments" and others
> are saying, "it would be really scary to run git as a transport layer
> on kernel.org servers because git won't scale well to that kind of
> load."
>
> Both may be correct.
>
> Cheers,
>
>                                         - Ted

Good point. I wonder if it's possible to choose a more-git-friendly
storage scheme and to optimize the OSS git to get to the necessary
scalability level. I am asking because "optimizing some piece of
software" looks like a smaller part in the grand scheme of things of
the overall problem (unless of course there are some fundamental
conflicts between git and efficient storage for this type of data).

However, I mainly wanted to point out a higher-level consideration.
Total doomsday resistance, assuming every party in the world is an
adversary and total decentralization are nice properties, but each of
them makes project design and implementation an order or magnitude
harder. So the question is: are the following requirements would be
enough:
 - open-source implementation
 - transparent raw data format
 - ability to export and backup all data natively (each user may even
have a complete replica of whole raw data for the "local patchwork"
thing)
 - ability to do most of the work locally
 - not owning user identities/be able to export user identities
 - maybe something else, but you get the idea
?

We already have patchwork and public inbox deployed on kernel.org, so
could this whole development support system also be something simply
deployed on kernel.org? That would make lots of things _much_ easier.