Re: thoughts on a Merge Request based development workflow

["Theodore Y. Ts'o" <tytso@mit.edu>]

On Fri, Oct 11, 2019 at 01:01:04PM +0200, Dmitry Vyukov wrote:
> Good point. I wonder if it's possible to choose a more-git-friendly
> storage scheme and to optimize the OSS git to get to the necessary
> scalability level. I am asking because "optimizing some piece of
> software" looks like a smaller part in the grand scheme of things of
> the overall problem (unless of course there are some fundamental
> conflicts between git and efficient storage for this type of data).

It's certainly possible; Jgit is open source (licensed under the
Eclipse Public License), and while G-o-B is not open sourced (since it
needs to interface with internal Google infrastructure, such as Big
Table, it wouldn't help if it was open sourced), I suspect it wouldn't
be hard to create new Java classes for jgit which used some other
backend storage, such as Postgres.  That being said running a
Jgit/Postgres server is probably going to be quite bit more
complicated than the OSS git.

> However, I mainly wanted to point out a higher-level consideration.
> Total doomsday resistance, assuming every party in the world is an
> adversary and total decentralization are nice properties, but each of
> them makes project design and implementation an order or magnitude
> harder. So the question is: are the following requirements would be
> enough:
>  - open-source implementation
>  - transparent raw data format
>  - ability to export and backup all data natively (each user may even
> have a complete replica of whole raw data for the "local patchwork"
> thing)
>  - ability to do most of the work locally
>  - not owning user identities/be able to export user identities
>  - maybe something else, but you get the idea
> ?

This is a good point, and I agree.  And whether we use something like
Jgit, or OSS git, or Google's G-o-B, or Microsoft's highly scalable
git reimplementation, is really is an implementation detail.  So long
as we know that our requirements have viable solutions, we don't need
to rathole on the which solution is best.

And if we have an independent implementation which _could_ scale, and
could be stood up and demonstrate could work for a small set of trees,
if some large company were to contribute the infrastructure so that
could run at scale for most of the kernel subsystems' git trees,
perhaps we would be OK with a solution which is centralized-at-least-in-practice?

After all, lore.kernel.org is centralized at least in practice, and
it's convenient that it archives many of the kernel's mailing lists
(on vger as well as others) because when a message is CC'ed to many
lists, it can be more efficient when it is run centrally.  But the
fact that public inbox *can* be run by others is enough to keep people
from being worried about things being too centralized.

						- Ted