Re: thoughts on a Merge Request based development workflow

[Eric Wong <e@80x24.org>]

Daniel Axtens <dja@axtens.net> wrote:
> >> For example:
> >> 
> >>  - is a given series a revision of a previous series? Humans can change
> >>    the name of the cover letter, they can re-order or drop patches,
> >>    split and merge series, even change sender, and other humans just
> >>    figure it out. But if I try to crystalise that logic into patchwork,
> >>    things get very tricky. This makes it hard to build powerful APIs
> >>    into patchwork, which makes it harder to build really cool tools on
> >>    top of patchwork.
> >
> > I'm confident that we can build much of that logic off search
> > and do similar things to what git does with rename detection.
> 
> A lot of people on this list are confident of a great many things :)
> 
> There should be an API in the next minor version of Patchwork that
> allows you to set patch relations. I would encourage you to try to build
> this - if it works, we can plug it in to the core.

Manually set relations should not be needed if people use
format-patch with --interdiff or --range-diff.

A well-tuned search engine will be able to figure out the
preceding series using the git blob IDs from interdiff or commit
IDs from range-diff.

No need to introduce extra metadata into the system, especially
not in a way that can't be reproduced.  Reuse what we have.

Even without interdiff or range-diff, it should be possible to
determine relationships based on common pre-image blob IDs
if the sender used the same base.

> >>  - what are the dependencies of a patch series? Does it need another
> >>    series first? Does it apply to a particular tree? (maintainer/next,
> >>    maintainer/fixes, stable?) This affects every CI system that I'm
> >>    aware of (some of which build on patchwork). Humans can understand
> >>    this pretty easily, computers not so much.
> >
> > I think can do all these things off existing data in archives.
> > We already have pre/post-image blob IDs in git patches.
> > To get there, I think we'll need:
> 
> > 1) efficient way to map blobs -> trees -> commits -> refs
> >    (a reverse-mapping for git's normal DAG)
> >
> > 2) automatic scanning of known repos (searching what appear to
> >    be pull-requests, similar to what pr-tracker-bot does).
> >
> > None of which requires patch senders to do anything differently.
> >
> > git format-patch features such as --base and --range-diff can
> > certainly help with this, and it's probably easier to train people
> > to use newer options in existing tools than new tools, entirely.
> 
> I don't understand any of what you're proposing, unfortunately.
> 
> AIUI snowpatch (to pick an open source patch CI example) tries applying
> patches to a set of different (instance-configured) trees until it finds
> one that works. I'm sure they'd be interested in seeing patches to make
> this more efficient.

Every patch from git format-patch has abbreviated pre/post-image
SHA-1 blob IDs.  If we had an efficient reverse mapping of those
blob IDs to trees, we could quickly figure out which trees those
patches can apply to.

I've already been using pre/post-image blob IDs to recreate blobs
efficiently:

  https://lore.kernel.org/workflows/20190924013920.GA22698@dcvr/

But it doesn't yet find which trees the patch can apply to;
since it cannot (yet) tell you which trees those blob exist in.

> >> Non-email systems have an easier time of this: with gerrit (which I'm
> >> not a big fan of, but just take it as an example) you push things up to
> >> a git repository, and it requires a change-id. So you can track the base
> >> tree, dependencies, and patch revisions easily, because you build on a
> >> richer, more structured data source.
> >
> > Right, decentralization is a HARD problem; but it starts off
> > with centralization-resistance, a slightly easier problem to
> > solve :)
> >
> > The key is: don't introduce things which mirrors can't reproduce
> 
> Mirrors already can't meaningfully reproduce patchwork. They can only
> make a read-only copy of some of the data, but it's not enough to spin
> up a new identical instance.

Right, that seems to be a consequence of not having the
prerequisite storage or search that public-inbox does:

> > Unlike in 2005 when git started; things like Xapian and SQLite
> > are much more mature and I'm comfortable leaning on them to
> > solve harder problems.