Re: [PATCH] refs: implement reference transaction hooks

[Patrick Steinhardt <ps@pks.im>]

[-- Attachment #1: Type: text/plain, Size: 4210 bytes --]

On Sun, Jun 07, 2020 at 10:12:33PM +0200, SZEDER Gábor wrote:
> On Wed, Jun 03, 2020 at 01:26:04PM +0200, Patrick Steinhardt wrote:
> > On Tue, Jun 02, 2020 at 10:47:55AM -0700, Junio C Hamano wrote:
> > > Patrick Steinhardt <ps@pks.im> writes:
> > > 
> > > > The above scenario is the motivation for a set of three new hooks that
> > > > reach directly into Git's reference transaction. Each of the following
> > > > new hooks (currently) doesn't accept any parameters and receives the set
> > > > of queued reference updates via stdin:
> > > 
> > > Do we have something (e.g. performance measurement) to convince
> > > ourselves that this won't incur unacceptable levels of overhead in
> > > null cases where there is no hook installed in the repository?
> > 
> > Not yet, but I'll try to come up with a benchmark in the next iteration.
> > I guess the best way to test is to directly exercise git-update-refs, as
> > it's nearly a direct wrapper around reference transactions.
> > 
> > > > +	proc.in = -1;
> > > > +	proc.stdout_to_stderr = 1;
> > > > +	proc.trace2_hook_name = hook_name;
> > > > +
> > > > +	code = start_command(&proc);
> > > > +	if (code)
> > > > +		return code;
> > > > +
> > > > +	sigchain_push(SIGPIPE, SIG_IGN);
> > > > +
> > > > +	for (i = 0; i < transaction->nr; i++) {
> > > > +		struct ref_update *update = transaction->updates[i];
> > > > +
> > > > +		strbuf_reset(&buf);
> > > > +		strbuf_addf(&buf, "%s %s %s\n",
> > > > +			    oid_to_hex(&update->old_oid),
> > > > +			    oid_to_hex(&update->new_oid),
> > > > +			    update->refname);
> > > > +
> > > > +		if (write_in_full(proc.in, buf.buf, buf.len) < 0)
> > > > +			break;
> > > 
> > > We leave the loop early when we detect a write failure here...
> > > 
> > > > +	}
> > > > +
> > > > +	close(proc.in);
> > > > +	sigchain_pop(SIGPIPE);
> > > > +
> > > > +	strbuf_release(&buf);
> > > > +	return finish_command(&proc);
> > > 
> > > ... but the caller does not get notified.  Intended?
> > 
> > This is semi-intended. In case the hook doesn't fully consume stdin and
> > exits early, writing to its stdin would fail as we ignore SIGPIPE. We
> > don't want to force the hook to care about consuming all of stdin,
> > though.
> 
> Why?  How could the prepared hook properly initialize the voting
> mechanism for the transaction without reading all the refs to be
> updated?

Because the hook might not want to implement a voting mechanism after
all but something entirely different which we're currently not
foreseeing as a valid usecase. We don't enforce this anywhere else
either, like e.g. for the pre-receive hook. If that one exits early
without consuming its stdin then that's totally fine.

> > We could improve error handling here by ignoring EPIPE, but making every
> > other write error fatal. If there's any other abnormal error condition
> > then we certainly don't want the hook to act on incomplete data and
> > pretend everything's fine.
> 
> As I read v2 of this patch, a prepared hook can exit(0) early without
> reading all the refs to be updated, cause EPIPE in the git process
> invoking the hook, and that process would interpret that as success.
> I haven't though it through how such a voting mechanism would work,
> but I have a gut feeling that this can't be good.

As said, I lean towards allowing more flexibility for the hook
implementation to also cater for other usecases. But I agree that in a
voting implementation, not reading all of stdin is a bad thing and may
point to a buggy hook implementation. Aborting the transaction if the
hook didn't read all of stdin would be a nice safeguard in that case.

With the current implementation of using a single hook for "prepared",
"committed" and "aborted", it'd also force the hook implementation to do
something in cases it doesn't care about. E.g.

    #!/bin/sh
    case "$1" in
        prepared)
            VOTE=$(sha1sum <&0)
            cast $VOTE
            ;;
        aborted|committed)
            cat <&0 >/dev/null
            ;;
    esac

That being said, I'm not opposed to enforce this and not treat EPIPE
differently.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]