From: Jonathan Tan <jonathantanmy@google.com>
To: emilyshaffer@google.com
Cc: jonathantanmy@google.com, git@vger.kernel.org
Subject: Re: [PATCH v7 15/17] hook: provide stdin by string_list or callback
Date: Tue, 2 Mar 2021 15:33:14 -0800 [thread overview]
Message-ID: <20210302233314.43539-1-jonathantanmy@google.com> (raw)
In-Reply-To: <YD2ZQIRfcwGzpFuQ@google.com>
> Jonathan and I discussed this a little more offline and agreed to leave
> the implementation as is.
>
> Jonathan had suggested "have one callback invocation apply to all hooks
> that are running now", either by having the callback iterate over the
> task queue or by having the run-command lib take the result from the
> callback and have *that* iterate over the task queue. The idea being,
> one pointer to one copy of source material is easier to handle than
> many.
>
> I suggested that the callback's implementation of the second version of
> that, where the library takes care of the "and do it for each task in
> progress" part, would be pretty much identical to the callback's
> implementation as it is in this patch, except that as it is here the
> context pointer is per-task and as Jonathan suggests the context pointer
> is per-entire-hook-invocation - so there isn't much complexity
> difference between the two, from the user's perspective.
>
> We also talked about cases where N=# of hooks > M=# of jobs, that is,
> where some hooks must wait for other hooks to finish executing before
> that could start. In this case, users' callback implementations would
> need to be able to start over from the beginning of the source material,
> and a long-running hook would block other short-running hooks from
> beginning (because the long-running hook would be confused by hearing
> the source material to its stdin again).
Yes - this (number of hooks greater than number of jobs allowed to run
in parallel) was the case in which my suggestion of not having
hook-specific state would not work. The case we were talking about is
when there's a large amount of dynamically-generated data to be
transmitted to the hooks' stdins and I was thinking that it would be
best anyway if the callback looped over all hooks as data was generated,
but it would not be possible to only do a single pass if the number of
hooks is greater than the number of jobs.
next prev parent reply other threads:[~2021-03-03 6:42 UTC|newest]
Thread overview: 170+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-21 18:54 [PATCH v2 0/4] propose config-based hooks Emily Shaffer
2020-05-21 18:54 ` [PATCH v2 1/4] doc: propose hooks managed by the config Emily Shaffer
2020-05-22 10:13 ` Phillip Wood
2020-06-09 20:26 ` Emily Shaffer
2020-05-21 18:54 ` [PATCH v2 2/4] hook: scaffolding for git-hook subcommand Emily Shaffer
2020-05-21 18:54 ` [PATCH v2 3/4] hook: add list command Emily Shaffer
2020-05-22 10:27 ` Phillip Wood
2020-06-09 21:49 ` Emily Shaffer
2020-08-17 13:36 ` Phillip Wood
2020-05-24 23:00 ` Johannes Schindelin
2020-05-27 23:37 ` Emily Shaffer
2020-05-21 18:54 ` [PATCH v2 4/4] hook: add --porcelain to " Emily Shaffer
2020-05-24 23:00 ` Johannes Schindelin
2020-05-25 0:29 ` Johannes Schindelin
2020-07-28 22:24 ` [PATCH v3 0/6] propose config-based hooks Emily Shaffer
2020-07-28 22:24 ` [PATCH v3 1/6] doc: propose hooks managed by the config Emily Shaffer
2020-07-28 22:24 ` [PATCH v3 2/6] hook: scaffolding for git-hook subcommand Emily Shaffer
2020-07-28 22:24 ` [PATCH v3 3/6] hook: add list command Emily Shaffer
2020-07-28 22:24 ` [PATCH v3 4/6] hook: add --porcelain to " Emily Shaffer
2020-07-28 22:24 ` [RFC PATCH v3 5/6] parse-options: parse into argv_array Emily Shaffer
2020-07-29 19:33 ` Junio C Hamano
2020-07-30 23:41 ` Junio C Hamano
2020-07-28 22:24 ` [RFC PATCH v3 6/6] hook: add 'run' subcommand Emily Shaffer
2020-09-09 0:49 ` [PATCH v4 0/9] propose config-based hooks Emily Shaffer
2020-09-09 0:49 ` [PATCH v4 1/9] doc: propose hooks managed by the config Emily Shaffer
2020-09-23 22:59 ` Jonathan Tan
2020-09-24 21:54 ` Emily Shaffer
2020-10-07 9:23 ` Ævar Arnfjörð Bjarmason
2020-10-22 0:58 ` Emily Shaffer
2020-10-23 19:10 ` Ævar Arnfjörð Bjarmason
2020-10-29 15:38 ` Emily Shaffer
2020-10-29 20:04 ` Ævar Arnfjörð Bjarmason
2020-09-09 0:49 ` [PATCH v4 2/9] hook: scaffolding for git-hook subcommand Emily Shaffer
2020-10-05 23:24 ` Jonathan Nieder
2020-10-06 19:06 ` Emily Shaffer
2020-09-09 0:49 ` [PATCH v4 3/9] hook: add list command Emily Shaffer
2020-09-11 13:27 ` Phillip Wood
2020-09-11 16:51 ` Emily Shaffer
2020-09-23 23:04 ` Jonathan Tan
2020-10-06 20:46 ` Emily Shaffer
2020-09-27 19:23 ` Martin Ågren
2020-10-06 20:20 ` Emily Shaffer
2020-10-05 23:27 ` Jonathan Nieder
2020-09-09 0:49 ` [PATCH v4 4/9] hook: add --porcelain to " Emily Shaffer
2020-09-28 19:29 ` Josh Steadmon
2020-09-09 0:49 ` [PATCH v4 5/9] parse-options: parse into strvec Emily Shaffer
2020-10-05 23:30 ` Jonathan Nieder
2020-10-06 4:49 ` Junio C Hamano
2020-09-09 0:49 ` [PATCH v4 6/9] hook: add 'run' subcommand Emily Shaffer
2020-09-11 13:30 ` Phillip Wood
2020-09-28 19:29 ` Josh Steadmon
2020-10-05 23:39 ` Jonathan Nieder
2020-10-06 22:57 ` Emily Shaffer
2020-09-09 0:49 ` [PATCH v4 7/9] hook: replace run-command.h:find_hook Emily Shaffer
2020-09-09 20:32 ` Junio C Hamano
2020-09-10 19:08 ` Emily Shaffer
2020-09-23 23:20 ` Jonathan Tan
2020-10-05 23:42 ` Jonathan Nieder
2020-09-09 0:49 ` [PATCH v4 8/9] commit: use config-based hooks Emily Shaffer
2020-09-10 13:50 ` Phillip Wood
2020-09-10 22:21 ` Junio C Hamano
2020-09-23 23:47 ` Jonathan Tan
2020-10-05 21:27 ` Emily Shaffer
2020-10-05 23:48 ` Jonathan Nieder
2020-10-06 19:08 ` Emily Shaffer
2020-09-09 0:49 ` [PATCH v4 9/9] run_commit_hook: take strvec instead of varargs Emily Shaffer
2020-09-10 14:16 ` Phillip Wood
2020-09-11 13:20 ` Phillip Wood
2020-09-09 21:04 ` [PATCH v4 0/9] propose config-based hooks Junio C Hamano
2020-10-14 23:24 ` [PATCH v5 0/8] propose config-based hooks (part I) Emily Shaffer
2020-10-14 23:24 ` [PATCH v5 1/8] doc: propose hooks managed by the config Emily Shaffer
2020-10-15 16:31 ` Ævar Arnfjörð Bjarmason
2020-10-16 17:29 ` Junio C Hamano
2020-10-21 23:37 ` Emily Shaffer
2020-10-14 23:24 ` [PATCH v5 2/8] hook: scaffolding for git-hook subcommand Emily Shaffer
2020-10-14 23:24 ` [PATCH v5 3/8] hook: add list command Emily Shaffer
2020-10-14 23:24 ` [PATCH v5 4/8] hook: include hookdir hook in list Emily Shaffer
2020-10-14 23:24 ` [PATCH v5 5/8] hook: implement hookcmd.<name>.skip Emily Shaffer
2020-10-14 23:24 ` [PATCH v5 6/8] parse-options: parse into strvec Emily Shaffer
2020-10-14 23:24 ` [PATCH v5 7/8] hook: add 'run' subcommand Emily Shaffer
2020-10-14 23:24 ` [PATCH v5 8/8] hook: replace find_hook() with hook_exists() Emily Shaffer
2020-12-05 1:45 ` [PATCH v6 00/17] propose config-based hooks (part I) Emily Shaffer
2020-12-05 1:45 ` [PATCH 01/17] doc: propose hooks managed by the config Emily Shaffer
2020-12-05 1:45 ` [PATCH 02/17] hook: scaffolding for git-hook subcommand Emily Shaffer
2020-12-05 1:45 ` [PATCH 03/17] hook: add list command Emily Shaffer
2020-12-05 1:45 ` [PATCH 04/17] hook: include hookdir hook in list Emily Shaffer
2020-12-05 1:45 ` [PATCH 05/17] hook: respect hook.runHookDir Emily Shaffer
2020-12-05 1:45 ` [PATCH 06/17] hook: implement hookcmd.<name>.skip Emily Shaffer
2020-12-05 1:45 ` [PATCH 07/17] parse-options: parse into strvec Emily Shaffer
2020-12-05 1:45 ` [PATCH 08/17] hook: add 'run' subcommand Emily Shaffer
2020-12-11 10:15 ` Phillip Wood
2020-12-15 21:41 ` Emily Shaffer
2020-12-05 1:45 ` [PATCH 09/17] hook: replace find_hook() with hook_exists() Emily Shaffer
2020-12-05 1:46 ` [PATCH 10/17] hook: support passing stdin to hooks Emily Shaffer
2020-12-05 1:46 ` [PATCH 11/17] run-command: allow stdin for run_processes_parallel Emily Shaffer
2020-12-05 1:46 ` [PATCH 12/17] hook: allow parallel hook execution Emily Shaffer
2020-12-05 1:46 ` [PATCH 13/17] hook: allow specifying working directory for hooks Emily Shaffer
2020-12-05 1:46 ` [PATCH 14/17] run-command: add stdin callback for parallelization Emily Shaffer
2020-12-05 1:46 ` [PATCH 15/17] hook: provide stdin by string_list or callback Emily Shaffer
2020-12-08 21:09 ` SZEDER Gábor
2020-12-08 22:11 ` Emily Shaffer
2020-12-05 1:46 ` [PATCH 16/17] run-command: allow capturing of collated output Emily Shaffer
2020-12-05 1:46 ` [PATCH 17/17] hooks: allow callers to capture output Emily Shaffer
2020-12-16 0:34 ` [PATCH v6 00/17] propose config-based hooks (part I) Josh Steadmon
2020-12-16 0:56 ` Junio C Hamano
2020-12-16 20:16 ` Emily Shaffer
2020-12-16 23:32 ` Junio C Hamano
2020-12-18 2:07 ` Emily Shaffer
2020-12-18 5:29 ` Junio C Hamano
2020-12-22 0:02 ` [PATCH v7 " Emily Shaffer
2020-12-22 0:02 ` [PATCH v7 01/17] doc: propose hooks managed by the config Emily Shaffer
2021-01-23 15:38 ` Ævar Arnfjörð Bjarmason
2021-01-29 23:52 ` Emily Shaffer
2021-02-01 22:11 ` Junio C Hamano
2021-03-10 19:30 ` Emily Shaffer
2020-12-22 0:02 ` [PATCH v7 02/17] hook: scaffolding for git-hook subcommand Emily Shaffer
2020-12-22 0:02 ` [PATCH v7 03/17] hook: add list command Emily Shaffer
2021-01-31 3:10 ` Jonathan Tan
2021-02-09 21:06 ` Emily Shaffer
2020-12-22 0:02 ` [PATCH v7 04/17] hook: include hookdir hook in list Emily Shaffer
2021-01-31 3:20 ` Jonathan Tan
2021-02-09 22:05 ` Emily Shaffer
2020-12-22 0:02 ` [PATCH v7 05/17] hook: respect hook.runHookDir Emily Shaffer
2021-01-31 3:35 ` Jonathan Tan
2021-02-09 22:31 ` Emily Shaffer
2020-12-22 0:02 ` [PATCH v7 06/17] hook: implement hookcmd.<name>.skip Emily Shaffer
2021-01-31 3:40 ` Jonathan Tan
2021-02-09 22:57 ` Emily Shaffer
2020-12-22 0:02 ` [PATCH v7 07/17] parse-options: parse into strvec Emily Shaffer
2020-12-22 0:02 ` [PATCH v7 08/17] hook: add 'run' subcommand Emily Shaffer
2021-01-31 4:22 ` Jonathan Tan
2021-02-11 22:44 ` Emily Shaffer
2020-12-22 0:02 ` [PATCH v7 09/17] hook: replace find_hook() with hook_exists() Emily Shaffer
2021-01-31 4:39 ` Jonathan Tan
2021-02-12 22:15 ` Emily Shaffer
2021-02-18 22:23 ` Emily Shaffer
2020-12-22 0:02 ` [PATCH v7 10/17] hook: support passing stdin to hooks Emily Shaffer
2020-12-22 0:02 ` [PATCH v7 11/17] run-command: allow stdin for run_processes_parallel Emily Shaffer
2021-02-01 5:38 ` Jonathan Tan
2021-02-19 20:23 ` Emily Shaffer
2020-12-22 0:02 ` [PATCH v7 12/17] hook: allow parallel hook execution Emily Shaffer
2021-02-01 6:04 ` Jonathan Tan
2021-02-22 21:46 ` Emily Shaffer
2020-12-22 0:02 ` [PATCH v7 13/17] hook: allow specifying working directory for hooks Emily Shaffer
2020-12-22 0:02 ` [PATCH v7 14/17] run-command: add stdin callback for parallelization Emily Shaffer
2021-02-01 6:51 ` Jonathan Tan
2021-02-22 23:38 ` Emily Shaffer
2021-02-23 19:33 ` Jonathan Tan
2021-03-10 18:24 ` Emily Shaffer
2020-12-22 0:02 ` [PATCH v7 15/17] hook: provide stdin by string_list or callback Emily Shaffer
2021-02-01 7:04 ` Jonathan Tan
2021-02-23 19:52 ` Emily Shaffer
2021-02-25 20:56 ` Jonathan Tan
2021-03-02 1:47 ` Emily Shaffer
2021-03-02 23:33 ` Jonathan Tan [this message]
2020-12-22 0:02 ` [PATCH v7 16/17] run-command: allow capturing of collated output Emily Shaffer
2020-12-22 0:02 ` [PATCH v7 17/17] hooks: allow callers to capture output Emily Shaffer
2020-12-22 2:11 ` [PATCH v7 00/17] propose config-based hooks (part I) Junio C Hamano
2020-12-28 18:34 ` Emily Shaffer
2020-12-28 22:50 ` Junio C Hamano
2020-12-28 22:37 ` [PATCH v3 18/17] doc: make git-hook.txt point of truth Emily Shaffer
2020-12-28 22:39 ` Emily Shaffer
2021-01-29 23:59 ` [PATCH v7 00/17] propose config-based hooks (part I) Emily Shaffer
2021-02-16 19:46 ` Josh Steadmon
2021-02-16 22:47 ` Junio C Hamano
2021-02-17 21:21 ` Josh Steadmon
2021-02-17 23:07 ` Junio C Hamano
2021-02-25 19:50 ` Junio C Hamano
2021-03-01 21:51 ` Emily Shaffer
2021-03-01 22:19 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210302233314.43539-1-jonathantanmy@google.com \
--to=jonathantanmy@google.com \
--cc=emilyshaffer@google.com \
--cc=git@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.