git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v8 00/37] config-based hooks
@ 2021-03-11  2:10 Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 01/37] doc: propose hooks managed by the config Emily Shaffer
                   ` (40 more replies)
  0 siblings, 41 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git
  Cc: Emily Shaffer, Jeff King, Junio C Hamano, James Ramsay,
	Jonathan Nieder, brian m. carlson,
	Ævar Arnfjörð Bjarmason, Phillip Wood,
	Josh Steadmon, Johannes Schindelin, Jonathan Tan

Since v7:
- Addressed Jonathan Tan's review of part I
- Addressed Junio's review of part I and II
- Combined parts I and II

I think the updates to patch 1 between the rest of the work I've been
doing probably have covered Ævar's comments.

More details about per-patch changes found in the notes on each mail (I
hope).

I know that Junio was talking about merging v7 after Josh Steadmon's
review and I asked him not to - this reroll has those changes from
Jonathan Tan's review that I was wanting to wait for.

Thanks!
 - Emily

Emily Shaffer (37):
  doc: propose hooks managed by the config
  hook: scaffolding for git-hook subcommand
  hook: add list command
  hook: include hookdir hook in list
  hook: teach hook.runHookDir
  hook: implement hookcmd.<name>.skip
  parse-options: parse into strvec
  hook: add 'run' subcommand
  hook: introduce hook_exists()
  hook: support passing stdin to hooks
  run-command: allow stdin for run_processes_parallel
  hook: allow parallel hook execution
  hook: allow specifying working directory for hooks
  run-command: add stdin callback for parallelization
  hook: provide stdin by string_list or callback
  run-command: allow capturing of collated output
  hooks: allow callers to capture output
  commit: use config-based hooks
  am: convert applypatch hooks to use config
  merge: use config-based hooks for post-merge hook
  gc: use hook library for pre-auto-gc hook
  rebase: teach pre-rebase to use hook.h
  read-cache: convert post-index-change hook to use config
  receive-pack: convert push-to-checkout hook to hook.h
  git-p4: use 'git hook' to run hooks
  hooks: convert 'post-checkout' hook to hook library
  hook: convert 'post-rewrite' hook to config
  transport: convert pre-push hook to use config
  reference-transaction: look for hooks in config
  receive-pack: convert 'update' hook to hook.h
  proc-receive: acquire hook list from hook.h
  post-update: use hook.h library
  receive-pack: convert receive hooks to hook.h
  bugreport: use hook_exists instead of find_hook
  git-send-email: use 'git hook run' for 'sendemail-validate'
  run-command: stop thinking about hooks
  docs: unify githooks and git-hook manpages

 .gitignore                                    |   1 +
 Documentation/Makefile                        |   1 +
 Documentation/config/hook.txt                 |  27 +
 Documentation/git-hook.txt                    | 161 ++++
 Documentation/githooks.txt                    | 655 +---------------
 Documentation/native-hooks.txt                | 708 ++++++++++++++++++
 Documentation/technical/api-parse-options.txt |   7 +
 .../technical/config-based-hooks.txt          | 369 +++++++++
 Makefile                                      |   2 +
 builtin.h                                     |   1 +
 builtin/am.c                                  |  33 +-
 builtin/bugreport.c                           |   4 +-
 builtin/checkout.c                            |  19 +-
 builtin/clone.c                               |   8 +-
 builtin/commit.c                              |  11 +-
 builtin/fetch.c                               |   1 +
 builtin/gc.c                                  |   5 +-
 builtin/hook.c                                | 176 +++++
 builtin/merge.c                               |  15 +-
 builtin/rebase.c                              |   9 +-
 builtin/receive-pack.c                        | 329 ++++----
 builtin/submodule--helper.c                   |   2 +-
 builtin/worktree.c                            |  31 +-
 command-list.txt                              |   1 +
 commit.c                                      |  22 +-
 commit.h                                      |   3 +-
 git-p4.py                                     |  67 +-
 git-send-email.perl                           |  21 +-
 git.c                                         |   1 +
 hook.c                                        | 480 ++++++++++++
 hook.h                                        | 138 ++++
 parse-options-cb.c                            |  16 +
 parse-options.h                               |   4 +
 read-cache.c                                  |  13 +-
 refs.c                                        |  43 +-
 reset.c                                       |  16 +-
 run-command.c                                 | 156 ++--
 run-command.h                                 |  55 +-
 sequencer.c                                   |  90 +--
 submodule.c                                   |   1 +
 t/helper/test-run-command.c                   |  46 +-
 t/t0061-run-command.sh                        |  37 +
 t/t1360-config-based-hooks.sh                 | 303 ++++++++
 t/t1416-ref-transaction-hooks.sh              |  12 +-
 t/t5411/test-0015-too-many-hooks-error.sh     |  47 ++
 ...3-pre-commit-and-pre-merge-commit-hooks.sh |  17 +-
 t/t9001-send-email.sh                         |  11 +-
 transport.c                                   |  59 +-
 48 files changed, 3052 insertions(+), 1182 deletions(-)
 create mode 100644 Documentation/config/hook.txt
 create mode 100644 Documentation/git-hook.txt
 create mode 100644 Documentation/native-hooks.txt
 create mode 100644 Documentation/technical/config-based-hooks.txt
 create mode 100644 builtin/hook.c
 create mode 100644 hook.c
 create mode 100644 hook.h
 create mode 100755 t/t1360-config-based-hooks.sh
 create mode 100644 t/t5411/test-0015-too-many-hooks-error.sh

-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH v8 01/37] doc: propose hooks managed by the config
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 02/37] hook: scaffolding for git-hook subcommand Emily Shaffer
                   ` (39 subsequent siblings)
  40 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Begin a design document for config-based hooks, managed via git-hook.
Focus on an overview of the implementation and motivation for design
decisions. Briefly discuss the alternatives considered before this
point. Also, attempt to redefine terms to fit into a multihook world.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

Notes:
    Since v7, made some wording changes based on reviewer comments
    (mostly Junio's, I think).

 - Emily

 Documentation/Makefile                        |   1 +
 .../technical/config-based-hooks.txt          | 369 ++++++++++++++++++
 2 files changed, 370 insertions(+)
 create mode 100644 Documentation/technical/config-based-hooks.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 81d1bf7a04..2743de8995 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -82,6 +82,7 @@ SP_ARTICLES += $(API_DOCS)
 TECH_DOCS += MyFirstContribution
 TECH_DOCS += MyFirstObjectWalk
 TECH_DOCS += SubmittingPatches
+TECH_DOCS += technical/config-based-hooks
 TECH_DOCS += technical/hash-function-transition
 TECH_DOCS += technical/http-protocol
 TECH_DOCS += technical/index-format
diff --git a/Documentation/technical/config-based-hooks.txt b/Documentation/technical/config-based-hooks.txt
new file mode 100644
index 0000000000..1f973117e4
--- /dev/null
+++ b/Documentation/technical/config-based-hooks.txt
@@ -0,0 +1,369 @@
+Configuration-based hook management
+===================================
+:sectanchors:
+
+[[motivation]]
+== Motivation
+
+Replace the `.git/hook/hookname` path as the only source of hooks to execute;
+allow users to define hooks using config files, in a way which is friendly to
+users with multiple repos which have similar needs - hooks can be easily shared
+between multiple Git repos.
+
+Redefine "hook" as an event rather than a single script, allowing users to
+perform multiple unrelated actions on a single event.
+
+Make it easier for users to discover Git's hook feature and automate their
+workflows.
+
+[[user-interfaces]]
+== User interfaces
+
+[[config-schema]]
+=== Config schema
+
+Hooks can be introduced by editing the configuration manually. There are two new
+sections added, `hook` and `hookcmd`.
+
+[[config-schema-hook]]
+==== `hook`
+
+Primarily contains subsections for each hook event. The order of variables in
+these subsections defines the hook command execution order; hook commands can be
+specified by setting the value directly to the command if no additional
+configuration is needed, or by setting the value as the name of a `hookcmd`. If
+Git does not find a `hookcmd` whose subsection matches the value of the given
+command string, Git will try to execute the string directly. Hooks are executed
+by passing the resolved command string to the shell. In the future, hook event
+subsections could also contain per-hook-event settings; see
+<<per-hook-event-settings,the section in Future Work>> for more details.
+
+Also contains top-level hook execution settings, for example, `hook.runHookDir`.
+(These settings are described more in <<library,Library>>.)
+
+----
+[hook "pre-commit"]
+  command = perl-linter
+  command = /usr/bin/git-secrets --pre-commit
+
+[hook "pre-applypatch"]
+  command = perl-linter
+  # for illustration purposes; error behavior isn't planned yet
+  error = ignore
+
+[hook]
+  runHookDir = interactive
+----
+
+[[config-schema-hookcmd]]
+==== `hookcmd`
+
+Defines a hook command and its attributes, which will be used when a hook event
+occurs. Unqualified attributes are assumed to apply to this hook during all hook
+events, but event-specific attributes can also be supplied. The example runs
+`/usr/bin/lint-it --language=perl <args passed by Git>`, but for repos which
+include this config, the hook command will be skipped for all events.
+Theoretically, the last line could be used to "un-skip" the hook command for
+`pre-commit` hooks, but this hasn't been scoped or implemented yet.
+
+----
+[hookcmd "perl-linter"]
+  command = /usr/bin/lint-it --language=perl
+  skip = true
+  # for illustration purposes; below hasn't been defined yet
+  pre-commit-skip = false
+----
+
+[[command-line-api]]
+=== Command-line API
+
+Users should be able to view, run, reorder, and create hook commands via the
+command line. External tools should be able to view a list of hooks in the
+correct order to run. Modifier commands (`edit` and `add`) have not been
+implemented yet and may not be if manually editing the config proves usable
+enough.
+
+*`git hook list <hook-event>`*
+
+*`git hook run <hook-event> [-a <arg>]... [-e <env-var>]...`*
+
+*`git hook edit <hook-event>`*
+
+*`git hook add <hook-command> <hook-event> <options...>`*
+
+[[hook-editor]]
+=== Hook editor
+
+The tool which is presented by `git hook edit <hook-command>`. Ideally, this
+tool should be easier to use than manually editing the config, and then produce
+a concise config afterwards. It may take a form similar to `git rebase
+--interactive`. This has not been designed or implemented yet and may not be if
+the config proves usable enough.
+
+[[implementation]]
+== Implementation
+
+[[library]]
+=== Library
+
+`hook.c` and `hook.h` are responsible for interacting with the config files. The
+hook library provides a basic API to call all hooks in config order with more
+complex options passed via `struct run_hooks_opt`:
+
+*`int run_hooks(const char *hookname, struct run_hooks_opt *options)`*
+
+`struct run_hooks_opt` allows callers to set:
+
+- environment variables
+- command-line arguments
+- behavior for the hook command provided by `run-command.h:find_hook()` (see
+  below)
+- a method to provide stdin to each hook, either via a file containing stdin, a
+  `struct string_list` containing a list of lines to print, or a callback
+  function to allow the caller to populate stdin manually
+- a method to process stdout from each hook, e.g. for printing to sideband
+  during a network operation
+- parallelism
+- a custom working directory for hooks to execute in
+
+And this struct can be extended with more options as necessary in the future.
+
+The "legacy" hook provided by `run-command.h:find_hook()` - that is, the hook
+present in `.git/hooks/<hookname>` or
+`$(git config --get core.hooksPath)/<hookname>` - can be handled in a number of
+ways, providing an avenue to deprecate these "legacy" hooks if desired. The
+handling is based on a config `hook.runHookDir`, which is checked against a
+number of cases:
+
+- "no": the legacy hook will not be run
+- "error": Git will print a warning to stderr before ignoring the legacy hook
+- "interactive": Git will prompt the user before running the legacy hook
+- "warn": Git will print a warning to stderr before running the legacy hook
+- "yes" (default): Git will silently run the legacy hook
+
+In case this list is expanded in the future, if a value for `hook.runHookDir` is
+given which Git does not recognize, Git should discard that config entry. For
+example, if "warn" was specified at system level and "junk" was specified at
+global level, Git would resolve the value to "warn"; if the only time the config
+was set was to "junk", Git would use the default value of "yes" (but print a
+warning to the user first to let them know their value is wrong).
+
+`struct hookcmd` is expected to grow in size over time as more functionality is
+added to hooks; so that other parts of the code don't need to understand the
+config schema, `struct hookcmd` should contain logical values instead of string
+pairs.
+
+By default, hook parallelism is chosen based on the semantics of each hook;
+callsites initialize their `struct run_hooks_opt` via one of two macros,
+`RUN_HOOKS_OPT_INIT_SYNC` or `RUN_HOOKS_OPT_INIT_ASYNC`. The default number of
+jobs can be configured in `hook.jobs`; this config applies across all hook
+events. If unset, the value of `online_cpus()` (equivalent to `nproc`) is used.
+
+[[builtin]]
+=== Builtin
+
+`builtin/hook.c` is responsible for providing the frontend. It's responsible for
+formatting user-provided data and then calling the library API to set the
+configs as appropriate. The builtin frontend is not responsible for calling the
+config directly, so that other areas of Git can rely on the hook library to
+understand the most recent config schema for hooks.
+
+[[migration]]
+=== Migration path
+
+[[stage-0]]
+==== Stage 0
+
+Hooks are called by running `run-command.h:find_hook()` with the hookname and
+executing the result. The hook library and builtin do not exist. Hooks only
+exist as specially named scripts within `.git/hooks/`.
+
+[[stage-1]]
+==== Stage 1
+
+`git hook list --porcelain <hook-event>` is implemented. `hook.h:run_hooks()` is
+taught to include `run-command.h:find_hook()` at the end; calls to `find_hook()`
+are replaced with calls to `run_hooks()`. Users can opt-in to config-based hooks
+simply by creating some in their config; otherwise users should remain
+unaffected by the change.
+
+[[stage-2]]
+==== Stage 2
+
+The call to `find_hook()` inside of `run_hooks()` learns to check for a config,
+`hook.runHookDir`. Users can opt into managing their hooks completely via the
+config this way.
+
+[[stage-3]]
+==== Stage 3
+
+`.git/hooks` is removed from the template and the hook directory is considered
+deprecated. To avoid breaking older repos, the default of `hook.runHookDir` is
+not changed, and `find_hook()` is not removed.
+
+[[caveats]]
+== Caveats
+
+[[security]]
+=== Security and repo config
+
+Part of the motivation behind this refactor is to mitigate hooks as an attack
+vector.footnote:[https://lore.kernel.org/git/20171002234517.GV19555@aiede.mtv.corp.google.com/]
+However, as the design stands, users can still provide hooks in the repo-level
+config, which is included when a repo is zipped and sent elsewhere. The
+security of the repo-level config is still under discussion; this design
+generally assumes the repo-level config is secure, which is not true yet. This
+assumption was made to avoid overcomplicating the design. So, this series
+doesn't particularly improve security or resistance to zip attacks.
+
+[[ease-of-use]]
+=== Ease of use
+
+The config schema is nontrivial; that's why it's important for the `git hook`
+modifier commands to be usable. Contributors with UX expertise are encouraged to
+share their suggestions.
+
+[[alternatives]]
+== Alternative approaches
+
+A previous summary of alternatives exists in the
+archives.footnote:[https://lore.kernel.org/git/20191116011125.GG22855@google.com]
+
+The table below shows a number of goals and how they might be achieved with
+config-based hooks, by implementing directory support (i.e.
+'.git/hooks/pre-commit.d'), or as hooks are run today.
+
+.Comparison of alternatives
+|===
+|Feature |Config-based hooks |Hook directories |Status quo
+
+|Supports multiple hooks
+|Natively
+|Natively
+|With user effort
+
+|Supports parallelization
+|Natively
+|Natively
+|No (user's multihook trampoline script would need to handle parallelism)
+
+|Safer for zipped repos
+|A little
+|No
+|No
+
+|Previous hooks just work
+|If configured
+|Yes
+|Yes
+
+|Can install one hook to many repos
+|Yes
+|With symlinks or core.hooksPath
+|With symlinks or core.hooksPath
+
+|Discoverability
+|Findable with 'git help git' or tab-completion via 'git hook' subcommand
+|Findable via improved documentation
+|Same as before
+
+|Hard to run unexpected hook
+|If configured
+|Could be made to warn or look for a config
+|No
+|===
+
+[[status-quo]]
+=== Status quo
+
+Today users can implement multihooks themselves by using a "trampoline script"
+as their hook, and pointing that script to a directory or list of other scripts
+they wish to run.
+
+[[hook-directories]]
+=== Hook directories
+
+Other contributors have suggested Git learn about the existence of a directory
+such as `.git/hooks/<hookname>.d` and execute those hooks in alphabetical order.
+
+[[future-work]]
+== Future work
+
+[[execution-ordering]]
+=== Execution ordering
+
+We may find that config order is insufficient for some users; for example,
+config order makes it difficult to add a new hook to the system or global config
+which runs at the end of the hook list. A new ordering schema should be:
+
+1) Specified by a `hook.order` config, so that users will not unexpectedly see
+their order change;
+
+2) Either dependency or numerically based.
+
+Dependency-based ordering is prone to classic linked-list problems, like a
+cycles and handling of missing dependencies. But, it paves the way for enabling
+parallelization if some tasks truly depend on others.
+
+Numerical ordering makes it tricky for Git to generate suggested ordering
+numbers for each command, but is easy to determine a definitive order.
+
+[[parallelization]]
+=== Parallelization with dependencies
+
+Currently hooks use a naive parallelization scheme or are run in series.  But if
+one hook depends on another's output, then users will want to specify those
+dependencies. If we decide to solve this problem, we may want to look to modern
+build systems for inspiration on how to manage dependencies and parallel tasks.
+
+[[nontrivial-hooks]]
+=== Multihooks and nontrivial output
+
+Some hooks - like 'proc-receive' - don't lend themselves well to multihooks at
+all. In the case of 'proc-receive', for now, multiple hook definitions are
+disallowed. In the future we might be able to conceive a better approach, for
+example, running the hooks in series and using the output from one hook as the
+input to the next.
+
+[[securing-hookdir-hooks]]
+=== Securing hookdir hooks
+
+With the design as written in this doc, it's still possible for a malicious user
+to modify `.git/config` to include `hook.pre-receive.command = rm -rf /`, then
+zip their repo and send it to another user. It may be necessary to teach Git to
+only allow inlined hooks like this if they were configured outside of the local
+scope (in other words, only run hookcmds, and only allow hookcmds to be
+configured in global or system scope); or another approach, like a list of safe
+projects, might be useful. It may also be sufficient (or at least useful) to
+teach a `hook.disableAll` config or similar flag to the Git executable.
+
+[[submodule-inheritance]]
+=== Submodule inheritance
+
+It's possible some submodules may want to run the identical set of hooks that
+their superrepo runs. While a globally-configured hook set is helpful, it's not
+a great solution for users who have multiple repos-with-submodules under the
+same user. It would be useful for submodules to learn how to run hooks from
+their superrepo's config, or inherit that hook setting.
+
+[[per-hook-event-settings]]
+=== Per-hook-event settings
+
+It might be desirable to keep settings specifically for some hook events, but
+not for others - for example, a user may wish to disable hookdir hooks for all
+events but pre-commit, which they haven't had time to convert yet; or, a user
+may wish for execution order settings to differ based on hook event. In that
+case, it would be useful to set something like `hook.pre-commit.executionOrder`
+which would not apply to the 'prepare-commit-msg' hook, for example.
+
+[[glossary]]
+== Glossary
+
+*hook event*
+
+A point during Git's execution where user scripts may be run, for example,
+_prepare-commit-msg_ or _pre-push_.
+
+*hook command*
+
+A user script or executable which will be run on one or more hook events.
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 02/37] hook: scaffolding for git-hook subcommand
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 01/37] doc: propose hooks managed by the config Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 03/37] hook: add list command Emily Shaffer
                   ` (38 subsequent siblings)
  40 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Introduce infrastructure for a new subcommand, git-hook, which will be
used to ease config-based hook management. This command will handle
parsing configs to compose a list of hooks to run for a given event, as
well as adding or modifying hook configs in an interactive fashion.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
No change since v7.

 .gitignore                    |  1 +
 Documentation/git-hook.txt    | 20 ++++++++++++++++++++
 Makefile                      |  1 +
 builtin.h                     |  1 +
 builtin/hook.c                | 21 +++++++++++++++++++++
 command-list.txt              |  1 +
 git.c                         |  1 +
 t/t1360-config-based-hooks.sh | 11 +++++++++++
 8 files changed, 57 insertions(+)
 create mode 100644 Documentation/git-hook.txt
 create mode 100644 builtin/hook.c
 create mode 100755 t/t1360-config-based-hooks.sh

diff --git a/.gitignore b/.gitignore
index 3dcdb6bb5a..3608c35b73 100644
--- a/.gitignore
+++ b/.gitignore
@@ -76,6 +76,7 @@
 /git-grep
 /git-hash-object
 /git-help
+/git-hook
 /git-http-backend
 /git-http-fetch
 /git-http-push
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
new file mode 100644
index 0000000000..9eeab0009d
--- /dev/null
+++ b/Documentation/git-hook.txt
@@ -0,0 +1,20 @@
+git-hook(1)
+===========
+
+NAME
+----
+git-hook - Manage configured hooks
+
+SYNOPSIS
+--------
+[verse]
+'git hook'
+
+DESCRIPTION
+-----------
+A placeholder command. Later, you will be able to list, add, and modify hooks
+with this command.
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index dfb0f1000f..8e904a1ab5 100644
--- a/Makefile
+++ b/Makefile
@@ -1087,6 +1087,7 @@ BUILTIN_OBJS += builtin/get-tar-commit-id.o
 BUILTIN_OBJS += builtin/grep.o
 BUILTIN_OBJS += builtin/hash-object.o
 BUILTIN_OBJS += builtin/help.o
+BUILTIN_OBJS += builtin/hook.o
 BUILTIN_OBJS += builtin/index-pack.o
 BUILTIN_OBJS += builtin/init-db.o
 BUILTIN_OBJS += builtin/interpret-trailers.o
diff --git a/builtin.h b/builtin.h
index b6ce981b73..8df1d36a7a 100644
--- a/builtin.h
+++ b/builtin.h
@@ -163,6 +163,7 @@ int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix);
 int cmd_grep(int argc, const char **argv, const char *prefix);
 int cmd_hash_object(int argc, const char **argv, const char *prefix);
 int cmd_help(int argc, const char **argv, const char *prefix);
+int cmd_hook(int argc, const char **argv, const char *prefix);
 int cmd_index_pack(int argc, const char **argv, const char *prefix);
 int cmd_init_db(int argc, const char **argv, const char *prefix);
 int cmd_interpret_trailers(int argc, const char **argv, const char *prefix);
diff --git a/builtin/hook.c b/builtin/hook.c
new file mode 100644
index 0000000000..b2bbc84d4d
--- /dev/null
+++ b/builtin/hook.c
@@ -0,0 +1,21 @@
+#include "cache.h"
+
+#include "builtin.h"
+#include "parse-options.h"
+
+static const char * const builtin_hook_usage[] = {
+	N_("git hook"),
+	NULL
+};
+
+int cmd_hook(int argc, const char **argv, const char *prefix)
+{
+	struct option builtin_hook_options[] = {
+		OPT_END(),
+	};
+
+	argc = parse_options(argc, argv, prefix, builtin_hook_options,
+			     builtin_hook_usage, 0);
+
+	return 0;
+}
diff --git a/command-list.txt b/command-list.txt
index a289f09ed6..9ccd8e5aeb 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -103,6 +103,7 @@ git-grep                                mainporcelain           info
 git-gui                                 mainporcelain
 git-hash-object                         plumbingmanipulators
 git-help                                ancillaryinterrogators          complete
+git-hook                                mainporcelain
 git-http-backend                        synchingrepositories
 git-http-fetch                          synchelpers
 git-http-push                           synchelpers
diff --git a/git.c b/git.c
index 9bc077a025..14adac716f 100644
--- a/git.c
+++ b/git.c
@@ -528,6 +528,7 @@ static struct cmd_struct commands[] = {
 	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
 	{ "hash-object", cmd_hash_object },
 	{ "help", cmd_help },
+	{ "hook", cmd_hook, RUN_SETUP_GENTLY },
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "init", cmd_init_db },
 	{ "init-db", cmd_init_db },
diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh
new file mode 100755
index 0000000000..34b0df5216
--- /dev/null
+++ b/t/t1360-config-based-hooks.sh
@@ -0,0 +1,11 @@
+#!/bin/bash
+
+test_description='config-managed multihooks, including git-hook command'
+
+. ./test-lib.sh
+
+test_expect_success 'git hook command does not crash' '
+	git hook
+'
+
+test_done
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 03/37] hook: add list command
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 01/37] doc: propose hooks managed by the config Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 02/37] hook: scaffolding for git-hook subcommand Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-12  8:20   ` Ævar Arnfjörð Bjarmason
  2021-03-11  2:10 ` [PATCH v8 04/37] hook: include hookdir hook in list Emily Shaffer
                   ` (37 subsequent siblings)
  40 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach 'git hook list <hookname>', which checks the known configs in
order to create an ordered list of hooks to run on a given hook event.

Multiple commands can be specified for a given hook by providing
multiple "hook.<hookname>.command = <path-to-hook>" lines. Hooks will be
run in config order. If more properties need to be set on a given hook
in the future, commands can also be specified by providing
"hook.<hookname>.command = <hookcmd-name>", as well as a "[hookcmd
<hookcmd-name>]" subsection; this subsection should contain a
"hookcmd.<hookcmd-name>.command = <path-to-hook>" line.

For example:

  $ git config --list | grep ^hook
  hook.pre-commit.command=baz
  hook.pre-commit.command=~/bar.sh
  hookcmd.baz.command=~/baz/from/hookcmd.sh

  $ git hook list pre-commit
  global: ~/baz/from/hookcmd.sh
  local: ~/bar.sh

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

Notes:
    Since v7, fixed some nits from Jonathan Tan, one of which revealed a bug in
    how I was adding hooks to the list.

 Documentation/config/hook.txt |   9 +++
 Documentation/git-hook.txt    |  59 ++++++++++++++++-
 Makefile                      |   1 +
 builtin/hook.c                |  56 ++++++++++++++--
 hook.c                        | 120 ++++++++++++++++++++++++++++++++++
 hook.h                        |  25 +++++++
 t/t1360-config-based-hooks.sh |  81 ++++++++++++++++++++++-
 7 files changed, 341 insertions(+), 10 deletions(-)
 create mode 100644 Documentation/config/hook.txt
 create mode 100644 hook.c
 create mode 100644 hook.h

diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
new file mode 100644
index 0000000000..71449ecbc7
--- /dev/null
+++ b/Documentation/config/hook.txt
@@ -0,0 +1,9 @@
+hook.<command>.command::
+	A command to execute during the <command> hook event. This can be an
+	executable on your device, a oneliner for your shell, or the name of a
+	hookcmd. See linkgit:git-hook[1].
+
+hookcmd.<name>.command::
+	A command to execute during a hook for which <name> has been specified
+	as a command. This can be an executable on your device or a oneliner for
+	your shell. See linkgit:git-hook[1].
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 9eeab0009d..f19875ed68 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -8,12 +8,65 @@ git-hook - Manage configured hooks
 SYNOPSIS
 --------
 [verse]
-'git hook'
+'git hook' list <hook-name>
 
 DESCRIPTION
 -----------
-A placeholder command. Later, you will be able to list, add, and modify hooks
-with this command.
+You can list configured hooks with this command. Later, you will be able to run,
+add, and modify hooks with this command.
+
+This command parses the default configuration files for sections `hook` and
+`hookcmd`. `hook` is used to describe the commands which will be run during a
+particular hook event; commands are run in the order Git encounters them during
+the configuration parse (see linkgit:git-config[1]). `hookcmd` is used to
+describe attributes of a specific command. If additional attributes don't need
+to be specified, a command to run can be specified directly in the `hook`
+section; if a `hookcmd` by that name isn't found, Git will attempt to run the
+provided value directly. For example:
+
+Global config
+----
+  [hook "post-commit"]
+    command = "linter"
+    command = "~/typocheck.sh"
+
+  [hookcmd "linter"]
+    command = "/bin/linter --c"
+----
+
+Local config
+----
+  [hook "prepare-commit-msg"]
+    command = "linter"
+  [hook "post-commit"]
+    command = "python ~/run-test-suite.py"
+----
+
+With these configs, you'd then see:
+
+----
+$ git hook list "post-commit"
+global: /bin/linter --c
+global: ~/typocheck.sh
+local: python ~/run-test-suite.py
+
+$ git hook list "prepare-commit-msg"
+local: /bin/linter --c
+----
+
+COMMANDS
+--------
+
+list `<hook-name>`::
+
+List the hooks which have been configured for `<hook-name>`. Hooks appear
+in the order they should be run, and print the config scope where the relevant
+`hook.<hook-name>.command` was specified, not the `hookcmd` (if applicable).
+This output is human-readable and the format is subject to change over time.
+
+CONFIGURATION
+-------------
+include::config/hook.txt[]
 
 GIT
 ---
diff --git a/Makefile b/Makefile
index 8e904a1ab5..3fa51597d8 100644
--- a/Makefile
+++ b/Makefile
@@ -891,6 +891,7 @@ LIB_OBJS += hash-lookup.o
 LIB_OBJS += hashmap.o
 LIB_OBJS += help.o
 LIB_OBJS += hex.o
+LIB_OBJS += hook.o
 LIB_OBJS += ident.o
 LIB_OBJS += json-writer.o
 LIB_OBJS += kwset.o
diff --git a/builtin/hook.c b/builtin/hook.c
index b2bbc84d4d..bb64cd77ca 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -1,21 +1,67 @@
 #include "cache.h"
-
 #include "builtin.h"
+#include "config.h"
+#include "hook.h"
 #include "parse-options.h"
+#include "strbuf.h"
 
 static const char * const builtin_hook_usage[] = {
-	N_("git hook"),
+	N_("git hook list <hookname>"),
 	NULL
 };
 
-int cmd_hook(int argc, const char **argv, const char *prefix)
+static int list(int argc, const char **argv, const char *prefix)
 {
-	struct option builtin_hook_options[] = {
+	struct list_head *head, *pos;
+	struct strbuf hookname = STRBUF_INIT;
+
+	struct option list_options[] = {
 		OPT_END(),
 	};
 
-	argc = parse_options(argc, argv, prefix, builtin_hook_options,
+	argc = parse_options(argc, argv, prefix, list_options,
 			     builtin_hook_usage, 0);
 
+	if (argc < 1) {
+		usage_msg_opt(_("You must specify a hook event name to list."),
+			      builtin_hook_usage, list_options);
+	}
+
+	strbuf_addstr(&hookname, argv[0]);
+
+	head = hook_list(&hookname);
+
+	if (list_empty(head)) {
+		printf(_("no commands configured for hook '%s'\n"),
+		       hookname.buf);
+		strbuf_release(&hookname);
+		return 0;
+	}
+
+	list_for_each(pos, head) {
+		struct hook *item = list_entry(pos, struct hook, list);
+		if (item)
+			printf("%s: %s\n",
+			       config_scope_name(item->origin),
+			       item->command.buf);
+	}
+
+	clear_hook_list(head);
+	strbuf_release(&hookname);
+
 	return 0;
 }
+
+int cmd_hook(int argc, const char **argv, const char *prefix)
+{
+	struct option builtin_hook_options[] = {
+		OPT_END(),
+	};
+	if (argc < 2)
+		usage_with_options(builtin_hook_usage, builtin_hook_options);
+
+	if (!strcmp(argv[1], "list"))
+		return list(argc - 1, argv + 1, prefix);
+
+	usage_with_options(builtin_hook_usage, builtin_hook_options);
+}
diff --git a/hook.c b/hook.c
new file mode 100644
index 0000000000..fede40e925
--- /dev/null
+++ b/hook.c
@@ -0,0 +1,120 @@
+#include "cache.h"
+
+#include "hook.h"
+#include "config.h"
+
+void free_hook(struct hook *ptr)
+{
+	if (ptr) {
+		strbuf_release(&ptr->command);
+		free(ptr);
+	}
+}
+
+static void append_or_move_hook(struct list_head *head, const char *command)
+{
+	struct list_head *pos = NULL, *tmp = NULL;
+	struct hook *to_add = NULL;
+
+	/*
+	 * remove the prior entry with this command; we'll replace it at the
+	 * end.
+	 */
+	list_for_each_safe(pos, tmp, head) {
+		struct hook *it = list_entry(pos, struct hook, list);
+		if (!strcmp(it->command.buf, command)) {
+		    list_del(pos);
+		    /* we'll simply move the hook to the end */
+		    to_add = it;
+		    break;
+		}
+	}
+
+	if (!to_add) {
+		/* adding a new hook, not moving an old one */
+		to_add = xmalloc(sizeof(*to_add));
+		strbuf_init(&to_add->command, 0);
+		strbuf_addstr(&to_add->command, command);
+	}
+
+	/* re-set the scope so we show where an override was specified */
+	to_add->origin = current_config_scope();
+
+	list_add_tail(&to_add->list, head);
+}
+
+static void remove_hook(struct list_head *to_remove)
+{
+	struct hook *hook_to_remove = list_entry(to_remove, struct hook, list);
+	list_del(to_remove);
+	free_hook(hook_to_remove);
+}
+
+void clear_hook_list(struct list_head *head)
+{
+	struct list_head *pos, *tmp;
+	list_for_each_safe(pos, tmp, head)
+		remove_hook(pos);
+}
+
+struct hook_config_cb
+{
+	struct strbuf *hookname;
+	struct list_head *list;
+};
+
+static int hook_config_lookup(const char *key, const char *value, void *cb_data)
+{
+	struct hook_config_cb *data = cb_data;
+	const char *hook_key = data->hookname->buf;
+	struct list_head *head = data->list;
+
+	if (!strcmp(key, hook_key)) {
+		const char *command = value;
+		struct strbuf hookcmd_name = STRBUF_INIT;
+
+		/*
+		 * Check if a hookcmd with that name exists. If it doesn't,
+		 * 'git_config_get_value()' is documented not to touch &command,
+		 * so we don't need to do anything.
+		 */
+		strbuf_addf(&hookcmd_name, "hookcmd.%s.command", command);
+		git_config_get_value(hookcmd_name.buf, &command);
+
+		if (!command) {
+			strbuf_release(&hookcmd_name);
+			BUG("git_config_get_value overwrote a string it shouldn't have");
+		}
+
+		/*
+		 * TODO: implement an option-getting callback, e.g.
+		 *   get configs by pattern hookcmd.$value.*
+		 *   for each key+value, do_callback(key, value, cb_data)
+		 */
+
+		append_or_move_hook(head, command);
+
+		strbuf_release(&hookcmd_name);
+	}
+
+	return 0;
+}
+
+struct list_head* hook_list(const struct strbuf* hookname)
+{
+	struct strbuf hook_key = STRBUF_INIT;
+	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
+	struct hook_config_cb cb_data = { &hook_key, hook_head };
+
+	INIT_LIST_HEAD(hook_head);
+
+	if (!hookname)
+		return NULL;
+
+	strbuf_addf(&hook_key, "hook.%s.command", hookname->buf);
+
+	git_config(hook_config_lookup, &cb_data);
+
+	strbuf_release(&hook_key);
+	return hook_head;
+}
diff --git a/hook.h b/hook.h
new file mode 100644
index 0000000000..e48dfc6d27
--- /dev/null
+++ b/hook.h
@@ -0,0 +1,25 @@
+#include "config.h"
+#include "list.h"
+#include "strbuf.h"
+
+struct hook {
+	struct list_head list;
+	/*
+	 * Config file which holds the hook.*.command definition.
+	 * (This has nothing to do with the hookcmd.<name>.* configs.)
+	 */
+	enum config_scope origin;
+	/* The literal command to run. */
+	struct strbuf command;
+};
+
+/*
+ * Provides a linked list of 'struct hook' detailing commands which should run
+ * in response to the 'hookname' event, in execution order.
+ */
+struct list_head* hook_list(const struct strbuf *hookname);
+
+/* Free memory associated with a 'struct hook' */
+void free_hook(struct hook *ptr);
+/* Empties the list at 'head', calling 'free_hook()' on each entry */
+void clear_hook_list(struct list_head *head);
diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh
index 34b0df5216..6e4a3e763f 100755
--- a/t/t1360-config-based-hooks.sh
+++ b/t/t1360-config-based-hooks.sh
@@ -4,8 +4,85 @@ test_description='config-managed multihooks, including git-hook command'
 
 . ./test-lib.sh
 
-test_expect_success 'git hook command does not crash' '
-	git hook
+ROOT=
+if test_have_prereq MINGW
+then
+	# In Git for Windows, Unix-like paths work only in shell scripts;
+	# `git.exe`, however, will prefix them with the pseudo root directory
+	# (of the Unix shell). Let's accommodate for that.
+	ROOT="$(cd / && pwd)"
+fi
+
+setup_hooks () {
+	test_config hook.pre-commit.command "/path/ghi" --add
+	test_config_global hook.pre-commit.command "/path/def" --add
+}
+
+setup_hookcmd () {
+	test_config hook.pre-commit.command "abc" --add
+	test_config_global hookcmd.abc.command "/path/abc" --add
+}
+
+test_expect_success 'git hook rejects commands without a mode' '
+	test_must_fail git hook pre-commit
+'
+
+
+test_expect_success 'git hook rejects commands without a hookname' '
+	test_must_fail git hook list
+'
+
+test_expect_success 'git hook runs outside of a repo' '
+	setup_hooks &&
+
+	cat >expected <<-EOF &&
+	global: $ROOT/path/def
+	EOF
+
+	nongit git config --list --global &&
+
+	nongit git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list orders by config order' '
+	setup_hooks &&
+
+	cat >expected <<-EOF &&
+	global: $ROOT/path/def
+	local: $ROOT/path/ghi
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list dereferences a hookcmd' '
+	setup_hooks &&
+	setup_hookcmd &&
+
+	cat >expected <<-EOF &&
+	global: $ROOT/path/def
+	local: $ROOT/path/ghi
+	local: $ROOT/path/abc
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list reorders on duplicate commands' '
+	setup_hooks &&
+
+	test_config hook.pre-commit.command "/path/def" --add &&
+
+	cat >expected <<-EOF &&
+	local: $ROOT/path/ghi
+	local: $ROOT/path/def
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
 '
 
 test_done
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 04/37] hook: include hookdir hook in list
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (2 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 03/37] hook: add list command Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-12  8:30   ` Ævar Arnfjörð Bjarmason
  2021-03-11  2:10 ` [PATCH v8 05/37] hook: teach hook.runHookDir Emily Shaffer
                   ` (36 subsequent siblings)
  40 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Historically, hooks are declared by placing an executable into
$GIT_DIR/hooks/$HOOKNAME (or $HOOKDIR/$HOOKNAME). Although hooks taken
from the config are more featureful than hooks placed in the $HOOKDIR,
those hooks should not stop working for users who already have them.
Let's list them to the user, but instead of displaying a config scope
(e.g. "global: blah") we can prefix them with "hookdir:".

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

Notes:
    Since v7, fix some nits from Jonathan Tan. The largest is to move reference to
    "hookdir annotation" from this commit to the next one which introduces the
    hook.runHookDir option.

 builtin/hook.c                | 11 +++++++++--
 hook.c                        | 17 +++++++++++++++++
 hook.h                        |  1 +
 t/t1360-config-based-hooks.sh | 19 +++++++++++++++++++
 4 files changed, 46 insertions(+), 2 deletions(-)

diff --git a/builtin/hook.c b/builtin/hook.c
index bb64cd77ca..c8fbfbb39d 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -40,10 +40,15 @@ static int list(int argc, const char **argv, const char *prefix)
 
 	list_for_each(pos, head) {
 		struct hook *item = list_entry(pos, struct hook, list);
-		if (item)
+		item = list_entry(pos, struct hook, list);
+		if (item) {
+			/* Don't translate 'hookdir' - it matches the config */
 			printf("%s: %s\n",
-			       config_scope_name(item->origin),
+			       (item->from_hookdir
+				? "hookdir"
+				: config_scope_name(item->origin)),
 			       item->command.buf);
+		}
 	}
 
 	clear_hook_list(head);
@@ -60,6 +65,8 @@ int cmd_hook(int argc, const char **argv, const char *prefix)
 	if (argc < 2)
 		usage_with_options(builtin_hook_usage, builtin_hook_options);
 
+	git_config(git_default_config, NULL);
+
 	if (!strcmp(argv[1], "list"))
 		return list(argc - 1, argv + 1, prefix);
 
diff --git a/hook.c b/hook.c
index fede40e925..080e25696b 100644
--- a/hook.c
+++ b/hook.c
@@ -2,6 +2,7 @@
 
 #include "hook.h"
 #include "config.h"
+#include "run-command.h"
 
 void free_hook(struct hook *ptr)
 {
@@ -35,6 +36,7 @@ static void append_or_move_hook(struct list_head *head, const char *command)
 		to_add = xmalloc(sizeof(*to_add));
 		strbuf_init(&to_add->command, 0);
 		strbuf_addstr(&to_add->command, command);
+		to_add->from_hookdir = 0;
 	}
 
 	/* re-set the scope so we show where an override was specified */
@@ -115,6 +117,21 @@ struct list_head* hook_list(const struct strbuf* hookname)
 
 	git_config(hook_config_lookup, &cb_data);
 
+	if (have_git_dir()) {
+		const char *legacy_hook_path = find_hook(hookname->buf);
+
+		/* Unconditionally add legacy hook, but annotate it. */
+		if (legacy_hook_path) {
+			struct hook *legacy_hook;
+
+			append_or_move_hook(hook_head,
+					    absolute_path(legacy_hook_path));
+			legacy_hook = list_entry(hook_head->prev, struct hook,
+						 list);
+			legacy_hook->from_hookdir = 1;
+		}
+	}
+
 	strbuf_release(&hook_key);
 	return hook_head;
 }
diff --git a/hook.h b/hook.h
index e48dfc6d27..a97d43670d 100644
--- a/hook.h
+++ b/hook.h
@@ -11,6 +11,7 @@ struct hook {
 	enum config_scope origin;
 	/* The literal command to run. */
 	struct strbuf command;
+	unsigned from_hookdir : 1;
 };
 
 /*
diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh
index 6e4a3e763f..0f12af4659 100755
--- a/t/t1360-config-based-hooks.sh
+++ b/t/t1360-config-based-hooks.sh
@@ -23,6 +23,14 @@ setup_hookcmd () {
 	test_config_global hookcmd.abc.command "/path/abc" --add
 }
 
+setup_hookdir () {
+	mkdir .git/hooks
+	write_script .git/hooks/pre-commit <<-EOF
+	echo \"Legacy Hook\"
+	EOF
+	test_when_finished rm -rf .git/hooks
+}
+
 test_expect_success 'git hook rejects commands without a mode' '
 	test_must_fail git hook pre-commit
 '
@@ -85,4 +93,15 @@ test_expect_success 'git hook list reorders on duplicate commands' '
 	test_cmp expected actual
 '
 
+test_expect_success 'git hook list shows hooks from the hookdir' '
+	setup_hookdir &&
+
+	cat >expected <<-EOF &&
+	hookdir: $(pwd)/.git/hooks/pre-commit
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
 test_done
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 05/37] hook: teach hook.runHookDir
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (3 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 04/37] hook: include hookdir hook in list Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-12  8:33   ` Ævar Arnfjörð Bjarmason
  2021-03-11  2:10 ` [PATCH v8 06/37] hook: implement hookcmd.<name>.skip Emily Shaffer
                   ` (35 subsequent siblings)
  40 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

For now, just give a hint about how these hooks will be run in 'git hook
list'. Later on, though, we will pay attention to this enum when running
the hooks.
---

Notes:
    Since v7, tidied up the behavior of the HOOK_UNKNOWN flag and added a test to
    enforce it - now it matches the design doc much better.
    
    Also, thanks Jonathan Tan for pointing out that the commit message made no sense
    and was targeted for a different change. Rewrote the commit message now.
    
    Plus, added HOOK_ERROR flag per Junio and Jonathan Nieder.

 - Emily

 Documentation/config/hook.txt |  5 +++
 builtin/hook.c                | 69 +++++++++++++++++++++++++++++++---
 hook.c                        | 24 ++++++++++++
 hook.h                        | 16 ++++++++
 t/t1360-config-based-hooks.sh | 71 +++++++++++++++++++++++++++++++++++
 5 files changed, 180 insertions(+), 5 deletions(-)

diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
index 71449ecbc7..75312754ae 100644
--- a/Documentation/config/hook.txt
+++ b/Documentation/config/hook.txt
@@ -7,3 +7,8 @@ hookcmd.<name>.command::
 	A command to execute during a hook for which <name> has been specified
 	as a command. This can be an executable on your device or a oneliner for
 	your shell. See linkgit:git-hook[1].
+
+hook.runHookDir::
+	Controls how hooks contained in your hookdir are executed. Can be any of
+	"yes", "warn", "interactive", or "no". Defaults to "yes". See
+	linkgit:git-hook[1] and linkgit:git-config[1] "core.hooksPath").
diff --git a/builtin/hook.c b/builtin/hook.c
index c8fbfbb39d..310f696ebf 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -10,10 +10,13 @@ static const char * const builtin_hook_usage[] = {
 	NULL
 };
 
+static enum hookdir_opt should_run_hookdir;
+
 static int list(int argc, const char **argv, const char *prefix)
 {
 	struct list_head *head, *pos;
 	struct strbuf hookname = STRBUF_INIT;
+	struct strbuf hookdir_annotation = STRBUF_INIT;
 
 	struct option list_options[] = {
 		OPT_END(),
@@ -38,20 +41,48 @@ static int list(int argc, const char **argv, const char *prefix)
 		return 0;
 	}
 
+	switch (should_run_hookdir) {
+		case HOOKDIR_NO:
+			strbuf_addstr(&hookdir_annotation, _(" (will not run)"));
+			break;
+		case HOOKDIR_ERROR:
+			strbuf_addstr(&hookdir_annotation, _(" (will error and not run)"));
+			break;
+		case HOOKDIR_INTERACTIVE:
+			strbuf_addstr(&hookdir_annotation, _(" (will prompt)"));
+			break;
+		case HOOKDIR_WARN:
+			strbuf_addstr(&hookdir_annotation, _(" (will warn but run)"));
+			break;
+		case HOOKDIR_YES:
+		/*
+		 * The default behavior should agree with
+		 * hook.c:configured_hookdir_opt(). HOOKDIR_UNKNOWN should just
+		 * do the default behavior.
+		 */
+		case HOOKDIR_UNKNOWN:
+		default:
+			break;
+	}
+
 	list_for_each(pos, head) {
 		struct hook *item = list_entry(pos, struct hook, list);
 		item = list_entry(pos, struct hook, list);
 		if (item) {
 			/* Don't translate 'hookdir' - it matches the config */
-			printf("%s: %s\n",
+			printf("%s: %s%s\n",
 			       (item->from_hookdir
 				? "hookdir"
 				: config_scope_name(item->origin)),
-			       item->command.buf);
+			       item->command.buf,
+			       (item->from_hookdir
+				? hookdir_annotation.buf
+				: ""));
 		}
 	}
 
 	clear_hook_list(head);
+	strbuf_release(&hookdir_annotation);
 	strbuf_release(&hookname);
 
 	return 0;
@@ -59,16 +90,44 @@ static int list(int argc, const char **argv, const char *prefix)
 
 int cmd_hook(int argc, const char **argv, const char *prefix)
 {
+	const char *run_hookdir = NULL;
+
 	struct option builtin_hook_options[] = {
+		OPT_STRING(0, "run-hookdir", &run_hookdir, N_("option"),
+			   N_("what to do with hooks found in the hookdir")),
 		OPT_END(),
 	};
-	if (argc < 2)
+
+	argc = parse_options(argc, argv, prefix, builtin_hook_options,
+			     builtin_hook_usage, 0);
+
+	/* after the parse, we should have "<command> <hookname> <args...>" */
+	if (argc < 1)
 		usage_with_options(builtin_hook_usage, builtin_hook_options);
 
 	git_config(git_default_config, NULL);
 
-	if (!strcmp(argv[1], "list"))
-		return list(argc - 1, argv + 1, prefix);
+
+	/* argument > config */
+	if (run_hookdir)
+		if (!strcmp(run_hookdir, "no"))
+			should_run_hookdir = HOOKDIR_NO;
+		else if (!strcmp(run_hookdir, "error"))
+			should_run_hookdir = HOOKDIR_ERROR;
+		else if (!strcmp(run_hookdir, "yes"))
+			should_run_hookdir = HOOKDIR_YES;
+		else if (!strcmp(run_hookdir, "warn"))
+			should_run_hookdir = HOOKDIR_WARN;
+		else if (!strcmp(run_hookdir, "interactive"))
+			should_run_hookdir = HOOKDIR_INTERACTIVE;
+		else
+			die(_("'%s' is not a valid option for --run-hookdir "
+			      "(yes, warn, interactive, no)"), run_hookdir);
+	else
+		should_run_hookdir = configured_hookdir_opt();
+
+	if (!strcmp(argv[0], "list"))
+		return list(argc, argv, prefix);
 
 	usage_with_options(builtin_hook_usage, builtin_hook_options);
 }
diff --git a/hook.c b/hook.c
index 080e25696b..039ff0a378 100644
--- a/hook.c
+++ b/hook.c
@@ -102,6 +102,30 @@ static int hook_config_lookup(const char *key, const char *value, void *cb_data)
 	return 0;
 }
 
+enum hookdir_opt configured_hookdir_opt(void)
+{
+	const char *key;
+	if (git_config_get_value("hook.runhookdir", &key))
+		return HOOKDIR_YES; /* by default, just run it. */
+
+	if (!strcmp(key, "no"))
+		return HOOKDIR_NO;
+
+	if (!strcmp(key, "error"))
+		return HOOKDIR_ERROR;
+
+	if (!strcmp(key, "yes"))
+		return HOOKDIR_YES;
+
+	if (!strcmp(key, "warn"))
+		return HOOKDIR_WARN;
+
+	if (!strcmp(key, "interactive"))
+		return HOOKDIR_INTERACTIVE;
+
+	return HOOKDIR_UNKNOWN;
+}
+
 struct list_head* hook_list(const struct strbuf* hookname)
 {
 	struct strbuf hook_key = STRBUF_INIT;
diff --git a/hook.h b/hook.h
index a97d43670d..1c4b953aec 100644
--- a/hook.h
+++ b/hook.h
@@ -20,6 +20,22 @@ struct hook {
  */
 struct list_head* hook_list(const struct strbuf *hookname);
 
+enum hookdir_opt
+{
+	HOOKDIR_NO,
+	HOOKDIR_ERROR,
+	HOOKDIR_WARN,
+	HOOKDIR_INTERACTIVE,
+	HOOKDIR_YES,
+	HOOKDIR_UNKNOWN,
+};
+
+/*
+ * Provides the hookdir_opt specified in the config without consulting any
+ * command line arguments.
+ */
+enum hookdir_opt configured_hookdir_opt(void);
+
 /* Free memory associated with a 'struct hook' */
 void free_hook(struct hook *ptr);
 /* Empties the list at 'head', calling 'free_hook()' on each entry */
diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh
index 0f12af4659..66b0b6b7ad 100755
--- a/t/t1360-config-based-hooks.sh
+++ b/t/t1360-config-based-hooks.sh
@@ -104,4 +104,75 @@ test_expect_success 'git hook list shows hooks from the hookdir' '
 	test_cmp expected actual
 '
 
+test_expect_success 'hook.runHookDir = no is respected by list' '
+	setup_hookdir &&
+
+	test_config hook.runHookDir "no" &&
+
+	cat >expected <<-EOF &&
+	hookdir: $(pwd)/.git/hooks/pre-commit (will not run)
+	EOF
+
+	git hook list pre-commit >actual &&
+	# the hookdir annotation is translated
+	test_i18ncmp expected actual
+'
+
+test_expect_success 'hook.runHookDir = error is respected by list' '
+	setup_hookdir &&
+
+	test_config hook.runHookDir "error" &&
+
+	cat >expected <<-EOF &&
+	hookdir: $(pwd)/.git/hooks/pre-commit (will error and not run)
+	EOF
+
+	git hook list pre-commit >actual &&
+	# the hookdir annotation is translated
+	test_i18ncmp expected actual
+'
+
+test_expect_success 'hook.runHookDir = warn is respected by list' '
+	setup_hookdir &&
+
+	test_config hook.runHookDir "warn" &&
+
+	cat >expected <<-EOF &&
+	hookdir: $(pwd)/.git/hooks/pre-commit (will warn but run)
+	EOF
+
+	git hook list pre-commit >actual &&
+	# the hookdir annotation is translated
+	test_i18ncmp expected actual
+'
+
+
+test_expect_success 'hook.runHookDir = interactive is respected by list' '
+	setup_hookdir &&
+
+	test_config hook.runHookDir "interactive" &&
+
+	cat >expected <<-EOF &&
+	hookdir: $(pwd)/.git/hooks/pre-commit (will prompt)
+	EOF
+
+	git hook list pre-commit >actual &&
+	# the hookdir annotation is translated
+	test_i18ncmp expected actual
+'
+
+test_expect_success 'hook.runHookDir is tolerant to unknown values' '
+	setup_hookdir &&
+
+	test_config hook.runHookDir "junk" &&
+
+	cat >expected <<-EOF &&
+	hookdir: $(pwd)/.git/hooks/pre-commit
+	EOF
+
+	git hook list pre-commit >actual &&
+	# the hookdir annotation is translated
+	test_i18ncmp expected actual
+'
+
 test_done
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 06/37] hook: implement hookcmd.<name>.skip
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (4 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 05/37] hook: teach hook.runHookDir Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-12  8:49   ` Ævar Arnfjörð Bjarmason
  2021-03-11  2:10 ` [PATCH v8 07/37] parse-options: parse into strvec Emily Shaffer
                   ` (34 subsequent siblings)
  40 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

If a user wants a specific repo to skip execution of a hook which is set
at a global or system level, they will be able to do so by specifying
'skip' in their repo config:

~/.gitconfig
  [hook.pre-commit]
    command = skippable-oneliner
    command = skippable-hookcmd

  [hookcmd.skippable-hookcmd]
    command = foo.sh

$GIT_DIR/.git/config
  [hookcmd.skippable-oneliner]
    skip = true
  [hookcmd.skippable-hookcmd]
    skip = true

Later it may make sense to add an option like
"hookcmd.<name>.<hook-event>-skip" - but for simplicity, let's start
with a universal skip setting like this.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

 Documentation/config/hook.txt |  8 ++++++++
 Documentation/git-hook.txt    | 33 +++++++++++++++++++++++++++++++++
 hook.c                        | 35 ++++++++++++++++++++++++++---------
 t/t1360-config-based-hooks.sh | 35 +++++++++++++++++++++++++++++++++++
 4 files changed, 102 insertions(+), 9 deletions(-)

diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
index 75312754ae..8b12512e33 100644
--- a/Documentation/config/hook.txt
+++ b/Documentation/config/hook.txt
@@ -8,6 +8,14 @@ hookcmd.<name>.command::
 	as a command. This can be an executable on your device or a oneliner for
 	your shell. See linkgit:git-hook[1].
 
+hookcmd.<name>.skip::
+	Specify this boolean to remove a command from earlier in the execution
+	order. Useful if you want to make a single repo an exception to hook
+	configured at the system or global scope. If there is no hookcmd
+	specified for the command you want to skip, you can use the value of
+	`hook.<command>.command` as <name> as a shortcut. The "skip" setting
+	must be specified after the "hook.<command>.command" to have an effect.
+
 hook.runHookDir::
 	Controls how hooks contained in your hookdir are executed. Can be any of
 	"yes", "warn", "interactive", or "no". Defaults to "yes". See
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index f19875ed68..c84520cb38 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -54,6 +54,39 @@ $ git hook list "prepare-commit-msg"
 local: /bin/linter --c
 ----
 
+If there is a command you wish to run in most cases but have one or two
+exceptional repos where it should be skipped, you can use specify
+`hookcmd.<name>.skip`, for example:
+
+System config
+----
+  [hook "pre-commit"]
+    command = check-for-secrets
+
+  [hookcmd "check-for-secrets"]
+    command = /bin/secret-checker --aggressive
+----
+
+Local config
+----
+  [hookcmd "check-for-secrets"]
+    skip = true
+  # This works for inlined hook commands, too:
+  [hookcmd "~/typocheck.sh"]
+    skip = true
+----
+
+After these configs are added, the hook list becomes:
+
+----
+$ git hook list "post-commit"
+global: /bin/linter --c
+local: python ~/run-test-suite.py
+
+$ git hook list "pre-commit"
+no commands configured for hook 'pre-commit'
+----
+
 COMMANDS
 --------
 
diff --git a/hook.c b/hook.c
index 039ff0a378..37b740d58d 100644
--- a/hook.c
+++ b/hook.c
@@ -12,24 +12,25 @@ void free_hook(struct hook *ptr)
 	}
 }
 
-static void append_or_move_hook(struct list_head *head, const char *command)
+static struct hook * find_hook_by_command(struct list_head *head, const char *command)
 {
 	struct list_head *pos = NULL, *tmp = NULL;
-	struct hook *to_add = NULL;
+	struct hook *found = NULL;
 
-	/*
-	 * remove the prior entry with this command; we'll replace it at the
-	 * end.
-	 */
 	list_for_each_safe(pos, tmp, head) {
 		struct hook *it = list_entry(pos, struct hook, list);
 		if (!strcmp(it->command.buf, command)) {
 		    list_del(pos);
-		    /* we'll simply move the hook to the end */
-		    to_add = it;
+		    found = it;
 		    break;
 		}
 	}
+	return found;
+}
+
+static void append_or_move_hook(struct list_head *head, const char *command)
+{
+	struct hook *to_add = find_hook_by_command(head, command);
 
 	if (!to_add) {
 		/* adding a new hook, not moving an old one */
@@ -74,12 +75,22 @@ static int hook_config_lookup(const char *key, const char *value, void *cb_data)
 	if (!strcmp(key, hook_key)) {
 		const char *command = value;
 		struct strbuf hookcmd_name = STRBUF_INIT;
+		int skip = 0;
+
+		/*
+		 * Check if we're removing that hook instead. Hookcmds are
+		 * removed by name, and inlined hooks are removed by command
+		 * content.
+		 */
+		strbuf_addf(&hookcmd_name, "hookcmd.%s.skip", command);
+		git_config_get_bool(hookcmd_name.buf, &skip);
 
 		/*
 		 * Check if a hookcmd with that name exists. If it doesn't,
 		 * 'git_config_get_value()' is documented not to touch &command,
 		 * so we don't need to do anything.
 		 */
+		strbuf_reset(&hookcmd_name);
 		strbuf_addf(&hookcmd_name, "hookcmd.%s.command", command);
 		git_config_get_value(hookcmd_name.buf, &command);
 
@@ -94,7 +105,13 @@ static int hook_config_lookup(const char *key, const char *value, void *cb_data)
 		 *   for each key+value, do_callback(key, value, cb_data)
 		 */
 
-		append_or_move_hook(head, command);
+		if (skip) {
+			struct hook *to_remove = find_hook_by_command(head, command);
+			if (to_remove)
+				remove_hook(&(to_remove->list));
+		} else {
+			append_or_move_hook(head, command);
+		}
 
 		strbuf_release(&hookcmd_name);
 	}
diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh
index 66b0b6b7ad..a9b1b046c1 100755
--- a/t/t1360-config-based-hooks.sh
+++ b/t/t1360-config-based-hooks.sh
@@ -146,6 +146,41 @@ test_expect_success 'hook.runHookDir = warn is respected by list' '
 	test_i18ncmp expected actual
 '
 
+test_expect_success 'git hook list removes skipped hookcmd' '
+	setup_hookcmd &&
+	test_config hookcmd.abc.skip "true" --add &&
+
+	cat >expected <<-EOF &&
+	no commands configured for hook '\''pre-commit'\''
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_i18ncmp expected actual
+'
+
+test_expect_success 'git hook list ignores skip referring to unused hookcmd' '
+	test_config hookcmd.abc.command "/path/abc" --add &&
+	test_config hookcmd.abc.skip "true" --add &&
+
+	cat >expected <<-EOF &&
+	no commands configured for hook '\''pre-commit'\''
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_i18ncmp expected actual
+'
+
+test_expect_success 'git hook list removes skipped inlined hook' '
+	setup_hooks &&
+	test_config hookcmd."$ROOT/path/ghi".skip "true" --add &&
+
+	cat >expected <<-EOF &&
+	global: $ROOT/path/def
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
 
 test_expect_success 'hook.runHookDir = interactive is respected by list' '
 	setup_hookdir &&
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 07/37] parse-options: parse into strvec
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (5 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 06/37] hook: implement hookcmd.<name>.skip Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-12  8:50   ` Ævar Arnfjörð Bjarmason
  2021-03-11  2:10 ` [PATCH v8 08/37] hook: add 'run' subcommand Emily Shaffer
                   ` (33 subsequent siblings)
  40 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

parse-options already knows how to read into a string_list, and it knows
how to read into an strvec as a passthrough (that is, including the
argument as well as its value). string_list and strvec serve similar
purposes but are somewhat painful to convert between; so, let's teach
parse-options to read values of string arguments directly into an
strvec without preserving the argument name.

This is useful if collecting generic arguments to pass through to
another command, for example, 'git hook run --arg "--quiet" --arg
"--format=pretty" some-hook'. The resulting strvec would contain
{ "--quiet", "--format=pretty" }.

The implementation is based on that of OPT_STRING_LIST.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

Notes:
    Since v7, updated the reference doc to make the intended usage for OPT_STRVEC
    more clear.
    
    Since v4, fixed one or two more places where I missed the argv_array->strvec
    rename.

 Documentation/technical/api-parse-options.txt |  7 +++++++
 parse-options-cb.c                            | 16 ++++++++++++++++
 parse-options.h                               |  4 ++++
 3 files changed, 27 insertions(+)

diff --git a/Documentation/technical/api-parse-options.txt b/Documentation/technical/api-parse-options.txt
index 5a60bbfa7f..f79b17e7fc 100644
--- a/Documentation/technical/api-parse-options.txt
+++ b/Documentation/technical/api-parse-options.txt
@@ -173,6 +173,13 @@ There are some macros to easily define options:
 	The string argument is stored as an element in `string_list`.
 	Use of `--no-option` will clear the list of preceding values.
 
+`OPT_STRVEC(short, long, &struct strvec, arg_str, description)`::
+	Introduce an option with a string argument, meant to be specified
+	multiple times.
+	The string argument is stored as an element in `strvec`, and later
+	arguments are added to the same `strvec`.
+	Use of `--no-option` will clear the list of preceding values.
+
 `OPT_INTEGER(short, long, &int_var, description)`::
 	Introduce an option with integer argument.
 	The integer is put into `int_var`.
diff --git a/parse-options-cb.c b/parse-options-cb.c
index 4542d4d3f9..c2451dfb1b 100644
--- a/parse-options-cb.c
+++ b/parse-options-cb.c
@@ -207,6 +207,22 @@ int parse_opt_string_list(const struct option *opt, const char *arg, int unset)
 	return 0;
 }
 
+int parse_opt_strvec(const struct option *opt, const char *arg, int unset)
+{
+	struct strvec *v = opt->value;
+
+	if (unset) {
+		strvec_clear(v);
+		return 0;
+	}
+
+	if (!arg)
+		return -1;
+
+	strvec_push(v, arg);
+	return 0;
+}
+
 int parse_opt_noop_cb(const struct option *opt, const char *arg, int unset)
 {
 	return 0;
diff --git a/parse-options.h b/parse-options.h
index ff6506a504..44c4ac08e9 100644
--- a/parse-options.h
+++ b/parse-options.h
@@ -177,6 +177,9 @@ struct option {
 #define OPT_STRING_LIST(s, l, v, a, h) \
 				    { OPTION_CALLBACK, (s), (l), (v), (a), \
 				      (h), 0, &parse_opt_string_list }
+#define OPT_STRVEC(s, l, v, a, h) \
+				    { OPTION_CALLBACK, (s), (l), (v), (a), \
+				      (h), 0, &parse_opt_strvec }
 #define OPT_UYN(s, l, v, h)         { OPTION_CALLBACK, (s), (l), (v), NULL, \
 				      (h), PARSE_OPT_NOARG, &parse_opt_tertiary }
 #define OPT_EXPIRY_DATE(s, l, v, h) \
@@ -296,6 +299,7 @@ int parse_opt_commits(const struct option *, const char *, int);
 int parse_opt_commit(const struct option *, const char *, int);
 int parse_opt_tertiary(const struct option *, const char *, int);
 int parse_opt_string_list(const struct option *, const char *, int);
+int parse_opt_strvec(const struct option *, const char *, int);
 int parse_opt_noop_cb(const struct option *, const char *, int);
 enum parse_opt_result parse_opt_unknown_cb(struct parse_opt_ctx_t *ctx,
 					   const struct option *,
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 08/37] hook: add 'run' subcommand
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (6 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 07/37] parse-options: parse into strvec Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-12  8:54   ` Ævar Arnfjörð Bjarmason
  2021-03-12 10:22   ` Junio C Hamano
  2021-03-11  2:10 ` [PATCH v8 09/37] hook: introduce hook_exists() Emily Shaffer
                   ` (32 subsequent siblings)
  40 siblings, 2 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

In order to enable hooks to be run as an external process, by a
standalone Git command, or by tools which wrap Git, provide an external
means to run all configured hook commands for a given hook event.

For now, the hook commands will run in config order, in series. As
alternate ordering or parallelism is supported in the future, we should
add knobs to use those to the command line as well.

As with the legacy hook implementation, all stdout generated by hook
commands is redirected to stderr. Piping from stdin is not yet
supported.

Legacy hooks (those present in $GITDIR/hooks) are run at the end of the
execution list. They can be disabled, or made to print warnings, or to
prompt before running, with the 'hook.runHookDir' config.

Users may wish to provide hook commands like 'git config
hook.pre-commit.command "~/linter.sh --pre-commit"'. To enable this,
config-defined hooks are run in a shell. (Since hooks in $GITDIR/hooks
can't be specified with included arguments or paths which need expansion
like this, they are run without a shell instead.)

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

Notes:
    Since v7, added support for "error" hook.runHookDir setting.
    
    Since v4, updated the docs, and did less local application of single
    quotes. In order for hookdir hooks to run successfully with a space in
    the path, though, they must not be run with 'sh -c'. So we can treat the
    hookdir hooks specially, and warn users via doc about special
    considerations for configured hooks with spaces in their path.

 Documentation/git-hook.txt    |  31 +++++++-
 builtin/hook.c                |  42 ++++++++++-
 hook.c                        | 128 ++++++++++++++++++++++++++++++++++
 hook.h                        |  26 +++++++
 t/t1360-config-based-hooks.sh |  72 ++++++++++++++++++-
 5 files changed, 292 insertions(+), 7 deletions(-)

diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index c84520cb38..8f96c347ea 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -9,11 +9,12 @@ SYNOPSIS
 --------
 [verse]
 'git hook' list <hook-name>
+'git hook' run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] <hook-name>
 
 DESCRIPTION
 -----------
-You can list configured hooks with this command. Later, you will be able to run,
-add, and modify hooks with this command.
+You can list and run configured hooks with this command. Later, you will be able
+to add and modify hooks with this command.
 
 This command parses the default configuration files for sections `hook` and
 `hookcmd`. `hook` is used to describe the commands which will be run during a
@@ -97,6 +98,32 @@ in the order they should be run, and print the config scope where the relevant
 `hook.<hook-name>.command` was specified, not the `hookcmd` (if applicable).
 This output is human-readable and the format is subject to change over time.
 
+run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] `<hook-name>`::
+
+Runs hooks configured for `<hook-name>`, in the same order displayed by `git
+hook list`. Hooks configured this way may be run prepended with `sh -c`, so
+paths containing special characters or spaces should be wrapped in single
+quotes: `command = '/my/path with spaces/script.sh' some args`.
+
+OPTIONS
+-------
+--run-hookdir::
+	Overrides the hook.runHookDir config. Must be 'yes', 'warn',
+	'interactive', or 'no'. Specifies how to handle hooks located in the Git
+	hook directory (core.hooksPath).
+
+-a::
+--arg::
+	Only valid for `run`.
++
+Specify arguments to pass to every hook that is run.
+
+-e::
+--env::
+	Only valid for `run`.
++
+Specify environment variables to set for every hook that is run.
+
 CONFIGURATION
 -------------
 include::config/hook.txt[]
diff --git a/builtin/hook.c b/builtin/hook.c
index 310f696ebf..e823a96238 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -4,9 +4,11 @@
 #include "hook.h"
 #include "parse-options.h"
 #include "strbuf.h"
+#include "strvec.h"
 
 static const char * const builtin_hook_usage[] = {
 	N_("git hook list <hookname>"),
+	N_("git hook run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] <hookname>"),
 	NULL
 };
 
@@ -88,6 +90,40 @@ static int list(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+static int run(int argc, const char **argv, const char *prefix)
+{
+	struct strbuf hookname = STRBUF_INIT;
+	struct run_hooks_opt opt;
+	int rc = 0;
+
+	struct option run_options[] = {
+		OPT_STRVEC('e', "env", &opt.env, N_("var"),
+			   N_("environment variables for hook to use")),
+		OPT_STRVEC('a', "arg", &opt.args, N_("args"),
+			   N_("argument to pass to hook")),
+		OPT_END(),
+	};
+
+	run_hooks_opt_init(&opt);
+
+	argc = parse_options(argc, argv, prefix, run_options,
+			     builtin_hook_usage, 0);
+
+	if (argc < 1)
+		usage_msg_opt(_("You must specify a hook event to run."),
+			      builtin_hook_usage, run_options);
+
+	strbuf_addstr(&hookname, argv[0]);
+	opt.run_hookdir = should_run_hookdir;
+
+	rc = run_hooks(hookname.buf, &opt);
+
+	strbuf_release(&hookname);
+	run_hooks_opt_clear(&opt);
+
+	return rc;
+}
+
 int cmd_hook(int argc, const char **argv, const char *prefix)
 {
 	const char *run_hookdir = NULL;
@@ -99,10 +135,10 @@ int cmd_hook(int argc, const char **argv, const char *prefix)
 	};
 
 	argc = parse_options(argc, argv, prefix, builtin_hook_options,
-			     builtin_hook_usage, 0);
+			     builtin_hook_usage, PARSE_OPT_KEEP_UNKNOWN);
 
 	/* after the parse, we should have "<command> <hookname> <args...>" */
-	if (argc < 1)
+	if (argc < 2)
 		usage_with_options(builtin_hook_usage, builtin_hook_options);
 
 	git_config(git_default_config, NULL);
@@ -128,6 +164,8 @@ int cmd_hook(int argc, const char **argv, const char *prefix)
 
 	if (!strcmp(argv[0], "list"))
 		return list(argc, argv, prefix);
+	if (!strcmp(argv[0], "run"))
+		return run(argc, argv, prefix);
 
 	usage_with_options(builtin_hook_usage, builtin_hook_options);
 }
diff --git a/hook.c b/hook.c
index 37b740d58d..d166d17fb0 100644
--- a/hook.c
+++ b/hook.c
@@ -3,6 +3,7 @@
 #include "hook.h"
 #include "config.h"
 #include "run-command.h"
+#include "prompt.h"
 
 void free_hook(struct hook *ptr)
 {
@@ -143,6 +144,64 @@ enum hookdir_opt configured_hookdir_opt(void)
 	return HOOKDIR_UNKNOWN;
 }
 
+static int should_include_hookdir(const char *path, enum hookdir_opt cfg)
+{
+	struct strbuf prompt = STRBUF_INIT;
+	/*
+	 * If the path doesn't exist, don't bother adding the empty hook and
+	 * don't bother checking the config or prompting the user.
+	 */
+	if (!path)
+		return 0;
+
+	switch (cfg)
+	{
+		case HOOKDIR_ERROR:
+			fprintf(stderr, _("Skipping legacy hook at '%s'\n"),
+				path);
+			/* FALLTHROUGH */
+		case HOOKDIR_NO:
+			return 0;
+		case HOOKDIR_WARN:
+			fprintf(stderr, _("Running legacy hook at '%s'\n"),
+				path);
+			return 1;
+		case HOOKDIR_INTERACTIVE:
+			do {
+				/*
+				 * TRANSLATORS: Make sure to include [Y] and [n]
+				 * in your translation. Only English input is
+				 * accepted. Default option is "yes".
+				 */
+				fprintf(stderr, _("Run '%s'? [Yn] "), path);
+				git_read_line_interactively(&prompt);
+				strbuf_tolower(&prompt);
+				if (starts_with(prompt.buf, "n")) {
+					strbuf_release(&prompt);
+					return 0;
+				} else if (starts_with(prompt.buf, "y")) {
+					strbuf_release(&prompt);
+					return 1;
+				}
+				/* otherwise, we didn't understand the input */
+			} while (prompt.len); /* an empty reply means "Yes" */
+			strbuf_release(&prompt);
+			return 1;
+		/*
+		 * HOOKDIR_UNKNOWN should match the default behavior, but let's
+		 * give a heads up to the user.
+		 */
+		case HOOKDIR_UNKNOWN:
+			fprintf(stderr,
+				_("Unrecognized value for 'hook.runHookDir'. "
+				  "Is there a typo? "));
+			/* FALLTHROUGH */
+		case HOOKDIR_YES:
+		default:
+			return 1;
+	}
+}
+
 struct list_head* hook_list(const struct strbuf* hookname)
 {
 	struct strbuf hook_key = STRBUF_INIT;
@@ -176,3 +235,72 @@ struct list_head* hook_list(const struct strbuf* hookname)
 	strbuf_release(&hook_key);
 	return hook_head;
 }
+
+void run_hooks_opt_init(struct run_hooks_opt *o)
+{
+	strvec_init(&o->env);
+	strvec_init(&o->args);
+	o->run_hookdir = configured_hookdir_opt();
+}
+
+void run_hooks_opt_clear(struct run_hooks_opt *o)
+{
+	strvec_clear(&o->env);
+	strvec_clear(&o->args);
+}
+
+static void prepare_hook_cp(struct hook *hook, struct run_hooks_opt *options,
+			    struct child_process *cp)
+{
+	if (!hook)
+		return;
+
+	cp->no_stdin = 1;
+	cp->env = options->env.v;
+	cp->stdout_to_stderr = 1;
+	cp->trace2_hook_name = hook->command.buf;
+
+	/*
+	 * Commands from the config could be oneliners, but we know
+	 * for certain that hookdir commands are not.
+	 */
+	cp->use_shell = !hook->from_hookdir;
+
+	/* add command */
+	strvec_push(&cp->args, hook->command.buf);
+
+	/*
+	 * add passed-in argv, without expanding - let the user get back
+	 * exactly what they put in
+	 */
+	strvec_pushv(&cp->args, options->args.v);
+}
+
+int run_hooks(const char *hookname, struct run_hooks_opt *options)
+{
+	struct strbuf hookname_str = STRBUF_INIT;
+	struct list_head *to_run, *pos = NULL, *tmp = NULL;
+	int rc = 0;
+
+	if (!options)
+		BUG("a struct run_hooks_opt must be provided to run_hooks");
+
+	strbuf_addstr(&hookname_str, hookname);
+
+	to_run = hook_list(&hookname_str);
+
+	list_for_each_safe(pos, tmp, to_run) {
+		struct child_process hook_proc = CHILD_PROCESS_INIT;
+		struct hook *hook = list_entry(pos, struct hook, list);
+
+		if (hook->from_hookdir &&
+		    !should_include_hookdir(hook->command.buf, options->run_hookdir))
+			continue;
+
+		prepare_hook_cp(hook, options, &hook_proc);
+
+		rc |= run_command(&hook_proc);
+	}
+
+	return rc;
+}
diff --git a/hook.h b/hook.h
index 1c4b953aec..c24b2c9ecd 100644
--- a/hook.h
+++ b/hook.h
@@ -1,6 +1,7 @@
 #include "config.h"
 #include "list.h"
 #include "strbuf.h"
+#include "strvec.h"
 
 struct hook {
 	struct list_head list;
@@ -36,6 +37,31 @@ enum hookdir_opt
  */
 enum hookdir_opt configured_hookdir_opt(void);
 
+struct run_hooks_opt
+{
+	/* Environment vars to be set for each hook */
+	struct strvec env;
+
+	/* Args to be passed to each hook */
+	struct strvec args;
+
+	/*
+	 * How should the hookdir be handled?
+	 * Leave the RUN_HOOKS_OPT_INIT default in most cases; this only needs
+	 * to be overridden if the user can override it at the command line.
+	 */
+	enum hookdir_opt run_hookdir;
+};
+
+void run_hooks_opt_init(struct run_hooks_opt *o);
+void run_hooks_opt_clear(struct run_hooks_opt *o);
+
+/*
+ * Runs all hooks associated to the 'hookname' event in order. Each hook will be
+ * passed 'env' and 'args'.
+ */
+int run_hooks(const char *hookname, struct run_hooks_opt *options);
+
 /* Free memory associated with a 'struct hook' */
 void free_hook(struct hook *ptr);
 /* Empties the list at 'head', calling 'free_hook()' on each entry */
diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh
index a9b1b046c1..1fca83d536 100755
--- a/t/t1360-config-based-hooks.sh
+++ b/t/t1360-config-based-hooks.sh
@@ -115,7 +115,10 @@ test_expect_success 'hook.runHookDir = no is respected by list' '
 
 	git hook list pre-commit >actual &&
 	# the hookdir annotation is translated
-	test_i18ncmp expected actual
+	test_i18ncmp expected actual &&
+
+	git hook run pre-commit 2>actual &&
+	test_must_be_empty actual
 '
 
 test_expect_success 'hook.runHookDir = error is respected by list' '
@@ -129,6 +132,13 @@ test_expect_success 'hook.runHookDir = error is respected by list' '
 
 	git hook list pre-commit >actual &&
 	# the hookdir annotation is translated
+	test_i18ncmp expected actual &&
+
+	cat >expected <<-EOF &&
+	Skipping legacy hook at '\''$(pwd)/.git/hooks/pre-commit'\''
+	EOF
+
+	git hook run pre-commit 2>actual &&
 	test_i18ncmp expected actual
 '
 
@@ -143,6 +153,14 @@ test_expect_success 'hook.runHookDir = warn is respected by list' '
 
 	git hook list pre-commit >actual &&
 	# the hookdir annotation is translated
+	test_i18ncmp expected actual &&
+
+	cat >expected <<-EOF &&
+	Running legacy hook at '\''$(pwd)/.git/hooks/pre-commit'\''
+	"Legacy Hook"
+	EOF
+
+	git hook run pre-commit 2>actual &&
 	test_i18ncmp expected actual
 '
 
@@ -182,7 +200,7 @@ test_expect_success 'git hook list removes skipped inlined hook' '
 	test_cmp expected actual
 '
 
-test_expect_success 'hook.runHookDir = interactive is respected by list' '
+test_expect_success 'hook.runHookDir = interactive is respected by list and run' '
 	setup_hookdir &&
 
 	test_config hook.runHookDir "interactive" &&
@@ -193,7 +211,55 @@ test_expect_success 'hook.runHookDir = interactive is respected by list' '
 
 	git hook list pre-commit >actual &&
 	# the hookdir annotation is translated
-	test_i18ncmp expected actual
+	test_i18ncmp expected actual &&
+
+	test_write_lines n | git hook run pre-commit 2>actual &&
+	! grep "Legacy Hook" actual &&
+
+	test_write_lines y | git hook run pre-commit 2>actual &&
+	grep "Legacy Hook" actual
+'
+
+test_expect_success 'inline hook definitions execute oneliners' '
+	test_config hook.pre-commit.command "echo \"Hello World\"" &&
+
+	echo "Hello World" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'inline hook definitions resolve paths' '
+	write_script sample-hook.sh <<-EOF &&
+	echo \"Sample Hook\"
+	EOF
+
+	test_when_finished "rm sample-hook.sh" &&
+
+	test_config hook.pre-commit.command "\"$(pwd)/sample-hook.sh\"" &&
+
+	echo \"Sample Hook\" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'hookdir hook included in git hook run' '
+	setup_hookdir &&
+
+	echo \"Legacy Hook\" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'out-of-repo runs excluded' '
+	setup_hooks &&
+
+	nongit test_must_fail git hook run pre-commit
 '
 
 test_expect_success 'hook.runHookDir is tolerant to unknown values' '
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 09/37] hook: introduce hook_exists()
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (7 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 08/37] hook: add 'run' subcommand Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 10/37] hook: support passing stdin to hooks Emily Shaffer
                   ` (31 subsequent siblings)
  40 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Add a helper to easily determine whether any hooks exist for a given
hook event.

Many callers want to check whether some state could be modified by a
hook; that check should include the config-based hooks as well. Optimize
by checking the config directly. Since commands which execute hooks
might want to take args to replace 'hook.runHookDir', let
'hook_exists()' take a hookdir_opt to override that config.

In some cases, external callers today use find_hook() to discover the
location of a hook and then run it manually with run-command.h (that is,
not with run_hook_le()). Later, those cases will call hook.h:run_hook()
directly instead.

Once the entire codebase is using hook_exists() instead of find_hook(),
find_hook() can be safely rolled into hook_exists() and removed from
run-command.h.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 hook.c | 19 +++++++++++++++++++
 hook.h | 10 ++++++++++
 2 files changed, 29 insertions(+)

diff --git a/hook.c b/hook.c
index d166d17fb0..118931f273 100644
--- a/hook.c
+++ b/hook.c
@@ -243,6 +243,25 @@ void run_hooks_opt_init(struct run_hooks_opt *o)
 	o->run_hookdir = configured_hookdir_opt();
 }
 
+int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir)
+{
+	const char *value = NULL; /* throwaway */
+	struct strbuf hook_key = STRBUF_INIT;
+	int could_run_hookdir;
+
+	if (should_run_hookdir == HOOKDIR_USE_CONFIG)
+		should_run_hookdir = configured_hookdir_opt();
+
+	could_run_hookdir = (should_run_hookdir == HOOKDIR_INTERACTIVE ||
+				should_run_hookdir == HOOKDIR_WARN ||
+				should_run_hookdir == HOOKDIR_YES)
+				&& !!find_hook(hookname);
+
+	strbuf_addf(&hook_key, "hook.%s.command", hookname);
+
+	return (!git_config_get_value(hook_key.buf, &value)) || could_run_hookdir;
+}
+
 void run_hooks_opt_clear(struct run_hooks_opt *o)
 {
 	strvec_clear(&o->env);
diff --git a/hook.h b/hook.h
index c24b2c9ecd..0df785add5 100644
--- a/hook.h
+++ b/hook.h
@@ -23,6 +23,7 @@ struct list_head* hook_list(const struct strbuf *hookname);
 
 enum hookdir_opt
 {
+	HOOKDIR_USE_CONFIG,
 	HOOKDIR_NO,
 	HOOKDIR_ERROR,
 	HOOKDIR_WARN,
@@ -56,6 +57,15 @@ struct run_hooks_opt
 void run_hooks_opt_init(struct run_hooks_opt *o);
 void run_hooks_opt_clear(struct run_hooks_opt *o);
 
+/*
+ * Returns 1 if any hooks are specified in the config or if a hook exists in the
+ * hookdir. Typically, invoke hook_exsts() like:
+ *   hook_exists(hookname, configured_hookdir_opt());
+ * Like with run_hooks, if you take a --run-hookdir flag, reflect that
+ * user-specified behavior here instead.
+ */
+int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir);
+
 /*
  * Runs all hooks associated to the 'hookname' event in order. Each hook will be
  * passed 'env' and 'args'.
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 10/37] hook: support passing stdin to hooks
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (8 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 09/37] hook: introduce hook_exists() Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-12  9:00   ` Ævar Arnfjörð Bjarmason
  2021-03-12 10:22   ` Junio C Hamano
  2021-03-11  2:10 ` [PATCH v8 11/37] run-command: allow stdin for run_processes_parallel Emily Shaffer
                   ` (30 subsequent siblings)
  40 siblings, 2 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Some hooks (such as post-rewrite) need to take input via stdin.
Previously, callers provided stdin to hooks by setting
run-command.h:child_process.in, which takes a FD. Callers would open the
file in question themselves before calling run-command(). However, since
we will now need to seek to the front of the file and read it again for
every hook which runs, hook.h:run_command() takes a path and handles FD
management itself. Since this file is opened for read only, it should
not prevent later parallel execution support.

On the frontend, this is supported by asking for a file path, rather
than by reading stdin. Reading directly from stdin would involve caching
the entire stdin (to memory or to disk) and reading it back from the
beginning to each hook. We'd want to support cases like insufficient
memory or storage for the file. While this may prove useful later, for
now the path of least resistance is to just ask the user to make this
interim file themselves.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-hook.txt    | 11 +++++++++--
 builtin/hook.c                |  5 ++++-
 hook.c                        |  8 +++++++-
 hook.h                        |  6 +++++-
 t/t1360-config-based-hooks.sh | 24 ++++++++++++++++++++++++
 5 files changed, 49 insertions(+), 5 deletions(-)

diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 8f96c347ea..96a857c682 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -9,7 +9,8 @@ SYNOPSIS
 --------
 [verse]
 'git hook' list <hook-name>
-'git hook' run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] <hook-name>
+'git hook' run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] [--to-stdin=<path>]
+	<hook-name>
 
 DESCRIPTION
 -----------
@@ -98,7 +99,7 @@ in the order they should be run, and print the config scope where the relevant
 `hook.<hook-name>.command` was specified, not the `hookcmd` (if applicable).
 This output is human-readable and the format is subject to change over time.
 
-run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] `<hook-name>`::
+run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] [--to-stdin=<path>] `<hook-name>`::
 
 Runs hooks configured for `<hook-name>`, in the same order displayed by `git
 hook list`. Hooks configured this way may be run prepended with `sh -c`, so
@@ -124,6 +125,12 @@ Specify arguments to pass to every hook that is run.
 +
 Specify environment variables to set for every hook that is run.
 
+--to-stdin::
+	Only valid for `run`.
++
+Specify a file which will be streamed into stdin for every hook that is run.
+Each hook will receive the entire file from beginning to EOF.
+
 CONFIGURATION
 -------------
 include::config/hook.txt[]
diff --git a/builtin/hook.c b/builtin/hook.c
index e823a96238..38a4555e05 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -8,7 +8,8 @@
 
 static const char * const builtin_hook_usage[] = {
 	N_("git hook list <hookname>"),
-	N_("git hook run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] <hookname>"),
+	N_("git hook run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...]"
+	   "[--to-stdin=<path>] <hookname>"),
 	NULL
 };
 
@@ -101,6 +102,8 @@ static int run(int argc, const char **argv, const char *prefix)
 			   N_("environment variables for hook to use")),
 		OPT_STRVEC('a', "arg", &opt.args, N_("args"),
 			   N_("argument to pass to hook")),
+		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
+			   N_("file to read into hooks' stdin")),
 		OPT_END(),
 	};
 
diff --git a/hook.c b/hook.c
index 118931f273..f906e8c61c 100644
--- a/hook.c
+++ b/hook.c
@@ -240,6 +240,7 @@ void run_hooks_opt_init(struct run_hooks_opt *o)
 {
 	strvec_init(&o->env);
 	strvec_init(&o->args);
+	o->path_to_stdin = NULL;
 	o->run_hookdir = configured_hookdir_opt();
 }
 
@@ -274,7 +275,12 @@ static void prepare_hook_cp(struct hook *hook, struct run_hooks_opt *options,
 	if (!hook)
 		return;
 
-	cp->no_stdin = 1;
+	/* reopen the file for stdin; run_command closes it. */
+	if (options->path_to_stdin)
+		cp->in = xopen(options->path_to_stdin, O_RDONLY);
+	else
+		cp->no_stdin = 1;
+
 	cp->env = options->env.v;
 	cp->stdout_to_stderr = 1;
 	cp->trace2_hook_name = hook->command.buf;
diff --git a/hook.h b/hook.h
index 0df785add5..2314ec5962 100644
--- a/hook.h
+++ b/hook.h
@@ -52,6 +52,9 @@ struct run_hooks_opt
 	 * to be overridden if the user can override it at the command line.
 	 */
 	enum hookdir_opt run_hookdir;
+
+	/* Path to file which should be piped to stdin for each hook */
+	const char *path_to_stdin;
 };
 
 void run_hooks_opt_init(struct run_hooks_opt *o);
@@ -68,7 +71,8 @@ int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir);
 
 /*
  * Runs all hooks associated to the 'hookname' event in order. Each hook will be
- * passed 'env' and 'args'.
+ * passed 'env' and 'args'. The file at 'stdin_path' will be closed and reopened
+ * for each hook that runs.
  */
 int run_hooks(const char *hookname, struct run_hooks_opt *options);
 
diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh
index 1fca83d536..cace5a23c1 100755
--- a/t/t1360-config-based-hooks.sh
+++ b/t/t1360-config-based-hooks.sh
@@ -276,4 +276,28 @@ test_expect_success 'hook.runHookDir is tolerant to unknown values' '
 	test_i18ncmp expected actual
 '
 
+test_expect_success 'stdin to multiple hooks' '
+	git config --add hook.test.command "xargs -P1 -I% echo a%" &&
+	git config --add hook.test.command "xargs -P1 -I% echo b%" &&
+	test_when_finished "test_unconfig hook.test.command" &&
+
+	cat >input <<-EOF &&
+	1
+	2
+	3
+	EOF
+
+	cat >expected <<-EOF &&
+	a1
+	a2
+	a3
+	b1
+	b2
+	b3
+	EOF
+
+	git hook run --to-stdin=input test 2>actual &&
+	test_cmp expected actual
+'
+
 test_done
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 11/37] run-command: allow stdin for run_processes_parallel
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (9 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 10/37] hook: support passing stdin to hooks Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 12/37] hook: allow parallel hook execution Emily Shaffer
                   ` (29 subsequent siblings)
  40 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

While it makes sense not to inherit stdin from the parent process to
avoid deadlocking, it's not necessary to completely ban stdin to
children. An informed user should be able to configure stdin safely. By
setting `some_child.process.no_stdin=1` before calling `get_next_task()`
we provide a reasonable default behavior but enable users to set up
stdin streaming for themselves during the callback.

`some_child.process.stdout_to_stderr`, however, remains unmodifiable by
`get_next_task()` - the rest of the run_processes_parallel() API depends
on child output in stderr.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 run-command.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/run-command.c b/run-command.c
index 4e34623e2e..e6d7541b84 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1693,6 +1693,14 @@ static int pp_start_one(struct parallel_processes *pp)
 	if (i == pp->max_processes)
 		BUG("bookkeeping is hard");
 
+	/*
+	 * By default, do not inherit stdin from the parent process - otherwise,
+	 * all children would share stdin! Users may overwrite this to provide
+	 * something to the child's stdin by having their 'get_next_task'
+	 * callback assign 0 to .no_stdin and an appropriate integer to .in.
+	 */
+	pp->children[i].process.no_stdin = 1;
+
 	code = pp->get_next_task(&pp->children[i].process,
 				 &pp->children[i].err,
 				 pp->data,
@@ -1704,7 +1712,6 @@ static int pp_start_one(struct parallel_processes *pp)
 	}
 	pp->children[i].process.err = -1;
 	pp->children[i].process.stdout_to_stderr = 1;
-	pp->children[i].process.no_stdin = 1;
 
 	if (start_command(&pp->children[i].process)) {
 		code = pp->start_failure(&pp->children[i].err,
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 12/37] hook: allow parallel hook execution
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (10 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 11/37] run-command: allow stdin for run_processes_parallel Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 13/37] hook: allow specifying working directory for hooks Emily Shaffer
                   ` (28 subsequent siblings)
  40 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

In many cases, there's no reason not to allow hooks to execute in
parallel. run_processes_parallel() is well-suited - it's a task queue
that runs its housekeeping in series, which means users don't
need to worry about thread safety on their callback data. True
multithreaded execution with the async_* functions isn't necessary here.
Synchronous hook execution can be achieved by only allowing 1 job to run
at a time.

Teach run_hooks() to use that function for simple hooks which don't
require stdin or capture of stderr.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

Notes:
    Per AEvar's request - parallel hook execution on day zero.
    
    In most ways run_processes_parallel() worked great for me - but it didn't
    have great support for hooks where we pipe to and from. I had to add this
    support later in the series.
    
    Since I modified an existing and in-use library I'd appreciate a keen look on
    these patches.
    
     - Emily

 Documentation/config/hook.txt |   5 ++
 Documentation/git-hook.txt    |  14 ++++-
 builtin/hook.c                |   6 +-
 hook.c                        | 108 +++++++++++++++++++++++++++++-----
 hook.h                        |  21 ++++++-
 5 files changed, 132 insertions(+), 22 deletions(-)

diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
index 8b12512e33..4f66bb35cf 100644
--- a/Documentation/config/hook.txt
+++ b/Documentation/config/hook.txt
@@ -20,3 +20,8 @@ hook.runHookDir::
 	Controls how hooks contained in your hookdir are executed. Can be any of
 	"yes", "warn", "interactive", or "no". Defaults to "yes". See
 	linkgit:git-hook[1] and linkgit:git-config[1] "core.hooksPath").
+
+hook.jobs::
+	Specifies how many hooks can be run simultaneously during parallelized
+	hook execution. If unspecified, defaults to the number of processors on
+	the current system.
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 96a857c682..81b8e94994 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -10,7 +10,7 @@ SYNOPSIS
 [verse]
 'git hook' list <hook-name>
 'git hook' run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] [--to-stdin=<path>]
-	<hook-name>
+	[(-j|--jobs) <n>] <hook-name>
 
 DESCRIPTION
 -----------
@@ -99,7 +99,7 @@ in the order they should be run, and print the config scope where the relevant
 `hook.<hook-name>.command` was specified, not the `hookcmd` (if applicable).
 This output is human-readable and the format is subject to change over time.
 
-run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] [--to-stdin=<path>] `<hook-name>`::
+run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] [--to-stdin=<path>] [(-j|--jobs)<n>] `<hook-name>`::
 
 Runs hooks configured for `<hook-name>`, in the same order displayed by `git
 hook list`. Hooks configured this way may be run prepended with `sh -c`, so
@@ -131,6 +131,16 @@ Specify environment variables to set for every hook that is run.
 Specify a file which will be streamed into stdin for every hook that is run.
 Each hook will receive the entire file from beginning to EOF.
 
+-j::
+--jobs::
+	Only valid for `run`.
++
+Specify how many hooks to run simultaneously. If this flag is not specified, use
+the value of the `hook.jobs` config. If the config is not specified, use the
+number of CPUs on the current system. Some hooks may be ineligible for
+parallelization: for example, 'commit-msg' intends hooks modify the commit
+message body and cannot be parallelized.
+
 CONFIGURATION
 -------------
 include::config/hook.txt[]
diff --git a/builtin/hook.c b/builtin/hook.c
index 38a4555e05..b4f4adb1de 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -9,7 +9,7 @@
 static const char * const builtin_hook_usage[] = {
 	N_("git hook list <hookname>"),
 	N_("git hook run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...]"
-	   "[--to-stdin=<path>] <hookname>"),
+	   "[--to-stdin=<path>] [(-j|--jobs) <count>] <hookname>"),
 	NULL
 };
 
@@ -104,10 +104,12 @@ static int run(int argc, const char **argv, const char *prefix)
 			   N_("argument to pass to hook")),
 		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
 			   N_("file to read into hooks' stdin")),
+		OPT_INTEGER('j', "jobs", &opt.jobs,
+			    N_("run up to <n> hooks simultaneously")),
 		OPT_END(),
 	};
 
-	run_hooks_opt_init(&opt);
+	run_hooks_opt_init_async(&opt);
 
 	argc = parse_options(argc, argv, prefix, run_options,
 			     builtin_hook_usage, 0);
diff --git a/hook.c b/hook.c
index f906e8c61c..fe8860860b 100644
--- a/hook.c
+++ b/hook.c
@@ -144,6 +144,14 @@ enum hookdir_opt configured_hookdir_opt(void)
 	return HOOKDIR_UNKNOWN;
 }
 
+int configured_hook_jobs(void)
+{
+	int n = online_cpus();
+	git_config_get_int("hook.jobs", &n);
+
+	return n;
+}
+
 static int should_include_hookdir(const char *path, enum hookdir_opt cfg)
 {
 	struct strbuf prompt = STRBUF_INIT;
@@ -236,12 +244,19 @@ struct list_head* hook_list(const struct strbuf* hookname)
 	return hook_head;
 }
 
-void run_hooks_opt_init(struct run_hooks_opt *o)
+void run_hooks_opt_init_sync(struct run_hooks_opt *o)
 {
 	strvec_init(&o->env);
 	strvec_init(&o->args);
 	o->path_to_stdin = NULL;
 	o->run_hookdir = configured_hookdir_opt();
+	o->jobs = 1;
+}
+
+void run_hooks_opt_init_async(struct run_hooks_opt *o)
+{
+	run_hooks_opt_init_sync(o);
+	o->jobs = configured_hook_jobs();
 }
 
 int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir)
@@ -269,19 +284,26 @@ void run_hooks_opt_clear(struct run_hooks_opt *o)
 	strvec_clear(&o->args);
 }
 
-static void prepare_hook_cp(struct hook *hook, struct run_hooks_opt *options,
-			    struct child_process *cp)
+static int pick_next_hook(struct child_process *cp,
+			  struct strbuf *out,
+			  void *pp_cb,
+			  void **pp_task_cb)
 {
+	struct hook_cb_data *hook_cb = pp_cb;
+	struct hook *hook = hook_cb->run_me;
+
 	if (!hook)
-		return;
+		return 0;
 
 	/* reopen the file for stdin; run_command closes it. */
-	if (options->path_to_stdin)
-		cp->in = xopen(options->path_to_stdin, O_RDONLY);
-	else
+	if (hook_cb->options->path_to_stdin) {
+		cp->no_stdin = 0;
+		cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY);
+	} else {
 		cp->no_stdin = 1;
+	}
 
-	cp->env = options->env.v;
+	cp->env = hook_cb->options->env.v;
 	cp->stdout_to_stderr = 1;
 	cp->trace2_hook_name = hook->command.buf;
 
@@ -298,14 +320,59 @@ static void prepare_hook_cp(struct hook *hook, struct run_hooks_opt *options,
 	 * add passed-in argv, without expanding - let the user get back
 	 * exactly what they put in
 	 */
-	strvec_pushv(&cp->args, options->args.v);
+	strvec_pushv(&cp->args, hook_cb->options->args.v);
+
+	/* Provide context for errors if necessary */
+	*pp_task_cb = hook;
+
+	/* Get the next entry ready */
+	if (hook_cb->run_me->list.next == hook_cb->head)
+		hook_cb->run_me = NULL;
+	else
+		hook_cb->run_me = list_entry(hook_cb->run_me->list.next,
+					     struct hook, list);
+
+	return 1;
+}
+
+static int notify_start_failure(struct strbuf *out,
+				void *pp_cb,
+				void *pp_task_cp)
+{
+	struct hook_cb_data *hook_cb = pp_cb;
+	struct hook *attempted = pp_task_cp;
+
+	/* |= rc in cb */
+	hook_cb->rc |= 1;
+
+	strbuf_addf(out, _("Couldn't start '%s', configured in '%s'\n"),
+		    attempted->command.buf,
+		    attempted->from_hookdir ? "hookdir"
+			: config_scope_name(attempted->origin));
+
+	/* NEEDSWORK: if halt_on_error is desired, do it here. */
+	return 0;
+}
+
+static int notify_hook_finished(int result,
+				struct strbuf *out,
+				void *pp_cb,
+				void *pp_task_cb)
+{
+	struct hook_cb_data *hook_cb = pp_cb;
+
+	/* |= rc in cb */
+	hook_cb->rc |= result;
+
+	/* NEEDSWORK: if halt_on_error is desired, do it here. */
+	return 0;
 }
 
 int run_hooks(const char *hookname, struct run_hooks_opt *options)
 {
 	struct strbuf hookname_str = STRBUF_INIT;
 	struct list_head *to_run, *pos = NULL, *tmp = NULL;
-	int rc = 0;
+	struct hook_cb_data cb_data = { 0, NULL, NULL, options };
 
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
@@ -315,17 +382,26 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options)
 	to_run = hook_list(&hookname_str);
 
 	list_for_each_safe(pos, tmp, to_run) {
-		struct child_process hook_proc = CHILD_PROCESS_INIT;
 		struct hook *hook = list_entry(pos, struct hook, list);
 
 		if (hook->from_hookdir &&
 		    !should_include_hookdir(hook->command.buf, options->run_hookdir))
-			continue;
+			    list_del(pos);
+	}
+
+	if (list_empty(to_run))
+		return 0;
 
-		prepare_hook_cp(hook, options, &hook_proc);
+	cb_data.head = to_run;
+	cb_data.run_me = list_entry(to_run->next, struct hook, list);
 
-		rc |= run_command(&hook_proc);
-	}
+	run_processes_parallel_tr2(options->jobs,
+				   pick_next_hook,
+				   notify_start_failure,
+				   notify_hook_finished,
+				   &cb_data,
+				   "hook",
+				   hookname);
 
-	return rc;
+	return cb_data.rc;
 }
diff --git a/hook.h b/hook.h
index 2314ec5962..2593f932c0 100644
--- a/hook.h
+++ b/hook.h
@@ -38,6 +38,9 @@ enum hookdir_opt
  */
 enum hookdir_opt configured_hookdir_opt(void);
 
+/* Provides the number of threads to use for parallel hook execution. */
+int configured_hook_jobs(void);
+
 struct run_hooks_opt
 {
 	/* Environment vars to be set for each hook */
@@ -48,16 +51,30 @@ struct run_hooks_opt
 
 	/*
 	 * How should the hookdir be handled?
-	 * Leave the RUN_HOOKS_OPT_INIT default in most cases; this only needs
+	 * Leave the run_hooks_opt_init_*() default in most cases; this only needs
 	 * to be overridden if the user can override it at the command line.
 	 */
 	enum hookdir_opt run_hookdir;
 
 	/* Path to file which should be piped to stdin for each hook */
 	const char *path_to_stdin;
+
+	/* Number of threads to parallelize across */
+	int jobs;
+};
+
+/*
+ * Callback provided to feed_pipe_fn and consume_sideband_fn.
+ */
+struct hook_cb_data {
+	int rc;
+	struct list_head *head;
+	struct hook *run_me;
+	struct run_hooks_opt *options;
 };
 
-void run_hooks_opt_init(struct run_hooks_opt *o);
+void run_hooks_opt_init_sync(struct run_hooks_opt *o);
+void run_hooks_opt_init_async(struct run_hooks_opt *o);
 void run_hooks_opt_clear(struct run_hooks_opt *o);
 
 /*
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 13/37] hook: allow specifying working directory for hooks
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (11 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 12/37] hook: allow parallel hook execution Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 14/37] run-command: add stdin callback for parallelization Emily Shaffer
                   ` (27 subsequent siblings)
  40 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Hooks like "post-checkout" require that hooks have a different working
directory than the initial process. Pipe that directly through to struct
child_process.

Because we can just run 'git -C <some-dir> hook run ...' it shouldn't be
necessary to pipe this option through the frontend. In fact, this
reduces the possibility of users running hooks which affect some part of
the filesystem outside of the repo in question.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

Notes:
    Needed later for "post-checkout" conversion.

 hook.c | 2 ++
 hook.h | 3 +++
 2 files changed, 5 insertions(+)

diff --git a/hook.c b/hook.c
index fe8860860b..67ad3aa747 100644
--- a/hook.c
+++ b/hook.c
@@ -251,6 +251,7 @@ void run_hooks_opt_init_sync(struct run_hooks_opt *o)
 	o->path_to_stdin = NULL;
 	o->run_hookdir = configured_hookdir_opt();
 	o->jobs = 1;
+	o->dir = NULL;
 }
 
 void run_hooks_opt_init_async(struct run_hooks_opt *o)
@@ -306,6 +307,7 @@ static int pick_next_hook(struct child_process *cp,
 	cp->env = hook_cb->options->env.v;
 	cp->stdout_to_stderr = 1;
 	cp->trace2_hook_name = hook->command.buf;
+	cp->dir = hook_cb->options->dir;
 
 	/*
 	 * Commands from the config could be oneliners, but we know
diff --git a/hook.h b/hook.h
index 2593f932c0..fcd8e99e39 100644
--- a/hook.h
+++ b/hook.h
@@ -61,6 +61,9 @@ struct run_hooks_opt
 
 	/* Number of threads to parallelize across */
 	int jobs;
+
+	/* Path to initial working directory for subprocess */
+	const char *dir;
 };
 
 /*
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 14/37] run-command: add stdin callback for parallelization
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (12 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 13/37] hook: allow specifying working directory for hooks Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 15/37] hook: provide stdin by string_list or callback Emily Shaffer
                   ` (26 subsequent siblings)
  40 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

If a user of the run_processes_parallel() API wants to pipe a large
amount of information to stdin of each parallel command, that
information could exceed the buffer of the pipe allocated for that
process's stdin.  Generally this is solved by repeatedly writing to
child_process.in between calls to start_command() and finish_command();
run_processes_parallel() did not provide users an opportunity to access
child_process at that time.

Because the data might be extremely large (for example, a list of all
refs received during a push from a client) simply taking a string_list
or strbuf is not as scalable as using a callback; the rest of the
run_processes_parallel() API also uses callbacks, so making this feature
match the rest of the API reduces mental load on the user.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 builtin/fetch.c             |  1 +
 builtin/submodule--helper.c |  2 +-
 hook.c                      |  1 +
 run-command.c               | 54 +++++++++++++++++++++++++++++++++++--
 run-command.h               | 17 +++++++++++-
 submodule.c                 |  1 +
 t/helper/test-run-command.c | 31 ++++++++++++++++++---
 t/t0061-run-command.sh      | 30 +++++++++++++++++++++
 8 files changed, 129 insertions(+), 8 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index 0b90de87c7..d8e798dc69 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1757,6 +1757,7 @@ static int fetch_multiple(struct string_list *list, int max_children)
 		result = run_processes_parallel_tr2(max_children,
 						    &fetch_next_remote,
 						    &fetch_failed_to_start,
+						    NULL,
 						    &fetch_finished,
 						    &state,
 						    "fetch", "parallel/fetch");
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 9d505a6329..14f6e4ee8c 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2294,7 +2294,7 @@ static int update_submodules(struct submodule_update_clone *suc)
 	int i;
 
 	run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task,
-				   update_clone_start_failure,
+				   update_clone_start_failure, NULL,
 				   update_clone_task_finished, suc, "submodule",
 				   "parallel/update");
 
diff --git a/hook.c b/hook.c
index 67ad3aa747..9088b520f3 100644
--- a/hook.c
+++ b/hook.c
@@ -400,6 +400,7 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options)
 	run_processes_parallel_tr2(options->jobs,
 				   pick_next_hook,
 				   notify_start_failure,
+				   NULL,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
diff --git a/run-command.c b/run-command.c
index e6d7541b84..e7eeb6c49b 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1558,6 +1558,7 @@ struct parallel_processes {
 
 	get_next_task_fn get_next_task;
 	start_failure_fn start_failure;
+	feed_pipe_fn feed_pipe;
 	task_finished_fn task_finished;
 
 	struct {
@@ -1585,6 +1586,13 @@ static int default_start_failure(struct strbuf *out,
 	return 0;
 }
 
+static int default_feed_pipe(struct strbuf *pipe,
+			     void *pp_cb,
+			     void *pp_task_cb)
+{
+	return 1;
+}
+
 static int default_task_finished(int result,
 				 struct strbuf *out,
 				 void *pp_cb,
@@ -1615,6 +1623,7 @@ static void pp_init(struct parallel_processes *pp,
 		    int n,
 		    get_next_task_fn get_next_task,
 		    start_failure_fn start_failure,
+		    feed_pipe_fn feed_pipe,
 		    task_finished_fn task_finished,
 		    void *data)
 {
@@ -1633,6 +1642,7 @@ static void pp_init(struct parallel_processes *pp,
 	pp->get_next_task = get_next_task;
 
 	pp->start_failure = start_failure ? start_failure : default_start_failure;
+	pp->feed_pipe = feed_pipe ? feed_pipe : default_feed_pipe;
 	pp->task_finished = task_finished ? task_finished : default_task_finished;
 
 	pp->nr_processes = 0;
@@ -1730,6 +1740,37 @@ static int pp_start_one(struct parallel_processes *pp)
 	return 0;
 }
 
+static void pp_buffer_stdin(struct parallel_processes *pp)
+{
+	int i;
+	struct strbuf sb = STRBUF_INIT;
+
+	/* Buffer stdin for each pipe. */
+	for (i = 0; i < pp->max_processes; i++) {
+		if (pp->children[i].state == GIT_CP_WORKING &&
+		    pp->children[i].process.in > 0) {
+			int done;
+			strbuf_reset(&sb);
+			done = pp->feed_pipe(&sb, pp->data,
+					      pp->children[i].data);
+			if (sb.len) {
+				if (write_in_full(pp->children[i].process.in,
+					      sb.buf, sb.len) < 0) {
+					if (errno != EPIPE)
+						die_errno("write");
+					done = 1;
+				}
+			}
+			if (done) {
+				close(pp->children[i].process.in);
+				pp->children[i].process.in = 0;
+			}
+		}
+	}
+
+	strbuf_release(&sb);
+}
+
 static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout)
 {
 	int i;
@@ -1794,6 +1835,7 @@ static int pp_collect_finished(struct parallel_processes *pp)
 		pp->nr_processes--;
 		pp->children[i].state = GIT_CP_FREE;
 		pp->pfd[i].fd = -1;
+		pp->children[i].process.in = 0;
 		child_process_init(&pp->children[i].process);
 
 		if (i != pp->output_owner) {
@@ -1827,6 +1869,7 @@ static int pp_collect_finished(struct parallel_processes *pp)
 int run_processes_parallel(int n,
 			   get_next_task_fn get_next_task,
 			   start_failure_fn start_failure,
+			   feed_pipe_fn feed_pipe,
 			   task_finished_fn task_finished,
 			   void *pp_cb)
 {
@@ -1835,7 +1878,9 @@ int run_processes_parallel(int n,
 	int spawn_cap = 4;
 	struct parallel_processes pp;
 
-	pp_init(&pp, n, get_next_task, start_failure, task_finished, pp_cb);
+	sigchain_push(SIGPIPE, SIG_IGN);
+
+	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, task_finished, pp_cb);
 	while (1) {
 		for (i = 0;
 		    i < spawn_cap && !pp.shutdown &&
@@ -1852,6 +1897,7 @@ int run_processes_parallel(int n,
 		}
 		if (!pp.nr_processes)
 			break;
+		pp_buffer_stdin(&pp);
 		pp_buffer_stderr(&pp, output_timeout);
 		pp_output(&pp);
 		code = pp_collect_finished(&pp);
@@ -1863,11 +1909,15 @@ int run_processes_parallel(int n,
 	}
 
 	pp_cleanup(&pp);
+
+	sigchain_pop(SIGPIPE);
+
 	return 0;
 }
 
 int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 			       start_failure_fn start_failure,
+			       feed_pipe_fn feed_pipe,
 			       task_finished_fn task_finished, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label)
 {
@@ -1877,7 +1927,7 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 				   ((n < 1) ? online_cpus() : n));
 
 	result = run_processes_parallel(n, get_next_task, start_failure,
-					task_finished, pp_cb);
+					feed_pipe, task_finished, pp_cb);
 
 	trace2_region_leave(tr2_category, tr2_label, NULL);
 
diff --git a/run-command.h b/run-command.h
index d08414a92e..1e3cf0999f 100644
--- a/run-command.h
+++ b/run-command.h
@@ -443,6 +443,20 @@ typedef int (*start_failure_fn)(struct strbuf *out,
 				void *pp_cb,
 				void *pp_task_cb);
 
+/**
+ * This callback is called repeatedly on every child process who requests
+ * start_command() to create a pipe by setting child_process.in < 0.
+ *
+ * pp_cb is the callback cookie as passed into run_processes_parallel, and
+ * pp_task_cb is the callback cookie as passed into get_next_task_fn.
+ * The contents of 'send' will be read into the pipe and passed to the pipe.
+ *
+ * Return nonzero to close the pipe.
+ */
+typedef int (*feed_pipe_fn)(struct strbuf *pipe,
+			    void *pp_cb,
+			    void *pp_task_cb);
+
 /**
  * This callback is called on every child process that finished processing.
  *
@@ -477,10 +491,11 @@ typedef int (*task_finished_fn)(int result,
 int run_processes_parallel(int n,
 			   get_next_task_fn,
 			   start_failure_fn,
+			   feed_pipe_fn,
 			   task_finished_fn,
 			   void *pp_cb);
 int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn,
-			       task_finished_fn, void *pp_cb,
+			       feed_pipe_fn, task_finished_fn, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label);
 
 #endif
diff --git a/submodule.c b/submodule.c
index 9767ba9893..dc4a6a60f4 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1644,6 +1644,7 @@ int fetch_populated_submodules(struct repository *r,
 	run_processes_parallel_tr2(max_parallel_jobs,
 				   get_next_submodule,
 				   fetch_start_failure,
+				   NULL,
 				   fetch_finish,
 				   &spf,
 				   "submodule", "parallel/fetch");
diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c
index 7ae03dc712..9348184d30 100644
--- a/t/helper/test-run-command.c
+++ b/t/helper/test-run-command.c
@@ -32,8 +32,13 @@ static int parallel_next(struct child_process *cp,
 		return 0;
 
 	strvec_pushv(&cp->args, d->argv);
+	cp->in = d->in;
+	cp->no_stdin = d->no_stdin;
 	strbuf_addstr(err, "preloaded output of a child\n");
 	number_callbacks++;
+
+	*task_cb = xmalloc(sizeof(int));
+	*(int*)(*task_cb) = 2;
 	return 1;
 }
 
@@ -55,6 +60,17 @@ static int task_finished(int result,
 	return 1;
 }
 
+static int test_stdin(struct strbuf *pipe, void *cb, void *task_cb)
+{
+	int *lines_remaining = task_cb;
+
+	if (*lines_remaining)
+		strbuf_addf(pipe, "sample stdin %d\n", --(*lines_remaining));
+
+	return !(*lines_remaining);
+}
+
+
 struct testsuite {
 	struct string_list tests, failed;
 	int next;
@@ -185,7 +201,7 @@ static int testsuite(int argc, const char **argv)
 		suite.tests.nr, max_jobs);
 
 	ret = run_processes_parallel(max_jobs, next_test, test_failed,
-				     test_finished, &suite);
+				     test_stdin, test_finished, &suite);
 
 	if (suite.failed.nr > 0) {
 		ret = 1;
@@ -413,15 +429,22 @@ int cmd__run_command(int argc, const char **argv)
 
 	if (!strcmp(argv[1], "run-command-parallel"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, &proc));
+					    NULL, NULL, NULL, &proc));
 
 	if (!strcmp(argv[1], "run-command-abort"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, task_finished, &proc));
+					    NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-no-jobs"))
 		exit(run_processes_parallel(jobs, no_job,
-					    NULL, task_finished, &proc));
+					    NULL, NULL, task_finished, &proc));
+
+	if (!strcmp(argv[1], "run-command-stdin")) {
+		proc.in = -1;
+		proc.no_stdin = 0;
+		exit (run_processes_parallel(jobs, parallel_next, NULL,
+					     test_stdin, NULL, &proc));
+	}
 
 	fprintf(stderr, "check usage\n");
 	return 1;
diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh
index 7d599675e3..87759482ad 100755
--- a/t/t0061-run-command.sh
+++ b/t/t0061-run-command.sh
@@ -143,6 +143,36 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai
 	test_cmp expect actual
 '
 
+cat >expect <<-EOF
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+EOF
+
+test_expect_success 'run_command listens to stdin' '
+	write_script stdin-script <<-\EOF &&
+	echo "listening for stdin:"
+	while read line; do
+		echo "$line"
+	done
+	EOF
+	test-tool run-command run-command-stdin 2 ./stdin-script 2>actual &&
+	test_cmp expect actual
+'
+
 cat >expect <<-EOF
 preloaded output of a child
 asking for a quick stop
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 15/37] hook: provide stdin by string_list or callback
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (13 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 14/37] run-command: add stdin callback for parallelization Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 16/37] run-command: allow capturing of collated output Emily Shaffer
                   ` (25 subsequent siblings)
  40 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

In cases where a hook requires only a small amount of information via
stdin, it should be simple for users to provide a string_list alone. But
in more complicated cases where the stdin is too large to hold in
memory, let's provide a callback the users can populate line after line
with instead.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 hook.c | 35 ++++++++++++++++++++++++++++++++++-
 hook.h | 28 ++++++++++++++++++++++++++++
 2 files changed, 62 insertions(+), 1 deletion(-)

diff --git a/hook.c b/hook.c
index 9088b520f3..a509d2d80e 100644
--- a/hook.c
+++ b/hook.c
@@ -9,6 +9,7 @@ void free_hook(struct hook *ptr)
 {
 	if (ptr) {
 		strbuf_release(&ptr->command);
+		free(ptr->feed_pipe_cb_data);
 		free(ptr);
 	}
 }
@@ -39,6 +40,7 @@ static void append_or_move_hook(struct list_head *head, const char *command)
 		strbuf_init(&to_add->command, 0);
 		strbuf_addstr(&to_add->command, command);
 		to_add->from_hookdir = 0;
+		to_add->feed_pipe_cb_data = NULL;
 	}
 
 	/* re-set the scope so we show where an override was specified */
@@ -252,6 +254,8 @@ void run_hooks_opt_init_sync(struct run_hooks_opt *o)
 	o->run_hookdir = configured_hookdir_opt();
 	o->jobs = 1;
 	o->dir = NULL;
+	o->feed_pipe = NULL;
+	o->feed_pipe_ctx = NULL;
 }
 
 void run_hooks_opt_init_async(struct run_hooks_opt *o)
@@ -285,6 +289,28 @@ void run_hooks_opt_clear(struct run_hooks_opt *o)
 	strvec_clear(&o->args);
 }
 
+int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb)
+{
+	int *item_idx;
+	struct hook *ctx = pp_task_cb;
+	struct string_list *to_pipe = ((struct hook_cb_data*)pp_cb)->options->feed_pipe_ctx;
+
+	/* Bootstrap the state manager if necessary. */
+	if (!ctx->feed_pipe_cb_data) {
+		ctx->feed_pipe_cb_data = xmalloc(sizeof(unsigned int));
+		*(int*)ctx->feed_pipe_cb_data = 0;
+	}
+
+	item_idx = ctx->feed_pipe_cb_data;
+
+	if (*item_idx < to_pipe->nr) {
+		strbuf_addf(pipe, "%s\n", to_pipe->items[*item_idx].string);
+		(*item_idx)++;
+		return 0;
+	}
+	return 1;
+}
+
 static int pick_next_hook(struct child_process *cp,
 			  struct strbuf *out,
 			  void *pp_cb,
@@ -300,6 +326,10 @@ static int pick_next_hook(struct child_process *cp,
 	if (hook_cb->options->path_to_stdin) {
 		cp->no_stdin = 0;
 		cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY);
+	} else if (hook_cb->options->feed_pipe) {
+		/* ask for start_command() to make a pipe for us */
+		cp->in = -1;
+		cp->no_stdin = 0;
 	} else {
 		cp->no_stdin = 1;
 	}
@@ -379,6 +409,9 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options)
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
 
+	if (options->path_to_stdin && options->feed_pipe)
+		BUG("choose only one method to populate stdin");
+
 	strbuf_addstr(&hookname_str, hookname);
 
 	to_run = hook_list(&hookname_str);
@@ -400,7 +433,7 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options)
 	run_processes_parallel_tr2(options->jobs,
 				   pick_next_hook,
 				   notify_start_failure,
-				   NULL,
+				   options->feed_pipe,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
diff --git a/hook.h b/hook.h
index fcd8e99e39..ecf0228a46 100644
--- a/hook.h
+++ b/hook.h
@@ -2,6 +2,7 @@
 #include "list.h"
 #include "strbuf.h"
 #include "strvec.h"
+#include "run-command.h"
 
 struct hook {
 	struct list_head list;
@@ -13,6 +14,12 @@ struct hook {
 	/* The literal command to run. */
 	struct strbuf command;
 	unsigned from_hookdir : 1;
+
+	/*
+	 * Use this to keep state for your feed_pipe_fn if you are using
+	 * run_hooks_opt.feed_pipe. Otherwise, do not touch it.
+	 */
+	void *feed_pipe_cb_data;
 };
 
 /*
@@ -58,14 +65,35 @@ struct run_hooks_opt
 
 	/* Path to file which should be piped to stdin for each hook */
 	const char *path_to_stdin;
+	/*
+	 * Callback and state pointer to ask for more content to pipe to stdin.
+	 * Will be called repeatedly, for each hook. See
+	 * hook.c:pipe_from_stdin() for an example. Keep per-hook state in
+	 * hook.feed_pipe_cb_data (per process). Keep initialization context in
+	 * feed_pipe_ctx (shared by all processes).
+	 *
+	 * See 'pipe_from_string_list()' for info about how to specify a
+	 * string_list as the stdin input instead of writing your own handler.
+	 */
+	feed_pipe_fn feed_pipe;
+	void *feed_pipe_ctx;
 
 	/* Number of threads to parallelize across */
 	int jobs;
 
 	/* Path to initial working directory for subprocess */
 	const char *dir;
+
 };
 
+/*
+ * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
+ * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'.
+ * This will pipe each string in the list to stdin, separated by newlines.  (Do
+ * not inject your own newlines.)
+ */
+int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb);
+
 /*
  * Callback provided to feed_pipe_fn and consume_sideband_fn.
  */
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 16/37] run-command: allow capturing of collated output
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (14 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 15/37] hook: provide stdin by string_list or callback Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 17/37] hooks: allow callers to capture output Emily Shaffer
                   ` (24 subsequent siblings)
  40 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Some callers, for example server-side hooks which wish to relay hook
output to clients across a transport, want to capture what would
normally print to stderr and do something else with it. Allow that via a
callback.

By calling the callback regardless of whether there's output available,
we allow clients to send e.g. a keepalive if necessary.

Because we expose a strbuf, not a fd or FILE*, there's no need to create
a temporary pipe or similar - we can just skip the print to stderr and
instead hand it to the caller.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

 builtin/fetch.c             |  2 +-
 builtin/submodule--helper.c |  2 +-
 hook.c                      |  1 +
 run-command.c               | 33 +++++++++++++++++++++++++--------
 run-command.h               | 18 +++++++++++++++++-
 submodule.c                 |  2 +-
 t/helper/test-run-command.c | 25 ++++++++++++++++++++-----
 t/t0061-run-command.sh      |  7 +++++++
 8 files changed, 73 insertions(+), 17 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index d8e798dc69..b6d45f8359 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1757,7 +1757,7 @@ static int fetch_multiple(struct string_list *list, int max_children)
 		result = run_processes_parallel_tr2(max_children,
 						    &fetch_next_remote,
 						    &fetch_failed_to_start,
-						    NULL,
+						    NULL, NULL,
 						    &fetch_finished,
 						    &state,
 						    "fetch", "parallel/fetch");
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 14f6e4ee8c..136e09a016 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2294,7 +2294,7 @@ static int update_submodules(struct submodule_update_clone *suc)
 	int i;
 
 	run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task,
-				   update_clone_start_failure, NULL,
+				   update_clone_start_failure, NULL, NULL,
 				   update_clone_task_finished, suc, "submodule",
 				   "parallel/update");
 
diff --git a/hook.c b/hook.c
index a509d2d80e..e16b082cbd 100644
--- a/hook.c
+++ b/hook.c
@@ -434,6 +434,7 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options)
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
+				   NULL,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
diff --git a/run-command.c b/run-command.c
index e7eeb6c49b..36a4edbacf 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1559,6 +1559,7 @@ struct parallel_processes {
 	get_next_task_fn get_next_task;
 	start_failure_fn start_failure;
 	feed_pipe_fn feed_pipe;
+	consume_sideband_fn consume_sideband;
 	task_finished_fn task_finished;
 
 	struct {
@@ -1624,6 +1625,7 @@ static void pp_init(struct parallel_processes *pp,
 		    get_next_task_fn get_next_task,
 		    start_failure_fn start_failure,
 		    feed_pipe_fn feed_pipe,
+		    consume_sideband_fn consume_sideband,
 		    task_finished_fn task_finished,
 		    void *data)
 {
@@ -1644,6 +1646,7 @@ static void pp_init(struct parallel_processes *pp,
 	pp->start_failure = start_failure ? start_failure : default_start_failure;
 	pp->feed_pipe = feed_pipe ? feed_pipe : default_feed_pipe;
 	pp->task_finished = task_finished ? task_finished : default_task_finished;
+	pp->consume_sideband = consume_sideband;
 
 	pp->nr_processes = 0;
 	pp->output_owner = 0;
@@ -1680,7 +1683,10 @@ static void pp_cleanup(struct parallel_processes *pp)
 	 * When get_next_task added messages to the buffer in its last
 	 * iteration, the buffered output is non empty.
 	 */
-	strbuf_write(&pp->buffered_output, stderr);
+	if (pp->consume_sideband)
+		pp->consume_sideband(&pp->buffered_output, pp->data);
+	else
+		strbuf_write(&pp->buffered_output, stderr);
 	strbuf_release(&pp->buffered_output);
 
 	sigchain_pop_common();
@@ -1801,9 +1807,13 @@ static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout)
 static void pp_output(struct parallel_processes *pp)
 {
 	int i = pp->output_owner;
+
 	if (pp->children[i].state == GIT_CP_WORKING &&
 	    pp->children[i].err.len) {
-		strbuf_write(&pp->children[i].err, stderr);
+		if (pp->consume_sideband)
+			pp->consume_sideband(&pp->children[i].err, pp->data);
+		else
+			strbuf_write(&pp->children[i].err, stderr);
 		strbuf_reset(&pp->children[i].err);
 	}
 }
@@ -1842,11 +1852,15 @@ static int pp_collect_finished(struct parallel_processes *pp)
 			strbuf_addbuf(&pp->buffered_output, &pp->children[i].err);
 			strbuf_reset(&pp->children[i].err);
 		} else {
-			strbuf_write(&pp->children[i].err, stderr);
+			/* Output errors, then all other finished child processes */
+			if (pp->consume_sideband) {
+				pp->consume_sideband(&pp->children[i].err, pp->data);
+				pp->consume_sideband(&pp->buffered_output, pp->data);
+			} else {
+				strbuf_write(&pp->children[i].err, stderr);
+				strbuf_write(&pp->buffered_output, stderr);
+			}
 			strbuf_reset(&pp->children[i].err);
-
-			/* Output all other finished child processes */
-			strbuf_write(&pp->buffered_output, stderr);
 			strbuf_reset(&pp->buffered_output);
 
 			/*
@@ -1870,6 +1884,7 @@ int run_processes_parallel(int n,
 			   get_next_task_fn get_next_task,
 			   start_failure_fn start_failure,
 			   feed_pipe_fn feed_pipe,
+			   consume_sideband_fn consume_sideband,
 			   task_finished_fn task_finished,
 			   void *pp_cb)
 {
@@ -1880,7 +1895,7 @@ int run_processes_parallel(int n,
 
 	sigchain_push(SIGPIPE, SIG_IGN);
 
-	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, task_finished, pp_cb);
+	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, consume_sideband, task_finished, pp_cb);
 	while (1) {
 		for (i = 0;
 		    i < spawn_cap && !pp.shutdown &&
@@ -1918,6 +1933,7 @@ int run_processes_parallel(int n,
 int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 			       start_failure_fn start_failure,
 			       feed_pipe_fn feed_pipe,
+			       consume_sideband_fn consume_sideband,
 			       task_finished_fn task_finished, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label)
 {
@@ -1927,7 +1943,8 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 				   ((n < 1) ? online_cpus() : n));
 
 	result = run_processes_parallel(n, get_next_task, start_failure,
-					feed_pipe, task_finished, pp_cb);
+					feed_pipe, consume_sideband,
+					task_finished, pp_cb);
 
 	trace2_region_leave(tr2_category, tr2_label, NULL);
 
diff --git a/run-command.h b/run-command.h
index 1e3cf0999f..ebc4a95a94 100644
--- a/run-command.h
+++ b/run-command.h
@@ -457,6 +457,20 @@ typedef int (*feed_pipe_fn)(struct strbuf *pipe,
 			    void *pp_cb,
 			    void *pp_task_cb);
 
+/**
+ * If this callback is provided, instead of collating process output to stderr,
+ * they will be collated into a new pipe. consume_sideband_fn will be called
+ * repeatedly. When output is available on that pipe, it will be contained in
+ * 'output'. But it will be called with an empty 'output' too, to allow for
+ * keepalives or similar operations if necessary.
+ *
+ * pp_cb is the callback cookie as passed into run_processes_parallel.
+ *
+ * Since this callback is provided with the collated output, no task cookie is
+ * provided.
+ */
+typedef void (*consume_sideband_fn)(struct strbuf *output, void *pp_cb);
+
 /**
  * This callback is called on every child process that finished processing.
  *
@@ -492,10 +506,12 @@ int run_processes_parallel(int n,
 			   get_next_task_fn,
 			   start_failure_fn,
 			   feed_pipe_fn,
+			   consume_sideband_fn,
 			   task_finished_fn,
 			   void *pp_cb);
 int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn,
-			       feed_pipe_fn, task_finished_fn, void *pp_cb,
+			       feed_pipe_fn, consume_sideband_fn,
+			       task_finished_fn, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label);
 
 #endif
diff --git a/submodule.c b/submodule.c
index dc4a6a60f4..4926642451 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1644,7 +1644,7 @@ int fetch_populated_submodules(struct repository *r,
 	run_processes_parallel_tr2(max_parallel_jobs,
 				   get_next_submodule,
 				   fetch_start_failure,
-				   NULL,
+				   NULL, NULL,
 				   fetch_finish,
 				   &spf,
 				   "submodule", "parallel/fetch");
diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c
index 9348184d30..d53db6d11c 100644
--- a/t/helper/test-run-command.c
+++ b/t/helper/test-run-command.c
@@ -51,6 +51,16 @@ static int no_job(struct child_process *cp,
 	return 0;
 }
 
+static void test_consume_sideband(struct strbuf *output, void *cb)
+{
+	FILE *sideband;
+
+	sideband = fopen("./sideband", "a");
+
+	strbuf_write(output, sideband);
+	fclose(sideband);
+}
+
 static int task_finished(int result,
 			 struct strbuf *err,
 			 void *pp_cb,
@@ -201,7 +211,7 @@ static int testsuite(int argc, const char **argv)
 		suite.tests.nr, max_jobs);
 
 	ret = run_processes_parallel(max_jobs, next_test, test_failed,
-				     test_stdin, test_finished, &suite);
+				     test_stdin, NULL, test_finished, &suite);
 
 	if (suite.failed.nr > 0) {
 		ret = 1;
@@ -429,23 +439,28 @@ int cmd__run_command(int argc, const char **argv)
 
 	if (!strcmp(argv[1], "run-command-parallel"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, NULL, &proc));
+					    NULL, NULL, NULL, NULL, &proc));
 
 	if (!strcmp(argv[1], "run-command-abort"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, task_finished, &proc));
+					    NULL, NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-no-jobs"))
 		exit(run_processes_parallel(jobs, no_job,
-					    NULL, NULL, task_finished, &proc));
+					    NULL, NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-stdin")) {
 		proc.in = -1;
 		proc.no_stdin = 0;
 		exit (run_processes_parallel(jobs, parallel_next, NULL,
-					     test_stdin, NULL, &proc));
+					     test_stdin, NULL, NULL, &proc));
 	}
 
+	if (!strcmp(argv[1], "run-command-sideband"))
+		exit(run_processes_parallel(jobs, parallel_next, NULL, NULL,
+					    test_consume_sideband, NULL,
+					    &proc));
+
 	fprintf(stderr, "check usage\n");
 	return 1;
 }
diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh
index 87759482ad..e99f6c7f44 100755
--- a/t/t0061-run-command.sh
+++ b/t/t0061-run-command.sh
@@ -143,6 +143,13 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai
 	test_cmp expect actual
 '
 
+test_expect_success 'run_command can divert output' '
+	test_when_finished rm sideband &&
+	test-tool run-command run-command-sideband 3 sh -c "printf \"%s\n%s\n\" Hello World" 2>actual &&
+	test_must_be_empty actual &&
+	test_cmp expect sideband
+'
+
 cat >expect <<-EOF
 preloaded output of a child
 listening for stdin:
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 17/37] hooks: allow callers to capture output
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (15 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 16/37] run-command: allow capturing of collated output Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-12  9:08   ` Ævar Arnfjörð Bjarmason
  2021-03-11  2:10 ` [PATCH v8 18/37] commit: use config-based hooks Emily Shaffer
                   ` (23 subsequent siblings)
  40 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Some server-side hooks will require capturing output to send over
sideband instead of printing directly to stderr. Expose that capability.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

Notes:
    You can see this in practice in the conversions for some of the push hooks,
    like 'receive-pack'.

 hook.c | 3 ++-
 hook.h | 8 ++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/hook.c b/hook.c
index e16b082cbd..2322720ffe 100644
--- a/hook.c
+++ b/hook.c
@@ -256,6 +256,7 @@ void run_hooks_opt_init_sync(struct run_hooks_opt *o)
 	o->dir = NULL;
 	o->feed_pipe = NULL;
 	o->feed_pipe_ctx = NULL;
+	o->consume_sideband = NULL;
 }
 
 void run_hooks_opt_init_async(struct run_hooks_opt *o)
@@ -434,7 +435,7 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options)
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
-				   NULL,
+				   options->consume_sideband,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
diff --git a/hook.h b/hook.h
index ecf0228a46..4ff9999b04 100644
--- a/hook.h
+++ b/hook.h
@@ -78,6 +78,14 @@ struct run_hooks_opt
 	feed_pipe_fn feed_pipe;
 	void *feed_pipe_ctx;
 
+	/*
+	 * Populate this to capture output and prevent it from being printed to
+	 * stderr. This will be passed directly through to
+	 * run_command:run_parallel_processes(). See t/helper/test-run-command.c
+	 * for an example.
+	 */
+	consume_sideband_fn consume_sideband;
+
 	/* Number of threads to parallelize across */
 	int jobs;
 
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 18/37] commit: use config-based hooks
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (16 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 17/37] hooks: allow callers to capture output Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-12 10:22   ` Junio C Hamano
  2021-03-11  2:10 ` [PATCH v8 19/37] am: convert applypatch hooks to use config Emily Shaffer
                   ` (22 subsequent siblings)
  40 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

As part of the adoption of config-based hooks, teach run_commit_hook()
to call hook.h instead of run-command.h. This covers 'pre-commit',
'commit-msg', and 'prepare-commit-msg'. Additionally, ask the hook
library - not run-command - whether any hooks will be run, as it's
possible hooks may exist in the config but not the hookdir.

Because all but 'post-commit' hooks are expected to make some state
change, force all but 'post-commit' hook to run in series. 'post-commit'
"is meant primarily for notification, and cannot affect the outcome of
`git commit`," so it is fine to run in parallel.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt                    | 13 +++++++++++
 builtin/commit.c                              | 11 +++++-----
 builtin/merge.c                               |  9 ++++----
 commit.c                                      | 22 ++++++++++++++-----
 commit.h                                      |  3 ++-
 sequencer.c                                   |  7 +++---
 ...3-pre-commit-and-pre-merge-commit-hooks.sh | 17 ++++++++++++--
 7 files changed, 61 insertions(+), 21 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 1f3b57d04d..984fb998b2 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -103,6 +103,8 @@ The default 'pre-commit' hook, when enabled--and with the
 `hooks.allownonascii` config option unset or set to false--prevents
 the use of non-ASCII filenames.
 
+Hooks executed during 'pre-commit' will not be parallelized.
+
 pre-merge-commit
 ~~~~~~~~~~~~~~~~
 
@@ -125,6 +127,8 @@ need to be resolved and the result committed separately (see
 linkgit:git-merge[1]). At that point, this hook will not be executed,
 but the 'pre-commit' hook will, if it is enabled.
 
+Hooks executed during 'pre-merge-commit' will not be parallelized.
+
 prepare-commit-msg
 ~~~~~~~~~~~~~~~~~~
 
@@ -150,6 +154,9 @@ be used as replacement for pre-commit hook.
 The sample `prepare-commit-msg` hook that comes with Git removes the
 help message found in the commented portion of the commit template.
 
+Hooks executed during 'prepare-commit-msg' will not be parallelized, because
+hooks are expected to edit the file containing the commit log message.
+
 commit-msg
 ~~~~~~~~~~
 
@@ -166,6 +173,9 @@ file.
 The default 'commit-msg' hook, when enabled, detects duplicate
 `Signed-off-by` trailers, and aborts the commit if one is found.
 
+Hooks executed during 'commit-msg' will not be parallelized, because hooks are
+expected to edit the file containing the proposed commit log message.
+
 post-commit
 ~~~~~~~~~~~
 
@@ -175,6 +185,9 @@ invoked after a commit is made.
 This hook is meant primarily for notification, and cannot affect
 the outcome of `git commit`.
 
+Hooks executed during 'post-commit' will run in parallel, unless hook.jobs is
+configured to 1.
+
 pre-rebase
 ~~~~~~~~~~
 
diff --git a/builtin/commit.c b/builtin/commit.c
index 739110c5a7..39f387e8f7 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -36,6 +36,7 @@
 #include "help.h"
 #include "commit-reach.h"
 #include "commit-graph.h"
+#include "hook.h"
 
 static const char * const builtin_commit_usage[] = {
 	N_("git commit [<options>] [--] <pathspec>..."),
@@ -699,7 +700,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	/* This checks and barfs if author is badly specified */
 	determine_author_info(author_ident);
 
-	if (!no_verify && run_commit_hook(use_editor, index_file, "pre-commit", NULL))
+	if (!no_verify && run_commit_hook(use_editor, 0, index_file, "pre-commit", NULL))
 		return 0;
 
 	if (squash_message) {
@@ -983,7 +984,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (!no_verify && find_hook("pre-commit")) {
+	if (!no_verify && hook_exists("pre-commit", HOOKDIR_USE_CONFIG)) {
 		/*
 		 * Re-read the index as pre-commit hook could have updated it,
 		 * and write it out as a tree.  We must do this before we invoke
@@ -998,7 +999,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (run_commit_hook(use_editor, index_file, "prepare-commit-msg",
+	if (run_commit_hook(use_editor, 0, index_file, "prepare-commit-msg",
 			    git_path_commit_editmsg(), hook_arg1, hook_arg2, NULL))
 		return 0;
 
@@ -1015,7 +1016,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	}
 
 	if (!no_verify &&
-	    run_commit_hook(use_editor, index_file, "commit-msg", git_path_commit_editmsg(), NULL)) {
+	    run_commit_hook(use_editor, 0, index_file, "commit-msg", git_path_commit_editmsg(), NULL)) {
 		return 0;
 	}
 
@@ -1701,7 +1702,7 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 
 	repo_rerere(the_repository, 0);
 	run_auto_maintenance(quiet);
-	run_commit_hook(use_editor, get_index_file(), "post-commit", NULL);
+	run_commit_hook(use_editor, 1, get_index_file(), "post-commit", NULL);
 	if (amend && !no_post_rewrite) {
 		commit_post_rewrite(the_repository, current_head, &oid);
 	}
diff --git a/builtin/merge.c b/builtin/merge.c
index eb00b273e6..33df744ab0 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -43,6 +43,7 @@
 #include "commit-reach.h"
 #include "wt-status.h"
 #include "commit-graph.h"
+#include "hook.h"
 
 #define DEFAULT_TWOHEAD (1<<0)
 #define DEFAULT_OCTOPUS (1<<1)
@@ -837,14 +838,14 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 	struct strbuf msg = STRBUF_INIT;
 	const char *index_file = get_index_file();
 
-	if (!no_verify && run_commit_hook(0 < option_edit, index_file, "pre-merge-commit", NULL))
+	if (!no_verify && run_commit_hook(0 < option_edit, 0, index_file, "pre-merge-commit", NULL))
 		abort_commit(remoteheads, NULL);
 	/*
 	 * Re-read the index as pre-merge-commit hook could have updated it,
 	 * and write it out as a tree.  We must do this before we invoke
 	 * the editor and after we invoke run_status above.
 	 */
-	if (find_hook("pre-merge-commit"))
+	if (hook_exists("pre-merge-commit", HOOKDIR_USE_CONFIG))
 		discard_cache();
 	read_cache_from(index_file);
 	strbuf_addbuf(&msg, &merge_msg);
@@ -865,7 +866,7 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 		append_signoff(&msg, ignore_non_trailer(msg.buf, msg.len), 0);
 	write_merge_heads(remoteheads);
 	write_file_buf(git_path_merge_msg(the_repository), msg.buf, msg.len);
-	if (run_commit_hook(0 < option_edit, get_index_file(), "prepare-commit-msg",
+	if (run_commit_hook(0 < option_edit, 0, get_index_file(), "prepare-commit-msg",
 			    git_path_merge_msg(the_repository), "merge", NULL))
 		abort_commit(remoteheads, NULL);
 	if (0 < option_edit) {
@@ -873,7 +874,7 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 			abort_commit(remoteheads, NULL);
 	}
 
-	if (!no_verify && run_commit_hook(0 < option_edit, get_index_file(),
+	if (!no_verify && run_commit_hook(0 < option_edit, 0, get_index_file(),
 					  "commit-msg",
 					  git_path_merge_msg(the_repository), NULL))
 		abort_commit(remoteheads, NULL);
diff --git a/commit.c b/commit.c
index 6ccd774841..b72158cb34 100644
--- a/commit.c
+++ b/commit.c
@@ -21,6 +21,7 @@
 #include "commit-reach.h"
 #include "run-command.h"
 #include "shallow.h"
+#include "hook.h"
 
 static struct commit_extra_header *read_commit_extra_header_lines(const char *buf, size_t len, const char **);
 
@@ -1681,25 +1682,34 @@ size_t ignore_non_trailer(const char *buf, size_t len)
 	return boc ? len - boc : len - cutoff;
 }
 
-int run_commit_hook(int editor_is_used, const char *index_file,
+int run_commit_hook(int editor_is_used, int parallelize, const char *index_file,
 		    const char *name, ...)
 {
-	struct strvec hook_env = STRVEC_INIT;
+	struct run_hooks_opt opt;
 	va_list args;
+	const char *arg;
 	int ret;
 
-	strvec_pushf(&hook_env, "GIT_INDEX_FILE=%s", index_file);
+	run_hooks_opt_init_sync(&opt);
+
+	if (parallelize)
+		opt.jobs = configured_hook_jobs();
+
+	strvec_pushf(&opt.env, "GIT_INDEX_FILE=%s", index_file);
 
 	/*
 	 * Let the hook know that no editor will be launched.
 	 */
 	if (!editor_is_used)
-		strvec_push(&hook_env, "GIT_EDITOR=:");
+		strvec_push(&opt.env, "GIT_EDITOR=:");
 
 	va_start(args, name);
-	ret = run_hook_ve(hook_env.v, name, args);
+	while ((arg = va_arg(args, const char *)))
+		strvec_push(&opt.args, arg);
 	va_end(args);
-	strvec_clear(&hook_env);
+
+	ret = run_hooks(name, &opt);
+	run_hooks_opt_clear(&opt);
 
 	return ret;
 }
diff --git a/commit.h b/commit.h
index 49c0f50396..abea90a3f9 100644
--- a/commit.h
+++ b/commit.h
@@ -360,7 +360,8 @@ int compare_commits_by_commit_date(const void *a_, const void *b_, void *unused)
 int compare_commits_by_gen_then_commit_date(const void *a_, const void *b_, void *unused);
 
 LAST_ARG_MUST_BE_NULL
-int run_commit_hook(int editor_is_used, const char *index_file, const char *name, ...);
+int run_commit_hook(int editor_is_used, int parallelize, const char *index_file,
+		    const char *name, ...);
 
 /* Sign a commit or tag buffer, storing the result in a header. */
 int sign_with_header(struct strbuf *buf, const char *keyid);
diff --git a/sequencer.c b/sequencer.c
index d2332d3e17..e3a951fbeb 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -34,6 +34,7 @@
 #include "commit-reach.h"
 #include "rebase-interactive.h"
 #include "reset.h"
+#include "hook.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -1206,7 +1207,7 @@ static int run_prepare_commit_msg_hook(struct repository *r,
 	} else {
 		arg1 = "message";
 	}
-	if (run_commit_hook(0, r->index_file, "prepare-commit-msg", name,
+	if (run_commit_hook(0, 0, r->index_file, "prepare-commit-msg", name,
 			    arg1, arg2, NULL))
 		ret = error(_("'prepare-commit-msg' hook failed"));
 
@@ -1444,7 +1445,7 @@ static int try_to_commit(struct repository *r,
 		}
 	}
 
-	if (find_hook("prepare-commit-msg")) {
+	if (hook_exists("prepare-commit-msg", HOOKDIR_USE_CONFIG)) {
 		res = run_prepare_commit_msg_hook(r, msg, hook_commit);
 		if (res)
 			goto out;
@@ -1536,7 +1537,7 @@ static int try_to_commit(struct repository *r,
 		goto out;
 	}
 
-	run_commit_hook(0, r->index_file, "post-commit", NULL);
+	run_commit_hook(0, 1, r->index_file, "post-commit", NULL);
 	if (flags & AMEND_MSG)
 		commit_post_rewrite(r, current_head, oid);
 
diff --git a/t/t7503-pre-commit-and-pre-merge-commit-hooks.sh b/t/t7503-pre-commit-and-pre-merge-commit-hooks.sh
index 606d8d0f08..e9e3713033 100755
--- a/t/t7503-pre-commit-and-pre-merge-commit-hooks.sh
+++ b/t/t7503-pre-commit-and-pre-merge-commit-hooks.sh
@@ -8,8 +8,8 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
 . ./test-lib.sh
 
 HOOKDIR="$(git rev-parse --git-dir)/hooks"
-PRECOMMIT="$HOOKDIR/pre-commit"
-PREMERGE="$HOOKDIR/pre-merge-commit"
+PRECOMMIT="$(pwd)/$HOOKDIR/pre-commit"
+PREMERGE="$(pwd)/$HOOKDIR/pre-merge-commit"
 
 # Prepare sample scripts that write their $0 to actual_hooks
 test_expect_success 'sample script setup' '
@@ -106,6 +106,19 @@ test_expect_success 'with succeeding hook' '
 	test_cmp expected_hooks actual_hooks
 '
 
+# NEEDSWORK: when 'git hook add' and 'git hook remove' have been added, use that
+# instead
+test_expect_success 'with succeeding hook (config-based)' '
+	test_when_finished "git config --unset hook.pre-commit.command success.sample" &&
+	test_when_finished "rm -f expected_hooks actual_hooks" &&
+	git config hook.pre-commit.command "$HOOKDIR/success.sample" &&
+	echo "$HOOKDIR/success.sample" >expected_hooks &&
+	echo "more" >>file &&
+	git add file &&
+	git commit -m "more" &&
+	test_cmp expected_hooks actual_hooks
+'
+
 test_expect_success 'with succeeding hook (merge)' '
 	test_when_finished "rm -f \"$PREMERGE\" expected_hooks actual_hooks" &&
 	cp "$HOOKDIR/success.sample" "$PREMERGE" &&
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 19/37] am: convert applypatch hooks to use config
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (17 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 18/37] commit: use config-based hooks Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-12 10:23   ` Junio C Hamano
  2021-03-11  2:10 ` [PATCH v8 20/37] merge: use config-based hooks for post-merge hook Emily Shaffer
                   ` (21 subsequent siblings)
  40 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach pre-applypatch, post-applypatch, and applypatch-msg to use the
hook.h library instead of the run-command.h library. This enables use of
hooks specified in the config, in addition to those in the hookdir.
These three hooks are called only by builtin/am.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt |  9 +++++++++
 builtin/am.c               | 14 +++++++++++---
 2 files changed, 20 insertions(+), 3 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 984fb998b2..0e7eb972ab 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -58,6 +58,9 @@ the message file.
 The default 'applypatch-msg' hook, when enabled, runs the
 'commit-msg' hook, if the latter is enabled.
 
+Hooks run during 'applypatch-msg' will not be parallelized, because hooks are
+expected to edit the file holding the commit log message.
+
 pre-applypatch
 ~~~~~~~~~~~~~~
 
@@ -73,6 +76,9 @@ make a commit if it does not pass certain test.
 The default 'pre-applypatch' hook, when enabled, runs the
 'pre-commit' hook, if the latter is enabled.
 
+Hooks run during 'pre-applypatch' will be run in parallel, unless hook.jobs is
+configured to 1.
+
 post-applypatch
 ~~~~~~~~~~~~~~~
 
@@ -82,6 +88,9 @@ and is invoked after the patch is applied and a commit is made.
 This hook is meant primarily for notification, and cannot affect
 the outcome of `git am`.
 
+Hooks run during 'post-applypatch' will be run in parallel, unless hook.jobs is
+configured to 1.
+
 pre-commit
 ~~~~~~~~~~
 
diff --git a/builtin/am.c b/builtin/am.c
index 8355e3566f..4467fd9e63 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -33,6 +33,7 @@
 #include "string-list.h"
 #include "packfile.h"
 #include "repository.h"
+#include "hook.h"
 
 /**
  * Returns the length of the first line of msg.
@@ -426,9 +427,13 @@ static void am_destroy(const struct am_state *state)
 static int run_applypatch_msg_hook(struct am_state *state)
 {
 	int ret;
+	struct run_hooks_opt opt;
+	run_hooks_opt_init_sync(&opt);
 
 	assert(state->msg);
-	ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL);
+	strvec_push(&opt.args, am_path(state, "final-commit"));
+	ret = run_hooks("applypatch-msg", &opt);
+	run_hooks_opt_clear(&opt);
 
 	if (!ret) {
 		FREE_AND_NULL(state->msg);
@@ -1558,8 +1563,10 @@ static void do_commit(const struct am_state *state)
 	struct commit_list *parents = NULL;
 	const char *reflog_msg, *author, *committer = NULL;
 	struct strbuf sb = STRBUF_INIT;
+	struct run_hooks_opt hook_opt;
+	run_hooks_opt_init_async(&hook_opt);
 
-	if (run_hook_le(NULL, "pre-applypatch", NULL))
+	if (run_hooks("pre-applypatch", &hook_opt))
 		exit(1);
 
 	if (write_cache_as_tree(&tree, 0, NULL))
@@ -1611,8 +1618,9 @@ static void do_commit(const struct am_state *state)
 		fclose(fp);
 	}
 
-	run_hook_le(NULL, "post-applypatch", NULL);
+	run_hooks("post-applypatch", &hook_opt);
 
+	run_hooks_opt_clear(&hook_opt);
 	strbuf_release(&sb);
 }
 
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 20/37] merge: use config-based hooks for post-merge hook
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (18 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 19/37] am: convert applypatch hooks to use config Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 21/37] gc: use hook library for pre-auto-gc hook Emily Shaffer
                   ` (20 subsequent siblings)
  40 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach post-merge to use the hook.h library instead of the run-command.h
library to run hooks. This means that post-merge hooks can come from the
config as well as from the hookdir. post-merge is invoked only from
builtin/merge.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt | 3 +++
 builtin/merge.c            | 6 +++++-
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 0e7eb972ab..664ad4803e 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -242,6 +242,9 @@ save and restore any form of metadata associated with the working tree
 (e.g.: permissions/ownership, ACLS, etc).  See contrib/hooks/setgitperms.perl
 for an example of how to do this.
 
+Hooks executed during 'post-merge' will run in parallel, unless hook.jobs is
+configured to 1.
+
 pre-push
 ~~~~~~~~
 
diff --git a/builtin/merge.c b/builtin/merge.c
index 33df744ab0..b473c8c5d3 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -444,7 +444,9 @@ static void finish(struct commit *head_commit,
 		   const struct object_id *new_head, const char *msg)
 {
 	struct strbuf reflog_message = STRBUF_INIT;
+	struct run_hooks_opt opt;
 	const struct object_id *head = &head_commit->object.oid;
+	run_hooks_opt_init_async(&opt);
 
 	if (!msg)
 		strbuf_addstr(&reflog_message, getenv("GIT_REFLOG_ACTION"));
@@ -485,7 +487,9 @@ static void finish(struct commit *head_commit,
 	}
 
 	/* Run a post-merge hook */
-	run_hook_le(NULL, "post-merge", squash ? "1" : "0", NULL);
+	strvec_push(&opt.args, squash ? "1" : "0");
+	run_hooks("post-merge", &opt);
+	run_hooks_opt_clear(&opt);
 
 	apply_autostash(git_path_merge_autostash(the_repository));
 	strbuf_release(&reflog_message);
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 21/37] gc: use hook library for pre-auto-gc hook
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (19 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 20/37] merge: use config-based hooks for post-merge hook Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 22/37] rebase: teach pre-rebase to use hook.h Emily Shaffer
                   ` (19 subsequent siblings)
  40 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Using the hook.h library instead of the run-command.h library to run
pre-auto-gc means that those hooks can be set up in config files, as
well as in the hookdir. pre-auto-gc is called only from builtin/gc.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt | 3 +++
 builtin/gc.c               | 5 ++++-
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 664ad4803e..00f88912cd 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -560,6 +560,9 @@ This hook is invoked by `git gc --auto` (see linkgit:git-gc[1]). It
 takes no parameter, and exiting with non-zero status from this script
 causes the `git gc --auto` to abort.
 
+Hooks run during 'pre-auto-gc' will be run in parallel, unless hook.jobs is
+configured to 1.
+
 post-rewrite
 ~~~~~~~~~~~~
 
diff --git a/builtin/gc.c b/builtin/gc.c
index ef7226d7bc..e62cb510ee 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -32,6 +32,7 @@
 #include "remote.h"
 #include "object-store.h"
 #include "exec-cmd.h"
+#include "hook.h"
 
 #define FAILED_RUN "failed to run %s"
 
@@ -348,6 +349,8 @@ static void add_repack_incremental_option(void)
 
 static int need_to_gc(void)
 {
+	struct run_hooks_opt hook_opt;
+	run_hooks_opt_init_async(&hook_opt);
 	/*
 	 * Setting gc.auto to 0 or negative can disable the
 	 * automatic gc.
@@ -394,7 +397,7 @@ static int need_to_gc(void)
 	else
 		return 0;
 
-	if (run_hook_le(NULL, "pre-auto-gc", NULL))
+	if (run_hooks("pre-auto-gc", &hook_opt))
 		return 0;
 	return 1;
 }
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 22/37] rebase: teach pre-rebase to use hook.h
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (20 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 21/37] gc: use hook library for pre-auto-gc hook Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-12 10:24   ` Junio C Hamano
  2021-03-11  2:10 ` [PATCH v8 23/37] read-cache: convert post-index-change hook to use config Emily Shaffer
                   ` (18 subsequent siblings)
  40 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using hook.h instead of run-command.h to run hooks, pre-rebase hooks
can now be specified in the config as well as in the hookdir. pre-rebase
is not called anywhere besides builtin/rebase.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt | 3 +++
 builtin/rebase.c           | 9 +++++++--
 2 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 00f88912cd..e3a0375827 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -206,6 +206,9 @@ two parameters.  The first parameter is the upstream from which
 the series was forked.  The second parameter is the branch being
 rebased, and is not set when rebasing the current branch.
 
+Hooks executed during 'pre-rebase' will run in parallel, unless hook.jobs is
+configured to 1.
+
 post-checkout
 ~~~~~~~~~~~~~
 
diff --git a/builtin/rebase.c b/builtin/rebase.c
index de400f9a19..c35b5ba452 100644
--- a/builtin/rebase.c
+++ b/builtin/rebase.c
@@ -28,6 +28,7 @@
 #include "sequencer.h"
 #include "rebase-interactive.h"
 #include "reset.h"
+#include "hook.h"
 
 #define DEFAULT_REFLOG_ACTION "rebase"
 
@@ -1318,6 +1319,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	char *squash_onto_name = NULL;
 	int reschedule_failed_exec = -1;
 	int allow_preemptive_ff = 1;
+	struct run_hooks_opt hook_opt;
 	struct option builtin_rebase_options[] = {
 		OPT_STRING(0, "onto", &options.onto_name,
 			   N_("revision"),
@@ -1431,6 +1433,8 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	};
 	int i;
 
+	run_hooks_opt_init_async(&hook_opt);
+
 	if (argc == 2 && !strcmp(argv[1], "-h"))
 		usage_with_options(builtin_rebase_usage,
 				   builtin_rebase_options);
@@ -2032,9 +2036,9 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	}
 
 	/* If a hook exists, give it a chance to interrupt*/
+	strvec_pushl(&hook_opt.args, options.upstream_arg, argc ? argv[0] : NULL, NULL);
 	if (!ok_to_skip_pre_rebase &&
-	    run_hook_le(NULL, "pre-rebase", options.upstream_arg,
-			argc ? argv[0] : NULL, NULL))
+	    run_hooks("pre-rebase", &hook_opt))
 		die(_("The pre-rebase hook refused to rebase."));
 
 	if (options.flags & REBASE_DIFFSTAT) {
@@ -2114,6 +2118,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	ret = !!run_specific_rebase(&options, action);
 
 cleanup:
+	run_hooks_opt_clear(&hook_opt);
 	strbuf_release(&buf);
 	strbuf_release(&revisions);
 	free(options.head_name);
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 23/37] read-cache: convert post-index-change hook to use config
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (21 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 22/37] rebase: teach pre-rebase to use hook.h Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-12 10:22   ` Junio C Hamano
  2021-03-11  2:10 ` [PATCH v8 24/37] receive-pack: convert push-to-checkout hook to hook.h Emily Shaffer
                   ` (17 subsequent siblings)
  40 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using hook.h instead of run-command.h to run, post-index-change hooks
can now be specified in the config in addition to the hookdir.
post-index-change is not run anywhere besides in read-cache.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt |  3 +++
 read-cache.c               | 13 ++++++++++---
 2 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index e3a0375827..e5c2cef271 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -720,6 +720,9 @@ and "0" meaning they were not.
 Only one parameter should be set to "1" when the hook runs.  The hook
 running passing "1", "1" should not be possible.
 
+Hooks run during 'post-index-change' will be run in parallel, unless hook.jobs
+is configured to 1.
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/read-cache.c b/read-cache.c
index 1e9a50c6c7..fd6c111372 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -25,6 +25,7 @@
 #include "fsmonitor.h"
 #include "thread-utils.h"
 #include "progress.h"
+#include "hook.h"
 
 /* Mask for the name length in ce_flags in the on-disk index */
 
@@ -3070,6 +3071,8 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 				 unsigned flags)
 {
 	int ret;
+	struct run_hooks_opt hook_opt;
+	run_hooks_opt_init_async(&hook_opt);
 
 	/*
 	 * TODO trace2: replace "the_repository" with the actual repo instance
@@ -3088,9 +3091,13 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 	else
 		ret = close_lock_file_gently(lock);
 
-	run_hook_le(NULL, "post-index-change",
-			istate->updated_workdir ? "1" : "0",
-			istate->updated_skipworktree ? "1" : "0", NULL);
+	strvec_pushl(&hook_opt.args,
+		     istate->updated_workdir ? "1" : "0",
+		     istate->updated_skipworktree ? "1" : "0",
+		     NULL);
+	run_hooks("post-index-change", &hook_opt);
+	run_hooks_opt_clear(&hook_opt);
+
 	istate->updated_workdir = 0;
 	istate->updated_skipworktree = 0;
 
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 24/37] receive-pack: convert push-to-checkout hook to hook.h
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (22 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 23/37] read-cache: convert post-index-change hook to use config Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-12 10:24   ` Junio C Hamano
  2021-03-11  2:10 ` [PATCH v8 25/37] git-p4: use 'git hook' to run hooks Emily Shaffer
                   ` (16 subsequent siblings)
  40 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using hook.h instead of run-command.h to invoke push-to-checkout,
hooks can now be specified in the config as well as in the hookdir.
push-to-checkout is not called anywhere but in builtin/receive-pack.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt |  1 +
 builtin/receive-pack.c     | 16 ++++++++++++----
 2 files changed, 13 insertions(+), 4 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index e5c2cef271..f2178dbc83 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -555,6 +555,7 @@ that switches branches while
 keeping the local changes in the working tree that do not interfere
 with the difference between the branches.
 
+Hooks executed during 'push-to-checkout' will not be parallelized.
 
 pre-auto-gc
 ~~~~~~~~~~~
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index d26040c477..234b70f0d1 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -29,6 +29,7 @@
 #include "commit-reach.h"
 #include "worktree.h"
 #include "shallow.h"
+#include "hook.h"
 
 static const char * const receive_pack_usage[] = {
 	N_("git receive-pack <git-dir>"),
@@ -1435,12 +1436,19 @@ static const char *push_to_checkout(unsigned char *hash,
 				    struct strvec *env,
 				    const char *work_tree)
 {
+	struct run_hooks_opt opt;
+	run_hooks_opt_init_sync(&opt);
+
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
-	if (run_hook_le(env->v, push_to_checkout_hook,
-			hash_to_hex(hash), NULL))
+	strvec_pushv(&opt.env, env->v);
+	strvec_push(&opt.args, hash_to_hex(hash));
+	if (run_hooks(push_to_checkout_hook, &opt)) {
+		run_hooks_opt_clear(&opt);
 		return "push-to-checkout hook declined";
-	else
+	} else {
+		run_hooks_opt_clear(&opt);
 		return NULL;
+	}
 }
 
 static const char *update_worktree(unsigned char *sha1, const struct worktree *worktree)
@@ -1464,7 +1472,7 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 
 	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
 
-	if (!find_hook(push_to_checkout_hook))
+	if (!hook_exists(push_to_checkout_hook, HOOKDIR_USE_CONFIG))
 		retval = push_to_deploy(sha1, &env, work_tree);
 	else
 		retval = push_to_checkout(sha1, &env, work_tree);
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 25/37] git-p4: use 'git hook' to run hooks
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (23 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 24/37] receive-pack: convert push-to-checkout hook to hook.h Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 26/37] hooks: convert 'post-checkout' hook to hook library Emily Shaffer
                   ` (15 subsequent siblings)
  40 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Instead of duplicating the behavior of run-command.h:run_hook_le() in
Python, we can directly call 'git hook run'. As a bonus, this means
git-p4 learns how to find hook specifications from the Git config as
well as from the hookdir.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

Notes:
    Maybe there is a better way to do this - I had a hard time getting this to run
    locally, and Python is not my forte, so if anybody has a better approach I'd
    love to just take that patch instead :)
    
    Since v6, removed the developer debug print statements.... :X
    
    Maybe there is a better way to do this - I had a hard time getting this to run
    locally, and Python is not my forte, so if anybody has a better approach I'd
    love to just take that patch instead :)

 git-p4.py | 67 +++++--------------------------------------------------
 1 file changed, 6 insertions(+), 61 deletions(-)

diff --git a/git-p4.py b/git-p4.py
index 09c9e93ac4..4b1c69822c 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -208,70 +208,15 @@ def decode_path(path):
 
 def run_git_hook(cmd, param=[]):
     """Execute a hook if the hook exists."""
-    if verbose:
-        sys.stderr.write("Looking for hook: %s\n" % cmd)
-        sys.stderr.flush()
-
-    hooks_path = gitConfig("core.hooksPath")
-    if len(hooks_path) <= 0:
-        hooks_path = os.path.join(os.environ["GIT_DIR"], "hooks")
-
-    if not isinstance(param, list):
-        param=[param]
-
-    # resolve hook file name, OS depdenent
-    hook_file = os.path.join(hooks_path, cmd)
-    if platform.system() == 'Windows':
-        if not os.path.isfile(hook_file):
-            # look for the file with an extension
-            files = glob.glob(hook_file + ".*")
-            if not files:
-                return True
-            files.sort()
-            hook_file = files.pop()
-            while hook_file.upper().endswith(".SAMPLE"):
-                # The file is a sample hook. We don't want it
-                if len(files) > 0:
-                    hook_file = files.pop()
-                else:
-                    return True
-
-    if not os.path.isfile(hook_file) or not os.access(hook_file, os.X_OK):
+    if not cmd:
         return True
 
-    return run_hook_command(hook_file, param) == 0
-
-def run_hook_command(cmd, param):
-    """Executes a git hook command
-       cmd = the command line file to be executed. This can be
-       a file that is run by OS association.
-
-       param = a list of parameters to pass to the cmd command
-
-       On windows, the extension is checked to see if it should
-       be run with the Git for Windows Bash shell.  If there
-       is no file extension, the file is deemed a bash shell
-       and will be handed off to sh.exe. Otherwise, Windows
-       will be called with the shell to handle the file assocation.
-
-       For non Windows operating systems, the file is called
-       as an executable.
-    """
-    cli = [cmd] + param
-    use_shell = False
-    if platform.system() == 'Windows':
-        (root,ext) = os.path.splitext(cmd)
-        if ext == "":
-            exe_path = os.environ.get("EXEPATH")
-            if exe_path is None:
-                exe_path = ""
-            else:
-                exe_path = os.path.join(exe_path, "bin")
-            cli = [os.path.join(exe_path, "SH.EXE")] + cli
-        else:
-            use_shell = True
-    return subprocess.call(cli, shell=use_shell)
+    """args are specified with -a <arg> -a <arg> -a <arg>"""
+    args = (['git', 'hook', 'run'] +
+	    ["-a" + arg for arg in param] +
+	    [cmd])
 
+    return subprocess.call(args) == 0
 
 def write_pipe(c, stdin):
     if verbose:
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 26/37] hooks: convert 'post-checkout' hook to hook library
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (24 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 25/37] git-p4: use 'git hook' to run hooks Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 27/37] hook: convert 'post-rewrite' hook to config Emily Shaffer
                   ` (14 subsequent siblings)
  40 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using the 'hook.h' library, 'post-checkout' hooks can now be
specified in the config as well as in the hook directory.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt |  2 ++
 builtin/checkout.c         | 19 ++++++++++++++-----
 builtin/clone.c            |  8 ++++++--
 builtin/worktree.c         | 31 +++++++++++++++----------------
 reset.c                    | 16 ++++++++++++----
 5 files changed, 49 insertions(+), 27 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index f2178dbc83..362224a03b 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -231,6 +231,8 @@ This hook can be used to perform repository validity checks, auto-display
 differences from the previous HEAD if different, or set working dir metadata
 properties.
 
+Hooks executed during 'post-checkout' will not be parallelized.
+
 post-merge
 ~~~~~~~~~~
 
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 2d6550bc3c..f287b5e643 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "diff.h"
 #include "dir.h"
+#include "hook.h"
 #include "ll-merge.h"
 #include "lockfile.h"
 #include "merge-recursive.h"
@@ -104,13 +105,21 @@ struct branch_info {
 static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit,
 			      int changed)
 {
-	return run_hook_le(NULL, "post-checkout",
-			   oid_to_hex(old_commit ? &old_commit->object.oid : &null_oid),
-			   oid_to_hex(new_commit ? &new_commit->object.oid : &null_oid),
-			   changed ? "1" : "0", NULL);
+	struct run_hooks_opt opt;
+	int rc;
+
+	run_hooks_opt_init_sync(&opt);
+
 	/* "new_commit" can be NULL when checking out from the index before
 	   a commit exists. */
-
+	strvec_pushl(&opt.args,
+		     oid_to_hex(old_commit ? &old_commit->object.oid : &null_oid),
+		     oid_to_hex(new_commit ? &new_commit->object.oid : &null_oid),
+		     changed ? "1" : "0",
+		     NULL);
+	rc = run_hooks("post-checkout", &opt);
+	run_hooks_opt_clear(&opt);
+	return rc;
 }
 
 static int update_some(const struct object_id *oid, struct strbuf *base,
diff --git a/builtin/clone.c b/builtin/clone.c
index 51e844a2de..52f2a5ecb4 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -32,6 +32,7 @@
 #include "connected.h"
 #include "packfile.h"
 #include "list-objects-filter-options.h"
+#include "hook.h"
 
 /*
  * Overall FIXMEs:
@@ -771,6 +772,8 @@ static int checkout(int submodule_progress)
 	struct tree *tree;
 	struct tree_desc t;
 	int err = 0;
+	struct run_hooks_opt hook_opt;
+	run_hooks_opt_init_sync(&hook_opt);
 
 	if (option_no_checkout)
 		return 0;
@@ -816,8 +819,9 @@ static int checkout(int submodule_progress)
 	if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
 		die(_("unable to write new index file"));
 
-	err |= run_hook_le(NULL, "post-checkout", oid_to_hex(&null_oid),
-			   oid_to_hex(&oid), "1", NULL);
+	strvec_pushl(&hook_opt.args, oid_to_hex(&null_oid), oid_to_hex(&oid), "1", NULL);
+	err |= run_hooks("post-checkout", &hook_opt);
+	run_hooks_opt_clear(&hook_opt);
 
 	if (!err && (option_recurse_submodules.nr > 0)) {
 		struct strvec args = STRVEC_INIT;
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 1cd5c2016e..8b06d121e5 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -13,6 +13,7 @@
 #include "utf8.h"
 #include "worktree.h"
 #include "quote.h"
+#include "hook.h"
 
 static const char * const worktree_usage[] = {
 	N_("git worktree add [<options>] <path> [<commit-ish>]"),
@@ -383,22 +384,20 @@ static int add_worktree(const char *path, const char *refname,
 	 * is_junk is cleared, but do return appropriate code when hook fails.
 	 */
 	if (!ret && opts->checkout) {
-		const char *hook = find_hook("post-checkout");
-		if (hook) {
-			const char *env[] = { "GIT_DIR", "GIT_WORK_TREE", NULL };
-			cp.git_cmd = 0;
-			cp.no_stdin = 1;
-			cp.stdout_to_stderr = 1;
-			cp.dir = path;
-			cp.env = env;
-			cp.argv = NULL;
-			cp.trace2_hook_name = "post-checkout";
-			strvec_pushl(&cp.args, absolute_path(hook),
-				     oid_to_hex(&null_oid),
-				     oid_to_hex(&commit->object.oid),
-				     "1", NULL);
-			ret = run_command(&cp);
-		}
+		struct run_hooks_opt opt;
+		run_hooks_opt_init_sync(&opt);
+
+		strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
+		strvec_pushl(&opt.args,
+			     oid_to_hex(&null_oid),
+			     oid_to_hex(&commit->object.oid),
+			     "1",
+			     NULL);
+		opt.dir = path;
+
+		ret = run_hooks("post-checkout", &opt);
+
+		run_hooks_opt_clear(&opt);
 	}
 
 	strvec_clear(&child_env);
diff --git a/reset.c b/reset.c
index 2f4fbd07c5..85ee75f7fd 100644
--- a/reset.c
+++ b/reset.c
@@ -7,6 +7,7 @@
 #include "tree-walk.h"
 #include "tree.h"
 #include "unpack-trees.h"
+#include "hook.h"
 
 int reset_head(struct repository *r, struct object_id *oid, const char *action,
 	       const char *switch_to_branch, unsigned flags,
@@ -126,10 +127,17 @@ int reset_head(struct repository *r, struct object_id *oid, const char *action,
 			ret = create_symref("HEAD", switch_to_branch,
 					    reflog_head);
 	}
-	if (run_hook)
-		run_hook_le(NULL, "post-checkout",
-			    oid_to_hex(orig ? orig : &null_oid),
-			    oid_to_hex(oid), "1", NULL);
+	if (run_hook) {
+		struct run_hooks_opt opt;
+		run_hooks_opt_init_sync(&opt);
+		strvec_pushl(&opt.args,
+			     oid_to_hex(orig ? orig : &null_oid),
+			     oid_to_hex(oid),
+			     "1",
+			     NULL);
+		run_hooks("post-checkout", &opt);
+		run_hooks_opt_clear(&opt);
+	}
 
 leave_reset_head:
 	strbuf_release(&msg);
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 27/37] hook: convert 'post-rewrite' hook to config
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (25 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 26/37] hooks: convert 'post-checkout' hook to hook library Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 28/37] transport: convert pre-push hook to use config Emily Shaffer
                   ` (13 subsequent siblings)
  40 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using 'hook.h' for 'post-rewrite', we simplify hook invocations by
not needing to put together our own 'struct child_process' and we also
learn to run hooks specified in the config as well as the hook dir.

The signal handling that's being removed by this commit now takes place
in run-command.h:run_processes_parallel(), so it is OK to remove them
here.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt |  3 ++
 builtin/am.c               | 19 +++------
 sequencer.c                | 83 +++++++++++++++++---------------------
 3 files changed, 45 insertions(+), 60 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 362224a03b..544238b381 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -594,6 +594,9 @@ The hook always runs after the automatic note copying (see
 "notes.rewrite.<command>" in linkgit:git-config[1]) has happened, and
 thus has access to these notes.
 
+Hooks run during 'post-rewrite' will be run in parallel, unless hook.jobs is
+configured to 1.
+
 The following command-specific comments apply:
 
 rebase::
diff --git a/builtin/am.c b/builtin/am.c
index 4467fd9e63..45425105e8 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -450,23 +450,16 @@ static int run_applypatch_msg_hook(struct am_state *state)
  */
 static int run_post_rewrite_hook(const struct am_state *state)
 {
-	struct child_process cp = CHILD_PROCESS_INIT;
-	const char *hook = find_hook("post-rewrite");
+	struct run_hooks_opt opt;
 	int ret;
+	run_hooks_opt_init_async(&opt);
 
-	if (!hook)
-		return 0;
-
-	strvec_push(&cp.args, hook);
-	strvec_push(&cp.args, "rebase");
-
-	cp.in = xopen(am_path(state, "rewritten"), O_RDONLY);
-	cp.stdout_to_stderr = 1;
-	cp.trace2_hook_name = "post-rewrite";
+	strvec_push(&opt.args, "rebase");
+	opt.path_to_stdin = am_path(state, "rewritten");
 
-	ret = run_command(&cp);
+	ret = run_hooks("post-rewrite", &opt);
 
-	close(cp.in);
+	run_hooks_opt_clear(&opt);
 	return ret;
 }
 
diff --git a/sequencer.c b/sequencer.c
index e3a951fbeb..8280ba828b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -35,6 +35,7 @@
 #include "rebase-interactive.h"
 #include "reset.h"
 #include "hook.h"
+#include "string-list.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -1146,33 +1147,29 @@ int update_head_with_reflog(const struct commit *old_head,
 static int run_rewrite_hook(const struct object_id *oldoid,
 			    const struct object_id *newoid)
 {
-	struct child_process proc = CHILD_PROCESS_INIT;
-	const char *argv[3];
+	struct run_hooks_opt opt;
+	struct strbuf tmp = STRBUF_INIT;
+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int code;
-	struct strbuf sb = STRBUF_INIT;
+	run_hooks_opt_init_async(&opt);
 
-	argv[0] = find_hook("post-rewrite");
-	if (!argv[0])
-		return 0;
+	strvec_push(&opt.args, "amend");
 
-	argv[1] = "amend";
-	argv[2] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = "post-rewrite";
-
-	code = start_command(&proc);
-	if (code)
-		return code;
-	strbuf_addf(&sb, "%s %s\n", oid_to_hex(oldoid), oid_to_hex(newoid));
-	sigchain_push(SIGPIPE, SIG_IGN);
-	write_in_full(proc.in, sb.buf, sb.len);
-	close(proc.in);
-	strbuf_release(&sb);
-	sigchain_pop(SIGPIPE);
-	return finish_command(&proc);
+	strbuf_addf(&tmp,
+		    "%s %s",
+		    oid_to_hex(oldoid),
+		    oid_to_hex(newoid));
+	string_list_append(&to_stdin, tmp.buf);
+
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
+
+	code = run_hooks("post-rewrite", &opt);
+
+	run_hooks_opt_clear(&opt);
+	strbuf_release(&tmp);
+	string_list_clear(&to_stdin, 0);
+	return code;
 }
 
 void commit_post_rewrite(struct repository *r,
@@ -4325,30 +4322,22 @@ static int pick_commits(struct repository *r,
 		flush_rewritten_pending();
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
-			struct child_process child = CHILD_PROCESS_INIT;
-			const char *post_rewrite_hook =
-				find_hook("post-rewrite");
-
-			child.in = open(rebase_path_rewritten_list(), O_RDONLY);
-			child.git_cmd = 1;
-			strvec_push(&child.args, "notes");
-			strvec_push(&child.args, "copy");
-			strvec_push(&child.args, "--for-rewrite=rebase");
+			struct child_process notes_cp = CHILD_PROCESS_INIT;
+			struct run_hooks_opt hook_opt;
+			run_hooks_opt_init_async(&hook_opt);
+
+			notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY);
+			notes_cp.git_cmd = 1;
+			strvec_push(&notes_cp.args, "notes");
+			strvec_push(&notes_cp.args, "copy");
+			strvec_push(&notes_cp.args, "--for-rewrite=rebase");
 			/* we don't care if this copying failed */
-			run_command(&child);
-
-			if (post_rewrite_hook) {
-				struct child_process hook = CHILD_PROCESS_INIT;
-
-				hook.in = open(rebase_path_rewritten_list(),
-					O_RDONLY);
-				hook.stdout_to_stderr = 1;
-				hook.trace2_hook_name = "post-rewrite";
-				strvec_push(&hook.args, post_rewrite_hook);
-				strvec_push(&hook.args, "rebase");
-				/* we don't care if this hook failed */
-				run_command(&hook);
-			}
+			run_command(&notes_cp);
+
+			hook_opt.path_to_stdin = rebase_path_rewritten_list();
+			strvec_push(&hook_opt.args, "rebase");
+			run_hooks("post-rewrite", &hook_opt);
+			run_hooks_opt_clear(&hook_opt);
 		}
 		apply_autostash(rebase_path_autostash());
 
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 28/37] transport: convert pre-push hook to use config
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (26 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 27/37] hook: convert 'post-rewrite' hook to config Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 29/37] reference-transaction: look for hooks in config Emily Shaffer
                   ` (12 subsequent siblings)
  40 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using the hook.h:run_hooks API, pre-push hooks can be specified in
the config as well as in the hookdir.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt |  3 ++
 transport.c                | 59 +++++++++++---------------------------
 2 files changed, 20 insertions(+), 42 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 544238b381..489c93a7cb 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -279,6 +279,9 @@ If this hook exits with a non-zero status, `git push` will abort without
 pushing anything.  Information about why the push is rejected may be sent
 to the user by writing to standard error.
 
+Hooks executed during 'pre-push' will run in parallel, unless hook.jobs is
+configured to 1.
+
 [[pre-receive]]
 pre-receive
 ~~~~~~~~~~~
diff --git a/transport.c b/transport.c
index b13fab5dc3..286b73881b 100644
--- a/transport.c
+++ b/transport.c
@@ -22,6 +22,7 @@
 #include "protocol.h"
 #include "object-store.h"
 #include "color.h"
+#include "hook.h"
 
 static int transport_use_color = -1;
 static char transport_colors[][COLOR_MAXLEN] = {
@@ -1172,31 +1173,15 @@ static void die_with_unpushed_submodules(struct string_list *needs_pushing)
 static int run_pre_push_hook(struct transport *transport,
 			     struct ref *remote_refs)
 {
-	int ret = 0, x;
+	int ret = 0;
+	struct run_hooks_opt opt;
+	struct strbuf tmp = STRBUF_INIT;
 	struct ref *r;
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct strbuf buf;
-	const char *argv[4];
-
-	if (!(argv[0] = find_hook("pre-push")))
-		return 0;
-
-	argv[1] = transport->remote->name;
-	argv[2] = transport->url;
-	argv[3] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.trace2_hook_name = "pre-push";
-
-	if (start_command(&proc)) {
-		finish_command(&proc);
-		return -1;
-	}
-
-	sigchain_push(SIGPIPE, SIG_IGN);
+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
+	run_hooks_opt_init_async(&opt);
 
-	strbuf_init(&buf, 256);
+	strvec_push(&opt.args, transport->remote->name);
+	strvec_push(&opt.args, transport->url);
 
 	for (r = remote_refs; r; r = r->next) {
 		if (!r->peer_ref) continue;
@@ -1205,30 +1190,20 @@ static int run_pre_push_hook(struct transport *transport,
 		if (r->status == REF_STATUS_REJECT_REMOTE_UPDATED) continue;
 		if (r->status == REF_STATUS_UPTODATE) continue;
 
-		strbuf_reset(&buf);
-		strbuf_addf( &buf, "%s %s %s %s\n",
+		strbuf_reset(&tmp);
+		strbuf_addf(&tmp, "%s %s %s %s",
 			 r->peer_ref->name, oid_to_hex(&r->new_oid),
 			 r->name, oid_to_hex(&r->old_oid));
-
-		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
-			/* We do not mind if a hook does not read all refs. */
-			if (errno != EPIPE)
-				ret = -1;
-			break;
-		}
+		string_list_append(&to_stdin, tmp.buf);
 	}
 
-	strbuf_release(&buf);
-
-	x = close(proc.in);
-	if (!ret)
-		ret = x;
-
-	sigchain_pop(SIGPIPE);
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
 
-	x = finish_command(&proc);
-	if (!ret)
-		ret = x;
+	ret = run_hooks("pre-push", &opt);
+	run_hooks_opt_clear(&opt);
+	strbuf_release(&tmp);
+	string_list_clear(&to_stdin, 0);
 
 	return ret;
 }
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 29/37] reference-transaction: look for hooks in config
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (27 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 28/37] transport: convert pre-push hook to use config Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 30/37] receive-pack: convert 'update' hook to hook.h Emily Shaffer
                   ` (11 subsequent siblings)
  40 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using the hook.h library, reference-transaction hooks can be
specified in the config instead.

The expected output of the test is not fully updated to reflect the
absolute path of the hook called because the 'update' hook has not yet
been converted to use hook.h.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt       |  3 +++
 refs.c                           | 43 +++++++++++++-------------------
 t/t1416-ref-transaction-hooks.sh |  8 +++---
 3 files changed, 24 insertions(+), 30 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 489c93a7cb..dc8b7111d5 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -530,6 +530,9 @@ The exit status of the hook is ignored for any state except for the
 cause the transaction to be aborted. The hook will not be called with
 "aborted" state in that case.
 
+Hooks run during 'reference-transaction' will be run in parallel, unless
+hook.jobs is configured to 1.
+
 push-to-checkout
 ~~~~~~~~~~~~~~~~
 
diff --git a/refs.c b/refs.c
index a665ed5e10..4fccbac3e6 100644
--- a/refs.c
+++ b/refs.c
@@ -18,6 +18,7 @@
 #include "strvec.h"
 #include "repository.h"
 #include "sigchain.h"
+#include "hook.h"
 
 /*
  * List of all available backends
@@ -2061,47 +2062,37 @@ int ref_update_reject_duplicates(struct string_list *refnames,
 static int run_transaction_hook(struct ref_transaction *transaction,
 				const char *state)
 {
-	struct child_process proc = CHILD_PROCESS_INIT;
 	struct strbuf buf = STRBUF_INIT;
-	const char *hook;
+	struct run_hooks_opt opt;
+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int ret = 0, i;
+	char o[GIT_MAX_HEXSZ + 1], n[GIT_MAX_HEXSZ + 1];
 
-	hook = find_hook("reference-transaction");
-	if (!hook)
-		return ret;
-
-	strvec_pushl(&proc.args, hook, state, NULL);
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = "reference-transaction";
+	run_hooks_opt_init_async(&opt);
 
-	ret = start_command(&proc);
-	if (ret)
+	if (!hook_exists("reference-transaction", HOOKDIR_USE_CONFIG))
 		return ret;
 
-	sigchain_push(SIGPIPE, SIG_IGN);
+	strvec_push(&opt.args, state);
 
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		oid_to_hex_r(o, &update->old_oid);
+		oid_to_hex_r(n, &update->new_oid);
 
 		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
-			    oid_to_hex(&update->old_oid),
-			    oid_to_hex(&update->new_oid),
-			    update->refname);
-
-		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
-			if (errno != EPIPE)
-				ret = -1;
-			break;
-		}
+		strbuf_addf(&buf, "%s %s %s", o, n, update->refname);
+		string_list_append(&to_stdin, buf.buf);
 	}
 
-	close(proc.in);
-	sigchain_pop(SIGPIPE);
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
+
+	ret = run_hooks("reference-transaction", &opt);
+	run_hooks_opt_clear(&opt);
 	strbuf_release(&buf);
+	string_list_clear(&to_stdin, 0);
 
-	ret |= finish_command(&proc);
 	return ret;
 }
 
diff --git a/t/t1416-ref-transaction-hooks.sh b/t/t1416-ref-transaction-hooks.sh
index 6c941027a8..3a90a59143 100755
--- a/t/t1416-ref-transaction-hooks.sh
+++ b/t/t1416-ref-transaction-hooks.sh
@@ -125,11 +125,11 @@ test_expect_success 'interleaving hook calls succeed' '
 
 	cat >expect <<-EOF &&
 		hooks/update refs/tags/PRE $ZERO_OID $PRE_OID
-		hooks/reference-transaction prepared
-		hooks/reference-transaction committed
+		$(pwd)/target-repo.git/hooks/reference-transaction prepared
+		$(pwd)/target-repo.git/hooks/reference-transaction committed
 		hooks/update refs/tags/POST $ZERO_OID $POST_OID
-		hooks/reference-transaction prepared
-		hooks/reference-transaction committed
+		$(pwd)/target-repo.git/hooks/reference-transaction prepared
+		$(pwd)/target-repo.git/hooks/reference-transaction committed
 	EOF
 
 	git push ./target-repo.git PRE POST &&
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 30/37] receive-pack: convert 'update' hook to hook.h
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (28 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 29/37] reference-transaction: look for hooks in config Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 31/37] proc-receive: acquire hook list from hook.h Emily Shaffer
                   ` (10 subsequent siblings)
  40 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using hook.h to invoke the 'update' hook, now hooks can be specified
in the config in addition to the hookdir.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt       |  3 ++
 builtin/receive-pack.c           | 66 ++++++++++++++++++++++----------
 t/t1416-ref-transaction-hooks.sh |  4 +-
 3 files changed, 50 insertions(+), 23 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index dc8b7111d5..60fd43d1da 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -368,6 +368,9 @@ The default 'update' hook, when enabled--and with
 `hooks.allowunannotated` config option unset or set to false--prevents
 unannotated tags to be pushed.
 
+Hooks executed during 'update' are run in parallel, unless hook.jobs is
+configured to 1.
+
 [[proc-receive]]
 proc-receive
 ~~~~~~~~~~~~
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 234b70f0d1..b34a27a303 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -938,33 +938,57 @@ static int run_receive_hook(struct command *commands,
 	return status;
 }
 
-static int run_update_hook(struct command *cmd)
+static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
 {
-	const char *argv[5];
-	struct child_process proc = CHILD_PROCESS_INIT;
-	int code;
+	int keepalive_active = 0;
 
-	argv[0] = find_hook("update");
-	if (!argv[0])
-		return 0;
+	if (keepalive_in_sec <= 0)
+		use_keepalive = KEEPALIVE_NEVER;
+	if (use_keepalive == KEEPALIVE_ALWAYS)
+		keepalive_active = 1;
 
-	argv[1] = cmd->ref_name;
-	argv[2] = oid_to_hex(&cmd->old_oid);
-	argv[3] = oid_to_hex(&cmd->new_oid);
-	argv[4] = NULL;
+	/* send a keepalive if there is no data to write */
+	if (keepalive_active && !output->len) {
+		static const char buf[] = "0005\1";
+		write_or_die(1, buf, sizeof(buf) - 1);
+		return;
+	}
 
-	proc.no_stdin = 1;
-	proc.stdout_to_stderr = 1;
-	proc.err = use_sideband ? -1 : 0;
-	proc.argv = argv;
-	proc.trace2_hook_name = "update";
+	if (use_keepalive == KEEPALIVE_AFTER_NUL && !keepalive_active) {
+		const char *first_null = memchr(output->buf, '\0', output->len);
+		if (first_null) {
+			/* The null bit is excluded. */
+			size_t before_null = first_null - output->buf;
+			size_t after_null = output->len - (before_null + 1);
+			keepalive_active = 1;
+			send_sideband(1, 2, output->buf, before_null, use_sideband);
+			send_sideband(1, 2, first_null + 1, after_null, use_sideband);
+
+			return;
+		}
+	}
+
+	send_sideband(1, 2, output->buf, output->len, use_sideband);
+}
+
+static int run_update_hook(struct command *cmd)
+{
+	struct run_hooks_opt opt;
+	int code;
+	run_hooks_opt_init_async(&opt);
+
+	strvec_pushl(&opt.args,
+		     cmd->ref_name,
+		     oid_to_hex(&cmd->old_oid),
+		     oid_to_hex(&cmd->new_oid),
+		     NULL);
 
-	code = start_command(&proc);
-	if (code)
-		return code;
 	if (use_sideband)
-		copy_to_sideband(proc.err, -1, NULL);
-	return finish_command(&proc);
+		opt.consume_sideband = hook_output_to_sideband;
+
+	code = run_hooks("update", &opt);
+	run_hooks_opt_clear(&opt);
+	return code;
 }
 
 static struct command *find_command_by_refname(struct command *list,
diff --git a/t/t1416-ref-transaction-hooks.sh b/t/t1416-ref-transaction-hooks.sh
index 3a90a59143..0a3c3e4a86 100755
--- a/t/t1416-ref-transaction-hooks.sh
+++ b/t/t1416-ref-transaction-hooks.sh
@@ -124,10 +124,10 @@ test_expect_success 'interleaving hook calls succeed' '
 	EOF
 
 	cat >expect <<-EOF &&
-		hooks/update refs/tags/PRE $ZERO_OID $PRE_OID
+		$(pwd)/target-repo.git/hooks/update refs/tags/PRE $ZERO_OID $PRE_OID
 		$(pwd)/target-repo.git/hooks/reference-transaction prepared
 		$(pwd)/target-repo.git/hooks/reference-transaction committed
-		hooks/update refs/tags/POST $ZERO_OID $POST_OID
+		$(pwd)/target-repo.git/hooks/update refs/tags/POST $ZERO_OID $POST_OID
 		$(pwd)/target-repo.git/hooks/reference-transaction prepared
 		$(pwd)/target-repo.git/hooks/reference-transaction committed
 	EOF
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 31/37] proc-receive: acquire hook list from hook.h
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (29 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 30/37] receive-pack: convert 'update' hook to hook.h Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 32/37] post-update: use hook.h library Emily Shaffer
                   ` (9 subsequent siblings)
  40 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

The proc-receive hook differs from most other hooks Git invokes because
the hook and the parent Git process engage in bidirectional
communication via stdin/stdout. This bidirectional communication is
unsuitable for multiple hooks, whether they are in series or in
parallel, and is incompatible with run-command.h:run_processes_parallel:

- The proc-receive hook is intended to modify the state of the Git repo.
  From 'git help githooks':
    This [proc-receive] hook is responsible for updating the relevant
    references and reporting the results back to 'receive-pack'.
  This prevents parallelization and implies, at least, specific ordering
  of hook execution.
- The proc-receive hook can reject a push by aborting early with an
  error code. If a former hook ran through the entire push contents
  successfully but a later hook rejects some of the push, the repo may
  be left in a partially-updated (and corrupt) state.
- The callback model of the run_processes_parallel() API is unsuited to
  the current implementation of proc-receive, which loops through
  "send-receive-consider" with the child process. proc-receive today
  relies on stateful communication with the child process, which would be
  unwieldy to implement with callbacks and saved state.
- Additionally, run_processes_parallel() is designed to collate the
  output of many child processes into a single output (stderr or callback),
  and would require significant work to tell the caller which process sent
  the output, and indeed to collect any output before the child process
  has exited.

So, rather than using hook.h:run_hooks() to invoke the proc-receive
hook, receive-pack.c can learn to ask hook.h:hook_list() for the
location of a hook to run. This allows users to configure their
proc-receive in a global config for all repos if they want, or a local
config if they just don't want to use the hookdir. Because running more
than one proc-receive hook doesn't make sense from a repo state
perspective, we can explicitly ban configuring more than one
proc-receive hook at a time.

If a user wants to globally configure one proc-receive hook for most of
their repos, but override that hook in a single repo, they should use
'skip' to manually remove the global hook in their special repo:

~/.gitconfig:
[hook.proc-receive]
  command = /usr/bin/usual-proc-receive

~/special-repo/.git/config:
[hookcmd./usr/bin/usual-proc-receive]
  skip = true
[hook.proc-receive]
  command = /usr/bin/special-proc-receive

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt                |  4 ++
 builtin/receive-pack.c                    | 33 +++++++++++++++-
 t/t5411/test-0015-too-many-hooks-error.sh | 47 +++++++++++++++++++++++
 3 files changed, 82 insertions(+), 2 deletions(-)
 create mode 100644 t/t5411/test-0015-too-many-hooks-error.sh

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 60fd43d1da..c16353be2d 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -433,6 +433,10 @@ the input.  The exit status of the 'proc-receive' hook only determines
 the success or failure of the group of commands sent to it, unless
 atomic push is in use.
 
+It is forbidden to specify more than one hook for 'proc-receive'. If a
+globally-configured 'proc-receive' must be overridden, use
+'hookcmd.<global-hook>.skip = true' to ignore it.
+
 [[post-receive]]
 post-receive
 ~~~~~~~~~~~~
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index b34a27a303..e448956a32 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1146,11 +1146,40 @@ static int run_proc_receive_hook(struct command *commands,
 	int version = 0;
 	int code;
 
-	argv[0] = find_hook("proc-receive");
-	if (!argv[0]) {
+	struct strbuf hookname = STRBUF_INIT;
+	struct hook *proc_receive = NULL;
+	struct list_head *pos, *hooks;
+
+	strbuf_addstr(&hookname, "proc-receive");
+	hooks = hook_list(&hookname);
+
+	list_for_each(pos, hooks) {
+		if (proc_receive) {
+			rp_error("only one 'proc-receive' hook can be specified");
+			return -1;
+		}
+		proc_receive = list_entry(pos, struct hook, list);
+		/* check if the hookdir hook should be ignored */
+		if (proc_receive->from_hookdir) {
+			switch (configured_hookdir_opt()) {
+			case HOOKDIR_INTERACTIVE:
+			case HOOKDIR_NO:
+				proc_receive = NULL;
+				break;
+			default:
+				break;
+			}
+		}
+
+	}
+
+	if (!proc_receive) {
 		rp_error("cannot find hook 'proc-receive'");
 		return -1;
 	}
+
+
+	argv[0] = proc_receive->command.buf;
 	argv[1] = NULL;
 
 	proc.argv = argv;
diff --git a/t/t5411/test-0015-too-many-hooks-error.sh b/t/t5411/test-0015-too-many-hooks-error.sh
new file mode 100644
index 0000000000..2d64534510
--- /dev/null
+++ b/t/t5411/test-0015-too-many-hooks-error.sh
@@ -0,0 +1,47 @@
+test_expect_success "setup too  many proc-receive hooks (ok, $PROTOCOL)" '
+	write_script "proc-receive" <<-EOF &&
+	printf >&2 "# proc-receive hook\n"
+	test-tool proc-receive -v \
+		-r "ok refs/for/main/topic"
+	EOF
+
+	git -C "$upstream" config --add "hook.proc-receive.command" proc-receive &&
+	cp proc-receive "$upstream/hooks/proc-receive"
+'
+
+# Refs of upstream : main(A)
+# Refs of workbench: main(A)  tags/v123
+# git push         :                       next(A)  refs/for/main/topic(A)
+test_expect_success "proc-receive: reject more than one configured hook" '
+	test_must_fail git -C workbench push origin \
+		HEAD:next \
+		HEAD:refs/for/main/topic \
+		>out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	cat >expect <<-EOF &&
+	remote: # pre-receive hook
+	remote: pre-receive< <ZERO-OID> <COMMIT-A> refs/heads/next
+	remote: pre-receive< <ZERO-OID> <COMMIT-A> refs/for/main/topic
+	remote: error: only one "proc-receive" hook can be specified
+	remote: # post-receive hook
+	remote: post-receive< <ZERO-OID> <COMMIT-A> refs/heads/next
+	To <URL/of/upstream.git>
+	 * [new branch] HEAD -> next
+	 ! [remote rejected] HEAD -> refs/for/main/topic (fail to run proc-receive hook)
+	EOF
+	test_cmp expect actual &&
+	git -C "$upstream" show-ref >out &&
+	make_user_friendly_and_stable_output <out >actual &&
+	cat >expect <<-EOF &&
+	<COMMIT-A> refs/heads/main
+	<COMMIT-A> refs/heads/next
+	EOF
+	test_cmp expect actual
+'
+
+# Refs of upstream : main(A)             next(A)
+# Refs of workbench: main(A)  tags/v123
+test_expect_success "cleanup ($PROTOCOL)" '
+	git -C "$upstream" config --unset "hook.proc-receive.command" "proc-receive" &&
+	git -C "$upstream" update-ref -d refs/heads/next
+'
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 32/37] post-update: use hook.h library
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (30 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 31/37] proc-receive: acquire hook list from hook.h Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-12  9:14   ` Ævar Arnfjörð Bjarmason
  2021-03-11  2:10 ` [PATCH v8 33/37] receive-pack: convert receive hooks to hook.h Emily Shaffer
                   ` (8 subsequent siblings)
  40 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using run_hooks() instead of run_hook_le(), 'post-update' hooks can
be specified in the config as well as the hookdir.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt |  3 +++
 builtin/receive-pack.c     | 27 ++++++++-------------------
 2 files changed, 11 insertions(+), 19 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index c16353be2d..fe5381b95b 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -508,6 +508,9 @@ Both standard output and standard error output are forwarded to
 `git send-pack` on the other end, so you can simply `echo` messages
 for the user.
 
+Hooks run during 'post-update' will be run in parallel, unless hook.jobs is
+configured to 1.
+
 reference-transaction
 ~~~~~~~~~~~~~~~~~~~~~
 
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index e448956a32..955efbdf6d 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1688,33 +1688,22 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 static void run_update_post_hook(struct command *commands)
 {
 	struct command *cmd;
-	struct child_process proc = CHILD_PROCESS_INIT;
-	const char *hook;
-
-	hook = find_hook("post-update");
-	if (!hook)
-		return;
+	struct run_hooks_opt opt;
+	run_hooks_opt_init_async(&opt);
 
 	for (cmd = commands; cmd; cmd = cmd->next) {
 		if (cmd->error_string || cmd->did_not_exist)
 			continue;
-		if (!proc.args.nr)
-			strvec_push(&proc.args, hook);
-		strvec_push(&proc.args, cmd->ref_name);
+		strvec_push(&opt.args, cmd->ref_name);
 	}
-	if (!proc.args.nr)
+	if (!opt.args.nr)
 		return;
 
-	proc.no_stdin = 1;
-	proc.stdout_to_stderr = 1;
-	proc.err = use_sideband ? -1 : 0;
-	proc.trace2_hook_name = "post-update";
+	if (use_sideband)
+		opt.consume_sideband = hook_output_to_sideband;
 
-	if (!start_command(&proc)) {
-		if (use_sideband)
-			copy_to_sideband(proc.err, -1, NULL);
-		finish_command(&proc);
-	}
+	run_hooks("post-update", &opt);
+	run_hooks_opt_clear(&opt);
 }
 
 static void check_aliased_update_internal(struct command *cmd,
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 33/37] receive-pack: convert receive hooks to hook.h
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (31 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 32/37] post-update: use hook.h library Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 34/37] bugreport: use hook_exists instead of find_hook Emily Shaffer
                   ` (7 subsequent siblings)
  40 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using the hook.h library to run receive hooks, they can be specified
in the config as well as in the hookdir.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt |   5 +
 builtin/receive-pack.c     | 199 +++++++++++++++++--------------------
 2 files changed, 97 insertions(+), 107 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index fe5381b95b..b63054b947 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -323,6 +323,8 @@ will be set to zero, `GIT_PUSH_OPTION_COUNT=0`.
 See the section on "Quarantine Environment" in
 linkgit:git-receive-pack[1] for some caveats.
 
+Hooks executed during 'pre-receive' will not be parallelized.
+
 [[update]]
 update
 ~~~~~~
@@ -476,6 +478,9 @@ environment variables will not be set. If the client selects
 to use push options, but doesn't transmit any, the count variable
 will be set to zero, `GIT_PUSH_OPTION_COUNT=0`.
 
+Hooks executed during 'post-receive' are run in parallel, unless hook.jobs is
+configured to 1.
+
 [[post-update]]
 post-update
 ~~~~~~~~~~~
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 955efbdf6d..d124718d0b 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -748,7 +748,7 @@ static int check_cert_push_options(const struct string_list *push_options)
 	return retval;
 }
 
-static void prepare_push_cert_sha1(struct child_process *proc)
+static void prepare_push_cert_sha1(struct run_hooks_opt *opt)
 {
 	static int already_done;
 
@@ -772,110 +772,42 @@ static void prepare_push_cert_sha1(struct child_process *proc)
 		nonce_status = check_nonce(push_cert.buf, bogs);
 	}
 	if (!is_null_oid(&push_cert_oid)) {
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT=%s",
 			     oid_to_hex(&push_cert_oid));
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_SIGNER=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_SIGNER=%s",
 			     sigcheck.signer ? sigcheck.signer : "");
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_KEY=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_KEY=%s",
 			     sigcheck.key ? sigcheck.key : "");
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_STATUS=%c",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_STATUS=%c",
 			     sigcheck.result);
 		if (push_cert_nonce) {
-			strvec_pushf(&proc->env_array,
+			strvec_pushf(&opt->env,
 				     "GIT_PUSH_CERT_NONCE=%s",
 				     push_cert_nonce);
-			strvec_pushf(&proc->env_array,
+			strvec_pushf(&opt->env,
 				     "GIT_PUSH_CERT_NONCE_STATUS=%s",
 				     nonce_status);
 			if (nonce_status == NONCE_SLOP)
-				strvec_pushf(&proc->env_array,
+				strvec_pushf(&opt->env,
 					     "GIT_PUSH_CERT_NONCE_SLOP=%ld",
 					     nonce_stamp_slop);
 		}
 	}
 }
 
+struct receive_hook_feed_context {
+	struct command *cmd;
+	int skip_broken;
+};
+
 struct receive_hook_feed_state {
 	struct command *cmd;
 	struct ref_push_report *report;
 	int skip_broken;
 	struct strbuf buf;
-	const struct string_list *push_options;
 };
 
-typedef int (*feed_fn)(void *, const char **, size_t *);
-static int run_and_feed_hook(const char *hook_name, feed_fn feed,
-			     struct receive_hook_feed_state *feed_state)
-{
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct async muxer;
-	const char *argv[2];
-	int code;
-
-	argv[0] = find_hook(hook_name);
-	if (!argv[0])
-		return 0;
-
-	argv[1] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = hook_name;
-
-	if (feed_state->push_options) {
-		int i;
-		for (i = 0; i < feed_state->push_options->nr; i++)
-			strvec_pushf(&proc.env_array,
-				     "GIT_PUSH_OPTION_%d=%s", i,
-				     feed_state->push_options->items[i].string);
-		strvec_pushf(&proc.env_array, "GIT_PUSH_OPTION_COUNT=%d",
-			     feed_state->push_options->nr);
-	} else
-		strvec_pushf(&proc.env_array, "GIT_PUSH_OPTION_COUNT");
-
-	if (tmp_objdir)
-		strvec_pushv(&proc.env_array, tmp_objdir_env(tmp_objdir));
-
-	if (use_sideband) {
-		memset(&muxer, 0, sizeof(muxer));
-		muxer.proc = copy_to_sideband;
-		muxer.in = -1;
-		code = start_async(&muxer);
-		if (code)
-			return code;
-		proc.err = muxer.in;
-	}
-
-	prepare_push_cert_sha1(&proc);
-
-	code = start_command(&proc);
-	if (code) {
-		if (use_sideband)
-			finish_async(&muxer);
-		return code;
-	}
-
-	sigchain_push(SIGPIPE, SIG_IGN);
-
-	while (1) {
-		const char *buf;
-		size_t n;
-		if (feed(feed_state, &buf, &n))
-			break;
-		if (write_in_full(proc.in, buf, n) < 0)
-			break;
-	}
-	close(proc.in);
-	if (use_sideband)
-		finish_async(&muxer);
-
-	sigchain_pop(SIGPIPE);
-
-	return finish_command(&proc);
-}
-
-static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
+static int feed_receive_hook(void *state_)
 {
 	struct receive_hook_feed_state *state = state_;
 	struct command *cmd = state->cmd;
@@ -884,9 +816,7 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
 	       state->skip_broken && (cmd->error_string || cmd->did_not_exist))
 		cmd = cmd->next;
 	if (!cmd)
-		return -1; /* EOF */
-	if (!bufp)
-		return 0; /* OK, can feed something. */
+		return 1; /* EOF - close the pipe*/
 	strbuf_reset(&state->buf);
 	if (!state->report)
 		state->report = cmd->report;
@@ -910,32 +840,36 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
 			    cmd->ref_name);
 		state->cmd = cmd->next;
 	}
-	if (bufp) {
-		*bufp = state->buf.buf;
-		*sizep = state->buf.len;
-	}
 	return 0;
 }
 
-static int run_receive_hook(struct command *commands,
-			    const char *hook_name,
-			    int skip_broken,
-			    const struct string_list *push_options)
+static int feed_receive_hook_cb(struct strbuf *pipe, void *pp_cb, void *pp_task_cb)
 {
-	struct receive_hook_feed_state state;
-	int status;
-
-	strbuf_init(&state.buf, 0);
-	state.cmd = commands;
-	state.skip_broken = skip_broken;
-	state.report = NULL;
-	if (feed_receive_hook(&state, NULL, NULL))
-		return 0;
-	state.cmd = commands;
-	state.push_options = push_options;
-	status = run_and_feed_hook(hook_name, feed_receive_hook, &state);
-	strbuf_release(&state.buf);
-	return status;
+	struct hook *hook = pp_task_cb;
+	struct receive_hook_feed_state *feed_state = hook->feed_pipe_cb_data;
+	int rc;
+
+	/* first-time setup */
+	if (!feed_state) {
+		struct hook_cb_data *hook_cb = pp_cb;
+		struct run_hooks_opt *opt = hook_cb->options;
+		struct receive_hook_feed_context *ctx = opt->feed_pipe_ctx;
+		if (!ctx)
+			BUG("run_hooks_opt.feed_pipe_ctx required for receive hook");
+
+		feed_state = xmalloc(sizeof(struct receive_hook_feed_state));
+		strbuf_init(&feed_state->buf, 0);
+		feed_state->cmd = ctx->cmd;
+		feed_state->skip_broken = ctx->skip_broken;
+		feed_state->report = NULL;
+
+		hook->feed_pipe_cb_data = feed_state;
+	}
+
+	rc = feed_receive_hook(feed_state);
+	if (!rc)
+		strbuf_addbuf(pipe, &feed_state->buf);
+	return rc;
 }
 
 static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
@@ -971,6 +905,57 @@ static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
 	send_sideband(1, 2, output->buf, output->len, use_sideband);
 }
 
+static int run_receive_hook(struct command *commands,
+			    const char *hook_name,
+			    int skip_broken,
+			    const struct string_list *push_options)
+{
+	struct run_hooks_opt opt;
+	struct receive_hook_feed_context ctx;
+	int rc;
+	struct command *iter = commands;
+
+	run_hooks_opt_init_async(&opt);
+
+	/* if there are no valid commands, don't invoke the hook at all. */
+	while (iter && skip_broken && (iter->error_string || iter->did_not_exist))
+		iter = iter->next;
+	if (!iter)
+		return 0;
+
+	/* pre-receive hooks should run in series as the hook updates refs */
+	if (!strcmp(hook_name, "pre-receive"))
+		opt.jobs = 1;
+
+	if (push_options) {
+		int i;
+		for (i = 0; i < push_options->nr; i++)
+			strvec_pushf(&opt.env, "GIT_PUSH_OPTION_%d=%s", i,
+				     push_options->items[i].string);
+		strvec_pushf(&opt.env, "GIT_PUSH_OPTION_COUNT=%d", push_options->nr);
+	} else
+		strvec_push(&opt.env, "GIT_PUSH_OPTION_COUNT");
+
+	if (tmp_objdir)
+		strvec_pushv(&opt.env, tmp_objdir_env(tmp_objdir));
+
+	prepare_push_cert_sha1(&opt);
+
+	/* set up sideband printer */
+	if (use_sideband)
+		opt.consume_sideband = hook_output_to_sideband;
+
+	/* set up stdin callback */
+	ctx.cmd = commands;
+	ctx.skip_broken = skip_broken;
+	opt.feed_pipe = feed_receive_hook_cb;
+	opt.feed_pipe_ctx = &ctx;
+
+	rc = run_hooks(hook_name, &opt);
+	run_hooks_opt_clear(&opt);
+	return rc;
+}
+
 static int run_update_hook(struct command *cmd)
 {
 	struct run_hooks_opt opt;
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 34/37] bugreport: use hook_exists instead of find_hook
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (32 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 33/37] receive-pack: convert receive hooks to hook.h Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 35/37] git-send-email: use 'git hook run' for 'sendemail-validate' Emily Shaffer
                   ` (6 subsequent siblings)
  40 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using the helper in hook.h instead of the one in run-command.h, we
can also check whether a hook exists in the config - not just whether it
exists in the hookdir.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 builtin/bugreport.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/builtin/bugreport.c b/builtin/bugreport.c
index ad3cc9c02f..eac3726527 100644
--- a/builtin/bugreport.c
+++ b/builtin/bugreport.c
@@ -3,7 +3,7 @@
 #include "strbuf.h"
 #include "help.h"
 #include "compat/compiler.h"
-#include "run-command.h"
+#include "hook.h"
 
 
 static void get_system_info(struct strbuf *sys_info)
@@ -82,7 +82,7 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 
 	for (i = 0; i < ARRAY_SIZE(hook); i++)
-		if (find_hook(hook[i]))
+		if (hook_exists(hook[i], HOOKDIR_USE_CONFIG))
 			strbuf_addf(hook_info, "%s\n", hook[i]);
 }
 
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 35/37] git-send-email: use 'git hook run' for 'sendemail-validate'
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (33 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 34/37] bugreport: use hook_exists instead of find_hook Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-12  9:21   ` Ævar Arnfjörð Bjarmason
  2021-03-12 23:29   ` [PATCH v8 35/37] git-send-email: use 'git hook run' for 'sendemail-validate' Emily Shaffer
  2021-03-11  2:10 ` [PATCH v8 36/37] run-command: stop thinking about hooks Emily Shaffer
                   ` (5 subsequent siblings)
  40 siblings, 2 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using the new 'git hook run' subcommand to run 'sendemail-validate',
we can reduce the boilerplate needed to run this hook in perl. Using
config-based hooks also allows us to run 'sendemail-validate' hooks that
were configured globally when running 'git send-email' from outside of a
Git directory, alongside other benefits like multihooks and
parallelization.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 git-send-email.perl   | 21 ++++-----------------
 t/t9001-send-email.sh | 11 +----------
 2 files changed, 5 insertions(+), 27 deletions(-)

diff --git a/git-send-email.perl b/git-send-email.perl
index 1f425c0809..73e1e0b51a 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -1941,23 +1941,10 @@ sub unique_email_list {
 sub validate_patch {
 	my ($fn, $xfer_encoding) = @_;
 
-	if ($repo) {
-		my $validate_hook = catfile(catdir($repo->repo_path(), 'hooks'),
-					    'sendemail-validate');
-		my $hook_error;
-		if (-x $validate_hook) {
-			my $target = abs_path($fn);
-			# The hook needs a correct cwd and GIT_DIR.
-			my $cwd_save = cwd();
-			chdir($repo->wc_path() or $repo->repo_path())
-				or die("chdir: $!");
-			local $ENV{"GIT_DIR"} = $repo->repo_path();
-			$hook_error = "rejected by sendemail-validate hook"
-				if system($validate_hook, $target);
-			chdir($cwd_save) or die("chdir: $!");
-		}
-		return $hook_error if $hook_error;
-	}
+	my $target = abs_path($fn);
+	return "rejected by sendemail-validate hook"
+		if system(("git", "hook", "run", "sendemail-validate", "-a",
+				$target));
 
 	# Any long lines will be automatically fixed if we use a suitable transfer
 	# encoding.
diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index 4eee9c3dcb..456b471c5c 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -2101,16 +2101,7 @@ test_expect_success $PREREQ 'invoke hook' '
 	mkdir -p .git/hooks &&
 
 	write_script .git/hooks/sendemail-validate <<-\EOF &&
-	# test that we have the correct environment variable, pwd, and
-	# argument
-	case "$GIT_DIR" in
-	*.git)
-		true
-		;;
-	*)
-		false
-		;;
-	esac &&
+	# test that we have the correct argument
 	test -f 0001-add-main.patch &&
 	grep "add main" "$1"
 	EOF
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 36/37] run-command: stop thinking about hooks
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (34 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 35/37] git-send-email: use 'git hook run' for 'sendemail-validate' Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-12  9:23   ` Ævar Arnfjörð Bjarmason
  2021-03-11  2:10 ` [PATCH v8 37/37] docs: unify githooks and git-hook manpages Emily Shaffer
                   ` (4 subsequent siblings)
  40 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

hook.h has replaced all run-command.h hook-related functionality.
run-command.h:run_hooks_le/ve and find_hook are no longer used anywhere
in the codebase. So, let's delete the dead code - or, in the one case
where it's still needed, move it to an internal function in hook.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 hook.c        | 39 ++++++++++++++++++++++++++++--
 run-command.c | 66 ---------------------------------------------------
 run-command.h | 24 -------------------
 3 files changed, 37 insertions(+), 92 deletions(-)

diff --git a/hook.c b/hook.c
index 2322720ffe..7f6f3b9a61 100644
--- a/hook.c
+++ b/hook.c
@@ -212,6 +212,41 @@ static int should_include_hookdir(const char *path, enum hookdir_opt cfg)
 	}
 }
 
+static const char *find_legacy_hook(const char *name)
+{
+	static struct strbuf path = STRBUF_INIT;
+
+	strbuf_reset(&path);
+	strbuf_git_path(&path, "hooks/%s", name);
+	if (access(path.buf, X_OK) < 0) {
+		int err = errno;
+
+#ifdef STRIP_EXTENSION
+		strbuf_addstr(&path, STRIP_EXTENSION);
+		if (access(path.buf, X_OK) >= 0)
+			return path.buf;
+		if (errno == EACCES)
+			err = errno;
+#endif
+
+		if (err == EACCES && advice_ignored_hook) {
+			static struct string_list advise_given = STRING_LIST_INIT_DUP;
+
+			if (!string_list_lookup(&advise_given, name)) {
+				string_list_insert(&advise_given, name);
+				advise(_("The '%s' hook was ignored because "
+					 "it's not set as executable.\n"
+					 "You can disable this warning with "
+					 "`git config advice.ignoredHook false`."),
+				       path.buf);
+			}
+		}
+		return NULL;
+	}
+	return path.buf;
+}
+
+
 struct list_head* hook_list(const struct strbuf* hookname)
 {
 	struct strbuf hook_key = STRBUF_INIT;
@@ -228,7 +263,7 @@ struct list_head* hook_list(const struct strbuf* hookname)
 	git_config(hook_config_lookup, &cb_data);
 
 	if (have_git_dir()) {
-		const char *legacy_hook_path = find_hook(hookname->buf);
+		const char *legacy_hook_path = find_legacy_hook(hookname->buf);
 
 		/* Unconditionally add legacy hook, but annotate it. */
 		if (legacy_hook_path) {
@@ -277,7 +312,7 @@ int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir)
 	could_run_hookdir = (should_run_hookdir == HOOKDIR_INTERACTIVE ||
 				should_run_hookdir == HOOKDIR_WARN ||
 				should_run_hookdir == HOOKDIR_YES)
-				&& !!find_hook(hookname);
+				&& !!find_legacy_hook(hookname);
 
 	strbuf_addf(&hook_key, "hook.%s.command", hookname);
 
diff --git a/run-command.c b/run-command.c
index 36a4edbacf..837415131d 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1320,72 +1320,6 @@ int async_with_fork(void)
 #endif
 }
 
-const char *find_hook(const char *name)
-{
-	static struct strbuf path = STRBUF_INIT;
-
-	strbuf_reset(&path);
-	strbuf_git_path(&path, "hooks/%s", name);
-	if (access(path.buf, X_OK) < 0) {
-		int err = errno;
-
-#ifdef STRIP_EXTENSION
-		strbuf_addstr(&path, STRIP_EXTENSION);
-		if (access(path.buf, X_OK) >= 0)
-			return path.buf;
-		if (errno == EACCES)
-			err = errno;
-#endif
-
-		if (err == EACCES && advice_ignored_hook) {
-			static struct string_list advise_given = STRING_LIST_INIT_DUP;
-
-			if (!string_list_lookup(&advise_given, name)) {
-				string_list_insert(&advise_given, name);
-				advise(_("The '%s' hook was ignored because "
-					 "it's not set as executable.\n"
-					 "You can disable this warning with "
-					 "`git config advice.ignoredHook false`."),
-				       path.buf);
-			}
-		}
-		return NULL;
-	}
-	return path.buf;
-}
-
-int run_hook_ve(const char *const *env, const char *name, va_list args)
-{
-	struct child_process hook = CHILD_PROCESS_INIT;
-	const char *p;
-
-	p = find_hook(name);
-	if (!p)
-		return 0;
-
-	strvec_push(&hook.args, p);
-	while ((p = va_arg(args, const char *)))
-		strvec_push(&hook.args, p);
-	hook.env = env;
-	hook.no_stdin = 1;
-	hook.stdout_to_stderr = 1;
-	hook.trace2_hook_name = name;
-
-	return run_command(&hook);
-}
-
-int run_hook_le(const char *const *env, const char *name, ...)
-{
-	va_list args;
-	int ret;
-
-	va_start(args, name);
-	ret = run_hook_ve(env, name, args);
-	va_end(args);
-
-	return ret;
-}
-
 struct io_pump {
 	/* initialized by caller */
 	int fd;
diff --git a/run-command.h b/run-command.h
index ebc4a95a94..7150da851a 100644
--- a/run-command.h
+++ b/run-command.h
@@ -201,30 +201,6 @@ int finish_command_in_signal(struct child_process *);
  */
 int run_command(struct child_process *);
 
-/*
- * Returns the path to the hook file, or NULL if the hook is missing
- * or disabled. Note that this points to static storage that will be
- * overwritten by further calls to find_hook and run_hook_*.
- */
-const char *find_hook(const char *name);
-
-/**
- * Run a hook.
- * The first argument is a pathname to an index file, or NULL
- * if the hook uses the default index file or no index is needed.
- * The second argument is the name of the hook.
- * The further arguments correspond to the hook arguments.
- * The last argument has to be NULL to terminate the arguments list.
- * If the hook does not exist or is not executable, the return
- * value will be zero.
- * If it is executable, the hook will be executed and the exit
- * status of the hook is returned.
- * On execution, .stdout_to_stderr and .no_stdin will be set.
- */
-LAST_ARG_MUST_BE_NULL
-int run_hook_le(const char *const *env, const char *name, ...);
-int run_hook_ve(const char *const *env, const char *name, va_list args);
-
 /*
  * Trigger an auto-gc
  */
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v8 37/37] docs: unify githooks and git-hook manpages
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (35 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 36/37] run-command: stop thinking about hooks Emily Shaffer
@ 2021-03-11  2:10 ` Emily Shaffer
  2021-03-12  9:29   ` Ævar Arnfjörð Bjarmason
  2021-04-07  2:36   ` Junio C Hamano
  2021-03-11 22:26 ` [PATCH v8 00/37] config-based hooks Junio C Hamano
                   ` (3 subsequent siblings)
  40 siblings, 2 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-11  2:10 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By showing the list of all hooks in 'git help hook' for users to refer
to, 'git help hook' becomes a one-stop shop for hook authorship. Since
some may still have muscle memory for 'git help githooks', though,
reference the 'git hook' commands and otherwise don't remove content.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-hook.txt     |  11 +
 Documentation/githooks.txt     | 716 +--------------------------------
 Documentation/native-hooks.txt | 708 ++++++++++++++++++++++++++++++++
 3 files changed, 724 insertions(+), 711 deletions(-)
 create mode 100644 Documentation/native-hooks.txt

diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 81b8e94994..4ad31ac360 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -17,6 +17,13 @@ DESCRIPTION
 You can list and run configured hooks with this command. Later, you will be able
 to add and modify hooks with this command.
 
+In general, when instructions suggest adding a script to
+`.git/hooks/<something>`, you can specify it in the config instead by running
+`git config --add hook.<something>.command <path-to-script>` - this way you can
+share the script between multiple repos. That is, `cp ~/my-script.sh
+~/project/.git/hooks/pre-commit` would become `git config --add
+hook.pre-commit.command ~/my-script.sh`.
+
 This command parses the default configuration files for sections `hook` and
 `hookcmd`. `hook` is used to describe the commands which will be run during a
 particular hook event; commands are run in the order Git encounters them during
@@ -145,6 +152,10 @@ CONFIGURATION
 -------------
 include::config/hook.txt[]
 
+HOOKS
+-----
+include::native-hooks.txt[]
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index b63054b947..9a25dfdc3f 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -7,15 +7,16 @@ githooks - Hooks used by Git
 
 SYNOPSIS
 --------
+'git hook'
 $GIT_DIR/hooks/* (or \`git config core.hooksPath`/*)
 
 
 DESCRIPTION
 -----------
 
-Hooks are programs you can place in a hooks directory to trigger
-actions at certain points in git's execution. Hooks that don't have
-the executable bit set are ignored.
+Hooks are programs you can specify in your config (see linkgit:git-hook[1]) or
+place in a hooks directory to trigger actions at certain points in git's
+execution. Hooks that don't have the executable bit set are ignored.
 
 By default the hooks directory is `$GIT_DIR/hooks`, but that can be
 changed via the `core.hooksPath` configuration variable (see
@@ -41,714 +42,7 @@ The currently supported hooks are described below.
 
 HOOKS
 -----
-
-applypatch-msg
-~~~~~~~~~~~~~~
-
-This hook is invoked by linkgit:git-am[1].  It takes a single
-parameter, the name of the file that holds the proposed commit
-log message.  Exiting with a non-zero status causes `git am` to abort
-before applying the patch.
-
-The hook is allowed to edit the message file in place, and can
-be used to normalize the message into some project standard
-format. It can also be used to refuse the commit after inspecting
-the message file.
-
-The default 'applypatch-msg' hook, when enabled, runs the
-'commit-msg' hook, if the latter is enabled.
-
-Hooks run during 'applypatch-msg' will not be parallelized, because hooks are
-expected to edit the file holding the commit log message.
-
-pre-applypatch
-~~~~~~~~~~~~~~
-
-This hook is invoked by linkgit:git-am[1].  It takes no parameter, and is
-invoked after the patch is applied, but before a commit is made.
-
-If it exits with non-zero status, then the working tree will not be
-committed after applying the patch.
-
-It can be used to inspect the current working tree and refuse to
-make a commit if it does not pass certain test.
-
-The default 'pre-applypatch' hook, when enabled, runs the
-'pre-commit' hook, if the latter is enabled.
-
-Hooks run during 'pre-applypatch' will be run in parallel, unless hook.jobs is
-configured to 1.
-
-post-applypatch
-~~~~~~~~~~~~~~~
-
-This hook is invoked by linkgit:git-am[1].  It takes no parameter,
-and is invoked after the patch is applied and a commit is made.
-
-This hook is meant primarily for notification, and cannot affect
-the outcome of `git am`.
-
-Hooks run during 'post-applypatch' will be run in parallel, unless hook.jobs is
-configured to 1.
-
-pre-commit
-~~~~~~~~~~
-
-This hook is invoked by linkgit:git-commit[1], and can be bypassed
-with the `--no-verify` option.  It takes no parameters, and is
-invoked before obtaining the proposed commit log message and
-making a commit.  Exiting with a non-zero status from this script
-causes the `git commit` command to abort before creating a commit.
-
-The default 'pre-commit' hook, when enabled, catches introduction
-of lines with trailing whitespaces and aborts the commit when
-such a line is found.
-
-All the `git commit` hooks are invoked with the environment
-variable `GIT_EDITOR=:` if the command will not bring up an editor
-to modify the commit message.
-
-The default 'pre-commit' hook, when enabled--and with the
-`hooks.allownonascii` config option unset or set to false--prevents
-the use of non-ASCII filenames.
-
-Hooks executed during 'pre-commit' will not be parallelized.
-
-pre-merge-commit
-~~~~~~~~~~~~~~~~
-
-This hook is invoked by linkgit:git-merge[1], and can be bypassed
-with the `--no-verify` option.  It takes no parameters, and is
-invoked after the merge has been carried out successfully and before
-obtaining the proposed commit log message to
-make a commit.  Exiting with a non-zero status from this script
-causes the `git merge` command to abort before creating a commit.
-
-The default 'pre-merge-commit' hook, when enabled, runs the
-'pre-commit' hook, if the latter is enabled.
-
-This hook is invoked with the environment variable
-`GIT_EDITOR=:` if the command will not bring up an editor
-to modify the commit message.
-
-If the merge cannot be carried out automatically, the conflicts
-need to be resolved and the result committed separately (see
-linkgit:git-merge[1]). At that point, this hook will not be executed,
-but the 'pre-commit' hook will, if it is enabled.
-
-Hooks executed during 'pre-merge-commit' will not be parallelized.
-
-prepare-commit-msg
-~~~~~~~~~~~~~~~~~~
-
-This hook is invoked by linkgit:git-commit[1] right after preparing the
-default log message, and before the editor is started.
-
-It takes one to three parameters.  The first is the name of the file
-that contains the commit log message.  The second is the source of the commit
-message, and can be: `message` (if a `-m` or `-F` option was
-given); `template` (if a `-t` option was given or the
-configuration option `commit.template` is set); `merge` (if the
-commit is a merge or a `.git/MERGE_MSG` file exists); `squash`
-(if a `.git/SQUASH_MSG` file exists); or `commit`, followed by
-a commit SHA-1 (if a `-c`, `-C` or `--amend` option was given).
-
-If the exit status is non-zero, `git commit` will abort.
-
-The purpose of the hook is to edit the message file in place, and
-it is not suppressed by the `--no-verify` option.  A non-zero exit
-means a failure of the hook and aborts the commit.  It should not
-be used as replacement for pre-commit hook.
-
-The sample `prepare-commit-msg` hook that comes with Git removes the
-help message found in the commented portion of the commit template.
-
-Hooks executed during 'prepare-commit-msg' will not be parallelized, because
-hooks are expected to edit the file containing the commit log message.
-
-commit-msg
-~~~~~~~~~~
-
-This hook is invoked by linkgit:git-commit[1] and linkgit:git-merge[1], and can be
-bypassed with the `--no-verify` option.  It takes a single parameter,
-the name of the file that holds the proposed commit log message.
-Exiting with a non-zero status causes the command to abort.
-
-The hook is allowed to edit the message file in place, and can be used
-to normalize the message into some project standard format. It
-can also be used to refuse the commit after inspecting the message
-file.
-
-The default 'commit-msg' hook, when enabled, detects duplicate
-`Signed-off-by` trailers, and aborts the commit if one is found.
-
-Hooks executed during 'commit-msg' will not be parallelized, because hooks are
-expected to edit the file containing the proposed commit log message.
-
-post-commit
-~~~~~~~~~~~
-
-This hook is invoked by linkgit:git-commit[1]. It takes no parameters, and is
-invoked after a commit is made.
-
-This hook is meant primarily for notification, and cannot affect
-the outcome of `git commit`.
-
-Hooks executed during 'post-commit' will run in parallel, unless hook.jobs is
-configured to 1.
-
-pre-rebase
-~~~~~~~~~~
-
-This hook is called by linkgit:git-rebase[1] and can be used to prevent a
-branch from getting rebased.  The hook may be called with one or
-two parameters.  The first parameter is the upstream from which
-the series was forked.  The second parameter is the branch being
-rebased, and is not set when rebasing the current branch.
-
-Hooks executed during 'pre-rebase' will run in parallel, unless hook.jobs is
-configured to 1.
-
-post-checkout
-~~~~~~~~~~~~~
-
-This hook is invoked when a linkgit:git-checkout[1] or
-linkgit:git-switch[1] is run after having updated the
-worktree.  The hook is given three parameters: the ref of the previous HEAD,
-the ref of the new HEAD (which may or may not have changed), and a flag
-indicating whether the checkout was a branch checkout (changing branches,
-flag=1) or a file checkout (retrieving a file from the index, flag=0).
-This hook cannot affect the outcome of `git switch` or `git checkout`,
-other than that the hook's exit status becomes the exit status of
-these two commands.
-
-It is also run after linkgit:git-clone[1], unless the `--no-checkout` (`-n`) option is
-used. The first parameter given to the hook is the null-ref, the second the
-ref of the new HEAD and the flag is always 1. Likewise for `git worktree add`
-unless `--no-checkout` is used.
-
-This hook can be used to perform repository validity checks, auto-display
-differences from the previous HEAD if different, or set working dir metadata
-properties.
-
-Hooks executed during 'post-checkout' will not be parallelized.
-
-post-merge
-~~~~~~~~~~
-
-This hook is invoked by linkgit:git-merge[1], which happens when a `git pull`
-is done on a local repository.  The hook takes a single parameter, a status
-flag specifying whether or not the merge being done was a squash merge.
-This hook cannot affect the outcome of `git merge` and is not executed,
-if the merge failed due to conflicts.
-
-This hook can be used in conjunction with a corresponding pre-commit hook to
-save and restore any form of metadata associated with the working tree
-(e.g.: permissions/ownership, ACLS, etc).  See contrib/hooks/setgitperms.perl
-for an example of how to do this.
-
-Hooks executed during 'post-merge' will run in parallel, unless hook.jobs is
-configured to 1.
-
-pre-push
-~~~~~~~~
-
-This hook is called by linkgit:git-push[1] and can be used to prevent
-a push from taking place.  The hook is called with two parameters
-which provide the name and location of the destination remote, if a
-named remote is not being used both values will be the same.
-
-Information about what is to be pushed is provided on the hook's standard
-input with lines of the form:
-
-  <local ref> SP <local sha1> SP <remote ref> SP <remote sha1> LF
-
-For instance, if the command +git push origin master:foreign+ were run the
-hook would receive a line like the following:
-
-  refs/heads/master 67890 refs/heads/foreign 12345
-
-although the full, 40-character SHA-1s would be supplied.  If the foreign ref
-does not yet exist the `<remote SHA-1>` will be 40 `0`.  If a ref is to be
-deleted, the `<local ref>` will be supplied as `(delete)` and the `<local
-SHA-1>` will be 40 `0`.  If the local commit was specified by something other
-than a name which could be expanded (such as `HEAD~`, or a SHA-1) it will be
-supplied as it was originally given.
-
-If this hook exits with a non-zero status, `git push` will abort without
-pushing anything.  Information about why the push is rejected may be sent
-to the user by writing to standard error.
-
-Hooks executed during 'pre-push' will run in parallel, unless hook.jobs is
-configured to 1.
-
-[[pre-receive]]
-pre-receive
-~~~~~~~~~~~
-
-This hook is invoked by linkgit:git-receive-pack[1] when it reacts to
-`git push` and updates reference(s) in its repository.
-Just before starting to update refs on the remote repository, the
-pre-receive hook is invoked.  Its exit status determines the success
-or failure of the update.
-
-This hook executes once for the receive operation. It takes no
-arguments, but for each ref to be updated it receives on standard
-input a line of the format:
-
-  <old-value> SP <new-value> SP <ref-name> LF
-
-where `<old-value>` is the old object name stored in the ref,
-`<new-value>` is the new object name to be stored in the ref and
-`<ref-name>` is the full name of the ref.
-When creating a new ref, `<old-value>` is 40 `0`.
-
-If the hook exits with non-zero status, none of the refs will be
-updated. If the hook exits with zero, updating of individual refs can
-still be prevented by the <<update,'update'>> hook.
-
-Both standard output and standard error output are forwarded to
-`git send-pack` on the other end, so you can simply `echo` messages
-for the user.
-
-The number of push options given on the command line of
-`git push --push-option=...` can be read from the environment
-variable `GIT_PUSH_OPTION_COUNT`, and the options themselves are
-found in `GIT_PUSH_OPTION_0`, `GIT_PUSH_OPTION_1`,...
-If it is negotiated to not use the push options phase, the
-environment variables will not be set. If the client selects
-to use push options, but doesn't transmit any, the count variable
-will be set to zero, `GIT_PUSH_OPTION_COUNT=0`.
-
-See the section on "Quarantine Environment" in
-linkgit:git-receive-pack[1] for some caveats.
-
-Hooks executed during 'pre-receive' will not be parallelized.
-
-[[update]]
-update
-~~~~~~
-
-This hook is invoked by linkgit:git-receive-pack[1] when it reacts to
-`git push` and updates reference(s) in its repository.
-Just before updating the ref on the remote repository, the update hook
-is invoked.  Its exit status determines the success or failure of
-the ref update.
-
-The hook executes once for each ref to be updated, and takes
-three parameters:
-
- - the name of the ref being updated,
- - the old object name stored in the ref,
- - and the new object name to be stored in the ref.
-
-A zero exit from the update hook allows the ref to be updated.
-Exiting with a non-zero status prevents `git receive-pack`
-from updating that ref.
-
-This hook can be used to prevent 'forced' update on certain refs by
-making sure that the object name is a commit object that is a
-descendant of the commit object named by the old object name.
-That is, to enforce a "fast-forward only" policy.
-
-It could also be used to log the old..new status.  However, it
-does not know the entire set of branches, so it would end up
-firing one e-mail per ref when used naively, though.  The
-<<post-receive,'post-receive'>> hook is more suited to that.
-
-In an environment that restricts the users' access only to git
-commands over the wire, this hook can be used to implement access
-control without relying on filesystem ownership and group
-membership. See linkgit:git-shell[1] for how you might use the login
-shell to restrict the user's access to only git commands.
-
-Both standard output and standard error output are forwarded to
-`git send-pack` on the other end, so you can simply `echo` messages
-for the user.
-
-The default 'update' hook, when enabled--and with
-`hooks.allowunannotated` config option unset or set to false--prevents
-unannotated tags to be pushed.
-
-Hooks executed during 'update' are run in parallel, unless hook.jobs is
-configured to 1.
-
-[[proc-receive]]
-proc-receive
-~~~~~~~~~~~~
-
-This hook is invoked by linkgit:git-receive-pack[1].  If the server has
-set the multi-valued config variable `receive.procReceiveRefs`, and the
-commands sent to 'receive-pack' have matching reference names, these
-commands will be executed by this hook, instead of by the internal
-`execute_commands()` function.  This hook is responsible for updating
-the relevant references and reporting the results back to 'receive-pack'.
-
-This hook executes once for the receive operation.  It takes no
-arguments, but uses a pkt-line format protocol to communicate with
-'receive-pack' to read commands, push-options and send results.  In the
-following example for the protocol, the letter 'S' stands for
-'receive-pack' and the letter 'H' stands for this hook.
-
-    # Version and features negotiation.
-    S: PKT-LINE(version=1\0push-options atomic...)
-    S: flush-pkt
-    H: PKT-LINE(version=1\0push-options...)
-    H: flush-pkt
-
-    # Send commands from server to the hook.
-    S: PKT-LINE(<old-oid> <new-oid> <ref>)
-    S: ... ...
-    S: flush-pkt
-    # Send push-options only if the 'push-options' feature is enabled.
-    S: PKT-LINE(push-option)
-    S: ... ...
-    S: flush-pkt
-
-    # Receive result from the hook.
-    # OK, run this command successfully.
-    H: PKT-LINE(ok <ref>)
-    # NO, I reject it.
-    H: PKT-LINE(ng <ref> <reason>)
-    # Fall through, let 'receive-pack' to execute it.
-    H: PKT-LINE(ok <ref>)
-    H: PKT-LINE(option fall-through)
-    # OK, but has an alternate reference.  The alternate reference name
-    # and other status can be given in option directives.
-    H: PKT-LINE(ok <ref>)
-    H: PKT-LINE(option refname <refname>)
-    H: PKT-LINE(option old-oid <old-oid>)
-    H: PKT-LINE(option new-oid <new-oid>)
-    H: PKT-LINE(option forced-update)
-    H: ... ...
-    H: flush-pkt
-
-Each command for the 'proc-receive' hook may point to a pseudo-reference
-and always has a zero-old as its old-oid, while the 'proc-receive' hook
-may update an alternate reference and the alternate reference may exist
-already with a non-zero old-oid.  For this case, this hook will use
-"option" directives to report extended attributes for the reference given
-by the leading "ok" directive.
-
-The report of the commands of this hook should have the same order as
-the input.  The exit status of the 'proc-receive' hook only determines
-the success or failure of the group of commands sent to it, unless
-atomic push is in use.
-
-It is forbidden to specify more than one hook for 'proc-receive'. If a
-globally-configured 'proc-receive' must be overridden, use
-'hookcmd.<global-hook>.skip = true' to ignore it.
-
-[[post-receive]]
-post-receive
-~~~~~~~~~~~~
-
-This hook is invoked by linkgit:git-receive-pack[1] when it reacts to
-`git push` and updates reference(s) in its repository.
-It executes on the remote repository once after all the refs have
-been updated.
-
-This hook executes once for the receive operation.  It takes no
-arguments, but gets the same information as the
-<<pre-receive,'pre-receive'>>
-hook does on its standard input.
-
-This hook does not affect the outcome of `git receive-pack`, as it
-is called after the real work is done.
-
-This supersedes the <<post-update,'post-update'>> hook in that it gets
-both old and new values of all the refs in addition to their
-names.
-
-Both standard output and standard error output are forwarded to
-`git send-pack` on the other end, so you can simply `echo` messages
-for the user.
-
-The default 'post-receive' hook is empty, but there is
-a sample script `post-receive-email` provided in the `contrib/hooks`
-directory in Git distribution, which implements sending commit
-emails.
-
-The number of push options given on the command line of
-`git push --push-option=...` can be read from the environment
-variable `GIT_PUSH_OPTION_COUNT`, and the options themselves are
-found in `GIT_PUSH_OPTION_0`, `GIT_PUSH_OPTION_1`,...
-If it is negotiated to not use the push options phase, the
-environment variables will not be set. If the client selects
-to use push options, but doesn't transmit any, the count variable
-will be set to zero, `GIT_PUSH_OPTION_COUNT=0`.
-
-Hooks executed during 'post-receive' are run in parallel, unless hook.jobs is
-configured to 1.
-
-[[post-update]]
-post-update
-~~~~~~~~~~~
-
-This hook is invoked by linkgit:git-receive-pack[1] when it reacts to
-`git push` and updates reference(s) in its repository.
-It executes on the remote repository once after all the refs have
-been updated.
-
-It takes a variable number of parameters, each of which is the
-name of ref that was actually updated.
-
-This hook is meant primarily for notification, and cannot affect
-the outcome of `git receive-pack`.
-
-The 'post-update' hook can tell what are the heads that were pushed,
-but it does not know what their original and updated values are,
-so it is a poor place to do log old..new. The
-<<post-receive,'post-receive'>> hook does get both original and
-updated values of the refs. You might consider it instead if you need
-them.
-
-When enabled, the default 'post-update' hook runs
-`git update-server-info` to keep the information used by dumb
-transports (e.g., HTTP) up to date.  If you are publishing
-a Git repository that is accessible via HTTP, you should
-probably enable this hook.
-
-Both standard output and standard error output are forwarded to
-`git send-pack` on the other end, so you can simply `echo` messages
-for the user.
-
-Hooks run during 'post-update' will be run in parallel, unless hook.jobs is
-configured to 1.
-
-reference-transaction
-~~~~~~~~~~~~~~~~~~~~~
-
-This hook is invoked by any Git command that performs reference
-updates. It executes whenever a reference transaction is prepared,
-committed or aborted and may thus get called multiple times.
-
-The hook takes exactly one argument, which is the current state the
-given reference transaction is in:
-
-    - "prepared": All reference updates have been queued to the
-      transaction and references were locked on disk.
-
-    - "committed": The reference transaction was committed and all
-      references now have their respective new value.
-
-    - "aborted": The reference transaction was aborted, no changes
-      were performed and the locks have been released.
-
-For each reference update that was added to the transaction, the hook
-receives on standard input a line of the format:
-
-  <old-value> SP <new-value> SP <ref-name> LF
-
-The exit status of the hook is ignored for any state except for the
-"prepared" state. In the "prepared" state, a non-zero exit status will
-cause the transaction to be aborted. The hook will not be called with
-"aborted" state in that case.
-
-Hooks run during 'reference-transaction' will be run in parallel, unless
-hook.jobs is configured to 1.
-
-push-to-checkout
-~~~~~~~~~~~~~~~~
-
-This hook is invoked by linkgit:git-receive-pack[1] when it reacts to
-`git push` and updates reference(s) in its repository, and when
-the push tries to update the branch that is currently checked out
-and the `receive.denyCurrentBranch` configuration variable is set to
-`updateInstead`.  Such a push by default is refused if the working
-tree and the index of the remote repository has any difference from
-the currently checked out commit; when both the working tree and the
-index match the current commit, they are updated to match the newly
-pushed tip of the branch.  This hook is to be used to override the
-default behaviour.
-
-The hook receives the commit with which the tip of the current
-branch is going to be updated.  It can exit with a non-zero status
-to refuse the push (when it does so, it must not modify the index or
-the working tree).  Or it can make any necessary changes to the
-working tree and to the index to bring them to the desired state
-when the tip of the current branch is updated to the new commit, and
-exit with a zero status.
-
-For example, the hook can simply run `git read-tree -u -m HEAD "$1"`
-in order to emulate `git fetch` that is run in the reverse direction
-with `git push`, as the two-tree form of `git read-tree -u -m` is
-essentially the same as `git switch` or `git checkout`
-that switches branches while
-keeping the local changes in the working tree that do not interfere
-with the difference between the branches.
-
-Hooks executed during 'push-to-checkout' will not be parallelized.
-
-pre-auto-gc
-~~~~~~~~~~~
-
-This hook is invoked by `git gc --auto` (see linkgit:git-gc[1]). It
-takes no parameter, and exiting with non-zero status from this script
-causes the `git gc --auto` to abort.
-
-Hooks run during 'pre-auto-gc' will be run in parallel, unless hook.jobs is
-configured to 1.
-
-post-rewrite
-~~~~~~~~~~~~
-
-This hook is invoked by commands that rewrite commits
-(linkgit:git-commit[1] when called with `--amend` and
-linkgit:git-rebase[1]; however, full-history (re)writing tools like
-linkgit:git-fast-import[1] or
-https://github.com/newren/git-filter-repo[git-filter-repo] typically
-do not call it!).  Its first argument denotes the command it was
-invoked by: currently one of `amend` or `rebase`.  Further
-command-dependent arguments may be passed in the future.
-
-The hook receives a list of the rewritten commits on stdin, in the
-format
-
-  <old-sha1> SP <new-sha1> [ SP <extra-info> ] LF
-
-The 'extra-info' is again command-dependent.  If it is empty, the
-preceding SP is also omitted.  Currently, no commands pass any
-'extra-info'.
-
-The hook always runs after the automatic note copying (see
-"notes.rewrite.<command>" in linkgit:git-config[1]) has happened, and
-thus has access to these notes.
-
-Hooks run during 'post-rewrite' will be run in parallel, unless hook.jobs is
-configured to 1.
-
-The following command-specific comments apply:
-
-rebase::
-	For the 'squash' and 'fixup' operation, all commits that were
-	squashed are listed as being rewritten to the squashed commit.
-	This means that there will be several lines sharing the same
-	'new-sha1'.
-+
-The commits are guaranteed to be listed in the order that they were
-processed by rebase.
-
-sendemail-validate
-~~~~~~~~~~~~~~~~~~
-
-This hook is invoked by linkgit:git-send-email[1].  It takes a single parameter,
-the name of the file that holds the e-mail to be sent.  Exiting with a
-non-zero status causes `git send-email` to abort before sending any
-e-mails.
-
-fsmonitor-watchman
-~~~~~~~~~~~~~~~~~~
-
-This hook is invoked when the configuration option `core.fsmonitor` is
-set to `.git/hooks/fsmonitor-watchman` or `.git/hooks/fsmonitor-watchmanv2`
-depending on the version of the hook to use.
-
-Version 1 takes two arguments, a version (1) and the time in elapsed
-nanoseconds since midnight, January 1, 1970.
-
-Version 2 takes two arguments, a version (2) and a token that is used
-for identifying changes since the token. For watchman this would be
-a clock id. This version must output to stdout the new token followed
-by a NUL before the list of files.
-
-The hook should output to stdout the list of all files in the working
-directory that may have changed since the requested time.  The logic
-should be inclusive so that it does not miss any potential changes.
-The paths should be relative to the root of the working directory
-and be separated by a single NUL.
-
-It is OK to include files which have not actually changed.  All changes
-including newly-created and deleted files should be included. When
-files are renamed, both the old and the new name should be included.
-
-Git will limit what files it checks for changes as well as which
-directories are checked for untracked files based on the path names
-given.
-
-An optimized way to tell git "all files have changed" is to return
-the filename `/`.
-
-The exit status determines whether git will use the data from the
-hook to limit its search.  On error, it will fall back to verifying
-all files and folders.
-
-p4-changelist
-~~~~~~~~~~~~~
-
-This hook is invoked by `git-p4 submit`.
-
-The `p4-changelist` hook is executed after the changelist
-message has been edited by the user. It can be bypassed with the
-`--no-verify` option. It takes a single parameter, the name
-of the file that holds the proposed changelist text. Exiting
-with a non-zero status causes the command to abort.
-
-The hook is allowed to edit the changelist file and can be used
-to normalize the text into some project standard format. It can
-also be used to refuse the Submit after inspect the message file.
-
-Run `git-p4 submit --help` for details.
-
-p4-prepare-changelist
-~~~~~~~~~~~~~~~~~~~~~
-
-This hook is invoked by `git-p4 submit`.
-
-The `p4-prepare-changelist` hook is executed right after preparing
-the default changelist message and before the editor is started.
-It takes one parameter, the name of the file that contains the
-changelist text. Exiting with a non-zero status from the script
-will abort the process.
-
-The purpose of the hook is to edit the message file in place,
-and it is not suppressed by the `--no-verify` option. This hook
-is called even if `--prepare-p4-only` is set.
-
-Run `git-p4 submit --help` for details.
-
-p4-post-changelist
-~~~~~~~~~~~~~~~~~~
-
-This hook is invoked by `git-p4 submit`.
-
-The `p4-post-changelist` hook is invoked after the submit has
-successfully occurred in P4. It takes no parameters and is meant
-primarily for notification and cannot affect the outcome of the
-git p4 submit action.
-
-Run `git-p4 submit --help` for details.
-
-p4-pre-submit
-~~~~~~~~~~~~~
-
-This hook is invoked by `git-p4 submit`. It takes no parameters and nothing
-from standard input. Exiting with non-zero status from this script prevent
-`git-p4 submit` from launching. It can be bypassed with the `--no-verify`
-command line option. Run `git-p4 submit --help` for details.
-
-
-
-post-index-change
-~~~~~~~~~~~~~~~~~
-
-This hook is invoked when the index is written in read-cache.c
-do_write_locked_index.
-
-The first parameter passed to the hook is the indicator for the
-working directory being updated.  "1" meaning working directory
-was updated or "0" when the working directory was not updated.
-
-The second parameter passed to the hook is the indicator for whether
-or not the index was updated and the skip-worktree bit could have
-changed.  "1" meaning skip-worktree bits could have been updated
-and "0" meaning they were not.
-
-Only one parameter should be set to "1" when the hook runs.  The hook
-running passing "1", "1" should not be possible.
-
-Hooks run during 'post-index-change' will be run in parallel, unless hook.jobs
-is configured to 1.
+include::native-hooks.txt[]
 
 GIT
 ---
diff --git a/Documentation/native-hooks.txt b/Documentation/native-hooks.txt
new file mode 100644
index 0000000000..6c4aad83e1
--- /dev/null
+++ b/Documentation/native-hooks.txt
@@ -0,0 +1,708 @@
+applypatch-msg
+~~~~~~~~~~~~~~
+
+This hook is invoked by linkgit:git-am[1].  It takes a single
+parameter, the name of the file that holds the proposed commit
+log message.  Exiting with a non-zero status causes `git am` to abort
+before applying the patch.
+
+The hook is allowed to edit the message file in place, and can
+be used to normalize the message into some project standard
+format. It can also be used to refuse the commit after inspecting
+the message file.
+
+The default 'applypatch-msg' hook, when enabled, runs the
+'commit-msg' hook, if the latter is enabled.
+
+Hooks run during 'applypatch-msg' will not be parallelized, because hooks are
+expected to edit the file holding the commit log message.
+
+pre-applypatch
+~~~~~~~~~~~~~~
+
+This hook is invoked by linkgit:git-am[1].  It takes no parameter, and is
+invoked after the patch is applied, but before a commit is made.
+
+If it exits with non-zero status, then the working tree will not be
+committed after applying the patch.
+
+It can be used to inspect the current working tree and refuse to
+make a commit if it does not pass certain test.
+
+The default 'pre-applypatch' hook, when enabled, runs the
+'pre-commit' hook, if the latter is enabled.
+
+Hooks run during 'pre-applypatch' will be run in parallel, unless hook.jobs is
+configured to 1.
+
+post-applypatch
+~~~~~~~~~~~~~~~
+
+This hook is invoked by linkgit:git-am[1].  It takes no parameter,
+and is invoked after the patch is applied and a commit is made.
+
+This hook is meant primarily for notification, and cannot affect
+the outcome of `git am`.
+
+Hooks run during 'post-applypatch' will be run in parallel, unless hook.jobs is
+configured to 1.
+
+pre-commit
+~~~~~~~~~~
+
+This hook is invoked by linkgit:git-commit[1], and can be bypassed
+with the `--no-verify` option.  It takes no parameters, and is
+invoked before obtaining the proposed commit log message and
+making a commit.  Exiting with a non-zero status from this script
+causes the `git commit` command to abort before creating a commit.
+
+The default 'pre-commit' hook, when enabled, catches introduction
+of lines with trailing whitespaces and aborts the commit when
+such a line is found.
+
+All the `git commit` hooks are invoked with the environment
+variable `GIT_EDITOR=:` if the command will not bring up an editor
+to modify the commit message.
+
+The default 'pre-commit' hook, when enabled--and with the
+`hooks.allownonascii` config option unset or set to false--prevents
+the use of non-ASCII filenames.
+
+Hooks executed during 'pre-commit' will not be parallelized.
+
+pre-merge-commit
+~~~~~~~~~~~~~~~~
+
+This hook is invoked by linkgit:git-merge[1], and can be bypassed
+with the `--no-verify` option.  It takes no parameters, and is
+invoked after the merge has been carried out successfully and before
+obtaining the proposed commit log message to
+make a commit.  Exiting with a non-zero status from this script
+causes the `git merge` command to abort before creating a commit.
+
+The default 'pre-merge-commit' hook, when enabled, runs the
+'pre-commit' hook, if the latter is enabled.
+
+This hook is invoked with the environment variable
+`GIT_EDITOR=:` if the command will not bring up an editor
+to modify the commit message.
+
+If the merge cannot be carried out automatically, the conflicts
+need to be resolved and the result committed separately (see
+linkgit:git-merge[1]). At that point, this hook will not be executed,
+but the 'pre-commit' hook will, if it is enabled.
+
+Hooks executed during 'pre-merge-commit' will not be parallelized.
+
+prepare-commit-msg
+~~~~~~~~~~~~~~~~~~
+
+This hook is invoked by linkgit:git-commit[1] right after preparing the
+default log message, and before the editor is started.
+
+It takes one to three parameters.  The first is the name of the file
+that contains the commit log message.  The second is the source of the commit
+message, and can be: `message` (if a `-m` or `-F` option was
+given); `template` (if a `-t` option was given or the
+configuration option `commit.template` is set); `merge` (if the
+commit is a merge or a `.git/MERGE_MSG` file exists); `squash`
+(if a `.git/SQUASH_MSG` file exists); or `commit`, followed by
+a commit SHA-1 (if a `-c`, `-C` or `--amend` option was given).
+
+If the exit status is non-zero, `git commit` will abort.
+
+The purpose of the hook is to edit the message file in place, and
+it is not suppressed by the `--no-verify` option.  A non-zero exit
+means a failure of the hook and aborts the commit.  It should not
+be used as replacement for pre-commit hook.
+
+The sample `prepare-commit-msg` hook that comes with Git removes the
+help message found in the commented portion of the commit template.
+
+Hooks executed during 'prepare-commit-msg' will not be parallelized, because
+hooks are expected to edit the file containing the commit log message.
+
+commit-msg
+~~~~~~~~~~
+
+This hook is invoked by linkgit:git-commit[1] and linkgit:git-merge[1], and can be
+bypassed with the `--no-verify` option.  It takes a single parameter,
+the name of the file that holds the proposed commit log message.
+Exiting with a non-zero status causes the command to abort.
+
+The hook is allowed to edit the message file in place, and can be used
+to normalize the message into some project standard format. It
+can also be used to refuse the commit after inspecting the message
+file.
+
+The default 'commit-msg' hook, when enabled, detects duplicate
+`Signed-off-by` trailers, and aborts the commit if one is found.
+
+Hooks executed during 'commit-msg' will not be parallelized, because hooks are
+expected to edit the file containing the proposed commit log message.
+
+post-commit
+~~~~~~~~~~~
+
+This hook is invoked by linkgit:git-commit[1]. It takes no parameters, and is
+invoked after a commit is made.
+
+This hook is meant primarily for notification, and cannot affect
+the outcome of `git commit`.
+
+Hooks executed during 'post-commit' will run in parallel, unless hook.jobs is
+configured to 1.
+
+pre-rebase
+~~~~~~~~~~
+
+This hook is called by linkgit:git-rebase[1] and can be used to prevent a
+branch from getting rebased.  The hook may be called with one or
+two parameters.  The first parameter is the upstream from which
+the series was forked.  The second parameter is the branch being
+rebased, and is not set when rebasing the current branch.
+
+Hooks executed during 'pre-rebase' will run in parallel, unless hook.jobs is
+configured to 1.
+
+post-checkout
+~~~~~~~~~~~~~
+
+This hook is invoked when a linkgit:git-checkout[1] or
+linkgit:git-switch[1] is run after having updated the
+worktree.  The hook is given three parameters: the ref of the previous HEAD,
+the ref of the new HEAD (which may or may not have changed), and a flag
+indicating whether the checkout was a branch checkout (changing branches,
+flag=1) or a file checkout (retrieving a file from the index, flag=0).
+This hook cannot affect the outcome of `git switch` or `git checkout`,
+other than that the hook's exit status becomes the exit status of
+these two commands.
+
+It is also run after linkgit:git-clone[1], unless the `--no-checkout` (`-n`) option is
+used. The first parameter given to the hook is the null-ref, the second the
+ref of the new HEAD and the flag is always 1. Likewise for `git worktree add`
+unless `--no-checkout` is used.
+
+This hook can be used to perform repository validity checks, auto-display
+differences from the previous HEAD if different, or set working dir metadata
+properties.
+
+Hooks executed during 'post-checkout' will not be parallelized.
+
+post-merge
+~~~~~~~~~~
+
+This hook is invoked by linkgit:git-merge[1], which happens when a `git pull`
+is done on a local repository.  The hook takes a single parameter, a status
+flag specifying whether or not the merge being done was a squash merge.
+This hook cannot affect the outcome of `git merge` and is not executed,
+if the merge failed due to conflicts.
+
+This hook can be used in conjunction with a corresponding pre-commit hook to
+save and restore any form of metadata associated with the working tree
+(e.g.: permissions/ownership, ACLS, etc).  See contrib/hooks/setgitperms.perl
+for an example of how to do this.
+
+Hooks executed during 'post-merge' will run in parallel, unless hook.jobs is
+configured to 1.
+
+pre-push
+~~~~~~~~
+
+This hook is called by linkgit:git-push[1] and can be used to prevent
+a push from taking place.  The hook is called with two parameters
+which provide the name and location of the destination remote, if a
+named remote is not being used both values will be the same.
+
+Information about what is to be pushed is provided on the hook's standard
+input with lines of the form:
+
+  <local ref> SP <local sha1> SP <remote ref> SP <remote sha1> LF
+
+For instance, if the command +git push origin master:foreign+ were run the
+hook would receive a line like the following:
+
+  refs/heads/master 67890 refs/heads/foreign 12345
+
+although the full, 40-character SHA-1s would be supplied.  If the foreign ref
+does not yet exist the `<remote SHA-1>` will be 40 `0`.  If a ref is to be
+deleted, the `<local ref>` will be supplied as `(delete)` and the `<local
+SHA-1>` will be 40 `0`.  If the local commit was specified by something other
+than a name which could be expanded (such as `HEAD~`, or a SHA-1) it will be
+supplied as it was originally given.
+
+If this hook exits with a non-zero status, `git push` will abort without
+pushing anything.  Information about why the push is rejected may be sent
+to the user by writing to standard error.
+
+Hooks executed during 'pre-push' will run in parallel, unless hook.jobs is
+configured to 1.
+
+[[pre-receive]]
+pre-receive
+~~~~~~~~~~~
+
+This hook is invoked by linkgit:git-receive-pack[1] when it reacts to
+`git push` and updates reference(s) in its repository.
+Just before starting to update refs on the remote repository, the
+pre-receive hook is invoked.  Its exit status determines the success
+or failure of the update.
+
+This hook executes once for the receive operation. It takes no
+arguments, but for each ref to be updated it receives on standard
+input a line of the format:
+
+  <old-value> SP <new-value> SP <ref-name> LF
+
+where `<old-value>` is the old object name stored in the ref,
+`<new-value>` is the new object name to be stored in the ref and
+`<ref-name>` is the full name of the ref.
+When creating a new ref, `<old-value>` is 40 `0`.
+
+If the hook exits with non-zero status, none of the refs will be
+updated. If the hook exits with zero, updating of individual refs can
+still be prevented by the <<update,'update'>> hook.
+
+Both standard output and standard error output are forwarded to
+`git send-pack` on the other end, so you can simply `echo` messages
+for the user.
+
+The number of push options given on the command line of
+`git push --push-option=...` can be read from the environment
+variable `GIT_PUSH_OPTION_COUNT`, and the options themselves are
+found in `GIT_PUSH_OPTION_0`, `GIT_PUSH_OPTION_1`,...
+If it is negotiated to not use the push options phase, the
+environment variables will not be set. If the client selects
+to use push options, but doesn't transmit any, the count variable
+will be set to zero, `GIT_PUSH_OPTION_COUNT=0`.
+
+See the section on "Quarantine Environment" in
+linkgit:git-receive-pack[1] for some caveats.
+
+Hooks executed during 'pre-receive' will not be parallelized.
+
+[[update]]
+update
+~~~~~~
+
+This hook is invoked by linkgit:git-receive-pack[1] when it reacts to
+`git push` and updates reference(s) in its repository.
+Just before updating the ref on the remote repository, the update hook
+is invoked.  Its exit status determines the success or failure of
+the ref update.
+
+The hook executes once for each ref to be updated, and takes
+three parameters:
+
+ - the name of the ref being updated,
+ - the old object name stored in the ref,
+ - and the new object name to be stored in the ref.
+
+A zero exit from the update hook allows the ref to be updated.
+Exiting with a non-zero status prevents `git receive-pack`
+from updating that ref.
+
+This hook can be used to prevent 'forced' update on certain refs by
+making sure that the object name is a commit object that is a
+descendant of the commit object named by the old object name.
+That is, to enforce a "fast-forward only" policy.
+
+It could also be used to log the old..new status.  However, it
+does not know the entire set of branches, so it would end up
+firing one e-mail per ref when used naively, though.  The
+<<post-receive,'post-receive'>> hook is more suited to that.
+
+In an environment that restricts the users' access only to git
+commands over the wire, this hook can be used to implement access
+control without relying on filesystem ownership and group
+membership. See linkgit:git-shell[1] for how you might use the login
+shell to restrict the user's access to only git commands.
+
+Both standard output and standard error output are forwarded to
+`git send-pack` on the other end, so you can simply `echo` messages
+for the user.
+
+The default 'update' hook, when enabled--and with
+`hooks.allowunannotated` config option unset or set to false--prevents
+unannotated tags to be pushed.
+
+Hooks executed during 'update' are run in parallel, unless hook.jobs is
+configured to 1.
+
+[[proc-receive]]
+proc-receive
+~~~~~~~~~~~~
+
+This hook is invoked by linkgit:git-receive-pack[1].  If the server has
+set the multi-valued config variable `receive.procReceiveRefs`, and the
+commands sent to 'receive-pack' have matching reference names, these
+commands will be executed by this hook, instead of by the internal
+`execute_commands()` function.  This hook is responsible for updating
+the relevant references and reporting the results back to 'receive-pack'.
+
+This hook executes once for the receive operation.  It takes no
+arguments, but uses a pkt-line format protocol to communicate with
+'receive-pack' to read commands, push-options and send results.  In the
+following example for the protocol, the letter 'S' stands for
+'receive-pack' and the letter 'H' stands for this hook.
+
+    # Version and features negotiation.
+    S: PKT-LINE(version=1\0push-options atomic...)
+    S: flush-pkt
+    H: PKT-LINE(version=1\0push-options...)
+    H: flush-pkt
+
+    # Send commands from server to the hook.
+    S: PKT-LINE(<old-oid> <new-oid> <ref>)
+    S: ... ...
+    S: flush-pkt
+    # Send push-options only if the 'push-options' feature is enabled.
+    S: PKT-LINE(push-option)
+    S: ... ...
+    S: flush-pkt
+
+    # Receive result from the hook.
+    # OK, run this command successfully.
+    H: PKT-LINE(ok <ref>)
+    # NO, I reject it.
+    H: PKT-LINE(ng <ref> <reason>)
+    # Fall through, let 'receive-pack' to execute it.
+    H: PKT-LINE(ok <ref>)
+    H: PKT-LINE(option fall-through)
+    # OK, but has an alternate reference.  The alternate reference name
+    # and other status can be given in option directives.
+    H: PKT-LINE(ok <ref>)
+    H: PKT-LINE(option refname <refname>)
+    H: PKT-LINE(option old-oid <old-oid>)
+    H: PKT-LINE(option new-oid <new-oid>)
+    H: PKT-LINE(option forced-update)
+    H: ... ...
+    H: flush-pkt
+
+Each command for the 'proc-receive' hook may point to a pseudo-reference
+and always has a zero-old as its old-oid, while the 'proc-receive' hook
+may update an alternate reference and the alternate reference may exist
+already with a non-zero old-oid.  For this case, this hook will use
+"option" directives to report extended attributes for the reference given
+by the leading "ok" directive.
+
+The report of the commands of this hook should have the same order as
+the input.  The exit status of the 'proc-receive' hook only determines
+the success or failure of the group of commands sent to it, unless
+atomic push is in use.
+
+It is forbidden to specify more than one hook for 'proc-receive'. If a
+globally-configured 'proc-receive' must be overridden, use
+'hookcmd.<global-hook>.skip = true' to ignore it.
+
+[[post-receive]]
+post-receive
+~~~~~~~~~~~~
+
+This hook is invoked by linkgit:git-receive-pack[1] when it reacts to
+`git push` and updates reference(s) in its repository.
+It executes on the remote repository once after all the refs have
+been updated.
+
+This hook executes once for the receive operation.  It takes no
+arguments, but gets the same information as the
+<<pre-receive,'pre-receive'>>
+hook does on its standard input.
+
+This hook does not affect the outcome of `git receive-pack`, as it
+is called after the real work is done.
+
+This supersedes the <<post-update,'post-update'>> hook in that it gets
+both old and new values of all the refs in addition to their
+names.
+
+Both standard output and standard error output are forwarded to
+`git send-pack` on the other end, so you can simply `echo` messages
+for the user.
+
+The default 'post-receive' hook is empty, but there is
+a sample script `post-receive-email` provided in the `contrib/hooks`
+directory in Git distribution, which implements sending commit
+emails.
+
+The number of push options given on the command line of
+`git push --push-option=...` can be read from the environment
+variable `GIT_PUSH_OPTION_COUNT`, and the options themselves are
+found in `GIT_PUSH_OPTION_0`, `GIT_PUSH_OPTION_1`,...
+If it is negotiated to not use the push options phase, the
+environment variables will not be set. If the client selects
+to use push options, but doesn't transmit any, the count variable
+will be set to zero, `GIT_PUSH_OPTION_COUNT=0`.
+
+Hooks executed during 'post-receive' are run in parallel, unless hook.jobs is
+configured to 1.
+
+[[post-update]]
+post-update
+~~~~~~~~~~~
+
+This hook is invoked by linkgit:git-receive-pack[1] when it reacts to
+`git push` and updates reference(s) in its repository.
+It executes on the remote repository once after all the refs have
+been updated.
+
+It takes a variable number of parameters, each of which is the
+name of ref that was actually updated.
+
+This hook is meant primarily for notification, and cannot affect
+the outcome of `git receive-pack`.
+
+The 'post-update' hook can tell what are the heads that were pushed,
+but it does not know what their original and updated values are,
+so it is a poor place to do log old..new. The
+<<post-receive,'post-receive'>> hook does get both original and
+updated values of the refs. You might consider it instead if you need
+them.
+
+When enabled, the default 'post-update' hook runs
+`git update-server-info` to keep the information used by dumb
+transports (e.g., HTTP) up to date.  If you are publishing
+a Git repository that is accessible via HTTP, you should
+probably enable this hook.
+
+Both standard output and standard error output are forwarded to
+`git send-pack` on the other end, so you can simply `echo` messages
+for the user.
+
+Hooks run during 'post-update' will be run in parallel, unless hook.jobs is
+configured to 1.
+
+reference-transaction
+~~~~~~~~~~~~~~~~~~~~~
+
+This hook is invoked by any Git command that performs reference
+updates. It executes whenever a reference transaction is prepared,
+committed or aborted and may thus get called multiple times.
+
+The hook takes exactly one argument, which is the current state the
+given reference transaction is in:
+
+    - "prepared": All reference updates have been queued to the
+      transaction and references were locked on disk.
+
+    - "committed": The reference transaction was committed and all
+      references now have their respective new value.
+
+    - "aborted": The reference transaction was aborted, no changes
+      were performed and the locks have been released.
+
+For each reference update that was added to the transaction, the hook
+receives on standard input a line of the format:
+
+  <old-value> SP <new-value> SP <ref-name> LF
+
+The exit status of the hook is ignored for any state except for the
+"prepared" state. In the "prepared" state, a non-zero exit status will
+cause the transaction to be aborted. The hook will not be called with
+"aborted" state in that case.
+
+Hooks run during 'reference-transaction' will be run in parallel, unless
+hook.jobs is configured to 1.
+
+push-to-checkout
+~~~~~~~~~~~~~~~~
+
+This hook is invoked by linkgit:git-receive-pack[1] when it reacts to
+`git push` and updates reference(s) in its repository, and when
+the push tries to update the branch that is currently checked out
+and the `receive.denyCurrentBranch` configuration variable is set to
+`updateInstead`.  Such a push by default is refused if the working
+tree and the index of the remote repository has any difference from
+the currently checked out commit; when both the working tree and the
+index match the current commit, they are updated to match the newly
+pushed tip of the branch.  This hook is to be used to override the
+default behaviour.
+
+The hook receives the commit with which the tip of the current
+branch is going to be updated.  It can exit with a non-zero status
+to refuse the push (when it does so, it must not modify the index or
+the working tree).  Or it can make any necessary changes to the
+working tree and to the index to bring them to the desired state
+when the tip of the current branch is updated to the new commit, and
+exit with a zero status.
+
+For example, the hook can simply run `git read-tree -u -m HEAD "$1"`
+in order to emulate `git fetch` that is run in the reverse direction
+with `git push`, as the two-tree form of `git read-tree -u -m` is
+essentially the same as `git switch` or `git checkout`
+that switches branches while
+keeping the local changes in the working tree that do not interfere
+with the difference between the branches.
+
+Hooks executed during 'push-to-checkout' will not be parallelized.
+
+pre-auto-gc
+~~~~~~~~~~~
+
+This hook is invoked by `git gc --auto` (see linkgit:git-gc[1]). It
+takes no parameter, and exiting with non-zero status from this script
+causes the `git gc --auto` to abort.
+
+Hooks run during 'pre-auto-gc' will be run in parallel, unless hook.jobs is
+configured to 1.
+
+post-rewrite
+~~~~~~~~~~~~
+
+This hook is invoked by commands that rewrite commits
+(linkgit:git-commit[1] when called with `--amend` and
+linkgit:git-rebase[1]; however, full-history (re)writing tools like
+linkgit:git-fast-import[1] or
+https://github.com/newren/git-filter-repo[git-filter-repo] typically
+do not call it!).  Its first argument denotes the command it was
+invoked by: currently one of `amend` or `rebase`.  Further
+command-dependent arguments may be passed in the future.
+
+The hook receives a list of the rewritten commits on stdin, in the
+format
+
+  <old-sha1> SP <new-sha1> [ SP <extra-info> ] LF
+
+The 'extra-info' is again command-dependent.  If it is empty, the
+preceding SP is also omitted.  Currently, no commands pass any
+'extra-info'.
+
+The hook always runs after the automatic note copying (see
+"notes.rewrite.<command>" in linkgit:git-config[1]) has happened, and
+thus has access to these notes.
+
+Hooks run during 'post-rewrite' will be run in parallel, unless hook.jobs is
+configured to 1.
+
+The following command-specific comments apply:
+
+rebase::
+	For the 'squash' and 'fixup' operation, all commits that were
+	squashed are listed as being rewritten to the squashed commit.
+	This means that there will be several lines sharing the same
+	'new-sha1'.
++
+The commits are guaranteed to be listed in the order that they were
+processed by rebase.
+
+sendemail-validate
+~~~~~~~~~~~~~~~~~~
+
+This hook is invoked by linkgit:git-send-email[1].  It takes a single parameter,
+the name of the file that holds the e-mail to be sent.  Exiting with a
+non-zero status causes `git send-email` to abort before sending any
+e-mails.
+
+fsmonitor-watchman
+~~~~~~~~~~~~~~~~~~
+
+This hook is invoked when the configuration option `core.fsmonitor` is
+set to `.git/hooks/fsmonitor-watchman` or `.git/hooks/fsmonitor-watchmanv2`
+depending on the version of the hook to use.
+
+Version 1 takes two arguments, a version (1) and the time in elapsed
+nanoseconds since midnight, January 1, 1970.
+
+Version 2 takes two arguments, a version (2) and a token that is used
+for identifying changes since the token. For watchman this would be
+a clock id. This version must output to stdout the new token followed
+by a NUL before the list of files.
+
+The hook should output to stdout the list of all files in the working
+directory that may have changed since the requested time.  The logic
+should be inclusive so that it does not miss any potential changes.
+The paths should be relative to the root of the working directory
+and be separated by a single NUL.
+
+It is OK to include files which have not actually changed.  All changes
+including newly-created and deleted files should be included. When
+files are renamed, both the old and the new name should be included.
+
+Git will limit what files it checks for changes as well as which
+directories are checked for untracked files based on the path names
+given.
+
+An optimized way to tell git "all files have changed" is to return
+the filename `/`.
+
+The exit status determines whether git will use the data from the
+hook to limit its search.  On error, it will fall back to verifying
+all files and folders.
+
+p4-changelist
+~~~~~~~~~~~~~
+
+This hook is invoked by `git-p4 submit`.
+
+The `p4-changelist` hook is executed after the changelist
+message has been edited by the user. It can be bypassed with the
+`--no-verify` option. It takes a single parameter, the name
+of the file that holds the proposed changelist text. Exiting
+with a non-zero status causes the command to abort.
+
+The hook is allowed to edit the changelist file and can be used
+to normalize the text into some project standard format. It can
+also be used to refuse the Submit after inspect the message file.
+
+Run `git-p4 submit --help` for details.
+
+p4-prepare-changelist
+~~~~~~~~~~~~~~~~~~~~~
+
+This hook is invoked by `git-p4 submit`.
+
+The `p4-prepare-changelist` hook is executed right after preparing
+the default changelist message and before the editor is started.
+It takes one parameter, the name of the file that contains the
+changelist text. Exiting with a non-zero status from the script
+will abort the process.
+
+The purpose of the hook is to edit the message file in place,
+and it is not suppressed by the `--no-verify` option. This hook
+is called even if `--prepare-p4-only` is set.
+
+Run `git-p4 submit --help` for details.
+
+p4-post-changelist
+~~~~~~~~~~~~~~~~~~
+
+This hook is invoked by `git-p4 submit`.
+
+The `p4-post-changelist` hook is invoked after the submit has
+successfully occurred in P4. It takes no parameters and is meant
+primarily for notification and cannot affect the outcome of the
+git p4 submit action.
+
+Run `git-p4 submit --help` for details.
+
+p4-pre-submit
+~~~~~~~~~~~~~
+
+This hook is invoked by `git-p4 submit`. It takes no parameters and nothing
+from standard input. Exiting with non-zero status from this script prevent
+`git-p4 submit` from launching. It can be bypassed with the `--no-verify`
+command line option. Run `git-p4 submit --help` for details.
+
+
+
+post-index-change
+~~~~~~~~~~~~~~~~~
+
+This hook is invoked when the index is written in read-cache.c
+do_write_locked_index.
+
+The first parameter passed to the hook is the indicator for the
+working directory being updated.  "1" meaning working directory
+was updated or "0" when the working directory was not updated.
+
+The second parameter passed to the hook is the indicator for whether
+or not the index was updated and the skip-worktree bit could have
+changed.  "1" meaning skip-worktree bits could have been updated
+and "0" meaning they were not.
+
+Only one parameter should be set to "1" when the hook runs.  The hook
+running passing "1", "1" should not be possible.
+
+Hooks run during 'post-index-change' will be run in parallel, unless hook.jobs
+is configured to 1.
+
-- 
2.31.0.rc2.261.g7f71774620-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 00/37] config-based hooks
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (36 preceding siblings ...)
  2021-03-11  2:10 ` [PATCH v8 37/37] docs: unify githooks and git-hook manpages Emily Shaffer
@ 2021-03-11 22:26 ` Junio C Hamano
  2021-03-12 23:27   ` Emily Shaffer
  2021-03-12  9:49 ` Ævar Arnfjörð Bjarmason
                   ` (2 subsequent siblings)
  40 siblings, 1 reply; 479+ messages in thread
From: Junio C Hamano @ 2021-03-11 22:26 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: git, Jeff King, James Ramsay, Jonathan Nieder, brian m. carlson,
	Ævar Arnfjörð Bjarmason, Phillip Wood,
	Josh Steadmon, Johannes Schindelin, Jonathan Tan

Emily Shaffer <emilyshaffer@google.com> writes:

> Since v7:
> - Addressed Jonathan Tan's review of part I
> - Addressed Junio's review of part I and II
> - Combined parts I and II
>
> I think the updates to patch 1 between the rest of the work I've been
> doing probably have covered Ævar's comments.
>
> More details about per-patch changes found in the notes on each mail (I
> hope).
>
> I know that Junio was talking about merging v7 after Josh Steadmon's
> review and I asked him not to - this reroll has those changes from
> Jonathan Tan's review that I was wanting to wait for.

I picked it up and replaced, not necessarily because it is an urgent
thing to do during the pre-release period, but primarily because I
wanted to be prepared for any nasty surprises by unmanageable
conflicts I may have to face once the current cycle is over.

It turns out that it was a bit painful to merge to 'seen' as there
are in-flight topics that touch the hooks documentation, and the
changes they make must be carried forward to the new file.

But it was not too bad.  

The merge into 'seen' is 3cdeaeab (Merge branch 'es/config-hooks'
into seen, 2021-03-11) as of this writing, and the output of

    $ git diff 3cdeaeab3a^:Documentation/githooks.txt \
               3cdeaeab3a:Documentation/native-hooks.txt

    (i.e. the version of the file before the merge, where your topic
    being merged took material to edit to produce the new "native-hooks"
    document, is compared with the result)

looks reasonable to me, but please double check.

Thanks.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 03/37] hook: add list command
  2021-03-11  2:10 ` [PATCH v8 03/37] hook: add list command Emily Shaffer
@ 2021-03-12  8:20   ` Ævar Arnfjörð Bjarmason
  2021-03-24 17:31     ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-12  8:20 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Mar 11 2021, Emily Shaffer wrote:

> diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
> new file mode 100644
> index 0000000000..71449ecbc7
> --- /dev/null
> +++ b/Documentation/config/hook.txt
> @@ -0,0 +1,9 @@
> +hook.<command>.command::
> +	A command to execute during the <command> hook event. This can be an
> +	executable on your device, a oneliner for your shell, or the name of a
> +	hookcmd. See linkgit:git-hook[1].
> +
> +hookcmd.<name>.command::
> +	A command to execute during a hook for which <name> has been specified
> +	as a command. This can be an executable on your device or a oneliner for
> +	your shell. See linkgit:git-hook[1].
> diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
> index 9eeab0009d..f19875ed68 100644
> --- a/Documentation/git-hook.txt
> +++ b/Documentation/git-hook.txt
> @@ -8,12 +8,65 @@ git-hook - Manage configured hooks
>  SYNOPSIS
>  --------
>  [verse]
> -'git hook'
> +'git hook' list <hook-name>

Having just read this far (maybe this pattern is shared in the rest of
the series): Let's just squash this and the 2nd patch together.

Sometimes it's worth doing the scaffolding first, but adding a new
built-in is so trivial that I don't think it's worth it, and it just
results in back & forth churn like the above...

>  DESCRIPTION
>  -----------
> -A placeholder command. Later, you will be able to list, add, and modify hooks
> -with this command.

...and this...

> +You can list configured hooks with this command. Later, you will be able to run,
> +add, and modify hooks with this command.
> +
> +This command parses the default configuration files for sections `hook` and
> +`hookcmd`. `hook` is used to describe the commands which will be run during a
> +particular hook event; commands are run in the order Git encounters them during
> +the configuration parse (see linkgit:git-config[1]). `hookcmd` is used to
> +describe attributes of a specific command. If additional attributes don't need
> +to be specified, a command to run can be specified directly in the `hook`
> +section; if a `hookcmd` by that name isn't found, Git will attempt to run the
> +provided value directly. For example:
> +
> +Global config
> +----
> +  [hook "post-commit"]
> +    command = "linter"
> +    command = "~/typocheck.sh"
> +
> +  [hookcmd "linter"]
> +    command = "/bin/linter --c"
> +----
> +
> +Local config
> +----
> +  [hook "prepare-commit-msg"]
> +    command = "linter"
> +  [hook "post-commit"]
> +    command = "python ~/run-test-suite.py"
> +----
> +
> +With these configs, you'd then see:
> +
> +----
> +$ git hook list "post-commit"
> +global: /bin/linter --c
> +global: ~/typocheck.sh
> +local: python ~/run-test-suite.py
> +
> +$ git hook list "prepare-commit-msg"
> +local: /bin/linter --c
> +----
> +
> +COMMANDS
> +--------
> +
> +list `<hook-name>`::
> +
> +List the hooks which have been configured for `<hook-name>`. Hooks appear
> +in the order they should be run, and print the config scope where the relevant
> +`hook.<hook-name>.command` was specified, not the `hookcmd` (if applicable).
> +This output is human-readable and the format is subject to change over time.
> +
> +CONFIGURATION
> +-------------
> +include::config/hook.txt[]
>  
>  GIT
>  ---
> diff --git a/Makefile b/Makefile
> index 8e904a1ab5..3fa51597d8 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -891,6 +891,7 @@ LIB_OBJS += hash-lookup.o
>  LIB_OBJS += hashmap.o
>  LIB_OBJS += help.o
>  LIB_OBJS += hex.o
> +LIB_OBJS += hook.o
>  LIB_OBJS += ident.o
>  LIB_OBJS += json-writer.o
>  LIB_OBJS += kwset.o
> diff --git a/builtin/hook.c b/builtin/hook.c
> index b2bbc84d4d..bb64cd77ca 100644
> --- a/builtin/hook.c
> +++ b/builtin/hook.c
> @@ -1,21 +1,67 @@
>  #include "cache.h"
> -

Stray back & forth whitespace churn?

>  #include "builtin.h"
> +#include "config.h"
> +#include "hook.h"
>  #include "parse-options.h"
> +#include "strbuf.h"
>  
>  static const char * const builtin_hook_usage[] = {
> -	N_("git hook"),
> +	N_("git hook list <hookname>"),
>  	NULL
>  };
>  
> -int cmd_hook(int argc, const char **argv, const char *prefix)
> +static int list(int argc, const char **argv, const char *prefix)

...and here the cmd_hook() function being replaced (not really, just
moved below, but you get my drift...)

>  {
> -	struct option builtin_hook_options[] = {
> +	struct list_head *head, *pos;
> +	struct strbuf hookname = STRBUF_INIT;
> +
> +	struct option list_options[] = {
>  		OPT_END(),
>  	};
>  
> -	argc = parse_options(argc, argv, prefix, builtin_hook_options,
> +	argc = parse_options(argc, argv, prefix, list_options,
>  			     builtin_hook_usage, 0);
>  
> +	if (argc < 1) {
> +		usage_msg_opt(_("You must specify a hook event name to list."),
> +			      builtin_hook_usage, list_options);
> +	}
> +
> +	strbuf_addstr(&hookname, argv[0]);
> +
> +	head = hook_list(&hookname);
> +

More on strbuf usage later in another soon-to-be-sent E-Mail.

> +	if (list_empty(head)) {
> +		printf(_("no commands configured for hook '%s'\n"),
> +		       hookname.buf);
> +		strbuf_release(&hookname);
> +		return 0;
> +	}
> +
> +	list_for_each(pos, head) {
> +		struct hook *item = list_entry(pos, struct hook, list);
> +		if (item)
> +			printf("%s: %s\n",
> +			       config_scope_name(item->origin),
> +			       item->command.buf);
> +	}
> +
> +	clear_hook_list(head);
> +	strbuf_release(&hookname);
> +
>  	return 0;
>  }
> +
> +int cmd_hook(int argc, const char **argv, const char *prefix)
> +{
> +	struct option builtin_hook_options[] = {
> +		OPT_END(),
> +	};
> +	if (argc < 2)
> +		usage_with_options(builtin_hook_usage, builtin_hook_options);
> +
> +	if (!strcmp(argv[1], "list"))
> +		return list(argc - 1, argv + 1, prefix);
> +
> +	usage_with_options(builtin_hook_usage, builtin_hook_options);
> +}
> diff --git a/hook.c b/hook.c
> new file mode 100644
> index 0000000000..fede40e925
> --- /dev/null
> +++ b/hook.c
> @@ -0,0 +1,120 @@
> +#include "cache.h"
> +
> +#include "hook.h"
> +#include "config.h"
> +
> +void free_hook(struct hook *ptr)
> +{
> +	if (ptr) {
> +		strbuf_release(&ptr->command);
> +		free(ptr);
> +	}
> +}

Neither strbuf_release() nor free() need or should have a "if (ptr)" guard.

> +
> +static void append_or_move_hook(struct list_head *head, const char *command)
> +{
> +	struct list_head *pos = NULL, *tmp = NULL;
> +	struct hook *to_add = NULL;
> +
> +	/*
> +	 * remove the prior entry with this command; we'll replace it at the
> +	 * end.
> +	 */
> +	list_for_each_safe(pos, tmp, head) {
> +		struct hook *it = list_entry(pos, struct hook, list);
> +		if (!strcmp(it->command.buf, command)) {
> +		    list_del(pos);
> +		    /* we'll simply move the hook to the end */
> +		    to_add = it;
> +		    break;
> +		}
> +	}
> +
> +	if (!to_add) {
> +		/* adding a new hook, not moving an old one */
> +		to_add = xmalloc(sizeof(*to_add));
> +		strbuf_init(&to_add->command, 0);
> +		strbuf_addstr(&to_add->command, command);
> +	}
> +
> +	/* re-set the scope so we show where an override was specified */
> +	to_add->origin = current_config_scope();
> +
> +	list_add_tail(&to_add->list, head);
> +}
> +
> +static void remove_hook(struct list_head *to_remove)
> +{
> +	struct hook *hook_to_remove = list_entry(to_remove, struct hook, list);
> +	list_del(to_remove);
> +	free_hook(hook_to_remove);
> +}
> +
> +void clear_hook_list(struct list_head *head)
> +{
> +	struct list_head *pos, *tmp;
> +	list_for_each_safe(pos, tmp, head)
> +		remove_hook(pos);
> +}
> +
> +struct hook_config_cb
> +{
> +	struct strbuf *hookname;
> +	struct list_head *list;
> +};
> +
> +static int hook_config_lookup(const char *key, const char *value, void *cb_data)
> +{
> +	struct hook_config_cb *data = cb_data;
> +	const char *hook_key = data->hookname->buf;
> +	struct list_head *head = data->list;
> +
> +	if (!strcmp(key, hook_key)) {
> +		const char *command = value;
> +		struct strbuf hookcmd_name = STRBUF_INIT;
> +
> +		/*
> +		 * Check if a hookcmd with that name exists. If it doesn't,
> +		 * 'git_config_get_value()' is documented not to touch &command,
> +		 * so we don't need to do anything.
> +		 */
> +		strbuf_addf(&hookcmd_name, "hookcmd.%s.command", command);
> +		git_config_get_value(hookcmd_name.buf, &command);
> +
> +		if (!command) {
> +			strbuf_release(&hookcmd_name);
> +			BUG("git_config_get_value overwrote a string it shouldn't have");
> +		}
> +
> +		/*
> +		 * TODO: implement an option-getting callback, e.g.
> +		 *   get configs by pattern hookcmd.$value.*
> +		 *   for each key+value, do_callback(key, value, cb_data)
> +		 */
> +
> +		append_or_move_hook(head, command);
> +
> +		strbuf_release(&hookcmd_name);
> +	}
> +
> +	return 0;
> +}
> +
> +struct list_head* hook_list(const struct strbuf* hookname)
> +{
> +	struct strbuf hook_key = STRBUF_INIT;
> +	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
> +	struct hook_config_cb cb_data = { &hook_key, hook_head };
> +
> +	INIT_LIST_HEAD(hook_head);
> +
> +	if (!hookname)
> +		return NULL;

...if a strbuf being passed in is NULL?

> [...]
> +ROOT=
> +if test_have_prereq MINGW
> +then
> +	# In Git for Windows, Unix-like paths work only in shell scripts;
> +	# `git.exe`, however, will prefix them with the pseudo root directory
> +	# (of the Unix shell). Let's accommodate for that.
> +	ROOT="$(cd / && pwd)"
> +fi

I didn't read up on previous rounds, but if we're squashing this into 02
having a seperate commit summarizing this little hack would be most
welcome, or have it in this commit message.

Isn't this sort of thing generally usable, maybe we can add it under a
longer variable name to test-lib.sh?

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 04/37] hook: include hookdir hook in list
  2021-03-11  2:10 ` [PATCH v8 04/37] hook: include hookdir hook in list Emily Shaffer
@ 2021-03-12  8:30   ` Ævar Arnfjörð Bjarmason
  2021-03-24 17:56     ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-12  8:30 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Mar 11 2021, Emily Shaffer wrote:

> Historically, hooks are declared by placing an executable into
> $GIT_DIR/hooks/$HOOKNAME (or $HOOKDIR/$HOOKNAME). Although hooks taken
> from the config are more featureful than hooks placed in the $HOOKDIR,
> those hooks should not stop working for users who already have them.
> Let's list them to the user, but instead of displaying a config scope
> (e.g. "global: blah") we can prefix them with "hookdir:".
>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>
> Notes:
>     Since v7, fix some nits from Jonathan Tan. The largest is to move reference to
>     "hookdir annotation" from this commit to the next one which introduces the
>     hook.runHookDir option.
>
>  builtin/hook.c                | 11 +++++++++--
>  hook.c                        | 17 +++++++++++++++++
>  hook.h                        |  1 +
>  t/t1360-config-based-hooks.sh | 19 +++++++++++++++++++
>  4 files changed, 46 insertions(+), 2 deletions(-)
>
> diff --git a/builtin/hook.c b/builtin/hook.c
> index bb64cd77ca..c8fbfbb39d 100644
> --- a/builtin/hook.c
> +++ b/builtin/hook.c
> @@ -40,10 +40,15 @@ static int list(int argc, const char **argv, const char *prefix)
>  
>  	list_for_each(pos, head) {
>  		struct hook *item = list_entry(pos, struct hook, list);
> -		if (item)
> +		item = list_entry(pos, struct hook, list);
> +		if (item) {
> +			/* Don't translate 'hookdir' - it matches the config */

Let's prefix comments for translators with /* TRANSLATORS: .., see the
coding style doc. That's what they'll see, and this is useful to them.

Better yet have a note here about the first argument being 'system',
'local' etc., which I had to source spelunge for, and translators won't
have any idea about unless the magic parameter is documented.

> +setup_hookdir () {
> +	mkdir .git/hooks
> +	write_script .git/hooks/pre-commit <<-EOF
> +	echo \"Legacy Hook\"

Nit, "'s not needed, but it also seems nothing uses this, so if it's
just a pass-through script either "exit 0", or actually check if it's
run or something?

> [...]
> +test_expect_success 'git hook list shows hooks from the hookdir' '
> +	setup_hookdir &&
> +
> +	cat >expected <<-EOF &&
> +	hookdir: $(pwd)/.git/hooks/pre-commit
> +	EOF
> +
> +	git hook list pre-commit >actual &&
> +	test_cmp expected actual
> +'

Ah, so it's just checking if it exists...

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 05/37] hook: teach hook.runHookDir
  2021-03-11  2:10 ` [PATCH v8 05/37] hook: teach hook.runHookDir Emily Shaffer
@ 2021-03-12  8:33   ` Ævar Arnfjörð Bjarmason
  2021-03-24 18:46     ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-12  8:33 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Mar 11 2021, Emily Shaffer wrote:

> +	switch (should_run_hookdir) {
> +		case HOOKDIR_NO:

Style: case shouldn't be indented

> +			strbuf_addstr(&hookdir_annotation, _(" (will not run)"));
> +			break;
> +		case HOOKDIR_ERROR:
> +			strbuf_addstr(&hookdir_annotation, _(" (will error and not run)"));
> +			break;
> +		case HOOKDIR_INTERACTIVE:
> +			strbuf_addstr(&hookdir_annotation, _(" (will prompt)"));
> +			break;
> +		case HOOKDIR_WARN:
> +			strbuf_addstr(&hookdir_annotation, _(" (will warn but run)"));
> +			break;
> +		case HOOKDIR_YES:
> +		/*
> +		 * The default behavior should agree with
> +		 * hook.c:configured_hookdir_opt(). HOOKDIR_UNKNOWN should just
> +		 * do the default behavior.
> +		 */
> +		case HOOKDIR_UNKNOWN:
> +		default:
> +			break;

We should avoid this sort of translation lego.

> +	}
> +
>  	list_for_each(pos, head) {
>  		struct hook *item = list_entry(pos, struct hook, list);
>  		item = list_entry(pos, struct hook, list);
>  		if (item) {
>  			/* Don't translate 'hookdir' - it matches the config */
> -			printf("%s: %s\n",
> +			printf("%s: %s%s\n",

native speakers in some languages to read the sentance backwards.
Because if you concatenate strings like this you force.

(We don't currently have a RTL language in po/, still, but let's not
create churn for if/when we do if we can help it)>


I have a patch on top to fix this, will send it as some general reply of
proposed fixup.s

>  			       (item->from_hookdir
> +	git hook list pre-commit >actual &&
> +	# the hookdir annotation is translated
> +	test_i18ncmp expected actual

This (and the rest of test_i18ncmp in this series) can and should just
be "test_cmp" or "test_i18ncmp", the poison mode is dead. See my recent
patches to search/replace test_i18ncmp.

The reason the function isn't gone entirely was to help a series like
yours in "seen", but if we're re-rolling...

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 06/37] hook: implement hookcmd.<name>.skip
  2021-03-11  2:10 ` [PATCH v8 06/37] hook: implement hookcmd.<name>.skip Emily Shaffer
@ 2021-03-12  8:49   ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-12  8:49 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Mar 11 2021, Emily Shaffer wrote:

> +	cat >expected <<-EOF &&
> +	no commands configured for hook '\''pre-commit'\''
> +	EOF
> +
> +	git hook list pre-commit >actual &&
> +	test_i18ncmp expected actual
> +'
> +
> +test_expect_success 'git hook list ignores skip referring to unused hookcmd' '
> +	test_config hookcmd.abc.command "/path/abc" --add &&
> +	test_config hookcmd.abc.skip "true" --add &&
> +
> +	cat >expected <<-EOF &&
> +	no commands configured for hook '\''pre-commit'\''

ditto on the "echo" comment in a previous mail, looks like we can avoid
both of these entirely.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 07/37] parse-options: parse into strvec
  2021-03-11  2:10 ` [PATCH v8 07/37] parse-options: parse into strvec Emily Shaffer
@ 2021-03-12  8:50   ` Ævar Arnfjörð Bjarmason
  2021-03-24 20:34     ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-12  8:50 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Mar 11 2021, Emily Shaffer wrote:

> diff --git a/Documentation/technical/api-parse-options.txt b/Documentation/technical/api-parse-options.txt
> index 5a60bbfa7f..f79b17e7fc 100644
> --- a/Documentation/technical/api-parse-options.txt
> +++ b/Documentation/technical/api-parse-options.txt
> @@ -173,6 +173,13 @@ There are some macros to easily define options:
>  	The string argument is stored as an element in `string_list`.
>  	Use of `--no-option` will clear the list of preceding values.
>  
> +`OPT_STRVEC(short, long, &struct strvec, arg_str, description)`::
> +	Introduce an option with a string argument, meant to be specified
> +	multiple times.
> +	The string argument is stored as an element in `strvec`, and later
> +	arguments are added to the same `strvec`.
> +	Use of `--no-option` will clear the list of preceding values.
> +
>  `OPT_INTEGER(short, long, &int_var, description)`::
>  	Introduce an option with integer argument.
>  	The integer is put into `int_var`.
> diff --git a/parse-options-cb.c b/parse-options-cb.c
> index 4542d4d3f9..c2451dfb1b 100644
> --- a/parse-options-cb.c
> +++ b/parse-options-cb.c
> @@ -207,6 +207,22 @@ int parse_opt_string_list(const struct option *opt, const char *arg, int unset)
>  	return 0;
>  }
>  
> +int parse_opt_strvec(const struct option *opt, const char *arg, int unset)
> +{
> +	struct strvec *v = opt->value;
> +
> +	if (unset) {
> +		strvec_clear(v);
> +		return 0;
> +	}
> +
> +	if (!arg)
> +		return -1;
> +
> +	strvec_push(v, arg);
> +	return 0;
> +}
> +
>  int parse_opt_noop_cb(const struct option *opt, const char *arg, int unset)
>  {
>  	return 0;
> diff --git a/parse-options.h b/parse-options.h
> index ff6506a504..44c4ac08e9 100644
> --- a/parse-options.h
> +++ b/parse-options.h
> @@ -177,6 +177,9 @@ struct option {
>  #define OPT_STRING_LIST(s, l, v, a, h) \
>  				    { OPTION_CALLBACK, (s), (l), (v), (a), \
>  				      (h), 0, &parse_opt_string_list }
> +#define OPT_STRVEC(s, l, v, a, h) \
> +				    { OPTION_CALLBACK, (s), (l), (v), (a), \
> +				      (h), 0, &parse_opt_strvec }
>  #define OPT_UYN(s, l, v, h)         { OPTION_CALLBACK, (s), (l), (v), NULL, \
>  				      (h), PARSE_OPT_NOARG, &parse_opt_tertiary }
>  #define OPT_EXPIRY_DATE(s, l, v, h) \
> @@ -296,6 +299,7 @@ int parse_opt_commits(const struct option *, const char *, int);
>  int parse_opt_commit(const struct option *, const char *, int);
>  int parse_opt_tertiary(const struct option *, const char *, int);
>  int parse_opt_string_list(const struct option *, const char *, int);
> +int parse_opt_strvec(const struct option *, const char *, int);
>  int parse_opt_noop_cb(const struct option *, const char *, int);
>  enum parse_opt_result parse_opt_unknown_cb(struct parse_opt_ctx_t *ctx,
>  					   const struct option *,

Nice, seems very useful.

But let's add a test in test-parse-options.c like we have for
string_list?

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 08/37] hook: add 'run' subcommand
  2021-03-11  2:10 ` [PATCH v8 08/37] hook: add 'run' subcommand Emily Shaffer
@ 2021-03-12  8:54   ` Ævar Arnfjörð Bjarmason
  2021-03-24 21:29     ` Emily Shaffer
  2021-03-12 10:22   ` Junio C Hamano
  1 sibling, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-12  8:54 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Mar 11 2021, Emily Shaffer wrote:

>  'git hook' list <hook-name>
> +'git hook' run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] <hook-name>

[...]

> +	switch (cfg)
> +	{
> +		case HOOKDIR_ERROR:

Overly indented case statements again.

> +			fprintf(stderr, _("Skipping legacy hook at '%s'\n"),
> +				path);
> +			/* FALLTHROUGH */
> +		case HOOKDIR_NO:
> +			return 0;
> +		case HOOKDIR_WARN:
> +			fprintf(stderr, _("Running legacy hook at '%s'\n"),
> +				path);
> +			return 1;
> +		case HOOKDIR_INTERACTIVE:
> +			do {
> +				/*
> +				 * TRANSLATORS: Make sure to include [Y] and [n]
> +				 * in your translation. Only English input is
> +				 * accepted. Default option is "yes".
> +				 */
> +				fprintf(stderr, _("Run '%s'? [Yn] "), path);

Nit: [Y/n]

> +				} else if (starts_with(prompt.buf, "y")) {

So also "Y", "yes" and "yellow"...

> [...]
>  	git hook list pre-commit >actual &&
>  	# the hookdir annotation is translated
> -	test_i18ncmp expected actual
> +	test_i18ncmp expected actual &&
> +
> +	test_write_lines n | git hook run pre-commit 2>actual &&
> +	! grep "Legacy Hook" actual &&
> +
> +	test_write_lines y | git hook run pre-commit 2>actual &&
> +	grep "Legacy Hook" actual
> +'
> +
> +test_expect_success 'inline hook definitions execute oneliners' '
> +	test_config hook.pre-commit.command "echo \"Hello World\"" &&
> +
> +	echo "Hello World" >expected &&
> +
> +	# hooks are run with stdout_to_stderr = 1
> +	git hook run pre-commit 2>actual &&
> +	test_cmp expected actual
> +'
> +
> +test_expect_success 'inline hook definitions resolve paths' '
> +	write_script sample-hook.sh <<-EOF &&
> +	echo \"Sample Hook\"
> +	EOF
> +
> +	test_when_finished "rm sample-hook.sh" &&
> +
> +	test_config hook.pre-commit.command "\"$(pwd)/sample-hook.sh\"" &&
> +
> +	echo \"Sample Hook\" >expected &&
> +
> +	# hooks are run with stdout_to_stderr = 1
> +	git hook run pre-commit 2>actual &&
> +	test_cmp expected actual
> +'
> +
> +test_expect_success 'hookdir hook included in git hook run' '
> +	setup_hookdir &&
> +
> +	echo \"Legacy Hook\" >expected &&
> +
> +	# hooks are run with stdout_to_stderr = 1
> +	git hook run pre-commit 2>actual &&
> +	test_cmp expected actual
> +'
> +
> +test_expect_success 'out-of-repo runs excluded' '
> +	setup_hooks &&
> +
> +	nongit test_must_fail git hook run pre-commit
>  '
>  
>  test_expect_success 'hook.runHookDir is tolerant to unknown values' '

No tests for --env or --arg?

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 10/37] hook: support passing stdin to hooks
  2021-03-11  2:10 ` [PATCH v8 10/37] hook: support passing stdin to hooks Emily Shaffer
@ 2021-03-12  9:00   ` Ævar Arnfjörð Bjarmason
  2021-03-12 10:22   ` Junio C Hamano
  1 sibling, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-12  9:00 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Mar 11 2021, Emily Shaffer wrote:

> On the frontend, this is supported by asking for a file path, rather
> than by reading stdin. Reading directly from stdin would involve caching
> the entire stdin (to memory or to disk) and reading it back from the
> beginning to each hook. We'd want to support cases like insufficient
> memory or storage for the file. While this may prove useful later, for
> now the path of least resistance is to just ask the user to make this
> interim file themselves.

We need to worry about cases where we wouldn't have enough memory to
buffer the stdin, but still need to then do repo operations on the input
from such a file, presumably some giant pre-receive update or something.

Seems unlikely, and the convenience of having stdin just work by just
allocating that seems appealing, but let's read on to the rest of the
series...

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 17/37] hooks: allow callers to capture output
  2021-03-11  2:10 ` [PATCH v8 17/37] hooks: allow callers to capture output Emily Shaffer
@ 2021-03-12  9:08   ` Ævar Arnfjörð Bjarmason
  2021-03-24 21:54     ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-12  9:08 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Mar 11 2021, Emily Shaffer wrote:

> Some server-side hooks will require capturing output to send over
> sideband instead of printing directly to stderr. Expose that capability.

So added here in 17/37 and not used until 30/37. As a point on
readability (this isn't the first such patch) I think it would be better
to just squash those together with some "since we now need access to
consume_sideband in hooks, do that ...".

If there's a much larger API it makes sense to do it as another step...

>  hook.c | 3 ++-
>  hook.h | 8 ++++++++
>  2 files changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/hook.c b/hook.c
> index e16b082cbd..2322720ffe 100644
> --- a/hook.c
> +++ b/hook.c
> @@ -256,6 +256,7 @@ void run_hooks_opt_init_sync(struct run_hooks_opt *o)
>  	o->dir = NULL;
>  	o->feed_pipe = NULL;
>  	o->feed_pipe_ctx = NULL;
> +	o->consume_sideband = NULL;
>  }
>  
>  void run_hooks_opt_init_async(struct run_hooks_opt *o)
> @@ -434,7 +435,7 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options)
>  				   pick_next_hook,
>  				   notify_start_failure,
>  				   options->feed_pipe,
> -				   NULL,
> +				   options->consume_sideband,
>  				   notify_hook_finished,
>  				   &cb_data,
>  				   "hook",
> diff --git a/hook.h b/hook.h
> index ecf0228a46..4ff9999b04 100644
> --- a/hook.h
> +++ b/hook.h
> @@ -78,6 +78,14 @@ struct run_hooks_opt
>  	feed_pipe_fn feed_pipe;
>  	void *feed_pipe_ctx;
>  
> +	/*
> +	 * Populate this to capture output and prevent it from being printed to
> +	 * stderr. This will be passed directly through to
> +	 * run_command:run_parallel_processes(). See t/helper/test-run-command.c
> +	 * for an example.
> +	 */
> +	consume_sideband_fn consume_sideband;
> +
>  	/* Number of threads to parallelize across */
>  	int jobs;

...but this scaffolding is rather trivial.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 32/37] post-update: use hook.h library
  2021-03-11  2:10 ` [PATCH v8 32/37] post-update: use hook.h library Emily Shaffer
@ 2021-03-12  9:14   ` Ævar Arnfjörð Bjarmason
  2021-03-30  0:01     ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-12  9:14 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Mar 11 2021, Emily Shaffer wrote:

> By using run_hooks() instead of run_hook_le(), 'post-update' hooks can
> be specified in the config as well as the hookdir.

Looking ahead in the series no tests for this, seems like a good thing
to have some at least trivial tests for each hook and their config
invocation.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 35/37] git-send-email: use 'git hook run' for 'sendemail-validate'
  2021-03-11  2:10 ` [PATCH v8 35/37] git-send-email: use 'git hook run' for 'sendemail-validate' Emily Shaffer
@ 2021-03-12  9:21   ` Ævar Arnfjörð Bjarmason
  2021-03-30  0:03     ` Emily Shaffer
  2021-03-31 21:47     ` Emily Shaffer
  2021-03-12 23:29   ` [PATCH v8 35/37] git-send-email: use 'git hook run' for 'sendemail-validate' Emily Shaffer
  1 sibling, 2 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-12  9:21 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Mar 11 2021, Emily Shaffer wrote:

> By using the new 'git hook run' subcommand to run 'sendemail-validate',
> we can reduce the boilerplate needed to run this hook in perl. Using
> config-based hooks also allows us to run 'sendemail-validate' hooks that
> were configured globally when running 'git send-email' from outside of a
> Git directory, alongside other benefits like multihooks and
> parallelization.
>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  git-send-email.perl   | 21 ++++-----------------
>  t/t9001-send-email.sh | 11 +----------
>  2 files changed, 5 insertions(+), 27 deletions(-)
>
> diff --git a/git-send-email.perl b/git-send-email.perl
> index 1f425c0809..73e1e0b51a 100755
> --- a/git-send-email.perl
> +++ b/git-send-email.perl
> @@ -1941,23 +1941,10 @@ sub unique_email_list {
>  sub validate_patch {
>  	my ($fn, $xfer_encoding) = @_;
>  
> -	if ($repo) {
> -		my $validate_hook = catfile(catdir($repo->repo_path(), 'hooks'),
> -					    'sendemail-validate');
> -		my $hook_error;
> -		if (-x $validate_hook) {
> -			my $target = abs_path($fn);
> -			# The hook needs a correct cwd and GIT_DIR.
> -			my $cwd_save = cwd();
> -			chdir($repo->wc_path() or $repo->repo_path())
> -				or die("chdir: $!");
> -			local $ENV{"GIT_DIR"} = $repo->repo_path();
> -			$hook_error = "rejected by sendemail-validate hook"
> -				if system($validate_hook, $target);
> -			chdir($cwd_save) or die("chdir: $!");
> -		}
> -		return $hook_error if $hook_error;
> -	}
> +	my $target = abs_path($fn);
> +	return "rejected by sendemail-validate hook"
> +		if system(("git", "hook", "run", "sendemail-validate", "-a",
> +				$target));

I see it's just moving code around, but since we're touching this:

This conflates the hook exit code with a general failure to invoke it,
Perl's system().

Not a big deal in this case, but there's two other existing system()
invocations which use the right blurb for it:


	system('sh', '-c', $editor.' "$@"', $editor, $_);
	if (($? & 127) || ($? >> 8)) {
		die(__("the editor exited uncleanly, aborting everything"));
	}

Makes sense to do something similar here for consistency. See "perldoc
-f system" for an example.

>  
>  	# Any long lines will be automatically fixed if we use a suitable transfer
>  	# encoding.
> diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
> index 4eee9c3dcb..456b471c5c 100755
> --- a/t/t9001-send-email.sh
> +++ b/t/t9001-send-email.sh
> @@ -2101,16 +2101,7 @@ test_expect_success $PREREQ 'invoke hook' '
>  	mkdir -p .git/hooks &&
>  
>  	write_script .git/hooks/sendemail-validate <<-\EOF &&
> -	# test that we have the correct environment variable, pwd, and
> -	# argument
> -	case "$GIT_DIR" in
> -	*.git)
> -		true
> -		;;
> -	*)
> -		false
> -		;;
> -	esac &&
> +	# test that we have the correct argument

This and getting rid of these Perl/Python/whatever special cases is very
nice.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 36/37] run-command: stop thinking about hooks
  2021-03-11  2:10 ` [PATCH v8 36/37] run-command: stop thinking about hooks Emily Shaffer
@ 2021-03-12  9:23   ` Ævar Arnfjörð Bjarmason
  2021-03-30  0:07     ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-12  9:23 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Mar 11 2021, Emily Shaffer wrote:

> hook.h has replaced all run-command.h hook-related functionality.
> run-command.h:run_hooks_le/ve and find_hook are no longer used anywhere
> in the codebase. So, let's delete the dead code - or, in the one case
> where it's still needed, move it to an internal function in hook.c.

Similar to other comments about squashing, I think just having this
happen incrementally as we remove whatever is the last user of the
function would be better.

E.g. find_hook() is last used in one commit, run_hook*() in another...

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 37/37] docs: unify githooks and git-hook manpages
  2021-03-11  2:10 ` [PATCH v8 37/37] docs: unify githooks and git-hook manpages Emily Shaffer
@ 2021-03-12  9:29   ` Ævar Arnfjörð Bjarmason
  2021-03-30  0:10     ` Emily Shaffer
  2021-04-07  2:36   ` Junio C Hamano
  1 sibling, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-12  9:29 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Mar 11 2021, Emily Shaffer wrote:

> By showing the list of all hooks in 'git help hook' for users to refer
> to, 'git help hook' becomes a one-stop shop for hook authorship. Since
> some may still have muscle memory for 'git help githooks', though,
> reference the 'git hook' commands and otherwise don't remove content.

I think this should at least have something like what my b6a8d09f6d8 (gc
docs: include the "gc.*" section from "config" in "gc", 2019-04-07) has
on top, i.e.:
    
    diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
    index 4ad31ac360a..5c9af30b43e 100644
    --- a/Documentation/git-hook.txt
    +++ b/Documentation/git-hook.txt
    @@ -150,10 +150,18 @@ message body and cannot be parallelized.
     
     CONFIGURATION
     -------------
    +
    +The below documentation is the same as what's found in
    +linkgit:git-config[1]:
    +
     include::config/hook.txt[]
     
     HOOKS
     -----
    +
    +The below documentation is the same as what's found in
    +linkgit:githooks[5]:
    +
     include::native-hooks.txt[]
     
     GIT

But I also don't think we should demote githooks(5) as the canonical doc
page for the hooks themselves.

If you run this in your terminal:

    man 5 git<TAB>

You'll get:

    gitattributes         gitignore             gitmailmap            gitrepository-layout  
    githooks              git-lfs-config        gitmodules            gitweb.conf 

(Well, maybe not the lfs-part, but whatever...).

We should move more in the direction of splitting up our "file format"
docs from implementation, like the git-hook runner.

I'm somewhat negative on including it at all in git-hook(1). For the
config section it makes sense, and it's consistent with established doc
convention.

But including githooks(5) is around 2/3 of the resulting manpage, I
think just a link is better.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 00/37] config-based hooks
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (37 preceding siblings ...)
  2021-03-11 22:26 ` [PATCH v8 00/37] config-based hooks Junio C Hamano
@ 2021-03-12  9:49 ` Ævar Arnfjörð Bjarmason
  2021-03-17 18:41   ` Emily Shaffer
  2021-03-12 11:13 ` Ævar Arnfjörð Bjarmason
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
  40 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-12  9:49 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: git, Jeff King, Junio C Hamano, James Ramsay, Jonathan Nieder,
	brian m. carlson, Phillip Wood, Josh Steadmon,
	Johannes Schindelin, Jonathan Tan


On Thu, Mar 11 2021, Emily Shaffer wrote:

> Since v7:
> - Addressed Jonathan Tan's review of part I
> - Addressed Junio's review of part I and II
> - Combined parts I and II
>
> I think the updates to patch 1 between the rest of the work I've been
> doing probably have covered Ævar's comments.

A range-diff between iterations of such a large series would be most
useful. Do you have a public repo with tags or whatever the different
versions, for those who'd like an easier way to follow along the
differing versions than scraping the ML archive?

While reading this I came up with the following fixup patches on top,
for discussion, maybe not something you want as-is:
	
	 Documentation/git-hook.txt |  8 +++++
	 builtin/bugreport.c        |  8 +++--
	 builtin/commit.c           |  3 +-
	 builtin/hook.c             | 79 ++++++++++++++++++++--------------------------
	 builtin/merge.c            |  3 +-
	 builtin/receive-pack.c     | 11 +++----
	 hook.c                     | 21 +++++-------
	 hook.h                     |  5 +--
	 refs.c                     |  4 ++-
	 sequencer.c                |  4 ++-
	 10 files changed, 73 insertions(+), 73 deletions(-)
	
	diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
	index 4ad31ac360a..5c9af30b43e 100644
	--- a/Documentation/git-hook.txt
	+++ b/Documentation/git-hook.txt
	@@ -150,10 +150,18 @@ message body and cannot be parallelized.
	 
	 CONFIGURATION
	 -------------
	+
	+The below documentation is the same as what's found in
	+linkgit:git-config[1]:
	+
	 include::config/hook.txt[]
	 
	 HOOKS
	 -----
	+
	+The below documentation is the same as what's found in
	+linkgit:githooks[5]:
	+
	 include::native-hooks.txt[]
	 
Noted in another reply, including it here for completeness.

	 GIT
	diff --git a/builtin/bugreport.c b/builtin/bugreport.c
	index 04467cd1d3a..b64e53fd625 100644
	--- a/builtin/bugreport.c
	+++ b/builtin/bugreport.c
	@@ -81,9 +81,13 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
	 		return;
	 	}
	 
	-	for (i = 0; i < ARRAY_SIZE(hook); i++)
	-		if (hook_exists(hook[i], HOOKDIR_USE_CONFIG))
	+	for (i = 0; i < ARRAY_SIZE(hook); i++) {
	+		struct strbuf config;
	+		strbuf_addf(&config, "hook.%s.config", hook[i]);
	+		if (hook_exists(hook[i], config.buf, HOOKDIR_USE_CONFIG))
	 			strbuf_addf(hook_info, "%s\n", hook[i]);
	+		strbuf_release(&config);
	+	}
	 }

Less strbuf, see below.
	 
	 static const char * const bugreport_usage[] = {
	diff --git a/builtin/commit.c b/builtin/commit.c
	index 31df571f123..fc9f1f5ee58 100644
	--- a/builtin/commit.c
	+++ b/builtin/commit.c
	@@ -984,7 +984,8 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
	 		return 0;
	 	}
	 
	-	if (!no_verify && hook_exists("pre-commit", HOOKDIR_USE_CONFIG)) {
	+	if (!no_verify && hook_exists("pre-commit", "hook.pre-commit.command",
	+				      HOOKDIR_USE_CONFIG)) {
	 		/*
	 		 * Re-read the index as pre-commit hook could have updated it,
	 		 * and write it out as a tree.  We must do this before we invoke


..ditto.

	diff --git a/builtin/hook.c b/builtin/hook.c
	index b4f4adb1dea..d0b56ee47f8 100644
	--- a/builtin/hook.c
	+++ b/builtin/hook.c
	@@ -18,8 +18,6 @@ static enum hookdir_opt should_run_hookdir;
	 static int list(int argc, const char **argv, const char *prefix)
	 {
	 	struct list_head *head, *pos;
	-	struct strbuf hookname = STRBUF_INIT;
	-	struct strbuf hookdir_annotation = STRBUF_INIT;
	 
	 	struct option list_options[] = {
	 		OPT_END(),
	@@ -33,67 +31,60 @@ static int list(int argc, const char **argv, const char *prefix)
	 			      builtin_hook_usage, list_options);
	 	}
	 
	-	strbuf_addstr(&hookname, argv[0]);
	-
	-	head = hook_list(&hookname);
	+	head = hook_list(argv[0]);
	 
	 	if (list_empty(head)) {
	 		printf(_("no commands configured for hook '%s'\n"),
	-		       hookname.buf);
	-		strbuf_release(&hookname);
	+		       argv[0]);
	 		return 0;
	 	}
	 
	-	switch (should_run_hookdir) {
	-		case HOOKDIR_NO:
	-			strbuf_addstr(&hookdir_annotation, _(" (will not run)"));
	-			break;
	-		case HOOKDIR_ERROR:
	-			strbuf_addstr(&hookdir_annotation, _(" (will error and not run)"));
	-			break;
	-		case HOOKDIR_INTERACTIVE:
	-			strbuf_addstr(&hookdir_annotation, _(" (will prompt)"));
	-			break;
	-		case HOOKDIR_WARN:
	-			strbuf_addstr(&hookdir_annotation, _(" (will warn but run)"));
	-			break;
	-		case HOOKDIR_YES:
	-		/*
	-		 * The default behavior should agree with
	-		 * hook.c:configured_hookdir_opt(). HOOKDIR_UNKNOWN should just
	-		 * do the default behavior.
	-		 */
	-		case HOOKDIR_UNKNOWN:
	-		default:
	-			break;
	-	}
	-
	 	list_for_each(pos, head) {
	 		struct hook *item = list_entry(pos, struct hook, list);
	 		item = list_entry(pos, struct hook, list);
	 		if (item) {
	-			/* Don't translate 'hookdir' - it matches the config */
	-			printf("%s: %s%s\n",
	-			       (item->from_hookdir
	+			const char *scope = item->from_hookdir
	 				? "hookdir"
	-				: config_scope_name(item->origin)),
	-			       item->command.buf,
	-			       (item->from_hookdir
	-				? hookdir_annotation.buf
	-				: ""));
	+				: config_scope_name(item->origin);
	+			switch (should_run_hookdir) {
	+			case HOOKDIR_NO:
	+				printf(_("%s: %s (will not run)\n"),
	+				       scope, item->command.buf);
	+				break;
	+			case HOOKDIR_ERROR:
	+				printf(_("%s: %s (will error and not run)\n"),
	+				       scope, item->command.buf);
	+				break;
	+			case HOOKDIR_INTERACTIVE:
	+				printf(_("%s: %s (will prompt)\n"),
	+				       scope, item->command.buf);
	+				break;
	+			case HOOKDIR_WARN:
	+				printf(_("%s: %s (will warn but run)\n"),
	+				       scope, item->command.buf);
	+				break;
	+			case HOOKDIR_YES:
	+				/*
	+				 * The default behavior should agree with
	+				 * hook.c:configured_hookdir_opt(). HOOKDIR_UNKNOWN should just
	+				 * do the default behavior.
	+				 */
	+			case HOOKDIR_UNKNOWN:
	+			default:
	+				printf(_("%s: %s\n"),
	+				       scope, item->command.buf);
	+				break;
	+			}
	 		}
	 	}
	 
	 	clear_hook_list(head);
	-	strbuf_release(&hookdir_annotation);
	-	strbuf_release(&hookname);
	 
	 	return 0;
	 }

I think this is better to avoid i18n lego, as noted in another reply
(but I didn't include the patch).

More on strbuf below:
	 
	 static int run(int argc, const char **argv, const char *prefix)
	 {
	-	struct strbuf hookname = STRBUF_INIT;
	 	struct run_hooks_opt opt;
	 	int rc = 0;
	 
	@@ -118,12 +109,10 @@ static int run(int argc, const char **argv, const char *prefix)
	 		usage_msg_opt(_("You must specify a hook event to run."),
	 			      builtin_hook_usage, run_options);
	 
	-	strbuf_addstr(&hookname, argv[0]);
	 	opt.run_hookdir = should_run_hookdir;
	 
	-	rc = run_hooks(hookname.buf, &opt);
	+	rc = run_hooks(argv[0], &opt);
	 
	-	strbuf_release(&hookname);
	 	run_hooks_opt_clear(&opt);
	 
	 	return rc;
	diff --git a/builtin/merge.c b/builtin/merge.c
	index 3a2af257a6b..df4ff72fbc7 100644
	--- a/builtin/merge.c
	+++ b/builtin/merge.c
	@@ -848,7 +848,8 @@ static void prepare_to_commit(struct commit_list *remoteheads)
	 	 * and write it out as a tree.  We must do this before we invoke
	 	 * the editor and after we invoke run_status above.
	 	 */
	-	if (hook_exists("pre-merge-commit", HOOKDIR_USE_CONFIG))
	+	if (hook_exists("pre-merge-commit", "hook.pre-merge-commit.command",
	+			HOOKDIR_USE_CONFIG))
	 		discard_cache();
	 	read_cache_from(index_file);
	 	strbuf_addbuf(&msg, &merge_msg);
	diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
	index eaedeeb1e8b..a76069ea592 100644
	--- a/builtin/receive-pack.c
	+++ b/builtin/receive-pack.c
	@@ -1123,12 +1123,10 @@ static int run_proc_receive_hook(struct command *commands,
	 	int version = 0;
	 	int code;
	 
	-	struct strbuf hookname = STRBUF_INIT;
	 	struct hook *proc_receive = NULL;
	 	struct list_head *pos, *hooks;
	 
	-	strbuf_addstr(&hookname, "proc-receive");
	-	hooks = hook_list(&hookname);
	+	hooks = hook_list("proc-receive");
	 
	 	list_for_each(pos, hooks) {
	 		if (proc_receive) {
	@@ -1460,8 +1458,6 @@ static const char *push_to_deploy(unsigned char *sha1,
	 	return NULL;
	 }
	 
	-static const char *push_to_checkout_hook = "push-to-checkout";
	-
	 static const char *push_to_checkout(unsigned char *hash,
	 				    struct strvec *env,
	 				    const char *work_tree)
	@@ -1472,7 +1468,7 @@ static const char *push_to_checkout(unsigned char *hash,
	 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
	 	strvec_pushv(&opt.env, env->v);
	 	strvec_push(&opt.args, hash_to_hex(hash));
	-	if (run_hooks(push_to_checkout_hook, &opt)) {
	+	if (run_hooks("push-to-checkout", &opt)) {
	 		run_hooks_opt_clear(&opt);
	 		return "push-to-checkout hook declined";
	 	} else {
	@@ -1502,7 +1498,8 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
	 
	 	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
	 
	-	if (!hook_exists(push_to_checkout_hook, HOOKDIR_USE_CONFIG))
	+	if (!hook_exists("push-to-checkout", "hook.push-to-checkout.command",
	+			 HOOKDIR_USE_CONFIG))
	 		retval = push_to_deploy(sha1, &env, work_tree);
	 	else
	 		retval = push_to_checkout(sha1, &env, work_tree);
	diff --git a/hook.c b/hook.c
	index 7f6f3b9a616..49c3861ce00 100644
	--- a/hook.c
	+++ b/hook.c
	@@ -247,7 +247,7 @@ static const char *find_legacy_hook(const char *name)
	 }
	 
	 
	-struct list_head* hook_list(const struct strbuf* hookname)
	+struct list_head* hook_list(const char *hookname)
	 {
	 	struct strbuf hook_key = STRBUF_INIT;
	 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
	@@ -256,14 +256,14 @@ struct list_head* hook_list(const struct strbuf* hookname)
	 	INIT_LIST_HEAD(hook_head);
	 
	 	if (!hookname)
	-		return NULL;
	+		BUG("???");;
	 
	-	strbuf_addf(&hook_key, "hook.%s.command", hookname->buf);
	+	strbuf_addf(&hook_key, "hook.%s.command", hookname);
	 
	 	git_config(hook_config_lookup, &cb_data);
	 
	 	if (have_git_dir()) {
	-		const char *legacy_hook_path = find_legacy_hook(hookname->buf);
	+		const char *legacy_hook_path = find_legacy_hook(hookname);
	 
	 		/* Unconditionally add legacy hook, but annotate it. */
	 		if (legacy_hook_path) {
	@@ -300,10 +300,10 @@ void run_hooks_opt_init_async(struct run_hooks_opt *o)
	 	o->jobs = configured_hook_jobs();
	 }
	 
	-int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir)
	+int hook_exists(const char *hookname, const char *hook_config,
	+		enum hookdir_opt should_run_hookdir)
	 {
	 	const char *value = NULL; /* throwaway */
	-	struct strbuf hook_key = STRBUF_INIT;
	 	int could_run_hookdir;
	 
	 	if (should_run_hookdir == HOOKDIR_USE_CONFIG)
	@@ -314,9 +314,7 @@ int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir)
	 				should_run_hookdir == HOOKDIR_YES)
	 				&& !!find_legacy_hook(hookname);
	 
	-	strbuf_addf(&hook_key, "hook.%s.command", hookname);
	-
	-	return (!git_config_get_value(hook_key.buf, &value)) || could_run_hookdir;
	+	return (!git_config_get_value(hook_config, &value)) || could_run_hookdir;
	 }
	 
	 void run_hooks_opt_clear(struct run_hooks_opt *o)
	@@ -438,7 +436,6 @@ static int notify_hook_finished(int result,
	 
	 int run_hooks(const char *hookname, struct run_hooks_opt *options)
	 {
	-	struct strbuf hookname_str = STRBUF_INIT;
	 	struct list_head *to_run, *pos = NULL, *tmp = NULL;
	 	struct hook_cb_data cb_data = { 0, NULL, NULL, options };
	 
	@@ -448,9 +445,7 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options)
	 	if (options->path_to_stdin && options->feed_pipe)
	 		BUG("choose only one method to populate stdin");
	 
	-	strbuf_addstr(&hookname_str, hookname);
	-
	-	to_run = hook_list(&hookname_str);
	+	to_run = hook_list(hookname);
	 
	 	list_for_each_safe(pos, tmp, to_run) {
	 		struct hook *hook = list_entry(pos, struct hook, list);
	diff --git a/hook.h b/hook.h
	index 4ff9999b049..bfbbf36882d 100644
	--- a/hook.h
	+++ b/hook.h
	@@ -26,7 +26,7 @@ struct hook {
	  * Provides a linked list of 'struct hook' detailing commands which should run
	  * in response to the 'hookname' event, in execution order.
	  */
	-struct list_head* hook_list(const struct strbuf *hookname);
	+struct list_head* hook_list(const char *hookname);
	 
	 enum hookdir_opt
	 {
	@@ -123,7 +123,8 @@ void run_hooks_opt_clear(struct run_hooks_opt *o);
	  * Like with run_hooks, if you take a --run-hookdir flag, reflect that
	  * user-specified behavior here instead.
	  */
	-int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir);
	+int hook_exists(const char *hookname, const char *hook_config,
	+		enum hookdir_opt should_run_hookdir);
	 
	 /*
	  * Runs all hooks associated to the 'hookname' event in order. Each hook will be
	diff --git a/refs.c b/refs.c
	index 334fdd9103c..f01995fe64f 100644
	--- a/refs.c
	+++ b/refs.c
	@@ -1966,7 +1966,9 @@ static int run_transaction_hook(struct ref_transaction *transaction,
	 
	 	run_hooks_opt_init_async(&opt);
	 
	-	if (!hook_exists("reference-transaction", HOOKDIR_USE_CONFIG))
	+	if (!hook_exists("reference-transaction",
	+			 "hook.reference-transaction.command",
	+			 HOOKDIR_USE_CONFIG))
	 		return ret;
	 
	 	strvec_push(&opt.args, state);
	diff --git a/sequencer.c b/sequencer.c
	index 34ff275f0d1..52c067c1688 100644
	--- a/sequencer.c
	+++ b/sequencer.c
	@@ -1436,7 +1436,9 @@ static int try_to_commit(struct repository *r,
	 		}
	 	}
	 
	-	if (hook_exists("prepare-commit-msg", HOOKDIR_USE_CONFIG)) {
	+	if (hook_exists("prepare-commit-msg",
	+			"hook.prepare-commit-msg.command",
	+			HOOKDIR_USE_CONFIG)) {
	 		res = run_prepare_commit_msg_hook(r, msg, hook_commit);
	 		if (res)
	 			goto out;

There was another reply (from JT I believe, but didn't go back and look
it up) about the over use of strbuf.

I tend to agree, as much as I love the API it's really not better to
write C with it if all you need is a const char* that's never modified,
particularly if you get it from elsewhere.

So it's really not meant for or good for "everything we need a const
char*", but to avoid verbose realloc() dances all over the place, and
for things like getline() loops without a hardcoded buffer size.

E.g. in the first hunk here we're creating a strbuf just to copy argv[0]
to it, and then throwing it away, let's just pass down argv[0].

For hook_exists I think just having the code more grep-able and having
the config value inline is better, but I admit that's a matter of taste.

I didn't try to find all such strbuf() occurrences, anyway, in the
overall scheme of things it's a relatively small nit.

I'm hoping to do some deeper diving into this series, in particular the
parallelism, but just sending the shallow-ish comments I have for now.

Thanks for working on this!

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 23/37] read-cache: convert post-index-change hook to use config
  2021-03-11  2:10 ` [PATCH v8 23/37] read-cache: convert post-index-change hook to use config Emily Shaffer
@ 2021-03-12 10:22   ` Junio C Hamano
  2021-03-29 23:56     ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Junio C Hamano @ 2021-03-12 10:22 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> @@ -3070,6 +3071,8 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
>  				 unsigned flags)
>  {
>  	int ret;
> +	struct run_hooks_opt hook_opt;
> +	run_hooks_opt_init_async(&hook_opt);
>  

Nit. blank line between the last of decls and the first stmt (many
identical nits exist everywhere in this series).

>  	/*
>  	 * TODO trace2: replace "the_repository" with the actual repo instance
> @@ -3088,9 +3091,13 @@ static int do_write_locked_index(s
>  	else
>  		ret = close_lock_file_gently(lock);
>  
> -	run_hook_le(NULL, "post-index-change",
> -			istate->updated_workdir ? "1" : "0",
> -			istate->updated_skipworktree ? "1" : "0", NULL);
> +	strvec_pushl(&hook_opt.args,
> +		     istate->updated_workdir ? "1" : "0",
> +		     istate->updated_skipworktree ? "1" : "0",
> +		     NULL);
> +	run_hooks("post-index-change", &hook_opt);
> +	run_hooks_opt_clear(&hook_opt);

There is one early return before the precontext of this hunk that
bypasses this opt_clear() call.  It is before any member of hook_opt
structure that was opt_init()'ed gets touched, so with the current
code, there is no leak, but it probably is laying a landmine for the
future, where opt_init() may allocate some resource to its member,
with the expectation that all users of the API would call
opt_clear() to release.  Or the caller of the API (like this one) may
start mucking with the opt structure before the existing early return,
at which point the current assumption that it is safe to return from
that point without opt_clear() would be broken.

I saw that there are other early returns in the series that are safe
right now but may become unsafe when the API implementation gets
extended that way.  If it does not involve too much code churning,
we may want to restructure the code to make these early returns into
"goto"s that jump to a single exit point, so that we can always
match opt_init() with opt_clear(), like the structure of the
existing code allowed cmd_rebase() to use the hooks API cleanly in
[v8 22/37].

Thanks.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 08/37] hook: add 'run' subcommand
  2021-03-11  2:10 ` [PATCH v8 08/37] hook: add 'run' subcommand Emily Shaffer
  2021-03-12  8:54   ` Ævar Arnfjörð Bjarmason
@ 2021-03-12 10:22   ` Junio C Hamano
  1 sibling, 0 replies; 479+ messages in thread
From: Junio C Hamano @ 2021-03-12 10:22 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> +static int run(int argc, const char **argv, const char *prefix)
> +{
> +	struct strbuf hookname = STRBUF_INIT;
> +	struct run_hooks_opt opt;
> +	int rc = 0;
> +
> +	struct option run_options[] = {
> +		OPT_STRVEC('e', "env", &opt.env, N_("var"),
> +			   N_("environment variables for hook to use")),
> +		OPT_STRVEC('a', "arg", &opt.args, N_("args"),
> +			   N_("argument to pass to hook")),
> +		OPT_END(),
> +	};
> +
> +	run_hooks_opt_init(&opt);
> +
> +	argc = parse_options(argc, argv, prefix, run_options,
> +			     builtin_hook_usage, 0);
> +
> +	if (argc < 1)
> +		usage_msg_opt(_("You must specify a hook event to run."),
> +			      builtin_hook_usage, run_options);
> +
> +	strbuf_addstr(&hookname, argv[0]);
> +	opt.run_hookdir = should_run_hookdir;
> +
> +	rc = run_hooks(hookname.buf, &opt);
> +
> +	strbuf_release(&hookname);
> +	run_hooks_opt_clear(&opt);
> +
> +	return rc;
> +}

This looks like a small and clean example that is good for people to
emulate when using the new run-hooks API.  You opt_init(), futz with
the its fields, call run_hooks(), and finally opt_clear() to release
the resources.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 10/37] hook: support passing stdin to hooks
  2021-03-11  2:10 ` [PATCH v8 10/37] hook: support passing stdin to hooks Emily Shaffer
  2021-03-12  9:00   ` Ævar Arnfjörð Bjarmason
@ 2021-03-12 10:22   ` Junio C Hamano
  1 sibling, 0 replies; 479+ messages in thread
From: Junio C Hamano @ 2021-03-12 10:22 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> diff --git a/hook.c b/hook.c
> index 118931f273..f906e8c61c 100644
> --- a/hook.c
> +++ b/hook.c
> @@ -240,6 +240,7 @@ void run_hooks_opt_init(struct run_hooks_opt *o)
>  {
>  	strvec_init(&o->env);
>  	strvec_init(&o->args);
> +	o->path_to_stdin = NULL;
>  	o->run_hookdir = configured_hookdir_opt();
>  }

The new member is initialized to NULL, and presumably the user of
the API would point an existing string with it.  Since there is no
free() in opt_clear() introduced by this patch, the member is
obviously a pointer to a borrowed piece of memory.

> diff --git a/hook.h b/hook.h
> index 0df785add5..2314ec5962 100644
> --- a/hook.h
> +++ b/hook.h
> @@ -52,6 +52,9 @@ struct run_hooks_opt
>  	 * to be overridden if the user can override it at the command line.
>  	 */
>  	enum hookdir_opt run_hookdir;
> +
> +	/* Path to file which should be piped to stdin for each hook */
> +	const char *path_to_stdin;
>  };

And we mark the fact that hook subsystem does not own it by making
it "const char *".  Looks quite consistent.  Good.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 18/37] commit: use config-based hooks
  2021-03-11  2:10 ` [PATCH v8 18/37] commit: use config-based hooks Emily Shaffer
@ 2021-03-12 10:22   ` Junio C Hamano
  0 siblings, 0 replies; 479+ messages in thread
From: Junio C Hamano @ 2021-03-12 10:22 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> -int run_commit_hook(int editor_is_used, const char *index_file,
> +int run_commit_hook(int editor_is_used, int parallelize, const char *index_file,
>  		    const char *name, ...)
>  {
> -	struct strvec hook_env = STRVEC_INIT;
> +	struct run_hooks_opt opt;
>  	va_list args;
> +	const char *arg;
>  	int ret;
>  
> -	strvec_pushf(&hook_env, "GIT_INDEX_FILE=%s", index_file);
> +	run_hooks_opt_init_sync(&opt);
> +
> +	if (parallelize)
> +		opt.jobs = configured_hook_jobs();
> +
> +	strvec_pushf(&opt.env, "GIT_INDEX_FILE=%s", index_file);
>  
>  	/*
>  	 * Let the hook know that no editor will be launched.
>  	 */
>  	if (!editor_is_used)
> -		strvec_push(&hook_env, "GIT_EDITOR=:");
> +		strvec_push(&opt.env, "GIT_EDITOR=:");
>  
>  	va_start(args, name);
> -	ret = run_hook_ve(hook_env.v, name, args);
> +	while ((arg = va_arg(args, const char *)))
> +		strvec_push(&opt.args, arg);
>  	va_end(args);
> -	strvec_clear(&hook_env);
> +
> +	ret = run_hooks(name, &opt);
> +	run_hooks_opt_clear(&opt);
>  
>  	return ret;
>  }

This follows the textbook pattern established earlier and
demonstrated in [v8 08/37].  opt_init() to initialize, populate its
members, call run_hooks() and finally opt_clear().

Quite nicely demonstrated.


^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 19/37] am: convert applypatch hooks to use config
  2021-03-11  2:10 ` [PATCH v8 19/37] am: convert applypatch hooks to use config Emily Shaffer
@ 2021-03-12 10:23   ` Junio C Hamano
  2021-03-29 23:39     ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Junio C Hamano @ 2021-03-12 10:23 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> @@ -1558,8 +1563,10 @@ static void do_commit(const struct am_state *state)
>  	struct commit_list *parents = NULL;
>  	const char *reflog_msg, *author, *committer = NULL;
>  	struct strbuf sb = STRBUF_INIT;
> +	struct run_hooks_opt hook_opt;
> +	run_hooks_opt_init_async(&hook_opt);
>  
> -	if (run_hook_le(NULL, "pre-applypatch", NULL))
> +	if (run_hooks("pre-applypatch", &hook_opt))
>  		exit(1);
>  
>  	if (write_cache_as_tree(&tree, 0, NULL))
> @@ -1611,8 +1618,9 @@ static void do_commit(const struct am_state *state)
>  		fclose(fp);
>  	}
>  
> -	run_hook_le(NULL, "post-applypatch", NULL);
> +	run_hooks("post-applypatch", &hook_opt);
>  
> +	run_hooks_opt_clear(&hook_opt);
>  	strbuf_release(&sb);
>  }

This one does opt_init(), run_hooks(), and another run_hooks() and
then opt_clear().  If run_hooks() is a read-only operation on the
hook_opt, then that would be alright, but it just smells iffy that
it is not done as two separate opt_init(), run_hooks(), opt_clear()
sequences for two separate run_hooks() invocations.  The same worry
about future safety I meantioned elsewhere in the series also
applies.

Thanks.



^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 22/37] rebase: teach pre-rebase to use hook.h
  2021-03-11  2:10 ` [PATCH v8 22/37] rebase: teach pre-rebase to use hook.h Emily Shaffer
@ 2021-03-12 10:24   ` Junio C Hamano
  0 siblings, 0 replies; 479+ messages in thread
From: Junio C Hamano @ 2021-03-12 10:24 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> @@ -1318,6 +1319,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
>  	char *squash_onto_name = NULL;
>  	int reschedule_failed_exec = -1;
>  	int allow_preemptive_ff = 1;
> +	struct run_hooks_opt hook_opt;
>  	struct option builtin_rebase_options[] = {
>  		OPT_STRING(0, "onto", &options.onto_name,
>  			   N_("revision"),
> @@ -1431,6 +1433,8 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
>  	};
>  	int i;
>  
> +	run_hooks_opt_init_async(&hook_opt);
> +
>  	if (argc == 2 && !strcmp(argv[1], "-h"))
>  		usage_with_options(builtin_rebase_usage,
>  				   builtin_rebase_options);
> @@ -2032,9 +2036,9 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
>  	}
>  
>  	/* If a hook exists, give it a chance to interrupt*/
> +	strvec_pushl(&hook_opt.args, options.upstream_arg, argc ? argv[0] : NULL, NULL);
>  	if (!ok_to_skip_pre_rebase &&
> -	    run_hook_le(NULL, "pre-rebase", options.upstream_arg,
> -			argc ? argv[0] : NULL, NULL))
> +	    run_hooks("pre-rebase", &hook_opt))
>  		die(_("The pre-rebase hook refused to rebase."));

This may needlessly populate hook_opt.args even when run_hooks() is
not triggered, but that probably is OK.  Except for a place or two
where we call die(), the exit path from this function after this
point all eventually passes ...

>  	if (options.flags & REBASE_DIFFSTAT) {
> @@ -2114,6 +2118,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
>  	ret = !!run_specific_rebase(&options, action);
>  
>  cleanup:

... this label, so everybody calls opt_clear() at the end, which is
good.

> +	run_hooks_opt_clear(&hook_opt);
>  	strbuf_release(&buf);
>  	strbuf_release(&revisions);
>  	free(options.head_name);

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 24/37] receive-pack: convert push-to-checkout hook to hook.h
  2021-03-11  2:10 ` [PATCH v8 24/37] receive-pack: convert push-to-checkout hook to hook.h Emily Shaffer
@ 2021-03-12 10:24   ` Junio C Hamano
  2021-03-29 23:59     ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Junio C Hamano @ 2021-03-12 10:24 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> @@ -1435,12 +1436,19 @@ static const char *push_to_checkout(unsigned char *hash,
>  				    struct strvec *env,
>  				    const char *work_tree)
>  {
> +	struct run_hooks_opt opt;
> +	run_hooks_opt_init_sync(&opt);
> +
>  	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
> -	if (run_hook_le(env->v, push_to_checkout_hook,
> -			hash_to_hex(hash), NULL))
> +	strvec_pushv(&opt.env, env->v);
> +	strvec_push(&opt.args, hash_to_hex(hash));
> +	if (run_hooks(push_to_checkout_hook, &opt)) {
> +		run_hooks_opt_clear(&opt);
>  		return "push-to-checkout hook declined";
> -	else
> +	} else {
> +		run_hooks_opt_clear(&opt);
>  		return NULL;
> +	}
>  }

OK, we opt_init(), futz with opt, call run_hooks() and opt_clear()
regardless of the outcome from run_hooks().  Narrow-sighted me
wonders if it makes the use of the API easier if run_hooks() did the
opt_clear() before it returns, but I haven't yet seen enough use at
this point to judge.

Thanks.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 00/37] config-based hooks
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (38 preceding siblings ...)
  2021-03-12  9:49 ` Ævar Arnfjörð Bjarmason
@ 2021-03-12 11:13 ` Ævar Arnfjörð Bjarmason
  2021-03-25 12:41   ` Ævar Arnfjörð Bjarmason
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
  40 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-12 11:13 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: git, Jeff King, Junio C Hamano, James Ramsay, Jonathan Nieder,
	brian m. carlson, Phillip Wood, Josh Steadmon,
	Johannes Schindelin, Jonathan Tan


On Thu, Mar 11 2021, Emily Shaffer wrote:

> Since v7:
> - Addressed Jonathan Tan's review of part I
> - Addressed Junio's review of part I and II
> - Combined parts I and II
>

Comments on the overall design / goals (I don't just have strbuf nits):

I think I mentioned this in earlier rounds, but I'm still very skeptical
of the need for a "git hook" command for anything except the "run" case
(which is very useful).

So I tried patching it with this on top:
	
	diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
	index 4f66bb35cf8..eb48da1dcf0 100644
	--- a/Documentation/config/hook.txt
	+++ b/Documentation/config/hook.txt
	@@ -1,20 +1,17 @@
	-hook.<command>.command::
	-	A command to execute during the <command> hook event. This can be an
	-	executable on your device, a oneliner for your shell, or the name of a
	-	hookcmd. See linkgit:git-hook[1].
	-
	-hookcmd.<name>.command::
	-	A command to execute during a hook for which <name> has been specified
	-	as a command. This can be an executable on your device or a oneliner for
	-	your shell. See linkgit:git-hook[1].
	-
	-hookcmd.<name>.skip::
	-	Specify this boolean to remove a command from earlier in the execution
	-	order. Useful if you want to make a single repo an exception to hook
	-	configured at the system or global scope. If there is no hookcmd
	-	specified for the command you want to skip, you can use the value of
	-	`hook.<command>.command` as <name> as a shortcut. The "skip" setting
	-	must be specified after the "hook.<command>.command" to have an effect.
	+hook.<name>.event::
	+hook.<name>.command::
	+	A command to execute during a given hook event for which
	+	<name> has been specified This can be an executable on your
	+	device or a oneliner for your shell. See linkgit:git-hook[1].
	++
	+As a convention setting this to the string `true` will clobber and
	+omit a command from earlier in the execution order. Similarly to the
	+"cat" special-case for `pager.<cmd>` we won't execute the hook at all
	+in that case.
	++
	+To have a single hook handle multiple types of events (such as
	+`pre-receive` and `post-receive`) specify `hook.<name>.event` multiple
	+times.
	 
	 hook.runHookDir::
	 	Controls how hooks contained in your hookdir are executed. Can be any of

I didn't finish that WIP patch, but I have yet to see any reason for why
it wouldn't work.

In experimenting with it further I tried just adding a "git config
--show-hook" as a convenience alias for "git config --show-origin
--show-scope --get-regexp '^hook\.<name>\.'", something like:
	
	diff --git a/builtin/config.c b/builtin/config.c
	index 963d65fd3fc..f62356b923a 100644
	--- a/builtin/config.c
	+++ b/builtin/config.c
	@@ -33,6 +33,7 @@ static int end_nul;
	 static int respect_includes_opt = -1;
	 static struct config_options config_options;
	 static int show_origin;
	+static int show_hook;
	 static int show_scope;
	 
	 #define ACTION_GET (1<<0)
	@@ -159,6 +160,7 @@ static struct option builtin_config_options[] = {
	 	OPT_BOOL('z', "null", &end_nul, N_("terminate values with NUL byte")),
	 	OPT_BOOL(0, "name-only", &omit_values, N_("show variable names only")),
	 	OPT_BOOL(0, "includes", &respect_includes_opt, N_("respect include directives on lookup")),
	+	OPT_BOOL(0, "show-hook", &show_hook, N_("show configuration for a given hook (convenience alias for --show-origin --show-scope --get-regexp '^hook\\.<name>\\.')")),
	 	OPT_BOOL(0, "show-origin", &show_origin, N_("show origin of config (file, standard input, blob, command line)")),
	 	OPT_BOOL(0, "show-scope", &show_scope, N_("show scope of config (worktree, local, global, system, command)")),
	 	OPT_STRING(0, "default", &default_value, N_("value"), N_("with --get, use default value when missing entry")),
	@@ -631,6 +633,7 @@ int cmd_config(int argc, const char **argv, const char *prefix)
	 {
	 	int nongit = !startup_info->have_repository;
	 	char *value;
	+	struct strbuf show_hook_arg = STRBUF_INIT;
	 
	 	given_config_source.file = xstrdup_or_null(getenv(CONFIG_ENVIRONMENT));
	 
	@@ -645,6 +648,14 @@ int cmd_config(int argc, const char **argv, const char *prefix)
	 		usage_builtin_config();
	 	}
	 
	+	if (show_hook) {
	+		strbuf_addf(&show_hook_arg, "^hook\\.%s\\.", argv[0]);
	+		actions = ACTION_GET_REGEXP;
	+		show_scope = 1;
	+		argv[0] = show_hook_arg.buf;
	+	}
	+		
	+
	 	if (nongit) {
	 		if (use_local_config)
	 			die(_("--local can only be used inside a git repository"));
	@@ -915,5 +926,8 @@ int cmd_config(int argc, const char **argv, const char *prefix)
	 		return get_colorbool(argv[0], argc == 2);
	 	}
	 
	+	/* TODO: Memory leak on non-zero return, do we care? */
	+	strbuf_release(&show_hook_arg);
	+
	 	return 0;
	 }

So the reason that naïve approach doesn't work is that the current
design has both a hook.<command>.command, *or* a
hookcmd.<command>.<cfg>. So it can't be just a single --get-regexp, you
need to statefully parse it, as indeed your implementation does.

But this seems like a bad idea to me for at least these reasons I've
thought of so far:

 1. If we just change the design a bit we can make this a much simpler
    git-config wrapper, or point to that directly.

 2. You're sticking full paths in the git config key, which is
    case-insensitive, and a feature of this format is being able to
    configure/override previously configured hooks.

    So the behavior of this feature depends on git's interaction with
    the case-sensitivity of filesystems, and not just one fs, any fs
    we're walking in our various config sources, and where the hook
    itself lives.

    As recent CVEs have shown that's a big can of worms, particularly
    for something whose goal is to address the security aspect of
    running hooks from other config.

    Arguably the case-sensitivity issue is just confusing since we
    canonicalize it anyway. But once you add in FS path canonicalization
    it becomes a real big can of worms. See the .gitmodules fsck code.

    Even if it wasn't for that it's relatively nastier to edit/maintain
    full paths and the appropriate escaping in the double-quoted key in
    the config file v.s. having it as an optionally quoted value.

 3. We're left with this "*.command = cmd", and "*.skip = true"
    special-case syntax. I can't see any reason for why it's needed over
    simply having "*.command = true" clobber earlier hooks as noted in
    the proposed docs above.

    And that doesn't require any magic to support, just like our
    existing "core.pager=cat" case.

    I mean, I suppose it's magical in that we might otherwise error on
    non-consumed stdin (do we?), anyway, documenting it as a synonym for
    "cat >/dev/null" would get around that :)

 4. It makes the common case of having the same hooks for N commands
    needlessly verbose, if you can just support "type" (or whatever we
    should call it) you can add that N times...

 5. At the end of this series we're left with the start of the docs
    saying:

      You can list and run configured hooks with this command. Later,
      you will be able to add and modify hooks with this command.

    But those patches have yet to land, and looking at the design
    document I'm skeptical of that being a good addition v.s. just
    adding the same thing to "git config".

    As just one exmaple; surely "git config edit <name>" would need to
    run around and find config files to edit, then open them in a loop
    for you, no?

    Which we'd eventually want for "git config" in general with an
    --edit-regexp option or whatever, which brings us (well, at least
    me) back to "then let's just add it to git-config?".

 6. The whole 'git hook' config special-casing doesn't help other
    commands or the security issue that seemed to have prompted (at
    least some of) its existence

    In the design doc we mention the "core.pager = rm -rf /" case for a
    .git/config.

    This series doesn't implement, but the design docs note a future
    want for solving that issue for the hooks.

    To me that's another case where we should just have general config
    syntax, not something hook-specific, e.g. if I could do this in my
    ~/.gitconfig:

       ;; We consider 'config.ignore' in reverse order, so e.g setting
       ;; it in. ~/.gitconfig will ignore any such keys for repo-level
       ;; config
       [config "ignore"]
       key = core.pager
       keyRegexp = "^hook\."

    We'd address both any hook security concerns, as well as core.pager
    etc. We could then just have e.g. some syntax sugar of:

       [include]
       path = built-in://gimme-safe-config

    Which would just be a thin layer of magit to include
    <path-to-git-prefix>/config-templates/gimme-safe-config or whatever.

    We'd thus address the issue for all config types without
    hook-specific magic.

Anyway. I'm very willing to be convinced otherwise. I just think that
for a first-draft implementation leaving aside 'hook.<command>.command'
and the whole 'list' thing makes sense.

We can consider the core code changes relatively separately from any
future aspirations, particularly with a 40-some patch series, and the
end-state of *this series* IMO not really justifying, that part of the
implementation, and thus requiring reviewers to look ahead beyond the
40-some patches.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 00/37] config-based hooks
  2021-03-11 22:26 ` [PATCH v8 00/37] config-based hooks Junio C Hamano
@ 2021-03-12 23:27   ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-12 23:27 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Jeff King, James Ramsay, Jonathan Nieder, brian m. carlson,
	Ævar Arnfjörð Bjarmason, Phillip Wood,
	Josh Steadmon, Johannes Schindelin, Jonathan Tan

On Thu, Mar 11, 2021 at 02:26:10PM -0800, Junio C Hamano wrote:
> 
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > Since v7:
> > - Addressed Jonathan Tan's review of part I
> > - Addressed Junio's review of part I and II
> > - Combined parts I and II
> >
> > I think the updates to patch 1 between the rest of the work I've been
> > doing probably have covered Ævar's comments.
> >
> > More details about per-patch changes found in the notes on each mail (I
> > hope).
> >
> > I know that Junio was talking about merging v7 after Josh Steadmon's
> > review and I asked him not to - this reroll has those changes from
> > Jonathan Tan's review that I was wanting to wait for.
> 
> I picked it up and replaced, not necessarily because it is an urgent
> thing to do during the pre-release period, but primarily because I
> wanted to be prepared for any nasty surprises by unmanageable
> conflicts I may have to face once the current cycle is over.
> 
> It turns out that it was a bit painful to merge to 'seen' as there
> are in-flight topics that touch the hooks documentation, and the
> changes they make must be carried forward to the new file.
> 
> But it was not too bad.  
> 
> The merge into 'seen' is 3cdeaeab (Merge branch 'es/config-hooks'
> into seen, 2021-03-11) as of this writing, and the output of
> 
>     $ git diff 3cdeaeab3a^:Documentation/githooks.txt \
>                3cdeaeab3a:Documentation/native-hooks.txt
> 
>     (i.e. the version of the file before the merge, where your topic
>     being merged took material to edit to produce the new "native-hooks"
>     document, is compared with the result)
> 
> looks reasonable to me, but please double check.

I had a look at that diff (but targeting 6da6893c, which is what I see
for "Merge branch 'es/config-hooks' into seen" when I fetch from
gitster/git today) and it looks fine to me, very reasonable. Thanks for
doing that.

> 
> Thanks.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 35/37] git-send-email: use 'git hook run' for 'sendemail-validate'
  2021-03-11  2:10 ` [PATCH v8 35/37] git-send-email: use 'git hook run' for 'sendemail-validate' Emily Shaffer
  2021-03-12  9:21   ` Ævar Arnfjörð Bjarmason
@ 2021-03-12 23:29   ` Emily Shaffer
  1 sibling, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-12 23:29 UTC (permalink / raw)
  To: git

On Wed, Mar 10, 2021 at 06:10:35PM -0800, Emily Shaffer wrote:
> 
> By using the new 'git hook run' subcommand to run 'sendemail-validate',
> we can reduce the boilerplate needed to run this hook in perl. Using
> config-based hooks also allows us to run 'sendemail-validate' hooks that
> were configured globally when running 'git send-email' from outside of a
> Git directory, alongside other benefits like multihooks and
> parallelization.
> 
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---

Without having had time to look at reviews to this (or the rest of the
series) yet - it occurred to me that this hook should be run in series
instead. That is, I should invoke 'git hook run' with '-j1'.

>  git-send-email.perl   | 21 ++++-----------------
>  t/t9001-send-email.sh | 11 +----------
>  2 files changed, 5 insertions(+), 27 deletions(-)
> 
> diff --git a/git-send-email.perl b/git-send-email.perl
> index 1f425c0809..73e1e0b51a 100755
> --- a/git-send-email.perl
> +++ b/git-send-email.perl
> @@ -1941,23 +1941,10 @@ sub unique_email_list {
>  sub validate_patch {
>  	my ($fn, $xfer_encoding) = @_;
>  
> -	if ($repo) {
> -		my $validate_hook = catfile(catdir($repo->repo_path(), 'hooks'),
> -					    'sendemail-validate');
> -		my $hook_error;
> -		if (-x $validate_hook) {
> -			my $target = abs_path($fn);
> -			# The hook needs a correct cwd and GIT_DIR.
> -			my $cwd_save = cwd();
> -			chdir($repo->wc_path() or $repo->repo_path())
> -				or die("chdir: $!");
> -			local $ENV{"GIT_DIR"} = $repo->repo_path();
> -			$hook_error = "rejected by sendemail-validate hook"
> -				if system($validate_hook, $target);
> -			chdir($cwd_save) or die("chdir: $!");
> -		}
> -		return $hook_error if $hook_error;
> -	}
> +	my $target = abs_path($fn);
> +	return "rejected by sendemail-validate hook"
> +		if system(("git", "hook", "run", "sendemail-validate", "-a",
> +				$target));
>  
>  	# Any long lines will be automatically fixed if we use a suitable transfer
>  	# encoding.
> diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
> index 4eee9c3dcb..456b471c5c 100755
> --- a/t/t9001-send-email.sh
> +++ b/t/t9001-send-email.sh
> @@ -2101,16 +2101,7 @@ test_expect_success $PREREQ 'invoke hook' '
>  	mkdir -p .git/hooks &&
>  
>  	write_script .git/hooks/sendemail-validate <<-\EOF &&
> -	# test that we have the correct environment variable, pwd, and
> -	# argument
> -	case "$GIT_DIR" in
> -	*.git)
> -		true
> -		;;
> -	*)
> -		false
> -		;;
> -	esac &&
> +	# test that we have the correct argument
>  	test -f 0001-add-main.patch &&
>  	grep "add main" "$1"
>  	EOF
> -- 
> 2.31.0.rc2.261.g7f71774620-goog
> 

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 00/37] config-based hooks
  2021-03-12  9:49 ` Ævar Arnfjörð Bjarmason
@ 2021-03-17 18:41   ` Emily Shaffer
  2021-03-17 19:16     ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-03-17 18:41 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Jeff King, Junio C Hamano, James Ramsay, Jonathan Nieder,
	brian m. carlson, Phillip Wood, Josh Steadmon,
	Johannes Schindelin, Jonathan Tan

On Fri, Mar 12, 2021 at 10:49:38AM +0100, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Thu, Mar 11 2021, Emily Shaffer wrote:
> 
> > Since v7:
> > - Addressed Jonathan Tan's review of part I
> > - Addressed Junio's review of part I and II
> > - Combined parts I and II
> >
> > I think the updates to patch 1 between the rest of the work I've been
> > doing probably have covered Ævar's comments.
> 
> A range-diff between iterations of such a large series would be most
> useful. Do you have a public repo with tags or whatever the different
> versions, for those who'd like an easier way to follow along the
> differing versions than scraping the ML archive?

I am really embarrassed to say that I don't have the
branches/tags/whatever up. I have not succeeded in building that habit
yet. I'll generate one from my local patches today and send it here.

> 
> While reading this I came up with the following fixup patches on top,
> for discussion, maybe not something you want as-is:

I was a little bit confused reading this fixup without seeing the rest of your
review, so I'll revisit this once I get through what else you wrote.

> 	
> 	 Documentation/git-hook.txt |  8 +++++
> 	 builtin/bugreport.c        |  8 +++--
> 	 builtin/commit.c           |  3 +-
> 	 builtin/hook.c             | 79 ++++++++++++++++++++--------------------------
> 	 builtin/merge.c            |  3 +-
> 	 builtin/receive-pack.c     | 11 +++----
> 	 hook.c                     | 21 +++++-------
> 	 hook.h                     |  5 +--
> 	 refs.c                     |  4 ++-
> 	 sequencer.c                |  4 ++-
> 	 10 files changed, 73 insertions(+), 73 deletions(-)
> 	
> 	diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
> 	index 4ad31ac360a..5c9af30b43e 100644
> 	--- a/Documentation/git-hook.txt
> 	+++ b/Documentation/git-hook.txt
> 	@@ -150,10 +150,18 @@ message body and cannot be parallelized.
> 	 
> 	 CONFIGURATION
> 	 -------------
> 	+
> 	+The below documentation is the same as what's found in
> 	+linkgit:git-config[1]:
> 	+
> 	 include::config/hook.txt[]
> 	 
> 	 HOOKS
> 	 -----
> 	+
> 	+The below documentation is the same as what's found in
> 	+linkgit:githooks[5]:
> 	+
> 	 include::native-hooks.txt[]
> 	 
> Noted in another reply, including it here for completeness.
> 
> 	 GIT
> 	diff --git a/builtin/bugreport.c b/builtin/bugreport.c
> 	index 04467cd1d3a..b64e53fd625 100644
> 	--- a/builtin/bugreport.c
> 	+++ b/builtin/bugreport.c
> 	@@ -81,9 +81,13 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
> 	 		return;
> 	 	}
> 	 
> 	-	for (i = 0; i < ARRAY_SIZE(hook); i++)
> 	-		if (hook_exists(hook[i], HOOKDIR_USE_CONFIG))
> 	+	for (i = 0; i < ARRAY_SIZE(hook); i++) {
> 	+		struct strbuf config;
> 	+		strbuf_addf(&config, "hook.%s.config", hook[i]);
> 	+		if (hook_exists(hook[i], config.buf, HOOKDIR_USE_CONFIG))
> 	 			strbuf_addf(hook_info, "%s\n", hook[i]);
> 	+		strbuf_release(&config);
> 	+	}
> 	 }
> 
> Less strbuf, see below.
> 	 
> 	 static const char * const bugreport_usage[] = {
> 	diff --git a/builtin/commit.c b/builtin/commit.c
> 	index 31df571f123..fc9f1f5ee58 100644
> 	--- a/builtin/commit.c
> 	+++ b/builtin/commit.c
> 	@@ -984,7 +984,8 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
> 	 		return 0;
> 	 	}
> 	 
> 	-	if (!no_verify && hook_exists("pre-commit", HOOKDIR_USE_CONFIG)) {
> 	+	if (!no_verify && hook_exists("pre-commit", "hook.pre-commit.command",
> 	+				      HOOKDIR_USE_CONFIG)) {
> 	 		/*
> 	 		 * Re-read the index as pre-commit hook could have updated it,
> 	 		 * and write it out as a tree.  We must do this before we invoke
> 
> 
> ..ditto.
> 
> 	diff --git a/builtin/hook.c b/builtin/hook.c
> 	index b4f4adb1dea..d0b56ee47f8 100644
> 	--- a/builtin/hook.c
> 	+++ b/builtin/hook.c
> 	@@ -18,8 +18,6 @@ static enum hookdir_opt should_run_hookdir;
> 	 static int list(int argc, const char **argv, const char *prefix)
> 	 {
> 	 	struct list_head *head, *pos;
> 	-	struct strbuf hookname = STRBUF_INIT;
> 	-	struct strbuf hookdir_annotation = STRBUF_INIT;
> 	 
> 	 	struct option list_options[] = {
> 	 		OPT_END(),
> 	@@ -33,67 +31,60 @@ static int list(int argc, const char **argv, const char *prefix)
> 	 			      builtin_hook_usage, list_options);
> 	 	}
> 	 
> 	-	strbuf_addstr(&hookname, argv[0]);
> 	-
> 	-	head = hook_list(&hookname);
> 	+	head = hook_list(argv[0]);
> 	 
> 	 	if (list_empty(head)) {
> 	 		printf(_("no commands configured for hook '%s'\n"),
> 	-		       hookname.buf);
> 	-		strbuf_release(&hookname);
> 	+		       argv[0]);
> 	 		return 0;
> 	 	}
> 	 
> 	-	switch (should_run_hookdir) {
> 	-		case HOOKDIR_NO:
> 	-			strbuf_addstr(&hookdir_annotation, _(" (will not run)"));
> 	-			break;
> 	-		case HOOKDIR_ERROR:
> 	-			strbuf_addstr(&hookdir_annotation, _(" (will error and not run)"));
> 	-			break;
> 	-		case HOOKDIR_INTERACTIVE:
> 	-			strbuf_addstr(&hookdir_annotation, _(" (will prompt)"));
> 	-			break;
> 	-		case HOOKDIR_WARN:
> 	-			strbuf_addstr(&hookdir_annotation, _(" (will warn but run)"));
> 	-			break;
> 	-		case HOOKDIR_YES:
> 	-		/*
> 	-		 * The default behavior should agree with
> 	-		 * hook.c:configured_hookdir_opt(). HOOKDIR_UNKNOWN should just
> 	-		 * do the default behavior.
> 	-		 */
> 	-		case HOOKDIR_UNKNOWN:
> 	-		default:
> 	-			break;
> 	-	}
> 	-
> 	 	list_for_each(pos, head) {
> 	 		struct hook *item = list_entry(pos, struct hook, list);
> 	 		item = list_entry(pos, struct hook, list);
> 	 		if (item) {
> 	-			/* Don't translate 'hookdir' - it matches the config */
> 	-			printf("%s: %s%s\n",
> 	-			       (item->from_hookdir
> 	+			const char *scope = item->from_hookdir
> 	 				? "hookdir"
> 	-				: config_scope_name(item->origin)),
> 	-			       item->command.buf,
> 	-			       (item->from_hookdir
> 	-				? hookdir_annotation.buf
> 	-				: ""));
> 	+				: config_scope_name(item->origin);
> 	+			switch (should_run_hookdir) {
> 	+			case HOOKDIR_NO:
> 	+				printf(_("%s: %s (will not run)\n"),
> 	+				       scope, item->command.buf);
> 	+				break;
> 	+			case HOOKDIR_ERROR:
> 	+				printf(_("%s: %s (will error and not run)\n"),
> 	+				       scope, item->command.buf);
> 	+				break;
> 	+			case HOOKDIR_INTERACTIVE:
> 	+				printf(_("%s: %s (will prompt)\n"),
> 	+				       scope, item->command.buf);
> 	+				break;
> 	+			case HOOKDIR_WARN:
> 	+				printf(_("%s: %s (will warn but run)\n"),
> 	+				       scope, item->command.buf);
> 	+				break;
> 	+			case HOOKDIR_YES:
> 	+				/*
> 	+				 * The default behavior should agree with
> 	+				 * hook.c:configured_hookdir_opt(). HOOKDIR_UNKNOWN should just
> 	+				 * do the default behavior.
> 	+				 */
> 	+			case HOOKDIR_UNKNOWN:
> 	+			default:
> 	+				printf(_("%s: %s\n"),
> 	+				       scope, item->command.buf);
> 	+				break;
> 	+			}
> 	 		}
> 	 	}
> 	 
> 	 	clear_hook_list(head);
> 	-	strbuf_release(&hookdir_annotation);
> 	-	strbuf_release(&hookname);
> 	 
> 	 	return 0;
> 	 }
> 
> I think this is better to avoid i18n lego, as noted in another reply
> (but I didn't include the patch).
> 
> More on strbuf below:
> 	 
> 	 static int run(int argc, const char **argv, const char *prefix)
> 	 {
> 	-	struct strbuf hookname = STRBUF_INIT;
> 	 	struct run_hooks_opt opt;
> 	 	int rc = 0;
> 	 
> 	@@ -118,12 +109,10 @@ static int run(int argc, const char **argv, const char *prefix)
> 	 		usage_msg_opt(_("You must specify a hook event to run."),
> 	 			      builtin_hook_usage, run_options);
> 	 
> 	-	strbuf_addstr(&hookname, argv[0]);
> 	 	opt.run_hookdir = should_run_hookdir;
> 	 
> 	-	rc = run_hooks(hookname.buf, &opt);
> 	+	rc = run_hooks(argv[0], &opt);
> 	 
> 	-	strbuf_release(&hookname);
> 	 	run_hooks_opt_clear(&opt);
> 	 
> 	 	return rc;
> 	diff --git a/builtin/merge.c b/builtin/merge.c
> 	index 3a2af257a6b..df4ff72fbc7 100644
> 	--- a/builtin/merge.c
> 	+++ b/builtin/merge.c
> 	@@ -848,7 +848,8 @@ static void prepare_to_commit(struct commit_list *remoteheads)
> 	 	 * and write it out as a tree.  We must do this before we invoke
> 	 	 * the editor and after we invoke run_status above.
> 	 	 */
> 	-	if (hook_exists("pre-merge-commit", HOOKDIR_USE_CONFIG))
> 	+	if (hook_exists("pre-merge-commit", "hook.pre-merge-commit.command",
> 	+			HOOKDIR_USE_CONFIG))
> 	 		discard_cache();
> 	 	read_cache_from(index_file);
> 	 	strbuf_addbuf(&msg, &merge_msg);
> 	diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
> 	index eaedeeb1e8b..a76069ea592 100644
> 	--- a/builtin/receive-pack.c
> 	+++ b/builtin/receive-pack.c
> 	@@ -1123,12 +1123,10 @@ static int run_proc_receive_hook(struct command *commands,
> 	 	int version = 0;
> 	 	int code;
> 	 
> 	-	struct strbuf hookname = STRBUF_INIT;
> 	 	struct hook *proc_receive = NULL;
> 	 	struct list_head *pos, *hooks;
> 	 
> 	-	strbuf_addstr(&hookname, "proc-receive");
> 	-	hooks = hook_list(&hookname);
> 	+	hooks = hook_list("proc-receive");
> 	 
> 	 	list_for_each(pos, hooks) {
> 	 		if (proc_receive) {
> 	@@ -1460,8 +1458,6 @@ static const char *push_to_deploy(unsigned char *sha1,
> 	 	return NULL;
> 	 }
> 	 
> 	-static const char *push_to_checkout_hook = "push-to-checkout";
> 	-
> 	 static const char *push_to_checkout(unsigned char *hash,
> 	 				    struct strvec *env,
> 	 				    const char *work_tree)
> 	@@ -1472,7 +1468,7 @@ static const char *push_to_checkout(unsigned char *hash,
> 	 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
> 	 	strvec_pushv(&opt.env, env->v);
> 	 	strvec_push(&opt.args, hash_to_hex(hash));
> 	-	if (run_hooks(push_to_checkout_hook, &opt)) {
> 	+	if (run_hooks("push-to-checkout", &opt)) {
> 	 		run_hooks_opt_clear(&opt);
> 	 		return "push-to-checkout hook declined";
> 	 	} else {
> 	@@ -1502,7 +1498,8 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
> 	 
> 	 	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
> 	 
> 	-	if (!hook_exists(push_to_checkout_hook, HOOKDIR_USE_CONFIG))
> 	+	if (!hook_exists("push-to-checkout", "hook.push-to-checkout.command",
> 	+			 HOOKDIR_USE_CONFIG))
> 	 		retval = push_to_deploy(sha1, &env, work_tree);
> 	 	else
> 	 		retval = push_to_checkout(sha1, &env, work_tree);
> 	diff --git a/hook.c b/hook.c
> 	index 7f6f3b9a616..49c3861ce00 100644
> 	--- a/hook.c
> 	+++ b/hook.c
> 	@@ -247,7 +247,7 @@ static const char *find_legacy_hook(const char *name)
> 	 }
> 	 
> 	 
> 	-struct list_head* hook_list(const struct strbuf* hookname)
> 	+struct list_head* hook_list(const char *hookname)
> 	 {
> 	 	struct strbuf hook_key = STRBUF_INIT;
> 	 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
> 	@@ -256,14 +256,14 @@ struct list_head* hook_list(const struct strbuf* hookname)
> 	 	INIT_LIST_HEAD(hook_head);
> 	 
> 	 	if (!hookname)
> 	-		return NULL;
> 	+		BUG("???");;
> 	 
> 	-	strbuf_addf(&hook_key, "hook.%s.command", hookname->buf);
> 	+	strbuf_addf(&hook_key, "hook.%s.command", hookname);
> 	 
> 	 	git_config(hook_config_lookup, &cb_data);
> 	 
> 	 	if (have_git_dir()) {
> 	-		const char *legacy_hook_path = find_legacy_hook(hookname->buf);
> 	+		const char *legacy_hook_path = find_legacy_hook(hookname);
> 	 
> 	 		/* Unconditionally add legacy hook, but annotate it. */
> 	 		if (legacy_hook_path) {
> 	@@ -300,10 +300,10 @@ void run_hooks_opt_init_async(struct run_hooks_opt *o)
> 	 	o->jobs = configured_hook_jobs();
> 	 }
> 	 
> 	-int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir)
> 	+int hook_exists(const char *hookname, const char *hook_config,
> 	+		enum hookdir_opt should_run_hookdir)
> 	 {
> 	 	const char *value = NULL; /* throwaway */
> 	-	struct strbuf hook_key = STRBUF_INIT;
> 	 	int could_run_hookdir;
> 	 
> 	 	if (should_run_hookdir == HOOKDIR_USE_CONFIG)
> 	@@ -314,9 +314,7 @@ int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir)
> 	 				should_run_hookdir == HOOKDIR_YES)
> 	 				&& !!find_legacy_hook(hookname);
> 	 
> 	-	strbuf_addf(&hook_key, "hook.%s.command", hookname);
> 	-
> 	-	return (!git_config_get_value(hook_key.buf, &value)) || could_run_hookdir;
> 	+	return (!git_config_get_value(hook_config, &value)) || could_run_hookdir;
> 	 }
> 	 
> 	 void run_hooks_opt_clear(struct run_hooks_opt *o)
> 	@@ -438,7 +436,6 @@ static int notify_hook_finished(int result,
> 	 
> 	 int run_hooks(const char *hookname, struct run_hooks_opt *options)
> 	 {
> 	-	struct strbuf hookname_str = STRBUF_INIT;
> 	 	struct list_head *to_run, *pos = NULL, *tmp = NULL;
> 	 	struct hook_cb_data cb_data = { 0, NULL, NULL, options };
> 	 
> 	@@ -448,9 +445,7 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options)
> 	 	if (options->path_to_stdin && options->feed_pipe)
> 	 		BUG("choose only one method to populate stdin");
> 	 
> 	-	strbuf_addstr(&hookname_str, hookname);
> 	-
> 	-	to_run = hook_list(&hookname_str);
> 	+	to_run = hook_list(hookname);
> 	 
> 	 	list_for_each_safe(pos, tmp, to_run) {
> 	 		struct hook *hook = list_entry(pos, struct hook, list);
> 	diff --git a/hook.h b/hook.h
> 	index 4ff9999b049..bfbbf36882d 100644
> 	--- a/hook.h
> 	+++ b/hook.h
> 	@@ -26,7 +26,7 @@ struct hook {
> 	  * Provides a linked list of 'struct hook' detailing commands which should run
> 	  * in response to the 'hookname' event, in execution order.
> 	  */
> 	-struct list_head* hook_list(const struct strbuf *hookname);
> 	+struct list_head* hook_list(const char *hookname);
> 	 
> 	 enum hookdir_opt
> 	 {
> 	@@ -123,7 +123,8 @@ void run_hooks_opt_clear(struct run_hooks_opt *o);
> 	  * Like with run_hooks, if you take a --run-hookdir flag, reflect that
> 	  * user-specified behavior here instead.
> 	  */
> 	-int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir);
> 	+int hook_exists(const char *hookname, const char *hook_config,
> 	+		enum hookdir_opt should_run_hookdir);
> 	 
> 	 /*
> 	  * Runs all hooks associated to the 'hookname' event in order. Each hook will be
> 	diff --git a/refs.c b/refs.c
> 	index 334fdd9103c..f01995fe64f 100644
> 	--- a/refs.c
> 	+++ b/refs.c
> 	@@ -1966,7 +1966,9 @@ static int run_transaction_hook(struct ref_transaction *transaction,
> 	 
> 	 	run_hooks_opt_init_async(&opt);
> 	 
> 	-	if (!hook_exists("reference-transaction", HOOKDIR_USE_CONFIG))
> 	+	if (!hook_exists("reference-transaction",
> 	+			 "hook.reference-transaction.command",
> 	+			 HOOKDIR_USE_CONFIG))
> 	 		return ret;
> 	 
> 	 	strvec_push(&opt.args, state);
> 	diff --git a/sequencer.c b/sequencer.c
> 	index 34ff275f0d1..52c067c1688 100644
> 	--- a/sequencer.c
> 	+++ b/sequencer.c
> 	@@ -1436,7 +1436,9 @@ static int try_to_commit(struct repository *r,
> 	 		}
> 	 	}
> 	 
> 	-	if (hook_exists("prepare-commit-msg", HOOKDIR_USE_CONFIG)) {
> 	+	if (hook_exists("prepare-commit-msg",
> 	+			"hook.prepare-commit-msg.command",
> 	+			HOOKDIR_USE_CONFIG)) {
> 	 		res = run_prepare_commit_msg_hook(r, msg, hook_commit);
> 	 		if (res)
> 	 			goto out;
> 
> There was another reply (from JT I believe, but didn't go back and look
> it up) about the over use of strbuf.
> 
> I tend to agree, as much as I love the API it's really not better to
> write C with it if all you need is a const char* that's never modified,
> particularly if you get it from elsewhere.
> 
> So it's really not meant for or good for "everything we need a const
> char*", but to avoid verbose realloc() dances all over the place, and
> for things like getline() loops without a hardcoded buffer size.
> 
> E.g. in the first hunk here we're creating a strbuf just to copy argv[0]
> to it, and then throwing it away, let's just pass down argv[0].
> 
> For hook_exists I think just having the code more grep-able and having
> the config value inline is better, but I admit that's a matter of taste.
> 
> I didn't try to find all such strbuf() occurrences, anyway, in the
> overall scheme of things it's a relatively small nit.
> 
> I'm hoping to do some deeper diving into this series, in particular the
> parallelism, but just sending the shallow-ish comments I have for now.
> 
> Thanks for working on this!

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 00/37] config-based hooks
  2021-03-17 18:41   ` Emily Shaffer
@ 2021-03-17 19:16     ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-17 19:16 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Jeff King, Junio C Hamano, James Ramsay, Jonathan Nieder,
	brian m. carlson, Phillip Wood, Josh Steadmon,
	Johannes Schindelin, Jonathan Tan

On Wed, Mar 17, 2021 at 11:41:59AM -0700, Emily Shaffer wrote:
> 
> On Fri, Mar 12, 2021 at 10:49:38AM +0100, �var Arnfj�r� Bjarmason wrote:
> > 
> > 
> > On Thu, Mar 11 2021, Emily Shaffer wrote:
> > 
> > > Since v7:
> > > - Addressed Jonathan Tan's review of part I
> > > - Addressed Junio's review of part I and II
> > > - Combined parts I and II
> > >
> > > I think the updates to patch 1 between the rest of the work I've been
> > > doing probably have covered �var's comments.
> > 
> > A range-diff between iterations of such a large series would be most
> > useful. Do you have a public repo with tags or whatever the different
> > versions, for those who'd like an easier way to follow along the
> > differing versions than scraping the ML archive?
> 
> I am really embarrassed to say that I don't have the
> branches/tags/whatever up. I have not succeeded in building that habit
> yet. I'll generate one from my local patches today and send it here.

 1:  be907f68b9 !  1:  a5e8c233c3 doc: propose hooks managed by the config
    @@ Documentation/technical/config-based-hooks.txt (new)
     +[[motivation]]
     +== Motivation
     +
    -+Replace the .git/hook/hookname path as the only source of hooks to execute;
    ++Replace the `.git/hook/hookname` path as the only source of hooks to execute;
     +allow users to define hooks using config files, in a way which is friendly to
    -+users with multiple repos which have similar needs.
    ++users with multiple repos which have similar needs - hooks can be easily shared
    ++between multiple Git repos.
     +
     +Redefine "hook" as an event rather than a single script, allowing users to
     +perform multiple unrelated actions on a single event.
     +
    -+Take a step closer to safety when copying zipped Git repositories from untrusted
    -+users by making it more apparent to users which scripts will be run during
    -+normal Git operations.
    -+
     +Make it easier for users to discover Git's hook feature and automate their
     +workflows.
     +
    @@ Documentation/technical/config-based-hooks.txt (new)
     +number of cases:
     +
     +- "no": the legacy hook will not be run
    ++- "error": Git will print a warning to stderr before ignoring the legacy hook
     +- "interactive": Git will prompt the user before running the legacy hook
     +- "warn": Git will print a warning to stderr before running the legacy hook
     +- "yes" (default): Git will silently run the legacy hook
    @@ Documentation/technical/config-based-hooks.txt (new)
     +given which Git does not recognize, Git should discard that config entry. For
     +example, if "warn" was specified at system level and "junk" was specified at
     +global level, Git would resolve the value to "warn"; if the only time the config
    -+was set was to "junk", Git would use the default value of "yes".
    ++was set was to "junk", Git would use the default value of "yes" (but print a
    ++warning to the user first to let them know their value is wrong).
     +
     +`struct hookcmd` is expected to grow in size over time as more functionality is
     +added to hooks; so that other parts of the code don't need to understand the
    @@ Documentation/technical/config-based-hooks.txt (new)
     +=== Security and repo config
     +
     +Part of the motivation behind this refactor is to mitigate hooks as an attack
    -+vector;footnote:[https://lore.kernel.org/git/20171002234517.GV19555@aiede.mtv.corp.google.com/]
    -+however, as the design stands, users can still provide hooks in the repo-level
    -+config, which is included when a repo is zipped and sent elsewhere.  The
    ++vector.footnote:[https://lore.kernel.org/git/20171002234517.GV19555@aiede.mtv.corp.google.com/]
    ++However, as the design stands, users can still provide hooks in the repo-level
    ++config, which is included when a repo is zipped and sent elsewhere. The
     +security of the repo-level config is still under discussion; this design
    -+generally assumes the repo-level config is secure, which is not true yet. The
    -+goal is to avoid an overcomplicated design to work around a problem which has
    -+ceased to exist.
    ++generally assumes the repo-level config is secure, which is not true yet. This
    ++assumption was made to avoid overcomplicating the design. So, this series
    ++doesn't particularly improve security or resistance to zip attacks.
     +
     +[[ease-of-use]]
     +=== Ease of use
    @@ Documentation/technical/config-based-hooks.txt (new)
     +A previous summary of alternatives exists in the
     +archives.footnote:[https://lore.kernel.org/git/20191116011125.GG22855@google.com]
     +
    -+[[status-quo]]
    -+=== Status quo
    -+
    -+Today users can implement multihooks themselves by using a "trampoline script"
    -+as their hook, and pointing that script to a directory or list of other scripts
    -+they wish to run.
    -+
    -+[[hook-directories]]
    -+=== Hook directories
    -+
    -+Other contributors have suggested Git learn about the existence of a directory
    -+such as `.git/hooks/<hookname>.d` and execute those hooks in alphabetical order.
    -+
    -+[[comparison]]
    -+=== Comparison table
    ++The table below shows a number of goals and how they might be achieved with
    ++config-based hooks, by implementing directory support (i.e.
    ++'.git/hooks/pre-commit.d'), or as hooks are run today.
     +
     +.Comparison of alternatives
     +|===
    @@ Documentation/technical/config-based-hooks.txt (new)
     +|Natively
     +|With user effort
     +
    ++|Supports parallelization
    ++|Natively
    ++|Natively
    ++|No (user's multihook trampoline script would need to handle parallelism)
    ++
     +|Safer for zipped repos
     +|A little
     +|No
    @@ Documentation/technical/config-based-hooks.txt (new)
     +
     +|Can install one hook to many repos
     +|Yes
    -+|No
    -+|No
    ++|With symlinks or core.hooksPath
    ++|With symlinks or core.hooksPath
     +
     +|Discoverability
    -+|Better (in `git help git`)
    -+|Same as before
    ++|Findable with 'git help git' or tab-completion via 'git hook' subcommand
    ++|Findable via improved documentation
     +|Same as before
     +
     +|Hard to run unexpected hook
     +|If configured
    -+|No
    ++|Could be made to warn or look for a config
     +|No
     +|===
     +
    ++[[status-quo]]
    ++=== Status quo
    ++
    ++Today users can implement multihooks themselves by using a "trampoline script"
    ++as their hook, and pointing that script to a directory or list of other scripts
    ++they wish to run.
    ++
    ++[[hook-directories]]
    ++=== Hook directories
    ++
    ++Other contributors have suggested Git learn about the existence of a directory
    ++such as `.git/hooks/<hookname>.d` and execute those hooks in alphabetical order.
    ++
     +[[future-work]]
     +== Future work
     +
    @@ Documentation/technical/config-based-hooks.txt (new)
     +dependencies. If we decide to solve this problem, we may want to look to modern
     +build systems for inspiration on how to manage dependencies and parallel tasks.
     +
    ++[[nontrivial-hooks]]
    ++=== Multihooks and nontrivial output
    ++
    ++Some hooks - like 'proc-receive' - don't lend themselves well to multihooks at
    ++all. In the case of 'proc-receive', for now, multiple hook definitions are
    ++disallowed. In the future we might be able to conceive a better approach, for
    ++example, running the hooks in series and using the output from one hook as the
    ++input to the next.
    ++
     +[[securing-hookdir-hooks]]
     +=== Securing hookdir hooks
     +
 2:  b1d37c3911 =  2:  a3e858d056 hook: scaffolding for git-hook subcommand
 3:  fea411c598 !  3:  60b28a652b hook: add list command
    @@ Commit message
         run in config order. If more properties need to be set on a given hook
         in the future, commands can also be specified by providing
         "hook.<hookname>.command = <hookcmd-name>", as well as a "[hookcmd
    -    <hookcmd-name>]" subsection; at minimum, this subsection must contain a
    +    <hookcmd-name>]" subsection; this subsection should contain a
         "hookcmd.<hookcmd-name>.command = <path-to-hook>" line.
     
         For example:
    @@ Makefile: LIB_OBJS += hash-lookup.o
      ## builtin/hook.c ##
     @@
      #include "cache.h"
    - 
    +-
      #include "builtin.h"
     +#include "config.h"
     +#include "hook.h"
    @@ builtin/hook.c
      {
     -	struct option builtin_hook_options[] = {
     +	struct list_head *head, *pos;
    -+	struct hook *item;
     +	struct strbuf hookname = STRBUF_INIT;
     +
     +	struct option list_options[] = {
    @@ builtin/hook.c
     +	}
     +
     +	list_for_each(pos, head) {
    -+		item = list_entry(pos, struct hook, list);
    ++		struct hook *item = list_entry(pos, struct hook, list);
     +		if (item)
     +			printf("%s: %s\n",
     +			       config_scope_name(item->origin),
    @@ hook.c (new)
     +		    list_del(pos);
     +		    /* we'll simply move the hook to the end */
     +		    to_add = it;
    ++		    break;
     +		}
     +	}
     +
     +	if (!to_add) {
     +		/* adding a new hook, not moving an old one */
    -+		to_add = xmalloc(sizeof(struct hook));
    ++		to_add = xmalloc(sizeof(*to_add));
     +		strbuf_init(&to_add->command, 0);
     +		strbuf_addstr(&to_add->command, command);
     +	}
    @@ hook.c (new)
     +	/* re-set the scope so we show where an override was specified */
     +	to_add->origin = current_config_scope();
     +
    -+	list_add_tail(&to_add->list, pos);
    ++	list_add_tail(&to_add->list, head);
     +}
     +
     +static void remove_hook(struct list_head *to_remove)
    @@ hook.c (new)
     +		const char *command = value;
     +		struct strbuf hookcmd_name = STRBUF_INIT;
     +
    -+		/* Check if a hookcmd with that name exists. */
    ++		/*
    ++		 * Check if a hookcmd with that name exists. If it doesn't,
    ++		 * 'git_config_get_value()' is documented not to touch &command,
    ++		 * so we don't need to do anything.
    ++		 */
     +		strbuf_addf(&hookcmd_name, "hookcmd.%s.command", command);
     +		git_config_get_value(hookcmd_name.buf, &command);
     +
    @@ hook.c (new)
     +
     +	strbuf_addf(&hook_key, "hook.%s.command", hookname->buf);
     +
    -+	git_config(hook_config_lookup, (void*)&cb_data);
    ++	git_config(hook_config_lookup, &cb_data);
     +
     +	strbuf_release(&hook_key);
     +	return hook_head;
    @@ hook.h (new)
     +#include "list.h"
     +#include "strbuf.h"
     +
    -+struct hook
    -+{
    ++struct hook {
     +	struct list_head list;
     +	/*
     +	 * Config file which holds the hook.*.command definition.
 4:  89f1adf34d !  4:  d8232a8254 hook: include hookdir hook in list
    @@ Commit message
         $GIT_DIR/hooks/$HOOKNAME (or $HOOKDIR/$HOOKNAME). Although hooks taken
         from the config are more featureful than hooks placed in the $HOOKDIR,
         those hooks should not stop working for users who already have them.
    -
    -    Legacy hooks should be run directly, not in shell. We know that they are
    -    a path to an executable, not a oneliner script - and running them
    -    directly takes care of path quoting concerns for us for free.
    +    Let's list them to the user, but instead of displaying a config scope
    +    (e.g. "global: blah") we can prefix them with "hookdir:".
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
     
      ## builtin/hook.c ##
     @@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
    - 	struct list_head *head, *pos;
    - 	struct hook *item;
    - 	struct strbuf hookname = STRBUF_INIT;
    -+	struct strbuf hookdir_annotation = STRBUF_INIT;
    - 
    - 	struct option list_options[] = {
    - 		OPT_END(),
    -@@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
      
      	list_for_each(pos, head) {
    - 		item = list_entry(pos, struct hook, list);
    + 		struct hook *item = list_entry(pos, struct hook, list);
     -		if (item)
    --			printf("%s: %s\n",
    --			       config_scope_name(item->origin),
    --			       item->command.buf);
    ++		item = list_entry(pos, struct hook, list);
     +		if (item) {
     +			/* Don't translate 'hookdir' - it matches the config */
    -+			printf("%s: %s%s\n",
    + 			printf("%s: %s\n",
    +-			       config_scope_name(item->origin),
     +			       (item->from_hookdir
     +				? "hookdir"
     +				: config_scope_name(item->origin)),
    -+			       item->command.buf,
    -+			       (item->from_hookdir
    -+				? hookdir_annotation.buf
    -+				: ""));
    + 			       item->command.buf);
     +		}
      	}
      
    @@ hook.c
      void free_hook(struct hook *ptr)
      {
     @@ hook.c: static void append_or_move_hook(struct list_head *head, const char *command)
    - 		to_add = xmalloc(sizeof(struct hook));
    + 		to_add = xmalloc(sizeof(*to_add));
      		strbuf_init(&to_add->command, 0);
      		strbuf_addstr(&to_add->command, command);
     +		to_add->from_hookdir = 0;
    @@ hook.c: static void append_or_move_hook(struct list_head *head, const char *comm
      
      	/* re-set the scope so we show where an override was specified */
     @@ hook.c: struct list_head* hook_list(const struct strbuf* hookname)
    - 	struct strbuf hook_key = STRBUF_INIT;
    - 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
    - 	struct hook_config_cb cb_data = { &hook_key, hook_head };
    -+	const char *legacy_hook_path = NULL;
    - 
    - 	INIT_LIST_HEAD(hook_head);
      
    -@@ hook.c: struct list_head* hook_list(const struct strbuf* hookname)
    + 	git_config(hook_config_lookup, &cb_data);
      
    - 	git_config(hook_config_lookup, (void*)&cb_data);
    - 
    -+	if (have_git_dir())
    -+		legacy_hook_path = find_hook(hookname->buf);
    ++	if (have_git_dir()) {
    ++		const char *legacy_hook_path = find_hook(hookname->buf);
     +
    -+	/* Unconditionally add legacy hook, but annotate it. */
    -+	if (legacy_hook_path) {
    -+		struct hook *legacy_hook;
    ++		/* Unconditionally add legacy hook, but annotate it. */
    ++		if (legacy_hook_path) {
    ++			struct hook *legacy_hook;
     +
    -+		append_or_move_hook(hook_head, absolute_path(legacy_hook_path));
    -+		legacy_hook = list_entry(hook_head->prev, struct hook, list);
    -+		legacy_hook->from_hookdir = 1;
    ++			append_or_move_hook(hook_head,
    ++					    absolute_path(legacy_hook_path));
    ++			legacy_hook = list_entry(hook_head->prev, struct hook,
    ++						 list);
    ++			legacy_hook->from_hookdir = 1;
    ++		}
     +	}
     +
      	strbuf_release(&hook_key);
    @@ hook.c: struct list_head* hook_list(const struct strbuf* hookname)
      }
     
      ## hook.h ##
    -@@ hook.h: struct hook
    +@@ hook.h: struct hook {
      	enum config_scope origin;
      	/* The literal command to run. */
      	struct strbuf command;
    -+	int from_hookdir;
    ++	unsigned from_hookdir : 1;
      };
      
      /*
 5:  723edcd785 !  5:  96c0a4838f hook: respect hook.runHookDir
    @@ Metadata
     Author: Emily Shaffer <emilyshaffer@google.com>
     
      ## Commit message ##
    -    hook: respect hook.runHookDir
    +    hook: teach hook.runHookDir
     
    -    Include hooks specified in the hook directory in the list of hooks to
    -    run. These hooks do need to be treated differently from config-specified
    -    ones - they do not need to run in a shell, and later on may be disabled
    -    or warned about based on a config setting.
    -
    -    Because they are at least as local as the local config, we'll run them
    -    last - to keep the hook execution order from global to local.
    -
    -    Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    +    For now, just give a hint about how these hooks will be run in 'git hook
    +    list'. Later on, though, we will pay attention to this enum when running
    +    the hooks.
     
      ## Documentation/config/hook.txt ##
     @@ Documentation/config/hook.txt: hookcmd.<name>.command::
    @@ builtin/hook.c: static const char * const builtin_hook_usage[] = {
      static int list(int argc, const char **argv, const char *prefix)
      {
      	struct list_head *head, *pos;
    + 	struct strbuf hookname = STRBUF_INIT;
    ++	struct strbuf hookdir_annotation = STRBUF_INIT;
    + 
    + 	struct option list_options[] = {
    + 		OPT_END(),
     @@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
      		return 0;
      	}
    @@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
     +		case HOOKDIR_NO:
     +			strbuf_addstr(&hookdir_annotation, _(" (will not run)"));
     +			break;
    ++		case HOOKDIR_ERROR:
    ++			strbuf_addstr(&hookdir_annotation, _(" (will error and not run)"));
    ++			break;
     +		case HOOKDIR_INTERACTIVE:
     +			strbuf_addstr(&hookdir_annotation, _(" (will prompt)"));
     +			break;
     +		case HOOKDIR_WARN:
    -+		case HOOKDIR_UNKNOWN:
    -+			strbuf_addstr(&hookdir_annotation, _(" (will warn)"));
    ++			strbuf_addstr(&hookdir_annotation, _(" (will warn but run)"));
     +			break;
     +		case HOOKDIR_YES:
     +		/*
     +		 * The default behavior should agree with
    -+		 * hook.c:configured_hookdir_opt().
    ++		 * hook.c:configured_hookdir_opt(). HOOKDIR_UNKNOWN should just
    ++		 * do the default behavior.
     +		 */
    ++		case HOOKDIR_UNKNOWN:
     +		default:
     +			break;
     +	}
     +
      	list_for_each(pos, head) {
    + 		struct hook *item = list_entry(pos, struct hook, list);
      		item = list_entry(pos, struct hook, list);
      		if (item) {
    + 			/* Don't translate 'hookdir' - it matches the config */
    +-			printf("%s: %s\n",
    ++			printf("%s: %s%s\n",
    + 			       (item->from_hookdir
    + 				? "hookdir"
    + 				: config_scope_name(item->origin)),
    +-			       item->command.buf);
    ++			       item->command.buf,
    ++			       (item->from_hookdir
    ++				? hookdir_annotation.buf
    ++				: ""));
    + 		}
    + 	}
    + 
    + 	clear_hook_list(head);
    ++	strbuf_release(&hookdir_annotation);
    + 	strbuf_release(&hookname);
    + 
    + 	return 0;
     @@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
      
      int cmd_hook(int argc, const char **argv, const char *prefix)
    @@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
     +	if (run_hookdir)
     +		if (!strcmp(run_hookdir, "no"))
     +			should_run_hookdir = HOOKDIR_NO;
    ++		else if (!strcmp(run_hookdir, "error"))
    ++			should_run_hookdir = HOOKDIR_ERROR;
     +		else if (!strcmp(run_hookdir, "yes"))
     +			should_run_hookdir = HOOKDIR_YES;
     +		else if (!strcmp(run_hookdir, "warn"))
    @@ hook.c: static int hook_config_lookup(const char *key, const char *value, void *
     +	if (!strcmp(key, "no"))
     +		return HOOKDIR_NO;
     +
    ++	if (!strcmp(key, "error"))
    ++		return HOOKDIR_ERROR;
    ++
     +	if (!strcmp(key, "yes"))
     +		return HOOKDIR_YES;
     +
    @@ hook.c: static int hook_config_lookup(const char *key, const char *value, void *
      	struct strbuf hook_key = STRBUF_INIT;
     
      ## hook.h ##
    -@@ hook.h: struct hook
    +@@ hook.h: struct hook {
       */
      struct list_head* hook_list(const struct strbuf *hookname);
      
     +enum hookdir_opt
     +{
     +	HOOKDIR_NO,
    ++	HOOKDIR_ERROR,
     +	HOOKDIR_WARN,
     +	HOOKDIR_INTERACTIVE,
     +	HOOKDIR_YES,
    @@ t/t1360-config-based-hooks.sh: test_expect_success 'git hook list shows hooks fr
     +	test_i18ncmp expected actual
     +'
     +
    ++test_expect_success 'hook.runHookDir = error is respected by list' '
    ++	setup_hookdir &&
    ++
    ++	test_config hook.runHookDir "error" &&
    ++
    ++	cat >expected <<-EOF &&
    ++	hookdir: $(pwd)/.git/hooks/pre-commit (will error and not run)
    ++	EOF
    ++
    ++	git hook list pre-commit >actual &&
    ++	# the hookdir annotation is translated
    ++	test_i18ncmp expected actual
    ++'
    ++
     +test_expect_success 'hook.runHookDir = warn is respected by list' '
     +	setup_hookdir &&
     +
     +	test_config hook.runHookDir "warn" &&
     +
     +	cat >expected <<-EOF &&
    -+	hookdir: $(pwd)/.git/hooks/pre-commit (will warn)
    ++	hookdir: $(pwd)/.git/hooks/pre-commit (will warn but run)
     +	EOF
     +
     +	git hook list pre-commit >actual &&
    @@ t/t1360-config-based-hooks.sh: test_expect_success 'git hook list shows hooks fr
     +	# the hookdir annotation is translated
     +	test_i18ncmp expected actual
     +'
    ++
    ++test_expect_success 'hook.runHookDir is tolerant to unknown values' '
    ++	setup_hookdir &&
    ++
    ++	test_config hook.runHookDir "junk" &&
    ++
    ++	cat >expected <<-EOF &&
    ++	hookdir: $(pwd)/.git/hooks/pre-commit
    ++	EOF
    ++
    ++	git hook list pre-commit >actual &&
    ++	# the hookdir annotation is translated
    ++	test_i18ncmp expected actual
    ++'
     +
      test_done
 6:  567f6d9d00 !  6:  9068e11679 hook: implement hookcmd.<name>.skip
    @@ Commit message
         hook: implement hookcmd.<name>.skip
     
         If a user wants a specific repo to skip execution of a hook which is set
    -    at a global or system level, they can now do so by specifying 'skip' in
    -    their repo config:
    +    at a global or system level, they will be able to do so by specifying
    +    'skip' in their repo config:
     
         ~/.gitconfig
           [hook.pre-commit]
    @@ Commit message
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
     
    + ## Documentation/config/hook.txt ##
    +@@ Documentation/config/hook.txt: hookcmd.<name>.command::
    + 	as a command. This can be an executable on your device or a oneliner for
    + 	your shell. See linkgit:git-hook[1].
    + 
    ++hookcmd.<name>.skip::
    ++	Specify this boolean to remove a command from earlier in the execution
    ++	order. Useful if you want to make a single repo an exception to hook
    ++	configured at the system or global scope. If there is no hookcmd
    ++	specified for the command you want to skip, you can use the value of
    ++	`hook.<command>.command` as <name> as a shortcut. The "skip" setting
    ++	must be specified after the "hook.<command>.command" to have an effect.
    ++
    + hook.runHookDir::
    + 	Controls how hooks contained in your hookdir are executed. Can be any of
    + 	"yes", "warn", "interactive", or "no". Defaults to "yes". See
    +
    + ## Documentation/git-hook.txt ##
    +@@ Documentation/git-hook.txt: $ git hook list "prepare-commit-msg"
    + local: /bin/linter --c
    + ----
    + 
    ++If there is a command you wish to run in most cases but have one or two
    ++exceptional repos where it should be skipped, you can use specify
    ++`hookcmd.<name>.skip`, for example:
    ++
    ++System config
    ++----
    ++  [hook "pre-commit"]
    ++    command = check-for-secrets
    ++
    ++  [hookcmd "check-for-secrets"]
    ++    command = /bin/secret-checker --aggressive
    ++----
    ++
    ++Local config
    ++----
    ++  [hookcmd "check-for-secrets"]
    ++    skip = true
    ++  # This works for inlined hook commands, too:
    ++  [hookcmd "~/typocheck.sh"]
    ++    skip = true
    ++----
    ++
    ++After these configs are added, the hook list becomes:
    ++
    ++----
    ++$ git hook list "post-commit"
    ++global: /bin/linter --c
    ++local: python ~/run-test-suite.py
    ++
    ++$ git hook list "pre-commit"
    ++no commands configured for hook 'pre-commit'
    ++----
    ++
    + COMMANDS
    + --------
    + 
    +
      ## hook.c ##
     @@ hook.c: void free_hook(struct hook *ptr)
      	}
      }
      
     -static void append_or_move_hook(struct list_head *head, const char *command)
    -+static struct hook* find_hook_by_command(struct list_head *head, const char *command)
    ++static struct hook * find_hook_by_command(struct list_head *head, const char *command)
      {
      	struct list_head *pos = NULL, *tmp = NULL;
     -	struct hook *to_add = NULL;
    @@ hook.c: void free_hook(struct hook *ptr)
     -		    /* we'll simply move the hook to the end */
     -		    to_add = it;
     +		    found = it;
    + 		    break;
      		}
      	}
     +	return found;
    @@ hook.c: void free_hook(struct hook *ptr)
      
      	if (!to_add) {
      		/* adding a new hook, not moving an old one */
    -@@ hook.c: static void append_or_move_hook(struct list_head *head, const char *command)
    - 	/* re-set the scope so we show where an override was specified */
    - 	to_add->origin = current_config_scope();
    - 
    --	list_add_tail(&to_add->list, pos);
    -+	list_add_tail(&to_add->list, head);
    - }
    - 
    - static void remove_hook(struct list_head *to_remove)
     @@ hook.c: static int hook_config_lookup(const char *key, const char *value, void *cb_data)
      	if (!strcmp(key, hook_key)) {
      		const char *command = value;
    @@ hook.c: static int hook_config_lookup(const char *key, const char *value, void *
     +		strbuf_addf(&hookcmd_name, "hookcmd.%s.skip", command);
     +		git_config_get_bool(hookcmd_name.buf, &skip);
      
    - 		/* Check if a hookcmd with that name exists. */
    + 		/*
    + 		 * Check if a hookcmd with that name exists. If it doesn't,
    + 		 * 'git_config_get_value()' is documented not to touch &command,
    + 		 * so we don't need to do anything.
    + 		 */
     +		strbuf_reset(&hookcmd_name);
      		strbuf_addf(&hookcmd_name, "hookcmd.%s.command", command);
      		git_config_get_value(hookcmd_name.buf, &command);
    @@ t/t1360-config-based-hooks.sh: test_expect_success 'hook.runHookDir = warn is re
     +	test_i18ncmp expected actual
     +'
     +
    ++test_expect_success 'git hook list ignores skip referring to unused hookcmd' '
    ++	test_config hookcmd.abc.command "/path/abc" --add &&
    ++	test_config hookcmd.abc.skip "true" --add &&
    ++
    ++	cat >expected <<-EOF &&
    ++	no commands configured for hook '\''pre-commit'\''
    ++	EOF
    ++
    ++	git hook list pre-commit >actual &&
    ++	test_i18ncmp expected actual
    ++'
    ++
     +test_expect_success 'git hook list removes skipped inlined hook' '
     +	setup_hooks &&
     +	test_config hookcmd."$ROOT/path/ghi".skip "true" --add &&
 7:  a1c02b6758 !  7:  a2867ab8c0 parse-options: parse into strvec
    @@ Documentation/technical/api-parse-options.txt: There are some macros to easily d
      	Use of `--no-option` will clear the list of preceding values.
      
     +`OPT_STRVEC(short, long, &struct strvec, arg_str, description)`::
    -+	Introduce an option with a string argument.
    -+	The string argument is stored as an element in `strvec`.
    ++	Introduce an option with a string argument, meant to be specified
    ++	multiple times.
    ++	The string argument is stored as an element in `strvec`, and later
    ++	arguments are added to the same `strvec`.
     +	Use of `--no-option` will clear the list of preceding values.
     +
      `OPT_INTEGER(short, long, &int_var, description)`::
 8:  d865772ebc !  8:  8848eeddf2 hook: add 'run' subcommand
    @@ Commit message
         supported.
     
         Legacy hooks (those present in $GITDIR/hooks) are run at the end of the
    -    execution list. For now, there is no way to disable them.
    +    execution list. They can be disabled, or made to print warnings, or to
    +    prompt before running, with the 'hook.runHookDir' config.
     
         Users may wish to provide hook commands like 'git config
         hook.pre-commit.command "~/linter.sh --pre-commit"'. To enable this,
    @@ Documentation/git-hook.txt: in the order they should be run, and print the confi
     +run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] `<hook-name>`::
     +
     +Runs hooks configured for `<hook-name>`, in the same order displayed by `git
    -+hook list`. Hooks configured this way are run prepended with `sh -c`, so paths
    -+containing special characters or spaces should be wrapped in single quotes:
    -+`command = '/my/path with spaces/script.sh' some args`.
    ++hook list`. Hooks configured this way may be run prepended with `sh -c`, so
    ++paths containing special characters or spaces should be wrapped in single
    ++quotes: `command = '/my/path with spaces/script.sh' some args`.
     +
     +OPTIONS
     +-------
    @@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
     +static int run(int argc, const char **argv, const char *prefix)
     +{
     +	struct strbuf hookname = STRBUF_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    ++	struct run_hooks_opt opt;
     +	int rc = 0;
     +
     +	struct option run_options[] = {
    @@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
     +		OPT_END(),
     +	};
     +
    -+	/*
    -+	 * While it makes sense to list hooks out-of-repo, it doesn't make sense
    -+	 * to execute them. Hooks usually want to look at repository artifacts.
    -+	 */
    -+	if (!have_git_dir())
    -+		usage_msg_opt(_("You must be in a Git repo to execute hooks."),
    -+			      builtin_hook_usage, run_options);
    ++	run_hooks_opt_init(&opt);
     +
     +	argc = parse_options(argc, argv, prefix, run_options,
     +			     builtin_hook_usage, 0);
    @@ hook.c: enum hookdir_opt configured_hookdir_opt(void)
     +
     +	switch (cfg)
     +	{
    ++		case HOOKDIR_ERROR:
    ++			fprintf(stderr, _("Skipping legacy hook at '%s'\n"),
    ++				path);
    ++			/* FALLTHROUGH */
     +		case HOOKDIR_NO:
     +			return 0;
    -+		case HOOKDIR_UNKNOWN:
    -+			fprintf(stderr,
    -+				_("Unrecognized value for 'hook.runHookDir'. "
    -+				  "Is there a typo? "));
    -+			/* FALLTHROUGH */
     +		case HOOKDIR_WARN:
     +			fprintf(stderr, _("Running legacy hook at '%s'\n"),
     +				path);
    @@ hook.c: enum hookdir_opt configured_hookdir_opt(void)
     +			} while (prompt.len); /* an empty reply means "Yes" */
     +			strbuf_release(&prompt);
     +			return 1;
    ++		/*
    ++		 * HOOKDIR_UNKNOWN should match the default behavior, but let's
    ++		 * give a heads up to the user.
    ++		 */
    ++		case HOOKDIR_UNKNOWN:
    ++			fprintf(stderr,
    ++				_("Unrecognized value for 'hook.runHookDir'. "
    ++				  "Is there a typo? "));
    ++			/* FALLTHROUGH */
     +		case HOOKDIR_YES:
     +		default:
     +			return 1;
    @@ hook.c: struct list_head* hook_list(const struct strbuf* hookname)
     +	strvec_clear(&o->args);
     +}
     +
    ++static void prepare_hook_cp(struct hook *hook, struct run_hooks_opt *options,
    ++			    struct child_process *cp)
    ++{
    ++	if (!hook)
    ++		return;
    ++
    ++	cp->no_stdin = 1;
    ++	cp->env = options->env.v;
    ++	cp->stdout_to_stderr = 1;
    ++	cp->trace2_hook_name = hook->command.buf;
    ++
    ++	/*
    ++	 * Commands from the config could be oneliners, but we know
    ++	 * for certain that hookdir commands are not.
    ++	 */
    ++	cp->use_shell = !hook->from_hookdir;
    ++
    ++	/* add command */
    ++	strvec_push(&cp->args, hook->command.buf);
    ++
    ++	/*
    ++	 * add passed-in argv, without expanding - let the user get back
    ++	 * exactly what they put in
    ++	 */
    ++	strvec_pushv(&cp->args, options->args.v);
    ++}
    ++
     +int run_hooks(const char *hookname, struct run_hooks_opt *options)
     +{
     +	struct strbuf hookname_str = STRBUF_INIT;
    @@ hook.c: struct list_head* hook_list(const struct strbuf* hookname)
     +		struct child_process hook_proc = CHILD_PROCESS_INIT;
     +		struct hook *hook = list_entry(pos, struct hook, list);
     +
    -+		hook_proc.env = options->env.v;
    -+		hook_proc.no_stdin = 1;
    -+		hook_proc.stdout_to_stderr = 1;
    -+		hook_proc.trace2_hook_name = hook->command.buf;
    -+		hook_proc.use_shell = 1;
    -+
    -+		if (hook->from_hookdir) {
    -+		    if (!should_include_hookdir(hook->command.buf, options->run_hookdir))
    ++		if (hook->from_hookdir &&
    ++		    !should_include_hookdir(hook->command.buf, options->run_hookdir))
     +			continue;
    -+		    /*
    -+		     * Commands from the config could be oneliners, but we know
    -+		     * for certain that hookdir commands are not.
    -+		     */
    -+		    hook_proc.use_shell = 0;
    -+		}
    -+
    -+		/* add command */
    -+		strvec_push(&hook_proc.args, hook->command.buf);
     +
    -+		/*
    -+		 * add passed-in argv, without expanding - let the user get back
    -+		 * exactly what they put in
    -+		 */
    -+		strvec_pushv(&hook_proc.args, options->args.v);
    ++		prepare_hook_cp(hook, options, &hook_proc);
     +
     +		rc |= run_command(&hook_proc);
     +	}
    @@ hook.h
      #include "strbuf.h"
     +#include "strvec.h"
      
    - struct hook
    - {
    + struct hook {
    + 	struct list_head list;
     @@ hook.h: enum hookdir_opt
       */
      enum hookdir_opt configured_hookdir_opt(void);
    @@ hook.h: enum hookdir_opt
     +	enum hookdir_opt run_hookdir;
     +};
     +
    -+#define RUN_HOOKS_OPT_INIT  {   		\
    -+	.env = STRVEC_INIT, 				\
    -+	.args = STRVEC_INIT, 			\
    -+	.run_hookdir = configured_hookdir_opt()	\
    -+}
    -+
     +void run_hooks_opt_init(struct run_hooks_opt *o);
     +void run_hooks_opt_clear(struct run_hooks_opt *o);
     +
    @@ t/t1360-config-based-hooks.sh: test_expect_success 'hook.runHookDir = no is resp
     +	test_must_be_empty actual
      '
      
    - test_expect_success 'hook.runHookDir = warn is respected by list' '
    + test_expect_success 'hook.runHookDir = error is respected by list' '
    +@@ t/t1360-config-based-hooks.sh: test_expect_success 'hook.runHookDir = error is respected by list' '
    + 
    + 	git hook list pre-commit >actual &&
    + 	# the hookdir annotation is translated
    ++	test_i18ncmp expected actual &&
    ++
    ++	cat >expected <<-EOF &&
    ++	Skipping legacy hook at '\''$(pwd)/.git/hooks/pre-commit'\''
    ++	EOF
    ++
    ++	git hook run pre-commit 2>actual &&
    + 	test_i18ncmp expected actual
    + '
    + 
     @@ t/t1360-config-based-hooks.sh: test_expect_success 'hook.runHookDir = warn is respected by list' '
      
      	git hook list pre-commit >actual &&
    @@ t/t1360-config-based-hooks.sh: test_expect_success 'hook.runHookDir = interactiv
     +	nongit test_must_fail git hook run pre-commit
      '
      
    - test_done
    + test_expect_success 'hook.runHookDir is tolerant to unknown values' '
 9:  53a655ed2c !  9:  452f7eea89 hook: replace find_hook() with hook_exists()
    @@ Metadata
     Author: Emily Shaffer <emilyshaffer@google.com>
     
      ## Commit message ##
    -    hook: replace find_hook() with hook_exists()
    +    hook: introduce hook_exists()
     
         Add a helper to easily determine whether any hooks exist for a given
         hook event.
    @@ Commit message
         hook; that check should include the config-based hooks as well. Optimize
         by checking the config directly. Since commands which execute hooks
         might want to take args to replace 'hook.runHookDir', let
    -    'hook_exists()' mirror the behavior of 'hook.runHookDir'.
    +    'hook_exists()' take a hookdir_opt to override that config.
     
    -    Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    +    In some cases, external callers today use find_hook() to discover the
    +    location of a hook and then run it manually with run-command.h (that is,
    +    not with run_hook_le()). Later, those cases will call hook.h:run_hook()
    +    directly instead.
     
    - ## builtin/bugreport.c ##
    -@@
    - #include "strbuf.h"
    - #include "help.h"
    - #include "compat/compiler.h"
    --#include "run-command.h"
    -+#include "hook.h"
    - 
    - 
    - static void get_system_info(struct strbuf *sys_info)
    -@@ builtin/bugreport.c: static void get_populated_hooks(struct strbuf *hook_info, int nongit)
    - 	}
    - 
    - 	for (i = 0; i < ARRAY_SIZE(hook); i++)
    --		if (find_hook(hook[i]))
    -+		if (hook_exists(hook[i], configured_hookdir_opt()))
    - 			strbuf_addf(hook_info, "%s\n", hook[i]);
    - }
    - 
    +    Once the entire codebase is using hook_exists() instead of find_hook(),
    +    find_hook() can be safely rolled into hook_exists() and removed from
    +    run-command.h.
    +
    +    Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
     
      ## hook.c ##
     @@ hook.c: void run_hooks_opt_init(struct run_hooks_opt *o)
    @@ hook.c: void run_hooks_opt_init(struct run_hooks_opt *o)
     +{
     +	const char *value = NULL; /* throwaway */
     +	struct strbuf hook_key = STRBUF_INIT;
    ++	int could_run_hookdir;
    ++
    ++	if (should_run_hookdir == HOOKDIR_USE_CONFIG)
    ++		should_run_hookdir = configured_hookdir_opt();
     +
    -+	int could_run_hookdir = (should_run_hookdir == HOOKDIR_INTERACTIVE ||
    ++	could_run_hookdir = (should_run_hookdir == HOOKDIR_INTERACTIVE ||
     +				should_run_hookdir == HOOKDIR_WARN ||
     +				should_run_hookdir == HOOKDIR_YES)
     +				&& !!find_hook(hookname);
    @@ hook.c: void run_hooks_opt_init(struct run_hooks_opt *o)
      	strvec_clear(&o->env);
     
      ## hook.h ##
    +@@ hook.h: struct list_head* hook_list(const struct strbuf *hookname);
    + 
    + enum hookdir_opt
    + {
    ++	HOOKDIR_USE_CONFIG,
    + 	HOOKDIR_NO,
    + 	HOOKDIR_ERROR,
    + 	HOOKDIR_WARN,
     @@ hook.h: struct run_hooks_opt
      void run_hooks_opt_init(struct run_hooks_opt *o);
      void run_hooks_opt_clear(struct run_hooks_opt *o);
10:  13abc6ce24 ! 10:  e76507b290 hook: support passing stdin to hooks
    @@ Documentation/git-hook.txt: in the order they should be run, and print the confi
     +run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] [--to-stdin=<path>] `<hook-name>`::
      
      Runs hooks configured for `<hook-name>`, in the same order displayed by `git
    - hook list`. Hooks configured this way are run prepended with `sh -c`, so paths
    + hook list`. Hooks configured this way may be run prepended with `sh -c`, so
     @@ Documentation/git-hook.txt: Specify arguments to pass to every hook that is run.
      +
      Specify environment variables to set for every hook that is run.
    @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
      
     
      ## hook.c ##
    -@@ hook.c: int run_hooks(const char *hookname, struct run_hooks_opt *options)
    - 		struct child_process hook_proc = CHILD_PROCESS_INIT;
    - 		struct hook *hook = list_entry(pos, struct hook, list);
    +@@ hook.c: void run_hooks_opt_init(struct run_hooks_opt *o)
    + {
    + 	strvec_init(&o->env);
    + 	strvec_init(&o->args);
    ++	o->path_to_stdin = NULL;
    + 	o->run_hookdir = configured_hookdir_opt();
    + }
    + 
    +@@ hook.c: static void prepare_hook_cp(struct hook *hook, struct run_hooks_opt *options,
    + 	if (!hook)
    + 		return;
      
    -+		/* reopen the file for stdin; run_command closes it. */
    -+		if (options->path_to_stdin)
    -+			hook_proc.in = xopen(options->path_to_stdin, O_RDONLY);
    -+		else
    -+			hook_proc.no_stdin = 1;
    +-	cp->no_stdin = 1;
    ++	/* reopen the file for stdin; run_command closes it. */
    ++	if (options->path_to_stdin)
    ++		cp->in = xopen(options->path_to_stdin, O_RDONLY);
    ++	else
    ++		cp->no_stdin = 1;
     +
    - 		hook_proc.env = options->env.v;
    --		hook_proc.no_stdin = 1;
    - 		hook_proc.stdout_to_stderr = 1;
    - 		hook_proc.trace2_hook_name = hook->command.buf;
    - 		hook_proc.use_shell = 1;
    + 	cp->env = options->env.v;
    + 	cp->stdout_to_stderr = 1;
    + 	cp->trace2_hook_name = hook->command.buf;
     
      ## hook.h ##
     @@ hook.h: struct run_hooks_opt
    @@ hook.h: struct run_hooks_opt
     +	const char *path_to_stdin;
      };
      
    - #define RUN_HOOKS_OPT_INIT  {   		\
    --	.env = STRVEC_INIT, 				\
    -+	.env = STRVEC_INIT, 			\
    - 	.args = STRVEC_INIT, 			\
    -+	.path_to_stdin = NULL,			\
    - 	.run_hookdir = configured_hookdir_opt()	\
    - }
    - 
    + void run_hooks_opt_init(struct run_hooks_opt *o);
     @@ hook.h: int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir);
      
      /*
    @@ hook.h: int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdi
      
     
      ## t/t1360-config-based-hooks.sh ##
    -@@ t/t1360-config-based-hooks.sh: test_expect_success 'out-of-repo runs excluded' '
    - 	nongit test_must_fail git hook run pre-commit
    +@@ t/t1360-config-based-hooks.sh: test_expect_success 'hook.runHookDir is tolerant to unknown values' '
    + 	test_i18ncmp expected actual
      '
      
     +test_expect_success 'stdin to multiple hooks' '
11:  0465a9ec94 ! 11:  5f41555e49 run-command: allow stdin for run_processes_parallel
    @@ run-command.c: static int pp_start_one(struct parallel_processes *pp)
      	if (i == pp->max_processes)
      		BUG("bookkeeping is hard");
      
    -+	/* disallow by default, but allow users to set up stdin if they wish */
    ++	/*
    ++	 * By default, do not inherit stdin from the parent process - otherwise,
    ++	 * all children would share stdin! Users may overwrite this to provide
    ++	 * something to the child's stdin by having their 'get_next_task'
    ++	 * callback assign 0 to .no_stdin and an appropriate integer to .in.
    ++	 */
     +	pp->children[i].process.no_stdin = 1;
     +
      	code = pp->get_next_task(&pp->children[i].process,
12:  83eb7805a4 ! 12:  a3bf826304 hook: allow parallel hook execution
    @@ Documentation/git-hook.txt: in the order they should be run, and print the confi
     +run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] [--to-stdin=<path>] [(-j|--jobs)<n>] `<hook-name>`::
      
      Runs hooks configured for `<hook-name>`, in the same order displayed by `git
    - hook list`. Hooks configured this way are run prepended with `sh -c`, so paths
    + hook list`. Hooks configured this way may be run prepended with `sh -c`, so
     @@ Documentation/git-hook.txt: Specify environment variables to set for every hook that is run.
      Specify a file which will be streamed into stdin for every hook that is run.
      Each hook will receive the entire file from beginning to EOF.
    @@ builtin/hook.c
      	NULL
      };
      
    -@@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
    - static int run(int argc, const char **argv, const char *prefix)
    - {
    - 	struct strbuf hookname = STRBUF_INIT;
    --	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
    - 	int rc = 0;
    - 
    - 	struct option run_options[] = {
     @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
      			   N_("argument to pass to hook")),
      		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
    @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
      		OPT_END(),
      	};
      
    +-	run_hooks_opt_init(&opt);
    ++	run_hooks_opt_init_async(&opt);
    + 
    + 	argc = parse_options(argc, argv, prefix, run_options,
    + 			     builtin_hook_usage, 0);
     
      ## hook.c ##
     @@ hook.c: enum hookdir_opt configured_hookdir_opt(void)
    @@ hook.c: enum hookdir_opt configured_hookdir_opt(void)
      static int should_include_hookdir(const char *path, enum hookdir_opt cfg)
      {
      	struct strbuf prompt = STRBUF_INIT;
    -@@ hook.c: void run_hooks_opt_init(struct run_hooks_opt *o)
    +@@ hook.c: struct list_head* hook_list(const struct strbuf* hookname)
    + 	return hook_head;
    + }
    + 
    +-void run_hooks_opt_init(struct run_hooks_opt *o)
    ++void run_hooks_opt_init_sync(struct run_hooks_opt *o)
    + {
      	strvec_init(&o->env);
      	strvec_init(&o->args);
    + 	o->path_to_stdin = NULL;
      	o->run_hookdir = configured_hookdir_opt();
    ++	o->jobs = 1;
    ++}
    ++
    ++void run_hooks_opt_init_async(struct run_hooks_opt *o)
    ++{
    ++	run_hooks_opt_init_sync(o);
     +	o->jobs = configured_hook_jobs();
      }
      
    @@ hook.c: void run_hooks_opt_clear(struct run_hooks_opt *o)
      	strvec_clear(&o->args);
      }
      
    -+
    +-static void prepare_hook_cp(struct hook *hook, struct run_hooks_opt *options,
    +-			    struct child_process *cp)
     +static int pick_next_hook(struct child_process *cp,
     +			  struct strbuf *out,
     +			  void *pp_cb,
     +			  void **pp_task_cb)
    -+{
    + {
     +	struct hook_cb_data *hook_cb = pp_cb;
    ++	struct hook *hook = hook_cb->run_me;
     +
    -+	struct hook *hook = list_entry(hook_cb->run_me, struct hook, list);
    -+
    -+	if (hook_cb->head == hook_cb->run_me)
    + 	if (!hook)
    +-		return;
     +		return 0;
    -+
    -+	cp->env = hook_cb->options->env.v;
    -+	cp->stdout_to_stderr = 1;
    -+	cp->trace2_hook_name = hook->command.buf;
    -+
    -+	/* reopen the file for stdin; run_command closes it. */
    + 
    + 	/* reopen the file for stdin; run_command closes it. */
    +-	if (options->path_to_stdin)
    +-		cp->in = xopen(options->path_to_stdin, O_RDONLY);
    +-	else
     +	if (hook_cb->options->path_to_stdin) {
     +		cp->no_stdin = 0;
     +		cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY);
     +	} else {
    -+		cp->no_stdin = 1;
    + 		cp->no_stdin = 1;
     +	}
    -+
    -+	/*
    -+	 * Commands from the config could be oneliners, but we know
    -+	 * for certain that hookdir commands are not.
    -+	 */
    -+	if (hook->from_hookdir)
    -+		cp->use_shell = 0;
    -+	else
    -+		cp->use_shell = 1;
    -+
    -+	/* add command */
    -+	strvec_push(&cp->args, hook->command.buf);
    -+
    -+	/*
    -+	 * add passed-in argv, without expanding - let the user get back
    -+	 * exactly what they put in
    -+	 */
    + 
    +-	cp->env = options->env.v;
    ++	cp->env = hook_cb->options->env.v;
    + 	cp->stdout_to_stderr = 1;
    + 	cp->trace2_hook_name = hook->command.buf;
    + 
    +@@ hook.c: static void prepare_hook_cp(struct hook *hook, struct run_hooks_opt *options,
    + 	 * add passed-in argv, without expanding - let the user get back
    + 	 * exactly what they put in
    + 	 */
    +-	strvec_pushv(&cp->args, options->args.v);
     +	strvec_pushv(&cp->args, hook_cb->options->args.v);
     +
     +	/* Provide context for errors if necessary */
     +	*pp_task_cb = hook;
     +
     +	/* Get the next entry ready */
    -+	hook_cb->run_me = hook_cb->run_me->next;
    ++	if (hook_cb->run_me->list.next == hook_cb->head)
    ++		hook_cb->run_me = NULL;
    ++	else
    ++		hook_cb->run_me = list_entry(hook_cb->run_me->list.next,
    ++					     struct hook, list);
     +
     +	return 1;
     +}
    @@ hook.c: void run_hooks_opt_clear(struct run_hooks_opt *o)
     +	strbuf_addf(out, _("Couldn't start '%s', configured in '%s'\n"),
     +		    attempted->command.buf,
     +		    attempted->from_hookdir ? "hookdir"
    -+		    	: config_scope_name(attempted->origin));
    ++			: config_scope_name(attempted->origin));
     +
     +	/* NEEDSWORK: if halt_on_error is desired, do it here. */
     +	return 0;
    @@ hook.c: void run_hooks_opt_clear(struct run_hooks_opt *o)
     +
     +	/* NEEDSWORK: if halt_on_error is desired, do it here. */
     +	return 0;
    -+}
    -+
    + }
    + 
      int run_hooks(const char *hookname, struct run_hooks_opt *options)
      {
      	struct strbuf hookname_str = STRBUF_INIT;
    @@ hook.c: int run_hooks(const char *hookname, struct run_hooks_opt *options)
     -		struct child_process hook_proc = CHILD_PROCESS_INIT;
      		struct hook *hook = list_entry(pos, struct hook, list);
      
    --		/* reopen the file for stdin; run_command closes it. */
    --		if (options->path_to_stdin)
    --			hook_proc.in = xopen(options->path_to_stdin, O_RDONLY);
    --		else
    --			hook_proc.no_stdin = 1;
    --
    --		hook_proc.env = options->env.v;
    --		hook_proc.stdout_to_stderr = 1;
    --		hook_proc.trace2_hook_name = hook->command.buf;
    --		hook_proc.use_shell = 1;
    --
    --		if (hook->from_hookdir) {
    --		    if (!should_include_hookdir(hook->command.buf, options->run_hookdir))
    + 		if (hook->from_hookdir &&
    + 		    !should_include_hookdir(hook->command.buf, options->run_hookdir))
     -			continue;
    --		    /*
    --		     * Commands from the config could be oneliners, but we know
    --		     * for certain that hookdir commands are not.
    --		     */
    --		    hook_proc.use_shell = 0;
    --		}
    --
    --		/* add command */
    --		strvec_push(&hook_proc.args, hook->command.buf);
    -+		if (hook->from_hookdir &&
    -+		    !should_include_hookdir(hook->command.buf, options->run_hookdir))
     +			    list_del(pos);
     +	}
    ++
    ++	if (list_empty(to_run))
    ++		return 0;
      
    --		/*
    --		 * add passed-in argv, without expanding - let the user get back
    --		 * exactly what they put in
    --		 */
    --		strvec_pushv(&hook_proc.args, options->args.v);
    +-		prepare_hook_cp(hook, options, &hook_proc);
     +	cb_data.head = to_run;
    -+	cb_data.run_me = to_run->next;
    ++	cb_data.run_me = list_entry(to_run->next, struct hook, list);
      
     -		rc |= run_command(&hook_proc);
     -	}
    @@ hook.h: enum hookdir_opt
      	/* Environment vars to be set for each hook */
     @@ hook.h: struct run_hooks_opt
      
    + 	/*
    + 	 * How should the hookdir be handled?
    +-	 * Leave the RUN_HOOKS_OPT_INIT default in most cases; this only needs
    ++	 * Leave the run_hooks_opt_init_*() default in most cases; this only needs
    + 	 * to be overridden if the user can override it at the command line.
    + 	 */
    + 	enum hookdir_opt run_hookdir;
    + 
      	/* Path to file which should be piped to stdin for each hook */
      	const char *path_to_stdin;
     +
     +	/* Number of threads to parallelize across */
     +	int jobs;
    - };
    - 
    --#define RUN_HOOKS_OPT_INIT  {   		\
    ++};
    ++
     +/*
     + * Callback provided to feed_pipe_fn and consume_sideband_fn.
     + */
     +struct hook_cb_data {
     +	int rc;
     +	struct list_head *head;
    -+	struct list_head *run_me;
    ++	struct hook *run_me;
     +	struct run_hooks_opt *options;
    -+};
    -+
    -+#define RUN_HOOKS_OPT_INIT_SYNC  {   		\
    - 	.env = STRVEC_INIT, 			\
    - 	.args = STRVEC_INIT, 			\
    - 	.path_to_stdin = NULL,			\
    -+	.jobs = 1,				\
    - 	.run_hookdir = configured_hookdir_opt()	\
    - }
    + };
      
    -+#define RUN_HOOKS_OPT_INIT_ASYNC {		\
    -+	.env = STRVEC_INIT, 			\
    -+	.args = STRVEC_INIT, 			\
    -+	.path_to_stdin = NULL,			\
    -+	.jobs = configured_hook_jobs(),		\
    -+	.run_hookdir = configured_hookdir_opt()	\
    -+}
    -+
    -+
    - void run_hooks_opt_init(struct run_hooks_opt *o);
    +-void run_hooks_opt_init(struct run_hooks_opt *o);
    ++void run_hooks_opt_init_sync(struct run_hooks_opt *o);
    ++void run_hooks_opt_init_async(struct run_hooks_opt *o);
      void run_hooks_opt_clear(struct run_hooks_opt *o);
      
    + /*
13:  f84c879d5a <  -:  ---------- hook: allow specifying working directory for hooks
 -:  ---------- > 13:  0c4add98a4 hook: allow specifying working directory for hooks
14:  ac9cec6587 = 14:  1847c4c675 run-command: add stdin callback for parallelization
15:  71fca28ccf ! 15:  ab781c94d7 hook: provide stdin by string_list or callback
    @@ hook.c: static void append_or_move_hook(struct list_head *head, const char *comm
      	}
      
      	/* re-set the scope so we show where an override was specified */
    +@@ hook.c: void run_hooks_opt_init_sync(struct run_hooks_opt *o)
    + 	o->run_hookdir = configured_hookdir_opt();
    + 	o->jobs = 1;
    + 	o->dir = NULL;
    ++	o->feed_pipe = NULL;
    ++	o->feed_pipe_ctx = NULL;
    + }
    + 
    + void run_hooks_opt_init_async(struct run_hooks_opt *o)
     @@ hook.c: void run_hooks_opt_clear(struct run_hooks_opt *o)
    - {
    - 	strvec_clear(&o->env);
      	strvec_clear(&o->args);
    -+	string_list_clear(&o->str_stdin, 0);
      }
      
    - 
    -+static int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb)
    ++int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb)
     +{
     +	int *item_idx;
     +	struct hook *ctx = pp_task_cb;
    -+	struct string_list *to_pipe = &((struct hook_cb_data*)pp_cb)->options->str_stdin;
    ++	struct string_list *to_pipe = ((struct hook_cb_data*)pp_cb)->options->feed_pipe_ctx;
     +
     +	/* Bootstrap the state manager if necessary. */
     +	if (!ctx->feed_pipe_cb_data) {
    @@ hook.c: int run_hooks(const char *hookname, struct run_hooks_opt *options)
      	if (!options)
      		BUG("a struct run_hooks_opt must be provided to run_hooks");
      
    -+	if ((options->path_to_stdin && options->str_stdin.nr) ||
    -+	    (options->path_to_stdin && options->feed_pipe) ||
    -+	    (options->str_stdin.nr && options->feed_pipe))
    ++	if (options->path_to_stdin && options->feed_pipe)
     +		BUG("choose only one method to populate stdin");
    -+
    -+	if (options->str_stdin.nr)
    -+		options->feed_pipe = &pipe_from_string_list;
     +
      	strbuf_addstr(&hookname_str, hookname);
      
    @@ hook.h
      #include "strvec.h"
     +#include "run-command.h"
      
    - struct hook
    - {
    -@@ hook.h: struct hook
    + struct hook {
    + 	struct list_head list;
    +@@ hook.h: struct hook {
      	/* The literal command to run. */
      	struct strbuf command;
    - 	int from_hookdir;
    + 	unsigned from_hookdir : 1;
     +
     +	/*
     +	 * Use this to keep state for your feed_pipe_fn if you are using
    @@ hook.h: struct run_hooks_opt
      
      	/* Path to file which should be piped to stdin for each hook */
      	const char *path_to_stdin;
    -+	/* Pipe each string to stdin, separated by newlines */
    -+	struct string_list str_stdin;
     +	/*
     +	 * Callback and state pointer to ask for more content to pipe to stdin.
     +	 * Will be called repeatedly, for each hook. See
     +	 * hook.c:pipe_from_stdin() for an example. Keep per-hook state in
     +	 * hook.feed_pipe_cb_data (per process). Keep initialization context in
     +	 * feed_pipe_ctx (shared by all processes).
    ++	 *
    ++	 * See 'pipe_from_string_list()' for info about how to specify a
    ++	 * string_list as the stdin input instead of writing your own handler.
     +	 */
     +	feed_pipe_fn feed_pipe;
     +	void *feed_pipe_ctx;
    @@ hook.h: struct run_hooks_opt
     +
      };
      
    ++/*
    ++ * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
    ++ * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'.
    ++ * This will pipe each string in the list to stdin, separated by newlines.  (Do
    ++ * not inject your own newlines.)
    ++ */
    ++int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb);
    ++
      /*
    -@@ hook.h: struct hook_cb_data {
    - 	.path_to_stdin = NULL,			\
    - 	.jobs = 1,				\
    - 	.dir = NULL,				\
    -+	.str_stdin = STRING_LIST_INIT_DUP,	\
    -+	.feed_pipe = NULL,			\
    -+	.feed_pipe_ctx = NULL,			\
    - 	.run_hookdir = configured_hookdir_opt()	\
    - }
    - 
    -@@ hook.h: struct hook_cb_data {
    - 	.path_to_stdin = NULL,			\
    - 	.jobs = configured_hook_jobs(),		\
    - 	.dir = NULL,				\
    -+	.str_stdin = STRING_LIST_INIT_DUP,	\
    -+	.feed_pipe = NULL,			\
    -+	.feed_pipe_ctx = NULL,			\
    - 	.run_hookdir = configured_hookdir_opt()	\
    - }
    - 
    +  * Callback provided to feed_pipe_fn and consume_sideband_fn.
    +  */
16:  98253fa8fd = 16:  c51bf46e8d run-command: allow capturing of collated output
17:  9505812b74 ! 17:  b90a4ee79b hooks: allow callers to capture output
    @@ Commit message
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
     
      ## hook.c ##
    +@@ hook.c: void run_hooks_opt_init_sync(struct run_hooks_opt *o)
    + 	o->dir = NULL;
    + 	o->feed_pipe = NULL;
    + 	o->feed_pipe_ctx = NULL;
    ++	o->consume_sideband = NULL;
    + }
    + 
    + void run_hooks_opt_init_async(struct run_hooks_opt *o)
     @@ hook.c: int run_hooks(const char *hookname, struct run_hooks_opt *options)
      				   pick_next_hook,
      				   notify_start_failure,
    @@ hook.h: struct run_hooks_opt
      	/* Number of threads to parallelize across */
      	int jobs;
      
    -@@ hook.h: struct hook_cb_data {
    - 	.str_stdin = STRING_LIST_INIT_DUP,	\
    - 	.feed_pipe = NULL,			\
    - 	.feed_pipe_ctx = NULL,			\
    -+	.consume_sideband = NULL,		\
    - 	.run_hookdir = configured_hookdir_opt()	\
    - }
    - 
    -@@ hook.h: struct hook_cb_data {
    - 	.str_stdin = STRING_LIST_INIT_DUP,	\
    - 	.feed_pipe = NULL,			\
    - 	.feed_pipe_ctx = NULL,			\
    -+	.consume_sideband = NULL,		\
    - 	.run_hookdir = configured_hookdir_opt()	\
    - }
    - 

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 03/37] hook: add list command
  2021-03-12  8:20   ` Ævar Arnfjörð Bjarmason
@ 2021-03-24 17:31     ` Emily Shaffer
  2021-03-25 12:36       ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-03-24 17:31 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Fri, Mar 12, 2021 at 09:20:05AM +0100, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Thu, Mar 11 2021, Emily Shaffer wrote:
> 
> > diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
> > new file mode 100644
> > index 0000000000..71449ecbc7
> > --- /dev/null
> > +++ b/Documentation/config/hook.txt
> > @@ -0,0 +1,9 @@
> > +hook.<command>.command::
> > +	A command to execute during the <command> hook event. This can be an
> > +	executable on your device, a oneliner for your shell, or the name of a
> > +	hookcmd. See linkgit:git-hook[1].
> > +
> > +hookcmd.<name>.command::
> > +	A command to execute during a hook for which <name> has been specified
> > +	as a command. This can be an executable on your device or a oneliner for
> > +	your shell. See linkgit:git-hook[1].
> > diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
> > index 9eeab0009d..f19875ed68 100644
> > --- a/Documentation/git-hook.txt
> > +++ b/Documentation/git-hook.txt
> > @@ -8,12 +8,65 @@ git-hook - Manage configured hooks
> >  SYNOPSIS
> >  --------
> >  [verse]
> > -'git hook'
> > +'git hook' list <hook-name>
> 
> Having just read this far (maybe this pattern is shared in the rest of
> the series): Let's just squash this and the 2nd patch together.
> 
> Sometimes it's worth doing the scaffolding first, but adding a new
> built-in is so trivial that I don't think it's worth it, and it just
> results in back & forth churn like the above...

Yeah, I think you are right here :)

> > +void free_hook(struct hook *ptr)
> > +{
> > +	if (ptr) {
> > +		strbuf_release(&ptr->command);
> > +		free(ptr);
> > +	}
> > +}
> 
> Neither strbuf_release() nor free() need or should have a "if (ptr)" guard.

I'll take free() out of the if guard, but I think
'strbuf_release(&<null>->command)' will go poorly - dereferencing the
NULL to even invoke strbuf_release will not be a happy time, and
strbuf_release internally is not NULL-resistant.

> > +struct list_head* hook_list(const struct strbuf* hookname)
> > +{
> > +	struct strbuf hook_key = STRBUF_INIT;
> > +	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
> > +	struct hook_config_cb cb_data = { &hook_key, hook_head };
> > +
> > +	INIT_LIST_HEAD(hook_head);
> > +
> > +	if (!hookname)
> > +		return NULL;
> 
> ...if a strbuf being passed in is NULL?

Yeah, I think this is misplaced. But since it sounds like generally
folks don't like having the strbuf at the input here, I will address the
error checking then also.

> 
> > [...]
> > +ROOT=
> > +if test_have_prereq MINGW
> > +then
> > +	# In Git for Windows, Unix-like paths work only in shell scripts;
> > +	# `git.exe`, however, will prefix them with the pseudo root directory
> > +	# (of the Unix shell). Let's accommodate for that.
> > +	ROOT="$(cd / && pwd)"
> > +fi
> 
> I didn't read up on previous rounds, but if we're squashing this into 02
> having a seperate commit summarizing this little hack would be most
> welcome, or have it in this commit message.

Sure. I squashed it in from a commit dscho sent, so I can preserve that
commit in tree instead.

> 
> Isn't this sort of thing generally usable, maybe we can add it under a
> longer variable name to test-lib.sh?

I wonder. `git grep cd \/ &&` shows me that this hack also happens in
t1509-root-work-tree.sh. I think most tests must use relative paths, so
this must not be in broad use? But since it's not used elsewhere I feel
ambivalent about adding a helper to test-lib.sh. I can if you feel
strongly :)

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 04/37] hook: include hookdir hook in list
  2021-03-12  8:30   ` Ævar Arnfjörð Bjarmason
@ 2021-03-24 17:56     ` Emily Shaffer
  2021-03-24 19:11       ` Junio C Hamano
  0 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-03-24 17:56 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Fri, Mar 12, 2021 at 09:30:04AM +0100, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Thu, Mar 11 2021, Emily Shaffer wrote:
> 
> > Historically, hooks are declared by placing an executable into
> > $GIT_DIR/hooks/$HOOKNAME (or $HOOKDIR/$HOOKNAME). Although hooks taken
> > from the config are more featureful than hooks placed in the $HOOKDIR,
> > those hooks should not stop working for users who already have them.
> > Let's list them to the user, but instead of displaying a config scope
> > (e.g. "global: blah") we can prefix them with "hookdir:".
> >
> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > ---
> >
> > Notes:
> >     Since v7, fix some nits from Jonathan Tan. The largest is to move reference to
> >     "hookdir annotation" from this commit to the next one which introduces the
> >     hook.runHookDir option.
> >
> >  builtin/hook.c                | 11 +++++++++--
> >  hook.c                        | 17 +++++++++++++++++
> >  hook.h                        |  1 +
> >  t/t1360-config-based-hooks.sh | 19 +++++++++++++++++++
> >  4 files changed, 46 insertions(+), 2 deletions(-)
> >
> > diff --git a/builtin/hook.c b/builtin/hook.c
> > index bb64cd77ca..c8fbfbb39d 100644
> > --- a/builtin/hook.c
> > +++ b/builtin/hook.c
> > @@ -40,10 +40,15 @@ static int list(int argc, const char **argv, const char *prefix)
> >  
> >  	list_for_each(pos, head) {
> >  		struct hook *item = list_entry(pos, struct hook, list);
> > -		if (item)
> > +		item = list_entry(pos, struct hook, list);
> > +		if (item) {
> > +			/* Don't translate 'hookdir' - it matches the config */
> 
> Let's prefix comments for translators with /* TRANSLATORS: .., see the
> coding style doc. That's what they'll see, and this is useful to them.
> 
> Better yet have a note here about the first argument being 'system',
> 'local' etc., which I had to source spelunge for, and translators won't
> have any idea about unless the magic parameter is documented.

It's not a comment for translators. It's a comment for someone helpful
who comes later and says "oh, none of this is marked for translation,
I'd better fix that."

> 
> > +setup_hookdir () {
> > +	mkdir .git/hooks
> > +	write_script .git/hooks/pre-commit <<-EOF
> > +	echo \"Legacy Hook\"
> 
> Nit, "'s not needed, but it also seems nothing uses this, so if it's
> just a pass-through script either "exit 0", or actually check if it's
> run or something?

The output is checked in the run tests later on. I can remove it for
this commit if you want.

> 
> > [...]
> > +test_expect_success 'git hook list shows hooks from the hookdir' '
> > +	setup_hookdir &&
> > +
> > +	cat >expected <<-EOF &&
> > +	hookdir: $(pwd)/.git/hooks/pre-commit
> > +	EOF
> > +
> > +	git hook list pre-commit >actual &&
> > +	test_cmp expected actual
> > +'
> 
> Ah, so it's just checking if it exists...

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 05/37] hook: teach hook.runHookDir
  2021-03-12  8:33   ` Ævar Arnfjörð Bjarmason
@ 2021-03-24 18:46     ` Emily Shaffer
  2021-03-24 22:38       ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-03-24 18:46 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Fri, Mar 12, 2021 at 09:33:46AM +0100, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Thu, Mar 11 2021, Emily Shaffer wrote:
> 
> > +	switch (should_run_hookdir) {
> > +		case HOOKDIR_NO:
> 
> Style: case shouldn't be indented

Done, thanks.

> 
> > +			strbuf_addstr(&hookdir_annotation, _(" (will not run)"));
> > +			break;
> > +		case HOOKDIR_ERROR:
> > +			strbuf_addstr(&hookdir_annotation, _(" (will error and not run)"));
> > +			break;
> > +		case HOOKDIR_INTERACTIVE:
> > +			strbuf_addstr(&hookdir_annotation, _(" (will prompt)"));
> > +			break;
> > +		case HOOKDIR_WARN:
> > +			strbuf_addstr(&hookdir_annotation, _(" (will warn but run)"));
> > +			break;
> > +		case HOOKDIR_YES:
> > +		/*
> > +		 * The default behavior should agree with
> > +		 * hook.c:configured_hookdir_opt(). HOOKDIR_UNKNOWN should just
> > +		 * do the default behavior.
> > +		 */
> > +		case HOOKDIR_UNKNOWN:
> > +		default:
> > +			break;
> 
> We should avoid this sort of translation lego.
> 
> > +	}
> > +
> >  	list_for_each(pos, head) {
> >  		struct hook *item = list_entry(pos, struct hook, list);
> >  		item = list_entry(pos, struct hook, list);
> >  		if (item) {
> >  			/* Don't translate 'hookdir' - it matches the config */
> > -			printf("%s: %s\n",
> > +			printf("%s: %s%s\n",
> 
> native speakers in some languages to read the sentance backwards.
> Because if you concatenate strings like this you force.
> 
> (We don't currently have a RTL language in po/, still, but let's not
> create churn for if/when we do if we can help it)>

Yeah, you are absolutely right. I'll take a look at your suggestion,
thanks.

> 
> 
> I have a patch on top to fix this, will send it as some general reply of
> proposed fixup.s

FWIW, I found this format of suggestions really hard to navigate. I had
to go find your fixup (and I think you sent two different ones) and then
had to scroll around and find what you're referring to from here. If I
were to apply the fixup directly, I'd have to split it up and find the
appropriate commits to associate each part of the diff with. I know
generating scissors patches for each review is a pain, but I'd even
prefer a prose "How about printing in each part of the case instead, so
each string can be translated in full" or a non-formatted example inline
to the catchall fixups patch you sent.

> 
> >  			       (item->from_hookdir
> > +	git hook list pre-commit >actual &&
> > +	# the hookdir annotation is translated
> > +	test_i18ncmp expected actual
> 
> This (and the rest of test_i18ncmp in this series) can and should just
> be "test_cmp" or "test_i18ncmp", the poison mode is dead. See my recent
> patches to search/replace test_i18ncmp.
> 
> The reason the function isn't gone entirely was to help a series like
> yours in "seen", but if we're re-rolling...

Oh cool, thanks, will do.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 04/37] hook: include hookdir hook in list
  2021-03-24 17:56     ` Emily Shaffer
@ 2021-03-24 19:11       ` Junio C Hamano
  2021-03-24 19:23         ` Eric Sunshine
  0 siblings, 1 reply; 479+ messages in thread
From: Junio C Hamano @ 2021-03-24 19:11 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Ævar Arnfjörð Bjarmason, git

Emily Shaffer <emilyshaffer@google.com> writes:

>> > @@ -40,10 +40,15 @@ static int list(int argc, const char **argv, const char *prefix)
>> >  
>> >  	list_for_each(pos, head) {
>> >  		struct hook *item = list_entry(pos, struct hook, list);
>> > -		if (item)
>> > +		item = list_entry(pos, struct hook, list);
>> > +		if (item) {
>> > +			/* Don't translate 'hookdir' - it matches the config */
>> 
>> Let's prefix comments for translators with /* TRANSLATORS: .., see the
>> coding style doc. That's what they'll see, and this is useful to them.
>> 
>> Better yet have a note here about the first argument being 'system',
>> 'local' etc., which I had to source spelunge for, and translators won't
>> have any idea about unless the magic parameter is documented.
>
> It's not a comment for translators. It's a comment for someone helpful
> who comes later and says "oh, none of this is marked for translation,
> I'd better fix that."

Then, it is not limited to "hookdir", is it?  Resurrecting the
elided part back here:

Not just we do not want "hookdir" placed inside _(),

 			printf("%s: %s\n",
+			       (item->from_hookdir
+				? "hookdir"
+				: config_scope_name(item->origin)),
 			       item->command.buf);

we do not want the "%s: %s\n" to be placed inside _() and get munged
into "%2$s: %1$s\n" for languages that want the order swapped, for
example.

So perhaps the comment should be about the entire output, i.e.
"don't translate the output from this helper, as it is meant to be
machine parseable", or something?


^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 04/37] hook: include hookdir hook in list
  2021-03-24 19:11       ` Junio C Hamano
@ 2021-03-24 19:23         ` Eric Sunshine
  2021-03-24 20:07           ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Eric Sunshine @ 2021-03-24 19:23 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Emily Shaffer, Ævar Arnfjörð Bjarmason, Git List

On Wed, Mar 24, 2021 at 3:12 PM Junio C Hamano <gitster@pobox.com> wrote:
> Not just we do not want "hookdir" placed inside _(),
>
>                         printf("%s: %s\n",
> +                              (item->from_hookdir
> +                               ? "hookdir"
> +                               : config_scope_name(item->origin)),
>                                item->command.buf);
>
> we do not want the "%s: %s\n" to be placed inside _() and get munged
> into "%2$s: %1$s\n" for languages that want the order swapped, for
> example.
>
> So perhaps the comment should be about the entire output, i.e.
> "don't translate the output from this helper, as it is meant to be
> machine parseable", or something?

Having the word "translate" in the comment automatically implies
localization, which confuses the issue. It would be clearer to avoid
that word altogether. Perhaps something along the lines of:

    /* machine-parseable output; do not apply _() localization */

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 04/37] hook: include hookdir hook in list
  2021-03-24 19:23         ` Eric Sunshine
@ 2021-03-24 20:07           ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-24 20:07 UTC (permalink / raw)
  To: Eric Sunshine
  Cc: Junio C Hamano, Ævar Arnfjörð Bjarmason, Git List

On Wed, Mar 24, 2021 at 03:23:30PM -0400, Eric Sunshine wrote:
> 
> On Wed, Mar 24, 2021 at 3:12 PM Junio C Hamano <gitster@pobox.com> wrote:
> > Not just we do not want "hookdir" placed inside _(),
> >
> >                         printf("%s: %s\n",
> > +                              (item->from_hookdir
> > +                               ? "hookdir"
> > +                               : config_scope_name(item->origin)),
> >                                item->command.buf);
> >
> > we do not want the "%s: %s\n" to be placed inside _() and get munged
> > into "%2$s: %1$s\n" for languages that want the order swapped, for
> > example.
> >
> > So perhaps the comment should be about the entire output, i.e.
> > "don't translate the output from this helper, as it is meant to be
> > machine parseable", or something?
> 
> Having the word "translate" in the comment automatically implies
> localization, which confuses the issue. It would be clearer to avoid
> that word altogether. Perhaps something along the lines of:
> 
>     /* machine-parseable output; do not apply _() localization */

After I read Ævar's comments on the next patch in this series, I decided
to rework the comments and translation markers for this whole section.

  if (item) {
          if (item->from_hookdir) {
                  /*
                   * TRANSLATORS: do not translate 'hookdir' as
                   * it matches the config setting.
                   */
                  switch (should_run_hookdir) {
                  case HOOKDIR_NO:
                          printf(_("hookdir: %s (will not run)\n"),
                                 item->command.buf);
                          break;
                  case HOOKDIR_ERROR:
                          printf(_("hookdir: %s (will error and not run)\n"),
                                 item->command.buf);
                          break;
                  case HOOKDIR_INTERACTIVE:
                          printf(_("hookdir: %s (will prompt)\n"),
                                 item->command.buf);
                          break;
                  case HOOKDIR_WARN:
                          printf(_("hookdir: %s (will warn but run)\n"),
                                 item->command.buf);
                          break;
                  case HOOKDIR_YES:
                  /*
                   * The default behavior should agree with
                   * hook.c:configured_hookdir_opt(). HOOKDIR_UNKNOWN should just
                   * do the default behavior.
                   */
                  case HOOKDIR_UNKNOWN:
                  default:
                          printf(_("hookdir: %s\n"),
                                   item->command.buf);
                          break;
                  }
          } else {
                  /*
                   * TRANSLATORS: "<config scope>: <path>". Both fields
                   * should be left untranslated; config scope matches the
                   * output of 'git config --show-scope'. Marked for
                   * translation to provide better RTL support later.
                   */
                  printf(_("%s: %s\n"),
                          config_scope_name(item->origin),
                          item->command.buf);
          }
  }

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 07/37] parse-options: parse into strvec
  2021-03-12  8:50   ` Ævar Arnfjörð Bjarmason
@ 2021-03-24 20:34     ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-24 20:34 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Fri, Mar 12, 2021 at 09:50:58AM +0100, Ævar Arnfjörð Bjarmason wrote:
[snip]
> Nice, seems very useful.
> 
> But let's add a test in test-parse-options.c like we have for
> string_list?

Sure, done. Thanks.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 08/37] hook: add 'run' subcommand
  2021-03-12  8:54   ` Ævar Arnfjörð Bjarmason
@ 2021-03-24 21:29     ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-24 21:29 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Fri, Mar 12, 2021 at 09:54:28AM +0100, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Thu, Mar 11 2021, Emily Shaffer wrote:
> 
> >  'git hook' list <hook-name>
> > +'git hook' run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] <hook-name>
> 
> [...]
> 
> > +	switch (cfg)
> > +	{
> > +		case HOOKDIR_ERROR:
> 
> Overly indented case statements again.
Thanks. Probably need to fix my autoformatter or something.
> 
> > +			fprintf(stderr, _("Skipping legacy hook at '%s'\n"),
> > +				path);
> > +			/* FALLTHROUGH */
> > +		case HOOKDIR_NO:
> > +			return 0;
> > +		case HOOKDIR_WARN:
> > +			fprintf(stderr, _("Running legacy hook at '%s'\n"),
> > +				path);
> > +			return 1;
> > +		case HOOKDIR_INTERACTIVE:
> > +			do {
> > +				/*
> > +				 * TRANSLATORS: Make sure to include [Y] and [n]
> > +				 * in your translation. Only English input is
> > +				 * accepted. Default option is "yes".
> > +				 */
> > +				fprintf(stderr, _("Run '%s'? [Yn] "), path);
> 
> Nit: [Y/n]
ACK

> 
> > +				} else if (starts_with(prompt.buf, "y")) {
> 
> So also "Y", "yes" and "yellow"...
That's also how add-patch.c:prompt_yesno(),
builtin/bisect--helper.c:decide_next(), and
git-add--interactive.perl:prompt_yesno (assuming I'm grokking the perl
correctly) work. builtin/clean.c:ask_each_cmd checks that the user's
reply matches a substring anchored to the beginning (e.g. "y", "ye",
"yes").  git-diftool--helper.sh:launch_merge_tool just checks for "not
'n'". And git-send-email.perl:ask just checks for "'y' or not".

So I think there's a little flexibility :) But I like builtin/clean.c's
approach most out of these, so I'll switch.

> 
> > [...]
> >  	git hook list pre-commit >actual &&
> >  	# the hookdir annotation is translated
> > -	test_i18ncmp expected actual
> > +	test_i18ncmp expected actual &&
> > +
> > +	test_write_lines n | git hook run pre-commit 2>actual &&
> > +	! grep "Legacy Hook" actual &&
> > +
> > +	test_write_lines y | git hook run pre-commit 2>actual &&
> > +	grep "Legacy Hook" actual
> > +'
> > +
> > +test_expect_success 'inline hook definitions execute oneliners' '
> > +	test_config hook.pre-commit.command "echo \"Hello World\"" &&
> > +
> > +	echo "Hello World" >expected &&
> > +
> > +	# hooks are run with stdout_to_stderr = 1
> > +	git hook run pre-commit 2>actual &&
> > +	test_cmp expected actual
> > +'
> > +
> > +test_expect_success 'inline hook definitions resolve paths' '
> > +	write_script sample-hook.sh <<-EOF &&
> > +	echo \"Sample Hook\"
> > +	EOF
> > +
> > +	test_when_finished "rm sample-hook.sh" &&
> > +
> > +	test_config hook.pre-commit.command "\"$(pwd)/sample-hook.sh\"" &&
> > +
> > +	echo \"Sample Hook\" >expected &&
> > +
> > +	# hooks are run with stdout_to_stderr = 1
> > +	git hook run pre-commit 2>actual &&
> > +	test_cmp expected actual
> > +'
> > +
> > +test_expect_success 'hookdir hook included in git hook run' '
> > +	setup_hookdir &&
> > +
> > +	echo \"Legacy Hook\" >expected &&
> > +
> > +	# hooks are run with stdout_to_stderr = 1
> > +	git hook run pre-commit 2>actual &&
> > +	test_cmp expected actual
> > +'
> > +
> > +test_expect_success 'out-of-repo runs excluded' '
> > +	setup_hooks &&
> > +
> > +	nongit test_must_fail git hook run pre-commit
> >  '
> >  
> >  test_expect_success 'hook.runHookDir is tolerant to unknown values' '
> 
> No tests for --env or --arg?

Yikes, I guess not. Thanks, will add.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 17/37] hooks: allow callers to capture output
  2021-03-12  9:08   ` Ævar Arnfjörð Bjarmason
@ 2021-03-24 21:54     ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-24 21:54 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Fri, Mar 12, 2021 at 10:08:04AM +0100, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Thu, Mar 11 2021, Emily Shaffer wrote:
> 
> > Some server-side hooks will require capturing output to send over
> > sideband instead of printing directly to stderr. Expose that capability.
> 
> So added here in 17/37 and not used until 30/37. As a point on
> readability (this isn't the first such patch) I think it would be better
> to just squash those together with some "since we now need access to
> consume_sideband in hooks, do that ...".

Yeah. When I was putting together the series I had two thoughts on how
best to organize it:

1. Adding functionality just-in-time for the hook that needs it (like
you describe)
or
2. Implementing the whole utility, then doing hook conversions in a
separate chunk or series (what I went with).

I chose 2 for a couple reasons: that it would be easier for people who
just care "did a hook I use start working differently?" to review only
the second chunk of the change, and that it would be easier if we wanted
to adopt the library part into the codebase without converting the hooks
to use it (this was listed as a step in the design doc, but I think we
ended up abandoning it). The differentiation was certainly easier when I
had the two "chunks" separated into a part I and part II series, but
Junio asked me to combine them starting with this revision so it would
be easier to merge to 'seen' (as I understood it).

At this point, I'd prefer not to rearrange the series, though.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 05/37] hook: teach hook.runHookDir
  2021-03-24 18:46     ` Emily Shaffer
@ 2021-03-24 22:38       ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-24 22:38 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Wed, Mar 24 2021, Emily Shaffer wrote:

> On Fri, Mar 12, 2021 at 09:33:46AM +0100, �var Arnfj�r� Bjarmason wrote:
>> 
>> 
>> On Thu, Mar 11 2021, Emily Shaffer wrote:
>> 
>> > +	switch (should_run_hookdir) {
>> > +		case HOOKDIR_NO:
>> 
>> Style: case shouldn't be indented
>
> Done, thanks.
>
>> 
>> > +			strbuf_addstr(&hookdir_annotation, _(" (will not run)"));
>> > +			break;
>> > +		case HOOKDIR_ERROR:
>> > +			strbuf_addstr(&hookdir_annotation, _(" (will error and not run)"));
>> > +			break;
>> > +		case HOOKDIR_INTERACTIVE:
>> > +			strbuf_addstr(&hookdir_annotation, _(" (will prompt)"));
>> > +			break;
>> > +		case HOOKDIR_WARN:
>> > +			strbuf_addstr(&hookdir_annotation, _(" (will warn but run)"));
>> > +			break;
>> > +		case HOOKDIR_YES:
>> > +		/*
>> > +		 * The default behavior should agree with
>> > +		 * hook.c:configured_hookdir_opt(). HOOKDIR_UNKNOWN should just
>> > +		 * do the default behavior.
>> > +		 */
>> > +		case HOOKDIR_UNKNOWN:
>> > +		default:
>> > +			break;
>> 
>> We should avoid this sort of translation lego.
>> 
>> > +	}
>> > +
>> >  	list_for_each(pos, head) {
>> >  		struct hook *item = list_entry(pos, struct hook, list);
>> >  		item = list_entry(pos, struct hook, list);
>> >  		if (item) {
>> >  			/* Don't translate 'hookdir' - it matches the config */
>> > -			printf("%s: %s\n",
>> > +			printf("%s: %s%s\n",
>> 
>> native speakers in some languages to read the sentance backwards.
>> Because if you concatenate strings like this you force.
>> 
>> (We don't currently have a RTL language in po/, still, but let's not
>> create churn for if/when we do if we can help it)>
>
> Yeah, you are absolutely right. I'll take a look at your suggestion,
> thanks.
>
>> 
>> 
>> I have a patch on top to fix this, will send it as some general reply of
>> proposed fixup.s
>
> FWIW, I found this format of suggestions really hard to navigate. I had
> to go find your fixup (and I think you sent two different ones) and then
> had to scroll around and find what you're referring to from here. If I
> were to apply the fixup directly, I'd have to split it up and find the
> appropriate commits to associate each part of the diff with. I know
> generating scissors patches for each review is a pain, but I'd even
> prefer a prose "How about printing in each part of the case instead, so
> each string can be translated in full" or a non-formatted example inline
> to the catchall fixups patch you sent.

Sorry about that. FWIW I did try, but found myself jumping a bit too
much between different commits (as noted in some "maybe squash?"
comments elsewhere), and eventually just gave up and produced some
RFC-just-for-discussion patch on top of the whole thing and mostly
looked at the entire diff forthe series.

>> 
>> >  			       (item->from_hookdir
>> > +	git hook list pre-commit >actual &&
>> > +	# the hookdir annotation is translated
>> > +	test_i18ncmp expected actual
>> 
>> This (and the rest of test_i18ncmp in this series) can and should just
>> be "test_cmp" or "test_i18ncmp", the poison mode is dead. See my recent
>> patches to search/replace test_i18ncmp.
>> 
>> The reason the function isn't gone entirely was to help a series like
>> yours in "seen", but if we're re-rolling...
>
> Oh cool, thanks, will do.


^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 03/37] hook: add list command
  2021-03-24 17:31     ` Emily Shaffer
@ 2021-03-25 12:36       ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-25 12:36 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Wed, Mar 24 2021, Emily Shaffer wrote:

> On Fri, Mar 12, 2021 at 09:20:05AM +0100, �var Arnfj�r� Bjarmason wrote:
>> 
>> 
>> On Thu, Mar 11 2021, Emily Shaffer wrote:
>> 
>> > diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
>> > new file mode 100644
>> > index 0000000000..71449ecbc7
>> > --- /dev/null
>> > +++ b/Documentation/config/hook.txt
>> > @@ -0,0 +1,9 @@
>> > +hook.<command>.command::
>> > +	A command to execute during the <command> hook event. This can be an
>> > +	executable on your device, a oneliner for your shell, or the name of a
>> > +	hookcmd. See linkgit:git-hook[1].
>> > +
>> > +hookcmd.<name>.command::
>> > +	A command to execute during a hook for which <name> has been specified
>> > +	as a command. This can be an executable on your device or a oneliner for
>> > +	your shell. See linkgit:git-hook[1].
>> > diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
>> > index 9eeab0009d..f19875ed68 100644
>> > --- a/Documentation/git-hook.txt
>> > +++ b/Documentation/git-hook.txt
>> > @@ -8,12 +8,65 @@ git-hook - Manage configured hooks
>> >  SYNOPSIS
>> >  --------
>> >  [verse]
>> > -'git hook'
>> > +'git hook' list <hook-name>
>> 
>> Having just read this far (maybe this pattern is shared in the rest of
>> the series): Let's just squash this and the 2nd patch together.
>> 
>> Sometimes it's worth doing the scaffolding first, but adding a new
>> built-in is so trivial that I don't think it's worth it, and it just
>> results in back & forth churn like the above...
>
> Yeah, I think you are right here :)
>
>> > +void free_hook(struct hook *ptr)
>> > +{
>> > +	if (ptr) {
>> > +		strbuf_release(&ptr->command);
>> > +		free(ptr);
>> > +	}
>> > +}
>> 
>> Neither strbuf_release() nor free() need or should have a "if (ptr)" guard.
>
> I'll take free() out of the if guard, but I think
> 'strbuf_release(&<null>->command)' will go poorly - dereferencing the
> NULL to even invoke strbuf_release will not be a happy time, and
> strbuf_release internally is not NULL-resistant.

Sorry I meant something like:

    if (ptr) strbuf_release(&ptr->command);
    free(ptr);

But maybe even more idiomatic would be:

    if (!ptr)
	return;
    strbuf_release(&ptr->command);
    free(ptr);

Or some other variant of checking teh container struct early. Anyway,
this doesn't really matter, per a below comment I had more meaningful
feedback in [1]. Most of my other traffic on this topic (including this)
was some stream-of-consciousness notes as I went along.

>> > +struct list_head* hook_list(const struct strbuf* hookname)
>> > +{
>> > +	struct strbuf hook_key = STRBUF_INIT;
>> > +	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
>> > +	struct hook_config_cb cb_data = { &hook_key, hook_head };
>> > +
>> > +	INIT_LIST_HEAD(hook_head);
>> > +
>> > +	if (!hookname)
>> > +		return NULL;
>> 
>> ...if a strbuf being passed in is NULL?
>
> Yeah, I think this is misplaced. But since it sounds like generally
> folks don't like having the strbuf at the input here, I will address the
> error checking then also.
>
>> 
>> > [...]
>> > +ROOT=
>> > +if test_have_prereq MINGW
>> > +then
>> > +	# In Git for Windows, Unix-like paths work only in shell scripts;
>> > +	# `git.exe`, however, will prefix them with the pseudo root directory
>> > +	# (of the Unix shell). Let's accommodate for that.
>> > +	ROOT="$(cd / && pwd)"
>> > +fi
>> 
>> I didn't read up on previous rounds, but if we're squashing this into 02
>> having a seperate commit summarizing this little hack would be most
>> welcome, or have it in this commit message.
>
> Sure. I squashed it in from a commit dscho sent, so I can preserve that
> commit in tree instead.
>
>> 
>> Isn't this sort of thing generally usable, maybe we can add it under a
>> longer variable name to test-lib.sh?
>
> I wonder. `git grep cd \/ &&` shows me that this hack also happens in
> t1509-root-work-tree.sh. I think most tests must use relative paths, so
> this must not be in broad use? But since it's not used elsewhere I feel
> ambivalent about adding a helper to test-lib.sh. I can if you feel
> strongly :)

After I sent this I saw that pretty much the same thing is happening in
t1300-config.sh for the --show-origin option.

    ! test_have_prereq MINGW ||
    HOME="$(pwd)" # convert to Windows path

I don't feel strongly about this at all, but per the outstanding
feedback I had in[1] I wondered whether this whole thing wouln't be
better as some variant of "git config --show-origin",

1. https://lore.kernel.org/git/87mtv8fww3.fsf@evledraar.gmail.com/#t

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 00/37] config-based hooks
  2021-03-12 11:13 ` Ævar Arnfjörð Bjarmason
@ 2021-03-25 12:41   ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-25 12:41 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: git, Jeff King, Junio C Hamano, James Ramsay, Jonathan Nieder,
	brian m. carlson, Phillip Wood, Josh Steadmon,
	Johannes Schindelin, Jonathan Tan


On Fri, Mar 12 2021, Ævar Arnfjörð Bjarmason wrote:

A small correction to one of my comments:

> On Thu, Mar 11 2021, Emily Shaffer wrote:

>  2. You're sticking full paths in the git config key, which is
>     case-insensitive, and a feature of this format is being able to
>     configure/override previously configured hooks.
>
>     So the behavior of this feature depends on git's interaction with
>     the case-sensitivity of filesystems, and not just one fs, any fs
>     we're walking in our various config sources, and where the hook
>     itself lives.
>
>     As recent CVEs have shown that's a big can of worms, particularly
>     for something whose goal is to address the security aspect of
>     running hooks from other config.
>
>     Arguably the case-sensitivity issue is just confusing since we
>     canonicalize it anyway. But once you add in FS path canonicalization
>     it becomes a real big can of worms. See the .gitmodules fsck code.
>
>     Even if it wasn't for that it's relatively nastier to edit/maintain
>     full paths and the appropriate escaping in the double-quoted key in
>     the config file v.s. having it as an optionally quoted value.

So the "case-insensitive" part of that *mostly* doesn't apply.

I'd forgotten that we don't consider the "LeVeL" part of
"ThReE.LeVeL.KeY" to be case-insensitive, but the other two components
are, as discussed in git-config(1)'s docs.

I say "mostly" because that's tolower()'s idea of case normalization,
which may or may not match the FS's, but anyway, I think that's probably
splitting hairs, but I worry more about the path normalization aspect
noted in the last two paragraphs there.

>  3. We're left with this "*.command = cmd", and "*.skip = true"
>     special-case syntax. I can't see any reason for why it's needed over
>     simply having "*.command = true" clobber earlier hooks as noted in
>     the proposed docs above.
>
>     And that doesn't require any magic to support, just like our
>     existing "core.pager=cat" case.
>
>     I mean, I suppose it's magical in that we might otherwise error on
>     non-consumed stdin (do we?), anyway, documenting it as a synonym for
>     "cat >/dev/null" would get around that :)
>
>  4. It makes the common case of having the same hooks for N commands
>     needlessly verbose, if you can just support "type" (or whatever we
>     should call it) you can add that N times...
>
>  5. At the end of this series we're left with the start of the docs
>     saying:
>
>       You can list and run configured hooks with this command. Later,
>       you will be able to add and modify hooks with this command.
>
>     But those patches have yet to land, and looking at the design
>     document I'm skeptical of that being a good addition v.s. just
>     adding the same thing to "git config".
>
>     As just one exmaple; surely "git config edit <name>" would need to
>     run around and find config files to edit, then open them in a loop
>     for you, no?
>
>     Which we'd eventually want for "git config" in general with an
>     --edit-regexp option or whatever, which brings us (well, at least
>     me) back to "then let's just add it to git-config?".
>
>  6. The whole 'git hook' config special-casing doesn't help other
>     commands or the security issue that seemed to have prompted (at
>     least some of) its existence
>
>     In the design doc we mention the "core.pager = rm -rf /" case for a
>     .git/config.
>
>     This series doesn't implement, but the design docs note a future
>     want for solving that issue for the hooks.
>
>     To me that's another case where we should just have general config
>     syntax, not something hook-specific, e.g. if I could do this in my
>     ~/.gitconfig:
>
>        ;; We consider 'config.ignore' in reverse order, so e.g setting
>        ;; it in. ~/.gitconfig will ignore any such keys for repo-level
>        ;; config
>        [config "ignore"]
>        key = core.pager
>        keyRegexp = "^hook\."
>
>     We'd address both any hook security concerns, as well as core.pager
>     etc. We could then just have e.g. some syntax sugar of:
>
>        [include]
>        path = built-in://gimme-safe-config
>
>     Which would just be a thin layer of magit to include
>     <path-to-git-prefix>/config-templates/gimme-safe-config or whatever.
>
>     We'd thus address the issue for all config types without
>     hook-specific magic.
>
> Anyway. I'm very willing to be convinced otherwise. I just think that
> for a first-draft implementation leaving aside 'hook.<command>.command'
> and the whole 'list' thing makes sense.
>
> We can consider the core code changes relatively separately from any
> future aspirations, particularly with a 40-some patch series, and the
> end-state of *this series* IMO not really justifying, that part of the
> implementation, and thus requiring reviewers to look ahead beyond the
> 40-some patches.

Emily: *Bump* on being interesed in what you think about the rest of
this though.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 19/37] am: convert applypatch hooks to use config
  2021-03-12 10:23   ` Junio C Hamano
@ 2021-03-29 23:39     ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-29 23:39 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Fri, Mar 12, 2021 at 02:23:39AM -0800, Junio C Hamano wrote:
> 
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > @@ -1558,8 +1563,10 @@ static void do_commit(const struct am_state *state)
> >  	struct commit_list *parents = NULL;
> >  	const char *reflog_msg, *author, *committer = NULL;
> >  	struct strbuf sb = STRBUF_INIT;
> > +	struct run_hooks_opt hook_opt;
> > +	run_hooks_opt_init_async(&hook_opt);
> >  
> > -	if (run_hook_le(NULL, "pre-applypatch", NULL))
> > +	if (run_hooks("pre-applypatch", &hook_opt))
> >  		exit(1);
> >  
> >  	if (write_cache_as_tree(&tree, 0, NULL))
> > @@ -1611,8 +1618,9 @@ static void do_commit(const struct am_state *state)
> >  		fclose(fp);
> >  	}
> >  
> > -	run_hook_le(NULL, "post-applypatch", NULL);
> > +	run_hooks("post-applypatch", &hook_opt);
> >  
> > +	run_hooks_opt_clear(&hook_opt);
> >  	strbuf_release(&sb);
> >  }
> 
> This one does opt_init(), run_hooks(), and another run_hooks() and
> then opt_clear().  If run_hooks() is a read-only operation on the
> hook_opt, then that would be alright, but it just smells iffy that
> it is not done as two separate opt_init(), run_hooks(), opt_clear()
> sequences for two separate run_hooks() invocations.  The same worry
> about future safety I meantioned elsewhere in the series also
> applies.

Interesting observation. I think the only thing that could be mutated in
the run_hooks_opt struct today is the caller-provided callback data
(run_hooks_opt.feed_pipe_ctx) - which presumably is being manipulated by the
caller in a callback they wrote. But I don't think it hurts particularly
to clear/init again between the two invocations, to be safe - so I will
change the code here.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 23/37] read-cache: convert post-index-change hook to use config
  2021-03-12 10:22   ` Junio C Hamano
@ 2021-03-29 23:56     ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-29 23:56 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Fri, Mar 12, 2021 at 02:22:08AM -0800, Junio C Hamano wrote:
> 
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > @@ -3070,6 +3071,8 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
> >  				 unsigned flags)
> >  {
> >  	int ret;
> > +	struct run_hooks_opt hook_opt;
> > +	run_hooks_opt_init_async(&hook_opt);
> >  
> 
> Nit. blank line between the last of decls and the first stmt (many
> identical nits exist everywhere in this series).
> 
> >  	/*
> >  	 * TODO trace2: replace "the_repository" with the actual repo instance
> > @@ -3088,9 +3091,13 @@ static int do_write_locked_index(s
> >  	else
> >  		ret = close_lock_file_gently(lock);
> >  
> > -	run_hook_le(NULL, "post-index-change",
> > -			istate->updated_workdir ? "1" : "0",
> > -			istate->updated_skipworktree ? "1" : "0", NULL);
> > +	strvec_pushl(&hook_opt.args,
> > +		     istate->updated_workdir ? "1" : "0",
> > +		     istate->updated_skipworktree ? "1" : "0",
> > +		     NULL);
> > +	run_hooks("post-index-change", &hook_opt);
> > +	run_hooks_opt_clear(&hook_opt);
> 
> There is one early return before the precontext of this hunk that
> bypasses this opt_clear() call.  It is before any member of hook_opt
> structure that was opt_init()'ed gets touched, so with the current
> code, there is no leak, but it probably is laying a landmine for the
> future, where opt_init() may allocate some resource to its member,
> with the expectation that all users of the API would call
> opt_clear() to release.  Or the caller of the API (like this one) may
> start mucking with the opt structure before the existing early return,
> at which point the current assumption that it is safe to return from
> that point without opt_clear() would be broken.
> 
> I saw that there are other early returns in the series that are safe
> right now but may become unsafe when the API implementation gets
> extended that way.  If it does not involve too much code churning,
> we may want to restructure the code to make these early returns into
> "goto"s that jump to a single exit point, so that we can always
> match opt_init() with opt_clear(), like the structure of the
> existing code allowed cmd_rebase() to use the hooks API cleanly in
> [v8 22/37].

OK. I'll audit this second half of the series looking for this type of
thing and try to clean up/use gotos if appropriate/etc. Thanks for
pointing it out.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 24/37] receive-pack: convert push-to-checkout hook to hook.h
  2021-03-12 10:24   ` Junio C Hamano
@ 2021-03-29 23:59     ` Emily Shaffer
  2021-03-30  0:10       ` Junio C Hamano
  0 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-03-29 23:59 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Fri, Mar 12, 2021 at 02:24:41AM -0800, Junio C Hamano wrote:
> 
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > @@ -1435,12 +1436,19 @@ static const char *push_to_checkout(unsigned char *hash,
> >  				    struct strvec *env,
> >  				    const char *work_tree)
> >  {
> > +	struct run_hooks_opt opt;
> > +	run_hooks_opt_init_sync(&opt);
> > +
> >  	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
> > -	if (run_hook_le(env->v, push_to_checkout_hook,
> > -			hash_to_hex(hash), NULL))
> > +	strvec_pushv(&opt.env, env->v);
> > +	strvec_push(&opt.args, hash_to_hex(hash));
> > +	if (run_hooks(push_to_checkout_hook, &opt)) {
> > +		run_hooks_opt_clear(&opt);
> >  		return "push-to-checkout hook declined";
> > -	else
> > +	} else {
> > +		run_hooks_opt_clear(&opt);
> >  		return NULL;
> > +	}
> >  }
> 
> OK, we opt_init(), futz with opt, call run_hooks() and opt_clear()
> regardless of the outcome from run_hooks().  Narrow-sighted me
> wonders if it makes the use of the API easier if run_hooks() did the
> opt_clear() before it returns, but I haven't yet seen enough use at
> this point to judge.

Hrm, is that idiomatic? I guess it would be convenient, and as long as
it doesn't touch explicitly caller-managed context pointer it should be
safe, but wouldn't it be surprising?

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 32/37] post-update: use hook.h library
  2021-03-12  9:14   ` Ævar Arnfjörð Bjarmason
@ 2021-03-30  0:01     ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-30  0:01 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Fri, Mar 12, 2021 at 10:14:31AM +0100, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Thu, Mar 11 2021, Emily Shaffer wrote:
> 
> > By using run_hooks() instead of run_hook_le(), 'post-update' hooks can
> > be specified in the config as well as the hookdir.
> 
> Looking ahead in the series no tests for this, seems like a good thing
> to have some at least trivial tests for each hook and their config
> invocation.

I'll look through the series and make sure the hooks being converted
have at least some test to make sure they worked; I think I checked that
for some of the early ones but got lazy :) I'll try and add some
config-specified tests too. Thanks, it's on my todo list.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 35/37] git-send-email: use 'git hook run' for 'sendemail-validate'
  2021-03-12  9:21   ` Ævar Arnfjörð Bjarmason
@ 2021-03-30  0:03     ` Emily Shaffer
  2021-03-31 21:47     ` Emily Shaffer
  1 sibling, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-30  0:03 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Fri, Mar 12, 2021 at 10:21:08AM +0100, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Thu, Mar 11 2021, Emily Shaffer wrote:
> 
> > By using the new 'git hook run' subcommand to run 'sendemail-validate',
> > we can reduce the boilerplate needed to run this hook in perl. Using
> > config-based hooks also allows us to run 'sendemail-validate' hooks that
> > were configured globally when running 'git send-email' from outside of a
> > Git directory, alongside other benefits like multihooks and
> > parallelization.
> >
> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > ---
> >  git-send-email.perl   | 21 ++++-----------------
> >  t/t9001-send-email.sh | 11 +----------
> >  2 files changed, 5 insertions(+), 27 deletions(-)
> >
> > diff --git a/git-send-email.perl b/git-send-email.perl
> > index 1f425c0809..73e1e0b51a 100755
> > --- a/git-send-email.perl
> > +++ b/git-send-email.perl
> > @@ -1941,23 +1941,10 @@ sub unique_email_list {
> >  sub validate_patch {
> >  	my ($fn, $xfer_encoding) = @_;
> >  
> > -	if ($repo) {
> > -		my $validate_hook = catfile(catdir($repo->repo_path(), 'hooks'),
> > -					    'sendemail-validate');
> > -		my $hook_error;
> > -		if (-x $validate_hook) {
> > -			my $target = abs_path($fn);
> > -			# The hook needs a correct cwd and GIT_DIR.
> > -			my $cwd_save = cwd();
> > -			chdir($repo->wc_path() or $repo->repo_path())
> > -				or die("chdir: $!");
> > -			local $ENV{"GIT_DIR"} = $repo->repo_path();
> > -			$hook_error = "rejected by sendemail-validate hook"
> > -				if system($validate_hook, $target);
> > -			chdir($cwd_save) or die("chdir: $!");
> > -		}
> > -		return $hook_error if $hook_error;
> > -	}
> > +	my $target = abs_path($fn);
> > +	return "rejected by sendemail-validate hook"
> > +		if system(("git", "hook", "run", "sendemail-validate", "-a",
> > +				$target));
> 
> I see it's just moving code around, but since we're touching this:
> 
> This conflates the hook exit code with a general failure to invoke it,
> Perl's system().
> 
> Not a big deal in this case, but there's two other existing system()
> invocations which use the right blurb for it:
> 
> 
> 	system('sh', '-c', $editor.' "$@"', $editor, $_);
> 	if (($? & 127) || ($? >> 8)) {
> 		die(__("the editor exited uncleanly, aborting everything"));
> 	}
> 
> Makes sense to do something similar here for consistency. See "perldoc
> -f system" for an example.

Oh cool, thanks. I'll do that.

> 
> >  
> >  	# Any long lines will be automatically fixed if we use a suitable transfer
> >  	# encoding.
> > diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
> > index 4eee9c3dcb..456b471c5c 100755
> > --- a/t/t9001-send-email.sh
> > +++ b/t/t9001-send-email.sh
> > @@ -2101,16 +2101,7 @@ test_expect_success $PREREQ 'invoke hook' '
> >  	mkdir -p .git/hooks &&
> >  
> >  	write_script .git/hooks/sendemail-validate <<-\EOF &&
> > -	# test that we have the correct environment variable, pwd, and
> > -	# argument
> > -	case "$GIT_DIR" in
> > -	*.git)
> > -		true
> > -		;;
> > -	*)
> > -		false
> > -		;;
> > -	esac &&
> > +	# test that we have the correct argument
> 
> This and getting rid of these Perl/Python/whatever special cases is very
> nice.

I thought so too :D :D

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 36/37] run-command: stop thinking about hooks
  2021-03-12  9:23   ` Ævar Arnfjörð Bjarmason
@ 2021-03-30  0:07     ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-30  0:07 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Fri, Mar 12, 2021 at 10:23:55AM +0100, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Thu, Mar 11 2021, Emily Shaffer wrote:
> 
> > hook.h has replaced all run-command.h hook-related functionality.
> > run-command.h:run_hooks_le/ve and find_hook are no longer used anywhere
> > in the codebase. So, let's delete the dead code - or, in the one case
> > where it's still needed, move it to an internal function in hook.c.
> 
> Similar to other comments about squashing, I think just having this
> happen incrementally as we remove whatever is the last user of the
> function would be better.
> 
> E.g. find_hook() is last used in one commit, run_hook*() in another...

Hm. I could see it, coupled with a blurb like, "Nobody is using this
anymore so delete."

But it feels odd to move the find_hook() impl from here to hook.c
internal in a commit about, say, bugreport.

I'll consider this, thanks. Maybe it fits in one case (like run_hook_*)
better than in another (like find_hook). I'll play with it :)

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 24/37] receive-pack: convert push-to-checkout hook to hook.h
  2021-03-29 23:59     ` Emily Shaffer
@ 2021-03-30  0:10       ` Junio C Hamano
  0 siblings, 0 replies; 479+ messages in thread
From: Junio C Hamano @ 2021-03-30  0:10 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

>> OK, we opt_init(), futz with opt, call run_hooks() and opt_clear()
>> regardless of the outcome from run_hooks().  Narrow-sighted me
>> wonders if it makes the use of the API easier if run_hooks() did the
>> opt_clear() before it returns, but I haven't yet seen enough use at
>> this point to judge.
>
> Hrm, is that idiomatic? I guess it would be convenient, and as long as
> it doesn't touch explicitly caller-managed context pointer it should be
> safe, but wouldn't it be surprising?

The precedent (at this point, I will not judge if it is a good
pattern to emulate or an anti-pattern to stay away from) I had in
mind was the run_command() which clears child_process structure
as the side effect of internally calling finish_command().

Leaving them separate is of course more flexible, but depending on
how small we can keep down the number of call patterns of this new
API, always having to clear after run might become an unnecessary
source of leaks.  When I gave that comment, I didn't have enough
input to decide, and now it has been so long since I gave my
reviews, I do not quite remember what my impression after reading
all the patches through was X-<.



^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 37/37] docs: unify githooks and git-hook manpages
  2021-03-12  9:29   ` Ævar Arnfjörð Bjarmason
@ 2021-03-30  0:10     ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-30  0:10 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Fri, Mar 12, 2021 at 10:29:52AM +0100, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Thu, Mar 11 2021, Emily Shaffer wrote:
> 
> > By showing the list of all hooks in 'git help hook' for users to refer
> > to, 'git help hook' becomes a one-stop shop for hook authorship. Since
> > some may still have muscle memory for 'git help githooks', though,
> > reference the 'git hook' commands and otherwise don't remove content.
> 
> I think this should at least have something like what my b6a8d09f6d8 (gc
> docs: include the "gc.*" section from "config" in "gc", 2019-04-07) has
> on top, i.e.:

Yeah, this seems reasonable.

>     
>     diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
>     index 4ad31ac360a..5c9af30b43e 100644
>     --- a/Documentation/git-hook.txt
>     +++ b/Documentation/git-hook.txt
>     @@ -150,10 +150,18 @@ message body and cannot be parallelized.
>      
>      CONFIGURATION
>      -------------
>     +
>     +The below documentation is the same as what's found in
>     +linkgit:git-config[1]:
>     +
>      include::config/hook.txt[]
>      
>      HOOKS
>      -----
>     +
>     +The below documentation is the same as what's found in
>     +linkgit:githooks[5]:
>     +
>      include::native-hooks.txt[]
>      
>      GIT
> 
> But I also don't think we should demote githooks(5) as the canonical doc
> page for the hooks themselves.
> 
> If you run this in your terminal:
> 
>     man 5 git<TAB>
> 
> You'll get:
> 
>     gitattributes         gitignore             gitmailmap            gitrepository-layout  
>     githooks              git-lfs-config        gitmodules            gitweb.conf 
> 
> (Well, maybe not the lfs-part, but whatever...).
> 
> We should move more in the direction of splitting up our "file format"
> docs from implementation, like the git-hook runner.
> 
> I'm somewhat negative on including it at all in git-hook(1). For the
> config section it makes sense, and it's consistent with established doc
> convention.
> 
> But including githooks(5) is around 2/3 of the resulting manpage, I
> think just a link is better.

Maybe so. What I really would like would be if `git help githooks` //
`man githooks` opened `git-hook` manpage, but I had trouble getting it to do
that and still publish to the `githooks` manpage (because the command
doc format doesn't match the guide format). (Or, really, if `git help
githooks` didn't exist so we didn't need to split the docs up. But that
ship has sailed.)

Regardless, I won't complain that much about using a link instead. I'll
make this change for v9.

 - Emily


^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 35/37] git-send-email: use 'git hook run' for 'sendemail-validate'
  2021-03-12  9:21   ` Ævar Arnfjörð Bjarmason
  2021-03-30  0:03     ` Emily Shaffer
@ 2021-03-31 21:47     ` Emily Shaffer
  2021-03-31 22:06       ` Junio C Hamano
  2021-04-02 11:34       ` [PATCH 0/2] git-send-email: refactor duplicate $? checks into a function Ævar Arnfjörð Bjarmason
  1 sibling, 2 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-03-31 21:47 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Fri, Mar 12, 2021 at 10:21:08AM +0100, Ævar Arnfjörð Bjarmason wrote:
> > +	my $target = abs_path($fn);
> > +	return "rejected by sendemail-validate hook"
> > +		if system(("git", "hook", "run", "sendemail-validate", "-a",
> > +				$target));
> 
> I see it's just moving code around, but since we're touching this:
> 
> This conflates the hook exit code with a general failure to invoke it,
> Perl's system().

Ah, at first I thought you meant "hook exit code vs. failure in 'git
hook run'" - but I think you are saying "system() can also exit
unhappily".

I had a look in 'perldoc -f system' like you suggested and saw that in
addition to $? & 127, it seems like I also should check $? == -1
("system() couldn't start the child process") and ($? >> 8) (the rc
from the child hangs out in the top byte). So then it seems like I want
something like so:

  system("git", "hook", "run", "sendemail-validate",
          "-j1", "-a", $target);

  return "git-send-email failed to launch hook process: $!"
          if ($? == -1) || ($? & 127))
  return "git-send-email invoked git-hook run incorrectly"
          if (($? >> 8) == 129);
  return "Rejected by 'sendemail-validate' hook"
          if ($? >> 8);

That seems really verbose, though. I guess ($? >> 8) includes -1 as well (since
0xFF... will meet that conditional), but do we care about the difference between
"system() couldn't run my thing" and "my thing returned upset"?

In this case, "my thing returned upset" - that is, $? >> 8 reflects an
error code from the hook exec - should already have some user output
associated with it, from the hook exec itself, but it's not guaranteed -
neither builtin/hook.c:run() nor hook.c:run_hooks() prints anything to
the user if rc != 0, because they're counting on either the hook execs
or the process that invoked the hook to do the tattling.

I think that means that it's a good idea to differentiate all these
things to the user:

 1. system() broke or your hook got a SIGINT (write a bug report or take
    out the infinite loop/memory violation from the hook you're
    developing)
 2. builtin/hook.c:run() wasn't invoked properly (fix the change you made
    to git-send-email.perl)
 3. your hook rejected your email (working as intended, fix the file you
    want to email)

I'd not expect users to encounter (1) or (2) so it seems fine to me to
include them; if (3) isn't present *and* the hook author did a bad job
communicating what failed, then I think the user experience would be
very confusing - even though they'd see some warning telling them their
patches didn't send, it wouldn't be clear whether it's because of an
issue in git-send-email or an issue with their patch.

Phew. I think I convinced myself that the wordy rc checking is OK. But I
am a perl noob so please correct me if I am wrong :)

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 35/37] git-send-email: use 'git hook run' for 'sendemail-validate'
  2021-03-31 21:47     ` Emily Shaffer
@ 2021-03-31 22:06       ` Junio C Hamano
  2021-04-01 18:08         ` Emily Shaffer
  2021-04-02 11:34       ` [PATCH 0/2] git-send-email: refactor duplicate $? checks into a function Ævar Arnfjörð Bjarmason
  1 sibling, 1 reply; 479+ messages in thread
From: Junio C Hamano @ 2021-03-31 22:06 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Ævar Arnfjörð Bjarmason, git

Emily Shaffer <emilyshaffer@google.com> writes:

> On Fri, Mar 12, 2021 at 10:21:08AM +0100, Ævar Arnfjörð Bjarmason wrote:
>> > +	my $target = abs_path($fn);
>> > +	return "rejected by sendemail-validate hook"
>> > +		if system(("git", "hook", "run", "sendemail-validate", "-a",
>> > +				$target));
>> 
>> I see it's just moving code around, but since we're touching this:
>> 
>> This conflates the hook exit code with a general failure to invoke it,
>> Perl's system().
>
> Ah, at first I thought you meant "hook exit code vs. failure in 'git
> hook run'" - but I think you are saying "system() can also exit
> unhappily".
>
> I had a look in 'perldoc -f system' like you suggested and saw that in
> addition to $? & 127, it seems like I also should check $? == -1
> ("system() couldn't start the child process") and ($? >> 8) (the rc
> from the child hangs out in the top byte). So then it seems like I want
> something like so:
>
>   system("git", "hook", "run", "sendemail-validate",
>           "-j1", "-a", $target);
>
>   return "git-send-email failed to launch hook process: $!"
>           if ($? == -1) || ($? & 127))
>   return "git-send-email invoked git-hook run incorrectly"
>           if (($? >> 8) == 129);
>   return "Rejected by 'sendemail-validate' hook"
>           if ($? >> 8);
>

The example in "perldoc -f system" distinguishes these two like so:

        if ($? == -1) {
                print "failed to execute: $!\n";
        }
        elsif ($? & 127) {
                printf "child died with signal %d, %s coredump\n",
                    ($? & 127), ($? & 128) ? 'with' : 'without';
        }
        else {
                printf "child exited with value %d\n", $? >> 8;
        }

> That seems really verbose, though. I guess ($? >> 8) includes -1 as well (since
> 0xFF... will meet that conditional), but do we care about the difference between
> "system() couldn't run my thing" and "my thing returned upset"?

If we classify the failure cases into three using the sample code in
the doc, I think the last one is the only case that we know the
logic in the hook is making a decision for us.  In the first case,
the hook did not even have a chance to decide for us, and in the
second case, the hook died with signal, most likely before it had a
chance to make a decision.  If we want to be conservative (sending
a message out is something you cannot easily undo), then it may make
sense to take the first two failure cases, even though the hook may
have said it is OK to send it out if it ran successfully, as a denial
to be safe, I would think.

Thanks.


^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 35/37] git-send-email: use 'git hook run' for 'sendemail-validate'
  2021-03-31 22:06       ` Junio C Hamano
@ 2021-04-01 18:08         ` Emily Shaffer
  2021-04-01 18:55           ` Junio C Hamano
  0 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-04-01 18:08 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Ævar Arnfjörð Bjarmason, git

On Wed, Mar 31, 2021 at 03:06:12PM -0700, Junio C Hamano wrote:
> 
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > On Fri, Mar 12, 2021 at 10:21:08AM +0100, Ævar Arnfjörð Bjarmason wrote:
> >> > +	my $target = abs_path($fn);
> >> > +	return "rejected by sendemail-validate hook"
> >> > +		if system(("git", "hook", "run", "sendemail-validate", "-a",
> >> > +				$target));
> >> 
> >> I see it's just moving code around, but since we're touching this:
> >> 
> >> This conflates the hook exit code with a general failure to invoke it,
> >> Perl's system().
> >
> > Ah, at first I thought you meant "hook exit code vs. failure in 'git
> > hook run'" - but I think you are saying "system() can also exit
> > unhappily".
> >
> > I had a look in 'perldoc -f system' like you suggested and saw that in
> > addition to $? & 127, it seems like I also should check $? == -1
> > ("system() couldn't start the child process") and ($? >> 8) (the rc
> > from the child hangs out in the top byte). So then it seems like I want
> > something like so:
> >
> >   system("git", "hook", "run", "sendemail-validate",
> >           "-j1", "-a", $target);
> >
> >   return "git-send-email failed to launch hook process: $!"
> >           if ($? == -1) || ($? & 127))
> >   return "git-send-email invoked git-hook run incorrectly"
> >           if (($? >> 8) == 129);
> >   return "Rejected by 'sendemail-validate' hook"
> >           if ($? >> 8);
> >
> 
> The example in "perldoc -f system" distinguishes these two like so:
> 
>         if ($? == -1) {
>                 print "failed to execute: $!\n";
>         }
>         elsif ($? & 127) {
>                 printf "child died with signal %d, %s coredump\n",
>                     ($? & 127), ($? & 128) ? 'with' : 'without';
>         }
>         else {
>                 printf "child exited with value %d\n", $? >> 8;
>         }
> 
> > That seems really verbose, though. I guess ($? >> 8) includes -1 as well (since
> > 0xFF... will meet that conditional), but do we care about the difference between
> > "system() couldn't run my thing" and "my thing returned upset"?
> 
> If we classify the failure cases into three using the sample code in
> the doc, I think the last one is the only case that we know the
> logic in the hook is making a decision for us.  In the first case,
> the hook did not even have a chance to decide for us, and in the
> second case, the hook died with signal, most likely before it had a
> chance to make a decision.  If we want to be conservative (sending
> a message out is something you cannot easily undo), then it may make
> sense to take the first two failure cases, even though the hook may
> have said it is OK to send it out if it ran successfully, as a denial
> to be safe, I would think.

Yeah, I tend to agree. In that case I think you are saying: "Please
split the first case into two and differentiate launch failure from
signal, but otherwise continue to return all these cases as errors and
halt the email."

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 35/37] git-send-email: use 'git hook run' for 'sendemail-validate'
  2021-04-01 18:08         ` Emily Shaffer
@ 2021-04-01 18:55           ` Junio C Hamano
  0 siblings, 0 replies; 479+ messages in thread
From: Junio C Hamano @ 2021-04-01 18:55 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Ævar Arnfjörð Bjarmason, git

Emily Shaffer <emilyshaffer@google.com> writes:

>> If we classify the failure cases into three using the sample code in
>> the doc, I think the last one is the only case that we know the
>> logic in the hook is making a decision for us.  In the first case,
>> the hook did not even have a chance to decide for us, and in the
>> second case, the hook died with signal, most likely before it had a
>> chance to make a decision.  If we want to be conservative (sending
>> a message out is something you cannot easily undo), then it may make
>> sense to take the first two failure cases, even though the hook may
>> have said it is OK to send it out if it ran successfully, as a denial
>> to be safe, I would think.
>
> Yeah, I tend to agree. In that case I think you are saying: "Please
> split the first case into two and differentiate launch failure from
> signal, but otherwise continue to return all these cases as errors and
> halt the email."

Not exactly.  I do not have a strong opinion either way to split the
first two cases apart or lump them together.  If I were pressed, I
probably would vote for the latter.

The doc's example classfies into three and I think that
classification is logical:

 * Lumping the first two together would make sense with respect to
   deciding what to do when we see a failure. The first two are
   "hook failed to approve or disapprove" case, while the last one
   is "the hook actively disapproved".  The former is not under
   hook's control.

 * Further, treating a failure even from the first "hook failed to
   approve or disapprove" as a signal to stop sending would be more
   conservative.

 * Which leads us to say, with respect to deciding what to do, any
   failure just stops the program from sending.

It is a separate matter how to phrase the diagnoses and hints for
recovery.  It could be that sendmail-validate hook failed to run due
to a simple misconfiguration (e.g. because it lacked the executable
bit).  Giving an error message with strerr would be helpful for the
"hook failed to approve or disapprove" case.


^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH 0/2] git-send-email: refactor duplicate $? checks into a function
  2021-03-31 21:47     ` Emily Shaffer
  2021-03-31 22:06       ` Junio C Hamano
@ 2021-04-02 11:34       ` Ævar Arnfjörð Bjarmason
  2021-04-02 11:34         ` [PATCH 1/2] git-send-email: replace "map" in void context with "for" Ævar Arnfjörð Bjarmason
                           ` (2 more replies)
  1 sibling, 3 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-04-02 11:34 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Emily Shaffer, Ævar Arnfjörð Bjarmason

A tiny series to help along the config-based hooks series[1]. Its
patch dealing with git-send-email.perl can now trivially be based on
top of this instead of adding another system() wrapper to
git-send-email.perl.

As alluded to in the TODO comment in 2/2 it's probably best to fix
things while we're at it to call validate_patch_error() instead of
just emitting the more brief "rejected by sendemail-validate hook".

But for now this series is just aiming for bug-for-bug compatibility
with the existing code, and to just reduce code duplication.

http://lore.kernel.org/git/20210311021037.3001235-1-emilyshaffer@google.com

Ævar Arnfjörð Bjarmason (2):
  git-send-email: replace "map" in void context with "for"
  git-send-email: refactor duplicate $? checks into a function

 git-send-email.perl | 49 +++++++++++++++++++++++++++++----------------
 1 file changed, 32 insertions(+), 17 deletions(-)

-- 
2.31.1.482.g6691c1be520


^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH 1/2] git-send-email: replace "map" in void context with "for"
  2021-04-02 11:34       ` [PATCH 0/2] git-send-email: refactor duplicate $? checks into a function Ævar Arnfjörð Bjarmason
@ 2021-04-02 11:34         ` Ævar Arnfjörð Bjarmason
  2021-04-02 21:31           ` Junio C Hamano
  2021-04-02 11:34         ` [PATCH 2/2] git-send-email: refactor duplicate $? checks into a function Ævar Arnfjörð Bjarmason
  2021-04-04  9:19         ` [PATCH v2 0/4] refactor duplicate $? checks into a function + improve errors Ævar Arnfjörð Bjarmason
  2 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-04-02 11:34 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Emily Shaffer, Ævar Arnfjörð Bjarmason

While using "map" instead of "for" or "map" instead of "grep" and
vice-versa makes for interesting trivia questions when interviewing
Perl programmers, it doesn't make for very readable code. Let's
refactor this loop initially added in 8fd5bb7f44b (git send-email: add
--annotate option, 2008-11-11) to be a for-loop instead.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-send-email.perl | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/git-send-email.perl b/git-send-email.perl
index f5bbf1647e3..6893c8e5808 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -217,12 +217,12 @@ sub do_edit {
 		$editor = Git::command_oneline('var', 'GIT_EDITOR');
 	}
 	if (defined($multiedit) && !$multiedit) {
-		map {
+		for (@_) {
 			system('sh', '-c', $editor.' "$@"', $editor, $_);
 			if (($? & 127) || ($? >> 8)) {
 				die(__("the editor exited uncleanly, aborting everything"));
 			}
-		} @_;
+		}
 	} else {
 		system('sh', '-c', $editor.' "$@"', $editor, @_);
 		if (($? & 127) || ($? >> 8)) {
-- 
2.31.1.482.g6691c1be520


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 2/2] git-send-email: refactor duplicate $? checks into a function
  2021-04-02 11:34       ` [PATCH 0/2] git-send-email: refactor duplicate $? checks into a function Ævar Arnfjörð Bjarmason
  2021-04-02 11:34         ` [PATCH 1/2] git-send-email: replace "map" in void context with "for" Ævar Arnfjörð Bjarmason
@ 2021-04-02 11:34         ` Ævar Arnfjörð Bjarmason
  2021-04-02 21:36           ` Junio C Hamano
  2021-04-04  9:19         ` [PATCH v2 0/4] refactor duplicate $? checks into a function + improve errors Ævar Arnfjörð Bjarmason
  2 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-04-02 11:34 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Emily Shaffer, Ævar Arnfjörð Bjarmason

Refactor the duplicate checking of $? into a function. There's an
outstanding series[1] wanting to add a third use of system() in this
file, let's not copy this boilerplate anymore when that happens.

1. http://lore.kernel.org/git/87y2esg22j.fsf@evledraar.gmail.com

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-send-email.perl | 49 +++++++++++++++++++++++++++++----------------
 1 file changed, 32 insertions(+), 17 deletions(-)

diff --git a/git-send-email.perl b/git-send-email.perl
index 6893c8e5808..901c935455d 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -212,22 +212,30 @@ sub format_2822_time {
 my $multiedit;
 my $editor;
 
+sub system_or_msg {
+	my ($args, $msg) = @_;
+	system(@$args);
+	return unless (($? & 127) || ($? >> 8));
+
+	die $msg if $msg;
+	return sprintf(__("failed to run command %s, died with code %d"),
+		       "@$args", $? >> 8);
+}
+
+sub system_or_die {
+	my $msg = system_or_msg(@_);
+	die $msg if $msg;
+}
+
 sub do_edit {
 	if (!defined($editor)) {
 		$editor = Git::command_oneline('var', 'GIT_EDITOR');
 	}
+	my $die_msg = __("the editor exited uncleanly, aborting everything");
 	if (defined($multiedit) && !$multiedit) {
-		for (@_) {
-			system('sh', '-c', $editor.' "$@"', $editor, $_);
-			if (($? & 127) || ($? >> 8)) {
-				die(__("the editor exited uncleanly, aborting everything"));
-			}
-		}
+		system_or_die(['sh', '-c', $editor.' "$@"', $editor, $_], $die_msg) for @_;
 	} else {
-		system('sh', '-c', $editor.' "$@"', $editor, @_);
-		if (($? & 127) || ($? >> 8)) {
-			die(__("the editor exited uncleanly, aborting everything"));
-		}
+		system_or_die(['sh', '-c', $editor.' "$@"', $editor, @_], $die_msg);
 	}
 }
 
@@ -698,9 +706,7 @@ sub is_format_patch_arg {
 if ($validate) {
 	foreach my $f (@files) {
 		unless (-p $f) {
-			my $error = validate_patch($f, $target_xfer_encoding);
-			$error and die sprintf(__("fatal: %s: %s\nwarning: no patches were sent\n"),
-						  $f, $error);
+			validate_patch($f, $target_xfer_encoding);
 		}
 	}
 }
@@ -1938,6 +1944,12 @@ sub unique_email_list {
 	return @emails;
 }
 
+sub validate_patch_error {
+	my ($fn, $error) = @_;
+	die sprintf(__("fatal: %s: %s\nwarning: no patches were sent\n"),
+		    $fn, $error);
+}
+
 sub validate_patch {
 	my ($fn, $xfer_encoding) = @_;
 
@@ -1952,11 +1964,14 @@ sub validate_patch {
 			chdir($repo->wc_path() or $repo->repo_path())
 				or die("chdir: $!");
 			local $ENV{"GIT_DIR"} = $repo->repo_path();
-			$hook_error = "rejected by sendemail-validate hook"
-				if system($validate_hook, $target);
+			if (my $msg = system_or_msg([$validate_hook, $target])) {
+				# TODO Use $msg and emit exit code on
+				# hook failures?
+				$hook_error = __("rejected by sendemail-validate hook");
+			}
 			chdir($cwd_save) or die("chdir: $!");
 		}
-		return $hook_error if $hook_error;
+		validate_patch_error($fn, $hook_error) if $hook_error;
 	}
 
 	# Any long lines will be automatically fixed if we use a suitable transfer
@@ -1966,7 +1981,7 @@ sub validate_patch {
 			or die sprintf(__("unable to open %s: %s\n"), $fn, $!);
 		while (my $line = <$fh>) {
 			if (length($line) > 998) {
-				return sprintf(__("%s: patch contains a line longer than 998 characters"), $.);
+				validate_patch_error($fn, sprintf(__("%s: patch contains a line longer than 998 characters"), $.));
 			}
 		}
 	}
-- 
2.31.1.482.g6691c1be520


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* Re: [PATCH 1/2] git-send-email: replace "map" in void context with "for"
  2021-04-02 11:34         ` [PATCH 1/2] git-send-email: replace "map" in void context with "for" Ævar Arnfjörð Bjarmason
@ 2021-04-02 21:31           ` Junio C Hamano
  2021-04-02 21:37             ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Junio C Hamano @ 2021-04-02 21:31 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git, Emily Shaffer

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> While using "map" instead of "for" or "map" instead of "grep" and
> vice-versa makes for interesting trivia questions when interviewing
> Perl programmers, it doesn't make for very readable code. Let's
> refactor this loop initially added in 8fd5bb7f44b (git send-email: add
> --annotate option, 2008-11-11) to be a for-loop instead.

;-)

Will queue.  Thanks.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 2/2] git-send-email: refactor duplicate $? checks into a function
  2021-04-02 11:34         ` [PATCH 2/2] git-send-email: refactor duplicate $? checks into a function Ævar Arnfjörð Bjarmason
@ 2021-04-02 21:36           ` Junio C Hamano
  0 siblings, 0 replies; 479+ messages in thread
From: Junio C Hamano @ 2021-04-02 21:36 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git, Emily Shaffer

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> Refactor the duplicate checking of $? into a function. There's an
> outstanding series[1] wanting to add a third use of system() in this
> file, let's not copy this boilerplate anymore when that happens.
>
> 1. http://lore.kernel.org/git/87y2esg22j.fsf@evledraar.gmail.com
>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>  git-send-email.perl | 49 +++++++++++++++++++++++++++++----------------
>  1 file changed, 32 insertions(+), 17 deletions(-)
>
> diff --git a/git-send-email.perl b/git-send-email.perl
> index 6893c8e5808..901c935455d 100755
> --- a/git-send-email.perl
> +++ b/git-send-email.perl
> @@ -212,22 +212,30 @@ sub format_2822_time {
>  my $multiedit;
>  my $editor;
>  
> +sub system_or_msg {
> +	my ($args, $msg) = @_;
> +	system(@$args);
> +	return unless (($? & 127) || ($? >> 8));
> +
> +	die $msg if $msg;
> +	return sprintf(__("failed to run command %s, died with code %d"),
> +		       "@$args", $? >> 8);
> +}

That sounds more like system_and_die_or_msg to me.  More
importantly, the name of the helper makes it clear what difference
this has with ...

> +sub system_or_die {
> +	my $msg = system_or_msg(@_);
> +	die $msg if $msg;
> +}

... this one.  The former does nto die but returns message only when
X?  If that X were in its name, readers who look at the caller of
system_or_msg vs system_or_die would immediately know that why the
callsite is using one and not the other variant.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 1/2] git-send-email: replace "map" in void context with "for"
  2021-04-02 21:31           ` Junio C Hamano
@ 2021-04-02 21:37             ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-04-02 21:37 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Ævar Arnfjörð Bjarmason, git

On Fri, Apr 02, 2021 at 02:31:38PM -0700, Junio C Hamano wrote:
> 
> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
> 
> > While using "map" instead of "for" or "map" instead of "grep" and
> > vice-versa makes for interesting trivia questions when interviewing
> > Perl programmers, it doesn't make for very readable code. Let's
> > refactor this loop initially added in 8fd5bb7f44b (git send-email: add
> > --annotate option, 2008-11-11) to be a for-loop instead.
> 
> ;-)
> 
> Will queue.  Thanks.

Oh cool, thanks both :)

^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH v2 0/4] refactor duplicate $? checks into a function + improve errors
  2021-04-02 11:34       ` [PATCH 0/2] git-send-email: refactor duplicate $? checks into a function Ævar Arnfjörð Bjarmason
  2021-04-02 11:34         ` [PATCH 1/2] git-send-email: replace "map" in void context with "for" Ævar Arnfjörð Bjarmason
  2021-04-02 11:34         ` [PATCH 2/2] git-send-email: refactor duplicate $? checks into a function Ævar Arnfjörð Bjarmason
@ 2021-04-04  9:19         ` Ævar Arnfjörð Bjarmason
  2021-04-04  9:19           ` [PATCH v2 1/4] git-send-email: replace "map" in void context with "for" Ævar Arnfjörð Bjarmason
                             ` (4 more replies)
  2 siblings, 5 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-04-04  9:19 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Emily Shaffer, Ævar Arnfjörð Bjarmason

There was a silly error in v1 noted by Junio. In practice we didn't
hit that "die $msg if $die" in system_or_msg(), but it didn't belong
there.

I see 1/2 of v1 of this series was merged to "next". I'm sending the
full thing anyway, but presumably just 3-4 will be picked up. They
apply cleanly on "next".

I added two patches at the end to improve the error output, the first
two patches in both v1 and v2 just reproduced the current output
bug-for-bug, but I've now made it more sensible.

Ævar Arnfjörð Bjarmason (4):
  git-send-email: replace "map" in void context with "for"
  git-send-email: refactor duplicate $? checks into a function
  git-send-email: test full --validate output
  git-send-email: improve --validate error output

 git-send-email.perl   | 45 +++++++++++++++++++++++++++----------------
 t/t9001-send-email.sh | 35 +++++++++++++++++++++++++--------
 2 files changed, 55 insertions(+), 25 deletions(-)

Range-diff:
1:  bea11504a67 = 1:  e37b861f239 git-send-email: replace "map" in void context with "for"
2:  f4bace5607c ! 2:  f236f083e36 git-send-email: refactor duplicate $? checks into a function
    @@ git-send-email.perl: sub format_2822_time {
     +sub system_or_msg {
     +	my ($args, $msg) = @_;
     +	system(@$args);
    -+	return unless (($? & 127) || ($? >> 8));
    ++	my $signalled = $? & 127;
    ++	my $exit_code = $? >> 8;
    ++	return unless $signalled or $exit_code;
     +
    -+	die $msg if $msg;
     +	return sprintf(__("failed to run command %s, died with code %d"),
    -+		       "@$args", $? >> 8);
    ++		       "@$args", $exit_code);
     +}
     +
     +sub system_or_die {
    @@ git-send-email.perl: sub validate_patch {
     -			$hook_error = "rejected by sendemail-validate hook"
     -				if system($validate_hook, $target);
     +			if (my $msg = system_or_msg([$validate_hook, $target])) {
    -+				# TODO Use $msg and emit exit code on
    -+				# hook failures?
     +				$hook_error = __("rejected by sendemail-validate hook");
     +			}
      			chdir($cwd_save) or die("chdir: $!");
-:  ----------- > 3:  15b59c226d4 git-send-email: test full --validate output
-:  ----------- > 4:  a1edceb4913 git-send-email: improve --validate error output
-- 
2.31.1.482.g6691c1be520


^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH v2 1/4] git-send-email: replace "map" in void context with "for"
  2021-04-04  9:19         ` [PATCH v2 0/4] refactor duplicate $? checks into a function + improve errors Ævar Arnfjörð Bjarmason
@ 2021-04-04  9:19           ` Ævar Arnfjörð Bjarmason
  2021-04-04  9:19           ` [PATCH v2 2/4] git-send-email: refactor duplicate $? checks into a function Ævar Arnfjörð Bjarmason
                             ` (3 subsequent siblings)
  4 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-04-04  9:19 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Emily Shaffer, Ævar Arnfjörð Bjarmason

While using "map" instead of "for" or "map" instead of "grep" and
vice-versa makes for interesting trivia questions when interviewing
Perl programmers, it doesn't make for very readable code. Let's
refactor this loop initially added in 8fd5bb7f44b (git send-email: add
--annotate option, 2008-11-11) to be a for-loop instead.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-send-email.perl | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/git-send-email.perl b/git-send-email.perl
index f5bbf1647e3..6893c8e5808 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -217,12 +217,12 @@ sub do_edit {
 		$editor = Git::command_oneline('var', 'GIT_EDITOR');
 	}
 	if (defined($multiedit) && !$multiedit) {
-		map {
+		for (@_) {
 			system('sh', '-c', $editor.' "$@"', $editor, $_);
 			if (($? & 127) || ($? >> 8)) {
 				die(__("the editor exited uncleanly, aborting everything"));
 			}
-		} @_;
+		}
 	} else {
 		system('sh', '-c', $editor.' "$@"', $editor, @_);
 		if (($? & 127) || ($? >> 8)) {
-- 
2.31.1.482.g6691c1be520


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 2/4] git-send-email: refactor duplicate $? checks into a function
  2021-04-04  9:19         ` [PATCH v2 0/4] refactor duplicate $? checks into a function + improve errors Ævar Arnfjörð Bjarmason
  2021-04-04  9:19           ` [PATCH v2 1/4] git-send-email: replace "map" in void context with "for" Ævar Arnfjörð Bjarmason
@ 2021-04-04  9:19           ` Ævar Arnfjörð Bjarmason
  2021-04-05 19:11             ` Junio C Hamano
  2021-04-05 23:47             ` Junio C Hamano
  2021-04-04  9:19           ` [PATCH v2 3/4] git-send-email: test full --validate output Ævar Arnfjörð Bjarmason
                             ` (2 subsequent siblings)
  4 siblings, 2 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-04-04  9:19 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Emily Shaffer, Ævar Arnfjörð Bjarmason

Refactor the duplicate checking of $? into a function. There's an
outstanding series[1] wanting to add a third use of system() in this
file, let's not copy this boilerplate anymore when that happens.

1. http://lore.kernel.org/git/87y2esg22j.fsf@evledraar.gmail.com

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-send-email.perl | 48 +++++++++++++++++++++++++++++----------------
 1 file changed, 31 insertions(+), 17 deletions(-)

diff --git a/git-send-email.perl b/git-send-email.perl
index 6893c8e5808..9724a9cae27 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -212,22 +212,31 @@ sub format_2822_time {
 my $multiedit;
 my $editor;
 
+sub system_or_msg {
+	my ($args, $msg) = @_;
+	system(@$args);
+	my $signalled = $? & 127;
+	my $exit_code = $? >> 8;
+	return unless $signalled or $exit_code;
+
+	return sprintf(__("failed to run command %s, died with code %d"),
+		       "@$args", $exit_code);
+}
+
+sub system_or_die {
+	my $msg = system_or_msg(@_);
+	die $msg if $msg;
+}
+
 sub do_edit {
 	if (!defined($editor)) {
 		$editor = Git::command_oneline('var', 'GIT_EDITOR');
 	}
+	my $die_msg = __("the editor exited uncleanly, aborting everything");
 	if (defined($multiedit) && !$multiedit) {
-		for (@_) {
-			system('sh', '-c', $editor.' "$@"', $editor, $_);
-			if (($? & 127) || ($? >> 8)) {
-				die(__("the editor exited uncleanly, aborting everything"));
-			}
-		}
+		system_or_die(['sh', '-c', $editor.' "$@"', $editor, $_], $die_msg) for @_;
 	} else {
-		system('sh', '-c', $editor.' "$@"', $editor, @_);
-		if (($? & 127) || ($? >> 8)) {
-			die(__("the editor exited uncleanly, aborting everything"));
-		}
+		system_or_die(['sh', '-c', $editor.' "$@"', $editor, @_], $die_msg);
 	}
 }
 
@@ -698,9 +707,7 @@ sub is_format_patch_arg {
 if ($validate) {
 	foreach my $f (@files) {
 		unless (-p $f) {
-			my $error = validate_patch($f, $target_xfer_encoding);
-			$error and die sprintf(__("fatal: %s: %s\nwarning: no patches were sent\n"),
-						  $f, $error);
+			validate_patch($f, $target_xfer_encoding);
 		}
 	}
 }
@@ -1938,6 +1945,12 @@ sub unique_email_list {
 	return @emails;
 }
 
+sub validate_patch_error {
+	my ($fn, $error) = @_;
+	die sprintf(__("fatal: %s: %s\nwarning: no patches were sent\n"),
+		    $fn, $error);
+}
+
 sub validate_patch {
 	my ($fn, $xfer_encoding) = @_;
 
@@ -1952,11 +1965,12 @@ sub validate_patch {
 			chdir($repo->wc_path() or $repo->repo_path())
 				or die("chdir: $!");
 			local $ENV{"GIT_DIR"} = $repo->repo_path();
-			$hook_error = "rejected by sendemail-validate hook"
-				if system($validate_hook, $target);
+			if (my $msg = system_or_msg([$validate_hook, $target])) {
+				$hook_error = __("rejected by sendemail-validate hook");
+			}
 			chdir($cwd_save) or die("chdir: $!");
 		}
-		return $hook_error if $hook_error;
+		validate_patch_error($fn, $hook_error) if $hook_error;
 	}
 
 	# Any long lines will be automatically fixed if we use a suitable transfer
@@ -1966,7 +1980,7 @@ sub validate_patch {
 			or die sprintf(__("unable to open %s: %s\n"), $fn, $!);
 		while (my $line = <$fh>) {
 			if (length($line) > 998) {
-				return sprintf(__("%s: patch contains a line longer than 998 characters"), $.);
+				validate_patch_error($fn, sprintf(__("%s: patch contains a line longer than 998 characters"), $.));
 			}
 		}
 	}
-- 
2.31.1.482.g6691c1be520


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 3/4] git-send-email: test full --validate output
  2021-04-04  9:19         ` [PATCH v2 0/4] refactor duplicate $? checks into a function + improve errors Ævar Arnfjörð Bjarmason
  2021-04-04  9:19           ` [PATCH v2 1/4] git-send-email: replace "map" in void context with "for" Ævar Arnfjörð Bjarmason
  2021-04-04  9:19           ` [PATCH v2 2/4] git-send-email: refactor duplicate $? checks into a function Ævar Arnfjörð Bjarmason
@ 2021-04-04  9:19           ` Ævar Arnfjörð Bjarmason
  2021-04-04  9:19           ` [PATCH v2 4/4] git-send-email: improve --validate error output Ævar Arnfjörð Bjarmason
  2021-04-06 14:00           ` [PATCH v3 0/3] refactor duplicate $? checks into a function + improve errors Ævar Arnfjörð Bjarmason
  4 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-04-04  9:19 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Emily Shaffer, Ævar Arnfjörð Bjarmason

Change the tests that grep substrings out of the output to use a full
test_cmp, in preparation for improving the output.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t9001-send-email.sh | 24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)

diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index 1a1caf8f2ed..74225e3dc7a 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -422,8 +422,12 @@ test_expect_success $PREREQ 'reject long lines' '
 		--smtp-server="$(pwd)/fake.sendmail" \
 		--transfer-encoding=8bit \
 		$patches longline.patch \
-		2>errors &&
-	grep longline.patch errors
+		2>actual &&
+	cat >expect <<-\EOF &&
+	fatal: longline.patch: 35: patch contains a line longer than 998 characters
+	warning: no patches were sent
+	EOF
+	test_cmp expect actual
 '
 
 test_expect_success $PREREQ 'no patch was sent' '
@@ -527,9 +531,13 @@ test_expect_success $PREREQ "--validate respects relative core.hooksPath path" '
 		--to=nobody@example.com \
 		--smtp-server="$(pwd)/fake.sendmail" \
 		--validate \
-		longline.patch 2>err &&
+		longline.patch 2>actual &&
 	test_path_is_file my-hooks.ran &&
-	grep "rejected by sendemail-validate" err
+	cat >expect <<-\EOF &&
+	fatal: longline.patch: rejected by sendemail-validate hook
+	warning: no patches were sent
+	EOF
+	test_cmp expect actual
 '
 
 test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" '
@@ -540,9 +548,13 @@ test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" '
 		--to=nobody@example.com \
 		--smtp-server="$(pwd)/fake.sendmail" \
 		--validate \
-		longline.patch 2>err &&
+		longline.patch 2>actual &&
 	test_path_is_file my-hooks.ran &&
-	grep "rejected by sendemail-validate" err
+	cat >expect <<-\EOF &&
+	fatal: longline.patch: rejected by sendemail-validate hook
+	warning: no patches were sent
+	EOF
+	test_cmp expect actual
 '
 
 for enc in 7bit 8bit quoted-printable base64
-- 
2.31.1.482.g6691c1be520


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 4/4] git-send-email: improve --validate error output
  2021-04-04  9:19         ` [PATCH v2 0/4] refactor duplicate $? checks into a function + improve errors Ævar Arnfjörð Bjarmason
                             ` (2 preceding siblings ...)
  2021-04-04  9:19           ` [PATCH v2 3/4] git-send-email: test full --validate output Ævar Arnfjörð Bjarmason
@ 2021-04-04  9:19           ` Ævar Arnfjörð Bjarmason
  2021-04-05 19:14             ` Junio C Hamano
  2021-04-06 14:00           ` [PATCH v3 0/3] refactor duplicate $? checks into a function + improve errors Ævar Arnfjörð Bjarmason
  4 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-04-04  9:19 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Emily Shaffer, Ævar Arnfjörð Bjarmason

Improve the output we emit on --validate error to:

 * Say "FILE:LINE" instead of "FILE: LINE".

 * Don't say "patch contains a" after just mentioning the filename,
   just leave it at "FILE:LINE: is longer than[...]. The "contains a"
   sounded like we were talking about the file in general, when we're
   actually checking it line-by-line.

 * Don't just say "rejected by sendemail-validate hook", but combine
   that with the system_or_msg() output to say what exit code the hook
   died with.

I had an aborted attempt to make the line length checker note all
lines that were longer than the limit. I didn't think that was worth
the effort, but I've left in the testing change to check that we die
as soon as we spot the first long line.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-send-email.perl   | 23 ++++++++++-------------
 t/t9001-send-email.sh | 17 ++++++++++++-----
 2 files changed, 22 insertions(+), 18 deletions(-)

diff --git a/git-send-email.perl b/git-send-email.perl
index 9724a9cae27..175da07d946 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -219,8 +219,8 @@ sub system_or_msg {
 	my $exit_code = $? >> 8;
 	return unless $signalled or $exit_code;
 
-	return sprintf(__("failed to run command %s, died with code %d"),
-		       "@$args", $exit_code);
+	return sprintf(__("fatal: command '%s' died with exit code %d"),
+		       $args->[0], $exit_code);
 }
 
 sub system_or_die {
@@ -1945,12 +1945,6 @@ sub unique_email_list {
 	return @emails;
 }
 
-sub validate_patch_error {
-	my ($fn, $error) = @_;
-	die sprintf(__("fatal: %s: %s\nwarning: no patches were sent\n"),
-		    $fn, $error);
-}
-
 sub validate_patch {
 	my ($fn, $xfer_encoding) = @_;
 
@@ -1965,12 +1959,14 @@ sub validate_patch {
 			chdir($repo->wc_path() or $repo->repo_path())
 				or die("chdir: $!");
 			local $ENV{"GIT_DIR"} = $repo->repo_path();
-			if (my $msg = system_or_msg([$validate_hook, $target])) {
-				$hook_error = __("rejected by sendemail-validate hook");
-			}
+			$hook_error = system_or_msg([$validate_hook, $target]);
 			chdir($cwd_save) or die("chdir: $!");
 		}
-		validate_patch_error($fn, $hook_error) if $hook_error;
+		if ($hook_error) {
+			die sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
+				       "%s\n" .
+				       "warning: no patches were sent\n"), $fn, $hook_error);
+		}
 	}
 
 	# Any long lines will be automatically fixed if we use a suitable transfer
@@ -1980,7 +1976,8 @@ sub validate_patch {
 			or die sprintf(__("unable to open %s: %s\n"), $fn, $!);
 		while (my $line = <$fh>) {
 			if (length($line) > 998) {
-				validate_patch_error($fn, sprintf(__("%s: patch contains a line longer than 998 characters"), $.));
+				die sprintf(__("fatal: %s:%d is longer than 998 characters\n" .
+					       "warning: no patches were sent\n"), $fn, $.);
 			}
 		}
 	}
diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index 74225e3dc7a..65b30353719 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -415,7 +415,11 @@ test_expect_success $PREREQ 'reject long lines' '
 	z512=$z64$z64$z64$z64$z64$z64$z64$z64 &&
 	clean_fake_sendmail &&
 	cp $patches longline.patch &&
-	echo $z512$z512 >>longline.patch &&
+	cat >>longline.patch <<-EOF &&
+	$z512$z512
+	not a long line
+	$z512$z512
+	EOF
 	test_must_fail git send-email \
 		--from="Example <nobody@example.com>" \
 		--to=nobody@example.com \
@@ -424,7 +428,7 @@ test_expect_success $PREREQ 'reject long lines' '
 		$patches longline.patch \
 		2>actual &&
 	cat >expect <<-\EOF &&
-	fatal: longline.patch: 35: patch contains a line longer than 998 characters
+	fatal: longline.patch:35 is longer than 998 characters
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
@@ -533,15 +537,17 @@ test_expect_success $PREREQ "--validate respects relative core.hooksPath path" '
 		--validate \
 		longline.patch 2>actual &&
 	test_path_is_file my-hooks.ran &&
-	cat >expect <<-\EOF &&
+	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
+	fatal: command '"'"'$(pwd)/my-hooks/sendemail-validate'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
 '
 
 test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" '
-	test_config core.hooksPath "$(pwd)/my-hooks" &&
+	hooks_path="$(pwd)/my-hooks" &&
+	test_config core.hooksPath "$hooks_path" &&
 	test_when_finished "rm my-hooks.ran" &&
 	test_must_fail git send-email \
 		--from="Example <nobody@example.com>" \
@@ -550,8 +556,9 @@ test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" '
 		--validate \
 		longline.patch 2>actual &&
 	test_path_is_file my-hooks.ran &&
-	cat >expect <<-\EOF &&
+	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
+	fatal: command '"'"'$hooks_path/sendemail-validate'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
-- 
2.31.1.482.g6691c1be520


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 2/4] git-send-email: refactor duplicate $? checks into a function
  2021-04-04  9:19           ` [PATCH v2 2/4] git-send-email: refactor duplicate $? checks into a function Ævar Arnfjörð Bjarmason
@ 2021-04-05 19:11             ` Junio C Hamano
  2021-04-05 23:47             ` Junio C Hamano
  1 sibling, 0 replies; 479+ messages in thread
From: Junio C Hamano @ 2021-04-05 19:11 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git, Emily Shaffer

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> @@ -1938,6 +1945,12 @@ sub unique_email_list {
>  	return @emails;
>  }
>  
> +sub validate_patch_error {
> +	my ($fn, $error) = @_;
> +	die sprintf(__("fatal: %s: %s\nwarning: no patches were sent\n"),
> +		    $fn, $error);
> +}
> +
>  sub validate_patch {
>  	my ($fn, $xfer_encoding) = @_;

I like the overall direction of this series, but this change will
soon be reverted back to have the die/sprintf in the two callsite
in 4/4 anyway, so this hunk looks more like "I thought this would
be a good way, but in the end I had to change my mind".

> @@ -1952,11 +1965,12 @@ sub validate_patch {
>  			chdir($repo->wc_path() or $repo->repo_path())
>  				or die("chdir: $!");
>  			local $ENV{"GIT_DIR"} = $repo->repo_path();
> -			$hook_error = "rejected by sendemail-validate hook"
> -				if system($validate_hook, $target);
> +			if (my $msg = system_or_msg([$validate_hook, $target])) {
> +				$hook_error = __("rejected by sendemail-validate hook");
> +			}
>  			chdir($cwd_save) or die("chdir: $!");
>  		}
> -		return $hook_error if $hook_error;
> +		validate_patch_error($fn, $hook_error) if $hook_error;
>  	}
>  
>  	# Any long lines will be automatically fixed if we use a suitable transfer
> @@ -1966,7 +1980,7 @@ sub validate_patch {
>  			or die sprintf(__("unable to open %s: %s\n"), $fn, $!);
>  		while (my $line = <$fh>) {
>  			if (length($line) > 998) {
> -				return sprintf(__("%s: patch contains a line longer than 998 characters"), $.);
> +				validate_patch_error($fn, sprintf(__("%s: patch contains a line longer than 998 characters"), $.));
>  			}
>  		}
>  	}

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 4/4] git-send-email: improve --validate error output
  2021-04-04  9:19           ` [PATCH v2 4/4] git-send-email: improve --validate error output Ævar Arnfjörð Bjarmason
@ 2021-04-05 19:14             ` Junio C Hamano
  0 siblings, 0 replies; 479+ messages in thread
From: Junio C Hamano @ 2021-04-05 19:14 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git, Emily Shaffer

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> Improve the output we emit on --validate error to:
>
>  * Say "FILE:LINE" instead of "FILE: LINE".

OK, that is an improvement because it matches "grep -n" hits,
compiler error messages, etc., to help the editors to jump to these
lines.

>  * Don't say "patch contains a" after just mentioning the filename,
>    just leave it at "FILE:LINE: is longer than[...]. The "contains a"
>    sounded like we were talking about the file in general, when we're
>    actually checking it line-by-line.

This, too.

>  * Don't just say "rejected by sendemail-validate hook", but combine
>    that with the system_or_msg() output to say what exit code the hook
>    died with.
>
> I had an aborted attempt to make the line length checker note all
> lines that were longer than the limit. I didn't think that was worth
> the effort, but I've left in the testing change to check that we die
> as soon as we spot the first long line.
>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>  git-send-email.perl   | 23 ++++++++++-------------
>  t/t9001-send-email.sh | 17 ++++++++++++-----
>  2 files changed, 22 insertions(+), 18 deletions(-)

Will queue.  I like the end result, but left a comment about
flipping-and-flopping between 2/4 and this step on an extra
"validate_patch_error" helper sub.

Thanks.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 2/4] git-send-email: refactor duplicate $? checks into a function
  2021-04-04  9:19           ` [PATCH v2 2/4] git-send-email: refactor duplicate $? checks into a function Ævar Arnfjörð Bjarmason
  2021-04-05 19:11             ` Junio C Hamano
@ 2021-04-05 23:47             ` Junio C Hamano
  2021-04-08 22:43               ` Junio C Hamano
  1 sibling, 1 reply; 479+ messages in thread
From: Junio C Hamano @ 2021-04-05 23:47 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason, Emily Shaffer; +Cc: git

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> Refactor the duplicate checking of $? into a function. There's an
> outstanding series[1] wanting to add a third use of system() in this
> file, let's not copy this boilerplate anymore when that happens.
>
> 1. http://lore.kernel.org/git/87y2esg22j.fsf@evledraar.gmail.com
>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>  git-send-email.perl | 48 +++++++++++++++++++++++++++++----------------
>  1 file changed, 31 insertions(+), 17 deletions(-)

>  sub validate_patch {
>  	my ($fn, $xfer_encoding) = @_;
>  
> @@ -1952,11 +1965,12 @@ sub validate_patch {
>  			chdir($repo->wc_path() or $repo->repo_path())
>  				or die("chdir: $!");
>  			local $ENV{"GIT_DIR"} = $repo->repo_path();
> -			$hook_error = "rejected by sendemail-validate hook"
> -				if system($validate_hook, $target);
> +			if (my $msg = system_or_msg([$validate_hook, $target])) {
> +				$hook_error = __("rejected by sendemail-validate hook");
> +			}
>  			chdir($cwd_save) or die("chdir: $!");
>  		}
> -		return $hook_error if $hook_error;
> +		validate_patch_error($fn, $hook_error) if $hook_error;
>  	}

One big thing that is different between this version and the one in
Emily's "config hook" topic is that this is still limited to the
case where $repo exists.  In the new world order, it will not matter
in what directory the command runs, as long as "git hook" finds the
hook, and details of the invocation is hidden behind the command.

I presume that Emily's series is expected to be updated soonish?
Please figure out who to go first and other details to work well
together between you two.

I'd drop the "config hook" topic for now, and I think the endpoint
of these four-patch series (the first "map vs for" can move more or
less independently) are more-or-less in a good shape (even though as
I said already, I think 2/4 and 4/4 want to be updated not to
introduce the intermediate "validate_patch_error()" sub in 2/4 only
to get rid of it in 4/4) and would require only one update.

Thanks.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH v3 0/3] refactor duplicate $? checks into a function + improve errors
  2021-04-04  9:19         ` [PATCH v2 0/4] refactor duplicate $? checks into a function + improve errors Ævar Arnfjörð Bjarmason
                             ` (3 preceding siblings ...)
  2021-04-04  9:19           ` [PATCH v2 4/4] git-send-email: improve --validate error output Ævar Arnfjörð Bjarmason
@ 2021-04-06 14:00           ` Ævar Arnfjörð Bjarmason
  2021-04-06 14:00             ` [PATCH v3 1/3] git-send-email: test full --validate output Ævar Arnfjörð Bjarmason
                               ` (3 more replies)
  4 siblings, 4 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-04-06 14:00 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Emily Shaffer, Ævar Arnfjörð Bjarmason

Version 3 yields the TREESAME end result as v2[1], but re-arranges the
way we get there to make the progression more understandable, along
with a minor commit message update.

I also peeled off the previous 1st patch, as Junio's picked it up
separately and marged it into "next" already.

1. http://lore.kernel.org/git/cover-0.5-00000000000-20210404T091649Z-avarab@gmail.com

Ævar Arnfjörð Bjarmason (3):
  git-send-email: test full --validate output
  git-send-email: refactor duplicate $? checks into a function
  git-send-email: improve --validate error output

 git-send-email.perl   | 45 +++++++++++++++++++++++++++----------------
 t/t9001-send-email.sh | 35 +++++++++++++++++++++++++--------
 2 files changed, 55 insertions(+), 25 deletions(-)

Range-diff:
2:  15b59c226d4 = 1:  6e1009e5bed git-send-email: test full --validate output
1:  f236f083e36 ! 2:  4ee582d8301 git-send-email: refactor duplicate $? checks into a function
    @@ git-send-email.perl: sub is_format_patch_arg {
      		}
      	}
      }
    -@@ git-send-email.perl: sub unique_email_list {
    - 	return @emails;
    - }
    - 
    -+sub validate_patch_error {
    -+	my ($fn, $error) = @_;
    -+	die sprintf(__("fatal: %s: %s\nwarning: no patches were sent\n"),
    -+		    $fn, $error);
    -+}
    -+
    - sub validate_patch {
    - 	my ($fn, $xfer_encoding) = @_;
    - 
     @@ git-send-email.perl: sub validate_patch {
      			chdir($repo->wc_path() or $repo->repo_path())
      				or die("chdir: $!");
      			local $ENV{"GIT_DIR"} = $repo->repo_path();
     -			$hook_error = "rejected by sendemail-validate hook"
     -				if system($validate_hook, $target);
    -+			if (my $msg = system_or_msg([$validate_hook, $target])) {
    -+				$hook_error = __("rejected by sendemail-validate hook");
    -+			}
    ++			$hook_error = system_or_msg([$validate_hook, $target]);
      			chdir($cwd_save) or die("chdir: $!");
      		}
     -		return $hook_error if $hook_error;
    -+		validate_patch_error($fn, $hook_error) if $hook_error;
    ++		if ($hook_error) {
    ++			die sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
    ++				       "warning: no patches were sent\n"), $fn);
    ++		}
      	}
      
      	# Any long lines will be automatically fixed if we use a suitable transfer
    @@ git-send-email.perl: sub validate_patch {
      		while (my $line = <$fh>) {
      			if (length($line) > 998) {
     -				return sprintf(__("%s: patch contains a line longer than 998 characters"), $.);
    -+				validate_patch_error($fn, sprintf(__("%s: patch contains a line longer than 998 characters"), $.));
    ++				die sprintf(__("fatal: %s: %d: patch contains a line longer than 998 characters\n" .
    ++					       "warning: no patches were sent\n"),
    ++					    $fn, $.);
      			}
      		}
      	}
3:  a1edceb4913 ! 3:  8a67afd3404 git-send-email: improve --validate error output
    @@ Commit message
     
         Improve the output we emit on --validate error to:
     
    -     * Say "FILE:LINE" instead of "FILE: LINE".
    +     * Say "FILE:LINE" instead of "FILE: LINE", to match "grep -n",
    +       compiler error messages etc.
     
          * Don't say "patch contains a" after just mentioning the filename,
            just leave it at "FILE:LINE: is longer than[...]. The "contains a"
    @@ git-send-email.perl: sub system_or_msg {
      }
      
      sub system_or_die {
    -@@ git-send-email.perl: sub unique_email_list {
    - 	return @emails;
    - }
    - 
    --sub validate_patch_error {
    --	my ($fn, $error) = @_;
    --	die sprintf(__("fatal: %s: %s\nwarning: no patches were sent\n"),
    --		    $fn, $error);
    --}
    --
    - sub validate_patch {
    - 	my ($fn, $xfer_encoding) = @_;
    - 
     @@ git-send-email.perl: sub validate_patch {
    - 			chdir($repo->wc_path() or $repo->repo_path())
    - 				or die("chdir: $!");
    - 			local $ENV{"GIT_DIR"} = $repo->repo_path();
    --			if (my $msg = system_or_msg([$validate_hook, $target])) {
    --				$hook_error = __("rejected by sendemail-validate hook");
    --			}
    -+			$hook_error = system_or_msg([$validate_hook, $target]);
    - 			chdir($cwd_save) or die("chdir: $!");
      		}
    --		validate_patch_error($fn, $hook_error) if $hook_error;
    -+		if ($hook_error) {
    -+			die sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
    + 		if ($hook_error) {
    + 			die sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
    +-				       "warning: no patches were sent\n"), $fn);
     +				       "%s\n" .
     +				       "warning: no patches were sent\n"), $fn, $hook_error);
    -+		}
    + 		}
      	}
      
    - 	# Any long lines will be automatically fixed if we use a suitable transfer
     @@ git-send-email.perl: sub validate_patch {
      			or die sprintf(__("unable to open %s: %s\n"), $fn, $!);
      		while (my $line = <$fh>) {
      			if (length($line) > 998) {
    --				validate_patch_error($fn, sprintf(__("%s: patch contains a line longer than 998 characters"), $.));
    +-				die sprintf(__("fatal: %s: %d: patch contains a line longer than 998 characters\n" .
    +-					       "warning: no patches were sent\n"),
    +-					    $fn, $.);
     +				die sprintf(__("fatal: %s:%d is longer than 998 characters\n" .
     +					       "warning: no patches were sent\n"), $fn, $.);
      			}
-- 
2.31.1.527.g9b8f7de2547


^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH v3 1/3] git-send-email: test full --validate output
  2021-04-06 14:00           ` [PATCH v3 0/3] refactor duplicate $? checks into a function + improve errors Ævar Arnfjörð Bjarmason
@ 2021-04-06 14:00             ` Ævar Arnfjörð Bjarmason
  2021-04-06 14:00             ` [PATCH v3 2/3] git-send-email: refactor duplicate $? checks into a function Ævar Arnfjörð Bjarmason
                               ` (2 subsequent siblings)
  3 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-04-06 14:00 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Emily Shaffer, Ævar Arnfjörð Bjarmason

Change the tests that grep substrings out of the output to use a full
test_cmp, in preparation for improving the output.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t9001-send-email.sh | 24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)

diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index 1a1caf8f2ed..74225e3dc7a 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -422,8 +422,12 @@ test_expect_success $PREREQ 'reject long lines' '
 		--smtp-server="$(pwd)/fake.sendmail" \
 		--transfer-encoding=8bit \
 		$patches longline.patch \
-		2>errors &&
-	grep longline.patch errors
+		2>actual &&
+	cat >expect <<-\EOF &&
+	fatal: longline.patch: 35: patch contains a line longer than 998 characters
+	warning: no patches were sent
+	EOF
+	test_cmp expect actual
 '
 
 test_expect_success $PREREQ 'no patch was sent' '
@@ -527,9 +531,13 @@ test_expect_success $PREREQ "--validate respects relative core.hooksPath path" '
 		--to=nobody@example.com \
 		--smtp-server="$(pwd)/fake.sendmail" \
 		--validate \
-		longline.patch 2>err &&
+		longline.patch 2>actual &&
 	test_path_is_file my-hooks.ran &&
-	grep "rejected by sendemail-validate" err
+	cat >expect <<-\EOF &&
+	fatal: longline.patch: rejected by sendemail-validate hook
+	warning: no patches were sent
+	EOF
+	test_cmp expect actual
 '
 
 test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" '
@@ -540,9 +548,13 @@ test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" '
 		--to=nobody@example.com \
 		--smtp-server="$(pwd)/fake.sendmail" \
 		--validate \
-		longline.patch 2>err &&
+		longline.patch 2>actual &&
 	test_path_is_file my-hooks.ran &&
-	grep "rejected by sendemail-validate" err
+	cat >expect <<-\EOF &&
+	fatal: longline.patch: rejected by sendemail-validate hook
+	warning: no patches were sent
+	EOF
+	test_cmp expect actual
 '
 
 for enc in 7bit 8bit quoted-printable base64
-- 
2.31.1.527.g9b8f7de2547


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v3 2/3] git-send-email: refactor duplicate $? checks into a function
  2021-04-06 14:00           ` [PATCH v3 0/3] refactor duplicate $? checks into a function + improve errors Ævar Arnfjörð Bjarmason
  2021-04-06 14:00             ` [PATCH v3 1/3] git-send-email: test full --validate output Ævar Arnfjörð Bjarmason
@ 2021-04-06 14:00             ` Ævar Arnfjörð Bjarmason
  2021-04-06 14:00             ` [PATCH v3 3/3] git-send-email: improve --validate error output Ævar Arnfjörð Bjarmason
  2021-04-06 20:33             ` [PATCH v3 0/3] refactor duplicate $? checks into a function + improve errors Junio C Hamano
  3 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-04-06 14:00 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Emily Shaffer, Ævar Arnfjörð Bjarmason

Refactor the duplicate checking of $? into a function. There's an
outstanding series[1] wanting to add a third use of system() in this
file, let's not copy this boilerplate anymore when that happens.

1. http://lore.kernel.org/git/87y2esg22j.fsf@evledraar.gmail.com

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-send-email.perl | 45 ++++++++++++++++++++++++++++-----------------
 1 file changed, 28 insertions(+), 17 deletions(-)

diff --git a/git-send-email.perl b/git-send-email.perl
index 6893c8e5808..2dd48621759 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -212,22 +212,31 @@ sub format_2822_time {
 my $multiedit;
 my $editor;
 
+sub system_or_msg {
+	my ($args, $msg) = @_;
+	system(@$args);
+	my $signalled = $? & 127;
+	my $exit_code = $? >> 8;
+	return unless $signalled or $exit_code;
+
+	return sprintf(__("failed to run command %s, died with code %d"),
+		       "@$args", $exit_code);
+}
+
+sub system_or_die {
+	my $msg = system_or_msg(@_);
+	die $msg if $msg;
+}
+
 sub do_edit {
 	if (!defined($editor)) {
 		$editor = Git::command_oneline('var', 'GIT_EDITOR');
 	}
+	my $die_msg = __("the editor exited uncleanly, aborting everything");
 	if (defined($multiedit) && !$multiedit) {
-		for (@_) {
-			system('sh', '-c', $editor.' "$@"', $editor, $_);
-			if (($? & 127) || ($? >> 8)) {
-				die(__("the editor exited uncleanly, aborting everything"));
-			}
-		}
+		system_or_die(['sh', '-c', $editor.' "$@"', $editor, $_], $die_msg) for @_;
 	} else {
-		system('sh', '-c', $editor.' "$@"', $editor, @_);
-		if (($? & 127) || ($? >> 8)) {
-			die(__("the editor exited uncleanly, aborting everything"));
-		}
+		system_or_die(['sh', '-c', $editor.' "$@"', $editor, @_], $die_msg);
 	}
 }
 
@@ -698,9 +707,7 @@ sub is_format_patch_arg {
 if ($validate) {
 	foreach my $f (@files) {
 		unless (-p $f) {
-			my $error = validate_patch($f, $target_xfer_encoding);
-			$error and die sprintf(__("fatal: %s: %s\nwarning: no patches were sent\n"),
-						  $f, $error);
+			validate_patch($f, $target_xfer_encoding);
 		}
 	}
 }
@@ -1952,11 +1959,13 @@ sub validate_patch {
 			chdir($repo->wc_path() or $repo->repo_path())
 				or die("chdir: $!");
 			local $ENV{"GIT_DIR"} = $repo->repo_path();
-			$hook_error = "rejected by sendemail-validate hook"
-				if system($validate_hook, $target);
+			$hook_error = system_or_msg([$validate_hook, $target]);
 			chdir($cwd_save) or die("chdir: $!");
 		}
-		return $hook_error if $hook_error;
+		if ($hook_error) {
+			die sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
+				       "warning: no patches were sent\n"), $fn);
+		}
 	}
 
 	# Any long lines will be automatically fixed if we use a suitable transfer
@@ -1966,7 +1975,9 @@ sub validate_patch {
 			or die sprintf(__("unable to open %s: %s\n"), $fn, $!);
 		while (my $line = <$fh>) {
 			if (length($line) > 998) {
-				return sprintf(__("%s: patch contains a line longer than 998 characters"), $.);
+				die sprintf(__("fatal: %s: %d: patch contains a line longer than 998 characters\n" .
+					       "warning: no patches were sent\n"),
+					    $fn, $.);
 			}
 		}
 	}
-- 
2.31.1.527.g9b8f7de2547


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v3 3/3] git-send-email: improve --validate error output
  2021-04-06 14:00           ` [PATCH v3 0/3] refactor duplicate $? checks into a function + improve errors Ævar Arnfjörð Bjarmason
  2021-04-06 14:00             ` [PATCH v3 1/3] git-send-email: test full --validate output Ævar Arnfjörð Bjarmason
  2021-04-06 14:00             ` [PATCH v3 2/3] git-send-email: refactor duplicate $? checks into a function Ævar Arnfjörð Bjarmason
@ 2021-04-06 14:00             ` Ævar Arnfjörð Bjarmason
  2021-04-06 20:33             ` [PATCH v3 0/3] refactor duplicate $? checks into a function + improve errors Junio C Hamano
  3 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-04-06 14:00 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Emily Shaffer, Ævar Arnfjörð Bjarmason

Improve the output we emit on --validate error to:

 * Say "FILE:LINE" instead of "FILE: LINE", to match "grep -n",
   compiler error messages etc.

 * Don't say "patch contains a" after just mentioning the filename,
   just leave it at "FILE:LINE: is longer than[...]. The "contains a"
   sounded like we were talking about the file in general, when we're
   actually checking it line-by-line.

 * Don't just say "rejected by sendemail-validate hook", but combine
   that with the system_or_msg() output to say what exit code the hook
   died with.

I had an aborted attempt to make the line length checker note all
lines that were longer than the limit. I didn't think that was worth
the effort, but I've left in the testing change to check that we die
as soon as we spot the first long line.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-send-email.perl   | 12 ++++++------
 t/t9001-send-email.sh | 17 ++++++++++++-----
 2 files changed, 18 insertions(+), 11 deletions(-)

diff --git a/git-send-email.perl b/git-send-email.perl
index 2dd48621759..175da07d946 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -219,8 +219,8 @@ sub system_or_msg {
 	my $exit_code = $? >> 8;
 	return unless $signalled or $exit_code;
 
-	return sprintf(__("failed to run command %s, died with code %d"),
-		       "@$args", $exit_code);
+	return sprintf(__("fatal: command '%s' died with exit code %d"),
+		       $args->[0], $exit_code);
 }
 
 sub system_or_die {
@@ -1964,7 +1964,8 @@ sub validate_patch {
 		}
 		if ($hook_error) {
 			die sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
-				       "warning: no patches were sent\n"), $fn);
+				       "%s\n" .
+				       "warning: no patches were sent\n"), $fn, $hook_error);
 		}
 	}
 
@@ -1975,9 +1976,8 @@ sub validate_patch {
 			or die sprintf(__("unable to open %s: %s\n"), $fn, $!);
 		while (my $line = <$fh>) {
 			if (length($line) > 998) {
-				die sprintf(__("fatal: %s: %d: patch contains a line longer than 998 characters\n" .
-					       "warning: no patches were sent\n"),
-					    $fn, $.);
+				die sprintf(__("fatal: %s:%d is longer than 998 characters\n" .
+					       "warning: no patches were sent\n"), $fn, $.);
 			}
 		}
 	}
diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index 74225e3dc7a..65b30353719 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -415,7 +415,11 @@ test_expect_success $PREREQ 'reject long lines' '
 	z512=$z64$z64$z64$z64$z64$z64$z64$z64 &&
 	clean_fake_sendmail &&
 	cp $patches longline.patch &&
-	echo $z512$z512 >>longline.patch &&
+	cat >>longline.patch <<-EOF &&
+	$z512$z512
+	not a long line
+	$z512$z512
+	EOF
 	test_must_fail git send-email \
 		--from="Example <nobody@example.com>" \
 		--to=nobody@example.com \
@@ -424,7 +428,7 @@ test_expect_success $PREREQ 'reject long lines' '
 		$patches longline.patch \
 		2>actual &&
 	cat >expect <<-\EOF &&
-	fatal: longline.patch: 35: patch contains a line longer than 998 characters
+	fatal: longline.patch:35 is longer than 998 characters
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
@@ -533,15 +537,17 @@ test_expect_success $PREREQ "--validate respects relative core.hooksPath path" '
 		--validate \
 		longline.patch 2>actual &&
 	test_path_is_file my-hooks.ran &&
-	cat >expect <<-\EOF &&
+	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
+	fatal: command '"'"'$(pwd)/my-hooks/sendemail-validate'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
 '
 
 test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" '
-	test_config core.hooksPath "$(pwd)/my-hooks" &&
+	hooks_path="$(pwd)/my-hooks" &&
+	test_config core.hooksPath "$hooks_path" &&
 	test_when_finished "rm my-hooks.ran" &&
 	test_must_fail git send-email \
 		--from="Example <nobody@example.com>" \
@@ -550,8 +556,9 @@ test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" '
 		--validate \
 		longline.patch 2>actual &&
 	test_path_is_file my-hooks.ran &&
-	cat >expect <<-\EOF &&
+	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
+	fatal: command '"'"'$hooks_path/sendemail-validate'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
-- 
2.31.1.527.g9b8f7de2547


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 0/3] refactor duplicate $? checks into a function + improve errors
  2021-04-06 14:00           ` [PATCH v3 0/3] refactor duplicate $? checks into a function + improve errors Ævar Arnfjörð Bjarmason
                               ` (2 preceding siblings ...)
  2021-04-06 14:00             ` [PATCH v3 3/3] git-send-email: improve --validate error output Ævar Arnfjörð Bjarmason
@ 2021-04-06 20:33             ` Junio C Hamano
  3 siblings, 0 replies; 479+ messages in thread
From: Junio C Hamano @ 2021-04-06 20:33 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git, Emily Shaffer

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> Version 3 yields the TREESAME end result as v2[1], but re-arranges the
> way we get there to make the progression more understandable, along
> with a minor commit message update.

Nice.  I see no more nits to pick ;-)

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 37/37] docs: unify githooks and git-hook manpages
  2021-03-11  2:10 ` [PATCH v8 37/37] docs: unify githooks and git-hook manpages Emily Shaffer
  2021-03-12  9:29   ` Ævar Arnfjörð Bjarmason
@ 2021-04-07  2:36   ` Junio C Hamano
  2021-04-08 20:20     ` Jeff Hostetler
  2021-04-08 23:46     ` Emily Shaffer
  1 sibling, 2 replies; 479+ messages in thread
From: Junio C Hamano @ 2021-04-07  2:36 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git, Jeff Hostetler

Emily Shaffer <emilyshaffer@google.com> writes:

> By showing the list of all hooks in 'git help hook' for users to refer
> to, 'git help hook' becomes a one-stop shop for hook authorship. Since
> some may still have muscle memory for 'git help githooks', though,
> reference the 'git hook' commands and otherwise don't remove content.
>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  Documentation/git-hook.txt     |  11 +
>  Documentation/githooks.txt     | 716 +--------------------------------
>  Documentation/native-hooks.txt | 708 ++++++++++++++++++++++++++++++++
>  3 files changed, 724 insertions(+), 711 deletions(-)
>  create mode 100644 Documentation/native-hooks.txt

While this would be a very good move when this were the only topic
juggling the hook related documentation, in the real world, it
creates rather nasty "ouch, the original hooks document was updated,
and we need to carry these changes over to the new native-hooks
file" conflicts with multiple commits on different topics.

$ git log --oneline --no-merges es/config-hooks..seen Documentation/githooks.txt
2d4e48b8ee fsmonitor--daemon: man page and documentation
23c781f173 githooks.txt: clarify documentation on reference-transaction hook
5f308a89d8 githooks.txt: replace mentions of SHA-1 specific properties
7efc378205 doc: fix some typos

$ git log --oneline --no-merges ^master es/config-hooks..seen Documentation/githooks.txt
2d4e48b8ee fsmonitor--daemon: man page and documentation

As three of the four changes are already in master, it probably is a
good idea to rebase this topic (and redo this step) to update the
native-hooks.txt

I am not sure offhand how ready fsmonitor--daemon stuff is, but if
it takes longer to stabilize than this topic, it might make sense to
hold off the changes to githooks.txt in that topic, until this topic
stabilizes enough to hit at least 'next', preferrably 'master', and
then base that topic (or at least the documentation part of it) on
the final shape of the native-hooks.txt.

Or better ideas?

Thanks.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 37/37] docs: unify githooks and git-hook manpages
  2021-04-07  2:36   ` Junio C Hamano
@ 2021-04-08 20:20     ` Jeff Hostetler
  2021-04-08 21:17       ` Junio C Hamano
  2021-04-08 23:46     ` Emily Shaffer
  1 sibling, 1 reply; 479+ messages in thread
From: Jeff Hostetler @ 2021-04-08 20:20 UTC (permalink / raw)
  To: Junio C Hamano, Emily Shaffer; +Cc: git, Jeff Hostetler



On 4/6/21 10:36 PM, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
>> By showing the list of all hooks in 'git help hook' for users to refer
>> to, 'git help hook' becomes a one-stop shop for hook authorship. Since
>> some may still have muscle memory for 'git help githooks', though,
>> reference the 'git hook' commands and otherwise don't remove content.
>>
>> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
>> ---
>>   Documentation/git-hook.txt     |  11 +
>>   Documentation/githooks.txt     | 716 +--------------------------------
>>   Documentation/native-hooks.txt | 708 ++++++++++++++++++++++++++++++++
>>   3 files changed, 724 insertions(+), 711 deletions(-)
>>   create mode 100644 Documentation/native-hooks.txt
> 
> While this would be a very good move when this were the only topic
> juggling the hook related documentation, in the real world, it
> creates rather nasty "ouch, the original hooks document was updated,
> and we need to carry these changes over to the new native-hooks
> file" conflicts with multiple commits on different topics.
> 
> $ git log --oneline --no-merges es/config-hooks..seen Documentation/githooks.txt
> 2d4e48b8ee fsmonitor--daemon: man page and documentation
> 23c781f173 githooks.txt: clarify documentation on reference-transaction hook
> 5f308a89d8 githooks.txt: replace mentions of SHA-1 specific properties
> 7efc378205 doc: fix some typos
> 
> $ git log --oneline --no-merges ^master es/config-hooks..seen Documentation/githooks.txt
> 2d4e48b8ee fsmonitor--daemon: man page and documentation
> 
> As three of the four changes are already in master, it probably is a
> good idea to rebase this topic (and redo this step) to update the
> native-hooks.txt
> 
> I am not sure offhand how ready fsmonitor--daemon stuff is, but if
> it takes longer to stabilize than this topic, it might make sense to
> hold off the changes to githooks.txt in that topic, until this topic
> stabilizes enough to hit at least 'next', preferrably 'master', and
> then base that topic (or at least the documentation part of it) on
> the final shape of the native-hooks.txt.
> 
> Or better ideas?
> 
> Thanks.
> 

I expect the fsmonitor stuff to take a while.  It is rather large
and complicated.  My changes in the Documentation are rather minor.
And I wouldn't want to be the sole reason to hold up Emily's changes.

If it would be helpful, you can add a "revert" commit on top of my
branch for my documentation commit -or- just drop it completely from
my series.  Then I can re-adjust/rebase my doc changes before
I send a V2.

Jeff

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 37/37] docs: unify githooks and git-hook manpages
  2021-04-08 20:20     ` Jeff Hostetler
@ 2021-04-08 21:17       ` Junio C Hamano
  0 siblings, 0 replies; 479+ messages in thread
From: Junio C Hamano @ 2021-04-08 21:17 UTC (permalink / raw)
  To: Jeff Hostetler; +Cc: Emily Shaffer, git, Jeff Hostetler

Jeff Hostetler <git@jeffhostetler.com> writes:

> I expect the fsmonitor stuff to take a while.  It is rather large
> and complicated.  My changes in the Documentation are rather minor.
> And I wouldn't want to be the sole reason to hold up Emily's changes.
>
> If it would be helpful, you can add a "revert" commit on top of my
> branch for my documentation commit -or- just drop it completely from
> my series.  Then I can re-adjust/rebase my doc changes before
> I send a V2.

Sounds like a plan.  I'll drop that step for now before the next
integration cycle.

There is another topic that interacts with es/config-hooks topic
badly (which I haven't resolved) in flight, though.

Thanks.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 2/4] git-send-email: refactor duplicate $? checks into a function
  2021-04-05 23:47             ` Junio C Hamano
@ 2021-04-08 22:43               ` Junio C Hamano
  2021-04-08 22:46                 ` Junio C Hamano
  0 siblings, 1 reply; 479+ messages in thread
From: Junio C Hamano @ 2021-04-08 22:43 UTC (permalink / raw)
  To: Emily Shaffer, Ævar Arnfjörð Bjarmason; +Cc: git

Junio C Hamano <gitster@pobox.com> writes:

> One big thing that is different between this version and the one in
> Emily's "config hook" topic is that this is still limited to the
> case where $repo exists.  In the new world order, it will not matter
> in what directory the command runs, as long as "git hook" finds the
> hook, and details of the invocation is hidden behind the command.
>
> I presume that Emily's series is expected to be updated soonish?
> Please figure out who to go first and other details to work well
> together between you two.

Since I didn't hear from either of you, I'll queue with this
possibly bogus conflict resolution for now.

Thanks.


diff --cc git-send-email.perl
index 175da07d94,73e1e0b51a..0000000000
--- i/git-send-email.perl
+++ w/git-send-email.perl
@@@ -1947,27 -1940,11 +1947,13 @@@ sub unique_email_list 
  
  sub validate_patch {
  	my ($fn, $xfer_encoding) = @_;
--
- 	if ($repo) {
- 		my $validate_hook = catfile($repo->hooks_path(),
- 					    'sendemail-validate');
- 		my $hook_error;
- 		if (-x $validate_hook) {
- 			my $target = abs_path($fn);
- 			# The hook needs a correct cwd and GIT_DIR.
- 			my $cwd_save = cwd();
- 			chdir($repo->wc_path() or $repo->repo_path())
- 				or die("chdir: $!");
- 			local $ENV{"GIT_DIR"} = $repo->repo_path();
- 			$hook_error = system_or_msg([$validate_hook, $target]);
- 			chdir($cwd_save) or die("chdir: $!");
- 		}
- 		if ($hook_error) {
- 			die sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
- 				       "%s\n" .
- 				       "warning: no patches were sent\n"), $fn, $hook_error);
- 		}
+ 	my $target = abs_path($fn);
 -	return "rejected by sendemail-validate hook"
 -		if system(("git", "hook", "run", "sendemail-validate", "-a",
 -				$target));
++	$hook_error = system_or_msg([qw(git hook run sendemail-validate -a), $target]);
++	if ($hook_error) {
++		die sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
++			       "%s\n" .
++			       "warning: no patches were sent\n"), $fn, $hook_error);
 +	}
  
  	# Any long lines will be automatically fixed if we use a suitable transfer
  	# encoding.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 2/4] git-send-email: refactor duplicate $? checks into a function
  2021-04-08 22:43               ` Junio C Hamano
@ 2021-04-08 22:46                 ` Junio C Hamano
  2021-04-08 23:54                   ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 479+ messages in thread
From: Junio C Hamano @ 2021-04-08 22:46 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Ævar Arnfjörð Bjarmason, git

Junio C Hamano <gitster@pobox.com> writes:

> Junio C Hamano <gitster@pobox.com> writes:
>
>> One big thing that is different between this version and the one in
>> Emily's "config hook" topic is that this is still limited to the
>> case where $repo exists.  In the new world order, it will not matter
>> in what directory the command runs, as long as "git hook" finds the
>> hook, and details of the invocation is hidden behind the command.
>>
>> I presume that Emily's series is expected to be updated soonish?
>> Please figure out who to go first and other details to work well
>> together between you two.
>
> Since I didn't hear from either of you, I'll queue with this
> possibly bogus conflict resolution for now.
>

Well, I retract it.  This makes many steps in send-email tests
fail.  For now, es/config-hooks topic is excluded from 'seen'.

What's the status of that topic, if there weren't other topics in
flight that interfere with it, by the way?  Is it otherwise a good
enough shape to be given priority and stable enough to get other
topics rebased on top of it?

Thanks.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 37/37] docs: unify githooks and git-hook manpages
  2021-04-07  2:36   ` Junio C Hamano
  2021-04-08 20:20     ` Jeff Hostetler
@ 2021-04-08 23:46     ` Emily Shaffer
  2021-04-09  0:03       ` Junio C Hamano
  1 sibling, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-04-08 23:46 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Jeff Hostetler

On Tue, Apr 06, 2021 at 07:36:15PM -0700, Junio C Hamano wrote:
> 
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > By showing the list of all hooks in 'git help hook' for users to refer
> > to, 'git help hook' becomes a one-stop shop for hook authorship. Since
> > some may still have muscle memory for 'git help githooks', though,
> > reference the 'git hook' commands and otherwise don't remove content.
> >
> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > ---
> >  Documentation/git-hook.txt     |  11 +
> >  Documentation/githooks.txt     | 716 +--------------------------------
> >  Documentation/native-hooks.txt | 708 ++++++++++++++++++++++++++++++++
> >  3 files changed, 724 insertions(+), 711 deletions(-)
> >  create mode 100644 Documentation/native-hooks.txt
> 
> While this would be a very good move when this were the only topic
> juggling the hook related documentation, in the real world, it
> creates rather nasty "ouch, the original hooks document was updated,
> and we need to carry these changes over to the new native-hooks
> file" conflicts with multiple commits on different topics.
> 
> $ git log --oneline --no-merges es/config-hooks..seen Documentation/githooks.txt
> 2d4e48b8ee fsmonitor--daemon: man page and documentation
> 23c781f173 githooks.txt: clarify documentation on reference-transaction hook
> 5f308a89d8 githooks.txt: replace mentions of SHA-1 specific properties
> 7efc378205 doc: fix some typos
> 
> $ git log --oneline --no-merges ^master es/config-hooks..seen Documentation/githooks.txt
> 2d4e48b8ee fsmonitor--daemon: man page and documentation
> 
> As three of the four changes are already in master, it probably is a
> good idea to rebase this topic (and redo this step) to update the
> native-hooks.txt
> 
> I am not sure offhand how ready fsmonitor--daemon stuff is, but if
> it takes longer to stabilize than this topic, it might make sense to
> hold off the changes to githooks.txt in that topic, until this topic
> stabilizes enough to hit at least 'next', preferrably 'master', and
> then base that topic (or at least the documentation part of it) on
> the final shape of the native-hooks.txt.
> 
> Or better ideas?
> 
> Thanks.

I got bitten by this same issue with native-hooks.txt while addressing
comments, too. Another commenter suggested to not inline those hook
definitions into "git help hook" - so I plan to drop that part of this
patch. If it makes it easier for you, I think you could revert this last
commit; if we decide later that we want to have "git help hook" share
the hook definitions after all, I think we should do that separately and
as a quick change not stuck behind 36 other complicated patches.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 2/4] git-send-email: refactor duplicate $? checks into a function
  2021-04-08 22:46                 ` Junio C Hamano
@ 2021-04-08 23:54                   ` Ævar Arnfjörð Bjarmason
  2021-04-09  0:08                     ` Junio C Hamano
  0 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-04-08 23:54 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Emily Shaffer, git


On Fri, Apr 09 2021, Junio C Hamano wrote:

> Junio C Hamano <gitster@pobox.com> writes:
>
>> Junio C Hamano <gitster@pobox.com> writes:
>>
>>> One big thing that is different between this version and the one in
>>> Emily's "config hook" topic is that this is still limited to the
>>> case where $repo exists.  In the new world order, it will not matter
>>> in what directory the command runs, as long as "git hook" finds the
>>> hook, and details of the invocation is hidden behind the command.
>>>
>>> I presume that Emily's series is expected to be updated soonish?
>>> Please figure out who to go first and other details to work well
>>> together between you two.
>>
>> Since I didn't hear from either of you, I'll queue with this
>> possibly bogus conflict resolution for now.
>>
>
> Well, I retract it.  This makes many steps in send-email tests
> fail.  For now, es/config-hooks topic is excluded from 'seen'.

Sorry about not replying earlier upthread. FWIW I didn't look deeply
into how the chdir etc. might interact with Emily's topic. I figured
we'd want the $? etc. cleanup first, and that just deleting most of that
code once we had some hook runner to shell out to would be easy.

> What's the status of that topic, if there weren't other topics in
> flight that interfere with it, by the way?  Is it otherwise a good
> enough shape to be given priority and stable enough to get other
> topics rebased on top of it?

I see I've mentioned [1] in passing to you before, but in summary I have
some major qualms about parts of it, but very much like the overall
direction/goal of having hooks in config.

Elevator pitch summary of the lengthy [1]: hooks in config: good, but
having a "git hook" command introduce some nascent UI for managing a
subset of git-config: somewhere between "meh" / "bad idea" (see security
concerns in [1]) / "not needed". I.e. I demonstrated that we can replace
it with a trivial git-config wrapper, if the series doesn't go out of
its way to make it difficult (i.e. we can/should stick all config for a
given hook in the same <prefix>, and not re-invent the
"sendemail.identity" special-case).

I'd very much like the author to respond to that :) And/or for others to
chime in with what they think.

1. https://lore.kernel.org/git/87mtv8fww3.fsf@evledraar.gmail.com/

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v8 37/37] docs: unify githooks and git-hook manpages
  2021-04-08 23:46     ` Emily Shaffer
@ 2021-04-09  0:03       ` Junio C Hamano
  0 siblings, 0 replies; 479+ messages in thread
From: Junio C Hamano @ 2021-04-09  0:03 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git, Jeff Hostetler

Emily Shaffer <emilyshaffer@google.com> writes:

> I got bitten by this same issue with native-hooks.txt while addressing
> comments, too. Another commenter suggested to not inline those hook
> definitions into "git help hook" - so I plan to drop that part of this
> patch. If it makes it easier for you, I think you could revert this last
> commit; if we decide later that we want to have "git help hook" share
> the hook definitions after all, I think we should do that separately and
> as a quick change not stuck behind 36 other complicated patches.

I've already discarded the step, and then I had to eject the whole
topic from 'seen' for now (see my other message to you earlier
today).  The "other complicated patches" need to be whipped into
shape to be at least in 'next' first; I do not know how close the
last round is from that state.

Thanks.


^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 2/4] git-send-email: refactor duplicate $? checks into a function
  2021-04-08 23:54                   ` Ævar Arnfjörð Bjarmason
@ 2021-04-09  0:08                     ` Junio C Hamano
  2021-05-03 20:30                       ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Junio C Hamano @ 2021-04-09  0:08 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: Emily Shaffer, git

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> On Fri, Apr 09 2021, Junio C Hamano wrote:
> ...
>> What's the status of that topic, if there weren't other topics in
>> flight that interfere with it, by the way?  Is it otherwise a good
>> enough shape to be given priority and stable enough to get other
>> topics rebased on top of it?
>
> I see I've mentioned [1] in passing to you before, but in summary I have
> some major qualms about parts of it, but very much like the overall
> direction/goal of having hooks in config.
>
> Elevator pitch summary of the lengthy [1]: hooks in config: good, but
> having a "git hook" command introduce some nascent UI for managing a
> subset of git-config: somewhere between "meh" / "bad idea" (see security
> concerns in [1]) / "not needed". I.e. I demonstrated that we can replace
> it with a trivial git-config wrapper, if the series doesn't go out of
> its way to make it difficult (i.e. we can/should stick all config for a
> given hook in the same <prefix>, and not re-invent the
> "sendemail.identity" special-case).
>
> I'd very much like the author to respond to that :) And/or for others to
> chime in with what they think.
>
> 1. https://lore.kernel.org/git/87mtv8fww3.fsf@evledraar.gmail.com/

OK, Emily, I guess the ball is in your court now?

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 2/4] git-send-email: refactor duplicate $? checks into a function
  2021-04-09  0:08                     ` Junio C Hamano
@ 2021-05-03 20:30                       ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-03 20:30 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Ævar Arnfjörð Bjarmason, git

On Thu, Apr 08, 2021 at 05:08:30PM -0700, Junio C Hamano wrote:
> 
> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
> 
> > On Fri, Apr 09 2021, Junio C Hamano wrote:
> > ...
> >> What's the status of that topic, if there weren't other topics in
> >> flight that interfere with it, by the way?  Is it otherwise a good
> >> enough shape to be given priority and stable enough to get other
> >> topics rebased on top of it?
> >
> > I see I've mentioned [1] in passing to you before, but in summary I have
> > some major qualms about parts of it, but very much like the overall
> > direction/goal of having hooks in config.
> >
> > Elevator pitch summary of the lengthy [1]: hooks in config: good, but
> > having a "git hook" command introduce some nascent UI for managing a
> > subset of git-config: somewhere between "meh" / "bad idea" (see security
> > concerns in [1]) / "not needed". I.e. I demonstrated that we can replace
> > it with a trivial git-config wrapper, if the series doesn't go out of
> > its way to make it difficult (i.e. we can/should stick all config for a
> > given hook in the same <prefix>, and not re-invent the
> > "sendemail.identity" special-case).
> >
> > I'd very much like the author to respond to that :) And/or for others to
> > chime in with what they think.
> >
> > 1. https://lore.kernel.org/git/87mtv8fww3.fsf@evledraar.gmail.com/
> 
> OK, Emily, I guess the ball is in your court now?

The topic is not ready for submission besides interference. I have a
list of things to do and was sidetracked with other work (the submodule
RFC, etc.). This week I am working on getting this series polished and
ready to go.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH v9 00/37] propose config-based hooks
  2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
                   ` (39 preceding siblings ...)
  2021-03-12 11:13 ` Ævar Arnfjörð Bjarmason
@ 2021-05-27  0:08 ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 01/37] doc: propose hooks managed by the config Emily Shaffer
                     ` (38 more replies)
  40 siblings, 39 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git
  Cc: Emily Shaffer, Jeff King, Junio C Hamano, James Ramsay,
	Jonathan Nieder, brian m. carlson,
	Ævar Arnfjörð Bjarmason, Phillip Wood,
	Josh Steadmon, Johannes Schindelin, Jonathan Tan

After much delay and $DAYJOB, here is v9.

- Addressed nits in reviews on v8
- sendemail-validate hook becomes non-parallelized; updated to use
  Ævar's updated system_or_die() function
- changed strbuf to char* in hooks_list
  - Attempted to do so in run_command's stdout callback, but this made
    length protection difficult, so stuck with strbuf there.
- test_i18ncmp -> test_cmp
- Stop doing i18n lego in run_hooks()
- Checked that run_hooks_opt_init() is always separated by a space from
  variable decl blocks
- Checked for early returns which may skip run_hooks_opt_clear(); this
  resulted in minimizing the scope of run_hooks_opt in most places
- Got rid of native-hooks.txt. It was a nice idea, but not attached to a
  large and slow series like this one.
- In traces, log the name of the hook (e.g. "pre-commit") instead of the
  name of the executable (e.g. "/home/emily/check-for-debug-strings");
  the executable name is tracelogged as part of argv anyways, and we
  want to be able to tell which hook was responsible for invoking the
  executable in question.

Thanks.
 - Emily

Emily Shaffer (37):
  doc: propose hooks managed by the config
  hook: introduce git-hook subcommand
  hook: include hookdir hook in list
  hook: teach hook.runHookDir
  hook: implement hookcmd.<name>.skip
  parse-options: parse into strvec
  hook: add 'run' subcommand
  hook: introduce hook_exists()
  hook: support passing stdin to hooks
  run-command: allow stdin for run_processes_parallel
  hook: allow parallel hook execution
  hook: allow specifying working directory for hooks
  run-command: add stdin callback for parallelization
  hook: provide stdin by string_list or callback
  run-command: allow capturing of collated output
  hooks: allow callers to capture output
  commit: use config-based hooks
  am: convert applypatch hooks to use config
  merge: use config-based hooks for post-merge hook
  gc: use hook library for pre-auto-gc hook
  rebase: teach pre-rebase to use hook.h
  read-cache: convert post-index-change hook to use config
  receive-pack: convert push-to-checkout hook to hook.h
  git-p4: use 'git hook' to run hooks
  hooks: convert 'post-checkout' hook to hook library
  hook: convert 'post-rewrite' hook to config
  transport: convert pre-push hook to use config
  reference-transaction: look for hooks in config
  receive-pack: convert 'update' hook to hook.h
  proc-receive: acquire hook list from hook.h
  post-update: use hook.h library
  receive-pack: convert receive hooks to hook.h
  bugreport: use hook_exists instead of find_hook
  git-send-email: use 'git hook run' for 'sendemail-validate'
  run-command: stop thinking about hooks
  doc: clarify fsmonitor-watchman specification
  docs: link githooks and git-hook manpages

 .gitignore                                    |   1 +
 Documentation/Makefile                        |   1 +
 Documentation/config/hook.txt                 |  27 +
 Documentation/git-hook.txt                    | 162 ++++++
 Documentation/githooks.txt                    |  77 ++-
 Documentation/technical/api-parse-options.txt |   7 +
 .../technical/config-based-hooks.txt          | 369 +++++++++++++
 Makefile                                      |   2 +
 builtin.h                                     |   1 +
 builtin/am.c                                  |  39 +-
 builtin/bugreport.c                           |   4 +-
 builtin/checkout.c                            |  19 +-
 builtin/clone.c                               |   8 +-
 builtin/commit.c                              |  11 +-
 builtin/fetch.c                               |   1 +
 builtin/gc.c                                  |   9 +-
 builtin/hook.c                                | 190 +++++++
 builtin/merge.c                               |  15 +-
 builtin/rebase.c                              |  10 +-
 builtin/receive-pack.c                        | 326 ++++++------
 builtin/submodule--helper.c                   |   2 +-
 builtin/worktree.c                            |  32 +-
 command-list.txt                              |   1 +
 commit.c                                      |  22 +-
 commit.h                                      |   3 +-
 git-p4.py                                     |  67 +--
 git-send-email.perl                           |  26 +-
 git.c                                         |   1 +
 hook.c                                        | 483 ++++++++++++++++++
 hook.h                                        | 139 +++++
 parse-options-cb.c                            |  16 +
 parse-options.h                               |   4 +
 read-cache.c                                  |  13 +-
 refs.c                                        |  43 +-
 reset.c                                       |  17 +-
 run-command.c                                 | 156 +++---
 run-command.h                                 |  55 +-
 sequencer.c                                   |  92 ++--
 submodule.c                                   |   1 +
 t/helper/test-parse-options.c                 |   6 +
 t/helper/test-run-command.c                   |  46 +-
 t/t0040-parse-options.sh                      |  27 +
 t/t0061-run-command.sh                        |  37 ++
 t/t1360-config-based-hooks.sh                 | 329 ++++++++++++
 t/t1416-ref-transaction-hooks.sh              |  12 +-
 t/t5411/test-0015-too-many-hooks-error.sh     |  47 ++
 ...3-pre-commit-and-pre-merge-commit-hooks.sh |  17 +-
 t/t9001-send-email.sh                         |  13 +-
 transport.c                                   |  58 +--
 49 files changed, 2505 insertions(+), 539 deletions(-)
 create mode 100644 Documentation/config/hook.txt
 create mode 100644 Documentation/git-hook.txt
 create mode 100644 Documentation/technical/config-based-hooks.txt
 create mode 100644 builtin/hook.c
 create mode 100644 hook.c
 create mode 100644 hook.h
 create mode 100755 t/t1360-config-based-hooks.sh
 create mode 100644 t/t5411/test-0015-too-many-hooks-error.sh



 1:  85b99369f1 !  1:  d2b7ee8317 doc: propose hooks managed by the config
    @@ Commit message
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
     
    +
    + ## Notes ##
    +    Since v6, checked for inconsistencies with implementation and added lots of
    +    caveats about whether 'git hook add' and 'git hook edit' will ever materialize.
    +
    +    Hopefully this reflects reality now; please review accordingly.
    +
    +    Since v6, checked for inconsistencies with implementation and added lots of
    +    caveats about whether 'git hook add' and 'git hook edit' will ever materialize.
    +
    +    Hopefully this reflects reality now; please review accordingly.
    +
    +    Since v4, addressed comments from Jonathan Tan about wording. However, I have
    +    not addressed AEvar's comments or done a full re-review of this document.
    +    I wanted to get the rest of the series out for initial review first.
    +
    +     - Emily
    +
    +    Since v4, addressed comments from Jonathan Tan about wording.
    +
      ## Documentation/Makefile ##
     @@ Documentation/Makefile: SP_ARTICLES += $(API_DOCS)
      TECH_DOCS += MyFirstContribution
 2:  1d19f1477c <  -:  ---------- hook: scaffolding for git-hook subcommand
 3:  c125c63880 !  2:  112a809f02 hook: add list command
    @@ Metadata
     Author: Emily Shaffer <emilyshaffer@google.com>
     
      ## Commit message ##
    -    hook: add list command
    +    hook: introduce git-hook subcommand
     
    -    Teach 'git hook list <hookname>', which checks the known configs in
    +    Add a new subcommand, git-hook, which will be used to ease config-based
    +    hook management. This command will handle parsing configs to compose a
    +    list of hooks to run for a given event, as well as adding or modifying
    +    hook configs in an interactive fashion.
    +
    +    Start with 'git hook list <hookname>', which checks the known configs in
         order to create an ordered list of hooks to run on a given hook event.
     
         Multiple commands can be specified for a given hook by providing
    @@ Commit message
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
     
    +
    + ## Notes ##
    +    Since v4, mainly changed to RUN_SETUP_GENTLY so that 'git hook list' can
    +    be executed outside of a repo.
    +
    + ## .gitignore ##
    +@@
    + /git-grep
    + /git-hash-object
    + /git-help
    ++/git-hook
    + /git-http-backend
    + /git-http-fetch
    + /git-http-push
    +
      ## Documentation/config/hook.txt (new) ##
     @@
     +hook.<command>.command::
    @@ Documentation/config/hook.txt (new)
     +	as a command. This can be an executable on your device or a oneliner for
     +	your shell. See linkgit:git-hook[1].
     
    - ## Documentation/git-hook.txt ##
    -@@ Documentation/git-hook.txt: git-hook - Manage configured hooks
    - SYNOPSIS
    - --------
    - [verse]
    --'git hook'
    + ## Documentation/git-hook.txt (new) ##
    +@@
    ++git-hook(1)
    ++===========
    ++
    ++NAME
    ++----
    ++git-hook - Manage configured hooks
    ++
    ++SYNOPSIS
    ++--------
    ++[verse]
     +'git hook' list <hook-name>
    - 
    - DESCRIPTION
    - -----------
    --A placeholder command. Later, you will be able to list, add, and modify hooks
    --with this command.
    ++
    ++DESCRIPTION
    ++-----------
     +You can list configured hooks with this command. Later, you will be able to run,
     +add, and modify hooks with this command.
     +
    @@ Documentation/git-hook.txt: git-hook - Manage configured hooks
     +CONFIGURATION
     +-------------
     +include::config/hook.txt[]
    - 
    - GIT
    - ---
    ++
    ++GIT
    ++---
    ++Part of the linkgit:git[1] suite
     
      ## Makefile ##
     @@ Makefile: LIB_OBJS += hash-lookup.o
    @@ Makefile: LIB_OBJS += hash-lookup.o
      LIB_OBJS += ident.o
      LIB_OBJS += json-writer.o
      LIB_OBJS += kwset.o
    +@@ Makefile: BUILTIN_OBJS += builtin/get-tar-commit-id.o
    + BUILTIN_OBJS += builtin/grep.o
    + BUILTIN_OBJS += builtin/hash-object.o
    + BUILTIN_OBJS += builtin/help.o
    ++BUILTIN_OBJS += builtin/hook.o
    + BUILTIN_OBJS += builtin/index-pack.o
    + BUILTIN_OBJS += builtin/init-db.o
    + BUILTIN_OBJS += builtin/interpret-trailers.o
     
    - ## builtin/hook.c ##
    + ## builtin.h ##
    +@@ builtin.h: int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix);
    + int cmd_grep(int argc, const char **argv, const char *prefix);
    + int cmd_hash_object(int argc, const char **argv, const char *prefix);
    + int cmd_help(int argc, const char **argv, const char *prefix);
    ++int cmd_hook(int argc, const char **argv, const char *prefix);
    + int cmd_index_pack(int argc, const char **argv, const char *prefix);
    + int cmd_init_db(int argc, const char **argv, const char *prefix);
    + int cmd_interpret_trailers(int argc, const char **argv, const char *prefix);
    +
    + ## builtin/hook.c (new) ##
     @@
    - #include "cache.h"
    --
    - #include "builtin.h"
    ++#include "cache.h"
    ++#include "builtin.h"
     +#include "config.h"
     +#include "hook.h"
    - #include "parse-options.h"
    ++#include "parse-options.h"
     +#include "strbuf.h"
    - 
    - static const char * const builtin_hook_usage[] = {
    --	N_("git hook"),
    ++
    ++static const char * const builtin_hook_usage[] = {
     +	N_("git hook list <hookname>"),
    - 	NULL
    - };
    - 
    --int cmd_hook(int argc, const char **argv, const char *prefix)
    ++	NULL
    ++};
    ++
     +static int list(int argc, const char **argv, const char *prefix)
    - {
    --	struct option builtin_hook_options[] = {
    ++{
     +	struct list_head *head, *pos;
    -+	struct strbuf hookname = STRBUF_INIT;
    ++	const char *hookname = NULL;
     +
     +	struct option list_options[] = {
    - 		OPT_END(),
    - 	};
    - 
    --	argc = parse_options(argc, argv, prefix, builtin_hook_options,
    ++		OPT_END(),
    ++	};
    ++
     +	argc = parse_options(argc, argv, prefix, list_options,
    - 			     builtin_hook_usage, 0);
    - 
    ++			     builtin_hook_usage, 0);
    ++
     +	if (argc < 1) {
     +		usage_msg_opt(_("You must specify a hook event name to list."),
     +			      builtin_hook_usage, list_options);
     +	}
     +
    -+	strbuf_addstr(&hookname, argv[0]);
    ++	hookname = argv[0];
     +
    -+	head = hook_list(&hookname);
    ++	head = hook_list(hookname);
     +
     +	if (list_empty(head)) {
     +		printf(_("no commands configured for hook '%s'\n"),
    -+		       hookname.buf);
    -+		strbuf_release(&hookname);
    ++		       hookname);
     +		return 0;
     +	}
     +
    @@ builtin/hook.c
     +	}
     +
     +	clear_hook_list(head);
    -+	strbuf_release(&hookname);
     +
    - 	return 0;
    - }
    ++	return 0;
    ++}
     +
     +int cmd_hook(int argc, const char **argv, const char *prefix)
     +{
    @@ builtin/hook.c
     +	usage_with_options(builtin_hook_usage, builtin_hook_options);
     +}
     
    + ## command-list.txt ##
    +@@ command-list.txt: git-grep                                mainporcelain           info
    + git-gui                                 mainporcelain
    + git-hash-object                         plumbingmanipulators
    + git-help                                ancillaryinterrogators          complete
    ++git-hook                                mainporcelain
    + git-http-backend                        synchingrepositories
    + git-http-fetch                          synchelpers
    + git-http-push                           synchelpers
    +
    + ## git.c ##
    +@@ git.c: static struct cmd_struct commands[] = {
    + 	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
    + 	{ "hash-object", cmd_hash_object },
    + 	{ "help", cmd_help },
    ++	{ "hook", cmd_hook, RUN_SETUP_GENTLY },
    + 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
    + 	{ "init", cmd_init_db },
    + 	{ "init-db", cmd_init_db },
    +
      ## hook.c (new) ##
     @@
     +#include "cache.h"
    @@ hook.c (new)
     +	return 0;
     +}
     +
    -+struct list_head* hook_list(const struct strbuf* hookname)
    ++struct list_head* hook_list(const char* hookname)
     +{
     +	struct strbuf hook_key = STRBUF_INIT;
     +	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
    @@ hook.c (new)
     +	if (!hookname)
     +		return NULL;
     +
    -+	strbuf_addf(&hook_key, "hook.%s.command", hookname->buf);
    ++	strbuf_addf(&hook_key, "hook.%s.command", hookname);
     +
     +	git_config(hook_config_lookup, &cb_data);
     +
    @@ hook.h (new)
     + * Provides a linked list of 'struct hook' detailing commands which should run
     + * in response to the 'hookname' event, in execution order.
     + */
    -+struct list_head* hook_list(const struct strbuf *hookname);
    ++struct list_head* hook_list(const char *hookname);
     +
     +/* Free memory associated with a 'struct hook' */
     +void free_hook(struct hook *ptr);
     +/* Empties the list at 'head', calling 'free_hook()' on each entry */
     +void clear_hook_list(struct list_head *head);
     
    - ## t/t1360-config-based-hooks.sh ##
    -@@ t/t1360-config-based-hooks.sh: test_description='config-managed multihooks, including git-hook command'
    - 
    - . ./test-lib.sh
    - 
    --test_expect_success 'git hook command does not crash' '
    --	git hook
    + ## t/t1360-config-based-hooks.sh (new) ##
    +@@
    ++#!/bin/bash
    ++
    ++test_description='config-managed multihooks, including git-hook command'
    ++
    ++. ./test-lib.sh
    ++
     +ROOT=
     +if test_have_prereq MINGW
     +then
    @@ t/t1360-config-based-hooks.sh: test_description='config-managed multihooks, incl
     +
     +	git hook list pre-commit >actual &&
     +	test_cmp expected actual
    - '
    - 
    - test_done
    ++'
    ++
    ++test_done
 4:  0b8cd46ff9 !  3:  3114306368 hook: include hookdir hook in list
    @@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
      	list_for_each(pos, head) {
      		struct hook *item = list_entry(pos, struct hook, list);
     -		if (item)
    +-			printf("%s: %s\n",
    +-			       config_scope_name(item->origin),
     +		item = list_entry(pos, struct hook, list);
     +		if (item) {
    -+			/* Don't translate 'hookdir' - it matches the config */
    - 			printf("%s: %s\n",
    --			       config_scope_name(item->origin),
    ++			/*
    ++			 * TRANSLATORS: "<config scope>: <path>". Both fields
    ++			 * should be left untranslated; config scope matches the
    ++			 * output of 'git config --show-scope'. Marked for
    ++			 * translation to provide better RTL support later.
    ++			 */
    ++			printf(_("%s: %s\n"),
     +			       (item->from_hookdir
     +				? "hookdir"
     +				: config_scope_name(item->origin)),
    @@ hook.c: static void append_or_move_hook(struct list_head *head, const char *comm
      	}
      
      	/* re-set the scope so we show where an override was specified */
    -@@ hook.c: struct list_head* hook_list(const struct strbuf* hookname)
    +@@ hook.c: struct list_head* hook_list(const char* hookname)
      
      	git_config(hook_config_lookup, &cb_data);
      
     +	if (have_git_dir()) {
    -+		const char *legacy_hook_path = find_hook(hookname->buf);
    ++		const char *legacy_hook_path = find_hook(hookname);
     +
     +		/* Unconditionally add legacy hook, but annotate it. */
     +		if (legacy_hook_path) {
 5:  05c503fbe1 !  4:  681013c32a hook: teach hook.runHookDir
    @@ Commit message
         list'. Later on, though, we will pay attention to this enum when running
         the hooks.
     
    +
    + ## Notes ##
    +    Since v7, tidied up the behavior of the HOOK_UNKNOWN flag and added a test to
    +    enforce it - now it matches the design doc much better.
    +
    +    Also, thanks Jonathan Tan for pointing out that the commit message made no sense
    +    and was targeted for a different change. Rewrote the commit message now.
    +
    +    Plus, added HOOK_ERROR flag per Junio and Jonathan Nieder.
    +
    +    Newly split into its own commit since v4, and taking place much sooner.
    +
    +    An unfortunate side effect of adding this support *before* the
    +    hook.runHookDir support is that the labels on the list are not clear -
    +    because we aren't yet flagging which hooks are from the hookdir versus
    +    the config. I suppose we could move the addition of that field to the
    +    struct hook up to this patch, but it didn't make a lot of sense to me to
    +    do it just for cosmetic purposes.
    +
    +    Since v7, tidied up the behavior of the HOOK_UNKNOWN flag and added a test to
    +    enforce it - now it matches the design doc much better.
    +
    +    Also, thanks Jonathan Tan for pointing out that the commit message made no sense
    +    and was targeted for a different change. Rewrote the commit message now.
    +
    +    Newly split into its own commit since v4, and taking place much sooner.
    +
    +    An unfortunate side effect of adding this support *before* the
    +    hook.runHookDir support is that the labels on the list are not clear -
    +    because we aren't yet flagging which hooks are from the hookdir versus
    +    the config. I suppose we could move the addition of that field to the
    +    struct hook up to this patch, but it didn't make a lot of sense to me to
    +    do it just for cosmetic purposes.
    +
      ## Documentation/config/hook.txt ##
     @@ Documentation/config/hook.txt: hookcmd.<name>.command::
      	A command to execute during a hook for which <name> has been specified
    @@ builtin/hook.c: static const char * const builtin_hook_usage[] = {
      static int list(int argc, const char **argv, const char *prefix)
      {
      	struct list_head *head, *pos;
    - 	struct strbuf hookname = STRBUF_INIT;
    + 	const char *hookname = NULL;
     +	struct strbuf hookdir_annotation = STRBUF_INIT;
      
      	struct option list_options[] = {
      		OPT_END(),
     @@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
    - 		return 0;
    - 	}
    - 
    -+	switch (should_run_hookdir) {
    -+		case HOOKDIR_NO:
    -+			strbuf_addstr(&hookdir_annotation, _(" (will not run)"));
    -+			break;
    -+		case HOOKDIR_ERROR:
    -+			strbuf_addstr(&hookdir_annotation, _(" (will error and not run)"));
    -+			break;
    -+		case HOOKDIR_INTERACTIVE:
    -+			strbuf_addstr(&hookdir_annotation, _(" (will prompt)"));
    -+			break;
    -+		case HOOKDIR_WARN:
    -+			strbuf_addstr(&hookdir_annotation, _(" (will warn but run)"));
    -+			break;
    -+		case HOOKDIR_YES:
    -+		/*
    -+		 * The default behavior should agree with
    -+		 * hook.c:configured_hookdir_opt(). HOOKDIR_UNKNOWN should just
    -+		 * do the default behavior.
    -+		 */
    -+		case HOOKDIR_UNKNOWN:
    -+		default:
    -+			break;
    -+	}
    -+
    - 	list_for_each(pos, head) {
      		struct hook *item = list_entry(pos, struct hook, list);
      		item = list_entry(pos, struct hook, list);
      		if (item) {
    - 			/* Don't translate 'hookdir' - it matches the config */
    --			printf("%s: %s\n",
    -+			printf("%s: %s%s\n",
    - 			       (item->from_hookdir
    - 				? "hookdir"
    - 				: config_scope_name(item->origin)),
    +-			/*
    +-			 * TRANSLATORS: "<config scope>: <path>". Both fields
    +-			 * should be left untranslated; config scope matches the
    +-			 * output of 'git config --show-scope'. Marked for
    +-			 * translation to provide better RTL support later.
    +-			 */
    +-			printf(_("%s: %s\n"),
    +-			       (item->from_hookdir
    +-				? "hookdir"
    +-				: config_scope_name(item->origin)),
     -			       item->command.buf);
    -+			       item->command.buf,
    -+			       (item->from_hookdir
    -+				? hookdir_annotation.buf
    -+				: ""));
    ++			if (item->from_hookdir) {
    ++				/*
    ++				 * TRANSLATORS: do not translate 'hookdir' as
    ++				 * it matches the config setting.
    ++				 */
    ++				switch (should_run_hookdir) {
    ++				case HOOKDIR_NO:
    ++					printf(_("hookdir: %s (will not run)\n"),
    ++					       item->command.buf);
    ++					break;
    ++				case HOOKDIR_ERROR:
    ++					printf(_("hookdir: %s (will error and not run)\n"),
    ++					       item->command.buf);
    ++					break;
    ++				case HOOKDIR_INTERACTIVE:
    ++					printf(_("hookdir: %s (will prompt)\n"),
    ++					       item->command.buf);
    ++					break;
    ++				case HOOKDIR_WARN:
    ++					printf(_("hookdir: %s (will warn but run)\n"),
    ++					       item->command.buf);
    ++					break;
    ++				case HOOKDIR_YES:
    ++				/*
    ++				 * The default behavior should agree with
    ++				 * hook.c:configured_hookdir_opt(). HOOKDIR_UNKNOWN should just
    ++				 * do the default behavior.
    ++				 */
    ++				case HOOKDIR_UNKNOWN:
    ++				default:
    ++					printf(_("hookdir: %s\n"),
    ++						 item->command.buf);
    ++					break;
    ++				}
    ++			} else {
    ++				/*
    ++				 * TRANSLATORS: "<config scope>: <path>". Both fields
    ++				 * should be left untranslated; config scope matches the
    ++				 * output of 'git config --show-scope'. Marked for
    ++				 * translation to provide better RTL support later.
    ++				 */
    ++				printf(_("%s: %s\n"),
    ++					config_scope_name(item->origin),
    ++					item->command.buf);
    ++			}
      		}
      	}
      
      	clear_hook_list(head);
     +	strbuf_release(&hookdir_annotation);
    - 	strbuf_release(&hookname);
      
      	return 0;
    -@@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
    + }
      
      int cmd_hook(int argc, const char **argv, const char *prefix)
      {
    @@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
     +		else if (!strcmp(run_hookdir, "interactive"))
     +			should_run_hookdir = HOOKDIR_INTERACTIVE;
     +		else
    ++			/*
    ++			 * TRANSLATORS: leave "yes/warn/interactive/no"
    ++			 * untranslated; the strings are compared literally.
    ++			 */
     +			die(_("'%s' is not a valid option for --run-hookdir "
     +			      "(yes, warn, interactive, no)"), run_hookdir);
     +	else
    @@ hook.c: static int hook_config_lookup(const char *key, const char *value, void *
     +	return HOOKDIR_UNKNOWN;
     +}
     +
    - struct list_head* hook_list(const struct strbuf* hookname)
    + struct list_head* hook_list(const char* hookname)
      {
      	struct strbuf hook_key = STRBUF_INIT;
     
      ## hook.h ##
     @@ hook.h: struct hook {
       */
    - struct list_head* hook_list(const struct strbuf *hookname);
    + struct list_head* hook_list(const char *hookname);
      
     +enum hookdir_opt
     +{
    @@ hook.h: struct hook {
     + * command line arguments.
     + */
     +enum hookdir_opt configured_hookdir_opt(void);
    ++
    ++/*
    ++ * Provides the hookdir_opt specified in the config without consulting any
    ++ * command line arguments.
    ++ */
    ++enum hookdir_opt configured_hookdir_opt(void);
     +
      /* Free memory associated with a 'struct hook' */
      void free_hook(struct hook *ptr);
    @@ t/t1360-config-based-hooks.sh: test_expect_success 'git hook list shows hooks fr
     +
     +	git hook list pre-commit >actual &&
     +	# the hookdir annotation is translated
    -+	test_i18ncmp expected actual
    ++	test_cmp expected actual
     +'
     +
     +test_expect_success 'hook.runHookDir = error is respected by list' '
    @@ t/t1360-config-based-hooks.sh: test_expect_success 'git hook list shows hooks fr
     +
     +	git hook list pre-commit >actual &&
     +	# the hookdir annotation is translated
    -+	test_i18ncmp expected actual
    ++	test_cmp expected actual
     +'
     +
     +test_expect_success 'hook.runHookDir = warn is respected by list' '
    @@ t/t1360-config-based-hooks.sh: test_expect_success 'git hook list shows hooks fr
     +
     +	git hook list pre-commit >actual &&
     +	# the hookdir annotation is translated
    -+	test_i18ncmp expected actual
    ++	test_cmp expected actual
     +'
     +
     +
    @@ t/t1360-config-based-hooks.sh: test_expect_success 'git hook list shows hooks fr
     +
     +	git hook list pre-commit >actual &&
     +	# the hookdir annotation is translated
    -+	test_i18ncmp expected actual
    ++	test_cmp expected actual
     +'
     +
     +test_expect_success 'hook.runHookDir is tolerant to unknown values' '
    @@ t/t1360-config-based-hooks.sh: test_expect_success 'git hook list shows hooks fr
     +
     +	git hook list pre-commit >actual &&
     +	# the hookdir annotation is translated
    -+	test_i18ncmp expected actual
    ++	test_cmp expected actual
     +'
     +
      test_done
 6:  e86025853a !  5:  0a4b9f27b3 hook: implement hookcmd.<name>.skip
    @@ hook.c: static int hook_config_lookup(const char *key, const char *value, void *
     
      ## t/t1360-config-based-hooks.sh ##
     @@ t/t1360-config-based-hooks.sh: test_expect_success 'hook.runHookDir = warn is respected by list' '
    - 	test_i18ncmp expected actual
    + 	test_cmp expected actual
      '
      
     +test_expect_success 'git hook list removes skipped hookcmd' '
    @@ t/t1360-config-based-hooks.sh: test_expect_success 'hook.runHookDir = warn is re
     +	EOF
     +
     +	git hook list pre-commit >actual &&
    -+	test_i18ncmp expected actual
    ++	test_cmp expected actual
     +'
     +
     +test_expect_success 'git hook list ignores skip referring to unused hookcmd' '
    @@ t/t1360-config-based-hooks.sh: test_expect_success 'hook.runHookDir = warn is re
     +	EOF
     +
     +	git hook list pre-commit >actual &&
    -+	test_i18ncmp expected actual
    ++	test_cmp expected actual
     +'
     +
     +test_expect_success 'git hook list removes skipped inlined hook' '
 7:  6e10593d75 !  6:  2ad4f44d08 parse-options: parse into strvec
    @@ Commit message
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
     
    +
    + ## Notes ##
    +    Since v7, updated the reference doc to make the intended usage for OPT_STRVEC
    +    more clear.
    +
    +    Since v4, fixed one or two more places where I missed the argv_array->strvec
    +    rename.
    +
      ## Documentation/technical/api-parse-options.txt ##
     @@ Documentation/technical/api-parse-options.txt: There are some macros to easily define options:
      	The string argument is stored as an element in `string_list`.
    @@ parse-options.h: int parse_opt_commits(const struct option *, const char *, int)
      int parse_opt_noop_cb(const struct option *, const char *, int);
      enum parse_opt_result parse_opt_unknown_cb(struct parse_opt_ctx_t *ctx,
      					   const struct option *,
    +
    + ## t/helper/test-parse-options.c ##
    +@@
    + #include "cache.h"
    + #include "parse-options.h"
    + #include "string-list.h"
    ++#include "strvec.h"
    + #include "trace2.h"
    + 
    + static int boolean = 0;
    +@@ t/helper/test-parse-options.c: static char *string = NULL;
    + static char *file = NULL;
    + static int ambiguous;
    + static struct string_list list = STRING_LIST_INIT_NODUP;
    ++static struct strvec vector = STRVEC_INIT;
    + 
    + static struct {
    + 	int called;
    +@@ t/helper/test-parse-options.c: int cmd__parse_options(int argc, const char **argv)
    + 		OPT_STRING('o', NULL, &string, "str", "get another string"),
    + 		OPT_NOOP_NOARG(0, "obsolete"),
    + 		OPT_STRING_LIST(0, "list", &list, "str", "add str to list"),
    ++		OPT_STRVEC(0, "vector", &vector, "str", "add str to strvec"),
    + 		OPT_GROUP("Magic arguments"),
    + 		OPT_ARGUMENT("quux", NULL, "means --quux"),
    + 		OPT_NUMBER_CALLBACK(&integer, "set integer to NUM",
    +@@ t/helper/test-parse-options.c: int cmd__parse_options(int argc, const char **argv)
    + 	for (i = 0; i < list.nr; i++)
    + 		show(&expect, &ret, "list: %s", list.items[i].string);
    + 
    ++	for (i = 0; i < vector.nr; i++)
    ++		show(&expect, &ret, "vector: %s", vector.v[i]);
    ++
    + 	for (i = 0; i < argc; i++)
    + 		show(&expect, &ret, "arg %02d: %s", i, argv[i]);
    + 
    +
    + ## t/t0040-parse-options.sh ##
    +@@ t/t0040-parse-options.sh: String options
    +     --st <st>             get another string (pervert ordering)
    +     -o <str>              get another string
    +     --list <str>          add str to list
    ++    --vector <str>        add str to strvec
    + 
    + Magic arguments
    +     --quux                means --quux
    +@@ t/t0040-parse-options.sh: test_expect_success '--no-list resets list' '
    + 	test_cmp expect output
    + '
    + 
    ++cat >expect <<\EOF
    ++boolean: 0
    ++integer: 0
    ++magnitude: 0
    ++timestamp: 0
    ++string: (not set)
    ++abbrev: 7
    ++verbose: -1
    ++quiet: 0
    ++dry run: no
    ++file: (not set)
    ++vector: foo
    ++vector: bar
    ++vector: baz
    ++EOF
    ++test_expect_success '--vector keeps list of strings' '
    ++	test-tool parse-options --vector foo --vector=bar --vector=baz >output &&
    ++	test_cmp expect output
    ++'
    ++
    ++test_expect_success '--no-vector resets list' '
    ++	test-tool parse-options --vector=other --vector=irrelevant --vector=options \
    ++		--no-vector --vector=foo --vector=bar --vector=baz >output &&
    ++	test_cmp expect output
    ++'
    ++
    + test_expect_success 'multiple quiet levels' '
    + 	test-tool parse-options --expect="quiet: 3" -q -q -q
    + '
 8:  0dc9284057 !  7:  27dd8e3edf hook: add 'run' subcommand
    @@ Commit message
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
     
    +
    + ## Notes ##
    +    Since v7, added support for "error" hook.runHookDir setting.
    +
    +    Since v4, updated the docs, and did less local application of single
    +    quotes. In order for hookdir hooks to run successfully with a space in
    +    the path, though, they must not be run with 'sh -c'. So we can treat the
    +    hookdir hooks specially, and warn users via doc about special
    +    considerations for configured hooks with spaces in their path.
    +
      ## Documentation/git-hook.txt ##
     @@ Documentation/git-hook.txt: SYNOPSIS
      --------
    @@ hook.c
      
      void free_hook(struct hook *ptr)
      {
    +-	if (ptr) {
    ++	if (ptr)
    + 		strbuf_release(&ptr->command);
    +-		free(ptr);
    +-	}
    ++	free(ptr);
    + }
    + 
    + static struct hook * find_hook_by_command(struct list_head *head, const char *command)
     @@ hook.c: enum hookdir_opt configured_hookdir_opt(void)
      	return HOOKDIR_UNKNOWN;
      }
    @@ hook.c: enum hookdir_opt configured_hookdir_opt(void)
     +
     +	switch (cfg)
     +	{
    -+		case HOOKDIR_ERROR:
    -+			fprintf(stderr, _("Skipping legacy hook at '%s'\n"),
    -+				path);
    -+			/* FALLTHROUGH */
    -+		case HOOKDIR_NO:
    -+			return 0;
    -+		case HOOKDIR_WARN:
    -+			fprintf(stderr, _("Running legacy hook at '%s'\n"),
    -+				path);
    -+			return 1;
    -+		case HOOKDIR_INTERACTIVE:
    -+			do {
    -+				/*
    -+				 * TRANSLATORS: Make sure to include [Y] and [n]
    -+				 * in your translation. Only English input is
    -+				 * accepted. Default option is "yes".
    -+				 */
    -+				fprintf(stderr, _("Run '%s'? [Yn] "), path);
    -+				git_read_line_interactively(&prompt);
    -+				strbuf_tolower(&prompt);
    -+				if (starts_with(prompt.buf, "n")) {
    -+					strbuf_release(&prompt);
    -+					return 0;
    -+				} else if (starts_with(prompt.buf, "y")) {
    -+					strbuf_release(&prompt);
    -+					return 1;
    -+				}
    -+				/* otherwise, we didn't understand the input */
    -+			} while (prompt.len); /* an empty reply means "Yes" */
    -+			strbuf_release(&prompt);
    -+			return 1;
    -+		/*
    -+		 * HOOKDIR_UNKNOWN should match the default behavior, but let's
    -+		 * give a heads up to the user.
    -+		 */
    -+		case HOOKDIR_UNKNOWN:
    -+			fprintf(stderr,
    -+				_("Unrecognized value for 'hook.runHookDir'. "
    -+				  "Is there a typo? "));
    -+			/* FALLTHROUGH */
    -+		case HOOKDIR_YES:
    -+		default:
    -+			return 1;
    ++	case HOOKDIR_ERROR:
    ++		fprintf(stderr, _("Skipping legacy hook at '%s'\n"),
    ++			path);
    ++		/* FALLTHROUGH */
    ++	case HOOKDIR_NO:
    ++		return 0;
    ++	case HOOKDIR_WARN:
    ++		fprintf(stderr, _("Running legacy hook at '%s'\n"),
    ++			path);
    ++		return 1;
    ++	case HOOKDIR_INTERACTIVE:
    ++		do {
    ++			/*
    ++			 * TRANSLATORS: Make sure to include [Y] and [n]
    ++			 * in your translation. Only English input is
    ++			 * accepted. Default option is "yes".
    ++			 */
    ++			fprintf(stderr, _("Run '%s'? [Y/n] "), path);
    ++			git_read_line_interactively(&prompt);
    ++			/*
    ++			 * In case of prompt = '' - that is, user hit enter,
    ++			 * saying "yes I want the default" - strncasecmp will
    ++			 * return 0 regardless. So list the default first.
    ++			 *
    ++			 * Case insensitively, accept "y", "ye", or "yes" as
    ++			 * "yes"; accept "n" or "no" as "no".
    ++			 */
    ++			if (!strncasecmp(prompt.buf, "yes", prompt.len)) {
    ++				strbuf_release(&prompt);
    ++				return 1;
    ++			} else if (!strncasecmp(prompt.buf, "no", prompt.len)) {
    ++				strbuf_release(&prompt);
    ++				return 0;
    ++			}
    ++			/* otherwise, we didn't understand the input */
    ++		} while (prompt.len); /* an empty reply means default (yes) */
    ++		return 1;
    ++	/*
    ++	 * HOOKDIR_UNKNOWN should match the default behavior, but let's
    ++	 * give a heads up to the user.
    ++	 */
    ++	case HOOKDIR_UNKNOWN:
    ++		fprintf(stderr,
    ++			_("Unrecognized value for 'hook.runHookDir'. "
    ++			  "Is there a typo? "));
    ++		/* FALLTHROUGH */
    ++	case HOOKDIR_YES:
    ++	default:
    ++		return 1;
     +	}
     +}
     +
    - struct list_head* hook_list(const struct strbuf* hookname)
    + struct list_head* hook_list(const char* hookname)
      {
      	struct strbuf hook_key = STRBUF_INIT;
    -@@ hook.c: struct list_head* hook_list(const struct strbuf* hookname)
    +@@ hook.c: struct list_head* hook_list(const char* hookname)
      	strbuf_release(&hook_key);
      	return hook_head;
      }
    @@ hook.c: struct list_head* hook_list(const struct strbuf* hookname)
     +	strvec_clear(&o->args);
     +}
     +
    -+static void prepare_hook_cp(struct hook *hook, struct run_hooks_opt *options,
    ++static void prepare_hook_cp(const char *hookname, struct hook *hook,
    ++			    struct run_hooks_opt *options,
     +			    struct child_process *cp)
     +{
     +	if (!hook)
    @@ hook.c: struct list_head* hook_list(const struct strbuf* hookname)
     +	cp->no_stdin = 1;
     +	cp->env = options->env.v;
     +	cp->stdout_to_stderr = 1;
    -+	cp->trace2_hook_name = hook->command.buf;
    ++	cp->trace2_hook_name = hookname;
     +
     +	/*
     +	 * Commands from the config could be oneliners, but we know
    @@ hook.c: struct list_head* hook_list(const struct strbuf* hookname)
     +
     +int run_hooks(const char *hookname, struct run_hooks_opt *options)
     +{
    -+	struct strbuf hookname_str = STRBUF_INIT;
     +	struct list_head *to_run, *pos = NULL, *tmp = NULL;
     +	int rc = 0;
     +
     +	if (!options)
     +		BUG("a struct run_hooks_opt must be provided to run_hooks");
     +
    -+	strbuf_addstr(&hookname_str, hookname);
    -+
    -+	to_run = hook_list(&hookname_str);
    ++	to_run = hook_list(hookname);
     +
     +	list_for_each_safe(pos, tmp, to_run) {
     +		struct child_process hook_proc = CHILD_PROCESS_INIT;
    @@ hook.c: struct list_head* hook_list(const struct strbuf* hookname)
     +		    !should_include_hookdir(hook->command.buf, options->run_hookdir))
     +			continue;
     +
    -+		prepare_hook_cp(hook, options, &hook_proc);
    ++		prepare_hook_cp(hookname, hook, options, &hook_proc);
     +
     +		rc |= run_command(&hook_proc);
     +	}
    @@ hook.h: enum hookdir_opt
     +void run_hooks_opt_init(struct run_hooks_opt *o);
     +void run_hooks_opt_clear(struct run_hooks_opt *o);
     +
    -+/*
    + /*
    +- * Provides the hookdir_opt specified in the config without consulting any
    +- * command line arguments.
     + * Runs all hooks associated to the 'hookname' event in order. Each hook will be
     + * passed 'env' and 'args'.
    -+ */
    +  */
    +-enum hookdir_opt configured_hookdir_opt(void);
     +int run_hooks(const char *hookname, struct run_hooks_opt *options);
    -+
    + 
      /* Free memory associated with a 'struct hook' */
      void free_hook(struct hook *ptr);
    - /* Empties the list at 'head', calling 'free_hook()' on each entry */
     
      ## t/t1360-config-based-hooks.sh ##
     @@ t/t1360-config-based-hooks.sh: test_expect_success 'hook.runHookDir = no is respected by list' '
      
      	git hook list pre-commit >actual &&
      	# the hookdir annotation is translated
    --	test_i18ncmp expected actual
    -+	test_i18ncmp expected actual &&
    +-	test_cmp expected actual
    ++	test_cmp expected actual &&
     +
     +	git hook run pre-commit 2>actual &&
     +	test_must_be_empty actual
    @@ t/t1360-config-based-hooks.sh: test_expect_success 'hook.runHookDir = error is r
      
      	git hook list pre-commit >actual &&
      	# the hookdir annotation is translated
    -+	test_i18ncmp expected actual &&
    ++	test_cmp expected actual &&
     +
     +	cat >expected <<-EOF &&
     +	Skipping legacy hook at '\''$(pwd)/.git/hooks/pre-commit'\''
     +	EOF
     +
     +	git hook run pre-commit 2>actual &&
    - 	test_i18ncmp expected actual
    + 	test_cmp expected actual
      '
      
     @@ t/t1360-config-based-hooks.sh: test_expect_success 'hook.runHookDir = warn is respected by list' '
      
      	git hook list pre-commit >actual &&
      	# the hookdir annotation is translated
    -+	test_i18ncmp expected actual &&
    ++	test_cmp expected actual &&
     +
     +	cat >expected <<-EOF &&
     +	Running legacy hook at '\''$(pwd)/.git/hooks/pre-commit'\''
    @@ t/t1360-config-based-hooks.sh: test_expect_success 'hook.runHookDir = warn is re
     +	EOF
     +
     +	git hook run pre-commit 2>actual &&
    - 	test_i18ncmp expected actual
    + 	test_cmp expected actual
      '
      
     @@ t/t1360-config-based-hooks.sh: test_expect_success 'git hook list removes skipped inlined hook' '
    @@ t/t1360-config-based-hooks.sh: test_expect_success 'hook.runHookDir = interactiv
      
      	git hook list pre-commit >actual &&
      	# the hookdir annotation is translated
    --	test_i18ncmp expected actual
    -+	test_i18ncmp expected actual &&
    ++	test_cmp expected actual &&
     +
     +	test_write_lines n | git hook run pre-commit 2>actual &&
     +	! grep "Legacy Hook" actual &&
    @@ t/t1360-config-based-hooks.sh: test_expect_success 'hook.runHookDir = interactiv
     +	test_cmp expected actual
     +'
     +
    ++test_expect_success 'git hook run can pass args and env vars' '
    ++	write_script sample-hook.sh <<-\EOF &&
    ++	echo $1
    ++	echo $2
    ++	echo $TEST_ENV_1
    ++	echo $TEST_ENV_2
    ++	EOF
    ++
    ++	test_config hook.pre-commit.command "\"$(pwd)/sample-hook.sh\"" &&
    ++
    ++	cat >expected <<-EOF &&
    ++	arg1
    ++	arg2
    ++	env1
    ++	env2
    ++	EOF
    ++
    ++	git hook run --arg arg1 \
    ++		--env TEST_ENV_1=env1 \
    ++		-a arg2 \
    ++		-e TEST_ENV_2=env2 \
    ++		pre-commit 2>actual &&
    ++
    + 	test_cmp expected actual
    + '
    + 
     +test_expect_success 'hookdir hook included in git hook run' '
     +	setup_hookdir &&
     +
    @@ t/t1360-config-based-hooks.sh: test_expect_success 'hook.runHookDir = interactiv
     +	setup_hooks &&
     +
     +	nongit test_must_fail git hook run pre-commit
    - '
    - 
    ++'
    ++
      test_expect_success 'hook.runHookDir is tolerant to unknown values' '
    + 	setup_hookdir &&
    + 
 9:  92c67ed9da !  8:  46975c11c8 hook: introduce hook_exists()
    @@ hook.c: void run_hooks_opt_init(struct run_hooks_opt *o)
      	strvec_clear(&o->env);
     
      ## hook.h ##
    -@@ hook.h: struct list_head* hook_list(const struct strbuf *hookname);
    +@@ hook.h: struct list_head* hook_list(const char *hookname);
      
      enum hookdir_opt
      {
10:  9b3bb0b655 !  9:  e11f9e28a3 hook: support passing stdin to hooks
    @@ hook.c: void run_hooks_opt_init(struct run_hooks_opt *o)
      	o->run_hookdir = configured_hookdir_opt();
      }
      
    -@@ hook.c: static void prepare_hook_cp(struct hook *hook, struct run_hooks_opt *options,
    +@@ hook.c: static void prepare_hook_cp(const char *hookname, struct hook *hook,
      	if (!hook)
      		return;
      
    @@ hook.c: static void prepare_hook_cp(struct hook *hook, struct run_hooks_opt *opt
     +
      	cp->env = options->env.v;
      	cp->stdout_to_stderr = 1;
    - 	cp->trace2_hook_name = hook->command.buf;
    + 	cp->trace2_hook_name = hookname;
     
      ## hook.h ##
     @@ hook.h: struct run_hooks_opt
    @@ hook.h: int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdi
     
      ## t/t1360-config-based-hooks.sh ##
     @@ t/t1360-config-based-hooks.sh: test_expect_success 'hook.runHookDir is tolerant to unknown values' '
    - 	test_i18ncmp expected actual
    + 	test_cmp expected actual
      '
      
     +test_expect_success 'stdin to multiple hooks' '
11:  9933985e78 = 10:  3ceb4156fd run-command: allow stdin for run_processes_parallel
12:  43caafe656 ! 11:  93a47f5242 hook: allow parallel hook execution
    @@ Commit message
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
     
    +
    + ## Notes ##
    +    Per AEvar's request - parallel hook execution on day zero.
    +
    +    In most ways run_processes_parallel() worked great for me - but it didn't
    +    have great support for hooks where we pipe to and from. I had to add this
    +    support later in the series.
    +
    +    Since I modified an existing and in-use library I'd appreciate a keen look on
    +    these patches.
    +
    +     - Emily
    +
      ## Documentation/config/hook.txt ##
     @@ Documentation/config/hook.txt: hook.runHookDir::
      	Controls how hooks contained in your hookdir are executed. Can be any of
    @@ hook.c: enum hookdir_opt configured_hookdir_opt(void)
      static int should_include_hookdir(const char *path, enum hookdir_opt cfg)
      {
      	struct strbuf prompt = STRBUF_INIT;
    -@@ hook.c: struct list_head* hook_list(const struct strbuf* hookname)
    +@@ hook.c: struct list_head* hook_list(const char* hookname)
      	return hook_head;
      }
      
    @@ hook.c: void run_hooks_opt_clear(struct run_hooks_opt *o)
      	strvec_clear(&o->args);
      }
      
    --static void prepare_hook_cp(struct hook *hook, struct run_hooks_opt *options,
    +-static void prepare_hook_cp(const char *hookname, struct hook *hook,
    +-			    struct run_hooks_opt *options,
     -			    struct child_process *cp)
     +static int pick_next_hook(struct child_process *cp,
     +			  struct strbuf *out,
    @@ hook.c: void run_hooks_opt_clear(struct run_hooks_opt *o)
     -	cp->env = options->env.v;
     +	cp->env = hook_cb->options->env.v;
      	cp->stdout_to_stderr = 1;
    - 	cp->trace2_hook_name = hook->command.buf;
    +-	cp->trace2_hook_name = hookname;
    ++	cp->trace2_hook_name = hook_cb->hookname;
      
    -@@ hook.c: static void prepare_hook_cp(struct hook *hook, struct run_hooks_opt *options,
    + 	/*
    + 	 * Commands from the config could be oneliners, but we know
    +@@ hook.c: static void prepare_hook_cp(const char *hookname, struct hook *hook,
      	 * add passed-in argv, without expanding - let the user get back
      	 * exactly what they put in
      	 */
    @@ hook.c: static void prepare_hook_cp(struct hook *hook, struct run_hooks_opt *opt
      
      int run_hooks(const char *hookname, struct run_hooks_opt *options)
      {
    - 	struct strbuf hookname_str = STRBUF_INIT;
      	struct list_head *to_run, *pos = NULL, *tmp = NULL;
     -	int rc = 0;
    -+	struct hook_cb_data cb_data = { 0, NULL, NULL, options };
    ++	struct hook_cb_data cb_data = { 0, hookname, NULL, NULL, options };
      
      	if (!options)
      		BUG("a struct run_hooks_opt must be provided to run_hooks");
     @@ hook.c: int run_hooks(const char *hookname, struct run_hooks_opt *options)
    - 	to_run = hook_list(&hookname_str);
    + 	to_run = hook_list(hookname);
      
      	list_for_each_safe(pos, tmp, to_run) {
     -		struct child_process hook_proc = CHILD_PROCESS_INIT;
    @@ hook.c: int run_hooks(const char *hookname, struct run_hooks_opt *options)
     +	if (list_empty(to_run))
     +		return 0;
      
    --		prepare_hook_cp(hook, options, &hook_proc);
    +-		prepare_hook_cp(hookname, hook, options, &hook_proc);
     +	cb_data.head = to_run;
     +	cb_data.run_me = list_entry(to_run->next, struct hook, list);
      
    @@ hook.h: struct run_hooks_opt
     + */
     +struct hook_cb_data {
     +	int rc;
    ++	const char *hookname;
     +	struct list_head *head;
     +	struct hook *run_me;
     +	struct run_hooks_opt *options;
13:  2e189a7566 ! 12:  7f8c886d3f hook: allow specifying working directory for hooks
    @@ Commit message
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
     
    +
    + ## Notes ##
    +    Needed later for "post-checkout" conversion.
    +
      ## hook.c ##
     @@ hook.c: void run_hooks_opt_init_sync(struct run_hooks_opt *o)
      	o->path_to_stdin = NULL;
    @@ hook.c: void run_hooks_opt_init_sync(struct run_hooks_opt *o)
     @@ hook.c: static int pick_next_hook(struct child_process *cp,
      	cp->env = hook_cb->options->env.v;
      	cp->stdout_to_stderr = 1;
    - 	cp->trace2_hook_name = hook->command.buf;
    + 	cp->trace2_hook_name = hook_cb->hookname;
     +	cp->dir = hook_cb->options->dir;
      
      	/*
14:  07899ad346 = 13:  8930baa9db run-command: add stdin callback for parallelization
15:  d3f18e433f ! 14:  d0f362591a hook: provide stdin by string_list or callback
    @@ Commit message
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
     
      ## hook.c ##
    -@@ hook.c: void free_hook(struct hook *ptr)
    +@@
    + 
    + void free_hook(struct hook *ptr)
      {
    - 	if (ptr) {
    +-	if (ptr)
    ++	if (ptr) {
      		strbuf_release(&ptr->command);
     +		free(ptr->feed_pipe_cb_data);
    - 		free(ptr);
    - 	}
    ++	}
    + 	free(ptr);
      }
    + 
     @@ hook.c: static void append_or_move_hook(struct list_head *head, const char *command)
      		strbuf_init(&to_add->command, 0);
      		strbuf_addstr(&to_add->command, command);
    @@ hook.c: int run_hooks(const char *hookname, struct run_hooks_opt *options)
     +	if (options->path_to_stdin && options->feed_pipe)
     +		BUG("choose only one method to populate stdin");
     +
    - 	strbuf_addstr(&hookname_str, hookname);
    + 	to_run = hook_list(hookname);
      
    - 	to_run = hook_list(&hookname_str);
    + 	list_for_each_safe(pos, tmp, to_run) {
     @@ hook.c: int run_hooks(const char *hookname, struct run_hooks_opt *options)
      	run_processes_parallel_tr2(options->jobs,
      				   pick_next_hook,
16:  417c3f054e ! 15:  83bbb405a5 run-command: allow capturing of collated output
    @@ Commit message
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
     
    +
    + ## Notes ##
    +    Originally when writing this patch I attempted to use a pipe in memory -
    +    but managing its lifetime was actually pretty tricky, and I found I could
    +    achieve the same thing with less code by doing it this way. Critique welcome,
    +    including "no, you really need to do it with a pipe".
    +
      ## builtin/fetch.c ##
     @@ builtin/fetch.c: static int fetch_multiple(struct string_list *list, int max_children)
      		result = run_processes_parallel_tr2(max_children,
17:  c0f3471bd1 ! 16:  73ed5de54c hooks: allow callers to capture output
    @@ Commit message
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
     
    +
    + ## Notes ##
    +    You can see this in practice in the conversions for some of the push hooks,
    +    like 'receive-pack'.
    +
      ## hook.c ##
     @@ hook.c: void run_hooks_opt_init_sync(struct run_hooks_opt *o)
      	o->dir = NULL;
18:  13446e4273 = 17:  900c4d8963 commit: use config-based hooks
19:  9380c43180 ! 18:  a562120e22 am: convert applypatch hooks to use config
    @@ builtin/am.c: static void am_destroy(const struct am_state *state)
      {
      	int ret;
     +	struct run_hooks_opt opt;
    ++
     +	run_hooks_opt_init_sync(&opt);
      
      	assert(state->msg);
    @@ builtin/am.c: static void do_commit(const struct am_state *state)
      	const char *reflog_msg, *author, *committer = NULL;
      	struct strbuf sb = STRBUF_INIT;
     +	struct run_hooks_opt hook_opt;
    ++
     +	run_hooks_opt_init_async(&hook_opt);
      
     -	if (run_hook_le(NULL, "pre-applypatch", NULL))
    -+	if (run_hooks("pre-applypatch", &hook_opt))
    ++	if (run_hooks("pre-applypatch", &hook_opt)) {
    ++		run_hooks_opt_clear(&hook_opt);
      		exit(1);
    ++	}
    ++
    ++	run_hooks_opt_clear(&hook_opt);
      
      	if (write_cache_as_tree(&tree, 0, NULL))
    + 		die(_("git write-tree failed to write a tree"));
     @@ builtin/am.c: static void do_commit(const struct am_state *state)
      		fclose(fp);
      	}
      
     -	run_hook_le(NULL, "post-applypatch", NULL);
    ++	run_hooks_opt_init_async(&hook_opt);
     +	run_hooks("post-applypatch", &hook_opt);
      
     +	run_hooks_opt_clear(&hook_opt);
20:  316a605606 ! 19:  e841ed4384 merge: use config-based hooks for post-merge hook
    @@ builtin/merge.c: static void finish(struct commit *head_commit,
      	struct strbuf reflog_message = STRBUF_INIT;
     +	struct run_hooks_opt opt;
      	const struct object_id *head = &head_commit->object.oid;
    -+	run_hooks_opt_init_async(&opt);
      
      	if (!msg)
    - 		strbuf_addstr(&reflog_message, getenv("GIT_REFLOG_ACTION"));
     @@ builtin/merge.c: static void finish(struct commit *head_commit,
      	}
      
      	/* Run a post-merge hook */
     -	run_hook_le(NULL, "post-merge", squash ? "1" : "0", NULL);
    ++	run_hooks_opt_init_async(&opt);
     +	strvec_push(&opt.args, squash ? "1" : "0");
     +	run_hooks("post-merge", &opt);
     +	run_hooks_opt_clear(&opt);
21:  a5132f14b3 ! 20:  7e99398f7d gc: use hook library for pre-auto-gc hook
    @@ builtin/gc.c: static void add_repack_incremental_option(void)
      static int need_to_gc(void)
      {
     +	struct run_hooks_opt hook_opt;
    -+	run_hooks_opt_init_async(&hook_opt);
    ++
      	/*
      	 * Setting gc.auto to 0 or negative can disable the
      	 * automatic gc.
    @@ builtin/gc.c: static int need_to_gc(void)
      		return 0;
      
     -	if (run_hook_le(NULL, "pre-auto-gc", NULL))
    -+	if (run_hooks("pre-auto-gc", &hook_opt))
    ++	run_hooks_opt_init_async(&hook_opt);
    ++	if (run_hooks("pre-auto-gc", &hook_opt)) {
    ++		run_hooks_opt_clear(&hook_opt);
      		return 0;
    ++	}
    ++	run_hooks_opt_clear(&hook_opt);
      	return 1;
      }
    + 
22:  01f1331cc9 ! 21:  5423217ef2 rebase: teach pre-rebase to use hook.h
    @@ builtin/rebase.c: int cmd_rebase(int argc, const char **argv, const char *prefix
      	struct option builtin_rebase_options[] = {
      		OPT_STRING(0, "onto", &options.onto_name,
      			   N_("revision"),
    -@@ builtin/rebase.c: int cmd_rebase(int argc, const char **argv, const char *prefix)
    - 	};
    - 	int i;
    - 
    -+	run_hooks_opt_init_async(&hook_opt);
    -+
    - 	if (argc == 2 && !strcmp(argv[1], "-h"))
    - 		usage_with_options(builtin_rebase_usage,
    - 				   builtin_rebase_options);
     @@ builtin/rebase.c: int cmd_rebase(int argc, const char **argv, const char *prefix)
      	}
      
      	/* If a hook exists, give it a chance to interrupt*/
    ++	run_hooks_opt_init_async(&hook_opt);
     +	strvec_pushl(&hook_opt.args, options.upstream_arg, argc ? argv[0] : NULL, NULL);
      	if (!ok_to_skip_pre_rebase &&
     -	    run_hook_le(NULL, "pre-rebase", options.upstream_arg,
     -			argc ? argv[0] : NULL, NULL))
    -+	    run_hooks("pre-rebase", &hook_opt))
    ++	    run_hooks("pre-rebase", &hook_opt)) {
    ++		run_hooks_opt_clear(&hook_opt);
      		die(_("The pre-rebase hook refused to rebase."));
    ++	}
    ++	run_hooks_opt_clear(&hook_opt);
      
      	if (options.flags & REBASE_DIFFSTAT) {
    -@@ builtin/rebase.c: int cmd_rebase(int argc, const char **argv, const char *prefix)
    - 	ret = !!run_specific_rebase(&options, action);
    - 
    - cleanup:
    -+	run_hooks_opt_clear(&hook_opt);
    - 	strbuf_release(&buf);
    - 	strbuf_release(&revisions);
    - 	free(options.head_name);
    + 		struct diff_options opts;
23:  85a7721adc ! 22:  1c0c5ad129 read-cache: convert post-index-change hook to use config
    @@ Documentation/githooks.txt: and "0" meaning they were not.
     
      ## read-cache.c ##
     @@
    - #include "fsmonitor.h"
      #include "thread-utils.h"
      #include "progress.h"
    + #include "sparse-index.h"
     +#include "hook.h"
    ++>>>>>>> 9524a9d29d (read-cache: convert post-index-change hook to use config)
      
      /* Mask for the name length in ce_flags in the on-disk index */
      
     @@ read-cache.c: static int do_write_locked_index(struct index_state *istate, struct lock_file *l
    - 				 unsigned flags)
      {
      	int ret;
    + 	int was_full = !istate->sparse_index;
     +	struct run_hooks_opt hook_opt;
    -+	run_hooks_opt_init_async(&hook_opt);
      
    - 	/*
    - 	 * TODO trace2: replace "the_repository" with the actual repo instance
    + 	ret = convert_to_sparse(istate);
    + 
     @@ read-cache.c: static int do_write_locked_index(struct index_state *istate, struct lock_file *l
      	else
      		ret = close_lock_file_gently(lock);
    @@ read-cache.c: static int do_write_locked_index(struct index_state *istate, struc
     -	run_hook_le(NULL, "post-index-change",
     -			istate->updated_workdir ? "1" : "0",
     -			istate->updated_skipworktree ? "1" : "0", NULL);
    ++	run_hooks_opt_init_async(&hook_opt);
     +	strvec_pushl(&hook_opt.args,
     +		     istate->updated_workdir ? "1" : "0",
     +		     istate->updated_skipworktree ? "1" : "0",
24:  21ec3e1a9d ! 23:  1193e856e6 receive-pack: convert push-to-checkout hook to hook.h
    @@ builtin/receive-pack.c: static const char *push_to_checkout(unsigned char *hash,
      				    const char *work_tree)
      {
     +	struct run_hooks_opt opt;
    ++
     +	run_hooks_opt_init_sync(&opt);
     +
      	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
25:  e0405e96ad ! 24:  1817b6851b git-p4: use 'git hook' to run hooks
    @@ Commit message
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
     
    +
    + ## Notes ##
    +    Maybe there is a better way to do this - I had a hard time getting this to run
    +    locally, and Python is not my forte, so if anybody has a better approach I'd
    +    love to just take that patch instead :)
    +
    +    Since v6, removed the developer debug print statements.... :X
    +
    +    Maybe there is a better way to do this - I had a hard time getting this to run
    +    locally, and Python is not my forte, so if anybody has a better approach I'd
    +    love to just take that patch instead :)
    +
      ## git-p4.py ##
     @@ git-p4.py: def decode_path(path):
      
26:  c52578e078 ! 25:  b3a354e4a8 hooks: convert 'post-checkout' hook to hook library
    @@ builtin/checkout.c: struct branch_info {
      			      int changed)
      {
     -	return run_hook_le(NULL, "post-checkout",
    --			   oid_to_hex(old_commit ? &old_commit->object.oid : &null_oid),
    --			   oid_to_hex(new_commit ? &new_commit->object.oid : &null_oid),
    +-			   oid_to_hex(old_commit ? &old_commit->object.oid : null_oid()),
    +-			   oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()),
     -			   changed ? "1" : "0", NULL);
     +	struct run_hooks_opt opt;
     +	int rc;
    @@ builtin/checkout.c: struct branch_info {
      	   a commit exists. */
     -
     +	strvec_pushl(&opt.args,
    -+		     oid_to_hex(old_commit ? &old_commit->object.oid : &null_oid),
    -+		     oid_to_hex(new_commit ? &new_commit->object.oid : &null_oid),
    ++		     oid_to_hex(old_commit ? &old_commit->object.oid : null_oid()),
    ++		     oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()),
     +		     changed ? "1" : "0",
     +		     NULL);
     +	rc = run_hooks("post-checkout", &opt);
    @@ builtin/clone.c: static int checkout(int submodule_progress)
      	struct tree_desc t;
      	int err = 0;
     +	struct run_hooks_opt hook_opt;
    -+	run_hooks_opt_init_sync(&hook_opt);
      
      	if (option_no_checkout)
      		return 0;
    @@ builtin/clone.c: static int checkout(int submodule_progress)
      	if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
      		die(_("unable to write new index file"));
      
    --	err |= run_hook_le(NULL, "post-checkout", oid_to_hex(&null_oid),
    +-	err |= run_hook_le(NULL, "post-checkout", oid_to_hex(null_oid()),
     -			   oid_to_hex(&oid), "1", NULL);
    -+	strvec_pushl(&hook_opt.args, oid_to_hex(&null_oid), oid_to_hex(&oid), "1", NULL);
    ++	run_hooks_opt_init_sync(&hook_opt);
    ++	strvec_pushl(&hook_opt.args, oid_to_hex(null_oid()), oid_to_hex(&oid), "1", NULL);
     +	err |= run_hooks("post-checkout", &hook_opt);
     +	run_hooks_opt_clear(&hook_opt);
      
    @@ builtin/worktree.c: static int add_worktree(const char *path, const char *refnam
     -			cp.argv = NULL;
     -			cp.trace2_hook_name = "post-checkout";
     -			strvec_pushl(&cp.args, absolute_path(hook),
    --				     oid_to_hex(&null_oid),
    +-				     oid_to_hex(null_oid()),
     -				     oid_to_hex(&commit->object.oid),
     -				     "1", NULL);
     -			ret = run_command(&cp);
     -		}
     +		struct run_hooks_opt opt;
    ++
     +		run_hooks_opt_init_sync(&opt);
     +
     +		strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
     +		strvec_pushl(&opt.args,
    -+			     oid_to_hex(&null_oid),
    ++			     oid_to_hex(null_oid()),
     +			     oid_to_hex(&commit->object.oid),
     +			     "1",
     +			     NULL);
    @@ builtin/worktree.c: static int add_worktree(const char *path, const char *refnam
      
      	strvec_clear(&child_env);
     
    + ## read-cache.c ##
    +@@
    + #include "progress.h"
    + #include "sparse-index.h"
    + #include "hook.h"
    +->>>>>>> 9524a9d29d (read-cache: convert post-index-change hook to use config)
    + 
    + /* Mask for the name length in ce_flags in the on-disk index */
    + 
    +
      ## reset.c ##
     @@
      #include "tree-walk.h"
    @@ reset.c: int reset_head(struct repository *r, struct object_id *oid, const char
      	}
     -	if (run_hook)
     -		run_hook_le(NULL, "post-checkout",
    --			    oid_to_hex(orig ? orig : &null_oid),
    +-			    oid_to_hex(orig ? orig : null_oid()),
     -			    oid_to_hex(oid), "1", NULL);
     +	if (run_hook) {
     +		struct run_hooks_opt opt;
    ++
     +		run_hooks_opt_init_sync(&opt);
     +		strvec_pushl(&opt.args,
    -+			     oid_to_hex(orig ? orig : &null_oid),
    ++			     oid_to_hex(orig ? orig : null_oid()),
     +			     oid_to_hex(oid),
     +			     "1",
     +			     NULL);
27:  316cb6f584 ! 26:  692352f9aa hook: convert 'post-rewrite' hook to config
    @@ builtin/am.c: static int run_applypatch_msg_hook(struct am_state *state)
     -	const char *hook = find_hook("post-rewrite");
     +	struct run_hooks_opt opt;
      	int ret;
    -+	run_hooks_opt_init_async(&opt);
      
     -	if (!hook)
     -		return 0;
    --
    ++	run_hooks_opt_init_async(&opt);
    + 
     -	strvec_push(&cp.args, hook);
     -	strvec_push(&cp.args, "rebase");
    --
    --	cp.in = xopen(am_path(state, "rewritten"), O_RDONLY);
    --	cp.stdout_to_stderr = 1;
    --	cp.trace2_hook_name = "post-rewrite";
     +	strvec_push(&opt.args, "rebase");
     +	opt.path_to_stdin = am_path(state, "rewritten");
      
    --	ret = run_command(&cp);
    +-	cp.in = xopen(am_path(state, "rewritten"), O_RDONLY);
    +-	cp.stdout_to_stderr = 1;
    +-	cp.trace2_hook_name = "post-rewrite";
     +	ret = run_hooks("post-rewrite", &opt);
      
    +-	ret = run_command(&cp);
    +-
     -	close(cp.in);
     +	run_hooks_opt_clear(&opt);
      	return ret;
    @@ sequencer.c: int update_head_with_reflog(const struct commit *old_head,
     +	struct string_list to_stdin = STRING_LIST_INIT_DUP;
      	int code;
     -	struct strbuf sb = STRBUF_INIT;
    -+	run_hooks_opt_init_async(&opt);
      
     -	argv[0] = find_hook("post-rewrite");
     -	if (!argv[0])
     -		return 0;
    -+	strvec_push(&opt.args, "amend");
    ++	run_hooks_opt_init_async(&opt);
      
     -	argv[1] = "amend";
     -	argv[2] = NULL;
    @@ sequencer.c: int update_head_with_reflog(const struct commit *old_head,
     -	strbuf_release(&sb);
     -	sigchain_pop(SIGPIPE);
     -	return finish_command(&proc);
    ++	strvec_push(&opt.args, "amend");
    ++
     +	strbuf_addf(&tmp,
     +		    "%s %s",
     +		    oid_to_hex(oldoid),
    @@ sequencer.c: static int pick_commits(struct repository *r,
     -			strvec_push(&child.args, "--for-rewrite=rebase");
     +			struct child_process notes_cp = CHILD_PROCESS_INIT;
     +			struct run_hooks_opt hook_opt;
    ++
     +			run_hooks_opt_init_async(&hook_opt);
     +
     +			notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY);
28:  0ab776068d = 27:  964011dfdd transport: convert pre-push hook to use config
29:  601dada804 ! 28:  c04822add9 reference-transaction: look for hooks in config
    @@ t/t1416-ref-transaction-hooks.sh: test_expect_success 'interleaving hook calls s
      	EOF
      
      	git push ./target-repo.git PRE POST &&
    +
    + ## transport.c ##
    +@@ transport.c: static int run_pre_push_hook(struct transport *transport,
    + 	struct strbuf tmp = STRBUF_INIT;
    + 	struct ref *r;
    + 	struct string_list to_stdin = STRING_LIST_INIT_DUP;
    ++
    + 	run_hooks_opt_init_async(&opt);
    + 
    + 	strvec_push(&opt.args, transport->remote->name);
30:  d60f2b146e ! 29:  ddc6f56bec receive-pack: convert 'update' hook to hook.h
    @@ builtin/receive-pack.c: static int run_receive_hook(struct command *commands,
      	return status;
      }
      
    --static int run_update_hook(struct command *cmd)
     +static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
    - {
    --	const char *argv[5];
    --	struct child_process proc = CHILD_PROCESS_INIT;
    --	int code;
    ++{
     +	int keepalive_active = 0;
    - 
    --	argv[0] = find_hook("update");
    --	if (!argv[0])
    --		return 0;
    ++
     +	if (keepalive_in_sec <= 0)
     +		use_keepalive = KEEPALIVE_NEVER;
     +	if (use_keepalive == KEEPALIVE_ALWAYS)
     +		keepalive_active = 1;
    - 
    --	argv[1] = cmd->ref_name;
    --	argv[2] = oid_to_hex(&cmd->old_oid);
    --	argv[3] = oid_to_hex(&cmd->new_oid);
    --	argv[4] = NULL;
    ++
     +	/* send a keepalive if there is no data to write */
     +	if (keepalive_active && !output->len) {
     +		static const char buf[] = "0005\1";
     +		write_or_die(1, buf, sizeof(buf) - 1);
     +		return;
     +	}
    - 
    --	proc.no_stdin = 1;
    --	proc.stdout_to_stderr = 1;
    --	proc.err = use_sideband ? -1 : 0;
    --	proc.argv = argv;
    --	proc.trace2_hook_name = "update";
    ++
     +	if (use_keepalive == KEEPALIVE_AFTER_NUL && !keepalive_active) {
     +		const char *first_null = memchr(output->buf, '\0', output->len);
     +		if (first_null) {
    @@ builtin/receive-pack.c: static int run_receive_hook(struct command *commands,
     +	send_sideband(1, 2, output->buf, output->len, use_sideband);
     +}
     +
    -+static int run_update_hook(struct command *cmd)
    -+{
    + static int run_update_hook(struct command *cmd)
    + {
    +-	const char *argv[5];
    +-	struct child_process proc = CHILD_PROCESS_INIT;
     +	struct run_hooks_opt opt;
    -+	int code;
    + 	int code;
    + 
    +-	argv[0] = find_hook("update");
    +-	if (!argv[0])
    +-		return 0;
     +	run_hooks_opt_init_async(&opt);
    -+
    + 
    +-	argv[1] = cmd->ref_name;
    +-	argv[2] = oid_to_hex(&cmd->old_oid);
    +-	argv[3] = oid_to_hex(&cmd->new_oid);
    +-	argv[4] = NULL;
     +	strvec_pushl(&opt.args,
     +		     cmd->ref_name,
     +		     oid_to_hex(&cmd->old_oid),
     +		     oid_to_hex(&cmd->new_oid),
     +		     NULL);
      
    +-	proc.no_stdin = 1;
    +-	proc.stdout_to_stderr = 1;
    +-	proc.err = use_sideband ? -1 : 0;
    +-	proc.argv = argv;
    +-	proc.trace2_hook_name = "update";
    +-
     -	code = start_command(&proc);
     -	if (code)
     -		return code;
31:  1e6898670b ! 30:  e1e810869f proc-receive: acquire hook list from hook.h
    @@ builtin/receive-pack.c: static int run_proc_receive_hook(struct command *command
      
     -	argv[0] = find_hook("proc-receive");
     -	if (!argv[0]) {
    -+	struct strbuf hookname = STRBUF_INIT;
     +	struct hook *proc_receive = NULL;
     +	struct list_head *pos, *hooks;
     +
    -+	strbuf_addstr(&hookname, "proc-receive");
    -+	hooks = hook_list(&hookname);
    ++	hooks = hook_list("proc-receive");
     +
     +	list_for_each(pos, hooks) {
     +		if (proc_receive) {
32:  012e3a7a79 ! 31:  b8be5a2288 post-update: use hook.h library
    @@ builtin/receive-pack.c: static const char *update(struct command *cmd, struct sh
      	struct command *cmd;
     -	struct child_process proc = CHILD_PROCESS_INIT;
     -	const char *hook;
    --
    ++	struct run_hooks_opt opt;
    + 
     -	hook = find_hook("post-update");
     -	if (!hook)
     -		return;
    -+	struct run_hooks_opt opt;
     +	run_hooks_opt_init_async(&opt);
      
      	for (cmd = commands; cmd; cmd = cmd->next) {
33:  2740bcda6d = 32:  1cc1384eae receive-pack: convert receive hooks to hook.h
34:  f201f3af5f = 33:  1bb9a3810c bugreport: use hook_exists instead of find_hook
35:  0956a94cc7 ! 34:  3db7bf3b0d git-send-email: use 'git hook run' for 'sendemail-validate'
    @@ git-send-email.perl: sub unique_email_list {
      	my ($fn, $xfer_encoding) = @_;
      
     -	if ($repo) {
    --		my $validate_hook = catfile(catdir($repo->repo_path(), 'hooks'),
    +-		my $validate_hook = catfile($repo->hooks_path(),
     -					    'sendemail-validate');
     -		my $hook_error;
     -		if (-x $validate_hook) {
    @@ git-send-email.perl: sub unique_email_list {
     -			chdir($repo->wc_path() or $repo->repo_path())
     -				or die("chdir: $!");
     -			local $ENV{"GIT_DIR"} = $repo->repo_path();
    --			$hook_error = "rejected by sendemail-validate hook"
    --				if system($validate_hook, $target);
    +-			$hook_error = system_or_msg([$validate_hook, $target]);
     -			chdir($cwd_save) or die("chdir: $!");
     -		}
    --		return $hook_error if $hook_error;
    +-		if ($hook_error) {
    +-			die sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
    +-				       "%s\n" .
    +-				       "warning: no patches were sent\n"), $fn, $hook_error);
    +-		}
     -	}
     +	my $target = abs_path($fn);
    -+	return "rejected by sendemail-validate hook"
    -+		if system(("git", "hook", "run", "sendemail-validate", "-a",
    -+				$target));
    ++
    ++	system_or_die(["git", "hook", "run", "sendemail-validate", "-j1", "-a", $target],
    ++		sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
    ++			   "warning: no patches were sent\n"),
    ++		        $fn));
      
      	# Any long lines will be automatically fixed if we use a suitable transfer
      	# encoding.
     
      ## t/t9001-send-email.sh ##
    +@@ t/t9001-send-email.sh: test_expect_success $PREREQ "--validate respects relative core.hooksPath path" '
    + 	test_path_is_file my-hooks.ran &&
    + 	cat >expect <<-EOF &&
    + 	fatal: longline.patch: rejected by sendemail-validate hook
    +-	fatal: command '"'"'$PWD/my-hooks/sendemail-validate'"'"' died with exit code 1
    + 	warning: no patches were sent
    + 	EOF
    + 	test_cmp expect actual
    +@@ t/t9001-send-email.sh: test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" '
    + 	test_path_is_file my-hooks.ran &&
    + 	cat >expect <<-EOF &&
    + 	fatal: longline.patch: rejected by sendemail-validate hook
    +-	fatal: command '"'"'$PWD/my-hooks/sendemail-validate'"'"' died with exit code 1
    + 	warning: no patches were sent
    + 	EOF
    + 	test_cmp expect actual
     @@ t/t9001-send-email.sh: test_expect_success $PREREQ 'invoke hook' '
      	mkdir -p .git/hooks &&
      
36:  7f05b25392 ! 35:  d2a477d9e3 run-command: stop thinking about hooks
    @@ hook.c: static int should_include_hookdir(const char *path, enum hookdir_opt cfg
     +}
     +
     +
    - struct list_head* hook_list(const struct strbuf* hookname)
    + struct list_head* hook_list(const char* hookname)
      {
      	struct strbuf hook_key = STRBUF_INIT;
    -@@ hook.c: struct list_head* hook_list(const struct strbuf* hookname)
    +@@ hook.c: struct list_head* hook_list(const char* hookname)
      	git_config(hook_config_lookup, &cb_data);
      
      	if (have_git_dir()) {
    --		const char *legacy_hook_path = find_hook(hookname->buf);
    -+		const char *legacy_hook_path = find_legacy_hook(hookname->buf);
    +-		const char *legacy_hook_path = find_hook(hookname);
    ++		const char *legacy_hook_path = find_legacy_hook(hookname);
      
      		/* Unconditionally add legacy hook, but annotate it. */
      		if (legacy_hook_path) {
37:  e9b1f847f2 <  -:  ---------- docs: unify githooks and git-hook manpages
 -:  ---------- > 36:  62a3e3b419 doc: clarify fsmonitor-watchman specification
 -:  ---------- > 37:  5c864de1aa docs: link githooks and git-hook manpages

-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH v9 01/37] doc: propose hooks managed by the config
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 02/37] hook: introduce git-hook subcommand Emily Shaffer
                     ` (37 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Begin a design document for config-based hooks, managed via git-hook.
Focus on an overview of the implementation and motivation for design
decisions. Briefly discuss the alternatives considered before this
point. Also, attempt to redefine terms to fit into a multihook world.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

Notes:
    Since v6, checked for inconsistencies with implementation and added lots of
    caveats about whether 'git hook add' and 'git hook edit' will ever materialize.
    
    Hopefully this reflects reality now; please review accordingly.
    
    Since v6, checked for inconsistencies with implementation and added lots of
    caveats about whether 'git hook add' and 'git hook edit' will ever materialize.
    
    Hopefully this reflects reality now; please review accordingly.
    
    Since v4, addressed comments from Jonathan Tan about wording. However, I have
    not addressed AEvar's comments or done a full re-review of this document.
    I wanted to get the rest of the series out for initial review first.
    
     - Emily
    
    Since v4, addressed comments from Jonathan Tan about wording.

 Documentation/Makefile                        |   1 +
 .../technical/config-based-hooks.txt          | 369 ++++++++++++++++++
 2 files changed, 370 insertions(+)
 create mode 100644 Documentation/technical/config-based-hooks.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 2aae4c9cbb..5d19eddb0e 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -90,6 +90,7 @@ SP_ARTICLES += $(API_DOCS)
 TECH_DOCS += MyFirstContribution
 TECH_DOCS += MyFirstObjectWalk
 TECH_DOCS += SubmittingPatches
+TECH_DOCS += technical/config-based-hooks
 TECH_DOCS += technical/hash-function-transition
 TECH_DOCS += technical/http-protocol
 TECH_DOCS += technical/index-format
diff --git a/Documentation/technical/config-based-hooks.txt b/Documentation/technical/config-based-hooks.txt
new file mode 100644
index 0000000000..1f973117e4
--- /dev/null
+++ b/Documentation/technical/config-based-hooks.txt
@@ -0,0 +1,369 @@
+Configuration-based hook management
+===================================
+:sectanchors:
+
+[[motivation]]
+== Motivation
+
+Replace the `.git/hook/hookname` path as the only source of hooks to execute;
+allow users to define hooks using config files, in a way which is friendly to
+users with multiple repos which have similar needs - hooks can be easily shared
+between multiple Git repos.
+
+Redefine "hook" as an event rather than a single script, allowing users to
+perform multiple unrelated actions on a single event.
+
+Make it easier for users to discover Git's hook feature and automate their
+workflows.
+
+[[user-interfaces]]
+== User interfaces
+
+[[config-schema]]
+=== Config schema
+
+Hooks can be introduced by editing the configuration manually. There are two new
+sections added, `hook` and `hookcmd`.
+
+[[config-schema-hook]]
+==== `hook`
+
+Primarily contains subsections for each hook event. The order of variables in
+these subsections defines the hook command execution order; hook commands can be
+specified by setting the value directly to the command if no additional
+configuration is needed, or by setting the value as the name of a `hookcmd`. If
+Git does not find a `hookcmd` whose subsection matches the value of the given
+command string, Git will try to execute the string directly. Hooks are executed
+by passing the resolved command string to the shell. In the future, hook event
+subsections could also contain per-hook-event settings; see
+<<per-hook-event-settings,the section in Future Work>> for more details.
+
+Also contains top-level hook execution settings, for example, `hook.runHookDir`.
+(These settings are described more in <<library,Library>>.)
+
+----
+[hook "pre-commit"]
+  command = perl-linter
+  command = /usr/bin/git-secrets --pre-commit
+
+[hook "pre-applypatch"]
+  command = perl-linter
+  # for illustration purposes; error behavior isn't planned yet
+  error = ignore
+
+[hook]
+  runHookDir = interactive
+----
+
+[[config-schema-hookcmd]]
+==== `hookcmd`
+
+Defines a hook command and its attributes, which will be used when a hook event
+occurs. Unqualified attributes are assumed to apply to this hook during all hook
+events, but event-specific attributes can also be supplied. The example runs
+`/usr/bin/lint-it --language=perl <args passed by Git>`, but for repos which
+include this config, the hook command will be skipped for all events.
+Theoretically, the last line could be used to "un-skip" the hook command for
+`pre-commit` hooks, but this hasn't been scoped or implemented yet.
+
+----
+[hookcmd "perl-linter"]
+  command = /usr/bin/lint-it --language=perl
+  skip = true
+  # for illustration purposes; below hasn't been defined yet
+  pre-commit-skip = false
+----
+
+[[command-line-api]]
+=== Command-line API
+
+Users should be able to view, run, reorder, and create hook commands via the
+command line. External tools should be able to view a list of hooks in the
+correct order to run. Modifier commands (`edit` and `add`) have not been
+implemented yet and may not be if manually editing the config proves usable
+enough.
+
+*`git hook list <hook-event>`*
+
+*`git hook run <hook-event> [-a <arg>]... [-e <env-var>]...`*
+
+*`git hook edit <hook-event>`*
+
+*`git hook add <hook-command> <hook-event> <options...>`*
+
+[[hook-editor]]
+=== Hook editor
+
+The tool which is presented by `git hook edit <hook-command>`. Ideally, this
+tool should be easier to use than manually editing the config, and then produce
+a concise config afterwards. It may take a form similar to `git rebase
+--interactive`. This has not been designed or implemented yet and may not be if
+the config proves usable enough.
+
+[[implementation]]
+== Implementation
+
+[[library]]
+=== Library
+
+`hook.c` and `hook.h` are responsible for interacting with the config files. The
+hook library provides a basic API to call all hooks in config order with more
+complex options passed via `struct run_hooks_opt`:
+
+*`int run_hooks(const char *hookname, struct run_hooks_opt *options)`*
+
+`struct run_hooks_opt` allows callers to set:
+
+- environment variables
+- command-line arguments
+- behavior for the hook command provided by `run-command.h:find_hook()` (see
+  below)
+- a method to provide stdin to each hook, either via a file containing stdin, a
+  `struct string_list` containing a list of lines to print, or a callback
+  function to allow the caller to populate stdin manually
+- a method to process stdout from each hook, e.g. for printing to sideband
+  during a network operation
+- parallelism
+- a custom working directory for hooks to execute in
+
+And this struct can be extended with more options as necessary in the future.
+
+The "legacy" hook provided by `run-command.h:find_hook()` - that is, the hook
+present in `.git/hooks/<hookname>` or
+`$(git config --get core.hooksPath)/<hookname>` - can be handled in a number of
+ways, providing an avenue to deprecate these "legacy" hooks if desired. The
+handling is based on a config `hook.runHookDir`, which is checked against a
+number of cases:
+
+- "no": the legacy hook will not be run
+- "error": Git will print a warning to stderr before ignoring the legacy hook
+- "interactive": Git will prompt the user before running the legacy hook
+- "warn": Git will print a warning to stderr before running the legacy hook
+- "yes" (default): Git will silently run the legacy hook
+
+In case this list is expanded in the future, if a value for `hook.runHookDir` is
+given which Git does not recognize, Git should discard that config entry. For
+example, if "warn" was specified at system level and "junk" was specified at
+global level, Git would resolve the value to "warn"; if the only time the config
+was set was to "junk", Git would use the default value of "yes" (but print a
+warning to the user first to let them know their value is wrong).
+
+`struct hookcmd` is expected to grow in size over time as more functionality is
+added to hooks; so that other parts of the code don't need to understand the
+config schema, `struct hookcmd` should contain logical values instead of string
+pairs.
+
+By default, hook parallelism is chosen based on the semantics of each hook;
+callsites initialize their `struct run_hooks_opt` via one of two macros,
+`RUN_HOOKS_OPT_INIT_SYNC` or `RUN_HOOKS_OPT_INIT_ASYNC`. The default number of
+jobs can be configured in `hook.jobs`; this config applies across all hook
+events. If unset, the value of `online_cpus()` (equivalent to `nproc`) is used.
+
+[[builtin]]
+=== Builtin
+
+`builtin/hook.c` is responsible for providing the frontend. It's responsible for
+formatting user-provided data and then calling the library API to set the
+configs as appropriate. The builtin frontend is not responsible for calling the
+config directly, so that other areas of Git can rely on the hook library to
+understand the most recent config schema for hooks.
+
+[[migration]]
+=== Migration path
+
+[[stage-0]]
+==== Stage 0
+
+Hooks are called by running `run-command.h:find_hook()` with the hookname and
+executing the result. The hook library and builtin do not exist. Hooks only
+exist as specially named scripts within `.git/hooks/`.
+
+[[stage-1]]
+==== Stage 1
+
+`git hook list --porcelain <hook-event>` is implemented. `hook.h:run_hooks()` is
+taught to include `run-command.h:find_hook()` at the end; calls to `find_hook()`
+are replaced with calls to `run_hooks()`. Users can opt-in to config-based hooks
+simply by creating some in their config; otherwise users should remain
+unaffected by the change.
+
+[[stage-2]]
+==== Stage 2
+
+The call to `find_hook()` inside of `run_hooks()` learns to check for a config,
+`hook.runHookDir`. Users can opt into managing their hooks completely via the
+config this way.
+
+[[stage-3]]
+==== Stage 3
+
+`.git/hooks` is removed from the template and the hook directory is considered
+deprecated. To avoid breaking older repos, the default of `hook.runHookDir` is
+not changed, and `find_hook()` is not removed.
+
+[[caveats]]
+== Caveats
+
+[[security]]
+=== Security and repo config
+
+Part of the motivation behind this refactor is to mitigate hooks as an attack
+vector.footnote:[https://lore.kernel.org/git/20171002234517.GV19555@aiede.mtv.corp.google.com/]
+However, as the design stands, users can still provide hooks in the repo-level
+config, which is included when a repo is zipped and sent elsewhere. The
+security of the repo-level config is still under discussion; this design
+generally assumes the repo-level config is secure, which is not true yet. This
+assumption was made to avoid overcomplicating the design. So, this series
+doesn't particularly improve security or resistance to zip attacks.
+
+[[ease-of-use]]
+=== Ease of use
+
+The config schema is nontrivial; that's why it's important for the `git hook`
+modifier commands to be usable. Contributors with UX expertise are encouraged to
+share their suggestions.
+
+[[alternatives]]
+== Alternative approaches
+
+A previous summary of alternatives exists in the
+archives.footnote:[https://lore.kernel.org/git/20191116011125.GG22855@google.com]
+
+The table below shows a number of goals and how they might be achieved with
+config-based hooks, by implementing directory support (i.e.
+'.git/hooks/pre-commit.d'), or as hooks are run today.
+
+.Comparison of alternatives
+|===
+|Feature |Config-based hooks |Hook directories |Status quo
+
+|Supports multiple hooks
+|Natively
+|Natively
+|With user effort
+
+|Supports parallelization
+|Natively
+|Natively
+|No (user's multihook trampoline script would need to handle parallelism)
+
+|Safer for zipped repos
+|A little
+|No
+|No
+
+|Previous hooks just work
+|If configured
+|Yes
+|Yes
+
+|Can install one hook to many repos
+|Yes
+|With symlinks or core.hooksPath
+|With symlinks or core.hooksPath
+
+|Discoverability
+|Findable with 'git help git' or tab-completion via 'git hook' subcommand
+|Findable via improved documentation
+|Same as before
+
+|Hard to run unexpected hook
+|If configured
+|Could be made to warn or look for a config
+|No
+|===
+
+[[status-quo]]
+=== Status quo
+
+Today users can implement multihooks themselves by using a "trampoline script"
+as their hook, and pointing that script to a directory or list of other scripts
+they wish to run.
+
+[[hook-directories]]
+=== Hook directories
+
+Other contributors have suggested Git learn about the existence of a directory
+such as `.git/hooks/<hookname>.d` and execute those hooks in alphabetical order.
+
+[[future-work]]
+== Future work
+
+[[execution-ordering]]
+=== Execution ordering
+
+We may find that config order is insufficient for some users; for example,
+config order makes it difficult to add a new hook to the system or global config
+which runs at the end of the hook list. A new ordering schema should be:
+
+1) Specified by a `hook.order` config, so that users will not unexpectedly see
+their order change;
+
+2) Either dependency or numerically based.
+
+Dependency-based ordering is prone to classic linked-list problems, like a
+cycles and handling of missing dependencies. But, it paves the way for enabling
+parallelization if some tasks truly depend on others.
+
+Numerical ordering makes it tricky for Git to generate suggested ordering
+numbers for each command, but is easy to determine a definitive order.
+
+[[parallelization]]
+=== Parallelization with dependencies
+
+Currently hooks use a naive parallelization scheme or are run in series.  But if
+one hook depends on another's output, then users will want to specify those
+dependencies. If we decide to solve this problem, we may want to look to modern
+build systems for inspiration on how to manage dependencies and parallel tasks.
+
+[[nontrivial-hooks]]
+=== Multihooks and nontrivial output
+
+Some hooks - like 'proc-receive' - don't lend themselves well to multihooks at
+all. In the case of 'proc-receive', for now, multiple hook definitions are
+disallowed. In the future we might be able to conceive a better approach, for
+example, running the hooks in series and using the output from one hook as the
+input to the next.
+
+[[securing-hookdir-hooks]]
+=== Securing hookdir hooks
+
+With the design as written in this doc, it's still possible for a malicious user
+to modify `.git/config` to include `hook.pre-receive.command = rm -rf /`, then
+zip their repo and send it to another user. It may be necessary to teach Git to
+only allow inlined hooks like this if they were configured outside of the local
+scope (in other words, only run hookcmds, and only allow hookcmds to be
+configured in global or system scope); or another approach, like a list of safe
+projects, might be useful. It may also be sufficient (or at least useful) to
+teach a `hook.disableAll` config or similar flag to the Git executable.
+
+[[submodule-inheritance]]
+=== Submodule inheritance
+
+It's possible some submodules may want to run the identical set of hooks that
+their superrepo runs. While a globally-configured hook set is helpful, it's not
+a great solution for users who have multiple repos-with-submodules under the
+same user. It would be useful for submodules to learn how to run hooks from
+their superrepo's config, or inherit that hook setting.
+
+[[per-hook-event-settings]]
+=== Per-hook-event settings
+
+It might be desirable to keep settings specifically for some hook events, but
+not for others - for example, a user may wish to disable hookdir hooks for all
+events but pre-commit, which they haven't had time to convert yet; or, a user
+may wish for execution order settings to differ based on hook event. In that
+case, it would be useful to set something like `hook.pre-commit.executionOrder`
+which would not apply to the 'prepare-commit-msg' hook, for example.
+
+[[glossary]]
+== Glossary
+
+*hook event*
+
+A point during Git's execution where user scripts may be run, for example,
+_prepare-commit-msg_ or _pre-push_.
+
+*hook command*
+
+A user script or executable which will be run on one or more hook events.
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 02/37] hook: introduce git-hook subcommand
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 01/37] doc: propose hooks managed by the config Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  2:18     ` Junio C Hamano
  2021-05-27  0:08   ` [PATCH v9 03/37] hook: include hookdir hook in list Emily Shaffer
                     ` (36 subsequent siblings)
  38 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Add a new subcommand, git-hook, which will be used to ease config-based
hook management. This command will handle parsing configs to compose a
list of hooks to run for a given event, as well as adding or modifying
hook configs in an interactive fashion.

Start with 'git hook list <hookname>', which checks the known configs in
order to create an ordered list of hooks to run on a given hook event.

Multiple commands can be specified for a given hook by providing
multiple "hook.<hookname>.command = <path-to-hook>" lines. Hooks will be
run in config order. If more properties need to be set on a given hook
in the future, commands can also be specified by providing
"hook.<hookname>.command = <hookcmd-name>", as well as a "[hookcmd
<hookcmd-name>]" subsection; this subsection should contain a
"hookcmd.<hookcmd-name>.command = <path-to-hook>" line.

For example:

  $ git config --list | grep ^hook
  hook.pre-commit.command=baz
  hook.pre-commit.command=~/bar.sh
  hookcmd.baz.command=~/baz/from/hookcmd.sh

  $ git hook list pre-commit
  global: ~/baz/from/hookcmd.sh
  local: ~/bar.sh

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

Notes:
    Since v4, mainly changed to RUN_SETUP_GENTLY so that 'git hook list' can
    be executed outside of a repo.

 .gitignore                    |   1 +
 Documentation/config/hook.txt |   9 +++
 Documentation/git-hook.txt    |  73 +++++++++++++++++++++
 Makefile                      |   2 +
 builtin.h                     |   1 +
 builtin/hook.c                |  65 ++++++++++++++++++
 command-list.txt              |   1 +
 git.c                         |   1 +
 hook.c                        | 120 ++++++++++++++++++++++++++++++++++
 hook.h                        |  25 +++++++
 t/t1360-config-based-hooks.sh |  88 +++++++++++++++++++++++++
 11 files changed, 386 insertions(+)
 create mode 100644 Documentation/config/hook.txt
 create mode 100644 Documentation/git-hook.txt
 create mode 100644 builtin/hook.c
 create mode 100644 hook.c
 create mode 100644 hook.h
 create mode 100755 t/t1360-config-based-hooks.sh

diff --git a/.gitignore b/.gitignore
index 311841f9be..de39dc9961 100644
--- a/.gitignore
+++ b/.gitignore
@@ -77,6 +77,7 @@
 /git-grep
 /git-hash-object
 /git-help
+/git-hook
 /git-http-backend
 /git-http-fetch
 /git-http-push
diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
new file mode 100644
index 0000000000..71449ecbc7
--- /dev/null
+++ b/Documentation/config/hook.txt
@@ -0,0 +1,9 @@
+hook.<command>.command::
+	A command to execute during the <command> hook event. This can be an
+	executable on your device, a oneliner for your shell, or the name of a
+	hookcmd. See linkgit:git-hook[1].
+
+hookcmd.<name>.command::
+	A command to execute during a hook for which <name> has been specified
+	as a command. This can be an executable on your device or a oneliner for
+	your shell. See linkgit:git-hook[1].
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
new file mode 100644
index 0000000000..f19875ed68
--- /dev/null
+++ b/Documentation/git-hook.txt
@@ -0,0 +1,73 @@
+git-hook(1)
+===========
+
+NAME
+----
+git-hook - Manage configured hooks
+
+SYNOPSIS
+--------
+[verse]
+'git hook' list <hook-name>
+
+DESCRIPTION
+-----------
+You can list configured hooks with this command. Later, you will be able to run,
+add, and modify hooks with this command.
+
+This command parses the default configuration files for sections `hook` and
+`hookcmd`. `hook` is used to describe the commands which will be run during a
+particular hook event; commands are run in the order Git encounters them during
+the configuration parse (see linkgit:git-config[1]). `hookcmd` is used to
+describe attributes of a specific command. If additional attributes don't need
+to be specified, a command to run can be specified directly in the `hook`
+section; if a `hookcmd` by that name isn't found, Git will attempt to run the
+provided value directly. For example:
+
+Global config
+----
+  [hook "post-commit"]
+    command = "linter"
+    command = "~/typocheck.sh"
+
+  [hookcmd "linter"]
+    command = "/bin/linter --c"
+----
+
+Local config
+----
+  [hook "prepare-commit-msg"]
+    command = "linter"
+  [hook "post-commit"]
+    command = "python ~/run-test-suite.py"
+----
+
+With these configs, you'd then see:
+
+----
+$ git hook list "post-commit"
+global: /bin/linter --c
+global: ~/typocheck.sh
+local: python ~/run-test-suite.py
+
+$ git hook list "prepare-commit-msg"
+local: /bin/linter --c
+----
+
+COMMANDS
+--------
+
+list `<hook-name>`::
+
+List the hooks which have been configured for `<hook-name>`. Hooks appear
+in the order they should be run, and print the config scope where the relevant
+`hook.<hook-name>.command` was specified, not the `hookcmd` (if applicable).
+This output is human-readable and the format is subject to change over time.
+
+CONFIGURATION
+-------------
+include::config/hook.txt[]
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index c3565fc0f8..a6b71a0fbe 100644
--- a/Makefile
+++ b/Makefile
@@ -901,6 +901,7 @@ LIB_OBJS += hash-lookup.o
 LIB_OBJS += hashmap.o
 LIB_OBJS += help.o
 LIB_OBJS += hex.o
+LIB_OBJS += hook.o
 LIB_OBJS += ident.o
 LIB_OBJS += json-writer.o
 LIB_OBJS += kwset.o
@@ -1101,6 +1102,7 @@ BUILTIN_OBJS += builtin/get-tar-commit-id.o
 BUILTIN_OBJS += builtin/grep.o
 BUILTIN_OBJS += builtin/hash-object.o
 BUILTIN_OBJS += builtin/help.o
+BUILTIN_OBJS += builtin/hook.o
 BUILTIN_OBJS += builtin/index-pack.o
 BUILTIN_OBJS += builtin/init-db.o
 BUILTIN_OBJS += builtin/interpret-trailers.o
diff --git a/builtin.h b/builtin.h
index 16ecd5586f..91740c1514 100644
--- a/builtin.h
+++ b/builtin.h
@@ -164,6 +164,7 @@ int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix);
 int cmd_grep(int argc, const char **argv, const char *prefix);
 int cmd_hash_object(int argc, const char **argv, const char *prefix);
 int cmd_help(int argc, const char **argv, const char *prefix);
+int cmd_hook(int argc, const char **argv, const char *prefix);
 int cmd_index_pack(int argc, const char **argv, const char *prefix);
 int cmd_init_db(int argc, const char **argv, const char *prefix);
 int cmd_interpret_trailers(int argc, const char **argv, const char *prefix);
diff --git a/builtin/hook.c b/builtin/hook.c
new file mode 100644
index 0000000000..79e150437e
--- /dev/null
+++ b/builtin/hook.c
@@ -0,0 +1,65 @@
+#include "cache.h"
+#include "builtin.h"
+#include "config.h"
+#include "hook.h"
+#include "parse-options.h"
+#include "strbuf.h"
+
+static const char * const builtin_hook_usage[] = {
+	N_("git hook list <hookname>"),
+	NULL
+};
+
+static int list(int argc, const char **argv, const char *prefix)
+{
+	struct list_head *head, *pos;
+	const char *hookname = NULL;
+
+	struct option list_options[] = {
+		OPT_END(),
+	};
+
+	argc = parse_options(argc, argv, prefix, list_options,
+			     builtin_hook_usage, 0);
+
+	if (argc < 1) {
+		usage_msg_opt(_("You must specify a hook event name to list."),
+			      builtin_hook_usage, list_options);
+	}
+
+	hookname = argv[0];
+
+	head = hook_list(hookname);
+
+	if (list_empty(head)) {
+		printf(_("no commands configured for hook '%s'\n"),
+		       hookname);
+		return 0;
+	}
+
+	list_for_each(pos, head) {
+		struct hook *item = list_entry(pos, struct hook, list);
+		if (item)
+			printf("%s: %s\n",
+			       config_scope_name(item->origin),
+			       item->command.buf);
+	}
+
+	clear_hook_list(head);
+
+	return 0;
+}
+
+int cmd_hook(int argc, const char **argv, const char *prefix)
+{
+	struct option builtin_hook_options[] = {
+		OPT_END(),
+	};
+	if (argc < 2)
+		usage_with_options(builtin_hook_usage, builtin_hook_options);
+
+	if (!strcmp(argv[1], "list"))
+		return list(argc - 1, argv + 1, prefix);
+
+	usage_with_options(builtin_hook_usage, builtin_hook_options);
+}
diff --git a/command-list.txt b/command-list.txt
index a289f09ed6..9ccd8e5aeb 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -103,6 +103,7 @@ git-grep                                mainporcelain           info
 git-gui                                 mainporcelain
 git-hash-object                         plumbingmanipulators
 git-help                                ancillaryinterrogators          complete
+git-hook                                mainporcelain
 git-http-backend                        synchingrepositories
 git-http-fetch                          synchelpers
 git-http-push                           synchelpers
diff --git a/git.c b/git.c
index 18bed9a996..39988ee3b0 100644
--- a/git.c
+++ b/git.c
@@ -538,6 +538,7 @@ static struct cmd_struct commands[] = {
 	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
 	{ "hash-object", cmd_hash_object },
 	{ "help", cmd_help },
+	{ "hook", cmd_hook, RUN_SETUP_GENTLY },
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "init", cmd_init_db },
 	{ "init-db", cmd_init_db },
diff --git a/hook.c b/hook.c
new file mode 100644
index 0000000000..d3e28aa73a
--- /dev/null
+++ b/hook.c
@@ -0,0 +1,120 @@
+#include "cache.h"
+
+#include "hook.h"
+#include "config.h"
+
+void free_hook(struct hook *ptr)
+{
+	if (ptr) {
+		strbuf_release(&ptr->command);
+		free(ptr);
+	}
+}
+
+static void append_or_move_hook(struct list_head *head, const char *command)
+{
+	struct list_head *pos = NULL, *tmp = NULL;
+	struct hook *to_add = NULL;
+
+	/*
+	 * remove the prior entry with this command; we'll replace it at the
+	 * end.
+	 */
+	list_for_each_safe(pos, tmp, head) {
+		struct hook *it = list_entry(pos, struct hook, list);
+		if (!strcmp(it->command.buf, command)) {
+		    list_del(pos);
+		    /* we'll simply move the hook to the end */
+		    to_add = it;
+		    break;
+		}
+	}
+
+	if (!to_add) {
+		/* adding a new hook, not moving an old one */
+		to_add = xmalloc(sizeof(*to_add));
+		strbuf_init(&to_add->command, 0);
+		strbuf_addstr(&to_add->command, command);
+	}
+
+	/* re-set the scope so we show where an override was specified */
+	to_add->origin = current_config_scope();
+
+	list_add_tail(&to_add->list, head);
+}
+
+static void remove_hook(struct list_head *to_remove)
+{
+	struct hook *hook_to_remove = list_entry(to_remove, struct hook, list);
+	list_del(to_remove);
+	free_hook(hook_to_remove);
+}
+
+void clear_hook_list(struct list_head *head)
+{
+	struct list_head *pos, *tmp;
+	list_for_each_safe(pos, tmp, head)
+		remove_hook(pos);
+}
+
+struct hook_config_cb
+{
+	struct strbuf *hookname;
+	struct list_head *list;
+};
+
+static int hook_config_lookup(const char *key, const char *value, void *cb_data)
+{
+	struct hook_config_cb *data = cb_data;
+	const char *hook_key = data->hookname->buf;
+	struct list_head *head = data->list;
+
+	if (!strcmp(key, hook_key)) {
+		const char *command = value;
+		struct strbuf hookcmd_name = STRBUF_INIT;
+
+		/*
+		 * Check if a hookcmd with that name exists. If it doesn't,
+		 * 'git_config_get_value()' is documented not to touch &command,
+		 * so we don't need to do anything.
+		 */
+		strbuf_addf(&hookcmd_name, "hookcmd.%s.command", command);
+		git_config_get_value(hookcmd_name.buf, &command);
+
+		if (!command) {
+			strbuf_release(&hookcmd_name);
+			BUG("git_config_get_value overwrote a string it shouldn't have");
+		}
+
+		/*
+		 * TODO: implement an option-getting callback, e.g.
+		 *   get configs by pattern hookcmd.$value.*
+		 *   for each key+value, do_callback(key, value, cb_data)
+		 */
+
+		append_or_move_hook(head, command);
+
+		strbuf_release(&hookcmd_name);
+	}
+
+	return 0;
+}
+
+struct list_head* hook_list(const char* hookname)
+{
+	struct strbuf hook_key = STRBUF_INIT;
+	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
+	struct hook_config_cb cb_data = { &hook_key, hook_head };
+
+	INIT_LIST_HEAD(hook_head);
+
+	if (!hookname)
+		return NULL;
+
+	strbuf_addf(&hook_key, "hook.%s.command", hookname);
+
+	git_config(hook_config_lookup, &cb_data);
+
+	strbuf_release(&hook_key);
+	return hook_head;
+}
diff --git a/hook.h b/hook.h
new file mode 100644
index 0000000000..042cab8446
--- /dev/null
+++ b/hook.h
@@ -0,0 +1,25 @@
+#include "config.h"
+#include "list.h"
+#include "strbuf.h"
+
+struct hook {
+	struct list_head list;
+	/*
+	 * Config file which holds the hook.*.command definition.
+	 * (This has nothing to do with the hookcmd.<name>.* configs.)
+	 */
+	enum config_scope origin;
+	/* The literal command to run. */
+	struct strbuf command;
+};
+
+/*
+ * Provides a linked list of 'struct hook' detailing commands which should run
+ * in response to the 'hookname' event, in execution order.
+ */
+struct list_head* hook_list(const char *hookname);
+
+/* Free memory associated with a 'struct hook' */
+void free_hook(struct hook *ptr);
+/* Empties the list at 'head', calling 'free_hook()' on each entry */
+void clear_hook_list(struct list_head *head);
diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh
new file mode 100755
index 0000000000..6e4a3e763f
--- /dev/null
+++ b/t/t1360-config-based-hooks.sh
@@ -0,0 +1,88 @@
+#!/bin/bash
+
+test_description='config-managed multihooks, including git-hook command'
+
+. ./test-lib.sh
+
+ROOT=
+if test_have_prereq MINGW
+then
+	# In Git for Windows, Unix-like paths work only in shell scripts;
+	# `git.exe`, however, will prefix them with the pseudo root directory
+	# (of the Unix shell). Let's accommodate for that.
+	ROOT="$(cd / && pwd)"
+fi
+
+setup_hooks () {
+	test_config hook.pre-commit.command "/path/ghi" --add
+	test_config_global hook.pre-commit.command "/path/def" --add
+}
+
+setup_hookcmd () {
+	test_config hook.pre-commit.command "abc" --add
+	test_config_global hookcmd.abc.command "/path/abc" --add
+}
+
+test_expect_success 'git hook rejects commands without a mode' '
+	test_must_fail git hook pre-commit
+'
+
+
+test_expect_success 'git hook rejects commands without a hookname' '
+	test_must_fail git hook list
+'
+
+test_expect_success 'git hook runs outside of a repo' '
+	setup_hooks &&
+
+	cat >expected <<-EOF &&
+	global: $ROOT/path/def
+	EOF
+
+	nongit git config --list --global &&
+
+	nongit git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list orders by config order' '
+	setup_hooks &&
+
+	cat >expected <<-EOF &&
+	global: $ROOT/path/def
+	local: $ROOT/path/ghi
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list dereferences a hookcmd' '
+	setup_hooks &&
+	setup_hookcmd &&
+
+	cat >expected <<-EOF &&
+	global: $ROOT/path/def
+	local: $ROOT/path/ghi
+	local: $ROOT/path/abc
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list reorders on duplicate commands' '
+	setup_hooks &&
+
+	test_config hook.pre-commit.command "/path/def" --add &&
+
+	cat >expected <<-EOF &&
+	local: $ROOT/path/ghi
+	local: $ROOT/path/def
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_done
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 03/37] hook: include hookdir hook in list
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 01/37] doc: propose hooks managed by the config Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 02/37] hook: introduce git-hook subcommand Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 04/37] hook: teach hook.runHookDir Emily Shaffer
                     ` (35 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Historically, hooks are declared by placing an executable into
$GIT_DIR/hooks/$HOOKNAME (or $HOOKDIR/$HOOKNAME). Although hooks taken
from the config are more featureful than hooks placed in the $HOOKDIR,
those hooks should not stop working for users who already have them.
Let's list them to the user, but instead of displaying a config scope
(e.g. "global: blah") we can prefix them with "hookdir:".

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 builtin/hook.c                | 18 +++++++++++++++---
 hook.c                        | 17 +++++++++++++++++
 hook.h                        |  1 +
 t/t1360-config-based-hooks.sh | 19 +++++++++++++++++++
 4 files changed, 52 insertions(+), 3 deletions(-)

diff --git a/builtin/hook.c b/builtin/hook.c
index 79e150437e..e82725f0a6 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -39,10 +39,20 @@ static int list(int argc, const char **argv, const char *prefix)
 
 	list_for_each(pos, head) {
 		struct hook *item = list_entry(pos, struct hook, list);
-		if (item)
-			printf("%s: %s\n",
-			       config_scope_name(item->origin),
+		item = list_entry(pos, struct hook, list);
+		if (item) {
+			/*
+			 * TRANSLATORS: "<config scope>: <path>". Both fields
+			 * should be left untranslated; config scope matches the
+			 * output of 'git config --show-scope'. Marked for
+			 * translation to provide better RTL support later.
+			 */
+			printf(_("%s: %s\n"),
+			       (item->from_hookdir
+				? "hookdir"
+				: config_scope_name(item->origin)),
 			       item->command.buf);
+		}
 	}
 
 	clear_hook_list(head);
@@ -58,6 +68,8 @@ int cmd_hook(int argc, const char **argv, const char *prefix)
 	if (argc < 2)
 		usage_with_options(builtin_hook_usage, builtin_hook_options);
 
+	git_config(git_default_config, NULL);
+
 	if (!strcmp(argv[1], "list"))
 		return list(argc - 1, argv + 1, prefix);
 
diff --git a/hook.c b/hook.c
index d3e28aa73a..b4994fc108 100644
--- a/hook.c
+++ b/hook.c
@@ -2,6 +2,7 @@
 
 #include "hook.h"
 #include "config.h"
+#include "run-command.h"
 
 void free_hook(struct hook *ptr)
 {
@@ -35,6 +36,7 @@ static void append_or_move_hook(struct list_head *head, const char *command)
 		to_add = xmalloc(sizeof(*to_add));
 		strbuf_init(&to_add->command, 0);
 		strbuf_addstr(&to_add->command, command);
+		to_add->from_hookdir = 0;
 	}
 
 	/* re-set the scope so we show where an override was specified */
@@ -115,6 +117,21 @@ struct list_head* hook_list(const char* hookname)
 
 	git_config(hook_config_lookup, &cb_data);
 
+	if (have_git_dir()) {
+		const char *legacy_hook_path = find_hook(hookname);
+
+		/* Unconditionally add legacy hook, but annotate it. */
+		if (legacy_hook_path) {
+			struct hook *legacy_hook;
+
+			append_or_move_hook(hook_head,
+					    absolute_path(legacy_hook_path));
+			legacy_hook = list_entry(hook_head->prev, struct hook,
+						 list);
+			legacy_hook->from_hookdir = 1;
+		}
+	}
+
 	strbuf_release(&hook_key);
 	return hook_head;
 }
diff --git a/hook.h b/hook.h
index 042cab8446..b6c5480325 100644
--- a/hook.h
+++ b/hook.h
@@ -11,6 +11,7 @@ struct hook {
 	enum config_scope origin;
 	/* The literal command to run. */
 	struct strbuf command;
+	unsigned from_hookdir : 1;
 };
 
 /*
diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh
index 6e4a3e763f..0f12af4659 100755
--- a/t/t1360-config-based-hooks.sh
+++ b/t/t1360-config-based-hooks.sh
@@ -23,6 +23,14 @@ setup_hookcmd () {
 	test_config_global hookcmd.abc.command "/path/abc" --add
 }
 
+setup_hookdir () {
+	mkdir .git/hooks
+	write_script .git/hooks/pre-commit <<-EOF
+	echo \"Legacy Hook\"
+	EOF
+	test_when_finished rm -rf .git/hooks
+}
+
 test_expect_success 'git hook rejects commands without a mode' '
 	test_must_fail git hook pre-commit
 '
@@ -85,4 +93,15 @@ test_expect_success 'git hook list reorders on duplicate commands' '
 	test_cmp expected actual
 '
 
+test_expect_success 'git hook list shows hooks from the hookdir' '
+	setup_hookdir &&
+
+	cat >expected <<-EOF &&
+	hookdir: $(pwd)/.git/hooks/pre-commit
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
 test_done
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 04/37] hook: teach hook.runHookDir
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (2 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 03/37] hook: include hookdir hook in list Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 05/37] hook: implement hookcmd.<name>.skip Emily Shaffer
                     ` (34 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

For now, just give a hint about how these hooks will be run in 'git hook
list'. Later on, though, we will pay attention to this enum when running
the hooks.
---

Notes:
    Since v7, tidied up the behavior of the HOOK_UNKNOWN flag and added a test to
    enforce it - now it matches the design doc much better.
    
    Also, thanks Jonathan Tan for pointing out that the commit message made no sense
    and was targeted for a different change. Rewrote the commit message now.
    
    Plus, added HOOK_ERROR flag per Junio and Jonathan Nieder.
    
    Newly split into its own commit since v4, and taking place much sooner.
    
    An unfortunate side effect of adding this support *before* the
    hook.runHookDir support is that the labels on the list are not clear -
    because we aren't yet flagging which hooks are from the hookdir versus
    the config. I suppose we could move the addition of that field to the
    struct hook up to this patch, but it didn't make a lot of sense to me to
    do it just for cosmetic purposes.
    
    Since v7, tidied up the behavior of the HOOK_UNKNOWN flag and added a test to
    enforce it - now it matches the design doc much better.
    
    Also, thanks Jonathan Tan for pointing out that the commit message made no sense
    and was targeted for a different change. Rewrote the commit message now.
    
    Newly split into its own commit since v4, and taking place much sooner.
    
    An unfortunate side effect of adding this support *before* the
    hook.runHookDir support is that the labels on the list are not clear -
    because we aren't yet flagging which hooks are from the hookdir versus
    the config. I suppose we could move the addition of that field to the
    struct hook up to this patch, but it didn't make a lot of sense to me to
    do it just for cosmetic purposes.

 Documentation/config/hook.txt |  5 ++
 builtin/hook.c                | 98 ++++++++++++++++++++++++++++++-----
 hook.c                        | 24 +++++++++
 hook.h                        | 22 ++++++++
 t/t1360-config-based-hooks.sh | 71 +++++++++++++++++++++++++
 5 files changed, 206 insertions(+), 14 deletions(-)

diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
index 71449ecbc7..75312754ae 100644
--- a/Documentation/config/hook.txt
+++ b/Documentation/config/hook.txt
@@ -7,3 +7,8 @@ hookcmd.<name>.command::
 	A command to execute during a hook for which <name> has been specified
 	as a command. This can be an executable on your device or a oneliner for
 	your shell. See linkgit:git-hook[1].
+
+hook.runHookDir::
+	Controls how hooks contained in your hookdir are executed. Can be any of
+	"yes", "warn", "interactive", or "no". Defaults to "yes". See
+	linkgit:git-hook[1] and linkgit:git-config[1] "core.hooksPath").
diff --git a/builtin/hook.c b/builtin/hook.c
index e82725f0a6..b1e63a9576 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -10,10 +10,13 @@ static const char * const builtin_hook_usage[] = {
 	NULL
 };
 
+static enum hookdir_opt should_run_hookdir;
+
 static int list(int argc, const char **argv, const char *prefix)
 {
 	struct list_head *head, *pos;
 	const char *hookname = NULL;
+	struct strbuf hookdir_annotation = STRBUF_INIT;
 
 	struct option list_options[] = {
 		OPT_END(),
@@ -41,37 +44,104 @@ static int list(int argc, const char **argv, const char *prefix)
 		struct hook *item = list_entry(pos, struct hook, list);
 		item = list_entry(pos, struct hook, list);
 		if (item) {
-			/*
-			 * TRANSLATORS: "<config scope>: <path>". Both fields
-			 * should be left untranslated; config scope matches the
-			 * output of 'git config --show-scope'. Marked for
-			 * translation to provide better RTL support later.
-			 */
-			printf(_("%s: %s\n"),
-			       (item->from_hookdir
-				? "hookdir"
-				: config_scope_name(item->origin)),
-			       item->command.buf);
+			if (item->from_hookdir) {
+				/*
+				 * TRANSLATORS: do not translate 'hookdir' as
+				 * it matches the config setting.
+				 */
+				switch (should_run_hookdir) {
+				case HOOKDIR_NO:
+					printf(_("hookdir: %s (will not run)\n"),
+					       item->command.buf);
+					break;
+				case HOOKDIR_ERROR:
+					printf(_("hookdir: %s (will error and not run)\n"),
+					       item->command.buf);
+					break;
+				case HOOKDIR_INTERACTIVE:
+					printf(_("hookdir: %s (will prompt)\n"),
+					       item->command.buf);
+					break;
+				case HOOKDIR_WARN:
+					printf(_("hookdir: %s (will warn but run)\n"),
+					       item->command.buf);
+					break;
+				case HOOKDIR_YES:
+				/*
+				 * The default behavior should agree with
+				 * hook.c:configured_hookdir_opt(). HOOKDIR_UNKNOWN should just
+				 * do the default behavior.
+				 */
+				case HOOKDIR_UNKNOWN:
+				default:
+					printf(_("hookdir: %s\n"),
+						 item->command.buf);
+					break;
+				}
+			} else {
+				/*
+				 * TRANSLATORS: "<config scope>: <path>". Both fields
+				 * should be left untranslated; config scope matches the
+				 * output of 'git config --show-scope'. Marked for
+				 * translation to provide better RTL support later.
+				 */
+				printf(_("%s: %s\n"),
+					config_scope_name(item->origin),
+					item->command.buf);
+			}
 		}
 	}
 
 	clear_hook_list(head);
+	strbuf_release(&hookdir_annotation);
 
 	return 0;
 }
 
 int cmd_hook(int argc, const char **argv, const char *prefix)
 {
+	const char *run_hookdir = NULL;
+
 	struct option builtin_hook_options[] = {
+		OPT_STRING(0, "run-hookdir", &run_hookdir, N_("option"),
+			   N_("what to do with hooks found in the hookdir")),
 		OPT_END(),
 	};
-	if (argc < 2)
+
+	argc = parse_options(argc, argv, prefix, builtin_hook_options,
+			     builtin_hook_usage, 0);
+
+	/* after the parse, we should have "<command> <hookname> <args...>" */
+	if (argc < 1)
 		usage_with_options(builtin_hook_usage, builtin_hook_options);
 
 	git_config(git_default_config, NULL);
 
-	if (!strcmp(argv[1], "list"))
-		return list(argc - 1, argv + 1, prefix);
+
+	/* argument > config */
+	if (run_hookdir)
+		if (!strcmp(run_hookdir, "no"))
+			should_run_hookdir = HOOKDIR_NO;
+		else if (!strcmp(run_hookdir, "error"))
+			should_run_hookdir = HOOKDIR_ERROR;
+		else if (!strcmp(run_hookdir, "yes"))
+			should_run_hookdir = HOOKDIR_YES;
+		else if (!strcmp(run_hookdir, "warn"))
+			should_run_hookdir = HOOKDIR_WARN;
+		else if (!strcmp(run_hookdir, "interactive"))
+			should_run_hookdir = HOOKDIR_INTERACTIVE;
+		else
+			/*
+			 * TRANSLATORS: leave "yes/warn/interactive/no"
+			 * untranslated; the strings are compared literally.
+			 */
+			die(_("'%s' is not a valid option for --run-hookdir "
+			      "(yes, warn, interactive, no)"), run_hookdir);
+	else
+		should_run_hookdir = configured_hookdir_opt();
+
+	if (!strcmp(argv[0], "list"))
+		return list(argc, argv, prefix);
 
 	usage_with_options(builtin_hook_usage, builtin_hook_options);
 }
diff --git a/hook.c b/hook.c
index b4994fc108..030051cab2 100644
--- a/hook.c
+++ b/hook.c
@@ -102,6 +102,30 @@ static int hook_config_lookup(const char *key, const char *value, void *cb_data)
 	return 0;
 }
 
+enum hookdir_opt configured_hookdir_opt(void)
+{
+	const char *key;
+	if (git_config_get_value("hook.runhookdir", &key))
+		return HOOKDIR_YES; /* by default, just run it. */
+
+	if (!strcmp(key, "no"))
+		return HOOKDIR_NO;
+
+	if (!strcmp(key, "error"))
+		return HOOKDIR_ERROR;
+
+	if (!strcmp(key, "yes"))
+		return HOOKDIR_YES;
+
+	if (!strcmp(key, "warn"))
+		return HOOKDIR_WARN;
+
+	if (!strcmp(key, "interactive"))
+		return HOOKDIR_INTERACTIVE;
+
+	return HOOKDIR_UNKNOWN;
+}
+
 struct list_head* hook_list(const char* hookname)
 {
 	struct strbuf hook_key = STRBUF_INIT;
diff --git a/hook.h b/hook.h
index b6c5480325..7f2b2ee8f2 100644
--- a/hook.h
+++ b/hook.h
@@ -20,6 +20,28 @@ struct hook {
  */
 struct list_head* hook_list(const char *hookname);
 
+enum hookdir_opt
+{
+	HOOKDIR_NO,
+	HOOKDIR_ERROR,
+	HOOKDIR_WARN,
+	HOOKDIR_INTERACTIVE,
+	HOOKDIR_YES,
+	HOOKDIR_UNKNOWN,
+};
+
+/*
+ * Provides the hookdir_opt specified in the config without consulting any
+ * command line arguments.
+ */
+enum hookdir_opt configured_hookdir_opt(void);
+
+/*
+ * Provides the hookdir_opt specified in the config without consulting any
+ * command line arguments.
+ */
+enum hookdir_opt configured_hookdir_opt(void);
+
 /* Free memory associated with a 'struct hook' */
 void free_hook(struct hook *ptr);
 /* Empties the list at 'head', calling 'free_hook()' on each entry */
diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh
index 0f12af4659..141e6f7590 100755
--- a/t/t1360-config-based-hooks.sh
+++ b/t/t1360-config-based-hooks.sh
@@ -104,4 +104,75 @@ test_expect_success 'git hook list shows hooks from the hookdir' '
 	test_cmp expected actual
 '
 
+test_expect_success 'hook.runHookDir = no is respected by list' '
+	setup_hookdir &&
+
+	test_config hook.runHookDir "no" &&
+
+	cat >expected <<-EOF &&
+	hookdir: $(pwd)/.git/hooks/pre-commit (will not run)
+	EOF
+
+	git hook list pre-commit >actual &&
+	# the hookdir annotation is translated
+	test_cmp expected actual
+'
+
+test_expect_success 'hook.runHookDir = error is respected by list' '
+	setup_hookdir &&
+
+	test_config hook.runHookDir "error" &&
+
+	cat >expected <<-EOF &&
+	hookdir: $(pwd)/.git/hooks/pre-commit (will error and not run)
+	EOF
+
+	git hook list pre-commit >actual &&
+	# the hookdir annotation is translated
+	test_cmp expected actual
+'
+
+test_expect_success 'hook.runHookDir = warn is respected by list' '
+	setup_hookdir &&
+
+	test_config hook.runHookDir "warn" &&
+
+	cat >expected <<-EOF &&
+	hookdir: $(pwd)/.git/hooks/pre-commit (will warn but run)
+	EOF
+
+	git hook list pre-commit >actual &&
+	# the hookdir annotation is translated
+	test_cmp expected actual
+'
+
+
+test_expect_success 'hook.runHookDir = interactive is respected by list' '
+	setup_hookdir &&
+
+	test_config hook.runHookDir "interactive" &&
+
+	cat >expected <<-EOF &&
+	hookdir: $(pwd)/.git/hooks/pre-commit (will prompt)
+	EOF
+
+	git hook list pre-commit >actual &&
+	# the hookdir annotation is translated
+	test_cmp expected actual
+'
+
+test_expect_success 'hook.runHookDir is tolerant to unknown values' '
+	setup_hookdir &&
+
+	test_config hook.runHookDir "junk" &&
+
+	cat >expected <<-EOF &&
+	hookdir: $(pwd)/.git/hooks/pre-commit
+	EOF
+
+	git hook list pre-commit >actual &&
+	# the hookdir annotation is translated
+	test_cmp expected actual
+'
+
 test_done
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 05/37] hook: implement hookcmd.<name>.skip
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (3 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 04/37] hook: teach hook.runHookDir Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 06/37] parse-options: parse into strvec Emily Shaffer
                     ` (33 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

If a user wants a specific repo to skip execution of a hook which is set
at a global or system level, they will be able to do so by specifying
'skip' in their repo config:

~/.gitconfig
  [hook.pre-commit]
    command = skippable-oneliner
    command = skippable-hookcmd

  [hookcmd.skippable-hookcmd]
    command = foo.sh

$GIT_DIR/.git/config
  [hookcmd.skippable-oneliner]
    skip = true
  [hookcmd.skippable-hookcmd]
    skip = true

Later it may make sense to add an option like
"hookcmd.<name>.<hook-event>-skip" - but for simplicity, let's start
with a universal skip setting like this.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/config/hook.txt |  8 ++++++++
 Documentation/git-hook.txt    | 33 +++++++++++++++++++++++++++++++++
 hook.c                        | 35 ++++++++++++++++++++++++++---------
 t/t1360-config-based-hooks.sh | 35 +++++++++++++++++++++++++++++++++++
 4 files changed, 102 insertions(+), 9 deletions(-)

diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
index 75312754ae..8b12512e33 100644
--- a/Documentation/config/hook.txt
+++ b/Documentation/config/hook.txt
@@ -8,6 +8,14 @@ hookcmd.<name>.command::
 	as a command. This can be an executable on your device or a oneliner for
 	your shell. See linkgit:git-hook[1].
 
+hookcmd.<name>.skip::
+	Specify this boolean to remove a command from earlier in the execution
+	order. Useful if you want to make a single repo an exception to hook
+	configured at the system or global scope. If there is no hookcmd
+	specified for the command you want to skip, you can use the value of
+	`hook.<command>.command` as <name> as a shortcut. The "skip" setting
+	must be specified after the "hook.<command>.command" to have an effect.
+
 hook.runHookDir::
 	Controls how hooks contained in your hookdir are executed. Can be any of
 	"yes", "warn", "interactive", or "no". Defaults to "yes". See
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index f19875ed68..c84520cb38 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -54,6 +54,39 @@ $ git hook list "prepare-commit-msg"
 local: /bin/linter --c
 ----
 
+If there is a command you wish to run in most cases but have one or two
+exceptional repos where it should be skipped, you can use specify
+`hookcmd.<name>.skip`, for example:
+
+System config
+----
+  [hook "pre-commit"]
+    command = check-for-secrets
+
+  [hookcmd "check-for-secrets"]
+    command = /bin/secret-checker --aggressive
+----
+
+Local config
+----
+  [hookcmd "check-for-secrets"]
+    skip = true
+  # This works for inlined hook commands, too:
+  [hookcmd "~/typocheck.sh"]
+    skip = true
+----
+
+After these configs are added, the hook list becomes:
+
+----
+$ git hook list "post-commit"
+global: /bin/linter --c
+local: python ~/run-test-suite.py
+
+$ git hook list "pre-commit"
+no commands configured for hook 'pre-commit'
+----
+
 COMMANDS
 --------
 
diff --git a/hook.c b/hook.c
index 030051cab2..65cbad8dba 100644
--- a/hook.c
+++ b/hook.c
@@ -12,24 +12,25 @@ void free_hook(struct hook *ptr)
 	}
 }
 
-static void append_or_move_hook(struct list_head *head, const char *command)
+static struct hook * find_hook_by_command(struct list_head *head, const char *command)
 {
 	struct list_head *pos = NULL, *tmp = NULL;
-	struct hook *to_add = NULL;
+	struct hook *found = NULL;
 
-	/*
-	 * remove the prior entry with this command; we'll replace it at the
-	 * end.
-	 */
 	list_for_each_safe(pos, tmp, head) {
 		struct hook *it = list_entry(pos, struct hook, list);
 		if (!strcmp(it->command.buf, command)) {
 		    list_del(pos);
-		    /* we'll simply move the hook to the end */
-		    to_add = it;
+		    found = it;
 		    break;
 		}
 	}
+	return found;
+}
+
+static void append_or_move_hook(struct list_head *head, const char *command)
+{
+	struct hook *to_add = find_hook_by_command(head, command);
 
 	if (!to_add) {
 		/* adding a new hook, not moving an old one */
@@ -74,12 +75,22 @@ static int hook_config_lookup(const char *key, const char *value, void *cb_data)
 	if (!strcmp(key, hook_key)) {
 		const char *command = value;
 		struct strbuf hookcmd_name = STRBUF_INIT;
+		int skip = 0;
+
+		/*
+		 * Check if we're removing that hook instead. Hookcmds are
+		 * removed by name, and inlined hooks are removed by command
+		 * content.
+		 */
+		strbuf_addf(&hookcmd_name, "hookcmd.%s.skip", command);
+		git_config_get_bool(hookcmd_name.buf, &skip);
 
 		/*
 		 * Check if a hookcmd with that name exists. If it doesn't,
 		 * 'git_config_get_value()' is documented not to touch &command,
 		 * so we don't need to do anything.
 		 */
+		strbuf_reset(&hookcmd_name);
 		strbuf_addf(&hookcmd_name, "hookcmd.%s.command", command);
 		git_config_get_value(hookcmd_name.buf, &command);
 
@@ -94,7 +105,13 @@ static int hook_config_lookup(const char *key, const char *value, void *cb_data)
 		 *   for each key+value, do_callback(key, value, cb_data)
 		 */
 
-		append_or_move_hook(head, command);
+		if (skip) {
+			struct hook *to_remove = find_hook_by_command(head, command);
+			if (to_remove)
+				remove_hook(&(to_remove->list));
+		} else {
+			append_or_move_hook(head, command);
+		}
 
 		strbuf_release(&hookcmd_name);
 	}
diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh
index 141e6f7590..33ac27aa97 100755
--- a/t/t1360-config-based-hooks.sh
+++ b/t/t1360-config-based-hooks.sh
@@ -146,6 +146,41 @@ test_expect_success 'hook.runHookDir = warn is respected by list' '
 	test_cmp expected actual
 '
 
+test_expect_success 'git hook list removes skipped hookcmd' '
+	setup_hookcmd &&
+	test_config hookcmd.abc.skip "true" --add &&
+
+	cat >expected <<-EOF &&
+	no commands configured for hook '\''pre-commit'\''
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list ignores skip referring to unused hookcmd' '
+	test_config hookcmd.abc.command "/path/abc" --add &&
+	test_config hookcmd.abc.skip "true" --add &&
+
+	cat >expected <<-EOF &&
+	no commands configured for hook '\''pre-commit'\''
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list removes skipped inlined hook' '
+	setup_hooks &&
+	test_config hookcmd."$ROOT/path/ghi".skip "true" --add &&
+
+	cat >expected <<-EOF &&
+	global: $ROOT/path/def
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
 
 test_expect_success 'hook.runHookDir = interactive is respected by list' '
 	setup_hookdir &&
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 06/37] parse-options: parse into strvec
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (4 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 05/37] hook: implement hookcmd.<name>.skip Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 07/37] hook: add 'run' subcommand Emily Shaffer
                     ` (32 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

parse-options already knows how to read into a string_list, and it knows
how to read into an strvec as a passthrough (that is, including the
argument as well as its value). string_list and strvec serve similar
purposes but are somewhat painful to convert between; so, let's teach
parse-options to read values of string arguments directly into an
strvec without preserving the argument name.

This is useful if collecting generic arguments to pass through to
another command, for example, 'git hook run --arg "--quiet" --arg
"--format=pretty" some-hook'. The resulting strvec would contain
{ "--quiet", "--format=pretty" }.

The implementation is based on that of OPT_STRING_LIST.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

Notes:
    Since v7, updated the reference doc to make the intended usage for OPT_STRVEC
    more clear.
    
    Since v4, fixed one or two more places where I missed the argv_array->strvec
    rename.

 Documentation/technical/api-parse-options.txt |  7 +++++
 parse-options-cb.c                            | 16 +++++++++++
 parse-options.h                               |  4 +++
 t/helper/test-parse-options.c                 |  6 +++++
 t/t0040-parse-options.sh                      | 27 +++++++++++++++++++
 5 files changed, 60 insertions(+)

diff --git a/Documentation/technical/api-parse-options.txt b/Documentation/technical/api-parse-options.txt
index 5a60bbfa7f..f79b17e7fc 100644
--- a/Documentation/technical/api-parse-options.txt
+++ b/Documentation/technical/api-parse-options.txt
@@ -173,6 +173,13 @@ There are some macros to easily define options:
 	The string argument is stored as an element in `string_list`.
 	Use of `--no-option` will clear the list of preceding values.
 
+`OPT_STRVEC(short, long, &struct strvec, arg_str, description)`::
+	Introduce an option with a string argument, meant to be specified
+	multiple times.
+	The string argument is stored as an element in `strvec`, and later
+	arguments are added to the same `strvec`.
+	Use of `--no-option` will clear the list of preceding values.
+
 `OPT_INTEGER(short, long, &int_var, description)`::
 	Introduce an option with integer argument.
 	The integer is put into `int_var`.
diff --git a/parse-options-cb.c b/parse-options-cb.c
index 3c811e1e4a..8227499eb6 100644
--- a/parse-options-cb.c
+++ b/parse-options-cb.c
@@ -207,6 +207,22 @@ int parse_opt_string_list(const struct option *opt, const char *arg, int unset)
 	return 0;
 }
 
+int parse_opt_strvec(const struct option *opt, const char *arg, int unset)
+{
+	struct strvec *v = opt->value;
+
+	if (unset) {
+		strvec_clear(v);
+		return 0;
+	}
+
+	if (!arg)
+		return -1;
+
+	strvec_push(v, arg);
+	return 0;
+}
+
 int parse_opt_noop_cb(const struct option *opt, const char *arg, int unset)
 {
 	return 0;
diff --git a/parse-options.h b/parse-options.h
index a845a9d952..fcb0f1f31e 100644
--- a/parse-options.h
+++ b/parse-options.h
@@ -178,6 +178,9 @@ struct option {
 #define OPT_STRING_LIST(s, l, v, a, h) \
 				    { OPTION_CALLBACK, (s), (l), (v), (a), \
 				      (h), 0, &parse_opt_string_list }
+#define OPT_STRVEC(s, l, v, a, h) \
+				    { OPTION_CALLBACK, (s), (l), (v), (a), \
+				      (h), 0, &parse_opt_strvec }
 #define OPT_UYN(s, l, v, h)         { OPTION_CALLBACK, (s), (l), (v), NULL, \
 				      (h), PARSE_OPT_NOARG, &parse_opt_tertiary }
 #define OPT_EXPIRY_DATE(s, l, v, h) \
@@ -297,6 +300,7 @@ int parse_opt_commits(const struct option *, const char *, int);
 int parse_opt_commit(const struct option *, const char *, int);
 int parse_opt_tertiary(const struct option *, const char *, int);
 int parse_opt_string_list(const struct option *, const char *, int);
+int parse_opt_strvec(const struct option *, const char *, int);
 int parse_opt_noop_cb(const struct option *, const char *, int);
 enum parse_opt_result parse_opt_unknown_cb(struct parse_opt_ctx_t *ctx,
 					   const struct option *,
diff --git a/t/helper/test-parse-options.c b/t/helper/test-parse-options.c
index 2051ce57db..922af56156 100644
--- a/t/helper/test-parse-options.c
+++ b/t/helper/test-parse-options.c
@@ -2,6 +2,7 @@
 #include "cache.h"
 #include "parse-options.h"
 #include "string-list.h"
+#include "strvec.h"
 #include "trace2.h"
 
 static int boolean = 0;
@@ -15,6 +16,7 @@ static char *string = NULL;
 static char *file = NULL;
 static int ambiguous;
 static struct string_list list = STRING_LIST_INIT_NODUP;
+static struct strvec vector = STRVEC_INIT;
 
 static struct {
 	int called;
@@ -133,6 +135,7 @@ int cmd__parse_options(int argc, const char **argv)
 		OPT_STRING('o', NULL, &string, "str", "get another string"),
 		OPT_NOOP_NOARG(0, "obsolete"),
 		OPT_STRING_LIST(0, "list", &list, "str", "add str to list"),
+		OPT_STRVEC(0, "vector", &vector, "str", "add str to strvec"),
 		OPT_GROUP("Magic arguments"),
 		OPT_ARGUMENT("quux", NULL, "means --quux"),
 		OPT_NUMBER_CALLBACK(&integer, "set integer to NUM",
@@ -183,6 +186,9 @@ int cmd__parse_options(int argc, const char **argv)
 	for (i = 0; i < list.nr; i++)
 		show(&expect, &ret, "list: %s", list.items[i].string);
 
+	for (i = 0; i < vector.nr; i++)
+		show(&expect, &ret, "vector: %s", vector.v[i]);
+
 	for (i = 0; i < argc; i++)
 		show(&expect, &ret, "arg %02d: %s", i, argv[i]);
 
diff --git a/t/t0040-parse-options.sh b/t/t0040-parse-options.sh
index ad4746d899..485e0170bf 100755
--- a/t/t0040-parse-options.sh
+++ b/t/t0040-parse-options.sh
@@ -35,6 +35,7 @@ String options
     --st <st>             get another string (pervert ordering)
     -o <str>              get another string
     --list <str>          add str to list
+    --vector <str>        add str to strvec
 
 Magic arguments
     --quux                means --quux
@@ -386,6 +387,32 @@ test_expect_success '--no-list resets list' '
 	test_cmp expect output
 '
 
+cat >expect <<\EOF
+boolean: 0
+integer: 0
+magnitude: 0
+timestamp: 0
+string: (not set)
+abbrev: 7
+verbose: -1
+quiet: 0
+dry run: no
+file: (not set)
+vector: foo
+vector: bar
+vector: baz
+EOF
+test_expect_success '--vector keeps list of strings' '
+	test-tool parse-options --vector foo --vector=bar --vector=baz >output &&
+	test_cmp expect output
+'
+
+test_expect_success '--no-vector resets list' '
+	test-tool parse-options --vector=other --vector=irrelevant --vector=options \
+		--no-vector --vector=foo --vector=bar --vector=baz >output &&
+	test_cmp expect output
+'
+
 test_expect_success 'multiple quiet levels' '
 	test-tool parse-options --expect="quiet: 3" -q -q -q
 '
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 07/37] hook: add 'run' subcommand
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (5 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 06/37] parse-options: parse into strvec Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-06-03  9:07     ` Ævar Arnfjörð Bjarmason
  2021-05-27  0:08   ` [PATCH v9 08/37] hook: introduce hook_exists() Emily Shaffer
                     ` (31 subsequent siblings)
  38 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

In order to enable hooks to be run as an external process, by a
standalone Git command, or by tools which wrap Git, provide an external
means to run all configured hook commands for a given hook event.

For now, the hook commands will run in config order, in series. As
alternate ordering or parallelism is supported in the future, we should
add knobs to use those to the command line as well.

As with the legacy hook implementation, all stdout generated by hook
commands is redirected to stderr. Piping from stdin is not yet
supported.

Legacy hooks (those present in $GITDIR/hooks) are run at the end of the
execution list. They can be disabled, or made to print warnings, or to
prompt before running, with the 'hook.runHookDir' config.

Users may wish to provide hook commands like 'git config
hook.pre-commit.command "~/linter.sh --pre-commit"'. To enable this,
config-defined hooks are run in a shell. (Since hooks in $GITDIR/hooks
can't be specified with included arguments or paths which need expansion
like this, they are run without a shell instead.)

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

Notes:
    Since v7, added support for "error" hook.runHookDir setting.
    
    Since v4, updated the docs, and did less local application of single
    quotes. In order for hookdir hooks to run successfully with a space in
    the path, though, they must not be run with 'sh -c'. So we can treat the
    hookdir hooks specially, and warn users via doc about special
    considerations for configured hooks with spaces in their path.

 Documentation/git-hook.txt    |  31 +++++++-
 builtin/hook.c                |  42 ++++++++++-
 hook.c                        | 137 +++++++++++++++++++++++++++++++++-
 hook.h                        |  26 ++++++-
 t/t1360-config-based-hooks.sh |  96 +++++++++++++++++++++++-
 5 files changed, 320 insertions(+), 12 deletions(-)

diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index c84520cb38..8f96c347ea 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -9,11 +9,12 @@ SYNOPSIS
 --------
 [verse]
 'git hook' list <hook-name>
+'git hook' run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] <hook-name>
 
 DESCRIPTION
 -----------
-You can list configured hooks with this command. Later, you will be able to run,
-add, and modify hooks with this command.
+You can list and run configured hooks with this command. Later, you will be able
+to add and modify hooks with this command.
 
 This command parses the default configuration files for sections `hook` and
 `hookcmd`. `hook` is used to describe the commands which will be run during a
@@ -97,6 +98,32 @@ in the order they should be run, and print the config scope where the relevant
 `hook.<hook-name>.command` was specified, not the `hookcmd` (if applicable).
 This output is human-readable and the format is subject to change over time.
 
+run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] `<hook-name>`::
+
+Runs hooks configured for `<hook-name>`, in the same order displayed by `git
+hook list`. Hooks configured this way may be run prepended with `sh -c`, so
+paths containing special characters or spaces should be wrapped in single
+quotes: `command = '/my/path with spaces/script.sh' some args`.
+
+OPTIONS
+-------
+--run-hookdir::
+	Overrides the hook.runHookDir config. Must be 'yes', 'warn',
+	'interactive', or 'no'. Specifies how to handle hooks located in the Git
+	hook directory (core.hooksPath).
+
+-a::
+--arg::
+	Only valid for `run`.
++
+Specify arguments to pass to every hook that is run.
+
+-e::
+--env::
+	Only valid for `run`.
++
+Specify environment variables to set for every hook that is run.
+
 CONFIGURATION
 -------------
 include::config/hook.txt[]
diff --git a/builtin/hook.c b/builtin/hook.c
index b1e63a9576..4673c9091c 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -4,9 +4,11 @@
 #include "hook.h"
 #include "parse-options.h"
 #include "strbuf.h"
+#include "strvec.h"
 
 static const char * const builtin_hook_usage[] = {
 	N_("git hook list <hookname>"),
+	N_("git hook run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] <hookname>"),
 	NULL
 };
 
@@ -98,6 +100,40 @@ static int list(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+static int run(int argc, const char **argv, const char *prefix)
+{
+	struct strbuf hookname = STRBUF_INIT;
+	struct run_hooks_opt opt;
+	int rc = 0;
+
+	struct option run_options[] = {
+		OPT_STRVEC('e', "env", &opt.env, N_("var"),
+			   N_("environment variables for hook to use")),
+		OPT_STRVEC('a', "arg", &opt.args, N_("args"),
+			   N_("argument to pass to hook")),
+		OPT_END(),
+	};
+
+	run_hooks_opt_init(&opt);
+
+	argc = parse_options(argc, argv, prefix, run_options,
+			     builtin_hook_usage, 0);
+
+	if (argc < 1)
+		usage_msg_opt(_("You must specify a hook event to run."),
+			      builtin_hook_usage, run_options);
+
+	strbuf_addstr(&hookname, argv[0]);
+	opt.run_hookdir = should_run_hookdir;
+
+	rc = run_hooks(hookname.buf, &opt);
+
+	strbuf_release(&hookname);
+	run_hooks_opt_clear(&opt);
+
+	return rc;
+}
+
 int cmd_hook(int argc, const char **argv, const char *prefix)
 {
 	const char *run_hookdir = NULL;
@@ -109,10 +145,10 @@ int cmd_hook(int argc, const char **argv, const char *prefix)
 	};
 
 	argc = parse_options(argc, argv, prefix, builtin_hook_options,
-			     builtin_hook_usage, 0);
+			     builtin_hook_usage, PARSE_OPT_KEEP_UNKNOWN);
 
 	/* after the parse, we should have "<command> <hookname> <args...>" */
-	if (argc < 1)
+	if (argc < 2)
 		usage_with_options(builtin_hook_usage, builtin_hook_options);
 
 	git_config(git_default_config, NULL);
@@ -142,6 +178,8 @@ int cmd_hook(int argc, const char **argv, const char *prefix)
 
 	if (!strcmp(argv[0], "list"))
 		return list(argc, argv, prefix);
+	if (!strcmp(argv[0], "run"))
+		return run(argc, argv, prefix);
 
 	usage_with_options(builtin_hook_usage, builtin_hook_options);
 }
diff --git a/hook.c b/hook.c
index 65cbad8dba..b631da659b 100644
--- a/hook.c
+++ b/hook.c
@@ -3,13 +3,13 @@
 #include "hook.h"
 #include "config.h"
 #include "run-command.h"
+#include "prompt.h"
 
 void free_hook(struct hook *ptr)
 {
-	if (ptr) {
+	if (ptr)
 		strbuf_release(&ptr->command);
-		free(ptr);
-	}
+	free(ptr);
 }
 
 static struct hook * find_hook_by_command(struct list_head *head, const char *command)
@@ -143,6 +143,70 @@ enum hookdir_opt configured_hookdir_opt(void)
 	return HOOKDIR_UNKNOWN;
 }
 
+static int should_include_hookdir(const char *path, enum hookdir_opt cfg)
+{
+	struct strbuf prompt = STRBUF_INIT;
+	/*
+	 * If the path doesn't exist, don't bother adding the empty hook and
+	 * don't bother checking the config or prompting the user.
+	 */
+	if (!path)
+		return 0;
+
+	switch (cfg)
+	{
+	case HOOKDIR_ERROR:
+		fprintf(stderr, _("Skipping legacy hook at '%s'\n"),
+			path);
+		/* FALLTHROUGH */
+	case HOOKDIR_NO:
+		return 0;
+	case HOOKDIR_WARN:
+		fprintf(stderr, _("Running legacy hook at '%s'\n"),
+			path);
+		return 1;
+	case HOOKDIR_INTERACTIVE:
+		do {
+			/*
+			 * TRANSLATORS: Make sure to include [Y] and [n]
+			 * in your translation. Only English input is
+			 * accepted. Default option is "yes".
+			 */
+			fprintf(stderr, _("Run '%s'? [Y/n] "), path);
+			git_read_line_interactively(&prompt);
+			/*
+			 * In case of prompt = '' - that is, user hit enter,
+			 * saying "yes I want the default" - strncasecmp will
+			 * return 0 regardless. So list the default first.
+			 *
+			 * Case insensitively, accept "y", "ye", or "yes" as
+			 * "yes"; accept "n" or "no" as "no".
+			 */
+			if (!strncasecmp(prompt.buf, "yes", prompt.len)) {
+				strbuf_release(&prompt);
+				return 1;
+			} else if (!strncasecmp(prompt.buf, "no", prompt.len)) {
+				strbuf_release(&prompt);
+				return 0;
+			}
+			/* otherwise, we didn't understand the input */
+		} while (prompt.len); /* an empty reply means default (yes) */
+		return 1;
+	/*
+	 * HOOKDIR_UNKNOWN should match the default behavior, but let's
+	 * give a heads up to the user.
+	 */
+	case HOOKDIR_UNKNOWN:
+		fprintf(stderr,
+			_("Unrecognized value for 'hook.runHookDir'. "
+			  "Is there a typo? "));
+		/* FALLTHROUGH */
+	case HOOKDIR_YES:
+	default:
+		return 1;
+	}
+}
+
 struct list_head* hook_list(const char* hookname)
 {
 	struct strbuf hook_key = STRBUF_INIT;
@@ -176,3 +240,70 @@ struct list_head* hook_list(const char* hookname)
 	strbuf_release(&hook_key);
 	return hook_head;
 }
+
+void run_hooks_opt_init(struct run_hooks_opt *o)
+{
+	strvec_init(&o->env);
+	strvec_init(&o->args);
+	o->run_hookdir = configured_hookdir_opt();
+}
+
+void run_hooks_opt_clear(struct run_hooks_opt *o)
+{
+	strvec_clear(&o->env);
+	strvec_clear(&o->args);
+}
+
+static void prepare_hook_cp(const char *hookname, struct hook *hook,
+			    struct run_hooks_opt *options,
+			    struct child_process *cp)
+{
+	if (!hook)
+		return;
+
+	cp->no_stdin = 1;
+	cp->env = options->env.v;
+	cp->stdout_to_stderr = 1;
+	cp->trace2_hook_name = hookname;
+
+	/*
+	 * Commands from the config could be oneliners, but we know
+	 * for certain that hookdir commands are not.
+	 */
+	cp->use_shell = !hook->from_hookdir;
+
+	/* add command */
+	strvec_push(&cp->args, hook->command.buf);
+
+	/*
+	 * add passed-in argv, without expanding - let the user get back
+	 * exactly what they put in
+	 */
+	strvec_pushv(&cp->args, options->args.v);
+}
+
+int run_hooks(const char *hookname, struct run_hooks_opt *options)
+{
+	struct list_head *to_run, *pos = NULL, *tmp = NULL;
+	int rc = 0;
+
+	if (!options)
+		BUG("a struct run_hooks_opt must be provided to run_hooks");
+
+	to_run = hook_list(hookname);
+
+	list_for_each_safe(pos, tmp, to_run) {
+		struct child_process hook_proc = CHILD_PROCESS_INIT;
+		struct hook *hook = list_entry(pos, struct hook, list);
+
+		if (hook->from_hookdir &&
+		    !should_include_hookdir(hook->command.buf, options->run_hookdir))
+			continue;
+
+		prepare_hook_cp(hookname, hook, options, &hook_proc);
+
+		rc |= run_command(&hook_proc);
+	}
+
+	return rc;
+}
diff --git a/hook.h b/hook.h
index 7f2b2ee8f2..fb5132305f 100644
--- a/hook.h
+++ b/hook.h
@@ -1,6 +1,7 @@
 #include "config.h"
 #include "list.h"
 #include "strbuf.h"
+#include "strvec.h"
 
 struct hook {
 	struct list_head list;
@@ -36,11 +37,30 @@ enum hookdir_opt
  */
 enum hookdir_opt configured_hookdir_opt(void);
 
+struct run_hooks_opt
+{
+	/* Environment vars to be set for each hook */
+	struct strvec env;
+
+	/* Args to be passed to each hook */
+	struct strvec args;
+
+	/*
+	 * How should the hookdir be handled?
+	 * Leave the RUN_HOOKS_OPT_INIT default in most cases; this only needs
+	 * to be overridden if the user can override it at the command line.
+	 */
+	enum hookdir_opt run_hookdir;
+};
+
+void run_hooks_opt_init(struct run_hooks_opt *o);
+void run_hooks_opt_clear(struct run_hooks_opt *o);
+
 /*
- * Provides the hookdir_opt specified in the config without consulting any
- * command line arguments.
+ * Runs all hooks associated to the 'hookname' event in order. Each hook will be
+ * passed 'env' and 'args'.
  */
-enum hookdir_opt configured_hookdir_opt(void);
+int run_hooks(const char *hookname, struct run_hooks_opt *options);
 
 /* Free memory associated with a 'struct hook' */
 void free_hook(struct hook *ptr);
diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh
index 33ac27aa97..3dddd41e4f 100755
--- a/t/t1360-config-based-hooks.sh
+++ b/t/t1360-config-based-hooks.sh
@@ -115,7 +115,10 @@ test_expect_success 'hook.runHookDir = no is respected by list' '
 
 	git hook list pre-commit >actual &&
 	# the hookdir annotation is translated
-	test_cmp expected actual
+	test_cmp expected actual &&
+
+	git hook run pre-commit 2>actual &&
+	test_must_be_empty actual
 '
 
 test_expect_success 'hook.runHookDir = error is respected by list' '
@@ -129,6 +132,13 @@ test_expect_success 'hook.runHookDir = error is respected by list' '
 
 	git hook list pre-commit >actual &&
 	# the hookdir annotation is translated
+	test_cmp expected actual &&
+
+	cat >expected <<-EOF &&
+	Skipping legacy hook at '\''$(pwd)/.git/hooks/pre-commit'\''
+	EOF
+
+	git hook run pre-commit 2>actual &&
 	test_cmp expected actual
 '
 
@@ -143,6 +153,14 @@ test_expect_success 'hook.runHookDir = warn is respected by list' '
 
 	git hook list pre-commit >actual &&
 	# the hookdir annotation is translated
+	test_cmp expected actual &&
+
+	cat >expected <<-EOF &&
+	Running legacy hook at '\''$(pwd)/.git/hooks/pre-commit'\''
+	"Legacy Hook"
+	EOF
+
+	git hook run pre-commit 2>actual &&
 	test_cmp expected actual
 '
 
@@ -182,7 +200,7 @@ test_expect_success 'git hook list removes skipped inlined hook' '
 	test_cmp expected actual
 '
 
-test_expect_success 'hook.runHookDir = interactive is respected by list' '
+test_expect_success 'hook.runHookDir = interactive is respected by list and run' '
 	setup_hookdir &&
 
 	test_config hook.runHookDir "interactive" &&
@@ -193,9 +211,83 @@ test_expect_success 'hook.runHookDir = interactive is respected by list' '
 
 	git hook list pre-commit >actual &&
 	# the hookdir annotation is translated
+	test_cmp expected actual &&
+
+	test_write_lines n | git hook run pre-commit 2>actual &&
+	! grep "Legacy Hook" actual &&
+
+	test_write_lines y | git hook run pre-commit 2>actual &&
+	grep "Legacy Hook" actual
+'
+
+test_expect_success 'inline hook definitions execute oneliners' '
+	test_config hook.pre-commit.command "echo \"Hello World\"" &&
+
+	echo "Hello World" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'inline hook definitions resolve paths' '
+	write_script sample-hook.sh <<-EOF &&
+	echo \"Sample Hook\"
+	EOF
+
+	test_when_finished "rm sample-hook.sh" &&
+
+	test_config hook.pre-commit.command "\"$(pwd)/sample-hook.sh\"" &&
+
+	echo \"Sample Hook\" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook run can pass args and env vars' '
+	write_script sample-hook.sh <<-\EOF &&
+	echo $1
+	echo $2
+	echo $TEST_ENV_1
+	echo $TEST_ENV_2
+	EOF
+
+	test_config hook.pre-commit.command "\"$(pwd)/sample-hook.sh\"" &&
+
+	cat >expected <<-EOF &&
+	arg1
+	arg2
+	env1
+	env2
+	EOF
+
+	git hook run --arg arg1 \
+		--env TEST_ENV_1=env1 \
+		-a arg2 \
+		-e TEST_ENV_2=env2 \
+		pre-commit 2>actual &&
+
 	test_cmp expected actual
 '
 
+test_expect_success 'hookdir hook included in git hook run' '
+	setup_hookdir &&
+
+	echo \"Legacy Hook\" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'out-of-repo runs excluded' '
+	setup_hooks &&
+
+	nongit test_must_fail git hook run pre-commit
+'
+
 test_expect_success 'hook.runHookDir is tolerant to unknown values' '
 	setup_hookdir &&
 
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 08/37] hook: introduce hook_exists()
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (6 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 07/37] hook: add 'run' subcommand Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 09/37] hook: support passing stdin to hooks Emily Shaffer
                     ` (30 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Add a helper to easily determine whether any hooks exist for a given
hook event.

Many callers want to check whether some state could be modified by a
hook; that check should include the config-based hooks as well. Optimize
by checking the config directly. Since commands which execute hooks
might want to take args to replace 'hook.runHookDir', let
'hook_exists()' take a hookdir_opt to override that config.

In some cases, external callers today use find_hook() to discover the
location of a hook and then run it manually with run-command.h (that is,
not with run_hook_le()). Later, those cases will call hook.h:run_hook()
directly instead.

Once the entire codebase is using hook_exists() instead of find_hook(),
find_hook() can be safely rolled into hook_exists() and removed from
run-command.h.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 hook.c | 19 +++++++++++++++++++
 hook.h | 10 ++++++++++
 2 files changed, 29 insertions(+)

diff --git a/hook.c b/hook.c
index b631da659b..008167dbe5 100644
--- a/hook.c
+++ b/hook.c
@@ -248,6 +248,25 @@ void run_hooks_opt_init(struct run_hooks_opt *o)
 	o->run_hookdir = configured_hookdir_opt();
 }
 
+int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir)
+{
+	const char *value = NULL; /* throwaway */
+	struct strbuf hook_key = STRBUF_INIT;
+	int could_run_hookdir;
+
+	if (should_run_hookdir == HOOKDIR_USE_CONFIG)
+		should_run_hookdir = configured_hookdir_opt();
+
+	could_run_hookdir = (should_run_hookdir == HOOKDIR_INTERACTIVE ||
+				should_run_hookdir == HOOKDIR_WARN ||
+				should_run_hookdir == HOOKDIR_YES)
+				&& !!find_hook(hookname);
+
+	strbuf_addf(&hook_key, "hook.%s.command", hookname);
+
+	return (!git_config_get_value(hook_key.buf, &value)) || could_run_hookdir;
+}
+
 void run_hooks_opt_clear(struct run_hooks_opt *o)
 {
 	strvec_clear(&o->env);
diff --git a/hook.h b/hook.h
index fb5132305f..5f770b53ed 100644
--- a/hook.h
+++ b/hook.h
@@ -23,6 +23,7 @@ struct list_head* hook_list(const char *hookname);
 
 enum hookdir_opt
 {
+	HOOKDIR_USE_CONFIG,
 	HOOKDIR_NO,
 	HOOKDIR_ERROR,
 	HOOKDIR_WARN,
@@ -56,6 +57,15 @@ struct run_hooks_opt
 void run_hooks_opt_init(struct run_hooks_opt *o);
 void run_hooks_opt_clear(struct run_hooks_opt *o);
 
+/*
+ * Returns 1 if any hooks are specified in the config or if a hook exists in the
+ * hookdir. Typically, invoke hook_exsts() like:
+ *   hook_exists(hookname, configured_hookdir_opt());
+ * Like with run_hooks, if you take a --run-hookdir flag, reflect that
+ * user-specified behavior here instead.
+ */
+int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir);
+
 /*
  * Runs all hooks associated to the 'hookname' event in order. Each hook will be
  * passed 'env' and 'args'.
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 09/37] hook: support passing stdin to hooks
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (7 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 08/37] hook: introduce hook_exists() Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 10/37] run-command: allow stdin for run_processes_parallel Emily Shaffer
                     ` (29 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Some hooks (such as post-rewrite) need to take input via stdin.
Previously, callers provided stdin to hooks by setting
run-command.h:child_process.in, which takes a FD. Callers would open the
file in question themselves before calling run-command(). However, since
we will now need to seek to the front of the file and read it again for
every hook which runs, hook.h:run_command() takes a path and handles FD
management itself. Since this file is opened for read only, it should
not prevent later parallel execution support.

On the frontend, this is supported by asking for a file path, rather
than by reading stdin. Reading directly from stdin would involve caching
the entire stdin (to memory or to disk) and reading it back from the
beginning to each hook. We'd want to support cases like insufficient
memory or storage for the file. While this may prove useful later, for
now the path of least resistance is to just ask the user to make this
interim file themselves.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-hook.txt    | 11 +++++++++--
 builtin/hook.c                |  5 ++++-
 hook.c                        |  8 +++++++-
 hook.h                        |  6 +++++-
 t/t1360-config-based-hooks.sh | 24 ++++++++++++++++++++++++
 5 files changed, 49 insertions(+), 5 deletions(-)

diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 8f96c347ea..96a857c682 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -9,7 +9,8 @@ SYNOPSIS
 --------
 [verse]
 'git hook' list <hook-name>
-'git hook' run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] <hook-name>
+'git hook' run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] [--to-stdin=<path>]
+	<hook-name>
 
 DESCRIPTION
 -----------
@@ -98,7 +99,7 @@ in the order they should be run, and print the config scope where the relevant
 `hook.<hook-name>.command` was specified, not the `hookcmd` (if applicable).
 This output is human-readable and the format is subject to change over time.
 
-run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] `<hook-name>`::
+run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] [--to-stdin=<path>] `<hook-name>`::
 
 Runs hooks configured for `<hook-name>`, in the same order displayed by `git
 hook list`. Hooks configured this way may be run prepended with `sh -c`, so
@@ -124,6 +125,12 @@ Specify arguments to pass to every hook that is run.
 +
 Specify environment variables to set for every hook that is run.
 
+--to-stdin::
+	Only valid for `run`.
++
+Specify a file which will be streamed into stdin for every hook that is run.
+Each hook will receive the entire file from beginning to EOF.
+
 CONFIGURATION
 -------------
 include::config/hook.txt[]
diff --git a/builtin/hook.c b/builtin/hook.c
index 4673c9091c..0d9c052e84 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -8,7 +8,8 @@
 
 static const char * const builtin_hook_usage[] = {
 	N_("git hook list <hookname>"),
-	N_("git hook run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] <hookname>"),
+	N_("git hook run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...]"
+	   "[--to-stdin=<path>] <hookname>"),
 	NULL
 };
 
@@ -111,6 +112,8 @@ static int run(int argc, const char **argv, const char *prefix)
 			   N_("environment variables for hook to use")),
 		OPT_STRVEC('a', "arg", &opt.args, N_("args"),
 			   N_("argument to pass to hook")),
+		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
+			   N_("file to read into hooks' stdin")),
 		OPT_END(),
 	};
 
diff --git a/hook.c b/hook.c
index 008167dbe5..a2eda57fb9 100644
--- a/hook.c
+++ b/hook.c
@@ -245,6 +245,7 @@ void run_hooks_opt_init(struct run_hooks_opt *o)
 {
 	strvec_init(&o->env);
 	strvec_init(&o->args);
+	o->path_to_stdin = NULL;
 	o->run_hookdir = configured_hookdir_opt();
 }
 
@@ -280,7 +281,12 @@ static void prepare_hook_cp(const char *hookname, struct hook *hook,
 	if (!hook)
 		return;
 
-	cp->no_stdin = 1;
+	/* reopen the file for stdin; run_command closes it. */
+	if (options->path_to_stdin)
+		cp->in = xopen(options->path_to_stdin, O_RDONLY);
+	else
+		cp->no_stdin = 1;
+
 	cp->env = options->env.v;
 	cp->stdout_to_stderr = 1;
 	cp->trace2_hook_name = hookname;
diff --git a/hook.h b/hook.h
index 5f770b53ed..37147f4c5e 100644
--- a/hook.h
+++ b/hook.h
@@ -52,6 +52,9 @@ struct run_hooks_opt
 	 * to be overridden if the user can override it at the command line.
 	 */
 	enum hookdir_opt run_hookdir;
+
+	/* Path to file which should be piped to stdin for each hook */
+	const char *path_to_stdin;
 };
 
 void run_hooks_opt_init(struct run_hooks_opt *o);
@@ -68,7 +71,8 @@ int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir);
 
 /*
  * Runs all hooks associated to the 'hookname' event in order. Each hook will be
- * passed 'env' and 'args'.
+ * passed 'env' and 'args'. The file at 'stdin_path' will be closed and reopened
+ * for each hook that runs.
  */
 int run_hooks(const char *hookname, struct run_hooks_opt *options);
 
diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh
index 3dddd41e4f..43917172d7 100755
--- a/t/t1360-config-based-hooks.sh
+++ b/t/t1360-config-based-hooks.sh
@@ -302,4 +302,28 @@ test_expect_success 'hook.runHookDir is tolerant to unknown values' '
 	test_cmp expected actual
 '
 
+test_expect_success 'stdin to multiple hooks' '
+	git config --add hook.test.command "xargs -P1 -I% echo a%" &&
+	git config --add hook.test.command "xargs -P1 -I% echo b%" &&
+	test_when_finished "test_unconfig hook.test.command" &&
+
+	cat >input <<-EOF &&
+	1
+	2
+	3
+	EOF
+
+	cat >expected <<-EOF &&
+	a1
+	a2
+	a3
+	b1
+	b2
+	b3
+	EOF
+
+	git hook run --to-stdin=input test 2>actual &&
+	test_cmp expected actual
+'
+
 test_done
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 10/37] run-command: allow stdin for run_processes_parallel
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (8 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 09/37] hook: support passing stdin to hooks Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 11/37] hook: allow parallel hook execution Emily Shaffer
                     ` (28 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

While it makes sense not to inherit stdin from the parent process to
avoid deadlocking, it's not necessary to completely ban stdin to
children. An informed user should be able to configure stdin safely. By
setting `some_child.process.no_stdin=1` before calling `get_next_task()`
we provide a reasonable default behavior but enable users to set up
stdin streaming for themselves during the callback.

`some_child.process.stdout_to_stderr`, however, remains unmodifiable by
`get_next_task()` - the rest of the run_processes_parallel() API depends
on child output in stderr.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 run-command.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/run-command.c b/run-command.c
index be6bc128cd..5237984ebe 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1693,6 +1693,14 @@ static int pp_start_one(struct parallel_processes *pp)
 	if (i == pp->max_processes)
 		BUG("bookkeeping is hard");
 
+	/*
+	 * By default, do not inherit stdin from the parent process - otherwise,
+	 * all children would share stdin! Users may overwrite this to provide
+	 * something to the child's stdin by having their 'get_next_task'
+	 * callback assign 0 to .no_stdin and an appropriate integer to .in.
+	 */
+	pp->children[i].process.no_stdin = 1;
+
 	code = pp->get_next_task(&pp->children[i].process,
 				 &pp->children[i].err,
 				 pp->data,
@@ -1704,7 +1712,6 @@ static int pp_start_one(struct parallel_processes *pp)
 	}
 	pp->children[i].process.err = -1;
 	pp->children[i].process.stdout_to_stderr = 1;
-	pp->children[i].process.no_stdin = 1;
 
 	if (start_command(&pp->children[i].process)) {
 		code = pp->start_failure(&pp->children[i].err,
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 11/37] hook: allow parallel hook execution
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (9 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 10/37] run-command: allow stdin for run_processes_parallel Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 12/37] hook: allow specifying working directory for hooks Emily Shaffer
                     ` (27 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

In many cases, there's no reason not to allow hooks to execute in
parallel. run_processes_parallel() is well-suited - it's a task queue
that runs its housekeeping in series, which means users don't
need to worry about thread safety on their callback data. True
multithreaded execution with the async_* functions isn't necessary here.
Synchronous hook execution can be achieved by only allowing 1 job to run
at a time.

Teach run_hooks() to use that function for simple hooks which don't
require stdin or capture of stderr.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

Notes:
    Per AEvar's request - parallel hook execution on day zero.
    
    In most ways run_processes_parallel() worked great for me - but it didn't
    have great support for hooks where we pipe to and from. I had to add this
    support later in the series.
    
    Since I modified an existing and in-use library I'd appreciate a keen look on
    these patches.
    
     - Emily

 Documentation/config/hook.txt |   5 ++
 Documentation/git-hook.txt    |  14 ++++-
 builtin/hook.c                |   6 +-
 hook.c                        | 111 ++++++++++++++++++++++++++++------
 hook.h                        |  22 ++++++-
 5 files changed, 134 insertions(+), 24 deletions(-)

diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
index 8b12512e33..4f66bb35cf 100644
--- a/Documentation/config/hook.txt
+++ b/Documentation/config/hook.txt
@@ -20,3 +20,8 @@ hook.runHookDir::
 	Controls how hooks contained in your hookdir are executed. Can be any of
 	"yes", "warn", "interactive", or "no". Defaults to "yes". See
 	linkgit:git-hook[1] and linkgit:git-config[1] "core.hooksPath").
+
+hook.jobs::
+	Specifies how many hooks can be run simultaneously during parallelized
+	hook execution. If unspecified, defaults to the number of processors on
+	the current system.
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 96a857c682..81b8e94994 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -10,7 +10,7 @@ SYNOPSIS
 [verse]
 'git hook' list <hook-name>
 'git hook' run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] [--to-stdin=<path>]
-	<hook-name>
+	[(-j|--jobs) <n>] <hook-name>
 
 DESCRIPTION
 -----------
@@ -99,7 +99,7 @@ in the order they should be run, and print the config scope where the relevant
 `hook.<hook-name>.command` was specified, not the `hookcmd` (if applicable).
 This output is human-readable and the format is subject to change over time.
 
-run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] [--to-stdin=<path>] `<hook-name>`::
+run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] [--to-stdin=<path>] [(-j|--jobs)<n>] `<hook-name>`::
 
 Runs hooks configured for `<hook-name>`, in the same order displayed by `git
 hook list`. Hooks configured this way may be run prepended with `sh -c`, so
@@ -131,6 +131,16 @@ Specify environment variables to set for every hook that is run.
 Specify a file which will be streamed into stdin for every hook that is run.
 Each hook will receive the entire file from beginning to EOF.
 
+-j::
+--jobs::
+	Only valid for `run`.
++
+Specify how many hooks to run simultaneously. If this flag is not specified, use
+the value of the `hook.jobs` config. If the config is not specified, use the
+number of CPUs on the current system. Some hooks may be ineligible for
+parallelization: for example, 'commit-msg' intends hooks modify the commit
+message body and cannot be parallelized.
+
 CONFIGURATION
 -------------
 include::config/hook.txt[]
diff --git a/builtin/hook.c b/builtin/hook.c
index 0d9c052e84..c79a961e80 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -9,7 +9,7 @@
 static const char * const builtin_hook_usage[] = {
 	N_("git hook list <hookname>"),
 	N_("git hook run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...]"
-	   "[--to-stdin=<path>] <hookname>"),
+	   "[--to-stdin=<path>] [(-j|--jobs) <count>] <hookname>"),
 	NULL
 };
 
@@ -114,10 +114,12 @@ static int run(int argc, const char **argv, const char *prefix)
 			   N_("argument to pass to hook")),
 		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
 			   N_("file to read into hooks' stdin")),
+		OPT_INTEGER('j', "jobs", &opt.jobs,
+			    N_("run up to <n> hooks simultaneously")),
 		OPT_END(),
 	};
 
-	run_hooks_opt_init(&opt);
+	run_hooks_opt_init_async(&opt);
 
 	argc = parse_options(argc, argv, prefix, run_options,
 			     builtin_hook_usage, 0);
diff --git a/hook.c b/hook.c
index a2eda57fb9..784529fe2b 100644
--- a/hook.c
+++ b/hook.c
@@ -143,6 +143,14 @@ enum hookdir_opt configured_hookdir_opt(void)
 	return HOOKDIR_UNKNOWN;
 }
 
+int configured_hook_jobs(void)
+{
+	int n = online_cpus();
+	git_config_get_int("hook.jobs", &n);
+
+	return n;
+}
+
 static int should_include_hookdir(const char *path, enum hookdir_opt cfg)
 {
 	struct strbuf prompt = STRBUF_INIT;
@@ -241,12 +249,19 @@ struct list_head* hook_list(const char* hookname)
 	return hook_head;
 }
 
-void run_hooks_opt_init(struct run_hooks_opt *o)
+void run_hooks_opt_init_sync(struct run_hooks_opt *o)
 {
 	strvec_init(&o->env);
 	strvec_init(&o->args);
 	o->path_to_stdin = NULL;
 	o->run_hookdir = configured_hookdir_opt();
+	o->jobs = 1;
+}
+
+void run_hooks_opt_init_async(struct run_hooks_opt *o)
+{
+	run_hooks_opt_init_sync(o);
+	o->jobs = configured_hook_jobs();
 }
 
 int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir)
@@ -274,22 +289,28 @@ void run_hooks_opt_clear(struct run_hooks_opt *o)
 	strvec_clear(&o->args);
 }
 
-static void prepare_hook_cp(const char *hookname, struct hook *hook,
-			    struct run_hooks_opt *options,
-			    struct child_process *cp)
+static int pick_next_hook(struct child_process *cp,
+			  struct strbuf *out,
+			  void *pp_cb,
+			  void **pp_task_cb)
 {
+	struct hook_cb_data *hook_cb = pp_cb;
+	struct hook *hook = hook_cb->run_me;
+
 	if (!hook)
-		return;
+		return 0;
 
 	/* reopen the file for stdin; run_command closes it. */
-	if (options->path_to_stdin)
-		cp->in = xopen(options->path_to_stdin, O_RDONLY);
-	else
+	if (hook_cb->options->path_to_stdin) {
+		cp->no_stdin = 0;
+		cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY);
+	} else {
 		cp->no_stdin = 1;
+	}
 
-	cp->env = options->env.v;
+	cp->env = hook_cb->options->env.v;
 	cp->stdout_to_stderr = 1;
-	cp->trace2_hook_name = hookname;
+	cp->trace2_hook_name = hook_cb->hookname;
 
 	/*
 	 * Commands from the config could be oneliners, but we know
@@ -304,13 +325,58 @@ static void prepare_hook_cp(const char *hookname, struct hook *hook,
 	 * add passed-in argv, without expanding - let the user get back
 	 * exactly what they put in
 	 */
-	strvec_pushv(&cp->args, options->args.v);
+	strvec_pushv(&cp->args, hook_cb->options->args.v);
+
+	/* Provide context for errors if necessary */
+	*pp_task_cb = hook;
+
+	/* Get the next entry ready */
+	if (hook_cb->run_me->list.next == hook_cb->head)
+		hook_cb->run_me = NULL;
+	else
+		hook_cb->run_me = list_entry(hook_cb->run_me->list.next,
+					     struct hook, list);
+
+	return 1;
+}
+
+static int notify_start_failure(struct strbuf *out,
+				void *pp_cb,
+				void *pp_task_cp)
+{
+	struct hook_cb_data *hook_cb = pp_cb;
+	struct hook *attempted = pp_task_cp;
+
+	/* |= rc in cb */
+	hook_cb->rc |= 1;
+
+	strbuf_addf(out, _("Couldn't start '%s', configured in '%s'\n"),
+		    attempted->command.buf,
+		    attempted->from_hookdir ? "hookdir"
+			: config_scope_name(attempted->origin));
+
+	/* NEEDSWORK: if halt_on_error is desired, do it here. */
+	return 0;
+}
+
+static int notify_hook_finished(int result,
+				struct strbuf *out,
+				void *pp_cb,
+				void *pp_task_cb)
+{
+	struct hook_cb_data *hook_cb = pp_cb;
+
+	/* |= rc in cb */
+	hook_cb->rc |= result;
+
+	/* NEEDSWORK: if halt_on_error is desired, do it here. */
+	return 0;
 }
 
 int run_hooks(const char *hookname, struct run_hooks_opt *options)
 {
 	struct list_head *to_run, *pos = NULL, *tmp = NULL;
-	int rc = 0;
+	struct hook_cb_data cb_data = { 0, hookname, NULL, NULL, options };
 
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
@@ -318,17 +384,26 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options)
 	to_run = hook_list(hookname);
 
 	list_for_each_safe(pos, tmp, to_run) {
-		struct child_process hook_proc = CHILD_PROCESS_INIT;
 		struct hook *hook = list_entry(pos, struct hook, list);
 
 		if (hook->from_hookdir &&
 		    !should_include_hookdir(hook->command.buf, options->run_hookdir))
-			continue;
+			    list_del(pos);
+	}
+
+	if (list_empty(to_run))
+		return 0;
 
-		prepare_hook_cp(hookname, hook, options, &hook_proc);
+	cb_data.head = to_run;
+	cb_data.run_me = list_entry(to_run->next, struct hook, list);
 
-		rc |= run_command(&hook_proc);
-	}
+	run_processes_parallel_tr2(options->jobs,
+				   pick_next_hook,
+				   notify_start_failure,
+				   notify_hook_finished,
+				   &cb_data,
+				   "hook",
+				   hookname);
 
-	return rc;
+	return cb_data.rc;
 }
diff --git a/hook.h b/hook.h
index 37147f4c5e..cdb1ac5510 100644
--- a/hook.h
+++ b/hook.h
@@ -38,6 +38,9 @@ enum hookdir_opt
  */
 enum hookdir_opt configured_hookdir_opt(void);
 
+/* Provides the number of threads to use for parallel hook execution. */
+int configured_hook_jobs(void);
+
 struct run_hooks_opt
 {
 	/* Environment vars to be set for each hook */
@@ -48,16 +51,31 @@ struct run_hooks_opt
 
 	/*
 	 * How should the hookdir be handled?
-	 * Leave the RUN_HOOKS_OPT_INIT default in most cases; this only needs
+	 * Leave the run_hooks_opt_init_*() default in most cases; this only needs
 	 * to be overridden if the user can override it at the command line.
 	 */
 	enum hookdir_opt run_hookdir;
 
 	/* Path to file which should be piped to stdin for each hook */
 	const char *path_to_stdin;
+
+	/* Number of threads to parallelize across */
+	int jobs;
+};
+
+/*
+ * Callback provided to feed_pipe_fn and consume_sideband_fn.
+ */
+struct hook_cb_data {
+	int rc;
+	const char *hookname;
+	struct list_head *head;
+	struct hook *run_me;
+	struct run_hooks_opt *options;
 };
 
-void run_hooks_opt_init(struct run_hooks_opt *o);
+void run_hooks_opt_init_sync(struct run_hooks_opt *o);
+void run_hooks_opt_init_async(struct run_hooks_opt *o);
 void run_hooks_opt_clear(struct run_hooks_opt *o);
 
 /*
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 12/37] hook: allow specifying working directory for hooks
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (10 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 11/37] hook: allow parallel hook execution Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 13/37] run-command: add stdin callback for parallelization Emily Shaffer
                     ` (26 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Hooks like "post-checkout" require that hooks have a different working
directory than the initial process. Pipe that directly through to struct
child_process.

Because we can just run 'git -C <some-dir> hook run ...' it shouldn't be
necessary to pipe this option through the frontend. In fact, this
reduces the possibility of users running hooks which affect some part of
the filesystem outside of the repo in question.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

Notes:
    Needed later for "post-checkout" conversion.

 hook.c | 2 ++
 hook.h | 3 +++
 2 files changed, 5 insertions(+)

diff --git a/hook.c b/hook.c
index 784529fe2b..920563607d 100644
--- a/hook.c
+++ b/hook.c
@@ -256,6 +256,7 @@ void run_hooks_opt_init_sync(struct run_hooks_opt *o)
 	o->path_to_stdin = NULL;
 	o->run_hookdir = configured_hookdir_opt();
 	o->jobs = 1;
+	o->dir = NULL;
 }
 
 void run_hooks_opt_init_async(struct run_hooks_opt *o)
@@ -311,6 +312,7 @@ static int pick_next_hook(struct child_process *cp,
 	cp->env = hook_cb->options->env.v;
 	cp->stdout_to_stderr = 1;
 	cp->trace2_hook_name = hook_cb->hookname;
+	cp->dir = hook_cb->options->dir;
 
 	/*
 	 * Commands from the config could be oneliners, but we know
diff --git a/hook.h b/hook.h
index cdb1ac5510..218b9e1721 100644
--- a/hook.h
+++ b/hook.h
@@ -61,6 +61,9 @@ struct run_hooks_opt
 
 	/* Number of threads to parallelize across */
 	int jobs;
+
+	/* Path to initial working directory for subprocess */
+	const char *dir;
 };
 
 /*
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 13/37] run-command: add stdin callback for parallelization
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (11 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 12/37] hook: allow specifying working directory for hooks Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 14/37] hook: provide stdin by string_list or callback Emily Shaffer
                     ` (25 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

If a user of the run_processes_parallel() API wants to pipe a large
amount of information to stdin of each parallel command, that
information could exceed the buffer of the pipe allocated for that
process's stdin.  Generally this is solved by repeatedly writing to
child_process.in between calls to start_command() and finish_command();
run_processes_parallel() did not provide users an opportunity to access
child_process at that time.

Because the data might be extremely large (for example, a list of all
refs received during a push from a client) simply taking a string_list
or strbuf is not as scalable as using a callback; the rest of the
run_processes_parallel() API also uses callbacks, so making this feature
match the rest of the API reduces mental load on the user.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 builtin/fetch.c             |  1 +
 builtin/submodule--helper.c |  2 +-
 hook.c                      |  1 +
 run-command.c               | 54 +++++++++++++++++++++++++++++++++++--
 run-command.h               | 17 +++++++++++-
 submodule.c                 |  1 +
 t/helper/test-run-command.c | 31 ++++++++++++++++++---
 t/t0061-run-command.sh      | 30 +++++++++++++++++++++
 8 files changed, 129 insertions(+), 8 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index dfde96a435..a07816b650 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1817,6 +1817,7 @@ static int fetch_multiple(struct string_list *list, int max_children)
 		result = run_processes_parallel_tr2(max_children,
 						    &fetch_next_remote,
 						    &fetch_failed_to_start,
+						    NULL,
 						    &fetch_finished,
 						    &state,
 						    "fetch", "parallel/fetch");
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index d55f6262e9..8d1e731073 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2295,7 +2295,7 @@ static int update_submodules(struct submodule_update_clone *suc)
 	int i;
 
 	run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task,
-				   update_clone_start_failure,
+				   update_clone_start_failure, NULL,
 				   update_clone_task_finished, suc, "submodule",
 				   "parallel/update");
 
diff --git a/hook.c b/hook.c
index 920563607d..1fa7976583 100644
--- a/hook.c
+++ b/hook.c
@@ -402,6 +402,7 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options)
 	run_processes_parallel_tr2(options->jobs,
 				   pick_next_hook,
 				   notify_start_failure,
+				   NULL,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
diff --git a/run-command.c b/run-command.c
index 5237984ebe..6dd33caa57 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1558,6 +1558,7 @@ struct parallel_processes {
 
 	get_next_task_fn get_next_task;
 	start_failure_fn start_failure;
+	feed_pipe_fn feed_pipe;
 	task_finished_fn task_finished;
 
 	struct {
@@ -1585,6 +1586,13 @@ static int default_start_failure(struct strbuf *out,
 	return 0;
 }
 
+static int default_feed_pipe(struct strbuf *pipe,
+			     void *pp_cb,
+			     void *pp_task_cb)
+{
+	return 1;
+}
+
 static int default_task_finished(int result,
 				 struct strbuf *out,
 				 void *pp_cb,
@@ -1615,6 +1623,7 @@ static void pp_init(struct parallel_processes *pp,
 		    int n,
 		    get_next_task_fn get_next_task,
 		    start_failure_fn start_failure,
+		    feed_pipe_fn feed_pipe,
 		    task_finished_fn task_finished,
 		    void *data)
 {
@@ -1633,6 +1642,7 @@ static void pp_init(struct parallel_processes *pp,
 	pp->get_next_task = get_next_task;
 
 	pp->start_failure = start_failure ? start_failure : default_start_failure;
+	pp->feed_pipe = feed_pipe ? feed_pipe : default_feed_pipe;
 	pp->task_finished = task_finished ? task_finished : default_task_finished;
 
 	pp->nr_processes = 0;
@@ -1730,6 +1740,37 @@ static int pp_start_one(struct parallel_processes *pp)
 	return 0;
 }
 
+static void pp_buffer_stdin(struct parallel_processes *pp)
+{
+	int i;
+	struct strbuf sb = STRBUF_INIT;
+
+	/* Buffer stdin for each pipe. */
+	for (i = 0; i < pp->max_processes; i++) {
+		if (pp->children[i].state == GIT_CP_WORKING &&
+		    pp->children[i].process.in > 0) {
+			int done;
+			strbuf_reset(&sb);
+			done = pp->feed_pipe(&sb, pp->data,
+					      pp->children[i].data);
+			if (sb.len) {
+				if (write_in_full(pp->children[i].process.in,
+					      sb.buf, sb.len) < 0) {
+					if (errno != EPIPE)
+						die_errno("write");
+					done = 1;
+				}
+			}
+			if (done) {
+				close(pp->children[i].process.in);
+				pp->children[i].process.in = 0;
+			}
+		}
+	}
+
+	strbuf_release(&sb);
+}
+
 static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout)
 {
 	int i;
@@ -1794,6 +1835,7 @@ static int pp_collect_finished(struct parallel_processes *pp)
 		pp->nr_processes--;
 		pp->children[i].state = GIT_CP_FREE;
 		pp->pfd[i].fd = -1;
+		pp->children[i].process.in = 0;
 		child_process_init(&pp->children[i].process);
 
 		if (i != pp->output_owner) {
@@ -1827,6 +1869,7 @@ static int pp_collect_finished(struct parallel_processes *pp)
 int run_processes_parallel(int n,
 			   get_next_task_fn get_next_task,
 			   start_failure_fn start_failure,
+			   feed_pipe_fn feed_pipe,
 			   task_finished_fn task_finished,
 			   void *pp_cb)
 {
@@ -1835,7 +1878,9 @@ int run_processes_parallel(int n,
 	int spawn_cap = 4;
 	struct parallel_processes pp;
 
-	pp_init(&pp, n, get_next_task, start_failure, task_finished, pp_cb);
+	sigchain_push(SIGPIPE, SIG_IGN);
+
+	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, task_finished, pp_cb);
 	while (1) {
 		for (i = 0;
 		    i < spawn_cap && !pp.shutdown &&
@@ -1852,6 +1897,7 @@ int run_processes_parallel(int n,
 		}
 		if (!pp.nr_processes)
 			break;
+		pp_buffer_stdin(&pp);
 		pp_buffer_stderr(&pp, output_timeout);
 		pp_output(&pp);
 		code = pp_collect_finished(&pp);
@@ -1863,11 +1909,15 @@ int run_processes_parallel(int n,
 	}
 
 	pp_cleanup(&pp);
+
+	sigchain_pop(SIGPIPE);
+
 	return 0;
 }
 
 int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 			       start_failure_fn start_failure,
+			       feed_pipe_fn feed_pipe,
 			       task_finished_fn task_finished, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label)
 {
@@ -1877,7 +1927,7 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 				   ((n < 1) ? online_cpus() : n));
 
 	result = run_processes_parallel(n, get_next_task, start_failure,
-					task_finished, pp_cb);
+					feed_pipe, task_finished, pp_cb);
 
 	trace2_region_leave(tr2_category, tr2_label, NULL);
 
diff --git a/run-command.h b/run-command.h
index d08414a92e..1e3cf0999f 100644
--- a/run-command.h
+++ b/run-command.h
@@ -443,6 +443,20 @@ typedef int (*start_failure_fn)(struct strbuf *out,
 				void *pp_cb,
 				void *pp_task_cb);
 
+/**
+ * This callback is called repeatedly on every child process who requests
+ * start_command() to create a pipe by setting child_process.in < 0.
+ *
+ * pp_cb is the callback cookie as passed into run_processes_parallel, and
+ * pp_task_cb is the callback cookie as passed into get_next_task_fn.
+ * The contents of 'send' will be read into the pipe and passed to the pipe.
+ *
+ * Return nonzero to close the pipe.
+ */
+typedef int (*feed_pipe_fn)(struct strbuf *pipe,
+			    void *pp_cb,
+			    void *pp_task_cb);
+
 /**
  * This callback is called on every child process that finished processing.
  *
@@ -477,10 +491,11 @@ typedef int (*task_finished_fn)(int result,
 int run_processes_parallel(int n,
 			   get_next_task_fn,
 			   start_failure_fn,
+			   feed_pipe_fn,
 			   task_finished_fn,
 			   void *pp_cb);
 int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn,
-			       task_finished_fn, void *pp_cb,
+			       feed_pipe_fn, task_finished_fn, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label);
 
 #endif
diff --git a/submodule.c b/submodule.c
index 0b1d9c1dde..ea026a8195 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1645,6 +1645,7 @@ int fetch_populated_submodules(struct repository *r,
 	run_processes_parallel_tr2(max_parallel_jobs,
 				   get_next_submodule,
 				   fetch_start_failure,
+				   NULL,
 				   fetch_finish,
 				   &spf,
 				   "submodule", "parallel/fetch");
diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c
index 7ae03dc712..9348184d30 100644
--- a/t/helper/test-run-command.c
+++ b/t/helper/test-run-command.c
@@ -32,8 +32,13 @@ static int parallel_next(struct child_process *cp,
 		return 0;
 
 	strvec_pushv(&cp->args, d->argv);
+	cp->in = d->in;
+	cp->no_stdin = d->no_stdin;
 	strbuf_addstr(err, "preloaded output of a child\n");
 	number_callbacks++;
+
+	*task_cb = xmalloc(sizeof(int));
+	*(int*)(*task_cb) = 2;
 	return 1;
 }
 
@@ -55,6 +60,17 @@ static int task_finished(int result,
 	return 1;
 }
 
+static int test_stdin(struct strbuf *pipe, void *cb, void *task_cb)
+{
+	int *lines_remaining = task_cb;
+
+	if (*lines_remaining)
+		strbuf_addf(pipe, "sample stdin %d\n", --(*lines_remaining));
+
+	return !(*lines_remaining);
+}
+
+
 struct testsuite {
 	struct string_list tests, failed;
 	int next;
@@ -185,7 +201,7 @@ static int testsuite(int argc, const char **argv)
 		suite.tests.nr, max_jobs);
 
 	ret = run_processes_parallel(max_jobs, next_test, test_failed,
-				     test_finished, &suite);
+				     test_stdin, test_finished, &suite);
 
 	if (suite.failed.nr > 0) {
 		ret = 1;
@@ -413,15 +429,22 @@ int cmd__run_command(int argc, const char **argv)
 
 	if (!strcmp(argv[1], "run-command-parallel"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, &proc));
+					    NULL, NULL, NULL, &proc));
 
 	if (!strcmp(argv[1], "run-command-abort"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, task_finished, &proc));
+					    NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-no-jobs"))
 		exit(run_processes_parallel(jobs, no_job,
-					    NULL, task_finished, &proc));
+					    NULL, NULL, task_finished, &proc));
+
+	if (!strcmp(argv[1], "run-command-stdin")) {
+		proc.in = -1;
+		proc.no_stdin = 0;
+		exit (run_processes_parallel(jobs, parallel_next, NULL,
+					     test_stdin, NULL, &proc));
+	}
 
 	fprintf(stderr, "check usage\n");
 	return 1;
diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh
index 7d599675e3..87759482ad 100755
--- a/t/t0061-run-command.sh
+++ b/t/t0061-run-command.sh
@@ -143,6 +143,36 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai
 	test_cmp expect actual
 '
 
+cat >expect <<-EOF
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+EOF
+
+test_expect_success 'run_command listens to stdin' '
+	write_script stdin-script <<-\EOF &&
+	echo "listening for stdin:"
+	while read line; do
+		echo "$line"
+	done
+	EOF
+	test-tool run-command run-command-stdin 2 ./stdin-script 2>actual &&
+	test_cmp expect actual
+'
+
 cat >expect <<-EOF
 preloaded output of a child
 asking for a quick stop
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 14/37] hook: provide stdin by string_list or callback
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (12 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 13/37] run-command: add stdin callback for parallelization Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 15/37] run-command: allow capturing of collated output Emily Shaffer
                     ` (24 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

In cases where a hook requires only a small amount of information via
stdin, it should be simple for users to provide a string_list alone. But
in more complicated cases where the stdin is too large to hold in
memory, let's provide a callback the users can populate line after line
with instead.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 hook.c | 38 ++++++++++++++++++++++++++++++++++++--
 hook.h | 28 ++++++++++++++++++++++++++++
 2 files changed, 64 insertions(+), 2 deletions(-)

diff --git a/hook.c b/hook.c
index 1fa7976583..8aa0d4fdf2 100644
--- a/hook.c
+++ b/hook.c
@@ -7,8 +7,10 @@
 
 void free_hook(struct hook *ptr)
 {
-	if (ptr)
+	if (ptr) {
 		strbuf_release(&ptr->command);
+		free(ptr->feed_pipe_cb_data);
+	}
 	free(ptr);
 }
 
@@ -38,6 +40,7 @@ static void append_or_move_hook(struct list_head *head, const char *command)
 		strbuf_init(&to_add->command, 0);
 		strbuf_addstr(&to_add->command, command);
 		to_add->from_hookdir = 0;
+		to_add->feed_pipe_cb_data = NULL;
 	}
 
 	/* re-set the scope so we show where an override was specified */
@@ -257,6 +260,8 @@ void run_hooks_opt_init_sync(struct run_hooks_opt *o)
 	o->run_hookdir = configured_hookdir_opt();
 	o->jobs = 1;
 	o->dir = NULL;
+	o->feed_pipe = NULL;
+	o->feed_pipe_ctx = NULL;
 }
 
 void run_hooks_opt_init_async(struct run_hooks_opt *o)
@@ -290,6 +295,28 @@ void run_hooks_opt_clear(struct run_hooks_opt *o)
 	strvec_clear(&o->args);
 }
 
+int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb)
+{
+	int *item_idx;
+	struct hook *ctx = pp_task_cb;
+	struct string_list *to_pipe = ((struct hook_cb_data*)pp_cb)->options->feed_pipe_ctx;
+
+	/* Bootstrap the state manager if necessary. */
+	if (!ctx->feed_pipe_cb_data) {
+		ctx->feed_pipe_cb_data = xmalloc(sizeof(unsigned int));
+		*(int*)ctx->feed_pipe_cb_data = 0;
+	}
+
+	item_idx = ctx->feed_pipe_cb_data;
+
+	if (*item_idx < to_pipe->nr) {
+		strbuf_addf(pipe, "%s\n", to_pipe->items[*item_idx].string);
+		(*item_idx)++;
+		return 0;
+	}
+	return 1;
+}
+
 static int pick_next_hook(struct child_process *cp,
 			  struct strbuf *out,
 			  void *pp_cb,
@@ -305,6 +332,10 @@ static int pick_next_hook(struct child_process *cp,
 	if (hook_cb->options->path_to_stdin) {
 		cp->no_stdin = 0;
 		cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY);
+	} else if (hook_cb->options->feed_pipe) {
+		/* ask for start_command() to make a pipe for us */
+		cp->in = -1;
+		cp->no_stdin = 0;
 	} else {
 		cp->no_stdin = 1;
 	}
@@ -383,6 +414,9 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options)
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
 
+	if (options->path_to_stdin && options->feed_pipe)
+		BUG("choose only one method to populate stdin");
+
 	to_run = hook_list(hookname);
 
 	list_for_each_safe(pos, tmp, to_run) {
@@ -402,7 +436,7 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options)
 	run_processes_parallel_tr2(options->jobs,
 				   pick_next_hook,
 				   notify_start_failure,
-				   NULL,
+				   options->feed_pipe,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
diff --git a/hook.h b/hook.h
index 218b9e1721..aba62d4112 100644
--- a/hook.h
+++ b/hook.h
@@ -2,6 +2,7 @@
 #include "list.h"
 #include "strbuf.h"
 #include "strvec.h"
+#include "run-command.h"
 
 struct hook {
 	struct list_head list;
@@ -13,6 +14,12 @@ struct hook {
 	/* The literal command to run. */
 	struct strbuf command;
 	unsigned from_hookdir : 1;
+
+	/*
+	 * Use this to keep state for your feed_pipe_fn if you are using
+	 * run_hooks_opt.feed_pipe. Otherwise, do not touch it.
+	 */
+	void *feed_pipe_cb_data;
 };
 
 /*
@@ -58,14 +65,35 @@ struct run_hooks_opt
 
 	/* Path to file which should be piped to stdin for each hook */
 	const char *path_to_stdin;
+	/*
+	 * Callback and state pointer to ask for more content to pipe to stdin.
+	 * Will be called repeatedly, for each hook. See
+	 * hook.c:pipe_from_stdin() for an example. Keep per-hook state in
+	 * hook.feed_pipe_cb_data (per process). Keep initialization context in
+	 * feed_pipe_ctx (shared by all processes).
+	 *
+	 * See 'pipe_from_string_list()' for info about how to specify a
+	 * string_list as the stdin input instead of writing your own handler.
+	 */
+	feed_pipe_fn feed_pipe;
+	void *feed_pipe_ctx;
 
 	/* Number of threads to parallelize across */
 	int jobs;
 
 	/* Path to initial working directory for subprocess */
 	const char *dir;
+
 };
 
+/*
+ * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
+ * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'.
+ * This will pipe each string in the list to stdin, separated by newlines.  (Do
+ * not inject your own newlines.)
+ */
+int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb);
+
 /*
  * Callback provided to feed_pipe_fn and consume_sideband_fn.
  */
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 15/37] run-command: allow capturing of collated output
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (13 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 14/37] hook: provide stdin by string_list or callback Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 16/37] hooks: allow callers to capture output Emily Shaffer
                     ` (23 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Some callers, for example server-side hooks which wish to relay hook
output to clients across a transport, want to capture what would
normally print to stderr and do something else with it. Allow that via a
callback.

By calling the callback regardless of whether there's output available,
we allow clients to send e.g. a keepalive if necessary.

Because we expose a strbuf, not a fd or FILE*, there's no need to create
a temporary pipe or similar - we can just skip the print to stderr and
instead hand it to the caller.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

Notes:
    Originally when writing this patch I attempted to use a pipe in memory -
    but managing its lifetime was actually pretty tricky, and I found I could
    achieve the same thing with less code by doing it this way. Critique welcome,
    including "no, you really need to do it with a pipe".

 builtin/fetch.c             |  2 +-
 builtin/submodule--helper.c |  2 +-
 hook.c                      |  1 +
 run-command.c               | 33 +++++++++++++++++++++++++--------
 run-command.h               | 18 +++++++++++++++++-
 submodule.c                 |  2 +-
 t/helper/test-run-command.c | 25 ++++++++++++++++++++-----
 t/t0061-run-command.sh      |  7 +++++++
 8 files changed, 73 insertions(+), 17 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index a07816b650..769af53ca4 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1817,7 +1817,7 @@ static int fetch_multiple(struct string_list *list, int max_children)
 		result = run_processes_parallel_tr2(max_children,
 						    &fetch_next_remote,
 						    &fetch_failed_to_start,
-						    NULL,
+						    NULL, NULL,
 						    &fetch_finished,
 						    &state,
 						    "fetch", "parallel/fetch");
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 8d1e731073..fef8392e1d 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2295,7 +2295,7 @@ static int update_submodules(struct submodule_update_clone *suc)
 	int i;
 
 	run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task,
-				   update_clone_start_failure, NULL,
+				   update_clone_start_failure, NULL, NULL,
 				   update_clone_task_finished, suc, "submodule",
 				   "parallel/update");
 
diff --git a/hook.c b/hook.c
index 8aa0d4fdf2..d48071bb1d 100644
--- a/hook.c
+++ b/hook.c
@@ -437,6 +437,7 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options)
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
+				   NULL,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
diff --git a/run-command.c b/run-command.c
index 6dd33caa57..27135defb8 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1559,6 +1559,7 @@ struct parallel_processes {
 	get_next_task_fn get_next_task;
 	start_failure_fn start_failure;
 	feed_pipe_fn feed_pipe;
+	consume_sideband_fn consume_sideband;
 	task_finished_fn task_finished;
 
 	struct {
@@ -1624,6 +1625,7 @@ static void pp_init(struct parallel_processes *pp,
 		    get_next_task_fn get_next_task,
 		    start_failure_fn start_failure,
 		    feed_pipe_fn feed_pipe,
+		    consume_sideband_fn consume_sideband,
 		    task_finished_fn task_finished,
 		    void *data)
 {
@@ -1644,6 +1646,7 @@ static void pp_init(struct parallel_processes *pp,
 	pp->start_failure = start_failure ? start_failure : default_start_failure;
 	pp->feed_pipe = feed_pipe ? feed_pipe : default_feed_pipe;
 	pp->task_finished = task_finished ? task_finished : default_task_finished;
+	pp->consume_sideband = consume_sideband;
 
 	pp->nr_processes = 0;
 	pp->output_owner = 0;
@@ -1680,7 +1683,10 @@ static void pp_cleanup(struct parallel_processes *pp)
 	 * When get_next_task added messages to the buffer in its last
 	 * iteration, the buffered output is non empty.
 	 */
-	strbuf_write(&pp->buffered_output, stderr);
+	if (pp->consume_sideband)
+		pp->consume_sideband(&pp->buffered_output, pp->data);
+	else
+		strbuf_write(&pp->buffered_output, stderr);
 	strbuf_release(&pp->buffered_output);
 
 	sigchain_pop_common();
@@ -1801,9 +1807,13 @@ static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout)
 static void pp_output(struct parallel_processes *pp)
 {
 	int i = pp->output_owner;
+
 	if (pp->children[i].state == GIT_CP_WORKING &&
 	    pp->children[i].err.len) {
-		strbuf_write(&pp->children[i].err, stderr);
+		if (pp->consume_sideband)
+			pp->consume_sideband(&pp->children[i].err, pp->data);
+		else
+			strbuf_write(&pp->children[i].err, stderr);
 		strbuf_reset(&pp->children[i].err);
 	}
 }
@@ -1842,11 +1852,15 @@ static int pp_collect_finished(struct parallel_processes *pp)
 			strbuf_addbuf(&pp->buffered_output, &pp->children[i].err);
 			strbuf_reset(&pp->children[i].err);
 		} else {
-			strbuf_write(&pp->children[i].err, stderr);
+			/* Output errors, then all other finished child processes */
+			if (pp->consume_sideband) {
+				pp->consume_sideband(&pp->children[i].err, pp->data);
+				pp->consume_sideband(&pp->buffered_output, pp->data);
+			} else {
+				strbuf_write(&pp->children[i].err, stderr);
+				strbuf_write(&pp->buffered_output, stderr);
+			}
 			strbuf_reset(&pp->children[i].err);
-
-			/* Output all other finished child processes */
-			strbuf_write(&pp->buffered_output, stderr);
 			strbuf_reset(&pp->buffered_output);
 
 			/*
@@ -1870,6 +1884,7 @@ int run_processes_parallel(int n,
 			   get_next_task_fn get_next_task,
 			   start_failure_fn start_failure,
 			   feed_pipe_fn feed_pipe,
+			   consume_sideband_fn consume_sideband,
 			   task_finished_fn task_finished,
 			   void *pp_cb)
 {
@@ -1880,7 +1895,7 @@ int run_processes_parallel(int n,
 
 	sigchain_push(SIGPIPE, SIG_IGN);
 
-	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, task_finished, pp_cb);
+	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, consume_sideband, task_finished, pp_cb);
 	while (1) {
 		for (i = 0;
 		    i < spawn_cap && !pp.shutdown &&
@@ -1918,6 +1933,7 @@ int run_processes_parallel(int n,
 int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 			       start_failure_fn start_failure,
 			       feed_pipe_fn feed_pipe,
+			       consume_sideband_fn consume_sideband,
 			       task_finished_fn task_finished, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label)
 {
@@ -1927,7 +1943,8 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 				   ((n < 1) ? online_cpus() : n));
 
 	result = run_processes_parallel(n, get_next_task, start_failure,
-					feed_pipe, task_finished, pp_cb);
+					feed_pipe, consume_sideband,
+					task_finished, pp_cb);
 
 	trace2_region_leave(tr2_category, tr2_label, NULL);
 
diff --git a/run-command.h b/run-command.h
index 1e3cf0999f..ebc4a95a94 100644
--- a/run-command.h
+++ b/run-command.h
@@ -457,6 +457,20 @@ typedef int (*feed_pipe_fn)(struct strbuf *pipe,
 			    void *pp_cb,
 			    void *pp_task_cb);
 
+/**
+ * If this callback is provided, instead of collating process output to stderr,
+ * they will be collated into a new pipe. consume_sideband_fn will be called
+ * repeatedly. When output is available on that pipe, it will be contained in
+ * 'output'. But it will be called with an empty 'output' too, to allow for
+ * keepalives or similar operations if necessary.
+ *
+ * pp_cb is the callback cookie as passed into run_processes_parallel.
+ *
+ * Since this callback is provided with the collated output, no task cookie is
+ * provided.
+ */
+typedef void (*consume_sideband_fn)(struct strbuf *output, void *pp_cb);
+
 /**
  * This callback is called on every child process that finished processing.
  *
@@ -492,10 +506,12 @@ int run_processes_parallel(int n,
 			   get_next_task_fn,
 			   start_failure_fn,
 			   feed_pipe_fn,
+			   consume_sideband_fn,
 			   task_finished_fn,
 			   void *pp_cb);
 int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn,
-			       feed_pipe_fn, task_finished_fn, void *pp_cb,
+			       feed_pipe_fn, consume_sideband_fn,
+			       task_finished_fn, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label);
 
 #endif
diff --git a/submodule.c b/submodule.c
index ea026a8195..7fe0c8f7c9 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1645,7 +1645,7 @@ int fetch_populated_submodules(struct repository *r,
 	run_processes_parallel_tr2(max_parallel_jobs,
 				   get_next_submodule,
 				   fetch_start_failure,
-				   NULL,
+				   NULL, NULL,
 				   fetch_finish,
 				   &spf,
 				   "submodule", "parallel/fetch");
diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c
index 9348184d30..d53db6d11c 100644
--- a/t/helper/test-run-command.c
+++ b/t/helper/test-run-command.c
@@ -51,6 +51,16 @@ static int no_job(struct child_process *cp,
 	return 0;
 }
 
+static void test_consume_sideband(struct strbuf *output, void *cb)
+{
+	FILE *sideband;
+
+	sideband = fopen("./sideband", "a");
+
+	strbuf_write(output, sideband);
+	fclose(sideband);
+}
+
 static int task_finished(int result,
 			 struct strbuf *err,
 			 void *pp_cb,
@@ -201,7 +211,7 @@ static int testsuite(int argc, const char **argv)
 		suite.tests.nr, max_jobs);
 
 	ret = run_processes_parallel(max_jobs, next_test, test_failed,
-				     test_stdin, test_finished, &suite);
+				     test_stdin, NULL, test_finished, &suite);
 
 	if (suite.failed.nr > 0) {
 		ret = 1;
@@ -429,23 +439,28 @@ int cmd__run_command(int argc, const char **argv)
 
 	if (!strcmp(argv[1], "run-command-parallel"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, NULL, &proc));
+					    NULL, NULL, NULL, NULL, &proc));
 
 	if (!strcmp(argv[1], "run-command-abort"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, task_finished, &proc));
+					    NULL, NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-no-jobs"))
 		exit(run_processes_parallel(jobs, no_job,
-					    NULL, NULL, task_finished, &proc));
+					    NULL, NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-stdin")) {
 		proc.in = -1;
 		proc.no_stdin = 0;
 		exit (run_processes_parallel(jobs, parallel_next, NULL,
-					     test_stdin, NULL, &proc));
+					     test_stdin, NULL, NULL, &proc));
 	}
 
+	if (!strcmp(argv[1], "run-command-sideband"))
+		exit(run_processes_parallel(jobs, parallel_next, NULL, NULL,
+					    test_consume_sideband, NULL,
+					    &proc));
+
 	fprintf(stderr, "check usage\n");
 	return 1;
 }
diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh
index 87759482ad..e99f6c7f44 100755
--- a/t/t0061-run-command.sh
+++ b/t/t0061-run-command.sh
@@ -143,6 +143,13 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai
 	test_cmp expect actual
 '
 
+test_expect_success 'run_command can divert output' '
+	test_when_finished rm sideband &&
+	test-tool run-command run-command-sideband 3 sh -c "printf \"%s\n%s\n\" Hello World" 2>actual &&
+	test_must_be_empty actual &&
+	test_cmp expect sideband
+'
+
 cat >expect <<-EOF
 preloaded output of a child
 listening for stdin:
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 16/37] hooks: allow callers to capture output
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (14 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 15/37] run-command: allow capturing of collated output Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 17/37] commit: use config-based hooks Emily Shaffer
                     ` (22 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Some server-side hooks will require capturing output to send over
sideband instead of printing directly to stderr. Expose that capability.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

Notes:
    You can see this in practice in the conversions for some of the push hooks,
    like 'receive-pack'.

 hook.c | 3 ++-
 hook.h | 8 ++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/hook.c b/hook.c
index d48071bb1d..27da1fdb32 100644
--- a/hook.c
+++ b/hook.c
@@ -262,6 +262,7 @@ void run_hooks_opt_init_sync(struct run_hooks_opt *o)
 	o->dir = NULL;
 	o->feed_pipe = NULL;
 	o->feed_pipe_ctx = NULL;
+	o->consume_sideband = NULL;
 }
 
 void run_hooks_opt_init_async(struct run_hooks_opt *o)
@@ -437,7 +438,7 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options)
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
-				   NULL,
+				   options->consume_sideband,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
diff --git a/hook.h b/hook.h
index aba62d4112..f32189380a 100644
--- a/hook.h
+++ b/hook.h
@@ -78,6 +78,14 @@ struct run_hooks_opt
 	feed_pipe_fn feed_pipe;
 	void *feed_pipe_ctx;
 
+	/*
+	 * Populate this to capture output and prevent it from being printed to
+	 * stderr. This will be passed directly through to
+	 * run_command:run_parallel_processes(). See t/helper/test-run-command.c
+	 * for an example.
+	 */
+	consume_sideband_fn consume_sideband;
+
 	/* Number of threads to parallelize across */
 	int jobs;
 
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 17/37] commit: use config-based hooks
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (15 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 16/37] hooks: allow callers to capture output Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 18/37] am: convert applypatch hooks to use config Emily Shaffer
                     ` (21 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

As part of the adoption of config-based hooks, teach run_commit_hook()
to call hook.h instead of run-command.h. This covers 'pre-commit',
'commit-msg', and 'prepare-commit-msg'. Additionally, ask the hook
library - not run-command - whether any hooks will be run, as it's
possible hooks may exist in the config but not the hookdir.

Because all but 'post-commit' hooks are expected to make some state
change, force all but 'post-commit' hook to run in series. 'post-commit'
"is meant primarily for notification, and cannot affect the outcome of
`git commit`," so it is fine to run in parallel.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt                    | 13 +++++++++++
 builtin/commit.c                              | 11 +++++-----
 builtin/merge.c                               |  9 ++++----
 commit.c                                      | 22 ++++++++++++++-----
 commit.h                                      |  3 ++-
 sequencer.c                                   |  7 +++---
 ...3-pre-commit-and-pre-merge-commit-hooks.sh | 17 ++++++++++++--
 7 files changed, 61 insertions(+), 21 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index b51959ff94..4af202b366 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -103,6 +103,8 @@ The default 'pre-commit' hook, when enabled--and with the
 `hooks.allownonascii` config option unset or set to false--prevents
 the use of non-ASCII filenames.
 
+Hooks executed during 'pre-commit' will not be parallelized.
+
 pre-merge-commit
 ~~~~~~~~~~~~~~~~
 
@@ -125,6 +127,8 @@ need to be resolved and the result committed separately (see
 linkgit:git-merge[1]). At that point, this hook will not be executed,
 but the 'pre-commit' hook will, if it is enabled.
 
+Hooks executed during 'pre-merge-commit' will not be parallelized.
+
 prepare-commit-msg
 ~~~~~~~~~~~~~~~~~~
 
@@ -150,6 +154,9 @@ be used as replacement for pre-commit hook.
 The sample `prepare-commit-msg` hook that comes with Git removes the
 help message found in the commented portion of the commit template.
 
+Hooks executed during 'prepare-commit-msg' will not be parallelized, because
+hooks are expected to edit the file containing the commit log message.
+
 commit-msg
 ~~~~~~~~~~
 
@@ -166,6 +173,9 @@ file.
 The default 'commit-msg' hook, when enabled, detects duplicate
 `Signed-off-by` trailers, and aborts the commit if one is found.
 
+Hooks executed during 'commit-msg' will not be parallelized, because hooks are
+expected to edit the file containing the proposed commit log message.
+
 post-commit
 ~~~~~~~~~~~
 
@@ -175,6 +185,9 @@ invoked after a commit is made.
 This hook is meant primarily for notification, and cannot affect
 the outcome of `git commit`.
 
+Hooks executed during 'post-commit' will run in parallel, unless hook.jobs is
+configured to 1.
+
 pre-rebase
 ~~~~~~~~~~
 
diff --git a/builtin/commit.c b/builtin/commit.c
index 190d215d43..7e01802961 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -36,6 +36,7 @@
 #include "help.h"
 #include "commit-reach.h"
 #include "commit-graph.h"
+#include "hook.h"
 
 static const char * const builtin_commit_usage[] = {
 	N_("git commit [<options>] [--] <pathspec>..."),
@@ -728,7 +729,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	/* This checks and barfs if author is badly specified */
 	determine_author_info(author_ident);
 
-	if (!no_verify && run_commit_hook(use_editor, index_file, "pre-commit", NULL))
+	if (!no_verify && run_commit_hook(use_editor, 0, index_file, "pre-commit", NULL))
 		return 0;
 
 	if (squash_message) {
@@ -1044,7 +1045,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (!no_verify && find_hook("pre-commit")) {
+	if (!no_verify && hook_exists("pre-commit", HOOKDIR_USE_CONFIG)) {
 		/*
 		 * Re-read the index as pre-commit hook could have updated it,
 		 * and write it out as a tree.  We must do this before we invoke
@@ -1059,7 +1060,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (run_commit_hook(use_editor, index_file, "prepare-commit-msg",
+	if (run_commit_hook(use_editor, 0, index_file, "prepare-commit-msg",
 			    git_path_commit_editmsg(), hook_arg1, hook_arg2, NULL))
 		return 0;
 
@@ -1076,7 +1077,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	}
 
 	if (!no_verify &&
-	    run_commit_hook(use_editor, index_file, "commit-msg", git_path_commit_editmsg(), NULL)) {
+	    run_commit_hook(use_editor, 0, index_file, "commit-msg", git_path_commit_editmsg(), NULL)) {
 		return 0;
 	}
 
@@ -1829,7 +1830,7 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 
 	repo_rerere(the_repository, 0);
 	run_auto_maintenance(quiet);
-	run_commit_hook(use_editor, get_index_file(), "post-commit", NULL);
+	run_commit_hook(use_editor, 1, get_index_file(), "post-commit", NULL);
 	if (amend && !no_post_rewrite) {
 		commit_post_rewrite(the_repository, current_head, &oid);
 	}
diff --git a/builtin/merge.c b/builtin/merge.c
index eddb8ae70d..182dea418a 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -43,6 +43,7 @@
 #include "commit-reach.h"
 #include "wt-status.h"
 #include "commit-graph.h"
+#include "hook.h"
 
 #define DEFAULT_TWOHEAD (1<<0)
 #define DEFAULT_OCTOPUS (1<<1)
@@ -841,14 +842,14 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 	struct strbuf msg = STRBUF_INIT;
 	const char *index_file = get_index_file();
 
-	if (!no_verify && run_commit_hook(0 < option_edit, index_file, "pre-merge-commit", NULL))
+	if (!no_verify && run_commit_hook(0 < option_edit, 0, index_file, "pre-merge-commit", NULL))
 		abort_commit(remoteheads, NULL);
 	/*
 	 * Re-read the index as pre-merge-commit hook could have updated it,
 	 * and write it out as a tree.  We must do this before we invoke
 	 * the editor and after we invoke run_status above.
 	 */
-	if (find_hook("pre-merge-commit"))
+	if (hook_exists("pre-merge-commit", HOOKDIR_USE_CONFIG))
 		discard_cache();
 	read_cache_from(index_file);
 	strbuf_addbuf(&msg, &merge_msg);
@@ -869,7 +870,7 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 		append_signoff(&msg, ignore_non_trailer(msg.buf, msg.len), 0);
 	write_merge_heads(remoteheads);
 	write_file_buf(git_path_merge_msg(the_repository), msg.buf, msg.len);
-	if (run_commit_hook(0 < option_edit, get_index_file(), "prepare-commit-msg",
+	if (run_commit_hook(0 < option_edit, 0, get_index_file(), "prepare-commit-msg",
 			    git_path_merge_msg(the_repository), "merge", NULL))
 		abort_commit(remoteheads, NULL);
 	if (0 < option_edit) {
@@ -877,7 +878,7 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 			abort_commit(remoteheads, NULL);
 	}
 
-	if (!no_verify && run_commit_hook(0 < option_edit, get_index_file(),
+	if (!no_verify && run_commit_hook(0 < option_edit, 0, get_index_file(),
 					  "commit-msg",
 					  git_path_merge_msg(the_repository), NULL))
 		abort_commit(remoteheads, NULL);
diff --git a/commit.c b/commit.c
index 8ea55a447f..0da5b7e7f1 100644
--- a/commit.c
+++ b/commit.c
@@ -21,6 +21,7 @@
 #include "commit-reach.h"
 #include "run-command.h"
 #include "shallow.h"
+#include "hook.h"
 
 static struct commit_extra_header *read_commit_extra_header_lines(const char *buf, size_t len, const char **);
 
@@ -1695,25 +1696,34 @@ size_t ignore_non_trailer(const char *buf, size_t len)
 	return boc ? len - boc : len - cutoff;
 }
 
-int run_commit_hook(int editor_is_used, const char *index_file,
+int run_commit_hook(int editor_is_used, int parallelize, const char *index_file,
 		    const char *name, ...)
 {
-	struct strvec hook_env = STRVEC_INIT;
+	struct run_hooks_opt opt;
 	va_list args;
+	const char *arg;
 	int ret;
 
-	strvec_pushf(&hook_env, "GIT_INDEX_FILE=%s", index_file);
+	run_hooks_opt_init_sync(&opt);
+
+	if (parallelize)
+		opt.jobs = configured_hook_jobs();
+
+	strvec_pushf(&opt.env, "GIT_INDEX_FILE=%s", index_file);
 
 	/*
 	 * Let the hook know that no editor will be launched.
 	 */
 	if (!editor_is_used)
-		strvec_push(&hook_env, "GIT_EDITOR=:");
+		strvec_push(&opt.env, "GIT_EDITOR=:");
 
 	va_start(args, name);
-	ret = run_hook_ve(hook_env.v, name, args);
+	while ((arg = va_arg(args, const char *)))
+		strvec_push(&opt.args, arg);
 	va_end(args);
-	strvec_clear(&hook_env);
+
+	ret = run_hooks(name, &opt);
+	run_hooks_opt_clear(&opt);
 
 	return ret;
 }
diff --git a/commit.h b/commit.h
index df42eb434f..a90c094ec2 100644
--- a/commit.h
+++ b/commit.h
@@ -363,7 +363,8 @@ int compare_commits_by_commit_date(const void *a_, const void *b_, void *unused)
 int compare_commits_by_gen_then_commit_date(const void *a_, const void *b_, void *unused);
 
 LAST_ARG_MUST_BE_NULL
-int run_commit_hook(int editor_is_used, const char *index_file, const char *name, ...);
+int run_commit_hook(int editor_is_used, int parallelize, const char *index_file,
+		    const char *name, ...);
 
 /* Sign a commit or tag buffer, storing the result in a header. */
 int sign_with_header(struct strbuf *buf, const char *keyid);
diff --git a/sequencer.c b/sequencer.c
index 0bec01cf38..72234af8ed 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -34,6 +34,7 @@
 #include "commit-reach.h"
 #include "rebase-interactive.h"
 #include "reset.h"
+#include "hook.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -1207,7 +1208,7 @@ static int run_prepare_commit_msg_hook(struct repository *r,
 	} else {
 		arg1 = "message";
 	}
-	if (run_commit_hook(0, r->index_file, "prepare-commit-msg", name,
+	if (run_commit_hook(0, 0, r->index_file, "prepare-commit-msg", name,
 			    arg1, arg2, NULL))
 		ret = error(_("'prepare-commit-msg' hook failed"));
 
@@ -1445,7 +1446,7 @@ static int try_to_commit(struct repository *r,
 		}
 	}
 
-	if (find_hook("prepare-commit-msg")) {
+	if (hook_exists("prepare-commit-msg", HOOKDIR_USE_CONFIG)) {
 		res = run_prepare_commit_msg_hook(r, msg, hook_commit);
 		if (res)
 			goto out;
@@ -1537,7 +1538,7 @@ static int try_to_commit(struct repository *r,
 		goto out;
 	}
 
-	run_commit_hook(0, r->index_file, "post-commit", NULL);
+	run_commit_hook(0, 1, r->index_file, "post-commit", NULL);
 	if (flags & AMEND_MSG)
 		commit_post_rewrite(r, current_head, oid);
 
diff --git a/t/t7503-pre-commit-and-pre-merge-commit-hooks.sh b/t/t7503-pre-commit-and-pre-merge-commit-hooks.sh
index 606d8d0f08..e9e3713033 100755
--- a/t/t7503-pre-commit-and-pre-merge-commit-hooks.sh
+++ b/t/t7503-pre-commit-and-pre-merge-commit-hooks.sh
@@ -8,8 +8,8 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
 . ./test-lib.sh
 
 HOOKDIR="$(git rev-parse --git-dir)/hooks"
-PRECOMMIT="$HOOKDIR/pre-commit"
-PREMERGE="$HOOKDIR/pre-merge-commit"
+PRECOMMIT="$(pwd)/$HOOKDIR/pre-commit"
+PREMERGE="$(pwd)/$HOOKDIR/pre-merge-commit"
 
 # Prepare sample scripts that write their $0 to actual_hooks
 test_expect_success 'sample script setup' '
@@ -106,6 +106,19 @@ test_expect_success 'with succeeding hook' '
 	test_cmp expected_hooks actual_hooks
 '
 
+# NEEDSWORK: when 'git hook add' and 'git hook remove' have been added, use that
+# instead
+test_expect_success 'with succeeding hook (config-based)' '
+	test_when_finished "git config --unset hook.pre-commit.command success.sample" &&
+	test_when_finished "rm -f expected_hooks actual_hooks" &&
+	git config hook.pre-commit.command "$HOOKDIR/success.sample" &&
+	echo "$HOOKDIR/success.sample" >expected_hooks &&
+	echo "more" >>file &&
+	git add file &&
+	git commit -m "more" &&
+	test_cmp expected_hooks actual_hooks
+'
+
 test_expect_success 'with succeeding hook (merge)' '
 	test_when_finished "rm -f \"$PREMERGE\" expected_hooks actual_hooks" &&
 	cp "$HOOKDIR/success.sample" "$PREMERGE" &&
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 18/37] am: convert applypatch hooks to use config
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (16 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 17/37] commit: use config-based hooks Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 19/37] merge: use config-based hooks for post-merge hook Emily Shaffer
                     ` (20 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach pre-applypatch, post-applypatch, and applypatch-msg to use the
hook.h library instead of the run-command.h library. This enables use of
hooks specified in the config, in addition to those in the hookdir.
These three hooks are called only by builtin/am.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt |  9 +++++++++
 builtin/am.c               | 21 ++++++++++++++++++---
 2 files changed, 27 insertions(+), 3 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 4af202b366..ec5020bfb6 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -58,6 +58,9 @@ the message file.
 The default 'applypatch-msg' hook, when enabled, runs the
 'commit-msg' hook, if the latter is enabled.
 
+Hooks run during 'applypatch-msg' will not be parallelized, because hooks are
+expected to edit the file holding the commit log message.
+
 pre-applypatch
 ~~~~~~~~~~~~~~
 
@@ -73,6 +76,9 @@ make a commit if it does not pass certain test.
 The default 'pre-applypatch' hook, when enabled, runs the
 'pre-commit' hook, if the latter is enabled.
 
+Hooks run during 'pre-applypatch' will be run in parallel, unless hook.jobs is
+configured to 1.
+
 post-applypatch
 ~~~~~~~~~~~~~~~
 
@@ -82,6 +88,9 @@ and is invoked after the patch is applied and a commit is made.
 This hook is meant primarily for notification, and cannot affect
 the outcome of `git am`.
 
+Hooks run during 'post-applypatch' will be run in parallel, unless hook.jobs is
+configured to 1.
+
 pre-commit
 ~~~~~~~~~~
 
diff --git a/builtin/am.c b/builtin/am.c
index 0b2d886c81..d84791859c 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -33,6 +33,7 @@
 #include "string-list.h"
 #include "packfile.h"
 #include "repository.h"
+#include "hook.h"
 
 /**
  * Returns the length of the first line of msg.
@@ -444,9 +445,14 @@ static void am_destroy(const struct am_state *state)
 static int run_applypatch_msg_hook(struct am_state *state)
 {
 	int ret;
+	struct run_hooks_opt opt;
+
+	run_hooks_opt_init_sync(&opt);
 
 	assert(state->msg);
-	ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL);
+	strvec_push(&opt.args, am_path(state, "final-commit"));
+	ret = run_hooks("applypatch-msg", &opt);
+	run_hooks_opt_clear(&opt);
 
 	if (!ret) {
 		FREE_AND_NULL(state->msg);
@@ -1606,9 +1612,16 @@ static void do_commit(const struct am_state *state)
 	struct commit_list *parents = NULL;
 	const char *reflog_msg, *author, *committer = NULL;
 	struct strbuf sb = STRBUF_INIT;
+	struct run_hooks_opt hook_opt;
+
+	run_hooks_opt_init_async(&hook_opt);
 
-	if (run_hook_le(NULL, "pre-applypatch", NULL))
+	if (run_hooks("pre-applypatch", &hook_opt)) {
+		run_hooks_opt_clear(&hook_opt);
 		exit(1);
+	}
+
+	run_hooks_opt_clear(&hook_opt);
 
 	if (write_cache_as_tree(&tree, 0, NULL))
 		die(_("git write-tree failed to write a tree"));
@@ -1659,8 +1672,10 @@ static void do_commit(const struct am_state *state)
 		fclose(fp);
 	}
 
-	run_hook_le(NULL, "post-applypatch", NULL);
+	run_hooks_opt_init_async(&hook_opt);
+	run_hooks("post-applypatch", &hook_opt);
 
+	run_hooks_opt_clear(&hook_opt);
 	strbuf_release(&sb);
 }
 
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 19/37] merge: use config-based hooks for post-merge hook
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (17 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 18/37] am: convert applypatch hooks to use config Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 20/37] gc: use hook library for pre-auto-gc hook Emily Shaffer
                     ` (19 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach post-merge to use the hook.h library instead of the run-command.h
library to run hooks. This means that post-merge hooks can come from the
config as well as from the hookdir. post-merge is invoked only from
builtin/merge.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt | 3 +++
 builtin/merge.c            | 6 +++++-
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index ec5020bfb6..c904b160dc 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -242,6 +242,9 @@ save and restore any form of metadata associated with the working tree
 (e.g.: permissions/ownership, ACLS, etc).  See contrib/hooks/setgitperms.perl
 for an example of how to do this.
 
+Hooks executed during 'post-merge' will run in parallel, unless hook.jobs is
+configured to 1.
+
 pre-push
 ~~~~~~~~
 
diff --git a/builtin/merge.c b/builtin/merge.c
index 182dea418a..7a524cb3e3 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -448,6 +448,7 @@ static void finish(struct commit *head_commit,
 		   const struct object_id *new_head, const char *msg)
 {
 	struct strbuf reflog_message = STRBUF_INIT;
+	struct run_hooks_opt opt;
 	const struct object_id *head = &head_commit->object.oid;
 
 	if (!msg)
@@ -489,7 +490,10 @@ static void finish(struct commit *head_commit,
 	}
 
 	/* Run a post-merge hook */
-	run_hook_le(NULL, "post-merge", squash ? "1" : "0", NULL);
+	run_hooks_opt_init_async(&opt);
+	strvec_push(&opt.args, squash ? "1" : "0");
+	run_hooks("post-merge", &opt);
+	run_hooks_opt_clear(&opt);
 
 	apply_autostash(git_path_merge_autostash(the_repository));
 	strbuf_release(&reflog_message);
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 20/37] gc: use hook library for pre-auto-gc hook
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (18 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 19/37] merge: use config-based hooks for post-merge hook Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 21/37] rebase: teach pre-rebase to use hook.h Emily Shaffer
                     ` (18 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Using the hook.h library instead of the run-command.h library to run
pre-auto-gc means that those hooks can be set up in config files, as
well as in the hookdir. pre-auto-gc is called only from builtin/gc.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt | 3 +++
 builtin/gc.c               | 9 ++++++++-
 2 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index c904b160dc..d77170dafb 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -569,6 +569,9 @@ This hook is invoked by `git gc --auto` (see linkgit:git-gc[1]). It
 takes no parameter, and exiting with non-zero status from this script
 causes the `git gc --auto` to abort.
 
+Hooks run during 'pre-auto-gc' will be run in parallel, unless hook.jobs is
+configured to 1.
+
 post-rewrite
 ~~~~~~~~~~~~
 
diff --git a/builtin/gc.c b/builtin/gc.c
index f05d2f0a1a..16890b097c 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -32,6 +32,7 @@
 #include "remote.h"
 #include "object-store.h"
 #include "exec-cmd.h"
+#include "hook.h"
 
 #define FAILED_RUN "failed to run %s"
 
@@ -348,6 +349,8 @@ static void add_repack_incremental_option(void)
 
 static int need_to_gc(void)
 {
+	struct run_hooks_opt hook_opt;
+
 	/*
 	 * Setting gc.auto to 0 or negative can disable the
 	 * automatic gc.
@@ -394,8 +397,12 @@ static int need_to_gc(void)
 	else
 		return 0;
 
-	if (run_hook_le(NULL, "pre-auto-gc", NULL))
+	run_hooks_opt_init_async(&hook_opt);
+	if (run_hooks("pre-auto-gc", &hook_opt)) {
+		run_hooks_opt_clear(&hook_opt);
 		return 0;
+	}
+	run_hooks_opt_clear(&hook_opt);
 	return 1;
 }
 
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 21/37] rebase: teach pre-rebase to use hook.h
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (19 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 20/37] gc: use hook library for pre-auto-gc hook Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 22/37] read-cache: convert post-index-change hook to use config Emily Shaffer
                     ` (17 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using hook.h instead of run-command.h to run hooks, pre-rebase hooks
can now be specified in the config as well as in the hookdir. pre-rebase
is not called anywhere besides builtin/rebase.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt |  3 +++
 builtin/rebase.c           | 10 ++++++++--
 2 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index d77170dafb..7ae24d65ec 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -206,6 +206,9 @@ two parameters.  The first parameter is the upstream from which
 the series was forked.  The second parameter is the branch being
 rebased, and is not set when rebasing the current branch.
 
+Hooks executed during 'pre-rebase' will run in parallel, unless hook.jobs is
+configured to 1.
+
 post-checkout
 ~~~~~~~~~~~~~
 
diff --git a/builtin/rebase.c b/builtin/rebase.c
index 12f093121d..fe9f144cad 100644
--- a/builtin/rebase.c
+++ b/builtin/rebase.c
@@ -28,6 +28,7 @@
 #include "sequencer.h"
 #include "rebase-interactive.h"
 #include "reset.h"
+#include "hook.h"
 
 #define DEFAULT_REFLOG_ACTION "rebase"
 
@@ -1313,6 +1314,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	char *squash_onto_name = NULL;
 	int reschedule_failed_exec = -1;
 	int allow_preemptive_ff = 1;
+	struct run_hooks_opt hook_opt;
 	struct option builtin_rebase_options[] = {
 		OPT_STRING(0, "onto", &options.onto_name,
 			   N_("revision"),
@@ -2022,10 +2024,14 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	}
 
 	/* If a hook exists, give it a chance to interrupt*/
+	run_hooks_opt_init_async(&hook_opt);
+	strvec_pushl(&hook_opt.args, options.upstream_arg, argc ? argv[0] : NULL, NULL);
 	if (!ok_to_skip_pre_rebase &&
-	    run_hook_le(NULL, "pre-rebase", options.upstream_arg,
-			argc ? argv[0] : NULL, NULL))
+	    run_hooks("pre-rebase", &hook_opt)) {
+		run_hooks_opt_clear(&hook_opt);
 		die(_("The pre-rebase hook refused to rebase."));
+	}
+	run_hooks_opt_clear(&hook_opt);
 
 	if (options.flags & REBASE_DIFFSTAT) {
 		struct diff_options opts;
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 22/37] read-cache: convert post-index-change hook to use config
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (20 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 21/37] rebase: teach pre-rebase to use hook.h Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27 23:04     ` Ævar Arnfjörð Bjarmason
  2021-05-27  0:08   ` [PATCH v9 23/37] receive-pack: convert push-to-checkout hook to hook.h Emily Shaffer
                     ` (16 subsequent siblings)
  38 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using hook.h instead of run-command.h to run, post-index-change hooks
can now be specified in the config in addition to the hookdir.
post-index-change is not run anywhere besides in read-cache.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt |  3 +++
 read-cache.c               | 14 +++++++++++---
 2 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 7ae24d65ec..5efa25a44a 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -729,6 +729,9 @@ and "0" meaning they were not.
 Only one parameter should be set to "1" when the hook runs.  The hook
 running passing "1", "1" should not be possible.
 
+Hooks run during 'post-index-change' will be run in parallel, unless hook.jobs
+is configured to 1.
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/read-cache.c b/read-cache.c
index 1b3c2eb408..6a5c9403f4 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -26,6 +26,8 @@
 #include "thread-utils.h"
 #include "progress.h"
 #include "sparse-index.h"
+#include "hook.h"
+>>>>>>> 9524a9d29d (read-cache: convert post-index-change hook to use config)
 
 /* Mask for the name length in ce_flags in the on-disk index */
 
@@ -3131,6 +3133,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 {
 	int ret;
 	int was_full = !istate->sparse_index;
+	struct run_hooks_opt hook_opt;
 
 	ret = convert_to_sparse(istate);
 
@@ -3159,9 +3162,14 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 	else
 		ret = close_lock_file_gently(lock);
 
-	run_hook_le(NULL, "post-index-change",
-			istate->updated_workdir ? "1" : "0",
-			istate->updated_skipworktree ? "1" : "0", NULL);
+	run_hooks_opt_init_async(&hook_opt);
+	strvec_pushl(&hook_opt.args,
+		     istate->updated_workdir ? "1" : "0",
+		     istate->updated_skipworktree ? "1" : "0",
+		     NULL);
+	run_hooks("post-index-change", &hook_opt);
+	run_hooks_opt_clear(&hook_opt);
+
 	istate->updated_workdir = 0;
 	istate->updated_skipworktree = 0;
 
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 23/37] receive-pack: convert push-to-checkout hook to hook.h
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (21 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 22/37] read-cache: convert post-index-change hook to use config Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 24/37] git-p4: use 'git hook' to run hooks Emily Shaffer
                     ` (15 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using hook.h instead of run-command.h to invoke push-to-checkout,
hooks can now be specified in the config as well as in the hookdir.
push-to-checkout is not called anywhere but in builtin/receive-pack.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt |  1 +
 builtin/receive-pack.c     | 17 +++++++++++++----
 2 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 5efa25a44a..17ffeebf07 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -564,6 +564,7 @@ that switches branches while
 keeping the local changes in the working tree that do not interfere
 with the difference between the branches.
 
+Hooks executed during 'push-to-checkout' will not be parallelized.
 
 pre-auto-gc
 ~~~~~~~~~~~
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index a34742513a..09e7cdee45 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -29,6 +29,7 @@
 #include "commit-reach.h"
 #include "worktree.h"
 #include "shallow.h"
+#include "hook.h"
 
 static const char * const receive_pack_usage[] = {
 	N_("git receive-pack <git-dir>"),
@@ -1435,12 +1436,20 @@ static const char *push_to_checkout(unsigned char *hash,
 				    struct strvec *env,
 				    const char *work_tree)
 {
+	struct run_hooks_opt opt;
+
+	run_hooks_opt_init_sync(&opt);
+
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
-	if (run_hook_le(env->v, push_to_checkout_hook,
-			hash_to_hex(hash), NULL))
+	strvec_pushv(&opt.env, env->v);
+	strvec_push(&opt.args, hash_to_hex(hash));
+	if (run_hooks(push_to_checkout_hook, &opt)) {
+		run_hooks_opt_clear(&opt);
 		return "push-to-checkout hook declined";
-	else
+	} else {
+		run_hooks_opt_clear(&opt);
 		return NULL;
+	}
 }
 
 static const char *update_worktree(unsigned char *sha1, const struct worktree *worktree)
@@ -1464,7 +1473,7 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 
 	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
 
-	if (!find_hook(push_to_checkout_hook))
+	if (!hook_exists(push_to_checkout_hook, HOOKDIR_USE_CONFIG))
 		retval = push_to_deploy(sha1, &env, work_tree);
 	else
 		retval = push_to_checkout(sha1, &env, work_tree);
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 24/37] git-p4: use 'git hook' to run hooks
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (22 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 23/37] receive-pack: convert push-to-checkout hook to hook.h Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 25/37] hooks: convert 'post-checkout' hook to hook library Emily Shaffer
                     ` (14 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Instead of duplicating the behavior of run-command.h:run_hook_le() in
Python, we can directly call 'git hook run'. As a bonus, this means
git-p4 learns how to find hook specifications from the Git config as
well as from the hookdir.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

Notes:
    Maybe there is a better way to do this - I had a hard time getting this to run
    locally, and Python is not my forte, so if anybody has a better approach I'd
    love to just take that patch instead :)
    
    Since v6, removed the developer debug print statements.... :X
    
    Maybe there is a better way to do this - I had a hard time getting this to run
    locally, and Python is not my forte, so if anybody has a better approach I'd
    love to just take that patch instead :)

 git-p4.py | 67 +++++--------------------------------------------------
 1 file changed, 6 insertions(+), 61 deletions(-)

diff --git a/git-p4.py b/git-p4.py
index d34a1946b7..b7192d1153 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -208,70 +208,15 @@ def decode_path(path):
 
 def run_git_hook(cmd, param=[]):
     """Execute a hook if the hook exists."""
-    if verbose:
-        sys.stderr.write("Looking for hook: %s\n" % cmd)
-        sys.stderr.flush()
-
-    hooks_path = gitConfig("core.hooksPath")
-    if len(hooks_path) <= 0:
-        hooks_path = os.path.join(os.environ["GIT_DIR"], "hooks")
-
-    if not isinstance(param, list):
-        param=[param]
-
-    # resolve hook file name, OS depdenent
-    hook_file = os.path.join(hooks_path, cmd)
-    if platform.system() == 'Windows':
-        if not os.path.isfile(hook_file):
-            # look for the file with an extension
-            files = glob.glob(hook_file + ".*")
-            if not files:
-                return True
-            files.sort()
-            hook_file = files.pop()
-            while hook_file.upper().endswith(".SAMPLE"):
-                # The file is a sample hook. We don't want it
-                if len(files) > 0:
-                    hook_file = files.pop()
-                else:
-                    return True
-
-    if not os.path.isfile(hook_file) or not os.access(hook_file, os.X_OK):
+    if not cmd:
         return True
 
-    return run_hook_command(hook_file, param) == 0
-
-def run_hook_command(cmd, param):
-    """Executes a git hook command
-       cmd = the command line file to be executed. This can be
-       a file that is run by OS association.
-
-       param = a list of parameters to pass to the cmd command
-
-       On windows, the extension is checked to see if it should
-       be run with the Git for Windows Bash shell.  If there
-       is no file extension, the file is deemed a bash shell
-       and will be handed off to sh.exe. Otherwise, Windows
-       will be called with the shell to handle the file assocation.
-
-       For non Windows operating systems, the file is called
-       as an executable.
-    """
-    cli = [cmd] + param
-    use_shell = False
-    if platform.system() == 'Windows':
-        (root,ext) = os.path.splitext(cmd)
-        if ext == "":
-            exe_path = os.environ.get("EXEPATH")
-            if exe_path is None:
-                exe_path = ""
-            else:
-                exe_path = os.path.join(exe_path, "bin")
-            cli = [os.path.join(exe_path, "SH.EXE")] + cli
-        else:
-            use_shell = True
-    return subprocess.call(cli, shell=use_shell)
+    """args are specified with -a <arg> -a <arg> -a <arg>"""
+    args = (['git', 'hook', 'run'] +
+	    ["-a" + arg for arg in param] +
+	    [cmd])
 
+    return subprocess.call(args) == 0
 
 def write_pipe(c, stdin):
     if verbose:
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 25/37] hooks: convert 'post-checkout' hook to hook library
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (23 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 24/37] git-p4: use 'git hook' to run hooks Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 26/37] hook: convert 'post-rewrite' hook to config Emily Shaffer
                     ` (13 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using the 'hook.h' library, 'post-checkout' hooks can now be
specified in the config as well as in the hook directory.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt |  2 ++
 builtin/checkout.c         | 19 ++++++++++++++-----
 builtin/clone.c            |  8 ++++++--
 builtin/worktree.c         | 32 ++++++++++++++++----------------
 read-cache.c               |  1 -
 reset.c                    | 17 +++++++++++++----
 6 files changed, 51 insertions(+), 28 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 17ffeebf07..6a5ff036ff 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -231,6 +231,8 @@ This hook can be used to perform repository validity checks, auto-display
 differences from the previous HEAD if different, or set working dir metadata
 properties.
 
+Hooks executed during 'post-checkout' will not be parallelized.
+
 post-merge
 ~~~~~~~~~~
 
diff --git a/builtin/checkout.c b/builtin/checkout.c
index f4cd7747d3..1797f05a50 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "diff.h"
 #include "dir.h"
+#include "hook.h"
 #include "ll-merge.h"
 #include "lockfile.h"
 #include "merge-recursive.h"
@@ -106,13 +107,21 @@ struct branch_info {
 static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit,
 			      int changed)
 {
-	return run_hook_le(NULL, "post-checkout",
-			   oid_to_hex(old_commit ? &old_commit->object.oid : null_oid()),
-			   oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()),
-			   changed ? "1" : "0", NULL);
+	struct run_hooks_opt opt;
+	int rc;
+
+	run_hooks_opt_init_sync(&opt);
+
 	/* "new_commit" can be NULL when checking out from the index before
 	   a commit exists. */
-
+	strvec_pushl(&opt.args,
+		     oid_to_hex(old_commit ? &old_commit->object.oid : null_oid()),
+		     oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()),
+		     changed ? "1" : "0",
+		     NULL);
+	rc = run_hooks("post-checkout", &opt);
+	run_hooks_opt_clear(&opt);
+	return rc;
 }
 
 static int update_some(const struct object_id *oid, struct strbuf *base,
diff --git a/builtin/clone.c b/builtin/clone.c
index eeb74c0217..2a2a03bf76 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -32,6 +32,7 @@
 #include "connected.h"
 #include "packfile.h"
 #include "list-objects-filter-options.h"
+#include "hook.h"
 
 /*
  * Overall FIXMEs:
@@ -775,6 +776,7 @@ static int checkout(int submodule_progress)
 	struct tree *tree;
 	struct tree_desc t;
 	int err = 0;
+	struct run_hooks_opt hook_opt;
 
 	if (option_no_checkout)
 		return 0;
@@ -820,8 +822,10 @@ static int checkout(int submodule_progress)
 	if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
 		die(_("unable to write new index file"));
 
-	err |= run_hook_le(NULL, "post-checkout", oid_to_hex(null_oid()),
-			   oid_to_hex(&oid), "1", NULL);
+	run_hooks_opt_init_sync(&hook_opt);
+	strvec_pushl(&hook_opt.args, oid_to_hex(null_oid()), oid_to_hex(&oid), "1", NULL);
+	err |= run_hooks("post-checkout", &hook_opt);
+	run_hooks_opt_clear(&hook_opt);
 
 	if (!err && (option_recurse_submodules.nr > 0)) {
 		struct strvec args = STRVEC_INIT;
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 976bf8ed06..017b2cfcb5 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -13,6 +13,7 @@
 #include "utf8.h"
 #include "worktree.h"
 #include "quote.h"
+#include "hook.h"
 
 static const char * const worktree_usage[] = {
 	N_("git worktree add [<options>] <path> [<commit-ish>]"),
@@ -381,22 +382,21 @@ static int add_worktree(const char *path, const char *refname,
 	 * is_junk is cleared, but do return appropriate code when hook fails.
 	 */
 	if (!ret && opts->checkout) {
-		const char *hook = find_hook("post-checkout");
-		if (hook) {
-			const char *env[] = { "GIT_DIR", "GIT_WORK_TREE", NULL };
-			cp.git_cmd = 0;
-			cp.no_stdin = 1;
-			cp.stdout_to_stderr = 1;
-			cp.dir = path;
-			cp.env = env;
-			cp.argv = NULL;
-			cp.trace2_hook_name = "post-checkout";
-			strvec_pushl(&cp.args, absolute_path(hook),
-				     oid_to_hex(null_oid()),
-				     oid_to_hex(&commit->object.oid),
-				     "1", NULL);
-			ret = run_command(&cp);
-		}
+		struct run_hooks_opt opt;
+
+		run_hooks_opt_init_sync(&opt);
+
+		strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
+		strvec_pushl(&opt.args,
+			     oid_to_hex(null_oid()),
+			     oid_to_hex(&commit->object.oid),
+			     "1",
+			     NULL);
+		opt.dir = path;
+
+		ret = run_hooks("post-checkout", &opt);
+
+		run_hooks_opt_clear(&opt);
 	}
 
 	strvec_clear(&child_env);
diff --git a/read-cache.c b/read-cache.c
index 6a5c9403f4..ebb9c19056 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -27,7 +27,6 @@
 #include "progress.h"
 #include "sparse-index.h"
 #include "hook.h"
->>>>>>> 9524a9d29d (read-cache: convert post-index-change hook to use config)
 
 /* Mask for the name length in ce_flags in the on-disk index */
 
diff --git a/reset.c b/reset.c
index 4bea758053..48d45f5b79 100644
--- a/reset.c
+++ b/reset.c
@@ -7,6 +7,7 @@
 #include "tree-walk.h"
 #include "tree.h"
 #include "unpack-trees.h"
+#include "hook.h"
 
 int reset_head(struct repository *r, struct object_id *oid, const char *action,
 	       const char *switch_to_branch, unsigned flags,
@@ -126,10 +127,18 @@ int reset_head(struct repository *r, struct object_id *oid, const char *action,
 			ret = create_symref("HEAD", switch_to_branch,
 					    reflog_head);
 	}
-	if (run_hook)
-		run_hook_le(NULL, "post-checkout",
-			    oid_to_hex(orig ? orig : null_oid()),
-			    oid_to_hex(oid), "1", NULL);
+	if (run_hook) {
+		struct run_hooks_opt opt;
+
+		run_hooks_opt_init_sync(&opt);
+		strvec_pushl(&opt.args,
+			     oid_to_hex(orig ? orig : null_oid()),
+			     oid_to_hex(oid),
+			     "1",
+			     NULL);
+		run_hooks("post-checkout", &opt);
+		run_hooks_opt_clear(&opt);
+	}
 
 leave_reset_head:
 	strbuf_release(&msg);
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 26/37] hook: convert 'post-rewrite' hook to config
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (24 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 25/37] hooks: convert 'post-checkout' hook to hook library Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 27/37] transport: convert pre-push hook to use config Emily Shaffer
                     ` (12 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using 'hook.h' for 'post-rewrite', we simplify hook invocations by
not needing to put together our own 'struct child_process' and we also
learn to run hooks specified in the config as well as the hook dir.

The signal handling that's being removed by this commit now takes place
in run-command.h:run_processes_parallel(), so it is OK to remove them
here.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt |  3 ++
 builtin/am.c               | 18 +++-----
 sequencer.c                | 85 +++++++++++++++++---------------------
 3 files changed, 47 insertions(+), 59 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 6a5ff036ff..156e4809f6 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -603,6 +603,9 @@ The hook always runs after the automatic note copying (see
 "notes.rewrite.<command>" in linkgit:git-config[1]) has happened, and
 thus has access to these notes.
 
+Hooks run during 'post-rewrite' will be run in parallel, unless hook.jobs is
+configured to 1.
+
 The following command-specific comments apply:
 
 rebase::
diff --git a/builtin/am.c b/builtin/am.c
index d84791859c..d2534f9a1f 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -469,23 +469,17 @@ static int run_applypatch_msg_hook(struct am_state *state)
  */
 static int run_post_rewrite_hook(const struct am_state *state)
 {
-	struct child_process cp = CHILD_PROCESS_INIT;
-	const char *hook = find_hook("post-rewrite");
+	struct run_hooks_opt opt;
 	int ret;
 
-	if (!hook)
-		return 0;
+	run_hooks_opt_init_async(&opt);
 
-	strvec_push(&cp.args, hook);
-	strvec_push(&cp.args, "rebase");
+	strvec_push(&opt.args, "rebase");
+	opt.path_to_stdin = am_path(state, "rewritten");
 
-	cp.in = xopen(am_path(state, "rewritten"), O_RDONLY);
-	cp.stdout_to_stderr = 1;
-	cp.trace2_hook_name = "post-rewrite";
+	ret = run_hooks("post-rewrite", &opt);
 
-	ret = run_command(&cp);
-
-	close(cp.in);
+	run_hooks_opt_clear(&opt);
 	return ret;
 }
 
diff --git a/sequencer.c b/sequencer.c
index 72234af8ed..3fa7668763 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -35,6 +35,7 @@
 #include "rebase-interactive.h"
 #include "reset.h"
 #include "hook.h"
+#include "string-list.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -1147,33 +1148,30 @@ int update_head_with_reflog(const struct commit *old_head,
 static int run_rewrite_hook(const struct object_id *oldoid,
 			    const struct object_id *newoid)
 {
-	struct child_process proc = CHILD_PROCESS_INIT;
-	const char *argv[3];
+	struct run_hooks_opt opt;
+	struct strbuf tmp = STRBUF_INIT;
+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int code;
-	struct strbuf sb = STRBUF_INIT;
 
-	argv[0] = find_hook("post-rewrite");
-	if (!argv[0])
-		return 0;
+	run_hooks_opt_init_async(&opt);
 
-	argv[1] = "amend";
-	argv[2] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = "post-rewrite";
-
-	code = start_command(&proc);
-	if (code)
-		return code;
-	strbuf_addf(&sb, "%s %s\n", oid_to_hex(oldoid), oid_to_hex(newoid));
-	sigchain_push(SIGPIPE, SIG_IGN);
-	write_in_full(proc.in, sb.buf, sb.len);
-	close(proc.in);
-	strbuf_release(&sb);
-	sigchain_pop(SIGPIPE);
-	return finish_command(&proc);
+	strvec_push(&opt.args, "amend");
+
+	strbuf_addf(&tmp,
+		    "%s %s",
+		    oid_to_hex(oldoid),
+		    oid_to_hex(newoid));
+	string_list_append(&to_stdin, tmp.buf);
+
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
+
+	code = run_hooks("post-rewrite", &opt);
+
+	run_hooks_opt_clear(&opt);
+	strbuf_release(&tmp);
+	string_list_clear(&to_stdin, 0);
+	return code;
 }
 
 void commit_post_rewrite(struct repository *r,
@@ -4527,30 +4525,23 @@ static int pick_commits(struct repository *r,
 		flush_rewritten_pending();
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
-			struct child_process child = CHILD_PROCESS_INIT;
-			const char *post_rewrite_hook =
-				find_hook("post-rewrite");
-
-			child.in = open(rebase_path_rewritten_list(), O_RDONLY);
-			child.git_cmd = 1;
-			strvec_push(&child.args, "notes");
-			strvec_push(&child.args, "copy");
-			strvec_push(&child.args, "--for-rewrite=rebase");
+			struct child_process notes_cp = CHILD_PROCESS_INIT;
+			struct run_hooks_opt hook_opt;
+
+			run_hooks_opt_init_async(&hook_opt);
+
+			notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY);
+			notes_cp.git_cmd = 1;
+			strvec_push(&notes_cp.args, "notes");
+			strvec_push(&notes_cp.args, "copy");
+			strvec_push(&notes_cp.args, "--for-rewrite=rebase");
 			/* we don't care if this copying failed */
-			run_command(&child);
-
-			if (post_rewrite_hook) {
-				struct child_process hook = CHILD_PROCESS_INIT;
-
-				hook.in = open(rebase_path_rewritten_list(),
-					O_RDONLY);
-				hook.stdout_to_stderr = 1;
-				hook.trace2_hook_name = "post-rewrite";
-				strvec_push(&hook.args, post_rewrite_hook);
-				strvec_push(&hook.args, "rebase");
-				/* we don't care if this hook failed */
-				run_command(&hook);
-			}
+			run_command(&notes_cp);
+
+			hook_opt.path_to_stdin = rebase_path_rewritten_list();
+			strvec_push(&hook_opt.args, "rebase");
+			run_hooks("post-rewrite", &hook_opt);
+			run_hooks_opt_clear(&hook_opt);
 		}
 		apply_autostash(rebase_path_autostash());
 
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 27/37] transport: convert pre-push hook to use config
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (25 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 26/37] hook: convert 'post-rewrite' hook to config Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 28/37] reference-transaction: look for hooks in config Emily Shaffer
                     ` (11 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using the hook.h:run_hooks API, pre-push hooks can be specified in
the config as well as in the hookdir.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt |  3 ++
 transport.c                | 59 +++++++++++---------------------------
 2 files changed, 20 insertions(+), 42 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 156e4809f6..543244ec0a 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -279,6 +279,9 @@ If this hook exits with a non-zero status, `git push` will abort without
 pushing anything.  Information about why the push is rejected may be sent
 to the user by writing to standard error.
 
+Hooks executed during 'pre-push' will run in parallel, unless hook.jobs is
+configured to 1.
+
 [[pre-receive]]
 pre-receive
 ~~~~~~~~~~~
diff --git a/transport.c b/transport.c
index 6cf3da19eb..dbc7bb7820 100644
--- a/transport.c
+++ b/transport.c
@@ -22,6 +22,7 @@
 #include "protocol.h"
 #include "object-store.h"
 #include "color.h"
+#include "hook.h"
 
 static int transport_use_color = -1;
 static char transport_colors[][COLOR_MAXLEN] = {
@@ -1196,31 +1197,15 @@ static void die_with_unpushed_submodules(struct string_list *needs_pushing)
 static int run_pre_push_hook(struct transport *transport,
 			     struct ref *remote_refs)
 {
-	int ret = 0, x;
+	int ret = 0;
+	struct run_hooks_opt opt;
+	struct strbuf tmp = STRBUF_INIT;
 	struct ref *r;
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct strbuf buf;
-	const char *argv[4];
-
-	if (!(argv[0] = find_hook("pre-push")))
-		return 0;
-
-	argv[1] = transport->remote->name;
-	argv[2] = transport->url;
-	argv[3] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.trace2_hook_name = "pre-push";
-
-	if (start_command(&proc)) {
-		finish_command(&proc);
-		return -1;
-	}
-
-	sigchain_push(SIGPIPE, SIG_IGN);
+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
+	run_hooks_opt_init_async(&opt);
 
-	strbuf_init(&buf, 256);
+	strvec_push(&opt.args, transport->remote->name);
+	strvec_push(&opt.args, transport->url);
 
 	for (r = remote_refs; r; r = r->next) {
 		if (!r->peer_ref) continue;
@@ -1229,30 +1214,20 @@ static int run_pre_push_hook(struct transport *transport,
 		if (r->status == REF_STATUS_REJECT_REMOTE_UPDATED) continue;
 		if (r->status == REF_STATUS_UPTODATE) continue;
 
-		strbuf_reset(&buf);
-		strbuf_addf( &buf, "%s %s %s %s\n",
+		strbuf_reset(&tmp);
+		strbuf_addf(&tmp, "%s %s %s %s",
 			 r->peer_ref->name, oid_to_hex(&r->new_oid),
 			 r->name, oid_to_hex(&r->old_oid));
-
-		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
-			/* We do not mind if a hook does not read all refs. */
-			if (errno != EPIPE)
-				ret = -1;
-			break;
-		}
+		string_list_append(&to_stdin, tmp.buf);
 	}
 
-	strbuf_release(&buf);
-
-	x = close(proc.in);
-	if (!ret)
-		ret = x;
-
-	sigchain_pop(SIGPIPE);
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
 
-	x = finish_command(&proc);
-	if (!ret)
-		ret = x;
+	ret = run_hooks("pre-push", &opt);
+	run_hooks_opt_clear(&opt);
+	strbuf_release(&tmp);
+	string_list_clear(&to_stdin, 0);
 
 	return ret;
 }
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 28/37] reference-transaction: look for hooks in config
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (26 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 27/37] transport: convert pre-push hook to use config Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 29/37] receive-pack: convert 'update' hook to hook.h Emily Shaffer
                     ` (10 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using the hook.h library, reference-transaction hooks can be
specified in the config instead.

The expected output of the test is not fully updated to reflect the
absolute path of the hook called because the 'update' hook has not yet
been converted to use hook.h.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt       |  3 +++
 refs.c                           | 43 +++++++++++++-------------------
 t/t1416-ref-transaction-hooks.sh |  8 +++---
 transport.c                      |  1 +
 4 files changed, 25 insertions(+), 30 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 543244ec0a..edb840dcdd 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -539,6 +539,9 @@ The exit status of the hook is ignored for any state except for the
 cause the transaction to be aborted. The hook will not be called with
 "aborted" state in that case.
 
+Hooks run during 'reference-transaction' will be run in parallel, unless
+hook.jobs is configured to 1.
+
 push-to-checkout
 ~~~~~~~~~~~~~~~~
 
diff --git a/refs.c b/refs.c
index 8c9490235e..32e993aaff 100644
--- a/refs.c
+++ b/refs.c
@@ -18,6 +18,7 @@
 #include "strvec.h"
 #include "repository.h"
 #include "sigchain.h"
+#include "hook.h"
 
 /*
  * List of all available backends
@@ -2061,47 +2062,37 @@ int ref_update_reject_duplicates(struct string_list *refnames,
 static int run_transaction_hook(struct ref_transaction *transaction,
 				const char *state)
 {
-	struct child_process proc = CHILD_PROCESS_INIT;
 	struct strbuf buf = STRBUF_INIT;
-	const char *hook;
+	struct run_hooks_opt opt;
+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int ret = 0, i;
+	char o[GIT_MAX_HEXSZ + 1], n[GIT_MAX_HEXSZ + 1];
 
-	hook = find_hook("reference-transaction");
-	if (!hook)
-		return ret;
-
-	strvec_pushl(&proc.args, hook, state, NULL);
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = "reference-transaction";
+	run_hooks_opt_init_async(&opt);
 
-	ret = start_command(&proc);
-	if (ret)
+	if (!hook_exists("reference-transaction", HOOKDIR_USE_CONFIG))
 		return ret;
 
-	sigchain_push(SIGPIPE, SIG_IGN);
+	strvec_push(&opt.args, state);
 
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		oid_to_hex_r(o, &update->old_oid);
+		oid_to_hex_r(n, &update->new_oid);
 
 		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
-			    oid_to_hex(&update->old_oid),
-			    oid_to_hex(&update->new_oid),
-			    update->refname);
-
-		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
-			if (errno != EPIPE)
-				ret = -1;
-			break;
-		}
+		strbuf_addf(&buf, "%s %s %s", o, n, update->refname);
+		string_list_append(&to_stdin, buf.buf);
 	}
 
-	close(proc.in);
-	sigchain_pop(SIGPIPE);
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
+
+	ret = run_hooks("reference-transaction", &opt);
+	run_hooks_opt_clear(&opt);
 	strbuf_release(&buf);
+	string_list_clear(&to_stdin, 0);
 
-	ret |= finish_command(&proc);
 	return ret;
 }
 
diff --git a/t/t1416-ref-transaction-hooks.sh b/t/t1416-ref-transaction-hooks.sh
index 6c941027a8..3a90a59143 100755
--- a/t/t1416-ref-transaction-hooks.sh
+++ b/t/t1416-ref-transaction-hooks.sh
@@ -125,11 +125,11 @@ test_expect_success 'interleaving hook calls succeed' '
 
 	cat >expect <<-EOF &&
 		hooks/update refs/tags/PRE $ZERO_OID $PRE_OID
-		hooks/reference-transaction prepared
-		hooks/reference-transaction committed
+		$(pwd)/target-repo.git/hooks/reference-transaction prepared
+		$(pwd)/target-repo.git/hooks/reference-transaction committed
 		hooks/update refs/tags/POST $ZERO_OID $POST_OID
-		hooks/reference-transaction prepared
-		hooks/reference-transaction committed
+		$(pwd)/target-repo.git/hooks/reference-transaction prepared
+		$(pwd)/target-repo.git/hooks/reference-transaction committed
 	EOF
 
 	git push ./target-repo.git PRE POST &&
diff --git a/transport.c b/transport.c
index dbc7bb7820..9191107626 100644
--- a/transport.c
+++ b/transport.c
@@ -1202,6 +1202,7 @@ static int run_pre_push_hook(struct transport *transport,
 	struct strbuf tmp = STRBUF_INIT;
 	struct ref *r;
 	struct string_list to_stdin = STRING_LIST_INIT_DUP;
+
 	run_hooks_opt_init_async(&opt);
 
 	strvec_push(&opt.args, transport->remote->name);
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 29/37] receive-pack: convert 'update' hook to hook.h
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (27 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 28/37] reference-transaction: look for hooks in config Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 30/37] proc-receive: acquire hook list from hook.h Emily Shaffer
                     ` (9 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using hook.h to invoke the 'update' hook, now hooks can be specified
in the config in addition to the hookdir.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt       |  3 ++
 builtin/receive-pack.c           | 65 ++++++++++++++++++++++----------
 t/t1416-ref-transaction-hooks.sh |  4 +-
 3 files changed, 50 insertions(+), 22 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index edb840dcdd..1d731474ac 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -368,6 +368,9 @@ The default 'update' hook, when enabled--and with
 `hooks.allowunannotated` config option unset or set to false--prevents
 unannotated tags to be pushed.
 
+Hooks executed during 'update' are run in parallel, unless hook.jobs is
+configured to 1.
+
 [[proc-receive]]
 proc-receive
 ~~~~~~~~~~~~
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 09e7cdee45..0f6bc8653f 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -938,33 +938,58 @@ static int run_receive_hook(struct command *commands,
 	return status;
 }
 
+static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
+{
+	int keepalive_active = 0;
+
+	if (keepalive_in_sec <= 0)
+		use_keepalive = KEEPALIVE_NEVER;
+	if (use_keepalive == KEEPALIVE_ALWAYS)
+		keepalive_active = 1;
+
+	/* send a keepalive if there is no data to write */
+	if (keepalive_active && !output->len) {
+		static const char buf[] = "0005\1";
+		write_or_die(1, buf, sizeof(buf) - 1);
+		return;
+	}
+
+	if (use_keepalive == KEEPALIVE_AFTER_NUL && !keepalive_active) {
+		const char *first_null = memchr(output->buf, '\0', output->len);
+		if (first_null) {
+			/* The null bit is excluded. */
+			size_t before_null = first_null - output->buf;
+			size_t after_null = output->len - (before_null + 1);
+			keepalive_active = 1;
+			send_sideband(1, 2, output->buf, before_null, use_sideband);
+			send_sideband(1, 2, first_null + 1, after_null, use_sideband);
+
+			return;
+		}
+	}
+
+	send_sideband(1, 2, output->buf, output->len, use_sideband);
+}
+
 static int run_update_hook(struct command *cmd)
 {
-	const char *argv[5];
-	struct child_process proc = CHILD_PROCESS_INIT;
+	struct run_hooks_opt opt;
 	int code;
 
-	argv[0] = find_hook("update");
-	if (!argv[0])
-		return 0;
+	run_hooks_opt_init_async(&opt);
 
-	argv[1] = cmd->ref_name;
-	argv[2] = oid_to_hex(&cmd->old_oid);
-	argv[3] = oid_to_hex(&cmd->new_oid);
-	argv[4] = NULL;
+	strvec_pushl(&opt.args,
+		     cmd->ref_name,
+		     oid_to_hex(&cmd->old_oid),
+		     oid_to_hex(&cmd->new_oid),
+		     NULL);
 
-	proc.no_stdin = 1;
-	proc.stdout_to_stderr = 1;
-	proc.err = use_sideband ? -1 : 0;
-	proc.argv = argv;
-	proc.trace2_hook_name = "update";
-
-	code = start_command(&proc);
-	if (code)
-		return code;
 	if (use_sideband)
-		copy_to_sideband(proc.err, -1, NULL);
-	return finish_command(&proc);
+		opt.consume_sideband = hook_output_to_sideband;
+
+	code = run_hooks("update", &opt);
+	run_hooks_opt_clear(&opt);
+	return code;
 }
 
 static struct command *find_command_by_refname(struct command *list,
diff --git a/t/t1416-ref-transaction-hooks.sh b/t/t1416-ref-transaction-hooks.sh
index 3a90a59143..0a3c3e4a86 100755
--- a/t/t1416-ref-transaction-hooks.sh
+++ b/t/t1416-ref-transaction-hooks.sh
@@ -124,10 +124,10 @@ test_expect_success 'interleaving hook calls succeed' '
 	EOF
 
 	cat >expect <<-EOF &&
-		hooks/update refs/tags/PRE $ZERO_OID $PRE_OID
+		$(pwd)/target-repo.git/hooks/update refs/tags/PRE $ZERO_OID $PRE_OID
 		$(pwd)/target-repo.git/hooks/reference-transaction prepared
 		$(pwd)/target-repo.git/hooks/reference-transaction committed
-		hooks/update refs/tags/POST $ZERO_OID $POST_OID
+		$(pwd)/target-repo.git/hooks/update refs/tags/POST $ZERO_OID $POST_OID
 		$(pwd)/target-repo.git/hooks/reference-transaction prepared
 		$(pwd)/target-repo.git/hooks/reference-transaction committed
 	EOF
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 30/37] proc-receive: acquire hook list from hook.h
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (28 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 29/37] receive-pack: convert 'update' hook to hook.h Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 31/37] post-update: use hook.h library Emily Shaffer
                     ` (8 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

The proc-receive hook differs from most other hooks Git invokes because
the hook and the parent Git process engage in bidirectional
communication via stdin/stdout. This bidirectional communication is
unsuitable for multiple hooks, whether they are in series or in
parallel, and is incompatible with run-command.h:run_processes_parallel:

- The proc-receive hook is intended to modify the state of the Git repo.
  From 'git help githooks':
    This [proc-receive] hook is responsible for updating the relevant
    references and reporting the results back to 'receive-pack'.
  This prevents parallelization and implies, at least, specific ordering
  of hook execution.
- The proc-receive hook can reject a push by aborting early with an
  error code. If a former hook ran through the entire push contents
  successfully but a later hook rejects some of the push, the repo may
  be left in a partially-updated (and corrupt) state.
- The callback model of the run_processes_parallel() API is unsuited to
  the current implementation of proc-receive, which loops through
  "send-receive-consider" with the child process. proc-receive today
  relies on stateful communication with the child process, which would be
  unwieldy to implement with callbacks and saved state.
- Additionally, run_processes_parallel() is designed to collate the
  output of many child processes into a single output (stderr or callback),
  and would require significant work to tell the caller which process sent
  the output, and indeed to collect any output before the child process
  has exited.

So, rather than using hook.h:run_hooks() to invoke the proc-receive
hook, receive-pack.c can learn to ask hook.h:hook_list() for the
location of a hook to run. This allows users to configure their
proc-receive in a global config for all repos if they want, or a local
config if they just don't want to use the hookdir. Because running more
than one proc-receive hook doesn't make sense from a repo state
perspective, we can explicitly ban configuring more than one
proc-receive hook at a time.

If a user wants to globally configure one proc-receive hook for most of
their repos, but override that hook in a single repo, they should use
'skip' to manually remove the global hook in their special repo:

~/.gitconfig:
[hook.proc-receive]
  command = /usr/bin/usual-proc-receive

~/special-repo/.git/config:
[hookcmd./usr/bin/usual-proc-receive]
  skip = true
[hook.proc-receive]
  command = /usr/bin/special-proc-receive

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt                |  4 ++
 builtin/receive-pack.c                    | 31 ++++++++++++++-
 t/t5411/test-0015-too-many-hooks-error.sh | 47 +++++++++++++++++++++++
 3 files changed, 80 insertions(+), 2 deletions(-)
 create mode 100644 t/t5411/test-0015-too-many-hooks-error.sh

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 1d731474ac..b6b13afb36 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -433,6 +433,10 @@ the input.  The exit status of the 'proc-receive' hook only determines
 the success or failure of the group of commands sent to it, unless
 atomic push is in use.
 
+It is forbidden to specify more than one hook for 'proc-receive'. If a
+globally-configured 'proc-receive' must be overridden, use
+'hookcmd.<global-hook>.skip = true' to ignore it.
+
 [[post-receive]]
 post-receive
 ~~~~~~~~~~~~
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 0f6bc8653f..5ecc97d8ab 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1147,11 +1147,38 @@ static int run_proc_receive_hook(struct command *commands,
 	int version = 0;
 	int code;
 
-	argv[0] = find_hook("proc-receive");
-	if (!argv[0]) {
+	struct hook *proc_receive = NULL;
+	struct list_head *pos, *hooks;
+
+	hooks = hook_list("proc-receive");
+
+	list_for_each(pos, hooks) {
+		if (proc_receive) {
+			rp_error("only one 'proc-receive' hook can be specified");
+			return -1;
+		}
+		proc_receive = list_entry(pos, struct hook, list);
+		/* check if the hookdir hook should be ignored */
+		if (proc_receive->from_hookdir) {
+			switch (configured_hookdir_opt()) {
+			case HOOKDIR_INTERACTIVE:
+			case HOOKDIR_NO:
+				proc_receive = NULL;
+				break;
+			default:
+				break;
+			}
+		}
+
+	}
+
+	if (!proc_receive) {
 		rp_error("cannot find hook 'proc-receive'");
 		return -1;
 	}
+
+
+	argv[0] = proc_receive->command.buf;
 	argv[1] = NULL;
 
 	proc.argv = argv;
diff --git a/t/t5411/test-0015-too-many-hooks-error.sh b/t/t5411/test-0015-too-many-hooks-error.sh
new file mode 100644
index 0000000000..2d64534510
--- /dev/null
+++ b/t/t5411/test-0015-too-many-hooks-error.sh
@@ -0,0 +1,47 @@
+test_expect_success "setup too  many proc-receive hooks (ok, $PROTOCOL)" '
+	write_script "proc-receive" <<-EOF &&
+	printf >&2 "# proc-receive hook\n"
+	test-tool proc-receive -v \
+		-r "ok refs/for/main/topic"
+	EOF
+
+	git -C "$upstream" config --add "hook.proc-receive.command" proc-receive &&
+	cp proc-receive "$upstream/hooks/proc-receive"
+'
+
+# Refs of upstream : main(A)
+# Refs of workbench: main(A)  tags/v123
+# git push         :                       next(A)  refs/for/main/topic(A)
+test_expect_success "proc-receive: reject more than one configured hook" '
+	test_must_fail git -C workbench push origin \
+		HEAD:next \
+		HEAD:refs/for/main/topic \
+		>out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	cat >expect <<-EOF &&
+	remote: # pre-receive hook
+	remote: pre-receive< <ZERO-OID> <COMMIT-A> refs/heads/next
+	remote: pre-receive< <ZERO-OID> <COMMIT-A> refs/for/main/topic
+	remote: error: only one "proc-receive" hook can be specified
+	remote: # post-receive hook
+	remote: post-receive< <ZERO-OID> <COMMIT-A> refs/heads/next
+	To <URL/of/upstream.git>
+	 * [new branch] HEAD -> next
+	 ! [remote rejected] HEAD -> refs/for/main/topic (fail to run proc-receive hook)
+	EOF
+	test_cmp expect actual &&
+	git -C "$upstream" show-ref >out &&
+	make_user_friendly_and_stable_output <out >actual &&
+	cat >expect <<-EOF &&
+	<COMMIT-A> refs/heads/main
+	<COMMIT-A> refs/heads/next
+	EOF
+	test_cmp expect actual
+'
+
+# Refs of upstream : main(A)             next(A)
+# Refs of workbench: main(A)  tags/v123
+test_expect_success "cleanup ($PROTOCOL)" '
+	git -C "$upstream" config --unset "hook.proc-receive.command" "proc-receive" &&
+	git -C "$upstream" update-ref -d refs/heads/next
+'
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 31/37] post-update: use hook.h library
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (29 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 30/37] proc-receive: acquire hook list from hook.h Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-06-14  9:09     ` Ævar Arnfjörð Bjarmason
  2021-05-27  0:08   ` [PATCH v9 32/37] receive-pack: convert receive hooks to hook.h Emily Shaffer
                     ` (7 subsequent siblings)
  38 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using run_hooks() instead of run_hook_le(), 'post-update' hooks can
be specified in the config as well as the hookdir.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt |  3 +++
 builtin/receive-pack.c     | 26 ++++++++------------------
 2 files changed, 11 insertions(+), 18 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index b6b13afb36..281970f910 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -508,6 +508,9 @@ Both standard output and standard error output are forwarded to
 `git send-pack` on the other end, so you can simply `echo` messages
 for the user.
 
+Hooks run during 'post-update' will be run in parallel, unless hook.jobs is
+configured to 1.
+
 reference-transaction
 ~~~~~~~~~~~~~~~~~~~~~
 
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 5ecc97d8ab..4236c0b268 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1688,33 +1688,23 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 static void run_update_post_hook(struct command *commands)
 {
 	struct command *cmd;
-	struct child_process proc = CHILD_PROCESS_INIT;
-	const char *hook;
+	struct run_hooks_opt opt;
 
-	hook = find_hook("post-update");
-	if (!hook)
-		return;
+	run_hooks_opt_init_async(&opt);
 
 	for (cmd = commands; cmd; cmd = cmd->next) {
 		if (cmd->error_string || cmd->did_not_exist)
 			continue;
-		if (!proc.args.nr)
-			strvec_push(&proc.args, hook);
-		strvec_push(&proc.args, cmd->ref_name);
+		strvec_push(&opt.args, cmd->ref_name);
 	}
-	if (!proc.args.nr)
+	if (!opt.args.nr)
 		return;
 
-	proc.no_stdin = 1;
-	proc.stdout_to_stderr = 1;
-	proc.err = use_sideband ? -1 : 0;
-	proc.trace2_hook_name = "post-update";
+	if (use_sideband)
+		opt.consume_sideband = hook_output_to_sideband;
 
-	if (!start_command(&proc)) {
-		if (use_sideband)
-			copy_to_sideband(proc.err, -1, NULL);
-		finish_command(&proc);
-	}
+	run_hooks("post-update", &opt);
+	run_hooks_opt_clear(&opt);
 }
 
 static void check_aliased_update_internal(struct command *cmd,
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 32/37] receive-pack: convert receive hooks to hook.h
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (30 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 31/37] post-update: use hook.h library Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 33/37] bugreport: use hook_exists instead of find_hook Emily Shaffer
                     ` (6 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using the hook.h library to run receive hooks, they can be specified
in the config as well as in the hookdir.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt |   5 +
 builtin/receive-pack.c     | 199 +++++++++++++++++--------------------
 2 files changed, 97 insertions(+), 107 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 281970f910..78cc4e0872 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -323,6 +323,8 @@ will be set to zero, `GIT_PUSH_OPTION_COUNT=0`.
 See the section on "Quarantine Environment" in
 linkgit:git-receive-pack[1] for some caveats.
 
+Hooks executed during 'pre-receive' will not be parallelized.
+
 [[update]]
 update
 ~~~~~~
@@ -476,6 +478,9 @@ environment variables will not be set. If the client selects
 to use push options, but doesn't transmit any, the count variable
 will be set to zero, `GIT_PUSH_OPTION_COUNT=0`.
 
+Hooks executed during 'post-receive' are run in parallel, unless hook.jobs is
+configured to 1.
+
 [[post-update]]
 post-update
 ~~~~~~~~~~~
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 4236c0b268..f44b58e456 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -748,7 +748,7 @@ static int check_cert_push_options(const struct string_list *push_options)
 	return retval;
 }
 
-static void prepare_push_cert_sha1(struct child_process *proc)
+static void prepare_push_cert_sha1(struct run_hooks_opt *opt)
 {
 	static int already_done;
 
@@ -772,110 +772,42 @@ static void prepare_push_cert_sha1(struct child_process *proc)
 		nonce_status = check_nonce(push_cert.buf, bogs);
 	}
 	if (!is_null_oid(&push_cert_oid)) {
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT=%s",
 			     oid_to_hex(&push_cert_oid));
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_SIGNER=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_SIGNER=%s",
 			     sigcheck.signer ? sigcheck.signer : "");
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_KEY=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_KEY=%s",
 			     sigcheck.key ? sigcheck.key : "");
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_STATUS=%c",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_STATUS=%c",
 			     sigcheck.result);
 		if (push_cert_nonce) {
-			strvec_pushf(&proc->env_array,
+			strvec_pushf(&opt->env,
 				     "GIT_PUSH_CERT_NONCE=%s",
 				     push_cert_nonce);
-			strvec_pushf(&proc->env_array,
+			strvec_pushf(&opt->env,
 				     "GIT_PUSH_CERT_NONCE_STATUS=%s",
 				     nonce_status);
 			if (nonce_status == NONCE_SLOP)
-				strvec_pushf(&proc->env_array,
+				strvec_pushf(&opt->env,
 					     "GIT_PUSH_CERT_NONCE_SLOP=%ld",
 					     nonce_stamp_slop);
 		}
 	}
 }
 
+struct receive_hook_feed_context {
+	struct command *cmd;
+	int skip_broken;
+};
+
 struct receive_hook_feed_state {
 	struct command *cmd;
 	struct ref_push_report *report;
 	int skip_broken;
 	struct strbuf buf;
-	const struct string_list *push_options;
 };
 
-typedef int (*feed_fn)(void *, const char **, size_t *);
-static int run_and_feed_hook(const char *hook_name, feed_fn feed,
-			     struct receive_hook_feed_state *feed_state)
-{
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct async muxer;
-	const char *argv[2];
-	int code;
-
-	argv[0] = find_hook(hook_name);
-	if (!argv[0])
-		return 0;
-
-	argv[1] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = hook_name;
-
-	if (feed_state->push_options) {
-		int i;
-		for (i = 0; i < feed_state->push_options->nr; i++)
-			strvec_pushf(&proc.env_array,
-				     "GIT_PUSH_OPTION_%d=%s", i,
-				     feed_state->push_options->items[i].string);
-		strvec_pushf(&proc.env_array, "GIT_PUSH_OPTION_COUNT=%d",
-			     feed_state->push_options->nr);
-	} else
-		strvec_pushf(&proc.env_array, "GIT_PUSH_OPTION_COUNT");
-
-	if (tmp_objdir)
-		strvec_pushv(&proc.env_array, tmp_objdir_env(tmp_objdir));
-
-	if (use_sideband) {
-		memset(&muxer, 0, sizeof(muxer));
-		muxer.proc = copy_to_sideband;
-		muxer.in = -1;
-		code = start_async(&muxer);
-		if (code)
-			return code;
-		proc.err = muxer.in;
-	}
-
-	prepare_push_cert_sha1(&proc);
-
-	code = start_command(&proc);
-	if (code) {
-		if (use_sideband)
-			finish_async(&muxer);
-		return code;
-	}
-
-	sigchain_push(SIGPIPE, SIG_IGN);
-
-	while (1) {
-		const char *buf;
-		size_t n;
-		if (feed(feed_state, &buf, &n))
-			break;
-		if (write_in_full(proc.in, buf, n) < 0)
-			break;
-	}
-	close(proc.in);
-	if (use_sideband)
-		finish_async(&muxer);
-
-	sigchain_pop(SIGPIPE);
-
-	return finish_command(&proc);
-}
-
-static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
+static int feed_receive_hook(void *state_)
 {
 	struct receive_hook_feed_state *state = state_;
 	struct command *cmd = state->cmd;
@@ -884,9 +816,7 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
 	       state->skip_broken && (cmd->error_string || cmd->did_not_exist))
 		cmd = cmd->next;
 	if (!cmd)
-		return -1; /* EOF */
-	if (!bufp)
-		return 0; /* OK, can feed something. */
+		return 1; /* EOF - close the pipe*/
 	strbuf_reset(&state->buf);
 	if (!state->report)
 		state->report = cmd->report;
@@ -910,32 +840,36 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
 			    cmd->ref_name);
 		state->cmd = cmd->next;
 	}
-	if (bufp) {
-		*bufp = state->buf.buf;
-		*sizep = state->buf.len;
-	}
 	return 0;
 }
 
-static int run_receive_hook(struct command *commands,
-			    const char *hook_name,
-			    int skip_broken,
-			    const struct string_list *push_options)
+static int feed_receive_hook_cb(struct strbuf *pipe, void *pp_cb, void *pp_task_cb)
 {
-	struct receive_hook_feed_state state;
-	int status;
-
-	strbuf_init(&state.buf, 0);
-	state.cmd = commands;
-	state.skip_broken = skip_broken;
-	state.report = NULL;
-	if (feed_receive_hook(&state, NULL, NULL))
-		return 0;
-	state.cmd = commands;
-	state.push_options = push_options;
-	status = run_and_feed_hook(hook_name, feed_receive_hook, &state);
-	strbuf_release(&state.buf);
-	return status;
+	struct hook *hook = pp_task_cb;
+	struct receive_hook_feed_state *feed_state = hook->feed_pipe_cb_data;
+	int rc;
+
+	/* first-time setup */
+	if (!feed_state) {
+		struct hook_cb_data *hook_cb = pp_cb;
+		struct run_hooks_opt *opt = hook_cb->options;
+		struct receive_hook_feed_context *ctx = opt->feed_pipe_ctx;
+		if (!ctx)
+			BUG("run_hooks_opt.feed_pipe_ctx required for receive hook");
+
+		feed_state = xmalloc(sizeof(struct receive_hook_feed_state));
+		strbuf_init(&feed_state->buf, 0);
+		feed_state->cmd = ctx->cmd;
+		feed_state->skip_broken = ctx->skip_broken;
+		feed_state->report = NULL;
+
+		hook->feed_pipe_cb_data = feed_state;
+	}
+
+	rc = feed_receive_hook(feed_state);
+	if (!rc)
+		strbuf_addbuf(pipe, &feed_state->buf);
+	return rc;
 }
 
 static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
@@ -971,6 +905,57 @@ static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
 	send_sideband(1, 2, output->buf, output->len, use_sideband);
 }
 
+static int run_receive_hook(struct command *commands,
+			    const char *hook_name,
+			    int skip_broken,
+			    const struct string_list *push_options)
+{
+	struct run_hooks_opt opt;
+	struct receive_hook_feed_context ctx;
+	int rc;
+	struct command *iter = commands;
+
+	run_hooks_opt_init_async(&opt);
+
+	/* if there are no valid commands, don't invoke the hook at all. */
+	while (iter && skip_broken && (iter->error_string || iter->did_not_exist))
+		iter = iter->next;
+	if (!iter)
+		return 0;
+
+	/* pre-receive hooks should run in series as the hook updates refs */
+	if (!strcmp(hook_name, "pre-receive"))
+		opt.jobs = 1;
+
+	if (push_options) {
+		int i;
+		for (i = 0; i < push_options->nr; i++)
+			strvec_pushf(&opt.env, "GIT_PUSH_OPTION_%d=%s", i,
+				     push_options->items[i].string);
+		strvec_pushf(&opt.env, "GIT_PUSH_OPTION_COUNT=%d", push_options->nr);
+	} else
+		strvec_push(&opt.env, "GIT_PUSH_OPTION_COUNT");
+
+	if (tmp_objdir)
+		strvec_pushv(&opt.env, tmp_objdir_env(tmp_objdir));
+
+	prepare_push_cert_sha1(&opt);
+
+	/* set up sideband printer */
+	if (use_sideband)
+		opt.consume_sideband = hook_output_to_sideband;
+
+	/* set up stdin callback */
+	ctx.cmd = commands;
+	ctx.skip_broken = skip_broken;
+	opt.feed_pipe = feed_receive_hook_cb;
+	opt.feed_pipe_ctx = &ctx;
+
+	rc = run_hooks(hook_name, &opt);
+	run_hooks_opt_clear(&opt);
+	return rc;
+}
+
 static int run_update_hook(struct command *cmd)
 {
 	struct run_hooks_opt opt;
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 33/37] bugreport: use hook_exists instead of find_hook
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (31 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 32/37] receive-pack: convert receive hooks to hook.h Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 34/37] git-send-email: use 'git hook run' for 'sendemail-validate' Emily Shaffer
                     ` (5 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using the helper in hook.h instead of the one in run-command.h, we
can also check whether a hook exists in the config - not just whether it
exists in the hookdir.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 builtin/bugreport.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/builtin/bugreport.c b/builtin/bugreport.c
index 9915a5841d..190272ba70 100644
--- a/builtin/bugreport.c
+++ b/builtin/bugreport.c
@@ -3,7 +3,7 @@
 #include "strbuf.h"
 #include "help.h"
 #include "compat/compiler.h"
-#include "run-command.h"
+#include "hook.h"
 
 
 static void get_system_info(struct strbuf *sys_info)
@@ -82,7 +82,7 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 
 	for (i = 0; i < ARRAY_SIZE(hook); i++)
-		if (find_hook(hook[i]))
+		if (hook_exists(hook[i], HOOKDIR_USE_CONFIG))
 			strbuf_addf(hook_info, "%s\n", hook[i]);
 }
 
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 34/37] git-send-email: use 'git hook run' for 'sendemail-validate'
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (32 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 33/37] bugreport: use hook_exists instead of find_hook Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27 11:56     ` Ævar Arnfjörð Bjarmason
  2021-05-27  0:08   ` [PATCH v9 35/37] run-command: stop thinking about hooks Emily Shaffer
                     ` (4 subsequent siblings)
  38 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

By using the new 'git hook run' subcommand to run 'sendemail-validate',
we can reduce the boilerplate needed to run this hook in perl. Using
config-based hooks also allows us to run 'sendemail-validate' hooks that
were configured globally when running 'git send-email' from outside of a
Git directory, alongside other benefits like multihooks and
parallelization.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 git-send-email.perl   | 26 ++++++--------------------
 t/t9001-send-email.sh | 13 +------------
 2 files changed, 7 insertions(+), 32 deletions(-)

diff --git a/git-send-email.perl b/git-send-email.perl
index 170f42fe21..b55687453e 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -1958,26 +1958,12 @@ sub unique_email_list {
 sub validate_patch {
 	my ($fn, $xfer_encoding) = @_;
 
-	if ($repo) {
-		my $validate_hook = catfile($repo->hooks_path(),
-					    'sendemail-validate');
-		my $hook_error;
-		if (-x $validate_hook) {
-			my $target = abs_path($fn);
-			# The hook needs a correct cwd and GIT_DIR.
-			my $cwd_save = cwd();
-			chdir($repo->wc_path() or $repo->repo_path())
-				or die("chdir: $!");
-			local $ENV{"GIT_DIR"} = $repo->repo_path();
-			$hook_error = system_or_msg([$validate_hook, $target]);
-			chdir($cwd_save) or die("chdir: $!");
-		}
-		if ($hook_error) {
-			die sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
-				       "%s\n" .
-				       "warning: no patches were sent\n"), $fn, $hook_error);
-		}
-	}
+	my $target = abs_path($fn);
+
+	system_or_die(["git", "hook", "run", "sendemail-validate", "-j1", "-a", $target],
+		sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
+			   "warning: no patches were sent\n"),
+		        $fn));
 
 	# Any long lines will be automatically fixed if we use a suitable transfer
 	# encoding.
diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index aa603cf4d0..bdf6472871 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -539,7 +539,6 @@ test_expect_success $PREREQ "--validate respects relative core.hooksPath path" '
 	test_path_is_file my-hooks.ran &&
 	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
-	fatal: command '"'"'$PWD/my-hooks/sendemail-validate'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
@@ -557,7 +556,6 @@ test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" '
 	test_path_is_file my-hooks.ran &&
 	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
-	fatal: command '"'"'$PWD/my-hooks/sendemail-validate'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
@@ -2170,16 +2168,7 @@ test_expect_success $PREREQ 'invoke hook' '
 	mkdir -p .git/hooks &&
 
 	write_script .git/hooks/sendemail-validate <<-\EOF &&
-	# test that we have the correct environment variable, pwd, and
-	# argument
-	case "$GIT_DIR" in
-	*.git)
-		true
-		;;
-	*)
-		false
-		;;
-	esac &&
+	# test that we have the correct argument
 	test -f 0001-add-main.patch &&
 	grep "add main" "$1"
 	EOF
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 35/37] run-command: stop thinking about hooks
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (33 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 34/37] git-send-email: use 'git hook run' for 'sendemail-validate' Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 36/37] doc: clarify fsmonitor-watchman specification Emily Shaffer
                     ` (3 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

hook.h has replaced all run-command.h hook-related functionality.
run-command.h:run_hooks_le/ve and find_hook are no longer used anywhere
in the codebase. So, let's delete the dead code - or, in the one case
where it's still needed, move it to an internal function in hook.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 hook.c        | 39 ++++++++++++++++++++++++++++--
 run-command.c | 66 ---------------------------------------------------
 run-command.h | 24 -------------------
 3 files changed, 37 insertions(+), 92 deletions(-)

diff --git a/hook.c b/hook.c
index 27da1fdb32..ff80e52edd 100644
--- a/hook.c
+++ b/hook.c
@@ -218,6 +218,41 @@ static int should_include_hookdir(const char *path, enum hookdir_opt cfg)
 	}
 }
 
+static const char *find_legacy_hook(const char *name)
+{
+	static struct strbuf path = STRBUF_INIT;
+
+	strbuf_reset(&path);
+	strbuf_git_path(&path, "hooks/%s", name);
+	if (access(path.buf, X_OK) < 0) {
+		int err = errno;
+
+#ifdef STRIP_EXTENSION
+		strbuf_addstr(&path, STRIP_EXTENSION);
+		if (access(path.buf, X_OK) >= 0)
+			return path.buf;
+		if (errno == EACCES)
+			err = errno;
+#endif
+
+		if (err == EACCES && advice_ignored_hook) {
+			static struct string_list advise_given = STRING_LIST_INIT_DUP;
+
+			if (!string_list_lookup(&advise_given, name)) {
+				string_list_insert(&advise_given, name);
+				advise(_("The '%s' hook was ignored because "
+					 "it's not set as executable.\n"
+					 "You can disable this warning with "
+					 "`git config advice.ignoredHook false`."),
+				       path.buf);
+			}
+		}
+		return NULL;
+	}
+	return path.buf;
+}
+
+
 struct list_head* hook_list(const char* hookname)
 {
 	struct strbuf hook_key = STRBUF_INIT;
@@ -234,7 +269,7 @@ struct list_head* hook_list(const char* hookname)
 	git_config(hook_config_lookup, &cb_data);
 
 	if (have_git_dir()) {
-		const char *legacy_hook_path = find_hook(hookname);
+		const char *legacy_hook_path = find_legacy_hook(hookname);
 
 		/* Unconditionally add legacy hook, but annotate it. */
 		if (legacy_hook_path) {
@@ -283,7 +318,7 @@ int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir)
 	could_run_hookdir = (should_run_hookdir == HOOKDIR_INTERACTIVE ||
 				should_run_hookdir == HOOKDIR_WARN ||
 				should_run_hookdir == HOOKDIR_YES)
-				&& !!find_hook(hookname);
+				&& !!find_legacy_hook(hookname);
 
 	strbuf_addf(&hook_key, "hook.%s.command", hookname);
 
diff --git a/run-command.c b/run-command.c
index 27135defb8..2ff76f3c2f 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1320,72 +1320,6 @@ int async_with_fork(void)
 #endif
 }
 
-const char *find_hook(const char *name)
-{
-	static struct strbuf path = STRBUF_INIT;
-
-	strbuf_reset(&path);
-	strbuf_git_path(&path, "hooks/%s", name);
-	if (access(path.buf, X_OK) < 0) {
-		int err = errno;
-
-#ifdef STRIP_EXTENSION
-		strbuf_addstr(&path, STRIP_EXTENSION);
-		if (access(path.buf, X_OK) >= 0)
-			return path.buf;
-		if (errno == EACCES)
-			err = errno;
-#endif
-
-		if (err == EACCES && advice_ignored_hook) {
-			static struct string_list advise_given = STRING_LIST_INIT_DUP;
-
-			if (!string_list_lookup(&advise_given, name)) {
-				string_list_insert(&advise_given, name);
-				advise(_("The '%s' hook was ignored because "
-					 "it's not set as executable.\n"
-					 "You can disable this warning with "
-					 "`git config advice.ignoredHook false`."),
-				       path.buf);
-			}
-		}
-		return NULL;
-	}
-	return path.buf;
-}
-
-int run_hook_ve(const char *const *env, const char *name, va_list args)
-{
-	struct child_process hook = CHILD_PROCESS_INIT;
-	const char *p;
-
-	p = find_hook(name);
-	if (!p)
-		return 0;
-
-	strvec_push(&hook.args, p);
-	while ((p = va_arg(args, const char *)))
-		strvec_push(&hook.args, p);
-	hook.env = env;
-	hook.no_stdin = 1;
-	hook.stdout_to_stderr = 1;
-	hook.trace2_hook_name = name;
-
-	return run_command(&hook);
-}
-
-int run_hook_le(const char *const *env, const char *name, ...)
-{
-	va_list args;
-	int ret;
-
-	va_start(args, name);
-	ret = run_hook_ve(env, name, args);
-	va_end(args);
-
-	return ret;
-}
-
 struct io_pump {
 	/* initialized by caller */
 	int fd;
diff --git a/run-command.h b/run-command.h
index ebc4a95a94..7150da851a 100644
--- a/run-command.h
+++ b/run-command.h
@@ -201,30 +201,6 @@ int finish_command_in_signal(struct child_process *);
  */
 int run_command(struct child_process *);
 
-/*
- * Returns the path to the hook file, or NULL if the hook is missing
- * or disabled. Note that this points to static storage that will be
- * overwritten by further calls to find_hook and run_hook_*.
- */
-const char *find_hook(const char *name);
-
-/**
- * Run a hook.
- * The first argument is a pathname to an index file, or NULL
- * if the hook uses the default index file or no index is needed.
- * The second argument is the name of the hook.
- * The further arguments correspond to the hook arguments.
- * The last argument has to be NULL to terminate the arguments list.
- * If the hook does not exist or is not executable, the return
- * value will be zero.
- * If it is executable, the hook will be executed and the exit
- * status of the hook is returned.
- * On execution, .stdout_to_stderr and .no_stdin will be set.
- */
-LAST_ARG_MUST_BE_NULL
-int run_hook_le(const char *const *env, const char *name, ...);
-int run_hook_ve(const char *const *env, const char *name, va_list args);
-
 /*
  * Trigger an auto-gc
  */
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 36/37] doc: clarify fsmonitor-watchman specification
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (34 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 35/37] run-command: stop thinking about hooks Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-05-27  0:08   ` [PATCH v9 37/37] docs: link githooks and git-hook manpages Emily Shaffer
                     ` (2 subsequent siblings)
  38 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

fsmonitor-watchman is not specified in the same way that other hooks
are. In fsmonitor.c:query_fsmonitor(), the path stored in
'core_fsmonitor' is executed directly via
run-command.h:capture_command(). 'core_fsmonitor' is set during 'git
update-index' via config.c:git_config_get_fsmonitor(). Neither
builtin/update-index.c, nor config.c, nor fsmonitor.c check  that the
path given is in '.git/hooks'.

None of the existing hook execution code is used by fsmonitor.c to
invoke fsmonitor-watchman, because that executable isn't expected to
reside in '.git/hooks'.

Furthermore, it doesn't make sense to specify the fsmonitor-watchman
hook more than once, and that hook itself may soon be superseded by
native logic to talk to an fsmonitor daemon directly from the Git
executable. (See <e019cc71-ac39-44a4-0a23-b3b7decec754@jeffhostetler.com>
for more information.)

Therefore, let's correct the user-facing documentation around
fsmonitor-watchman and clarify that it won't be supported by
config-based hooks.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/githooks.txt | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 78cc4e0872..42e66d4e2d 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -649,9 +649,12 @@ e-mails.
 fsmonitor-watchman
 ~~~~~~~~~~~~~~~~~~
 
-This hook is invoked when the configuration option `core.fsmonitor` is
-set to `.git/hooks/fsmonitor-watchman` or `.git/hooks/fsmonitor-watchmanv2`
-depending on the version of the hook to use.
+This hook is invoked when the configuration option `core.fsmonitor` is set to a
+path containing an executable. It *cannot* be specified via the usual
+`hook.fsmonitor-watchman.command` configuration or by providing an executable
+in `.git/hooks/fsmonitor-watchman`. The arguments provided to the hook are
+determined by the value of the `core.fsmonitorHookVersion` configuration
+option.
 
 Version 1 takes two arguments, a version (1) and the time in elapsed
 nanoseconds since midnight, January 1, 1970.
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v9 37/37] docs: link githooks and git-hook manpages
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (35 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 36/37] doc: clarify fsmonitor-watchman specification Emily Shaffer
@ 2021-05-27  0:08   ` Emily Shaffer
  2021-06-03  9:18     ` Ævar Arnfjörð Bjarmason
  2021-05-27 11:49   ` [PATCH v9 00/37] propose config-based hooks Ævar Arnfjörð Bjarmason
  2021-05-27 13:36   ` Ævar Arnfjörð Bjarmason
  38 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-05-27  0:08 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Since users may have an easier time finding 'man githooks' or 'git help
githooks' through tab-completion or muscle memory, reference the 'git
hook' commands. And in the 'git hook' manual, point users back to 'man
githooks' for specifics about the hook events themselves.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-hook.txt | 12 ++++++++++++
 Documentation/githooks.txt |  7 ++++---
 2 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 81b8e94994..24e00a6f4a 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -17,6 +17,13 @@ DESCRIPTION
 You can list and run configured hooks with this command. Later, you will be able
 to add and modify hooks with this command.
 
+In general, when instructions suggest adding a script to
+`.git/hooks/<something>`, you can specify it in the config instead by running
+`git config --add hook.<something>.command <path-to-script>` - this way you can
+share the script between multiple repos. That is, `cp ~/my-script.sh
+~/project/.git/hooks/pre-commit` would become `git config --add
+hook.pre-commit.command ~/my-script.sh`.
+
 This command parses the default configuration files for sections `hook` and
 `hookcmd`. `hook` is used to describe the commands which will be run during a
 particular hook event; commands are run in the order Git encounters them during
@@ -141,6 +148,11 @@ number of CPUs on the current system. Some hooks may be ineligible for
 parallelization: for example, 'commit-msg' intends hooks modify the commit
 message body and cannot be parallelized.
 
+HOOKS
+-----
+For a list of hooks which can be configured and how they work, see
+linkgit:githooks[5].
+
 CONFIGURATION
 -------------
 include::config/hook.txt[]
diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 42e66d4e2d..d780cb3b18 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -7,15 +7,16 @@ githooks - Hooks used by Git
 
 SYNOPSIS
 --------
+'git hook'
 $GIT_DIR/hooks/* (or \`git config core.hooksPath`/*)
 
 
 DESCRIPTION
 -----------
 
-Hooks are programs you can place in a hooks directory to trigger
-actions at certain points in git's execution. Hooks that don't have
-the executable bit set are ignored.
+Hooks are programs you can specify in your config (see linkgit:git-hook[1]) or
+place in a hooks directory to trigger actions at certain points in git's
+execution. Hooks that don't have the executable bit set are ignored.
 
 By default the hooks directory is `$GIT_DIR/hooks`, but that can be
 changed via the `core.hooksPath` configuration variable (see
-- 
2.31.1.818.g46aad6cb9e-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* Re: [PATCH v9 02/37] hook: introduce git-hook subcommand
  2021-05-27  0:08   ` [PATCH v9 02/37] hook: introduce git-hook subcommand Emily Shaffer
@ 2021-05-27  2:18     ` Junio C Hamano
  0 siblings, 0 replies; 479+ messages in thread
From: Junio C Hamano @ 2021-05-27  2:18 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> diff --git a/hook.c b/hook.c
> new file mode 100644
> index 0000000000..d3e28aa73a
> --- /dev/null
> +++ b/hook.c
> @@ -0,0 +1,120 @@
> +#include "cache.h"
> +
> +#include "hook.h"
> +#include "config.h"
> +
> +void free_hook(struct hook *ptr)

Should this be "static" (and be removed from hook.h)?  Even after
the whole series, nobody outside seems to use this function.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v9 00/37] propose config-based hooks
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (36 preceding siblings ...)
  2021-05-27  0:08   ` [PATCH v9 37/37] docs: link githooks and git-hook manpages Emily Shaffer
@ 2021-05-27 11:49   ` Ævar Arnfjörð Bjarmason
  2021-05-27 13:36   ` Ævar Arnfjörð Bjarmason
  38 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-27 11:49 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: git, Jeff King, Junio C Hamano, James Ramsay, Jonathan Nieder,
	brian m. carlson, Phillip Wood, Josh Steadmon,
	Johannes Schindelin, Jonathan Tan


On Wed, May 26 2021, Emily Shaffer wrote:

> After much delay and $DAYJOB, here is v9.

Thanks. Haven't done any deep review of this yet. Just skimming things
from v8 & commenting as I go along...

> - Addressed nits in reviews on v8
> [...
>   Ævar's updated system_or_die() function
> - changed strbuf to char* in hooks_list
>   - Attempted to do so in run_command's stdout callback, but this made
>     length protection difficult, so stuck with strbuf there.

I see there's still quite a bit of that strbuf churn still in this
series, e.g. unfixed issues noted in
https://lore.kernel.org/git/87pn04g0r1.fsf@evledraar.gmail.com/ e.g. in
07/37 you're still doing this:
	
	+	struct strbuf hookname = STRBUF_INIT;
	+ [...]
	+	strbuf_addstr(&hookname, argv[0]);
	+	opt.run_hookdir = should_run_hookdir;
	+
	+	rc = run_hooks(hookname.buf, &opt);
	+
	+	strbuf_release(&hookname);

So fair enough n the run_command's stdout callback, but it seems there's
still quite a bit of strbuf encapsulating for no apparent benefit.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v9 34/37] git-send-email: use 'git hook run' for 'sendemail-validate'
  2021-05-27  0:08   ` [PATCH v9 34/37] git-send-email: use 'git hook run' for 'sendemail-validate' Emily Shaffer
@ 2021-05-27 11:56     ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-27 11:56 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Wed, May 26 2021, Emily Shaffer wrote:

> By using the new 'git hook run' subcommand to run 'sendemail-validate',
> we can reduce the boilerplate needed to run this hook in perl. Using
> config-based hooks also allows us to run 'sendemail-validate' hooks that
> were configured globally when running 'git send-email' from outside of a
> Git directory, alongside other benefits like multihooks and
> parallelization.
>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  git-send-email.perl   | 26 ++++++--------------------
>  t/t9001-send-email.sh | 13 +------------
>  2 files changed, 7 insertions(+), 32 deletions(-)
>
> diff --git a/git-send-email.perl b/git-send-email.perl
> index 170f42fe21..b55687453e 100755
> --- a/git-send-email.perl
> +++ b/git-send-email.perl
> @@ -1958,26 +1958,12 @@ sub unique_email_list {
>  sub validate_patch {
>  	my ($fn, $xfer_encoding) = @_;
>  
> -	if ($repo) {
> -		my $validate_hook = catfile($repo->hooks_path(),
> -					    'sendemail-validate');
> -		my $hook_error;
> -		if (-x $validate_hook) {
> -			my $target = abs_path($fn);
> -			# The hook needs a correct cwd and GIT_DIR.
> -			my $cwd_save = cwd();
> -			chdir($repo->wc_path() or $repo->repo_path())
> -				or die("chdir: $!");
> -			local $ENV{"GIT_DIR"} = $repo->repo_path();
> -			$hook_error = system_or_msg([$validate_hook, $target]);
> -			chdir($cwd_save) or die("chdir: $!");
> -		}
> -		if ($hook_error) {
> -			die sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
> -				       "%s\n" .
> -				       "warning: no patches were sent\n"), $fn, $hook_error);
> -		}
> -	}
> +	my $target = abs_path($fn);
> +
> +	system_or_die(["git", "hook", "run", "sendemail-validate", "-j1", "-a", $target],

How do we get the "benefit[ ... of ...] parallelization" here if -j1 is
hardcoded as parameter. Shouldn't that be removed so it'll use the
hook.jobs config like everything else?

> +		sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
> +			   "warning: no patches were sent\n"),
> +		        $fn));
>  
>  	# Any long lines will be automatically fixed if we use a suitable transfer
>  	# encoding.
> diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
> index aa603cf4d0..bdf6472871 100755
> --- a/t/t9001-send-email.sh
> +++ b/t/t9001-send-email.sh
> @@ -539,7 +539,6 @@ test_expect_success $PREREQ "--validate respects relative core.hooksPath path" '
>  	test_path_is_file my-hooks.ran &&
>  	cat >expect <<-EOF &&
>  	fatal: longline.patch: rejected by sendemail-validate hook
> -	fatal: command '"'"'$PWD/my-hooks/sendemail-validate'"'"' died with exit code 1
>  	warning: no patches were sent
>  	EOF
>  	test_cmp expect actual
> @@ -557,7 +556,6 @@ test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" '
>  	test_path_is_file my-hooks.ran &&
>  	cat >expect <<-EOF &&
>  	fatal: longline.patch: rejected by sendemail-validate hook
> -	fatal: command '"'"'$PWD/my-hooks/sendemail-validate'"'"' died with exit code 1
>  	warning: no patches were sent
>  	EOF
>  	test_cmp expect actual

Wouldn't it be better to keep the "died with exit code N" here. I.e. something like:

	fatal: longline.patch: rejected by sendemail-validate hook, 'git hook run' died with exit code %d

I.e. isn't that exit code carried forward by "git hook run"?

> @@ -2170,16 +2168,7 @@ test_expect_success $PREREQ 'invoke hook' '
>  	mkdir -p .git/hooks &&
>  
>  	write_script .git/hooks/sendemail-validate <<-\EOF &&
> -	# test that we have the correct environment variable, pwd, and
> -	# argument
> -	case "$GIT_DIR" in
> -	*.git)
> -		true
> -		;;
> -	*)
> -		false
> -		;;
> -	esac &&
> +	# test that we have the correct argument
>  	test -f 0001-add-main.patch &&
>  	grep "add main" "$1"
>  	EOF

We don't want to or need to check $GIT_DIR at all? Maybe not, but the
commit message doesn't say anything about it...

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v9 00/37] propose config-based hooks
  2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
                     ` (37 preceding siblings ...)
  2021-05-27 11:49   ` [PATCH v9 00/37] propose config-based hooks Ævar Arnfjörð Bjarmason
@ 2021-05-27 13:36   ` Ævar Arnfjörð Bjarmason
  2021-05-27 17:46     ` Felipe Contreras
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
  38 siblings, 2 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-27 13:36 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: git, Jeff King, Junio C Hamano, James Ramsay, Jonathan Nieder,
	brian m. carlson, Phillip Wood, Josh Steadmon,
	Johannes Schindelin, Jonathan Tan


On Wed, May 26 2021, Emily Shaffer wrote:

> After much delay and $DAYJOB, here is v9.
>
> - Addressed nits in reviews on v8
> - sendemail-validate hook becomes non-parallelized; updated to use
>   Ævar's updated system_or_die() function
> - changed strbuf to char* in hooks_list
>   - Attempted to do so in run_command's stdout callback, but this made
>     length protection difficult, so stuck with strbuf there.
> - test_i18ncmp -> test_cmp
> - Stop doing i18n lego in run_hooks()
> - Checked that run_hooks_opt_init() is always separated by a space from
>   variable decl blocks
> - Checked for early returns which may skip run_hooks_opt_clear(); this
>   resulted in minimizing the scope of run_hooks_opt in most places
> - Got rid of native-hooks.txt. It was a nice idea, but not attached to a
>   large and slow series like this one.
> - In traces, log the name of the hook (e.g. "pre-commit") instead of the
>   name of the executable (e.g. "/home/emily/check-for-debug-strings");
>   the executable name is tracelogged as part of argv anyways, and we
>   want to be able to tell which hook was responsible for invoking the
>   executable in question.

In v8 I had some comments about the overall approach here. I've got to
say I'm disappointed that that's neither been replied to or in any way
addressed:

    https://lore.kernel.org/git/87mtv8fww3.fsf@evledraar.gmail.com/

Also referenced in several follow-up discussions, including Junio's
comment of "OK, Emily, I guess the ball is in your court now?":

    https://lore.kernel.org/git/?q=87mtv8fww3.fsf%40evledraar.gmail.com

Basically I think that this very large series could be easily split into
much more digestable chunks if it were re-arranged. Right now it's
essentially a 37 patch mix of, in order:

 1. Design doc
 2. Add "git hook" with "run", "list" etc.
 3. Make that more fully featured, support config, legacy hooks et.c
 4. Implement parallel hooks
 5. Go through N existing hook callers and migrate them to "git hook run"
 6. Some misc updates, e.g. to docs

If it were instead:

 1. Add "git hook run", only supports "old/legacy" .git/hooks/ hooks
 2. Go through N existing hook callers and migrate them to "git hook run"

That would be at least half of this series, and it would be much more
narrow change that would demonstrably retain all existing
semantics. We'd simply call our hooks via a command wrapper instead of
via run_command(<path to name>) as we do now. So we could have that land
and then focus on the actual behavior changes later.

In earlier rounds/the above E-Mail I asked something to the effect of if
you thought that was a good approach, whether disagreed, or just thought
you didn't have time for it.

I'm still keen to help get this in, but given the non-responses don't
know where you stand on it. I suppose I could re-arrange this myself and
submit an alternate "prep" series to rebase this on, but wanted to get
your take first.

Aside from the suggestion of splitting it up:

In the above referenced correspondence I expressed concerns that the
config layout you're proposing needlessly creates complexity requiring a
"git hook list" etc. command to solve, as opposed to not having that, or
have it by a trivial synonym for a "git config --show-origin --list"
invocation.

I'm still interested in what you think of that, as being able to
normalize that more invites getting rid of more complexity without
impacting the end-goal of hooks via config.

But in any case such a discussion could be had later and on a smaller
series if the refactoring of existing hook running was split up from the
big semantic changes here, which are currently tied up with that
refactoring.

Thanks!

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v9 00/37] propose config-based hooks
  2021-05-27 13:36   ` Ævar Arnfjörð Bjarmason
@ 2021-05-27 17:46     ` Felipe Contreras
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
  1 sibling, 0 replies; 479+ messages in thread
From: Felipe Contreras @ 2021-05-27 17:46 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason, Emily Shaffer
  Cc: git, Jeff King, Junio C Hamano, James Ramsay, Jonathan Nieder,
	brian m. carlson, Phillip Wood, Josh Steadmon,
	Johannes Schindelin, Jonathan Tan

Ævar Arnfjörð Bjarmason wrote:
> If it were instead:
> 
>  1. Add "git hook run", only supports "old/legacy" .git/hooks/ hooks
>  2. Go through N existing hook callers and migrate them to "git hook run"

Yes please. It would make the series much easier to review, and also to
understand what the point of it is.

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v9 22/37] read-cache: convert post-index-change hook to use config
  2021-05-27  0:08   ` [PATCH v9 22/37] read-cache: convert post-index-change hook to use config Emily Shaffer
@ 2021-05-27 23:04     ` Ævar Arnfjörð Bjarmason
  2021-05-28  1:09       ` Taylor Blau
  2021-05-31 19:21       ` Felipe Contreras
  0 siblings, 2 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-27 23:04 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Wed, May 26 2021, Emily Shaffer wrote:

>  Part of the linkgit:git[1] suite
> diff --git a/read-cache.c b/read-cache.c
> index 1b3c2eb408..6a5c9403f4 100644
> --- a/read-cache.c
> +++ b/read-cache.c
> @@ -26,6 +26,8 @@
>  #include "thread-utils.h"
>  #include "progress.h"
>  #include "sparse-index.h"
> +#include "hook.h"
> +>>>>>>> 9524a9d29d (read-cache: convert post-index-change hook to use config)

This adds a conflict marker, which is removed later in the series.

Obviously a trivial mistake, but it's a good idea to use git rebase -i
-x 'make test' or equivalent for such a large series, perhaps there are
other inter-patch issues lurking here...

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v9 22/37] read-cache: convert post-index-change hook to use config
  2021-05-27 23:04     ` Ævar Arnfjörð Bjarmason
@ 2021-05-28  1:09       ` Taylor Blau
  2021-05-31 19:21       ` Felipe Contreras
  1 sibling, 0 replies; 479+ messages in thread
From: Taylor Blau @ 2021-05-28  1:09 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: Emily Shaffer, git

On Fri, May 28, 2021 at 01:04:40AM +0200, Ævar Arnfjörð Bjarmason wrote:
> Obviously a trivial mistake, but it's a good idea to use git rebase -i
> -x 'make test' or equivalent for such a large series, perhaps there are
> other inter-patch issues lurking here...

This is tangential to this series, but I would add that `git rebase -x
'make DEVELOPER=1 git' @{u}` can be useful to run often while developing
the series and reorganizing patches to make sure that everything
compiles.

The benefit of `make git` is that it ensures everything still compiles
while avoiding having to link everything together. This ends up being
quick enough that I find myself running it often while developing a
series.

I do `s/git/test` before submitting, though.

Thanks,
Taylor

^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH 00/31] minimal restart of "config-based-hooks"
  2021-05-27 13:36   ` Ævar Arnfjörð Bjarmason
  2021-05-27 17:46     ` Felipe Contreras
@ 2021-05-28 12:11     ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 01/31] hooks tests: don't leave "actual" nonexisting on failure Ævar Arnfjörð Bjarmason
                         ` (32 more replies)
  1 sibling, 33 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

After suggesting[1] an another round that the config-based-hook
topic[2] should take a more incremental approach to reach its end goal
I thought I'd try hacking that up.

So this is a proposed restart of that topic which if the consensus
favors it should replace it, and the config-based hooks topic should
be rebased on top of this.

As noted in [1] there are no user-visible behavior changes here. All
that's being done is to consolidate hook dispatch into the new
hook.[ch] library, and the "git hook run" command (used by
git-send-email and git-p4).

The range-diff below is scary, I recommend just reading this from
scratch. Some incomplete notes on changed things from Emily's v9:

 * First off, we've gone from:

    54 files changed, 2689 insertions(+), 713 deletions(-)

   to:

    41 files changed, 1293 insertions(+), 555 deletions(-)

   So while there's a similar number of patches the change is much
   smaller.

 * There's no config via hooks, design doc etc. in this series. It's
   meant as a basis on which to build that. We still only support
   running .git/hook/<hook>, and we run those with .jobs=1 by
   definition.

 * There's absolutely no behavior change in how we run these hooks,
   unlike in Emily's version. E.g. in hers sometimes we'd always pass
   absolute paths when before we didn't, now we just do that for "git
   worktree" via a flag as on the current "master". There were also
   some other subtle changes as seen in the tests. Now the only test
   changes are the addition of more and missing tests.

   Some of this (e.g. git-send-email's insistance on GIT_DIR being
   set) is something we'll probably need to change sooner than later,
   but for now we've got bug-for-bug compatibility.

 * This has been re-arranged as much as possible to start using
   minimal bits of the library as soon as we can. E.g. in Emily's the
   whole of hook.c is implemented before any hook is migrated over to
   it.

   In this version we barely know how to do anything yet (no stdin
   handling etc.) when we can move the pre-auto-gc hook over, we then
   do a few more hooks and add stdin support, migrate the hooks that
   need that etc.

 * In Emily's there's a mid-series switch from the run_command() API
   to run_processes_parallel_tr2() as we learn parallel execution. I
   thought this was more complex to read and just reflected the past
   evolution of the topic.

   That's now squashed together, we always have the parallel API use,
   we just have .jobs=1 throughout.

 * I tried as much as possible to similarly squash together
   e.g. removing an unused API along with the commit that removes the
   last user of it, not introducing APIs without simultaneously
   introducing users of them etc.

 * The "git hook run" command now takes arguments as:

        git hook run a-hook -- some arg u ments

   Not:

        git hook run -a some -a arg -a u -a ments a-hook

    The "-e" for "pass this env variable" is also gone, nothing
    actually made use of it.

    If we need it in the future surely we can just set them in the
    environment instead, and ask the relevant command run APIs not to
    clobber things.

    I think the -- (or --end-of-options) is easier to work
    with. Changing this allowed removing the whole
    parse-options-strvec part of Emily's series. It's now replaced
    with PARSE_OPT_KEEP_DASHDASH + 2-4 lines of code around that in
    builtin/hook.c.

 * Even though this extracted the "config based hooks" part, and .jobs
   != 1 support I've tried as much as possible to keep the same code
   layout, to the point where hooks e.g. set .jobs=1 still if they
   insist that one job should be run. It should thus be fairly drop-in
   compatible for building .jobs > 1 support on top, config-based hooks
   etc.

 * The two last patches are new, a couple of minor bugfixes of mine
   that I noticed while hacking on this. One solves a long-standing
   TOCTOU in run_a_hook() and then have_hook() to see if we ran it (we
   now just remember if we ran it), and there's now a generated and
   canonical hook-list.h similar to config-list.h, except this one's
   more strict. We'll die on unknown hooks unless they're in
   Documentation/githooks.txt.

1. https://lore.kernel.org/git/87lf80l1m6.fsf@evledraar.gmail.com/
2. https://lore.kernel.org/git/20210527000856.695702-1-emilyshaffer@google.com/

Emily Shaffer (25):
  hook: add 'run' subcommand
  hook.c: add a hook_exists() wrapper and use it in bugreport.c
  gc: use hook library for pre-auto-gc hook
  rebase: teach pre-rebase to use hook.h
  am: convert applypatch hooks to use config
  hooks: convert 'post-checkout' hook to hook library
  merge: use config-based hooks for post-merge hook
  send-email: use 'git hook run' for 'sendemail-validate'
  git-p4: use 'git hook' to run hooks
  commit: use hook.h to execute hooks
  read-cache: convert post-index-change hook to use config
  receive-pack: convert push-to-checkout hook to hook.h
  run-command: allow stdin for run_processes_parallel
  hook: support passing stdin to hooks
  am: convert 'post-rewrite' hook to hook.h
  run-command: add stdin callback for parallelization
  hook: provide stdin by string_list or callback
  hook: convert 'post-rewrite' hook in sequencer.c to hook.h
  transport: convert pre-push hook to use config
  reference-transaction: use hook.h to run hooks
  run-command: allow capturing of collated output
  hooks: allow callers to capture output
  receive-pack: convert 'update' hook to hook.h
  post-update: use hook.h library
  receive-pack: convert receive hooks to hook.h

Ævar Arnfjörð Bjarmason (6):
  hooks tests: don't leave "actual" nonexisting on failure
  gc tests: add a test for the "pre-auto-gc" hook
  run-command.h: move find_hook() to hook.h
  git hook run: add an --ignore-missing flag
  hooks: fix a TOCTOU in "did we run a hook?" heuristic
  hook-list.h: add a generated list of hooks, like config-list.h

 .gitignore                   |   2 +
 Documentation/git-hook.txt   |  49 ++++++
 Documentation/githooks.txt   |   4 +
 Makefile                     |  16 +-
 builtin.h                    |   1 +
 builtin/am.c                 |  34 ++--
 builtin/bugreport.c          |  46 ++----
 builtin/checkout.c           |  17 +-
 builtin/clone.c              |   7 +-
 builtin/commit.c             |  19 ++-
 builtin/fetch.c              |   1 +
 builtin/gc.c                 |   8 +-
 builtin/hook.c               |  72 +++++++++
 builtin/merge.c              |  22 ++-
 builtin/rebase.c             |   9 +-
 builtin/receive-pack.c       | 299 ++++++++++++++++++-----------------
 builtin/submodule--helper.c  |   2 +-
 builtin/worktree.c           |  31 ++--
 command-list.txt             |   1 +
 commit.c                     |  17 +-
 commit.h                     |   3 +-
 generate-hooklist.sh         |  20 +++
 git-p4.py                    |  72 +--------
 git-send-email.perl          |  20 ++-
 git.c                        |   1 +
 hook.c                       | 224 ++++++++++++++++++++++++++
 hook.h                       | 122 ++++++++++++++
 read-cache.c                 |  12 +-
 refs.c                       |  43 ++---
 reset.c                      |  15 +-
 run-command.c                | 157 +++++++++---------
 run-command.h                |  55 ++++---
 sequencer.c                  |  88 +++++------
 submodule.c                  |   1 +
 t/helper/test-run-command.c  |  46 +++++-
 t/t0061-run-command.sh       |  37 +++++
 t/t1350-config-hooks-path.sh |   1 +
 t/t1800-hook.sh              | 166 +++++++++++++++++++
 t/t6500-gc.sh                |  46 ++++++
 t/t9001-send-email.sh        |   4 +-
 transport.c                  |  58 ++-----
 41 files changed, 1293 insertions(+), 555 deletions(-)
 create mode 100644 Documentation/git-hook.txt
 create mode 100644 builtin/hook.c
 create mode 100755 generate-hooklist.sh
 create mode 100644 hook.c
 create mode 100644 hook.h
 create mode 100755 t/t1800-hook.sh

Range-diff:
 1:  9540c006dc0 <  -:  ----------- doc: propose hooks managed by the config
 -:  ----------- >  1:  8ac2efc71a0 hooks tests: don't leave "actual" nonexisting on failure
 -:  ----------- >  2:  eb37693f7dc gc tests: add a test for the "pre-auto-gc" hook
 2:  2f50cfe7b92 !  3:  1ad4e69f7da hook: introduce git-hook subcommand
    @@ Metadata
     Author: Emily Shaffer <emilyshaffer@google.com>
     
      ## Commit message ##
    -    hook: introduce git-hook subcommand
    +    hook: add 'run' subcommand
     
    -    Add a new subcommand, git-hook, which will be used to ease config-based
    -    hook management. This command will handle parsing configs to compose a
    -    list of hooks to run for a given event, as well as adding or modifying
    -    hook configs in an interactive fashion.
    +    In order to enable hooks to be run as an external process, by a
    +    standalone Git command, or by tools which wrap Git, provide an external
    +    means to run all configured hook commands for a given hook event.
     
    -    Start with 'git hook list <hookname>', which checks the known configs in
    -    order to create an ordered list of hooks to run on a given hook event.
    -
    -    Multiple commands can be specified for a given hook by providing
    -    multiple "hook.<hookname>.command = <path-to-hook>" lines. Hooks will be
    -    run in config order. If more properties need to be set on a given hook
    -    in the future, commands can also be specified by providing
    -    "hook.<hookname>.command = <hookcmd-name>", as well as a "[hookcmd
    -    <hookcmd-name>]" subsection; this subsection should contain a
    -    "hookcmd.<hookcmd-name>.command = <path-to-hook>" line.
    -
    -    For example:
    -
    -      $ git config --list | grep ^hook
    -      hook.pre-commit.command=baz
    -      hook.pre-commit.command=~/bar.sh
    -      hookcmd.baz.command=~/baz/from/hookcmd.sh
    -
    -      $ git hook list pre-commit
    -      global: ~/baz/from/hookcmd.sh
    -      local: ~/bar.sh
    +    Most of our hooks require more complex functionality than this, but
    +    let's start with the bare minimum required to support our simplest
    +    hooks.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## .gitignore ##
     @@
    @@ .gitignore
      /git-http-fetch
      /git-http-push
     
    - ## Documentation/config/hook.txt (new) ##
    -@@
    -+hook.<command>.command::
    -+	A command to execute during the <command> hook event. This can be an
    -+	executable on your device, a oneliner for your shell, or the name of a
    -+	hookcmd. See linkgit:git-hook[1].
    -+
    -+hookcmd.<name>.command::
    -+	A command to execute during a hook for which <name> has been specified
    -+	as a command. This can be an executable on your device or a oneliner for
    -+	your shell. See linkgit:git-hook[1].
    -
      ## Documentation/git-hook.txt (new) ##
     @@
     +git-hook(1)
    @@ Documentation/git-hook.txt (new)
     +
     +NAME
     +----
    -+git-hook - Manage configured hooks
    ++git-hook - run git hooks
     +
     +SYNOPSIS
     +--------
     +[verse]
    -+'git hook' list <hook-name>
    ++'git hook' run <hook-name> [-- <hook-args>]
     +
     +DESCRIPTION
     +-----------
    -+You can list configured hooks with this command. Later, you will be able to run,
    -+add, and modify hooks with this command.
    -+
    -+This command parses the default configuration files for sections `hook` and
    -+`hookcmd`. `hook` is used to describe the commands which will be run during a
    -+particular hook event; commands are run in the order Git encounters them during
    -+the configuration parse (see linkgit:git-config[1]). `hookcmd` is used to
    -+describe attributes of a specific command. If additional attributes don't need
    -+to be specified, a command to run can be specified directly in the `hook`
    -+section; if a `hookcmd` by that name isn't found, Git will attempt to run the
    -+provided value directly. For example:
    -+
    -+Global config
    -+----
    -+  [hook "post-commit"]
    -+    command = "linter"
    -+    command = "~/typocheck.sh"
     +
    -+  [hookcmd "linter"]
    -+    command = "/bin/linter --c"
    -+----
    ++This command is an interface to git hooks (see linkgit:githooks[5]).
    ++Currently it only provides a convenience wrapper for running hooks for
    ++use by git itself. In the future it might gain other functionality.
     +
    -+Local config
    -+----
    -+  [hook "prepare-commit-msg"]
    -+    command = "linter"
    -+  [hook "post-commit"]
    -+    command = "python ~/run-test-suite.py"
    -+----
    ++SUBCOMMANDS
    ++-----------
     +
    -+With these configs, you'd then see:
    ++run::
     +
    -+----
    -+$ git hook list "post-commit"
    -+global: /bin/linter --c
    -+global: ~/typocheck.sh
    -+local: python ~/run-test-suite.py
    ++	Run the `<hook-name>` hook. Any positional arguments to the
    ++	hook should be passed after an optional "--" (or
    ++	"--end-of-options"). See "OPTIONS" below for the arguments
    ++	this accepts.
     +
    -+$ git hook list "prepare-commit-msg"
    -+local: /bin/linter --c
    -+----
    -+
    -+COMMANDS
    ++SEE ALSO
     +--------
    -+
    -+list `<hook-name>`::
    -+
    -+List the hooks which have been configured for `<hook-name>`. Hooks appear
    -+in the order they should be run, and print the config scope where the relevant
    -+`hook.<hook-name>.command` was specified, not the `hookcmd` (if applicable).
    -+This output is human-readable and the format is subject to change over time.
    -+
    -+CONFIGURATION
    -+-------------
    -+include::config/hook.txt[]
    ++linkgit:githooks[5]
     +
     +GIT
     +---
     +Part of the linkgit:git[1] suite
     
    + ## Documentation/githooks.txt ##
    +@@ Documentation/githooks.txt: and "0" meaning they were not.
    + Only one parameter should be set to "1" when the hook runs.  The hook
    + running passing "1", "1" should not be possible.
    + 
    ++SEE ALSO
    ++--------
    ++linkgit:git-hook[1]
    ++
    + GIT
    + ---
    + Part of the linkgit:git[1] suite
    +
      ## Makefile ##
     @@ Makefile: LIB_OBJS += hash-lookup.o
      LIB_OBJS += hashmap.o
    @@ builtin/hook.c (new)
     +#include "hook.h"
     +#include "parse-options.h"
     +#include "strbuf.h"
    ++#include "strvec.h"
     +
     +static const char * const builtin_hook_usage[] = {
    -+	N_("git hook list <hookname>"),
    ++	N_("git hook run <hook-name> [-- <hook-args>]"),
     +	NULL
     +};
     +
    -+static int list(int argc, const char **argv, const char *prefix)
    ++static int run(int argc, const char **argv, const char *prefix)
     +{
    -+	struct list_head *head, *pos;
    -+	const char *hookname = NULL;
    ++	int i;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    ++	int rc = 0;
    ++	const char *hook_name;
    ++	const char *hook_path;
     +
    -+	struct option list_options[] = {
    ++	struct option run_options[] = {
     +		OPT_END(),
     +	};
     +
    -+	argc = parse_options(argc, argv, prefix, list_options,
    -+			     builtin_hook_usage, 0);
    -+
    -+	if (argc < 1) {
    -+		usage_msg_opt(_("You must specify a hook event name to list."),
    -+			      builtin_hook_usage, list_options);
    ++	argc = parse_options(argc, argv, prefix, run_options,
    ++			     builtin_hook_usage, PARSE_OPT_KEEP_UNKNOWN | PARSE_OPT_KEEP_DASHDASH);
    ++
    ++	if (argc > 2) {
    ++		if (strcmp(argv[2], "--") &&
    ++		    strcmp(argv[2], "--end-of-options"))
    ++			/* Having a -- for "run" is mandatory */
    ++			usage_with_options(builtin_hook_usage, run_options);
    ++		/* Add our arguments, start after -- */
    ++		for (i = 3 ; i < argc; i++)
    ++			strvec_push(&opt.args, argv[i]);
     +	}
     +
    -+	hookname = argv[0];
    -+
    -+	head = hook_list(hookname);
    -+
    -+	if (list_empty(head)) {
    -+		printf(_("no commands configured for hook '%s'\n"),
    -+		       hookname);
    -+		return 0;
    -+	}
    ++	/* Need to take into account core.hooksPath */
    ++	git_config(git_default_config, NULL);
     +
    -+	list_for_each(pos, head) {
    -+		struct hook *item = list_entry(pos, struct hook, list);
    -+		if (item)
    -+			printf("%s: %s\n",
    -+			       config_scope_name(item->origin),
    -+			       item->command.buf);
    ++	hook_name = argv[1];
    ++	hook_path = find_hook(hook_name);
    ++	if (!hook_path) {
    ++		error("cannot find a hook named %s", hook_name);
    ++		return 1;
     +	}
    ++	rc = run_found_hooks(hook_name, hook_path, &opt);
     +
    -+	clear_hook_list(head);
    ++	run_hooks_opt_clear(&opt);
     +
    -+	return 0;
    ++	return rc;
     +}
     +
     +int cmd_hook(int argc, const char **argv, const char *prefix)
    @@ builtin/hook.c (new)
     +	struct option builtin_hook_options[] = {
     +		OPT_END(),
     +	};
    -+	if (argc < 2)
    -+		usage_with_options(builtin_hook_usage, builtin_hook_options);
    -+
    -+	if (!strcmp(argv[1], "list"))
    -+		return list(argc - 1, argv + 1, prefix);
     +
    ++	if (!strcmp(argv[1], "run"))
    ++		return run(argc, argv, prefix);
     +	usage_with_options(builtin_hook_usage, builtin_hook_options);
    ++	return 1;
     +}
     
      ## command-list.txt ##
    @@ git.c: static struct cmd_struct commands[] = {
      	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
      	{ "hash-object", cmd_hash_object },
      	{ "help", cmd_help },
    -+	{ "hook", cmd_hook, RUN_SETUP_GENTLY },
    ++	{ "hook", cmd_hook, RUN_SETUP },
      	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
      	{ "init", cmd_init_db },
      	{ "init-db", cmd_init_db },
    @@ git.c: static struct cmd_struct commands[] = {
      ## hook.c (new) ##
     @@
     +#include "cache.h"
    -+
     +#include "hook.h"
    -+#include "config.h"
    ++#include "run-command.h"
     +
    -+void free_hook(struct hook *ptr)
    ++void run_hooks_opt_clear(struct run_hooks_opt *o)
     +{
    -+	if (ptr) {
    -+		strbuf_release(&ptr->command);
    -+		free(ptr);
    -+	}
    ++	strvec_clear(&o->env);
    ++	strvec_clear(&o->args);
     +}
     +
    -+static void append_or_move_hook(struct list_head *head, const char *command)
    ++static int pick_next_hook(struct child_process *cp,
    ++			  struct strbuf *out,
    ++			  void *pp_cb,
    ++			  void **pp_task_cb)
     +{
    -+	struct list_head *pos = NULL, *tmp = NULL;
    -+	struct hook *to_add = NULL;
    ++	struct hook_cb_data *hook_cb = pp_cb;
    ++	struct hook *run_me = hook_cb->run_me;
    ++
    ++	if (!run_me)
    ++		BUG("did we not return 1 in notify_hook_finished?");
    ++
    ++	cp->no_stdin = 1;
    ++	cp->env = hook_cb->options->env.v;
    ++	cp->stdout_to_stderr = 1;
    ++	cp->trace2_hook_name = hook_cb->hook_name;
    ++
    ++	/* add command */
    ++	strvec_push(&cp->args, run_me->hook_path);
     +
     +	/*
    -+	 * remove the prior entry with this command; we'll replace it at the
    -+	 * end.
    ++	 * add passed-in argv, without expanding - let the user get back
    ++	 * exactly what they put in
     +	 */
    -+	list_for_each_safe(pos, tmp, head) {
    -+		struct hook *it = list_entry(pos, struct hook, list);
    -+		if (!strcmp(it->command.buf, command)) {
    -+		    list_del(pos);
    -+		    /* we'll simply move the hook to the end */
    -+		    to_add = it;
    -+		    break;
    -+		}
    -+	}
    -+
    -+	if (!to_add) {
    -+		/* adding a new hook, not moving an old one */
    -+		to_add = xmalloc(sizeof(*to_add));
    -+		strbuf_init(&to_add->command, 0);
    -+		strbuf_addstr(&to_add->command, command);
    -+	}
    ++	strvec_pushv(&cp->args, hook_cb->options->args.v);
     +
    -+	/* re-set the scope so we show where an override was specified */
    -+	to_add->origin = current_config_scope();
    ++	/* Provide context for errors if necessary */
    ++	*pp_task_cb = run_me;
     +
    -+	list_add_tail(&to_add->list, head);
    ++	return 1;
     +}
     +
    -+static void remove_hook(struct list_head *to_remove)
    ++static int notify_start_failure(struct strbuf *out,
    ++				void *pp_cb,
    ++				void *pp_task_cp)
     +{
    -+	struct hook *hook_to_remove = list_entry(to_remove, struct hook, list);
    -+	list_del(to_remove);
    -+	free_hook(hook_to_remove);
    -+}
    ++	struct hook_cb_data *hook_cb = pp_cb;
    ++	struct hook *attempted = pp_task_cp;
     +
    -+void clear_hook_list(struct list_head *head)
    -+{
    -+	struct list_head *pos, *tmp;
    -+	list_for_each_safe(pos, tmp, head)
    -+		remove_hook(pos);
    ++	/* |= rc in cb */
    ++	hook_cb->rc |= 1;
    ++
    ++	strbuf_addf(out, _("Couldn't start hook '%s'\n"),
    ++		    attempted->hook_path);
    ++
    ++	return 1;
     +}
     +
    -+struct hook_config_cb
    ++static int notify_hook_finished(int result,
    ++				struct strbuf *out,
    ++				void *pp_cb,
    ++				void *pp_task_cb)
     +{
    -+	struct strbuf *hookname;
    -+	struct list_head *list;
    -+};
    ++	struct hook_cb_data *hook_cb = pp_cb;
     +
    -+static int hook_config_lookup(const char *key, const char *value, void *cb_data)
    -+{
    -+	struct hook_config_cb *data = cb_data;
    -+	const char *hook_key = data->hookname->buf;
    -+	struct list_head *head = data->list;
    -+
    -+	if (!strcmp(key, hook_key)) {
    -+		const char *command = value;
    -+		struct strbuf hookcmd_name = STRBUF_INIT;
    -+
    -+		/*
    -+		 * Check if a hookcmd with that name exists. If it doesn't,
    -+		 * 'git_config_get_value()' is documented not to touch &command,
    -+		 * so we don't need to do anything.
    -+		 */
    -+		strbuf_addf(&hookcmd_name, "hookcmd.%s.command", command);
    -+		git_config_get_value(hookcmd_name.buf, &command);
    -+
    -+		if (!command) {
    -+			strbuf_release(&hookcmd_name);
    -+			BUG("git_config_get_value overwrote a string it shouldn't have");
    -+		}
    -+
    -+		/*
    -+		 * TODO: implement an option-getting callback, e.g.
    -+		 *   get configs by pattern hookcmd.$value.*
    -+		 *   for each key+value, do_callback(key, value, cb_data)
    -+		 */
    -+
    -+		append_or_move_hook(head, command);
    -+
    -+		strbuf_release(&hookcmd_name);
    -+	}
    ++	/* |= rc in cb */
    ++	hook_cb->rc |= result;
     +
    -+	return 0;
    ++	return 1;
     +}
     +
    -+struct list_head* hook_list(const char* hookname)
    ++
    ++int run_found_hooks(const char *hook_name, const char *hook_path,
    ++		    struct run_hooks_opt *options)
     +{
    -+	struct strbuf hook_key = STRBUF_INIT;
    -+	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
    -+	struct hook_config_cb cb_data = { &hook_key, hook_head };
    ++	struct hook my_hook = {
    ++		.hook_path = hook_path,
    ++	};
    ++	struct hook_cb_data cb_data = {
    ++		.rc = 0,
    ++		.hook_name = hook_name,
    ++		.options = options,
    ++	};
    ++	cb_data.run_me = &my_hook;
    ++
    ++	if (options->jobs != 1)
    ++		BUG("we do not handle %d or any other != 1 job number yet", options->jobs);
     +
    -+	INIT_LIST_HEAD(hook_head);
    ++	run_processes_parallel_tr2(options->jobs,
    ++				   pick_next_hook,
    ++				   notify_start_failure,
    ++				   notify_hook_finished,
    ++				   &cb_data,
    ++				   "hook",
    ++				   hook_name);
     +
    -+	if (!hookname)
    -+		return NULL;
    ++	return cb_data.rc;
    ++}
    ++
    ++int run_hooks(const char *hook_name, struct run_hooks_opt *options)
    ++{
    ++	const char *hook_path;
    ++	int ret;
    ++	if (!options)
    ++		BUG("a struct run_hooks_opt must be provided to run_hooks");
     +
    -+	strbuf_addf(&hook_key, "hook.%s.command", hookname);
    ++	hook_path = find_hook(hook_name);
     +
    -+	git_config(hook_config_lookup, &cb_data);
    ++	/* Care about nonexistence? Use run_found_hooks() */
    ++	if (!hook_path)
    ++		return 0;
     +
    -+	strbuf_release(&hook_key);
    -+	return hook_head;
    ++	ret = run_found_hooks(hook_name, hook_path, options);
    ++	return ret;
     +}
     
      ## hook.h (new) ##
     @@
    -+#include "config.h"
    -+#include "list.h"
    ++#ifndef HOOK_H
    ++#define HOOK_H
     +#include "strbuf.h"
    ++#include "strvec.h"
    ++#include "run-command.h"
     +
     +struct hook {
    -+	struct list_head list;
    -+	/*
    -+	 * Config file which holds the hook.*.command definition.
    -+	 * (This has nothing to do with the hookcmd.<name>.* configs.)
    -+	 */
    -+	enum config_scope origin;
    -+	/* The literal command to run. */
    -+	struct strbuf command;
    ++	/* The path to the hook */
    ++	const char *hook_path;
    ++};
    ++
    ++struct run_hooks_opt
    ++{
    ++	/* Environment vars to be set for each hook */
    ++	struct strvec env;
    ++
    ++	/* Args to be passed to each hook */
    ++	struct strvec args;
    ++
    ++	/* Number of threads to parallelize across */
    ++	int jobs;
     +};
     +
    ++#define RUN_HOOKS_OPT_INIT { \
    ++	.jobs = 1, \
    ++	.env = STRVEC_INIT, \
    ++	.args = STRVEC_INIT, \
    ++}
    ++
     +/*
    -+ * Provides a linked list of 'struct hook' detailing commands which should run
    -+ * in response to the 'hookname' event, in execution order.
    ++ * Callback provided to feed_pipe_fn and consume_sideband_fn.
     + */
    -+struct list_head* hook_list(const char *hookname);
    ++struct hook_cb_data {
    ++	int rc;
    ++	const char *hook_name;
    ++	struct hook *run_me;
    ++	struct run_hooks_opt *options;
    ++};
     +
    -+/* Free memory associated with a 'struct hook' */
    -+void free_hook(struct hook *ptr);
    -+/* Empties the list at 'head', calling 'free_hook()' on each entry */
    -+void clear_hook_list(struct list_head *head);
    ++void run_hooks_opt_clear(struct run_hooks_opt *o);
    ++
    ++/*
    ++ * Calls find_hook(hookname) and runs the hooks (if any) with
    ++ * run_found_hooks().
    ++ */
    ++int run_hooks(const char *hook_name, struct run_hooks_opt *options);
    ++
    ++/*
    ++ * Takes an already resolved hook and runs it. Internally the simpler
    ++ * run_hooks() will call this.
    ++ */
    ++int run_found_hooks(const char *hookname, const char *hook_path,
    ++		    struct run_hooks_opt *options);
    ++#endif
     
    - ## t/t1360-config-based-hooks.sh (new) ##
    + ## t/t1800-hook.sh (new) ##
     @@
     +#!/bin/bash
     +
    -+test_description='config-managed multihooks, including git-hook command'
    ++test_description='git-hook command'
     +
     +. ./test-lib.sh
     +
    -+ROOT=
    -+if test_have_prereq MINGW
    -+then
    -+	# In Git for Windows, Unix-like paths work only in shell scripts;
    -+	# `git.exe`, however, will prefix them with the pseudo root directory
    -+	# (of the Unix shell). Let's accommodate for that.
    -+	ROOT="$(cd / && pwd)"
    -+fi
    -+
    -+setup_hooks () {
    -+	test_config hook.pre-commit.command "/path/ghi" --add
    -+	test_config_global hook.pre-commit.command "/path/def" --add
    -+}
    ++test_expect_success 'setup .git/hooks' '
    ++	mkdir .git/hooks
    ++'
     +
    -+setup_hookcmd () {
    -+	test_config hook.pre-commit.command "abc" --add
    -+	test_config_global hookcmd.abc.command "/path/abc" --add
    -+}
    ++test_expect_success 'git hook run -- nonexistent hook' '
    ++	cat >stderr.expect <<-\EOF &&
    ++	error: cannot find a hook named does-not-exist
    ++	EOF
    ++	test_expect_code 1 git hook run does-not-exist 2>stderr.actual &&
    ++	test_cmp stderr.expect stderr.actual
    ++'
     +
    -+test_expect_success 'git hook rejects commands without a mode' '
    -+	test_must_fail git hook pre-commit
    ++test_expect_success 'git hook run -- basic' '
    ++	write_script .git/hooks/test-hook <<-EOF &&
    ++	echo Test hook
    ++	EOF
    ++
    ++	cat >expect <<-\EOF &&
    ++	Test hook
    ++	EOF
    ++	git hook run test-hook 2>actual &&
    ++	test_cmp expect actual
     +'
     +
    ++test_expect_success 'git hook run -- stdout and stderr handling' '
    ++	write_script .git/hooks/test-hook <<-EOF &&
    ++	echo >&1 Will end up on stderr
    ++	echo >&2 Will end up on stderr
    ++	EOF
     +
    -+test_expect_success 'git hook rejects commands without a hookname' '
    -+	test_must_fail git hook list
    ++	cat >stderr.expect <<-\EOF &&
    ++	Will end up on stderr
    ++	Will end up on stderr
    ++	EOF
    ++	git hook run test-hook >stdout.actual 2>stderr.actual &&
    ++	test_cmp stderr.expect stderr.actual &&
    ++	test_must_be_empty stdout.actual
     +'
     +
    -+test_expect_success 'git hook runs outside of a repo' '
    -+	setup_hooks &&
    ++test_expect_success 'git hook run -- exit codes are passed along' '
    ++	write_script .git/hooks/test-hook <<-EOF &&
    ++	exit 1
    ++	EOF
    ++
    ++	test_expect_code 1 git hook run test-hook &&
     +
    -+	cat >expected <<-EOF &&
    -+	global: $ROOT/path/def
    ++	write_script .git/hooks/test-hook <<-EOF &&
    ++	exit 2
     +	EOF
     +
    -+	nongit git config --list --global &&
    ++	test_expect_code 2 git hook run test-hook &&
     +
    -+	nongit git hook list pre-commit >actual &&
    -+	test_cmp expected actual
    -+'
    ++	write_script .git/hooks/test-hook <<-EOF &&
    ++	exit 128
    ++	EOF
     +
    -+test_expect_success 'git hook list orders by config order' '
    -+	setup_hooks &&
    ++	test_expect_code 128 git hook run test-hook &&
     +
    -+	cat >expected <<-EOF &&
    -+	global: $ROOT/path/def
    -+	local: $ROOT/path/ghi
    ++	write_script .git/hooks/test-hook <<-EOF &&
    ++	exit 129
     +	EOF
     +
    -+	git hook list pre-commit >actual &&
    -+	test_cmp expected actual
    ++	test_expect_code 129 git hook run test-hook
     +'
     +
    -+test_expect_success 'git hook list dereferences a hookcmd' '
    -+	setup_hooks &&
    -+	setup_hookcmd &&
    ++test_expect_success 'git hook run arg u ments without -- is not allowed' '
    ++	test_expect_code 129 git hook run test-hook arg u ments
    ++'
    ++
    ++test_expect_success 'git hook run -- pass arguments' '
    ++	write_script .git/hooks/test-hook <<-\EOF &&
    ++	echo $1
    ++	echo $2
    ++	EOF
     +
    -+	cat >expected <<-EOF &&
    -+	global: $ROOT/path/def
    -+	local: $ROOT/path/ghi
    -+	local: $ROOT/path/abc
    ++	cat >expect <<-EOF &&
    ++	arg
    ++	u ments
     +	EOF
     +
    -+	git hook list pre-commit >actual &&
    -+	test_cmp expected actual
    ++	git hook run test-hook -- arg "u ments" 2>actual &&
    ++	test_cmp expect actual
     +'
     +
    -+test_expect_success 'git hook list reorders on duplicate commands' '
    -+	setup_hooks &&
    ++test_expect_success 'git hook run -- out-of-repo runs excluded' '
    ++	write_script .git/hooks/test-hook <<-EOF &&
    ++	echo Test hook
    ++	EOF
     +
    -+	test_config hook.pre-commit.command "/path/def" --add &&
    ++	nongit test_must_fail git hook run test-hook
    ++'
     +
    -+	cat >expected <<-EOF &&
    -+	local: $ROOT/path/ghi
    -+	local: $ROOT/path/def
    ++test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
    ++	mkdir my-hooks &&
    ++	write_script my-hooks/test-hook <<-EOF &&
    ++	echo Hook ran >>actual
     +	EOF
     +
    -+	git hook list pre-commit >actual &&
    -+	test_cmp expected actual
    ++	cat >expect <<-\EOF &&
    ++	Test hook
    ++	Hook ran
    ++	Hook ran
    ++	Hook ran
    ++	Hook ran
    ++	EOF
    ++
    ++	# Test various ways of specifying the path. See also
    ++	# t1350-config-hooks-path.sh
    ++	>actual &&
    ++	git hook run test-hook 2>>actual &&
    ++	git -c core.hooksPath=my-hooks hook run test-hook 2>>actual &&
    ++	git -c core.hooksPath=my-hooks/ hook run test-hook 2>>actual &&
    ++	git -c core.hooksPath="$PWD/my-hooks" hook run test-hook 2>>actual &&
    ++	git -c core.hooksPath="$PWD/my-hooks/" hook run test-hook 2>>actual &&
    ++	test_cmp expect actual
    ++'
    ++
    ++test_expect_success 'set up a pre-commit hook in core.hooksPath' '
    ++	>actual &&
    ++	mkdir -p .git/custom-hooks .git/hooks &&
    ++	write_script .git/custom-hooks/pre-commit <<-\EOF &&
    ++	echo CUSTOM >>actual
    ++	EOF
    ++	write_script .git/hooks/pre-commit <<-\EOF
    ++	echo NORMAL >>actual
    ++	EOF
     +'
     +
     +test_done
 3:  faa4a655183 <  -:  ----------- hook: include hookdir hook in list
 4:  c43a7e0dd52 <  -:  ----------- hook: teach hook.runHookDir
 5:  81e453baea2 <  -:  ----------- hook: implement hookcmd.<name>.skip
 6:  71f9ee9ab82 <  -:  ----------- parse-options: parse into strvec
 7:  e43a7a94163 <  -:  ----------- hook: add 'run' subcommand
 8:  807ad9cf2f3 <  -:  ----------- hook: introduce hook_exists()
35:  43d2383af49 !  4:  1a67a1cc065 run-command: stop thinking about hooks
    @@
      ## Metadata ##
    -Author: Emily Shaffer <emilyshaffer@google.com>
    +Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## Commit message ##
    -    run-command: stop thinking about hooks
    +    run-command.h: move find_hook() to hook.h
     
    -    hook.h has replaced all run-command.h hook-related functionality.
    -    run-command.h:run_hooks_le/ve and find_hook are no longer used anywhere
    -    in the codebase. So, let's delete the dead code - or, in the one case
    -    where it's still needed, move it to an internal function in hook.c.
    +    Move the find_hook() command to hook.h. Eventually all the hook
    +    related code will live there, let's move this function over as-is.
     
    -    Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    +
    + ## builtin/am.c ##
    +@@
    + #include "parse-options.h"
    + #include "dir.h"
    + #include "run-command.h"
    ++#include "hook.h"
    + #include "quote.h"
    + #include "tempfile.h"
    + #include "lockfile.h"
    +
    + ## builtin/bugreport.c ##
    +@@
    + #include "strbuf.h"
    + #include "help.h"
    + #include "compat/compiler.h"
    +-#include "run-command.h"
    ++#include "hook.h"
    + 
    + 
    + static void get_system_info(struct strbuf *sys_info)
    +
    + ## builtin/commit.c ##
    +@@
    + #include "revision.h"
    + #include "wt-status.h"
    + #include "run-command.h"
    ++#include "hook.h"
    + #include "refs.h"
    + #include "log-tree.h"
    + #include "strbuf.h"
    +
    + ## builtin/merge.c ##
    +@@
    + #include "builtin.h"
    + #include "lockfile.h"
    + #include "run-command.h"
    ++#include "hook.h"
    + #include "diff.h"
    + #include "diff-merges.h"
    + #include "refs.h"
    +
    + ## builtin/receive-pack.c ##
    +@@
    + #include "pkt-line.h"
    + #include "sideband.h"
    + #include "run-command.h"
    ++#include "hook.h"
    + #include "exec-cmd.h"
    + #include "commit.h"
    + #include "object.h"
    +
    + ## builtin/worktree.c ##
    +@@
    + #include "branch.h"
    + #include "refs.h"
    + #include "run-command.h"
    ++#include "hook.h"
    + #include "sigchain.h"
    + #include "submodule.h"
    + #include "utf8.h"
     
      ## hook.c ##
    -@@ hook.c: static int should_include_hookdir(const char *path, enum hookdir_opt cfg)
    - 	}
    - }
    +@@
    + #include "hook.h"
    + #include "run-command.h"
      
    -+static const char *find_legacy_hook(const char *name)
    ++const char *find_hook(const char *name)
     +{
     +	static struct strbuf path = STRBUF_INIT;
     +
    @@ hook.c: static int should_include_hookdir(const char *path, enum hookdir_opt cfg
     +}
     +
     +
    - struct list_head* hook_list(const char* hookname)
    ++
    + void run_hooks_opt_clear(struct run_hooks_opt *o)
      {
    - 	struct strbuf hook_key = STRBUF_INIT;
    -@@ hook.c: struct list_head* hook_list(const char* hookname)
    - 	git_config(hook_config_lookup, &cb_data);
    - 
    - 	if (have_git_dir()) {
    --		const char *legacy_hook_path = find_hook(hookname);
    -+		const char *legacy_hook_path = find_legacy_hook(hookname);
    - 
    - 		/* Unconditionally add legacy hook, but annotate it. */
    - 		if (legacy_hook_path) {
    -@@ hook.c: int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir)
    - 	could_run_hookdir = (should_run_hookdir == HOOKDIR_INTERACTIVE ||
    - 				should_run_hookdir == HOOKDIR_WARN ||
    - 				should_run_hookdir == HOOKDIR_YES)
    --				&& !!find_hook(hookname);
    -+				&& !!find_legacy_hook(hookname);
    + 	strvec_clear(&o->env);
    +
    + ## hook.h ##
    +@@ hook.h: struct hook_cb_data {
    + 	struct run_hooks_opt *options;
    + };
      
    - 	strbuf_addf(&hook_key, "hook.%s.command", hookname);
    ++/*
    ++ * Returns the path to the hook file, or NULL if the hook is missing
    ++ * or disabled. Note that this points to static storage that will be
    ++ * overwritten by further calls to find_hook and run_hook_*.
    ++ */
    ++const char *find_hook(const char *name);
    ++
    + void run_hooks_opt_clear(struct run_hooks_opt *o);
      
    + /*
    +
    + ## refs.c ##
    +@@
    + #include "refs.h"
    + #include "refs/refs-internal.h"
    + #include "run-command.h"
    ++#include "hook.h"
    + #include "object-store.h"
    + #include "object.h"
    + #include "tag.h"
     
      ## run-command.c ##
    +@@
    + #include "string-list.h"
    + #include "quote.h"
    + #include "config.h"
    ++#include "hook.h"
    + 
    + void child_process_init(struct child_process *child)
    + {
     @@ run-command.c: int async_with_fork(void)
      #endif
      }
    @@ run-command.c: int async_with_fork(void)
     -	return path.buf;
     -}
     -
    --int run_hook_ve(const char *const *env, const char *name, va_list args)
    --{
    --	struct child_process hook = CHILD_PROCESS_INIT;
    --	const char *p;
    --
    --	p = find_hook(name);
    --	if (!p)
    --		return 0;
    --
    --	strvec_push(&hook.args, p);
    --	while ((p = va_arg(args, const char *)))
    --		strvec_push(&hook.args, p);
    --	hook.env = env;
    --	hook.no_stdin = 1;
    --	hook.stdout_to_stderr = 1;
    --	hook.trace2_hook_name = name;
    --
    --	return run_command(&hook);
    --}
    --
    --int run_hook_le(const char *const *env, const char *name, ...)
    --{
    --	va_list args;
    --	int ret;
    --
    --	va_start(args, name);
    --	ret = run_hook_ve(env, name, args);
    --	va_end(args);
    --
    --	return ret;
    --}
    --
    - struct io_pump {
    - 	/* initialized by caller */
    - 	int fd;
    + int run_hook_ve(const char *const *env, const char *name, va_list args)
    + {
    + 	struct child_process hook = CHILD_PROCESS_INIT;
     
      ## run-command.h ##
     @@ run-command.h: int finish_command_in_signal(struct child_process *);
    @@ run-command.h: int finish_command_in_signal(struct child_process *);
     - */
     -const char *find_hook(const char *name);
     -
    --/**
    -- * Run a hook.
    -- * The first argument is a pathname to an index file, or NULL
    -- * if the hook uses the default index file or no index is needed.
    -- * The second argument is the name of the hook.
    -- * The further arguments correspond to the hook arguments.
    -- * The last argument has to be NULL to terminate the arguments list.
    -- * If the hook does not exist or is not executable, the return
    -- * value will be zero.
    -- * If it is executable, the hook will be executed and the exit
    -- * status of the hook is returned.
    -- * On execution, .stdout_to_stderr and .no_stdin will be set.
    -- */
    --LAST_ARG_MUST_BE_NULL
    --int run_hook_le(const char *const *env, const char *name, ...);
    --int run_hook_ve(const char *const *env, const char *name, va_list args);
    --
    - /*
    -  * Trigger an auto-gc
    -  */
    + /**
    +  * Run a hook.
    +  * The first argument is a pathname to an index file, or NULL
    +
    + ## sequencer.c ##
    +@@
    + #include "sequencer.h"
    + #include "tag.h"
    + #include "run-command.h"
    ++#include "hook.h"
    + #include "exec-cmd.h"
    + #include "utf8.h"
    + #include "cache-tree.h"
    +
    + ## transport.c ##
    +@@
    + #include "config.h"
    + #include "transport.h"
    + #include "run-command.h"
    ++#include "hook.h"
    + #include "pkt-line.h"
    + #include "fetch-pack.h"
    + #include "remote.h"
33:  33043b33b04 !  5:  a6f0817ad81 bugreport: use hook_exists instead of find_hook
    @@ Metadata
     Author: Emily Shaffer <emilyshaffer@google.com>
     
      ## Commit message ##
    -    bugreport: use hook_exists instead of find_hook
    +    hook.c: add a hook_exists() wrapper and use it in bugreport.c
     
    -    By using the helper in hook.h instead of the one in run-command.h, we
    -    can also check whether a hook exists in the config - not just whether it
    -    exists in the hookdir.
    +    Add a boolean version of the find_hook() function for those callers
    +    who are only interested in checking whether the hook exists, not what
    +    the path to it is.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## builtin/bugreport.c ##
    -@@
    - #include "strbuf.h"
    - #include "help.h"
    - #include "compat/compiler.h"
    --#include "run-command.h"
    -+#include "hook.h"
    - 
    - 
    - static void get_system_info(struct strbuf *sys_info)
     @@ builtin/bugreport.c: static void get_populated_hooks(struct strbuf *hook_info, int nongit)
      	}
      
      	for (i = 0; i < ARRAY_SIZE(hook); i++)
     -		if (find_hook(hook[i]))
    -+		if (hook_exists(hook[i], HOOKDIR_USE_CONFIG))
    ++		if (hook_exists(hook[i]))
      			strbuf_addf(hook_info, "%s\n", hook[i]);
      }
      
    +
    + ## hook.c ##
    +@@ hook.c: const char *find_hook(const char *name)
    + 	return path.buf;
    + }
    + 
    +-
    ++int hook_exists(const char *name)
    ++{
    ++	return !!find_hook(name);
    ++}
    + 
    + void run_hooks_opt_clear(struct run_hooks_opt *o)
    + {
    +
    + ## hook.h ##
    +@@ hook.h: struct hook_cb_data {
    +  */
    + const char *find_hook(const char *name);
    + 
    ++/*
    ++ * A boolean version of find_hook()
    ++ */
    ++int hook_exists(const char *hookname);
    ++
    + void run_hooks_opt_clear(struct run_hooks_opt *o);
    + 
    + /*
20:  2006e2709f6 !  6:  b186fde43e1 gc: use hook library for pre-auto-gc hook
    @@ Commit message
         well as in the hookdir. pre-auto-gc is called only from builtin/gc.c.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    -
    - ## Documentation/githooks.txt ##
    -@@ Documentation/githooks.txt: This hook is invoked by `git gc --auto` (see linkgit:git-gc[1]). It
    - takes no parameter, and exiting with non-zero status from this script
    - causes the `git gc --auto` to abort.
    - 
    -+Hooks run during 'pre-auto-gc' will be run in parallel, unless hook.jobs is
    -+configured to 1.
    -+
    - post-rewrite
    - ~~~~~~~~~~~~
    - 
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## builtin/gc.c ##
     @@
    @@ builtin/gc.c: static void add_repack_incremental_option(void)
      
      static int need_to_gc(void)
      {
    -+	struct run_hooks_opt hook_opt;
    ++	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
     +
      	/*
      	 * Setting gc.auto to 0 or negative can disable the
    @@ builtin/gc.c: static int need_to_gc(void)
      		return 0;
      
     -	if (run_hook_le(NULL, "pre-auto-gc", NULL))
    -+	run_hooks_opt_init_async(&hook_opt);
     +	if (run_hooks("pre-auto-gc", &hook_opt)) {
     +		run_hooks_opt_clear(&hook_opt);
      		return 0;
21:  62f6f9ab90d !  7:  528402fac69 rebase: teach pre-rebase to use hook.h
    @@ Metadata
      ## Commit message ##
         rebase: teach pre-rebase to use hook.h
     
    -    By using hook.h instead of run-command.h to run hooks, pre-rebase hooks
    -    can now be specified in the config as well as in the hookdir. pre-rebase
    -    is not called anywhere besides builtin/rebase.c.
    +    Move the pre-rebase hook away from run-command.h to and over to the
    +    new hook.h library.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    -
    - ## Documentation/githooks.txt ##
    -@@ Documentation/githooks.txt: two parameters.  The first parameter is the upstream from which
    - the series was forked.  The second parameter is the branch being
    - rebased, and is not set when rebasing the current branch.
    - 
    -+Hooks executed during 'pre-rebase' will run in parallel, unless hook.jobs is
    -+configured to 1.
    -+
    - post-checkout
    - ~~~~~~~~~~~~~
    - 
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## builtin/rebase.c ##
     @@
    @@ builtin/rebase.c: int cmd_rebase(int argc, const char **argv, const char *prefix
      	char *squash_onto_name = NULL;
      	int reschedule_failed_exec = -1;
      	int allow_preemptive_ff = 1;
    -+	struct run_hooks_opt hook_opt;
    ++	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
      	struct option builtin_rebase_options[] = {
      		OPT_STRING(0, "onto", &options.onto_name,
      			   N_("revision"),
    @@ builtin/rebase.c: int cmd_rebase(int argc, const char **argv, const char *prefix
      	}
      
      	/* If a hook exists, give it a chance to interrupt*/
    -+	run_hooks_opt_init_async(&hook_opt);
     +	strvec_pushl(&hook_opt.args, options.upstream_arg, argc ? argv[0] : NULL, NULL);
      	if (!ok_to_skip_pre_rebase &&
     -	    run_hook_le(NULL, "pre-rebase", options.upstream_arg,
18:  744e156ae91 !  8:  69842c74383 am: convert applypatch hooks to use config
    @@ Commit message
         am: convert applypatch hooks to use config
     
         Teach pre-applypatch, post-applypatch, and applypatch-msg to use the
    -    hook.h library instead of the run-command.h library. This enables use of
    -    hooks specified in the config, in addition to those in the hookdir.
    -    These three hooks are called only by builtin/am.c.
    +    hook.h library instead of the run-command.h library.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    -
    - ## Documentation/githooks.txt ##
    -@@ Documentation/githooks.txt: the message file.
    - The default 'applypatch-msg' hook, when enabled, runs the
    - 'commit-msg' hook, if the latter is enabled.
    - 
    -+Hooks run during 'applypatch-msg' will not be parallelized, because hooks are
    -+expected to edit the file holding the commit log message.
    -+
    - pre-applypatch
    - ~~~~~~~~~~~~~~
    - 
    -@@ Documentation/githooks.txt: make a commit if it does not pass certain test.
    - The default 'pre-applypatch' hook, when enabled, runs the
    - 'pre-commit' hook, if the latter is enabled.
    - 
    -+Hooks run during 'pre-applypatch' will be run in parallel, unless hook.jobs is
    -+configured to 1.
    -+
    - post-applypatch
    - ~~~~~~~~~~~~~~~
    - 
    -@@ Documentation/githooks.txt: and is invoked after the patch is applied and a commit is made.
    - This hook is meant primarily for notification, and cannot affect
    - the outcome of `git am`.
    - 
    -+Hooks run during 'post-applypatch' will be run in parallel, unless hook.jobs is
    -+configured to 1.
    -+
    - pre-commit
    - ~~~~~~~~~~
    - 
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## builtin/am.c ##
    -@@
    - #include "string-list.h"
    - #include "packfile.h"
    - #include "repository.h"
    -+#include "hook.h"
    - 
    - /**
    -  * Returns the length of the first line of msg.
     @@ builtin/am.c: static void am_destroy(const struct am_state *state)
      static int run_applypatch_msg_hook(struct am_state *state)
      {
      	int ret;
    -+	struct run_hooks_opt opt;
    -+
    -+	run_hooks_opt_init_sync(&opt);
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
      
      	assert(state->msg);
     -	ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL);
    @@ builtin/am.c: static void do_commit(const struct am_state *state)
      	struct commit_list *parents = NULL;
      	const char *reflog_msg, *author, *committer = NULL;
      	struct strbuf sb = STRBUF_INIT;
    -+	struct run_hooks_opt hook_opt;
    -+
    -+	run_hooks_opt_init_async(&hook_opt);
    ++	struct run_hooks_opt hook_opt_pre = RUN_HOOKS_OPT_INIT;
    ++	struct run_hooks_opt hook_opt_post = RUN_HOOKS_OPT_INIT;
      
     -	if (run_hook_le(NULL, "pre-applypatch", NULL))
    -+	if (run_hooks("pre-applypatch", &hook_opt)) {
    -+		run_hooks_opt_clear(&hook_opt);
    ++	if (run_hooks("pre-applypatch", &hook_opt_pre)) {
    ++		run_hooks_opt_clear(&hook_opt_pre);
      		exit(1);
     +	}
    -+
    -+	run_hooks_opt_clear(&hook_opt);
      
      	if (write_cache_as_tree(&tree, 0, NULL))
      		die(_("git write-tree failed to write a tree"));
    @@ builtin/am.c: static void do_commit(const struct am_state *state)
      	}
      
     -	run_hook_le(NULL, "post-applypatch", NULL);
    -+	run_hooks_opt_init_async(&hook_opt);
    -+	run_hooks("post-applypatch", &hook_opt);
    ++	run_hooks("post-applypatch", &hook_opt_post);
      
    -+	run_hooks_opt_clear(&hook_opt);
    ++	run_hooks_opt_clear(&hook_opt_pre);
    ++	run_hooks_opt_clear(&hook_opt_post);
      	strbuf_release(&sb);
      }
      
25:  a13a690bebb !  9:  9b32c14669b hooks: convert 'post-checkout' hook to hook library
    @@ Metadata
      ## Commit message ##
         hooks: convert 'post-checkout' hook to hook library
     
    -    By using the 'hook.h' library, 'post-checkout' hooks can now be
    -    specified in the config as well as in the hook directory.
    +    Move the running of the 'post-checkout' hook away from run-command.h
    +    to the new hook.h library. For "worktree" this requires a change to it
    +    to run the hooks from a given directory.
     
    -    Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    +    We could strictly speaking skip the "absolute_path" flag and just
    +    check if "dir" is specified, but let's split them up for clarity, as
    +    well as for any future user who'd like to set "dir" but not implicitly
    +    change the argument to an absolute path.
     
    - ## Documentation/githooks.txt ##
    -@@ Documentation/githooks.txt: This hook can be used to perform repository validity checks, auto-display
    - differences from the previous HEAD if different, or set working dir metadata
    - properties.
    - 
    -+Hooks executed during 'post-checkout' will not be parallelized.
    -+
    - post-merge
    - ~~~~~~~~~~
    - 
    +    Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## builtin/checkout.c ##
     @@
    @@ builtin/checkout.c: struct branch_info {
     -			   oid_to_hex(old_commit ? &old_commit->object.oid : null_oid()),
     -			   oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()),
     -			   changed ? "1" : "0", NULL);
    -+	struct run_hooks_opt opt;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
     +	int rc;
    -+
    -+	run_hooks_opt_init_sync(&opt);
     +
      	/* "new_commit" can be NULL when checking out from the index before
      	   a commit exists. */
    @@ builtin/clone.c: static int checkout(int submodule_progress)
      	struct tree *tree;
      	struct tree_desc t;
      	int err = 0;
    -+	struct run_hooks_opt hook_opt;
    ++	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
      
      	if (option_no_checkout)
      		return 0;
    @@ builtin/clone.c: static int checkout(int submodule_progress)
      
     -	err |= run_hook_le(NULL, "post-checkout", oid_to_hex(null_oid()),
     -			   oid_to_hex(&oid), "1", NULL);
    -+	run_hooks_opt_init_sync(&hook_opt);
     +	strvec_pushl(&hook_opt.args, oid_to_hex(null_oid()), oid_to_hex(&oid), "1", NULL);
     +	err |= run_hooks("post-checkout", &hook_opt);
     +	run_hooks_opt_clear(&hook_opt);
    @@ builtin/clone.c: static int checkout(int submodule_progress)
      		struct strvec args = STRVEC_INIT;
     
      ## builtin/worktree.c ##
    -@@
    - #include "utf8.h"
    - #include "worktree.h"
    - #include "quote.h"
    -+#include "hook.h"
    - 
    - static const char * const worktree_usage[] = {
    - 	N_("git worktree add [<options>] <path> [<commit-ish>]"),
     @@ builtin/worktree.c: static int add_worktree(const char *path, const char *refname,
      	 * is_junk is cleared, but do return appropriate code when hook fails.
      	 */
    @@ builtin/worktree.c: static int add_worktree(const char *path, const char *refnam
     -				     "1", NULL);
     -			ret = run_command(&cp);
     -		}
    -+		struct run_hooks_opt opt;
    -+
    -+		run_hooks_opt_init_sync(&opt);
    ++		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
     +
     +		strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
     +		strvec_pushl(&opt.args,
    @@ builtin/worktree.c: static int add_worktree(const char *path, const char *refnam
     +			     "1",
     +			     NULL);
     +		opt.dir = path;
    ++		opt.absolute_path = 1;
     +
     +		ret = run_hooks("post-checkout", &opt);
     +
    @@ builtin/worktree.c: static int add_worktree(const char *path, const char *refnam
      
      	strvec_clear(&child_env);
     
    + ## hook.c ##
    +@@ hook.c: static int pick_next_hook(struct child_process *cp,
    + 	cp->env = hook_cb->options->env.v;
    + 	cp->stdout_to_stderr = 1;
    + 	cp->trace2_hook_name = hook_cb->hook_name;
    ++	cp->dir = hook_cb->options->dir;
    + 
    + 	/* add command */
    + 	strvec_push(&cp->args, run_me->hook_path);
    +@@ hook.c: static int notify_hook_finished(int result,
    + int run_found_hooks(const char *hook_name, const char *hook_path,
    + 		    struct run_hooks_opt *options)
    + {
    ++	struct strbuf abs_path = STRBUF_INIT;
    + 	struct hook my_hook = {
    + 		.hook_path = hook_path,
    + 	};
    +@@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
    + 		.hook_name = hook_name,
    + 		.options = options,
    + 	};
    ++	if (options->absolute_path) {
    ++		strbuf_add_absolute_path(&abs_path, hook_path);
    ++		my_hook.hook_path = abs_path.buf;
    ++	}
    + 	cb_data.run_me = &my_hook;
    + 
    + 	if (options->jobs != 1)
    +@@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
    + 				   &cb_data,
    + 				   "hook",
    + 				   hook_name);
    ++	if (options->absolute_path)
    ++		strbuf_release(&abs_path);
    + 
    + 	return cb_data.rc;
    + }
    +
    + ## hook.h ##
    +@@ hook.h: struct run_hooks_opt
    + 
    + 	/* Number of threads to parallelize across */
    + 	int jobs;
    ++
    ++	/* Resolve and run the "absolute_path(hook)" instead of
    ++	 * "hook". Used for "git worktree" hooks
    ++	 */
    ++	int absolute_path;
    ++
    ++	/* Path to initial working directory for subprocess */
    ++	const char *dir;
    ++
    + };
    + 
    + #define RUN_HOOKS_OPT_INIT { \
    +
      ## read-cache.c ##
     @@
    + #include "thread-utils.h"
      #include "progress.h"
      #include "sparse-index.h"
    - #include "hook.h"
    -->>>>>>> 9524a9d29d (read-cache: convert post-index-change hook to use config)
    ++#include "hook.h"
      
      /* Mask for the name length in ce_flags in the on-disk index */
      
    @@ reset.c: int reset_head(struct repository *r, struct object_id *oid, const char
     -			    oid_to_hex(orig ? orig : null_oid()),
     -			    oid_to_hex(oid), "1", NULL);
     +	if (run_hook) {
    -+		struct run_hooks_opt opt;
    -+
    -+		run_hooks_opt_init_sync(&opt);
    ++		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
     +		strvec_pushl(&opt.args,
     +			     oid_to_hex(orig ? orig : null_oid()),
     +			     oid_to_hex(oid),
19:  be1d7c7636b ! 10:  201c654bb0c merge: use config-based hooks for post-merge hook
    @@ Commit message
         builtin/merge.c.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    -
    - ## Documentation/githooks.txt ##
    -@@ Documentation/githooks.txt: save and restore any form of metadata associated with the working tree
    - (e.g.: permissions/ownership, ACLS, etc).  See contrib/hooks/setgitperms.perl
    - for an example of how to do this.
    - 
    -+Hooks executed during 'post-merge' will run in parallel, unless hook.jobs is
    -+configured to 1.
    -+
    - pre-push
    - ~~~~~~~~
    - 
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## builtin/merge.c ##
     @@ builtin/merge.c: static void finish(struct commit *head_commit,
      		   const struct object_id *new_head, const char *msg)
      {
      	struct strbuf reflog_message = STRBUF_INIT;
    -+	struct run_hooks_opt opt;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
      	const struct object_id *head = &head_commit->object.oid;
      
      	if (!msg)
    @@ builtin/merge.c: static void finish(struct commit *head_commit,
      
      	/* Run a post-merge hook */
     -	run_hook_le(NULL, "post-merge", squash ? "1" : "0", NULL);
    -+	run_hooks_opt_init_async(&opt);
     +	strvec_push(&opt.args, squash ? "1" : "0");
     +	run_hooks("post-merge", &opt);
     +	run_hooks_opt_clear(&opt);
      
      	apply_autostash(git_path_merge_autostash(the_repository));
      	strbuf_release(&reflog_message);
    +@@ builtin/merge.c: static void prepare_to_commit(struct commit_list *remoteheads)
    + 	 * and write it out as a tree.  We must do this before we invoke
    + 	 * the editor and after we invoke run_status above.
    + 	 */
    +-	if (find_hook("pre-merge-commit"))
    ++	if (hook_exists("pre-merge-commit"))
    + 		discard_cache();
    + 	read_cache_from(index_file);
    + 	strbuf_addbuf(&msg, &merge_msg);
 -:  ----------- > 11:  e65d8bd6e6f git hook run: add an --ignore-missing flag
34:  832136eb930 ! 12:  8795e9ceab8 git-send-email: use 'git hook run' for 'sendemail-validate'
    @@ Metadata
     Author: Emily Shaffer <emilyshaffer@google.com>
     
      ## Commit message ##
    -    git-send-email: use 'git hook run' for 'sendemail-validate'
    +    send-email: use 'git hook run' for 'sendemail-validate'
     
    -    By using the new 'git hook run' subcommand to run 'sendemail-validate',
    -    we can reduce the boilerplate needed to run this hook in perl. Using
    -    config-based hooks also allows us to run 'sendemail-validate' hooks that
    -    were configured globally when running 'git send-email' from outside of a
    -    Git directory, alongside other benefits like multihooks and
    -    parallelization.
    +    Change the "sendmail-validate" hook to be run via the "git hook run"
    +    wrapper instead of via a direct invocation.
    +
    +    This is the smallest possibly change to get "send-email" using "git
    +    hook run". We still check the hook itself with "-x", and set a
    +    "GIT_DIR" variable, both of which are asserted by our tests. We'll
    +    need to get rid of this special behavior if we start running N hooks,
    +    but for now let's be as close to bug-for-bug compatible as possible.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## git-send-email.perl ##
    -@@ git-send-email.perl: sub unique_email_list {
    - sub validate_patch {
    +@@ git-send-email.perl: sub format_2822_time {
    + my $editor;
    + 
    + sub system_or_msg {
    +-	my ($args, $msg) = @_;
    ++	my ($args, $msg, $cmd_name) = @_;
    + 	system(@$args);
    + 	my $signalled = $? & 127;
    + 	my $exit_code = $? >> 8;
    + 	return unless $signalled or $exit_code;
    + 
    +-	my @sprintf_args = ($args->[0], $exit_code);
    ++	my @sprintf_args = ($cmd_name ? $cmd_name : $args->[0], $exit_code);
    + 	if (defined $msg) {
    + 		# Quiet the 'redundant' warning category, except we
    + 		# need to support down to Perl 5.8, so we can't do a
    +@@ git-send-email.perl: sub validate_patch {
      	my ($fn, $xfer_encoding) = @_;
      
    --	if ($repo) {
    --		my $validate_hook = catfile($repo->hooks_path(),
    + 	if ($repo) {
    ++		my $hook_name = 'sendemail-validate';
    + 		my $hooks_path = $repo->command_oneline('rev-parse', '--git-path', 'hooks');
    +-		my $validate_hook = catfile($hooks_path,
     -					    'sendemail-validate');
    --		my $hook_error;
    --		if (-x $validate_hook) {
    --			my $target = abs_path($fn);
    --			# The hook needs a correct cwd and GIT_DIR.
    --			my $cwd_save = cwd();
    --			chdir($repo->wc_path() or $repo->repo_path())
    --				or die("chdir: $!");
    --			local $ENV{"GIT_DIR"} = $repo->repo_path();
    ++		my $validate_hook = catfile($hooks_path, $hook_name);
    + 		my $hook_error;
    + 		if (-x $validate_hook) {
    + 			my $target = abs_path($fn);
    +@@ git-send-email.perl: sub validate_patch {
    + 			chdir($repo->wc_path() or $repo->repo_path())
    + 				or die("chdir: $!");
    + 			local $ENV{"GIT_DIR"} = $repo->repo_path();
     -			$hook_error = system_or_msg([$validate_hook, $target]);
    --			chdir($cwd_save) or die("chdir: $!");
    --		}
    --		if ($hook_error) {
    ++			my @validate_hook = ("git", "hook", "run", "--ignore-missing", $hook_name, "--", $target);
    ++			$hook_error = system_or_msg(\@validate_hook, undef,
    ++						       "git hook run $hook_name -- <patch>");
    + 			chdir($cwd_save) or die("chdir: $!");
    + 		}
    + 		if ($hook_error) {
     -			die sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
     -				       "%s\n" .
     -				       "warning: no patches were sent\n"), $fn, $hook_error);
    --		}
    --	}
    -+	my $target = abs_path($fn);
    -+
    -+	system_or_die(["git", "hook", "run", "sendemail-validate", "-j1", "-a", $target],
    -+		sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
    -+			   "warning: no patches were sent\n"),
    -+		        $fn));
    ++			$hook_error = sprintf(__("fatal: %s: rejected by %s hook\n" .
    ++						 $hook_error . "\n" .
    ++						 "warning: no patches were sent\n"),
    ++					      $fn, $hook_name);
    ++			die $hook_error;
    + 		}
    + 	}
      
    - 	# Any long lines will be automatically fixed if we use a suitable transfer
    - 	# encoding.
     
      ## t/t9001-send-email.sh ##
     @@ t/t9001-send-email.sh: test_expect_success $PREREQ "--validate respects relative core.hooksPath path" '
      	test_path_is_file my-hooks.ran &&
      	cat >expect <<-EOF &&
      	fatal: longline.patch: rejected by sendemail-validate hook
    --	fatal: command '"'"'$PWD/my-hooks/sendemail-validate'"'"' died with exit code 1
    +-	fatal: command '"'"'my-hooks/sendemail-validate'"'"' died with exit code 1
    ++	fatal: command '"'"'git hook run sendemail-validate -- <patch>'"'"' died with exit code 1
      	warning: no patches were sent
      	EOF
      	test_cmp expect actual
    @@ t/t9001-send-email.sh: test_expect_success $PREREQ "--validate respects absolute
      	test_path_is_file my-hooks.ran &&
      	cat >expect <<-EOF &&
      	fatal: longline.patch: rejected by sendemail-validate hook
    --	fatal: command '"'"'$PWD/my-hooks/sendemail-validate'"'"' died with exit code 1
    +-	fatal: command '"'"'$hooks_path/sendemail-validate'"'"' died with exit code 1
    ++	fatal: command '"'"'git hook run sendemail-validate -- <patch>'"'"' died with exit code 1
      	warning: no patches were sent
      	EOF
      	test_cmp expect actual
    -@@ t/t9001-send-email.sh: test_expect_success $PREREQ 'invoke hook' '
    - 	mkdir -p .git/hooks &&
    - 
    - 	write_script .git/hooks/sendemail-validate <<-\EOF &&
    --	# test that we have the correct environment variable, pwd, and
    --	# argument
    --	case "$GIT_DIR" in
    --	*.git)
    --		true
    --		;;
    --	*)
    --		false
    --		;;
    --	esac &&
    -+	# test that we have the correct argument
    - 	test -f 0001-add-main.patch &&
    - 	grep "add main" "$1"
    - 	EOF
24:  f4c92f0dbc1 ! 13:  03129460fd2 git-p4: use 'git hook' to run hooks
    @@ Commit message
         git-p4: use 'git hook' to run hooks
     
         Instead of duplicating the behavior of run-command.h:run_hook_le() in
    -    Python, we can directly call 'git hook run'. As a bonus, this means
    -    git-p4 learns how to find hook specifications from the Git config as
    -    well as from the hookdir.
    +    Python, we can directly call 'git hook run'. We emulate the existence
    +    check with the --ignore-missing flag.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## git-p4.py ##
     @@ git-p4.py: def decode_path(path):
    +         return path
      
      def run_git_hook(cmd, param=[]):
    -     """Execute a hook if the hook exists."""
    +-    """Execute a hook if the hook exists."""
     -    if verbose:
     -        sys.stderr.write("Looking for hook: %s\n" % cmd)
     -        sys.stderr.flush()
    @@ git-p4.py: def decode_path(path):
     -                    return True
     -
     -    if not os.path.isfile(hook_file) or not os.access(hook_file, os.X_OK):
    -+    if not cmd:
    -         return True
    - 
    +-        return True
    +-
     -    return run_hook_command(hook_file, param) == 0
     -
     -def run_hook_command(cmd, param):
    @@ git-p4.py: def decode_path(path):
     -        else:
     -            use_shell = True
     -    return subprocess.call(cli, shell=use_shell)
    +-
     +    """args are specified with -a <arg> -a <arg> -a <arg>"""
    -+    args = (['git', 'hook', 'run'] +
    -+            ["-a" + arg for arg in param] +
    -+            [cmd])
    - 
    ++    args = ['git', 'hook', 'run', '--ignore-missing', cmd]
    ++    if param:
    ++        args.append("--")
    ++        for p in param:
    ++            args.append(p)
     +    return subprocess.call(args) == 0
      
      def write_pipe(c, stdin):
 -:  ----------- > 14:  3f3610f5ed3 commit: use hook.h to execute hooks
22:  0e405276551 ! 15:  6482a3e4cb8 read-cache: convert post-index-change hook to use config
    @@ Commit message
         can now be specified in the config in addition to the hookdir.
         post-index-change is not run anywhere besides in read-cache.c.
     
    -    Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    +    This removes the last direct user of run_hook_ve(), so we can make the
    +    function static now. It'll be removed entirely soon.
     
    - ## Documentation/githooks.txt ##
    -@@ Documentation/githooks.txt: and "0" meaning they were not.
    - Only one parameter should be set to "1" when the hook runs.  The hook
    - running passing "1", "1" should not be possible.
    - 
    -+Hooks run during 'post-index-change' will be run in parallel, unless hook.jobs
    -+is configured to 1.
    -+
    - GIT
    - ---
    - Part of the linkgit:git[1] suite
    +    Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## read-cache.c ##
    -@@
    - #include "thread-utils.h"
    - #include "progress.h"
    - #include "sparse-index.h"
    -+#include "hook.h"
    -+>>>>>>> 9524a9d29d (read-cache: convert post-index-change hook to use config)
    - 
    - /* Mask for the name length in ce_flags in the on-disk index */
    - 
     @@ read-cache.c: static int do_write_locked_index(struct index_state *istate, struct lock_file *l
      {
      	int ret;
      	int was_full = !istate->sparse_index;
    -+	struct run_hooks_opt hook_opt;
    ++	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
      
      	ret = convert_to_sparse(istate);
      
    @@ read-cache.c: static int do_write_locked_index(struct index_state *istate, struc
     -	run_hook_le(NULL, "post-index-change",
     -			istate->updated_workdir ? "1" : "0",
     -			istate->updated_skipworktree ? "1" : "0", NULL);
    -+	run_hooks_opt_init_async(&hook_opt);
     +	strvec_pushl(&hook_opt.args,
     +		     istate->updated_workdir ? "1" : "0",
     +		     istate->updated_skipworktree ? "1" : "0",
    @@ read-cache.c: static int do_write_locked_index(struct index_state *istate, struc
      	istate->updated_workdir = 0;
      	istate->updated_skipworktree = 0;
      
    +
    + ## run-command.c ##
    +@@ run-command.c: int async_with_fork(void)
    + #endif
    + }
    + 
    +-int run_hook_ve(const char *const *env, const char *name, va_list args)
    ++static int run_hook_ve(const char *const *env, const char *name, va_list args)
    + {
    + 	struct child_process hook = CHILD_PROCESS_INIT;
    + 	const char *p;
    +
    + ## run-command.h ##
    +@@ run-command.h: int run_command(struct child_process *);
    +  */
    + LAST_ARG_MUST_BE_NULL
    + int run_hook_le(const char *const *env, const char *name, ...);
    +-int run_hook_ve(const char *const *env, const char *name, va_list args);
    + 
    + /*
    +  * Trigger an auto-gc
23:  a793508373e ! 16:  a16163d4fb5 receive-pack: convert push-to-checkout hook to hook.h
    @@ Commit message
         hooks can now be specified in the config as well as in the hookdir.
         push-to-checkout is not called anywhere but in builtin/receive-pack.c.
     
    -    Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    +    This is the last user of the run_hook_le() API, so let's remove it
    +    while we're at it, since run_hook_le() itself is the last user of
    +    run_hook_ve() we can remove that too. The last direct user of
    +    run_hook_le() was removed in the commit preceding this one.
     
    - ## Documentation/githooks.txt ##
    -@@ Documentation/githooks.txt: that switches branches while
    - keeping the local changes in the working tree that do not interfere
    - with the difference between the branches.
    - 
    -+Hooks executed during 'push-to-checkout' will not be parallelized.
    - 
    - pre-auto-gc
    - ~~~~~~~~~~~
    +    Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## builtin/receive-pack.c ##
    -@@
    - #include "commit-reach.h"
    - #include "worktree.h"
    - #include "shallow.h"
    -+#include "hook.h"
    - 
    - static const char * const receive_pack_usage[] = {
    - 	N_("git receive-pack <git-dir>"),
     @@ builtin/receive-pack.c: static const char *push_to_checkout(unsigned char *hash,
      				    struct strvec *env,
      				    const char *work_tree)
      {
    -+	struct run_hooks_opt opt;
    -+
    -+	run_hooks_opt_init_sync(&opt);
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
     +
      	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
     -	if (run_hook_le(env->v, push_to_checkout_hook,
    @@ builtin/receive-pack.c: static const char *update_worktree(unsigned char *sha1,
      	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
      
     -	if (!find_hook(push_to_checkout_hook))
    -+	if (!hook_exists(push_to_checkout_hook, HOOKDIR_USE_CONFIG))
    ++	if (!hook_exists(push_to_checkout_hook))
      		retval = push_to_deploy(sha1, &env, work_tree);
      	else
      		retval = push_to_checkout(sha1, &env, work_tree);
    +
    + ## run-command.c ##
    +@@ run-command.c: int async_with_fork(void)
    + #endif
    + }
    + 
    +-static int run_hook_ve(const char *const *env, const char *name, va_list args)
    +-{
    +-	struct child_process hook = CHILD_PROCESS_INIT;
    +-	const char *p;
    +-
    +-	p = find_hook(name);
    +-	if (!p)
    +-		return 0;
    +-
    +-	strvec_push(&hook.args, p);
    +-	while ((p = va_arg(args, const char *)))
    +-		strvec_push(&hook.args, p);
    +-	hook.env = env;
    +-	hook.no_stdin = 1;
    +-	hook.stdout_to_stderr = 1;
    +-	hook.trace2_hook_name = name;
    +-
    +-	return run_command(&hook);
    +-}
    +-
    +-int run_hook_le(const char *const *env, const char *name, ...)
    +-{
    +-	va_list args;
    +-	int ret;
    +-
    +-	va_start(args, name);
    +-	ret = run_hook_ve(env, name, args);
    +-	va_end(args);
    +-
    +-	return ret;
    +-}
    +-
    + struct io_pump {
    + 	/* initialized by caller */
    + 	int fd;
    +
    + ## run-command.h ##
    +@@ run-command.h: int finish_command_in_signal(struct child_process *);
    +  */
    + int run_command(struct child_process *);
    + 
    +-/**
    +- * Run a hook.
    +- * The first argument is a pathname to an index file, or NULL
    +- * if the hook uses the default index file or no index is needed.
    +- * The second argument is the name of the hook.
    +- * The further arguments correspond to the hook arguments.
    +- * The last argument has to be NULL to terminate the arguments list.
    +- * If the hook does not exist or is not executable, the return
    +- * value will be zero.
    +- * If it is executable, the hook will be executed and the exit
    +- * status of the hook is returned.
    +- * On execution, .stdout_to_stderr and .no_stdin will be set.
    +- */
    +-LAST_ARG_MUST_BE_NULL
    +-int run_hook_le(const char *const *env, const char *name, ...);
    +-
    + /*
    +  * Trigger an auto-gc
    +  */
10:  744437273de ! 17:  7020cf10c8e run-command: allow stdin for run_processes_parallel
    @@ Commit message
         on child output in stderr.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## run-command.c ##
     @@ run-command.c: static int pp_start_one(struct parallel_processes *pp)
 9:  3e07045894f ! 18:  4745dcfce49 hook: support passing stdin to hooks
    @@ Commit message
         interim file themselves.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## Documentation/git-hook.txt ##
    -@@ Documentation/git-hook.txt: SYNOPSIS
    +@@ Documentation/git-hook.txt: git-hook - run git hooks
    + SYNOPSIS
      --------
      [verse]
    - 'git hook' list <hook-name>
    --'git hook' run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] <hook-name>
    -+'git hook' run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] [--to-stdin=<path>]
    -+	<hook-name>
    +-'git hook' run [--ignore-missing] <hook-name> [-- <hook-args>]
    ++'git hook' run [--to-stdin=<path>] [--ignore-missing] <hook-name> [-- <hook-args>]
      
      DESCRIPTION
      -----------
    -@@ Documentation/git-hook.txt: in the order they should be run, and print the config scope where the relevant
    - `hook.<hook-name>.command` was specified, not the `hookcmd` (if applicable).
    - This output is human-readable and the format is subject to change over time.
    - 
    --run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] `<hook-name>`::
    -+run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] [--to-stdin=<path>] `<hook-name>`::
    - 
    - Runs hooks configured for `<hook-name>`, in the same order displayed by `git
    - hook list`. Hooks configured this way may be run prepended with `sh -c`, so
    -@@ Documentation/git-hook.txt: Specify arguments to pass to every hook that is run.
    - +
    - Specify environment variables to set for every hook that is run.
    +@@ Documentation/git-hook.txt: run::
    + OPTIONS
    + -------
      
     +--to-stdin::
    -+	Only valid for `run`.
    -++
    -+Specify a file which will be streamed into stdin for every hook that is run.
    -+Each hook will receive the entire file from beginning to EOF.
    ++	For "run"; Specify a file which will be streamed into the
    ++	hook's stdin. The hook will receive the entire file from
    ++	beginning to EOF.
     +
    - CONFIGURATION
    - -------------
    - include::config/hook.txt[]
    + --ignore-missing::
    + 	Ignore any missing hook by quietly returning zero. Used for
    + 	tools that want to do a blind one-shot run of a hook that may
     
      ## builtin/hook.c ##
     @@
    + #include "strvec.h"
      
      static const char * const builtin_hook_usage[] = {
    - 	N_("git hook list <hookname>"),
    --	N_("git hook run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] <hookname>"),
    -+	N_("git hook run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...]"
    -+	   "[--to-stdin=<path>] <hookname>"),
    +-	N_("git hook run <hook-name> [-- <hook-args>]"),
    ++	N_("git hook run [--to-stdin=<path>] <hook-name> [-- <hook-args>]"),
      	NULL
      };
      
     @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
    - 			   N_("environment variables for hook to use")),
    - 		OPT_STRVEC('a', "arg", &opt.args, N_("args"),
    - 			   N_("argument to pass to hook")),
    + 	struct option run_options[] = {
    + 		OPT_BOOL(0, "ignore-missing", &ignore_missing,
    + 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
     +		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
     +			   N_("file to read into hooks' stdin")),
      		OPT_END(),
    @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
      
     
      ## hook.c ##
    -@@ hook.c: void run_hooks_opt_init(struct run_hooks_opt *o)
    - {
    - 	strvec_init(&o->env);
    - 	strvec_init(&o->args);
    -+	o->path_to_stdin = NULL;
    - 	o->run_hookdir = configured_hookdir_opt();
    - }
    - 
    -@@ hook.c: static void prepare_hook_cp(const char *hookname, struct hook *hook,
    - 	if (!hook)
    - 		return;
    +@@ hook.c: static int pick_next_hook(struct child_process *cp,
    + 	if (!run_me)
    + 		BUG("did we not return 1 in notify_hook_finished?");
      
     -	cp->no_stdin = 1;
     +	/* reopen the file for stdin; run_command closes it. */
    -+	if (options->path_to_stdin)
    -+		cp->in = xopen(options->path_to_stdin, O_RDONLY);
    -+	else
    ++	if (hook_cb->options->path_to_stdin) {
    ++		cp->no_stdin = 0;
    ++		cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY);
    ++	} else {
     +		cp->no_stdin = 1;
    -+
    - 	cp->env = options->env.v;
    ++	}
    + 	cp->env = hook_cb->options->env.v;
      	cp->stdout_to_stderr = 1;
    - 	cp->trace2_hook_name = hookname;
    + 	cp->trace2_hook_name = hook_cb->hook_name;
     
      ## hook.h ##
     @@ hook.h: struct run_hooks_opt
    - 	 * to be overridden if the user can override it at the command line.
    - 	 */
    - 	enum hookdir_opt run_hookdir;
    -+
    + 	/* Path to initial working directory for subprocess */
    + 	const char *dir;
    + 
     +	/* Path to file which should be piped to stdin for each hook */
     +	const char *path_to_stdin;
      };
      
    - void run_hooks_opt_init(struct run_hooks_opt *o);
    -@@ hook.h: int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir);
    - 
    - /*
    -  * Runs all hooks associated to the 'hookname' event in order. Each hook will be
    -- * passed 'env' and 'args'.
    -+ * passed 'env' and 'args'. The file at 'stdin_path' will be closed and reopened
    -+ * for each hook that runs.
    -  */
    - int run_hooks(const char *hookname, struct run_hooks_opt *options);
    - 
    + #define RUN_HOOKS_OPT_INIT { \
     
    - ## t/t1360-config-based-hooks.sh ##
    -@@ t/t1360-config-based-hooks.sh: test_expect_success 'hook.runHookDir is tolerant to unknown values' '
    - 	test_cmp expected actual
    + ## t/t1800-hook.sh ##
    +@@ t/t1800-hook.sh: test_expect_success 'set up a pre-commit hook in core.hooksPath' '
    + 	EOF
      '
      
    -+test_expect_success 'stdin to multiple hooks' '
    -+	git config --add hook.test.command "xargs -P1 -I% echo a%" &&
    -+	git config --add hook.test.command "xargs -P1 -I% echo b%" &&
    -+	test_when_finished "test_unconfig hook.test.command" &&
    -+
    -+	cat >input <<-EOF &&
    -+	1
    -+	2
    -+	3
    ++test_expect_success 'stdin to hooks' '
    ++	write_script .git/hooks/test-hook <<-\EOF &&
    ++	echo BEGIN stdin
    ++	cat
    ++	echo END stdin
     +	EOF
     +
    -+	cat >expected <<-EOF &&
    -+	a1
    -+	a2
    -+	a3
    -+	b1
    -+	b2
    -+	b3
    ++	cat >expect <<-EOF &&
    ++	BEGIN stdin
    ++	hello
    ++	END stdin
     +	EOF
     +
    -+	git hook run --to-stdin=input test 2>actual &&
    -+	test_cmp expected actual
    ++	echo hello >input &&
    ++	git hook run --to-stdin=input test-hook 2>actual &&
    ++	test_cmp expect actual
     +'
     +
      test_done
11:  e6c789629d6 <  -:  ----------- hook: allow parallel hook execution
12:  26e2d14bc1a <  -:  ----------- hook: allow specifying working directory for hooks
 -:  ----------- > 19:  986bfd89a54 am: convert 'post-rewrite' hook to hook.h
13:  e721d45efc6 ! 20:  756f52af22d run-command: add stdin callback for parallelization
    @@ Commit message
         match the rest of the API reduces mental load on the user.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## builtin/fetch.c ##
     @@ builtin/fetch.c: static int fetch_multiple(struct string_list *list, int max_children)
    @@ builtin/submodule--helper.c: static int update_submodules(struct submodule_updat
      
     
      ## hook.c ##
    -@@ hook.c: int run_hooks(const char *hookname, struct run_hooks_opt *options)
    +@@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
      	run_processes_parallel_tr2(options->jobs,
      				   pick_next_hook,
      				   notify_start_failure,
14:  5d74ea3ed3f ! 21:  3748f128763 hook: provide stdin by string_list or callback
    @@ Commit message
         with instead.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## hook.c ##
    -@@
    - 
    - void free_hook(struct hook *ptr)
    - {
    --	if (ptr)
    -+	if (ptr) {
    - 		strbuf_release(&ptr->command);
    -+		free(ptr->feed_pipe_cb_data);
    -+	}
    - 	free(ptr);
    - }
    - 
    -@@ hook.c: static void append_or_move_hook(struct list_head *head, const char *command)
    - 		strbuf_init(&to_add->command, 0);
    - 		strbuf_addstr(&to_add->command, command);
    - 		to_add->from_hookdir = 0;
    -+		to_add->feed_pipe_cb_data = NULL;
    - 	}
    - 
    - 	/* re-set the scope so we show where an override was specified */
    -@@ hook.c: void run_hooks_opt_init_sync(struct run_hooks_opt *o)
    - 	o->run_hookdir = configured_hookdir_opt();
    - 	o->jobs = 1;
    - 	o->dir = NULL;
    -+	o->feed_pipe = NULL;
    -+	o->feed_pipe_ctx = NULL;
    - }
    - 
    - void run_hooks_opt_init_async(struct run_hooks_opt *o)
     @@ hook.c: void run_hooks_opt_clear(struct run_hooks_opt *o)
      	strvec_clear(&o->args);
      }
    @@ hook.c: void run_hooks_opt_clear(struct run_hooks_opt *o)
     +{
     +	int *item_idx;
     +	struct hook *ctx = pp_task_cb;
    -+	struct string_list *to_pipe = ((struct hook_cb_data*)pp_cb)->options->feed_pipe_ctx;
    ++	struct hook_cb_data *hook_cb = pp_cb;
    ++	struct string_list *to_pipe = hook_cb->options->feed_pipe_ctx;
     +
     +	/* Bootstrap the state manager if necessary. */
     +	if (!ctx->feed_pipe_cb_data) {
    @@ hook.c: static int pick_next_hook(struct child_process *cp,
      	} else {
      		cp->no_stdin = 1;
      	}
    -@@ hook.c: int run_hooks(const char *hookname, struct run_hooks_opt *options)
    - 	if (!options)
    - 		BUG("a struct run_hooks_opt must be provided to run_hooks");
    - 
    -+	if (options->path_to_stdin && options->feed_pipe)
    -+		BUG("choose only one method to populate stdin");
    -+
    - 	to_run = hook_list(hookname);
    +@@ hook.c: static int notify_hook_finished(int result,
    + 	return 1;
    + }
      
    - 	list_for_each_safe(pos, tmp, to_run) {
    -@@ hook.c: int run_hooks(const char *hookname, struct run_hooks_opt *options)
    +-
    + int run_found_hooks(const char *hook_name, const char *hook_path,
    + 		    struct run_hooks_opt *options)
    + {
    +@@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
      	run_processes_parallel_tr2(options->jobs,
      				   pick_next_hook,
      				   notify_start_failure,
    @@ hook.c: int run_hooks(const char *hookname, struct run_hooks_opt *options)
      				   notify_hook_finished,
      				   &cb_data,
      				   "hook",
    +@@ hook.c: int run_hooks(const char *hook_name, struct run_hooks_opt *options)
    + 	if (!options)
    + 		BUG("a struct run_hooks_opt must be provided to run_hooks");
    + 
    ++	if (options->path_to_stdin && options->feed_pipe)
    ++		BUG("choose only one method to populate stdin");
    ++
    + 	hook_path = find_hook(hook_name);
    + 
    + 	/* Care about nonexistence? Use run_found_hooks() */
     
      ## hook.h ##
     @@
    - #include "list.h"
    - #include "strbuf.h"
    - #include "strvec.h"
    -+#include "run-command.h"
    - 
      struct hook {
    - 	struct list_head list;
    -@@ hook.h: struct hook {
    - 	/* The literal command to run. */
    - 	struct strbuf command;
    - 	unsigned from_hookdir : 1;
    + 	/* The path to the hook */
    + 	const char *hook_path;
     +
     +	/*
     +	 * Use this to keep state for your feed_pipe_fn if you are using
    @@ hook.h: struct hook {
     +	void *feed_pipe_cb_data;
      };
      
    - /*
    + struct run_hooks_opt
     @@ hook.h: struct run_hooks_opt
      
      	/* Path to file which should be piped to stdin for each hook */
      	const char *path_to_stdin;
    ++
     +	/*
     +	 * Callback and state pointer to ask for more content to pipe to stdin.
     +	 * Will be called repeatedly, for each hook. See
    @@ hook.h: struct run_hooks_opt
     +	 */
     +	feed_pipe_fn feed_pipe;
     +	void *feed_pipe_ctx;
    - 
    - 	/* Number of threads to parallelize across */
    - 	int jobs;
    - 
    - 	/* Path to initial working directory for subprocess */
    - 	const char *dir;
    -+
      };
      
    + #define RUN_HOOKS_OPT_INIT { \
    +@@ hook.h: struct run_hooks_opt
    + 	.args = STRVEC_INIT, \
    + }
    + 
     +/*
     + * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
     + * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'.
26:  6d804ca20fd ! 22:  0cf0b1fea93 hook: convert 'post-rewrite' hook to config
    @@ Metadata
     Author: Emily Shaffer <emilyshaffer@google.com>
     
      ## Commit message ##
    -    hook: convert 'post-rewrite' hook to config
    +    hook: convert 'post-rewrite' hook in sequencer.c to hook.h
     
         By using 'hook.h' for 'post-rewrite', we simplify hook invocations by
    -    not needing to put together our own 'struct child_process' and we also
    -    learn to run hooks specified in the config as well as the hook dir.
    +    not needing to put together our own 'struct child_process'.
     
    -    The signal handling that's being removed by this commit now takes place
    -    in run-command.h:run_processes_parallel(), so it is OK to remove them
    -    here.
    +    The signal handling that's being removed by this commit now takes
    +    place in run-command.h:run_processes_parallel(), so it is OK to remove
    +    them here.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    -
    - ## Documentation/githooks.txt ##
    -@@ Documentation/githooks.txt: The hook always runs after the automatic note copying (see
    - "notes.rewrite.<command>" in linkgit:git-config[1]) has happened, and
    - thus has access to these notes.
    - 
    -+Hooks run during 'post-rewrite' will be run in parallel, unless hook.jobs is
    -+configured to 1.
    -+
    - The following command-specific comments apply:
    - 
    - rebase::
    -
    - ## builtin/am.c ##
    -@@ builtin/am.c: static int run_applypatch_msg_hook(struct am_state *state)
    -  */
    - static int run_post_rewrite_hook(const struct am_state *state)
    - {
    --	struct child_process cp = CHILD_PROCESS_INIT;
    --	const char *hook = find_hook("post-rewrite");
    -+	struct run_hooks_opt opt;
    - 	int ret;
    - 
    --	if (!hook)
    --		return 0;
    -+	run_hooks_opt_init_async(&opt);
    - 
    --	strvec_push(&cp.args, hook);
    --	strvec_push(&cp.args, "rebase");
    -+	strvec_push(&opt.args, "rebase");
    -+	opt.path_to_stdin = am_path(state, "rewritten");
    - 
    --	cp.in = xopen(am_path(state, "rewritten"), O_RDONLY);
    --	cp.stdout_to_stderr = 1;
    --	cp.trace2_hook_name = "post-rewrite";
    -+	ret = run_hooks("post-rewrite", &opt);
    - 
    --	ret = run_command(&cp);
    --
    --	close(cp.in);
    -+	run_hooks_opt_clear(&opt);
    - 	return ret;
    - }
    - 
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## sequencer.c ##
     @@
    + #include "commit-reach.h"
      #include "rebase-interactive.h"
      #include "reset.h"
    - #include "hook.h"
     +#include "string-list.h"
      
      #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
    @@ sequencer.c: int update_head_with_reflog(const struct commit *old_head,
      {
     -	struct child_process proc = CHILD_PROCESS_INIT;
     -	const char *argv[3];
    -+	struct run_hooks_opt opt;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
     +	struct strbuf tmp = STRBUF_INIT;
     +	struct string_list to_stdin = STRING_LIST_INIT_DUP;
      	int code;
    @@ sequencer.c: int update_head_with_reflog(const struct commit *old_head,
     -	argv[0] = find_hook("post-rewrite");
     -	if (!argv[0])
     -		return 0;
    -+	run_hooks_opt_init_async(&opt);
    ++	strvec_push(&opt.args, "amend");
      
     -	argv[1] = "amend";
     -	argv[2] = NULL;
    @@ sequencer.c: int update_head_with_reflog(const struct commit *old_head,
     -	strbuf_release(&sb);
     -	sigchain_pop(SIGPIPE);
     -	return finish_command(&proc);
    -+	strvec_push(&opt.args, "amend");
    -+
     +	strbuf_addf(&tmp,
     +		    "%s %s",
     +		    oid_to_hex(oldoid),
    @@ sequencer.c: static int pick_commits(struct repository *r,
     -			strvec_push(&child.args, "copy");
     -			strvec_push(&child.args, "--for-rewrite=rebase");
     +			struct child_process notes_cp = CHILD_PROCESS_INIT;
    -+			struct run_hooks_opt hook_opt;
    -+
    -+			run_hooks_opt_init_async(&hook_opt);
    ++			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
     +
     +			notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY);
     +			notes_cp.git_cmd = 1;
27:  e761410a1e8 ! 23:  c59443a3b05 transport: convert pre-push hook to use config
    @@ Commit message
         the config as well as in the hookdir.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    -
    - ## Documentation/githooks.txt ##
    -@@ Documentation/githooks.txt: If this hook exits with a non-zero status, `git push` will abort without
    - pushing anything.  Information about why the push is rejected may be sent
    - to the user by writing to standard error.
    - 
    -+Hooks executed during 'pre-push' will run in parallel, unless hook.jobs is
    -+configured to 1.
    -+
    - [[pre-receive]]
    - pre-receive
    - ~~~~~~~~~~~
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## transport.c ##
    -@@
    - #include "protocol.h"
    - #include "object-store.h"
    - #include "color.h"
    -+#include "hook.h"
    - 
    - static int transport_use_color = -1;
    - static char transport_colors[][COLOR_MAXLEN] = {
     @@ transport.c: static void die_with_unpushed_submodules(struct string_list *needs_pushing)
      static int run_pre_push_hook(struct transport *transport,
      			     struct ref *remote_refs)
      {
     -	int ret = 0, x;
     +	int ret = 0;
    -+	struct run_hooks_opt opt;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
     +	struct strbuf tmp = STRBUF_INIT;
      	struct ref *r;
     -	struct child_process proc = CHILD_PROCESS_INIT;
    @@ transport.c: static void die_with_unpushed_submodules(struct string_list *needs_
     -
     -	sigchain_push(SIGPIPE, SIG_IGN);
     +	struct string_list to_stdin = STRING_LIST_INIT_DUP;
    -+	run_hooks_opt_init_async(&opt);
      
     -	strbuf_init(&buf, 256);
     +	strvec_push(&opt.args, transport->remote->name);
28:  e43f9f93d7b ! 24:  f7c8c97cb81 reference-transaction: look for hooks in config
    @@ Metadata
     Author: Emily Shaffer <emilyshaffer@google.com>
     
      ## Commit message ##
    -    reference-transaction: look for hooks in config
    +    reference-transaction: use hook.h to run hooks
     
    -    By using the hook.h library, reference-transaction hooks can be
    -    specified in the config instead.
    -
    -    The expected output of the test is not fully updated to reflect the
    -    absolute path of the hook called because the 'update' hook has not yet
    -    been converted to use hook.h.
    +    By using the hook.h library, we get closer to removing the hook code
    +    in run-command.c.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    -
    - ## Documentation/githooks.txt ##
    -@@ Documentation/githooks.txt: The exit status of the hook is ignored for any state except for the
    - cause the transaction to be aborted. The hook will not be called with
    - "aborted" state in that case.
    - 
    -+Hooks run during 'reference-transaction' will be run in parallel, unless
    -+hook.jobs is configured to 1.
    -+
    - push-to-checkout
    - ~~~~~~~~~~~~~~~~
    - 
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## refs.c ##
    -@@
    - #include "strvec.h"
    - #include "repository.h"
    - #include "sigchain.h"
    -+#include "hook.h"
    - 
    - /*
    -  * List of all available backends
     @@ refs.c: int ref_update_reject_duplicates(struct string_list *refnames,
      static int run_transaction_hook(struct ref_transaction *transaction,
      				const char *state)
    @@ refs.c: int ref_update_reject_duplicates(struct string_list *refnames,
     -	struct child_process proc = CHILD_PROCESS_INIT;
      	struct strbuf buf = STRBUF_INIT;
     -	const char *hook;
    -+	struct run_hooks_opt opt;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
     +	struct string_list to_stdin = STRING_LIST_INIT_DUP;
      	int ret = 0, i;
     +	char o[GIT_MAX_HEXSZ + 1], n[GIT_MAX_HEXSZ + 1];
      
     -	hook = find_hook("reference-transaction");
     -	if (!hook)
    --		return ret;
    --
    ++	if (!hook_exists("reference-transaction"))
    + 		return ret;
    + 
     -	strvec_pushl(&proc.args, hook, state, NULL);
     -	proc.in = -1;
     -	proc.stdout_to_stderr = 1;
     -	proc.trace2_hook_name = "reference-transaction";
    -+	run_hooks_opt_init_async(&opt);
    - 
    +-
     -	ret = start_command(&proc);
     -	if (ret)
    -+	if (!hook_exists("reference-transaction", HOOKDIR_USE_CONFIG))
    - 		return ret;
    - 
    +-		return ret;
    +-
     -	sigchain_push(SIGPIPE, SIG_IGN);
     +	strvec_push(&opt.args, state);
      
    @@ refs.c: int ref_update_reject_duplicates(struct string_list *refnames,
      	return ret;
      }
      
    -
    - ## t/t1416-ref-transaction-hooks.sh ##
    -@@ t/t1416-ref-transaction-hooks.sh: test_expect_success 'interleaving hook calls succeed' '
    - 
    - 	cat >expect <<-EOF &&
    - 		hooks/update refs/tags/PRE $ZERO_OID $PRE_OID
    --		hooks/reference-transaction prepared
    --		hooks/reference-transaction committed
    -+		$(pwd)/target-repo.git/hooks/reference-transaction prepared
    -+		$(pwd)/target-repo.git/hooks/reference-transaction committed
    - 		hooks/update refs/tags/POST $ZERO_OID $POST_OID
    --		hooks/reference-transaction prepared
    --		hooks/reference-transaction committed
    -+		$(pwd)/target-repo.git/hooks/reference-transaction prepared
    -+		$(pwd)/target-repo.git/hooks/reference-transaction committed
    - 	EOF
    - 
    - 	git push ./target-repo.git PRE POST &&
    -
    - ## transport.c ##
    -@@ transport.c: static int run_pre_push_hook(struct transport *transport,
    - 	struct strbuf tmp = STRBUF_INIT;
    - 	struct ref *r;
    - 	struct string_list to_stdin = STRING_LIST_INIT_DUP;
    -+
    - 	run_hooks_opt_init_async(&opt);
    - 
    - 	strvec_push(&opt.args, transport->remote->name);
15:  84fe7a21976 ! 25:  f240a51ec4e run-command: allow capturing of collated output
    @@ Commit message
         instead hand it to the caller.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## builtin/fetch.c ##
     @@ builtin/fetch.c: static int fetch_multiple(struct string_list *list, int max_children)
    @@ builtin/submodule--helper.c: static int update_submodules(struct submodule_updat
      
     
      ## hook.c ##
    -@@ hook.c: int run_hooks(const char *hookname, struct run_hooks_opt *options)
    +@@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
      				   pick_next_hook,
      				   notify_start_failure,
      				   options->feed_pipe,
16:  f23bcba44a0 ! 26:  7f10efb7858 hooks: allow callers to capture output
    @@ Commit message
         sideband instead of printing directly to stderr. Expose that capability.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## hook.c ##
    -@@ hook.c: void run_hooks_opt_init_sync(struct run_hooks_opt *o)
    - 	o->dir = NULL;
    - 	o->feed_pipe = NULL;
    - 	o->feed_pipe_ctx = NULL;
    -+	o->consume_sideband = NULL;
    - }
    - 
    - void run_hooks_opt_init_async(struct run_hooks_opt *o)
    -@@ hook.c: int run_hooks(const char *hookname, struct run_hooks_opt *options)
    +@@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
      				   pick_next_hook,
      				   notify_start_failure,
      				   options->feed_pipe,
    @@ hook.c: int run_hooks(const char *hookname, struct run_hooks_opt *options)
     
      ## hook.h ##
     @@ hook.h: struct run_hooks_opt
    + 	 */
      	feed_pipe_fn feed_pipe;
      	void *feed_pipe_ctx;
    - 
    ++
     +	/*
     +	 * Populate this to capture output and prevent it from being printed to
     +	 * stderr. This will be passed directly through to
    @@ hook.h: struct run_hooks_opt
     +	 * for an example.
     +	 */
     +	consume_sideband_fn consume_sideband;
    -+
    - 	/* Number of threads to parallelize across */
    - 	int jobs;
    + };
      
    + #define RUN_HOOKS_OPT_INIT { \
29:  7931831dc6e ! 27:  c39c608e5cc receive-pack: convert 'update' hook to hook.h
    @@ Metadata
      ## Commit message ##
         receive-pack: convert 'update' hook to hook.h
     
    -    By using hook.h to invoke the 'update' hook, now hooks can be specified
    -    in the config in addition to the hookdir.
    +    By using hook.h to invoke the 'update' hook we closer to removing the
    +    hooks code in run-command.c.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    -
    - ## Documentation/githooks.txt ##
    -@@ Documentation/githooks.txt: The default 'update' hook, when enabled--and with
    - `hooks.allowunannotated` config option unset or set to false--prevents
    - unannotated tags to be pushed.
    - 
    -+Hooks executed during 'update' are run in parallel, unless hook.jobs is
    -+configured to 1.
    -+
    - [[proc-receive]]
    - proc-receive
    - ~~~~~~~~~~~~
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## builtin/receive-pack.c ##
     @@ builtin/receive-pack.c: static int run_receive_hook(struct command *commands,
      	return status;
      }
      
    +-static int run_update_hook(struct command *cmd)
     +static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
    -+{
    + {
    +-	const char *argv[5];
    +-	struct child_process proc = CHILD_PROCESS_INIT;
    +-	int code;
     +	int keepalive_active = 0;
    -+
    + 
    +-	argv[0] = find_hook("update");
    +-	if (!argv[0])
    +-		return 0;
     +	if (keepalive_in_sec <= 0)
     +		use_keepalive = KEEPALIVE_NEVER;
     +	if (use_keepalive == KEEPALIVE_ALWAYS)
     +		keepalive_active = 1;
    -+
    + 
    +-	argv[1] = cmd->ref_name;
    +-	argv[2] = oid_to_hex(&cmd->old_oid);
    +-	argv[3] = oid_to_hex(&cmd->new_oid);
    +-	argv[4] = NULL;
     +	/* send a keepalive if there is no data to write */
     +	if (keepalive_active && !output->len) {
     +		static const char buf[] = "0005\1";
     +		write_or_die(1, buf, sizeof(buf) - 1);
     +		return;
     +	}
    -+
    + 
    +-	proc.no_stdin = 1;
    +-	proc.stdout_to_stderr = 1;
    +-	proc.err = use_sideband ? -1 : 0;
    +-	proc.argv = argv;
    +-	proc.trace2_hook_name = "update";
     +	if (use_keepalive == KEEPALIVE_AFTER_NUL && !keepalive_active) {
     +		const char *first_null = memchr(output->buf, '\0', output->len);
     +		if (first_null) {
    @@ builtin/receive-pack.c: static int run_receive_hook(struct command *commands,
     +	send_sideband(1, 2, output->buf, output->len, use_sideband);
     +}
     +
    - static int run_update_hook(struct command *cmd)
    - {
    --	const char *argv[5];
    --	struct child_process proc = CHILD_PROCESS_INIT;
    -+	struct run_hooks_opt opt;
    - 	int code;
    - 
    --	argv[0] = find_hook("update");
    --	if (!argv[0])
    --		return 0;
    -+	run_hooks_opt_init_async(&opt);
    - 
    --	argv[1] = cmd->ref_name;
    --	argv[2] = oid_to_hex(&cmd->old_oid);
    --	argv[3] = oid_to_hex(&cmd->new_oid);
    --	argv[4] = NULL;
    ++static int run_update_hook(struct command *cmd)
    ++{
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    ++	int code;
    ++
     +	strvec_pushl(&opt.args,
     +		     cmd->ref_name,
     +		     oid_to_hex(&cmd->old_oid),
     +		     oid_to_hex(&cmd->new_oid),
     +		     NULL);
      
    --	proc.no_stdin = 1;
    --	proc.stdout_to_stderr = 1;
    --	proc.err = use_sideband ? -1 : 0;
    --	proc.argv = argv;
    --	proc.trace2_hook_name = "update";
    --
     -	code = start_command(&proc);
     -	if (code)
     -		return code;
    @@ builtin/receive-pack.c: static int run_receive_hook(struct command *commands,
      }
      
      static struct command *find_command_by_refname(struct command *list,
    -
    - ## t/t1416-ref-transaction-hooks.sh ##
    -@@ t/t1416-ref-transaction-hooks.sh: test_expect_success 'interleaving hook calls succeed' '
    - 	EOF
    - 
    - 	cat >expect <<-EOF &&
    --		hooks/update refs/tags/PRE $ZERO_OID $PRE_OID
    -+		$(pwd)/target-repo.git/hooks/update refs/tags/PRE $ZERO_OID $PRE_OID
    - 		$(pwd)/target-repo.git/hooks/reference-transaction prepared
    - 		$(pwd)/target-repo.git/hooks/reference-transaction committed
    --		hooks/update refs/tags/POST $ZERO_OID $POST_OID
    -+		$(pwd)/target-repo.git/hooks/update refs/tags/POST $ZERO_OID $POST_OID
    - 		$(pwd)/target-repo.git/hooks/reference-transaction prepared
    - 		$(pwd)/target-repo.git/hooks/reference-transaction committed
    - 	EOF
31:  efbe55fcb13 ! 28:  3519068a634 post-update: use hook.h library
    @@ Commit message
         be specified in the config as well as the hookdir.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    -
    - ## Documentation/githooks.txt ##
    -@@ Documentation/githooks.txt: Both standard output and standard error output are forwarded to
    - `git send-pack` on the other end, so you can simply `echo` messages
    - for the user.
    - 
    -+Hooks run during 'post-update' will be run in parallel, unless hook.jobs is
    -+configured to 1.
    -+
    - reference-transaction
    - ~~~~~~~~~~~~~~~~~~~~~
    - 
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## builtin/receive-pack.c ##
     @@ builtin/receive-pack.c: static const char *update(struct command *cmd, struct shallow_info *si)
    @@ builtin/receive-pack.c: static const char *update(struct command *cmd, struct sh
      	struct command *cmd;
     -	struct child_process proc = CHILD_PROCESS_INIT;
     -	const char *hook;
    -+	struct run_hooks_opt opt;
    - 
    +-
     -	hook = find_hook("post-update");
     -	if (!hook)
     -		return;
    -+	run_hooks_opt_init_async(&opt);
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
      
      	for (cmd = commands; cmd; cmd = cmd->next) {
      		if (cmd->error_string || cmd->did_not_exist)
32:  26e08bd5257 ! 29:  3466f17af08 receive-pack: convert receive hooks to hook.h
    @@ Metadata
      ## Commit message ##
         receive-pack: convert receive hooks to hook.h
     
    -    By using the hook.h library to run receive hooks, they can be specified
    -    in the config as well as in the hookdir.
    +    By using the hook.h library to run receive hooks we get closer to
    +    deleting the hook functions in run-command.c
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    -
    - ## Documentation/githooks.txt ##
    -@@ Documentation/githooks.txt: will be set to zero, `GIT_PUSH_OPTION_COUNT=0`.
    - See the section on "Quarantine Environment" in
    - linkgit:git-receive-pack[1] for some caveats.
    - 
    -+Hooks executed during 'pre-receive' will not be parallelized.
    -+
    - [[update]]
    - update
    - ~~~~~~
    -@@ Documentation/githooks.txt: environment variables will not be set. If the client selects
    - to use push options, but doesn't transmit any, the count variable
    - will be set to zero, `GIT_PUSH_OPTION_COUNT=0`.
    - 
    -+Hooks executed during 'post-receive' are run in parallel, unless hook.jobs is
    -+configured to 1.
    -+
    - [[post-update]]
    - post-update
    - ~~~~~~~~~~~
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## builtin/receive-pack.c ##
     @@ builtin/receive-pack.c: static int check_cert_push_options(const struct string_list *push_options)
    @@ builtin/receive-pack.c: static void hook_output_to_sideband(struct strbuf *outpu
     +			    int skip_broken,
     +			    const struct string_list *push_options)
     +{
    -+	struct run_hooks_opt opt;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
     +	struct receive_hook_feed_context ctx;
     +	int rc;
     +	struct command *iter = commands;
     +
    -+	run_hooks_opt_init_async(&opt);
    -+
     +	/* if there are no valid commands, don't invoke the hook at all. */
     +	while (iter && skip_broken && (iter->error_string || iter->did_not_exist))
     +		iter = iter->next;
    @@ builtin/receive-pack.c: static void hook_output_to_sideband(struct strbuf *outpu
     +
      static int run_update_hook(struct command *cmd)
      {
    - 	struct run_hooks_opt opt;
    + 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
17:  edb22d675b7 ! 30:  d93bdc0c294 commit: use config-based hooks
    @@
      ## Metadata ##
    -Author: Emily Shaffer <emilyshaffer@google.com>
    +Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## Commit message ##
    -    commit: use config-based hooks
    +    hooks: fix a TOCTOU in "did we run a hook?" heuristic
     
    -    As part of the adoption of config-based hooks, teach run_commit_hook()
    -    to call hook.h instead of run-command.h. This covers 'pre-commit',
    -    'commit-msg', and 'prepare-commit-msg'. Additionally, ask the hook
    -    library - not run-command - whether any hooks will be run, as it's
    -    possible hooks may exist in the config but not the hookdir.
    +    Fix a race in code added in 680ee550d72 (commit: skip discarding the
    +    index if there is no pre-commit hook, 2017-08-14) by changing the
    +    hook.c API to optionally indicate whether or not the requested hook
    +    ran or not. This was suggested in the discussion around
    +    680ee550d72[1].
     
    -    Because all but 'post-commit' hooks are expected to make some state
    -    change, force all but 'post-commit' hook to run in series. 'post-commit'
    -    "is meant primarily for notification, and cannot affect the outcome of
    -    `git commit`," so it is fine to run in parallel.
    +    Let's also change this for the pre-merge-commit hook, see
    +    6098817fd7f (git-merge: honor pre-merge-commit hook, 2019-08-07) for
    +    the introduction of the previous behavior.
     
    -    Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
    +    Let's also change this for the push-to-checkout hook. Now instead of
    +    checking if the hook exists and either doing a push to checkout or a
    +    push to deploy we'll always attempt a push to checkout. If the hook
    +    doesn't exist we'll fall back on push to deploy. The same behavior as
    +    before, without the TOCTOU race. See 0855331941b (receive-pack:
    +    support push-to-checkout hook, 2014-12-01) for the introduction of the
    +    previous behavior.
     
    - ## Documentation/githooks.txt ##
    -@@ Documentation/githooks.txt: The default 'pre-commit' hook, when enabled--and with the
    - `hooks.allownonascii` config option unset or set to false--prevents
    - the use of non-ASCII filenames.
    - 
    -+Hooks executed during 'pre-commit' will not be parallelized.
    -+
    - pre-merge-commit
    - ~~~~~~~~~~~~~~~~
    - 
    -@@ Documentation/githooks.txt: need to be resolved and the result committed separately (see
    - linkgit:git-merge[1]). At that point, this hook will not be executed,
    - but the 'pre-commit' hook will, if it is enabled.
    - 
    -+Hooks executed during 'pre-merge-commit' will not be parallelized.
    -+
    - prepare-commit-msg
    - ~~~~~~~~~~~~~~~~~~
    - 
    -@@ Documentation/githooks.txt: be used as replacement for pre-commit hook.
    - The sample `prepare-commit-msg` hook that comes with Git removes the
    - help message found in the commented portion of the commit template.
    - 
    -+Hooks executed during 'prepare-commit-msg' will not be parallelized, because
    -+hooks are expected to edit the file containing the commit log message.
    -+
    - commit-msg
    - ~~~~~~~~~~
    - 
    -@@ Documentation/githooks.txt: file.
    - The default 'commit-msg' hook, when enabled, detects duplicate
    - `Signed-off-by` trailers, and aborts the commit if one is found.
    - 
    -+Hooks executed during 'commit-msg' will not be parallelized, because hooks are
    -+expected to edit the file containing the proposed commit log message.
    -+
    - post-commit
    - ~~~~~~~~~~~
    - 
    -@@ Documentation/githooks.txt: invoked after a commit is made.
    - This hook is meant primarily for notification, and cannot affect
    - the outcome of `git commit`.
    - 
    -+Hooks executed during 'post-commit' will run in parallel, unless hook.jobs is
    -+configured to 1.
    -+
    - pre-rebase
    - ~~~~~~~~~~
    - 
    +    This leaves uses of hook_exists() in two places that matter. The
    +    "reference-transaction" check in refs.c, see 67541597670 (refs:
    +    implement reference transaction hook, 2020-06-19), and the
    +    prepare-commit-msg hook, see 66618a50f9c (sequencer: run
    +    'prepare-commit-msg' hook, 2018-01-24).
    +
    +    In both of those cases we're saving ourselves CPU time by not
    +    preparing data for the hook that we'll then do nothing with if we
    +    don't have the hook, so using this "invoked_hook" pattern doesn't make
    +    sense there purely for optimization purposes.
    +
    +    More importantly, in those cases the worst we'll do is miss that we
    +    "should" run the hook because a new hook appeared, whereas in the
    +    pre-commit and pre-merge-commit cases we'll skip an important
    +    discard_cache() on the bases of our faulty guess.
    +
    +    I do think none of these races really matter in practice. It would be
    +    some one-off issue as a hook was added or removed. I did think it was
    +    stupid that we didn't pass a "did this run?" flag instead of doing
    +    this guessing at a distance though, so now we're not guessing anymore.
    +
    +    1. https://lore.kernel.org/git/20170810191613.kpmhzg4seyxy3cpq@sigill.intra.peff.net/
    +
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## builtin/commit.c ##
    -@@
    - #include "help.h"
    - #include "commit-reach.h"
    - #include "commit-graph.h"
    -+#include "hook.h"
    - 
    - static const char * const builtin_commit_usage[] = {
    - 	N_("git commit [<options>] [--] <pathspec>..."),
     @@ builtin/commit.c: static int prepare_to_commit(const char *index_file, const char *prefix,
    + 	int clean_message_contents = (cleanup_mode != COMMIT_MSG_CLEANUP_NONE);
    + 	int old_display_comment_prefix;
    + 	int merge_contains_scissors = 0;
    ++	int invoked_hook = 0;
    + 
      	/* This checks and barfs if author is badly specified */
      	determine_author_info(author_ident);
      
     -	if (!no_verify && run_commit_hook(use_editor, index_file, "pre-commit", NULL))
    -+	if (!no_verify && run_commit_hook(use_editor, 0, index_file, "pre-commit", NULL))
    ++	if (!no_verify && run_commit_hook(use_editor, index_file, &invoked_hook,
    ++					  "pre-commit", NULL))
      		return 0;
      
      	if (squash_message) {
    @@ builtin/commit.c: static int prepare_to_commit(const char *index_file, const cha
      		return 0;
      	}
      
    --	if (!no_verify && find_hook("pre-commit")) {
    -+	if (!no_verify && hook_exists("pre-commit", HOOKDIR_USE_CONFIG)) {
    +-	if (!no_verify && hook_exists("pre-commit")) {
    ++	if (!no_verify && invoked_hook) {
      		/*
    - 		 * Re-read the index as pre-commit hook could have updated it,
    - 		 * and write it out as a tree.  We must do this before we invoke
    +-		 * Re-read the index as pre-commit hook could have updated it,
    +-		 * and write it out as a tree.  We must do this before we invoke
    ++		 * Re-read the index as the pre-commit-commit hook was invoked
    ++		 * and could have updated it. We must do this before we invoke
    + 		 * the editor and after we invoke run_status above.
    + 		 */
    + 		discard_cache();
     @@ builtin/commit.c: static int prepare_to_commit(const char *index_file, const char *prefix,
      		return 0;
      	}
      
     -	if (run_commit_hook(use_editor, index_file, "prepare-commit-msg",
    -+	if (run_commit_hook(use_editor, 0, index_file, "prepare-commit-msg",
    ++	if (run_commit_hook(use_editor, index_file, NULL, "prepare-commit-msg",
      			    git_path_commit_editmsg(), hook_arg1, hook_arg2, NULL))
      		return 0;
      
    @@ builtin/commit.c: static int prepare_to_commit(const char *index_file, const cha
      
      	if (!no_verify &&
     -	    run_commit_hook(use_editor, index_file, "commit-msg", git_path_commit_editmsg(), NULL)) {
    -+	    run_commit_hook(use_editor, 0, index_file, "commit-msg", git_path_commit_editmsg(), NULL)) {
    ++	    run_commit_hook(use_editor, index_file, NULL, "commit-msg",
    ++			    git_path_commit_editmsg(), NULL)) {
      		return 0;
      	}
      
    @@ builtin/commit.c: int cmd_commit(int argc, const char **argv, const char *prefix
      	repo_rerere(the_repository, 0);
      	run_auto_maintenance(quiet);
     -	run_commit_hook(use_editor, get_index_file(), "post-commit", NULL);
    -+	run_commit_hook(use_editor, 1, get_index_file(), "post-commit", NULL);
    ++	run_commit_hook(use_editor, get_index_file(), NULL, "post-commit",
    ++			NULL);
      	if (amend && !no_post_rewrite) {
      		commit_post_rewrite(the_repository, current_head, &oid);
      	}
     
      ## builtin/merge.c ##
    -@@
    - #include "commit-reach.h"
    - #include "wt-status.h"
    - #include "commit-graph.h"
    -+#include "hook.h"
    - 
    - #define DEFAULT_TWOHEAD (1<<0)
    - #define DEFAULT_OCTOPUS (1<<1)
     @@ builtin/merge.c: static void prepare_to_commit(struct commit_list *remoteheads)
    + {
      	struct strbuf msg = STRBUF_INIT;
      	const char *index_file = get_index_file();
    ++	int invoked_hook = 0;
      
     -	if (!no_verify && run_commit_hook(0 < option_edit, index_file, "pre-merge-commit", NULL))
    -+	if (!no_verify && run_commit_hook(0 < option_edit, 0, index_file, "pre-merge-commit", NULL))
    ++	if (!no_verify && run_commit_hook(0 < option_edit, index_file,
    ++					  &invoked_hook, "pre-merge-commit",
    ++					  NULL))
      		abort_commit(remoteheads, NULL);
      	/*
    - 	 * Re-read the index as pre-merge-commit hook could have updated it,
    - 	 * and write it out as a tree.  We must do this before we invoke
    +-	 * Re-read the index as pre-merge-commit hook could have updated it,
    +-	 * and write it out as a tree.  We must do this before we invoke
    ++	 * Re-read the index as the pre-merge-commit hook was invoked
    ++	 * and could have updated it. We must do this before we invoke
      	 * the editor and after we invoke run_status above.
      	 */
    --	if (find_hook("pre-merge-commit"))
    -+	if (hook_exists("pre-merge-commit", HOOKDIR_USE_CONFIG))
    +-	if (hook_exists("pre-merge-commit"))
    ++	if (invoked_hook)
      		discard_cache();
      	read_cache_from(index_file);
      	strbuf_addbuf(&msg, &merge_msg);
    @@ builtin/merge.c: static void prepare_to_commit(struct commit_list *remoteheads)
      	write_merge_heads(remoteheads);
      	write_file_buf(git_path_merge_msg(the_repository), msg.buf, msg.len);
     -	if (run_commit_hook(0 < option_edit, get_index_file(), "prepare-commit-msg",
    -+	if (run_commit_hook(0 < option_edit, 0, get_index_file(), "prepare-commit-msg",
    ++	if (run_commit_hook(0 < option_edit, get_index_file(), NULL,
    ++			    "prepare-commit-msg",
      			    git_path_merge_msg(the_repository), "merge", NULL))
      		abort_commit(remoteheads, NULL);
      	if (0 < option_edit) {
     @@ builtin/merge.c: static void prepare_to_commit(struct commit_list *remoteheads)
    - 			abort_commit(remoteheads, NULL);
      	}
      
    --	if (!no_verify && run_commit_hook(0 < option_edit, get_index_file(),
    -+	if (!no_verify && run_commit_hook(0 < option_edit, 0, get_index_file(),
    - 					  "commit-msg",
    + 	if (!no_verify && run_commit_hook(0 < option_edit, get_index_file(),
    +-					  "commit-msg",
    ++					  NULL, "commit-msg",
      					  git_path_merge_msg(the_repository), NULL))
      		abort_commit(remoteheads, NULL);
    -
    - ## commit.c ##
    -@@
    - #include "commit-reach.h"
    - #include "run-command.h"
    - #include "shallow.h"
    -+#include "hook.h"
      
    - static struct commit_extra_header *read_commit_extra_header_lines(const char *buf, size_t len, const char **);
    - 
    -@@ commit.c: size_t ignore_non_trailer(const char *buf, size_t len)
    - 	return boc ? len - boc : len - cutoff;
    - }
    +
    + ## builtin/receive-pack.c ##
    +@@ builtin/receive-pack.c: static const char *push_to_deploy(unsigned char *sha1,
    + static const char *push_to_checkout_hook = "push-to-checkout";
    + 
    + static const char *push_to_checkout(unsigned char *hash,
    ++				    int *invoked_hook,
    + 				    struct strvec *env,
    + 				    const char *work_tree)
    + {
    + 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    ++	opt.invoked_hook = invoked_hook;
      
    --int run_commit_hook(int editor_is_used, const char *index_file,
    -+int run_commit_hook(int editor_is_used, int parallelize, const char *index_file,
    - 		    const char *name, ...)
    + 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
    + 	strvec_pushv(&opt.env, env->v);
    +@@ builtin/receive-pack.c: static const char *update_worktree(unsigned char *sha1, const struct worktree *w
      {
    --	struct strvec hook_env = STRVEC_INIT;
    -+	struct run_hooks_opt opt;
    - 	va_list args;
    -+	const char *arg;
    - 	int ret;
    + 	const char *retval, *work_tree, *git_dir = NULL;
    + 	struct strvec env = STRVEC_INIT;
    ++	int invoked_hook = 0;
      
    --	strvec_pushf(&hook_env, "GIT_INDEX_FILE=%s", index_file);
    -+	run_hooks_opt_init_sync(&opt);
    -+
    -+	if (parallelize)
    -+		opt.jobs = configured_hook_jobs();
    -+
    -+	strvec_pushf(&opt.env, "GIT_INDEX_FILE=%s", index_file);
    + 	if (worktree && worktree->path)
    + 		work_tree = worktree->path;
    +@@ builtin/receive-pack.c: static const char *update_worktree(unsigned char *sha1, const struct worktree *w
      
    - 	/*
    - 	 * Let the hook know that no editor will be launched.
    - 	 */
    - 	if (!editor_is_used)
    --		strvec_push(&hook_env, "GIT_EDITOR=:");
    -+		strvec_push(&opt.env, "GIT_EDITOR=:");
    + 	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
      
    - 	va_start(args, name);
    --	ret = run_hook_ve(hook_env.v, name, args);
    -+	while ((arg = va_arg(args, const char *)))
    -+		strvec_push(&opt.args, arg);
    - 	va_end(args);
    --	strvec_clear(&hook_env);
    -+
    -+	ret = run_hooks(name, &opt);
    -+	run_hooks_opt_clear(&opt);
    +-	if (!hook_exists(push_to_checkout_hook))
    ++	retval = push_to_checkout(sha1, &invoked_hook, &env, work_tree);
    ++	if (!invoked_hook)
    + 		retval = push_to_deploy(sha1, &env, work_tree);
    +-	else
    +-		retval = push_to_checkout(sha1, &env, work_tree);
      
    - 	return ret;
    + 	strvec_clear(&env);
    + 	return retval;
    +
    + ## commit.c ##
    +@@ commit.c: size_t ignore_non_trailer(const char *buf, size_t len)
      }
    + 
    + int run_commit_hook(int editor_is_used, const char *index_file,
    ++		    int *invoked_hook,
    + 		    const char *name, ...)
    + {
    + 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
     
      ## commit.h ##
     @@ commit.h: int compare_commits_by_commit_date(const void *a_, const void *b_, void *unused)
    @@ commit.h: int compare_commits_by_commit_date(const void *a_, const void *b_, voi
      
      LAST_ARG_MUST_BE_NULL
     -int run_commit_hook(int editor_is_used, const char *index_file, const char *name, ...);
    -+int run_commit_hook(int editor_is_used, int parallelize, const char *index_file,
    -+		    const char *name, ...);
    ++int run_commit_hook(int editor_is_used, const char *index_file,
    ++		    int *invoked_hook, const char *name, ...);
      
      /* Sign a commit or tag buffer, storing the result in a header. */
      int sign_with_header(struct strbuf *buf, const char *keyid);
     
    - ## sequencer.c ##
    -@@
    - #include "commit-reach.h"
    - #include "rebase-interactive.h"
    - #include "reset.h"
    -+#include "hook.h"
    + ## hook.c ##
    +@@ hook.c: static int notify_hook_finished(int result,
    + 	/* |= rc in cb */
    + 	hook_cb->rc |= result;
      
    - #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
    ++	if (hook_cb->invoked_hook)
    ++		*hook_cb->invoked_hook = 1;
    ++
    + 	return 1;
    + }
      
    +@@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
    + 		.rc = 0,
    + 		.hook_name = hook_name,
    + 		.options = options,
    ++		.invoked_hook = options->invoked_hook,
    + 	};
    + 	if (options->absolute_path) {
    + 		strbuf_add_absolute_path(&abs_path, hook_path);
    +
    + ## hook.h ##
    +@@ hook.h: struct run_hooks_opt
    + 	 * for an example.
    + 	 */
    + 	consume_sideband_fn consume_sideband;
    ++
    ++	/*
    ++	 * A pointer which if provided will be set to 1 or 0 depending
    ++	 * on if a hook was invoked (i.e. existed), regardless of
    ++	 * whether or not that was successful. Used for avoiding
    ++	 * TOCTOU races in code that would otherwise call hook_exist()
    ++	 * after a "maybe hook run" to see if a hook was invoked.
    ++	 */
    ++	int *invoked_hook;
    + };
    + 
    + #define RUN_HOOKS_OPT_INIT { \
    +@@ hook.h: struct hook_cb_data {
    + 	const char *hook_name;
    + 	struct hook *run_me;
    + 	struct run_hooks_opt *options;
    ++	int *invoked_hook;
    + };
    + 
    + /*
    +
    + ## sequencer.c ##
     @@ sequencer.c: static int run_prepare_commit_msg_hook(struct repository *r,
      	} else {
      		arg1 = "message";
      	}
     -	if (run_commit_hook(0, r->index_file, "prepare-commit-msg", name,
    -+	if (run_commit_hook(0, 0, r->index_file, "prepare-commit-msg", name,
    ++	if (run_commit_hook(0, r->index_file, NULL, "prepare-commit-msg", name,
      			    arg1, arg2, NULL))
      		ret = error(_("'prepare-commit-msg' hook failed"));
      
    -@@ sequencer.c: static int try_to_commit(struct repository *r,
    - 		}
    - 	}
    - 
    --	if (find_hook("prepare-commit-msg")) {
    -+	if (hook_exists("prepare-commit-msg", HOOKDIR_USE_CONFIG)) {
    - 		res = run_prepare_commit_msg_hook(r, msg, hook_commit);
    - 		if (res)
    - 			goto out;
     @@ sequencer.c: static int try_to_commit(struct repository *r,
      		goto out;
      	}
      
     -	run_commit_hook(0, r->index_file, "post-commit", NULL);
    -+	run_commit_hook(0, 1, r->index_file, "post-commit", NULL);
    ++	run_commit_hook(0, r->index_file, NULL, "post-commit", NULL);
      	if (flags & AMEND_MSG)
      		commit_post_rewrite(r, current_head, oid);
      
    -
    - ## t/t7503-pre-commit-and-pre-merge-commit-hooks.sh ##
    -@@ t/t7503-pre-commit-and-pre-merge-commit-hooks.sh: export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
    - . ./test-lib.sh
    - 
    - HOOKDIR="$(git rev-parse --git-dir)/hooks"
    --PRECOMMIT="$HOOKDIR/pre-commit"
    --PREMERGE="$HOOKDIR/pre-merge-commit"
    -+PRECOMMIT="$(pwd)/$HOOKDIR/pre-commit"
    -+PREMERGE="$(pwd)/$HOOKDIR/pre-merge-commit"
    - 
    - # Prepare sample scripts that write their $0 to actual_hooks
    - test_expect_success 'sample script setup' '
    -@@ t/t7503-pre-commit-and-pre-merge-commit-hooks.sh: test_expect_success 'with succeeding hook' '
    - 	test_cmp expected_hooks actual_hooks
    - '
    - 
    -+# NEEDSWORK: when 'git hook add' and 'git hook remove' have been added, use that
    -+# instead
    -+test_expect_success 'with succeeding hook (config-based)' '
    -+	test_when_finished "git config --unset hook.pre-commit.command success.sample" &&
    -+	test_when_finished "rm -f expected_hooks actual_hooks" &&
    -+	git config hook.pre-commit.command "$HOOKDIR/success.sample" &&
    -+	echo "$HOOKDIR/success.sample" >expected_hooks &&
    -+	echo "more" >>file &&
    -+	git add file &&
    -+	git commit -m "more" &&
    -+	test_cmp expected_hooks actual_hooks
    -+'
    -+
    - test_expect_success 'with succeeding hook (merge)' '
    - 	test_when_finished "rm -f \"$PREMERGE\" expected_hooks actual_hooks" &&
    - 	cp "$HOOKDIR/success.sample" "$PREMERGE" &&
30:  c9d8c1581de <  -:  ----------- proc-receive: acquire hook list from hook.h
36:  4caea3e6805 <  -:  ----------- doc: clarify fsmonitor-watchman specification
37:  f2e1003b62a <  -:  ----------- docs: link githooks and git-hook manpages
 -:  ----------- > 31:  896956250f6 hook-list.h: add a generated list of hooks, like config-list.h
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH 01/31] hooks tests: don't leave "actual" nonexisting on failure
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 02/31] gc tests: add a test for the "pre-auto-gc" hook Ævar Arnfjörð Bjarmason
                         ` (31 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

Start by creating an "actual" file in a core.hooksPath test that has
the hook echoing to the "actual" file. We later test_cmp that file to
see what hooks were run. Having that list of files be empty if we fail
instead of throwing an error about it being nonexistent makes for
friendlier output, we'll see what hooks we should have run.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t1350-config-hooks-path.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/t/t1350-config-hooks-path.sh b/t/t1350-config-hooks-path.sh
index f1f9aee9f5d..fa9647a7c0b 100755
--- a/t/t1350-config-hooks-path.sh
+++ b/t/t1350-config-hooks-path.sh
@@ -5,6 +5,7 @@ test_description='Test the core.hooksPath configuration variable'
 . ./test-lib.sh
 
 test_expect_success 'set up a pre-commit hook in core.hooksPath' '
+	>actual &&
 	mkdir -p .git/custom-hooks .git/hooks &&
 	write_script .git/custom-hooks/pre-commit <<-\EOF &&
 	echo CUSTOM >>actual
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 02/31] gc tests: add a test for the "pre-auto-gc" hook
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 01/31] hooks tests: don't leave "actual" nonexisting on failure Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 03/31] hook: add 'run' subcommand Ævar Arnfjörð Bjarmason
                         ` (30 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

Add a missing test for the behavior of the pre-auto-gc hook added in
0b85d92661e (Documentation/hooks: add pre-auto-gc hook, 2008-04-02).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t6500-gc.sh | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh
index 60d961b5260..10c7ae7f09c 100755
--- a/t/t6500-gc.sh
+++ b/t/t6500-gc.sh
@@ -95,6 +95,52 @@ test_expect_success 'gc --keep-largest-pack' '
 	)
 '
 
+test_expect_success 'pre-auto-gc hook can stop auto gc' '
+	cat >err.expect <<-\EOF &&
+	no gc for you
+	EOF
+
+	git init pre-auto-gc-hook &&
+	(
+		cd pre-auto-gc-hook &&
+		write_script ".git/hooks/pre-auto-gc" <<-\EOF &&
+		echo >&2 no gc for you &&
+		exit 1
+		EOF
+
+		git config gc.auto 3 &&
+		git config gc.autoDetach false &&
+
+		# We need to create two object whose sha1s start with 17
+		# since this is what git gc counts.  As it happens, these
+		# two blobs will do so.
+		test_commit "$(test_oid obj1)" &&
+		test_commit "$(test_oid obj2)" &&
+
+		git gc --auto >../out.actual 2>../err.actual
+	) &&
+	test_must_be_empty out.actual &&
+	test_cmp err.expect err.actual &&
+
+	cat >err.expect <<-\EOF &&
+	will gc for you
+	Auto packing the repository for optimum performance.
+	See "git help gc" for manual housekeeping.
+	EOF
+
+	(
+		cd pre-auto-gc-hook &&
+		write_script ".git/hooks/pre-auto-gc" <<-\EOF &&
+		echo >&2 will gc for you &&
+		exit 0
+		EOF
+		git gc --auto >../out.actual 2>../err.actual
+	) &&
+
+	test_must_be_empty out.actual &&
+	test_cmp err.expect err.actual
+'
+
 test_expect_success 'auto gc with too many loose objects does not attempt to create bitmaps' '
 	test_config gc.auto 3 &&
 	test_config gc.autodetach false &&
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 03/31] hook: add 'run' subcommand
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 01/31] hooks tests: don't leave "actual" nonexisting on failure Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 02/31] gc tests: add a test for the "pre-auto-gc" hook Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 04/31] run-command.h: move find_hook() to hook.h Ævar Arnfjörð Bjarmason
                         ` (29 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

In order to enable hooks to be run as an external process, by a
standalone Git command, or by tools which wrap Git, provide an external
means to run all configured hook commands for a given hook event.

Most of our hooks require more complex functionality than this, but
let's start with the bare minimum required to support our simplest
hooks.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .gitignore                 |   1 +
 Documentation/git-hook.txt |  36 ++++++++++
 Documentation/githooks.txt |   4 ++
 Makefile                   |   2 +
 builtin.h                  |   1 +
 builtin/hook.c             |  65 ++++++++++++++++++
 command-list.txt           |   1 +
 git.c                      |   1 +
 hook.c                     | 114 +++++++++++++++++++++++++++++++
 hook.h                     |  54 +++++++++++++++
 t/t1800-hook.sh            | 135 +++++++++++++++++++++++++++++++++++++
 11 files changed, 414 insertions(+)
 create mode 100644 Documentation/git-hook.txt
 create mode 100644 builtin/hook.c
 create mode 100644 hook.c
 create mode 100644 hook.h
 create mode 100755 t/t1800-hook.sh

diff --git a/.gitignore b/.gitignore
index 311841f9bed..de39dc9961b 100644
--- a/.gitignore
+++ b/.gitignore
@@ -77,6 +77,7 @@
 /git-grep
 /git-hash-object
 /git-help
+/git-hook
 /git-http-backend
 /git-http-fetch
 /git-http-push
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
new file mode 100644
index 00000000000..902b9cffaef
--- /dev/null
+++ b/Documentation/git-hook.txt
@@ -0,0 +1,36 @@
+git-hook(1)
+===========
+
+NAME
+----
+git-hook - run git hooks
+
+SYNOPSIS
+--------
+[verse]
+'git hook' run <hook-name> [-- <hook-args>]
+
+DESCRIPTION
+-----------
+
+This command is an interface to git hooks (see linkgit:githooks[5]).
+Currently it only provides a convenience wrapper for running hooks for
+use by git itself. In the future it might gain other functionality.
+
+SUBCOMMANDS
+-----------
+
+run::
+
+	Run the `<hook-name>` hook. Any positional arguments to the
+	hook should be passed after an optional "--" (or
+	"--end-of-options"). See "OPTIONS" below for the arguments
+	this accepts.
+
+SEE ALSO
+--------
+linkgit:githooks[5]
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index b51959ff941..a16e62bc8c8 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -698,6 +698,10 @@ and "0" meaning they were not.
 Only one parameter should be set to "1" when the hook runs.  The hook
 running passing "1", "1" should not be possible.
 
+SEE ALSO
+--------
+linkgit:git-hook[1]
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index c3565fc0f8f..a6b71a0fbed 100644
--- a/Makefile
+++ b/Makefile
@@ -901,6 +901,7 @@ LIB_OBJS += hash-lookup.o
 LIB_OBJS += hashmap.o
 LIB_OBJS += help.o
 LIB_OBJS += hex.o
+LIB_OBJS += hook.o
 LIB_OBJS += ident.o
 LIB_OBJS += json-writer.o
 LIB_OBJS += kwset.o
@@ -1101,6 +1102,7 @@ BUILTIN_OBJS += builtin/get-tar-commit-id.o
 BUILTIN_OBJS += builtin/grep.o
 BUILTIN_OBJS += builtin/hash-object.o
 BUILTIN_OBJS += builtin/help.o
+BUILTIN_OBJS += builtin/hook.o
 BUILTIN_OBJS += builtin/index-pack.o
 BUILTIN_OBJS += builtin/init-db.o
 BUILTIN_OBJS += builtin/interpret-trailers.o
diff --git a/builtin.h b/builtin.h
index 16ecd5586f0..91740c15149 100644
--- a/builtin.h
+++ b/builtin.h
@@ -164,6 +164,7 @@ int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix);
 int cmd_grep(int argc, const char **argv, const char *prefix);
 int cmd_hash_object(int argc, const char **argv, const char *prefix);
 int cmd_help(int argc, const char **argv, const char *prefix);
+int cmd_hook(int argc, const char **argv, const char *prefix);
 int cmd_index_pack(int argc, const char **argv, const char *prefix);
 int cmd_init_db(int argc, const char **argv, const char *prefix);
 int cmd_interpret_trailers(int argc, const char **argv, const char *prefix);
diff --git a/builtin/hook.c b/builtin/hook.c
new file mode 100644
index 00000000000..1b1a594fd00
--- /dev/null
+++ b/builtin/hook.c
@@ -0,0 +1,65 @@
+#include "cache.h"
+#include "builtin.h"
+#include "config.h"
+#include "hook.h"
+#include "parse-options.h"
+#include "strbuf.h"
+#include "strvec.h"
+
+static const char * const builtin_hook_usage[] = {
+	N_("git hook run <hook-name> [-- <hook-args>]"),
+	NULL
+};
+
+static int run(int argc, const char **argv, const char *prefix)
+{
+	int i;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	int rc = 0;
+	const char *hook_name;
+	const char *hook_path;
+
+	struct option run_options[] = {
+		OPT_END(),
+	};
+
+	argc = parse_options(argc, argv, prefix, run_options,
+			     builtin_hook_usage, PARSE_OPT_KEEP_UNKNOWN | PARSE_OPT_KEEP_DASHDASH);
+
+	if (argc > 2) {
+		if (strcmp(argv[2], "--") &&
+		    strcmp(argv[2], "--end-of-options"))
+			/* Having a -- for "run" is mandatory */
+			usage_with_options(builtin_hook_usage, run_options);
+		/* Add our arguments, start after -- */
+		for (i = 3 ; i < argc; i++)
+			strvec_push(&opt.args, argv[i]);
+	}
+
+	/* Need to take into account core.hooksPath */
+	git_config(git_default_config, NULL);
+
+	hook_name = argv[1];
+	hook_path = find_hook(hook_name);
+	if (!hook_path) {
+		error("cannot find a hook named %s", hook_name);
+		return 1;
+	}
+	rc = run_found_hooks(hook_name, hook_path, &opt);
+
+	run_hooks_opt_clear(&opt);
+
+	return rc;
+}
+
+int cmd_hook(int argc, const char **argv, const char *prefix)
+{
+	struct option builtin_hook_options[] = {
+		OPT_END(),
+	};
+
+	if (!strcmp(argv[1], "run"))
+		return run(argc, argv, prefix);
+	usage_with_options(builtin_hook_usage, builtin_hook_options);
+	return 1;
+}
diff --git a/command-list.txt b/command-list.txt
index a289f09ed6f..9ccd8e5aebe 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -103,6 +103,7 @@ git-grep                                mainporcelain           info
 git-gui                                 mainporcelain
 git-hash-object                         plumbingmanipulators
 git-help                                ancillaryinterrogators          complete
+git-hook                                mainporcelain
 git-http-backend                        synchingrepositories
 git-http-fetch                          synchelpers
 git-http-push                           synchelpers
diff --git a/git.c b/git.c
index 18bed9a9964..540909c391f 100644
--- a/git.c
+++ b/git.c
@@ -538,6 +538,7 @@ static struct cmd_struct commands[] = {
 	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
 	{ "hash-object", cmd_hash_object },
 	{ "help", cmd_help },
+	{ "hook", cmd_hook, RUN_SETUP },
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "init", cmd_init_db },
 	{ "init-db", cmd_init_db },
diff --git a/hook.c b/hook.c
new file mode 100644
index 00000000000..aa66c968186
--- /dev/null
+++ b/hook.c
@@ -0,0 +1,114 @@
+#include "cache.h"
+#include "hook.h"
+#include "run-command.h"
+
+void run_hooks_opt_clear(struct run_hooks_opt *o)
+{
+	strvec_clear(&o->env);
+	strvec_clear(&o->args);
+}
+
+static int pick_next_hook(struct child_process *cp,
+			  struct strbuf *out,
+			  void *pp_cb,
+			  void **pp_task_cb)
+{
+	struct hook_cb_data *hook_cb = pp_cb;
+	struct hook *run_me = hook_cb->run_me;
+
+	if (!run_me)
+		BUG("did we not return 1 in notify_hook_finished?");
+
+	cp->no_stdin = 1;
+	cp->env = hook_cb->options->env.v;
+	cp->stdout_to_stderr = 1;
+	cp->trace2_hook_name = hook_cb->hook_name;
+
+	/* add command */
+	strvec_push(&cp->args, run_me->hook_path);
+
+	/*
+	 * add passed-in argv, without expanding - let the user get back
+	 * exactly what they put in
+	 */
+	strvec_pushv(&cp->args, hook_cb->options->args.v);
+
+	/* Provide context for errors if necessary */
+	*pp_task_cb = run_me;
+
+	return 1;
+}
+
+static int notify_start_failure(struct strbuf *out,
+				void *pp_cb,
+				void *pp_task_cp)
+{
+	struct hook_cb_data *hook_cb = pp_cb;
+	struct hook *attempted = pp_task_cp;
+
+	/* |= rc in cb */
+	hook_cb->rc |= 1;
+
+	strbuf_addf(out, _("Couldn't start hook '%s'\n"),
+		    attempted->hook_path);
+
+	return 1;
+}
+
+static int notify_hook_finished(int result,
+				struct strbuf *out,
+				void *pp_cb,
+				void *pp_task_cb)
+{
+	struct hook_cb_data *hook_cb = pp_cb;
+
+	/* |= rc in cb */
+	hook_cb->rc |= result;
+
+	return 1;
+}
+
+
+int run_found_hooks(const char *hook_name, const char *hook_path,
+		    struct run_hooks_opt *options)
+{
+	struct hook my_hook = {
+		.hook_path = hook_path,
+	};
+	struct hook_cb_data cb_data = {
+		.rc = 0,
+		.hook_name = hook_name,
+		.options = options,
+	};
+	cb_data.run_me = &my_hook;
+
+	if (options->jobs != 1)
+		BUG("we do not handle %d or any other != 1 job number yet", options->jobs);
+
+	run_processes_parallel_tr2(options->jobs,
+				   pick_next_hook,
+				   notify_start_failure,
+				   notify_hook_finished,
+				   &cb_data,
+				   "hook",
+				   hook_name);
+
+	return cb_data.rc;
+}
+
+int run_hooks(const char *hook_name, struct run_hooks_opt *options)
+{
+	const char *hook_path;
+	int ret;
+	if (!options)
+		BUG("a struct run_hooks_opt must be provided to run_hooks");
+
+	hook_path = find_hook(hook_name);
+
+	/* Care about nonexistence? Use run_found_hooks() */
+	if (!hook_path)
+		return 0;
+
+	ret = run_found_hooks(hook_name, hook_path, options);
+	return ret;
+}
diff --git a/hook.h b/hook.h
new file mode 100644
index 00000000000..ebfee26bcf2
--- /dev/null
+++ b/hook.h
@@ -0,0 +1,54 @@
+#ifndef HOOK_H
+#define HOOK_H
+#include "strbuf.h"
+#include "strvec.h"
+#include "run-command.h"
+
+struct hook {
+	/* The path to the hook */
+	const char *hook_path;
+};
+
+struct run_hooks_opt
+{
+	/* Environment vars to be set for each hook */
+	struct strvec env;
+
+	/* Args to be passed to each hook */
+	struct strvec args;
+
+	/* Number of threads to parallelize across */
+	int jobs;
+};
+
+#define RUN_HOOKS_OPT_INIT { \
+	.jobs = 1, \
+	.env = STRVEC_INIT, \
+	.args = STRVEC_INIT, \
+}
+
+/*
+ * Callback provided to feed_pipe_fn and consume_sideband_fn.
+ */
+struct hook_cb_data {
+	int rc;
+	const char *hook_name;
+	struct hook *run_me;
+	struct run_hooks_opt *options;
+};
+
+void run_hooks_opt_clear(struct run_hooks_opt *o);
+
+/*
+ * Calls find_hook(hookname) and runs the hooks (if any) with
+ * run_found_hooks().
+ */
+int run_hooks(const char *hook_name, struct run_hooks_opt *options);
+
+/*
+ * Takes an already resolved hook and runs it. Internally the simpler
+ * run_hooks() will call this.
+ */
+int run_found_hooks(const char *hookname, const char *hook_path,
+		    struct run_hooks_opt *options);
+#endif
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
new file mode 100755
index 00000000000..7cacd27c748
--- /dev/null
+++ b/t/t1800-hook.sh
@@ -0,0 +1,135 @@
+#!/bin/bash
+
+test_description='git-hook command'
+
+. ./test-lib.sh
+
+test_expect_success 'setup .git/hooks' '
+	mkdir .git/hooks
+'
+
+test_expect_success 'git hook run -- nonexistent hook' '
+	cat >stderr.expect <<-\EOF &&
+	error: cannot find a hook named does-not-exist
+	EOF
+	test_expect_code 1 git hook run does-not-exist 2>stderr.actual &&
+	test_cmp stderr.expect stderr.actual
+'
+
+test_expect_success 'git hook run -- basic' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo Test hook
+	EOF
+
+	cat >expect <<-\EOF &&
+	Test hook
+	EOF
+	git hook run test-hook 2>actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git hook run -- stdout and stderr handling' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo >&1 Will end up on stderr
+	echo >&2 Will end up on stderr
+	EOF
+
+	cat >stderr.expect <<-\EOF &&
+	Will end up on stderr
+	Will end up on stderr
+	EOF
+	git hook run test-hook >stdout.actual 2>stderr.actual &&
+	test_cmp stderr.expect stderr.actual &&
+	test_must_be_empty stdout.actual
+'
+
+test_expect_success 'git hook run -- exit codes are passed along' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	exit 1
+	EOF
+
+	test_expect_code 1 git hook run test-hook &&
+
+	write_script .git/hooks/test-hook <<-EOF &&
+	exit 2
+	EOF
+
+	test_expect_code 2 git hook run test-hook &&
+
+	write_script .git/hooks/test-hook <<-EOF &&
+	exit 128
+	EOF
+
+	test_expect_code 128 git hook run test-hook &&
+
+	write_script .git/hooks/test-hook <<-EOF &&
+	exit 129
+	EOF
+
+	test_expect_code 129 git hook run test-hook
+'
+
+test_expect_success 'git hook run arg u ments without -- is not allowed' '
+	test_expect_code 129 git hook run test-hook arg u ments
+'
+
+test_expect_success 'git hook run -- pass arguments' '
+	write_script .git/hooks/test-hook <<-\EOF &&
+	echo $1
+	echo $2
+	EOF
+
+	cat >expect <<-EOF &&
+	arg
+	u ments
+	EOF
+
+	git hook run test-hook -- arg "u ments" 2>actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git hook run -- out-of-repo runs excluded' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo Test hook
+	EOF
+
+	nongit test_must_fail git hook run test-hook
+'
+
+test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
+	mkdir my-hooks &&
+	write_script my-hooks/test-hook <<-EOF &&
+	echo Hook ran >>actual
+	EOF
+
+	cat >expect <<-\EOF &&
+	Test hook
+	Hook ran
+	Hook ran
+	Hook ran
+	Hook ran
+	EOF
+
+	# Test various ways of specifying the path. See also
+	# t1350-config-hooks-path.sh
+	>actual &&
+	git hook run test-hook 2>>actual &&
+	git -c core.hooksPath=my-hooks hook run test-hook 2>>actual &&
+	git -c core.hooksPath=my-hooks/ hook run test-hook 2>>actual &&
+	git -c core.hooksPath="$PWD/my-hooks" hook run test-hook 2>>actual &&
+	git -c core.hooksPath="$PWD/my-hooks/" hook run test-hook 2>>actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'set up a pre-commit hook in core.hooksPath' '
+	>actual &&
+	mkdir -p .git/custom-hooks .git/hooks &&
+	write_script .git/custom-hooks/pre-commit <<-\EOF &&
+	echo CUSTOM >>actual
+	EOF
+	write_script .git/hooks/pre-commit <<-\EOF
+	echo NORMAL >>actual
+	EOF
+'
+
+test_done
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 04/31] run-command.h: move find_hook() to hook.h
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (2 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 03/31] hook: add 'run' subcommand Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 05/31] hook.c: add a hook_exists() wrapper and use it in bugreport.c Ævar Arnfjörð Bjarmason
                         ` (28 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

Move the find_hook() command to hook.h. Eventually all the hook
related code will live there, let's move this function over as-is.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/am.c           |  1 +
 builtin/bugreport.c    |  2 +-
 builtin/commit.c       |  1 +
 builtin/merge.c        |  1 +
 builtin/receive-pack.c |  1 +
 builtin/worktree.c     |  1 +
 hook.c                 | 36 ++++++++++++++++++++++++++++++++++++
 hook.h                 |  7 +++++++
 refs.c                 |  1 +
 run-command.c          | 35 +----------------------------------
 run-command.h          |  7 -------
 sequencer.c            |  1 +
 transport.c            |  1 +
 13 files changed, 53 insertions(+), 42 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index 0b2d886c81b..1c8a5489035 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -11,6 +11,7 @@
 #include "parse-options.h"
 #include "dir.h"
 #include "run-command.h"
+#include "hook.h"
 #include "quote.h"
 #include "tempfile.h"
 #include "lockfile.h"
diff --git a/builtin/bugreport.c b/builtin/bugreport.c
index 9915a5841de..596f079a7f9 100644
--- a/builtin/bugreport.c
+++ b/builtin/bugreport.c
@@ -3,7 +3,7 @@
 #include "strbuf.h"
 #include "help.h"
 #include "compat/compiler.h"
-#include "run-command.h"
+#include "hook.h"
 
 
 static void get_system_info(struct strbuf *sys_info)
diff --git a/builtin/commit.c b/builtin/commit.c
index 190d215d43b..f1aafd67d46 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -19,6 +19,7 @@
 #include "revision.h"
 #include "wt-status.h"
 #include "run-command.h"
+#include "hook.h"
 #include "refs.h"
 #include "log-tree.h"
 #include "strbuf.h"
diff --git a/builtin/merge.c b/builtin/merge.c
index eddb8ae70d6..5a10f6e3c96 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -13,6 +13,7 @@
 #include "builtin.h"
 #include "lockfile.h"
 #include "run-command.h"
+#include "hook.h"
 #include "diff.h"
 #include "diff-merges.h"
 #include "refs.h"
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index a34742513ac..1e0e04c62fc 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -7,6 +7,7 @@
 #include "pkt-line.h"
 #include "sideband.h"
 #include "run-command.h"
+#include "hook.h"
 #include "exec-cmd.h"
 #include "commit.h"
 #include "object.h"
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 976bf8ed063..b1350640fed 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -8,6 +8,7 @@
 #include "branch.h"
 #include "refs.h"
 #include "run-command.h"
+#include "hook.h"
 #include "sigchain.h"
 #include "submodule.h"
 #include "utf8.h"
diff --git a/hook.c b/hook.c
index aa66c968186..06842f50e5c 100644
--- a/hook.c
+++ b/hook.c
@@ -2,6 +2,42 @@
 #include "hook.h"
 #include "run-command.h"
 
+const char *find_hook(const char *name)
+{
+	static struct strbuf path = STRBUF_INIT;
+
+	strbuf_reset(&path);
+	strbuf_git_path(&path, "hooks/%s", name);
+	if (access(path.buf, X_OK) < 0) {
+		int err = errno;
+
+#ifdef STRIP_EXTENSION
+		strbuf_addstr(&path, STRIP_EXTENSION);
+		if (access(path.buf, X_OK) >= 0)
+			return path.buf;
+		if (errno == EACCES)
+			err = errno;
+#endif
+
+		if (err == EACCES && advice_ignored_hook) {
+			static struct string_list advise_given = STRING_LIST_INIT_DUP;
+
+			if (!string_list_lookup(&advise_given, name)) {
+				string_list_insert(&advise_given, name);
+				advise(_("The '%s' hook was ignored because "
+					 "it's not set as executable.\n"
+					 "You can disable this warning with "
+					 "`git config advice.ignoredHook false`."),
+				       path.buf);
+			}
+		}
+		return NULL;
+	}
+	return path.buf;
+}
+
+
+
 void run_hooks_opt_clear(struct run_hooks_opt *o)
 {
 	strvec_clear(&o->env);
diff --git a/hook.h b/hook.h
index ebfee26bcf2..291ee19469a 100644
--- a/hook.h
+++ b/hook.h
@@ -37,6 +37,13 @@ struct hook_cb_data {
 	struct run_hooks_opt *options;
 };
 
+/*
+ * Returns the path to the hook file, or NULL if the hook is missing
+ * or disabled. Note that this points to static storage that will be
+ * overwritten by further calls to find_hook and run_hook_*.
+ */
+const char *find_hook(const char *name);
+
 void run_hooks_opt_clear(struct run_hooks_opt *o);
 
 /*
diff --git a/refs.c b/refs.c
index 8c9490235ea..59be29cf081 100644
--- a/refs.c
+++ b/refs.c
@@ -10,6 +10,7 @@
 #include "refs.h"
 #include "refs/refs-internal.h"
 #include "run-command.h"
+#include "hook.h"
 #include "object-store.h"
 #include "object.h"
 #include "tag.h"
diff --git a/run-command.c b/run-command.c
index be6bc128cd9..82fdf296569 100644
--- a/run-command.c
+++ b/run-command.c
@@ -8,6 +8,7 @@
 #include "string-list.h"
 #include "quote.h"
 #include "config.h"
+#include "hook.h"
 
 void child_process_init(struct child_process *child)
 {
@@ -1320,40 +1321,6 @@ int async_with_fork(void)
 #endif
 }
 
-const char *find_hook(const char *name)
-{
-	static struct strbuf path = STRBUF_INIT;
-
-	strbuf_reset(&path);
-	strbuf_git_path(&path, "hooks/%s", name);
-	if (access(path.buf, X_OK) < 0) {
-		int err = errno;
-
-#ifdef STRIP_EXTENSION
-		strbuf_addstr(&path, STRIP_EXTENSION);
-		if (access(path.buf, X_OK) >= 0)
-			return path.buf;
-		if (errno == EACCES)
-			err = errno;
-#endif
-
-		if (err == EACCES && advice_ignored_hook) {
-			static struct string_list advise_given = STRING_LIST_INIT_DUP;
-
-			if (!string_list_lookup(&advise_given, name)) {
-				string_list_insert(&advise_given, name);
-				advise(_("The '%s' hook was ignored because "
-					 "it's not set as executable.\n"
-					 "You can disable this warning with "
-					 "`git config advice.ignoredHook false`."),
-				       path.buf);
-			}
-		}
-		return NULL;
-	}
-	return path.buf;
-}
-
 int run_hook_ve(const char *const *env, const char *name, va_list args)
 {
 	struct child_process hook = CHILD_PROCESS_INIT;
diff --git a/run-command.h b/run-command.h
index d08414a92e7..b58531a7eb3 100644
--- a/run-command.h
+++ b/run-command.h
@@ -201,13 +201,6 @@ int finish_command_in_signal(struct child_process *);
  */
 int run_command(struct child_process *);
 
-/*
- * Returns the path to the hook file, or NULL if the hook is missing
- * or disabled. Note that this points to static storage that will be
- * overwritten by further calls to find_hook and run_hook_*.
- */
-const char *find_hook(const char *name);
-
 /**
  * Run a hook.
  * The first argument is a pathname to an index file, or NULL
diff --git a/sequencer.c b/sequencer.c
index 0bec01cf38e..3de479f90e1 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -8,6 +8,7 @@
 #include "sequencer.h"
 #include "tag.h"
 #include "run-command.h"
+#include "hook.h"
 #include "exec-cmd.h"
 #include "utf8.h"
 #include "cache-tree.h"
diff --git a/transport.c b/transport.c
index 6cf3da19ebd..e6c46adf60c 100644
--- a/transport.c
+++ b/transport.c
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "transport.h"
 #include "run-command.h"
+#include "hook.h"
 #include "pkt-line.h"
 #include "fetch-pack.h"
 #include "remote.h"
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 05/31] hook.c: add a hook_exists() wrapper and use it in bugreport.c
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (3 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 04/31] run-command.h: move find_hook() to hook.h Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 06/31] gc: use hook library for pre-auto-gc hook Ævar Arnfjörð Bjarmason
                         ` (27 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Add a boolean version of the find_hook() function for those callers
who are only interested in checking whether the hook exists, not what
the path to it is.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/bugreport.c | 2 +-
 hook.c              | 5 ++++-
 hook.h              | 5 +++++
 3 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/builtin/bugreport.c b/builtin/bugreport.c
index 596f079a7f9..941c8d5e270 100644
--- a/builtin/bugreport.c
+++ b/builtin/bugreport.c
@@ -82,7 +82,7 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 
 	for (i = 0; i < ARRAY_SIZE(hook); i++)
-		if (find_hook(hook[i]))
+		if (hook_exists(hook[i]))
 			strbuf_addf(hook_info, "%s\n", hook[i]);
 }
 
diff --git a/hook.c b/hook.c
index 06842f50e5c..c7da273822d 100644
--- a/hook.c
+++ b/hook.c
@@ -36,7 +36,10 @@ const char *find_hook(const char *name)
 	return path.buf;
 }
 
-
+int hook_exists(const char *name)
+{
+	return !!find_hook(name);
+}
 
 void run_hooks_opt_clear(struct run_hooks_opt *o)
 {
diff --git a/hook.h b/hook.h
index 291ee19469a..cbda7746a5d 100644
--- a/hook.h
+++ b/hook.h
@@ -44,6 +44,11 @@ struct hook_cb_data {
  */
 const char *find_hook(const char *name);
 
+/*
+ * A boolean version of find_hook()
+ */
+int hook_exists(const char *hookname);
+
 void run_hooks_opt_clear(struct run_hooks_opt *o);
 
 /*
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 06/31] gc: use hook library for pre-auto-gc hook
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (4 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 05/31] hook.c: add a hook_exists() wrapper and use it in bugreport.c Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 07/31] rebase: teach pre-rebase to use hook.h Ævar Arnfjörð Bjarmason
                         ` (26 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Using the hook.h library instead of the run-command.h library to run
pre-auto-gc means that those hooks can be set up in config files, as
well as in the hookdir. pre-auto-gc is called only from builtin/gc.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/gc.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/builtin/gc.c b/builtin/gc.c
index f05d2f0a1ac..a12641a691d 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -32,6 +32,7 @@
 #include "remote.h"
 #include "object-store.h"
 #include "exec-cmd.h"
+#include "hook.h"
 
 #define FAILED_RUN "failed to run %s"
 
@@ -348,6 +349,8 @@ static void add_repack_incremental_option(void)
 
 static int need_to_gc(void)
 {
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+
 	/*
 	 * Setting gc.auto to 0 or negative can disable the
 	 * automatic gc.
@@ -394,8 +397,11 @@ static int need_to_gc(void)
 	else
 		return 0;
 
-	if (run_hook_le(NULL, "pre-auto-gc", NULL))
+	if (run_hooks("pre-auto-gc", &hook_opt)) {
+		run_hooks_opt_clear(&hook_opt);
 		return 0;
+	}
+	run_hooks_opt_clear(&hook_opt);
 	return 1;
 }
 
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 07/31] rebase: teach pre-rebase to use hook.h
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (5 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 06/31] gc: use hook library for pre-auto-gc hook Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 08/31] am: convert applypatch hooks to use config Ævar Arnfjörð Bjarmason
                         ` (25 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Move the pre-rebase hook away from run-command.h to and over to the
new hook.h library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/rebase.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/builtin/rebase.c b/builtin/rebase.c
index 12f093121d9..2081f6fa8db 100644
--- a/builtin/rebase.c
+++ b/builtin/rebase.c
@@ -28,6 +28,7 @@
 #include "sequencer.h"
 #include "rebase-interactive.h"
 #include "reset.h"
+#include "hook.h"
 
 #define DEFAULT_REFLOG_ACTION "rebase"
 
@@ -1313,6 +1314,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	char *squash_onto_name = NULL;
 	int reschedule_failed_exec = -1;
 	int allow_preemptive_ff = 1;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
 	struct option builtin_rebase_options[] = {
 		OPT_STRING(0, "onto", &options.onto_name,
 			   N_("revision"),
@@ -2022,10 +2024,13 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	}
 
 	/* If a hook exists, give it a chance to interrupt*/
+	strvec_pushl(&hook_opt.args, options.upstream_arg, argc ? argv[0] : NULL, NULL);
 	if (!ok_to_skip_pre_rebase &&
-	    run_hook_le(NULL, "pre-rebase", options.upstream_arg,
-			argc ? argv[0] : NULL, NULL))
+	    run_hooks("pre-rebase", &hook_opt)) {
+		run_hooks_opt_clear(&hook_opt);
 		die(_("The pre-rebase hook refused to rebase."));
+	}
+	run_hooks_opt_clear(&hook_opt);
 
 	if (options.flags & REBASE_DIFFSTAT) {
 		struct diff_options opts;
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 08/31] am: convert applypatch hooks to use config
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (6 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 07/31] rebase: teach pre-rebase to use hook.h Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 09/31] hooks: convert 'post-checkout' hook to hook library Ævar Arnfjörð Bjarmason
                         ` (24 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Teach pre-applypatch, post-applypatch, and applypatch-msg to use the
hook.h library instead of the run-command.h library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/am.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index 1c8a5489035..9e9c1b5e9f2 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -445,9 +445,12 @@ static void am_destroy(const struct am_state *state)
 static int run_applypatch_msg_hook(struct am_state *state)
 {
 	int ret;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 
 	assert(state->msg);
-	ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL);
+	strvec_push(&opt.args, am_path(state, "final-commit"));
+	ret = run_hooks("applypatch-msg", &opt);
+	run_hooks_opt_clear(&opt);
 
 	if (!ret) {
 		FREE_AND_NULL(state->msg);
@@ -1607,9 +1610,13 @@ static void do_commit(const struct am_state *state)
 	struct commit_list *parents = NULL;
 	const char *reflog_msg, *author, *committer = NULL;
 	struct strbuf sb = STRBUF_INIT;
+	struct run_hooks_opt hook_opt_pre = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt_post = RUN_HOOKS_OPT_INIT;
 
-	if (run_hook_le(NULL, "pre-applypatch", NULL))
+	if (run_hooks("pre-applypatch", &hook_opt_pre)) {
+		run_hooks_opt_clear(&hook_opt_pre);
 		exit(1);
+	}
 
 	if (write_cache_as_tree(&tree, 0, NULL))
 		die(_("git write-tree failed to write a tree"));
@@ -1660,8 +1667,10 @@ static void do_commit(const struct am_state *state)
 		fclose(fp);
 	}
 
-	run_hook_le(NULL, "post-applypatch", NULL);
+	run_hooks("post-applypatch", &hook_opt_post);
 
+	run_hooks_opt_clear(&hook_opt_pre);
+	run_hooks_opt_clear(&hook_opt_post);
 	strbuf_release(&sb);
 }
 
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 09/31] hooks: convert 'post-checkout' hook to hook library
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (7 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 08/31] am: convert applypatch hooks to use config Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 10/31] merge: use config-based hooks for post-merge hook Ævar Arnfjörð Bjarmason
                         ` (23 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Move the running of the 'post-checkout' hook away from run-command.h
to the new hook.h library. For "worktree" this requires a change to it
to run the hooks from a given directory.

We could strictly speaking skip the "absolute_path" flag and just
check if "dir" is specified, but let's split them up for clarity, as
well as for any future user who'd like to set "dir" but not implicitly
change the argument to an absolute path.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/checkout.c | 17 ++++++++++++-----
 builtin/clone.c    |  7 +++++--
 builtin/worktree.c | 30 ++++++++++++++----------------
 hook.c             |  8 ++++++++
 hook.h             |  9 +++++++++
 read-cache.c       |  1 +
 reset.c            | 15 +++++++++++----
 7 files changed, 60 insertions(+), 27 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index f4cd7747d35..6205ace09f6 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "diff.h"
 #include "dir.h"
+#include "hook.h"
 #include "ll-merge.h"
 #include "lockfile.h"
 #include "merge-recursive.h"
@@ -106,13 +107,19 @@ struct branch_info {
 static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit,
 			      int changed)
 {
-	return run_hook_le(NULL, "post-checkout",
-			   oid_to_hex(old_commit ? &old_commit->object.oid : null_oid()),
-			   oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()),
-			   changed ? "1" : "0", NULL);
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	int rc;
+
 	/* "new_commit" can be NULL when checking out from the index before
 	   a commit exists. */
-
+	strvec_pushl(&opt.args,
+		     oid_to_hex(old_commit ? &old_commit->object.oid : null_oid()),
+		     oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()),
+		     changed ? "1" : "0",
+		     NULL);
+	rc = run_hooks("post-checkout", &opt);
+	run_hooks_opt_clear(&opt);
+	return rc;
 }
 
 static int update_some(const struct object_id *oid, struct strbuf *base,
diff --git a/builtin/clone.c b/builtin/clone.c
index eeb74c0217c..6687025bea5 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -32,6 +32,7 @@
 #include "connected.h"
 #include "packfile.h"
 #include "list-objects-filter-options.h"
+#include "hook.h"
 
 /*
  * Overall FIXMEs:
@@ -775,6 +776,7 @@ static int checkout(int submodule_progress)
 	struct tree *tree;
 	struct tree_desc t;
 	int err = 0;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
 
 	if (option_no_checkout)
 		return 0;
@@ -820,8 +822,9 @@ static int checkout(int submodule_progress)
 	if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
 		die(_("unable to write new index file"));
 
-	err |= run_hook_le(NULL, "post-checkout", oid_to_hex(null_oid()),
-			   oid_to_hex(&oid), "1", NULL);
+	strvec_pushl(&hook_opt.args, oid_to_hex(null_oid()), oid_to_hex(&oid), "1", NULL);
+	err |= run_hooks("post-checkout", &hook_opt);
+	run_hooks_opt_clear(&hook_opt);
 
 	if (!err && (option_recurse_submodules.nr > 0)) {
 		struct strvec args = STRVEC_INIT;
diff --git a/builtin/worktree.c b/builtin/worktree.c
index b1350640fed..2ad26a76f4c 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -382,22 +382,20 @@ static int add_worktree(const char *path, const char *refname,
 	 * is_junk is cleared, but do return appropriate code when hook fails.
 	 */
 	if (!ret && opts->checkout) {
-		const char *hook = find_hook("post-checkout");
-		if (hook) {
-			const char *env[] = { "GIT_DIR", "GIT_WORK_TREE", NULL };
-			cp.git_cmd = 0;
-			cp.no_stdin = 1;
-			cp.stdout_to_stderr = 1;
-			cp.dir = path;
-			cp.env = env;
-			cp.argv = NULL;
-			cp.trace2_hook_name = "post-checkout";
-			strvec_pushl(&cp.args, absolute_path(hook),
-				     oid_to_hex(null_oid()),
-				     oid_to_hex(&commit->object.oid),
-				     "1", NULL);
-			ret = run_command(&cp);
-		}
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
+		strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
+		strvec_pushl(&opt.args,
+			     oid_to_hex(null_oid()),
+			     oid_to_hex(&commit->object.oid),
+			     "1",
+			     NULL);
+		opt.dir = path;
+		opt.absolute_path = 1;
+
+		ret = run_hooks("post-checkout", &opt);
+
+		run_hooks_opt_clear(&opt);
 	}
 
 	strvec_clear(&child_env);
diff --git a/hook.c b/hook.c
index c7da273822d..51337f9798f 100644
--- a/hook.c
+++ b/hook.c
@@ -62,6 +62,7 @@ static int pick_next_hook(struct child_process *cp,
 	cp->env = hook_cb->options->env.v;
 	cp->stdout_to_stderr = 1;
 	cp->trace2_hook_name = hook_cb->hook_name;
+	cp->dir = hook_cb->options->dir;
 
 	/* add command */
 	strvec_push(&cp->args, run_me->hook_path);
@@ -111,6 +112,7 @@ static int notify_hook_finished(int result,
 int run_found_hooks(const char *hook_name, const char *hook_path,
 		    struct run_hooks_opt *options)
 {
+	struct strbuf abs_path = STRBUF_INIT;
 	struct hook my_hook = {
 		.hook_path = hook_path,
 	};
@@ -119,6 +121,10 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 		.hook_name = hook_name,
 		.options = options,
 	};
+	if (options->absolute_path) {
+		strbuf_add_absolute_path(&abs_path, hook_path);
+		my_hook.hook_path = abs_path.buf;
+	}
 	cb_data.run_me = &my_hook;
 
 	if (options->jobs != 1)
@@ -131,6 +137,8 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 				   &cb_data,
 				   "hook",
 				   hook_name);
+	if (options->absolute_path)
+		strbuf_release(&abs_path);
 
 	return cb_data.rc;
 }
diff --git a/hook.h b/hook.h
index cbda7746a5d..2d7724bbb50 100644
--- a/hook.h
+++ b/hook.h
@@ -19,6 +19,15 @@ struct run_hooks_opt
 
 	/* Number of threads to parallelize across */
 	int jobs;
+
+	/* Resolve and run the "absolute_path(hook)" instead of
+	 * "hook". Used for "git worktree" hooks
+	 */
+	int absolute_path;
+
+	/* Path to initial working directory for subprocess */
+	const char *dir;
+
 };
 
 #define RUN_HOOKS_OPT_INIT { \
diff --git a/read-cache.c b/read-cache.c
index 1b3c2eb408b..775e970402c 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -26,6 +26,7 @@
 #include "thread-utils.h"
 #include "progress.h"
 #include "sparse-index.h"
+#include "hook.h"
 
 /* Mask for the name length in ce_flags in the on-disk index */
 
diff --git a/reset.c b/reset.c
index 4bea758053b..e6af33b901c 100644
--- a/reset.c
+++ b/reset.c
@@ -7,6 +7,7 @@
 #include "tree-walk.h"
 #include "tree.h"
 #include "unpack-trees.h"
+#include "hook.h"
 
 int reset_head(struct repository *r, struct object_id *oid, const char *action,
 	       const char *switch_to_branch, unsigned flags,
@@ -126,10 +127,16 @@ int reset_head(struct repository *r, struct object_id *oid, const char *action,
 			ret = create_symref("HEAD", switch_to_branch,
 					    reflog_head);
 	}
-	if (run_hook)
-		run_hook_le(NULL, "post-checkout",
-			    oid_to_hex(orig ? orig : null_oid()),
-			    oid_to_hex(oid), "1", NULL);
+	if (run_hook) {
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		strvec_pushl(&opt.args,
+			     oid_to_hex(orig ? orig : null_oid()),
+			     oid_to_hex(oid),
+			     "1",
+			     NULL);
+		run_hooks("post-checkout", &opt);
+		run_hooks_opt_clear(&opt);
+	}
 
 leave_reset_head:
 	strbuf_release(&msg);
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 10/31] merge: use config-based hooks for post-merge hook
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (8 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 09/31] hooks: convert 'post-checkout' hook to hook library Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 11/31] git hook run: add an --ignore-missing flag Ævar Arnfjörð Bjarmason
                         ` (22 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Teach post-merge to use the hook.h library instead of the run-command.h
library to run hooks. This means that post-merge hooks can come from the
config as well as from the hookdir. post-merge is invoked only from
builtin/merge.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/merge.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/builtin/merge.c b/builtin/merge.c
index 5a10f6e3c96..a9363b94065 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -448,6 +448,7 @@ static void finish(struct commit *head_commit,
 		   const struct object_id *new_head, const char *msg)
 {
 	struct strbuf reflog_message = STRBUF_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	const struct object_id *head = &head_commit->object.oid;
 
 	if (!msg)
@@ -489,7 +490,9 @@ static void finish(struct commit *head_commit,
 	}
 
 	/* Run a post-merge hook */
-	run_hook_le(NULL, "post-merge", squash ? "1" : "0", NULL);
+	strvec_push(&opt.args, squash ? "1" : "0");
+	run_hooks("post-merge", &opt);
+	run_hooks_opt_clear(&opt);
 
 	apply_autostash(git_path_merge_autostash(the_repository));
 	strbuf_release(&reflog_message);
@@ -849,7 +852,7 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 	 * and write it out as a tree.  We must do this before we invoke
 	 * the editor and after we invoke run_status above.
 	 */
-	if (find_hook("pre-merge-commit"))
+	if (hook_exists("pre-merge-commit"))
 		discard_cache();
 	read_cache_from(index_file);
 	strbuf_addbuf(&msg, &merge_msg);
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 11/31] git hook run: add an --ignore-missing flag
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (9 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 10/31] merge: use config-based hooks for post-merge hook Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 12/31] send-email: use 'git hook run' for 'sendemail-validate' Ævar Arnfjörð Bjarmason
                         ` (21 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

For certain one-shot hooks we'd like to optimistically run them, and
not complain if they don't exist. This will be used by send-email in a
subsequent commit.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-hook.txt | 10 +++++++++-
 builtin/hook.c             |  5 +++++
 t/t1800-hook.sh            |  5 +++++
 3 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 902b9cffaef..1528c860cf1 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -8,7 +8,7 @@ git-hook - run git hooks
 SYNOPSIS
 --------
 [verse]
-'git hook' run <hook-name> [-- <hook-args>]
+'git hook' run [--ignore-missing] <hook-name> [-- <hook-args>]
 
 DESCRIPTION
 -----------
@@ -27,6 +27,14 @@ run::
 	"--end-of-options"). See "OPTIONS" below for the arguments
 	this accepts.
 
+OPTIONS
+-------
+
+--ignore-missing::
+	Ignore any missing hook by quietly returning zero. Used for
+	tools that want to do a blind one-shot run of a hook that may
+	or may not be present.
+
 SEE ALSO
 --------
 linkgit:githooks[5]
diff --git a/builtin/hook.c b/builtin/hook.c
index 1b1a594fd00..275dd5b0ed0 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -16,10 +16,13 @@ static int run(int argc, const char **argv, const char *prefix)
 	int i;
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	int rc = 0;
+	int ignore_missing = 0;
 	const char *hook_name;
 	const char *hook_path;
 
 	struct option run_options[] = {
+		OPT_BOOL(0, "ignore-missing", &ignore_missing,
+			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
 		OPT_END(),
 	};
 
@@ -42,6 +45,8 @@ static int run(int argc, const char **argv, const char *prefix)
 	hook_name = argv[1];
 	hook_path = find_hook(hook_name);
 	if (!hook_path) {
+		if (ignore_missing)
+			return 0;
 		error("cannot find a hook named %s", hook_name);
 		return 1;
 	}
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
index 7cacd27c748..9077afa1ed9 100755
--- a/t/t1800-hook.sh
+++ b/t/t1800-hook.sh
@@ -16,6 +16,11 @@ test_expect_success 'git hook run -- nonexistent hook' '
 	test_cmp stderr.expect stderr.actual
 '
 
+test_expect_success 'git hook run -- nonexistent hook with --ignore-missing' '
+	git hook run --ignore-missing does-not-exist 2>stderr.actual &&
+	test_must_be_empty stderr.actual
+'
+
 test_expect_success 'git hook run -- basic' '
 	write_script .git/hooks/test-hook <<-EOF &&
 	echo Test hook
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 12/31] send-email: use 'git hook run' for 'sendemail-validate'
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (10 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 11/31] git hook run: add an --ignore-missing flag Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 13/31] git-p4: use 'git hook' to run hooks Ævar Arnfjörð Bjarmason
                         ` (20 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Change the "sendmail-validate" hook to be run via the "git hook run"
wrapper instead of via a direct invocation.

This is the smallest possibly change to get "send-email" using "git
hook run". We still check the hook itself with "-x", and set a
"GIT_DIR" variable, both of which are asserted by our tests. We'll
need to get rid of this special behavior if we start running N hooks,
but for now let's be as close to bug-for-bug compatible as possible.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-send-email.perl   | 20 ++++++++++++--------
 t/t9001-send-email.sh |  4 ++--
 2 files changed, 14 insertions(+), 10 deletions(-)

diff --git a/git-send-email.perl b/git-send-email.perl
index 25be2ebd2af..2ab8dfdbded 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -213,13 +213,13 @@ sub format_2822_time {
 my $editor;
 
 sub system_or_msg {
-	my ($args, $msg) = @_;
+	my ($args, $msg, $cmd_name) = @_;
 	system(@$args);
 	my $signalled = $? & 127;
 	my $exit_code = $? >> 8;
 	return unless $signalled or $exit_code;
 
-	my @sprintf_args = ($args->[0], $exit_code);
+	my @sprintf_args = ($cmd_name ? $cmd_name : $args->[0], $exit_code);
 	if (defined $msg) {
 		# Quiet the 'redundant' warning category, except we
 		# need to support down to Perl 5.8, so we can't do a
@@ -1959,9 +1959,9 @@ sub validate_patch {
 	my ($fn, $xfer_encoding) = @_;
 
 	if ($repo) {
+		my $hook_name = 'sendemail-validate';
 		my $hooks_path = $repo->command_oneline('rev-parse', '--git-path', 'hooks');
-		my $validate_hook = catfile($hooks_path,
-					    'sendemail-validate');
+		my $validate_hook = catfile($hooks_path, $hook_name);
 		my $hook_error;
 		if (-x $validate_hook) {
 			my $target = abs_path($fn);
@@ -1970,13 +1970,17 @@ sub validate_patch {
 			chdir($repo->wc_path() or $repo->repo_path())
 				or die("chdir: $!");
 			local $ENV{"GIT_DIR"} = $repo->repo_path();
-			$hook_error = system_or_msg([$validate_hook, $target]);
+			my @validate_hook = ("git", "hook", "run", "--ignore-missing", $hook_name, "--", $target);
+			$hook_error = system_or_msg(\@validate_hook, undef,
+						       "git hook run $hook_name -- <patch>");
 			chdir($cwd_save) or die("chdir: $!");
 		}
 		if ($hook_error) {
-			die sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
-				       "%s\n" .
-				       "warning: no patches were sent\n"), $fn, $hook_error);
+			$hook_error = sprintf(__("fatal: %s: rejected by %s hook\n" .
+						 $hook_error . "\n" .
+						 "warning: no patches were sent\n"),
+					      $fn, $hook_name);
+			die $hook_error;
 		}
 	}
 
diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index 3b7540050ca..35b513c015f 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -539,7 +539,7 @@ test_expect_success $PREREQ "--validate respects relative core.hooksPath path" '
 	test_path_is_file my-hooks.ran &&
 	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
-	fatal: command '"'"'my-hooks/sendemail-validate'"'"' died with exit code 1
+	fatal: command '"'"'git hook run sendemail-validate -- <patch>'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
@@ -558,7 +558,7 @@ test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" '
 	test_path_is_file my-hooks.ran &&
 	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
-	fatal: command '"'"'$hooks_path/sendemail-validate'"'"' died with exit code 1
+	fatal: command '"'"'git hook run sendemail-validate -- <patch>'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 13/31] git-p4: use 'git hook' to run hooks
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (11 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 12/31] send-email: use 'git hook run' for 'sendemail-validate' Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 14/31] commit: use hook.h to execute hooks Ævar Arnfjörð Bjarmason
                         ` (19 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Instead of duplicating the behavior of run-command.h:run_hook_le() in
Python, we can directly call 'git hook run'. We emulate the existence
check with the --ignore-missing flag.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-p4.py | 72 ++++++-------------------------------------------------
 1 file changed, 7 insertions(+), 65 deletions(-)

diff --git a/git-p4.py b/git-p4.py
index d34a1946b75..e76d8df3139 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -207,71 +207,13 @@ def decode_path(path):
         return path
 
 def run_git_hook(cmd, param=[]):
-    """Execute a hook if the hook exists."""
-    if verbose:
-        sys.stderr.write("Looking for hook: %s\n" % cmd)
-        sys.stderr.flush()
-
-    hooks_path = gitConfig("core.hooksPath")
-    if len(hooks_path) <= 0:
-        hooks_path = os.path.join(os.environ["GIT_DIR"], "hooks")
-
-    if not isinstance(param, list):
-        param=[param]
-
-    # resolve hook file name, OS depdenent
-    hook_file = os.path.join(hooks_path, cmd)
-    if platform.system() == 'Windows':
-        if not os.path.isfile(hook_file):
-            # look for the file with an extension
-            files = glob.glob(hook_file + ".*")
-            if not files:
-                return True
-            files.sort()
-            hook_file = files.pop()
-            while hook_file.upper().endswith(".SAMPLE"):
-                # The file is a sample hook. We don't want it
-                if len(files) > 0:
-                    hook_file = files.pop()
-                else:
-                    return True
-
-    if not os.path.isfile(hook_file) or not os.access(hook_file, os.X_OK):
-        return True
-
-    return run_hook_command(hook_file, param) == 0
-
-def run_hook_command(cmd, param):
-    """Executes a git hook command
-       cmd = the command line file to be executed. This can be
-       a file that is run by OS association.
-
-       param = a list of parameters to pass to the cmd command
-
-       On windows, the extension is checked to see if it should
-       be run with the Git for Windows Bash shell.  If there
-       is no file extension, the file is deemed a bash shell
-       and will be handed off to sh.exe. Otherwise, Windows
-       will be called with the shell to handle the file assocation.
-
-       For non Windows operating systems, the file is called
-       as an executable.
-    """
-    cli = [cmd] + param
-    use_shell = False
-    if platform.system() == 'Windows':
-        (root,ext) = os.path.splitext(cmd)
-        if ext == "":
-            exe_path = os.environ.get("EXEPATH")
-            if exe_path is None:
-                exe_path = ""
-            else:
-                exe_path = os.path.join(exe_path, "bin")
-            cli = [os.path.join(exe_path, "SH.EXE")] + cli
-        else:
-            use_shell = True
-    return subprocess.call(cli, shell=use_shell)
-
+    """args are specified with -a <arg> -a <arg> -a <arg>"""
+    args = ['git', 'hook', 'run', '--ignore-missing', cmd]
+    if param:
+        args.append("--")
+        for p in param:
+            args.append(p)
+    return subprocess.call(args) == 0
 
 def write_pipe(c, stdin):
     if verbose:
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 14/31] commit: use hook.h to execute hooks
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (12 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 13/31] git-p4: use 'git hook' to run hooks Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 15/31] read-cache: convert post-index-change hook to use config Ævar Arnfjörð Bjarmason
                         ` (18 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Teach run_commit_hook() to call hook.h instead of run-command.h. This
covers 'pre-commit', 'commit-msg', and
'prepare-commit-msg'.

Additionally, ask the hook library - not run-command - whether any
hooks will be run, as it's possible hooks may exist in the config but
not the hookdir.

Because all but 'post-commit' hooks are expected to make some state
change, force all but 'post-commit' hook to run in series. 'post-commit'
"is meant primarily for notification, and cannot affect the outcome of
`git commit`," so it is fine to run in parallel.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/commit.c |  2 +-
 commit.c         | 16 ++++++++++------
 sequencer.c      |  2 +-
 3 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index f1aafd67d46..dad4e565443 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1045,7 +1045,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (!no_verify && find_hook("pre-commit")) {
+	if (!no_verify && hook_exists("pre-commit")) {
 		/*
 		 * Re-read the index as pre-commit hook could have updated it,
 		 * and write it out as a tree.  We must do this before we invoke
diff --git a/commit.c b/commit.c
index 8ea55a447fa..e8147a88fc6 100644
--- a/commit.c
+++ b/commit.c
@@ -21,6 +21,7 @@
 #include "commit-reach.h"
 #include "run-command.h"
 #include "shallow.h"
+#include "hook.h"
 
 static struct commit_extra_header *read_commit_extra_header_lines(const char *buf, size_t len, const char **);
 
@@ -1698,22 +1699,25 @@ size_t ignore_non_trailer(const char *buf, size_t len)
 int run_commit_hook(int editor_is_used, const char *index_file,
 		    const char *name, ...)
 {
-	struct strvec hook_env = STRVEC_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	va_list args;
+	const char *arg;
 	int ret;
-
-	strvec_pushf(&hook_env, "GIT_INDEX_FILE=%s", index_file);
+	strvec_pushf(&opt.env, "GIT_INDEX_FILE=%s", index_file);
 
 	/*
 	 * Let the hook know that no editor will be launched.
 	 */
 	if (!editor_is_used)
-		strvec_push(&hook_env, "GIT_EDITOR=:");
+		strvec_push(&opt.env, "GIT_EDITOR=:");
 
 	va_start(args, name);
-	ret = run_hook_ve(hook_env.v, name, args);
+	while ((arg = va_arg(args, const char *)))
+		strvec_push(&opt.args, arg);
 	va_end(args);
-	strvec_clear(&hook_env);
+
+	ret = run_hooks(name, &opt);
+	run_hooks_opt_clear(&opt);
 
 	return ret;
 }
diff --git a/sequencer.c b/sequencer.c
index 3de479f90e1..8f46984ffb7 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1446,7 +1446,7 @@ static int try_to_commit(struct repository *r,
 		}
 	}
 
-	if (find_hook("prepare-commit-msg")) {
+	if (hook_exists("prepare-commit-msg")) {
 		res = run_prepare_commit_msg_hook(r, msg, hook_commit);
 		if (res)
 			goto out;
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 15/31] read-cache: convert post-index-change hook to use config
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (13 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 14/31] commit: use hook.h to execute hooks Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 16/31] receive-pack: convert push-to-checkout hook to hook.h Ævar Arnfjörð Bjarmason
                         ` (17 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

By using hook.h instead of run-command.h to run, post-index-change hooks
can now be specified in the config in addition to the hookdir.
post-index-change is not run anywhere besides in read-cache.c.

This removes the last direct user of run_hook_ve(), so we can make the
function static now. It'll be removed entirely soon.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 read-cache.c  | 11 ++++++++---
 run-command.c |  2 +-
 run-command.h |  1 -
 3 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/read-cache.c b/read-cache.c
index 775e970402c..a17bc30f870 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -3132,6 +3132,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 {
 	int ret;
 	int was_full = !istate->sparse_index;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
 
 	ret = convert_to_sparse(istate);
 
@@ -3160,9 +3161,13 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 	else
 		ret = close_lock_file_gently(lock);
 
-	run_hook_le(NULL, "post-index-change",
-			istate->updated_workdir ? "1" : "0",
-			istate->updated_skipworktree ? "1" : "0", NULL);
+	strvec_pushl(&hook_opt.args,
+		     istate->updated_workdir ? "1" : "0",
+		     istate->updated_skipworktree ? "1" : "0",
+		     NULL);
+	run_hooks("post-index-change", &hook_opt);
+	run_hooks_opt_clear(&hook_opt);
+
 	istate->updated_workdir = 0;
 	istate->updated_skipworktree = 0;
 
diff --git a/run-command.c b/run-command.c
index 82fdf296569..eecdef5a0c8 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1321,7 +1321,7 @@ int async_with_fork(void)
 #endif
 }
 
-int run_hook_ve(const char *const *env, const char *name, va_list args)
+static int run_hook_ve(const char *const *env, const char *name, va_list args)
 {
 	struct child_process hook = CHILD_PROCESS_INIT;
 	const char *p;
diff --git a/run-command.h b/run-command.h
index b58531a7eb3..24ab5d63c4c 100644
--- a/run-command.h
+++ b/run-command.h
@@ -216,7 +216,6 @@ int run_command(struct child_process *);
  */
 LAST_ARG_MUST_BE_NULL
 int run_hook_le(const char *const *env, const char *name, ...);
-int run_hook_ve(const char *const *env, const char *name, va_list args);
 
 /*
  * Trigger an auto-gc
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 16/31] receive-pack: convert push-to-checkout hook to hook.h
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (14 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 15/31] read-cache: convert post-index-change hook to use config Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-06-02  0:51         ` Felipe Contreras
  2021-05-28 12:11       ` [PATCH 17/31] run-command: allow stdin for run_processes_parallel Ævar Arnfjörð Bjarmason
                         ` (16 subsequent siblings)
  32 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

By using hook.h instead of run-command.h to invoke push-to-checkout,
hooks can now be specified in the config as well as in the hookdir.
push-to-checkout is not called anywhere but in builtin/receive-pack.c.

This is the last user of the run_hook_le() API, so let's remove it
while we're at it, since run_hook_le() itself is the last user of
run_hook_ve() we can remove that too. The last direct user of
run_hook_le() was removed in the commit preceding this one.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/receive-pack.c | 14 ++++++++++----
 run-command.c          | 32 --------------------------------
 run-command.h          | 16 ----------------
 3 files changed, 10 insertions(+), 52 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 1e0e04c62fc..5248228ebfe 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1436,12 +1436,18 @@ static const char *push_to_checkout(unsigned char *hash,
 				    struct strvec *env,
 				    const char *work_tree)
 {
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
-	if (run_hook_le(env->v, push_to_checkout_hook,
-			hash_to_hex(hash), NULL))
+	strvec_pushv(&opt.env, env->v);
+	strvec_push(&opt.args, hash_to_hex(hash));
+	if (run_hooks(push_to_checkout_hook, &opt)) {
+		run_hooks_opt_clear(&opt);
 		return "push-to-checkout hook declined";
-	else
+	} else {
+		run_hooks_opt_clear(&opt);
 		return NULL;
+	}
 }
 
 static const char *update_worktree(unsigned char *sha1, const struct worktree *worktree)
@@ -1465,7 +1471,7 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 
 	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
 
-	if (!find_hook(push_to_checkout_hook))
+	if (!hook_exists(push_to_checkout_hook))
 		retval = push_to_deploy(sha1, &env, work_tree);
 	else
 		retval = push_to_checkout(sha1, &env, work_tree);
diff --git a/run-command.c b/run-command.c
index eecdef5a0c8..95c950a4a2b 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1321,38 +1321,6 @@ int async_with_fork(void)
 #endif
 }
 
-static int run_hook_ve(const char *const *env, const char *name, va_list args)
-{
-	struct child_process hook = CHILD_PROCESS_INIT;
-	const char *p;
-
-	p = find_hook(name);
-	if (!p)
-		return 0;
-
-	strvec_push(&hook.args, p);
-	while ((p = va_arg(args, const char *)))
-		strvec_push(&hook.args, p);
-	hook.env = env;
-	hook.no_stdin = 1;
-	hook.stdout_to_stderr = 1;
-	hook.trace2_hook_name = name;
-
-	return run_command(&hook);
-}
-
-int run_hook_le(const char *const *env, const char *name, ...)
-{
-	va_list args;
-	int ret;
-
-	va_start(args, name);
-	ret = run_hook_ve(env, name, args);
-	va_end(args);
-
-	return ret;
-}
-
 struct io_pump {
 	/* initialized by caller */
 	int fd;
diff --git a/run-command.h b/run-command.h
index 24ab5d63c4c..748d4fc2a72 100644
--- a/run-command.h
+++ b/run-command.h
@@ -201,22 +201,6 @@ int finish_command_in_signal(struct child_process *);
  */
 int run_command(struct child_process *);
 
-/**
- * Run a hook.
- * The first argument is a pathname to an index file, or NULL
- * if the hook uses the default index file or no index is needed.
- * The second argument is the name of the hook.
- * The further arguments correspond to the hook arguments.
- * The last argument has to be NULL to terminate the arguments list.
- * If the hook does not exist or is not executable, the return
- * value will be zero.
- * If it is executable, the hook will be executed and the exit
- * status of the hook is returned.
- * On execution, .stdout_to_stderr and .no_stdin will be set.
- */
-LAST_ARG_MUST_BE_NULL
-int run_hook_le(const char *const *env, const char *name, ...);
-
 /*
  * Trigger an auto-gc
  */
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 17/31] run-command: allow stdin for run_processes_parallel
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (15 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 16/31] receive-pack: convert push-to-checkout hook to hook.h Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 18/31] hook: support passing stdin to hooks Ævar Arnfjörð Bjarmason
                         ` (15 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

While it makes sense not to inherit stdin from the parent process to
avoid deadlocking, it's not necessary to completely ban stdin to
children. An informed user should be able to configure stdin safely. By
setting `some_child.process.no_stdin=1` before calling `get_next_task()`
we provide a reasonable default behavior but enable users to set up
stdin streaming for themselves during the callback.

`some_child.process.stdout_to_stderr`, however, remains unmodifiable by
`get_next_task()` - the rest of the run_processes_parallel() API depends
on child output in stderr.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 run-command.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/run-command.c b/run-command.c
index 95c950a4a2b..0bf771845e4 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1628,6 +1628,14 @@ static int pp_start_one(struct parallel_processes *pp)
 	if (i == pp->max_processes)
 		BUG("bookkeeping is hard");
 
+	/*
+	 * By default, do not inherit stdin from the parent process - otherwise,
+	 * all children would share stdin! Users may overwrite this to provide
+	 * something to the child's stdin by having their 'get_next_task'
+	 * callback assign 0 to .no_stdin and an appropriate integer to .in.
+	 */
+	pp->children[i].process.no_stdin = 1;
+
 	code = pp->get_next_task(&pp->children[i].process,
 				 &pp->children[i].err,
 				 pp->data,
@@ -1639,7 +1647,6 @@ static int pp_start_one(struct parallel_processes *pp)
 	}
 	pp->children[i].process.err = -1;
 	pp->children[i].process.stdout_to_stderr = 1;
-	pp->children[i].process.no_stdin = 1;
 
 	if (start_command(&pp->children[i].process)) {
 		code = pp->start_failure(&pp->children[i].err,
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 18/31] hook: support passing stdin to hooks
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (16 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 17/31] run-command: allow stdin for run_processes_parallel Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 19/31] am: convert 'post-rewrite' hook to hook.h Ævar Arnfjörð Bjarmason
                         ` (14 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Some hooks (such as post-rewrite) need to take input via stdin.
Previously, callers provided stdin to hooks by setting
run-command.h:child_process.in, which takes a FD. Callers would open the
file in question themselves before calling run-command(). However, since
we will now need to seek to the front of the file and read it again for
every hook which runs, hook.h:run_command() takes a path and handles FD
management itself. Since this file is opened for read only, it should
not prevent later parallel execution support.

On the frontend, this is supported by asking for a file path, rather
than by reading stdin. Reading directly from stdin would involve caching
the entire stdin (to memory or to disk) and reading it back from the
beginning to each hook. We'd want to support cases like insufficient
memory or storage for the file. While this may prove useful later, for
now the path of least resistance is to just ask the user to make this
interim file themselves.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-hook.txt |  7 ++++++-
 builtin/hook.c             |  4 +++-
 hook.c                     |  8 +++++++-
 hook.h                     |  2 ++
 t/t1800-hook.sh            | 18 ++++++++++++++++++
 5 files changed, 36 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 1528c860cf1..816b3eda460 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -8,7 +8,7 @@ git-hook - run git hooks
 SYNOPSIS
 --------
 [verse]
-'git hook' run [--ignore-missing] <hook-name> [-- <hook-args>]
+'git hook' run [--to-stdin=<path>] [--ignore-missing] <hook-name> [-- <hook-args>]
 
 DESCRIPTION
 -----------
@@ -30,6 +30,11 @@ run::
 OPTIONS
 -------
 
+--to-stdin::
+	For "run"; Specify a file which will be streamed into the
+	hook's stdin. The hook will receive the entire file from
+	beginning to EOF.
+
 --ignore-missing::
 	Ignore any missing hook by quietly returning zero. Used for
 	tools that want to do a blind one-shot run of a hook that may
diff --git a/builtin/hook.c b/builtin/hook.c
index 275dd5b0ed0..baaef4dce49 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -7,7 +7,7 @@
 #include "strvec.h"
 
 static const char * const builtin_hook_usage[] = {
-	N_("git hook run <hook-name> [-- <hook-args>]"),
+	N_("git hook run [--to-stdin=<path>] <hook-name> [-- <hook-args>]"),
 	NULL
 };
 
@@ -23,6 +23,8 @@ static int run(int argc, const char **argv, const char *prefix)
 	struct option run_options[] = {
 		OPT_BOOL(0, "ignore-missing", &ignore_missing,
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
+		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
+			   N_("file to read into hooks' stdin")),
 		OPT_END(),
 	};
 
diff --git a/hook.c b/hook.c
index 51337f9798f..daf39f61741 100644
--- a/hook.c
+++ b/hook.c
@@ -58,7 +58,13 @@ static int pick_next_hook(struct child_process *cp,
 	if (!run_me)
 		BUG("did we not return 1 in notify_hook_finished?");
 
-	cp->no_stdin = 1;
+	/* reopen the file for stdin; run_command closes it. */
+	if (hook_cb->options->path_to_stdin) {
+		cp->no_stdin = 0;
+		cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY);
+	} else {
+		cp->no_stdin = 1;
+	}
 	cp->env = hook_cb->options->env.v;
 	cp->stdout_to_stderr = 1;
 	cp->trace2_hook_name = hook_cb->hook_name;
diff --git a/hook.h b/hook.h
index 2d7724bbb50..74a8c76a94c 100644
--- a/hook.h
+++ b/hook.h
@@ -28,6 +28,8 @@ struct run_hooks_opt
 	/* Path to initial working directory for subprocess */
 	const char *dir;
 
+	/* Path to file which should be piped to stdin for each hook */
+	const char *path_to_stdin;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
index 9077afa1ed9..9e2dd64275c 100755
--- a/t/t1800-hook.sh
+++ b/t/t1800-hook.sh
@@ -137,4 +137,22 @@ test_expect_success 'set up a pre-commit hook in core.hooksPath' '
 	EOF
 '
 
+test_expect_success 'stdin to hooks' '
+	write_script .git/hooks/test-hook <<-\EOF &&
+	echo BEGIN stdin
+	cat
+	echo END stdin
+	EOF
+
+	cat >expect <<-EOF &&
+	BEGIN stdin
+	hello
+	END stdin
+	EOF
+
+	echo hello >input &&
+	git hook run --to-stdin=input test-hook 2>actual &&
+	test_cmp expect actual
+'
+
 test_done
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 19/31] am: convert 'post-rewrite' hook to hook.h
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (17 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 18/31] hook: support passing stdin to hooks Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 20/31] run-command: add stdin callback for parallelization Ævar Arnfjörð Bjarmason
                         ` (13 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/am.c | 18 +++++-------------
 1 file changed, 5 insertions(+), 13 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index 9e9c1b5e9f2..6e4f9c80360 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -467,23 +467,15 @@ static int run_applypatch_msg_hook(struct am_state *state)
  */
 static int run_post_rewrite_hook(const struct am_state *state)
 {
-	struct child_process cp = CHILD_PROCESS_INIT;
-	const char *hook = find_hook("post-rewrite");
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	int ret;
 
-	if (!hook)
-		return 0;
-
-	strvec_push(&cp.args, hook);
-	strvec_push(&cp.args, "rebase");
-
-	cp.in = xopen(am_path(state, "rewritten"), O_RDONLY);
-	cp.stdout_to_stderr = 1;
-	cp.trace2_hook_name = "post-rewrite";
+	strvec_push(&opt.args, "rebase");
+	opt.path_to_stdin = am_path(state, "rewritten");
 
-	ret = run_command(&cp);
+	ret = run_hooks("post-rewrite", &opt);
 
-	close(cp.in);
+	run_hooks_opt_clear(&opt);
 	return ret;
 }
 
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 20/31] run-command: add stdin callback for parallelization
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (18 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 19/31] am: convert 'post-rewrite' hook to hook.h Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 21/31] hook: provide stdin by string_list or callback Ævar Arnfjörð Bjarmason
                         ` (12 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

If a user of the run_processes_parallel() API wants to pipe a large
amount of information to stdin of each parallel command, that
information could exceed the buffer of the pipe allocated for that
process's stdin.  Generally this is solved by repeatedly writing to
child_process.in between calls to start_command() and finish_command();
run_processes_parallel() did not provide users an opportunity to access
child_process at that time.

Because the data might be extremely large (for example, a list of all
refs received during a push from a client) simply taking a string_list
or strbuf is not as scalable as using a callback; the rest of the
run_processes_parallel() API also uses callbacks, so making this feature
match the rest of the API reduces mental load on the user.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/fetch.c             |  1 +
 builtin/submodule--helper.c |  2 +-
 hook.c                      |  1 +
 run-command.c               | 54 +++++++++++++++++++++++++++++++++++--
 run-command.h               | 17 +++++++++++-
 submodule.c                 |  1 +
 t/helper/test-run-command.c | 31 ++++++++++++++++++---
 t/t0061-run-command.sh      | 30 +++++++++++++++++++++
 8 files changed, 129 insertions(+), 8 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index dfde96a4354..a07816b6504 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1817,6 +1817,7 @@ static int fetch_multiple(struct string_list *list, int max_children)
 		result = run_processes_parallel_tr2(max_children,
 						    &fetch_next_remote,
 						    &fetch_failed_to_start,
+						    NULL,
 						    &fetch_finished,
 						    &state,
 						    "fetch", "parallel/fetch");
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index d55f6262e9c..8d1e7310737 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2295,7 +2295,7 @@ static int update_submodules(struct submodule_update_clone *suc)
 	int i;
 
 	run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task,
-				   update_clone_start_failure,
+				   update_clone_start_failure, NULL,
 				   update_clone_task_finished, suc, "submodule",
 				   "parallel/update");
 
diff --git a/hook.c b/hook.c
index daf39f61741..5f6335bac3f 100644
--- a/hook.c
+++ b/hook.c
@@ -139,6 +139,7 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 	run_processes_parallel_tr2(options->jobs,
 				   pick_next_hook,
 				   notify_start_failure,
+				   NULL,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
diff --git a/run-command.c b/run-command.c
index 0bf771845e4..3392640f17b 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1493,6 +1493,7 @@ struct parallel_processes {
 
 	get_next_task_fn get_next_task;
 	start_failure_fn start_failure;
+	feed_pipe_fn feed_pipe;
 	task_finished_fn task_finished;
 
 	struct {
@@ -1520,6 +1521,13 @@ static int default_start_failure(struct strbuf *out,
 	return 0;
 }
 
+static int default_feed_pipe(struct strbuf *pipe,
+			     void *pp_cb,
+			     void *pp_task_cb)
+{
+	return 1;
+}
+
 static int default_task_finished(int result,
 				 struct strbuf *out,
 				 void *pp_cb,
@@ -1550,6 +1558,7 @@ static void pp_init(struct parallel_processes *pp,
 		    int n,
 		    get_next_task_fn get_next_task,
 		    start_failure_fn start_failure,
+		    feed_pipe_fn feed_pipe,
 		    task_finished_fn task_finished,
 		    void *data)
 {
@@ -1568,6 +1577,7 @@ static void pp_init(struct parallel_processes *pp,
 	pp->get_next_task = get_next_task;
 
 	pp->start_failure = start_failure ? start_failure : default_start_failure;
+	pp->feed_pipe = feed_pipe ? feed_pipe : default_feed_pipe;
 	pp->task_finished = task_finished ? task_finished : default_task_finished;
 
 	pp->nr_processes = 0;
@@ -1665,6 +1675,37 @@ static int pp_start_one(struct parallel_processes *pp)
 	return 0;
 }
 
+static void pp_buffer_stdin(struct parallel_processes *pp)
+{
+	int i;
+	struct strbuf sb = STRBUF_INIT;
+
+	/* Buffer stdin for each pipe. */
+	for (i = 0; i < pp->max_processes; i++) {
+		if (pp->children[i].state == GIT_CP_WORKING &&
+		    pp->children[i].process.in > 0) {
+			int done;
+			strbuf_reset(&sb);
+			done = pp->feed_pipe(&sb, pp->data,
+					      pp->children[i].data);
+			if (sb.len) {
+				if (write_in_full(pp->children[i].process.in,
+					      sb.buf, sb.len) < 0) {
+					if (errno != EPIPE)
+						die_errno("write");
+					done = 1;
+				}
+			}
+			if (done) {
+				close(pp->children[i].process.in);
+				pp->children[i].process.in = 0;
+			}
+		}
+	}
+
+	strbuf_release(&sb);
+}
+
 static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout)
 {
 	int i;
@@ -1729,6 +1770,7 @@ static int pp_collect_finished(struct parallel_processes *pp)
 		pp->nr_processes--;
 		pp->children[i].state = GIT_CP_FREE;
 		pp->pfd[i].fd = -1;
+		pp->children[i].process.in = 0;
 		child_process_init(&pp->children[i].process);
 
 		if (i != pp->output_owner) {
@@ -1762,6 +1804,7 @@ static int pp_collect_finished(struct parallel_processes *pp)
 int run_processes_parallel(int n,
 			   get_next_task_fn get_next_task,
 			   start_failure_fn start_failure,
+			   feed_pipe_fn feed_pipe,
 			   task_finished_fn task_finished,
 			   void *pp_cb)
 {
@@ -1770,7 +1813,9 @@ int run_processes_parallel(int n,
 	int spawn_cap = 4;
 	struct parallel_processes pp;
 
-	pp_init(&pp, n, get_next_task, start_failure, task_finished, pp_cb);
+	sigchain_push(SIGPIPE, SIG_IGN);
+
+	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, task_finished, pp_cb);
 	while (1) {
 		for (i = 0;
 		    i < spawn_cap && !pp.shutdown &&
@@ -1787,6 +1832,7 @@ int run_processes_parallel(int n,
 		}
 		if (!pp.nr_processes)
 			break;
+		pp_buffer_stdin(&pp);
 		pp_buffer_stderr(&pp, output_timeout);
 		pp_output(&pp);
 		code = pp_collect_finished(&pp);
@@ -1798,11 +1844,15 @@ int run_processes_parallel(int n,
 	}
 
 	pp_cleanup(&pp);
+
+	sigchain_pop(SIGPIPE);
+
 	return 0;
 }
 
 int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 			       start_failure_fn start_failure,
+			       feed_pipe_fn feed_pipe,
 			       task_finished_fn task_finished, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label)
 {
@@ -1812,7 +1862,7 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 				   ((n < 1) ? online_cpus() : n));
 
 	result = run_processes_parallel(n, get_next_task, start_failure,
-					task_finished, pp_cb);
+					feed_pipe, task_finished, pp_cb);
 
 	trace2_region_leave(tr2_category, tr2_label, NULL);
 
diff --git a/run-command.h b/run-command.h
index 748d4fc2a72..41e36d26cb1 100644
--- a/run-command.h
+++ b/run-command.h
@@ -419,6 +419,20 @@ typedef int (*start_failure_fn)(struct strbuf *out,
 				void *pp_cb,
 				void *pp_task_cb);
 
+/**
+ * This callback is called repeatedly on every child process who requests
+ * start_command() to create a pipe by setting child_process.in < 0.
+ *
+ * pp_cb is the callback cookie as passed into run_processes_parallel, and
+ * pp_task_cb is the callback cookie as passed into get_next_task_fn.
+ * The contents of 'send' will be read into the pipe and passed to the pipe.
+ *
+ * Return nonzero to close the pipe.
+ */
+typedef int (*feed_pipe_fn)(struct strbuf *pipe,
+			    void *pp_cb,
+			    void *pp_task_cb);
+
 /**
  * This callback is called on every child process that finished processing.
  *
@@ -453,10 +467,11 @@ typedef int (*task_finished_fn)(int result,
 int run_processes_parallel(int n,
 			   get_next_task_fn,
 			   start_failure_fn,
+			   feed_pipe_fn,
 			   task_finished_fn,
 			   void *pp_cb);
 int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn,
-			       task_finished_fn, void *pp_cb,
+			       feed_pipe_fn, task_finished_fn, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label);
 
 #endif
diff --git a/submodule.c b/submodule.c
index 0b1d9c1dde5..ea026a8195f 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1645,6 +1645,7 @@ int fetch_populated_submodules(struct repository *r,
 	run_processes_parallel_tr2(max_parallel_jobs,
 				   get_next_submodule,
 				   fetch_start_failure,
+				   NULL,
 				   fetch_finish,
 				   &spf,
 				   "submodule", "parallel/fetch");
diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c
index 7ae03dc7123..9348184d303 100644
--- a/t/helper/test-run-command.c
+++ b/t/helper/test-run-command.c
@@ -32,8 +32,13 @@ static int parallel_next(struct child_process *cp,
 		return 0;
 
 	strvec_pushv(&cp->args, d->argv);
+	cp->in = d->in;
+	cp->no_stdin = d->no_stdin;
 	strbuf_addstr(err, "preloaded output of a child\n");
 	number_callbacks++;
+
+	*task_cb = xmalloc(sizeof(int));
+	*(int*)(*task_cb) = 2;
 	return 1;
 }
 
@@ -55,6 +60,17 @@ static int task_finished(int result,
 	return 1;
 }
 
+static int test_stdin(struct strbuf *pipe, void *cb, void *task_cb)
+{
+	int *lines_remaining = task_cb;
+
+	if (*lines_remaining)
+		strbuf_addf(pipe, "sample stdin %d\n", --(*lines_remaining));
+
+	return !(*lines_remaining);
+}
+
+
 struct testsuite {
 	struct string_list tests, failed;
 	int next;
@@ -185,7 +201,7 @@ static int testsuite(int argc, const char **argv)
 		suite.tests.nr, max_jobs);
 
 	ret = run_processes_parallel(max_jobs, next_test, test_failed,
-				     test_finished, &suite);
+				     test_stdin, test_finished, &suite);
 
 	if (suite.failed.nr > 0) {
 		ret = 1;
@@ -413,15 +429,22 @@ int cmd__run_command(int argc, const char **argv)
 
 	if (!strcmp(argv[1], "run-command-parallel"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, &proc));
+					    NULL, NULL, NULL, &proc));
 
 	if (!strcmp(argv[1], "run-command-abort"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, task_finished, &proc));
+					    NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-no-jobs"))
 		exit(run_processes_parallel(jobs, no_job,
-					    NULL, task_finished, &proc));
+					    NULL, NULL, task_finished, &proc));
+
+	if (!strcmp(argv[1], "run-command-stdin")) {
+		proc.in = -1;
+		proc.no_stdin = 0;
+		exit (run_processes_parallel(jobs, parallel_next, NULL,
+					     test_stdin, NULL, &proc));
+	}
 
 	fprintf(stderr, "check usage\n");
 	return 1;
diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh
index 7d599675e35..87759482ad1 100755
--- a/t/t0061-run-command.sh
+++ b/t/t0061-run-command.sh
@@ -143,6 +143,36 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai
 	test_cmp expect actual
 '
 
+cat >expect <<-EOF
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+EOF
+
+test_expect_success 'run_command listens to stdin' '
+	write_script stdin-script <<-\EOF &&
+	echo "listening for stdin:"
+	while read line; do
+		echo "$line"
+	done
+	EOF
+	test-tool run-command run-command-stdin 2 ./stdin-script 2>actual &&
+	test_cmp expect actual
+'
+
 cat >expect <<-EOF
 preloaded output of a child
 asking for a quick stop
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 21/31] hook: provide stdin by string_list or callback
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (19 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 20/31] run-command: add stdin callback for parallelization Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-06-02  1:00         ` Felipe Contreras
  2021-05-28 12:11       ` [PATCH 22/31] hook: convert 'post-rewrite' hook in sequencer.c to hook.h Ævar Arnfjörð Bjarmason
                         ` (11 subsequent siblings)
  32 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

In cases where a hook requires only a small amount of information via
stdin, it should be simple for users to provide a string_list alone. But
in more complicated cases where the stdin is too large to hold in
memory, let's provide a callback the users can populate line after line
with instead.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 hook.c | 33 +++++++++++++++++++++++++++++++--
 hook.h | 27 +++++++++++++++++++++++++++
 2 files changed, 58 insertions(+), 2 deletions(-)

diff --git a/hook.c b/hook.c
index 5f6335bac3f..ac5e3988fec 100644
--- a/hook.c
+++ b/hook.c
@@ -47,6 +47,29 @@ void run_hooks_opt_clear(struct run_hooks_opt *o)
 	strvec_clear(&o->args);
 }
 
+int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb)
+{
+	int *item_idx;
+	struct hook *ctx = pp_task_cb;
+	struct hook_cb_data *hook_cb = pp_cb;
+	struct string_list *to_pipe = hook_cb->options->feed_pipe_ctx;
+
+	/* Bootstrap the state manager if necessary. */
+	if (!ctx->feed_pipe_cb_data) {
+		ctx->feed_pipe_cb_data = xmalloc(sizeof(unsigned int));
+		*(int*)ctx->feed_pipe_cb_data = 0;
+	}
+
+	item_idx = ctx->feed_pipe_cb_data;
+
+	if (*item_idx < to_pipe->nr) {
+		strbuf_addf(pipe, "%s\n", to_pipe->items[*item_idx].string);
+		(*item_idx)++;
+		return 0;
+	}
+	return 1;
+}
+
 static int pick_next_hook(struct child_process *cp,
 			  struct strbuf *out,
 			  void *pp_cb,
@@ -62,6 +85,10 @@ static int pick_next_hook(struct child_process *cp,
 	if (hook_cb->options->path_to_stdin) {
 		cp->no_stdin = 0;
 		cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY);
+	} else if (hook_cb->options->feed_pipe) {
+		/* ask for start_command() to make a pipe for us */
+		cp->in = -1;
+		cp->no_stdin = 0;
 	} else {
 		cp->no_stdin = 1;
 	}
@@ -114,7 +141,6 @@ static int notify_hook_finished(int result,
 	return 1;
 }
 
-
 int run_found_hooks(const char *hook_name, const char *hook_path,
 		    struct run_hooks_opt *options)
 {
@@ -139,7 +165,7 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 	run_processes_parallel_tr2(options->jobs,
 				   pick_next_hook,
 				   notify_start_failure,
-				   NULL,
+				   options->feed_pipe,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
@@ -157,6 +183,9 @@ int run_hooks(const char *hook_name, struct run_hooks_opt *options)
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
 
+	if (options->path_to_stdin && options->feed_pipe)
+		BUG("choose only one method to populate stdin");
+
 	hook_path = find_hook(hook_name);
 
 	/* Care about nonexistence? Use run_found_hooks() */
diff --git a/hook.h b/hook.h
index 74a8c76a94c..ff1697d1087 100644
--- a/hook.h
+++ b/hook.h
@@ -7,6 +7,12 @@
 struct hook {
 	/* The path to the hook */
 	const char *hook_path;
+
+	/*
+	 * Use this to keep state for your feed_pipe_fn if you are using
+	 * run_hooks_opt.feed_pipe. Otherwise, do not touch it.
+	 */
+	void *feed_pipe_cb_data;
 };
 
 struct run_hooks_opt
@@ -30,6 +36,19 @@ struct run_hooks_opt
 
 	/* Path to file which should be piped to stdin for each hook */
 	const char *path_to_stdin;
+
+	/*
+	 * Callback and state pointer to ask for more content to pipe to stdin.
+	 * Will be called repeatedly, for each hook. See
+	 * hook.c:pipe_from_stdin() for an example. Keep per-hook state in
+	 * hook.feed_pipe_cb_data (per process). Keep initialization context in
+	 * feed_pipe_ctx (shared by all processes).
+	 *
+	 * See 'pipe_from_string_list()' for info about how to specify a
+	 * string_list as the stdin input instead of writing your own handler.
+	 */
+	feed_pipe_fn feed_pipe;
+	void *feed_pipe_ctx;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
@@ -38,6 +57,14 @@ struct run_hooks_opt
 	.args = STRVEC_INIT, \
 }
 
+/*
+ * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
+ * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'.
+ * This will pipe each string in the list to stdin, separated by newlines.  (Do
+ * not inject your own newlines.)
+ */
+int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb);
+
 /*
  * Callback provided to feed_pipe_fn and consume_sideband_fn.
  */
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 22/31] hook: convert 'post-rewrite' hook in sequencer.c to hook.h
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (20 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 21/31] hook: provide stdin by string_list or callback Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 23/31] transport: convert pre-push hook to use config Ævar Arnfjörð Bjarmason
                         ` (10 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

By using 'hook.h' for 'post-rewrite', we simplify hook invocations by
not needing to put together our own 'struct child_process'.

The signal handling that's being removed by this commit now takes
place in run-command.h:run_processes_parallel(), so it is OK to remove
them here.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 sequencer.c | 81 ++++++++++++++++++++++-------------------------------
 1 file changed, 34 insertions(+), 47 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 8f46984ffb7..ec2761e47d9 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -35,6 +35,7 @@
 #include "commit-reach.h"
 #include "rebase-interactive.h"
 #include "reset.h"
+#include "string-list.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -1147,33 +1148,28 @@ int update_head_with_reflog(const struct commit *old_head,
 static int run_rewrite_hook(const struct object_id *oldoid,
 			    const struct object_id *newoid)
 {
-	struct child_process proc = CHILD_PROCESS_INIT;
-	const char *argv[3];
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct strbuf tmp = STRBUF_INIT;
+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int code;
-	struct strbuf sb = STRBUF_INIT;
 
-	argv[0] = find_hook("post-rewrite");
-	if (!argv[0])
-		return 0;
+	strvec_push(&opt.args, "amend");
 
-	argv[1] = "amend";
-	argv[2] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = "post-rewrite";
-
-	code = start_command(&proc);
-	if (code)
-		return code;
-	strbuf_addf(&sb, "%s %s\n", oid_to_hex(oldoid), oid_to_hex(newoid));
-	sigchain_push(SIGPIPE, SIG_IGN);
-	write_in_full(proc.in, sb.buf, sb.len);
-	close(proc.in);
-	strbuf_release(&sb);
-	sigchain_pop(SIGPIPE);
-	return finish_command(&proc);
+	strbuf_addf(&tmp,
+		    "%s %s",
+		    oid_to_hex(oldoid),
+		    oid_to_hex(newoid));
+	string_list_append(&to_stdin, tmp.buf);
+
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
+
+	code = run_hooks("post-rewrite", &opt);
+
+	run_hooks_opt_clear(&opt);
+	strbuf_release(&tmp);
+	string_list_clear(&to_stdin, 0);
+	return code;
 }
 
 void commit_post_rewrite(struct repository *r,
@@ -4527,30 +4523,21 @@ static int pick_commits(struct repository *r,
 		flush_rewritten_pending();
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
-			struct child_process child = CHILD_PROCESS_INIT;
-			const char *post_rewrite_hook =
-				find_hook("post-rewrite");
-
-			child.in = open(rebase_path_rewritten_list(), O_RDONLY);
-			child.git_cmd = 1;
-			strvec_push(&child.args, "notes");
-			strvec_push(&child.args, "copy");
-			strvec_push(&child.args, "--for-rewrite=rebase");
+			struct child_process notes_cp = CHILD_PROCESS_INIT;
+			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+
+			notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY);
+			notes_cp.git_cmd = 1;
+			strvec_push(&notes_cp.args, "notes");
+			strvec_push(&notes_cp.args, "copy");
+			strvec_push(&notes_cp.args, "--for-rewrite=rebase");
 			/* we don't care if this copying failed */
-			run_command(&child);
-
-			if (post_rewrite_hook) {
-				struct child_process hook = CHILD_PROCESS_INIT;
-
-				hook.in = open(rebase_path_rewritten_list(),
-					O_RDONLY);
-				hook.stdout_to_stderr = 1;
-				hook.trace2_hook_name = "post-rewrite";
-				strvec_push(&hook.args, post_rewrite_hook);
-				strvec_push(&hook.args, "rebase");
-				/* we don't care if this hook failed */
-				run_command(&hook);
-			}
+			run_command(&notes_cp);
+
+			hook_opt.path_to_stdin = rebase_path_rewritten_list();
+			strvec_push(&hook_opt.args, "rebase");
+			run_hooks("post-rewrite", &hook_opt);
+			run_hooks_opt_clear(&hook_opt);
 		}
 		apply_autostash(rebase_path_autostash());
 
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 23/31] transport: convert pre-push hook to use config
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (21 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 22/31] hook: convert 'post-rewrite' hook in sequencer.c to hook.h Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 24/31] reference-transaction: use hook.h to run hooks Ævar Arnfjörð Bjarmason
                         ` (9 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

By using the hook.h:run_hooks API, pre-push hooks can be specified in
the config as well as in the hookdir.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 transport.c | 57 ++++++++++++++---------------------------------------
 1 file changed, 15 insertions(+), 42 deletions(-)

diff --git a/transport.c b/transport.c
index e6c46adf60c..1146ed3143c 100644
--- a/transport.c
+++ b/transport.c
@@ -1197,31 +1197,14 @@ static void die_with_unpushed_submodules(struct string_list *needs_pushing)
 static int run_pre_push_hook(struct transport *transport,
 			     struct ref *remote_refs)
 {
-	int ret = 0, x;
+	int ret = 0;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct strbuf tmp = STRBUF_INIT;
 	struct ref *r;
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct strbuf buf;
-	const char *argv[4];
-
-	if (!(argv[0] = find_hook("pre-push")))
-		return 0;
-
-	argv[1] = transport->remote->name;
-	argv[2] = transport->url;
-	argv[3] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.trace2_hook_name = "pre-push";
-
-	if (start_command(&proc)) {
-		finish_command(&proc);
-		return -1;
-	}
-
-	sigchain_push(SIGPIPE, SIG_IGN);
+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 
-	strbuf_init(&buf, 256);
+	strvec_push(&opt.args, transport->remote->name);
+	strvec_push(&opt.args, transport->url);
 
 	for (r = remote_refs; r; r = r->next) {
 		if (!r->peer_ref) continue;
@@ -1230,30 +1213,20 @@ static int run_pre_push_hook(struct transport *transport,
 		if (r->status == REF_STATUS_REJECT_REMOTE_UPDATED) continue;
 		if (r->status == REF_STATUS_UPTODATE) continue;
 
-		strbuf_reset(&buf);
-		strbuf_addf( &buf, "%s %s %s %s\n",
+		strbuf_reset(&tmp);
+		strbuf_addf(&tmp, "%s %s %s %s",
 			 r->peer_ref->name, oid_to_hex(&r->new_oid),
 			 r->name, oid_to_hex(&r->old_oid));
-
-		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
-			/* We do not mind if a hook does not read all refs. */
-			if (errno != EPIPE)
-				ret = -1;
-			break;
-		}
+		string_list_append(&to_stdin, tmp.buf);
 	}
 
-	strbuf_release(&buf);
-
-	x = close(proc.in);
-	if (!ret)
-		ret = x;
-
-	sigchain_pop(SIGPIPE);
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
 
-	x = finish_command(&proc);
-	if (!ret)
-		ret = x;
+	ret = run_hooks("pre-push", &opt);
+	run_hooks_opt_clear(&opt);
+	strbuf_release(&tmp);
+	string_list_clear(&to_stdin, 0);
 
 	return ret;
 }
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 24/31] reference-transaction: use hook.h to run hooks
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (22 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 23/31] transport: convert pre-push hook to use config Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 25/31] run-command: allow capturing of collated output Ævar Arnfjörð Bjarmason
                         ` (8 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

By using the hook.h library, we get closer to removing the hook code
in run-command.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c | 42 +++++++++++++++---------------------------
 1 file changed, 15 insertions(+), 27 deletions(-)

diff --git a/refs.c b/refs.c
index 59be29cf081..1149e7e7dcb 100644
--- a/refs.c
+++ b/refs.c
@@ -2062,47 +2062,35 @@ int ref_update_reject_duplicates(struct string_list *refnames,
 static int run_transaction_hook(struct ref_transaction *transaction,
 				const char *state)
 {
-	struct child_process proc = CHILD_PROCESS_INIT;
 	struct strbuf buf = STRBUF_INIT;
-	const char *hook;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int ret = 0, i;
+	char o[GIT_MAX_HEXSZ + 1], n[GIT_MAX_HEXSZ + 1];
 
-	hook = find_hook("reference-transaction");
-	if (!hook)
+	if (!hook_exists("reference-transaction"))
 		return ret;
 
-	strvec_pushl(&proc.args, hook, state, NULL);
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = "reference-transaction";
-
-	ret = start_command(&proc);
-	if (ret)
-		return ret;
-
-	sigchain_push(SIGPIPE, SIG_IGN);
+	strvec_push(&opt.args, state);
 
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		oid_to_hex_r(o, &update->old_oid);
+		oid_to_hex_r(n, &update->new_oid);
 
 		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
-			    oid_to_hex(&update->old_oid),
-			    oid_to_hex(&update->new_oid),
-			    update->refname);
-
-		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
-			if (errno != EPIPE)
-				ret = -1;
-			break;
-		}
+		strbuf_addf(&buf, "%s %s %s", o, n, update->refname);
+		string_list_append(&to_stdin, buf.buf);
 	}
 
-	close(proc.in);
-	sigchain_pop(SIGPIPE);
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
+
+	ret = run_hooks("reference-transaction", &opt);
+	run_hooks_opt_clear(&opt);
 	strbuf_release(&buf);
+	string_list_clear(&to_stdin, 0);
 
-	ret |= finish_command(&proc);
 	return ret;
 }
 
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 25/31] run-command: allow capturing of collated output
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (23 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 24/31] reference-transaction: use hook.h to run hooks Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 26/31] hooks: allow callers to capture output Ævar Arnfjörð Bjarmason
                         ` (7 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Some callers, for example server-side hooks which wish to relay hook
output to clients across a transport, want to capture what would
normally print to stderr and do something else with it. Allow that via a
callback.

By calling the callback regardless of whether there's output available,
we allow clients to send e.g. a keepalive if necessary.

Because we expose a strbuf, not a fd or FILE*, there's no need to create
a temporary pipe or similar - we can just skip the print to stderr and
instead hand it to the caller.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/fetch.c             |  2 +-
 builtin/submodule--helper.c |  2 +-
 hook.c                      |  1 +
 run-command.c               | 33 +++++++++++++++++++++++++--------
 run-command.h               | 18 +++++++++++++++++-
 submodule.c                 |  2 +-
 t/helper/test-run-command.c | 25 ++++++++++++++++++++-----
 t/t0061-run-command.sh      |  7 +++++++
 8 files changed, 73 insertions(+), 17 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index a07816b6504..769af53ca45 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1817,7 +1817,7 @@ static int fetch_multiple(struct string_list *list, int max_children)
 		result = run_processes_parallel_tr2(max_children,
 						    &fetch_next_remote,
 						    &fetch_failed_to_start,
-						    NULL,
+						    NULL, NULL,
 						    &fetch_finished,
 						    &state,
 						    "fetch", "parallel/fetch");
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 8d1e7310737..fef8392e1da 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2295,7 +2295,7 @@ static int update_submodules(struct submodule_update_clone *suc)
 	int i;
 
 	run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task,
-				   update_clone_start_failure, NULL,
+				   update_clone_start_failure, NULL, NULL,
 				   update_clone_task_finished, suc, "submodule",
 				   "parallel/update");
 
diff --git a/hook.c b/hook.c
index ac5e3988fec..0faa24ec825 100644
--- a/hook.c
+++ b/hook.c
@@ -166,6 +166,7 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
+				   NULL,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
diff --git a/run-command.c b/run-command.c
index 3392640f17b..4a1a7a10820 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1494,6 +1494,7 @@ struct parallel_processes {
 	get_next_task_fn get_next_task;
 	start_failure_fn start_failure;
 	feed_pipe_fn feed_pipe;
+	consume_sideband_fn consume_sideband;
 	task_finished_fn task_finished;
 
 	struct {
@@ -1559,6 +1560,7 @@ static void pp_init(struct parallel_processes *pp,
 		    get_next_task_fn get_next_task,
 		    start_failure_fn start_failure,
 		    feed_pipe_fn feed_pipe,
+		    consume_sideband_fn consume_sideband,
 		    task_finished_fn task_finished,
 		    void *data)
 {
@@ -1579,6 +1581,7 @@ static void pp_init(struct parallel_processes *pp,
 	pp->start_failure = start_failure ? start_failure : default_start_failure;
 	pp->feed_pipe = feed_pipe ? feed_pipe : default_feed_pipe;
 	pp->task_finished = task_finished ? task_finished : default_task_finished;
+	pp->consume_sideband = consume_sideband;
 
 	pp->nr_processes = 0;
 	pp->output_owner = 0;
@@ -1615,7 +1618,10 @@ static void pp_cleanup(struct parallel_processes *pp)
 	 * When get_next_task added messages to the buffer in its last
 	 * iteration, the buffered output is non empty.
 	 */
-	strbuf_write(&pp->buffered_output, stderr);
+	if (pp->consume_sideband)
+		pp->consume_sideband(&pp->buffered_output, pp->data);
+	else
+		strbuf_write(&pp->buffered_output, stderr);
 	strbuf_release(&pp->buffered_output);
 
 	sigchain_pop_common();
@@ -1736,9 +1742,13 @@ static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout)
 static void pp_output(struct parallel_processes *pp)
 {
 	int i = pp->output_owner;
+
 	if (pp->children[i].state == GIT_CP_WORKING &&
 	    pp->children[i].err.len) {
-		strbuf_write(&pp->children[i].err, stderr);
+		if (pp->consume_sideband)
+			pp->consume_sideband(&pp->children[i].err, pp->data);
+		else
+			strbuf_write(&pp->children[i].err, stderr);
 		strbuf_reset(&pp->children[i].err);
 	}
 }
@@ -1777,11 +1787,15 @@ static int pp_collect_finished(struct parallel_processes *pp)
 			strbuf_addbuf(&pp->buffered_output, &pp->children[i].err);
 			strbuf_reset(&pp->children[i].err);
 		} else {
-			strbuf_write(&pp->children[i].err, stderr);
+			/* Output errors, then all other finished child processes */
+			if (pp->consume_sideband) {
+				pp->consume_sideband(&pp->children[i].err, pp->data);
+				pp->consume_sideband(&pp->buffered_output, pp->data);
+			} else {
+				strbuf_write(&pp->children[i].err, stderr);
+				strbuf_write(&pp->buffered_output, stderr);
+			}
 			strbuf_reset(&pp->children[i].err);
-
-			/* Output all other finished child processes */
-			strbuf_write(&pp->buffered_output, stderr);
 			strbuf_reset(&pp->buffered_output);
 
 			/*
@@ -1805,6 +1819,7 @@ int run_processes_parallel(int n,
 			   get_next_task_fn get_next_task,
 			   start_failure_fn start_failure,
 			   feed_pipe_fn feed_pipe,
+			   consume_sideband_fn consume_sideband,
 			   task_finished_fn task_finished,
 			   void *pp_cb)
 {
@@ -1815,7 +1830,7 @@ int run_processes_parallel(int n,
 
 	sigchain_push(SIGPIPE, SIG_IGN);
 
-	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, task_finished, pp_cb);
+	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, consume_sideband, task_finished, pp_cb);
 	while (1) {
 		for (i = 0;
 		    i < spawn_cap && !pp.shutdown &&
@@ -1853,6 +1868,7 @@ int run_processes_parallel(int n,
 int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 			       start_failure_fn start_failure,
 			       feed_pipe_fn feed_pipe,
+			       consume_sideband_fn consume_sideband,
 			       task_finished_fn task_finished, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label)
 {
@@ -1862,7 +1878,8 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 				   ((n < 1) ? online_cpus() : n));
 
 	result = run_processes_parallel(n, get_next_task, start_failure,
-					feed_pipe, task_finished, pp_cb);
+					feed_pipe, consume_sideband,
+					task_finished, pp_cb);
 
 	trace2_region_leave(tr2_category, tr2_label, NULL);
 
diff --git a/run-command.h b/run-command.h
index 41e36d26cb1..7150da851a8 100644
--- a/run-command.h
+++ b/run-command.h
@@ -433,6 +433,20 @@ typedef int (*feed_pipe_fn)(struct strbuf *pipe,
 			    void *pp_cb,
 			    void *pp_task_cb);
 
+/**
+ * If this callback is provided, instead of collating process output to stderr,
+ * they will be collated into a new pipe. consume_sideband_fn will be called
+ * repeatedly. When output is available on that pipe, it will be contained in
+ * 'output'. But it will be called with an empty 'output' too, to allow for
+ * keepalives or similar operations if necessary.
+ *
+ * pp_cb is the callback cookie as passed into run_processes_parallel.
+ *
+ * Since this callback is provided with the collated output, no task cookie is
+ * provided.
+ */
+typedef void (*consume_sideband_fn)(struct strbuf *output, void *pp_cb);
+
 /**
  * This callback is called on every child process that finished processing.
  *
@@ -468,10 +482,12 @@ int run_processes_parallel(int n,
 			   get_next_task_fn,
 			   start_failure_fn,
 			   feed_pipe_fn,
+			   consume_sideband_fn,
 			   task_finished_fn,
 			   void *pp_cb);
 int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn,
-			       feed_pipe_fn, task_finished_fn, void *pp_cb,
+			       feed_pipe_fn, consume_sideband_fn,
+			       task_finished_fn, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label);
 
 #endif
diff --git a/submodule.c b/submodule.c
index ea026a8195f..7fe0c8f7c9f 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1645,7 +1645,7 @@ int fetch_populated_submodules(struct repository *r,
 	run_processes_parallel_tr2(max_parallel_jobs,
 				   get_next_submodule,
 				   fetch_start_failure,
-				   NULL,
+				   NULL, NULL,
 				   fetch_finish,
 				   &spf,
 				   "submodule", "parallel/fetch");
diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c
index 9348184d303..d53db6d11c4 100644
--- a/t/helper/test-run-command.c
+++ b/t/helper/test-run-command.c
@@ -51,6 +51,16 @@ static int no_job(struct child_process *cp,
 	return 0;
 }
 
+static void test_consume_sideband(struct strbuf *output, void *cb)
+{
+	FILE *sideband;
+
+	sideband = fopen("./sideband", "a");
+
+	strbuf_write(output, sideband);
+	fclose(sideband);
+}
+
 static int task_finished(int result,
 			 struct strbuf *err,
 			 void *pp_cb,
@@ -201,7 +211,7 @@ static int testsuite(int argc, const char **argv)
 		suite.tests.nr, max_jobs);
 
 	ret = run_processes_parallel(max_jobs, next_test, test_failed,
-				     test_stdin, test_finished, &suite);
+				     test_stdin, NULL, test_finished, &suite);
 
 	if (suite.failed.nr > 0) {
 		ret = 1;
@@ -429,23 +439,28 @@ int cmd__run_command(int argc, const char **argv)
 
 	if (!strcmp(argv[1], "run-command-parallel"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, NULL, &proc));
+					    NULL, NULL, NULL, NULL, &proc));
 
 	if (!strcmp(argv[1], "run-command-abort"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, task_finished, &proc));
+					    NULL, NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-no-jobs"))
 		exit(run_processes_parallel(jobs, no_job,
-					    NULL, NULL, task_finished, &proc));
+					    NULL, NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-stdin")) {
 		proc.in = -1;
 		proc.no_stdin = 0;
 		exit (run_processes_parallel(jobs, parallel_next, NULL,
-					     test_stdin, NULL, &proc));
+					     test_stdin, NULL, NULL, &proc));
 	}
 
+	if (!strcmp(argv[1], "run-command-sideband"))
+		exit(run_processes_parallel(jobs, parallel_next, NULL, NULL,
+					    test_consume_sideband, NULL,
+					    &proc));
+
 	fprintf(stderr, "check usage\n");
 	return 1;
 }
diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh
index 87759482ad1..e99f6c7f445 100755
--- a/t/t0061-run-command.sh
+++ b/t/t0061-run-command.sh
@@ -143,6 +143,13 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai
 	test_cmp expect actual
 '
 
+test_expect_success 'run_command can divert output' '
+	test_when_finished rm sideband &&
+	test-tool run-command run-command-sideband 3 sh -c "printf \"%s\n%s\n\" Hello World" 2>actual &&
+	test_must_be_empty actual &&
+	test_cmp expect sideband
+'
+
 cat >expect <<-EOF
 preloaded output of a child
 listening for stdin:
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 26/31] hooks: allow callers to capture output
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (24 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 25/31] run-command: allow capturing of collated output Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 27/31] receive-pack: convert 'update' hook to hook.h Ævar Arnfjörð Bjarmason
                         ` (6 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Some server-side hooks will require capturing output to send over
sideband instead of printing directly to stderr. Expose that capability.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 hook.c | 2 +-
 hook.h | 8 ++++++++
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/hook.c b/hook.c
index 0faa24ec825..17ae65eca31 100644
--- a/hook.c
+++ b/hook.c
@@ -166,7 +166,7 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
-				   NULL,
+				   options->consume_sideband,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
diff --git a/hook.h b/hook.h
index ff1697d1087..5f895032341 100644
--- a/hook.h
+++ b/hook.h
@@ -49,6 +49,14 @@ struct run_hooks_opt
 	 */
 	feed_pipe_fn feed_pipe;
 	void *feed_pipe_ctx;
+
+	/*
+	 * Populate this to capture output and prevent it from being printed to
+	 * stderr. This will be passed directly through to
+	 * run_command:run_parallel_processes(). See t/helper/test-run-command.c
+	 * for an example.
+	 */
+	consume_sideband_fn consume_sideband;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 27/31] receive-pack: convert 'update' hook to hook.h
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (25 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 26/31] hooks: allow callers to capture output Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-06-02  1:04         ` Felipe Contreras
  2021-05-28 12:11       ` [PATCH 28/31] post-update: use hook.h library Ævar Arnfjörð Bjarmason
                         ` (5 subsequent siblings)
  32 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

By using hook.h to invoke the 'update' hook we closer to removing the
hooks code in run-command.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/receive-pack.c | 65 ++++++++++++++++++++++++++++--------------
 1 file changed, 44 insertions(+), 21 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 5248228ebfe..378f8f6b5d1 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -938,33 +938,56 @@ static int run_receive_hook(struct command *commands,
 	return status;
 }
 
-static int run_update_hook(struct command *cmd)
+static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
 {
-	const char *argv[5];
-	struct child_process proc = CHILD_PROCESS_INIT;
-	int code;
+	int keepalive_active = 0;
 
-	argv[0] = find_hook("update");
-	if (!argv[0])
-		return 0;
+	if (keepalive_in_sec <= 0)
+		use_keepalive = KEEPALIVE_NEVER;
+	if (use_keepalive == KEEPALIVE_ALWAYS)
+		keepalive_active = 1;
 
-	argv[1] = cmd->ref_name;
-	argv[2] = oid_to_hex(&cmd->old_oid);
-	argv[3] = oid_to_hex(&cmd->new_oid);
-	argv[4] = NULL;
+	/* send a keepalive if there is no data to write */
+	if (keepalive_active && !output->len) {
+		static const char buf[] = "0005\1";
+		write_or_die(1, buf, sizeof(buf) - 1);
+		return;
+	}
 
-	proc.no_stdin = 1;
-	proc.stdout_to_stderr = 1;
-	proc.err = use_sideband ? -1 : 0;
-	proc.argv = argv;
-	proc.trace2_hook_name = "update";
+	if (use_keepalive == KEEPALIVE_AFTER_NUL && !keepalive_active) {
+		const char *first_null = memchr(output->buf, '\0', output->len);
+		if (first_null) {
+			/* The null bit is excluded. */
+			size_t before_null = first_null - output->buf;
+			size_t after_null = output->len - (before_null + 1);
+			keepalive_active = 1;
+			send_sideband(1, 2, output->buf, before_null, use_sideband);
+			send_sideband(1, 2, first_null + 1, after_null, use_sideband);
+
+			return;
+		}
+	}
+
+	send_sideband(1, 2, output->buf, output->len, use_sideband);
+}
+
+static int run_update_hook(struct command *cmd)
+{
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	int code;
+
+	strvec_pushl(&opt.args,
+		     cmd->ref_name,
+		     oid_to_hex(&cmd->old_oid),
+		     oid_to_hex(&cmd->new_oid),
+		     NULL);
 
-	code = start_command(&proc);
-	if (code)
-		return code;
 	if (use_sideband)
-		copy_to_sideband(proc.err, -1, NULL);
-	return finish_command(&proc);
+		opt.consume_sideband = hook_output_to_sideband;
+
+	code = run_hooks("update", &opt);
+	run_hooks_opt_clear(&opt);
+	return code;
 }
 
 static struct command *find_command_by_refname(struct command *list,
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 28/31] post-update: use hook.h library
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (26 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 27/31] receive-pack: convert 'update' hook to hook.h Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 29/31] receive-pack: convert receive hooks to hook.h Ævar Arnfjörð Bjarmason
                         ` (4 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

By using run_hooks() instead of run_hook_le(), 'post-update' hooks can
be specified in the config as well as the hookdir.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/receive-pack.c | 26 +++++++-------------------
 1 file changed, 7 insertions(+), 19 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 378f8f6b5d1..b2ccdb66daa 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1657,33 +1657,21 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 static void run_update_post_hook(struct command *commands)
 {
 	struct command *cmd;
-	struct child_process proc = CHILD_PROCESS_INIT;
-	const char *hook;
-
-	hook = find_hook("post-update");
-	if (!hook)
-		return;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 
 	for (cmd = commands; cmd; cmd = cmd->next) {
 		if (cmd->error_string || cmd->did_not_exist)
 			continue;
-		if (!proc.args.nr)
-			strvec_push(&proc.args, hook);
-		strvec_push(&proc.args, cmd->ref_name);
+		strvec_push(&opt.args, cmd->ref_name);
 	}
-	if (!proc.args.nr)
+	if (!opt.args.nr)
 		return;
 
-	proc.no_stdin = 1;
-	proc.stdout_to_stderr = 1;
-	proc.err = use_sideband ? -1 : 0;
-	proc.trace2_hook_name = "post-update";
+	if (use_sideband)
+		opt.consume_sideband = hook_output_to_sideband;
 
-	if (!start_command(&proc)) {
-		if (use_sideband)
-			copy_to_sideband(proc.err, -1, NULL);
-		finish_command(&proc);
-	}
+	run_hooks("post-update", &opt);
+	run_hooks_opt_clear(&opt);
 }
 
 static void check_aliased_update_internal(struct command *cmd,
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 29/31] receive-pack: convert receive hooks to hook.h
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (27 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 28/31] post-update: use hook.h library Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:11       ` [PATCH 30/31] hooks: fix a TOCTOU in "did we run a hook?" heuristic Ævar Arnfjörð Bjarmason
                         ` (3 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

By using the hook.h library to run receive hooks we get closer to
deleting the hook functions in run-command.c

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/receive-pack.c | 197 +++++++++++++++++++----------------------
 1 file changed, 90 insertions(+), 107 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index b2ccdb66daa..ec90e10477a 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -748,7 +748,7 @@ static int check_cert_push_options(const struct string_list *push_options)
 	return retval;
 }
 
-static void prepare_push_cert_sha1(struct child_process *proc)
+static void prepare_push_cert_sha1(struct run_hooks_opt *opt)
 {
 	static int already_done;
 
@@ -772,110 +772,42 @@ static void prepare_push_cert_sha1(struct child_process *proc)
 		nonce_status = check_nonce(push_cert.buf, bogs);
 	}
 	if (!is_null_oid(&push_cert_oid)) {
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT=%s",
 			     oid_to_hex(&push_cert_oid));
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_SIGNER=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_SIGNER=%s",
 			     sigcheck.signer ? sigcheck.signer : "");
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_KEY=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_KEY=%s",
 			     sigcheck.key ? sigcheck.key : "");
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_STATUS=%c",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_STATUS=%c",
 			     sigcheck.result);
 		if (push_cert_nonce) {
-			strvec_pushf(&proc->env_array,
+			strvec_pushf(&opt->env,
 				     "GIT_PUSH_CERT_NONCE=%s",
 				     push_cert_nonce);
-			strvec_pushf(&proc->env_array,
+			strvec_pushf(&opt->env,
 				     "GIT_PUSH_CERT_NONCE_STATUS=%s",
 				     nonce_status);
 			if (nonce_status == NONCE_SLOP)
-				strvec_pushf(&proc->env_array,
+				strvec_pushf(&opt->env,
 					     "GIT_PUSH_CERT_NONCE_SLOP=%ld",
 					     nonce_stamp_slop);
 		}
 	}
 }
 
+struct receive_hook_feed_context {
+	struct command *cmd;
+	int skip_broken;
+};
+
 struct receive_hook_feed_state {
 	struct command *cmd;
 	struct ref_push_report *report;
 	int skip_broken;
 	struct strbuf buf;
-	const struct string_list *push_options;
 };
 
-typedef int (*feed_fn)(void *, const char **, size_t *);
-static int run_and_feed_hook(const char *hook_name, feed_fn feed,
-			     struct receive_hook_feed_state *feed_state)
-{
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct async muxer;
-	const char *argv[2];
-	int code;
-
-	argv[0] = find_hook(hook_name);
-	if (!argv[0])
-		return 0;
-
-	argv[1] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = hook_name;
-
-	if (feed_state->push_options) {
-		int i;
-		for (i = 0; i < feed_state->push_options->nr; i++)
-			strvec_pushf(&proc.env_array,
-				     "GIT_PUSH_OPTION_%d=%s", i,
-				     feed_state->push_options->items[i].string);
-		strvec_pushf(&proc.env_array, "GIT_PUSH_OPTION_COUNT=%d",
-			     feed_state->push_options->nr);
-	} else
-		strvec_pushf(&proc.env_array, "GIT_PUSH_OPTION_COUNT");
-
-	if (tmp_objdir)
-		strvec_pushv(&proc.env_array, tmp_objdir_env(tmp_objdir));
-
-	if (use_sideband) {
-		memset(&muxer, 0, sizeof(muxer));
-		muxer.proc = copy_to_sideband;
-		muxer.in = -1;
-		code = start_async(&muxer);
-		if (code)
-			return code;
-		proc.err = muxer.in;
-	}
-
-	prepare_push_cert_sha1(&proc);
-
-	code = start_command(&proc);
-	if (code) {
-		if (use_sideband)
-			finish_async(&muxer);
-		return code;
-	}
-
-	sigchain_push(SIGPIPE, SIG_IGN);
-
-	while (1) {
-		const char *buf;
-		size_t n;
-		if (feed(feed_state, &buf, &n))
-			break;
-		if (write_in_full(proc.in, buf, n) < 0)
-			break;
-	}
-	close(proc.in);
-	if (use_sideband)
-		finish_async(&muxer);
-
-	sigchain_pop(SIGPIPE);
-
-	return finish_command(&proc);
-}
-
-static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
+static int feed_receive_hook(void *state_)
 {
 	struct receive_hook_feed_state *state = state_;
 	struct command *cmd = state->cmd;
@@ -884,9 +816,7 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
 	       state->skip_broken && (cmd->error_string || cmd->did_not_exist))
 		cmd = cmd->next;
 	if (!cmd)
-		return -1; /* EOF */
-	if (!bufp)
-		return 0; /* OK, can feed something. */
+		return 1; /* EOF - close the pipe*/
 	strbuf_reset(&state->buf);
 	if (!state->report)
 		state->report = cmd->report;
@@ -910,32 +840,36 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
 			    cmd->ref_name);
 		state->cmd = cmd->next;
 	}
-	if (bufp) {
-		*bufp = state->buf.buf;
-		*sizep = state->buf.len;
-	}
 	return 0;
 }
 
-static int run_receive_hook(struct command *commands,
-			    const char *hook_name,
-			    int skip_broken,
-			    const struct string_list *push_options)
+static int feed_receive_hook_cb(struct strbuf *pipe, void *pp_cb, void *pp_task_cb)
 {
-	struct receive_hook_feed_state state;
-	int status;
-
-	strbuf_init(&state.buf, 0);
-	state.cmd = commands;
-	state.skip_broken = skip_broken;
-	state.report = NULL;
-	if (feed_receive_hook(&state, NULL, NULL))
-		return 0;
-	state.cmd = commands;
-	state.push_options = push_options;
-	status = run_and_feed_hook(hook_name, feed_receive_hook, &state);
-	strbuf_release(&state.buf);
-	return status;
+	struct hook *hook = pp_task_cb;
+	struct receive_hook_feed_state *feed_state = hook->feed_pipe_cb_data;
+	int rc;
+
+	/* first-time setup */
+	if (!feed_state) {
+		struct hook_cb_data *hook_cb = pp_cb;
+		struct run_hooks_opt *opt = hook_cb->options;
+		struct receive_hook_feed_context *ctx = opt->feed_pipe_ctx;
+		if (!ctx)
+			BUG("run_hooks_opt.feed_pipe_ctx required for receive hook");
+
+		feed_state = xmalloc(sizeof(struct receive_hook_feed_state));
+		strbuf_init(&feed_state->buf, 0);
+		feed_state->cmd = ctx->cmd;
+		feed_state->skip_broken = ctx->skip_broken;
+		feed_state->report = NULL;
+
+		hook->feed_pipe_cb_data = feed_state;
+	}
+
+	rc = feed_receive_hook(feed_state);
+	if (!rc)
+		strbuf_addbuf(pipe, &feed_state->buf);
+	return rc;
 }
 
 static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
@@ -971,6 +905,55 @@ static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
 	send_sideband(1, 2, output->buf, output->len, use_sideband);
 }
 
+static int run_receive_hook(struct command *commands,
+			    const char *hook_name,
+			    int skip_broken,
+			    const struct string_list *push_options)
+{
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct receive_hook_feed_context ctx;
+	int rc;
+	struct command *iter = commands;
+
+	/* if there are no valid commands, don't invoke the hook at all. */
+	while (iter && skip_broken && (iter->error_string || iter->did_not_exist))
+		iter = iter->next;
+	if (!iter)
+		return 0;
+
+	/* pre-receive hooks should run in series as the hook updates refs */
+	if (!strcmp(hook_name, "pre-receive"))
+		opt.jobs = 1;
+
+	if (push_options) {
+		int i;
+		for (i = 0; i < push_options->nr; i++)
+			strvec_pushf(&opt.env, "GIT_PUSH_OPTION_%d=%s", i,
+				     push_options->items[i].string);
+		strvec_pushf(&opt.env, "GIT_PUSH_OPTION_COUNT=%d", push_options->nr);
+	} else
+		strvec_push(&opt.env, "GIT_PUSH_OPTION_COUNT");
+
+	if (tmp_objdir)
+		strvec_pushv(&opt.env, tmp_objdir_env(tmp_objdir));
+
+	prepare_push_cert_sha1(&opt);
+
+	/* set up sideband printer */
+	if (use_sideband)
+		opt.consume_sideband = hook_output_to_sideband;
+
+	/* set up stdin callback */
+	ctx.cmd = commands;
+	ctx.skip_broken = skip_broken;
+	opt.feed_pipe = feed_receive_hook_cb;
+	opt.feed_pipe_ctx = &ctx;
+
+	rc = run_hooks(hook_name, &opt);
+	run_hooks_opt_clear(&opt);
+	return rc;
+}
+
 static int run_update_hook(struct command *cmd)
 {
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 30/31] hooks: fix a TOCTOU in "did we run a hook?" heuristic
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (28 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 29/31] receive-pack: convert receive hooks to hook.h Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-05-28 12:48         ` Bagas Sanjaya
  2021-05-28 12:11       ` [PATCH 31/31] hook-list.h: add a generated list of hooks, like config-list.h Ævar Arnfjörð Bjarmason
                         ` (2 subsequent siblings)
  32 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

Fix a race in code added in 680ee550d72 (commit: skip discarding the
index if there is no pre-commit hook, 2017-08-14) by changing the
hook.c API to optionally indicate whether or not the requested hook
ran or not. This was suggested in the discussion around
680ee550d72[1].

Let's also change this for the pre-merge-commit hook, see
6098817fd7f (git-merge: honor pre-merge-commit hook, 2019-08-07) for
the introduction of the previous behavior.

Let's also change this for the push-to-checkout hook. Now instead of
checking if the hook exists and either doing a push to checkout or a
push to deploy we'll always attempt a push to checkout. If the hook
doesn't exist we'll fall back on push to deploy. The same behavior as
before, without the TOCTOU race. See 0855331941b (receive-pack:
support push-to-checkout hook, 2014-12-01) for the introduction of the
previous behavior.

This leaves uses of hook_exists() in two places that matter. The
"reference-transaction" check in refs.c, see 67541597670 (refs:
implement reference transaction hook, 2020-06-19), and the
prepare-commit-msg hook, see 66618a50f9c (sequencer: run
'prepare-commit-msg' hook, 2018-01-24).

In both of those cases we're saving ourselves CPU time by not
preparing data for the hook that we'll then do nothing with if we
don't have the hook, so using this "invoked_hook" pattern doesn't make
sense there purely for optimization purposes.

More importantly, in those cases the worst we'll do is miss that we
"should" run the hook because a new hook appeared, whereas in the
pre-commit and pre-merge-commit cases we'll skip an important
discard_cache() on the bases of our faulty guess.

I do think none of these races really matter in practice. It would be
some one-off issue as a hook was added or removed. I did think it was
stupid that we didn't pass a "did this run?" flag instead of doing
this guessing at a distance though, so now we're not guessing anymore.

1. https://lore.kernel.org/git/20170810191613.kpmhzg4seyxy3cpq@sigill.intra.peff.net/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/commit.c       | 18 +++++++++++-------
 builtin/merge.c        | 16 ++++++++++------
 builtin/receive-pack.c |  8 +++++---
 commit.c               |  1 +
 commit.h               |  3 ++-
 hook.c                 |  4 ++++
 hook.h                 | 10 ++++++++++
 sequencer.c            |  4 ++--
 8 files changed, 45 insertions(+), 19 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index dad4e565443..a66727a612a 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -725,11 +725,13 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	int clean_message_contents = (cleanup_mode != COMMIT_MSG_CLEANUP_NONE);
 	int old_display_comment_prefix;
 	int merge_contains_scissors = 0;
+	int invoked_hook = 0;
 
 	/* This checks and barfs if author is badly specified */
 	determine_author_info(author_ident);
 
-	if (!no_verify && run_commit_hook(use_editor, index_file, "pre-commit", NULL))
+	if (!no_verify && run_commit_hook(use_editor, index_file, &invoked_hook,
+					  "pre-commit", NULL))
 		return 0;
 
 	if (squash_message) {
@@ -1045,10 +1047,10 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (!no_verify && hook_exists("pre-commit")) {
+	if (!no_verify && invoked_hook) {
 		/*
-		 * Re-read the index as pre-commit hook could have updated it,
-		 * and write it out as a tree.  We must do this before we invoke
+		 * Re-read the index as the pre-commit-commit hook was invoked
+		 * and could have updated it. We must do this before we invoke
 		 * the editor and after we invoke run_status above.
 		 */
 		discard_cache();
@@ -1060,7 +1062,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (run_commit_hook(use_editor, index_file, "prepare-commit-msg",
+	if (run_commit_hook(use_editor, index_file, NULL, "prepare-commit-msg",
 			    git_path_commit_editmsg(), hook_arg1, hook_arg2, NULL))
 		return 0;
 
@@ -1077,7 +1079,8 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	}
 
 	if (!no_verify &&
-	    run_commit_hook(use_editor, index_file, "commit-msg", git_path_commit_editmsg(), NULL)) {
+	    run_commit_hook(use_editor, index_file, NULL, "commit-msg",
+			    git_path_commit_editmsg(), NULL)) {
 		return 0;
 	}
 
@@ -1830,7 +1833,8 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 
 	repo_rerere(the_repository, 0);
 	run_auto_maintenance(quiet);
-	run_commit_hook(use_editor, get_index_file(), "post-commit", NULL);
+	run_commit_hook(use_editor, get_index_file(), NULL, "post-commit",
+			NULL);
 	if (amend && !no_post_rewrite) {
 		commit_post_rewrite(the_repository, current_head, &oid);
 	}
diff --git a/builtin/merge.c b/builtin/merge.c
index a9363b94065..2bbdfd6080d 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -844,15 +844,18 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 {
 	struct strbuf msg = STRBUF_INIT;
 	const char *index_file = get_index_file();
+	int invoked_hook = 0;
 
-	if (!no_verify && run_commit_hook(0 < option_edit, index_file, "pre-merge-commit", NULL))
+	if (!no_verify && run_commit_hook(0 < option_edit, index_file,
+					  &invoked_hook, "pre-merge-commit",
+					  NULL))
 		abort_commit(remoteheads, NULL);
 	/*
-	 * Re-read the index as pre-merge-commit hook could have updated it,
-	 * and write it out as a tree.  We must do this before we invoke
+	 * Re-read the index as the pre-merge-commit hook was invoked
+	 * and could have updated it. We must do this before we invoke
 	 * the editor and after we invoke run_status above.
 	 */
-	if (hook_exists("pre-merge-commit"))
+	if (invoked_hook)
 		discard_cache();
 	read_cache_from(index_file);
 	strbuf_addbuf(&msg, &merge_msg);
@@ -873,7 +876,8 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 		append_signoff(&msg, ignore_non_trailer(msg.buf, msg.len), 0);
 	write_merge_heads(remoteheads);
 	write_file_buf(git_path_merge_msg(the_repository), msg.buf, msg.len);
-	if (run_commit_hook(0 < option_edit, get_index_file(), "prepare-commit-msg",
+	if (run_commit_hook(0 < option_edit, get_index_file(), NULL,
+			    "prepare-commit-msg",
 			    git_path_merge_msg(the_repository), "merge", NULL))
 		abort_commit(remoteheads, NULL);
 	if (0 < option_edit) {
@@ -882,7 +886,7 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 	}
 
 	if (!no_verify && run_commit_hook(0 < option_edit, get_index_file(),
-					  "commit-msg",
+					  NULL, "commit-msg",
 					  git_path_merge_msg(the_repository), NULL))
 		abort_commit(remoteheads, NULL);
 
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index ec90e10477a..cd658f41d58 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1439,10 +1439,12 @@ static const char *push_to_deploy(unsigned char *sha1,
 static const char *push_to_checkout_hook = "push-to-checkout";
 
 static const char *push_to_checkout(unsigned char *hash,
+				    int *invoked_hook,
 				    struct strvec *env,
 				    const char *work_tree)
 {
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	opt.invoked_hook = invoked_hook;
 
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
 	strvec_pushv(&opt.env, env->v);
@@ -1460,6 +1462,7 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 {
 	const char *retval, *work_tree, *git_dir = NULL;
 	struct strvec env = STRVEC_INIT;
+	int invoked_hook = 0;
 
 	if (worktree && worktree->path)
 		work_tree = worktree->path;
@@ -1477,10 +1480,9 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 
 	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
 
-	if (!hook_exists(push_to_checkout_hook))
+	retval = push_to_checkout(sha1, &invoked_hook, &env, work_tree);
+	if (!invoked_hook)
 		retval = push_to_deploy(sha1, &env, work_tree);
-	else
-		retval = push_to_checkout(sha1, &env, work_tree);
 
 	strvec_clear(&env);
 	return retval;
diff --git a/commit.c b/commit.c
index e8147a88fc6..cf62ebceae5 100644
--- a/commit.c
+++ b/commit.c
@@ -1697,6 +1697,7 @@ size_t ignore_non_trailer(const char *buf, size_t len)
 }
 
 int run_commit_hook(int editor_is_used, const char *index_file,
+		    int *invoked_hook,
 		    const char *name, ...)
 {
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
diff --git a/commit.h b/commit.h
index df42eb434f3..b5a542993c6 100644
--- a/commit.h
+++ b/commit.h
@@ -363,7 +363,8 @@ int compare_commits_by_commit_date(const void *a_, const void *b_, void *unused)
 int compare_commits_by_gen_then_commit_date(const void *a_, const void *b_, void *unused);
 
 LAST_ARG_MUST_BE_NULL
-int run_commit_hook(int editor_is_used, const char *index_file, const char *name, ...);
+int run_commit_hook(int editor_is_used, const char *index_file,
+		    int *invoked_hook, const char *name, ...);
 
 /* Sign a commit or tag buffer, storing the result in a header. */
 int sign_with_header(struct strbuf *buf, const char *keyid);
diff --git a/hook.c b/hook.c
index 17ae65eca31..3cf51460279 100644
--- a/hook.c
+++ b/hook.c
@@ -138,6 +138,9 @@ static int notify_hook_finished(int result,
 	/* |= rc in cb */
 	hook_cb->rc |= result;
 
+	if (hook_cb->invoked_hook)
+		*hook_cb->invoked_hook = 1;
+
 	return 1;
 }
 
@@ -152,6 +155,7 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 		.rc = 0,
 		.hook_name = hook_name,
 		.options = options,
+		.invoked_hook = options->invoked_hook,
 	};
 	if (options->absolute_path) {
 		strbuf_add_absolute_path(&abs_path, hook_path);
diff --git a/hook.h b/hook.h
index 5f895032341..9d163e6f992 100644
--- a/hook.h
+++ b/hook.h
@@ -57,6 +57,15 @@ struct run_hooks_opt
 	 * for an example.
 	 */
 	consume_sideband_fn consume_sideband;
+
+	/*
+	 * A pointer which if provided will be set to 1 or 0 depending
+	 * on if a hook was invoked (i.e. existed), regardless of
+	 * whether or not that was successful. Used for avoiding
+	 * TOCTOU races in code that would otherwise call hook_exist()
+	 * after a "maybe hook run" to see if a hook was invoked.
+	 */
+	int *invoked_hook;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
@@ -81,6 +90,7 @@ struct hook_cb_data {
 	const char *hook_name;
 	struct hook *run_me;
 	struct run_hooks_opt *options;
+	int *invoked_hook;
 };
 
 /*
diff --git a/sequencer.c b/sequencer.c
index ec2761e47d9..2440b9dccd8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1204,7 +1204,7 @@ static int run_prepare_commit_msg_hook(struct repository *r,
 	} else {
 		arg1 = "message";
 	}
-	if (run_commit_hook(0, r->index_file, "prepare-commit-msg", name,
+	if (run_commit_hook(0, r->index_file, NULL, "prepare-commit-msg", name,
 			    arg1, arg2, NULL))
 		ret = error(_("'prepare-commit-msg' hook failed"));
 
@@ -1534,7 +1534,7 @@ static int try_to_commit(struct repository *r,
 		goto out;
 	}
 
-	run_commit_hook(0, r->index_file, "post-commit", NULL);
+	run_commit_hook(0, r->index_file, NULL, "post-commit", NULL);
 	if (flags & AMEND_MSG)
 		commit_post_rewrite(r, current_head, oid);
 
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 31/31] hook-list.h: add a generated list of hooks, like config-list.h
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (29 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 30/31] hooks: fix a TOCTOU in "did we run a hook?" heuristic Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:11       ` Ævar Arnfjörð Bjarmason
  2021-06-01  1:00         ` Ævar Arnfjörð Bjarmason
  2021-06-01 18:14       ` [PATCH 00/31] minimal restart of "config-based-hooks" Emily Shaffer
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
  32 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 12:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

Resolve a long-standing TODO item in bugreport.c of there being no
centralized listing of hooks, this fixes a bug with the bugreport
listing only knowing about 1/4 of the p4 hooks. It didn't know about
the "reference-transaction" hook either.

We can now make sure this is kept up-to-date, as the hook.c library
will die if asked to find a hook we don't know about yet. The only
(undocumented) exception is the artificial "test-hook" used in our own
test suite. Move some of the tests away from the "does-not-exist"
pseudo-hook, and test for the new behavior.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .gitignore           |  1 +
 Makefile             | 14 +++++++++++---
 builtin/bugreport.c  | 44 ++++++++------------------------------------
 generate-hooklist.sh | 20 ++++++++++++++++++++
 hook.c               | 22 ++++++++++++++++++++++
 t/t1800-hook.sh      | 14 +++++++++++---
 6 files changed, 73 insertions(+), 42 deletions(-)
 create mode 100755 generate-hooklist.sh

diff --git a/.gitignore b/.gitignore
index de39dc9961b..66189ca3cdc 100644
--- a/.gitignore
+++ b/.gitignore
@@ -191,6 +191,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/hook-list.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Makefile b/Makefile
index a6b71a0fbed..d0532f3c744 100644
--- a/Makefile
+++ b/Makefile
@@ -817,6 +817,7 @@ XDIFF_LIB = xdiff/lib.a
 
 GENERATED_H += command-list.h
 GENERATED_H += config-list.h
+GENERATED_H += hook-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2207,7 +2208,9 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
+hook.sp hook.s hook.o: hook-list.h
+
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h hook-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2240,6 +2243,11 @@ command-list.h: $(wildcard Documentation/git*.txt)
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+hook-list.h: generate-hooklist.sh
+hook-list.h: Documentation/githooks.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-hooklist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
@@ -2890,7 +2898,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
+EXCEPT_HDRS := command-list.h config-list.h hook-list.h unicode-width.h compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
 endif
@@ -2912,7 +2920,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: config-list.h command-list.h
+check: config-list.h command-list.h hook-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
diff --git a/builtin/bugreport.c b/builtin/bugreport.c
index 941c8d5e270..a7a1fcb8a7a 100644
--- a/builtin/bugreport.c
+++ b/builtin/bugreport.c
@@ -4,6 +4,7 @@
 #include "help.h"
 #include "compat/compiler.h"
 #include "hook.h"
+#include "hook-list.h"
 
 
 static void get_system_info(struct strbuf *sys_info)
@@ -41,39 +42,7 @@ static void get_system_info(struct strbuf *sys_info)
 
 static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 {
-	/*
-	 * NEEDSWORK: Doesn't look like there is a list of all possible hooks;
-	 * so below is a transcription of `git help hooks`. Later, this should
-	 * be replaced with some programmatically generated list (generated from
-	 * doc or else taken from some library which tells us about all the
-	 * hooks)
-	 */
-	static const char *hook[] = {
-		"applypatch-msg",
-		"pre-applypatch",
-		"post-applypatch",
-		"pre-commit",
-		"pre-merge-commit",
-		"prepare-commit-msg",
-		"commit-msg",
-		"post-commit",
-		"pre-rebase",
-		"post-checkout",
-		"post-merge",
-		"pre-push",
-		"pre-receive",
-		"update",
-		"post-receive",
-		"post-update",
-		"push-to-checkout",
-		"pre-auto-gc",
-		"post-rewrite",
-		"sendemail-validate",
-		"fsmonitor-watchman",
-		"p4-pre-submit",
-		"post-index-change",
-	};
-	int i;
+	const char **p;
 
 	if (nongit) {
 		strbuf_addstr(hook_info,
@@ -81,9 +50,12 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 		return;
 	}
 
-	for (i = 0; i < ARRAY_SIZE(hook); i++)
-		if (hook_exists(hook[i]))
-			strbuf_addf(hook_info, "%s\n", hook[i]);
+	for (p = hook_name_list; *p; p++) {
+		const char *hook = *p;
+
+		if (hook_exists(hook))
+			strbuf_addf(hook_info, "%s\n", hook);
+	}
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/generate-hooklist.sh b/generate-hooklist.sh
new file mode 100755
index 00000000000..25a7207b276
--- /dev/null
+++ b/generate-hooklist.sh
@@ -0,0 +1,20 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-hooklist.sh */"
+
+print_hook_list () {
+	cat <<EOF
+static const char *hook_name_list[] = {
+EOF
+	grep -C1 -h '^~~~' Documentation/githooks.txt |
+	grep '^[a-z0-9][a-z0-9-]*$' |
+	sort |
+	sed 's/^.*$/	"&",/'
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_hook_list
diff --git a/hook.c b/hook.c
index 3cf51460279..68e8c1ba868 100644
--- a/hook.c
+++ b/hook.c
@@ -1,11 +1,33 @@
 #include "cache.h"
 #include "hook.h"
 #include "run-command.h"
+#include "hook-list.h"
+
+static int known_hook(const char *name)
+{
+	const char **p;
+	size_t len = strlen(name);
+	for (p = hook_name_list; *p; p++) {
+		const char *hook = *p;
+
+		if (!strncmp(name, hook, len) && hook[len] == '\0')
+			return 1;
+	}
+	if (!strcmp(name, "test-hook") ||
+	    !strcmp(name, "does-not-exist"))
+		return 1;
+
+	return 0;
+}
 
 const char *find_hook(const char *name)
 {
 	static struct strbuf path = STRBUF_INIT;
 
+	if (!known_hook(name))
+		die(_("the hook '%s' is not known to git, should be in hook-list.h via githooks(5)"),
+		    name);
+
 	strbuf_reset(&path);
 	strbuf_git_path(&path, "hooks/%s", name);
 	if (access(path.buf, X_OK) < 0) {
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
index 9e2dd64275c..80e985a81bd 100755
--- a/t/t1800-hook.sh
+++ b/t/t1800-hook.sh
@@ -10,17 +10,25 @@ test_expect_success 'setup .git/hooks' '
 
 test_expect_success 'git hook run -- nonexistent hook' '
 	cat >stderr.expect <<-\EOF &&
-	error: cannot find a hook named does-not-exist
+	error: cannot find a hook named test-hook
 	EOF
-	test_expect_code 1 git hook run does-not-exist 2>stderr.actual &&
+	test_expect_code 1 git hook run test-hook 2>stderr.actual &&
 	test_cmp stderr.expect stderr.actual
 '
 
 test_expect_success 'git hook run -- nonexistent hook with --ignore-missing' '
-	git hook run --ignore-missing does-not-exist 2>stderr.actual &&
+	git hook run --ignore-missing test-hook 2>stderr.actual &&
 	test_must_be_empty stderr.actual
 '
 
+test_expect_success 'git hook run -- nonexistent hook with --ignore-missing' '
+	cat >stderr.expect <<-\EOF &&
+	fatal: the hook '"'"'unknown-hook'"'"' is not known to git, should be in hook-list.h via githooks(5)
+	EOF
+	test_expect_code 128 git hook run unknown-hook 2>stderr.actual &&
+	test_cmp stderr.expect stderr.actual
+'
+
 test_expect_success 'git hook run -- basic' '
 	write_script .git/hooks/test-hook <<-EOF &&
 	echo Test hook
-- 
2.32.0.rc1.458.gd885d4f985c


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* Re: [PATCH 30/31] hooks: fix a TOCTOU in "did we run a hook?" heuristic
  2021-05-28 12:11       ` [PATCH 30/31] hooks: fix a TOCTOU in "did we run a hook?" heuristic Ævar Arnfjörð Bjarmason
@ 2021-05-28 12:48         ` Bagas Sanjaya
  2021-05-28 14:11           ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 479+ messages in thread
From: Bagas Sanjaya @ 2021-05-28 12:48 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason, git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan

On 28/05/21 19.11, Ævar Arnfjörð Bjarmason wrote:
> Let's also change this for the push-to-checkout hook. Now instead of
> checking if the hook exists and either doing a push to checkout or a
> push to deploy we'll always attempt a push to checkout. If the hook
> doesn't exist we'll fall back on push to deploy. The same behavior as
> before, without the TOCTOU race. See 0855331941b (receive-pack:
> support push-to-checkout hook, 2014-12-01) for the introduction of the
> previous behavior.

What is TOCTOU? I never hear this term before.

-- 
An old man doll... just what I always wanted! - Clara

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 30/31] hooks: fix a TOCTOU in "did we run a hook?" heuristic
  2021-05-28 12:48         ` Bagas Sanjaya
@ 2021-05-28 14:11           ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-28 14:11 UTC (permalink / raw)
  To: Bagas Sanjaya
  Cc: git, Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan


On Fri, May 28 2021, Bagas Sanjaya wrote:

> On 28/05/21 19.11, Ævar Arnfjörð Bjarmason wrote:
>> Let's also change this for the push-to-checkout hook. Now instead of
>> checking if the hook exists and either doing a push to checkout or a
>> push to deploy we'll always attempt a push to checkout. If the hook
>> doesn't exist we'll fall back on push to deploy. The same behavior as
>> before, without the TOCTOU race. See 0855331941b (receive-pack:
>> support push-to-checkout hook, 2014-12-01) for the introduction of the
>> previous behavior.
>
> What is TOCTOU? I never hear this term before.

A common race condition[1], time of use, time of check. I.e. in this
case we use the hook first, and then later we check if there's a
hook. Thus we've got a logic error where we're assuming we'll get the
same answer[2] in both instances.

In this case what we plainly want is to just save away whether we did X,
not do X, and then later check if we can do X to see if we did X
earlier.

I think this whole thing is still suspect even with my fix, there's an
inherent assumption here of a single-thread view of the world. I.e. that
you have a user running a git command at a time, and that while our hook
run didn't change the index, a background process might have.

But this probably handles the most common case, i.e. a hook explicitly
changing the index and wanting its invoker to notice, and in any case we
have the single-thread assumption already, so we might as well make it
less buggy/racy.

If you dig through the history we used to do this without the check for
"earlier X?" before for at least one of the hooks, but it got changed to
the current state as an optimization.

1. https://en.wikipedia.org/wiki/TOCTOU
2. I mean, I don't think any programmer assumed it in this case, it was
   just intentional lazyness or a legitimate trade-off, but now that
   it's easy to eliminate the race...

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v9 22/37] read-cache: convert post-index-change hook to use config
  2021-05-27 23:04     ` Ævar Arnfjörð Bjarmason
  2021-05-28  1:09       ` Taylor Blau
@ 2021-05-31 19:21       ` Felipe Contreras
  1 sibling, 0 replies; 479+ messages in thread
From: Felipe Contreras @ 2021-05-31 19:21 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason, Emily Shaffer; +Cc: git

Ævar Arnfjörð Bjarmason wrote:
> 
> On Wed, May 26 2021, Emily Shaffer wrote:
> 
> >  Part of the linkgit:git[1] suite
> > diff --git a/read-cache.c b/read-cache.c
> > index 1b3c2eb408..6a5c9403f4 100644
> > --- a/read-cache.c
> > +++ b/read-cache.c
> > @@ -26,6 +26,8 @@
> >  #include "thread-utils.h"
> >  #include "progress.h"
> >  #include "sparse-index.h"
> > +#include "hook.h"
> > +>>>>>>> 9524a9d29d (read-cache: convert post-index-change hook to use config)
> 
> This adds a conflict marker, which is removed later in the series.
> 
> Obviously a trivial mistake, but it's a good idea to use git rebase -i
> -x 'make test' or equivalent for such a large series, perhaps there are
> other inter-patch issues lurking here...

Ahh, thanks so much for this. I've often wanted to do this and sometimes
wrote simple scripts. It's good to know there a straightforward way from
standard commands.

That way I have less excuse not to do the check... and I just found a
couple of transitory issues on my latest patch series.

Cheers.

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 31/31] hook-list.h: add a generated list of hooks, like config-list.h
  2021-05-28 12:11       ` [PATCH 31/31] hook-list.h: add a generated list of hooks, like config-list.h Ævar Arnfjörð Bjarmason
@ 2021-06-01  1:00         ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-01  1:00 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason


On Fri, May 28 2021, Ævar Arnfjörð Bjarmason wrote:

> Resolve a long-standing TODO item in bugreport.c of there being no
> centralized listing of hooks, this fixes a bug with the bugreport
> listing only knowing about 1/4 of the p4 hooks. It didn't know about
> the "reference-transaction" hook either.
>
> We can now make sure this is kept up-to-date, as the hook.c library
> will die if asked to find a hook we don't know about yet. The only
> (undocumented) exception is the artificial "test-hook" used in our own
> test suite. Move some of the tests away from the "does-not-exist"
> pseudo-hook, and test for the new behavior.
>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>  .gitignore           |  1 +
>  Makefile             | 14 +++++++++++---
>  builtin/bugreport.c  | 44 ++++++++------------------------------------
>  generate-hooklist.sh | 20 ++++++++++++++++++++
>  hook.c               | 22 ++++++++++++++++++++++
>  t/t1800-hook.sh      | 14 +++++++++++---
>  6 files changed, 73 insertions(+), 42 deletions(-)
>  create mode 100755 generate-hooklist.sh
>
> diff --git a/.gitignore b/.gitignore
> index de39dc9961b..66189ca3cdc 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -191,6 +191,7 @@
>  /gitweb/static/gitweb.min.*
>  /config-list.h
>  /command-list.h
> +/hook-list.h
>  *.tar.gz
>  *.dsc
>  *.deb
> diff --git a/Makefile b/Makefile
> index a6b71a0fbed..d0532f3c744 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -817,6 +817,7 @@ XDIFF_LIB = xdiff/lib.a
>  
>  GENERATED_H += command-list.h
>  GENERATED_H += config-list.h
> +GENERATED_H += hook-list.h
>  
>  LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
>  	$(FIND) . \
> @@ -2207,7 +2208,9 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
>  
>  help.sp help.s help.o: command-list.h
>  
> -builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
> +hook.sp hook.s hook.o: hook-list.h
> +
> +builtin/help.sp builtin/help.s builtin/help.o: config-list.h hook-list.h GIT-PREFIX
>  builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
>  	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
>  	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
> @@ -2240,6 +2243,11 @@ command-list.h: $(wildcard Documentation/git*.txt)
>  		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
>  		command-list.txt >$@+ && mv $@+ $@
>  
> +hook-list.h: generate-hooklist.sh
> +hook-list.h: Documentation/githooks.txt
> +	$(QUIET_GEN)$(SHELL_PATH) ./generate-hooklist.sh \
> +		>$@+ && mv $@+ $@
> +
>  SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
>  	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
>  	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
> @@ -2890,7 +2898,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
>  .PHONY: sparse $(SP_OBJ)
>  sparse: $(SP_OBJ)
>  
> -EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
> +EXCEPT_HDRS := command-list.h config-list.h hook-list.h unicode-width.h compat/% xdiff/%
>  ifndef GCRYPT_SHA256
>  	EXCEPT_HDRS += sha256/gcrypt.h
>  endif
> @@ -2912,7 +2920,7 @@ hdr-check: $(HCO)
>  style:
>  	git clang-format --style file --diff --extensions c,h
>  
> -check: config-list.h command-list.h
> +check: config-list.h command-list.h hook-list.h
>  	@if sparse; \
>  	then \
>  		echo >&2 "Use 'make sparse' instead"; \
> diff --git a/builtin/bugreport.c b/builtin/bugreport.c
> index 941c8d5e270..a7a1fcb8a7a 100644
> --- a/builtin/bugreport.c
> +++ b/builtin/bugreport.c
> @@ -4,6 +4,7 @@
>  #include "help.h"
>  #include "compat/compiler.h"
>  #include "hook.h"
> +#include "hook-list.h"
>  
>  
>  static void get_system_info(struct strbuf *sys_info)
> @@ -41,39 +42,7 @@ static void get_system_info(struct strbuf *sys_info)
>  
>  static void get_populated_hooks(struct strbuf *hook_info, int nongit)
>  {
> -	/*
> -	 * NEEDSWORK: Doesn't look like there is a list of all possible hooks;
> -	 * so below is a transcription of `git help hooks`. Later, this should
> -	 * be replaced with some programmatically generated list (generated from
> -	 * doc or else taken from some library which tells us about all the
> -	 * hooks)
> -	 */
> -	static const char *hook[] = {
> -		"applypatch-msg",
> -		"pre-applypatch",
> -		"post-applypatch",
> -		"pre-commit",
> -		"pre-merge-commit",
> -		"prepare-commit-msg",
> -		"commit-msg",
> -		"post-commit",
> -		"pre-rebase",
> -		"post-checkout",
> -		"post-merge",
> -		"pre-push",
> -		"pre-receive",
> -		"update",
> -		"post-receive",
> -		"post-update",
> -		"push-to-checkout",
> -		"pre-auto-gc",
> -		"post-rewrite",
> -		"sendemail-validate",
> -		"fsmonitor-watchman",
> -		"p4-pre-submit",
> -		"post-index-change",
> -	};
> -	int i;
> +	const char **p;
>  
>  	if (nongit) {
>  		strbuf_addstr(hook_info,
> @@ -81,9 +50,12 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
>  		return;
>  	}
>  
> -	for (i = 0; i < ARRAY_SIZE(hook); i++)
> -		if (hook_exists(hook[i]))
> -			strbuf_addf(hook_info, "%s\n", hook[i]);
> +	for (p = hook_name_list; *p; p++) {
> +		const char *hook = *p;
> +
> +		if (hook_exists(hook))
> +			strbuf_addf(hook_info, "%s\n", hook);
> +	}
>  }
>  
>  static const char * const bugreport_usage[] = {
> diff --git a/generate-hooklist.sh b/generate-hooklist.sh
> new file mode 100755
> index 00000000000..25a7207b276
> --- /dev/null
> +++ b/generate-hooklist.sh
> @@ -0,0 +1,20 @@
> +#!/bin/sh
> +
> +echo "/* Automatically generated by generate-hooklist.sh */"
> +
> +print_hook_list () {
> +	cat <<EOF
> +static const char *hook_name_list[] = {
> +EOF
> +	grep -C1 -h '^~~~' Documentation/githooks.txt |
> +	grep '^[a-z0-9][a-z0-9-]*$' |
> +	sort |
> +	sed 's/^.*$/	"&",/'
> +	cat <<EOF
> +	NULL,
> +};
> +EOF
> +}
> +

I found that grep -C isn't portable, so (pending any feedback on this
series) I've locally replaced it with:

diff --git a/generate-hooklist.sh b/generate-hooklist.sh
new file mode 100755
index 0000000000..5a3f7f849c
--- /dev/null
+++ b/generate-hooklist.sh
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-hooklist.sh */"
+
+print_hook_list () {
+	cat <<EOF
+static const char *hook_name_list[] = {
+EOF
+	perl -ne '
+		chomp;
+		@l[$.] = $_;
+		push @h => $l[$. - 1] if /^~~~+$/s;
+		END {
+			print qq[\t"$_",\n] for sort @h;
+		}
+	' <Documentation/githooks.txt
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_hook_list

^ permalink raw reply related	[flat|nested] 479+ messages in thread

* Re: [PATCH 00/31] minimal restart of "config-based-hooks"
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (30 preceding siblings ...)
  2021-05-28 12:11       ` [PATCH 31/31] hook-list.h: add a generated list of hooks, like config-list.h Ævar Arnfjörð Bjarmason
@ 2021-06-01 18:14       ` Emily Shaffer
  2021-06-01 20:50         ` Derrick Stolee
                           ` (2 more replies)
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
  32 siblings, 3 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-06-01 18:14 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan

On Fri, May 28, 2021 at 02:11:02PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> After suggesting[1] an another round that the config-based-hook
> topic[2] should take a more incremental approach to reach its end goal
> I thought I'd try hacking that up.
> 
> So this is a proposed restart of that topic which if the consensus
> favors it should replace it, and the config-based hooks topic should
> be rebased on top of this.

I'm not entirely sure what you're trying to achieve by sending this
series. It was my impression that the existing config-based-hooks topic
was close to being ready to submit anyway (since Junio mentioned
submitting it a couple revisions ago); rather than churning by reviewing
a different 31-patch topic, and then re-rolling and re-reviewing a
(reduced) config hook topic, wouldn't it be easier on everyone's time to
do a final incremental review on the existing topic and then start in on
bugfixes/feature patches afterwards?

It would have been nice to see a more clear discussion of patch
organization sometime much sooner in the past year and a half since the
project was proposed[3], like maybe in the few iterations of the design
doc which included a rollout plan in July of last year[4]. To me, it
seems late to be overhauling the direction like this, especially after I
asked for opinions and approval on the direction before I started work
in earnest.

Anyway, I'd personally rather spend effort getting the existing series
the last few yards to the finish line than to head most of the way back
to the start.

 - Emily

> 1. https://lore.kernel.org/git/87lf80l1m6.fsf@evledraar.gmail.com/
> 2. https://lore.kernel.org/git/20210527000856.695702-1-emilyshaffer@google.com/
3. https://lore.kernel.org/git/20191116011125.GG22855@google.com/
4. https://lore.kernel.org/git/20200728222455.3023400-1-emilyshaffer@google.com/

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 00/31] minimal restart of "config-based-hooks"
  2021-06-01 18:14       ` [PATCH 00/31] minimal restart of "config-based-hooks" Emily Shaffer
@ 2021-06-01 20:50         ` Derrick Stolee
  2021-06-02  5:42           ` Felipe Contreras
  2021-06-02  9:34           ` Ævar Arnfjörð Bjarmason
  2021-06-02  5:30         ` Felipe Contreras
  2021-06-02  7:56         ` Ævar Arnfjörð Bjarmason
  2 siblings, 2 replies; 479+ messages in thread
From: Derrick Stolee @ 2021-06-01 20:50 UTC (permalink / raw)
  To: Emily Shaffer, Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan

On 6/1/2021 2:14 PM, Emily Shaffer wrote:
> On Fri, May 28, 2021 at 02:11:02PM +0200, Ævar Arnfjörð Bjarmason wrote:
>>
>> After suggesting[1] an another round that the config-based-hook
>> topic[2] should take a more incremental approach to reach its end goal
>> I thought I'd try hacking that up.

I think sending this complete reorganization of a long-lived topic
is not helpful, especially because the end-to-end diff is significant.
This series has been extensively tested and incrementally improved
for months, and it would be a waste to start over and lose all of that
hardening.

It's also a but rushed that this comes only a day after the previous
message recommending a reorganization. It would be best to at least
give the original author an opportunity to comment on your idea before
working on this.

>> So this is a proposed restart of that topic which if the consensus
>> favors it should replace it, and the config-based hooks topic should
>> be rebased on top of this.
> 
> I'm not entirely sure what you're trying to achieve by sending this
> series. It was my impression that the existing config-based-hooks topic
> was close to being ready to submit anyway (since Junio mentioned
> submitting it a couple revisions ago); rather than churning by reviewing
> a different 31-patch topic, and then re-rolling and re-reviewing a
> (reduced) config hook topic, wouldn't it be easier on everyone's time to
> do a final incremental review on the existing topic and then start in on
> bugfixes/feature patches afterwards?

I completely agree here.

> It would have been nice to see a more clear discussion of patch
> organization sometime much sooner in the past year and a half since the
> project was proposed[3], like maybe in the few iterations of the design
> doc which included a rollout plan in July of last year[4]. To me, it
> seems late to be overhauling the direction like this, especially after I
> asked for opinions and approval on the direction before I started work
> in earnest.

I've also seen messages as early as January where Ævar mentioned
wanting to review the series, but not finding the time to do so.
It is reasonable to expect that contributors attempt such major
reorganizations according to reviewers feedback, as long as the
reviewers are timely about delivering that feedback.

Thanks,
-Stolee

^ permalink raw reply	[flat|nested] 479+ messages in thread

* RE: [PATCH 16/31] receive-pack: convert push-to-checkout hook to hook.h
  2021-05-28 12:11       ` [PATCH 16/31] receive-pack: convert push-to-checkout hook to hook.h Ævar Arnfjörð Bjarmason
@ 2021-06-02  0:51         ` Felipe Contreras
  0 siblings, 0 replies; 479+ messages in thread
From: Felipe Contreras @ 2021-06-02  0:51 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason, git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

Ævar Arnfjörð Bjarmason wrote:
> From: Emily Shaffer <emilyshaffer@google.com>
> 
> By using hook.h instead of run-command.h to invoke push-to-checkout,
> hooks can now be specified in the config as well as in the hookdir.
> push-to-checkout is not called anywhere but in builtin/receive-pack.c.
> 
> This is the last user of the run_hook_le() API, so let's remove it
> while we're at it, since run_hook_le() itself is the last user of
> run_hook_ve() we can remove that too. The last direct user of
> run_hook_le() was removed in the commit preceding this one.

I think this should be split in two patches.

Yes, fundamentally there's not much of a difference, but if you are
already at 31 patches one that removes the last user and nother that
removes the API will make the progression a little bit cleaner.

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 479+ messages in thread

* RE: [PATCH 21/31] hook: provide stdin by string_list or callback
  2021-05-28 12:11       ` [PATCH 21/31] hook: provide stdin by string_list or callback Ævar Arnfjörð Bjarmason
@ 2021-06-02  1:00         ` Felipe Contreras
  0 siblings, 0 replies; 479+ messages in thread
From: Felipe Contreras @ 2021-06-02  1:00 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason, git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

Ævar Arnfjörð Bjarmason wrote:
> From: Emily Shaffer <emilyshaffer@google.com>
> 
> In cases where a hook requires only a small amount of information via
> stdin, it should be simple for users to provide a string_list alone. But
> in more complicated cases where the stdin is too large to hold in
> memory,

> let's provide a callback the users can populate line after line
> with instead.

let's instead provide a callback the users can populate line after line.

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 479+ messages in thread

* RE: [PATCH 27/31] receive-pack: convert 'update' hook to hook.h
  2021-05-28 12:11       ` [PATCH 27/31] receive-pack: convert 'update' hook to hook.h Ævar Arnfjörð Bjarmason
@ 2021-06-02  1:04         ` Felipe Contreras
  0 siblings, 0 replies; 479+ messages in thread
From: Felipe Contreras @ 2021-06-02  1:04 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason, git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan,
	Ævar Arnfjörð Bjarmason

Ævar Arnfjörð Bjarmason wrote:
> From: Emily Shaffer <emilyshaffer@google.com>
> 
> By using hook.h to invoke the 'update' hook we closer to removing the
> hooks code in run-command.c.

We *are* closer? We _get_ closer?

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 00/31] minimal restart of "config-based-hooks"
  2021-06-01 18:14       ` [PATCH 00/31] minimal restart of "config-based-hooks" Emily Shaffer
  2021-06-01 20:50         ` Derrick Stolee
@ 2021-06-02  5:30         ` Felipe Contreras
  2021-06-02  7:56         ` Ævar Arnfjörð Bjarmason
  2 siblings, 0 replies; 479+ messages in thread
From: Felipe Contreras @ 2021-06-02  5:30 UTC (permalink / raw)
  To: Emily Shaffer, Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan

Emily Shaffer wrote:
> On Fri, May 28, 2021 at 02:11:02PM +0200, Ævar Arnfjörð Bjarmason wrote:
> > 
> > After suggesting[1] an another round that the config-based-hook
> > topic[2] should take a more incremental approach to reach its end goal
> > I thought I'd try hacking that up.
> > 
> > So this is a proposed restart of that topic which if the consensus
> > favors it should replace it, and the config-based hooks topic should
> > be rebased on top of this.
> 
> I'm not entirely sure what you're trying to achieve by sending this
> series.

Have a better review? By having a more reviwable series?

> It was my impression that the existing config-based-hooks topic
> was close to being ready to submit anyway (since Junio mentioned
> submitting it a couple revisions ago); rather than churning by reviewing
> a different 31-patch topic, and then re-rolling and re-reviewing a
> (reduced) config hook topic, wouldn't it be easier on everyone's time to
> do a final incremental review on the existing topic and then start in on
> bugfixes/feature patches afterwards?

Not to me. I tried to review your series and I just couldn't.

Maybe it's something about me, but the whole point of a review is to
have as many eyes as possible on the code in order to spot any potential
issues. If some eyes have trouble parsing the patches, that's not
optimal.

Ævar's approach is easy for me to follow.

> It would have been nice to see a more clear discussion of patch
> organization sometime much sooner in the past year and a half since the
> project was proposed[3], like maybe in the few iterations of the design
> doc which included a rollout plan in July of last year[4].

Not all of us are being paid by a big corporation to work on Git.

Some of us are doing the work on our own free time. You can't demand we
spend our own free time on a certain patch series as soon as possible
because it's more convenient for $corporation.

Git is an open source community project.

> To me, it seems late to be overhauling the direction like this,
> especially after I asked for opinions and approval on the direction
> before I started work in earnest.

To me it's completely the opposite; it's never too late to overwhaul a
patch series.

In my opinion you should be thankful that somebody took the time to try
to improve your series *for free*.

Cheers.

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 00/31] minimal restart of "config-based-hooks"
  2021-06-01 20:50         ` Derrick Stolee
@ 2021-06-02  5:42           ` Felipe Contreras
  2021-06-02  7:46             ` Ævar Arnfjörð Bjarmason
  2021-06-02  9:34           ` Ævar Arnfjörð Bjarmason
  1 sibling, 1 reply; 479+ messages in thread
From: Felipe Contreras @ 2021-06-02  5:42 UTC (permalink / raw)
  To: Derrick Stolee, Emily Shaffer, Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan

Derrick Stolee wrote:
> On 6/1/2021 2:14 PM, Emily Shaffer wrote:
> > On Fri, May 28, 2021 at 02:11:02PM +0200, Ævar Arnfjörð Bjarmason wrote:
> >>
> >> After suggesting[1] an another round that the config-based-hook
> >> topic[2] should take a more incremental approach to reach its end goal
> >> I thought I'd try hacking that up.
> 
> I think sending this complete reorganization of a long-lived topic
> is not helpful, especially because the end-to-end diff is significant.

The end-to-end diff is significant because it's not the full patch
series.

Give me pointers to the two branches and I'll get you a patch series on
top of Ævar's that gets you *exactly* zero end-to-end diff to Emily's
series.

There's many paths to end up with exactly the same code.

I do it all the time.

> I've also seen messages as early as January where Ævar mentioned
> wanting to review the series, but not finding the time to do so.
> It is reasonable to expect that contributors attempt such major
> reorganizations according to reviewers feedback, as long as the
> reviewers are timely about delivering that feedback.

The Git project doesn't have deadlines.

Code should be merged when it's ready to be merged, not when it's
convenient for $company.

I have patches that have been stuck for 7 years. Why should $company get
a fast-path, an exception, or preferential treatment?

Cheers.

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 00/31] minimal restart of "config-based-hooks"
  2021-06-02  5:42           ` Felipe Contreras
@ 2021-06-02  7:46             ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-02  7:46 UTC (permalink / raw)
  To: Felipe Contreras
  Cc: Derrick Stolee, Emily Shaffer, git, Junio C Hamano, Jeff King,
	Taylor Blau, Eric Sunshine, brian m . carlson, Josh Steadmon,
	Jonathan Tan


On Wed, Jun 02 2021, Felipe Contreras wrote:

> Derrick Stolee wrote:
>> On 6/1/2021 2:14 PM, Emily Shaffer wrote:
>> > On Fri, May 28, 2021 at 02:11:02PM +0200, Ævar Arnfjörð Bjarmason wrote:
>> >>
>> >> After suggesting[1] an another round that the config-based-hook
>> >> topic[2] should take a more incremental approach to reach its end goal
>> >> I thought I'd try hacking that up.
>> 
>> I think sending this complete reorganization of a long-lived topic
>> is not helpful, especially because the end-to-end diff is significant.
>
> The end-to-end diff is significant because it's not the full patch
> series.
>
> Give me pointers to the two branches and I'll get you a patch series on
> top of Ævar's that gets you *exactly* zero end-to-end diff to Emily's
> series.

At https://github.com/avar/git this series is at
es-avar/config-based-hooks-2 (there's an unsubmitted *-3 with the minor
fix I already noted, pending any other feedback).

Then es/config-based-hooks-v9-basis-for-es-avar/config-based-hooks-2 is
Emily's v9 as applied by Junio, which is what I started work on. Right
now it's identical to Junio's (at https://github.com/gitster/git)
gitster/es/config-based-hooks, I just saved it away in case that branch
moves.

A diff --stat -p between es-avar/config-based-hooks-2~2 and that topic
follows, i.e. my topic submitted here minus my last two fixes-on-top
patches.

I.e. this would be more-or-less the diff we'd apply to get to the
end-state (minus reverting a few bugfixes etc. I made along the line,
e.g. I forgot to mention that I added the "git hook run
--ignore-missing" to fix another TOCTOU race, except one introduced by
Emily's series).

 Documentation/Makefile                           |   1 +
 Documentation/config/hook.txt                    |  27 ++
 Documentation/git-hook.txt                       | 157 +++++++--
 Documentation/githooks.txt                       |  79 ++++-
 Documentation/technical/api-parse-options.txt    |   7 +
 Documentation/technical/config-based-hooks.txt   | 369 +++++++++++++++++++++
 builtin/am.c                                     |  27 +-
 builtin/bugreport.c                              |   2 +-
 builtin/checkout.c                               |   4 +-
 builtin/clone.c                                  |   3 +-
 builtin/commit.c                                 |  12 +-
 builtin/gc.c                                     |   3 +-
 builtin/hook.c                                   | 182 +++++++++--
 builtin/merge.c                                  |  13 +-
 builtin/pack-objects.c                           | 174 +---------
 builtin/rebase.c                                 |   3 +-
 builtin/receive-pack.c                           |  51 ++-
 builtin/worktree.c                               |   7 +-
 commit.c                                         |  10 +-
 commit.h                                         |   3 +-
 git-p4.py                                        |  13 +-
 git-send-email.perl                              |  35 +-
 git.c                                            |   2 +-
 hook.c                                           | 397 +++++++++++++++++++----
 hook.h                                           | 109 ++++---
 pack-objects.h                                   | 159 +++++++++
 parse-options-cb.c                               |  16 +
 parse-options.h                                  |   4 +
 perl/Git.pm                                      |  13 +
 read-cache.c                                     |   3 +-
 refs.c                                           |   8 +-
 reset.c                                          |   4 +-
 run-command.c                                    |   1 -
 sequencer.c                                      |  16 +-
 t/helper/test-parse-options.c                    |   6 +
 t/t0040-parse-options.sh                         |  27 ++
 t/t1092-sparse-checkout-compatibility.sh         |   6 +-
 t/t1350-config-hooks-path.sh                     |   1 -
 t/t1360-config-based-hooks.sh                    | 329 +++++++++++++++++++
 t/t1416-ref-transaction-hooks.sh                 |  12 +-
 t/t1800-hook.sh                                  | 158 ---------
 t/t2080-parallel-checkout-basics.sh              |   2 +-
 t/t5411/test-0015-too-many-hooks-error.sh        |  47 +++
 t/t6500-gc.sh                                    |  46 ---
 t/t7503-pre-commit-and-pre-merge-commit-hooks.sh |  17 +-
 t/t9001-send-email.sh                            |  16 +-
 transport.c                                      |   6 +-
 47 files changed, 1941 insertions(+), 646 deletions(-)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 2aae4c9cbb3..5d19eddb0eb 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -90,6 +90,7 @@ SP_ARTICLES += $(API_DOCS)
 TECH_DOCS += MyFirstContribution
 TECH_DOCS += MyFirstObjectWalk
 TECH_DOCS += SubmittingPatches
+TECH_DOCS += technical/config-based-hooks
 TECH_DOCS += technical/hash-function-transition
 TECH_DOCS += technical/http-protocol
 TECH_DOCS += technical/index-format
diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
new file mode 100644
index 00000000000..4f66bb35cf8
--- /dev/null
+++ b/Documentation/config/hook.txt
@@ -0,0 +1,27 @@
+hook.<command>.command::
+	A command to execute during the <command> hook event. This can be an
+	executable on your device, a oneliner for your shell, or the name of a
+	hookcmd. See linkgit:git-hook[1].
+
+hookcmd.<name>.command::
+	A command to execute during a hook for which <name> has been specified
+	as a command. This can be an executable on your device or a oneliner for
+	your shell. See linkgit:git-hook[1].
+
+hookcmd.<name>.skip::
+	Specify this boolean to remove a command from earlier in the execution
+	order. Useful if you want to make a single repo an exception to hook
+	configured at the system or global scope. If there is no hookcmd
+	specified for the command you want to skip, you can use the value of
+	`hook.<command>.command` as <name> as a shortcut. The "skip" setting
+	must be specified after the "hook.<command>.command" to have an effect.
+
+hook.runHookDir::
+	Controls how hooks contained in your hookdir are executed. Can be any of
+	"yes", "warn", "interactive", or "no". Defaults to "yes". See
+	linkgit:git-hook[1] and linkgit:git-config[1] "core.hooksPath").
+
+hook.jobs::
+	Specifies how many hooks can be run simultaneously during parallelized
+	hook execution. If unspecified, defaults to the number of processors on
+	the current system.
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 816b3eda460..24e00a6f4af 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -3,46 +3,159 @@ git-hook(1)
 
 NAME
 ----
-git-hook - run git hooks
+git-hook - Manage configured hooks
 
 SYNOPSIS
 --------
 [verse]
-'git hook' run [--to-stdin=<path>] [--ignore-missing] <hook-name> [-- <hook-args>]
+'git hook' list <hook-name>
+'git hook' run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] [--to-stdin=<path>]
+	[(-j|--jobs) <n>] <hook-name>
 
 DESCRIPTION
 -----------
+You can list and run configured hooks with this command. Later, you will be able
+to add and modify hooks with this command.
 
-This command is an interface to git hooks (see linkgit:githooks[5]).
-Currently it only provides a convenience wrapper for running hooks for
-use by git itself. In the future it might gain other functionality.
+In general, when instructions suggest adding a script to
+`.git/hooks/<something>`, you can specify it in the config instead by running
+`git config --add hook.<something>.command <path-to-script>` - this way you can
+share the script between multiple repos. That is, `cp ~/my-script.sh
+~/project/.git/hooks/pre-commit` would become `git config --add
+hook.pre-commit.command ~/my-script.sh`.
 
-SUBCOMMANDS
------------
+This command parses the default configuration files for sections `hook` and
+`hookcmd`. `hook` is used to describe the commands which will be run during a
+particular hook event; commands are run in the order Git encounters them during
+the configuration parse (see linkgit:git-config[1]). `hookcmd` is used to
+describe attributes of a specific command. If additional attributes don't need
+to be specified, a command to run can be specified directly in the `hook`
+section; if a `hookcmd` by that name isn't found, Git will attempt to run the
+provided value directly. For example:
+
+Global config
+----
+  [hook "post-commit"]
+    command = "linter"
+    command = "~/typocheck.sh"
+
+  [hookcmd "linter"]
+    command = "/bin/linter --c"
+----
+
+Local config
+----
+  [hook "prepare-commit-msg"]
+    command = "linter"
+  [hook "post-commit"]
+    command = "python ~/run-test-suite.py"
+----
+
+With these configs, you'd then see:
+
+----
+$ git hook list "post-commit"
+global: /bin/linter --c
+global: ~/typocheck.sh
+local: python ~/run-test-suite.py
+
+$ git hook list "prepare-commit-msg"
+local: /bin/linter --c
+----
+
+If there is a command you wish to run in most cases but have one or two
+exceptional repos where it should be skipped, you can use specify
+`hookcmd.<name>.skip`, for example:
+
+System config
+----
+  [hook "pre-commit"]
+    command = check-for-secrets
+
+  [hookcmd "check-for-secrets"]
+    command = /bin/secret-checker --aggressive
+----
 
-run::
+Local config
+----
+  [hookcmd "check-for-secrets"]
+    skip = true
+  # This works for inlined hook commands, too:
+  [hookcmd "~/typocheck.sh"]
+    skip = true
+----
 
-	Run the `<hook-name>` hook. Any positional arguments to the
-	hook should be passed after an optional "--" (or
-	"--end-of-options"). See "OPTIONS" below for the arguments
-	this accepts.
+After these configs are added, the hook list becomes:
+
+----
+$ git hook list "post-commit"
+global: /bin/linter --c
+local: python ~/run-test-suite.py
+
+$ git hook list "pre-commit"
+no commands configured for hook 'pre-commit'
+----
+
+COMMANDS
+--------
+
+list `<hook-name>`::
+
+List the hooks which have been configured for `<hook-name>`. Hooks appear
+in the order they should be run, and print the config scope where the relevant
+`hook.<hook-name>.command` was specified, not the `hookcmd` (if applicable).
+This output is human-readable and the format is subject to change over time.
+
+run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] [--to-stdin=<path>] [(-j|--jobs)<n>] `<hook-name>`::
+
+Runs hooks configured for `<hook-name>`, in the same order displayed by `git
+hook list`. Hooks configured this way may be run prepended with `sh -c`, so
+paths containing special characters or spaces should be wrapped in single
+quotes: `command = '/my/path with spaces/script.sh' some args`.
 
 OPTIONS
 -------
+--run-hookdir::
+	Overrides the hook.runHookDir config. Must be 'yes', 'warn',
+	'interactive', or 'no'. Specifies how to handle hooks located in the Git
+	hook directory (core.hooksPath).
+
+-a::
+--arg::
+	Only valid for `run`.
++
+Specify arguments to pass to every hook that is run.
+
+-e::
+--env::
+	Only valid for `run`.
++
+Specify environment variables to set for every hook that is run.
 
 --to-stdin::
-	For "run"; Specify a file which will be streamed into the
-	hook's stdin. The hook will receive the entire file from
-	beginning to EOF.
+	Only valid for `run`.
++
+Specify a file which will be streamed into stdin for every hook that is run.
+Each hook will receive the entire file from beginning to EOF.
 
---ignore-missing::
-	Ignore any missing hook by quietly returning zero. Used for
-	tools that want to do a blind one-shot run of a hook that may
-	or may not be present.
+-j::
+--jobs::
+	Only valid for `run`.
++
+Specify how many hooks to run simultaneously. If this flag is not specified, use
+the value of the `hook.jobs` config. If the config is not specified, use the
+number of CPUs on the current system. Some hooks may be ineligible for
+parallelization: for example, 'commit-msg' intends hooks modify the commit
+message body and cannot be parallelized.
 
-SEE ALSO
---------
-linkgit:githooks[5]
+HOOKS
+-----
+For a list of hooks which can be configured and how they work, see
+linkgit:githooks[5].
+
+CONFIGURATION
+-------------
+include::config/hook.txt[]
 
 GIT
 ---
diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index a16e62bc8c8..d780cb3b18d 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -7,15 +7,16 @@ githooks - Hooks used by Git
 
 SYNOPSIS
 --------
+'git hook'
 $GIT_DIR/hooks/* (or \`git config core.hooksPath`/*)
 
 
 DESCRIPTION
 -----------
 
-Hooks are programs you can place in a hooks directory to trigger
-actions at certain points in git's execution. Hooks that don't have
-the executable bit set are ignored.
+Hooks are programs you can specify in your config (see linkgit:git-hook[1]) or
+place in a hooks directory to trigger actions at certain points in git's
+execution. Hooks that don't have the executable bit set are ignored.
 
 By default the hooks directory is `$GIT_DIR/hooks`, but that can be
 changed via the `core.hooksPath` configuration variable (see
@@ -58,6 +59,9 @@ the message file.
 The default 'applypatch-msg' hook, when enabled, runs the
 'commit-msg' hook, if the latter is enabled.
 
+Hooks run during 'applypatch-msg' will not be parallelized, because hooks are
+expected to edit the file holding the commit log message.
+
 pre-applypatch
 ~~~~~~~~~~~~~~
 
@@ -73,6 +77,9 @@ make a commit if it does not pass certain test.
 The default 'pre-applypatch' hook, when enabled, runs the
 'pre-commit' hook, if the latter is enabled.
 
+Hooks run during 'pre-applypatch' will be run in parallel, unless hook.jobs is
+configured to 1.
+
 post-applypatch
 ~~~~~~~~~~~~~~~
 
@@ -82,6 +89,9 @@ and is invoked after the patch is applied and a commit is made.
 This hook is meant primarily for notification, and cannot affect
 the outcome of `git am`.
 
+Hooks run during 'post-applypatch' will be run in parallel, unless hook.jobs is
+configured to 1.
+
 pre-commit
 ~~~~~~~~~~
 
@@ -103,6 +113,8 @@ The default 'pre-commit' hook, when enabled--and with the
 `hooks.allownonascii` config option unset or set to false--prevents
 the use of non-ASCII filenames.
 
+Hooks executed during 'pre-commit' will not be parallelized.
+
 pre-merge-commit
 ~~~~~~~~~~~~~~~~
 
@@ -125,6 +137,8 @@ need to be resolved and the result committed separately (see
 linkgit:git-merge[1]). At that point, this hook will not be executed,
 but the 'pre-commit' hook will, if it is enabled.
 
+Hooks executed during 'pre-merge-commit' will not be parallelized.
+
 prepare-commit-msg
 ~~~~~~~~~~~~~~~~~~
 
@@ -150,6 +164,9 @@ be used as replacement for pre-commit hook.
 The sample `prepare-commit-msg` hook that comes with Git removes the
 help message found in the commented portion of the commit template.
 
+Hooks executed during 'prepare-commit-msg' will not be parallelized, because
+hooks are expected to edit the file containing the commit log message.
+
 commit-msg
 ~~~~~~~~~~
 
@@ -166,6 +183,9 @@ file.
 The default 'commit-msg' hook, when enabled, detects duplicate
 `Signed-off-by` trailers, and aborts the commit if one is found.
 
+Hooks executed during 'commit-msg' will not be parallelized, because hooks are
+expected to edit the file containing the proposed commit log message.
+
 post-commit
 ~~~~~~~~~~~
 
@@ -175,6 +195,9 @@ invoked after a commit is made.
 This hook is meant primarily for notification, and cannot affect
 the outcome of `git commit`.
 
+Hooks executed during 'post-commit' will run in parallel, unless hook.jobs is
+configured to 1.
+
 pre-rebase
 ~~~~~~~~~~
 
@@ -184,6 +207,9 @@ two parameters.  The first parameter is the upstream from which
 the series was forked.  The second parameter is the branch being
 rebased, and is not set when rebasing the current branch.
 
+Hooks executed during 'pre-rebase' will run in parallel, unless hook.jobs is
+configured to 1.
+
 post-checkout
 ~~~~~~~~~~~~~
 
@@ -206,6 +232,8 @@ This hook can be used to perform repository validity checks, auto-display
 differences from the previous HEAD if different, or set working dir metadata
 properties.
 
+Hooks executed during 'post-checkout' will not be parallelized.
+
 post-merge
 ~~~~~~~~~~
 
@@ -220,6 +248,9 @@ save and restore any form of metadata associated with the working tree
 (e.g.: permissions/ownership, ACLS, etc).  See contrib/hooks/setgitperms.perl
 for an example of how to do this.
 
+Hooks executed during 'post-merge' will run in parallel, unless hook.jobs is
+configured to 1.
+
 pre-push
 ~~~~~~~~
 
@@ -249,6 +280,9 @@ If this hook exits with a non-zero status, `git push` will abort without
 pushing anything.  Information about why the push is rejected may be sent
 to the user by writing to standard error.
 
+Hooks executed during 'pre-push' will run in parallel, unless hook.jobs is
+configured to 1.
+
 [[pre-receive]]
 pre-receive
 ~~~~~~~~~~~
@@ -290,6 +324,8 @@ will be set to zero, `GIT_PUSH_OPTION_COUNT=0`.
 See the section on "Quarantine Environment" in
 linkgit:git-receive-pack[1] for some caveats.
 
+Hooks executed during 'pre-receive' will not be parallelized.
+
 [[update]]
 update
 ~~~~~~
@@ -335,6 +371,9 @@ The default 'update' hook, when enabled--and with
 `hooks.allowunannotated` config option unset or set to false--prevents
 unannotated tags to be pushed.
 
+Hooks executed during 'update' are run in parallel, unless hook.jobs is
+configured to 1.
+
 [[proc-receive]]
 proc-receive
 ~~~~~~~~~~~~
@@ -397,6 +436,10 @@ the input.  The exit status of the 'proc-receive' hook only determines
 the success or failure of the group of commands sent to it, unless
 atomic push is in use.
 
+It is forbidden to specify more than one hook for 'proc-receive'. If a
+globally-configured 'proc-receive' must be overridden, use
+'hookcmd.<global-hook>.skip = true' to ignore it.
+
 [[post-receive]]
 post-receive
 ~~~~~~~~~~~~
@@ -436,6 +479,9 @@ environment variables will not be set. If the client selects
 to use push options, but doesn't transmit any, the count variable
 will be set to zero, `GIT_PUSH_OPTION_COUNT=0`.
 
+Hooks executed during 'post-receive' are run in parallel, unless hook.jobs is
+configured to 1.
+
 [[post-update]]
 post-update
 ~~~~~~~~~~~
@@ -468,6 +514,9 @@ Both standard output and standard error output are forwarded to
 `git send-pack` on the other end, so you can simply `echo` messages
 for the user.
 
+Hooks run during 'post-update' will be run in parallel, unless hook.jobs is
+configured to 1.
+
 reference-transaction
 ~~~~~~~~~~~~~~~~~~~~~
 
@@ -506,6 +555,9 @@ The exit status of the hook is ignored for any state except for the
 cause the transaction to be aborted. The hook will not be called with
 "aborted" state in that case.
 
+Hooks run during 'reference-transaction' will be run in parallel, unless
+hook.jobs is configured to 1.
+
 push-to-checkout
 ~~~~~~~~~~~~~~~~
 
@@ -536,6 +588,7 @@ that switches branches while
 keeping the local changes in the working tree that do not interfere
 with the difference between the branches.
 
+Hooks executed during 'push-to-checkout' will not be parallelized.
 
 pre-auto-gc
 ~~~~~~~~~~~
@@ -544,6 +597,9 @@ This hook is invoked by `git gc --auto` (see linkgit:git-gc[1]). It
 takes no parameter, and exiting with non-zero status from this script
 causes the `git gc --auto` to abort.
 
+Hooks run during 'pre-auto-gc' will be run in parallel, unless hook.jobs is
+configured to 1.
+
 post-rewrite
 ~~~~~~~~~~~~
 
@@ -569,6 +625,9 @@ The hook always runs after the automatic note copying (see
 "notes.rewrite.<command>" in linkgit:git-config[1]) has happened, and
 thus has access to these notes.
 
+Hooks run during 'post-rewrite' will be run in parallel, unless hook.jobs is
+configured to 1.
+
 The following command-specific comments apply:
 
 rebase::
@@ -591,9 +650,12 @@ e-mails.
 fsmonitor-watchman
 ~~~~~~~~~~~~~~~~~~
 
-This hook is invoked when the configuration option `core.fsmonitor` is
-set to `.git/hooks/fsmonitor-watchman` or `.git/hooks/fsmonitor-watchmanv2`
-depending on the version of the hook to use.
+This hook is invoked when the configuration option `core.fsmonitor` is set to a
+path containing an executable. It *cannot* be specified via the usual
+`hook.fsmonitor-watchman.command` configuration or by providing an executable
+in `.git/hooks/fsmonitor-watchman`. The arguments provided to the hook are
+determined by the value of the `core.fsmonitorHookVersion` configuration
+option.
 
 Version 1 takes two arguments, a version (1) and the time in elapsed
 nanoseconds since midnight, January 1, 1970.
@@ -698,9 +760,8 @@ and "0" meaning they were not.
 Only one parameter should be set to "1" when the hook runs.  The hook
 running passing "1", "1" should not be possible.
 
-SEE ALSO
---------
-linkgit:git-hook[1]
+Hooks run during 'post-index-change' will be run in parallel, unless hook.jobs
+is configured to 1.
 
 GIT
 ---
diff --git a/Documentation/technical/api-parse-options.txt b/Documentation/technical/api-parse-options.txt
index 5a60bbfa7f4..f79b17e7fcd 100644
--- a/Documentation/technical/api-parse-options.txt
+++ b/Documentation/technical/api-parse-options.txt
@@ -173,6 +173,13 @@ There are some macros to easily define options:
 	The string argument is stored as an element in `string_list`.
 	Use of `--no-option` will clear the list of preceding values.
 
+`OPT_STRVEC(short, long, &struct strvec, arg_str, description)`::
+	Introduce an option with a string argument, meant to be specified
+	multiple times.
+	The string argument is stored as an element in `strvec`, and later
+	arguments are added to the same `strvec`.
+	Use of `--no-option` will clear the list of preceding values.
+
 `OPT_INTEGER(short, long, &int_var, description)`::
 	Introduce an option with integer argument.
 	The integer is put into `int_var`.
diff --git a/Documentation/technical/config-based-hooks.txt b/Documentation/technical/config-based-hooks.txt
new file mode 100644
index 00000000000..1f973117e44
--- /dev/null
+++ b/Documentation/technical/config-based-hooks.txt
@@ -0,0 +1,369 @@
+Configuration-based hook management
+===================================
+:sectanchors:
+
+[[motivation]]
+== Motivation
+
+Replace the `.git/hook/hookname` path as the only source of hooks to execute;
+allow users to define hooks using config files, in a way which is friendly to
+users with multiple repos which have similar needs - hooks can be easily shared
+between multiple Git repos.
+
+Redefine "hook" as an event rather than a single script, allowing users to
+perform multiple unrelated actions on a single event.
+
+Make it easier for users to discover Git's hook feature and automate their
+workflows.
+
+[[user-interfaces]]
+== User interfaces
+
+[[config-schema]]
+=== Config schema
+
+Hooks can be introduced by editing the configuration manually. There are two new
+sections added, `hook` and `hookcmd`.
+
+[[config-schema-hook]]
+==== `hook`
+
+Primarily contains subsections for each hook event. The order of variables in
+these subsections defines the hook command execution order; hook commands can be
+specified by setting the value directly to the command if no additional
+configuration is needed, or by setting the value as the name of a `hookcmd`. If
+Git does not find a `hookcmd` whose subsection matches the value of the given
+command string, Git will try to execute the string directly. Hooks are executed
+by passing the resolved command string to the shell. In the future, hook event
+subsections could also contain per-hook-event settings; see
+<<per-hook-event-settings,the section in Future Work>> for more details.
+
+Also contains top-level hook execution settings, for example, `hook.runHookDir`.
+(These settings are described more in <<library,Library>>.)
+
+----
+[hook "pre-commit"]
+  command = perl-linter
+  command = /usr/bin/git-secrets --pre-commit
+
+[hook "pre-applypatch"]
+  command = perl-linter
+  # for illustration purposes; error behavior isn't planned yet
+  error = ignore
+
+[hook]
+  runHookDir = interactive
+----
+
+[[config-schema-hookcmd]]
+==== `hookcmd`
+
+Defines a hook command and its attributes, which will be used when a hook event
+occurs. Unqualified attributes are assumed to apply to this hook during all hook
+events, but event-specific attributes can also be supplied. The example runs
+`/usr/bin/lint-it --language=perl <args passed by Git>`, but for repos which
+include this config, the hook command will be skipped for all events.
+Theoretically, the last line could be used to "un-skip" the hook command for
+`pre-commit` hooks, but this hasn't been scoped or implemented yet.
+
+----
+[hookcmd "perl-linter"]
+  command = /usr/bin/lint-it --language=perl
+  skip = true
+  # for illustration purposes; below hasn't been defined yet
+  pre-commit-skip = false
+----
+
+[[command-line-api]]
+=== Command-line API
+
+Users should be able to view, run, reorder, and create hook commands via the
+command line. External tools should be able to view a list of hooks in the
+correct order to run. Modifier commands (`edit` and `add`) have not been
+implemented yet and may not be if manually editing the config proves usable
+enough.
+
+*`git hook list <hook-event>`*
+
+*`git hook run <hook-event> [-a <arg>]... [-e <env-var>]...`*
+
+*`git hook edit <hook-event>`*
+
+*`git hook add <hook-command> <hook-event> <options...>`*
+
+[[hook-editor]]
+=== Hook editor
+
+The tool which is presented by `git hook edit <hook-command>`. Ideally, this
+tool should be easier to use than manually editing the config, and then produce
+a concise config afterwards. It may take a form similar to `git rebase
+--interactive`. This has not been designed or implemented yet and may not be if
+the config proves usable enough.
+
+[[implementation]]
+== Implementation
+
+[[library]]
+=== Library
+
+`hook.c` and `hook.h` are responsible for interacting with the config files. The
+hook library provides a basic API to call all hooks in config order with more
+complex options passed via `struct run_hooks_opt`:
+
+*`int run_hooks(const char *hookname, struct run_hooks_opt *options)`*
+
+`struct run_hooks_opt` allows callers to set:
+
+- environment variables
+- command-line arguments
+- behavior for the hook command provided by `run-command.h:find_hook()` (see
+  below)
+- a method to provide stdin to each hook, either via a file containing stdin, a
+  `struct string_list` containing a list of lines to print, or a callback
+  function to allow the caller to populate stdin manually
+- a method to process stdout from each hook, e.g. for printing to sideband
+  during a network operation
+- parallelism
+- a custom working directory for hooks to execute in
+
+And this struct can be extended with more options as necessary in the future.
+
+The "legacy" hook provided by `run-command.h:find_hook()` - that is, the hook
+present in `.git/hooks/<hookname>` or
+`$(git config --get core.hooksPath)/<hookname>` - can be handled in a number of
+ways, providing an avenue to deprecate these "legacy" hooks if desired. The
+handling is based on a config `hook.runHookDir`, which is checked against a
+number of cases:
+
+- "no": the legacy hook will not be run
+- "error": Git will print a warning to stderr before ignoring the legacy hook
+- "interactive": Git will prompt the user before running the legacy hook
+- "warn": Git will print a warning to stderr before running the legacy hook
+- "yes" (default): Git will silently run the legacy hook
+
+In case this list is expanded in the future, if a value for `hook.runHookDir` is
+given which Git does not recognize, Git should discard that config entry. For
+example, if "warn" was specified at system level and "junk" was specified at
+global level, Git would resolve the value to "warn"; if the only time the config
+was set was to "junk", Git would use the default value of "yes" (but print a
+warning to the user first to let them know their value is wrong).
+
+`struct hookcmd` is expected to grow in size over time as more functionality is
+added to hooks; so that other parts of the code don't need to understand the
+config schema, `struct hookcmd` should contain logical values instead of string
+pairs.
+
+By default, hook parallelism is chosen based on the semantics of each hook;
+callsites initialize their `struct run_hooks_opt` via one of two macros,
+`RUN_HOOKS_OPT_INIT_SYNC` or `RUN_HOOKS_OPT_INIT_ASYNC`. The default number of
+jobs can be configured in `hook.jobs`; this config applies across all hook
+events. If unset, the value of `online_cpus()` (equivalent to `nproc`) is used.
+
+[[builtin]]
+=== Builtin
+
+`builtin/hook.c` is responsible for providing the frontend. It's responsible for
+formatting user-provided data and then calling the library API to set the
+configs as appropriate. The builtin frontend is not responsible for calling the
+config directly, so that other areas of Git can rely on the hook library to
+understand the most recent config schema for hooks.
+
+[[migration]]
+=== Migration path
+
+[[stage-0]]
+==== Stage 0
+
+Hooks are called by running `run-command.h:find_hook()` with the hookname and
+executing the result. The hook library and builtin do not exist. Hooks only
+exist as specially named scripts within `.git/hooks/`.
+
+[[stage-1]]
+==== Stage 1
+
+`git hook list --porcelain <hook-event>` is implemented. `hook.h:run_hooks()` is
+taught to include `run-command.h:find_hook()` at the end; calls to `find_hook()`
+are replaced with calls to `run_hooks()`. Users can opt-in to config-based hooks
+simply by creating some in their config; otherwise users should remain
+unaffected by the change.
+
+[[stage-2]]
+==== Stage 2
+
+The call to `find_hook()` inside of `run_hooks()` learns to check for a config,
+`hook.runHookDir`. Users can opt into managing their hooks completely via the
+config this way.
+
+[[stage-3]]
+==== Stage 3
+
+`.git/hooks` is removed from the template and the hook directory is considered
+deprecated. To avoid breaking older repos, the default of `hook.runHookDir` is
+not changed, and `find_hook()` is not removed.
+
+[[caveats]]
+== Caveats
+
+[[security]]
+=== Security and repo config
+
+Part of the motivation behind this refactor is to mitigate hooks as an attack
+vector.footnote:[https://lore.kernel.org/git/20171002234517.GV19555@aiede.mtv.corp.google.com/]
+However, as the design stands, users can still provide hooks in the repo-level
+config, which is included when a repo is zipped and sent elsewhere. The
+security of the repo-level config is still under discussion; this design
+generally assumes the repo-level config is secure, which is not true yet. This
+assumption was made to avoid overcomplicating the design. So, this series
+doesn't particularly improve security or resistance to zip attacks.
+
+[[ease-of-use]]
+=== Ease of use
+
+The config schema is nontrivial; that's why it's important for the `git hook`
+modifier commands to be usable. Contributors with UX expertise are encouraged to
+share their suggestions.
+
+[[alternatives]]
+== Alternative approaches
+
+A previous summary of alternatives exists in the
+archives.footnote:[https://lore.kernel.org/git/20191116011125.GG22855@google.com]
+
+The table below shows a number of goals and how they might be achieved with
+config-based hooks, by implementing directory support (i.e.
+'.git/hooks/pre-commit.d'), or as hooks are run today.
+
+.Comparison of alternatives
+|===
+|Feature |Config-based hooks |Hook directories |Status quo
+
+|Supports multiple hooks
+|Natively
+|Natively
+|With user effort
+
+|Supports parallelization
+|Natively
+|Natively
+|No (user's multihook trampoline script would need to handle parallelism)
+
+|Safer for zipped repos
+|A little
+|No
+|No
+
+|Previous hooks just work
+|If configured
+|Yes
+|Yes
+
+|Can install one hook to many repos
+|Yes
+|With symlinks or core.hooksPath
+|With symlinks or core.hooksPath
+
+|Discoverability
+|Findable with 'git help git' or tab-completion via 'git hook' subcommand
+|Findable via improved documentation
+|Same as before
+
+|Hard to run unexpected hook
+|If configured
+|Could be made to warn or look for a config
+|No
+|===
+
+[[status-quo]]
+=== Status quo
+
+Today users can implement multihooks themselves by using a "trampoline script"
+as their hook, and pointing that script to a directory or list of other scripts
+they wish to run.
+
+[[hook-directories]]
+=== Hook directories
+
+Other contributors have suggested Git learn about the existence of a directory
+such as `.git/hooks/<hookname>.d` and execute those hooks in alphabetical order.
+
+[[future-work]]
+== Future work
+
+[[execution-ordering]]
+=== Execution ordering
+
+We may find that config order is insufficient for some users; for example,
+config order makes it difficult to add a new hook to the system or global config
+which runs at the end of the hook list. A new ordering schema should be:
+
+1) Specified by a `hook.order` config, so that users will not unexpectedly see
+their order change;
+
+2) Either dependency or numerically based.
+
+Dependency-based ordering is prone to classic linked-list problems, like a
+cycles and handling of missing dependencies. But, it paves the way for enabling
+parallelization if some tasks truly depend on others.
+
+Numerical ordering makes it tricky for Git to generate suggested ordering
+numbers for each command, but is easy to determine a definitive order.
+
+[[parallelization]]
+=== Parallelization with dependencies
+
+Currently hooks use a naive parallelization scheme or are run in series.  But if
+one hook depends on another's output, then users will want to specify those
+dependencies. If we decide to solve this problem, we may want to look to modern
+build systems for inspiration on how to manage dependencies and parallel tasks.
+
+[[nontrivial-hooks]]
+=== Multihooks and nontrivial output
+
+Some hooks - like 'proc-receive' - don't lend themselves well to multihooks at
+all. In the case of 'proc-receive', for now, multiple hook definitions are
+disallowed. In the future we might be able to conceive a better approach, for
+example, running the hooks in series and using the output from one hook as the
+input to the next.
+
+[[securing-hookdir-hooks]]
+=== Securing hookdir hooks
+
+With the design as written in this doc, it's still possible for a malicious user
+to modify `.git/config` to include `hook.pre-receive.command = rm -rf /`, then
+zip their repo and send it to another user. It may be necessary to teach Git to
+only allow inlined hooks like this if they were configured outside of the local
+scope (in other words, only run hookcmds, and only allow hookcmds to be
+configured in global or system scope); or another approach, like a list of safe
+projects, might be useful. It may also be sufficient (or at least useful) to
+teach a `hook.disableAll` config or similar flag to the Git executable.
+
+[[submodule-inheritance]]
+=== Submodule inheritance
+
+It's possible some submodules may want to run the identical set of hooks that
+their superrepo runs. While a globally-configured hook set is helpful, it's not
+a great solution for users who have multiple repos-with-submodules under the
+same user. It would be useful for submodules to learn how to run hooks from
+their superrepo's config, or inherit that hook setting.
+
+[[per-hook-event-settings]]
+=== Per-hook-event settings
+
+It might be desirable to keep settings specifically for some hook events, but
+not for others - for example, a user may wish to disable hookdir hooks for all
+events but pre-commit, which they haven't had time to convert yet; or, a user
+may wish for execution order settings to differ based on hook event. In that
+case, it would be useful to set something like `hook.pre-commit.executionOrder`
+which would not apply to the 'prepare-commit-msg' hook, for example.
+
+[[glossary]]
+== Glossary
+
+*hook event*
+
+A point during Git's execution where user scripts may be run, for example,
+_prepare-commit-msg_ or _pre-push_.
+
+*hook command*
+
+A user script or executable which will be run on one or more hook events.
diff --git a/builtin/am.c b/builtin/am.c
index 6e4f9c80360..d2534f9a1ff 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -11,7 +11,6 @@
 #include "parse-options.h"
 #include "dir.h"
 #include "run-command.h"
-#include "hook.h"
 #include "quote.h"
 #include "tempfile.h"
 #include "lockfile.h"
@@ -34,6 +33,7 @@
 #include "string-list.h"
 #include "packfile.h"
 #include "repository.h"
+#include "hook.h"
 
 /**
  * Returns the length of the first line of msg.
@@ -445,7 +445,9 @@ static void am_destroy(const struct am_state *state)
 static int run_applypatch_msg_hook(struct am_state *state)
 {
 	int ret;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
+
+	run_hooks_opt_init_sync(&opt);
 
 	assert(state->msg);
 	strvec_push(&opt.args, am_path(state, "final-commit"));
@@ -467,9 +469,11 @@ static int run_applypatch_msg_hook(struct am_state *state)
  */
 static int run_post_rewrite_hook(const struct am_state *state)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
 	int ret;
 
+	run_hooks_opt_init_async(&opt);
+
 	strvec_push(&opt.args, "rebase");
 	opt.path_to_stdin = am_path(state, "rewritten");
 
@@ -1602,14 +1606,17 @@ static void do_commit(const struct am_state *state)
 	struct commit_list *parents = NULL;
 	const char *reflog_msg, *author, *committer = NULL;
 	struct strbuf sb = STRBUF_INIT;
-	struct run_hooks_opt hook_opt_pre = RUN_HOOKS_OPT_INIT;
-	struct run_hooks_opt hook_opt_post = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt;
+
+	run_hooks_opt_init_async(&hook_opt);
 
-	if (run_hooks("pre-applypatch", &hook_opt_pre)) {
-		run_hooks_opt_clear(&hook_opt_pre);
+	if (run_hooks("pre-applypatch", &hook_opt)) {
+		run_hooks_opt_clear(&hook_opt);
 		exit(1);
 	}
 
+	run_hooks_opt_clear(&hook_opt);
+
 	if (write_cache_as_tree(&tree, 0, NULL))
 		die(_("git write-tree failed to write a tree"));
 
@@ -1659,10 +1666,10 @@ static void do_commit(const struct am_state *state)
 		fclose(fp);
 	}
 
-	run_hooks("post-applypatch", &hook_opt_post);
+	run_hooks_opt_init_async(&hook_opt);
+	run_hooks("post-applypatch", &hook_opt);
 
-	run_hooks_opt_clear(&hook_opt_pre);
-	run_hooks_opt_clear(&hook_opt_post);
+	run_hooks_opt_clear(&hook_opt);
 	strbuf_release(&sb);
 }
 
diff --git a/builtin/bugreport.c b/builtin/bugreport.c
index 941c8d5e270..190272ba70f 100644
--- a/builtin/bugreport.c
+++ b/builtin/bugreport.c
@@ -82,7 +82,7 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 
 	for (i = 0; i < ARRAY_SIZE(hook); i++)
-		if (hook_exists(hook[i]))
+		if (hook_exists(hook[i], HOOKDIR_USE_CONFIG))
 			strbuf_addf(hook_info, "%s\n", hook[i]);
 }
 
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 6205ace09f6..1797f05a50e 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -107,9 +107,11 @@ struct branch_info {
 static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit,
 			      int changed)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
 	int rc;
 
+	run_hooks_opt_init_sync(&opt);
+
 	/* "new_commit" can be NULL when checking out from the index before
 	   a commit exists. */
 	strvec_pushl(&opt.args,
diff --git a/builtin/clone.c b/builtin/clone.c
index 6687025bea5..2a2a03bf768 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -776,7 +776,7 @@ static int checkout(int submodule_progress)
 	struct tree *tree;
 	struct tree_desc t;
 	int err = 0;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt;
 
 	if (option_no_checkout)
 		return 0;
@@ -822,6 +822,7 @@ static int checkout(int submodule_progress)
 	if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
 		die(_("unable to write new index file"));
 
+	run_hooks_opt_init_sync(&hook_opt);
 	strvec_pushl(&hook_opt.args, oid_to_hex(null_oid()), oid_to_hex(&oid), "1", NULL);
 	err |= run_hooks("post-checkout", &hook_opt);
 	run_hooks_opt_clear(&hook_opt);
diff --git a/builtin/commit.c b/builtin/commit.c
index dad4e565443..7e01802961f 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -19,7 +19,6 @@
 #include "revision.h"
 #include "wt-status.h"
 #include "run-command.h"
-#include "hook.h"
 #include "refs.h"
 #include "log-tree.h"
 #include "strbuf.h"
@@ -37,6 +36,7 @@
 #include "help.h"
 #include "commit-reach.h"
 #include "commit-graph.h"
+#include "hook.h"
 
 static const char * const builtin_commit_usage[] = {
 	N_("git commit [<options>] [--] <pathspec>..."),
@@ -729,7 +729,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	/* This checks and barfs if author is badly specified */
 	determine_author_info(author_ident);
 
-	if (!no_verify && run_commit_hook(use_editor, index_file, "pre-commit", NULL))
+	if (!no_verify && run_commit_hook(use_editor, 0, index_file, "pre-commit", NULL))
 		return 0;
 
 	if (squash_message) {
@@ -1045,7 +1045,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (!no_verify && hook_exists("pre-commit")) {
+	if (!no_verify && hook_exists("pre-commit", HOOKDIR_USE_CONFIG)) {
 		/*
 		 * Re-read the index as pre-commit hook could have updated it,
 		 * and write it out as a tree.  We must do this before we invoke
@@ -1060,7 +1060,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (run_commit_hook(use_editor, index_file, "prepare-commit-msg",
+	if (run_commit_hook(use_editor, 0, index_file, "prepare-commit-msg",
 			    git_path_commit_editmsg(), hook_arg1, hook_arg2, NULL))
 		return 0;
 
@@ -1077,7 +1077,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	}
 
 	if (!no_verify &&
-	    run_commit_hook(use_editor, index_file, "commit-msg", git_path_commit_editmsg(), NULL)) {
+	    run_commit_hook(use_editor, 0, index_file, "commit-msg", git_path_commit_editmsg(), NULL)) {
 		return 0;
 	}
 
@@ -1830,7 +1830,7 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 
 	repo_rerere(the_repository, 0);
 	run_auto_maintenance(quiet);
-	run_commit_hook(use_editor, get_index_file(), "post-commit", NULL);
+	run_commit_hook(use_editor, 1, get_index_file(), "post-commit", NULL);
 	if (amend && !no_post_rewrite) {
 		commit_post_rewrite(the_repository, current_head, &oid);
 	}
diff --git a/builtin/gc.c b/builtin/gc.c
index a12641a691d..16890b097cd 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -349,7 +349,7 @@ static void add_repack_incremental_option(void)
 
 static int need_to_gc(void)
 {
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt;
 
 	/*
 	 * Setting gc.auto to 0 or negative can disable the
@@ -397,6 +397,7 @@ static int need_to_gc(void)
 	else
 		return 0;
 
+	run_hooks_opt_init_async(&hook_opt);
 	if (run_hooks("pre-auto-gc", &hook_opt)) {
 		run_hooks_opt_clear(&hook_opt);
 		return 0;
diff --git a/builtin/hook.c b/builtin/hook.c
index baaef4dce49..c79a961e801 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -7,53 +7,133 @@
 #include "strvec.h"
 
 static const char * const builtin_hook_usage[] = {
-	N_("git hook run [--to-stdin=<path>] <hook-name> [-- <hook-args>]"),
+	N_("git hook list <hookname>"),
+	N_("git hook run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...]"
+	   "[--to-stdin=<path>] [(-j|--jobs) <count>] <hookname>"),
 	NULL
 };
 
+static enum hookdir_opt should_run_hookdir;
+
+static int list(int argc, const char **argv, const char *prefix)
+{
+	struct list_head *head, *pos;
+	const char *hookname = NULL;
+	struct strbuf hookdir_annotation = STRBUF_INIT;
+
+	struct option list_options[] = {
+		OPT_END(),
+	};
+
+	argc = parse_options(argc, argv, prefix, list_options,
+			     builtin_hook_usage, 0);
+
+	if (argc < 1) {
+		usage_msg_opt(_("You must specify a hook event name to list."),
+			      builtin_hook_usage, list_options);
+	}
+
+	hookname = argv[0];
+
+	head = hook_list(hookname);
+
+	if (list_empty(head)) {
+		printf(_("no commands configured for hook '%s'\n"),
+		       hookname);
+		return 0;
+	}
+
+	list_for_each(pos, head) {
+		struct hook *item = list_entry(pos, struct hook, list);
+		item = list_entry(pos, struct hook, list);
+		if (item) {
+			if (item->from_hookdir) {
+				/*
+				 * TRANSLATORS: do not translate 'hookdir' as
+				 * it matches the config setting.
+				 */
+				switch (should_run_hookdir) {
+				case HOOKDIR_NO:
+					printf(_("hookdir: %s (will not run)\n"),
+					       item->command.buf);
+					break;
+				case HOOKDIR_ERROR:
+					printf(_("hookdir: %s (will error and not run)\n"),
+					       item->command.buf);
+					break;
+				case HOOKDIR_INTERACTIVE:
+					printf(_("hookdir: %s (will prompt)\n"),
+					       item->command.buf);
+					break;
+				case HOOKDIR_WARN:
+					printf(_("hookdir: %s (will warn but run)\n"),
+					       item->command.buf);
+					break;
+				case HOOKDIR_YES:
+				/*
+				 * The default behavior should agree with
+				 * hook.c:configured_hookdir_opt(). HOOKDIR_UNKNOWN should just
+				 * do the default behavior.
+				 */
+				case HOOKDIR_UNKNOWN:
+				default:
+					printf(_("hookdir: %s\n"),
+						 item->command.buf);
+					break;
+				}
+			} else {
+				/*
+				 * TRANSLATORS: "<config scope>: <path>". Both fields
+				 * should be left untranslated; config scope matches the
+				 * output of 'git config --show-scope'. Marked for
+				 * translation to provide better RTL support later.
+				 */
+				printf(_("%s: %s\n"),
+					config_scope_name(item->origin),
+					item->command.buf);
+			}
+		}
+	}
+
+	clear_hook_list(head);
+	strbuf_release(&hookdir_annotation);
+
+	return 0;
+}
+
 static int run(int argc, const char **argv, const char *prefix)
 {
-	int i;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct strbuf hookname = STRBUF_INIT;
+	struct run_hooks_opt opt;
 	int rc = 0;
-	int ignore_missing = 0;
-	const char *hook_name;
-	const char *hook_path;
 
 	struct option run_options[] = {
-		OPT_BOOL(0, "ignore-missing", &ignore_missing,
-			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
+		OPT_STRVEC('e', "env", &opt.env, N_("var"),
+			   N_("environment variables for hook to use")),
+		OPT_STRVEC('a', "arg", &opt.args, N_("args"),
+			   N_("argument to pass to hook")),
 		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
 			   N_("file to read into hooks' stdin")),
+		OPT_INTEGER('j', "jobs", &opt.jobs,
+			    N_("run up to <n> hooks simultaneously")),
 		OPT_END(),
 	};
 
+	run_hooks_opt_init_async(&opt);
+
 	argc = parse_options(argc, argv, prefix, run_options,
-			     builtin_hook_usage, PARSE_OPT_KEEP_UNKNOWN | PARSE_OPT_KEEP_DASHDASH);
-
-	if (argc > 2) {
-		if (strcmp(argv[2], "--") &&
-		    strcmp(argv[2], "--end-of-options"))
-			/* Having a -- for "run" is mandatory */
-			usage_with_options(builtin_hook_usage, run_options);
-		/* Add our arguments, start after -- */
-		for (i = 3 ; i < argc; i++)
-			strvec_push(&opt.args, argv[i]);
-	}
+			     builtin_hook_usage, 0);
 
-	/* Need to take into account core.hooksPath */
-	git_config(git_default_config, NULL);
+	if (argc < 1)
+		usage_msg_opt(_("You must specify a hook event to run."),
+			      builtin_hook_usage, run_options);
 
-	hook_name = argv[1];
-	hook_path = find_hook(hook_name);
-	if (!hook_path) {
-		if (ignore_missing)
-			return 0;
-		error("cannot find a hook named %s", hook_name);
-		return 1;
-	}
-	rc = run_found_hooks(hook_name, hook_path, &opt);
+	strbuf_addstr(&hookname, argv[0]);
+	opt.run_hookdir = should_run_hookdir;
 
+	rc = run_hooks(hookname.buf, &opt);
+
+	strbuf_release(&hookname);
 	run_hooks_opt_clear(&opt);
 
 	return rc;
@@ -61,12 +141,50 @@ static int run(int argc, const char **argv, const char *prefix)
 
 int cmd_hook(int argc, const char **argv, const char *prefix)
 {
+	const char *run_hookdir = NULL;
+
 	struct option builtin_hook_options[] = {
+		OPT_STRING(0, "run-hookdir", &run_hookdir, N_("option"),
+			   N_("what to do with hooks found in the hookdir")),
 		OPT_END(),
 	};
 
-	if (!strcmp(argv[1], "run"))
+	argc = parse_options(argc, argv, prefix, builtin_hook_options,
+			     builtin_hook_usage, PARSE_OPT_KEEP_UNKNOWN);
+
+	/* after the parse, we should have "<command> <hookname> <args...>" */
+	if (argc < 2)
+		usage_with_options(builtin_hook_usage, builtin_hook_options);
+
+	git_config(git_default_config, NULL);
+
+
+	/* argument > config */
+	if (run_hookdir)
+		if (!strcmp(run_hookdir, "no"))
+			should_run_hookdir = HOOKDIR_NO;
+		else if (!strcmp(run_hookdir, "error"))
+			should_run_hookdir = HOOKDIR_ERROR;
+		else if (!strcmp(run_hookdir, "yes"))
+			should_run_hookdir = HOOKDIR_YES;
+		else if (!strcmp(run_hookdir, "warn"))
+			should_run_hookdir = HOOKDIR_WARN;
+		else if (!strcmp(run_hookdir, "interactive"))
+			should_run_hookdir = HOOKDIR_INTERACTIVE;
+		else
+			/*
+			 * TRANSLATORS: leave "yes/warn/interactive/no"
+			 * untranslated; the strings are compared literally.
+			 */
+			die(_("'%s' is not a valid option for --run-hookdir "
+			      "(yes, warn, interactive, no)"), run_hookdir);
+	else
+		should_run_hookdir = configured_hookdir_opt();
+
+	if (!strcmp(argv[0], "list"))
+		return list(argc, argv, prefix);
+	if (!strcmp(argv[0], "run"))
 		return run(argc, argv, prefix);
+
 	usage_with_options(builtin_hook_usage, builtin_hook_options);
-	return 1;
 }
diff --git a/builtin/merge.c b/builtin/merge.c
index a9363b94065..7a524cb3e35 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -13,7 +13,6 @@
 #include "builtin.h"
 #include "lockfile.h"
 #include "run-command.h"
-#include "hook.h"
 #include "diff.h"
 #include "diff-merges.h"
 #include "refs.h"
@@ -44,6 +43,7 @@
 #include "commit-reach.h"
 #include "wt-status.h"
 #include "commit-graph.h"
+#include "hook.h"
 
 #define DEFAULT_TWOHEAD (1<<0)
 #define DEFAULT_OCTOPUS (1<<1)
@@ -448,7 +448,7 @@ static void finish(struct commit *head_commit,
 		   const struct object_id *new_head, const char *msg)
 {
 	struct strbuf reflog_message = STRBUF_INIT;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
 	const struct object_id *head = &head_commit->object.oid;
 
 	if (!msg)
@@ -490,6 +490,7 @@ static void finish(struct commit *head_commit,
 	}
 
 	/* Run a post-merge hook */
+	run_hooks_opt_init_async(&opt);
 	strvec_push(&opt.args, squash ? "1" : "0");
 	run_hooks("post-merge", &opt);
 	run_hooks_opt_clear(&opt);
@@ -845,14 +846,14 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 	struct strbuf msg = STRBUF_INIT;
 	const char *index_file = get_index_file();
 
-	if (!no_verify && run_commit_hook(0 < option_edit, index_file, "pre-merge-commit", NULL))
+	if (!no_verify && run_commit_hook(0 < option_edit, 0, index_file, "pre-merge-commit", NULL))
 		abort_commit(remoteheads, NULL);
 	/*
 	 * Re-read the index as pre-merge-commit hook could have updated it,
 	 * and write it out as a tree.  We must do this before we invoke
 	 * the editor and after we invoke run_status above.
 	 */
-	if (hook_exists("pre-merge-commit"))
+	if (hook_exists("pre-merge-commit", HOOKDIR_USE_CONFIG))
 		discard_cache();
 	read_cache_from(index_file);
 	strbuf_addbuf(&msg, &merge_msg);
@@ -873,7 +874,7 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 		append_signoff(&msg, ignore_non_trailer(msg.buf, msg.len), 0);
 	write_merge_heads(remoteheads);
 	write_file_buf(git_path_merge_msg(the_repository), msg.buf, msg.len);
-	if (run_commit_hook(0 < option_edit, get_index_file(), "prepare-commit-msg",
+	if (run_commit_hook(0 < option_edit, 0, get_index_file(), "prepare-commit-msg",
 			    git_path_merge_msg(the_repository), "merge", NULL))
 		abort_commit(remoteheads, NULL);
 	if (0 < option_edit) {
@@ -881,7 +882,7 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 			abort_commit(remoteheads, NULL);
 	}
 
-	if (!no_verify && run_commit_hook(0 < option_edit, get_index_file(),
+	if (!no_verify && run_commit_hook(0 < option_edit, 0, get_index_file(),
 					  "commit-msg",
 					  git_path_merge_msg(the_repository), NULL))
 		abort_commit(remoteheads, NULL);
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index de00adbb9e0..6ded130e45b 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -37,134 +37,6 @@
 #include "shallow.h"
 #include "promisor-remote.h"
 
-/*
- * Objects we are going to pack are collected in the `to_pack` structure.
- * It contains an array (dynamically expanded) of the object data, and a map
- * that can resolve SHA1s to their position in the array.
- */
-static struct packing_data to_pack;
-
-static inline struct object_entry *oe_delta(
-		const struct packing_data *pack,
-		const struct object_entry *e)
-{
-	if (!e->delta_idx)
-		return NULL;
-	if (e->ext_base)
-		return &pack->ext_bases[e->delta_idx - 1];
-	else
-		return &pack->objects[e->delta_idx - 1];
-}
-
-static inline unsigned long oe_delta_size(struct packing_data *pack,
-					  const struct object_entry *e)
-{
-	if (e->delta_size_valid)
-		return e->delta_size_;
-
-	/*
-	 * pack->delta_size[] can't be NULL because oe_set_delta_size()
-	 * must have been called when a new delta is saved with
-	 * oe_set_delta().
-	 * If oe_delta() returns NULL (i.e. default state, which means
-	 * delta_size_valid is also false), then the caller must never
-	 * call oe_delta_size().
-	 */
-	return pack->delta_size[e - pack->objects];
-}
-
-unsigned long oe_get_size_slow(struct packing_data *pack,
-			       const struct object_entry *e);
-
-static inline unsigned long oe_size(struct packing_data *pack,
-				    const struct object_entry *e)
-{
-	if (e->size_valid)
-		return e->size_;
-
-	return oe_get_size_slow(pack, e);
-}
-
-static inline void oe_set_delta(struct packing_data *pack,
-				struct object_entry *e,
-				struct object_entry *delta)
-{
-	if (delta)
-		e->delta_idx = (delta - pack->objects) + 1;
-	else
-		e->delta_idx = 0;
-}
-
-static inline struct object_entry *oe_delta_sibling(
-		const struct packing_data *pack,
-		const struct object_entry *e)
-{
-	if (e->delta_sibling_idx)
-		return &pack->objects[e->delta_sibling_idx - 1];
-	return NULL;
-}
-
-static inline struct object_entry *oe_delta_child(
-		const struct packing_data *pack,
-		const struct object_entry *e)
-{
-	if (e->delta_child_idx)
-		return &pack->objects[e->delta_child_idx - 1];
-	return NULL;
-}
-
-static inline void oe_set_delta_child(struct packing_data *pack,
-				      struct object_entry *e,
-				      struct object_entry *delta)
-{
-	if (delta)
-		e->delta_child_idx = (delta - pack->objects) + 1;
-	else
-		e->delta_child_idx = 0;
-}
-
-static inline void oe_set_delta_sibling(struct packing_data *pack,
-					struct object_entry *e,
-					struct object_entry *delta)
-{
-	if (delta)
-		e->delta_sibling_idx = (delta - pack->objects) + 1;
-	else
-		e->delta_sibling_idx = 0;
-}
-
-static inline void oe_set_size(struct packing_data *pack,
-			       struct object_entry *e,
-			       unsigned long size)
-{
-	if (size < pack->oe_size_limit) {
-		e->size_ = size;
-		e->size_valid = 1;
-	} else {
-		e->size_valid = 0;
-		if (oe_get_size_slow(pack, e) != size)
-			BUG("'size' is supposed to be the object size!");
-	}
-}
-
-static inline void oe_set_delta_size(struct packing_data *pack,
-				     struct object_entry *e,
-				     unsigned long size)
-{
-	if (size < pack->oe_delta_size_limit) {
-		e->delta_size_ = size;
-		e->delta_size_valid = 1;
-	} else {
-		packing_data_lock(pack);
-		if (!pack->delta_size)
-			ALLOC_ARRAY(pack->delta_size, pack->nr_alloc);
-		packing_data_unlock(pack);
-
-		pack->delta_size[e - pack->objects] = size;
-		e->delta_size_valid = 0;
-	}
-}
-
 #define IN_PACK(obj) oe_in_pack(&to_pack, obj)
 #define SIZE(obj) oe_size(&to_pack, obj)
 #define SET_SIZE(obj,size) oe_set_size(&to_pack, obj, size)
@@ -184,6 +56,13 @@ static const char *pack_usage[] = {
 	NULL
 };
 
+/*
+ * Objects we are going to pack are collected in the `to_pack` structure.
+ * It contains an array (dynamically expanded) of the object data, and a map
+ * that can resolve SHA1s to their position in the array.
+ */
+static struct packing_data to_pack;
+
 static struct pack_idx_entry **written_list;
 static uint32_t nr_result, nr_written, nr_seen;
 static struct bitmap_index *bitmap_git;
@@ -422,17 +301,6 @@ static void copy_pack_data(struct hashfile *f,
 	}
 }
 
-static inline int oe_size_greater_than(struct packing_data *pack,
-				       const struct object_entry *lhs,
-				       unsigned long rhs)
-{
-	if (lhs->size_valid)
-		return lhs->size_ > rhs;
-	if (rhs < pack->oe_size_limit) /* rhs < 2^x <= lhs ? */
-		return 1;
-	return oe_get_size_slow(pack, lhs) > rhs;
-}
-
 /* Return 0 if we will bust the pack-size limit */
 static unsigned long write_no_reuse_object(struct hashfile *f, struct object_entry *entry,
 					   unsigned long limit, int usable_delta)
@@ -774,14 +642,6 @@ static int mark_tagged(const char *path, const struct object_id *oid, int flag,
 	return 0;
 }
 
-static inline unsigned char oe_layer(struct packing_data *pack,
-				     struct object_entry *e)
-{
-	if (!pack->layer)
-		return 0;
-	return pack->layer[e - pack->objects];
-}
-
 static inline void add_to_write_order(struct object_entry **wo,
 			       unsigned int *endp,
 			       struct object_entry *e)
@@ -2371,26 +2231,6 @@ static pthread_mutex_t progress_mutex;
  * progress_mutex for protection.
  */
 
-static inline int oe_size_less_than(struct packing_data *pack,
-				    const struct object_entry *lhs,
-				    unsigned long rhs)
-{
-	if (lhs->size_valid)
-		return lhs->size_ < rhs;
-	if (rhs < pack->oe_size_limit) /* rhs < 2^x <= lhs ? */
-		return 0;
-	return oe_get_size_slow(pack, lhs) < rhs;
-}
-
-static inline void oe_set_tree_depth(struct packing_data *pack,
-				     struct object_entry *e,
-				     unsigned int tree_depth)
-{
-	if (!pack->tree_depth)
-		CALLOC_ARRAY(pack->tree_depth, pack->nr_alloc);
-	pack->tree_depth[e - pack->objects] = tree_depth;
-}
-
 /*
  * Return the size of the object without doing any delta
  * reconstruction (so non-deltas are true object sizes, but deltas
diff --git a/builtin/rebase.c b/builtin/rebase.c
index 2081f6fa8db..fe9f144cad6 100644
--- a/builtin/rebase.c
+++ b/builtin/rebase.c
@@ -1314,7 +1314,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	char *squash_onto_name = NULL;
 	int reschedule_failed_exec = -1;
 	int allow_preemptive_ff = 1;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt;
 	struct option builtin_rebase_options[] = {
 		OPT_STRING(0, "onto", &options.onto_name,
 			   N_("revision"),
@@ -2024,6 +2024,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	}
 
 	/* If a hook exists, give it a chance to interrupt*/
+	run_hooks_opt_init_async(&hook_opt);
 	strvec_pushl(&hook_opt.args, options.upstream_arg, argc ? argv[0] : NULL, NULL);
 	if (!ok_to_skip_pre_rebase &&
 	    run_hooks("pre-rebase", &hook_opt)) {
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index ec90e10477a..f44b58e456d 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -7,7 +7,6 @@
 #include "pkt-line.h"
 #include "sideband.h"
 #include "run-command.h"
-#include "hook.h"
 #include "exec-cmd.h"
 #include "commit.h"
 #include "object.h"
@@ -30,6 +29,7 @@
 #include "commit-reach.h"
 #include "worktree.h"
 #include "shallow.h"
+#include "hook.h"
 
 static const char * const receive_pack_usage[] = {
 	N_("git receive-pack <git-dir>"),
@@ -910,11 +910,13 @@ static int run_receive_hook(struct command *commands,
 			    int skip_broken,
 			    const struct string_list *push_options)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
 	struct receive_hook_feed_context ctx;
 	int rc;
 	struct command *iter = commands;
 
+	run_hooks_opt_init_async(&opt);
+
 	/* if there are no valid commands, don't invoke the hook at all. */
 	while (iter && skip_broken && (iter->error_string || iter->did_not_exist))
 		iter = iter->next;
@@ -956,9 +958,11 @@ static int run_receive_hook(struct command *commands,
 
 static int run_update_hook(struct command *cmd)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
 	int code;
 
+	run_hooks_opt_init_async(&opt);
+
 	strvec_pushl(&opt.args,
 		     cmd->ref_name,
 		     oid_to_hex(&cmd->old_oid),
@@ -1128,11 +1132,38 @@ static int run_proc_receive_hook(struct command *commands,
 	int version = 0;
 	int code;
 
-	argv[0] = find_hook("proc-receive");
-	if (!argv[0]) {
+	struct hook *proc_receive = NULL;
+	struct list_head *pos, *hooks;
+
+	hooks = hook_list("proc-receive");
+
+	list_for_each(pos, hooks) {
+		if (proc_receive) {
+			rp_error("only one 'proc-receive' hook can be specified");
+			return -1;
+		}
+		proc_receive = list_entry(pos, struct hook, list);
+		/* check if the hookdir hook should be ignored */
+		if (proc_receive->from_hookdir) {
+			switch (configured_hookdir_opt()) {
+			case HOOKDIR_INTERACTIVE:
+			case HOOKDIR_NO:
+				proc_receive = NULL;
+				break;
+			default:
+				break;
+			}
+		}
+
+	}
+
+	if (!proc_receive) {
 		rp_error("cannot find hook 'proc-receive'");
 		return -1;
 	}
+
+
+	argv[0] = proc_receive->command.buf;
 	argv[1] = NULL;
 
 	proc.argv = argv;
@@ -1442,7 +1473,9 @@ static const char *push_to_checkout(unsigned char *hash,
 				    struct strvec *env,
 				    const char *work_tree)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
+
+	run_hooks_opt_init_sync(&opt);
 
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
 	strvec_pushv(&opt.env, env->v);
@@ -1477,7 +1510,7 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 
 	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
 
-	if (!hook_exists(push_to_checkout_hook))
+	if (!hook_exists(push_to_checkout_hook, HOOKDIR_USE_CONFIG))
 		retval = push_to_deploy(sha1, &env, work_tree);
 	else
 		retval = push_to_checkout(sha1, &env, work_tree);
@@ -1640,7 +1673,9 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 static void run_update_post_hook(struct command *commands)
 {
 	struct command *cmd;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
+
+	run_hooks_opt_init_async(&opt);
 
 	for (cmd = commands; cmd; cmd = cmd->next) {
 		if (cmd->error_string || cmd->did_not_exist)
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 2ad26a76f4c..017b2cfcb58 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -8,12 +8,12 @@
 #include "branch.h"
 #include "refs.h"
 #include "run-command.h"
-#include "hook.h"
 #include "sigchain.h"
 #include "submodule.h"
 #include "utf8.h"
 #include "worktree.h"
 #include "quote.h"
+#include "hook.h"
 
 static const char * const worktree_usage[] = {
 	N_("git worktree add [<options>] <path> [<commit-ish>]"),
@@ -382,7 +382,9 @@ static int add_worktree(const char *path, const char *refname,
 	 * is_junk is cleared, but do return appropriate code when hook fails.
 	 */
 	if (!ret && opts->checkout) {
-		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		struct run_hooks_opt opt;
+
+		run_hooks_opt_init_sync(&opt);
 
 		strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
 		strvec_pushl(&opt.args,
@@ -391,7 +393,6 @@ static int add_worktree(const char *path, const char *refname,
 			     "1",
 			     NULL);
 		opt.dir = path;
-		opt.absolute_path = 1;
 
 		ret = run_hooks("post-checkout", &opt);
 
diff --git a/commit.c b/commit.c
index e8147a88fc6..0da5b7e7f1a 100644
--- a/commit.c
+++ b/commit.c
@@ -1696,13 +1696,19 @@ size_t ignore_non_trailer(const char *buf, size_t len)
 	return boc ? len - boc : len - cutoff;
 }
 
-int run_commit_hook(int editor_is_used, const char *index_file,
+int run_commit_hook(int editor_is_used, int parallelize, const char *index_file,
 		    const char *name, ...)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
 	va_list args;
 	const char *arg;
 	int ret;
+
+	run_hooks_opt_init_sync(&opt);
+
+	if (parallelize)
+		opt.jobs = configured_hook_jobs();
+
 	strvec_pushf(&opt.env, "GIT_INDEX_FILE=%s", index_file);
 
 	/*
diff --git a/commit.h b/commit.h
index df42eb434f3..a90c094ec27 100644
--- a/commit.h
+++ b/commit.h
@@ -363,7 +363,8 @@ int compare_commits_by_commit_date(const void *a_, const void *b_, void *unused)
 int compare_commits_by_gen_then_commit_date(const void *a_, const void *b_, void *unused);
 
 LAST_ARG_MUST_BE_NULL
-int run_commit_hook(int editor_is_used, const char *index_file, const char *name, ...);
+int run_commit_hook(int editor_is_used, int parallelize, const char *index_file,
+		    const char *name, ...);
 
 /* Sign a commit or tag buffer, storing the result in a header. */
 int sign_with_header(struct strbuf *buf, const char *keyid);
diff --git a/git-p4.py b/git-p4.py
index e76d8df3139..7d770957719 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -207,12 +207,15 @@ def decode_path(path):
         return path
 
 def run_git_hook(cmd, param=[]):
+    """Execute a hook if the hook exists."""
+    if not cmd:
+        return True
+
     """args are specified with -a <arg> -a <arg> -a <arg>"""
-    args = ['git', 'hook', 'run', '--ignore-missing', cmd]
-    if param:
-        args.append("--")
-        for p in param:
-            args.append(p)
+    args = (['git', 'hook', 'run'] +
+            ["-a" + arg for arg in param] +
+            [cmd])
+
     return subprocess.call(args) == 0
 
 def write_pipe(c, stdin):
diff --git a/git-send-email.perl b/git-send-email.perl
index 2ab8dfdbded..b55687453e0 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -213,13 +213,13 @@ sub format_2822_time {
 my $editor;
 
 sub system_or_msg {
-	my ($args, $msg, $cmd_name) = @_;
+	my ($args, $msg) = @_;
 	system(@$args);
 	my $signalled = $? & 127;
 	my $exit_code = $? >> 8;
 	return unless $signalled or $exit_code;
 
-	my @sprintf_args = ($cmd_name ? $cmd_name : $args->[0], $exit_code);
+	my @sprintf_args = ($args->[0], $exit_code);
 	if (defined $msg) {
 		# Quiet the 'redundant' warning category, except we
 		# need to support down to Perl 5.8, so we can't do a
@@ -1958,31 +1958,12 @@ sub unique_email_list {
 sub validate_patch {
 	my ($fn, $xfer_encoding) = @_;
 
-	if ($repo) {
-		my $hook_name = 'sendemail-validate';
-		my $hooks_path = $repo->command_oneline('rev-parse', '--git-path', 'hooks');
-		my $validate_hook = catfile($hooks_path, $hook_name);
-		my $hook_error;
-		if (-x $validate_hook) {
-			my $target = abs_path($fn);
-			# The hook needs a correct cwd and GIT_DIR.
-			my $cwd_save = cwd();
-			chdir($repo->wc_path() or $repo->repo_path())
-				or die("chdir: $!");
-			local $ENV{"GIT_DIR"} = $repo->repo_path();
-			my @validate_hook = ("git", "hook", "run", "--ignore-missing", $hook_name, "--", $target);
-			$hook_error = system_or_msg(\@validate_hook, undef,
-						       "git hook run $hook_name -- <patch>");
-			chdir($cwd_save) or die("chdir: $!");
-		}
-		if ($hook_error) {
-			$hook_error = sprintf(__("fatal: %s: rejected by %s hook\n" .
-						 $hook_error . "\n" .
-						 "warning: no patches were sent\n"),
-					      $fn, $hook_name);
-			die $hook_error;
-		}
-	}
+	my $target = abs_path($fn);
+
+	system_or_die(["git", "hook", "run", "sendemail-validate", "-j1", "-a", $target],
+		sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
+			   "warning: no patches were sent\n"),
+		        $fn));
 
 	# Any long lines will be automatically fixed if we use a suitable transfer
 	# encoding.
diff --git a/git.c b/git.c
index 540909c391f..39988ee3b02 100644
--- a/git.c
+++ b/git.c
@@ -538,7 +538,7 @@ static struct cmd_struct commands[] = {
 	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
 	{ "hash-object", cmd_hash_object },
 	{ "help", cmd_help },
-	{ "hook", cmd_hook, RUN_SETUP },
+	{ "hook", cmd_hook, RUN_SETUP_GENTLY },
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "init", cmd_init_db },
 	{ "init-db", cmd_init_db },
diff --git a/hook.c b/hook.c
index 17ae65eca31..ff80e52eddd 100644
--- a/hook.c
+++ b/hook.c
@@ -1,8 +1,224 @@
 #include "cache.h"
+
 #include "hook.h"
+#include "config.h"
 #include "run-command.h"
+#include "prompt.h"
+
+void free_hook(struct hook *ptr)
+{
+	if (ptr) {
+		strbuf_release(&ptr->command);
+		free(ptr->feed_pipe_cb_data);
+	}
+	free(ptr);
+}
+
+static struct hook * find_hook_by_command(struct list_head *head, const char *command)
+{
+	struct list_head *pos = NULL, *tmp = NULL;
+	struct hook *found = NULL;
+
+	list_for_each_safe(pos, tmp, head) {
+		struct hook *it = list_entry(pos, struct hook, list);
+		if (!strcmp(it->command.buf, command)) {
+		    list_del(pos);
+		    found = it;
+		    break;
+		}
+	}
+	return found;
+}
+
+static void append_or_move_hook(struct list_head *head, const char *command)
+{
+	struct hook *to_add = find_hook_by_command(head, command);
+
+	if (!to_add) {
+		/* adding a new hook, not moving an old one */
+		to_add = xmalloc(sizeof(*to_add));
+		strbuf_init(&to_add->command, 0);
+		strbuf_addstr(&to_add->command, command);
+		to_add->from_hookdir = 0;
+		to_add->feed_pipe_cb_data = NULL;
+	}
+
+	/* re-set the scope so we show where an override was specified */
+	to_add->origin = current_config_scope();
+
+	list_add_tail(&to_add->list, head);
+}
+
+static void remove_hook(struct list_head *to_remove)
+{
+	struct hook *hook_to_remove = list_entry(to_remove, struct hook, list);
+	list_del(to_remove);
+	free_hook(hook_to_remove);
+}
+
+void clear_hook_list(struct list_head *head)
+{
+	struct list_head *pos, *tmp;
+	list_for_each_safe(pos, tmp, head)
+		remove_hook(pos);
+}
+
+struct hook_config_cb
+{
+	struct strbuf *hookname;
+	struct list_head *list;
+};
+
+static int hook_config_lookup(const char *key, const char *value, void *cb_data)
+{
+	struct hook_config_cb *data = cb_data;
+	const char *hook_key = data->hookname->buf;
+	struct list_head *head = data->list;
+
+	if (!strcmp(key, hook_key)) {
+		const char *command = value;
+		struct strbuf hookcmd_name = STRBUF_INIT;
+		int skip = 0;
+
+		/*
+		 * Check if we're removing that hook instead. Hookcmds are
+		 * removed by name, and inlined hooks are removed by command
+		 * content.
+		 */
+		strbuf_addf(&hookcmd_name, "hookcmd.%s.skip", command);
+		git_config_get_bool(hookcmd_name.buf, &skip);
+
+		/*
+		 * Check if a hookcmd with that name exists. If it doesn't,
+		 * 'git_config_get_value()' is documented not to touch &command,
+		 * so we don't need to do anything.
+		 */
+		strbuf_reset(&hookcmd_name);
+		strbuf_addf(&hookcmd_name, "hookcmd.%s.command", command);
+		git_config_get_value(hookcmd_name.buf, &command);
+
+		if (!command) {
+			strbuf_release(&hookcmd_name);
+			BUG("git_config_get_value overwrote a string it shouldn't have");
+		}
+
+		/*
+		 * TODO: implement an option-getting callback, e.g.
+		 *   get configs by pattern hookcmd.$value.*
+		 *   for each key+value, do_callback(key, value, cb_data)
+		 */
+
+		if (skip) {
+			struct hook *to_remove = find_hook_by_command(head, command);
+			if (to_remove)
+				remove_hook(&(to_remove->list));
+		} else {
+			append_or_move_hook(head, command);
+		}
+
+		strbuf_release(&hookcmd_name);
+	}
+
+	return 0;
+}
+
+enum hookdir_opt configured_hookdir_opt(void)
+{
+	const char *key;
+	if (git_config_get_value("hook.runhookdir", &key))
+		return HOOKDIR_YES; /* by default, just run it. */
+
+	if (!strcmp(key, "no"))
+		return HOOKDIR_NO;
+
+	if (!strcmp(key, "error"))
+		return HOOKDIR_ERROR;
+
+	if (!strcmp(key, "yes"))
+		return HOOKDIR_YES;
+
+	if (!strcmp(key, "warn"))
+		return HOOKDIR_WARN;
+
+	if (!strcmp(key, "interactive"))
+		return HOOKDIR_INTERACTIVE;
+
+	return HOOKDIR_UNKNOWN;
+}
+
+int configured_hook_jobs(void)
+{
+	int n = online_cpus();
+	git_config_get_int("hook.jobs", &n);
+
+	return n;
+}
+
+static int should_include_hookdir(const char *path, enum hookdir_opt cfg)
+{
+	struct strbuf prompt = STRBUF_INIT;
+	/*
+	 * If the path doesn't exist, don't bother adding the empty hook and
+	 * don't bother checking the config or prompting the user.
+	 */
+	if (!path)
+		return 0;
+
+	switch (cfg)
+	{
+	case HOOKDIR_ERROR:
+		fprintf(stderr, _("Skipping legacy hook at '%s'\n"),
+			path);
+		/* FALLTHROUGH */
+	case HOOKDIR_NO:
+		return 0;
+	case HOOKDIR_WARN:
+		fprintf(stderr, _("Running legacy hook at '%s'\n"),
+			path);
+		return 1;
+	case HOOKDIR_INTERACTIVE:
+		do {
+			/*
+			 * TRANSLATORS: Make sure to include [Y] and [n]
+			 * in your translation. Only English input is
+			 * accepted. Default option is "yes".
+			 */
+			fprintf(stderr, _("Run '%s'? [Y/n] "), path);
+			git_read_line_interactively(&prompt);
+			/*
+			 * In case of prompt = '' - that is, user hit enter,
+			 * saying "yes I want the default" - strncasecmp will
+			 * return 0 regardless. So list the default first.
+			 *
+			 * Case insensitively, accept "y", "ye", or "yes" as
+			 * "yes"; accept "n" or "no" as "no".
+			 */
+			if (!strncasecmp(prompt.buf, "yes", prompt.len)) {
+				strbuf_release(&prompt);
+				return 1;
+			} else if (!strncasecmp(prompt.buf, "no", prompt.len)) {
+				strbuf_release(&prompt);
+				return 0;
+			}
+			/* otherwise, we didn't understand the input */
+		} while (prompt.len); /* an empty reply means default (yes) */
+		return 1;
+	/*
+	 * HOOKDIR_UNKNOWN should match the default behavior, but let's
+	 * give a heads up to the user.
+	 */
+	case HOOKDIR_UNKNOWN:
+		fprintf(stderr,
+			_("Unrecognized value for 'hook.runHookDir'. "
+			  "Is there a typo? "));
+		/* FALLTHROUGH */
+	case HOOKDIR_YES:
+	default:
+		return 1;
+	}
+}
 
-const char *find_hook(const char *name)
+static const char *find_legacy_hook(const char *name)
 {
 	static struct strbuf path = STRBUF_INIT;
 
@@ -36,9 +252,77 @@ const char *find_hook(const char *name)
 	return path.buf;
 }
 
-int hook_exists(const char *name)
+
+struct list_head* hook_list(const char* hookname)
+{
+	struct strbuf hook_key = STRBUF_INIT;
+	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
+	struct hook_config_cb cb_data = { &hook_key, hook_head };
+
+	INIT_LIST_HEAD(hook_head);
+
+	if (!hookname)
+		return NULL;
+
+	strbuf_addf(&hook_key, "hook.%s.command", hookname);
+
+	git_config(hook_config_lookup, &cb_data);
+
+	if (have_git_dir()) {
+		const char *legacy_hook_path = find_legacy_hook(hookname);
+
+		/* Unconditionally add legacy hook, but annotate it. */
+		if (legacy_hook_path) {
+			struct hook *legacy_hook;
+
+			append_or_move_hook(hook_head,
+					    absolute_path(legacy_hook_path));
+			legacy_hook = list_entry(hook_head->prev, struct hook,
+						 list);
+			legacy_hook->from_hookdir = 1;
+		}
+	}
+
+	strbuf_release(&hook_key);
+	return hook_head;
+}
+
+void run_hooks_opt_init_sync(struct run_hooks_opt *o)
 {
-	return !!find_hook(name);
+	strvec_init(&o->env);
+	strvec_init(&o->args);
+	o->path_to_stdin = NULL;
+	o->run_hookdir = configured_hookdir_opt();
+	o->jobs = 1;
+	o->dir = NULL;
+	o->feed_pipe = NULL;
+	o->feed_pipe_ctx = NULL;
+	o->consume_sideband = NULL;
+}
+
+void run_hooks_opt_init_async(struct run_hooks_opt *o)
+{
+	run_hooks_opt_init_sync(o);
+	o->jobs = configured_hook_jobs();
+}
+
+int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir)
+{
+	const char *value = NULL; /* throwaway */
+	struct strbuf hook_key = STRBUF_INIT;
+	int could_run_hookdir;
+
+	if (should_run_hookdir == HOOKDIR_USE_CONFIG)
+		should_run_hookdir = configured_hookdir_opt();
+
+	could_run_hookdir = (should_run_hookdir == HOOKDIR_INTERACTIVE ||
+				should_run_hookdir == HOOKDIR_WARN ||
+				should_run_hookdir == HOOKDIR_YES)
+				&& !!find_legacy_hook(hookname);
+
+	strbuf_addf(&hook_key, "hook.%s.command", hookname);
+
+	return (!git_config_get_value(hook_key.buf, &value)) || could_run_hookdir;
 }
 
 void run_hooks_opt_clear(struct run_hooks_opt *o)
@@ -51,8 +335,7 @@ int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb)
 {
 	int *item_idx;
 	struct hook *ctx = pp_task_cb;
-	struct hook_cb_data *hook_cb = pp_cb;
-	struct string_list *to_pipe = hook_cb->options->feed_pipe_ctx;
+	struct string_list *to_pipe = ((struct hook_cb_data*)pp_cb)->options->feed_pipe_ctx;
 
 	/* Bootstrap the state manager if necessary. */
 	if (!ctx->feed_pipe_cb_data) {
@@ -76,10 +359,10 @@ static int pick_next_hook(struct child_process *cp,
 			  void **pp_task_cb)
 {
 	struct hook_cb_data *hook_cb = pp_cb;
-	struct hook *run_me = hook_cb->run_me;
+	struct hook *hook = hook_cb->run_me;
 
-	if (!run_me)
-		BUG("did we not return 1 in notify_hook_finished?");
+	if (!hook)
+		return 0;
 
 	/* reopen the file for stdin; run_command closes it. */
 	if (hook_cb->options->path_to_stdin) {
@@ -92,13 +375,20 @@ static int pick_next_hook(struct child_process *cp,
 	} else {
 		cp->no_stdin = 1;
 	}
+
 	cp->env = hook_cb->options->env.v;
 	cp->stdout_to_stderr = 1;
-	cp->trace2_hook_name = hook_cb->hook_name;
+	cp->trace2_hook_name = hook_cb->hookname;
 	cp->dir = hook_cb->options->dir;
 
+	/*
+	 * Commands from the config could be oneliners, but we know
+	 * for certain that hookdir commands are not.
+	 */
+	cp->use_shell = !hook->from_hookdir;
+
 	/* add command */
-	strvec_push(&cp->args, run_me->hook_path);
+	strvec_push(&cp->args, hook->command.buf);
 
 	/*
 	 * add passed-in argv, without expanding - let the user get back
@@ -107,7 +397,14 @@ static int pick_next_hook(struct child_process *cp,
 	strvec_pushv(&cp->args, hook_cb->options->args.v);
 
 	/* Provide context for errors if necessary */
-	*pp_task_cb = run_me;
+	*pp_task_cb = hook;
+
+	/* Get the next entry ready */
+	if (hook_cb->run_me->list.next == hook_cb->head)
+		hook_cb->run_me = NULL;
+	else
+		hook_cb->run_me = list_entry(hook_cb->run_me->list.next,
+					     struct hook, list);
 
 	return 1;
 }
@@ -122,10 +419,13 @@ static int notify_start_failure(struct strbuf *out,
 	/* |= rc in cb */
 	hook_cb->rc |= 1;
 
-	strbuf_addf(out, _("Couldn't start hook '%s'\n"),
-		    attempted->hook_path);
+	strbuf_addf(out, _("Couldn't start '%s', configured in '%s'\n"),
+		    attempted->command.buf,
+		    attempted->from_hookdir ? "hookdir"
+			: config_scope_name(attempted->origin));
 
-	return 1;
+	/* NEEDSWORK: if halt_on_error is desired, do it here. */
+	return 0;
 }
 
 static int notify_hook_finished(int result,
@@ -138,29 +438,36 @@ static int notify_hook_finished(int result,
 	/* |= rc in cb */
 	hook_cb->rc |= result;
 
-	return 1;
+	/* NEEDSWORK: if halt_on_error is desired, do it here. */
+	return 0;
 }
 
-int run_found_hooks(const char *hook_name, const char *hook_path,
-		    struct run_hooks_opt *options)
-{
-	struct strbuf abs_path = STRBUF_INIT;
-	struct hook my_hook = {
-		.hook_path = hook_path,
-	};
-	struct hook_cb_data cb_data = {
-		.rc = 0,
-		.hook_name = hook_name,
-		.options = options,
-	};
-	if (options->absolute_path) {
-		strbuf_add_absolute_path(&abs_path, hook_path);
-		my_hook.hook_path = abs_path.buf;
+int run_hooks(const char *hookname, struct run_hooks_opt *options)
+{
+	struct list_head *to_run, *pos = NULL, *tmp = NULL;
+	struct hook_cb_data cb_data = { 0, hookname, NULL, NULL, options };
+
+	if (!options)
+		BUG("a struct run_hooks_opt must be provided to run_hooks");
+
+	if (options->path_to_stdin && options->feed_pipe)
+		BUG("choose only one method to populate stdin");
+
+	to_run = hook_list(hookname);
+
+	list_for_each_safe(pos, tmp, to_run) {
+		struct hook *hook = list_entry(pos, struct hook, list);
+
+		if (hook->from_hookdir &&
+		    !should_include_hookdir(hook->command.buf, options->run_hookdir))
+			    list_del(pos);
 	}
-	cb_data.run_me = &my_hook;
 
-	if (options->jobs != 1)
-		BUG("we do not handle %d or any other != 1 job number yet", options->jobs);
+	if (list_empty(to_run))
+		return 0;
+
+	cb_data.head = to_run;
+	cb_data.run_me = list_entry(to_run->next, struct hook, list);
 
 	run_processes_parallel_tr2(options->jobs,
 				   pick_next_hook,
@@ -170,29 +477,7 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
-				   hook_name);
-	if (options->absolute_path)
-		strbuf_release(&abs_path);
+				   hookname);
 
 	return cb_data.rc;
 }
-
-int run_hooks(const char *hook_name, struct run_hooks_opt *options)
-{
-	const char *hook_path;
-	int ret;
-	if (!options)
-		BUG("a struct run_hooks_opt must be provided to run_hooks");
-
-	if (options->path_to_stdin && options->feed_pipe)
-		BUG("choose only one method to populate stdin");
-
-	hook_path = find_hook(hook_name);
-
-	/* Care about nonexistence? Use run_found_hooks() */
-	if (!hook_path)
-		return 0;
-
-	ret = run_found_hooks(hook_name, hook_path, options);
-	return ret;
-}
diff --git a/hook.h b/hook.h
index 5f895032341..f32189380a8 100644
--- a/hook.h
+++ b/hook.h
@@ -1,12 +1,19 @@
-#ifndef HOOK_H
-#define HOOK_H
+#include "config.h"
+#include "list.h"
 #include "strbuf.h"
 #include "strvec.h"
 #include "run-command.h"
 
 struct hook {
-	/* The path to the hook */
-	const char *hook_path;
+	struct list_head list;
+	/*
+	 * Config file which holds the hook.*.command definition.
+	 * (This has nothing to do with the hookcmd.<name>.* configs.)
+	 */
+	enum config_scope origin;
+	/* The literal command to run. */
+	struct strbuf command;
+	unsigned from_hookdir : 1;
 
 	/*
 	 * Use this to keep state for your feed_pipe_fn if you are using
@@ -15,6 +22,32 @@ struct hook {
 	void *feed_pipe_cb_data;
 };
 
+/*
+ * Provides a linked list of 'struct hook' detailing commands which should run
+ * in response to the 'hookname' event, in execution order.
+ */
+struct list_head* hook_list(const char *hookname);
+
+enum hookdir_opt
+{
+	HOOKDIR_USE_CONFIG,
+	HOOKDIR_NO,
+	HOOKDIR_ERROR,
+	HOOKDIR_WARN,
+	HOOKDIR_INTERACTIVE,
+	HOOKDIR_YES,
+	HOOKDIR_UNKNOWN,
+};
+
+/*
+ * Provides the hookdir_opt specified in the config without consulting any
+ * command line arguments.
+ */
+enum hookdir_opt configured_hookdir_opt(void);
+
+/* Provides the number of threads to use for parallel hook execution. */
+int configured_hook_jobs(void);
+
 struct run_hooks_opt
 {
 	/* Environment vars to be set for each hook */
@@ -23,20 +56,15 @@ struct run_hooks_opt
 	/* Args to be passed to each hook */
 	struct strvec args;
 
-	/* Number of threads to parallelize across */
-	int jobs;
-
-	/* Resolve and run the "absolute_path(hook)" instead of
-	 * "hook". Used for "git worktree" hooks
+	/*
+	 * How should the hookdir be handled?
+	 * Leave the run_hooks_opt_init_*() default in most cases; this only needs
+	 * to be overridden if the user can override it at the command line.
 	 */
-	int absolute_path;
-
-	/* Path to initial working directory for subprocess */
-	const char *dir;
+	enum hookdir_opt run_hookdir;
 
 	/* Path to file which should be piped to stdin for each hook */
 	const char *path_to_stdin;
-
 	/*
 	 * Callback and state pointer to ask for more content to pipe to stdin.
 	 * Will be called repeatedly, for each hook. See
@@ -57,13 +85,14 @@ struct run_hooks_opt
 	 * for an example.
 	 */
 	consume_sideband_fn consume_sideband;
-};
 
-#define RUN_HOOKS_OPT_INIT { \
-	.jobs = 1, \
-	.env = STRVEC_INIT, \
-	.args = STRVEC_INIT, \
-}
+	/* Number of threads to parallelize across */
+	int jobs;
+
+	/* Path to initial working directory for subprocess */
+	const char *dir;
+
+};
 
 /*
  * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
@@ -78,35 +107,33 @@ int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb);
  */
 struct hook_cb_data {
 	int rc;
-	const char *hook_name;
+	const char *hookname;
+	struct list_head *head;
 	struct hook *run_me;
 	struct run_hooks_opt *options;
 };
 
-/*
- * Returns the path to the hook file, or NULL if the hook is missing
- * or disabled. Note that this points to static storage that will be
- * overwritten by further calls to find_hook and run_hook_*.
- */
-const char *find_hook(const char *name);
-
-/*
- * A boolean version of find_hook()
- */
-int hook_exists(const char *hookname);
-
+void run_hooks_opt_init_sync(struct run_hooks_opt *o);
+void run_hooks_opt_init_async(struct run_hooks_opt *o);
 void run_hooks_opt_clear(struct run_hooks_opt *o);
 
 /*
- * Calls find_hook(hookname) and runs the hooks (if any) with
- * run_found_hooks().
+ * Returns 1 if any hooks are specified in the config or if a hook exists in the
+ * hookdir. Typically, invoke hook_exsts() like:
+ *   hook_exists(hookname, configured_hookdir_opt());
+ * Like with run_hooks, if you take a --run-hookdir flag, reflect that
+ * user-specified behavior here instead.
  */
-int run_hooks(const char *hook_name, struct run_hooks_opt *options);
+int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir);
 
 /*
- * Takes an already resolved hook and runs it. Internally the simpler
- * run_hooks() will call this.
+ * Runs all hooks associated to the 'hookname' event in order. Each hook will be
+ * passed 'env' and 'args'. The file at 'stdin_path' will be closed and reopened
+ * for each hook that runs.
  */
-int run_found_hooks(const char *hookname, const char *hook_path,
-		    struct run_hooks_opt *options);
-#endif
+int run_hooks(const char *hookname, struct run_hooks_opt *options);
+
+/* Free memory associated with a 'struct hook' */
+void free_hook(struct hook *ptr);
+/* Empties the list at 'head', calling 'free_hook()' on each entry */
+void clear_hook_list(struct list_head *head);
diff --git a/pack-objects.h b/pack-objects.h
index dca2351ef94..9d88e3e518f 100644
--- a/pack-objects.h
+++ b/pack-objects.h
@@ -268,10 +268,152 @@ static inline void oe_set_in_pack(struct packing_data *pack,
 	pack->in_pack[e - pack->objects] = p;
 }
 
+static inline struct object_entry *oe_delta(
+		const struct packing_data *pack,
+		const struct object_entry *e)
+{
+	if (!e->delta_idx)
+		return NULL;
+	if (e->ext_base)
+		return &pack->ext_bases[e->delta_idx - 1];
+	else
+		return &pack->objects[e->delta_idx - 1];
+}
+
+static inline void oe_set_delta(struct packing_data *pack,
+				struct object_entry *e,
+				struct object_entry *delta)
+{
+	if (delta)
+		e->delta_idx = (delta - pack->objects) + 1;
+	else
+		e->delta_idx = 0;
+}
+
 void oe_set_delta_ext(struct packing_data *pack,
 		      struct object_entry *e,
 		      const struct object_id *oid);
 
+static inline struct object_entry *oe_delta_child(
+		const struct packing_data *pack,
+		const struct object_entry *e)
+{
+	if (e->delta_child_idx)
+		return &pack->objects[e->delta_child_idx - 1];
+	return NULL;
+}
+
+static inline void oe_set_delta_child(struct packing_data *pack,
+				      struct object_entry *e,
+				      struct object_entry *delta)
+{
+	if (delta)
+		e->delta_child_idx = (delta - pack->objects) + 1;
+	else
+		e->delta_child_idx = 0;
+}
+
+static inline struct object_entry *oe_delta_sibling(
+		const struct packing_data *pack,
+		const struct object_entry *e)
+{
+	if (e->delta_sibling_idx)
+		return &pack->objects[e->delta_sibling_idx - 1];
+	return NULL;
+}
+
+static inline void oe_set_delta_sibling(struct packing_data *pack,
+					struct object_entry *e,
+					struct object_entry *delta)
+{
+	if (delta)
+		e->delta_sibling_idx = (delta - pack->objects) + 1;
+	else
+		e->delta_sibling_idx = 0;
+}
+
+unsigned long oe_get_size_slow(struct packing_data *pack,
+			       const struct object_entry *e);
+static inline unsigned long oe_size(struct packing_data *pack,
+				    const struct object_entry *e)
+{
+	if (e->size_valid)
+		return e->size_;
+
+	return oe_get_size_slow(pack, e);
+}
+
+static inline int oe_size_less_than(struct packing_data *pack,
+				    const struct object_entry *lhs,
+				    unsigned long rhs)
+{
+	if (lhs->size_valid)
+		return lhs->size_ < rhs;
+	if (rhs < pack->oe_size_limit) /* rhs < 2^x <= lhs ? */
+		return 0;
+	return oe_get_size_slow(pack, lhs) < rhs;
+}
+
+static inline int oe_size_greater_than(struct packing_data *pack,
+				       const struct object_entry *lhs,
+				       unsigned long rhs)
+{
+	if (lhs->size_valid)
+		return lhs->size_ > rhs;
+	if (rhs < pack->oe_size_limit) /* rhs < 2^x <= lhs ? */
+		return 1;
+	return oe_get_size_slow(pack, lhs) > rhs;
+}
+
+static inline void oe_set_size(struct packing_data *pack,
+			       struct object_entry *e,
+			       unsigned long size)
+{
+	if (size < pack->oe_size_limit) {
+		e->size_ = size;
+		e->size_valid = 1;
+	} else {
+		e->size_valid = 0;
+		if (oe_get_size_slow(pack, e) != size)
+			BUG("'size' is supposed to be the object size!");
+	}
+}
+
+static inline unsigned long oe_delta_size(struct packing_data *pack,
+					  const struct object_entry *e)
+{
+	if (e->delta_size_valid)
+		return e->delta_size_;
+
+	/*
+	 * pack->delta_size[] can't be NULL because oe_set_delta_size()
+	 * must have been called when a new delta is saved with
+	 * oe_set_delta().
+	 * If oe_delta() returns NULL (i.e. default state, which means
+	 * delta_size_valid is also false), then the caller must never
+	 * call oe_delta_size().
+	 */
+	return pack->delta_size[e - pack->objects];
+}
+
+static inline void oe_set_delta_size(struct packing_data *pack,
+				     struct object_entry *e,
+				     unsigned long size)
+{
+	if (size < pack->oe_delta_size_limit) {
+		e->delta_size_ = size;
+		e->delta_size_valid = 1;
+	} else {
+		packing_data_lock(pack);
+		if (!pack->delta_size)
+			ALLOC_ARRAY(pack->delta_size, pack->nr_alloc);
+		packing_data_unlock(pack);
+
+		pack->delta_size[e - pack->objects] = size;
+		e->delta_size_valid = 0;
+	}
+}
+
 static inline unsigned int oe_tree_depth(struct packing_data *pack,
 					 struct object_entry *e)
 {
@@ -280,6 +422,23 @@ static inline unsigned int oe_tree_depth(struct packing_data *pack,
 	return pack->tree_depth[e - pack->objects];
 }
 
+static inline void oe_set_tree_depth(struct packing_data *pack,
+				     struct object_entry *e,
+				     unsigned int tree_depth)
+{
+	if (!pack->tree_depth)
+		CALLOC_ARRAY(pack->tree_depth, pack->nr_alloc);
+	pack->tree_depth[e - pack->objects] = tree_depth;
+}
+
+static inline unsigned char oe_layer(struct packing_data *pack,
+				     struct object_entry *e)
+{
+	if (!pack->layer)
+		return 0;
+	return pack->layer[e - pack->objects];
+}
+
 static inline void oe_set_layer(struct packing_data *pack,
 				struct object_entry *e,
 				unsigned char layer)
diff --git a/parse-options-cb.c b/parse-options-cb.c
index 3c811e1e4a7..8227499eb6f 100644
--- a/parse-options-cb.c
+++ b/parse-options-cb.c
@@ -207,6 +207,22 @@ int parse_opt_string_list(const struct option *opt, const char *arg, int unset)
 	return 0;
 }
 
+int parse_opt_strvec(const struct option *opt, const char *arg, int unset)
+{
+	struct strvec *v = opt->value;
+
+	if (unset) {
+		strvec_clear(v);
+		return 0;
+	}
+
+	if (!arg)
+		return -1;
+
+	strvec_push(v, arg);
+	return 0;
+}
+
 int parse_opt_noop_cb(const struct option *opt, const char *arg, int unset)
 {
 	return 0;
diff --git a/parse-options.h b/parse-options.h
index a845a9d9527..fcb0f1f31eb 100644
--- a/parse-options.h
+++ b/parse-options.h
@@ -178,6 +178,9 @@ struct option {
 #define OPT_STRING_LIST(s, l, v, a, h) \
 				    { OPTION_CALLBACK, (s), (l), (v), (a), \
 				      (h), 0, &parse_opt_string_list }
+#define OPT_STRVEC(s, l, v, a, h) \
+				    { OPTION_CALLBACK, (s), (l), (v), (a), \
+				      (h), 0, &parse_opt_strvec }
 #define OPT_UYN(s, l, v, h)         { OPTION_CALLBACK, (s), (l), (v), NULL, \
 				      (h), PARSE_OPT_NOARG, &parse_opt_tertiary }
 #define OPT_EXPIRY_DATE(s, l, v, h) \
@@ -297,6 +300,7 @@ int parse_opt_commits(const struct option *, const char *, int);
 int parse_opt_commit(const struct option *, const char *, int);
 int parse_opt_tertiary(const struct option *, const char *, int);
 int parse_opt_string_list(const struct option *, const char *, int);
+int parse_opt_strvec(const struct option *, const char *, int);
 int parse_opt_noop_cb(const struct option *, const char *, int);
 enum parse_opt_result parse_opt_unknown_cb(struct parse_opt_ctx_t *ctx,
 					   const struct option *,
diff --git a/perl/Git.pm b/perl/Git.pm
index 02eacef0c2a..73ebbf80cc6 100644
--- a/perl/Git.pm
+++ b/perl/Git.pm
@@ -619,6 +619,19 @@ sub _prompt {
 
 sub repo_path { $_[0]->{opts}->{Repository} }
 
+=item hooks_path ()
+
+Return path to the hooks directory. Must be called on a repository instance.
+
+=cut
+
+sub hooks_path {
+	my ($self) = @_;
+
+	my $dir = $self->command_oneline('rev-parse', '--git-path', 'hooks');
+	my $abs = abs_path($dir);
+	return $abs;
+}
 
 =item wc_path ()
 
diff --git a/read-cache.c b/read-cache.c
index a17bc30f870..ebb9c190562 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -3132,7 +3132,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 {
 	int ret;
 	int was_full = !istate->sparse_index;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt;
 
 	ret = convert_to_sparse(istate);
 
@@ -3161,6 +3161,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 	else
 		ret = close_lock_file_gently(lock);
 
+	run_hooks_opt_init_async(&hook_opt);
 	strvec_pushl(&hook_opt.args,
 		     istate->updated_workdir ? "1" : "0",
 		     istate->updated_skipworktree ? "1" : "0",
diff --git a/refs.c b/refs.c
index 1149e7e7dcb..32e993aaff3 100644
--- a/refs.c
+++ b/refs.c
@@ -10,7 +10,6 @@
 #include "refs.h"
 #include "refs/refs-internal.h"
 #include "run-command.h"
-#include "hook.h"
 #include "object-store.h"
 #include "object.h"
 #include "tag.h"
@@ -19,6 +18,7 @@
 #include "strvec.h"
 #include "repository.h"
 #include "sigchain.h"
+#include "hook.h"
 
 /*
  * List of all available backends
@@ -2063,12 +2063,14 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 				const char *state)
 {
 	struct strbuf buf = STRBUF_INIT;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
 	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int ret = 0, i;
 	char o[GIT_MAX_HEXSZ + 1], n[GIT_MAX_HEXSZ + 1];
 
-	if (!hook_exists("reference-transaction"))
+	run_hooks_opt_init_async(&opt);
+
+	if (!hook_exists("reference-transaction", HOOKDIR_USE_CONFIG))
 		return ret;
 
 	strvec_push(&opt.args, state);
diff --git a/reset.c b/reset.c
index e6af33b901c..48d45f5b792 100644
--- a/reset.c
+++ b/reset.c
@@ -128,7 +128,9 @@ int reset_head(struct repository *r, struct object_id *oid, const char *action,
 					    reflog_head);
 	}
 	if (run_hook) {
-		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		struct run_hooks_opt opt;
+
+		run_hooks_opt_init_sync(&opt);
 		strvec_pushl(&opt.args,
 			     oid_to_hex(orig ? orig : null_oid()),
 			     oid_to_hex(oid),
diff --git a/run-command.c b/run-command.c
index 4a1a7a10820..2ff76f3c2f1 100644
--- a/run-command.c
+++ b/run-command.c
@@ -8,7 +8,6 @@
 #include "string-list.h"
 #include "quote.h"
 #include "config.h"
-#include "hook.h"
 
 void child_process_init(struct child_process *child)
 {
diff --git a/sequencer.c b/sequencer.c
index ec2761e47d9..3fa76687639 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -8,7 +8,6 @@
 #include "sequencer.h"
 #include "tag.h"
 #include "run-command.h"
-#include "hook.h"
 #include "exec-cmd.h"
 #include "utf8.h"
 #include "cache-tree.h"
@@ -35,6 +34,7 @@
 #include "commit-reach.h"
 #include "rebase-interactive.h"
 #include "reset.h"
+#include "hook.h"
 #include "string-list.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
@@ -1148,11 +1148,13 @@ int update_head_with_reflog(const struct commit *old_head,
 static int run_rewrite_hook(const struct object_id *oldoid,
 			    const struct object_id *newoid)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
 	struct strbuf tmp = STRBUF_INIT;
 	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int code;
 
+	run_hooks_opt_init_async(&opt);
+
 	strvec_push(&opt.args, "amend");
 
 	strbuf_addf(&tmp,
@@ -1204,7 +1206,7 @@ static int run_prepare_commit_msg_hook(struct repository *r,
 	} else {
 		arg1 = "message";
 	}
-	if (run_commit_hook(0, r->index_file, "prepare-commit-msg", name,
+	if (run_commit_hook(0, 0, r->index_file, "prepare-commit-msg", name,
 			    arg1, arg2, NULL))
 		ret = error(_("'prepare-commit-msg' hook failed"));
 
@@ -1442,7 +1444,7 @@ static int try_to_commit(struct repository *r,
 		}
 	}
 
-	if (hook_exists("prepare-commit-msg")) {
+	if (hook_exists("prepare-commit-msg", HOOKDIR_USE_CONFIG)) {
 		res = run_prepare_commit_msg_hook(r, msg, hook_commit);
 		if (res)
 			goto out;
@@ -1534,7 +1536,7 @@ static int try_to_commit(struct repository *r,
 		goto out;
 	}
 
-	run_commit_hook(0, r->index_file, "post-commit", NULL);
+	run_commit_hook(0, 1, r->index_file, "post-commit", NULL);
 	if (flags & AMEND_MSG)
 		commit_post_rewrite(r, current_head, oid);
 
@@ -4524,7 +4526,9 @@ static int pick_commits(struct repository *r,
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
 			struct child_process notes_cp = CHILD_PROCESS_INIT;
-			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+			struct run_hooks_opt hook_opt;
+
+			run_hooks_opt_init_async(&hook_opt);
 
 			notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY);
 			notes_cp.git_cmd = 1;
diff --git a/t/helper/test-parse-options.c b/t/helper/test-parse-options.c
index 2051ce57db7..922af561567 100644
--- a/t/helper/test-parse-options.c
+++ b/t/helper/test-parse-options.c
@@ -2,6 +2,7 @@
 #include "cache.h"
 #include "parse-options.h"
 #include "string-list.h"
+#include "strvec.h"
 #include "trace2.h"
 
 static int boolean = 0;
@@ -15,6 +16,7 @@ static char *string = NULL;
 static char *file = NULL;
 static int ambiguous;
 static struct string_list list = STRING_LIST_INIT_NODUP;
+static struct strvec vector = STRVEC_INIT;
 
 static struct {
 	int called;
@@ -133,6 +135,7 @@ int cmd__parse_options(int argc, const char **argv)
 		OPT_STRING('o', NULL, &string, "str", "get another string"),
 		OPT_NOOP_NOARG(0, "obsolete"),
 		OPT_STRING_LIST(0, "list", &list, "str", "add str to list"),
+		OPT_STRVEC(0, "vector", &vector, "str", "add str to strvec"),
 		OPT_GROUP("Magic arguments"),
 		OPT_ARGUMENT("quux", NULL, "means --quux"),
 		OPT_NUMBER_CALLBACK(&integer, "set integer to NUM",
@@ -183,6 +186,9 @@ int cmd__parse_options(int argc, const char **argv)
 	for (i = 0; i < list.nr; i++)
 		show(&expect, &ret, "list: %s", list.items[i].string);
 
+	for (i = 0; i < vector.nr; i++)
+		show(&expect, &ret, "vector: %s", vector.v[i]);
+
 	for (i = 0; i < argc; i++)
 		show(&expect, &ret, "arg %02d: %s", i, argv[i]);
 
diff --git a/t/t0040-parse-options.sh b/t/t0040-parse-options.sh
index ad4746d899a..485e0170bff 100755
--- a/t/t0040-parse-options.sh
+++ b/t/t0040-parse-options.sh
@@ -35,6 +35,7 @@ String options
     --st <st>             get another string (pervert ordering)
     -o <str>              get another string
     --list <str>          add str to list
+    --vector <str>        add str to strvec
 
 Magic arguments
     --quux                means --quux
@@ -386,6 +387,32 @@ test_expect_success '--no-list resets list' '
 	test_cmp expect output
 '
 
+cat >expect <<\EOF
+boolean: 0
+integer: 0
+magnitude: 0
+timestamp: 0
+string: (not set)
+abbrev: 7
+verbose: -1
+quiet: 0
+dry run: no
+file: (not set)
+vector: foo
+vector: bar
+vector: baz
+EOF
+test_expect_success '--vector keeps list of strings' '
+	test-tool parse-options --vector foo --vector=bar --vector=baz >output &&
+	test_cmp expect output
+'
+
+test_expect_success '--no-vector resets list' '
+	test-tool parse-options --vector=other --vector=irrelevant --vector=options \
+		--no-vector --vector=foo --vector=bar --vector=baz >output &&
+	test_cmp expect output
+'
+
 test_expect_success 'multiple quiet levels' '
 	test-tool parse-options --expect="quiet: 3" -q -q -q
 '
diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh
index e9a815ca7aa..12e6c453024 100755
--- a/t/t1092-sparse-checkout-compatibility.sh
+++ b/t/t1092-sparse-checkout-compatibility.sh
@@ -106,18 +106,18 @@ init_repos () {
 run_on_sparse () {
 	(
 		cd sparse-checkout &&
-		GIT_PROGRESS_DELAY=100000 "$@" >../sparse-checkout-out 2>../sparse-checkout-err
+		"$@" >../sparse-checkout-out 2>../sparse-checkout-err
 	) &&
 	(
 		cd sparse-index &&
-		GIT_PROGRESS_DELAY=100000 "$@" >../sparse-index-out 2>../sparse-index-err
+		"$@" >../sparse-index-out 2>../sparse-index-err
 	)
 }
 
 run_on_all () {
 	(
 		cd full-checkout &&
-		GIT_PROGRESS_DELAY=100000 "$@" >../full-checkout-out 2>../full-checkout-err
+		"$@" >../full-checkout-out 2>../full-checkout-err
 	) &&
 	run_on_sparse "$@"
 }
diff --git a/t/t1350-config-hooks-path.sh b/t/t1350-config-hooks-path.sh
index fa9647a7c0b..f1f9aee9f5d 100755
--- a/t/t1350-config-hooks-path.sh
+++ b/t/t1350-config-hooks-path.sh
@@ -5,7 +5,6 @@ test_description='Test the core.hooksPath configuration variable'
 . ./test-lib.sh
 
 test_expect_success 'set up a pre-commit hook in core.hooksPath' '
-	>actual &&
 	mkdir -p .git/custom-hooks .git/hooks &&
 	write_script .git/custom-hooks/pre-commit <<-\EOF &&
 	echo CUSTOM >>actual
diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh
new file mode 100755
index 00000000000..43917172d74
--- /dev/null
+++ b/t/t1360-config-based-hooks.sh
@@ -0,0 +1,329 @@
+#!/bin/bash
+
+test_description='config-managed multihooks, including git-hook command'
+
+. ./test-lib.sh
+
+ROOT=
+if test_have_prereq MINGW
+then
+	# In Git for Windows, Unix-like paths work only in shell scripts;
+	# `git.exe`, however, will prefix them with the pseudo root directory
+	# (of the Unix shell). Let's accommodate for that.
+	ROOT="$(cd / && pwd)"
+fi
+
+setup_hooks () {
+	test_config hook.pre-commit.command "/path/ghi" --add
+	test_config_global hook.pre-commit.command "/path/def" --add
+}
+
+setup_hookcmd () {
+	test_config hook.pre-commit.command "abc" --add
+	test_config_global hookcmd.abc.command "/path/abc" --add
+}
+
+setup_hookdir () {
+	mkdir .git/hooks
+	write_script .git/hooks/pre-commit <<-EOF
+	echo \"Legacy Hook\"
+	EOF
+	test_when_finished rm -rf .git/hooks
+}
+
+test_expect_success 'git hook rejects commands without a mode' '
+	test_must_fail git hook pre-commit
+'
+
+
+test_expect_success 'git hook rejects commands without a hookname' '
+	test_must_fail git hook list
+'
+
+test_expect_success 'git hook runs outside of a repo' '
+	setup_hooks &&
+
+	cat >expected <<-EOF &&
+	global: $ROOT/path/def
+	EOF
+
+	nongit git config --list --global &&
+
+	nongit git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list orders by config order' '
+	setup_hooks &&
+
+	cat >expected <<-EOF &&
+	global: $ROOT/path/def
+	local: $ROOT/path/ghi
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list dereferences a hookcmd' '
+	setup_hooks &&
+	setup_hookcmd &&
+
+	cat >expected <<-EOF &&
+	global: $ROOT/path/def
+	local: $ROOT/path/ghi
+	local: $ROOT/path/abc
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list reorders on duplicate commands' '
+	setup_hooks &&
+
+	test_config hook.pre-commit.command "/path/def" --add &&
+
+	cat >expected <<-EOF &&
+	local: $ROOT/path/ghi
+	local: $ROOT/path/def
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list shows hooks from the hookdir' '
+	setup_hookdir &&
+
+	cat >expected <<-EOF &&
+	hookdir: $(pwd)/.git/hooks/pre-commit
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'hook.runHookDir = no is respected by list' '
+	setup_hookdir &&
+
+	test_config hook.runHookDir "no" &&
+
+	cat >expected <<-EOF &&
+	hookdir: $(pwd)/.git/hooks/pre-commit (will not run)
+	EOF
+
+	git hook list pre-commit >actual &&
+	# the hookdir annotation is translated
+	test_cmp expected actual &&
+
+	git hook run pre-commit 2>actual &&
+	test_must_be_empty actual
+'
+
+test_expect_success 'hook.runHookDir = error is respected by list' '
+	setup_hookdir &&
+
+	test_config hook.runHookDir "error" &&
+
+	cat >expected <<-EOF &&
+	hookdir: $(pwd)/.git/hooks/pre-commit (will error and not run)
+	EOF
+
+	git hook list pre-commit >actual &&
+	# the hookdir annotation is translated
+	test_cmp expected actual &&
+
+	cat >expected <<-EOF &&
+	Skipping legacy hook at '\''$(pwd)/.git/hooks/pre-commit'\''
+	EOF
+
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'hook.runHookDir = warn is respected by list' '
+	setup_hookdir &&
+
+	test_config hook.runHookDir "warn" &&
+
+	cat >expected <<-EOF &&
+	hookdir: $(pwd)/.git/hooks/pre-commit (will warn but run)
+	EOF
+
+	git hook list pre-commit >actual &&
+	# the hookdir annotation is translated
+	test_cmp expected actual &&
+
+	cat >expected <<-EOF &&
+	Running legacy hook at '\''$(pwd)/.git/hooks/pre-commit'\''
+	"Legacy Hook"
+	EOF
+
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list removes skipped hookcmd' '
+	setup_hookcmd &&
+	test_config hookcmd.abc.skip "true" --add &&
+
+	cat >expected <<-EOF &&
+	no commands configured for hook '\''pre-commit'\''
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list ignores skip referring to unused hookcmd' '
+	test_config hookcmd.abc.command "/path/abc" --add &&
+	test_config hookcmd.abc.skip "true" --add &&
+
+	cat >expected <<-EOF &&
+	no commands configured for hook '\''pre-commit'\''
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list removes skipped inlined hook' '
+	setup_hooks &&
+	test_config hookcmd."$ROOT/path/ghi".skip "true" --add &&
+
+	cat >expected <<-EOF &&
+	global: $ROOT/path/def
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'hook.runHookDir = interactive is respected by list and run' '
+	setup_hookdir &&
+
+	test_config hook.runHookDir "interactive" &&
+
+	cat >expected <<-EOF &&
+	hookdir: $(pwd)/.git/hooks/pre-commit (will prompt)
+	EOF
+
+	git hook list pre-commit >actual &&
+	# the hookdir annotation is translated
+	test_cmp expected actual &&
+
+	test_write_lines n | git hook run pre-commit 2>actual &&
+	! grep "Legacy Hook" actual &&
+
+	test_write_lines y | git hook run pre-commit 2>actual &&
+	grep "Legacy Hook" actual
+'
+
+test_expect_success 'inline hook definitions execute oneliners' '
+	test_config hook.pre-commit.command "echo \"Hello World\"" &&
+
+	echo "Hello World" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'inline hook definitions resolve paths' '
+	write_script sample-hook.sh <<-EOF &&
+	echo \"Sample Hook\"
+	EOF
+
+	test_when_finished "rm sample-hook.sh" &&
+
+	test_config hook.pre-commit.command "\"$(pwd)/sample-hook.sh\"" &&
+
+	echo \"Sample Hook\" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook run can pass args and env vars' '
+	write_script sample-hook.sh <<-\EOF &&
+	echo $1
+	echo $2
+	echo $TEST_ENV_1
+	echo $TEST_ENV_2
+	EOF
+
+	test_config hook.pre-commit.command "\"$(pwd)/sample-hook.sh\"" &&
+
+	cat >expected <<-EOF &&
+	arg1
+	arg2
+	env1
+	env2
+	EOF
+
+	git hook run --arg arg1 \
+		--env TEST_ENV_1=env1 \
+		-a arg2 \
+		-e TEST_ENV_2=env2 \
+		pre-commit 2>actual &&
+
+	test_cmp expected actual
+'
+
+test_expect_success 'hookdir hook included in git hook run' '
+	setup_hookdir &&
+
+	echo \"Legacy Hook\" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'out-of-repo runs excluded' '
+	setup_hooks &&
+
+	nongit test_must_fail git hook run pre-commit
+'
+
+test_expect_success 'hook.runHookDir is tolerant to unknown values' '
+	setup_hookdir &&
+
+	test_config hook.runHookDir "junk" &&
+
+	cat >expected <<-EOF &&
+	hookdir: $(pwd)/.git/hooks/pre-commit
+	EOF
+
+	git hook list pre-commit >actual &&
+	# the hookdir annotation is translated
+	test_cmp expected actual
+'
+
+test_expect_success 'stdin to multiple hooks' '
+	git config --add hook.test.command "xargs -P1 -I% echo a%" &&
+	git config --add hook.test.command "xargs -P1 -I% echo b%" &&
+	test_when_finished "test_unconfig hook.test.command" &&
+
+	cat >input <<-EOF &&
+	1
+	2
+	3
+	EOF
+
+	cat >expected <<-EOF &&
+	a1
+	a2
+	a3
+	b1
+	b2
+	b3
+	EOF
+
+	git hook run --to-stdin=input test 2>actual &&
+	test_cmp expected actual
+'
+
+test_done
diff --git a/t/t1416-ref-transaction-hooks.sh b/t/t1416-ref-transaction-hooks.sh
index 6c941027a81..0a3c3e4a861 100755
--- a/t/t1416-ref-transaction-hooks.sh
+++ b/t/t1416-ref-transaction-hooks.sh
@@ -124,12 +124,12 @@ test_expect_success 'interleaving hook calls succeed' '
 	EOF
 
 	cat >expect <<-EOF &&
-		hooks/update refs/tags/PRE $ZERO_OID $PRE_OID
-		hooks/reference-transaction prepared
-		hooks/reference-transaction committed
-		hooks/update refs/tags/POST $ZERO_OID $POST_OID
-		hooks/reference-transaction prepared
-		hooks/reference-transaction committed
+		$(pwd)/target-repo.git/hooks/update refs/tags/PRE $ZERO_OID $PRE_OID
+		$(pwd)/target-repo.git/hooks/reference-transaction prepared
+		$(pwd)/target-repo.git/hooks/reference-transaction committed
+		$(pwd)/target-repo.git/hooks/update refs/tags/POST $ZERO_OID $POST_OID
+		$(pwd)/target-repo.git/hooks/reference-transaction prepared
+		$(pwd)/target-repo.git/hooks/reference-transaction committed
 	EOF
 
 	git push ./target-repo.git PRE POST &&
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
deleted file mode 100755
index 9e2dd64275c..00000000000
--- a/t/t1800-hook.sh
+++ /dev/null
@@ -1,158 +0,0 @@
-#!/bin/bash
-
-test_description='git-hook command'
-
-. ./test-lib.sh
-
-test_expect_success 'setup .git/hooks' '
-	mkdir .git/hooks
-'
-
-test_expect_success 'git hook run -- nonexistent hook' '
-	cat >stderr.expect <<-\EOF &&
-	error: cannot find a hook named does-not-exist
-	EOF
-	test_expect_code 1 git hook run does-not-exist 2>stderr.actual &&
-	test_cmp stderr.expect stderr.actual
-'
-
-test_expect_success 'git hook run -- nonexistent hook with --ignore-missing' '
-	git hook run --ignore-missing does-not-exist 2>stderr.actual &&
-	test_must_be_empty stderr.actual
-'
-
-test_expect_success 'git hook run -- basic' '
-	write_script .git/hooks/test-hook <<-EOF &&
-	echo Test hook
-	EOF
-
-	cat >expect <<-\EOF &&
-	Test hook
-	EOF
-	git hook run test-hook 2>actual &&
-	test_cmp expect actual
-'
-
-test_expect_success 'git hook run -- stdout and stderr handling' '
-	write_script .git/hooks/test-hook <<-EOF &&
-	echo >&1 Will end up on stderr
-	echo >&2 Will end up on stderr
-	EOF
-
-	cat >stderr.expect <<-\EOF &&
-	Will end up on stderr
-	Will end up on stderr
-	EOF
-	git hook run test-hook >stdout.actual 2>stderr.actual &&
-	test_cmp stderr.expect stderr.actual &&
-	test_must_be_empty stdout.actual
-'
-
-test_expect_success 'git hook run -- exit codes are passed along' '
-	write_script .git/hooks/test-hook <<-EOF &&
-	exit 1
-	EOF
-
-	test_expect_code 1 git hook run test-hook &&
-
-	write_script .git/hooks/test-hook <<-EOF &&
-	exit 2
-	EOF
-
-	test_expect_code 2 git hook run test-hook &&
-
-	write_script .git/hooks/test-hook <<-EOF &&
-	exit 128
-	EOF
-
-	test_expect_code 128 git hook run test-hook &&
-
-	write_script .git/hooks/test-hook <<-EOF &&
-	exit 129
-	EOF
-
-	test_expect_code 129 git hook run test-hook
-'
-
-test_expect_success 'git hook run arg u ments without -- is not allowed' '
-	test_expect_code 129 git hook run test-hook arg u ments
-'
-
-test_expect_success 'git hook run -- pass arguments' '
-	write_script .git/hooks/test-hook <<-\EOF &&
-	echo $1
-	echo $2
-	EOF
-
-	cat >expect <<-EOF &&
-	arg
-	u ments
-	EOF
-
-	git hook run test-hook -- arg "u ments" 2>actual &&
-	test_cmp expect actual
-'
-
-test_expect_success 'git hook run -- out-of-repo runs excluded' '
-	write_script .git/hooks/test-hook <<-EOF &&
-	echo Test hook
-	EOF
-
-	nongit test_must_fail git hook run test-hook
-'
-
-test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
-	mkdir my-hooks &&
-	write_script my-hooks/test-hook <<-EOF &&
-	echo Hook ran >>actual
-	EOF
-
-	cat >expect <<-\EOF &&
-	Test hook
-	Hook ran
-	Hook ran
-	Hook ran
-	Hook ran
-	EOF
-
-	# Test various ways of specifying the path. See also
-	# t1350-config-hooks-path.sh
-	>actual &&
-	git hook run test-hook 2>>actual &&
-	git -c core.hooksPath=my-hooks hook run test-hook 2>>actual &&
-	git -c core.hooksPath=my-hooks/ hook run test-hook 2>>actual &&
-	git -c core.hooksPath="$PWD/my-hooks" hook run test-hook 2>>actual &&
-	git -c core.hooksPath="$PWD/my-hooks/" hook run test-hook 2>>actual &&
-	test_cmp expect actual
-'
-
-test_expect_success 'set up a pre-commit hook in core.hooksPath' '
-	>actual &&
-	mkdir -p .git/custom-hooks .git/hooks &&
-	write_script .git/custom-hooks/pre-commit <<-\EOF &&
-	echo CUSTOM >>actual
-	EOF
-	write_script .git/hooks/pre-commit <<-\EOF
-	echo NORMAL >>actual
-	EOF
-'
-
-test_expect_success 'stdin to hooks' '
-	write_script .git/hooks/test-hook <<-\EOF &&
-	echo BEGIN stdin
-	cat
-	echo END stdin
-	EOF
-
-	cat >expect <<-EOF &&
-	BEGIN stdin
-	hello
-	END stdin
-	EOF
-
-	echo hello >input &&
-	git hook run --to-stdin=input test-hook 2>actual &&
-	test_cmp expect actual
-'
-
-test_done
diff --git a/t/t2080-parallel-checkout-basics.sh b/t/t2080-parallel-checkout-basics.sh
index 3e0f8c675f7..7087818550f 100755
--- a/t/t2080-parallel-checkout-basics.sh
+++ b/t/t2080-parallel-checkout-basics.sh
@@ -114,7 +114,7 @@ do
 
 	test_expect_success "$mode checkout" '
 		repo=various_$mode &&
-		cp -R -P various $repo &&
+		cp -R various $repo &&
 
 		# The just copied files have more recent timestamps than their
 		# associated index entries. So refresh the cached timestamps
diff --git a/t/t5411/test-0015-too-many-hooks-error.sh b/t/t5411/test-0015-too-many-hooks-error.sh
new file mode 100644
index 00000000000..2d645345101
--- /dev/null
+++ b/t/t5411/test-0015-too-many-hooks-error.sh
@@ -0,0 +1,47 @@
+test_expect_success "setup too  many proc-receive hooks (ok, $PROTOCOL)" '
+	write_script "proc-receive" <<-EOF &&
+	printf >&2 "# proc-receive hook\n"
+	test-tool proc-receive -v \
+		-r "ok refs/for/main/topic"
+	EOF
+
+	git -C "$upstream" config --add "hook.proc-receive.command" proc-receive &&
+	cp proc-receive "$upstream/hooks/proc-receive"
+'
+
+# Refs of upstream : main(A)
+# Refs of workbench: main(A)  tags/v123
+# git push         :                       next(A)  refs/for/main/topic(A)
+test_expect_success "proc-receive: reject more than one configured hook" '
+	test_must_fail git -C workbench push origin \
+		HEAD:next \
+		HEAD:refs/for/main/topic \
+		>out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	cat >expect <<-EOF &&
+	remote: # pre-receive hook
+	remote: pre-receive< <ZERO-OID> <COMMIT-A> refs/heads/next
+	remote: pre-receive< <ZERO-OID> <COMMIT-A> refs/for/main/topic
+	remote: error: only one "proc-receive" hook can be specified
+	remote: # post-receive hook
+	remote: post-receive< <ZERO-OID> <COMMIT-A> refs/heads/next
+	To <URL/of/upstream.git>
+	 * [new branch] HEAD -> next
+	 ! [remote rejected] HEAD -> refs/for/main/topic (fail to run proc-receive hook)
+	EOF
+	test_cmp expect actual &&
+	git -C "$upstream" show-ref >out &&
+	make_user_friendly_and_stable_output <out >actual &&
+	cat >expect <<-EOF &&
+	<COMMIT-A> refs/heads/main
+	<COMMIT-A> refs/heads/next
+	EOF
+	test_cmp expect actual
+'
+
+# Refs of upstream : main(A)             next(A)
+# Refs of workbench: main(A)  tags/v123
+test_expect_success "cleanup ($PROTOCOL)" '
+	git -C "$upstream" config --unset "hook.proc-receive.command" "proc-receive" &&
+	git -C "$upstream" update-ref -d refs/heads/next
+'
diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh
index 10c7ae7f09c..60d961b5260 100755
--- a/t/t6500-gc.sh
+++ b/t/t6500-gc.sh
@@ -95,52 +95,6 @@ test_expect_success 'gc --keep-largest-pack' '
 	)
 '
 
-test_expect_success 'pre-auto-gc hook can stop auto gc' '
-	cat >err.expect <<-\EOF &&
-	no gc for you
-	EOF
-
-	git init pre-auto-gc-hook &&
-	(
-		cd pre-auto-gc-hook &&
-		write_script ".git/hooks/pre-auto-gc" <<-\EOF &&
-		echo >&2 no gc for you &&
-		exit 1
-		EOF
-
-		git config gc.auto 3 &&
-		git config gc.autoDetach false &&
-
-		# We need to create two object whose sha1s start with 17
-		# since this is what git gc counts.  As it happens, these
-		# two blobs will do so.
-		test_commit "$(test_oid obj1)" &&
-		test_commit "$(test_oid obj2)" &&
-
-		git gc --auto >../out.actual 2>../err.actual
-	) &&
-	test_must_be_empty out.actual &&
-	test_cmp err.expect err.actual &&
-
-	cat >err.expect <<-\EOF &&
-	will gc for you
-	Auto packing the repository for optimum performance.
-	See "git help gc" for manual housekeeping.
-	EOF
-
-	(
-		cd pre-auto-gc-hook &&
-		write_script ".git/hooks/pre-auto-gc" <<-\EOF &&
-		echo >&2 will gc for you &&
-		exit 0
-		EOF
-		git gc --auto >../out.actual 2>../err.actual
-	) &&
-
-	test_must_be_empty out.actual &&
-	test_cmp err.expect err.actual
-'
-
 test_expect_success 'auto gc with too many loose objects does not attempt to create bitmaps' '
 	test_config gc.auto 3 &&
 	test_config gc.autodetach false &&
diff --git a/t/t7503-pre-commit-and-pre-merge-commit-hooks.sh b/t/t7503-pre-commit-and-pre-merge-commit-hooks.sh
index 606d8d0f089..e9e37130332 100755
--- a/t/t7503-pre-commit-and-pre-merge-commit-hooks.sh
+++ b/t/t7503-pre-commit-and-pre-merge-commit-hooks.sh
@@ -8,8 +8,8 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
 . ./test-lib.sh
 
 HOOKDIR="$(git rev-parse --git-dir)/hooks"
-PRECOMMIT="$HOOKDIR/pre-commit"
-PREMERGE="$HOOKDIR/pre-merge-commit"
+PRECOMMIT="$(pwd)/$HOOKDIR/pre-commit"
+PREMERGE="$(pwd)/$HOOKDIR/pre-merge-commit"
 
 # Prepare sample scripts that write their $0 to actual_hooks
 test_expect_success 'sample script setup' '
@@ -106,6 +106,19 @@ test_expect_success 'with succeeding hook' '
 	test_cmp expected_hooks actual_hooks
 '
 
+# NEEDSWORK: when 'git hook add' and 'git hook remove' have been added, use that
+# instead
+test_expect_success 'with succeeding hook (config-based)' '
+	test_when_finished "git config --unset hook.pre-commit.command success.sample" &&
+	test_when_finished "rm -f expected_hooks actual_hooks" &&
+	git config hook.pre-commit.command "$HOOKDIR/success.sample" &&
+	echo "$HOOKDIR/success.sample" >expected_hooks &&
+	echo "more" >>file &&
+	git add file &&
+	git commit -m "more" &&
+	test_cmp expected_hooks actual_hooks
+'
+
 test_expect_success 'with succeeding hook (merge)' '
 	test_when_finished "rm -f \"$PREMERGE\" expected_hooks actual_hooks" &&
 	cp "$HOOKDIR/success.sample" "$PREMERGE" &&
diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index 35b513c015f..bdf64728710 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -539,15 +539,13 @@ test_expect_success $PREREQ "--validate respects relative core.hooksPath path" '
 	test_path_is_file my-hooks.ran &&
 	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
-	fatal: command '"'"'git hook run sendemail-validate -- <patch>'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
 '
 
 test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" '
-	hooks_path="$(pwd)/my-hooks" &&
-	test_config core.hooksPath "$hooks_path" &&
+	test_config core.hooksPath "$(pwd)/my-hooks" &&
 	test_when_finished "rm my-hooks.ran" &&
 	test_must_fail git send-email \
 		--from="Example <nobody@example.com>" \
@@ -558,7 +556,6 @@ test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" '
 	test_path_is_file my-hooks.ran &&
 	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
-	fatal: command '"'"'git hook run sendemail-validate -- <patch>'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
@@ -2171,16 +2168,7 @@ test_expect_success $PREREQ 'invoke hook' '
 	mkdir -p .git/hooks &&
 
 	write_script .git/hooks/sendemail-validate <<-\EOF &&
-	# test that we have the correct environment variable, pwd, and
-	# argument
-	case "$GIT_DIR" in
-	*.git)
-		true
-		;;
-	*)
-		false
-		;;
-	esac &&
+	# test that we have the correct argument
 	test -f 0001-add-main.patch &&
 	grep "add main" "$1"
 	EOF
diff --git a/transport.c b/transport.c
index 1146ed3143c..91911076264 100644
--- a/transport.c
+++ b/transport.c
@@ -2,7 +2,6 @@
 #include "config.h"
 #include "transport.h"
 #include "run-command.h"
-#include "hook.h"
 #include "pkt-line.h"
 #include "fetch-pack.h"
 #include "remote.h"
@@ -23,6 +22,7 @@
 #include "protocol.h"
 #include "object-store.h"
 #include "color.h"
+#include "hook.h"
 
 static int transport_use_color = -1;
 static char transport_colors[][COLOR_MAXLEN] = {
@@ -1198,11 +1198,13 @@ static int run_pre_push_hook(struct transport *transport,
 			     struct ref *remote_refs)
 {
 	int ret = 0;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
 	struct strbuf tmp = STRBUF_INIT;
 	struct ref *r;
 	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 
+	run_hooks_opt_init_async(&opt);
+
 	strvec_push(&opt.args, transport->remote->name);
 	strvec_push(&opt.args, transport->url);
 

^ permalink raw reply related	[flat|nested] 479+ messages in thread

* Re: [PATCH 00/31] minimal restart of "config-based-hooks"
  2021-06-01 18:14       ` [PATCH 00/31] minimal restart of "config-based-hooks" Emily Shaffer
  2021-06-01 20:50         ` Derrick Stolee
  2021-06-02  5:30         ` Felipe Contreras
@ 2021-06-02  7:56         ` Ævar Arnfjörð Bjarmason
  2021-06-02  9:39           ` Ævar Arnfjörð Bjarmason
  2021-06-25 18:14           ` Felipe Contreras
  2 siblings, 2 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-02  7:56 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan


On Tue, Jun 01 2021, Emily Shaffer wrote:

> On Fri, May 28, 2021 at 02:11:02PM +0200, Ævar Arnfjörð Bjarmason wrote:
>> 
>> After suggesting[1] an another round that the config-based-hook
>> topic[2] should take a more incremental approach to reach its end goal
>> I thought I'd try hacking that up.
>> 
>> So this is a proposed restart of that topic which if the consensus
>> favors it should replace it, and the config-based hooks topic should
>> be rebased on top of this.
>
> I'm not entirely sure what you're trying to achieve by sending this
> series.

I'm trying to convince you and others that an approach where we split up
the refactoring part of your large series from the behavior changes +
new behaviors it introduces is a better approach in getting us to your
currently proposed (or some small variation thereof) desired end-state.

I think the opening line of your reply doesn't bode for the "convince
you" part of that, maybe I'll do better on the "and others" front :)

I'm a bit surprised at what seems to be some hostility or annoyance that
I submitted this as a set of patches. That's ultimately something that
saves everyone involved time (well, except me by coming up with said
patches). To borrow some words:

    "Talk is cheap. Show me the code." ― Linus Torvalds.

If I give you feedback suggesting that maybe we should reorganize this
thing to split out refactorings from behavior changes I'm asking you to
do extra work. Ultimately neither I, you nor anyone else can really know
if such a proposed effort is going to be better until it happens.

When you get feedback in the form of restructured patches we skip past
all that. We know it compiles, passes tests, and we can more concretely
evaluate the proposal with diff, range-diff etc.

> It was my impression that the existing config-based-hooks topic
> was close to being ready to submit anyway (since Junio mentioned
> submitting it a couple revisions ago); rather than churning by reviewing
> a different 31-patch topic, and then re-rolling and re-reviewing a
> (reduced) config hook topic, wouldn't it be easier on everyone's time to
> do a final incremental review on the existing topic and then start in on
> bugfixes/feature patches afterwards?

I think it's fair to say that nobody's read your code in its current
state more thoroughly than I have at this point, but still, going back
to it and paging through it even now a couple of days after reading the
whole thing line-by-line I find myself getting lost again.

That's because the whole structure of it is to conflate changing
existing behavior with the introduction of new behavior. This makes
things *much harder* for reviewers, because they need to be on toes
about regressions *and* evaluating the function/viability/sanity of new
proposed semantics.

I very much would like to see some approximation (sans my outstanding
comments, more on that below) of your topic land sooner than later.

I think this approach gets us there faster, not slower. How long it
takes to review something isn't about the number of patches, it can be
harder to review a 10 patch series than a 100 patch series. It's mainly
about structuring things in such a way as to make it readable to other
people.

If you've followed any of my own topics you'll probably correctly form
the opinion that that the last few paragraphs at best amount to throwing
some rather large pebbles from a glass house, which you'd be right about
:)

It's can be really hard to see how/where to split things when you're the
author of the code. It's really hard in the "theory of mind" sense of
things to explain an idea to someone who doesn't have the information
you have.

Still, I do think there's a near-universal rules of thumb that the
structure of your series thoroughly runs afoul of, i.e. changing
existing behavior at the same time as introducing new behavior. We
should split such changes up whenever possible. This alternate approach
shows that's it's possible.

> their would have been nice to see a more clear discussion of patch
> organization sometime much sooner in the past year and a half since the
> project was proposed[3], like maybe in the few iterations of the design
> doc which included a rollout plan in July of last year[4]. To me, it
> seems late to be overhauling the direction like this, especially after I
> asked for opinions and approval on the direction before I started work
> in earnest.

We've had some version of this series going back to at least May last
year, and none of it has landed on "master" or "next" yet. To me that's
more reason, not less, for it benefiting from a more incremental
approach.

But yes, I also wish I'd submitted this much earlier. Sorry I didn't
have time to review in this detail until now. I think Felipe's
downthread "never too late" comments apply here though[Æ.1]

I've had outstanding significant feedback on the
"function/viability/sanity of new proposed semantics" part of this
almost 3 months ago that you still haven't addressed in any way[Æ.2]
except this comment[Æ.3] around a month ago on v8 (which I took to mean
that you would). So the lateness of us discussing this is at least
partially a two-way street.

There was also the "why do we need strbuf here?" feedback[Æ.4] around the
same time from me, and as shown in the diff between our two versions
things like your run_hooks_opt_init_async() being better done as a
"struct ... = *_INIT" idiom.

Small issues, but especially the latter to me suggests that your v9 is
too big a chunk for reviewers to consider. I daresay if this was a
smaller series there's no way that wouldn't have been pointed out
already, ditto your "git hook run" introducing an "--env" which never
gets used etc.

> Anyway, I'd personally rather spend effort getting the existing series
> the last few yards to the finish line than to head most of the way back
> to the start.

I think I've made a good argument above for why this takes you a step
forward, not backwards, I'm hoping despite this initial reply that
you'll come to agree on that.

In any case, writing code is hard, but splitting it up like I've done
here is rather easily done. It took me about a day with waiting for
"rebase -i --exec='make test'" equivalent, and that's from being mostly
unfamiliar with the code in question beforehand.

If this alternate series were to go in ahead of yours that doesn't land
you back on square one, you still have a version you could relatively
easily rebase on top. The patches here are mostly smaller versions (no
hook config changes) of corresponding patches of yours.

If anything I don't see how it doesn't make things much easier for you,
if you do get around to replying to my outstanding feedback in [Æ.1]
that'll involve rebasing/commit message rewording of patches that now
conflate significant refactoring and behavior changes.

You'll instead cleanly have just the behavior changes, which at that
point will be both easier to justify in a briefer commit message, and
easier for others to review.

>> 1. https://lore.kernel.org/git/87lf80l1m6.fsf@evledraar.gmail.com/
>> 2. https://lore.kernel.org/git/20210527000856.695702-1-emilyshaffer@google.com/
> 3. https://lore.kernel.org/git/20191116011125.GG22855@google.com/
> 4. https://lore.kernel.org/git/20200728222455.3023400-1-emilyshaffer@google.com/

Æ.1. https://lore.kernel.org/git/60b71788c0e6d_67d0208d4@natae.notmuch/
Æ.2. https://lore.kernel.org/git/87mtv8fww3.fsf@evledraar.gmail.com/
Æ.3. https://lore.kernel.org/git/YJBdbi50Hz+ekOtt@google.com/
Æ.4. https://lore.kernel.org/git/87pn04g0r1.fsf@evledraar.gmail.com/

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 00/31] minimal restart of "config-based-hooks"
  2021-06-01 20:50         ` Derrick Stolee
  2021-06-02  5:42           ` Felipe Contreras
@ 2021-06-02  9:34           ` Ævar Arnfjörð Bjarmason
  1 sibling, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-02  9:34 UTC (permalink / raw)
  To: Derrick Stolee
  Cc: Emily Shaffer, git, Junio C Hamano, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan


On Tue, Jun 01 2021, Derrick Stolee wrote:

> On 6/1/2021 2:14 PM, Emily Shaffer wrote:
>> On Fri, May 28, 2021 at 02:11:02PM +0200, Ævar Arnfjörð Bjarmason wrote:
>>>
>>> After suggesting[1] an another round that the config-based-hook
>>> topic[2] should take a more incremental approach to reach its end goal
>>> I thought I'd try hacking that up.
>
> I think sending this complete reorganization of a long-lived topic
> is not helpful, especially because the end-to-end diff is significant.
> This series has been extensively tested and incrementally improved
> for months, and it would be a waste to start over and lose all of that
> hardening.

Felipe already commented downthread on the end-to-end diff aspect of
this. I replied in
http://lore.kernel.org/git/871r9k8zui.fsf@evledraar.gmail.com with the
diff you're probably more interested in looking at.

> It's also a but rushed that this comes only a day after the previous
> message recommending a reorganization. It would be best to at least
> give the original author an opportunity to comment on your idea before
> working on this.

I replied toth is as "When you get feedback in the form of restructured
patches[...]" in  https://lore.kernel.org/git/87y2bs7gyc.fsf@evledraar.gmail.com/

As noted there I really don't see how sending "here's patches that
implement my suggestion" as opposed to "maybe xyz would be better" is
worse for anyone.

>>> So this is a proposed restart of that topic which if the consensus
>>> favors it should replace it, and the config-based hooks topic should
>>> be rebased on top of this.
>> 
>> I'm not entirely sure what you're trying to achieve by sending this
>> series. It was my impression that the existing config-based-hooks topic
>> was close to being ready to submit anyway (since Junio mentioned
>> submitting it a couple revisions ago); rather than churning by reviewing
>> a different 31-patch topic, and then re-rolling and re-reviewing a
>> (reduced) config hook topic, wouldn't it be easier on everyone's time to
>> do a final incremental review on the existing topic and then start in on
>> bugfixes/feature patches afterwards?
>
> I completely agree here.

What do you think of the issues I raised in [1] ?

Emily's series isn't a pure internal refactoring of existing
functionality, but also an opinionated introduction of new user-facing
functionality.

Now, you may be 100% in agreement with the "opinionated" or
"user-facing" part of that, I'm personally somewhere around 80%.

But to me that clearly makes it a case where we need to as much as
possible get it right *before* it lands on master, because we can't just
incrementally tweak it after it lands without backwards compatibility
concerns. Soon after landing it'll be in a release, have users in the
wild etc.

By default that turns any suggestion of tweaking existing behavior from
a "oh that makes sense, let's fix it" into "well, that's unfortunate,
but it's existing/documented functionality used in the wild".

[2] goes on to mention a couple of other such issues (e.g. the --env
switch to "git hook run"), and this time around I've just been narrowly
focusing on any such issues in the not-config-base-hooks part of this
topic. I.e. the "no user-facing behavior changes yet" part I carved out
into a separate topic.

I'm not telling you which issues those are because I'd like you to tell
me, and then we'll compare notes afterwards. I bet <insert round of
beers at git dev summit or equivalent here> that they're not the same
list.

Anyway, I don't think you will, and I'm not entirely serious. But it's a
real rhetorical point about us being unlikely to come up with the same
result, and thus about the outstanding v9 series being too large to be
readily understood.

>> It would have been nice to see a more clear discussion of patch
>> organization sometime much sooner in the past year and a half since the
>> project was proposed[3], like maybe in the few iterations of the design
>> doc which included a rollout plan in July of last year[4]. To me, it
>> seems late to be overhauling the direction like this, especially after I
>> asked for opinions and approval on the direction before I started work
>> in earnest.
>
> I've also seen messages as early as January where Ævar mentioned
> wanting to review the series, but not finding the time to do so.
> It is reasonable to expect that contributors attempt such major
> reorganizations according to reviewers feedback, as long as the
> reviewers are timely about delivering that feedback.

I made a case in [2] for it not being too late.

For what it's worth (and some of which is noted in [2]) that's partially
because I for one have found it really hard to keep track of this
series.

Multiple re-rolls (including v8->v9) have some outstanding
discussion/feedback, some of which is addressed in a re-roll, but other
parts (as noted in my [2]) which aren't and are silently omitted.

I'm not blaming Emily for that, I think it's a rather inevitable result
of this thing just being too big to begin with and needing to be split
up.

But that's at least part of the story of feedback at such a late
time. Whenever I've looked at this topic I've spent quite a lot of time
doing that re-reading of past discussions / comparing with the newest
cover letter and noting any omissions etc. myself before even getting to
the point of reading the first patch, and I've sometimes just given up.

1. https://lore.kernel.org/git/87mtv8fww3.fsf@evledraar.gmail.com/ 
2. https://lore.kernel.org/git/87y2bs7gyc.fsf@evledraar.gmail.com/

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 00/31] minimal restart of "config-based-hooks"
  2021-06-02  7:56         ` Ævar Arnfjörð Bjarmason
@ 2021-06-02  9:39           ` Ævar Arnfjörð Bjarmason
  2021-06-25 18:14           ` Felipe Contreras
  1 sibling, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-02  9:39 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan


On Wed, Jun 02 2021, Ævar Arnfjörð Bjarmason wrote:

> [...]
> If anything I don't see how it doesn't make things much easier for you,
> if you do get around to replying to my outstanding feedback in [Æ.1]

Hopefully obvious from context, but I meant [Æ.2] here.

> [...]
> Æ.1. https://lore.kernel.org/git/60b71788c0e6d_67d0208d4@natae.notmuch/
> Æ.2. https://lore.kernel.org/git/87mtv8fww3.fsf@evledraar.gmail.com/

i.e. this.

> Æ.3. https://lore.kernel.org/git/YJBdbi50Hz+ekOtt@google.com/
> Æ.4. https://lore.kernel.org/git/87pn04g0r1.fsf@evledraar.gmail.com/


^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v9 07/37] hook: add 'run' subcommand
  2021-05-27  0:08   ` [PATCH v9 07/37] hook: add 'run' subcommand Emily Shaffer
@ 2021-06-03  9:07     ` Ævar Arnfjörð Bjarmason
  2021-06-03 22:29       ` Junio C Hamano
  0 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-03  9:07 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Wed, May 26 2021, Emily Shaffer wrote:

> +void run_hooks_opt_init(struct run_hooks_opt *o)
> +{
> +	strvec_init(&o->env);
> +	strvec_init(&o->args);
> +	o->run_hookdir = configured_hookdir_opt();
> +}

I suggested in
https://lore.kernel.org/git/87y2bs7gyc.fsf@evledraar.gmail.com/ that
this could and should be a RUN_HOOKS_OPT_INIT

After some digging I see that was the case in an earlier version of your
series, i.e. before:
https://lore.kernel.org/git/20210131042254.1032233-1-jonathantanmy@google.com/

You came up with this current pattern because of
configured_hookdir_opt().

But a better option here is to use a RUN_HOOKS_OPT_INIT still and just
defer the initialization of this "run_hookdir" member. I.e. set it to
"we have not asked the config yet" in the initializer. I.e. the diff on
top your series at the end of this E-Mail[1].

That along with doing the same for the "jobs" member means we can move
back to a RUN_HOOKS_OPT_INIT, and also that the final version of this
function in this series, i.e.:
	
	void run_hooks_opt_init_sync(struct run_hooks_opt *o)
	{
		strvec_init(&o->env);
		strvec_init(&o->args);
		o->path_to_stdin = NULL;
		o->run_hookdir = HOOKDIR_UNINITIALIZED;
		o->jobs = 1;
		o->dir = NULL;
		o->feed_pipe = NULL;
		o->feed_pipe_ctx = NULL;
		o->consume_sideband = NULL;
	}

Is now mostly redundant to a designated initializer. I.e. you don't need
to NULL any of these out anymore. Then either don't set "jobs" and have
"0" mean "ask config" or set it to "-1" or whatever for "uninitialized".

1.

diff --git a/hook.c b/hook.c
index ff80e52eddd..daf3ddcc188 100644
--- a/hook.c
+++ b/hook.c
@@ -154,7 +154,7 @@ int configured_hook_jobs(void)
 	return n;
 }
 
-static int should_include_hookdir(const char *path, enum hookdir_opt cfg)
+static int should_include_hookdir(struct run_hooks_opt *options, const char *path)
 {
 	struct strbuf prompt = STRBUF_INIT;
 	/*
@@ -164,7 +164,10 @@ static int should_include_hookdir(const char *path, enum hookdir_opt cfg)
 	if (!path)
 		return 0;
 
-	switch (cfg)
+	if (options->run_hookdir == HOOKDIR_UNINITIALIZED)
+		options->run_hookdir = configured_hookdir_opt();
+
+	switch (options->run_hookdir)
 	{
 	case HOOKDIR_ERROR:
 		fprintf(stderr, _("Skipping legacy hook at '%s'\n"),
@@ -292,7 +295,7 @@ void run_hooks_opt_init_sync(struct run_hooks_opt *o)
 	strvec_init(&o->env);
 	strvec_init(&o->args);
 	o->path_to_stdin = NULL;
-	o->run_hookdir = configured_hookdir_opt();
+	o->run_hookdir = HOOKDIR_UNINITIALIZED;
 	o->jobs = 1;
 	o->dir = NULL;
 	o->feed_pipe = NULL;
@@ -312,6 +315,9 @@ int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir)
 	struct strbuf hook_key = STRBUF_INIT;
 	int could_run_hookdir;
 
+	if (should_run_hookdir != HOOKDIR_USE_CONFIG)
+		BUG("no callers want !HOOKDIR_USE_CONFIG?");
+
 	if (should_run_hookdir == HOOKDIR_USE_CONFIG)
 		should_run_hookdir = configured_hookdir_opt();
 
@@ -459,7 +465,7 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options)
 		struct hook *hook = list_entry(pos, struct hook, list);
 
 		if (hook->from_hookdir &&
-		    !should_include_hookdir(hook->command.buf, options->run_hookdir))
+		    !should_include_hookdir(options, hook->command.buf))
 			    list_del(pos);
 	}
 
diff --git a/hook.h b/hook.h
index f32189380a8..3c4491a74e7 100644
--- a/hook.h
+++ b/hook.h
@@ -30,6 +30,7 @@ struct list_head* hook_list(const char *hookname);
 
 enum hookdir_opt
 {
+	HOOKDIR_UNINITIALIZED,
 	HOOKDIR_USE_CONFIG,
 	HOOKDIR_NO,
 	HOOKDIR_ERROR,

^ permalink raw reply related	[flat|nested] 479+ messages in thread

* Re: [PATCH v9 37/37] docs: link githooks and git-hook manpages
  2021-05-27  0:08   ` [PATCH v9 37/37] docs: link githooks and git-hook manpages Emily Shaffer
@ 2021-06-03  9:18     ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-03  9:18 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Wed, May 26 2021, Emily Shaffer wrote:

> Since users may have an easier time finding 'man githooks' or 'git help
> githooks' through tab-completion or muscle memory, reference the 'git
> hook' commands. And in the 'git hook' manual, point users back to 'man
> githooks' for specifics about the hook events themselves.

Ok, there should be a cross-reference...

> +HOOKS
> +-----

But this should be a "SEE ALSO" section.

> +For a list of hooks which can be configured and how they work, see
> +linkgit:githooks[5].
> +
>  CONFIGURATION
>  -------------
>  include::config/hook.txt[]
> diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
> index 42e66d4e2d..d780cb3b18 100644
> --- a/Documentation/githooks.txt
> +++ b/Documentation/githooks.txt
> @@ -7,15 +7,16 @@ githooks - Hooks used by Git
>  
>  SYNOPSIS
>  --------
> +'git hook'

And ditto here, it makes no sense in a githooks(5) to put "git hook" in
the SYNOPSIS section (which is usually commands to be run), that'll just
spew out the --help output for "git hook" itself.

>  $GIT_DIR/hooks/* (or \`git config core.hooksPath`/*)
>  
>  
>  DESCRIPTION
>  -----------
>  
> -Hooks are programs you can place in a hooks directory to trigger
> -actions at certain points in git's execution. Hooks that don't have
> -the executable bit set are ignored.
> +Hooks are programs you can specify in your config (see linkgit:git-hook[1]) or

For most other things we link back to git-config[1] for such "see", even
though we have the included config in the specific command.

I can see how this makes more sense in a way, but would prefer to have
us be consistent.

> +place in a hooks directory to trigger actions at certain points in git's
> +execution. Hooks that don't have the executable bit set are ignored.
>  
>  By default the hooks directory is `$GIT_DIR/hooks`, but that can be
>  changed via the `core.hooksPath` configuration variable (see

Not a new issue, but is that "are ignored" not something that pre-dates
"advice.ignoredHook"? I.e. we don't ignore them anymore, we warn about
them, no?

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v9 07/37] hook: add 'run' subcommand
  2021-06-03  9:07     ` Ævar Arnfjörð Bjarmason
@ 2021-06-03 22:29       ` Junio C Hamano
  0 siblings, 0 replies; 479+ messages in thread
From: Junio C Hamano @ 2021-06-03 22:29 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: Emily Shaffer, git

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> On Wed, May 26 2021, Emily Shaffer wrote:
>
>> +void run_hooks_opt_init(struct run_hooks_opt *o)
>> +{
>> +	strvec_init(&o->env);
>> +	strvec_init(&o->args);
>> +	o->run_hookdir = configured_hookdir_opt();
>> +}
>
> I suggested in
> https://lore.kernel.org/git/87y2bs7gyc.fsf@evledraar.gmail.com/ that
> this could and should be a RUN_HOOKS_OPT_INIT
>
> After some digging I see that was the case in an earlier version of your
> series, i.e. before:
> https://lore.kernel.org/git/20210131042254.1032233-1-jonathantanmy@google.com/
>
> You came up with this current pattern because of
> configured_hookdir_opt().
>
> But a better option here is to use a RUN_HOOKS_OPT_INIT still and just
> defer the initialization of this "run_hookdir" member. I.e. set it to
> "we have not asked the config yet" in the initializer. I.e. the diff on
> top your series at the end of this E-Mail[1].

When I compared the result of applying your 31-patch series to
2.32-rc2 and the result of rebasing this series on the same base,
before sending out a responce to Emily's reaction, I found that the
31-patch series did a good job of not stepping on the "hook defined
with configuration" part and concentrated on providing a clean base
to build on with a better structure in the series, and there weren't
many changes that overlapped with Emily's series in a significant
way.  The above was one of the "overlapping" differences that stood
out.

Thanks.




^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v9 31/37] post-update: use hook.h library
  2021-05-27  0:08   ` [PATCH v9 31/37] post-update: use hook.h library Emily Shaffer
@ 2021-06-14  9:09     ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14  9:09 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Wed, May 26 2021, Emily Shaffer wrote:

> By using run_hooks() instead of run_hook_le(), 'post-update' hooks can
> be specified in the config as well as the hookdir.

But even though we delete run_hook_le() in a later commit this is not
one of the users of that API, it's doing its own start_command(). This
is exposed in my alternate restart series where I removed that API
before this commit.

Spotted while rephrasing the commit message for a re-roll of that, just
leaving a note here...

^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH v2 00/30] Minimal restart of "config-based-hooks"
  2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
                         ` (31 preceding siblings ...)
  2021-06-01 18:14       ` [PATCH 00/31] minimal restart of "config-based-hooks" Emily Shaffer
@ 2021-06-14 10:32       ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:32         ` [PATCH v2 01/30] hook: add 'run' subcommand Ævar Arnfjörð Bjarmason
                           ` (32 more replies)
  32 siblings, 33 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:32 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

I proposed splitting Emily's "hook config" topic[1] into at least a
topic that retains all current behavior of the codebase, and merely
refactors existing behavior to new APIs, and then doing behavior
changes later.

This is a re-roll of an attempt to do that. See v1's CL [2] for much
more details.

I was hoping to get more feedback from Emily in reply to [3] and
related E-Mails on the v1, but as it's been almost 2 weeks with no
reply, and both her topic and mine semantically conflict with changes
since merged to "master" I thought I'd send this re-roll.

Changes since v2:

 A. Addressed comments by Felipe, thanks! The "remove the [old] API"
    is now a separate commit / other minor nits fixed.

 B. Fixed a test failure due to "mkdir .git/hooks"; semantic conflict
    with my now-landed f0d4d398e28 (test-lib: split up and deprecate
    test_create_repo(), 2021-05-10)

 C. Fixed a portibility issue in the new generate-hooklist.sh.

 D. Clarified what a TOCTOU is, reworded commit message.

 E. I'd added various "we get closer to removing the hook code" in
    commit messages after we'd removed the hook code. We were really
    just porting bespoke behavior to hook.h.

 F. Fixed trivial conflict with master's 410334ed521 (read-cache: use
    hashfile instead of git_hash_ctx, 2021-05-18) in read-cache.c.

 G. In my v1 I added two patches at the beginning to fix/add
    hook-related tests. I'll just submit those as separate
    patches. There's no dependency between those and this topic, other
    than basic this one on top of them granting more assurances that
    this one doesn't break any existing hook behavior than the current
    tests on "master".

1. http://lore.kernel.org/git/20210527000856.695702-1-emilyshaffer@google.com
2. https://lore.kernel.org/git/cover-00.31-00000000000-20210528T110515Z-avarab@gmail.com/
3. https://lore.kernel.org/git/87y2bs7gyc.fsf@evledraar.gmail.com/

Emily Shaffer (26):
  hook: add 'run' subcommand
  hook.c: add a hook_exists() wrapper and use it in bugreport.c
  gc: use hook library for pre-auto-gc hook
  rebase: teach pre-rebase to use hook.h
  am: convert applypatch hooks to use config
  hooks: convert 'post-checkout' hook to hook library
  merge: use config-based hooks for post-merge hook
  send-email: use 'git hook run' for 'sendemail-validate'
  git-p4: use 'git hook' to run hooks
  commit: use hook.h to execute hooks
  read-cache: convert post-index-change hook to use config
  receive-pack: convert push-to-checkout hook to hook.h
  run-command: remove old run_hook_{le,ve}() hook API
  run-command: allow stdin for run_processes_parallel
  hook: support passing stdin to hooks
  am: convert 'post-rewrite' hook to hook.h
  run-command: add stdin callback for parallelization
  hook: provide stdin by string_list or callback
  hook: convert 'post-rewrite' hook in sequencer.c to hook.h
  transport: convert pre-push hook to use config
  reference-transaction: use hook.h to run hooks
  run-command: allow capturing of collated output
  hooks: allow callers to capture output
  receive-pack: convert 'update' hook to hook.h
  post-update: use hook.h library
  receive-pack: convert receive hooks to hook.h

Ævar Arnfjörð Bjarmason (4):
  run-command.h: move find_hook() to hook.h
  git hook run: add an --ignore-missing flag
  hooks: fix a TOCTOU in "did we run a hook?" heuristic
  hook-list.h: add a generated list of hooks, like config-list.h

 .gitignore                  |   2 +
 Documentation/git-hook.txt  |  49 ++++++
 Documentation/githooks.txt  |   4 +
 Makefile                    |  16 +-
 builtin.h                   |   1 +
 builtin/am.c                |  34 ++--
 builtin/bugreport.c         |  46 ++----
 builtin/checkout.c          |  17 +-
 builtin/clone.c             |   7 +-
 builtin/commit.c            |  19 ++-
 builtin/fetch.c             |   1 +
 builtin/gc.c                |   8 +-
 builtin/hook.c              |  72 +++++++++
 builtin/merge.c             |  22 ++-
 builtin/rebase.c            |   9 +-
 builtin/receive-pack.c      | 299 ++++++++++++++++++------------------
 builtin/submodule--helper.c |   2 +-
 builtin/worktree.c          |  31 ++--
 command-list.txt            |   1 +
 commit.c                    |  17 +-
 commit.h                    |   3 +-
 generate-hooklist.sh        |  24 +++
 git-p4.py                   |  72 +--------
 git-send-email.perl         |  20 ++-
 git.c                       |   1 +
 hook.c                      | 224 +++++++++++++++++++++++++++
 hook.h                      | 122 +++++++++++++++
 read-cache.c                |  12 +-
 refs.c                      |  43 ++----
 reset.c                     |  15 +-
 run-command.c               | 157 ++++++++++---------
 run-command.h               |  55 ++++---
 sequencer.c                 |  88 +++++------
 submodule.c                 |   1 +
 t/helper/test-run-command.c |  46 +++++-
 t/t0061-run-command.sh      |  37 +++++
 t/t1800-hook.sh             | 162 +++++++++++++++++++
 t/t9001-send-email.sh       |   4 +-
 transport.c                 |  58 ++-----
 39 files changed, 1246 insertions(+), 555 deletions(-)
 create mode 100644 Documentation/git-hook.txt
 create mode 100644 builtin/hook.c
 create mode 100755 generate-hooklist.sh
 create mode 100644 hook.c
 create mode 100644 hook.h
 create mode 100755 t/t1800-hook.sh

Range-diff against v1:
 1:  8ac2efc71a0 <  -:  ----------- hooks tests: don't leave "actual" nonexisting on failure
 2:  eb37693f7dc <  -:  ----------- gc tests: add a test for the "pre-auto-gc" hook
 3:  1ad4e69f7da !  1:  447d349c738 hook: add 'run' subcommand
    @@ t/t1800-hook.sh (new)
     +
     +. ./test-lib.sh
     +
    -+test_expect_success 'setup .git/hooks' '
    -+	mkdir .git/hooks
    -+'
    -+
     +test_expect_success 'git hook run -- nonexistent hook' '
     +	cat >stderr.expect <<-\EOF &&
     +	error: cannot find a hook named does-not-exist
 4:  1a67a1cc065 =  2:  85195a78cfb run-command.h: move find_hook() to hook.h
 5:  a6f0817ad81 =  3:  eb5bdd993c8 hook.c: add a hook_exists() wrapper and use it in bugreport.c
 6:  b186fde43e1 =  4:  da2763192ae gc: use hook library for pre-auto-gc hook
 7:  528402fac69 =  5:  51e6e72f239 rebase: teach pre-rebase to use hook.h
 8:  69842c74383 =  6:  d2f3b26d464 am: convert applypatch hooks to use config
 9:  9b32c14669b !  7:  d884465aab3 hooks: convert 'post-checkout' hook to hook library
    @@ hook.h: struct run_hooks_opt
     
      ## read-cache.c ##
     @@
    - #include "thread-utils.h"
      #include "progress.h"
      #include "sparse-index.h"
    + #include "csum-file.h"
     +#include "hook.h"
      
      /* Mask for the name length in ce_flags in the on-disk index */
10:  201c654bb0c =  8:  ad6e7507841 merge: use config-based hooks for post-merge hook
11:  e65d8bd6e6f =  9:  8d8b2d26453 git hook run: add an --ignore-missing flag
12:  8795e9ceab8 = 10:  1953326d1db send-email: use 'git hook run' for 'sendemail-validate'
13:  03129460fd2 = 11:  aa970a81752 git-p4: use 'git hook' to run hooks
14:  3f3610f5ed3 = 12:  7756f10aac9 commit: use hook.h to execute hooks
15:  6482a3e4cb8 = 13:  0300607a9b4 read-cache: convert post-index-change hook to use config
 -:  ----------- > 14:  ec4ad437f86 receive-pack: convert push-to-checkout hook to hook.h
16:  a16163d4fb5 ! 15:  477eb2245c3 receive-pack: convert push-to-checkout hook to hook.h
    @@ Metadata
     Author: Emily Shaffer <emilyshaffer@google.com>
     
      ## Commit message ##
    -    receive-pack: convert push-to-checkout hook to hook.h
    +    run-command: remove old run_hook_{le,ve}() hook API
     
    -    By using hook.h instead of run-command.h to invoke push-to-checkout,
    -    hooks can now be specified in the config as well as in the hookdir.
    -    push-to-checkout is not called anywhere but in builtin/receive-pack.c.
    -
    -    This is the last user of the run_hook_le() API, so let's remove it
    -    while we're at it, since run_hook_le() itself is the last user of
    -    run_hook_ve() we can remove that too. The last direct user of
    -    run_hook_le() was removed in the commit preceding this one.
    +    The new hook.h library has replaced all run-command.h hook-related
    +    functionality. So let's delete this dead code.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
    - ## builtin/receive-pack.c ##
    -@@ builtin/receive-pack.c: static const char *push_to_checkout(unsigned char *hash,
    - 				    struct strvec *env,
    - 				    const char *work_tree)
    - {
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+
    - 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
    --	if (run_hook_le(env->v, push_to_checkout_hook,
    --			hash_to_hex(hash), NULL))
    -+	strvec_pushv(&opt.env, env->v);
    -+	strvec_push(&opt.args, hash_to_hex(hash));
    -+	if (run_hooks(push_to_checkout_hook, &opt)) {
    -+		run_hooks_opt_clear(&opt);
    - 		return "push-to-checkout hook declined";
    --	else
    -+	} else {
    -+		run_hooks_opt_clear(&opt);
    - 		return NULL;
    -+	}
    - }
    - 
    - static const char *update_worktree(unsigned char *sha1, const struct worktree *worktree)
    -@@ builtin/receive-pack.c: static const char *update_worktree(unsigned char *sha1, const struct worktree *w
    - 
    - 	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
    - 
    --	if (!find_hook(push_to_checkout_hook))
    -+	if (!hook_exists(push_to_checkout_hook))
    - 		retval = push_to_deploy(sha1, &env, work_tree);
    - 	else
    - 		retval = push_to_checkout(sha1, &env, work_tree);
    -
      ## run-command.c ##
     @@ run-command.c: int async_with_fork(void)
      #endif
17:  7020cf10c8e = 16:  53a3877a476 run-command: allow stdin for run_processes_parallel
18:  4745dcfce49 = 17:  c4f60db606d hook: support passing stdin to hooks
19:  986bfd89a54 = 18:  febf05ef232 am: convert 'post-rewrite' hook to hook.h
20:  756f52af22d = 19:  7baf2469d50 run-command: add stdin callback for parallelization
21:  3748f128763 ! 20:  2edf9dea41a hook: provide stdin by string_list or callback
    @@ Commit message
         In cases where a hook requires only a small amount of information via
         stdin, it should be simple for users to provide a string_list alone. But
         in more complicated cases where the stdin is too large to hold in
    -    memory, let's provide a callback the users can populate line after line
    -    with instead.
    +    memory, let's instead provide a callback the users can populate line
    +    after line.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
22:  0cf0b1fea93 = 21:  303b31ee620 hook: convert 'post-rewrite' hook in sequencer.c to hook.h
23:  c59443a3b05 = 22:  62eecafb3ff transport: convert pre-push hook to use config
24:  f7c8c97cb81 ! 23:  6049b1cdc74 reference-transaction: use hook.h to run hooks
    @@ Metadata
      ## Commit message ##
         reference-transaction: use hook.h to run hooks
     
    -    By using the hook.h library, we get closer to removing the hook code
    -    in run-command.c.
    -
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
25:  f240a51ec4e = 24:  26ebbe4c545 run-command: allow capturing of collated output
26:  7f10efb7858 = 25:  251085b7525 hooks: allow callers to capture output
27:  c39c608e5cc ! 26:  e11a8f60071 receive-pack: convert 'update' hook to hook.h
    @@ Metadata
      ## Commit message ##
         receive-pack: convert 'update' hook to hook.h
     
    -    By using hook.h to invoke the 'update' hook we closer to removing the
    -    hooks code in run-command.c.
    +    This makes use of the new sideband API in hook.h added in the
    +    preceding commit.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
28:  3519068a634 ! 27:  ecaedd13b89 post-update: use hook.h library
    @@ Metadata
      ## Commit message ##
         post-update: use hook.h library
     
    -    By using run_hooks() instead of run_hook_le(), 'post-update' hooks can
    -    be specified in the config as well as the hookdir.
    -
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
29:  3466f17af08 ! 28:  bb9d57f809a receive-pack: convert receive hooks to hook.h
    @@ Metadata
      ## Commit message ##
         receive-pack: convert receive hooks to hook.h
     
    -    By using the hook.h library to run receive hooks we get closer to
    -    deleting the hook functions in run-command.c
    -
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
30:  d93bdc0c294 ! 29:  793f112f7ab hooks: fix a TOCTOU in "did we run a hook?" heuristic
    @@ Metadata
      ## Commit message ##
         hooks: fix a TOCTOU in "did we run a hook?" heuristic
     
    -    Fix a race in code added in 680ee550d72 (commit: skip discarding the
    -    index if there is no pre-commit hook, 2017-08-14) by changing the
    -    hook.c API to optionally indicate whether or not the requested hook
    -    ran or not. This was suggested in the discussion around
    -    680ee550d72[1].
    +    Fix a Time-of-check to time-of-use (TOCTOU) race in code added in
    +    680ee550d72 (commit: skip discarding the index if there is no
    +    pre-commit hook, 2017-08-14).
     
    -    Let's also change this for the pre-merge-commit hook, see
    +    We can fix the race passing around information about whether or not we
    +    ran the hook in question, instead of running hook_exists() after the
    +    fact to check if the hook in question exists. This problem has been
    +    noted on-list when 680ee550d72 was discussed[1], but had not been
    +    fixed.
    +
    +    In addition to fixing this for the pre-commit hook as suggested there
    +    I'm also fixing this for the pre-merge-commit hook. See
         6098817fd7f (git-merge: honor pre-merge-commit hook, 2019-08-07) for
    -    the introduction of the previous behavior.
    +    the introduction of its previous behavior.
     
         Let's also change this for the push-to-checkout hook. Now instead of
         checking if the hook exists and either doing a push to checkout or a
    @@ Commit message
     
         In both of those cases we're saving ourselves CPU time by not
         preparing data for the hook that we'll then do nothing with if we
    -    don't have the hook, so using this "invoked_hook" pattern doesn't make
    -    sense there purely for optimization purposes.
    +    don't have the hook. So using this "invoked_hook" pattern doesn't make
    +    sense in those cases.
     
         More importantly, in those cases the worst we'll do is miss that we
         "should" run the hook because a new hook appeared, whereas in the
31:  896956250f6 ! 30:  bc086454d68 hook-list.h: add a generated list of hooks, like config-list.h
    @@ generate-hooklist.sh (new)
     +	cat <<EOF
     +static const char *hook_name_list[] = {
     +EOF
    -+	grep -C1 -h '^~~~' Documentation/githooks.txt |
    -+	grep '^[a-z0-9][a-z0-9-]*$' |
    -+	sort |
    -+	sed 's/^.*$/	"&",/'
    ++	perl -ne '
    ++		chomp;
    ++		@l[$.] = $_;
    ++		push @h => $l[$. - 1] if /^~~~+$/s;
    ++		END {
    ++			print qq[\t"$_",\n] for sort @h;
    ++		}
    ++	' <Documentation/githooks.txt
     +	cat <<EOF
     +	NULL,
     +};
    @@ hook.c
      	if (access(path.buf, X_OK) < 0) {
     
      ## t/t1800-hook.sh ##
    -@@ t/t1800-hook.sh: test_expect_success 'setup .git/hooks' '
    +@@ t/t1800-hook.sh: test_description='git-hook command'
      
      test_expect_success 'git hook run -- nonexistent hook' '
      	cat >stderr.expect <<-\EOF &&
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH v2 01/30] hook: add 'run' subcommand
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:32         ` Ævar Arnfjörð Bjarmason
  2021-06-14 21:33           ` Emily Shaffer
  2021-06-14 10:32         ` [PATCH v2 02/30] run-command.h: move find_hook() to hook.h Ævar Arnfjörð Bjarmason
                           ` (31 subsequent siblings)
  32 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:32 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

In order to enable hooks to be run as an external process, by a
standalone Git command, or by tools which wrap Git, provide an external
means to run all configured hook commands for a given hook event.

Most of our hooks require more complex functionality than this, but
let's start with the bare minimum required to support our simplest
hooks.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .gitignore                 |   1 +
 Documentation/git-hook.txt |  36 ++++++++++
 Documentation/githooks.txt |   4 ++
 Makefile                   |   2 +
 builtin.h                  |   1 +
 builtin/hook.c             |  65 ++++++++++++++++++
 command-list.txt           |   1 +
 git.c                      |   1 +
 hook.c                     | 114 ++++++++++++++++++++++++++++++++
 hook.h                     |  54 +++++++++++++++
 t/t1800-hook.sh            | 131 +++++++++++++++++++++++++++++++++++++
 11 files changed, 410 insertions(+)
 create mode 100644 Documentation/git-hook.txt
 create mode 100644 builtin/hook.c
 create mode 100644 hook.c
 create mode 100644 hook.h
 create mode 100755 t/t1800-hook.sh

diff --git a/.gitignore b/.gitignore
index 311841f9bed..de39dc9961b 100644
--- a/.gitignore
+++ b/.gitignore
@@ -77,6 +77,7 @@
 /git-grep
 /git-hash-object
 /git-help
+/git-hook
 /git-http-backend
 /git-http-fetch
 /git-http-push
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
new file mode 100644
index 00000000000..902b9cffaef
--- /dev/null
+++ b/Documentation/git-hook.txt
@@ -0,0 +1,36 @@
+git-hook(1)
+===========
+
+NAME
+----
+git-hook - run git hooks
+
+SYNOPSIS
+--------
+[verse]
+'git hook' run <hook-name> [-- <hook-args>]
+
+DESCRIPTION
+-----------
+
+This command is an interface to git hooks (see linkgit:githooks[5]).
+Currently it only provides a convenience wrapper for running hooks for
+use by git itself. In the future it might gain other functionality.
+
+SUBCOMMANDS
+-----------
+
+run::
+
+	Run the `<hook-name>` hook. Any positional arguments to the
+	hook should be passed after an optional "--" (or
+	"--end-of-options"). See "OPTIONS" below for the arguments
+	this accepts.
+
+SEE ALSO
+--------
+linkgit:githooks[5]
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index b51959ff941..a16e62bc8c8 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -698,6 +698,10 @@ and "0" meaning they were not.
 Only one parameter should be set to "1" when the hook runs.  The hook
 running passing "1", "1" should not be possible.
 
+SEE ALSO
+--------
+linkgit:git-hook[1]
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index c3565fc0f8f..a6b71a0fbed 100644
--- a/Makefile
+++ b/Makefile
@@ -901,6 +901,7 @@ LIB_OBJS += hash-lookup.o
 LIB_OBJS += hashmap.o
 LIB_OBJS += help.o
 LIB_OBJS += hex.o
+LIB_OBJS += hook.o
 LIB_OBJS += ident.o
 LIB_OBJS += json-writer.o
 LIB_OBJS += kwset.o
@@ -1101,6 +1102,7 @@ BUILTIN_OBJS += builtin/get-tar-commit-id.o
 BUILTIN_OBJS += builtin/grep.o
 BUILTIN_OBJS += builtin/hash-object.o
 BUILTIN_OBJS += builtin/help.o
+BUILTIN_OBJS += builtin/hook.o
 BUILTIN_OBJS += builtin/index-pack.o
 BUILTIN_OBJS += builtin/init-db.o
 BUILTIN_OBJS += builtin/interpret-trailers.o
diff --git a/builtin.h b/builtin.h
index 16ecd5586f0..91740c15149 100644
--- a/builtin.h
+++ b/builtin.h
@@ -164,6 +164,7 @@ int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix);
 int cmd_grep(int argc, const char **argv, const char *prefix);
 int cmd_hash_object(int argc, const char **argv, const char *prefix);
 int cmd_help(int argc, const char **argv, const char *prefix);
+int cmd_hook(int argc, const char **argv, const char *prefix);
 int cmd_index_pack(int argc, const char **argv, const char *prefix);
 int cmd_init_db(int argc, const char **argv, const char *prefix);
 int cmd_interpret_trailers(int argc, const char **argv, const char *prefix);
diff --git a/builtin/hook.c b/builtin/hook.c
new file mode 100644
index 00000000000..1b1a594fd00
--- /dev/null
+++ b/builtin/hook.c
@@ -0,0 +1,65 @@
+#include "cache.h"
+#include "builtin.h"
+#include "config.h"
+#include "hook.h"
+#include "parse-options.h"
+#include "strbuf.h"
+#include "strvec.h"
+
+static const char * const builtin_hook_usage[] = {
+	N_("git hook run <hook-name> [-- <hook-args>]"),
+	NULL
+};
+
+static int run(int argc, const char **argv, const char *prefix)
+{
+	int i;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	int rc = 0;
+	const char *hook_name;
+	const char *hook_path;
+
+	struct option run_options[] = {
+		OPT_END(),
+	};
+
+	argc = parse_options(argc, argv, prefix, run_options,
+			     builtin_hook_usage, PARSE_OPT_KEEP_UNKNOWN | PARSE_OPT_KEEP_DASHDASH);
+
+	if (argc > 2) {
+		if (strcmp(argv[2], "--") &&
+		    strcmp(argv[2], "--end-of-options"))
+			/* Having a -- for "run" is mandatory */
+			usage_with_options(builtin_hook_usage, run_options);
+		/* Add our arguments, start after -- */
+		for (i = 3 ; i < argc; i++)
+			strvec_push(&opt.args, argv[i]);
+	}
+
+	/* Need to take into account core.hooksPath */
+	git_config(git_default_config, NULL);
+
+	hook_name = argv[1];
+	hook_path = find_hook(hook_name);
+	if (!hook_path) {
+		error("cannot find a hook named %s", hook_name);
+		return 1;
+	}
+	rc = run_found_hooks(hook_name, hook_path, &opt);
+
+	run_hooks_opt_clear(&opt);
+
+	return rc;
+}
+
+int cmd_hook(int argc, const char **argv, const char *prefix)
+{
+	struct option builtin_hook_options[] = {
+		OPT_END(),
+	};
+
+	if (!strcmp(argv[1], "run"))
+		return run(argc, argv, prefix);
+	usage_with_options(builtin_hook_usage, builtin_hook_options);
+	return 1;
+}
diff --git a/command-list.txt b/command-list.txt
index a289f09ed6f..9ccd8e5aebe 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -103,6 +103,7 @@ git-grep                                mainporcelain           info
 git-gui                                 mainporcelain
 git-hash-object                         plumbingmanipulators
 git-help                                ancillaryinterrogators          complete
+git-hook                                mainporcelain
 git-http-backend                        synchingrepositories
 git-http-fetch                          synchelpers
 git-http-push                           synchelpers
diff --git a/git.c b/git.c
index 18bed9a9964..540909c391f 100644
--- a/git.c
+++ b/git.c
@@ -538,6 +538,7 @@ static struct cmd_struct commands[] = {
 	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
 	{ "hash-object", cmd_hash_object },
 	{ "help", cmd_help },
+	{ "hook", cmd_hook, RUN_SETUP },
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "init", cmd_init_db },
 	{ "init-db", cmd_init_db },
diff --git a/hook.c b/hook.c
new file mode 100644
index 00000000000..aa66c968186
--- /dev/null
+++ b/hook.c
@@ -0,0 +1,114 @@
+#include "cache.h"
+#include "hook.h"
+#include "run-command.h"
+
+void run_hooks_opt_clear(struct run_hooks_opt *o)
+{
+	strvec_clear(&o->env);
+	strvec_clear(&o->args);
+}
+
+static int pick_next_hook(struct child_process *cp,
+			  struct strbuf *out,
+			  void *pp_cb,
+			  void **pp_task_cb)
+{
+	struct hook_cb_data *hook_cb = pp_cb;
+	struct hook *run_me = hook_cb->run_me;
+
+	if (!run_me)
+		BUG("did we not return 1 in notify_hook_finished?");
+
+	cp->no_stdin = 1;
+	cp->env = hook_cb->options->env.v;
+	cp->stdout_to_stderr = 1;
+	cp->trace2_hook_name = hook_cb->hook_name;
+
+	/* add command */
+	strvec_push(&cp->args, run_me->hook_path);
+
+	/*
+	 * add passed-in argv, without expanding - let the user get back
+	 * exactly what they put in
+	 */
+	strvec_pushv(&cp->args, hook_cb->options->args.v);
+
+	/* Provide context for errors if necessary */
+	*pp_task_cb = run_me;
+
+	return 1;
+}
+
+static int notify_start_failure(struct strbuf *out,
+				void *pp_cb,
+				void *pp_task_cp)
+{
+	struct hook_cb_data *hook_cb = pp_cb;
+	struct hook *attempted = pp_task_cp;
+
+	/* |= rc in cb */
+	hook_cb->rc |= 1;
+
+	strbuf_addf(out, _("Couldn't start hook '%s'\n"),
+		    attempted->hook_path);
+
+	return 1;
+}
+
+static int notify_hook_finished(int result,
+				struct strbuf *out,
+				void *pp_cb,
+				void *pp_task_cb)
+{
+	struct hook_cb_data *hook_cb = pp_cb;
+
+	/* |= rc in cb */
+	hook_cb->rc |= result;
+
+	return 1;
+}
+
+
+int run_found_hooks(const char *hook_name, const char *hook_path,
+		    struct run_hooks_opt *options)
+{
+	struct hook my_hook = {
+		.hook_path = hook_path,
+	};
+	struct hook_cb_data cb_data = {
+		.rc = 0,
+		.hook_name = hook_name,
+		.options = options,
+	};
+	cb_data.run_me = &my_hook;
+
+	if (options->jobs != 1)
+		BUG("we do not handle %d or any other != 1 job number yet", options->jobs);
+
+	run_processes_parallel_tr2(options->jobs,
+				   pick_next_hook,
+				   notify_start_failure,
+				   notify_hook_finished,
+				   &cb_data,
+				   "hook",
+				   hook_name);
+
+	return cb_data.rc;
+}
+
+int run_hooks(const char *hook_name, struct run_hooks_opt *options)
+{
+	const char *hook_path;
+	int ret;
+	if (!options)
+		BUG("a struct run_hooks_opt must be provided to run_hooks");
+
+	hook_path = find_hook(hook_name);
+
+	/* Care about nonexistence? Use run_found_hooks() */
+	if (!hook_path)
+		return 0;
+
+	ret = run_found_hooks(hook_name, hook_path, options);
+	return ret;
+}
diff --git a/hook.h b/hook.h
new file mode 100644
index 00000000000..ebfee26bcf2
--- /dev/null
+++ b/hook.h
@@ -0,0 +1,54 @@
+#ifndef HOOK_H
+#define HOOK_H
+#include "strbuf.h"
+#include "strvec.h"
+#include "run-command.h"
+
+struct hook {
+	/* The path to the hook */
+	const char *hook_path;
+};
+
+struct run_hooks_opt
+{
+	/* Environment vars to be set for each hook */
+	struct strvec env;
+
+	/* Args to be passed to each hook */
+	struct strvec args;
+
+	/* Number of threads to parallelize across */
+	int jobs;
+};
+
+#define RUN_HOOKS_OPT_INIT { \
+	.jobs = 1, \
+	.env = STRVEC_INIT, \
+	.args = STRVEC_INIT, \
+}
+
+/*
+ * Callback provided to feed_pipe_fn and consume_sideband_fn.
+ */
+struct hook_cb_data {
+	int rc;
+	const char *hook_name;
+	struct hook *run_me;
+	struct run_hooks_opt *options;
+};
+
+void run_hooks_opt_clear(struct run_hooks_opt *o);
+
+/*
+ * Calls find_hook(hookname) and runs the hooks (if any) with
+ * run_found_hooks().
+ */
+int run_hooks(const char *hook_name, struct run_hooks_opt *options);
+
+/*
+ * Takes an already resolved hook and runs it. Internally the simpler
+ * run_hooks() will call this.
+ */
+int run_found_hooks(const char *hookname, const char *hook_path,
+		    struct run_hooks_opt *options);
+#endif
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
new file mode 100755
index 00000000000..f6ff6c4a493
--- /dev/null
+++ b/t/t1800-hook.sh
@@ -0,0 +1,131 @@
+#!/bin/bash
+
+test_description='git-hook command'
+
+. ./test-lib.sh
+
+test_expect_success 'git hook run -- nonexistent hook' '
+	cat >stderr.expect <<-\EOF &&
+	error: cannot find a hook named does-not-exist
+	EOF
+	test_expect_code 1 git hook run does-not-exist 2>stderr.actual &&
+	test_cmp stderr.expect stderr.actual
+'
+
+test_expect_success 'git hook run -- basic' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo Test hook
+	EOF
+
+	cat >expect <<-\EOF &&
+	Test hook
+	EOF
+	git hook run test-hook 2>actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git hook run -- stdout and stderr handling' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo >&1 Will end up on stderr
+	echo >&2 Will end up on stderr
+	EOF
+
+	cat >stderr.expect <<-\EOF &&
+	Will end up on stderr
+	Will end up on stderr
+	EOF
+	git hook run test-hook >stdout.actual 2>stderr.actual &&
+	test_cmp stderr.expect stderr.actual &&
+	test_must_be_empty stdout.actual
+'
+
+test_expect_success 'git hook run -- exit codes are passed along' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	exit 1
+	EOF
+
+	test_expect_code 1 git hook run test-hook &&
+
+	write_script .git/hooks/test-hook <<-EOF &&
+	exit 2
+	EOF
+
+	test_expect_code 2 git hook run test-hook &&
+
+	write_script .git/hooks/test-hook <<-EOF &&
+	exit 128
+	EOF
+
+	test_expect_code 128 git hook run test-hook &&
+
+	write_script .git/hooks/test-hook <<-EOF &&
+	exit 129
+	EOF
+
+	test_expect_code 129 git hook run test-hook
+'
+
+test_expect_success 'git hook run arg u ments without -- is not allowed' '
+	test_expect_code 129 git hook run test-hook arg u ments
+'
+
+test_expect_success 'git hook run -- pass arguments' '
+	write_script .git/hooks/test-hook <<-\EOF &&
+	echo $1
+	echo $2
+	EOF
+
+	cat >expect <<-EOF &&
+	arg
+	u ments
+	EOF
+
+	git hook run test-hook -- arg "u ments" 2>actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git hook run -- out-of-repo runs excluded' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo Test hook
+	EOF
+
+	nongit test_must_fail git hook run test-hook
+'
+
+test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
+	mkdir my-hooks &&
+	write_script my-hooks/test-hook <<-EOF &&
+	echo Hook ran >>actual
+	EOF
+
+	cat >expect <<-\EOF &&
+	Test hook
+	Hook ran
+	Hook ran
+	Hook ran
+	Hook ran
+	EOF
+
+	# Test various ways of specifying the path. See also
+	# t1350-config-hooks-path.sh
+	>actual &&
+	git hook run test-hook 2>>actual &&
+	git -c core.hooksPath=my-hooks hook run test-hook 2>>actual &&
+	git -c core.hooksPath=my-hooks/ hook run test-hook 2>>actual &&
+	git -c core.hooksPath="$PWD/my-hooks" hook run test-hook 2>>actual &&
+	git -c core.hooksPath="$PWD/my-hooks/" hook run test-hook 2>>actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'set up a pre-commit hook in core.hooksPath' '
+	>actual &&
+	mkdir -p .git/custom-hooks .git/hooks &&
+	write_script .git/custom-hooks/pre-commit <<-\EOF &&
+	echo CUSTOM >>actual
+	EOF
+	write_script .git/hooks/pre-commit <<-\EOF
+	echo NORMAL >>actual
+	EOF
+'
+
+test_done
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 02/30] run-command.h: move find_hook() to hook.h
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
  2021-06-14 10:32         ` [PATCH v2 01/30] hook: add 'run' subcommand Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:32         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:32         ` [PATCH v2 03/30] hook.c: add a hook_exists() wrapper and use it in bugreport.c Ævar Arnfjörð Bjarmason
                           ` (30 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:32 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Move the find_hook() command to hook.h. Eventually all the hook
related code will live there, let's move this function over as-is.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/am.c           |  1 +
 builtin/bugreport.c    |  2 +-
 builtin/commit.c       |  1 +
 builtin/merge.c        |  1 +
 builtin/receive-pack.c |  1 +
 builtin/worktree.c     |  1 +
 hook.c                 | 36 ++++++++++++++++++++++++++++++++++++
 hook.h                 |  7 +++++++
 refs.c                 |  1 +
 run-command.c          | 35 +----------------------------------
 run-command.h          |  7 -------
 sequencer.c            |  1 +
 transport.c            |  1 +
 13 files changed, 53 insertions(+), 42 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index 0b2d886c81b..1c8a5489035 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -11,6 +11,7 @@
 #include "parse-options.h"
 #include "dir.h"
 #include "run-command.h"
+#include "hook.h"
 #include "quote.h"
 #include "tempfile.h"
 #include "lockfile.h"
diff --git a/builtin/bugreport.c b/builtin/bugreport.c
index 9915a5841de..596f079a7f9 100644
--- a/builtin/bugreport.c
+++ b/builtin/bugreport.c
@@ -3,7 +3,7 @@
 #include "strbuf.h"
 #include "help.h"
 #include "compat/compiler.h"
-#include "run-command.h"
+#include "hook.h"
 
 
 static void get_system_info(struct strbuf *sys_info)
diff --git a/builtin/commit.c b/builtin/commit.c
index 190d215d43b..f1aafd67d46 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -19,6 +19,7 @@
 #include "revision.h"
 #include "wt-status.h"
 #include "run-command.h"
+#include "hook.h"
 #include "refs.h"
 #include "log-tree.h"
 #include "strbuf.h"
diff --git a/builtin/merge.c b/builtin/merge.c
index a8a843b1f54..be98d66b0a8 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -13,6 +13,7 @@
 #include "builtin.h"
 #include "lockfile.h"
 #include "run-command.h"
+#include "hook.h"
 #include "diff.h"
 #include "diff-merges.h"
 #include "refs.h"
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index a34742513ac..1e0e04c62fc 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -7,6 +7,7 @@
 #include "pkt-line.h"
 #include "sideband.h"
 #include "run-command.h"
+#include "hook.h"
 #include "exec-cmd.h"
 #include "commit.h"
 #include "object.h"
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 976bf8ed063..b1350640fed 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -8,6 +8,7 @@
 #include "branch.h"
 #include "refs.h"
 #include "run-command.h"
+#include "hook.h"
 #include "sigchain.h"
 #include "submodule.h"
 #include "utf8.h"
diff --git a/hook.c b/hook.c
index aa66c968186..06842f50e5c 100644
--- a/hook.c
+++ b/hook.c
@@ -2,6 +2,42 @@
 #include "hook.h"
 #include "run-command.h"
 
+const char *find_hook(const char *name)
+{
+	static struct strbuf path = STRBUF_INIT;
+
+	strbuf_reset(&path);
+	strbuf_git_path(&path, "hooks/%s", name);
+	if (access(path.buf, X_OK) < 0) {
+		int err = errno;
+
+#ifdef STRIP_EXTENSION
+		strbuf_addstr(&path, STRIP_EXTENSION);
+		if (access(path.buf, X_OK) >= 0)
+			return path.buf;
+		if (errno == EACCES)
+			err = errno;
+#endif
+
+		if (err == EACCES && advice_ignored_hook) {
+			static struct string_list advise_given = STRING_LIST_INIT_DUP;
+
+			if (!string_list_lookup(&advise_given, name)) {
+				string_list_insert(&advise_given, name);
+				advise(_("The '%s' hook was ignored because "
+					 "it's not set as executable.\n"
+					 "You can disable this warning with "
+					 "`git config advice.ignoredHook false`."),
+				       path.buf);
+			}
+		}
+		return NULL;
+	}
+	return path.buf;
+}
+
+
+
 void run_hooks_opt_clear(struct run_hooks_opt *o)
 {
 	strvec_clear(&o->env);
diff --git a/hook.h b/hook.h
index ebfee26bcf2..291ee19469a 100644
--- a/hook.h
+++ b/hook.h
@@ -37,6 +37,13 @@ struct hook_cb_data {
 	struct run_hooks_opt *options;
 };
 
+/*
+ * Returns the path to the hook file, or NULL if the hook is missing
+ * or disabled. Note that this points to static storage that will be
+ * overwritten by further calls to find_hook and run_hook_*.
+ */
+const char *find_hook(const char *name);
+
 void run_hooks_opt_clear(struct run_hooks_opt *o);
 
 /*
diff --git a/refs.c b/refs.c
index 8c9490235ea..59be29cf081 100644
--- a/refs.c
+++ b/refs.c
@@ -10,6 +10,7 @@
 #include "refs.h"
 #include "refs/refs-internal.h"
 #include "run-command.h"
+#include "hook.h"
 #include "object-store.h"
 #include "object.h"
 #include "tag.h"
diff --git a/run-command.c b/run-command.c
index be6bc128cd9..82fdf296569 100644
--- a/run-command.c
+++ b/run-command.c
@@ -8,6 +8,7 @@
 #include "string-list.h"
 #include "quote.h"
 #include "config.h"
+#include "hook.h"
 
 void child_process_init(struct child_process *child)
 {
@@ -1320,40 +1321,6 @@ int async_with_fork(void)
 #endif
 }
 
-const char *find_hook(const char *name)
-{
-	static struct strbuf path = STRBUF_INIT;
-
-	strbuf_reset(&path);
-	strbuf_git_path(&path, "hooks/%s", name);
-	if (access(path.buf, X_OK) < 0) {
-		int err = errno;
-
-#ifdef STRIP_EXTENSION
-		strbuf_addstr(&path, STRIP_EXTENSION);
-		if (access(path.buf, X_OK) >= 0)
-			return path.buf;
-		if (errno == EACCES)
-			err = errno;
-#endif
-
-		if (err == EACCES && advice_ignored_hook) {
-			static struct string_list advise_given = STRING_LIST_INIT_DUP;
-
-			if (!string_list_lookup(&advise_given, name)) {
-				string_list_insert(&advise_given, name);
-				advise(_("The '%s' hook was ignored because "
-					 "it's not set as executable.\n"
-					 "You can disable this warning with "
-					 "`git config advice.ignoredHook false`."),
-				       path.buf);
-			}
-		}
-		return NULL;
-	}
-	return path.buf;
-}
-
 int run_hook_ve(const char *const *env, const char *name, va_list args)
 {
 	struct child_process hook = CHILD_PROCESS_INIT;
diff --git a/run-command.h b/run-command.h
index d08414a92e7..b58531a7eb3 100644
--- a/run-command.h
+++ b/run-command.h
@@ -201,13 +201,6 @@ int finish_command_in_signal(struct child_process *);
  */
 int run_command(struct child_process *);
 
-/*
- * Returns the path to the hook file, or NULL if the hook is missing
- * or disabled. Note that this points to static storage that will be
- * overwritten by further calls to find_hook and run_hook_*.
- */
-const char *find_hook(const char *name);
-
 /**
  * Run a hook.
  * The first argument is a pathname to an index file, or NULL
diff --git a/sequencer.c b/sequencer.c
index 0bec01cf38e..3de479f90e1 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -8,6 +8,7 @@
 #include "sequencer.h"
 #include "tag.h"
 #include "run-command.h"
+#include "hook.h"
 #include "exec-cmd.h"
 #include "utf8.h"
 #include "cache-tree.h"
diff --git a/transport.c b/transport.c
index 50f5830eb6b..2ed270171f0 100644
--- a/transport.c
+++ b/transport.c
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "transport.h"
 #include "run-command.h"
+#include "hook.h"
 #include "pkt-line.h"
 #include "fetch-pack.h"
 #include "remote.h"
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 03/30] hook.c: add a hook_exists() wrapper and use it in bugreport.c
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
  2021-06-14 10:32         ` [PATCH v2 01/30] hook: add 'run' subcommand Ævar Arnfjörð Bjarmason
  2021-06-14 10:32         ` [PATCH v2 02/30] run-command.h: move find_hook() to hook.h Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:32         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:32         ` [PATCH v2 04/30] gc: use hook library for pre-auto-gc hook Ævar Arnfjörð Bjarmason
                           ` (29 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:32 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Add a boolean version of the find_hook() function for those callers
who are only interested in checking whether the hook exists, not what
the path to it is.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/bugreport.c | 2 +-
 hook.c              | 5 ++++-
 hook.h              | 5 +++++
 3 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/builtin/bugreport.c b/builtin/bugreport.c
index 596f079a7f9..941c8d5e270 100644
--- a/builtin/bugreport.c
+++ b/builtin/bugreport.c
@@ -82,7 +82,7 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 
 	for (i = 0; i < ARRAY_SIZE(hook); i++)
-		if (find_hook(hook[i]))
+		if (hook_exists(hook[i]))
 			strbuf_addf(hook_info, "%s\n", hook[i]);
 }
 
diff --git a/hook.c b/hook.c
index 06842f50e5c..c7da273822d 100644
--- a/hook.c
+++ b/hook.c
@@ -36,7 +36,10 @@ const char *find_hook(const char *name)
 	return path.buf;
 }
 
-
+int hook_exists(const char *name)
+{
+	return !!find_hook(name);
+}
 
 void run_hooks_opt_clear(struct run_hooks_opt *o)
 {
diff --git a/hook.h b/hook.h
index 291ee19469a..cbda7746a5d 100644
--- a/hook.h
+++ b/hook.h
@@ -44,6 +44,11 @@ struct hook_cb_data {
  */
 const char *find_hook(const char *name);
 
+/*
+ * A boolean version of find_hook()
+ */
+int hook_exists(const char *hookname);
+
 void run_hooks_opt_clear(struct run_hooks_opt *o);
 
 /*
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 04/30] gc: use hook library for pre-auto-gc hook
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (2 preceding siblings ...)
  2021-06-14 10:32         ` [PATCH v2 03/30] hook.c: add a hook_exists() wrapper and use it in bugreport.c Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:32         ` Ævar Arnfjörð Bjarmason
  2021-06-14 23:57           ` Emily Shaffer
  2021-06-14 10:32         ` [PATCH v2 05/30] rebase: teach pre-rebase to use hook.h Ævar Arnfjörð Bjarmason
                           ` (28 subsequent siblings)
  32 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:32 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Using the hook.h library instead of the run-command.h library to run
pre-auto-gc means that those hooks can be set up in config files, as
well as in the hookdir. pre-auto-gc is called only from builtin/gc.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/gc.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/builtin/gc.c b/builtin/gc.c
index f05d2f0a1ac..a12641a691d 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -32,6 +32,7 @@
 #include "remote.h"
 #include "object-store.h"
 #include "exec-cmd.h"
+#include "hook.h"
 
 #define FAILED_RUN "failed to run %s"
 
@@ -348,6 +349,8 @@ static void add_repack_incremental_option(void)
 
 static int need_to_gc(void)
 {
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+
 	/*
 	 * Setting gc.auto to 0 or negative can disable the
 	 * automatic gc.
@@ -394,8 +397,11 @@ static int need_to_gc(void)
 	else
 		return 0;
 
-	if (run_hook_le(NULL, "pre-auto-gc", NULL))
+	if (run_hooks("pre-auto-gc", &hook_opt)) {
+		run_hooks_opt_clear(&hook_opt);
 		return 0;
+	}
+	run_hooks_opt_clear(&hook_opt);
 	return 1;
 }
 
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 05/30] rebase: teach pre-rebase to use hook.h
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (3 preceding siblings ...)
  2021-06-14 10:32         ` [PATCH v2 04/30] gc: use hook library for pre-auto-gc hook Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:32         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:32         ` [PATCH v2 06/30] am: convert applypatch hooks to use config Ævar Arnfjörð Bjarmason
                           ` (27 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:32 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Move the pre-rebase hook away from run-command.h to and over to the
new hook.h library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/rebase.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/builtin/rebase.c b/builtin/rebase.c
index 12f093121d9..2081f6fa8db 100644
--- a/builtin/rebase.c
+++ b/builtin/rebase.c
@@ -28,6 +28,7 @@
 #include "sequencer.h"
 #include "rebase-interactive.h"
 #include "reset.h"
+#include "hook.h"
 
 #define DEFAULT_REFLOG_ACTION "rebase"
 
@@ -1313,6 +1314,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	char *squash_onto_name = NULL;
 	int reschedule_failed_exec = -1;
 	int allow_preemptive_ff = 1;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
 	struct option builtin_rebase_options[] = {
 		OPT_STRING(0, "onto", &options.onto_name,
 			   N_("revision"),
@@ -2022,10 +2024,13 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	}
 
 	/* If a hook exists, give it a chance to interrupt*/
+	strvec_pushl(&hook_opt.args, options.upstream_arg, argc ? argv[0] : NULL, NULL);
 	if (!ok_to_skip_pre_rebase &&
-	    run_hook_le(NULL, "pre-rebase", options.upstream_arg,
-			argc ? argv[0] : NULL, NULL))
+	    run_hooks("pre-rebase", &hook_opt)) {
+		run_hooks_opt_clear(&hook_opt);
 		die(_("The pre-rebase hook refused to rebase."));
+	}
+	run_hooks_opt_clear(&hook_opt);
 
 	if (options.flags & REBASE_DIFFSTAT) {
 		struct diff_options opts;
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 06/30] am: convert applypatch hooks to use config
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (4 preceding siblings ...)
  2021-06-14 10:32         ` [PATCH v2 05/30] rebase: teach pre-rebase to use hook.h Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:32         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:32         ` [PATCH v2 07/30] hooks: convert 'post-checkout' hook to hook library Ævar Arnfjörð Bjarmason
                           ` (26 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:32 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Teach pre-applypatch, post-applypatch, and applypatch-msg to use the
hook.h library instead of the run-command.h library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/am.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index 1c8a5489035..9e9c1b5e9f2 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -445,9 +445,12 @@ static void am_destroy(const struct am_state *state)
 static int run_applypatch_msg_hook(struct am_state *state)
 {
 	int ret;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 
 	assert(state->msg);
-	ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL);
+	strvec_push(&opt.args, am_path(state, "final-commit"));
+	ret = run_hooks("applypatch-msg", &opt);
+	run_hooks_opt_clear(&opt);
 
 	if (!ret) {
 		FREE_AND_NULL(state->msg);
@@ -1607,9 +1610,13 @@ static void do_commit(const struct am_state *state)
 	struct commit_list *parents = NULL;
 	const char *reflog_msg, *author, *committer = NULL;
 	struct strbuf sb = STRBUF_INIT;
+	struct run_hooks_opt hook_opt_pre = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt_post = RUN_HOOKS_OPT_INIT;
 
-	if (run_hook_le(NULL, "pre-applypatch", NULL))
+	if (run_hooks("pre-applypatch", &hook_opt_pre)) {
+		run_hooks_opt_clear(&hook_opt_pre);
 		exit(1);
+	}
 
 	if (write_cache_as_tree(&tree, 0, NULL))
 		die(_("git write-tree failed to write a tree"));
@@ -1660,8 +1667,10 @@ static void do_commit(const struct am_state *state)
 		fclose(fp);
 	}
 
-	run_hook_le(NULL, "post-applypatch", NULL);
+	run_hooks("post-applypatch", &hook_opt_post);
 
+	run_hooks_opt_clear(&hook_opt_pre);
+	run_hooks_opt_clear(&hook_opt_post);
 	strbuf_release(&sb);
 }
 
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 07/30] hooks: convert 'post-checkout' hook to hook library
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (5 preceding siblings ...)
  2021-06-14 10:32         ` [PATCH v2 06/30] am: convert applypatch hooks to use config Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:32         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:32         ` [PATCH v2 08/30] merge: use config-based hooks for post-merge hook Ævar Arnfjörð Bjarmason
                           ` (25 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:32 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Move the running of the 'post-checkout' hook away from run-command.h
to the new hook.h library. For "worktree" this requires a change to it
to run the hooks from a given directory.

We could strictly speaking skip the "absolute_path" flag and just
check if "dir" is specified, but let's split them up for clarity, as
well as for any future user who'd like to set "dir" but not implicitly
change the argument to an absolute path.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/checkout.c | 17 ++++++++++++-----
 builtin/clone.c    |  7 +++++--
 builtin/worktree.c | 30 ++++++++++++++----------------
 hook.c             |  8 ++++++++
 hook.h             |  9 +++++++++
 read-cache.c       |  1 +
 reset.c            | 15 +++++++++++----
 7 files changed, 60 insertions(+), 27 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index f4cd7747d35..6205ace09f6 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "diff.h"
 #include "dir.h"
+#include "hook.h"
 #include "ll-merge.h"
 #include "lockfile.h"
 #include "merge-recursive.h"
@@ -106,13 +107,19 @@ struct branch_info {
 static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit,
 			      int changed)
 {
-	return run_hook_le(NULL, "post-checkout",
-			   oid_to_hex(old_commit ? &old_commit->object.oid : null_oid()),
-			   oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()),
-			   changed ? "1" : "0", NULL);
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	int rc;
+
 	/* "new_commit" can be NULL when checking out from the index before
 	   a commit exists. */
-
+	strvec_pushl(&opt.args,
+		     oid_to_hex(old_commit ? &old_commit->object.oid : null_oid()),
+		     oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()),
+		     changed ? "1" : "0",
+		     NULL);
+	rc = run_hooks("post-checkout", &opt);
+	run_hooks_opt_clear(&opt);
+	return rc;
 }
 
 static int update_some(const struct object_id *oid, struct strbuf *base,
diff --git a/builtin/clone.c b/builtin/clone.c
index 66fe66679c8..de57a3119b7 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -32,6 +32,7 @@
 #include "connected.h"
 #include "packfile.h"
 #include "list-objects-filter-options.h"
+#include "hook.h"
 
 /*
  * Overall FIXMEs:
@@ -775,6 +776,7 @@ static int checkout(int submodule_progress)
 	struct tree *tree;
 	struct tree_desc t;
 	int err = 0;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
 
 	if (option_no_checkout)
 		return 0;
@@ -820,8 +822,9 @@ static int checkout(int submodule_progress)
 	if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
 		die(_("unable to write new index file"));
 
-	err |= run_hook_le(NULL, "post-checkout", oid_to_hex(null_oid()),
-			   oid_to_hex(&oid), "1", NULL);
+	strvec_pushl(&hook_opt.args, oid_to_hex(null_oid()), oid_to_hex(&oid), "1", NULL);
+	err |= run_hooks("post-checkout", &hook_opt);
+	run_hooks_opt_clear(&hook_opt);
 
 	if (!err && (option_recurse_submodules.nr > 0)) {
 		struct strvec args = STRVEC_INIT;
diff --git a/builtin/worktree.c b/builtin/worktree.c
index b1350640fed..2ad26a76f4c 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -382,22 +382,20 @@ static int add_worktree(const char *path, const char *refname,
 	 * is_junk is cleared, but do return appropriate code when hook fails.
 	 */
 	if (!ret && opts->checkout) {
-		const char *hook = find_hook("post-checkout");
-		if (hook) {
-			const char *env[] = { "GIT_DIR", "GIT_WORK_TREE", NULL };
-			cp.git_cmd = 0;
-			cp.no_stdin = 1;
-			cp.stdout_to_stderr = 1;
-			cp.dir = path;
-			cp.env = env;
-			cp.argv = NULL;
-			cp.trace2_hook_name = "post-checkout";
-			strvec_pushl(&cp.args, absolute_path(hook),
-				     oid_to_hex(null_oid()),
-				     oid_to_hex(&commit->object.oid),
-				     "1", NULL);
-			ret = run_command(&cp);
-		}
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
+		strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
+		strvec_pushl(&opt.args,
+			     oid_to_hex(null_oid()),
+			     oid_to_hex(&commit->object.oid),
+			     "1",
+			     NULL);
+		opt.dir = path;
+		opt.absolute_path = 1;
+
+		ret = run_hooks("post-checkout", &opt);
+
+		run_hooks_opt_clear(&opt);
 	}
 
 	strvec_clear(&child_env);
diff --git a/hook.c b/hook.c
index c7da273822d..51337f9798f 100644
--- a/hook.c
+++ b/hook.c
@@ -62,6 +62,7 @@ static int pick_next_hook(struct child_process *cp,
 	cp->env = hook_cb->options->env.v;
 	cp->stdout_to_stderr = 1;
 	cp->trace2_hook_name = hook_cb->hook_name;
+	cp->dir = hook_cb->options->dir;
 
 	/* add command */
 	strvec_push(&cp->args, run_me->hook_path);
@@ -111,6 +112,7 @@ static int notify_hook_finished(int result,
 int run_found_hooks(const char *hook_name, const char *hook_path,
 		    struct run_hooks_opt *options)
 {
+	struct strbuf abs_path = STRBUF_INIT;
 	struct hook my_hook = {
 		.hook_path = hook_path,
 	};
@@ -119,6 +121,10 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 		.hook_name = hook_name,
 		.options = options,
 	};
+	if (options->absolute_path) {
+		strbuf_add_absolute_path(&abs_path, hook_path);
+		my_hook.hook_path = abs_path.buf;
+	}
 	cb_data.run_me = &my_hook;
 
 	if (options->jobs != 1)
@@ -131,6 +137,8 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 				   &cb_data,
 				   "hook",
 				   hook_name);
+	if (options->absolute_path)
+		strbuf_release(&abs_path);
 
 	return cb_data.rc;
 }
diff --git a/hook.h b/hook.h
index cbda7746a5d..2d7724bbb50 100644
--- a/hook.h
+++ b/hook.h
@@ -19,6 +19,15 @@ struct run_hooks_opt
 
 	/* Number of threads to parallelize across */
 	int jobs;
+
+	/* Resolve and run the "absolute_path(hook)" instead of
+	 * "hook". Used for "git worktree" hooks
+	 */
+	int absolute_path;
+
+	/* Path to initial working directory for subprocess */
+	const char *dir;
+
 };
 
 #define RUN_HOOKS_OPT_INIT { \
diff --git a/read-cache.c b/read-cache.c
index 77961a38854..af5b97104cf 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -27,6 +27,7 @@
 #include "progress.h"
 #include "sparse-index.h"
 #include "csum-file.h"
+#include "hook.h"
 
 /* Mask for the name length in ce_flags in the on-disk index */
 
diff --git a/reset.c b/reset.c
index 4bea758053b..e6af33b901c 100644
--- a/reset.c
+++ b/reset.c
@@ -7,6 +7,7 @@
 #include "tree-walk.h"
 #include "tree.h"
 #include "unpack-trees.h"
+#include "hook.h"
 
 int reset_head(struct repository *r, struct object_id *oid, const char *action,
 	       const char *switch_to_branch, unsigned flags,
@@ -126,10 +127,16 @@ int reset_head(struct repository *r, struct object_id *oid, const char *action,
 			ret = create_symref("HEAD", switch_to_branch,
 					    reflog_head);
 	}
-	if (run_hook)
-		run_hook_le(NULL, "post-checkout",
-			    oid_to_hex(orig ? orig : null_oid()),
-			    oid_to_hex(oid), "1", NULL);
+	if (run_hook) {
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		strvec_pushl(&opt.args,
+			     oid_to_hex(orig ? orig : null_oid()),
+			     oid_to_hex(oid),
+			     "1",
+			     NULL);
+		run_hooks("post-checkout", &opt);
+		run_hooks_opt_clear(&opt);
+	}
 
 leave_reset_head:
 	strbuf_release(&msg);
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 08/30] merge: use config-based hooks for post-merge hook
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (6 preceding siblings ...)
  2021-06-14 10:32         ` [PATCH v2 07/30] hooks: convert 'post-checkout' hook to hook library Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:32         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:32         ` [PATCH v2 09/30] git hook run: add an --ignore-missing flag Ævar Arnfjörð Bjarmason
                           ` (24 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:32 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Teach post-merge to use the hook.h library instead of the run-command.h
library to run hooks. This means that post-merge hooks can come from the
config as well as from the hookdir. post-merge is invoked only from
builtin/merge.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/merge.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/builtin/merge.c b/builtin/merge.c
index be98d66b0a8..6128b60942f 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -448,6 +448,7 @@ static void finish(struct commit *head_commit,
 		   const struct object_id *new_head, const char *msg)
 {
 	struct strbuf reflog_message = STRBUF_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	const struct object_id *head = &head_commit->object.oid;
 
 	if (!msg)
@@ -489,7 +490,9 @@ static void finish(struct commit *head_commit,
 	}
 
 	/* Run a post-merge hook */
-	run_hook_le(NULL, "post-merge", squash ? "1" : "0", NULL);
+	strvec_push(&opt.args, squash ? "1" : "0");
+	run_hooks("post-merge", &opt);
+	run_hooks_opt_clear(&opt);
 
 	apply_autostash(git_path_merge_autostash(the_repository));
 	strbuf_release(&reflog_message);
@@ -849,7 +852,7 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 	 * and write it out as a tree.  We must do this before we invoke
 	 * the editor and after we invoke run_status above.
 	 */
-	if (find_hook("pre-merge-commit"))
+	if (hook_exists("pre-merge-commit"))
 		discard_cache();
 	read_cache_from(index_file);
 	strbuf_addbuf(&msg, &merge_msg);
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 09/30] git hook run: add an --ignore-missing flag
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (7 preceding siblings ...)
  2021-06-14 10:32         ` [PATCH v2 08/30] merge: use config-based hooks for post-merge hook Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:32         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:32         ` [PATCH v2 10/30] send-email: use 'git hook run' for 'sendemail-validate' Ævar Arnfjörð Bjarmason
                           ` (23 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:32 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

For certain one-shot hooks we'd like to optimistically run them, and
not complain if they don't exist. This will be used by send-email in a
subsequent commit.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-hook.txt | 10 +++++++++-
 builtin/hook.c             |  5 +++++
 t/t1800-hook.sh            |  5 +++++
 3 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 902b9cffaef..1528c860cf1 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -8,7 +8,7 @@ git-hook - run git hooks
 SYNOPSIS
 --------
 [verse]
-'git hook' run <hook-name> [-- <hook-args>]
+'git hook' run [--ignore-missing] <hook-name> [-- <hook-args>]
 
 DESCRIPTION
 -----------
@@ -27,6 +27,14 @@ run::
 	"--end-of-options"). See "OPTIONS" below for the arguments
 	this accepts.
 
+OPTIONS
+-------
+
+--ignore-missing::
+	Ignore any missing hook by quietly returning zero. Used for
+	tools that want to do a blind one-shot run of a hook that may
+	or may not be present.
+
 SEE ALSO
 --------
 linkgit:githooks[5]
diff --git a/builtin/hook.c b/builtin/hook.c
index 1b1a594fd00..275dd5b0ed0 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -16,10 +16,13 @@ static int run(int argc, const char **argv, const char *prefix)
 	int i;
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	int rc = 0;
+	int ignore_missing = 0;
 	const char *hook_name;
 	const char *hook_path;
 
 	struct option run_options[] = {
+		OPT_BOOL(0, "ignore-missing", &ignore_missing,
+			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
 		OPT_END(),
 	};
 
@@ -42,6 +45,8 @@ static int run(int argc, const char **argv, const char *prefix)
 	hook_name = argv[1];
 	hook_path = find_hook(hook_name);
 	if (!hook_path) {
+		if (ignore_missing)
+			return 0;
 		error("cannot find a hook named %s", hook_name);
 		return 1;
 	}
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
index f6ff6c4a493..714dfd08cd8 100755
--- a/t/t1800-hook.sh
+++ b/t/t1800-hook.sh
@@ -12,6 +12,11 @@ test_expect_success 'git hook run -- nonexistent hook' '
 	test_cmp stderr.expect stderr.actual
 '
 
+test_expect_success 'git hook run -- nonexistent hook with --ignore-missing' '
+	git hook run --ignore-missing does-not-exist 2>stderr.actual &&
+	test_must_be_empty stderr.actual
+'
+
 test_expect_success 'git hook run -- basic' '
 	write_script .git/hooks/test-hook <<-EOF &&
 	echo Test hook
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 10/30] send-email: use 'git hook run' for 'sendemail-validate'
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (8 preceding siblings ...)
  2021-06-14 10:32         ` [PATCH v2 09/30] git hook run: add an --ignore-missing flag Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:32         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:33         ` [PATCH v2 11/30] git-p4: use 'git hook' to run hooks Ævar Arnfjörð Bjarmason
                           ` (22 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:32 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Change the "sendmail-validate" hook to be run via the "git hook run"
wrapper instead of via a direct invocation.

This is the smallest possibly change to get "send-email" using "git
hook run". We still check the hook itself with "-x", and set a
"GIT_DIR" variable, both of which are asserted by our tests. We'll
need to get rid of this special behavior if we start running N hooks,
but for now let's be as close to bug-for-bug compatible as possible.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-send-email.perl   | 20 ++++++++++++--------
 t/t9001-send-email.sh |  4 ++--
 2 files changed, 14 insertions(+), 10 deletions(-)

diff --git a/git-send-email.perl b/git-send-email.perl
index 7ba0b3433d7..9e474304036 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -214,13 +214,13 @@ sub format_2822_time {
 my $editor;
 
 sub system_or_msg {
-	my ($args, $msg) = @_;
+	my ($args, $msg, $cmd_name) = @_;
 	system(@$args);
 	my $signalled = $? & 127;
 	my $exit_code = $? >> 8;
 	return unless $signalled or $exit_code;
 
-	my @sprintf_args = ($args->[0], $exit_code);
+	my @sprintf_args = ($cmd_name ? $cmd_name : $args->[0], $exit_code);
 	if (defined $msg) {
 		# Quiet the 'redundant' warning category, except we
 		# need to support down to Perl 5.8, so we can't do a
@@ -1979,9 +1979,9 @@ sub validate_patch {
 	my ($fn, $xfer_encoding) = @_;
 
 	if ($repo) {
+		my $hook_name = 'sendemail-validate';
 		my $hooks_path = $repo->command_oneline('rev-parse', '--git-path', 'hooks');
-		my $validate_hook = catfile($hooks_path,
-					    'sendemail-validate');
+		my $validate_hook = catfile($hooks_path, $hook_name);
 		my $hook_error;
 		if (-x $validate_hook) {
 			my $target = abs_path($fn);
@@ -1990,13 +1990,17 @@ sub validate_patch {
 			chdir($repo->wc_path() or $repo->repo_path())
 				or die("chdir: $!");
 			local $ENV{"GIT_DIR"} = $repo->repo_path();
-			$hook_error = system_or_msg([$validate_hook, $target]);
+			my @validate_hook = ("git", "hook", "run", "--ignore-missing", $hook_name, "--", $target);
+			$hook_error = system_or_msg(\@validate_hook, undef,
+						       "git hook run $hook_name -- <patch>");
 			chdir($cwd_save) or die("chdir: $!");
 		}
 		if ($hook_error) {
-			die sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
-				       "%s\n" .
-				       "warning: no patches were sent\n"), $fn, $hook_error);
+			$hook_error = sprintf(__("fatal: %s: rejected by %s hook\n" .
+						 $hook_error . "\n" .
+						 "warning: no patches were sent\n"),
+					      $fn, $hook_name);
+			die $hook_error;
 		}
 	}
 
diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index 30eff725a96..6d4e25df8dc 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -539,7 +539,7 @@ test_expect_success $PREREQ "--validate respects relative core.hooksPath path" '
 	test_path_is_file my-hooks.ran &&
 	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
-	fatal: command '"'"'my-hooks/sendemail-validate'"'"' died with exit code 1
+	fatal: command '"'"'git hook run sendemail-validate -- <patch>'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
@@ -558,7 +558,7 @@ test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" '
 	test_path_is_file my-hooks.ran &&
 	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
-	fatal: command '"'"'$hooks_path/sendemail-validate'"'"' died with exit code 1
+	fatal: command '"'"'git hook run sendemail-validate -- <patch>'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 11/30] git-p4: use 'git hook' to run hooks
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (9 preceding siblings ...)
  2021-06-14 10:32         ` [PATCH v2 10/30] send-email: use 'git hook run' for 'sendemail-validate' Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:33         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:33         ` [PATCH v2 12/30] commit: use hook.h to execute hooks Ævar Arnfjörð Bjarmason
                           ` (21 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Instead of duplicating the behavior of run-command.h:run_hook_le() in
Python, we can directly call 'git hook run'. We emulate the existence
check with the --ignore-missing flag.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-p4.py | 72 ++++++-------------------------------------------------
 1 file changed, 7 insertions(+), 65 deletions(-)

diff --git a/git-p4.py b/git-p4.py
index d34a1946b75..e76d8df3139 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -207,71 +207,13 @@ def decode_path(path):
         return path
 
 def run_git_hook(cmd, param=[]):
-    """Execute a hook if the hook exists."""
-    if verbose:
-        sys.stderr.write("Looking for hook: %s\n" % cmd)
-        sys.stderr.flush()
-
-    hooks_path = gitConfig("core.hooksPath")
-    if len(hooks_path) <= 0:
-        hooks_path = os.path.join(os.environ["GIT_DIR"], "hooks")
-
-    if not isinstance(param, list):
-        param=[param]
-
-    # resolve hook file name, OS depdenent
-    hook_file = os.path.join(hooks_path, cmd)
-    if platform.system() == 'Windows':
-        if not os.path.isfile(hook_file):
-            # look for the file with an extension
-            files = glob.glob(hook_file + ".*")
-            if not files:
-                return True
-            files.sort()
-            hook_file = files.pop()
-            while hook_file.upper().endswith(".SAMPLE"):
-                # The file is a sample hook. We don't want it
-                if len(files) > 0:
-                    hook_file = files.pop()
-                else:
-                    return True
-
-    if not os.path.isfile(hook_file) or not os.access(hook_file, os.X_OK):
-        return True
-
-    return run_hook_command(hook_file, param) == 0
-
-def run_hook_command(cmd, param):
-    """Executes a git hook command
-       cmd = the command line file to be executed. This can be
-       a file that is run by OS association.
-
-       param = a list of parameters to pass to the cmd command
-
-       On windows, the extension is checked to see if it should
-       be run with the Git for Windows Bash shell.  If there
-       is no file extension, the file is deemed a bash shell
-       and will be handed off to sh.exe. Otherwise, Windows
-       will be called with the shell to handle the file assocation.
-
-       For non Windows operating systems, the file is called
-       as an executable.
-    """
-    cli = [cmd] + param
-    use_shell = False
-    if platform.system() == 'Windows':
-        (root,ext) = os.path.splitext(cmd)
-        if ext == "":
-            exe_path = os.environ.get("EXEPATH")
-            if exe_path is None:
-                exe_path = ""
-            else:
-                exe_path = os.path.join(exe_path, "bin")
-            cli = [os.path.join(exe_path, "SH.EXE")] + cli
-        else:
-            use_shell = True
-    return subprocess.call(cli, shell=use_shell)
-
+    """args are specified with -a <arg> -a <arg> -a <arg>"""
+    args = ['git', 'hook', 'run', '--ignore-missing', cmd]
+    if param:
+        args.append("--")
+        for p in param:
+            args.append(p)
+    return subprocess.call(args) == 0
 
 def write_pipe(c, stdin):
     if verbose:
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 12/30] commit: use hook.h to execute hooks
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (10 preceding siblings ...)
  2021-06-14 10:33         ` [PATCH v2 11/30] git-p4: use 'git hook' to run hooks Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:33         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:33         ` [PATCH v2 13/30] read-cache: convert post-index-change hook to use config Ævar Arnfjörð Bjarmason
                           ` (20 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Teach run_commit_hook() to call hook.h instead of run-command.h. This
covers 'pre-commit', 'commit-msg', and
'prepare-commit-msg'.

Additionally, ask the hook library - not run-command - whether any
hooks will be run, as it's possible hooks may exist in the config but
not the hookdir.

Because all but 'post-commit' hooks are expected to make some state
change, force all but 'post-commit' hook to run in series. 'post-commit'
"is meant primarily for notification, and cannot affect the outcome of
`git commit`," so it is fine to run in parallel.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/commit.c |  2 +-
 commit.c         | 16 ++++++++++------
 sequencer.c      |  2 +-
 3 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index f1aafd67d46..dad4e565443 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1045,7 +1045,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (!no_verify && find_hook("pre-commit")) {
+	if (!no_verify && hook_exists("pre-commit")) {
 		/*
 		 * Re-read the index as pre-commit hook could have updated it,
 		 * and write it out as a tree.  We must do this before we invoke
diff --git a/commit.c b/commit.c
index 8ea55a447fa..e8147a88fc6 100644
--- a/commit.c
+++ b/commit.c
@@ -21,6 +21,7 @@
 #include "commit-reach.h"
 #include "run-command.h"
 #include "shallow.h"
+#include "hook.h"
 
 static struct commit_extra_header *read_commit_extra_header_lines(const char *buf, size_t len, const char **);
 
@@ -1698,22 +1699,25 @@ size_t ignore_non_trailer(const char *buf, size_t len)
 int run_commit_hook(int editor_is_used, const char *index_file,
 		    const char *name, ...)
 {
-	struct strvec hook_env = STRVEC_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	va_list args;
+	const char *arg;
 	int ret;
-
-	strvec_pushf(&hook_env, "GIT_INDEX_FILE=%s", index_file);
+	strvec_pushf(&opt.env, "GIT_INDEX_FILE=%s", index_file);
 
 	/*
 	 * Let the hook know that no editor will be launched.
 	 */
 	if (!editor_is_used)
-		strvec_push(&hook_env, "GIT_EDITOR=:");
+		strvec_push(&opt.env, "GIT_EDITOR=:");
 
 	va_start(args, name);
-	ret = run_hook_ve(hook_env.v, name, args);
+	while ((arg = va_arg(args, const char *)))
+		strvec_push(&opt.args, arg);
 	va_end(args);
-	strvec_clear(&hook_env);
+
+	ret = run_hooks(name, &opt);
+	run_hooks_opt_clear(&opt);
 
 	return ret;
 }
diff --git a/sequencer.c b/sequencer.c
index 3de479f90e1..8f46984ffb7 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1446,7 +1446,7 @@ static int try_to_commit(struct repository *r,
 		}
 	}
 
-	if (find_hook("prepare-commit-msg")) {
+	if (hook_exists("prepare-commit-msg")) {
 		res = run_prepare_commit_msg_hook(r, msg, hook_commit);
 		if (res)
 			goto out;
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 13/30] read-cache: convert post-index-change hook to use config
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (11 preceding siblings ...)
  2021-06-14 10:33         ` [PATCH v2 12/30] commit: use hook.h to execute hooks Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:33         ` Ævar Arnfjörð Bjarmason
  2021-06-25 18:32           ` Felipe Contreras
  2021-06-14 10:33         ` [PATCH v2 14/30] receive-pack: convert push-to-checkout hook to hook.h Ævar Arnfjörð Bjarmason
                           ` (19 subsequent siblings)
  32 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

By using hook.h instead of run-command.h to run, post-index-change hooks
can now be specified in the config in addition to the hookdir.
post-index-change is not run anywhere besides in read-cache.c.

This removes the last direct user of run_hook_ve(), so we can make the
function static now. It'll be removed entirely soon.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 read-cache.c  | 11 ++++++++---
 run-command.c |  2 +-
 run-command.h |  1 -
 3 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/read-cache.c b/read-cache.c
index af5b97104cf..f801313cc95 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -3063,6 +3063,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 {
 	int ret;
 	int was_full = !istate->sparse_index;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
 
 	ret = convert_to_sparse(istate);
 
@@ -3091,9 +3092,13 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 	else
 		ret = close_lock_file_gently(lock);
 
-	run_hook_le(NULL, "post-index-change",
-			istate->updated_workdir ? "1" : "0",
-			istate->updated_skipworktree ? "1" : "0", NULL);
+	strvec_pushl(&hook_opt.args,
+		     istate->updated_workdir ? "1" : "0",
+		     istate->updated_skipworktree ? "1" : "0",
+		     NULL);
+	run_hooks("post-index-change", &hook_opt);
+	run_hooks_opt_clear(&hook_opt);
+
 	istate->updated_workdir = 0;
 	istate->updated_skipworktree = 0;
 
diff --git a/run-command.c b/run-command.c
index 82fdf296569..eecdef5a0c8 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1321,7 +1321,7 @@ int async_with_fork(void)
 #endif
 }
 
-int run_hook_ve(const char *const *env, const char *name, va_list args)
+static int run_hook_ve(const char *const *env, const char *name, va_list args)
 {
 	struct child_process hook = CHILD_PROCESS_INIT;
 	const char *p;
diff --git a/run-command.h b/run-command.h
index b58531a7eb3..24ab5d63c4c 100644
--- a/run-command.h
+++ b/run-command.h
@@ -216,7 +216,6 @@ int run_command(struct child_process *);
  */
 LAST_ARG_MUST_BE_NULL
 int run_hook_le(const char *const *env, const char *name, ...);
-int run_hook_ve(const char *const *env, const char *name, va_list args);
 
 /*
  * Trigger an auto-gc
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 14/30] receive-pack: convert push-to-checkout hook to hook.h
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (12 preceding siblings ...)
  2021-06-14 10:33         ` [PATCH v2 13/30] read-cache: convert post-index-change hook to use config Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:33         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:33         ` [PATCH v2 15/30] run-command: remove old run_hook_{le,ve}() hook API Ævar Arnfjörð Bjarmason
                           ` (18 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

By using hook.h instead of run-command.h to invoke push-to-checkout,
hooks can now be specified in the config as well as in the hookdir.
push-to-checkout is not called anywhere but in builtin/receive-pack.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/receive-pack.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 1e0e04c62fc..5248228ebfe 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1436,12 +1436,18 @@ static const char *push_to_checkout(unsigned char *hash,
 				    struct strvec *env,
 				    const char *work_tree)
 {
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
-	if (run_hook_le(env->v, push_to_checkout_hook,
-			hash_to_hex(hash), NULL))
+	strvec_pushv(&opt.env, env->v);
+	strvec_push(&opt.args, hash_to_hex(hash));
+	if (run_hooks(push_to_checkout_hook, &opt)) {
+		run_hooks_opt_clear(&opt);
 		return "push-to-checkout hook declined";
-	else
+	} else {
+		run_hooks_opt_clear(&opt);
 		return NULL;
+	}
 }
 
 static const char *update_worktree(unsigned char *sha1, const struct worktree *worktree)
@@ -1465,7 +1471,7 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 
 	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
 
-	if (!find_hook(push_to_checkout_hook))
+	if (!hook_exists(push_to_checkout_hook))
 		retval = push_to_deploy(sha1, &env, work_tree);
 	else
 		retval = push_to_checkout(sha1, &env, work_tree);
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 15/30] run-command: remove old run_hook_{le,ve}() hook API
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (13 preceding siblings ...)
  2021-06-14 10:33         ` [PATCH v2 14/30] receive-pack: convert push-to-checkout hook to hook.h Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:33         ` Ævar Arnfjörð Bjarmason
  2021-06-25 18:34           ` Felipe Contreras
  2021-06-14 10:33         ` [PATCH v2 16/30] run-command: allow stdin for run_processes_parallel Ævar Arnfjörð Bjarmason
                           ` (17 subsequent siblings)
  32 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

The new hook.h library has replaced all run-command.h hook-related
functionality. So let's delete this dead code.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 run-command.c | 32 --------------------------------
 run-command.h | 16 ----------------
 2 files changed, 48 deletions(-)

diff --git a/run-command.c b/run-command.c
index eecdef5a0c8..95c950a4a2b 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1321,38 +1321,6 @@ int async_with_fork(void)
 #endif
 }
 
-static int run_hook_ve(const char *const *env, const char *name, va_list args)
-{
-	struct child_process hook = CHILD_PROCESS_INIT;
-	const char *p;
-
-	p = find_hook(name);
-	if (!p)
-		return 0;
-
-	strvec_push(&hook.args, p);
-	while ((p = va_arg(args, const char *)))
-		strvec_push(&hook.args, p);
-	hook.env = env;
-	hook.no_stdin = 1;
-	hook.stdout_to_stderr = 1;
-	hook.trace2_hook_name = name;
-
-	return run_command(&hook);
-}
-
-int run_hook_le(const char *const *env, const char *name, ...)
-{
-	va_list args;
-	int ret;
-
-	va_start(args, name);
-	ret = run_hook_ve(env, name, args);
-	va_end(args);
-
-	return ret;
-}
-
 struct io_pump {
 	/* initialized by caller */
 	int fd;
diff --git a/run-command.h b/run-command.h
index 24ab5d63c4c..748d4fc2a72 100644
--- a/run-command.h
+++ b/run-command.h
@@ -201,22 +201,6 @@ int finish_command_in_signal(struct child_process *);
  */
 int run_command(struct child_process *);
 
-/**
- * Run a hook.
- * The first argument is a pathname to an index file, or NULL
- * if the hook uses the default index file or no index is needed.
- * The second argument is the name of the hook.
- * The further arguments correspond to the hook arguments.
- * The last argument has to be NULL to terminate the arguments list.
- * If the hook does not exist or is not executable, the return
- * value will be zero.
- * If it is executable, the hook will be executed and the exit
- * status of the hook is returned.
- * On execution, .stdout_to_stderr and .no_stdin will be set.
- */
-LAST_ARG_MUST_BE_NULL
-int run_hook_le(const char *const *env, const char *name, ...);
-
 /*
  * Trigger an auto-gc
  */
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 16/30] run-command: allow stdin for run_processes_parallel
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (14 preceding siblings ...)
  2021-06-14 10:33         ` [PATCH v2 15/30] run-command: remove old run_hook_{le,ve}() hook API Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:33         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:33         ` [PATCH v2 17/30] hook: support passing stdin to hooks Ævar Arnfjörð Bjarmason
                           ` (16 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

While it makes sense not to inherit stdin from the parent process to
avoid deadlocking, it's not necessary to completely ban stdin to
children. An informed user should be able to configure stdin safely. By
setting `some_child.process.no_stdin=1` before calling `get_next_task()`
we provide a reasonable default behavior but enable users to set up
stdin streaming for themselves during the callback.

`some_child.process.stdout_to_stderr`, however, remains unmodifiable by
`get_next_task()` - the rest of the run_processes_parallel() API depends
on child output in stderr.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 run-command.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/run-command.c b/run-command.c
index 95c950a4a2b..0bf771845e4 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1628,6 +1628,14 @@ static int pp_start_one(struct parallel_processes *pp)
 	if (i == pp->max_processes)
 		BUG("bookkeeping is hard");
 
+	/*
+	 * By default, do not inherit stdin from the parent process - otherwise,
+	 * all children would share stdin! Users may overwrite this to provide
+	 * something to the child's stdin by having their 'get_next_task'
+	 * callback assign 0 to .no_stdin and an appropriate integer to .in.
+	 */
+	pp->children[i].process.no_stdin = 1;
+
 	code = pp->get_next_task(&pp->children[i].process,
 				 &pp->children[i].err,
 				 pp->data,
@@ -1639,7 +1647,6 @@ static int pp_start_one(struct parallel_processes *pp)
 	}
 	pp->children[i].process.err = -1;
 	pp->children[i].process.stdout_to_stderr = 1;
-	pp->children[i].process.no_stdin = 1;
 
 	if (start_command(&pp->children[i].process)) {
 		code = pp->start_failure(&pp->children[i].err,
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 17/30] hook: support passing stdin to hooks
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (15 preceding siblings ...)
  2021-06-14 10:33         ` [PATCH v2 16/30] run-command: allow stdin for run_processes_parallel Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:33         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:33         ` [PATCH v2 18/30] am: convert 'post-rewrite' hook to hook.h Ævar Arnfjörð Bjarmason
                           ` (15 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Some hooks (such as post-rewrite) need to take input via stdin.
Previously, callers provided stdin to hooks by setting
run-command.h:child_process.in, which takes a FD. Callers would open the
file in question themselves before calling run-command(). However, since
we will now need to seek to the front of the file and read it again for
every hook which runs, hook.h:run_command() takes a path and handles FD
management itself. Since this file is opened for read only, it should
not prevent later parallel execution support.

On the frontend, this is supported by asking for a file path, rather
than by reading stdin. Reading directly from stdin would involve caching
the entire stdin (to memory or to disk) and reading it back from the
beginning to each hook. We'd want to support cases like insufficient
memory or storage for the file. While this may prove useful later, for
now the path of least resistance is to just ask the user to make this
interim file themselves.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-hook.txt |  7 ++++++-
 builtin/hook.c             |  4 +++-
 hook.c                     |  8 +++++++-
 hook.h                     |  2 ++
 t/t1800-hook.sh            | 18 ++++++++++++++++++
 5 files changed, 36 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 1528c860cf1..816b3eda460 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -8,7 +8,7 @@ git-hook - run git hooks
 SYNOPSIS
 --------
 [verse]
-'git hook' run [--ignore-missing] <hook-name> [-- <hook-args>]
+'git hook' run [--to-stdin=<path>] [--ignore-missing] <hook-name> [-- <hook-args>]
 
 DESCRIPTION
 -----------
@@ -30,6 +30,11 @@ run::
 OPTIONS
 -------
 
+--to-stdin::
+	For "run"; Specify a file which will be streamed into the
+	hook's stdin. The hook will receive the entire file from
+	beginning to EOF.
+
 --ignore-missing::
 	Ignore any missing hook by quietly returning zero. Used for
 	tools that want to do a blind one-shot run of a hook that may
diff --git a/builtin/hook.c b/builtin/hook.c
index 275dd5b0ed0..baaef4dce49 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -7,7 +7,7 @@
 #include "strvec.h"
 
 static const char * const builtin_hook_usage[] = {
-	N_("git hook run <hook-name> [-- <hook-args>]"),
+	N_("git hook run [--to-stdin=<path>] <hook-name> [-- <hook-args>]"),
 	NULL
 };
 
@@ -23,6 +23,8 @@ static int run(int argc, const char **argv, const char *prefix)
 	struct option run_options[] = {
 		OPT_BOOL(0, "ignore-missing", &ignore_missing,
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
+		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
+			   N_("file to read into hooks' stdin")),
 		OPT_END(),
 	};
 
diff --git a/hook.c b/hook.c
index 51337f9798f..daf39f61741 100644
--- a/hook.c
+++ b/hook.c
@@ -58,7 +58,13 @@ static int pick_next_hook(struct child_process *cp,
 	if (!run_me)
 		BUG("did we not return 1 in notify_hook_finished?");
 
-	cp->no_stdin = 1;
+	/* reopen the file for stdin; run_command closes it. */
+	if (hook_cb->options->path_to_stdin) {
+		cp->no_stdin = 0;
+		cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY);
+	} else {
+		cp->no_stdin = 1;
+	}
 	cp->env = hook_cb->options->env.v;
 	cp->stdout_to_stderr = 1;
 	cp->trace2_hook_name = hook_cb->hook_name;
diff --git a/hook.h b/hook.h
index 2d7724bbb50..74a8c76a94c 100644
--- a/hook.h
+++ b/hook.h
@@ -28,6 +28,8 @@ struct run_hooks_opt
 	/* Path to initial working directory for subprocess */
 	const char *dir;
 
+	/* Path to file which should be piped to stdin for each hook */
+	const char *path_to_stdin;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
index 714dfd08cd8..1a6b708137e 100755
--- a/t/t1800-hook.sh
+++ b/t/t1800-hook.sh
@@ -133,4 +133,22 @@ test_expect_success 'set up a pre-commit hook in core.hooksPath' '
 	EOF
 '
 
+test_expect_success 'stdin to hooks' '
+	write_script .git/hooks/test-hook <<-\EOF &&
+	echo BEGIN stdin
+	cat
+	echo END stdin
+	EOF
+
+	cat >expect <<-EOF &&
+	BEGIN stdin
+	hello
+	END stdin
+	EOF
+
+	echo hello >input &&
+	git hook run --to-stdin=input test-hook 2>actual &&
+	test_cmp expect actual
+'
+
 test_done
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 18/30] am: convert 'post-rewrite' hook to hook.h
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (16 preceding siblings ...)
  2021-06-14 10:33         ` [PATCH v2 17/30] hook: support passing stdin to hooks Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:33         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:33         ` [PATCH v2 19/30] run-command: add stdin callback for parallelization Ævar Arnfjörð Bjarmason
                           ` (14 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/am.c | 18 +++++-------------
 1 file changed, 5 insertions(+), 13 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index 9e9c1b5e9f2..6e4f9c80360 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -467,23 +467,15 @@ static int run_applypatch_msg_hook(struct am_state *state)
  */
 static int run_post_rewrite_hook(const struct am_state *state)
 {
-	struct child_process cp = CHILD_PROCESS_INIT;
-	const char *hook = find_hook("post-rewrite");
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	int ret;
 
-	if (!hook)
-		return 0;
-
-	strvec_push(&cp.args, hook);
-	strvec_push(&cp.args, "rebase");
-
-	cp.in = xopen(am_path(state, "rewritten"), O_RDONLY);
-	cp.stdout_to_stderr = 1;
-	cp.trace2_hook_name = "post-rewrite";
+	strvec_push(&opt.args, "rebase");
+	opt.path_to_stdin = am_path(state, "rewritten");
 
-	ret = run_command(&cp);
+	ret = run_hooks("post-rewrite", &opt);
 
-	close(cp.in);
+	run_hooks_opt_clear(&opt);
 	return ret;
 }
 
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 19/30] run-command: add stdin callback for parallelization
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (17 preceding siblings ...)
  2021-06-14 10:33         ` [PATCH v2 18/30] am: convert 'post-rewrite' hook to hook.h Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:33         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:33         ` [PATCH v2 20/30] hook: provide stdin by string_list or callback Ævar Arnfjörð Bjarmason
                           ` (13 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

If a user of the run_processes_parallel() API wants to pipe a large
amount of information to stdin of each parallel command, that
information could exceed the buffer of the pipe allocated for that
process's stdin.  Generally this is solved by repeatedly writing to
child_process.in between calls to start_command() and finish_command();
run_processes_parallel() did not provide users an opportunity to access
child_process at that time.

Because the data might be extremely large (for example, a list of all
refs received during a push from a client) simply taking a string_list
or strbuf is not as scalable as using a callback; the rest of the
run_processes_parallel() API also uses callbacks, so making this feature
match the rest of the API reduces mental load on the user.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/fetch.c             |  1 +
 builtin/submodule--helper.c |  2 +-
 hook.c                      |  1 +
 run-command.c               | 54 +++++++++++++++++++++++++++++++++++--
 run-command.h               | 17 +++++++++++-
 submodule.c                 |  1 +
 t/helper/test-run-command.c | 31 ++++++++++++++++++---
 t/t0061-run-command.sh      | 30 +++++++++++++++++++++
 8 files changed, 129 insertions(+), 8 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index 9191620e50c..3d8f04b392d 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1817,6 +1817,7 @@ static int fetch_multiple(struct string_list *list, int max_children)
 		result = run_processes_parallel_tr2(max_children,
 						    &fetch_next_remote,
 						    &fetch_failed_to_start,
+						    NULL,
 						    &fetch_finished,
 						    &state,
 						    "fetch", "parallel/fetch");
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index ae6174ab05b..818494dd18e 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2295,7 +2295,7 @@ static int update_submodules(struct submodule_update_clone *suc)
 	int i;
 
 	run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task,
-				   update_clone_start_failure,
+				   update_clone_start_failure, NULL,
 				   update_clone_task_finished, suc, "submodule",
 				   "parallel/update");
 
diff --git a/hook.c b/hook.c
index daf39f61741..5f6335bac3f 100644
--- a/hook.c
+++ b/hook.c
@@ -139,6 +139,7 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 	run_processes_parallel_tr2(options->jobs,
 				   pick_next_hook,
 				   notify_start_failure,
+				   NULL,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
diff --git a/run-command.c b/run-command.c
index 0bf771845e4..3392640f17b 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1493,6 +1493,7 @@ struct parallel_processes {
 
 	get_next_task_fn get_next_task;
 	start_failure_fn start_failure;
+	feed_pipe_fn feed_pipe;
 	task_finished_fn task_finished;
 
 	struct {
@@ -1520,6 +1521,13 @@ static int default_start_failure(struct strbuf *out,
 	return 0;
 }
 
+static int default_feed_pipe(struct strbuf *pipe,
+			     void *pp_cb,
+			     void *pp_task_cb)
+{
+	return 1;
+}
+
 static int default_task_finished(int result,
 				 struct strbuf *out,
 				 void *pp_cb,
@@ -1550,6 +1558,7 @@ static void pp_init(struct parallel_processes *pp,
 		    int n,
 		    get_next_task_fn get_next_task,
 		    start_failure_fn start_failure,
+		    feed_pipe_fn feed_pipe,
 		    task_finished_fn task_finished,
 		    void *data)
 {
@@ -1568,6 +1577,7 @@ static void pp_init(struct parallel_processes *pp,
 	pp->get_next_task = get_next_task;
 
 	pp->start_failure = start_failure ? start_failure : default_start_failure;
+	pp->feed_pipe = feed_pipe ? feed_pipe : default_feed_pipe;
 	pp->task_finished = task_finished ? task_finished : default_task_finished;
 
 	pp->nr_processes = 0;
@@ -1665,6 +1675,37 @@ static int pp_start_one(struct parallel_processes *pp)
 	return 0;
 }
 
+static void pp_buffer_stdin(struct parallel_processes *pp)
+{
+	int i;
+	struct strbuf sb = STRBUF_INIT;
+
+	/* Buffer stdin for each pipe. */
+	for (i = 0; i < pp->max_processes; i++) {
+		if (pp->children[i].state == GIT_CP_WORKING &&
+		    pp->children[i].process.in > 0) {
+			int done;
+			strbuf_reset(&sb);
+			done = pp->feed_pipe(&sb, pp->data,
+					      pp->children[i].data);
+			if (sb.len) {
+				if (write_in_full(pp->children[i].process.in,
+					      sb.buf, sb.len) < 0) {
+					if (errno != EPIPE)
+						die_errno("write");
+					done = 1;
+				}
+			}
+			if (done) {
+				close(pp->children[i].process.in);
+				pp->children[i].process.in = 0;
+			}
+		}
+	}
+
+	strbuf_release(&sb);
+}
+
 static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout)
 {
 	int i;
@@ -1729,6 +1770,7 @@ static int pp_collect_finished(struct parallel_processes *pp)
 		pp->nr_processes--;
 		pp->children[i].state = GIT_CP_FREE;
 		pp->pfd[i].fd = -1;
+		pp->children[i].process.in = 0;
 		child_process_init(&pp->children[i].process);
 
 		if (i != pp->output_owner) {
@@ -1762,6 +1804,7 @@ static int pp_collect_finished(struct parallel_processes *pp)
 int run_processes_parallel(int n,
 			   get_next_task_fn get_next_task,
 			   start_failure_fn start_failure,
+			   feed_pipe_fn feed_pipe,
 			   task_finished_fn task_finished,
 			   void *pp_cb)
 {
@@ -1770,7 +1813,9 @@ int run_processes_parallel(int n,
 	int spawn_cap = 4;
 	struct parallel_processes pp;
 
-	pp_init(&pp, n, get_next_task, start_failure, task_finished, pp_cb);
+	sigchain_push(SIGPIPE, SIG_IGN);
+
+	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, task_finished, pp_cb);
 	while (1) {
 		for (i = 0;
 		    i < spawn_cap && !pp.shutdown &&
@@ -1787,6 +1832,7 @@ int run_processes_parallel(int n,
 		}
 		if (!pp.nr_processes)
 			break;
+		pp_buffer_stdin(&pp);
 		pp_buffer_stderr(&pp, output_timeout);
 		pp_output(&pp);
 		code = pp_collect_finished(&pp);
@@ -1798,11 +1844,15 @@ int run_processes_parallel(int n,
 	}
 
 	pp_cleanup(&pp);
+
+	sigchain_pop(SIGPIPE);
+
 	return 0;
 }
 
 int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 			       start_failure_fn start_failure,
+			       feed_pipe_fn feed_pipe,
 			       task_finished_fn task_finished, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label)
 {
@@ -1812,7 +1862,7 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 				   ((n < 1) ? online_cpus() : n));
 
 	result = run_processes_parallel(n, get_next_task, start_failure,
-					task_finished, pp_cb);
+					feed_pipe, task_finished, pp_cb);
 
 	trace2_region_leave(tr2_category, tr2_label, NULL);
 
diff --git a/run-command.h b/run-command.h
index 748d4fc2a72..41e36d26cb1 100644
--- a/run-command.h
+++ b/run-command.h
@@ -419,6 +419,20 @@ typedef int (*start_failure_fn)(struct strbuf *out,
 				void *pp_cb,
 				void *pp_task_cb);
 
+/**
+ * This callback is called repeatedly on every child process who requests
+ * start_command() to create a pipe by setting child_process.in < 0.
+ *
+ * pp_cb is the callback cookie as passed into run_processes_parallel, and
+ * pp_task_cb is the callback cookie as passed into get_next_task_fn.
+ * The contents of 'send' will be read into the pipe and passed to the pipe.
+ *
+ * Return nonzero to close the pipe.
+ */
+typedef int (*feed_pipe_fn)(struct strbuf *pipe,
+			    void *pp_cb,
+			    void *pp_task_cb);
+
 /**
  * This callback is called on every child process that finished processing.
  *
@@ -453,10 +467,11 @@ typedef int (*task_finished_fn)(int result,
 int run_processes_parallel(int n,
 			   get_next_task_fn,
 			   start_failure_fn,
+			   feed_pipe_fn,
 			   task_finished_fn,
 			   void *pp_cb);
 int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn,
-			       task_finished_fn, void *pp_cb,
+			       feed_pipe_fn, task_finished_fn, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label);
 
 #endif
diff --git a/submodule.c b/submodule.c
index 0b1d9c1dde5..ea026a8195f 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1645,6 +1645,7 @@ int fetch_populated_submodules(struct repository *r,
 	run_processes_parallel_tr2(max_parallel_jobs,
 				   get_next_submodule,
 				   fetch_start_failure,
+				   NULL,
 				   fetch_finish,
 				   &spf,
 				   "submodule", "parallel/fetch");
diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c
index 7ae03dc7123..9348184d303 100644
--- a/t/helper/test-run-command.c
+++ b/t/helper/test-run-command.c
@@ -32,8 +32,13 @@ static int parallel_next(struct child_process *cp,
 		return 0;
 
 	strvec_pushv(&cp->args, d->argv);
+	cp->in = d->in;
+	cp->no_stdin = d->no_stdin;
 	strbuf_addstr(err, "preloaded output of a child\n");
 	number_callbacks++;
+
+	*task_cb = xmalloc(sizeof(int));
+	*(int*)(*task_cb) = 2;
 	return 1;
 }
 
@@ -55,6 +60,17 @@ static int task_finished(int result,
 	return 1;
 }
 
+static int test_stdin(struct strbuf *pipe, void *cb, void *task_cb)
+{
+	int *lines_remaining = task_cb;
+
+	if (*lines_remaining)
+		strbuf_addf(pipe, "sample stdin %d\n", --(*lines_remaining));
+
+	return !(*lines_remaining);
+}
+
+
 struct testsuite {
 	struct string_list tests, failed;
 	int next;
@@ -185,7 +201,7 @@ static int testsuite(int argc, const char **argv)
 		suite.tests.nr, max_jobs);
 
 	ret = run_processes_parallel(max_jobs, next_test, test_failed,
-				     test_finished, &suite);
+				     test_stdin, test_finished, &suite);
 
 	if (suite.failed.nr > 0) {
 		ret = 1;
@@ -413,15 +429,22 @@ int cmd__run_command(int argc, const char **argv)
 
 	if (!strcmp(argv[1], "run-command-parallel"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, &proc));
+					    NULL, NULL, NULL, &proc));
 
 	if (!strcmp(argv[1], "run-command-abort"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, task_finished, &proc));
+					    NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-no-jobs"))
 		exit(run_processes_parallel(jobs, no_job,
-					    NULL, task_finished, &proc));
+					    NULL, NULL, task_finished, &proc));
+
+	if (!strcmp(argv[1], "run-command-stdin")) {
+		proc.in = -1;
+		proc.no_stdin = 0;
+		exit (run_processes_parallel(jobs, parallel_next, NULL,
+					     test_stdin, NULL, &proc));
+	}
 
 	fprintf(stderr, "check usage\n");
 	return 1;
diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh
index 7d599675e35..87759482ad1 100755
--- a/t/t0061-run-command.sh
+++ b/t/t0061-run-command.sh
@@ -143,6 +143,36 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai
 	test_cmp expect actual
 '
 
+cat >expect <<-EOF
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+EOF
+
+test_expect_success 'run_command listens to stdin' '
+	write_script stdin-script <<-\EOF &&
+	echo "listening for stdin:"
+	while read line; do
+		echo "$line"
+	done
+	EOF
+	test-tool run-command run-command-stdin 2 ./stdin-script 2>actual &&
+	test_cmp expect actual
+'
+
 cat >expect <<-EOF
 preloaded output of a child
 asking for a quick stop
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 20/30] hook: provide stdin by string_list or callback
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (18 preceding siblings ...)
  2021-06-14 10:33         ` [PATCH v2 19/30] run-command: add stdin callback for parallelization Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:33         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:33         ` [PATCH v2 21/30] hook: convert 'post-rewrite' hook in sequencer.c to hook.h Ævar Arnfjörð Bjarmason
                           ` (12 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

In cases where a hook requires only a small amount of information via
stdin, it should be simple for users to provide a string_list alone. But
in more complicated cases where the stdin is too large to hold in
memory, let's instead provide a callback the users can populate line
after line.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 hook.c | 33 +++++++++++++++++++++++++++++++--
 hook.h | 27 +++++++++++++++++++++++++++
 2 files changed, 58 insertions(+), 2 deletions(-)

diff --git a/hook.c b/hook.c
index 5f6335bac3f..ac5e3988fec 100644
--- a/hook.c
+++ b/hook.c
@@ -47,6 +47,29 @@ void run_hooks_opt_clear(struct run_hooks_opt *o)
 	strvec_clear(&o->args);
 }
 
+int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb)
+{
+	int *item_idx;
+	struct hook *ctx = pp_task_cb;
+	struct hook_cb_data *hook_cb = pp_cb;
+	struct string_list *to_pipe = hook_cb->options->feed_pipe_ctx;
+
+	/* Bootstrap the state manager if necessary. */
+	if (!ctx->feed_pipe_cb_data) {
+		ctx->feed_pipe_cb_data = xmalloc(sizeof(unsigned int));
+		*(int*)ctx->feed_pipe_cb_data = 0;
+	}
+
+	item_idx = ctx->feed_pipe_cb_data;
+
+	if (*item_idx < to_pipe->nr) {
+		strbuf_addf(pipe, "%s\n", to_pipe->items[*item_idx].string);
+		(*item_idx)++;
+		return 0;
+	}
+	return 1;
+}
+
 static int pick_next_hook(struct child_process *cp,
 			  struct strbuf *out,
 			  void *pp_cb,
@@ -62,6 +85,10 @@ static int pick_next_hook(struct child_process *cp,
 	if (hook_cb->options->path_to_stdin) {
 		cp->no_stdin = 0;
 		cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY);
+	} else if (hook_cb->options->feed_pipe) {
+		/* ask for start_command() to make a pipe for us */
+		cp->in = -1;
+		cp->no_stdin = 0;
 	} else {
 		cp->no_stdin = 1;
 	}
@@ -114,7 +141,6 @@ static int notify_hook_finished(int result,
 	return 1;
 }
 
-
 int run_found_hooks(const char *hook_name, const char *hook_path,
 		    struct run_hooks_opt *options)
 {
@@ -139,7 +165,7 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 	run_processes_parallel_tr2(options->jobs,
 				   pick_next_hook,
 				   notify_start_failure,
-				   NULL,
+				   options->feed_pipe,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
@@ -157,6 +183,9 @@ int run_hooks(const char *hook_name, struct run_hooks_opt *options)
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
 
+	if (options->path_to_stdin && options->feed_pipe)
+		BUG("choose only one method to populate stdin");
+
 	hook_path = find_hook(hook_name);
 
 	/* Care about nonexistence? Use run_found_hooks() */
diff --git a/hook.h b/hook.h
index 74a8c76a94c..ff1697d1087 100644
--- a/hook.h
+++ b/hook.h
@@ -7,6 +7,12 @@
 struct hook {
 	/* The path to the hook */
 	const char *hook_path;
+
+	/*
+	 * Use this to keep state for your feed_pipe_fn if you are using
+	 * run_hooks_opt.feed_pipe. Otherwise, do not touch it.
+	 */
+	void *feed_pipe_cb_data;
 };
 
 struct run_hooks_opt
@@ -30,6 +36,19 @@ struct run_hooks_opt
 
 	/* Path to file which should be piped to stdin for each hook */
 	const char *path_to_stdin;
+
+	/*
+	 * Callback and state pointer to ask for more content to pipe to stdin.
+	 * Will be called repeatedly, for each hook. See
+	 * hook.c:pipe_from_stdin() for an example. Keep per-hook state in
+	 * hook.feed_pipe_cb_data (per process). Keep initialization context in
+	 * feed_pipe_ctx (shared by all processes).
+	 *
+	 * See 'pipe_from_string_list()' for info about how to specify a
+	 * string_list as the stdin input instead of writing your own handler.
+	 */
+	feed_pipe_fn feed_pipe;
+	void *feed_pipe_ctx;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
@@ -38,6 +57,14 @@ struct run_hooks_opt
 	.args = STRVEC_INIT, \
 }
 
+/*
+ * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
+ * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'.
+ * This will pipe each string in the list to stdin, separated by newlines.  (Do
+ * not inject your own newlines.)
+ */
+int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb);
+
 /*
  * Callback provided to feed_pipe_fn and consume_sideband_fn.
  */
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 21/30] hook: convert 'post-rewrite' hook in sequencer.c to hook.h
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (19 preceding siblings ...)
  2021-06-14 10:33         ` [PATCH v2 20/30] hook: provide stdin by string_list or callback Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:33         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:33         ` [PATCH v2 22/30] transport: convert pre-push hook to use config Ævar Arnfjörð Bjarmason
                           ` (11 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

By using 'hook.h' for 'post-rewrite', we simplify hook invocations by
not needing to put together our own 'struct child_process'.

The signal handling that's being removed by this commit now takes
place in run-command.h:run_processes_parallel(), so it is OK to remove
them here.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 sequencer.c | 81 ++++++++++++++++++++++-------------------------------
 1 file changed, 34 insertions(+), 47 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 8f46984ffb7..ec2761e47d9 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -35,6 +35,7 @@
 #include "commit-reach.h"
 #include "rebase-interactive.h"
 #include "reset.h"
+#include "string-list.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -1147,33 +1148,28 @@ int update_head_with_reflog(const struct commit *old_head,
 static int run_rewrite_hook(const struct object_id *oldoid,
 			    const struct object_id *newoid)
 {
-	struct child_process proc = CHILD_PROCESS_INIT;
-	const char *argv[3];
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct strbuf tmp = STRBUF_INIT;
+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int code;
-	struct strbuf sb = STRBUF_INIT;
 
-	argv[0] = find_hook("post-rewrite");
-	if (!argv[0])
-		return 0;
+	strvec_push(&opt.args, "amend");
 
-	argv[1] = "amend";
-	argv[2] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = "post-rewrite";
-
-	code = start_command(&proc);
-	if (code)
-		return code;
-	strbuf_addf(&sb, "%s %s\n", oid_to_hex(oldoid), oid_to_hex(newoid));
-	sigchain_push(SIGPIPE, SIG_IGN);
-	write_in_full(proc.in, sb.buf, sb.len);
-	close(proc.in);
-	strbuf_release(&sb);
-	sigchain_pop(SIGPIPE);
-	return finish_command(&proc);
+	strbuf_addf(&tmp,
+		    "%s %s",
+		    oid_to_hex(oldoid),
+		    oid_to_hex(newoid));
+	string_list_append(&to_stdin, tmp.buf);
+
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
+
+	code = run_hooks("post-rewrite", &opt);
+
+	run_hooks_opt_clear(&opt);
+	strbuf_release(&tmp);
+	string_list_clear(&to_stdin, 0);
+	return code;
 }
 
 void commit_post_rewrite(struct repository *r,
@@ -4527,30 +4523,21 @@ static int pick_commits(struct repository *r,
 		flush_rewritten_pending();
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
-			struct child_process child = CHILD_PROCESS_INIT;
-			const char *post_rewrite_hook =
-				find_hook("post-rewrite");
-
-			child.in = open(rebase_path_rewritten_list(), O_RDONLY);
-			child.git_cmd = 1;
-			strvec_push(&child.args, "notes");
-			strvec_push(&child.args, "copy");
-			strvec_push(&child.args, "--for-rewrite=rebase");
+			struct child_process notes_cp = CHILD_PROCESS_INIT;
+			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+
+			notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY);
+			notes_cp.git_cmd = 1;
+			strvec_push(&notes_cp.args, "notes");
+			strvec_push(&notes_cp.args, "copy");
+			strvec_push(&notes_cp.args, "--for-rewrite=rebase");
 			/* we don't care if this copying failed */
-			run_command(&child);
-
-			if (post_rewrite_hook) {
-				struct child_process hook = CHILD_PROCESS_INIT;
-
-				hook.in = open(rebase_path_rewritten_list(),
-					O_RDONLY);
-				hook.stdout_to_stderr = 1;
-				hook.trace2_hook_name = "post-rewrite";
-				strvec_push(&hook.args, post_rewrite_hook);
-				strvec_push(&hook.args, "rebase");
-				/* we don't care if this hook failed */
-				run_command(&hook);
-			}
+			run_command(&notes_cp);
+
+			hook_opt.path_to_stdin = rebase_path_rewritten_list();
+			strvec_push(&hook_opt.args, "rebase");
+			run_hooks("post-rewrite", &hook_opt);
+			run_hooks_opt_clear(&hook_opt);
 		}
 		apply_autostash(rebase_path_autostash());
 
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 22/30] transport: convert pre-push hook to use config
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (20 preceding siblings ...)
  2021-06-14 10:33         ` [PATCH v2 21/30] hook: convert 'post-rewrite' hook in sequencer.c to hook.h Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:33         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:33         ` [PATCH v2 23/30] reference-transaction: use hook.h to run hooks Ævar Arnfjörð Bjarmason
                           ` (10 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

By using the hook.h:run_hooks API, pre-push hooks can be specified in
the config as well as in the hookdir.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 transport.c | 57 ++++++++++++++---------------------------------------
 1 file changed, 15 insertions(+), 42 deletions(-)

diff --git a/transport.c b/transport.c
index 2ed270171f0..9969ed2cdde 100644
--- a/transport.c
+++ b/transport.c
@@ -1199,31 +1199,14 @@ static void die_with_unpushed_submodules(struct string_list *needs_pushing)
 static int run_pre_push_hook(struct transport *transport,
 			     struct ref *remote_refs)
 {
-	int ret = 0, x;
+	int ret = 0;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct strbuf tmp = STRBUF_INIT;
 	struct ref *r;
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct strbuf buf;
-	const char *argv[4];
-
-	if (!(argv[0] = find_hook("pre-push")))
-		return 0;
-
-	argv[1] = transport->remote->name;
-	argv[2] = transport->url;
-	argv[3] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.trace2_hook_name = "pre-push";
-
-	if (start_command(&proc)) {
-		finish_command(&proc);
-		return -1;
-	}
-
-	sigchain_push(SIGPIPE, SIG_IGN);
+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 
-	strbuf_init(&buf, 256);
+	strvec_push(&opt.args, transport->remote->name);
+	strvec_push(&opt.args, transport->url);
 
 	for (r = remote_refs; r; r = r->next) {
 		if (!r->peer_ref) continue;
@@ -1232,30 +1215,20 @@ static int run_pre_push_hook(struct transport *transport,
 		if (r->status == REF_STATUS_REJECT_REMOTE_UPDATED) continue;
 		if (r->status == REF_STATUS_UPTODATE) continue;
 
-		strbuf_reset(&buf);
-		strbuf_addf( &buf, "%s %s %s %s\n",
+		strbuf_reset(&tmp);
+		strbuf_addf(&tmp, "%s %s %s %s",
 			 r->peer_ref->name, oid_to_hex(&r->new_oid),
 			 r->name, oid_to_hex(&r->old_oid));
-
-		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
-			/* We do not mind if a hook does not read all refs. */
-			if (errno != EPIPE)
-				ret = -1;
-			break;
-		}
+		string_list_append(&to_stdin, tmp.buf);
 	}
 
-	strbuf_release(&buf);
-
-	x = close(proc.in);
-	if (!ret)
-		ret = x;
-
-	sigchain_pop(SIGPIPE);
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
 
-	x = finish_command(&proc);
-	if (!ret)
-		ret = x;
+	ret = run_hooks("pre-push", &opt);
+	run_hooks_opt_clear(&opt);
+	strbuf_release(&tmp);
+	string_list_clear(&to_stdin, 0);
 
 	return ret;
 }
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 23/30] reference-transaction: use hook.h to run hooks
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (21 preceding siblings ...)
  2021-06-14 10:33         ` [PATCH v2 22/30] transport: convert pre-push hook to use config Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:33         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:33         ` [PATCH v2 24/30] run-command: allow capturing of collated output Ævar Arnfjörð Bjarmason
                           ` (9 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c | 42 +++++++++++++++---------------------------
 1 file changed, 15 insertions(+), 27 deletions(-)

diff --git a/refs.c b/refs.c
index 59be29cf081..1149e7e7dcb 100644
--- a/refs.c
+++ b/refs.c
@@ -2062,47 +2062,35 @@ int ref_update_reject_duplicates(struct string_list *refnames,
 static int run_transaction_hook(struct ref_transaction *transaction,
 				const char *state)
 {
-	struct child_process proc = CHILD_PROCESS_INIT;
 	struct strbuf buf = STRBUF_INIT;
-	const char *hook;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int ret = 0, i;
+	char o[GIT_MAX_HEXSZ + 1], n[GIT_MAX_HEXSZ + 1];
 
-	hook = find_hook("reference-transaction");
-	if (!hook)
+	if (!hook_exists("reference-transaction"))
 		return ret;
 
-	strvec_pushl(&proc.args, hook, state, NULL);
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = "reference-transaction";
-
-	ret = start_command(&proc);
-	if (ret)
-		return ret;
-
-	sigchain_push(SIGPIPE, SIG_IGN);
+	strvec_push(&opt.args, state);
 
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		oid_to_hex_r(o, &update->old_oid);
+		oid_to_hex_r(n, &update->new_oid);
 
 		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
-			    oid_to_hex(&update->old_oid),
-			    oid_to_hex(&update->new_oid),
-			    update->refname);
-
-		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
-			if (errno != EPIPE)
-				ret = -1;
-			break;
-		}
+		strbuf_addf(&buf, "%s %s %s", o, n, update->refname);
+		string_list_append(&to_stdin, buf.buf);
 	}
 
-	close(proc.in);
-	sigchain_pop(SIGPIPE);
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
+
+	ret = run_hooks("reference-transaction", &opt);
+	run_hooks_opt_clear(&opt);
 	strbuf_release(&buf);
+	string_list_clear(&to_stdin, 0);
 
-	ret |= finish_command(&proc);
 	return ret;
 }
 
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 24/30] run-command: allow capturing of collated output
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (22 preceding siblings ...)
  2021-06-14 10:33         ` [PATCH v2 23/30] reference-transaction: use hook.h to run hooks Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:33         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:33         ` [PATCH v2 25/30] hooks: allow callers to capture output Ævar Arnfjörð Bjarmason
                           ` (8 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Some callers, for example server-side hooks which wish to relay hook
output to clients across a transport, want to capture what would
normally print to stderr and do something else with it. Allow that via a
callback.

By calling the callback regardless of whether there's output available,
we allow clients to send e.g. a keepalive if necessary.

Because we expose a strbuf, not a fd or FILE*, there's no need to create
a temporary pipe or similar - we can just skip the print to stderr and
instead hand it to the caller.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/fetch.c             |  2 +-
 builtin/submodule--helper.c |  2 +-
 hook.c                      |  1 +
 run-command.c               | 33 +++++++++++++++++++++++++--------
 run-command.h               | 18 +++++++++++++++++-
 submodule.c                 |  2 +-
 t/helper/test-run-command.c | 25 ++++++++++++++++++++-----
 t/t0061-run-command.sh      |  7 +++++++
 8 files changed, 73 insertions(+), 17 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index 3d8f04b392d..0595b88b7cc 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1817,7 +1817,7 @@ static int fetch_multiple(struct string_list *list, int max_children)
 		result = run_processes_parallel_tr2(max_children,
 						    &fetch_next_remote,
 						    &fetch_failed_to_start,
-						    NULL,
+						    NULL, NULL,
 						    &fetch_finished,
 						    &state,
 						    "fetch", "parallel/fetch");
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 818494dd18e..69782f62044 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2295,7 +2295,7 @@ static int update_submodules(struct submodule_update_clone *suc)
 	int i;
 
 	run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task,
-				   update_clone_start_failure, NULL,
+				   update_clone_start_failure, NULL, NULL,
 				   update_clone_task_finished, suc, "submodule",
 				   "parallel/update");
 
diff --git a/hook.c b/hook.c
index ac5e3988fec..0faa24ec825 100644
--- a/hook.c
+++ b/hook.c
@@ -166,6 +166,7 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
+				   NULL,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
diff --git a/run-command.c b/run-command.c
index 3392640f17b..4a1a7a10820 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1494,6 +1494,7 @@ struct parallel_processes {
 	get_next_task_fn get_next_task;
 	start_failure_fn start_failure;
 	feed_pipe_fn feed_pipe;
+	consume_sideband_fn consume_sideband;
 	task_finished_fn task_finished;
 
 	struct {
@@ -1559,6 +1560,7 @@ static void pp_init(struct parallel_processes *pp,
 		    get_next_task_fn get_next_task,
 		    start_failure_fn start_failure,
 		    feed_pipe_fn feed_pipe,
+		    consume_sideband_fn consume_sideband,
 		    task_finished_fn task_finished,
 		    void *data)
 {
@@ -1579,6 +1581,7 @@ static void pp_init(struct parallel_processes *pp,
 	pp->start_failure = start_failure ? start_failure : default_start_failure;
 	pp->feed_pipe = feed_pipe ? feed_pipe : default_feed_pipe;
 	pp->task_finished = task_finished ? task_finished : default_task_finished;
+	pp->consume_sideband = consume_sideband;
 
 	pp->nr_processes = 0;
 	pp->output_owner = 0;
@@ -1615,7 +1618,10 @@ static void pp_cleanup(struct parallel_processes *pp)
 	 * When get_next_task added messages to the buffer in its last
 	 * iteration, the buffered output is non empty.
 	 */
-	strbuf_write(&pp->buffered_output, stderr);
+	if (pp->consume_sideband)
+		pp->consume_sideband(&pp->buffered_output, pp->data);
+	else
+		strbuf_write(&pp->buffered_output, stderr);
 	strbuf_release(&pp->buffered_output);
 
 	sigchain_pop_common();
@@ -1736,9 +1742,13 @@ static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout)
 static void pp_output(struct parallel_processes *pp)
 {
 	int i = pp->output_owner;
+
 	if (pp->children[i].state == GIT_CP_WORKING &&
 	    pp->children[i].err.len) {
-		strbuf_write(&pp->children[i].err, stderr);
+		if (pp->consume_sideband)
+			pp->consume_sideband(&pp->children[i].err, pp->data);
+		else
+			strbuf_write(&pp->children[i].err, stderr);
 		strbuf_reset(&pp->children[i].err);
 	}
 }
@@ -1777,11 +1787,15 @@ static int pp_collect_finished(struct parallel_processes *pp)
 			strbuf_addbuf(&pp->buffered_output, &pp->children[i].err);
 			strbuf_reset(&pp->children[i].err);
 		} else {
-			strbuf_write(&pp->children[i].err, stderr);
+			/* Output errors, then all other finished child processes */
+			if (pp->consume_sideband) {
+				pp->consume_sideband(&pp->children[i].err, pp->data);
+				pp->consume_sideband(&pp->buffered_output, pp->data);
+			} else {
+				strbuf_write(&pp->children[i].err, stderr);
+				strbuf_write(&pp->buffered_output, stderr);
+			}
 			strbuf_reset(&pp->children[i].err);
-
-			/* Output all other finished child processes */
-			strbuf_write(&pp->buffered_output, stderr);
 			strbuf_reset(&pp->buffered_output);
 
 			/*
@@ -1805,6 +1819,7 @@ int run_processes_parallel(int n,
 			   get_next_task_fn get_next_task,
 			   start_failure_fn start_failure,
 			   feed_pipe_fn feed_pipe,
+			   consume_sideband_fn consume_sideband,
 			   task_finished_fn task_finished,
 			   void *pp_cb)
 {
@@ -1815,7 +1830,7 @@ int run_processes_parallel(int n,
 
 	sigchain_push(SIGPIPE, SIG_IGN);
 
-	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, task_finished, pp_cb);
+	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, consume_sideband, task_finished, pp_cb);
 	while (1) {
 		for (i = 0;
 		    i < spawn_cap && !pp.shutdown &&
@@ -1853,6 +1868,7 @@ int run_processes_parallel(int n,
 int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 			       start_failure_fn start_failure,
 			       feed_pipe_fn feed_pipe,
+			       consume_sideband_fn consume_sideband,
 			       task_finished_fn task_finished, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label)
 {
@@ -1862,7 +1878,8 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 				   ((n < 1) ? online_cpus() : n));
 
 	result = run_processes_parallel(n, get_next_task, start_failure,
-					feed_pipe, task_finished, pp_cb);
+					feed_pipe, consume_sideband,
+					task_finished, pp_cb);
 
 	trace2_region_leave(tr2_category, tr2_label, NULL);
 
diff --git a/run-command.h b/run-command.h
index 41e36d26cb1..7150da851a8 100644
--- a/run-command.h
+++ b/run-command.h
@@ -433,6 +433,20 @@ typedef int (*feed_pipe_fn)(struct strbuf *pipe,
 			    void *pp_cb,
 			    void *pp_task_cb);
 
+/**
+ * If this callback is provided, instead of collating process output to stderr,
+ * they will be collated into a new pipe. consume_sideband_fn will be called
+ * repeatedly. When output is available on that pipe, it will be contained in
+ * 'output'. But it will be called with an empty 'output' too, to allow for
+ * keepalives or similar operations if necessary.
+ *
+ * pp_cb is the callback cookie as passed into run_processes_parallel.
+ *
+ * Since this callback is provided with the collated output, no task cookie is
+ * provided.
+ */
+typedef void (*consume_sideband_fn)(struct strbuf *output, void *pp_cb);
+
 /**
  * This callback is called on every child process that finished processing.
  *
@@ -468,10 +482,12 @@ int run_processes_parallel(int n,
 			   get_next_task_fn,
 			   start_failure_fn,
 			   feed_pipe_fn,
+			   consume_sideband_fn,
 			   task_finished_fn,
 			   void *pp_cb);
 int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn,
-			       feed_pipe_fn, task_finished_fn, void *pp_cb,
+			       feed_pipe_fn, consume_sideband_fn,
+			       task_finished_fn, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label);
 
 #endif
diff --git a/submodule.c b/submodule.c
index ea026a8195f..7fe0c8f7c9f 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1645,7 +1645,7 @@ int fetch_populated_submodules(struct repository *r,
 	run_processes_parallel_tr2(max_parallel_jobs,
 				   get_next_submodule,
 				   fetch_start_failure,
-				   NULL,
+				   NULL, NULL,
 				   fetch_finish,
 				   &spf,
 				   "submodule", "parallel/fetch");
diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c
index 9348184d303..d53db6d11c4 100644
--- a/t/helper/test-run-command.c
+++ b/t/helper/test-run-command.c
@@ -51,6 +51,16 @@ static int no_job(struct child_process *cp,
 	return 0;
 }
 
+static void test_consume_sideband(struct strbuf *output, void *cb)
+{
+	FILE *sideband;
+
+	sideband = fopen("./sideband", "a");
+
+	strbuf_write(output, sideband);
+	fclose(sideband);
+}
+
 static int task_finished(int result,
 			 struct strbuf *err,
 			 void *pp_cb,
@@ -201,7 +211,7 @@ static int testsuite(int argc, const char **argv)
 		suite.tests.nr, max_jobs);
 
 	ret = run_processes_parallel(max_jobs, next_test, test_failed,
-				     test_stdin, test_finished, &suite);
+				     test_stdin, NULL, test_finished, &suite);
 
 	if (suite.failed.nr > 0) {
 		ret = 1;
@@ -429,23 +439,28 @@ int cmd__run_command(int argc, const char **argv)
 
 	if (!strcmp(argv[1], "run-command-parallel"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, NULL, &proc));
+					    NULL, NULL, NULL, NULL, &proc));
 
 	if (!strcmp(argv[1], "run-command-abort"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, task_finished, &proc));
+					    NULL, NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-no-jobs"))
 		exit(run_processes_parallel(jobs, no_job,
-					    NULL, NULL, task_finished, &proc));
+					    NULL, NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-stdin")) {
 		proc.in = -1;
 		proc.no_stdin = 0;
 		exit (run_processes_parallel(jobs, parallel_next, NULL,
-					     test_stdin, NULL, &proc));
+					     test_stdin, NULL, NULL, &proc));
 	}
 
+	if (!strcmp(argv[1], "run-command-sideband"))
+		exit(run_processes_parallel(jobs, parallel_next, NULL, NULL,
+					    test_consume_sideband, NULL,
+					    &proc));
+
 	fprintf(stderr, "check usage\n");
 	return 1;
 }
diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh
index 87759482ad1..e99f6c7f445 100755
--- a/t/t0061-run-command.sh
+++ b/t/t0061-run-command.sh
@@ -143,6 +143,13 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai
 	test_cmp expect actual
 '
 
+test_expect_success 'run_command can divert output' '
+	test_when_finished rm sideband &&
+	test-tool run-command run-command-sideband 3 sh -c "printf \"%s\n%s\n\" Hello World" 2>actual &&
+	test_must_be_empty actual &&
+	test_cmp expect sideband
+'
+
 cat >expect <<-EOF
 preloaded output of a child
 listening for stdin:
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 25/30] hooks: allow callers to capture output
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (23 preceding siblings ...)
  2021-06-14 10:33         ` [PATCH v2 24/30] run-command: allow capturing of collated output Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:33         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:33         ` [PATCH v2 26/30] receive-pack: convert 'update' hook to hook.h Ævar Arnfjörð Bjarmason
                           ` (7 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Some server-side hooks will require capturing output to send over
sideband instead of printing directly to stderr. Expose that capability.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 hook.c | 2 +-
 hook.h | 8 ++++++++
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/hook.c b/hook.c
index 0faa24ec825..17ae65eca31 100644
--- a/hook.c
+++ b/hook.c
@@ -166,7 +166,7 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
-				   NULL,
+				   options->consume_sideband,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
diff --git a/hook.h b/hook.h
index ff1697d1087..5f895032341 100644
--- a/hook.h
+++ b/hook.h
@@ -49,6 +49,14 @@ struct run_hooks_opt
 	 */
 	feed_pipe_fn feed_pipe;
 	void *feed_pipe_ctx;
+
+	/*
+	 * Populate this to capture output and prevent it from being printed to
+	 * stderr. This will be passed directly through to
+	 * run_command:run_parallel_processes(). See t/helper/test-run-command.c
+	 * for an example.
+	 */
+	consume_sideband_fn consume_sideband;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 26/30] receive-pack: convert 'update' hook to hook.h
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (24 preceding siblings ...)
  2021-06-14 10:33         ` [PATCH v2 25/30] hooks: allow callers to capture output Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:33         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:33         ` [PATCH v2 27/30] post-update: use hook.h library Ævar Arnfjörð Bjarmason
                           ` (6 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

This makes use of the new sideband API in hook.h added in the
preceding commit.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/receive-pack.c | 65 ++++++++++++++++++++++++++++--------------
 1 file changed, 44 insertions(+), 21 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 5248228ebfe..378f8f6b5d1 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -938,33 +938,56 @@ static int run_receive_hook(struct command *commands,
 	return status;
 }
 
-static int run_update_hook(struct command *cmd)
+static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
 {
-	const char *argv[5];
-	struct child_process proc = CHILD_PROCESS_INIT;
-	int code;
+	int keepalive_active = 0;
 
-	argv[0] = find_hook("update");
-	if (!argv[0])
-		return 0;
+	if (keepalive_in_sec <= 0)
+		use_keepalive = KEEPALIVE_NEVER;
+	if (use_keepalive == KEEPALIVE_ALWAYS)
+		keepalive_active = 1;
 
-	argv[1] = cmd->ref_name;
-	argv[2] = oid_to_hex(&cmd->old_oid);
-	argv[3] = oid_to_hex(&cmd->new_oid);
-	argv[4] = NULL;
+	/* send a keepalive if there is no data to write */
+	if (keepalive_active && !output->len) {
+		static const char buf[] = "0005\1";
+		write_or_die(1, buf, sizeof(buf) - 1);
+		return;
+	}
 
-	proc.no_stdin = 1;
-	proc.stdout_to_stderr = 1;
-	proc.err = use_sideband ? -1 : 0;
-	proc.argv = argv;
-	proc.trace2_hook_name = "update";
+	if (use_keepalive == KEEPALIVE_AFTER_NUL && !keepalive_active) {
+		const char *first_null = memchr(output->buf, '\0', output->len);
+		if (first_null) {
+			/* The null bit is excluded. */
+			size_t before_null = first_null - output->buf;
+			size_t after_null = output->len - (before_null + 1);
+			keepalive_active = 1;
+			send_sideband(1, 2, output->buf, before_null, use_sideband);
+			send_sideband(1, 2, first_null + 1, after_null, use_sideband);
+
+			return;
+		}
+	}
+
+	send_sideband(1, 2, output->buf, output->len, use_sideband);
+}
+
+static int run_update_hook(struct command *cmd)
+{
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	int code;
+
+	strvec_pushl(&opt.args,
+		     cmd->ref_name,
+		     oid_to_hex(&cmd->old_oid),
+		     oid_to_hex(&cmd->new_oid),
+		     NULL);
 
-	code = start_command(&proc);
-	if (code)
-		return code;
 	if (use_sideband)
-		copy_to_sideband(proc.err, -1, NULL);
-	return finish_command(&proc);
+		opt.consume_sideband = hook_output_to_sideband;
+
+	code = run_hooks("update", &opt);
+	run_hooks_opt_clear(&opt);
+	return code;
 }
 
 static struct command *find_command_by_refname(struct command *list,
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 27/30] post-update: use hook.h library
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (25 preceding siblings ...)
  2021-06-14 10:33         ` [PATCH v2 26/30] receive-pack: convert 'update' hook to hook.h Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:33         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:33         ` [PATCH v2 28/30] receive-pack: convert receive hooks to hook.h Ævar Arnfjörð Bjarmason
                           ` (5 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/receive-pack.c | 26 +++++++-------------------
 1 file changed, 7 insertions(+), 19 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 378f8f6b5d1..b2ccdb66daa 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1657,33 +1657,21 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 static void run_update_post_hook(struct command *commands)
 {
 	struct command *cmd;
-	struct child_process proc = CHILD_PROCESS_INIT;
-	const char *hook;
-
-	hook = find_hook("post-update");
-	if (!hook)
-		return;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 
 	for (cmd = commands; cmd; cmd = cmd->next) {
 		if (cmd->error_string || cmd->did_not_exist)
 			continue;
-		if (!proc.args.nr)
-			strvec_push(&proc.args, hook);
-		strvec_push(&proc.args, cmd->ref_name);
+		strvec_push(&opt.args, cmd->ref_name);
 	}
-	if (!proc.args.nr)
+	if (!opt.args.nr)
 		return;
 
-	proc.no_stdin = 1;
-	proc.stdout_to_stderr = 1;
-	proc.err = use_sideband ? -1 : 0;
-	proc.trace2_hook_name = "post-update";
+	if (use_sideband)
+		opt.consume_sideband = hook_output_to_sideband;
 
-	if (!start_command(&proc)) {
-		if (use_sideband)
-			copy_to_sideband(proc.err, -1, NULL);
-		finish_command(&proc);
-	}
+	run_hooks("post-update", &opt);
+	run_hooks_opt_clear(&opt);
 }
 
 static void check_aliased_update_internal(struct command *cmd,
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 28/30] receive-pack: convert receive hooks to hook.h
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (26 preceding siblings ...)
  2021-06-14 10:33         ` [PATCH v2 27/30] post-update: use hook.h library Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:33         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:33         ` [PATCH v2 29/30] hooks: fix a TOCTOU in "did we run a hook?" heuristic Ævar Arnfjörð Bjarmason
                           ` (4 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/receive-pack.c | 197 +++++++++++++++++++----------------------
 1 file changed, 90 insertions(+), 107 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index b2ccdb66daa..ec90e10477a 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -748,7 +748,7 @@ static int check_cert_push_options(const struct string_list *push_options)
 	return retval;
 }
 
-static void prepare_push_cert_sha1(struct child_process *proc)
+static void prepare_push_cert_sha1(struct run_hooks_opt *opt)
 {
 	static int already_done;
 
@@ -772,110 +772,42 @@ static void prepare_push_cert_sha1(struct child_process *proc)
 		nonce_status = check_nonce(push_cert.buf, bogs);
 	}
 	if (!is_null_oid(&push_cert_oid)) {
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT=%s",
 			     oid_to_hex(&push_cert_oid));
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_SIGNER=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_SIGNER=%s",
 			     sigcheck.signer ? sigcheck.signer : "");
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_KEY=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_KEY=%s",
 			     sigcheck.key ? sigcheck.key : "");
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_STATUS=%c",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_STATUS=%c",
 			     sigcheck.result);
 		if (push_cert_nonce) {
-			strvec_pushf(&proc->env_array,
+			strvec_pushf(&opt->env,
 				     "GIT_PUSH_CERT_NONCE=%s",
 				     push_cert_nonce);
-			strvec_pushf(&proc->env_array,
+			strvec_pushf(&opt->env,
 				     "GIT_PUSH_CERT_NONCE_STATUS=%s",
 				     nonce_status);
 			if (nonce_status == NONCE_SLOP)
-				strvec_pushf(&proc->env_array,
+				strvec_pushf(&opt->env,
 					     "GIT_PUSH_CERT_NONCE_SLOP=%ld",
 					     nonce_stamp_slop);
 		}
 	}
 }
 
+struct receive_hook_feed_context {
+	struct command *cmd;
+	int skip_broken;
+};
+
 struct receive_hook_feed_state {
 	struct command *cmd;
 	struct ref_push_report *report;
 	int skip_broken;
 	struct strbuf buf;
-	const struct string_list *push_options;
 };
 
-typedef int (*feed_fn)(void *, const char **, size_t *);
-static int run_and_feed_hook(const char *hook_name, feed_fn feed,
-			     struct receive_hook_feed_state *feed_state)
-{
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct async muxer;
-	const char *argv[2];
-	int code;
-
-	argv[0] = find_hook(hook_name);
-	if (!argv[0])
-		return 0;
-
-	argv[1] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = hook_name;
-
-	if (feed_state->push_options) {
-		int i;
-		for (i = 0; i < feed_state->push_options->nr; i++)
-			strvec_pushf(&proc.env_array,
-				     "GIT_PUSH_OPTION_%d=%s", i,
-				     feed_state->push_options->items[i].string);
-		strvec_pushf(&proc.env_array, "GIT_PUSH_OPTION_COUNT=%d",
-			     feed_state->push_options->nr);
-	} else
-		strvec_pushf(&proc.env_array, "GIT_PUSH_OPTION_COUNT");
-
-	if (tmp_objdir)
-		strvec_pushv(&proc.env_array, tmp_objdir_env(tmp_objdir));
-
-	if (use_sideband) {
-		memset(&muxer, 0, sizeof(muxer));
-		muxer.proc = copy_to_sideband;
-		muxer.in = -1;
-		code = start_async(&muxer);
-		if (code)
-			return code;
-		proc.err = muxer.in;
-	}
-
-	prepare_push_cert_sha1(&proc);
-
-	code = start_command(&proc);
-	if (code) {
-		if (use_sideband)
-			finish_async(&muxer);
-		return code;
-	}
-
-	sigchain_push(SIGPIPE, SIG_IGN);
-
-	while (1) {
-		const char *buf;
-		size_t n;
-		if (feed(feed_state, &buf, &n))
-			break;
-		if (write_in_full(proc.in, buf, n) < 0)
-			break;
-	}
-	close(proc.in);
-	if (use_sideband)
-		finish_async(&muxer);
-
-	sigchain_pop(SIGPIPE);
-
-	return finish_command(&proc);
-}
-
-static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
+static int feed_receive_hook(void *state_)
 {
 	struct receive_hook_feed_state *state = state_;
 	struct command *cmd = state->cmd;
@@ -884,9 +816,7 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
 	       state->skip_broken && (cmd->error_string || cmd->did_not_exist))
 		cmd = cmd->next;
 	if (!cmd)
-		return -1; /* EOF */
-	if (!bufp)
-		return 0; /* OK, can feed something. */
+		return 1; /* EOF - close the pipe*/
 	strbuf_reset(&state->buf);
 	if (!state->report)
 		state->report = cmd->report;
@@ -910,32 +840,36 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
 			    cmd->ref_name);
 		state->cmd = cmd->next;
 	}
-	if (bufp) {
-		*bufp = state->buf.buf;
-		*sizep = state->buf.len;
-	}
 	return 0;
 }
 
-static int run_receive_hook(struct command *commands,
-			    const char *hook_name,
-			    int skip_broken,
-			    const struct string_list *push_options)
+static int feed_receive_hook_cb(struct strbuf *pipe, void *pp_cb, void *pp_task_cb)
 {
-	struct receive_hook_feed_state state;
-	int status;
-
-	strbuf_init(&state.buf, 0);
-	state.cmd = commands;
-	state.skip_broken = skip_broken;
-	state.report = NULL;
-	if (feed_receive_hook(&state, NULL, NULL))
-		return 0;
-	state.cmd = commands;
-	state.push_options = push_options;
-	status = run_and_feed_hook(hook_name, feed_receive_hook, &state);
-	strbuf_release(&state.buf);
-	return status;
+	struct hook *hook = pp_task_cb;
+	struct receive_hook_feed_state *feed_state = hook->feed_pipe_cb_data;
+	int rc;
+
+	/* first-time setup */
+	if (!feed_state) {
+		struct hook_cb_data *hook_cb = pp_cb;
+		struct run_hooks_opt *opt = hook_cb->options;
+		struct receive_hook_feed_context *ctx = opt->feed_pipe_ctx;
+		if (!ctx)
+			BUG("run_hooks_opt.feed_pipe_ctx required for receive hook");
+
+		feed_state = xmalloc(sizeof(struct receive_hook_feed_state));
+		strbuf_init(&feed_state->buf, 0);
+		feed_state->cmd = ctx->cmd;
+		feed_state->skip_broken = ctx->skip_broken;
+		feed_state->report = NULL;
+
+		hook->feed_pipe_cb_data = feed_state;
+	}
+
+	rc = feed_receive_hook(feed_state);
+	if (!rc)
+		strbuf_addbuf(pipe, &feed_state->buf);
+	return rc;
 }
 
 static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
@@ -971,6 +905,55 @@ static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
 	send_sideband(1, 2, output->buf, output->len, use_sideband);
 }
 
+static int run_receive_hook(struct command *commands,
+			    const char *hook_name,
+			    int skip_broken,
+			    const struct string_list *push_options)
+{
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct receive_hook_feed_context ctx;
+	int rc;
+	struct command *iter = commands;
+
+	/* if there are no valid commands, don't invoke the hook at all. */
+	while (iter && skip_broken && (iter->error_string || iter->did_not_exist))
+		iter = iter->next;
+	if (!iter)
+		return 0;
+
+	/* pre-receive hooks should run in series as the hook updates refs */
+	if (!strcmp(hook_name, "pre-receive"))
+		opt.jobs = 1;
+
+	if (push_options) {
+		int i;
+		for (i = 0; i < push_options->nr; i++)
+			strvec_pushf(&opt.env, "GIT_PUSH_OPTION_%d=%s", i,
+				     push_options->items[i].string);
+		strvec_pushf(&opt.env, "GIT_PUSH_OPTION_COUNT=%d", push_options->nr);
+	} else
+		strvec_push(&opt.env, "GIT_PUSH_OPTION_COUNT");
+
+	if (tmp_objdir)
+		strvec_pushv(&opt.env, tmp_objdir_env(tmp_objdir));
+
+	prepare_push_cert_sha1(&opt);
+
+	/* set up sideband printer */
+	if (use_sideband)
+		opt.consume_sideband = hook_output_to_sideband;
+
+	/* set up stdin callback */
+	ctx.cmd = commands;
+	ctx.skip_broken = skip_broken;
+	opt.feed_pipe = feed_receive_hook_cb;
+	opt.feed_pipe_ctx = &ctx;
+
+	rc = run_hooks(hook_name, &opt);
+	run_hooks_opt_clear(&opt);
+	return rc;
+}
+
 static int run_update_hook(struct command *cmd)
 {
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 29/30] hooks: fix a TOCTOU in "did we run a hook?" heuristic
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (27 preceding siblings ...)
  2021-06-14 10:33         ` [PATCH v2 28/30] receive-pack: convert receive hooks to hook.h Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:33         ` Ævar Arnfjörð Bjarmason
  2021-06-14 10:33         ` [PATCH v2 30/30] hook-list.h: add a generated list of hooks, like config-list.h Ævar Arnfjörð Bjarmason
                           ` (3 subsequent siblings)
  32 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Fix a Time-of-check to time-of-use (TOCTOU) race in code added in
680ee550d72 (commit: skip discarding the index if there is no
pre-commit hook, 2017-08-14).

We can fix the race passing around information about whether or not we
ran the hook in question, instead of running hook_exists() after the
fact to check if the hook in question exists. This problem has been
noted on-list when 680ee550d72 was discussed[1], but had not been
fixed.

In addition to fixing this for the pre-commit hook as suggested there
I'm also fixing this for the pre-merge-commit hook. See
6098817fd7f (git-merge: honor pre-merge-commit hook, 2019-08-07) for
the introduction of its previous behavior.

Let's also change this for the push-to-checkout hook. Now instead of
checking if the hook exists and either doing a push to checkout or a
push to deploy we'll always attempt a push to checkout. If the hook
doesn't exist we'll fall back on push to deploy. The same behavior as
before, without the TOCTOU race. See 0855331941b (receive-pack:
support push-to-checkout hook, 2014-12-01) for the introduction of the
previous behavior.

This leaves uses of hook_exists() in two places that matter. The
"reference-transaction" check in refs.c, see 67541597670 (refs:
implement reference transaction hook, 2020-06-19), and the
prepare-commit-msg hook, see 66618a50f9c (sequencer: run
'prepare-commit-msg' hook, 2018-01-24).

In both of those cases we're saving ourselves CPU time by not
preparing data for the hook that we'll then do nothing with if we
don't have the hook. So using this "invoked_hook" pattern doesn't make
sense in those cases.

More importantly, in those cases the worst we'll do is miss that we
"should" run the hook because a new hook appeared, whereas in the
pre-commit and pre-merge-commit cases we'll skip an important
discard_cache() on the bases of our faulty guess.

I do think none of these races really matter in practice. It would be
some one-off issue as a hook was added or removed. I did think it was
stupid that we didn't pass a "did this run?" flag instead of doing
this guessing at a distance though, so now we're not guessing anymore.

1. https://lore.kernel.org/git/20170810191613.kpmhzg4seyxy3cpq@sigill.intra.peff.net/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/commit.c       | 18 +++++++++++-------
 builtin/merge.c        | 16 ++++++++++------
 builtin/receive-pack.c |  8 +++++---
 commit.c               |  1 +
 commit.h               |  3 ++-
 hook.c                 |  4 ++++
 hook.h                 | 10 ++++++++++
 sequencer.c            |  4 ++--
 8 files changed, 45 insertions(+), 19 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index dad4e565443..a66727a612a 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -725,11 +725,13 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	int clean_message_contents = (cleanup_mode != COMMIT_MSG_CLEANUP_NONE);
 	int old_display_comment_prefix;
 	int merge_contains_scissors = 0;
+	int invoked_hook = 0;
 
 	/* This checks and barfs if author is badly specified */
 	determine_author_info(author_ident);
 
-	if (!no_verify && run_commit_hook(use_editor, index_file, "pre-commit", NULL))
+	if (!no_verify && run_commit_hook(use_editor, index_file, &invoked_hook,
+					  "pre-commit", NULL))
 		return 0;
 
 	if (squash_message) {
@@ -1045,10 +1047,10 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (!no_verify && hook_exists("pre-commit")) {
+	if (!no_verify && invoked_hook) {
 		/*
-		 * Re-read the index as pre-commit hook could have updated it,
-		 * and write it out as a tree.  We must do this before we invoke
+		 * Re-read the index as the pre-commit-commit hook was invoked
+		 * and could have updated it. We must do this before we invoke
 		 * the editor and after we invoke run_status above.
 		 */
 		discard_cache();
@@ -1060,7 +1062,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (run_commit_hook(use_editor, index_file, "prepare-commit-msg",
+	if (run_commit_hook(use_editor, index_file, NULL, "prepare-commit-msg",
 			    git_path_commit_editmsg(), hook_arg1, hook_arg2, NULL))
 		return 0;
 
@@ -1077,7 +1079,8 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	}
 
 	if (!no_verify &&
-	    run_commit_hook(use_editor, index_file, "commit-msg", git_path_commit_editmsg(), NULL)) {
+	    run_commit_hook(use_editor, index_file, NULL, "commit-msg",
+			    git_path_commit_editmsg(), NULL)) {
 		return 0;
 	}
 
@@ -1830,7 +1833,8 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 
 	repo_rerere(the_repository, 0);
 	run_auto_maintenance(quiet);
-	run_commit_hook(use_editor, get_index_file(), "post-commit", NULL);
+	run_commit_hook(use_editor, get_index_file(), NULL, "post-commit",
+			NULL);
 	if (amend && !no_post_rewrite) {
 		commit_post_rewrite(the_repository, current_head, &oid);
 	}
diff --git a/builtin/merge.c b/builtin/merge.c
index 6128b60942f..0425c9bf2b5 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -844,15 +844,18 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 {
 	struct strbuf msg = STRBUF_INIT;
 	const char *index_file = get_index_file();
+	int invoked_hook = 0;
 
-	if (!no_verify && run_commit_hook(0 < option_edit, index_file, "pre-merge-commit", NULL))
+	if (!no_verify && run_commit_hook(0 < option_edit, index_file,
+					  &invoked_hook, "pre-merge-commit",
+					  NULL))
 		abort_commit(remoteheads, NULL);
 	/*
-	 * Re-read the index as pre-merge-commit hook could have updated it,
-	 * and write it out as a tree.  We must do this before we invoke
+	 * Re-read the index as the pre-merge-commit hook was invoked
+	 * and could have updated it. We must do this before we invoke
 	 * the editor and after we invoke run_status above.
 	 */
-	if (hook_exists("pre-merge-commit"))
+	if (invoked_hook)
 		discard_cache();
 	read_cache_from(index_file);
 	strbuf_addbuf(&msg, &merge_msg);
@@ -873,7 +876,8 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 		append_signoff(&msg, ignore_non_trailer(msg.buf, msg.len), 0);
 	write_merge_heads(remoteheads);
 	write_file_buf(git_path_merge_msg(the_repository), msg.buf, msg.len);
-	if (run_commit_hook(0 < option_edit, get_index_file(), "prepare-commit-msg",
+	if (run_commit_hook(0 < option_edit, get_index_file(), NULL,
+			    "prepare-commit-msg",
 			    git_path_merge_msg(the_repository), "merge", NULL))
 		abort_commit(remoteheads, NULL);
 	if (0 < option_edit) {
@@ -882,7 +886,7 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 	}
 
 	if (!no_verify && run_commit_hook(0 < option_edit, get_index_file(),
-					  "commit-msg",
+					  NULL, "commit-msg",
 					  git_path_merge_msg(the_repository), NULL))
 		abort_commit(remoteheads, NULL);
 
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index ec90e10477a..cd658f41d58 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1439,10 +1439,12 @@ static const char *push_to_deploy(unsigned char *sha1,
 static const char *push_to_checkout_hook = "push-to-checkout";
 
 static const char *push_to_checkout(unsigned char *hash,
+				    int *invoked_hook,
 				    struct strvec *env,
 				    const char *work_tree)
 {
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	opt.invoked_hook = invoked_hook;
 
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
 	strvec_pushv(&opt.env, env->v);
@@ -1460,6 +1462,7 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 {
 	const char *retval, *work_tree, *git_dir = NULL;
 	struct strvec env = STRVEC_INIT;
+	int invoked_hook = 0;
 
 	if (worktree && worktree->path)
 		work_tree = worktree->path;
@@ -1477,10 +1480,9 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 
 	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
 
-	if (!hook_exists(push_to_checkout_hook))
+	retval = push_to_checkout(sha1, &invoked_hook, &env, work_tree);
+	if (!invoked_hook)
 		retval = push_to_deploy(sha1, &env, work_tree);
-	else
-		retval = push_to_checkout(sha1, &env, work_tree);
 
 	strvec_clear(&env);
 	return retval;
diff --git a/commit.c b/commit.c
index e8147a88fc6..cf62ebceae5 100644
--- a/commit.c
+++ b/commit.c
@@ -1697,6 +1697,7 @@ size_t ignore_non_trailer(const char *buf, size_t len)
 }
 
 int run_commit_hook(int editor_is_used, const char *index_file,
+		    int *invoked_hook,
 		    const char *name, ...)
 {
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
diff --git a/commit.h b/commit.h
index df42eb434f3..b5a542993c6 100644
--- a/commit.h
+++ b/commit.h
@@ -363,7 +363,8 @@ int compare_commits_by_commit_date(const void *a_, const void *b_, void *unused)
 int compare_commits_by_gen_then_commit_date(const void *a_, const void *b_, void *unused);
 
 LAST_ARG_MUST_BE_NULL
-int run_commit_hook(int editor_is_used, const char *index_file, const char *name, ...);
+int run_commit_hook(int editor_is_used, const char *index_file,
+		    int *invoked_hook, const char *name, ...);
 
 /* Sign a commit or tag buffer, storing the result in a header. */
 int sign_with_header(struct strbuf *buf, const char *keyid);
diff --git a/hook.c b/hook.c
index 17ae65eca31..3cf51460279 100644
--- a/hook.c
+++ b/hook.c
@@ -138,6 +138,9 @@ static int notify_hook_finished(int result,
 	/* |= rc in cb */
 	hook_cb->rc |= result;
 
+	if (hook_cb->invoked_hook)
+		*hook_cb->invoked_hook = 1;
+
 	return 1;
 }
 
@@ -152,6 +155,7 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 		.rc = 0,
 		.hook_name = hook_name,
 		.options = options,
+		.invoked_hook = options->invoked_hook,
 	};
 	if (options->absolute_path) {
 		strbuf_add_absolute_path(&abs_path, hook_path);
diff --git a/hook.h b/hook.h
index 5f895032341..9d163e6f992 100644
--- a/hook.h
+++ b/hook.h
@@ -57,6 +57,15 @@ struct run_hooks_opt
 	 * for an example.
 	 */
 	consume_sideband_fn consume_sideband;
+
+	/*
+	 * A pointer which if provided will be set to 1 or 0 depending
+	 * on if a hook was invoked (i.e. existed), regardless of
+	 * whether or not that was successful. Used for avoiding
+	 * TOCTOU races in code that would otherwise call hook_exist()
+	 * after a "maybe hook run" to see if a hook was invoked.
+	 */
+	int *invoked_hook;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
@@ -81,6 +90,7 @@ struct hook_cb_data {
 	const char *hook_name;
 	struct hook *run_me;
 	struct run_hooks_opt *options;
+	int *invoked_hook;
 };
 
 /*
diff --git a/sequencer.c b/sequencer.c
index ec2761e47d9..2440b9dccd8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1204,7 +1204,7 @@ static int run_prepare_commit_msg_hook(struct repository *r,
 	} else {
 		arg1 = "message";
 	}
-	if (run_commit_hook(0, r->index_file, "prepare-commit-msg", name,
+	if (run_commit_hook(0, r->index_file, NULL, "prepare-commit-msg", name,
 			    arg1, arg2, NULL))
 		ret = error(_("'prepare-commit-msg' hook failed"));
 
@@ -1534,7 +1534,7 @@ static int try_to_commit(struct repository *r,
 		goto out;
 	}
 
-	run_commit_hook(0, r->index_file, "post-commit", NULL);
+	run_commit_hook(0, r->index_file, NULL, "post-commit", NULL);
 	if (flags & AMEND_MSG)
 		commit_post_rewrite(r, current_head, oid);
 
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 30/30] hook-list.h: add a generated list of hooks, like config-list.h
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (28 preceding siblings ...)
  2021-06-14 10:33         ` [PATCH v2 29/30] hooks: fix a TOCTOU in "did we run a hook?" heuristic Ævar Arnfjörð Bjarmason
@ 2021-06-14 10:33         ` Ævar Arnfjörð Bjarmason
  2021-06-15 10:02           ` Ævar Arnfjörð Bjarmason
  2021-06-14 20:22         ` [PATCH v2 00/30] Minimal restart of "config-based-hooks" Emily Shaffer
                           ` (2 subsequent siblings)
  32 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-14 10:33 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Resolve a long-standing TODO item in bugreport.c of there being no
centralized listing of hooks, this fixes a bug with the bugreport
listing only knowing about 1/4 of the p4 hooks. It didn't know about
the "reference-transaction" hook either.

We can now make sure this is kept up-to-date, as the hook.c library
will die if asked to find a hook we don't know about yet. The only
(undocumented) exception is the artificial "test-hook" used in our own
test suite. Move some of the tests away from the "does-not-exist"
pseudo-hook, and test for the new behavior.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .gitignore           |  1 +
 Makefile             | 14 +++++++++++---
 builtin/bugreport.c  | 44 ++++++++------------------------------------
 generate-hooklist.sh | 24 ++++++++++++++++++++++++
 hook.c               | 22 ++++++++++++++++++++++
 t/t1800-hook.sh      | 14 +++++++++++---
 6 files changed, 77 insertions(+), 42 deletions(-)
 create mode 100755 generate-hooklist.sh

diff --git a/.gitignore b/.gitignore
index de39dc9961b..66189ca3cdc 100644
--- a/.gitignore
+++ b/.gitignore
@@ -191,6 +191,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/hook-list.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Makefile b/Makefile
index a6b71a0fbed..d0532f3c744 100644
--- a/Makefile
+++ b/Makefile
@@ -817,6 +817,7 @@ XDIFF_LIB = xdiff/lib.a
 
 GENERATED_H += command-list.h
 GENERATED_H += config-list.h
+GENERATED_H += hook-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2207,7 +2208,9 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
+hook.sp hook.s hook.o: hook-list.h
+
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h hook-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2240,6 +2243,11 @@ command-list.h: $(wildcard Documentation/git*.txt)
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+hook-list.h: generate-hooklist.sh
+hook-list.h: Documentation/githooks.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-hooklist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
@@ -2890,7 +2898,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
+EXCEPT_HDRS := command-list.h config-list.h hook-list.h unicode-width.h compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
 endif
@@ -2912,7 +2920,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: config-list.h command-list.h
+check: config-list.h command-list.h hook-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
diff --git a/builtin/bugreport.c b/builtin/bugreport.c
index 941c8d5e270..a7a1fcb8a7a 100644
--- a/builtin/bugreport.c
+++ b/builtin/bugreport.c
@@ -4,6 +4,7 @@
 #include "help.h"
 #include "compat/compiler.h"
 #include "hook.h"
+#include "hook-list.h"
 
 
 static void get_system_info(struct strbuf *sys_info)
@@ -41,39 +42,7 @@ static void get_system_info(struct strbuf *sys_info)
 
 static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 {
-	/*
-	 * NEEDSWORK: Doesn't look like there is a list of all possible hooks;
-	 * so below is a transcription of `git help hooks`. Later, this should
-	 * be replaced with some programmatically generated list (generated from
-	 * doc or else taken from some library which tells us about all the
-	 * hooks)
-	 */
-	static const char *hook[] = {
-		"applypatch-msg",
-		"pre-applypatch",
-		"post-applypatch",
-		"pre-commit",
-		"pre-merge-commit",
-		"prepare-commit-msg",
-		"commit-msg",
-		"post-commit",
-		"pre-rebase",
-		"post-checkout",
-		"post-merge",
-		"pre-push",
-		"pre-receive",
-		"update",
-		"post-receive",
-		"post-update",
-		"push-to-checkout",
-		"pre-auto-gc",
-		"post-rewrite",
-		"sendemail-validate",
-		"fsmonitor-watchman",
-		"p4-pre-submit",
-		"post-index-change",
-	};
-	int i;
+	const char **p;
 
 	if (nongit) {
 		strbuf_addstr(hook_info,
@@ -81,9 +50,12 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 		return;
 	}
 
-	for (i = 0; i < ARRAY_SIZE(hook); i++)
-		if (hook_exists(hook[i]))
-			strbuf_addf(hook_info, "%s\n", hook[i]);
+	for (p = hook_name_list; *p; p++) {
+		const char *hook = *p;
+
+		if (hook_exists(hook))
+			strbuf_addf(hook_info, "%s\n", hook);
+	}
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/generate-hooklist.sh b/generate-hooklist.sh
new file mode 100755
index 00000000000..5a3f7f849c8
--- /dev/null
+++ b/generate-hooklist.sh
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-hooklist.sh */"
+
+print_hook_list () {
+	cat <<EOF
+static const char *hook_name_list[] = {
+EOF
+	perl -ne '
+		chomp;
+		@l[$.] = $_;
+		push @h => $l[$. - 1] if /^~~~+$/s;
+		END {
+			print qq[\t"$_",\n] for sort @h;
+		}
+	' <Documentation/githooks.txt
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_hook_list
diff --git a/hook.c b/hook.c
index 3cf51460279..68e8c1ba868 100644
--- a/hook.c
+++ b/hook.c
@@ -1,11 +1,33 @@
 #include "cache.h"
 #include "hook.h"
 #include "run-command.h"
+#include "hook-list.h"
+
+static int known_hook(const char *name)
+{
+	const char **p;
+	size_t len = strlen(name);
+	for (p = hook_name_list; *p; p++) {
+		const char *hook = *p;
+
+		if (!strncmp(name, hook, len) && hook[len] == '\0')
+			return 1;
+	}
+	if (!strcmp(name, "test-hook") ||
+	    !strcmp(name, "does-not-exist"))
+		return 1;
+
+	return 0;
+}
 
 const char *find_hook(const char *name)
 {
 	static struct strbuf path = STRBUF_INIT;
 
+	if (!known_hook(name))
+		die(_("the hook '%s' is not known to git, should be in hook-list.h via githooks(5)"),
+		    name);
+
 	strbuf_reset(&path);
 	strbuf_git_path(&path, "hooks/%s", name);
 	if (access(path.buf, X_OK) < 0) {
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
index 1a6b708137e..a4b089eb7fe 100755
--- a/t/t1800-hook.sh
+++ b/t/t1800-hook.sh
@@ -6,17 +6,25 @@ test_description='git-hook command'
 
 test_expect_success 'git hook run -- nonexistent hook' '
 	cat >stderr.expect <<-\EOF &&
-	error: cannot find a hook named does-not-exist
+	error: cannot find a hook named test-hook
 	EOF
-	test_expect_code 1 git hook run does-not-exist 2>stderr.actual &&
+	test_expect_code 1 git hook run test-hook 2>stderr.actual &&
 	test_cmp stderr.expect stderr.actual
 '
 
 test_expect_success 'git hook run -- nonexistent hook with --ignore-missing' '
-	git hook run --ignore-missing does-not-exist 2>stderr.actual &&
+	git hook run --ignore-missing test-hook 2>stderr.actual &&
 	test_must_be_empty stderr.actual
 '
 
+test_expect_success 'git hook run -- nonexistent hook with --ignore-missing' '
+	cat >stderr.expect <<-\EOF &&
+	fatal: the hook '"'"'unknown-hook'"'"' is not known to git, should be in hook-list.h via githooks(5)
+	EOF
+	test_expect_code 128 git hook run unknown-hook 2>stderr.actual &&
+	test_cmp stderr.expect stderr.actual
+'
+
 test_expect_success 'git hook run -- basic' '
 	write_script .git/hooks/test-hook <<-EOF &&
 	echo Test hook
-- 
2.32.0.rc3.434.gd8aed1f08a7


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 00/30] Minimal restart of "config-based-hooks"
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (29 preceding siblings ...)
  2021-06-14 10:33         ` [PATCH v2 30/30] hook-list.h: add a generated list of hooks, like config-list.h Ævar Arnfjörð Bjarmason
@ 2021-06-14 20:22         ` Emily Shaffer
  2021-06-16  0:45           ` Junio C Hamano
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
  2021-06-25 18:22         ` [PATCH v2 00/30] Minimal restart of "config-based-hooks" Felipe Contreras
  32 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-06-14 20:22 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

On Mon, Jun 14, 2021 at 12:32:49PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> I proposed splitting Emily's "hook config" topic[1] into at least a
> topic that retains all current behavior of the codebase, and merely
> refactors existing behavior to new APIs, and then doing behavior
> changes later.
> 
> This is a re-roll of an attempt to do that. See v1's CL [2] for much
> more details.
> 
> I was hoping to get more feedback from Emily in reply to [3] and
> related E-Mails on the v1, but as it's been almost 2 weeks with no
> reply, and both her topic and mine semantically conflict with changes
> since merged to "master" I thought I'd send this re-roll.

Yes, sorry about this. I'm planning on reviewing your series this week.
Thanks for sending the v2.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 01/30] hook: add 'run' subcommand
  2021-06-14 10:32         ` [PATCH v2 01/30] hook: add 'run' subcommand Ævar Arnfjörð Bjarmason
@ 2021-06-14 21:33           ` Emily Shaffer
  2021-06-15  9:36             ` Ævar Arnfjörð Bjarmason
  2021-06-25 19:08             ` Felipe Contreras
  0 siblings, 2 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-06-14 21:33 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

On Mon, Jun 14, 2021 at 12:32:50PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> In order to enable hooks to be run as an external process, by a
> standalone Git command, or by tools which wrap Git, provide an external
> means to run all configured hook commands for a given hook event.

From what it says on the box, I'm slightly worried about this patch
doing too much at once, but let's see... (I think this is also a common
thing you and I disagree on - how much work to do per commit - so feel
free to ignore me ;) )

> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

Thanks for including attribution - I appreciate it.

> diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
> new file mode 100644
> index 00000000000..902b9cffaef
> --- /dev/null
> +++ b/Documentation/git-hook.txt
> @@ -0,0 +1,36 @@
> +git-hook(1)
> +===========
> +
> +NAME
> +----
> +git-hook - run git hooks
> +
> +SYNOPSIS
> +--------
> +[verse]
> +'git hook' run <hook-name> [-- <hook-args>]

Interesting. This is definitely more user friendly than `-a foo -a bar -a
aagh` ;)

Can we think of a scenario when a user might want to alias to part of
'git hook run' with an argument provided, but still wish to use the hook
more generally? I thought maybe something like `git hook run -a
"--user=Bob"` for a user who has a handful of hooks that can take some
additional argument, but then I realized that most hooks need to meet a
contract with which args they accept, so this isn't a reasonable use
case.

I also wondered whether accepting hook args this way implied that we
can't also provide environment vars for the hooks later on, but I think
it's fine to have those two interfaces be asymmetrical, e.g. `git hook
run -e "USERID=Bob" -- blah.txt`.

So I like this way of accepting them :)

[snip]
> +run::
> +
> +	Run the `<hook-name>` hook. Any positional arguments to the
> +	hook should be passed after an optional "--" (or
> +	"--end-of-options"). See "OPTIONS" below for the arguments
> +	this accepts.

Is it clear enough that users will need to provide arguments to certain
hooks? (Should this have some reference to githooks.txt?)

The "OPTIONS" reference is stale - there is no OPTIONS header in the
manpage now.

> --- /dev/null
> +++ b/builtin/hook.c
> @@ -0,0 +1,65 @@
> +#include "cache.h"
> +#include "builtin.h"
> +#include "config.h"
> +#include "hook.h"
> +#include "parse-options.h"
> +#include "strbuf.h"
> +#include "strvec.h"
> +
> +static const char * const builtin_hook_usage[] = {
> +	N_("git hook run <hook-name> [-- <hook-args>]"),
> +	NULL
> +};
> +
> +static int run(int argc, const char **argv, const char *prefix)
> +{
> +	int i;
> +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
> +	int rc = 0;
> +	const char *hook_name;
> +	const char *hook_path;
> +
> +	struct option run_options[] = {
> +		OPT_END(),
> +	};
> +
> +	argc = parse_options(argc, argv, prefix, run_options,
> +			     builtin_hook_usage, PARSE_OPT_KEEP_UNKNOWN | PARSE_OPT_KEEP_DASHDASH);
> +
> +	if (argc > 2) {
> +		if (strcmp(argv[2], "--") &&
> +		    strcmp(argv[2], "--end-of-options"))
> +			/* Having a -- for "run" is mandatory */
> +			usage_with_options(builtin_hook_usage, run_options);
> +		/* Add our arguments, start after -- */
> +		for (i = 3 ; i < argc; i++)
> +			strvec_push(&opt.args, argv[i]);
> +	}
> +
> +	/* Need to take into account core.hooksPath */
> +	git_config(git_default_config, NULL);
> +
> +	hook_name = argv[1];
> +	hook_path = find_hook(hook_name);
> +	if (!hook_path) {
> +		error("cannot find a hook named %s", hook_name);
> +		return 1;
> +	}
> +	rc = run_found_hooks(hook_name, hook_path, &opt);

Hum, what's the reasoning for not letting the hook.h call look up the
hook path for itself? I scanned through the v1 cover and older version
of this patch and didn't see any reasoning. To me, having the builtin
look up paths feels like incorrect layering.

> +int cmd_hook(int argc, const char **argv, const char *prefix)
> +{
> +	struct option builtin_hook_options[] = {
> +		OPT_END(),
> +	};
> +
> +	if (!strcmp(argv[1], "run"))
> +		return run(argc, argv, prefix);

Hum. This means that 'run' will still be included in argv for run(),
which I see that it works around silently. I personally find that to be
confusing - maybe at least a comment pointing it out, if you don't like
the idea of adjusting argv before passing it to run()?

> diff --git a/git.c b/git.c
> index 18bed9a9964..540909c391f 100644
> --- a/git.c
> +++ b/git.c
> @@ -538,6 +538,7 @@ static struct cmd_struct commands[] = {
>  	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
>  	{ "hash-object", cmd_hash_object },
>  	{ "help", cmd_help },
> +	{ "hook", cmd_hook, RUN_SETUP },

Hm. RUN_SETUP requires a gitdir, which I suppose makes sense as this is
a pre-config-hooks world.

Does this mean "git send-email" will abort if I try to run it with no
gitdir (which I often do)? I looked ahead to patch 10 and it doesn't
look like there's a significant change to the error handling, so I guess
that if it works for me today, it will work for me with this change too.

Later config-based hooks will mean that such hooks could exist without a
gitdir, but we can cross that bridge when we get there :)

> --- /dev/null
> +++ b/hook.c
> @@ -0,0 +1,114 @@
> +#include "cache.h"
> +#include "hook.h"
> +#include "run-command.h"
> +
> +void run_hooks_opt_clear(struct run_hooks_opt *o)
> +{
> +	strvec_clear(&o->env);
> +	strvec_clear(&o->args);

Maybe more graceful to nullcheck within the _clear() function before
dereferencing 'o'? That way callers don't need to worry about NULL
checking on their end.

> +static int pick_next_hook(struct child_process *cp,
> +			  struct strbuf *out,
> +			  void *pp_cb,
> +			  void **pp_task_cb)
> +{
> +	struct hook_cb_data *hook_cb = pp_cb;
> +	struct hook *run_me = hook_cb->run_me;
> +
> +	if (!run_me)
> +		BUG("did we not return 1 in notify_hook_finished?");

I'm not sure I like this message, even as a BUG(), although the things
I'd rather say ("run_me was NULL unexpectedly!") are obvious as soon as
you grep the codebase. So I think I dislike it for no reason :)

[...]

> +static int notify_start_failure(struct strbuf *out,
> +				void *pp_cb,
> +				void *pp_task_cp)
> +{
> +	struct hook_cb_data *hook_cb = pp_cb;
> +	struct hook *attempted = pp_task_cp;
> +
> +	/* |= rc in cb */
> +	hook_cb->rc |= 1;

Yuck, I think I wrote this comment... yikes. Maybe something like
"hook_cb->rc reflects cumulative failure state" instead?

> +static int notify_hook_finished(int result,
> +				struct strbuf *out,
> +				void *pp_cb,
> +				void *pp_task_cb)
> +{
> +	struct hook_cb_data *hook_cb = pp_cb;
> +
> +	/* |= rc in cb */
> +	hook_cb->rc |= result;

(And same as above.)

> +int run_found_hooks(const char *hook_name, const char *hook_path,
> +		    struct run_hooks_opt *options)
> +{
> +	struct hook my_hook = {
> +		.hook_path = hook_path,

As mentioned earlier, I think it is neater - and better for config-based
hooks in the future - if my_hook.hook_path is set by find_hooks()
directly instead of by being passed in, here. (I expect you did it this
way because one of the later hooks lives in an odd place - but I seem to
remember that one being strange in other ways, too, and I ended up
letting it manage its own affairs in my attempt. So I'll look forward to
seeing whether you handled that differently.)

[...]

> +int run_hooks(const char *hook_name, struct run_hooks_opt *options)
> +{
> +	const char *hook_path;
> +	int ret;
> +	if (!options)
> +		BUG("a struct run_hooks_opt must be provided to run_hooks");
> +
> +	hook_path = find_hook(hook_name);
> +
> +	/* Care about nonexistence? Use run_found_hooks() */
> +	if (!hook_path)
> +		return 0;

Ah, I see - you've done it this way so that builtin/hook.c can complain
"You tried to run pre-commit hook but you don't even have one!".

Hm. I think I dislike this comment for the same reason I dislike the one
much earlier in this patch - it's different from how I would have
written it. But I do think it still conveys the exact same information (I
would have said "If you need to act on a missing hook, use
run_found_hooks() instead") so chalk it up to difference in tone
preferences and ignore me :)

> diff --git a/hook.h b/hook.h
[...]
> +	/* Number of threads to parallelize across */
> +	int jobs;

I wonder whether it's worth changing the comments here...

[...]
> +/*
> + * Callback provided to feed_pipe_fn and consume_sideband_fn.
> + */

...and here, since they don't mean anything in the context of this
specific commit? But they will mean something later on in the series.

> +/*
> + * Calls find_hook(hookname) and runs the hooks (if any) with
> + * run_found_hooks().
> + */
> +int run_hooks(const char *hook_name, struct run_hooks_opt *options);
> +
> +/*
> + * Takes an already resolved hook and runs it. Internally the simpler
> + * run_hooks() will call this.
> + */
> +int run_found_hooks(const char *hookname, const char *hook_path,
> +		    struct run_hooks_opt *options);

The comments in the header here resolve any concerns I had about the
comments in the run_hooks() implementation. I like these a lot.

> diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
> new file mode 100755
> index 00000000000..f6ff6c4a493
> --- /dev/null
> +++ b/t/t1800-hook.sh
> @@ -0,0 +1,131 @@
> +#!/bin/bash
> +
> +test_description='git-hook command'
> +
> +. ./test-lib.sh
> +
> +test_expect_success 'git hook run -- nonexistent hook' '

Nit: Since you take '--' in 'git hook run' now, can you use something else as
a delimiter in the test names? I keep reading these as "here we will
call `git hook run -- nonexistent hook`" :/

> +	cat >stderr.expect <<-\EOF &&
> +	error: cannot find a hook named does-not-exist
> +	EOF
> +	test_expect_code 1 git hook run does-not-exist 2>stderr.actual &&
> +	test_cmp stderr.expect stderr.actual

I'm not wild about matching directly against the error message; that
means that the test will be a pain to update any time we update the
error message language. I'd prefer an approach where we check that the
error is for the reason we expect (by ensuring .git/hooks/does-not-exist
is not there in the fs) and then check that 'git hook run' fails, but do
not particularly care about the error message.

> +test_expect_success 'git hook run -- stdout and stderr handling' '

I have a slight preference towards "the name of the test tells me
exactly what is supposed to happen" - which means I'd prefer to see this
named "stdout and stderr both write to hook's stderr". Too chatty,
maybe, though.

> +test_expect_success 'git hook run -- out-of-repo runs excluded' '
> +	write_script .git/hooks/test-hook <<-EOF &&
> +	echo Test hook
> +	EOF
> +
> +	nongit test_must_fail git hook run test-hook

I wonder if it's necessary to enforce this. I'm just thinking, in a
config-based hook world later on, it will make sense to allow nongit
runs - specifically, I'd use the heck out of a send-email hook to fixup
my In-Reply-To lines, and I always run git-send-email from a nongit dir,
because I keep all my mailed patches stored away out of repo.

What's the general feeling towards "this is how it works, but we don't
have a good reason to require it"?

> +test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
> +	mkdir my-hooks &&
> +	write_script my-hooks/test-hook <<-EOF &&
> +	echo Hook ran >>actual
> +	EOF
> +
> +	cat >expect <<-\EOF &&
> +	Test hook
> +	Hook ran
> +	Hook ran
> +	Hook ran
> +	Hook ran
> +	EOF

I'm not sure I like this - collecting multiple runs into one "actual"
and only comparing it once at the end. Are there other places in the
codebase that do this?

> +
> +	# Test various ways of specifying the path. See also
> +	# t1350-config-hooks-path.sh
> +	>actual &&
> +	git hook run test-hook 2>>actual &&
> +	git -c core.hooksPath=my-hooks hook run test-hook 2>>actual &&
> +	git -c core.hooksPath=my-hooks/ hook run test-hook 2>>actual &&
> +	git -c core.hooksPath="$PWD/my-hooks" hook run test-hook 2>>actual &&
> +	git -c core.hooksPath="$PWD/my-hooks/" hook run test-hook 2>>actual &&
> +	test_cmp expect actual
> +'
> +
> +test_expect_success 'set up a pre-commit hook in core.hooksPath' '
> +	>actual &&
> +	mkdir -p .git/custom-hooks .git/hooks &&
> +	write_script .git/custom-hooks/pre-commit <<-\EOF &&
> +	echo CUSTOM >>actual
> +	EOF
> +	write_script .git/hooks/pre-commit <<-\EOF
> +	echo NORMAL >>actual
> +	EOF
> +'

Is this setup test a leftover from a later commit?


Overall, I think I like the direction your reroll is going - I've needed
some time to process it. Hopefully I'll be able to get through all or
most of the series this week, but there's a lot going on here, too. I'll
do what I can. Thanks for the help.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 04/30] gc: use hook library for pre-auto-gc hook
  2021-06-14 10:32         ` [PATCH v2 04/30] gc: use hook library for pre-auto-gc hook Ævar Arnfjörð Bjarmason
@ 2021-06-14 23:57           ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-06-14 23:57 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

On Mon, Jun 14, 2021 at 12:32:53PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> Using the hook.h library instead of the run-command.h library to run
> pre-auto-gc means that those hooks can be set up in config files, as
> well as in the hookdir. pre-auto-gc is called only from builtin/gc.c.

I think all the commit messages in "convert x to hook library" commits I
wrote extol the virtues of config-based hooks. Since that's not part of
your reroll, I expect we should change the commit messages too.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 01/30] hook: add 'run' subcommand
  2021-06-14 21:33           ` Emily Shaffer
@ 2021-06-15  9:36             ` Ævar Arnfjörð Bjarmason
  2021-06-18 22:13               ` Emily Shaffer
  2021-06-25 19:08             ` Felipe Contreras
  1 sibling, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-15  9:36 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee


On Mon, Jun 14 2021, Emily Shaffer wrote:

> On Mon, Jun 14, 2021 at 12:32:50PM +0200, Ævar Arnfjörð Bjarmason wrote:
>> 
>> In order to enable hooks to be run as an external process, by a
>> standalone Git command, or by tools which wrap Git, provide an external
>> means to run all configured hook commands for a given hook event.
>
> From what it says on the box, I'm slightly worried about this patch
> doing too much at once, but let's see... (I think this is also a common
> thing you and I disagree on - how much work to do per commit - so feel
> free to ignore me ;) )
>
>> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>
> Thanks for including attribution - I appreciate it.

They are almost entirely subsets of your patches, so I retained the
authorship, but added my own SOB. The only ones with my authorship are
something where I added fixes in-between or on top.

>> diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
>> new file mode 100644
>> index 00000000000..902b9cffaef
>> --- /dev/null
>> +++ b/Documentation/git-hook.txt
>> @@ -0,0 +1,36 @@
>> +git-hook(1)
>> +===========
>> +
>> +NAME
>> +----
>> +git-hook - run git hooks
>> +
>> +SYNOPSIS
>> +--------
>> +[verse]
>> +'git hook' run <hook-name> [-- <hook-args>]
>
> Interesting. This is definitely more user friendly than `-a foo -a bar -a
> aagh` ;)
>
> Can we think of a scenario when a user might want to alias to part of
> 'git hook run' with an argument provided, but still wish to use the hook
> more generally? I thought maybe something like `git hook run -a
> "--user=Bob"` for a user who has a handful of hooks that can take some
> additional argument, but then I realized that most hooks need to meet a
> contract with which args they accept, so this isn't a reasonable use
> case.

I can only think of ones where the [-- <hook-args>] would be sufficient.

> I also wondered whether accepting hook args this way implied that we
> can't also provide environment vars for the hooks later on, but I think
> it's fine to have those two interfaces be asymmetrical, e.g. `git hook
> run -e "USERID=Bob" -- blah.txt`.
>
> So I like this way of accepting them :)

I carved out the "-e" support from "run" as part of a general removal of
things not needed for the bug-for-bug compatibility with existing hook
interfaces, i.e. nothing needed it in this phase.

But I also can't see why a "hook run" would need to have an "-e",
anything that invokes the command has other more native ways of setting
the environment first, so the command just needs to avoid actively
clearing its own environment, no?

> [snip]
>> +run::
>> +
>> +	Run the `<hook-name>` hook. Any positional arguments to the
>> +	hook should be passed after an optional "--" (or
>> +	"--end-of-options"). See "OPTIONS" below for the arguments
>> +	this accepts.
>
> Is it clear enough that users will need to provide arguments to certain
> hooks? (Should this have some reference to githooks.txt?)

Probably, I also figured at this point it could be left as almost a
*--helper command.

> The "OPTIONS" reference is stale - there is no OPTIONS header in the
> manpage now.

Well spotted, will fix, or maybe you will :)

>> --- /dev/null
>> +++ b/builtin/hook.c
>> @@ -0,0 +1,65 @@
>> +#include "cache.h"
>> +#include "builtin.h"
>> +#include "config.h"
>> +#include "hook.h"
>> +#include "parse-options.h"
>> +#include "strbuf.h"
>> +#include "strvec.h"
>> +
>> +static const char * const builtin_hook_usage[] = {
>> +	N_("git hook run <hook-name> [-- <hook-args>]"),
>> +	NULL
>> +};
>> +
>> +static int run(int argc, const char **argv, const char *prefix)
>> +{
>> +	int i;
>> +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
>> +	int rc = 0;
>> +	const char *hook_name;
>> +	const char *hook_path;
>> +
>> +	struct option run_options[] = {
>> +		OPT_END(),
>> +	};
>> +
>> +	argc = parse_options(argc, argv, prefix, run_options,
>> +			     builtin_hook_usage, PARSE_OPT_KEEP_UNKNOWN | PARSE_OPT_KEEP_DASHDASH);
>> +
>> +	if (argc > 2) {
>> +		if (strcmp(argv[2], "--") &&
>> +		    strcmp(argv[2], "--end-of-options"))
>> +			/* Having a -- for "run" is mandatory */
>> +			usage_with_options(builtin_hook_usage, run_options);
>> +		/* Add our arguments, start after -- */
>> +		for (i = 3 ; i < argc; i++)
>> +			strvec_push(&opt.args, argv[i]);
>> +	}
>> +
>> +	/* Need to take into account core.hooksPath */
>> +	git_config(git_default_config, NULL);
>> +
>> +	hook_name = argv[1];
>> +	hook_path = find_hook(hook_name);
>> +	if (!hook_path) {
>> +		error("cannot find a hook named %s", hook_name);
>> +		return 1;
>> +	}
>> +	rc = run_found_hooks(hook_name, hook_path, &opt);
>
> Hum, what's the reasoning for not letting the hook.h call look up the
> hook path for itself? I scanned through the v1 cover and older version
> of this patch and didn't see any reasoning. To me, having the builtin
> look up paths feels like incorrect layering.

I see you end up addressing this yourself below, i.e. some users want to
silently ignore missing hooks, others want to error (as in this case).

>> +int cmd_hook(int argc, const char **argv, const char *prefix)
>> +{
>> +	struct option builtin_hook_options[] = {
>> +		OPT_END(),
>> +	};
>> +
>> +	if (!strcmp(argv[1], "run"))
>> +		return run(argc, argv, prefix);
>
> Hum. This means that 'run' will still be included in argv for run(),
> which I see that it works around silently. I personally find that to be
> confusing - maybe at least a comment pointing it out, if you don't like
> the idea of adjusting argv before passing it to run()?

It's not something I actively changed IIRC, i.e. it just came out of the
general refactoring of minimizing the command. I tihnk it would also
make sense (probably more sense) to adjust argv/argc as you suggest.

>> diff --git a/git.c b/git.c
>> index 18bed9a9964..540909c391f 100644
>> --- a/git.c
>> +++ b/git.c
>> @@ -538,6 +538,7 @@ static struct cmd_struct commands[] = {
>>  	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
>>  	{ "hash-object", cmd_hash_object },
>>  	{ "help", cmd_help },
>> +	{ "hook", cmd_hook, RUN_SETUP },
>
> Hm. RUN_SETUP requires a gitdir, which I suppose makes sense as this is
> a pre-config-hooks world.
>
> Does this mean "git send-email" will abort if I try to run it with no
> gitdir (which I often do)? I looked ahead to patch 10 and it doesn't
> look like there's a significant change to the error handling, so I guess
> that if it works for me today, it will work for me with this change too.
>
> Later config-based hooks will mean that such hooks could exist without a
> gitdir, but we can cross that bridge when we get there :)

Right, since it's a way more limited command used for only running hooks
by git itself it's a RUN_SETUP. I had some preliminary WIP work (based
on yours, obviously) to make it a RUN_SETUP_GENTLY, but within this
series it doesn't become needed.

>> --- /dev/null
>> +++ b/hook.c
>> @@ -0,0 +1,114 @@
>> +#include "cache.h"
>> +#include "hook.h"
>> +#include "run-command.h"
>> +
>> +void run_hooks_opt_clear(struct run_hooks_opt *o)
>> +{
>> +	strvec_clear(&o->env);
>> +	strvec_clear(&o->args);
>
> Maybe more graceful to nullcheck within the _clear() function before
> dereferencing 'o'? That way callers don't need to worry about NULL
> checking on their end.

I don't feel strongly about it, but I tihnk doing it this way is
consistent with other APIs in git. I.e. you do:

    struct something blah = BLAH_INIT;
    [...]
    blah_release(&blah);

And we simply assume that "blah" isn't NULL, and that any field
mandatory via the "INIT" initialization can be assumed not to be NULL.

>> +static int pick_next_hook(struct child_process *cp,
>> +			  struct strbuf *out,
>> +			  void *pp_cb,
>> +			  void **pp_task_cb)
>> +{
>> +	struct hook_cb_data *hook_cb = pp_cb;
>> +	struct hook *run_me = hook_cb->run_me;
>> +
>> +	if (!run_me)
>> +		BUG("did we not return 1 in notify_hook_finished?");
>
> I'm not sure I like this message, even as a BUG(), although the things
> I'd rather say ("run_me was NULL unexpectedly!") are obvious as soon as
> you grep the codebase. So I think I dislike it for no reason :)

I think I just added that as an assertion during development, could be
turned into somethin gelse.

> [...]
>
>> +static int notify_start_failure(struct strbuf *out,
>> +				void *pp_cb,
>> +				void *pp_task_cp)
>> +{
>> +	struct hook_cb_data *hook_cb = pp_cb;
>> +	struct hook *attempted = pp_task_cp;
>> +
>> +	/* |= rc in cb */
>> +	hook_cb->rc |= 1;
>
> Yuck, I think I wrote this comment... yikes. Maybe something like
> "hook_cb->rc reflects cumulative failure state" instead?

*nod*, more on this below...

>> +static int notify_hook_finished(int result,
>> +				struct strbuf *out,
>> +				void *pp_cb,
>> +				void *pp_task_cb)
>> +{
>> +	struct hook_cb_data *hook_cb = pp_cb;
>> +
>> +	/* |= rc in cb */
>> +	hook_cb->rc |= result;
>
> (And same as above.)

..and this...

>> +int run_found_hooks(const char *hook_name, const char *hook_path,
>> +		    struct run_hooks_opt *options)
>> +{
>> +	struct hook my_hook = {
>> +		.hook_path = hook_path,
>
> As mentioned earlier, I think it is neater - and better for config-based
> hooks in the future - if my_hook.hook_path is set by find_hooks()
> directly instead of by being passed in, here. (I expect you did it this
> way because one of the later hooks lives in an odd place - but I seem to
> remember that one being strange in other ways, too, and I ended up
> letting it manage its own affairs in my attempt. So I'll look forward to
> seeing whether you handled that differently.)

Yeah, the designated initializers made a few things much nicer.


>
> [...]
>
>> +int run_hooks(const char *hook_name, struct run_hooks_opt *options)
>> +{
>> +	const char *hook_path;
>> +	int ret;
>> +	if (!options)
>> +		BUG("a struct run_hooks_opt must be provided to run_hooks");
>> +
>> +	hook_path = find_hook(hook_name);
>> +
>> +	/* Care about nonexistence? Use run_found_hooks() */
>> +	if (!hook_path)
>> +		return 0;
>
> Ah, I see - you've done it this way so that builtin/hook.c can complain
> "You tried to run pre-commit hook but you don't even have one!".

Sounds better.

> Hm. I think I dislike this comment for the same reason I dislike the one
> much earlier in this patch - it's different from how I would have
> written it. But I do think it still conveys the exact same information (I
> would have said "If you need to act on a missing hook, use
> run_found_hooks() instead") so chalk it up to difference in tone
> preferences and ignore me :)
>
>> diff --git a/hook.h b/hook.h
> [...]
>> +	/* Number of threads to parallelize across */
>> +	int jobs;
>
> I wonder whether it's worth changing the comments here...

So more generally (continuing the "and this" above), I really tried to
not change the comments, structure, names, order etc. of your initial
series unless it was necessary for the stated aims of minimizing and
simplifying this.

That was all in order to make it easier for you to eventually rebase on
top of this, or to pick it up. I.e. it leaves most things as subsets of
corresponding patches of yours.

But if you think this approach is good / would want to pick it up and
run with it that would make me happy, and I think at that point you
should continue tweaking/adjusting anything like this that you want,
since the non-tweaking of it was something I did for your benefit, but
if you don't mind...

> [...]
>> +/*
>> + * Callback provided to feed_pipe_fn and consume_sideband_fn.
>> + */
>
> ...and here, since they don't mean anything in the context of this
> specific commit? But they will mean something later on in the series.

*ditto*

>> +/*
>> + * Calls find_hook(hookname) and runs the hooks (if any) with
>> + * run_found_hooks().
>> + */
>> +int run_hooks(const char *hook_name, struct run_hooks_opt *options);
>> +
>> +/*
>> + * Takes an already resolved hook and runs it. Internally the simpler
>> + * run_hooks() will call this.
>> + */
>> +int run_found_hooks(const char *hookname, const char *hook_path,
>> +		    struct run_hooks_opt *options);
>
> The comments in the header here resolve any concerns I had about the
> comments in the run_hooks() implementation. I like these a lot.
>
>> diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
>> new file mode 100755
>> index 00000000000..f6ff6c4a493
>> --- /dev/null
>> +++ b/t/t1800-hook.sh
>> @@ -0,0 +1,131 @@
>> +#!/bin/bash
>> +
>> +test_description='git-hook command'
>> +
>> +. ./test-lib.sh
>> +
>> +test_expect_success 'git hook run -- nonexistent hook' '
>
> Nit: Since you take '--' in 'git hook run' now, can you use something else as
> a delimiter in the test names? I keep reading these as "here we will
> call `git hook run -- nonexistent hook`" :/

Yeah, makes sense, maybe ": " more generally in this test?

>> +	cat >stderr.expect <<-\EOF &&
>> +	error: cannot find a hook named does-not-exist
>> +	EOF
>> +	test_expect_code 1 git hook run does-not-exist 2>stderr.actual &&
>> +	test_cmp stderr.expect stderr.actual
>
> I'm not wild about matching directly against the error message; that
> means that the test will be a pain to update any time we update the
> error message language. I'd prefer an approach where we check that the
> error is for the reason we expect (by ensuring .git/hooks/does-not-exist
> is not there in the fs) and then check that 'git hook run' fails, but do
> not particularly care about the error message.

I generally prefer it, but as noted above, "if you want to pick it up..." :)

FWIW yes I agree it's a bit of a pain, but having run into a lot of
things of the form:

    cmd 2>err &&
    grep "some message I expect" err

hiding issues like the message being duplicated, or an unexpected
additional error/warning being there, I think it's best just to test_cmp
the full output (inserting relevant OIDs if needed) when possible.

>> +test_expect_success 'git hook run -- stdout and stderr handling' '
>
> I have a slight preference towards "the name of the test tells me
> exactly what is supposed to happen" - which means I'd prefer to see this
> named "stdout and stderr both write to hook's stderr". Too chatty,
> maybe, though.
>
>> +test_expect_success 'git hook run -- out-of-repo runs excluded' '
>> +	write_script .git/hooks/test-hook <<-EOF &&
>> +	echo Test hook
>> +	EOF
>> +
>> +	nongit test_must_fail git hook run test-hook
>
> I wonder if it's necessary to enforce this. I'm just thinking, in a
> config-based hook world later on, it will make sense to allow nongit
> runs - specifically, I'd use the heck out of a send-email hook to fixup
> my In-Reply-To lines, and I always run git-send-email from a nongit dir,
> because I keep all my mailed patches stored away out of repo.
>
> What's the general feeling towards "this is how it works, but we don't
> have a good reason to require it"?

I for one think tests should assert existing behavior, we now have
RUN_SETUP so it would be a bug for thaht not to error, so a test is
asserting it. It doesn't mean that we shouldn't willy-nilly change this
later on if and when it's expected that we have a RUN_SETUP_GENTLY.

>> +test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
>> +	mkdir my-hooks &&
>> +	write_script my-hooks/test-hook <<-EOF &&
>> +	echo Hook ran >>actual
>> +	EOF
>> +
>> +	cat >expect <<-\EOF &&
>> +	Test hook
>> +	Hook ran
>> +	Hook ran
>> +	Hook ran
>> +	Hook ran
>> +	EOF
>
> I'm not sure I like this - collecting multiple runs into one "actual"
> and only comparing it once at the end. Are there other places in the
> codebase that do this?

Yeah, that's some ad-hoc nastyness, would be better if it attributed
specific runs or otherwise asserted the "we expected to run" otherwise.

>> +
>> +	# Test various ways of specifying the path. See also
>> +	# t1350-config-hooks-path.sh
>> +	>actual &&
>> +	git hook run test-hook 2>>actual &&
>> +	git -c core.hooksPath=my-hooks hook run test-hook 2>>actual &&
>> +	git -c core.hooksPath=my-hooks/ hook run test-hook 2>>actual &&
>> +	git -c core.hooksPath="$PWD/my-hooks" hook run test-hook 2>>actual &&
>> +	git -c core.hooksPath="$PWD/my-hooks/" hook run test-hook 2>>actual &&
>> +	test_cmp expect actual
>> +'
>> +
>> +test_expect_success 'set up a pre-commit hook in core.hooksPath' '
>> +	>actual &&
>> +	mkdir -p .git/custom-hooks .git/hooks &&
>> +	write_script .git/custom-hooks/pre-commit <<-\EOF &&
>> +	echo CUSTOM >>actual
>> +	EOF
>> +	write_script .git/hooks/pre-commit <<-\EOF
>> +	echo NORMAL >>actual
>> +	EOF
>> +'
>
> Is this setup test a leftover from a later commit?

Hrm, maybe... :)

> Overall, I think I like the direction your reroll is going - I've needed
> some time to process it. Hopefully I'll be able to get through all or
> most of the series this week, but there's a lot going on here, too. I'll
> do what I can. Thanks for the help.

Yeah, will reply to any qusetions etc; and as noted above my initial
goal here was "hey, what about this approach", so if you wanted to pick
this up & run with it...

This particular version of the series is at github.com/avar/git.git's
es-avar/config-based-hooks-3 b.t.w.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 30/30] hook-list.h: add a generated list of hooks, like config-list.h
  2021-06-14 10:33         ` [PATCH v2 30/30] hook-list.h: add a generated list of hooks, like config-list.h Ævar Arnfjörð Bjarmason
@ 2021-06-15 10:02           ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-15 10:02 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason, Sibi Siddharthan


On Mon, Jun 14 2021, Ævar Arnfjörð Bjarmason wrote:

> Resolve a long-standing TODO item in bugreport.c of there being no
> centralized listing of hooks, this fixes a bug with the bugreport
> listing only knowing about 1/4 of the p4 hooks. It didn't know about
> the "reference-transaction" hook either.
>
> We can now make sure this is kept up-to-date, as the hook.c library
> will die if asked to find a hook we don't know about yet. The only
> (undocumented) exception is the artificial "test-hook" used in our own
> test suite. Move some of the tests away from the "does-not-exist"
> pseudo-hook, and test for the new behavior.
>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>  .gitignore           |  1 +
>  Makefile             | 14 +++++++++++---
>  builtin/bugreport.c  | 44 ++++++++------------------------------------
>  generate-hooklist.sh | 24 ++++++++++++++++++++++++
>  hook.c               | 22 ++++++++++++++++++++++
>  t/t1800-hook.sh      | 14 +++++++++++---
>  6 files changed, 77 insertions(+), 42 deletions(-)
>  create mode 100755 generate-hooklist.sh
>
> diff --git a/.gitignore b/.gitignore
> index de39dc9961b..66189ca3cdc 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -191,6 +191,7 @@
>  /gitweb/static/gitweb.min.*
>  /config-list.h
>  /command-list.h
> +/hook-list.h
>  *.tar.gz
>  *.dsc
>  *.deb
> diff --git a/Makefile b/Makefile
> index a6b71a0fbed..d0532f3c744 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -817,6 +817,7 @@ XDIFF_LIB = xdiff/lib.a
>  
>  GENERATED_H += command-list.h
>  GENERATED_H += config-list.h
> +GENERATED_H += hook-list.h

This fails CI on the Windows boxes (or some of them...) because we hard
rely on cmake there since semi-recently; See 4c2c38e800f (ci:
modification of main.yml to use cmake for vs-build job, 2020-06-26).

Fixing that seems to be a matter of copying some similar boilerplate
around, but I haven't tried.

I was somewhat surprised that cmake support being semi-mandatory seems
to have snuck up on us (or at least on me). The docs in
compat/vcbuild/README still suggest you need make, so perhaps the CI on
Windows could be split up into something that uses "make" and one that
just checks for the cmake compatibility.

It's not that I much prefer the Makefile, but I do think the state of
affairs of needing to duplicate any work of patching the Makefile would
be a step back for me, but maybe the benefit on Windows is so large as
to be worth it...

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 00/30] Minimal restart of "config-based-hooks"
  2021-06-14 20:22         ` [PATCH v2 00/30] Minimal restart of "config-based-hooks" Emily Shaffer
@ 2021-06-16  0:45           ` Junio C Hamano
  0 siblings, 0 replies; 479+ messages in thread
From: Junio C Hamano @ 2021-06-16  0:45 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: Ævar Arnfjörð Bjarmason, git, Jeff King,
	Taylor Blau, Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee

Emily Shaffer <emilyshaffer@google.com> writes:

> On Mon, Jun 14, 2021 at 12:32:49PM +0200, Ævar Arnfjörð Bjarmason wrote:
>> 
>> I proposed splitting Emily's "hook config" topic[1] into at least a
>> topic that retains all current behavior of the codebase, and merely
>> refactors existing behavior to new APIs, and then doing behavior
>> changes later.
>> 
>> This is a re-roll of an attempt to do that. See v1's CL [2] for much
>> more details.
>> 
>> I was hoping to get more feedback from Emily in reply to [3] and
>> related E-Mails on the v1, but as it's been almost 2 weeks with no
>> reply, and both her topic and mine semantically conflict with changes
>> since merged to "master" I thought I'd send this re-roll.
>
> Yes, sorry about this. I'm planning on reviewing your series this week.
> Thanks for sending the v2.

Thanks, both.  In the meantime, I will eject es/config-based-hooks
out of 'seen' and replace it with these patches, to see if there are
unexpected interactions with other topics in flight.


^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH 00/27] Base for "config-based-hooks"
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (30 preceding siblings ...)
  2021-06-14 20:22         ` [PATCH v2 00/30] Minimal restart of "config-based-hooks" Emily Shaffer
@ 2021-06-17 10:22         ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 01/27] hook: add 'run' subcommand Ævar Arnfjörð Bjarmason
                             ` (29 more replies)
  2021-06-25 18:22         ` [PATCH v2 00/30] Minimal restart of "config-based-hooks" Felipe Contreras
  32 siblings, 30 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

This v3 of the "Base for 'config-based-hooks'" topic is >95% a
slimmed-down versio nof Emily Schaffer's work to introduce a mechanism
to drive hooks via config.

This topic doesn't do that, but moves all hook execution (C libary and
Perl|Python script) to either the new hook.[ch] library code, or the
"git hook run" utility.

See previous iterations for more details:

 v0 (Emily's): http://lore.kernel.org/git/20210527000856.695702-1-emilyshaffer@google.com
 v1: https://lore.kernel.org/git/cover-00.31-00000000000-20210528T110515Z-avarab@gmail.com/
 v2: http://lore.kernel.org/git/cover-00.30-00000000000-20210614T101920Z-avarab@gmail.com

This series gained two new dependencies since v2, my just-submitted
preparatory topics:

    https://lore.kernel.org/git/cover-0.3-0000000000-20210617T095827Z-avarab@gmail.com/
    https://lore.kernel.org/git/cover-0.3-0000000000-20210617T100239Z-avarab@gmail.com/

And hopefully addreses all the feedback on v2, mainly/entirely from
Emily at https://lore.kernel.org/git/YMfLO9CT+iIDR3OA@google.com

I'd normally have waited longer for a v3, but as discussed in the
small hook.[ch] topic that precedes this one building the new
hook-list.h had an error on Windows CI due to a missing CMake change,
that's now fixed.

Other changes can be seen in the range-diff, all rather trivial fixes
like comment fixes, trimming off "argc/argv" before we pass things to
"git hook run", removing a redundant test setup etc.

I also added a GIT_TEST_FAKE_HOOKS=true for use in the test suite to
make us support a "test-hook" and a "does-not-exist" hook, we'd
previously accept those outside the test environment.

Emily Shaffer (25):
  hook: add 'run' subcommand
  gc: use hook library for pre-auto-gc hook
  rebase: teach pre-rebase to use hook.h
  am: convert applypatch hooks to use config
  hooks: convert 'post-checkout' hook to hook library
  merge: use config-based hooks for post-merge hook
  send-email: use 'git hook run' for 'sendemail-validate'
  git-p4: use 'git hook' to run hooks
  commit: use hook.h to execute hooks
  read-cache: convert post-index-change hook to use config
  receive-pack: convert push-to-checkout hook to hook.h
  run-command: remove old run_hook_{le,ve}() hook API
  run-command: allow stdin for run_processes_parallel
  hook: support passing stdin to hooks
  am: convert 'post-rewrite' hook to hook.h
  run-command: add stdin callback for parallelization
  hook: provide stdin by string_list or callback
  hook: convert 'post-rewrite' hook in sequencer.c to hook.h
  transport: convert pre-push hook to use config
  reference-transaction: use hook.h to run hooks
  run-command: allow capturing of collated output
  hooks: allow callers to capture output
  receive-pack: convert 'update' hook to hook.h
  post-update: use hook.h library
  receive-pack: convert receive hooks to hook.h

Ævar Arnfjörð Bjarmason (2):
  git hook run: add an --ignore-missing flag
  hooks: fix a TOCTOU in "did we run a hook?" heuristic

 .gitignore                  |   1 +
 Documentation/git-hook.txt  |  51 ++++++
 Documentation/githooks.txt  |   4 +
 Makefile                    |   1 +
 builtin.h                   |   1 +
 builtin/am.c                |  33 ++--
 builtin/checkout.c          |  17 +-
 builtin/clone.c             |   7 +-
 builtin/commit.c            |  18 ++-
 builtin/fetch.c             |   1 +
 builtin/gc.c                |   8 +-
 builtin/hook.c              |  90 +++++++++++
 builtin/merge.c             |  21 ++-
 builtin/rebase.c            |   9 +-
 builtin/receive-pack.c      | 298 ++++++++++++++++++------------------
 builtin/submodule--helper.c |   2 +-
 builtin/worktree.c          |  30 ++--
 command-list.txt            |   1 +
 commit.c                    |  17 +-
 commit.h                    |   3 +-
 git-p4.py                   |  72 +--------
 git-send-email.perl         |  20 ++-
 git.c                       |   1 +
 hook.c                      | 170 ++++++++++++++++++++
 hook.h                      | 109 +++++++++++++
 read-cache.c                |  12 +-
 refs.c                      |  42 ++---
 reset.c                     |  15 +-
 run-command.c               | 122 ++++++++++-----
 run-command.h               |  48 ++++--
 sequencer.c                 |  87 +++++------
 submodule.c                 |   1 +
 t/helper/test-run-command.c |  46 +++++-
 t/t0061-run-command.sh      |  37 +++++
 t/t1800-hook.sh             | 154 +++++++++++++++++++
 t/t9001-send-email.sh       |   4 +-
 transport.c                 |  57 ++-----
 37 files changed, 1136 insertions(+), 474 deletions(-)
 create mode 100644 Documentation/git-hook.txt
 create mode 100644 builtin/hook.c
 create mode 100755 t/t1800-hook.sh

Range-diff:
 1:  447d349c73 !  1:  cf4b06bfdf hook: add 'run' subcommand
    @@ Documentation/git-hook.txt (new)
     +-----------
     +
     +run::
    -+
    -+	Run the `<hook-name>` hook. Any positional arguments to the
    -+	hook should be passed after an optional "--" (or
    -+	"--end-of-options"). See "OPTIONS" below for the arguments
    -+	this accepts.
    ++	Run the `<hook-name>` hook. See linkgit:githooks[5] for
    ++	the hook names we support.
    +++
    ++Any positional arguments to the hook should be passed after an
    ++optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
    ++arguments (if any) differ by hook name, see linkgit:githooks[5] for
    ++what those are.
     +
     +SEE ALSO
     +--------
    @@ Documentation/githooks.txt: and "0" meaning they were not.
      Part of the linkgit:git[1] suite
     
      ## Makefile ##
    -@@ Makefile: LIB_OBJS += hash-lookup.o
    - LIB_OBJS += hashmap.o
    - LIB_OBJS += help.o
    - LIB_OBJS += hex.o
    -+LIB_OBJS += hook.o
    - LIB_OBJS += ident.o
    - LIB_OBJS += json-writer.o
    - LIB_OBJS += kwset.o
     @@ Makefile: BUILTIN_OBJS += builtin/get-tar-commit-id.o
      BUILTIN_OBJS += builtin/grep.o
      BUILTIN_OBJS += builtin/hash-object.o
    @@ builtin/hook.c (new)
     +#include "strvec.h"
     +
     +static const char * const builtin_hook_usage[] = {
    ++	N_("git hook <command> [...]"),
    ++	N_("git hook run <hook-name> [-- <hook-args>]"),
    ++	NULL
    ++};
    ++
    ++static const char * const builtin_hook_run_usage[] = {
     +	N_("git hook run <hook-name> [-- <hook-args>]"),
     +	NULL
     +};
    @@ builtin/hook.c (new)
     +	};
     +
     +	argc = parse_options(argc, argv, prefix, run_options,
    -+			     builtin_hook_usage, PARSE_OPT_KEEP_UNKNOWN | PARSE_OPT_KEEP_DASHDASH);
    ++			     builtin_hook_run_usage,
    ++			     PARSE_OPT_KEEP_UNKNOWN | PARSE_OPT_KEEP_DASHDASH);
     +
    -+	if (argc > 2) {
    -+		if (strcmp(argv[2], "--") &&
    -+		    strcmp(argv[2], "--end-of-options"))
    ++	if (argc > 1) {
    ++		if (strcmp(argv[1], "--") &&
    ++		    strcmp(argv[1], "--end-of-options"))
     +			/* Having a -- for "run" is mandatory */
     +			usage_with_options(builtin_hook_usage, run_options);
     +		/* Add our arguments, start after -- */
    -+		for (i = 3 ; i < argc; i++)
    ++		for (i = 2 ; i < argc; i++)
     +			strvec_push(&opt.args, argv[i]);
     +	}
     +
     +	/* Need to take into account core.hooksPath */
     +	git_config(git_default_config, NULL);
     +
    -+	hook_name = argv[1];
    ++	/*
    ++	 * We are not using run_hooks() because we'd like to detect
    ++	 * missing hooks. Let's find it ourselves and call
    ++	 * run_found_hooks() instead.
    ++	 */
    ++	hook_name = argv[0];
     +	hook_path = find_hook(hook_name);
     +	if (!hook_path) {
     +		error("cannot find a hook named %s", hook_name);
    @@ builtin/hook.c (new)
     +	struct option builtin_hook_options[] = {
     +		OPT_END(),
     +	};
    ++	argc = parse_options(argc, argv, NULL, builtin_hook_options,
    ++			     builtin_hook_usage, PARSE_OPT_STOP_AT_NON_OPTION);
    ++	if (!argc)
    ++		usage_with_options(builtin_hook_usage, builtin_hook_options);
     +
    -+	if (!strcmp(argv[1], "run"))
    ++	if (!strcmp(argv[0], "run"))
     +		return run(argc, argv, prefix);
    -+	usage_with_options(builtin_hook_usage, builtin_hook_options);
    -+	return 1;
    ++	else
    ++		usage_with_options(builtin_hook_usage, builtin_hook_options);
     +}
     
      ## command-list.txt ##
    @@ git.c: static struct cmd_struct commands[] = {
      	{ "init", cmd_init_db },
      	{ "init-db", cmd_init_db },
     
    - ## hook.c (new) ##
    + ## hook.c ##
     @@
    -+#include "cache.h"
    -+#include "hook.h"
    -+#include "run-command.h"
    + #include "hook.h"
    + #include "run-command.h"
    + #include "hook-list.h"
    ++#include "config.h"
    + 
    + static int known_hook(const char *name)
    + {
    + 	const char **p;
    + 	size_t len = strlen(name);
    ++	static int test_hooks_ok = -1;
    ++
    + 	for (p = hook_name_list; *p; p++) {
    + 		const char *hook = *p;
    + 
    +@@ hook.c: static int known_hook(const char *name)
    + 			return 1;
    + 	}
    + 
    ++	if (test_hooks_ok == -1)
    ++		test_hooks_ok = git_env_bool("GIT_TEST_FAKE_HOOKS", 0);
    ++
    ++	if (test_hooks_ok &&
    ++	    (!strcmp(name, "test-hook") ||
    ++	     !strcmp(name, "does-not-exist")))
    ++		return 1;
    ++
    + 	return 0;
    + }
    + 
    +@@ hook.c: int hook_exists(const char *name)
    + {
    + 	return !!find_hook(name);
    + }
     +
     +void run_hooks_opt_clear(struct run_hooks_opt *o)
     +{
    @@ hook.c (new)
     +	struct hook_cb_data *hook_cb = pp_cb;
     +	struct hook *run_me = hook_cb->run_me;
     +
    -+	if (!run_me)
    -+		BUG("did we not return 1 in notify_hook_finished?");
    -+
     +	cp->no_stdin = 1;
     +	cp->env = hook_cb->options->env.v;
     +	cp->stdout_to_stderr = 1;
    @@ hook.c (new)
     +	struct hook_cb_data *hook_cb = pp_cb;
     +	struct hook *attempted = pp_task_cp;
     +
    -+	/* |= rc in cb */
     +	hook_cb->rc |= 1;
     +
     +	strbuf_addf(out, _("Couldn't start hook '%s'\n"),
    @@ hook.c (new)
     +{
     +	struct hook_cb_data *hook_cb = pp_cb;
     +
    -+	/* |= rc in cb */
     +	hook_cb->rc |= result;
     +
     +	return 1;
     +}
     +
    -+
     +int run_found_hooks(const char *hook_name, const char *hook_path,
     +		    struct run_hooks_opt *options)
     +{
    @@ hook.c (new)
     +
     +	hook_path = find_hook(hook_name);
     +
    -+	/* Care about nonexistence? Use run_found_hooks() */
    ++	/*
    ++	 * If you need to act on a missing hook, use run_found_hooks()
    ++	 * instead
    ++	 */
     +	if (!hook_path)
     +		return 0;
     +
    @@ hook.c (new)
     +	return ret;
     +}
     
    - ## hook.h (new) ##
    + ## hook.h ##
     @@
    -+#ifndef HOOK_H
    -+#define HOOK_H
    + #ifndef HOOK_H
    + #define HOOK_H
     +#include "strbuf.h"
     +#include "strvec.h"
     +#include "run-command.h"
    -+
    + 
    + /*
    +  * Returns the path to the hook file, or NULL if the hook is missing
    +@@ hook.h: const char *find_hook(const char *name);
    +  */
    + int hook_exists(const char *hookname);
    + 
     +struct hook {
     +	/* The path to the hook */
     +	const char *hook_path;
    @@ hook.h (new)
     +	/* Args to be passed to each hook */
     +	struct strvec args;
     +
    -+	/* Number of threads to parallelize across */
    ++	/*
    ++	 * Number of threads to parallelize across, currently a stub,
    ++	 * we use the parallel API for future-proofing, but we always
    ++	 * have one hook of a given name, so this is always an
    ++	 * implicit 1 for now.
    ++	 */
     +	int jobs;
     +};
     +
    @@ hook.h (new)
     +	.args = STRVEC_INIT, \
     +}
     +
    -+/*
    -+ * Callback provided to feed_pipe_fn and consume_sideband_fn.
    -+ */
     +struct hook_cb_data {
    ++	/* rc reflects the cumulative failure state */
     +	int rc;
     +	const char *hook_name;
     +	struct hook *run_me;
    @@ hook.h (new)
     + */
     +int run_found_hooks(const char *hookname, const char *hook_path,
     +		    struct run_hooks_opt *options);
    -+#endif
    + #endif
     
      ## t/t1800-hook.sh (new) ##
     @@
    @@ t/t1800-hook.sh (new)
     +
     +. ./test-lib.sh
     +
    -+test_expect_success 'git hook run -- nonexistent hook' '
    ++test_expect_success 'git hook usage' '
    ++	test_expect_code 129 git hook &&
    ++	test_expect_code 129 git hook -h &&
    ++	test_expect_code 129 git hook run -h
    ++'
    ++
    ++test_expect_success 'setup GIT_TEST_FAKE_HOOKS=true to permit "test-hook" and "does-not-exist" names"' '
    ++	GIT_TEST_FAKE_HOOKS=true &&
    ++	export GIT_TEST_FAKE_HOOKS
    ++'
    ++
    ++test_expect_success 'git hook run: nonexistent hook' '
     +	cat >stderr.expect <<-\EOF &&
    -+	error: cannot find a hook named does-not-exist
    ++	error: cannot find a hook named test-hook
     +	EOF
    -+	test_expect_code 1 git hook run does-not-exist 2>stderr.actual &&
    ++	test_expect_code 1 git hook run test-hook 2>stderr.actual &&
     +	test_cmp stderr.expect stderr.actual
     +'
     +
    -+test_expect_success 'git hook run -- basic' '
    ++test_expect_success 'git hook run: basic' '
     +	write_script .git/hooks/test-hook <<-EOF &&
     +	echo Test hook
     +	EOF
    @@ t/t1800-hook.sh (new)
     +	test_cmp expect actual
     +'
     +
    -+test_expect_success 'git hook run -- stdout and stderr handling' '
    ++test_expect_success 'git hook run: stdout and stderr both write to our stderr' '
     +	write_script .git/hooks/test-hook <<-EOF &&
     +	echo >&1 Will end up on stderr
     +	echo >&2 Will end up on stderr
    @@ t/t1800-hook.sh (new)
     +	test_must_be_empty stdout.actual
     +'
     +
    -+test_expect_success 'git hook run -- exit codes are passed along' '
    ++test_expect_success 'git hook run: exit codes are passed along' '
     +	write_script .git/hooks/test-hook <<-EOF &&
     +	exit 1
     +	EOF
    @@ t/t1800-hook.sh (new)
     +
     +test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
     +	mkdir my-hooks &&
    -+	write_script my-hooks/test-hook <<-EOF &&
    -+	echo Hook ran >>actual
    ++	write_script my-hooks/test-hook <<-\EOF &&
    ++	echo Hook ran $1 >>actual
     +	EOF
     +
     +	cat >expect <<-\EOF &&
     +	Test hook
    -+	Hook ran
    -+	Hook ran
    -+	Hook ran
    -+	Hook ran
    ++	Hook ran one
    ++	Hook ran two
    ++	Hook ran three
    ++	Hook ran four
     +	EOF
     +
     +	# Test various ways of specifying the path. See also
     +	# t1350-config-hooks-path.sh
     +	>actual &&
    -+	git hook run test-hook 2>>actual &&
    -+	git -c core.hooksPath=my-hooks hook run test-hook 2>>actual &&
    -+	git -c core.hooksPath=my-hooks/ hook run test-hook 2>>actual &&
    -+	git -c core.hooksPath="$PWD/my-hooks" hook run test-hook 2>>actual &&
    -+	git -c core.hooksPath="$PWD/my-hooks/" hook run test-hook 2>>actual &&
    ++	git hook run test-hook -- ignored 2>>actual &&
    ++	git -c core.hooksPath=my-hooks hook run test-hook -- one 2>>actual &&
    ++	git -c core.hooksPath=my-hooks/ hook run test-hook -- two 2>>actual &&
    ++	git -c core.hooksPath="$PWD/my-hooks" hook run test-hook -- three 2>>actual &&
    ++	git -c core.hooksPath="$PWD/my-hooks/" hook run test-hook -- four 2>>actual &&
     +	test_cmp expect actual
     +'
     +
    -+test_expect_success 'set up a pre-commit hook in core.hooksPath' '
    -+	>actual &&
    -+	mkdir -p .git/custom-hooks .git/hooks &&
    -+	write_script .git/custom-hooks/pre-commit <<-\EOF &&
    -+	echo CUSTOM >>actual
    -+	EOF
    -+	write_script .git/hooks/pre-commit <<-\EOF
    -+	echo NORMAL >>actual
    -+	EOF
    -+'
    -+
     +test_done
 2:  85195a78cf <  -:  ---------- run-command.h: move find_hook() to hook.h
 3:  eb5bdd993c <  -:  ---------- hook.c: add a hook_exists() wrapper and use it in bugreport.c
 4:  da2763192a =  2:  7209f73f28 gc: use hook library for pre-auto-gc hook
 5:  51e6e72f23 =  3:  e9a1e7cf61 rebase: teach pre-rebase to use hook.h
 6:  d2f3b26d46 =  4:  1d08726930 am: convert applypatch hooks to use config
 7:  d884465aab !  5:  32eec5dc2f hooks: convert 'post-checkout' hook to hook library
    @@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
     
      ## hook.h ##
     @@ hook.h: struct run_hooks_opt
    - 
    - 	/* Number of threads to parallelize across */
    + 	 * implicit 1 for now.
    + 	 */
      	int jobs;
     +
     +	/* Resolve and run the "absolute_path(hook)" instead of
 8:  ad6e750784 =  6:  e9fa3f6759 merge: use config-based hooks for post-merge hook
 9:  8d8b2d2645 !  7:  12347d901b git hook run: add an --ignore-missing flag
    @@ Documentation/git-hook.txt: git-hook - run git hooks
      
      DESCRIPTION
      -----------
    -@@ Documentation/git-hook.txt: run::
    - 	"--end-of-options"). See "OPTIONS" below for the arguments
    - 	this accepts.
    +@@ Documentation/git-hook.txt: optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
    + arguments (if any) differ by hook name, see linkgit:githooks[5] for
    + what those are.
      
     +OPTIONS
     +-------
    @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
      	};
      
     @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
    - 	hook_name = argv[1];
    + 	/*
    + 	 * We are not using run_hooks() because we'd like to detect
    + 	 * missing hooks. Let's find it ourselves and call
    +-	 * run_found_hooks() instead.
    ++	 * run_found_hooks() instead...
    + 	 */
    + 	hook_name = argv[0];
      	hook_path = find_hook(hook_name);
      	if (!hook_path) {
    ++		/* ... act like run_hooks() under --ignore-missing */
     +		if (ignore_missing)
     +			return 0;
      		error("cannot find a hook named %s", hook_name);
    @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
      	}
     
      ## t/t1800-hook.sh ##
    -@@ t/t1800-hook.sh: test_expect_success 'git hook run -- nonexistent hook' '
    +@@ t/t1800-hook.sh: test_expect_success 'git hook run: nonexistent hook' '
      	test_cmp stderr.expect stderr.actual
      '
      
    -+test_expect_success 'git hook run -- nonexistent hook with --ignore-missing' '
    +-test_expect_success 'git hook run: basic' '
    ++test_expect_success 'git hook run: nonexistent hook with --ignore-missing' '
     +	git hook run --ignore-missing does-not-exist 2>stderr.actual &&
     +	test_must_be_empty stderr.actual
     +'
     +
    - test_expect_success 'git hook run -- basic' '
    ++test_expect_success 'git hook run -- basic' '
      	write_script .git/hooks/test-hook <<-EOF &&
      	echo Test hook
    + 	EOF
10:  1953326d1d =  8:  71d209b407 send-email: use 'git hook run' for 'sendemail-validate'
11:  aa970a8175 !  9:  246a82b55b git-p4: use 'git hook' to run hooks
    @@ Commit message
         Python, we can directly call 'git hook run'. We emulate the existence
         check with the --ignore-missing flag.
     
    +    As this is the last hook execution in git.git to not go through "git
    +    hook run" or the hook.[ch] library we can now be absolutely sure that
    +    our assertion in hook.c that only hooks known by the generated (from
    +    githooks(5)) hook-list.h are permitted.
    +
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
12:  7756f10aac = 10:  e3f8482d80 commit: use hook.h to execute hooks
13:  0300607a9b = 11:  6ed61071c5 read-cache: convert post-index-change hook to use config
14:  ec4ad437f8 = 12:  e4ef3f4548 receive-pack: convert push-to-checkout hook to hook.h
15:  477eb2245c = 13:  e3dda367ec run-command: remove old run_hook_{le,ve}() hook API
16:  53a3877a47 = 14:  477d75bf57 run-command: allow stdin for run_processes_parallel
17:  c4f60db606 ! 15:  b7c0ee9719 hook: support passing stdin to hooks
    @@ Documentation/git-hook.txt: git-hook - run git hooks
      
      DESCRIPTION
      -----------
    -@@ Documentation/git-hook.txt: run::
    +@@ Documentation/git-hook.txt: what those are.
      OPTIONS
      -------
      
    @@ Documentation/git-hook.txt: run::
     
      ## builtin/hook.c ##
     @@
    - #include "strvec.h"
      
      static const char * const builtin_hook_usage[] = {
    + 	N_("git hook <command> [...]"),
     -	N_("git hook run <hook-name> [-- <hook-args>]"),
    ++	N_("git hook run [<args>] <hook-name> [-- <hook-args>]"),
    + 	NULL
    + };
    + 
    + static const char * const builtin_hook_run_usage[] = {
    + 	N_("git hook run <hook-name> [-- <hook-args>]"),
     +	N_("git hook run [--to-stdin=<path>] <hook-name> [-- <hook-args>]"),
      	NULL
      };
    @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
     
      ## hook.c ##
     @@ hook.c: static int pick_next_hook(struct child_process *cp,
    - 	if (!run_me)
    - 		BUG("did we not return 1 in notify_hook_finished?");
    + 	struct hook_cb_data *hook_cb = pp_cb;
    + 	struct hook *run_me = hook_cb->run_me;
      
     -	cp->no_stdin = 1;
    ++
     +	/* reopen the file for stdin; run_command closes it. */
     +	if (hook_cb->options->path_to_stdin) {
     +		cp->no_stdin = 0;
    @@ hook.h: struct run_hooks_opt
      #define RUN_HOOKS_OPT_INIT { \
     
      ## t/t1800-hook.sh ##
    -@@ t/t1800-hook.sh: test_expect_success 'set up a pre-commit hook in core.hooksPath' '
    - 	EOF
    +@@ t/t1800-hook.sh: test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
    + 	test_cmp expect actual
      '
      
     +test_expect_success 'stdin to hooks' '
18:  febf05ef23 = 16:  4035069a98 am: convert 'post-rewrite' hook to hook.h
19:  7baf2469d5 = 17:  c847a19581 run-command: add stdin callback for parallelization
20:  2edf9dea41 ! 18:  da46c859c1 hook: provide stdin by string_list or callback
    @@ hook.c: static int pick_next_hook(struct child_process *cp,
      	} else {
      		cp->no_stdin = 1;
      	}
    -@@ hook.c: static int notify_hook_finished(int result,
    - 	return 1;
    - }
    - 
    --
    - int run_found_hooks(const char *hook_name, const char *hook_path,
    - 		    struct run_hooks_opt *options)
    - {
     @@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
      	run_processes_parallel_tr2(options->jobs,
      				   pick_next_hook,
    @@ hook.c: int run_hooks(const char *hook_name, struct run_hooks_opt *options)
     +
      	hook_path = find_hook(hook_name);
      
    - 	/* Care about nonexistence? Use run_found_hooks() */
    + 	/*
     
      ## hook.h ##
    -@@
    +@@ hook.h: int hook_exists(const char *hookname);
      struct hook {
      	/* The path to the hook */
      	const char *hook_path;
    @@ hook.h: struct run_hooks_opt
     + */
     +int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb);
     +
    - /*
    -  * Callback provided to feed_pipe_fn and consume_sideband_fn.
    -  */
    + struct hook_cb_data {
    + 	/* rc reflects the cumulative failure state */
    + 	int rc;
21:  303b31ee62 = 19:  7343be28ef hook: convert 'post-rewrite' hook in sequencer.c to hook.h
22:  62eecafb3f = 20:  85bf13a083 transport: convert pre-push hook to use config
23:  6049b1cdc7 = 21:  331014bad1 reference-transaction: use hook.h to run hooks
24:  26ebbe4c54 = 22:  f7f56d0a3b run-command: allow capturing of collated output
25:  251085b752 = 23:  7f7fcc0688 hooks: allow callers to capture output
26:  e11a8f6007 = 24:  e74d49e559 receive-pack: convert 'update' hook to hook.h
27:  ecaedd13b8 = 25:  0bdc4878ac post-update: use hook.h library
28:  bb9d57f809 = 26:  db70b59b3b receive-pack: convert receive hooks to hook.h
29:  793f112f7a ! 27:  d86fedf041 hooks: fix a TOCTOU in "did we run a hook?" heuristic
    @@ commit.h: int compare_commits_by_commit_date(const void *a_, const void *b_, voi
     
      ## hook.c ##
     @@ hook.c: static int notify_hook_finished(int result,
    - 	/* |= rc in cb */
    + 
      	hook_cb->rc |= result;
      
     +	if (hook_cb->invoked_hook)
    @@ hook.h: struct hook_cb_data {
     +	int *invoked_hook;
      };
      
    - /*
    + void run_hooks_opt_clear(struct run_hooks_opt *o);
     
      ## sequencer.c ##
     @@ sequencer.c: static int run_prepare_commit_msg_hook(struct repository *r,
30:  bc086454d6 <  -:  ---------- hook-list.h: add a generated list of hooks, like config-list.h
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH 01/27] hook: add 'run' subcommand
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 02/27] gc: use hook library for pre-auto-gc hook Ævar Arnfjörð Bjarmason
                             ` (28 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

In order to enable hooks to be run as an external process, by a
standalone Git command, or by tools which wrap Git, provide an external
means to run all configured hook commands for a given hook event.

Most of our hooks require more complex functionality than this, but
let's start with the bare minimum required to support our simplest
hooks.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .gitignore                 |   1 +
 Documentation/git-hook.txt |  38 +++++++++++
 Documentation/githooks.txt |   4 ++
 Makefile                   |   1 +
 builtin.h                  |   1 +
 builtin/hook.c             |  81 +++++++++++++++++++++++
 command-list.txt           |   1 +
 git.c                      |   1 +
 hook.c                     | 119 +++++++++++++++++++++++++++++++++
 hook.h                     |  53 +++++++++++++++
 t/t1800-hook.sh            | 131 +++++++++++++++++++++++++++++++++++++
 11 files changed, 431 insertions(+)
 create mode 100644 Documentation/git-hook.txt
 create mode 100644 builtin/hook.c
 create mode 100755 t/t1800-hook.sh

diff --git a/.gitignore b/.gitignore
index 6be9de41ae..66189ca3cd 100644
--- a/.gitignore
+++ b/.gitignore
@@ -77,6 +77,7 @@
 /git-grep
 /git-hash-object
 /git-help
+/git-hook
 /git-http-backend
 /git-http-fetch
 /git-http-push
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
new file mode 100644
index 0000000000..660d6a992a
--- /dev/null
+++ b/Documentation/git-hook.txt
@@ -0,0 +1,38 @@
+git-hook(1)
+===========
+
+NAME
+----
+git-hook - run git hooks
+
+SYNOPSIS
+--------
+[verse]
+'git hook' run <hook-name> [-- <hook-args>]
+
+DESCRIPTION
+-----------
+
+This command is an interface to git hooks (see linkgit:githooks[5]).
+Currently it only provides a convenience wrapper for running hooks for
+use by git itself. In the future it might gain other functionality.
+
+SUBCOMMANDS
+-----------
+
+run::
+	Run the `<hook-name>` hook. See linkgit:githooks[5] for
+	the hook names we support.
++
+Any positional arguments to the hook should be passed after an
+optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
+arguments (if any) differ by hook name, see linkgit:githooks[5] for
+what those are.
+
+SEE ALSO
+--------
+linkgit:githooks[5]
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index b51959ff94..a16e62bc8c 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -698,6 +698,10 @@ and "0" meaning they were not.
 Only one parameter should be set to "1" when the hook runs.  The hook
 running passing "1", "1" should not be possible.
 
+SEE ALSO
+--------
+linkgit:git-hook[1]
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index 9bd31e1ac5..a52d509162 100644
--- a/Makefile
+++ b/Makefile
@@ -1106,6 +1106,7 @@ BUILTIN_OBJS += builtin/get-tar-commit-id.o
 BUILTIN_OBJS += builtin/grep.o
 BUILTIN_OBJS += builtin/hash-object.o
 BUILTIN_OBJS += builtin/help.o
+BUILTIN_OBJS += builtin/hook.o
 BUILTIN_OBJS += builtin/index-pack.o
 BUILTIN_OBJS += builtin/init-db.o
 BUILTIN_OBJS += builtin/interpret-trailers.o
diff --git a/builtin.h b/builtin.h
index 16ecd5586f..91740c1514 100644
--- a/builtin.h
+++ b/builtin.h
@@ -164,6 +164,7 @@ int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix);
 int cmd_grep(int argc, const char **argv, const char *prefix);
 int cmd_hash_object(int argc, const char **argv, const char *prefix);
 int cmd_help(int argc, const char **argv, const char *prefix);
+int cmd_hook(int argc, const char **argv, const char *prefix);
 int cmd_index_pack(int argc, const char **argv, const char *prefix);
 int cmd_init_db(int argc, const char **argv, const char *prefix);
 int cmd_interpret_trailers(int argc, const char **argv, const char *prefix);
diff --git a/builtin/hook.c b/builtin/hook.c
new file mode 100644
index 0000000000..7714d31ef1
--- /dev/null
+++ b/builtin/hook.c
@@ -0,0 +1,81 @@
+#include "cache.h"
+#include "builtin.h"
+#include "config.h"
+#include "hook.h"
+#include "parse-options.h"
+#include "strbuf.h"
+#include "strvec.h"
+
+static const char * const builtin_hook_usage[] = {
+	N_("git hook <command> [...]"),
+	N_("git hook run <hook-name> [-- <hook-args>]"),
+	NULL
+};
+
+static const char * const builtin_hook_run_usage[] = {
+	N_("git hook run <hook-name> [-- <hook-args>]"),
+	NULL
+};
+
+static int run(int argc, const char **argv, const char *prefix)
+{
+	int i;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	int rc = 0;
+	const char *hook_name;
+	const char *hook_path;
+
+	struct option run_options[] = {
+		OPT_END(),
+	};
+
+	argc = parse_options(argc, argv, prefix, run_options,
+			     builtin_hook_run_usage,
+			     PARSE_OPT_KEEP_UNKNOWN | PARSE_OPT_KEEP_DASHDASH);
+
+	if (argc > 1) {
+		if (strcmp(argv[1], "--") &&
+		    strcmp(argv[1], "--end-of-options"))
+			/* Having a -- for "run" is mandatory */
+			usage_with_options(builtin_hook_usage, run_options);
+		/* Add our arguments, start after -- */
+		for (i = 2 ; i < argc; i++)
+			strvec_push(&opt.args, argv[i]);
+	}
+
+	/* Need to take into account core.hooksPath */
+	git_config(git_default_config, NULL);
+
+	/*
+	 * We are not using run_hooks() because we'd like to detect
+	 * missing hooks. Let's find it ourselves and call
+	 * run_found_hooks() instead.
+	 */
+	hook_name = argv[0];
+	hook_path = find_hook(hook_name);
+	if (!hook_path) {
+		error("cannot find a hook named %s", hook_name);
+		return 1;
+	}
+	rc = run_found_hooks(hook_name, hook_path, &opt);
+
+	run_hooks_opt_clear(&opt);
+
+	return rc;
+}
+
+int cmd_hook(int argc, const char **argv, const char *prefix)
+{
+	struct option builtin_hook_options[] = {
+		OPT_END(),
+	};
+	argc = parse_options(argc, argv, NULL, builtin_hook_options,
+			     builtin_hook_usage, PARSE_OPT_STOP_AT_NON_OPTION);
+	if (!argc)
+		usage_with_options(builtin_hook_usage, builtin_hook_options);
+
+	if (!strcmp(argv[0], "run"))
+		return run(argc, argv, prefix);
+	else
+		usage_with_options(builtin_hook_usage, builtin_hook_options);
+}
diff --git a/command-list.txt b/command-list.txt
index a289f09ed6..9ccd8e5aeb 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -103,6 +103,7 @@ git-grep                                mainporcelain           info
 git-gui                                 mainporcelain
 git-hash-object                         plumbingmanipulators
 git-help                                ancillaryinterrogators          complete
+git-hook                                mainporcelain
 git-http-backend                        synchingrepositories
 git-http-fetch                          synchelpers
 git-http-push                           synchelpers
diff --git a/git.c b/git.c
index 18bed9a996..540909c391 100644
--- a/git.c
+++ b/git.c
@@ -538,6 +538,7 @@ static struct cmd_struct commands[] = {
 	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
 	{ "hash-object", cmd_hash_object },
 	{ "help", cmd_help },
+	{ "hook", cmd_hook, RUN_SETUP },
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "init", cmd_init_db },
 	{ "init-db", cmd_init_db },
diff --git a/hook.c b/hook.c
index 1f1db1ec9b..4d87e98255 100644
--- a/hook.c
+++ b/hook.c
@@ -2,11 +2,14 @@
 #include "hook.h"
 #include "run-command.h"
 #include "hook-list.h"
+#include "config.h"
 
 static int known_hook(const char *name)
 {
 	const char **p;
 	size_t len = strlen(name);
+	static int test_hooks_ok = -1;
+
 	for (p = hook_name_list; *p; p++) {
 		const char *hook = *p;
 
@@ -14,6 +17,14 @@ static int known_hook(const char *name)
 			return 1;
 	}
 
+	if (test_hooks_ok == -1)
+		test_hooks_ok = git_env_bool("GIT_TEST_FAKE_HOOKS", 0);
+
+	if (test_hooks_ok &&
+	    (!strcmp(name, "test-hook") ||
+	     !strcmp(name, "does-not-exist")))
+		return 1;
+
 	return 0;
 }
 
@@ -59,3 +70,111 @@ int hook_exists(const char *name)
 {
 	return !!find_hook(name);
 }
+
+void run_hooks_opt_clear(struct run_hooks_opt *o)
+{
+	strvec_clear(&o->env);
+	strvec_clear(&o->args);
+}
+
+static int pick_next_hook(struct child_process *cp,
+			  struct strbuf *out,
+			  void *pp_cb,
+			  void **pp_task_cb)
+{
+	struct hook_cb_data *hook_cb = pp_cb;
+	struct hook *run_me = hook_cb->run_me;
+
+	cp->no_stdin = 1;
+	cp->env = hook_cb->options->env.v;
+	cp->stdout_to_stderr = 1;
+	cp->trace2_hook_name = hook_cb->hook_name;
+
+	/* add command */
+	strvec_push(&cp->args, run_me->hook_path);
+
+	/*
+	 * add passed-in argv, without expanding - let the user get back
+	 * exactly what they put in
+	 */
+	strvec_pushv(&cp->args, hook_cb->options->args.v);
+
+	/* Provide context for errors if necessary */
+	*pp_task_cb = run_me;
+
+	return 1;
+}
+
+static int notify_start_failure(struct strbuf *out,
+				void *pp_cb,
+				void *pp_task_cp)
+{
+	struct hook_cb_data *hook_cb = pp_cb;
+	struct hook *attempted = pp_task_cp;
+
+	hook_cb->rc |= 1;
+
+	strbuf_addf(out, _("Couldn't start hook '%s'\n"),
+		    attempted->hook_path);
+
+	return 1;
+}
+
+static int notify_hook_finished(int result,
+				struct strbuf *out,
+				void *pp_cb,
+				void *pp_task_cb)
+{
+	struct hook_cb_data *hook_cb = pp_cb;
+
+	hook_cb->rc |= result;
+
+	return 1;
+}
+
+int run_found_hooks(const char *hook_name, const char *hook_path,
+		    struct run_hooks_opt *options)
+{
+	struct hook my_hook = {
+		.hook_path = hook_path,
+	};
+	struct hook_cb_data cb_data = {
+		.rc = 0,
+		.hook_name = hook_name,
+		.options = options,
+	};
+	cb_data.run_me = &my_hook;
+
+	if (options->jobs != 1)
+		BUG("we do not handle %d or any other != 1 job number yet", options->jobs);
+
+	run_processes_parallel_tr2(options->jobs,
+				   pick_next_hook,
+				   notify_start_failure,
+				   notify_hook_finished,
+				   &cb_data,
+				   "hook",
+				   hook_name);
+
+	return cb_data.rc;
+}
+
+int run_hooks(const char *hook_name, struct run_hooks_opt *options)
+{
+	const char *hook_path;
+	int ret;
+	if (!options)
+		BUG("a struct run_hooks_opt must be provided to run_hooks");
+
+	hook_path = find_hook(hook_name);
+
+	/*
+	 * If you need to act on a missing hook, use run_found_hooks()
+	 * instead
+	 */
+	if (!hook_path)
+		return 0;
+
+	ret = run_found_hooks(hook_name, hook_path, options);
+	return ret;
+}
diff --git a/hook.h b/hook.h
index 4c547ac15e..dcc2cc2112 100644
--- a/hook.h
+++ b/hook.h
@@ -1,5 +1,8 @@
 #ifndef HOOK_H
 #define HOOK_H
+#include "strbuf.h"
+#include "strvec.h"
+#include "run-command.h"
 
 /*
  * Returns the path to the hook file, or NULL if the hook is missing
@@ -13,4 +16,54 @@ const char *find_hook(const char *name);
  */
 int hook_exists(const char *hookname);
 
+struct hook {
+	/* The path to the hook */
+	const char *hook_path;
+};
+
+struct run_hooks_opt
+{
+	/* Environment vars to be set for each hook */
+	struct strvec env;
+
+	/* Args to be passed to each hook */
+	struct strvec args;
+
+	/*
+	 * Number of threads to parallelize across, currently a stub,
+	 * we use the parallel API for future-proofing, but we always
+	 * have one hook of a given name, so this is always an
+	 * implicit 1 for now.
+	 */
+	int jobs;
+};
+
+#define RUN_HOOKS_OPT_INIT { \
+	.jobs = 1, \
+	.env = STRVEC_INIT, \
+	.args = STRVEC_INIT, \
+}
+
+struct hook_cb_data {
+	/* rc reflects the cumulative failure state */
+	int rc;
+	const char *hook_name;
+	struct hook *run_me;
+	struct run_hooks_opt *options;
+};
+
+void run_hooks_opt_clear(struct run_hooks_opt *o);
+
+/*
+ * Calls find_hook(hookname) and runs the hooks (if any) with
+ * run_found_hooks().
+ */
+int run_hooks(const char *hook_name, struct run_hooks_opt *options);
+
+/*
+ * Takes an already resolved hook and runs it. Internally the simpler
+ * run_hooks() will call this.
+ */
+int run_found_hooks(const char *hookname, const char *hook_path,
+		    struct run_hooks_opt *options);
 #endif
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
new file mode 100755
index 0000000000..ecd517b162
--- /dev/null
+++ b/t/t1800-hook.sh
@@ -0,0 +1,131 @@
+#!/bin/bash
+
+test_description='git-hook command'
+
+. ./test-lib.sh
+
+test_expect_success 'git hook usage' '
+	test_expect_code 129 git hook &&
+	test_expect_code 129 git hook -h &&
+	test_expect_code 129 git hook run -h
+'
+
+test_expect_success 'setup GIT_TEST_FAKE_HOOKS=true to permit "test-hook" and "does-not-exist" names"' '
+	GIT_TEST_FAKE_HOOKS=true &&
+	export GIT_TEST_FAKE_HOOKS
+'
+
+test_expect_success 'git hook run: nonexistent hook' '
+	cat >stderr.expect <<-\EOF &&
+	error: cannot find a hook named test-hook
+	EOF
+	test_expect_code 1 git hook run test-hook 2>stderr.actual &&
+	test_cmp stderr.expect stderr.actual
+'
+
+test_expect_success 'git hook run: basic' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo Test hook
+	EOF
+
+	cat >expect <<-\EOF &&
+	Test hook
+	EOF
+	git hook run test-hook 2>actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git hook run: stdout and stderr both write to our stderr' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo >&1 Will end up on stderr
+	echo >&2 Will end up on stderr
+	EOF
+
+	cat >stderr.expect <<-\EOF &&
+	Will end up on stderr
+	Will end up on stderr
+	EOF
+	git hook run test-hook >stdout.actual 2>stderr.actual &&
+	test_cmp stderr.expect stderr.actual &&
+	test_must_be_empty stdout.actual
+'
+
+test_expect_success 'git hook run: exit codes are passed along' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	exit 1
+	EOF
+
+	test_expect_code 1 git hook run test-hook &&
+
+	write_script .git/hooks/test-hook <<-EOF &&
+	exit 2
+	EOF
+
+	test_expect_code 2 git hook run test-hook &&
+
+	write_script .git/hooks/test-hook <<-EOF &&
+	exit 128
+	EOF
+
+	test_expect_code 128 git hook run test-hook &&
+
+	write_script .git/hooks/test-hook <<-EOF &&
+	exit 129
+	EOF
+
+	test_expect_code 129 git hook run test-hook
+'
+
+test_expect_success 'git hook run arg u ments without -- is not allowed' '
+	test_expect_code 129 git hook run test-hook arg u ments
+'
+
+test_expect_success 'git hook run -- pass arguments' '
+	write_script .git/hooks/test-hook <<-\EOF &&
+	echo $1
+	echo $2
+	EOF
+
+	cat >expect <<-EOF &&
+	arg
+	u ments
+	EOF
+
+	git hook run test-hook -- arg "u ments" 2>actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git hook run -- out-of-repo runs excluded' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo Test hook
+	EOF
+
+	nongit test_must_fail git hook run test-hook
+'
+
+test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
+	mkdir my-hooks &&
+	write_script my-hooks/test-hook <<-\EOF &&
+	echo Hook ran $1 >>actual
+	EOF
+
+	cat >expect <<-\EOF &&
+	Test hook
+	Hook ran one
+	Hook ran two
+	Hook ran three
+	Hook ran four
+	EOF
+
+	# Test various ways of specifying the path. See also
+	# t1350-config-hooks-path.sh
+	>actual &&
+	git hook run test-hook -- ignored 2>>actual &&
+	git -c core.hooksPath=my-hooks hook run test-hook -- one 2>>actual &&
+	git -c core.hooksPath=my-hooks/ hook run test-hook -- two 2>>actual &&
+	git -c core.hooksPath="$PWD/my-hooks" hook run test-hook -- three 2>>actual &&
+	git -c core.hooksPath="$PWD/my-hooks/" hook run test-hook -- four 2>>actual &&
+	test_cmp expect actual
+'
+
+test_done
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 02/27] gc: use hook library for pre-auto-gc hook
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 01/27] hook: add 'run' subcommand Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-07-22 21:58             ` Emily Shaffer
  2021-06-17 10:22           ` [PATCH 03/27] rebase: teach pre-rebase to use hook.h Ævar Arnfjörð Bjarmason
                             ` (27 subsequent siblings)
  29 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Using the hook.h library instead of the run-command.h library to run
pre-auto-gc means that those hooks can be set up in config files, as
well as in the hookdir. pre-auto-gc is called only from builtin/gc.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/gc.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/builtin/gc.c b/builtin/gc.c
index f05d2f0a1a..a12641a691 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -32,6 +32,7 @@
 #include "remote.h"
 #include "object-store.h"
 #include "exec-cmd.h"
+#include "hook.h"
 
 #define FAILED_RUN "failed to run %s"
 
@@ -348,6 +349,8 @@ static void add_repack_incremental_option(void)
 
 static int need_to_gc(void)
 {
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+
 	/*
 	 * Setting gc.auto to 0 or negative can disable the
 	 * automatic gc.
@@ -394,8 +397,11 @@ static int need_to_gc(void)
 	else
 		return 0;
 
-	if (run_hook_le(NULL, "pre-auto-gc", NULL))
+	if (run_hooks("pre-auto-gc", &hook_opt)) {
+		run_hooks_opt_clear(&hook_opt);
 		return 0;
+	}
+	run_hooks_opt_clear(&hook_opt);
 	return 1;
 }
 
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 03/27] rebase: teach pre-rebase to use hook.h
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 01/27] hook: add 'run' subcommand Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 02/27] gc: use hook library for pre-auto-gc hook Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 04/27] am: convert applypatch hooks to use config Ævar Arnfjörð Bjarmason
                             ` (26 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Move the pre-rebase hook away from run-command.h to and over to the
new hook.h library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/rebase.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/builtin/rebase.c b/builtin/rebase.c
index 12f093121d..2081f6fa8d 100644
--- a/builtin/rebase.c
+++ b/builtin/rebase.c
@@ -28,6 +28,7 @@
 #include "sequencer.h"
 #include "rebase-interactive.h"
 #include "reset.h"
+#include "hook.h"
 
 #define DEFAULT_REFLOG_ACTION "rebase"
 
@@ -1313,6 +1314,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	char *squash_onto_name = NULL;
 	int reschedule_failed_exec = -1;
 	int allow_preemptive_ff = 1;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
 	struct option builtin_rebase_options[] = {
 		OPT_STRING(0, "onto", &options.onto_name,
 			   N_("revision"),
@@ -2022,10 +2024,13 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	}
 
 	/* If a hook exists, give it a chance to interrupt*/
+	strvec_pushl(&hook_opt.args, options.upstream_arg, argc ? argv[0] : NULL, NULL);
 	if (!ok_to_skip_pre_rebase &&
-	    run_hook_le(NULL, "pre-rebase", options.upstream_arg,
-			argc ? argv[0] : NULL, NULL))
+	    run_hooks("pre-rebase", &hook_opt)) {
+		run_hooks_opt_clear(&hook_opt);
 		die(_("The pre-rebase hook refused to rebase."));
+	}
+	run_hooks_opt_clear(&hook_opt);
 
 	if (options.flags & REBASE_DIFFSTAT) {
 		struct diff_options opts;
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 04/27] am: convert applypatch hooks to use config
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (2 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 03/27] rebase: teach pre-rebase to use hook.h Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 05/27] hooks: convert 'post-checkout' hook to hook library Ævar Arnfjörð Bjarmason
                             ` (25 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Teach pre-applypatch, post-applypatch, and applypatch-msg to use the
hook.h library instead of the run-command.h library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/am.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index 1c8a548903..9e9c1b5e9f 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -445,9 +445,12 @@ static void am_destroy(const struct am_state *state)
 static int run_applypatch_msg_hook(struct am_state *state)
 {
 	int ret;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 
 	assert(state->msg);
-	ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL);
+	strvec_push(&opt.args, am_path(state, "final-commit"));
+	ret = run_hooks("applypatch-msg", &opt);
+	run_hooks_opt_clear(&opt);
 
 	if (!ret) {
 		FREE_AND_NULL(state->msg);
@@ -1607,9 +1610,13 @@ static void do_commit(const struct am_state *state)
 	struct commit_list *parents = NULL;
 	const char *reflog_msg, *author, *committer = NULL;
 	struct strbuf sb = STRBUF_INIT;
+	struct run_hooks_opt hook_opt_pre = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt_post = RUN_HOOKS_OPT_INIT;
 
-	if (run_hook_le(NULL, "pre-applypatch", NULL))
+	if (run_hooks("pre-applypatch", &hook_opt_pre)) {
+		run_hooks_opt_clear(&hook_opt_pre);
 		exit(1);
+	}
 
 	if (write_cache_as_tree(&tree, 0, NULL))
 		die(_("git write-tree failed to write a tree"));
@@ -1660,8 +1667,10 @@ static void do_commit(const struct am_state *state)
 		fclose(fp);
 	}
 
-	run_hook_le(NULL, "post-applypatch", NULL);
+	run_hooks("post-applypatch", &hook_opt_post);
 
+	run_hooks_opt_clear(&hook_opt_pre);
+	run_hooks_opt_clear(&hook_opt_post);
 	strbuf_release(&sb);
 }
 
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 05/27] hooks: convert 'post-checkout' hook to hook library
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (3 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 04/27] am: convert applypatch hooks to use config Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 06/27] merge: use config-based hooks for post-merge hook Ævar Arnfjörð Bjarmason
                             ` (24 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Move the running of the 'post-checkout' hook away from run-command.h
to the new hook.h library. For "worktree" this requires a change to it
to run the hooks from a given directory.

We could strictly speaking skip the "absolute_path" flag and just
check if "dir" is specified, but let's split them up for clarity, as
well as for any future user who'd like to set "dir" but not implicitly
change the argument to an absolute path.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/checkout.c | 17 ++++++++++++-----
 builtin/clone.c    |  7 +++++--
 builtin/worktree.c | 30 ++++++++++++++----------------
 hook.c             |  8 ++++++++
 hook.h             |  9 +++++++++
 read-cache.c       |  1 +
 reset.c            | 15 +++++++++++----
 7 files changed, 60 insertions(+), 27 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index f4cd7747d3..6205ace09f 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "diff.h"
 #include "dir.h"
+#include "hook.h"
 #include "ll-merge.h"
 #include "lockfile.h"
 #include "merge-recursive.h"
@@ -106,13 +107,19 @@ struct branch_info {
 static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit,
 			      int changed)
 {
-	return run_hook_le(NULL, "post-checkout",
-			   oid_to_hex(old_commit ? &old_commit->object.oid : null_oid()),
-			   oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()),
-			   changed ? "1" : "0", NULL);
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	int rc;
+
 	/* "new_commit" can be NULL when checking out from the index before
 	   a commit exists. */
-
+	strvec_pushl(&opt.args,
+		     oid_to_hex(old_commit ? &old_commit->object.oid : null_oid()),
+		     oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()),
+		     changed ? "1" : "0",
+		     NULL);
+	rc = run_hooks("post-checkout", &opt);
+	run_hooks_opt_clear(&opt);
+	return rc;
 }
 
 static int update_some(const struct object_id *oid, struct strbuf *base,
diff --git a/builtin/clone.c b/builtin/clone.c
index 66fe66679c..de57a3119b 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -32,6 +32,7 @@
 #include "connected.h"
 #include "packfile.h"
 #include "list-objects-filter-options.h"
+#include "hook.h"
 
 /*
  * Overall FIXMEs:
@@ -775,6 +776,7 @@ static int checkout(int submodule_progress)
 	struct tree *tree;
 	struct tree_desc t;
 	int err = 0;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
 
 	if (option_no_checkout)
 		return 0;
@@ -820,8 +822,9 @@ static int checkout(int submodule_progress)
 	if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
 		die(_("unable to write new index file"));
 
-	err |= run_hook_le(NULL, "post-checkout", oid_to_hex(null_oid()),
-			   oid_to_hex(&oid), "1", NULL);
+	strvec_pushl(&hook_opt.args, oid_to_hex(null_oid()), oid_to_hex(&oid), "1", NULL);
+	err |= run_hooks("post-checkout", &hook_opt);
+	run_hooks_opt_clear(&hook_opt);
 
 	if (!err && (option_recurse_submodules.nr > 0)) {
 		struct strvec args = STRVEC_INIT;
diff --git a/builtin/worktree.c b/builtin/worktree.c
index b1350640fe..2ad26a76f4 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -382,22 +382,20 @@ static int add_worktree(const char *path, const char *refname,
 	 * is_junk is cleared, but do return appropriate code when hook fails.
 	 */
 	if (!ret && opts->checkout) {
-		const char *hook = find_hook("post-checkout");
-		if (hook) {
-			const char *env[] = { "GIT_DIR", "GIT_WORK_TREE", NULL };
-			cp.git_cmd = 0;
-			cp.no_stdin = 1;
-			cp.stdout_to_stderr = 1;
-			cp.dir = path;
-			cp.env = env;
-			cp.argv = NULL;
-			cp.trace2_hook_name = "post-checkout";
-			strvec_pushl(&cp.args, absolute_path(hook),
-				     oid_to_hex(null_oid()),
-				     oid_to_hex(&commit->object.oid),
-				     "1", NULL);
-			ret = run_command(&cp);
-		}
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
+		strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
+		strvec_pushl(&opt.args,
+			     oid_to_hex(null_oid()),
+			     oid_to_hex(&commit->object.oid),
+			     "1",
+			     NULL);
+		opt.dir = path;
+		opt.absolute_path = 1;
+
+		ret = run_hooks("post-checkout", &opt);
+
+		run_hooks_opt_clear(&opt);
 	}
 
 	strvec_clear(&child_env);
diff --git a/hook.c b/hook.c
index 4d87e98255..cb39404ac8 100644
--- a/hook.c
+++ b/hook.c
@@ -89,6 +89,7 @@ static int pick_next_hook(struct child_process *cp,
 	cp->env = hook_cb->options->env.v;
 	cp->stdout_to_stderr = 1;
 	cp->trace2_hook_name = hook_cb->hook_name;
+	cp->dir = hook_cb->options->dir;
 
 	/* add command */
 	strvec_push(&cp->args, run_me->hook_path);
@@ -135,6 +136,7 @@ static int notify_hook_finished(int result,
 int run_found_hooks(const char *hook_name, const char *hook_path,
 		    struct run_hooks_opt *options)
 {
+	struct strbuf abs_path = STRBUF_INIT;
 	struct hook my_hook = {
 		.hook_path = hook_path,
 	};
@@ -143,6 +145,10 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 		.hook_name = hook_name,
 		.options = options,
 	};
+	if (options->absolute_path) {
+		strbuf_add_absolute_path(&abs_path, hook_path);
+		my_hook.hook_path = abs_path.buf;
+	}
 	cb_data.run_me = &my_hook;
 
 	if (options->jobs != 1)
@@ -155,6 +161,8 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 				   &cb_data,
 				   "hook",
 				   hook_name);
+	if (options->absolute_path)
+		strbuf_release(&abs_path);
 
 	return cb_data.rc;
 }
diff --git a/hook.h b/hook.h
index dcc2cc2112..8bd0fc8d1f 100644
--- a/hook.h
+++ b/hook.h
@@ -36,6 +36,15 @@ struct run_hooks_opt
 	 * implicit 1 for now.
 	 */
 	int jobs;
+
+	/* Resolve and run the "absolute_path(hook)" instead of
+	 * "hook". Used for "git worktree" hooks
+	 */
+	int absolute_path;
+
+	/* Path to initial working directory for subprocess */
+	const char *dir;
+
 };
 
 #define RUN_HOOKS_OPT_INIT { \
diff --git a/read-cache.c b/read-cache.c
index 77961a3885..af5b97104c 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -27,6 +27,7 @@
 #include "progress.h"
 #include "sparse-index.h"
 #include "csum-file.h"
+#include "hook.h"
 
 /* Mask for the name length in ce_flags in the on-disk index */
 
diff --git a/reset.c b/reset.c
index 4bea758053..e6af33b901 100644
--- a/reset.c
+++ b/reset.c
@@ -7,6 +7,7 @@
 #include "tree-walk.h"
 #include "tree.h"
 #include "unpack-trees.h"
+#include "hook.h"
 
 int reset_head(struct repository *r, struct object_id *oid, const char *action,
 	       const char *switch_to_branch, unsigned flags,
@@ -126,10 +127,16 @@ int reset_head(struct repository *r, struct object_id *oid, const char *action,
 			ret = create_symref("HEAD", switch_to_branch,
 					    reflog_head);
 	}
-	if (run_hook)
-		run_hook_le(NULL, "post-checkout",
-			    oid_to_hex(orig ? orig : null_oid()),
-			    oid_to_hex(oid), "1", NULL);
+	if (run_hook) {
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		strvec_pushl(&opt.args,
+			     oid_to_hex(orig ? orig : null_oid()),
+			     oid_to_hex(oid),
+			     "1",
+			     NULL);
+		run_hooks("post-checkout", &opt);
+		run_hooks_opt_clear(&opt);
+	}
 
 leave_reset_head:
 	strbuf_release(&msg);
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 06/27] merge: use config-based hooks for post-merge hook
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (4 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 05/27] hooks: convert 'post-checkout' hook to hook library Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 07/27] git hook run: add an --ignore-missing flag Ævar Arnfjörð Bjarmason
                             ` (23 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Teach post-merge to use the hook.h library instead of the run-command.h
library to run hooks. This means that post-merge hooks can come from the
config as well as from the hookdir. post-merge is invoked only from
builtin/merge.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/merge.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/builtin/merge.c b/builtin/merge.c
index be98d66b0a..6128b60942 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -448,6 +448,7 @@ static void finish(struct commit *head_commit,
 		   const struct object_id *new_head, const char *msg)
 {
 	struct strbuf reflog_message = STRBUF_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	const struct object_id *head = &head_commit->object.oid;
 
 	if (!msg)
@@ -489,7 +490,9 @@ static void finish(struct commit *head_commit,
 	}
 
 	/* Run a post-merge hook */
-	run_hook_le(NULL, "post-merge", squash ? "1" : "0", NULL);
+	strvec_push(&opt.args, squash ? "1" : "0");
+	run_hooks("post-merge", &opt);
+	run_hooks_opt_clear(&opt);
 
 	apply_autostash(git_path_merge_autostash(the_repository));
 	strbuf_release(&reflog_message);
@@ -849,7 +852,7 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 	 * and write it out as a tree.  We must do this before we invoke
 	 * the editor and after we invoke run_status above.
 	 */
-	if (find_hook("pre-merge-commit"))
+	if (hook_exists("pre-merge-commit"))
 		discard_cache();
 	read_cache_from(index_file);
 	strbuf_addbuf(&msg, &merge_msg);
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 07/27] git hook run: add an --ignore-missing flag
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (5 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 06/27] merge: use config-based hooks for post-merge hook Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-07-02 23:47             ` Emily Shaffer
  2021-06-17 10:22           ` [PATCH 08/27] send-email: use 'git hook run' for 'sendemail-validate' Ævar Arnfjörð Bjarmason
                             ` (22 subsequent siblings)
  29 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

For certain one-shot hooks we'd like to optimistically run them, and
not complain if they don't exist. This will be used by send-email in a
subsequent commit.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-hook.txt | 10 +++++++++-
 builtin/hook.c             |  8 +++++++-
 t/t1800-hook.sh            |  7 ++++++-
 3 files changed, 22 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 660d6a992a..097fb9de63 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -8,7 +8,7 @@ git-hook - run git hooks
 SYNOPSIS
 --------
 [verse]
-'git hook' run <hook-name> [-- <hook-args>]
+'git hook' run [--ignore-missing] <hook-name> [-- <hook-args>]
 
 DESCRIPTION
 -----------
@@ -29,6 +29,14 @@ optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
 arguments (if any) differ by hook name, see linkgit:githooks[5] for
 what those are.
 
+OPTIONS
+-------
+
+--ignore-missing::
+	Ignore any missing hook by quietly returning zero. Used for
+	tools that want to do a blind one-shot run of a hook that may
+	or may not be present.
+
 SEE ALSO
 --------
 linkgit:githooks[5]
diff --git a/builtin/hook.c b/builtin/hook.c
index 7714d31ef1..47e0de7bbc 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -22,10 +22,13 @@ static int run(int argc, const char **argv, const char *prefix)
 	int i;
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	int rc = 0;
+	int ignore_missing = 0;
 	const char *hook_name;
 	const char *hook_path;
 
 	struct option run_options[] = {
+		OPT_BOOL(0, "ignore-missing", &ignore_missing,
+			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
 		OPT_END(),
 	};
 
@@ -49,11 +52,14 @@ static int run(int argc, const char **argv, const char *prefix)
 	/*
 	 * We are not using run_hooks() because we'd like to detect
 	 * missing hooks. Let's find it ourselves and call
-	 * run_found_hooks() instead.
+	 * run_found_hooks() instead...
 	 */
 	hook_name = argv[0];
 	hook_path = find_hook(hook_name);
 	if (!hook_path) {
+		/* ... act like run_hooks() under --ignore-missing */
+		if (ignore_missing)
+			return 0;
 		error("cannot find a hook named %s", hook_name);
 		return 1;
 	}
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
index ecd517b162..542e551628 100755
--- a/t/t1800-hook.sh
+++ b/t/t1800-hook.sh
@@ -23,7 +23,12 @@ test_expect_success 'git hook run: nonexistent hook' '
 	test_cmp stderr.expect stderr.actual
 '
 
-test_expect_success 'git hook run: basic' '
+test_expect_success 'git hook run: nonexistent hook with --ignore-missing' '
+	git hook run --ignore-missing does-not-exist 2>stderr.actual &&
+	test_must_be_empty stderr.actual
+'
+
+test_expect_success 'git hook run -- basic' '
 	write_script .git/hooks/test-hook <<-EOF &&
 	echo Test hook
 	EOF
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 08/27] send-email: use 'git hook run' for 'sendemail-validate'
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (6 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 07/27] git hook run: add an --ignore-missing flag Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 09/27] git-p4: use 'git hook' to run hooks Ævar Arnfjörð Bjarmason
                             ` (21 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Change the "sendmail-validate" hook to be run via the "git hook run"
wrapper instead of via a direct invocation.

This is the smallest possibly change to get "send-email" using "git
hook run". We still check the hook itself with "-x", and set a
"GIT_DIR" variable, both of which are asserted by our tests. We'll
need to get rid of this special behavior if we start running N hooks,
but for now let's be as close to bug-for-bug compatible as possible.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-send-email.perl   | 20 ++++++++++++--------
 t/t9001-send-email.sh |  4 ++--
 2 files changed, 14 insertions(+), 10 deletions(-)

diff --git a/git-send-email.perl b/git-send-email.perl
index 7ba0b3433d..9e47430403 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -214,13 +214,13 @@ sub format_2822_time {
 my $editor;
 
 sub system_or_msg {
-	my ($args, $msg) = @_;
+	my ($args, $msg, $cmd_name) = @_;
 	system(@$args);
 	my $signalled = $? & 127;
 	my $exit_code = $? >> 8;
 	return unless $signalled or $exit_code;
 
-	my @sprintf_args = ($args->[0], $exit_code);
+	my @sprintf_args = ($cmd_name ? $cmd_name : $args->[0], $exit_code);
 	if (defined $msg) {
 		# Quiet the 'redundant' warning category, except we
 		# need to support down to Perl 5.8, so we can't do a
@@ -1979,9 +1979,9 @@ sub validate_patch {
 	my ($fn, $xfer_encoding) = @_;
 
 	if ($repo) {
+		my $hook_name = 'sendemail-validate';
 		my $hooks_path = $repo->command_oneline('rev-parse', '--git-path', 'hooks');
-		my $validate_hook = catfile($hooks_path,
-					    'sendemail-validate');
+		my $validate_hook = catfile($hooks_path, $hook_name);
 		my $hook_error;
 		if (-x $validate_hook) {
 			my $target = abs_path($fn);
@@ -1990,13 +1990,17 @@ sub validate_patch {
 			chdir($repo->wc_path() or $repo->repo_path())
 				or die("chdir: $!");
 			local $ENV{"GIT_DIR"} = $repo->repo_path();
-			$hook_error = system_or_msg([$validate_hook, $target]);
+			my @validate_hook = ("git", "hook", "run", "--ignore-missing", $hook_name, "--", $target);
+			$hook_error = system_or_msg(\@validate_hook, undef,
+						       "git hook run $hook_name -- <patch>");
 			chdir($cwd_save) or die("chdir: $!");
 		}
 		if ($hook_error) {
-			die sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
-				       "%s\n" .
-				       "warning: no patches were sent\n"), $fn, $hook_error);
+			$hook_error = sprintf(__("fatal: %s: rejected by %s hook\n" .
+						 $hook_error . "\n" .
+						 "warning: no patches were sent\n"),
+					      $fn, $hook_name);
+			die $hook_error;
 		}
 	}
 
diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index 30eff725a9..6d4e25df8d 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -539,7 +539,7 @@ test_expect_success $PREREQ "--validate respects relative core.hooksPath path" '
 	test_path_is_file my-hooks.ran &&
 	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
-	fatal: command '"'"'my-hooks/sendemail-validate'"'"' died with exit code 1
+	fatal: command '"'"'git hook run sendemail-validate -- <patch>'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
@@ -558,7 +558,7 @@ test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" '
 	test_path_is_file my-hooks.ran &&
 	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
-	fatal: command '"'"'$hooks_path/sendemail-validate'"'"' died with exit code 1
+	fatal: command '"'"'git hook run sendemail-validate -- <patch>'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 09/27] git-p4: use 'git hook' to run hooks
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (7 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 08/27] send-email: use 'git hook run' for 'sendemail-validate' Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 10/27] commit: use hook.h to execute hooks Ævar Arnfjörð Bjarmason
                             ` (20 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Instead of duplicating the behavior of run-command.h:run_hook_le() in
Python, we can directly call 'git hook run'. We emulate the existence
check with the --ignore-missing flag.

As this is the last hook execution in git.git to not go through "git
hook run" or the hook.[ch] library we can now be absolutely sure that
our assertion in hook.c that only hooks known by the generated (from
githooks(5)) hook-list.h are permitted.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-p4.py | 72 ++++++-------------------------------------------------
 1 file changed, 7 insertions(+), 65 deletions(-)

diff --git a/git-p4.py b/git-p4.py
index d34a1946b7..e76d8df313 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -207,71 +207,13 @@ def decode_path(path):
         return path
 
 def run_git_hook(cmd, param=[]):
-    """Execute a hook if the hook exists."""
-    if verbose:
-        sys.stderr.write("Looking for hook: %s\n" % cmd)
-        sys.stderr.flush()
-
-    hooks_path = gitConfig("core.hooksPath")
-    if len(hooks_path) <= 0:
-        hooks_path = os.path.join(os.environ["GIT_DIR"], "hooks")
-
-    if not isinstance(param, list):
-        param=[param]
-
-    # resolve hook file name, OS depdenent
-    hook_file = os.path.join(hooks_path, cmd)
-    if platform.system() == 'Windows':
-        if not os.path.isfile(hook_file):
-            # look for the file with an extension
-            files = glob.glob(hook_file + ".*")
-            if not files:
-                return True
-            files.sort()
-            hook_file = files.pop()
-            while hook_file.upper().endswith(".SAMPLE"):
-                # The file is a sample hook. We don't want it
-                if len(files) > 0:
-                    hook_file = files.pop()
-                else:
-                    return True
-
-    if not os.path.isfile(hook_file) or not os.access(hook_file, os.X_OK):
-        return True
-
-    return run_hook_command(hook_file, param) == 0
-
-def run_hook_command(cmd, param):
-    """Executes a git hook command
-       cmd = the command line file to be executed. This can be
-       a file that is run by OS association.
-
-       param = a list of parameters to pass to the cmd command
-
-       On windows, the extension is checked to see if it should
-       be run with the Git for Windows Bash shell.  If there
-       is no file extension, the file is deemed a bash shell
-       and will be handed off to sh.exe. Otherwise, Windows
-       will be called with the shell to handle the file assocation.
-
-       For non Windows operating systems, the file is called
-       as an executable.
-    """
-    cli = [cmd] + param
-    use_shell = False
-    if platform.system() == 'Windows':
-        (root,ext) = os.path.splitext(cmd)
-        if ext == "":
-            exe_path = os.environ.get("EXEPATH")
-            if exe_path is None:
-                exe_path = ""
-            else:
-                exe_path = os.path.join(exe_path, "bin")
-            cli = [os.path.join(exe_path, "SH.EXE")] + cli
-        else:
-            use_shell = True
-    return subprocess.call(cli, shell=use_shell)
-
+    """args are specified with -a <arg> -a <arg> -a <arg>"""
+    args = ['git', 'hook', 'run', '--ignore-missing', cmd]
+    if param:
+        args.append("--")
+        for p in param:
+            args.append(p)
+    return subprocess.call(args) == 0
 
 def write_pipe(c, stdin):
     if verbose:
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 10/27] commit: use hook.h to execute hooks
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (8 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 09/27] git-p4: use 'git hook' to run hooks Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 11/27] read-cache: convert post-index-change hook to use config Ævar Arnfjörð Bjarmason
                             ` (19 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Teach run_commit_hook() to call hook.h instead of run-command.h. This
covers 'pre-commit', 'commit-msg', and
'prepare-commit-msg'.

Additionally, ask the hook library - not run-command - whether any
hooks will be run, as it's possible hooks may exist in the config but
not the hookdir.

Because all but 'post-commit' hooks are expected to make some state
change, force all but 'post-commit' hook to run in series. 'post-commit'
"is meant primarily for notification, and cannot affect the outcome of
`git commit`," so it is fine to run in parallel.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/commit.c |  2 +-
 commit.c         | 16 ++++++++++------
 sequencer.c      |  2 +-
 3 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index f1aafd67d4..dad4e56544 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1045,7 +1045,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (!no_verify && find_hook("pre-commit")) {
+	if (!no_verify && hook_exists("pre-commit")) {
 		/*
 		 * Re-read the index as pre-commit hook could have updated it,
 		 * and write it out as a tree.  We must do this before we invoke
diff --git a/commit.c b/commit.c
index 8ea55a447f..e8147a88fc 100644
--- a/commit.c
+++ b/commit.c
@@ -21,6 +21,7 @@
 #include "commit-reach.h"
 #include "run-command.h"
 #include "shallow.h"
+#include "hook.h"
 
 static struct commit_extra_header *read_commit_extra_header_lines(const char *buf, size_t len, const char **);
 
@@ -1698,22 +1699,25 @@ size_t ignore_non_trailer(const char *buf, size_t len)
 int run_commit_hook(int editor_is_used, const char *index_file,
 		    const char *name, ...)
 {
-	struct strvec hook_env = STRVEC_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	va_list args;
+	const char *arg;
 	int ret;
-
-	strvec_pushf(&hook_env, "GIT_INDEX_FILE=%s", index_file);
+	strvec_pushf(&opt.env, "GIT_INDEX_FILE=%s", index_file);
 
 	/*
 	 * Let the hook know that no editor will be launched.
 	 */
 	if (!editor_is_used)
-		strvec_push(&hook_env, "GIT_EDITOR=:");
+		strvec_push(&opt.env, "GIT_EDITOR=:");
 
 	va_start(args, name);
-	ret = run_hook_ve(hook_env.v, name, args);
+	while ((arg = va_arg(args, const char *)))
+		strvec_push(&opt.args, arg);
 	va_end(args);
-	strvec_clear(&hook_env);
+
+	ret = run_hooks(name, &opt);
+	run_hooks_opt_clear(&opt);
 
 	return ret;
 }
diff --git a/sequencer.c b/sequencer.c
index 3de479f90e..8f46984ffb 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1446,7 +1446,7 @@ static int try_to_commit(struct repository *r,
 		}
 	}
 
-	if (find_hook("prepare-commit-msg")) {
+	if (hook_exists("prepare-commit-msg")) {
 		res = run_prepare_commit_msg_hook(r, msg, hook_commit);
 		if (res)
 			goto out;
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 11/27] read-cache: convert post-index-change hook to use config
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (9 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 10/27] commit: use hook.h to execute hooks Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 12/27] receive-pack: convert push-to-checkout hook to hook.h Ævar Arnfjörð Bjarmason
                             ` (18 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

By using hook.h instead of run-command.h to run, post-index-change hooks
can now be specified in the config in addition to the hookdir.
post-index-change is not run anywhere besides in read-cache.c.

This removes the last direct user of run_hook_ve(), so we can make the
function static now. It'll be removed entirely soon.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 read-cache.c  | 11 ++++++++---
 run-command.c |  2 +-
 run-command.h |  1 -
 3 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/read-cache.c b/read-cache.c
index af5b97104c..f801313cc9 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -3063,6 +3063,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 {
 	int ret;
 	int was_full = !istate->sparse_index;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
 
 	ret = convert_to_sparse(istate);
 
@@ -3091,9 +3092,13 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 	else
 		ret = close_lock_file_gently(lock);
 
-	run_hook_le(NULL, "post-index-change",
-			istate->updated_workdir ? "1" : "0",
-			istate->updated_skipworktree ? "1" : "0", NULL);
+	strvec_pushl(&hook_opt.args,
+		     istate->updated_workdir ? "1" : "0",
+		     istate->updated_skipworktree ? "1" : "0",
+		     NULL);
+	run_hooks("post-index-change", &hook_opt);
+	run_hooks_opt_clear(&hook_opt);
+
 	istate->updated_workdir = 0;
 	istate->updated_skipworktree = 0;
 
diff --git a/run-command.c b/run-command.c
index 82fdf29656..eecdef5a0c 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1321,7 +1321,7 @@ int async_with_fork(void)
 #endif
 }
 
-int run_hook_ve(const char *const *env, const char *name, va_list args)
+static int run_hook_ve(const char *const *env, const char *name, va_list args)
 {
 	struct child_process hook = CHILD_PROCESS_INIT;
 	const char *p;
diff --git a/run-command.h b/run-command.h
index b58531a7eb..24ab5d63c4 100644
--- a/run-command.h
+++ b/run-command.h
@@ -216,7 +216,6 @@ int run_command(struct child_process *);
  */
 LAST_ARG_MUST_BE_NULL
 int run_hook_le(const char *const *env, const char *name, ...);
-int run_hook_ve(const char *const *env, const char *name, va_list args);
 
 /*
  * Trigger an auto-gc
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 12/27] receive-pack: convert push-to-checkout hook to hook.h
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (10 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 11/27] read-cache: convert post-index-change hook to use config Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 13/27] run-command: remove old run_hook_{le,ve}() hook API Ævar Arnfjörð Bjarmason
                             ` (17 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

By using hook.h instead of run-command.h to invoke push-to-checkout,
hooks can now be specified in the config as well as in the hookdir.
push-to-checkout is not called anywhere but in builtin/receive-pack.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/receive-pack.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 1e0e04c62f..5248228ebf 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1436,12 +1436,18 @@ static const char *push_to_checkout(unsigned char *hash,
 				    struct strvec *env,
 				    const char *work_tree)
 {
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
-	if (run_hook_le(env->v, push_to_checkout_hook,
-			hash_to_hex(hash), NULL))
+	strvec_pushv(&opt.env, env->v);
+	strvec_push(&opt.args, hash_to_hex(hash));
+	if (run_hooks(push_to_checkout_hook, &opt)) {
+		run_hooks_opt_clear(&opt);
 		return "push-to-checkout hook declined";
-	else
+	} else {
+		run_hooks_opt_clear(&opt);
 		return NULL;
+	}
 }
 
 static const char *update_worktree(unsigned char *sha1, const struct worktree *worktree)
@@ -1465,7 +1471,7 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 
 	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
 
-	if (!find_hook(push_to_checkout_hook))
+	if (!hook_exists(push_to_checkout_hook))
 		retval = push_to_deploy(sha1, &env, work_tree);
 	else
 		retval = push_to_checkout(sha1, &env, work_tree);
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 13/27] run-command: remove old run_hook_{le,ve}() hook API
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (11 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 12/27] receive-pack: convert push-to-checkout hook to hook.h Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 14/27] run-command: allow stdin for run_processes_parallel Ævar Arnfjörð Bjarmason
                             ` (16 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

The new hook.h library has replaced all run-command.h hook-related
functionality. So let's delete this dead code.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 run-command.c | 32 --------------------------------
 run-command.h | 16 ----------------
 2 files changed, 48 deletions(-)

diff --git a/run-command.c b/run-command.c
index eecdef5a0c..95c950a4a2 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1321,38 +1321,6 @@ int async_with_fork(void)
 #endif
 }
 
-static int run_hook_ve(const char *const *env, const char *name, va_list args)
-{
-	struct child_process hook = CHILD_PROCESS_INIT;
-	const char *p;
-
-	p = find_hook(name);
-	if (!p)
-		return 0;
-
-	strvec_push(&hook.args, p);
-	while ((p = va_arg(args, const char *)))
-		strvec_push(&hook.args, p);
-	hook.env = env;
-	hook.no_stdin = 1;
-	hook.stdout_to_stderr = 1;
-	hook.trace2_hook_name = name;
-
-	return run_command(&hook);
-}
-
-int run_hook_le(const char *const *env, const char *name, ...)
-{
-	va_list args;
-	int ret;
-
-	va_start(args, name);
-	ret = run_hook_ve(env, name, args);
-	va_end(args);
-
-	return ret;
-}
-
 struct io_pump {
 	/* initialized by caller */
 	int fd;
diff --git a/run-command.h b/run-command.h
index 24ab5d63c4..748d4fc2a7 100644
--- a/run-command.h
+++ b/run-command.h
@@ -201,22 +201,6 @@ int finish_command_in_signal(struct child_process *);
  */
 int run_command(struct child_process *);
 
-/**
- * Run a hook.
- * The first argument is a pathname to an index file, or NULL
- * if the hook uses the default index file or no index is needed.
- * The second argument is the name of the hook.
- * The further arguments correspond to the hook arguments.
- * The last argument has to be NULL to terminate the arguments list.
- * If the hook does not exist or is not executable, the return
- * value will be zero.
- * If it is executable, the hook will be executed and the exit
- * status of the hook is returned.
- * On execution, .stdout_to_stderr and .no_stdin will be set.
- */
-LAST_ARG_MUST_BE_NULL
-int run_hook_le(const char *const *env, const char *name, ...);
-
 /*
  * Trigger an auto-gc
  */
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 14/27] run-command: allow stdin for run_processes_parallel
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (12 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 13/27] run-command: remove old run_hook_{le,ve}() hook API Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 15/27] hook: support passing stdin to hooks Ævar Arnfjörð Bjarmason
                             ` (15 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

While it makes sense not to inherit stdin from the parent process to
avoid deadlocking, it's not necessary to completely ban stdin to
children. An informed user should be able to configure stdin safely. By
setting `some_child.process.no_stdin=1` before calling `get_next_task()`
we provide a reasonable default behavior but enable users to set up
stdin streaming for themselves during the callback.

`some_child.process.stdout_to_stderr`, however, remains unmodifiable by
`get_next_task()` - the rest of the run_processes_parallel() API depends
on child output in stderr.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 run-command.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/run-command.c b/run-command.c
index 95c950a4a2..0bf771845e 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1628,6 +1628,14 @@ static int pp_start_one(struct parallel_processes *pp)
 	if (i == pp->max_processes)
 		BUG("bookkeeping is hard");
 
+	/*
+	 * By default, do not inherit stdin from the parent process - otherwise,
+	 * all children would share stdin! Users may overwrite this to provide
+	 * something to the child's stdin by having their 'get_next_task'
+	 * callback assign 0 to .no_stdin and an appropriate integer to .in.
+	 */
+	pp->children[i].process.no_stdin = 1;
+
 	code = pp->get_next_task(&pp->children[i].process,
 				 &pp->children[i].err,
 				 pp->data,
@@ -1639,7 +1647,6 @@ static int pp_start_one(struct parallel_processes *pp)
 	}
 	pp->children[i].process.err = -1;
 	pp->children[i].process.stdout_to_stderr = 1;
-	pp->children[i].process.no_stdin = 1;
 
 	if (start_command(&pp->children[i].process)) {
 		code = pp->start_failure(&pp->children[i].err,
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 15/27] hook: support passing stdin to hooks
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (13 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 14/27] run-command: allow stdin for run_processes_parallel Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 16/27] am: convert 'post-rewrite' hook to hook.h Ævar Arnfjörð Bjarmason
                             ` (14 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Some hooks (such as post-rewrite) need to take input via stdin.
Previously, callers provided stdin to hooks by setting
run-command.h:child_process.in, which takes a FD. Callers would open the
file in question themselves before calling run-command(). However, since
we will now need to seek to the front of the file and read it again for
every hook which runs, hook.h:run_command() takes a path and handles FD
management itself. Since this file is opened for read only, it should
not prevent later parallel execution support.

On the frontend, this is supported by asking for a file path, rather
than by reading stdin. Reading directly from stdin would involve caching
the entire stdin (to memory or to disk) and reading it back from the
beginning to each hook. We'd want to support cases like insufficient
memory or storage for the file. While this may prove useful later, for
now the path of least resistance is to just ask the user to make this
interim file themselves.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-hook.txt |  7 ++++++-
 builtin/hook.c             |  5 ++++-
 hook.c                     |  9 ++++++++-
 hook.h                     |  2 ++
 t/t1800-hook.sh            | 18 ++++++++++++++++++
 5 files changed, 38 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 097fb9de63..fa68c1f391 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -8,7 +8,7 @@ git-hook - run git hooks
 SYNOPSIS
 --------
 [verse]
-'git hook' run [--ignore-missing] <hook-name> [-- <hook-args>]
+'git hook' run [--to-stdin=<path>] [--ignore-missing] <hook-name> [-- <hook-args>]
 
 DESCRIPTION
 -----------
@@ -32,6 +32,11 @@ what those are.
 OPTIONS
 -------
 
+--to-stdin::
+	For "run"; Specify a file which will be streamed into the
+	hook's stdin. The hook will receive the entire file from
+	beginning to EOF.
+
 --ignore-missing::
 	Ignore any missing hook by quietly returning zero. Used for
 	tools that want to do a blind one-shot run of a hook that may
diff --git a/builtin/hook.c b/builtin/hook.c
index 47e0de7bbc..169a8dd08f 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -8,12 +8,13 @@
 
 static const char * const builtin_hook_usage[] = {
 	N_("git hook <command> [...]"),
-	N_("git hook run <hook-name> [-- <hook-args>]"),
+	N_("git hook run [<args>] <hook-name> [-- <hook-args>]"),
 	NULL
 };
 
 static const char * const builtin_hook_run_usage[] = {
 	N_("git hook run <hook-name> [-- <hook-args>]"),
+	N_("git hook run [--to-stdin=<path>] <hook-name> [-- <hook-args>]"),
 	NULL
 };
 
@@ -29,6 +30,8 @@ static int run(int argc, const char **argv, const char *prefix)
 	struct option run_options[] = {
 		OPT_BOOL(0, "ignore-missing", &ignore_missing,
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
+		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
+			   N_("file to read into hooks' stdin")),
 		OPT_END(),
 	};
 
diff --git a/hook.c b/hook.c
index cb39404ac8..cab9cced82 100644
--- a/hook.c
+++ b/hook.c
@@ -85,7 +85,14 @@ static int pick_next_hook(struct child_process *cp,
 	struct hook_cb_data *hook_cb = pp_cb;
 	struct hook *run_me = hook_cb->run_me;
 
-	cp->no_stdin = 1;
+
+	/* reopen the file for stdin; run_command closes it. */
+	if (hook_cb->options->path_to_stdin) {
+		cp->no_stdin = 0;
+		cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY);
+	} else {
+		cp->no_stdin = 1;
+	}
 	cp->env = hook_cb->options->env.v;
 	cp->stdout_to_stderr = 1;
 	cp->trace2_hook_name = hook_cb->hook_name;
diff --git a/hook.h b/hook.h
index 8bd0fc8d1f..c4ac054ee3 100644
--- a/hook.h
+++ b/hook.h
@@ -45,6 +45,8 @@ struct run_hooks_opt
 	/* Path to initial working directory for subprocess */
 	const char *dir;
 
+	/* Path to file which should be piped to stdin for each hook */
+	const char *path_to_stdin;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
index 542e551628..f3510379e0 100755
--- a/t/t1800-hook.sh
+++ b/t/t1800-hook.sh
@@ -133,4 +133,22 @@ test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
 	test_cmp expect actual
 '
 
+test_expect_success 'stdin to hooks' '
+	write_script .git/hooks/test-hook <<-\EOF &&
+	echo BEGIN stdin
+	cat
+	echo END stdin
+	EOF
+
+	cat >expect <<-EOF &&
+	BEGIN stdin
+	hello
+	END stdin
+	EOF
+
+	echo hello >input &&
+	git hook run --to-stdin=input test-hook 2>actual &&
+	test_cmp expect actual
+'
+
 test_done
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 16/27] am: convert 'post-rewrite' hook to hook.h
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (14 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 15/27] hook: support passing stdin to hooks Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 17/27] run-command: add stdin callback for parallelization Ævar Arnfjörð Bjarmason
                             ` (13 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/am.c | 18 +++++-------------
 1 file changed, 5 insertions(+), 13 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index 9e9c1b5e9f..6e4f9c8036 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -467,23 +467,15 @@ static int run_applypatch_msg_hook(struct am_state *state)
  */
 static int run_post_rewrite_hook(const struct am_state *state)
 {
-	struct child_process cp = CHILD_PROCESS_INIT;
-	const char *hook = find_hook("post-rewrite");
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	int ret;
 
-	if (!hook)
-		return 0;
-
-	strvec_push(&cp.args, hook);
-	strvec_push(&cp.args, "rebase");
-
-	cp.in = xopen(am_path(state, "rewritten"), O_RDONLY);
-	cp.stdout_to_stderr = 1;
-	cp.trace2_hook_name = "post-rewrite";
+	strvec_push(&opt.args, "rebase");
+	opt.path_to_stdin = am_path(state, "rewritten");
 
-	ret = run_command(&cp);
+	ret = run_hooks("post-rewrite", &opt);
 
-	close(cp.in);
+	run_hooks_opt_clear(&opt);
 	return ret;
 }
 
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 17/27] run-command: add stdin callback for parallelization
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (15 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 16/27] am: convert 'post-rewrite' hook to hook.h Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 18/27] hook: provide stdin by string_list or callback Ævar Arnfjörð Bjarmason
                             ` (12 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

If a user of the run_processes_parallel() API wants to pipe a large
amount of information to stdin of each parallel command, that
information could exceed the buffer of the pipe allocated for that
process's stdin.  Generally this is solved by repeatedly writing to
child_process.in between calls to start_command() and finish_command();
run_processes_parallel() did not provide users an opportunity to access
child_process at that time.

Because the data might be extremely large (for example, a list of all
refs received during a push from a client) simply taking a string_list
or strbuf is not as scalable as using a callback; the rest of the
run_processes_parallel() API also uses callbacks, so making this feature
match the rest of the API reduces mental load on the user.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/fetch.c             |  1 +
 builtin/submodule--helper.c |  2 +-
 hook.c                      |  1 +
 run-command.c               | 54 +++++++++++++++++++++++++++++++++++--
 run-command.h               | 17 +++++++++++-
 submodule.c                 |  1 +
 t/helper/test-run-command.c | 31 ++++++++++++++++++---
 t/t0061-run-command.sh      | 30 +++++++++++++++++++++
 8 files changed, 129 insertions(+), 8 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index 9191620e50..3d8f04b392 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1817,6 +1817,7 @@ static int fetch_multiple(struct string_list *list, int max_children)
 		result = run_processes_parallel_tr2(max_children,
 						    &fetch_next_remote,
 						    &fetch_failed_to_start,
+						    NULL,
 						    &fetch_finished,
 						    &state,
 						    "fetch", "parallel/fetch");
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index ae6174ab05..818494dd18 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2295,7 +2295,7 @@ static int update_submodules(struct submodule_update_clone *suc)
 	int i;
 
 	run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task,
-				   update_clone_start_failure,
+				   update_clone_start_failure, NULL,
 				   update_clone_task_finished, suc, "submodule",
 				   "parallel/update");
 
diff --git a/hook.c b/hook.c
index cab9cced82..a012c3d458 100644
--- a/hook.c
+++ b/hook.c
@@ -164,6 +164,7 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 	run_processes_parallel_tr2(options->jobs,
 				   pick_next_hook,
 				   notify_start_failure,
+				   NULL,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
diff --git a/run-command.c b/run-command.c
index 0bf771845e..3392640f17 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1493,6 +1493,7 @@ struct parallel_processes {
 
 	get_next_task_fn get_next_task;
 	start_failure_fn start_failure;
+	feed_pipe_fn feed_pipe;
 	task_finished_fn task_finished;
 
 	struct {
@@ -1520,6 +1521,13 @@ static int default_start_failure(struct strbuf *out,
 	return 0;
 }
 
+static int default_feed_pipe(struct strbuf *pipe,
+			     void *pp_cb,
+			     void *pp_task_cb)
+{
+	return 1;
+}
+
 static int default_task_finished(int result,
 				 struct strbuf *out,
 				 void *pp_cb,
@@ -1550,6 +1558,7 @@ static void pp_init(struct parallel_processes *pp,
 		    int n,
 		    get_next_task_fn get_next_task,
 		    start_failure_fn start_failure,
+		    feed_pipe_fn feed_pipe,
 		    task_finished_fn task_finished,
 		    void *data)
 {
@@ -1568,6 +1577,7 @@ static void pp_init(struct parallel_processes *pp,
 	pp->get_next_task = get_next_task;
 
 	pp->start_failure = start_failure ? start_failure : default_start_failure;
+	pp->feed_pipe = feed_pipe ? feed_pipe : default_feed_pipe;
 	pp->task_finished = task_finished ? task_finished : default_task_finished;
 
 	pp->nr_processes = 0;
@@ -1665,6 +1675,37 @@ static int pp_start_one(struct parallel_processes *pp)
 	return 0;
 }
 
+static void pp_buffer_stdin(struct parallel_processes *pp)
+{
+	int i;
+	struct strbuf sb = STRBUF_INIT;
+
+	/* Buffer stdin for each pipe. */
+	for (i = 0; i < pp->max_processes; i++) {
+		if (pp->children[i].state == GIT_CP_WORKING &&
+		    pp->children[i].process.in > 0) {
+			int done;
+			strbuf_reset(&sb);
+			done = pp->feed_pipe(&sb, pp->data,
+					      pp->children[i].data);
+			if (sb.len) {
+				if (write_in_full(pp->children[i].process.in,
+					      sb.buf, sb.len) < 0) {
+					if (errno != EPIPE)
+						die_errno("write");
+					done = 1;
+				}
+			}
+			if (done) {
+				close(pp->children[i].process.in);
+				pp->children[i].process.in = 0;
+			}
+		}
+	}
+
+	strbuf_release(&sb);
+}
+
 static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout)
 {
 	int i;
@@ -1729,6 +1770,7 @@ static int pp_collect_finished(struct parallel_processes *pp)
 		pp->nr_processes--;
 		pp->children[i].state = GIT_CP_FREE;
 		pp->pfd[i].fd = -1;
+		pp->children[i].process.in = 0;
 		child_process_init(&pp->children[i].process);
 
 		if (i != pp->output_owner) {
@@ -1762,6 +1804,7 @@ static int pp_collect_finished(struct parallel_processes *pp)
 int run_processes_parallel(int n,
 			   get_next_task_fn get_next_task,
 			   start_failure_fn start_failure,
+			   feed_pipe_fn feed_pipe,
 			   task_finished_fn task_finished,
 			   void *pp_cb)
 {
@@ -1770,7 +1813,9 @@ int run_processes_parallel(int n,
 	int spawn_cap = 4;
 	struct parallel_processes pp;
 
-	pp_init(&pp, n, get_next_task, start_failure, task_finished, pp_cb);
+	sigchain_push(SIGPIPE, SIG_IGN);
+
+	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, task_finished, pp_cb);
 	while (1) {
 		for (i = 0;
 		    i < spawn_cap && !pp.shutdown &&
@@ -1787,6 +1832,7 @@ int run_processes_parallel(int n,
 		}
 		if (!pp.nr_processes)
 			break;
+		pp_buffer_stdin(&pp);
 		pp_buffer_stderr(&pp, output_timeout);
 		pp_output(&pp);
 		code = pp_collect_finished(&pp);
@@ -1798,11 +1844,15 @@ int run_processes_parallel(int n,
 	}
 
 	pp_cleanup(&pp);
+
+	sigchain_pop(SIGPIPE);
+
 	return 0;
 }
 
 int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 			       start_failure_fn start_failure,
+			       feed_pipe_fn feed_pipe,
 			       task_finished_fn task_finished, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label)
 {
@@ -1812,7 +1862,7 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 				   ((n < 1) ? online_cpus() : n));
 
 	result = run_processes_parallel(n, get_next_task, start_failure,
-					task_finished, pp_cb);
+					feed_pipe, task_finished, pp_cb);
 
 	trace2_region_leave(tr2_category, tr2_label, NULL);
 
diff --git a/run-command.h b/run-command.h
index 748d4fc2a7..41e36d26cb 100644
--- a/run-command.h
+++ b/run-command.h
@@ -419,6 +419,20 @@ typedef int (*start_failure_fn)(struct strbuf *out,
 				void *pp_cb,
 				void *pp_task_cb);
 
+/**
+ * This callback is called repeatedly on every child process who requests
+ * start_command() to create a pipe by setting child_process.in < 0.
+ *
+ * pp_cb is the callback cookie as passed into run_processes_parallel, and
+ * pp_task_cb is the callback cookie as passed into get_next_task_fn.
+ * The contents of 'send' will be read into the pipe and passed to the pipe.
+ *
+ * Return nonzero to close the pipe.
+ */
+typedef int (*feed_pipe_fn)(struct strbuf *pipe,
+			    void *pp_cb,
+			    void *pp_task_cb);
+
 /**
  * This callback is called on every child process that finished processing.
  *
@@ -453,10 +467,11 @@ typedef int (*task_finished_fn)(int result,
 int run_processes_parallel(int n,
 			   get_next_task_fn,
 			   start_failure_fn,
+			   feed_pipe_fn,
 			   task_finished_fn,
 			   void *pp_cb);
 int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn,
-			       task_finished_fn, void *pp_cb,
+			       feed_pipe_fn, task_finished_fn, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label);
 
 #endif
diff --git a/submodule.c b/submodule.c
index 0b1d9c1dde..ea026a8195 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1645,6 +1645,7 @@ int fetch_populated_submodules(struct repository *r,
 	run_processes_parallel_tr2(max_parallel_jobs,
 				   get_next_submodule,
 				   fetch_start_failure,
+				   NULL,
 				   fetch_finish,
 				   &spf,
 				   "submodule", "parallel/fetch");
diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c
index 7ae03dc712..9348184d30 100644
--- a/t/helper/test-run-command.c
+++ b/t/helper/test-run-command.c
@@ -32,8 +32,13 @@ static int parallel_next(struct child_process *cp,
 		return 0;
 
 	strvec_pushv(&cp->args, d->argv);
+	cp->in = d->in;
+	cp->no_stdin = d->no_stdin;
 	strbuf_addstr(err, "preloaded output of a child\n");
 	number_callbacks++;
+
+	*task_cb = xmalloc(sizeof(int));
+	*(int*)(*task_cb) = 2;
 	return 1;
 }
 
@@ -55,6 +60,17 @@ static int task_finished(int result,
 	return 1;
 }
 
+static int test_stdin(struct strbuf *pipe, void *cb, void *task_cb)
+{
+	int *lines_remaining = task_cb;
+
+	if (*lines_remaining)
+		strbuf_addf(pipe, "sample stdin %d\n", --(*lines_remaining));
+
+	return !(*lines_remaining);
+}
+
+
 struct testsuite {
 	struct string_list tests, failed;
 	int next;
@@ -185,7 +201,7 @@ static int testsuite(int argc, const char **argv)
 		suite.tests.nr, max_jobs);
 
 	ret = run_processes_parallel(max_jobs, next_test, test_failed,
-				     test_finished, &suite);
+				     test_stdin, test_finished, &suite);
 
 	if (suite.failed.nr > 0) {
 		ret = 1;
@@ -413,15 +429,22 @@ int cmd__run_command(int argc, const char **argv)
 
 	if (!strcmp(argv[1], "run-command-parallel"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, &proc));
+					    NULL, NULL, NULL, &proc));
 
 	if (!strcmp(argv[1], "run-command-abort"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, task_finished, &proc));
+					    NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-no-jobs"))
 		exit(run_processes_parallel(jobs, no_job,
-					    NULL, task_finished, &proc));
+					    NULL, NULL, task_finished, &proc));
+
+	if (!strcmp(argv[1], "run-command-stdin")) {
+		proc.in = -1;
+		proc.no_stdin = 0;
+		exit (run_processes_parallel(jobs, parallel_next, NULL,
+					     test_stdin, NULL, &proc));
+	}
 
 	fprintf(stderr, "check usage\n");
 	return 1;
diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh
index 7d599675e3..87759482ad 100755
--- a/t/t0061-run-command.sh
+++ b/t/t0061-run-command.sh
@@ -143,6 +143,36 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai
 	test_cmp expect actual
 '
 
+cat >expect <<-EOF
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+EOF
+
+test_expect_success 'run_command listens to stdin' '
+	write_script stdin-script <<-\EOF &&
+	echo "listening for stdin:"
+	while read line; do
+		echo "$line"
+	done
+	EOF
+	test-tool run-command run-command-stdin 2 ./stdin-script 2>actual &&
+	test_cmp expect actual
+'
+
 cat >expect <<-EOF
 preloaded output of a child
 asking for a quick stop
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 18/27] hook: provide stdin by string_list or callback
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (16 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 17/27] run-command: add stdin callback for parallelization Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 19/27] hook: convert 'post-rewrite' hook in sequencer.c to hook.h Ævar Arnfjörð Bjarmason
                             ` (11 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

In cases where a hook requires only a small amount of information via
stdin, it should be simple for users to provide a string_list alone. But
in more complicated cases where the stdin is too large to hold in
memory, let's instead provide a callback the users can populate line
after line.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 hook.c | 32 +++++++++++++++++++++++++++++++-
 hook.h | 27 +++++++++++++++++++++++++++
 2 files changed, 58 insertions(+), 1 deletion(-)

diff --git a/hook.c b/hook.c
index a012c3d458..c4df787f13 100644
--- a/hook.c
+++ b/hook.c
@@ -77,6 +77,29 @@ void run_hooks_opt_clear(struct run_hooks_opt *o)
 	strvec_clear(&o->args);
 }
 
+int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb)
+{
+	int *item_idx;
+	struct hook *ctx = pp_task_cb;
+	struct hook_cb_data *hook_cb = pp_cb;
+	struct string_list *to_pipe = hook_cb->options->feed_pipe_ctx;
+
+	/* Bootstrap the state manager if necessary. */
+	if (!ctx->feed_pipe_cb_data) {
+		ctx->feed_pipe_cb_data = xmalloc(sizeof(unsigned int));
+		*(int*)ctx->feed_pipe_cb_data = 0;
+	}
+
+	item_idx = ctx->feed_pipe_cb_data;
+
+	if (*item_idx < to_pipe->nr) {
+		strbuf_addf(pipe, "%s\n", to_pipe->items[*item_idx].string);
+		(*item_idx)++;
+		return 0;
+	}
+	return 1;
+}
+
 static int pick_next_hook(struct child_process *cp,
 			  struct strbuf *out,
 			  void *pp_cb,
@@ -90,6 +113,10 @@ static int pick_next_hook(struct child_process *cp,
 	if (hook_cb->options->path_to_stdin) {
 		cp->no_stdin = 0;
 		cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY);
+	} else if (hook_cb->options->feed_pipe) {
+		/* ask for start_command() to make a pipe for us */
+		cp->in = -1;
+		cp->no_stdin = 0;
 	} else {
 		cp->no_stdin = 1;
 	}
@@ -164,7 +191,7 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 	run_processes_parallel_tr2(options->jobs,
 				   pick_next_hook,
 				   notify_start_failure,
-				   NULL,
+				   options->feed_pipe,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
@@ -182,6 +209,9 @@ int run_hooks(const char *hook_name, struct run_hooks_opt *options)
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
 
+	if (options->path_to_stdin && options->feed_pipe)
+		BUG("choose only one method to populate stdin");
+
 	hook_path = find_hook(hook_name);
 
 	/*
diff --git a/hook.h b/hook.h
index c4ac054ee3..b78a8e4805 100644
--- a/hook.h
+++ b/hook.h
@@ -19,6 +19,12 @@ int hook_exists(const char *hookname);
 struct hook {
 	/* The path to the hook */
 	const char *hook_path;
+
+	/*
+	 * Use this to keep state for your feed_pipe_fn if you are using
+	 * run_hooks_opt.feed_pipe. Otherwise, do not touch it.
+	 */
+	void *feed_pipe_cb_data;
 };
 
 struct run_hooks_opt
@@ -47,6 +53,19 @@ struct run_hooks_opt
 
 	/* Path to file which should be piped to stdin for each hook */
 	const char *path_to_stdin;
+
+	/*
+	 * Callback and state pointer to ask for more content to pipe to stdin.
+	 * Will be called repeatedly, for each hook. See
+	 * hook.c:pipe_from_stdin() for an example. Keep per-hook state in
+	 * hook.feed_pipe_cb_data (per process). Keep initialization context in
+	 * feed_pipe_ctx (shared by all processes).
+	 *
+	 * See 'pipe_from_string_list()' for info about how to specify a
+	 * string_list as the stdin input instead of writing your own handler.
+	 */
+	feed_pipe_fn feed_pipe;
+	void *feed_pipe_ctx;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
@@ -55,6 +74,14 @@ struct run_hooks_opt
 	.args = STRVEC_INIT, \
 }
 
+/*
+ * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
+ * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'.
+ * This will pipe each string in the list to stdin, separated by newlines.  (Do
+ * not inject your own newlines.)
+ */
+int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb);
+
 struct hook_cb_data {
 	/* rc reflects the cumulative failure state */
 	int rc;
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 19/27] hook: convert 'post-rewrite' hook in sequencer.c to hook.h
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (17 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 18/27] hook: provide stdin by string_list or callback Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 20/27] transport: convert pre-push hook to use config Ævar Arnfjörð Bjarmason
                             ` (10 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

By using 'hook.h' for 'post-rewrite', we simplify hook invocations by
not needing to put together our own 'struct child_process'.

The signal handling that's being removed by this commit now takes
place in run-command.h:run_processes_parallel(), so it is OK to remove
them here.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 sequencer.c | 81 ++++++++++++++++++++++-------------------------------
 1 file changed, 34 insertions(+), 47 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 8f46984ffb..ec2761e47d 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -35,6 +35,7 @@
 #include "commit-reach.h"
 #include "rebase-interactive.h"
 #include "reset.h"
+#include "string-list.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -1147,33 +1148,28 @@ int update_head_with_reflog(const struct commit *old_head,
 static int run_rewrite_hook(const struct object_id *oldoid,
 			    const struct object_id *newoid)
 {
-	struct child_process proc = CHILD_PROCESS_INIT;
-	const char *argv[3];
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct strbuf tmp = STRBUF_INIT;
+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int code;
-	struct strbuf sb = STRBUF_INIT;
 
-	argv[0] = find_hook("post-rewrite");
-	if (!argv[0])
-		return 0;
+	strvec_push(&opt.args, "amend");
 
-	argv[1] = "amend";
-	argv[2] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = "post-rewrite";
-
-	code = start_command(&proc);
-	if (code)
-		return code;
-	strbuf_addf(&sb, "%s %s\n", oid_to_hex(oldoid), oid_to_hex(newoid));
-	sigchain_push(SIGPIPE, SIG_IGN);
-	write_in_full(proc.in, sb.buf, sb.len);
-	close(proc.in);
-	strbuf_release(&sb);
-	sigchain_pop(SIGPIPE);
-	return finish_command(&proc);
+	strbuf_addf(&tmp,
+		    "%s %s",
+		    oid_to_hex(oldoid),
+		    oid_to_hex(newoid));
+	string_list_append(&to_stdin, tmp.buf);
+
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
+
+	code = run_hooks("post-rewrite", &opt);
+
+	run_hooks_opt_clear(&opt);
+	strbuf_release(&tmp);
+	string_list_clear(&to_stdin, 0);
+	return code;
 }
 
 void commit_post_rewrite(struct repository *r,
@@ -4527,30 +4523,21 @@ static int pick_commits(struct repository *r,
 		flush_rewritten_pending();
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
-			struct child_process child = CHILD_PROCESS_INIT;
-			const char *post_rewrite_hook =
-				find_hook("post-rewrite");
-
-			child.in = open(rebase_path_rewritten_list(), O_RDONLY);
-			child.git_cmd = 1;
-			strvec_push(&child.args, "notes");
-			strvec_push(&child.args, "copy");
-			strvec_push(&child.args, "--for-rewrite=rebase");
+			struct child_process notes_cp = CHILD_PROCESS_INIT;
+			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+
+			notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY);
+			notes_cp.git_cmd = 1;
+			strvec_push(&notes_cp.args, "notes");
+			strvec_push(&notes_cp.args, "copy");
+			strvec_push(&notes_cp.args, "--for-rewrite=rebase");
 			/* we don't care if this copying failed */
-			run_command(&child);
-
-			if (post_rewrite_hook) {
-				struct child_process hook = CHILD_PROCESS_INIT;
-
-				hook.in = open(rebase_path_rewritten_list(),
-					O_RDONLY);
-				hook.stdout_to_stderr = 1;
-				hook.trace2_hook_name = "post-rewrite";
-				strvec_push(&hook.args, post_rewrite_hook);
-				strvec_push(&hook.args, "rebase");
-				/* we don't care if this hook failed */
-				run_command(&hook);
-			}
+			run_command(&notes_cp);
+
+			hook_opt.path_to_stdin = rebase_path_rewritten_list();
+			strvec_push(&hook_opt.args, "rebase");
+			run_hooks("post-rewrite", &hook_opt);
+			run_hooks_opt_clear(&hook_opt);
 		}
 		apply_autostash(rebase_path_autostash());
 
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 20/27] transport: convert pre-push hook to use config
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (18 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 19/27] hook: convert 'post-rewrite' hook in sequencer.c to hook.h Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 21/27] reference-transaction: use hook.h to run hooks Ævar Arnfjörð Bjarmason
                             ` (9 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

By using the hook.h:run_hooks API, pre-push hooks can be specified in
the config as well as in the hookdir.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 transport.c | 57 ++++++++++++++---------------------------------------
 1 file changed, 15 insertions(+), 42 deletions(-)

diff --git a/transport.c b/transport.c
index 2ed270171f..9969ed2cdd 100644
--- a/transport.c
+++ b/transport.c
@@ -1199,31 +1199,14 @@ static void die_with_unpushed_submodules(struct string_list *needs_pushing)
 static int run_pre_push_hook(struct transport *transport,
 			     struct ref *remote_refs)
 {
-	int ret = 0, x;
+	int ret = 0;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct strbuf tmp = STRBUF_INIT;
 	struct ref *r;
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct strbuf buf;
-	const char *argv[4];
-
-	if (!(argv[0] = find_hook("pre-push")))
-		return 0;
-
-	argv[1] = transport->remote->name;
-	argv[2] = transport->url;
-	argv[3] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.trace2_hook_name = "pre-push";
-
-	if (start_command(&proc)) {
-		finish_command(&proc);
-		return -1;
-	}
-
-	sigchain_push(SIGPIPE, SIG_IGN);
+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 
-	strbuf_init(&buf, 256);
+	strvec_push(&opt.args, transport->remote->name);
+	strvec_push(&opt.args, transport->url);
 
 	for (r = remote_refs; r; r = r->next) {
 		if (!r->peer_ref) continue;
@@ -1232,30 +1215,20 @@ static int run_pre_push_hook(struct transport *transport,
 		if (r->status == REF_STATUS_REJECT_REMOTE_UPDATED) continue;
 		if (r->status == REF_STATUS_UPTODATE) continue;
 
-		strbuf_reset(&buf);
-		strbuf_addf( &buf, "%s %s %s %s\n",
+		strbuf_reset(&tmp);
+		strbuf_addf(&tmp, "%s %s %s %s",
 			 r->peer_ref->name, oid_to_hex(&r->new_oid),
 			 r->name, oid_to_hex(&r->old_oid));
-
-		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
-			/* We do not mind if a hook does not read all refs. */
-			if (errno != EPIPE)
-				ret = -1;
-			break;
-		}
+		string_list_append(&to_stdin, tmp.buf);
 	}
 
-	strbuf_release(&buf);
-
-	x = close(proc.in);
-	if (!ret)
-		ret = x;
-
-	sigchain_pop(SIGPIPE);
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
 
-	x = finish_command(&proc);
-	if (!ret)
-		ret = x;
+	ret = run_hooks("pre-push", &opt);
+	run_hooks_opt_clear(&opt);
+	strbuf_release(&tmp);
+	string_list_clear(&to_stdin, 0);
 
 	return ret;
 }
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 21/27] reference-transaction: use hook.h to run hooks
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (19 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 20/27] transport: convert pre-push hook to use config Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 22/27] run-command: allow capturing of collated output Ævar Arnfjörð Bjarmason
                             ` (8 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c | 42 +++++++++++++++---------------------------
 1 file changed, 15 insertions(+), 27 deletions(-)

diff --git a/refs.c b/refs.c
index 59be29cf08..1149e7e7dc 100644
--- a/refs.c
+++ b/refs.c
@@ -2062,47 +2062,35 @@ int ref_update_reject_duplicates(struct string_list *refnames,
 static int run_transaction_hook(struct ref_transaction *transaction,
 				const char *state)
 {
-	struct child_process proc = CHILD_PROCESS_INIT;
 	struct strbuf buf = STRBUF_INIT;
-	const char *hook;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int ret = 0, i;
+	char o[GIT_MAX_HEXSZ + 1], n[GIT_MAX_HEXSZ + 1];
 
-	hook = find_hook("reference-transaction");
-	if (!hook)
+	if (!hook_exists("reference-transaction"))
 		return ret;
 
-	strvec_pushl(&proc.args, hook, state, NULL);
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = "reference-transaction";
-
-	ret = start_command(&proc);
-	if (ret)
-		return ret;
-
-	sigchain_push(SIGPIPE, SIG_IGN);
+	strvec_push(&opt.args, state);
 
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		oid_to_hex_r(o, &update->old_oid);
+		oid_to_hex_r(n, &update->new_oid);
 
 		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
-			    oid_to_hex(&update->old_oid),
-			    oid_to_hex(&update->new_oid),
-			    update->refname);
-
-		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
-			if (errno != EPIPE)
-				ret = -1;
-			break;
-		}
+		strbuf_addf(&buf, "%s %s %s", o, n, update->refname);
+		string_list_append(&to_stdin, buf.buf);
 	}
 
-	close(proc.in);
-	sigchain_pop(SIGPIPE);
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
+
+	ret = run_hooks("reference-transaction", &opt);
+	run_hooks_opt_clear(&opt);
 	strbuf_release(&buf);
+	string_list_clear(&to_stdin, 0);
 
-	ret |= finish_command(&proc);
 	return ret;
 }
 
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 22/27] run-command: allow capturing of collated output
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (20 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 21/27] reference-transaction: use hook.h to run hooks Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 23/27] hooks: allow callers to capture output Ævar Arnfjörð Bjarmason
                             ` (7 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Some callers, for example server-side hooks which wish to relay hook
output to clients across a transport, want to capture what would
normally print to stderr and do something else with it. Allow that via a
callback.

By calling the callback regardless of whether there's output available,
we allow clients to send e.g. a keepalive if necessary.

Because we expose a strbuf, not a fd or FILE*, there's no need to create
a temporary pipe or similar - we can just skip the print to stderr and
instead hand it to the caller.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/fetch.c             |  2 +-
 builtin/submodule--helper.c |  2 +-
 hook.c                      |  1 +
 run-command.c               | 33 +++++++++++++++++++++++++--------
 run-command.h               | 18 +++++++++++++++++-
 submodule.c                 |  2 +-
 t/helper/test-run-command.c | 25 ++++++++++++++++++++-----
 t/t0061-run-command.sh      |  7 +++++++
 8 files changed, 73 insertions(+), 17 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index 3d8f04b392..0595b88b7c 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1817,7 +1817,7 @@ static int fetch_multiple(struct string_list *list, int max_children)
 		result = run_processes_parallel_tr2(max_children,
 						    &fetch_next_remote,
 						    &fetch_failed_to_start,
-						    NULL,
+						    NULL, NULL,
 						    &fetch_finished,
 						    &state,
 						    "fetch", "parallel/fetch");
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 818494dd18..69782f6204 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2295,7 +2295,7 @@ static int update_submodules(struct submodule_update_clone *suc)
 	int i;
 
 	run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task,
-				   update_clone_start_failure, NULL,
+				   update_clone_start_failure, NULL, NULL,
 				   update_clone_task_finished, suc, "submodule",
 				   "parallel/update");
 
diff --git a/hook.c b/hook.c
index c4df787f13..2dca969f9e 100644
--- a/hook.c
+++ b/hook.c
@@ -192,6 +192,7 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
+				   NULL,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
diff --git a/run-command.c b/run-command.c
index 3392640f17..4a1a7a1082 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1494,6 +1494,7 @@ struct parallel_processes {
 	get_next_task_fn get_next_task;
 	start_failure_fn start_failure;
 	feed_pipe_fn feed_pipe;
+	consume_sideband_fn consume_sideband;
 	task_finished_fn task_finished;
 
 	struct {
@@ -1559,6 +1560,7 @@ static void pp_init(struct parallel_processes *pp,
 		    get_next_task_fn get_next_task,
 		    start_failure_fn start_failure,
 		    feed_pipe_fn feed_pipe,
+		    consume_sideband_fn consume_sideband,
 		    task_finished_fn task_finished,
 		    void *data)
 {
@@ -1579,6 +1581,7 @@ static void pp_init(struct parallel_processes *pp,
 	pp->start_failure = start_failure ? start_failure : default_start_failure;
 	pp->feed_pipe = feed_pipe ? feed_pipe : default_feed_pipe;
 	pp->task_finished = task_finished ? task_finished : default_task_finished;
+	pp->consume_sideband = consume_sideband;
 
 	pp->nr_processes = 0;
 	pp->output_owner = 0;
@@ -1615,7 +1618,10 @@ static void pp_cleanup(struct parallel_processes *pp)
 	 * When get_next_task added messages to the buffer in its last
 	 * iteration, the buffered output is non empty.
 	 */
-	strbuf_write(&pp->buffered_output, stderr);
+	if (pp->consume_sideband)
+		pp->consume_sideband(&pp->buffered_output, pp->data);
+	else
+		strbuf_write(&pp->buffered_output, stderr);
 	strbuf_release(&pp->buffered_output);
 
 	sigchain_pop_common();
@@ -1736,9 +1742,13 @@ static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout)
 static void pp_output(struct parallel_processes *pp)
 {
 	int i = pp->output_owner;
+
 	if (pp->children[i].state == GIT_CP_WORKING &&
 	    pp->children[i].err.len) {
-		strbuf_write(&pp->children[i].err, stderr);
+		if (pp->consume_sideband)
+			pp->consume_sideband(&pp->children[i].err, pp->data);
+		else
+			strbuf_write(&pp->children[i].err, stderr);
 		strbuf_reset(&pp->children[i].err);
 	}
 }
@@ -1777,11 +1787,15 @@ static int pp_collect_finished(struct parallel_processes *pp)
 			strbuf_addbuf(&pp->buffered_output, &pp->children[i].err);
 			strbuf_reset(&pp->children[i].err);
 		} else {
-			strbuf_write(&pp->children[i].err, stderr);
+			/* Output errors, then all other finished child processes */
+			if (pp->consume_sideband) {
+				pp->consume_sideband(&pp->children[i].err, pp->data);
+				pp->consume_sideband(&pp->buffered_output, pp->data);
+			} else {
+				strbuf_write(&pp->children[i].err, stderr);
+				strbuf_write(&pp->buffered_output, stderr);
+			}
 			strbuf_reset(&pp->children[i].err);
-
-			/* Output all other finished child processes */
-			strbuf_write(&pp->buffered_output, stderr);
 			strbuf_reset(&pp->buffered_output);
 
 			/*
@@ -1805,6 +1819,7 @@ int run_processes_parallel(int n,
 			   get_next_task_fn get_next_task,
 			   start_failure_fn start_failure,
 			   feed_pipe_fn feed_pipe,
+			   consume_sideband_fn consume_sideband,
 			   task_finished_fn task_finished,
 			   void *pp_cb)
 {
@@ -1815,7 +1830,7 @@ int run_processes_parallel(int n,
 
 	sigchain_push(SIGPIPE, SIG_IGN);
 
-	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, task_finished, pp_cb);
+	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, consume_sideband, task_finished, pp_cb);
 	while (1) {
 		for (i = 0;
 		    i < spawn_cap && !pp.shutdown &&
@@ -1853,6 +1868,7 @@ int run_processes_parallel(int n,
 int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 			       start_failure_fn start_failure,
 			       feed_pipe_fn feed_pipe,
+			       consume_sideband_fn consume_sideband,
 			       task_finished_fn task_finished, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label)
 {
@@ -1862,7 +1878,8 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 				   ((n < 1) ? online_cpus() : n));
 
 	result = run_processes_parallel(n, get_next_task, start_failure,
-					feed_pipe, task_finished, pp_cb);
+					feed_pipe, consume_sideband,
+					task_finished, pp_cb);
 
 	trace2_region_leave(tr2_category, tr2_label, NULL);
 
diff --git a/run-command.h b/run-command.h
index 41e36d26cb..7150da851a 100644
--- a/run-command.h
+++ b/run-command.h
@@ -433,6 +433,20 @@ typedef int (*feed_pipe_fn)(struct strbuf *pipe,
 			    void *pp_cb,
 			    void *pp_task_cb);
 
+/**
+ * If this callback is provided, instead of collating process output to stderr,
+ * they will be collated into a new pipe. consume_sideband_fn will be called
+ * repeatedly. When output is available on that pipe, it will be contained in
+ * 'output'. But it will be called with an empty 'output' too, to allow for
+ * keepalives or similar operations if necessary.
+ *
+ * pp_cb is the callback cookie as passed into run_processes_parallel.
+ *
+ * Since this callback is provided with the collated output, no task cookie is
+ * provided.
+ */
+typedef void (*consume_sideband_fn)(struct strbuf *output, void *pp_cb);
+
 /**
  * This callback is called on every child process that finished processing.
  *
@@ -468,10 +482,12 @@ int run_processes_parallel(int n,
 			   get_next_task_fn,
 			   start_failure_fn,
 			   feed_pipe_fn,
+			   consume_sideband_fn,
 			   task_finished_fn,
 			   void *pp_cb);
 int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn,
-			       feed_pipe_fn, task_finished_fn, void *pp_cb,
+			       feed_pipe_fn, consume_sideband_fn,
+			       task_finished_fn, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label);
 
 #endif
diff --git a/submodule.c b/submodule.c
index ea026a8195..7fe0c8f7c9 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1645,7 +1645,7 @@ int fetch_populated_submodules(struct repository *r,
 	run_processes_parallel_tr2(max_parallel_jobs,
 				   get_next_submodule,
 				   fetch_start_failure,
-				   NULL,
+				   NULL, NULL,
 				   fetch_finish,
 				   &spf,
 				   "submodule", "parallel/fetch");
diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c
index 9348184d30..d53db6d11c 100644
--- a/t/helper/test-run-command.c
+++ b/t/helper/test-run-command.c
@@ -51,6 +51,16 @@ static int no_job(struct child_process *cp,
 	return 0;
 }
 
+static void test_consume_sideband(struct strbuf *output, void *cb)
+{
+	FILE *sideband;
+
+	sideband = fopen("./sideband", "a");
+
+	strbuf_write(output, sideband);
+	fclose(sideband);
+}
+
 static int task_finished(int result,
 			 struct strbuf *err,
 			 void *pp_cb,
@@ -201,7 +211,7 @@ static int testsuite(int argc, const char **argv)
 		suite.tests.nr, max_jobs);
 
 	ret = run_processes_parallel(max_jobs, next_test, test_failed,
-				     test_stdin, test_finished, &suite);
+				     test_stdin, NULL, test_finished, &suite);
 
 	if (suite.failed.nr > 0) {
 		ret = 1;
@@ -429,23 +439,28 @@ int cmd__run_command(int argc, const char **argv)
 
 	if (!strcmp(argv[1], "run-command-parallel"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, NULL, &proc));
+					    NULL, NULL, NULL, NULL, &proc));
 
 	if (!strcmp(argv[1], "run-command-abort"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, task_finished, &proc));
+					    NULL, NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-no-jobs"))
 		exit(run_processes_parallel(jobs, no_job,
-					    NULL, NULL, task_finished, &proc));
+					    NULL, NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-stdin")) {
 		proc.in = -1;
 		proc.no_stdin = 0;
 		exit (run_processes_parallel(jobs, parallel_next, NULL,
-					     test_stdin, NULL, &proc));
+					     test_stdin, NULL, NULL, &proc));
 	}
 
+	if (!strcmp(argv[1], "run-command-sideband"))
+		exit(run_processes_parallel(jobs, parallel_next, NULL, NULL,
+					    test_consume_sideband, NULL,
+					    &proc));
+
 	fprintf(stderr, "check usage\n");
 	return 1;
 }
diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh
index 87759482ad..e99f6c7f44 100755
--- a/t/t0061-run-command.sh
+++ b/t/t0061-run-command.sh
@@ -143,6 +143,13 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai
 	test_cmp expect actual
 '
 
+test_expect_success 'run_command can divert output' '
+	test_when_finished rm sideband &&
+	test-tool run-command run-command-sideband 3 sh -c "printf \"%s\n%s\n\" Hello World" 2>actual &&
+	test_must_be_empty actual &&
+	test_cmp expect sideband
+'
+
 cat >expect <<-EOF
 preloaded output of a child
 listening for stdin:
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 23/27] hooks: allow callers to capture output
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (21 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 22/27] run-command: allow capturing of collated output Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 24/27] receive-pack: convert 'update' hook to hook.h Ævar Arnfjörð Bjarmason
                             ` (6 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Some server-side hooks will require capturing output to send over
sideband instead of printing directly to stderr. Expose that capability.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 hook.c | 2 +-
 hook.h | 8 ++++++++
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/hook.c b/hook.c
index 2dca969f9e..e7f3b468ea 100644
--- a/hook.c
+++ b/hook.c
@@ -192,7 +192,7 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
-				   NULL,
+				   options->consume_sideband,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
diff --git a/hook.h b/hook.h
index b78a8e4805..1302efa590 100644
--- a/hook.h
+++ b/hook.h
@@ -66,6 +66,14 @@ struct run_hooks_opt
 	 */
 	feed_pipe_fn feed_pipe;
 	void *feed_pipe_ctx;
+
+	/*
+	 * Populate this to capture output and prevent it from being printed to
+	 * stderr. This will be passed directly through to
+	 * run_command:run_parallel_processes(). See t/helper/test-run-command.c
+	 * for an example.
+	 */
+	consume_sideband_fn consume_sideband;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 24/27] receive-pack: convert 'update' hook to hook.h
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (22 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 23/27] hooks: allow callers to capture output Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:22           ` [PATCH 25/27] post-update: use hook.h library Ævar Arnfjörð Bjarmason
                             ` (5 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

This makes use of the new sideband API in hook.h added in the
preceding commit.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/receive-pack.c | 65 ++++++++++++++++++++++++++++--------------
 1 file changed, 44 insertions(+), 21 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 5248228ebf..378f8f6b5d 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -938,33 +938,56 @@ static int run_receive_hook(struct command *commands,
 	return status;
 }
 
-static int run_update_hook(struct command *cmd)
+static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
 {
-	const char *argv[5];
-	struct child_process proc = CHILD_PROCESS_INIT;
-	int code;
+	int keepalive_active = 0;
 
-	argv[0] = find_hook("update");
-	if (!argv[0])
-		return 0;
+	if (keepalive_in_sec <= 0)
+		use_keepalive = KEEPALIVE_NEVER;
+	if (use_keepalive == KEEPALIVE_ALWAYS)
+		keepalive_active = 1;
 
-	argv[1] = cmd->ref_name;
-	argv[2] = oid_to_hex(&cmd->old_oid);
-	argv[3] = oid_to_hex(&cmd->new_oid);
-	argv[4] = NULL;
+	/* send a keepalive if there is no data to write */
+	if (keepalive_active && !output->len) {
+		static const char buf[] = "0005\1";
+		write_or_die(1, buf, sizeof(buf) - 1);
+		return;
+	}
 
-	proc.no_stdin = 1;
-	proc.stdout_to_stderr = 1;
-	proc.err = use_sideband ? -1 : 0;
-	proc.argv = argv;
-	proc.trace2_hook_name = "update";
+	if (use_keepalive == KEEPALIVE_AFTER_NUL && !keepalive_active) {
+		const char *first_null = memchr(output->buf, '\0', output->len);
+		if (first_null) {
+			/* The null bit is excluded. */
+			size_t before_null = first_null - output->buf;
+			size_t after_null = output->len - (before_null + 1);
+			keepalive_active = 1;
+			send_sideband(1, 2, output->buf, before_null, use_sideband);
+			send_sideband(1, 2, first_null + 1, after_null, use_sideband);
+
+			return;
+		}
+	}
+
+	send_sideband(1, 2, output->buf, output->len, use_sideband);
+}
+
+static int run_update_hook(struct command *cmd)
+{
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	int code;
+
+	strvec_pushl(&opt.args,
+		     cmd->ref_name,
+		     oid_to_hex(&cmd->old_oid),
+		     oid_to_hex(&cmd->new_oid),
+		     NULL);
 
-	code = start_command(&proc);
-	if (code)
-		return code;
 	if (use_sideband)
-		copy_to_sideband(proc.err, -1, NULL);
-	return finish_command(&proc);
+		opt.consume_sideband = hook_output_to_sideband;
+
+	code = run_hooks("update", &opt);
+	run_hooks_opt_clear(&opt);
+	return code;
 }
 
 static struct command *find_command_by_refname(struct command *list,
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 25/27] post-update: use hook.h library
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (23 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 24/27] receive-pack: convert 'update' hook to hook.h Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:22           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:23           ` [PATCH 26/27] receive-pack: convert receive hooks to hook.h Ævar Arnfjörð Bjarmason
                             ` (4 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:22 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/receive-pack.c | 26 +++++++-------------------
 1 file changed, 7 insertions(+), 19 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 378f8f6b5d..b2ccdb66da 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1657,33 +1657,21 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 static void run_update_post_hook(struct command *commands)
 {
 	struct command *cmd;
-	struct child_process proc = CHILD_PROCESS_INIT;
-	const char *hook;
-
-	hook = find_hook("post-update");
-	if (!hook)
-		return;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 
 	for (cmd = commands; cmd; cmd = cmd->next) {
 		if (cmd->error_string || cmd->did_not_exist)
 			continue;
-		if (!proc.args.nr)
-			strvec_push(&proc.args, hook);
-		strvec_push(&proc.args, cmd->ref_name);
+		strvec_push(&opt.args, cmd->ref_name);
 	}
-	if (!proc.args.nr)
+	if (!opt.args.nr)
 		return;
 
-	proc.no_stdin = 1;
-	proc.stdout_to_stderr = 1;
-	proc.err = use_sideband ? -1 : 0;
-	proc.trace2_hook_name = "post-update";
+	if (use_sideband)
+		opt.consume_sideband = hook_output_to_sideband;
 
-	if (!start_command(&proc)) {
-		if (use_sideband)
-			copy_to_sideband(proc.err, -1, NULL);
-		finish_command(&proc);
-	}
+	run_hooks("post-update", &opt);
+	run_hooks_opt_clear(&opt);
 }
 
 static void check_aliased_update_internal(struct command *cmd,
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 26/27] receive-pack: convert receive hooks to hook.h
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (24 preceding siblings ...)
  2021-06-17 10:22           ` [PATCH 25/27] post-update: use hook.h library Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:23           ` Ævar Arnfjörð Bjarmason
  2021-06-17 10:23           ` [PATCH 27/27] hooks: fix a TOCTOU in "did we run a hook?" heuristic Ævar Arnfjörð Bjarmason
                             ` (3 subsequent siblings)
  29 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:23 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/receive-pack.c | 197 +++++++++++++++++++----------------------
 1 file changed, 90 insertions(+), 107 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index b2ccdb66da..ec90e10477 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -748,7 +748,7 @@ static int check_cert_push_options(const struct string_list *push_options)
 	return retval;
 }
 
-static void prepare_push_cert_sha1(struct child_process *proc)
+static void prepare_push_cert_sha1(struct run_hooks_opt *opt)
 {
 	static int already_done;
 
@@ -772,110 +772,42 @@ static void prepare_push_cert_sha1(struct child_process *proc)
 		nonce_status = check_nonce(push_cert.buf, bogs);
 	}
 	if (!is_null_oid(&push_cert_oid)) {
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT=%s",
 			     oid_to_hex(&push_cert_oid));
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_SIGNER=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_SIGNER=%s",
 			     sigcheck.signer ? sigcheck.signer : "");
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_KEY=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_KEY=%s",
 			     sigcheck.key ? sigcheck.key : "");
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_STATUS=%c",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_STATUS=%c",
 			     sigcheck.result);
 		if (push_cert_nonce) {
-			strvec_pushf(&proc->env_array,
+			strvec_pushf(&opt->env,
 				     "GIT_PUSH_CERT_NONCE=%s",
 				     push_cert_nonce);
-			strvec_pushf(&proc->env_array,
+			strvec_pushf(&opt->env,
 				     "GIT_PUSH_CERT_NONCE_STATUS=%s",
 				     nonce_status);
 			if (nonce_status == NONCE_SLOP)
-				strvec_pushf(&proc->env_array,
+				strvec_pushf(&opt->env,
 					     "GIT_PUSH_CERT_NONCE_SLOP=%ld",
 					     nonce_stamp_slop);
 		}
 	}
 }
 
+struct receive_hook_feed_context {
+	struct command *cmd;
+	int skip_broken;
+};
+
 struct receive_hook_feed_state {
 	struct command *cmd;
 	struct ref_push_report *report;
 	int skip_broken;
 	struct strbuf buf;
-	const struct string_list *push_options;
 };
 
-typedef int (*feed_fn)(void *, const char **, size_t *);
-static int run_and_feed_hook(const char *hook_name, feed_fn feed,
-			     struct receive_hook_feed_state *feed_state)
-{
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct async muxer;
-	const char *argv[2];
-	int code;
-
-	argv[0] = find_hook(hook_name);
-	if (!argv[0])
-		return 0;
-
-	argv[1] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = hook_name;
-
-	if (feed_state->push_options) {
-		int i;
-		for (i = 0; i < feed_state->push_options->nr; i++)
-			strvec_pushf(&proc.env_array,
-				     "GIT_PUSH_OPTION_%d=%s", i,
-				     feed_state->push_options->items[i].string);
-		strvec_pushf(&proc.env_array, "GIT_PUSH_OPTION_COUNT=%d",
-			     feed_state->push_options->nr);
-	} else
-		strvec_pushf(&proc.env_array, "GIT_PUSH_OPTION_COUNT");
-
-	if (tmp_objdir)
-		strvec_pushv(&proc.env_array, tmp_objdir_env(tmp_objdir));
-
-	if (use_sideband) {
-		memset(&muxer, 0, sizeof(muxer));
-		muxer.proc = copy_to_sideband;
-		muxer.in = -1;
-		code = start_async(&muxer);
-		if (code)
-			return code;
-		proc.err = muxer.in;
-	}
-
-	prepare_push_cert_sha1(&proc);
-
-	code = start_command(&proc);
-	if (code) {
-		if (use_sideband)
-			finish_async(&muxer);
-		return code;
-	}
-
-	sigchain_push(SIGPIPE, SIG_IGN);
-
-	while (1) {
-		const char *buf;
-		size_t n;
-		if (feed(feed_state, &buf, &n))
-			break;
-		if (write_in_full(proc.in, buf, n) < 0)
-			break;
-	}
-	close(proc.in);
-	if (use_sideband)
-		finish_async(&muxer);
-
-	sigchain_pop(SIGPIPE);
-
-	return finish_command(&proc);
-}
-
-static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
+static int feed_receive_hook(void *state_)
 {
 	struct receive_hook_feed_state *state = state_;
 	struct command *cmd = state->cmd;
@@ -884,9 +816,7 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
 	       state->skip_broken && (cmd->error_string || cmd->did_not_exist))
 		cmd = cmd->next;
 	if (!cmd)
-		return -1; /* EOF */
-	if (!bufp)
-		return 0; /* OK, can feed something. */
+		return 1; /* EOF - close the pipe*/
 	strbuf_reset(&state->buf);
 	if (!state->report)
 		state->report = cmd->report;
@@ -910,32 +840,36 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
 			    cmd->ref_name);
 		state->cmd = cmd->next;
 	}
-	if (bufp) {
-		*bufp = state->buf.buf;
-		*sizep = state->buf.len;
-	}
 	return 0;
 }
 
-static int run_receive_hook(struct command *commands,
-			    const char *hook_name,
-			    int skip_broken,
-			    const struct string_list *push_options)
+static int feed_receive_hook_cb(struct strbuf *pipe, void *pp_cb, void *pp_task_cb)
 {
-	struct receive_hook_feed_state state;
-	int status;
-
-	strbuf_init(&state.buf, 0);
-	state.cmd = commands;
-	state.skip_broken = skip_broken;
-	state.report = NULL;
-	if (feed_receive_hook(&state, NULL, NULL))
-		return 0;
-	state.cmd = commands;
-	state.push_options = push_options;
-	status = run_and_feed_hook(hook_name, feed_receive_hook, &state);
-	strbuf_release(&state.buf);
-	return status;
+	struct hook *hook = pp_task_cb;
+	struct receive_hook_feed_state *feed_state = hook->feed_pipe_cb_data;
+	int rc;
+
+	/* first-time setup */
+	if (!feed_state) {
+		struct hook_cb_data *hook_cb = pp_cb;
+		struct run_hooks_opt *opt = hook_cb->options;
+		struct receive_hook_feed_context *ctx = opt->feed_pipe_ctx;
+		if (!ctx)
+			BUG("run_hooks_opt.feed_pipe_ctx required for receive hook");
+
+		feed_state = xmalloc(sizeof(struct receive_hook_feed_state));
+		strbuf_init(&feed_state->buf, 0);
+		feed_state->cmd = ctx->cmd;
+		feed_state->skip_broken = ctx->skip_broken;
+		feed_state->report = NULL;
+
+		hook->feed_pipe_cb_data = feed_state;
+	}
+
+	rc = feed_receive_hook(feed_state);
+	if (!rc)
+		strbuf_addbuf(pipe, &feed_state->buf);
+	return rc;
 }
 
 static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
@@ -971,6 +905,55 @@ static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
 	send_sideband(1, 2, output->buf, output->len, use_sideband);
 }
 
+static int run_receive_hook(struct command *commands,
+			    const char *hook_name,
+			    int skip_broken,
+			    const struct string_list *push_options)
+{
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct receive_hook_feed_context ctx;
+	int rc;
+	struct command *iter = commands;
+
+	/* if there are no valid commands, don't invoke the hook at all. */
+	while (iter && skip_broken && (iter->error_string || iter->did_not_exist))
+		iter = iter->next;
+	if (!iter)
+		return 0;
+
+	/* pre-receive hooks should run in series as the hook updates refs */
+	if (!strcmp(hook_name, "pre-receive"))
+		opt.jobs = 1;
+
+	if (push_options) {
+		int i;
+		for (i = 0; i < push_options->nr; i++)
+			strvec_pushf(&opt.env, "GIT_PUSH_OPTION_%d=%s", i,
+				     push_options->items[i].string);
+		strvec_pushf(&opt.env, "GIT_PUSH_OPTION_COUNT=%d", push_options->nr);
+	} else
+		strvec_push(&opt.env, "GIT_PUSH_OPTION_COUNT");
+
+	if (tmp_objdir)
+		strvec_pushv(&opt.env, tmp_objdir_env(tmp_objdir));
+
+	prepare_push_cert_sha1(&opt);
+
+	/* set up sideband printer */
+	if (use_sideband)
+		opt.consume_sideband = hook_output_to_sideband;
+
+	/* set up stdin callback */
+	ctx.cmd = commands;
+	ctx.skip_broken = skip_broken;
+	opt.feed_pipe = feed_receive_hook_cb;
+	opt.feed_pipe_ctx = &ctx;
+
+	rc = run_hooks(hook_name, &opt);
+	run_hooks_opt_clear(&opt);
+	return rc;
+}
+
 static int run_update_hook(struct command *cmd)
 {
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 27/27] hooks: fix a TOCTOU in "did we run a hook?" heuristic
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (25 preceding siblings ...)
  2021-06-17 10:23           ` [PATCH 26/27] receive-pack: convert receive hooks to hook.h Ævar Arnfjörð Bjarmason
@ 2021-06-17 10:23           ` Ævar Arnfjörð Bjarmason
  2021-06-18 22:09             ` Emily Shaffer
  2021-07-15 23:25           ` [PATCH 0/9] config-based hooks restarted Emily Shaffer
                             ` (2 subsequent siblings)
  29 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-17 10:23 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Fix a Time-of-check to time-of-use (TOCTOU) race in code added in
680ee550d72 (commit: skip discarding the index if there is no
pre-commit hook, 2017-08-14).

We can fix the race passing around information about whether or not we
ran the hook in question, instead of running hook_exists() after the
fact to check if the hook in question exists. This problem has been
noted on-list when 680ee550d72 was discussed[1], but had not been
fixed.

In addition to fixing this for the pre-commit hook as suggested there
I'm also fixing this for the pre-merge-commit hook. See
6098817fd7f (git-merge: honor pre-merge-commit hook, 2019-08-07) for
the introduction of its previous behavior.

Let's also change this for the push-to-checkout hook. Now instead of
checking if the hook exists and either doing a push to checkout or a
push to deploy we'll always attempt a push to checkout. If the hook
doesn't exist we'll fall back on push to deploy. The same behavior as
before, without the TOCTOU race. See 0855331941b (receive-pack:
support push-to-checkout hook, 2014-12-01) for the introduction of the
previous behavior.

This leaves uses of hook_exists() in two places that matter. The
"reference-transaction" check in refs.c, see 67541597670 (refs:
implement reference transaction hook, 2020-06-19), and the
prepare-commit-msg hook, see 66618a50f9c (sequencer: run
'prepare-commit-msg' hook, 2018-01-24).

In both of those cases we're saving ourselves CPU time by not
preparing data for the hook that we'll then do nothing with if we
don't have the hook. So using this "invoked_hook" pattern doesn't make
sense in those cases.

More importantly, in those cases the worst we'll do is miss that we
"should" run the hook because a new hook appeared, whereas in the
pre-commit and pre-merge-commit cases we'll skip an important
discard_cache() on the bases of our faulty guess.

I do think none of these races really matter in practice. It would be
some one-off issue as a hook was added or removed. I did think it was
stupid that we didn't pass a "did this run?" flag instead of doing
this guessing at a distance though, so now we're not guessing anymore.

1. https://lore.kernel.org/git/20170810191613.kpmhzg4seyxy3cpq@sigill.intra.peff.net/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/commit.c       | 18 +++++++++++-------
 builtin/merge.c        | 16 ++++++++++------
 builtin/receive-pack.c |  8 +++++---
 commit.c               |  1 +
 commit.h               |  3 ++-
 hook.c                 |  4 ++++
 hook.h                 | 10 ++++++++++
 sequencer.c            |  4 ++--
 8 files changed, 45 insertions(+), 19 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index dad4e56544..a66727a612 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -725,11 +725,13 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	int clean_message_contents = (cleanup_mode != COMMIT_MSG_CLEANUP_NONE);
 	int old_display_comment_prefix;
 	int merge_contains_scissors = 0;
+	int invoked_hook = 0;
 
 	/* This checks and barfs if author is badly specified */
 	determine_author_info(author_ident);
 
-	if (!no_verify && run_commit_hook(use_editor, index_file, "pre-commit", NULL))
+	if (!no_verify && run_commit_hook(use_editor, index_file, &invoked_hook,
+					  "pre-commit", NULL))
 		return 0;
 
 	if (squash_message) {
@@ -1045,10 +1047,10 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (!no_verify && hook_exists("pre-commit")) {
+	if (!no_verify && invoked_hook) {
 		/*
-		 * Re-read the index as pre-commit hook could have updated it,
-		 * and write it out as a tree.  We must do this before we invoke
+		 * Re-read the index as the pre-commit-commit hook was invoked
+		 * and could have updated it. We must do this before we invoke
 		 * the editor and after we invoke run_status above.
 		 */
 		discard_cache();
@@ -1060,7 +1062,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (run_commit_hook(use_editor, index_file, "prepare-commit-msg",
+	if (run_commit_hook(use_editor, index_file, NULL, "prepare-commit-msg",
 			    git_path_commit_editmsg(), hook_arg1, hook_arg2, NULL))
 		return 0;
 
@@ -1077,7 +1079,8 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	}
 
 	if (!no_verify &&
-	    run_commit_hook(use_editor, index_file, "commit-msg", git_path_commit_editmsg(), NULL)) {
+	    run_commit_hook(use_editor, index_file, NULL, "commit-msg",
+			    git_path_commit_editmsg(), NULL)) {
 		return 0;
 	}
 
@@ -1830,7 +1833,8 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 
 	repo_rerere(the_repository, 0);
 	run_auto_maintenance(quiet);
-	run_commit_hook(use_editor, get_index_file(), "post-commit", NULL);
+	run_commit_hook(use_editor, get_index_file(), NULL, "post-commit",
+			NULL);
 	if (amend && !no_post_rewrite) {
 		commit_post_rewrite(the_repository, current_head, &oid);
 	}
diff --git a/builtin/merge.c b/builtin/merge.c
index 6128b60942..0425c9bf2b 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -844,15 +844,18 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 {
 	struct strbuf msg = STRBUF_INIT;
 	const char *index_file = get_index_file();
+	int invoked_hook = 0;
 
-	if (!no_verify && run_commit_hook(0 < option_edit, index_file, "pre-merge-commit", NULL))
+	if (!no_verify && run_commit_hook(0 < option_edit, index_file,
+					  &invoked_hook, "pre-merge-commit",
+					  NULL))
 		abort_commit(remoteheads, NULL);
 	/*
-	 * Re-read the index as pre-merge-commit hook could have updated it,
-	 * and write it out as a tree.  We must do this before we invoke
+	 * Re-read the index as the pre-merge-commit hook was invoked
+	 * and could have updated it. We must do this before we invoke
 	 * the editor and after we invoke run_status above.
 	 */
-	if (hook_exists("pre-merge-commit"))
+	if (invoked_hook)
 		discard_cache();
 	read_cache_from(index_file);
 	strbuf_addbuf(&msg, &merge_msg);
@@ -873,7 +876,8 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 		append_signoff(&msg, ignore_non_trailer(msg.buf, msg.len), 0);
 	write_merge_heads(remoteheads);
 	write_file_buf(git_path_merge_msg(the_repository), msg.buf, msg.len);
-	if (run_commit_hook(0 < option_edit, get_index_file(), "prepare-commit-msg",
+	if (run_commit_hook(0 < option_edit, get_index_file(), NULL,
+			    "prepare-commit-msg",
 			    git_path_merge_msg(the_repository), "merge", NULL))
 		abort_commit(remoteheads, NULL);
 	if (0 < option_edit) {
@@ -882,7 +886,7 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 	}
 
 	if (!no_verify && run_commit_hook(0 < option_edit, get_index_file(),
-					  "commit-msg",
+					  NULL, "commit-msg",
 					  git_path_merge_msg(the_repository), NULL))
 		abort_commit(remoteheads, NULL);
 
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index ec90e10477..cd658f41d5 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1439,10 +1439,12 @@ static const char *push_to_deploy(unsigned char *sha1,
 static const char *push_to_checkout_hook = "push-to-checkout";
 
 static const char *push_to_checkout(unsigned char *hash,
+				    int *invoked_hook,
 				    struct strvec *env,
 				    const char *work_tree)
 {
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	opt.invoked_hook = invoked_hook;
 
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
 	strvec_pushv(&opt.env, env->v);
@@ -1460,6 +1462,7 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 {
 	const char *retval, *work_tree, *git_dir = NULL;
 	struct strvec env = STRVEC_INIT;
+	int invoked_hook = 0;
 
 	if (worktree && worktree->path)
 		work_tree = worktree->path;
@@ -1477,10 +1480,9 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 
 	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
 
-	if (!hook_exists(push_to_checkout_hook))
+	retval = push_to_checkout(sha1, &invoked_hook, &env, work_tree);
+	if (!invoked_hook)
 		retval = push_to_deploy(sha1, &env, work_tree);
-	else
-		retval = push_to_checkout(sha1, &env, work_tree);
 
 	strvec_clear(&env);
 	return retval;
diff --git a/commit.c b/commit.c
index e8147a88fc..cf62ebceae 100644
--- a/commit.c
+++ b/commit.c
@@ -1697,6 +1697,7 @@ size_t ignore_non_trailer(const char *buf, size_t len)
 }
 
 int run_commit_hook(int editor_is_used, const char *index_file,
+		    int *invoked_hook,
 		    const char *name, ...)
 {
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
diff --git a/commit.h b/commit.h
index df42eb434f..b5a542993c 100644
--- a/commit.h
+++ b/commit.h
@@ -363,7 +363,8 @@ int compare_commits_by_commit_date(const void *a_, const void *b_, void *unused)
 int compare_commits_by_gen_then_commit_date(const void *a_, const void *b_, void *unused);
 
 LAST_ARG_MUST_BE_NULL
-int run_commit_hook(int editor_is_used, const char *index_file, const char *name, ...);
+int run_commit_hook(int editor_is_used, const char *index_file,
+		    int *invoked_hook, const char *name, ...);
 
 /* Sign a commit or tag buffer, storing the result in a header. */
 int sign_with_header(struct strbuf *buf, const char *keyid);
diff --git a/hook.c b/hook.c
index e7f3b468ea..31e822bf51 100644
--- a/hook.c
+++ b/hook.c
@@ -164,6 +164,9 @@ static int notify_hook_finished(int result,
 
 	hook_cb->rc |= result;
 
+	if (hook_cb->invoked_hook)
+		*hook_cb->invoked_hook = 1;
+
 	return 1;
 }
 
@@ -178,6 +181,7 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 		.rc = 0,
 		.hook_name = hook_name,
 		.options = options,
+		.invoked_hook = options->invoked_hook,
 	};
 	if (options->absolute_path) {
 		strbuf_add_absolute_path(&abs_path, hook_path);
diff --git a/hook.h b/hook.h
index 1302efa590..9d9171672d 100644
--- a/hook.h
+++ b/hook.h
@@ -74,6 +74,15 @@ struct run_hooks_opt
 	 * for an example.
 	 */
 	consume_sideband_fn consume_sideband;
+
+	/*
+	 * A pointer which if provided will be set to 1 or 0 depending
+	 * on if a hook was invoked (i.e. existed), regardless of
+	 * whether or not that was successful. Used for avoiding
+	 * TOCTOU races in code that would otherwise call hook_exist()
+	 * after a "maybe hook run" to see if a hook was invoked.
+	 */
+	int *invoked_hook;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
@@ -96,6 +105,7 @@ struct hook_cb_data {
 	const char *hook_name;
 	struct hook *run_me;
 	struct run_hooks_opt *options;
+	int *invoked_hook;
 };
 
 void run_hooks_opt_clear(struct run_hooks_opt *o);
diff --git a/sequencer.c b/sequencer.c
index ec2761e47d..2440b9dccd 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1204,7 +1204,7 @@ static int run_prepare_commit_msg_hook(struct repository *r,
 	} else {
 		arg1 = "message";
 	}
-	if (run_commit_hook(0, r->index_file, "prepare-commit-msg", name,
+	if (run_commit_hook(0, r->index_file, NULL, "prepare-commit-msg", name,
 			    arg1, arg2, NULL))
 		ret = error(_("'prepare-commit-msg' hook failed"));
 
@@ -1534,7 +1534,7 @@ static int try_to_commit(struct repository *r,
 		goto out;
 	}
 
-	run_commit_hook(0, r->index_file, "post-commit", NULL);
+	run_commit_hook(0, r->index_file, NULL, "post-commit", NULL);
 	if (flags & AMEND_MSG)
 		commit_post_rewrite(r, current_head, oid);
 
-- 
2.32.0.576.g59759b6ca7d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* Re: [PATCH 27/27] hooks: fix a TOCTOU in "did we run a hook?" heuristic
  2021-06-17 10:23           ` [PATCH 27/27] hooks: fix a TOCTOU in "did we run a hook?" heuristic Ævar Arnfjörð Bjarmason
@ 2021-06-18 22:09             ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-06-18 22:09 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

On Thu, Jun 17, 2021 at 12:23:01PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> Fix a Time-of-check to time-of-use (TOCTOU) race in code added in
> 680ee550d72 (commit: skip discarding the index if there is no
> pre-commit hook, 2017-08-14).
> 
> We can fix the race passing around information about whether or not we
> ran the hook in question, instead of running hook_exists() after the
> fact to check if the hook in question exists. This problem has been
> noted on-list when 680ee550d72 was discussed[1], but had not been
> fixed.
> 
> In addition to fixing this for the pre-commit hook as suggested there
> I'm also fixing this for the pre-merge-commit hook. See
> 6098817fd7f (git-merge: honor pre-merge-commit hook, 2019-08-07) for
> the introduction of its previous behavior.
> 
> Let's also change this for the push-to-checkout hook. Now instead of
> checking if the hook exists and either doing a push to checkout or a
> push to deploy we'll always attempt a push to checkout. If the hook
> doesn't exist we'll fall back on push to deploy. The same behavior as
> before, without the TOCTOU race. See 0855331941b (receive-pack:
> support push-to-checkout hook, 2014-12-01) for the introduction of the
> previous behavior.
> 
> This leaves uses of hook_exists() in two places that matter. The
> "reference-transaction" check in refs.c, see 67541597670 (refs:
> implement reference transaction hook, 2020-06-19), and the
> prepare-commit-msg hook, see 66618a50f9c (sequencer: run
> 'prepare-commit-msg' hook, 2018-01-24).
> 
> In both of those cases we're saving ourselves CPU time by not
> preparing data for the hook that we'll then do nothing with if we
> don't have the hook. So using this "invoked_hook" pattern doesn't make
> sense in those cases.
> 
> More importantly, in those cases the worst we'll do is miss that we
> "should" run the hook because a new hook appeared, whereas in the
> pre-commit and pre-merge-commit cases we'll skip an important
> discard_cache() on the bases of our faulty guess.
> 
> I do think none of these races really matter in practice. It would be
> some one-off issue as a hook was added or removed. I did think it was
> stupid that we didn't pass a "did this run?" flag instead of doing
> this guessing at a distance though, so now we're not guessing anymore.

Yeah, I think your solution is very neat. I like this patch.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 01/30] hook: add 'run' subcommand
  2021-06-15  9:36             ` Ævar Arnfjörð Bjarmason
@ 2021-06-18 22:13               ` Emily Shaffer
  2021-06-20 19:30                 ` Ævar Arnfjörð Bjarmason
  2021-06-25 19:02                 ` Felipe Contreras
  0 siblings, 2 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-06-18 22:13 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

On Tue, Jun 15, 2021 at 11:36:26AM +0200, Ævar Arnfjörð Bjarmason wrote:
> > Overall, I think I like the direction your reroll is going - I've needed
> > some time to process it. Hopefully I'll be able to get through all or
> > most of the series this week, but there's a lot going on here, too. I'll
> > do what I can. Thanks for the help.
> 
> Yeah, will reply to any qusetions etc; and as noted above my initial
> goal here was "hey, what about this approach", so if you wanted to pick
> this up & run with it...
> 
> This particular version of the series is at github.com/avar/git.git's
> es-avar/config-based-hooks-3 b.t.w.

Have finished scanning through the rest of the series, and I think I
understand your goal a little better - you are not saying "let me take
over and drive this part of the feature set", which is what I thought
initially. Instead, you seem to be saying "let's chop it up this way
instead".

I don't dislike the reorganization, but I do still wonder whether it's
a setback to the progress the original series had made. I guess it is
hard to know - I had thought the original series was pretty much ready
to go in, therefore making "what if we ordered it this way" moot. But it
seems that you disagree.

Anyway, I do hear also that you don't have interest in driving this
subset to completion, and that's fine. Correct me if I'm wrong.

I'll keep thinking on this over the weekend. Thanks for the suggestion.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 01/30] hook: add 'run' subcommand
  2021-06-18 22:13               ` Emily Shaffer
@ 2021-06-20 19:30                 ` Ævar Arnfjörð Bjarmason
  2021-06-21  3:44                   ` Junio C Hamano
  2021-06-22  0:00                   ` Emily Shaffer
  2021-06-25 19:02                 ` Felipe Contreras
  1 sibling, 2 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-06-20 19:30 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee


On Fri, Jun 18 2021, Emily Shaffer wrote:

> On Tue, Jun 15, 2021 at 11:36:26AM +0200, Ævar Arnfjörð Bjarmason wrote:
>> > Overall, I think I like the direction your reroll is going - I've needed
>> > some time to process it. Hopefully I'll be able to get through all or
>> > most of the series this week, but there's a lot going on here, too. I'll
>> > do what I can. Thanks for the help.
>> 
>> Yeah, will reply to any qusetions etc; and as noted above my initial
>> goal here was "hey, what about this approach", so if you wanted to pick
>> this up & run with it...
>> 
>> This particular version of the series is at github.com/avar/git.git's
>> es-avar/config-based-hooks-3 b.t.w.
>
> Have finished scanning through the rest of the series, and I think I
> understand your goal a little better - you are not saying "let me take
> over and drive this part of the feature set", which is what I thought
> initially. Instead, you seem to be saying "let's chop it up this way
> instead".

Yes, 30-some patches that both refactor and introduce new behavior are
harder to reason about.

I've also had suggestions about the end-state, but I think whatever we
arrive at doing the scaffolding first without behavior changes makes
sense.

> I don't dislike the reorganization, but I do still wonder whether it's
> a setback to the progress the original series had made. I guess it is
> hard to know - I had thought the original series was pretty much ready
> to go in, therefore making "what if we ordered it this way" moot. But it
> seems that you disagree.

I'm still not sure if I disagree, well, I'm 95% sure I disagree with
some of the end-state, but you never replied to my questions about that:
https://lore.kernel.org/git/87mtv8fww3.fsf@evledaraar.gmail.com/ &
https://lore.kernel.org/git/87lf80l1m6.fsf@evledraar.gmail.com/; So I
don't know for sure, maybe there's things I missed there.

I think since Junio picked up the "base" version of this and it looks
like we're going that way first that's not something we need to discuss
now if you'd like to punt it, but I'd really like to get that cleared up
post-base topic.

In brief summary:

I'm 100% with you on hooks being driven by config, that they aren't is
in hindsight a historical wart. Ditto the parallel execution etc. (which
I'd suggested in an earlier iteration). That's all great.

Where you lose me is the need for having "git hook" be an administrative
interface for it, particularly (as noted in the linked E-Mail) since the
need for that over simply using "git config", or a trivial "git config"
wrapper seems to be fallout from other arbitrary design choices.

I.e. that all the config for a hook needing to be discovered by a
two-pass iteration over the config space (or keeping state), as opposed
to a "hookcfg.<name>.*" (or whatever) prefix.

Maybe that makes sense in the eventual end-state, your series has the
equivalent of "WIP, more will be added later" around that "git hook"
command; but not having the full overview of that I think we can make
simpler inroads into getting us all of the practical featureset we want,
without regretting our choices in command & config interfaces later.

> Anyway, I do hear also that you don't have interest in driving this
> subset to completion, and that's fine. Correct me if I'm wrong.

I submitted a v3 of this (which I forgot to label as such in the
subject) at
https://lore.kernel.org/git/cover-00.27-0000000000-20210617T101216Z-avarab@gmail.com/;
given the timing our E-Mails may have crossed.

But no, I will drive this subset to completion. What I meant with the
"run with it" comment and the earlier reply on v1 of my "base" version
here: https://lore.kernel.org/git/87y2bs7gyc.fsf@evledraar.gmail.com/

... is that I'd be happier if I managed to just convince you that the
more piecemeal approach is better, and something you'd want to pick up &
drive going forward.

I.e. it's still >95% your code, just re-arranged and split into subsets
of your patches. I really did not mean to "steal" it, it's just
something I hacked up one day to see if the more incremental approach
I'd been suggesting (and felt you were either ignoring or were too busy
to address) was something that could work.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 01/30] hook: add 'run' subcommand
  2021-06-20 19:30                 ` Ævar Arnfjörð Bjarmason
@ 2021-06-21  3:44                   ` Junio C Hamano
  2021-06-22  0:00                   ` Emily Shaffer
  1 sibling, 0 replies; 479+ messages in thread
From: Junio C Hamano @ 2021-06-21  3:44 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Emily Shaffer, git, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> I think since Junio picked up the "base" version of this and it looks
> like we're going that way first that's not something we need to discuss
> now if you'd like to punt it, but I'd really like to get that cleared up
> post-base topic.

I queued this 'base' thing only because (1) Emily promised to review
it and I wanted to give her a version of "seen" she can try to build
on the part beyond "refactoring", (2) I wanted to see how the 'base'
looks myself and (3) I wanted to see what possible interactions with
other topics in flight I should anticipate, if we were to decide to
go that route.  Please do not read more than that into what is and
what is not in 'seen'.


^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 01/30] hook: add 'run' subcommand
  2021-06-20 19:30                 ` Ævar Arnfjörð Bjarmason
  2021-06-21  3:44                   ` Junio C Hamano
@ 2021-06-22  0:00                   ` Emily Shaffer
  2021-06-29  1:12                     ` Junio C Hamano
  1 sibling, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-06-22  0:00 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

On Sun, Jun 20, 2021 at 09:30:59PM +0200, Ævar Arnfjörð Bjarmason wrote:
> On Fri, Jun 18 2021, Emily Shaffer wrote:
> Yes, 30-some patches that both refactor and introduce new behavior are
> harder to reason about.
> 
> I've also had suggestions about the end-state, but I think whatever we
> arrive at doing the scaffolding first without behavior changes makes
> sense.
> 
> > I don't dislike the reorganization, but I do still wonder whether it's
> > a setback to the progress the original series had made. I guess it is
> > hard to know - I had thought the original series was pretty much ready
> > to go in, therefore making "what if we ordered it this way" moot. But it
> > seems that you disagree.
> 
> I'm still not sure if I disagree, well, I'm 95% sure I disagree with
> some of the end-state, but you never replied to my questions about that:
> https://lore.kernel.org/git/87mtv8fww3.fsf@evledaraar.gmail.com/ &
> https://lore.kernel.org/git/87lf80l1m6.fsf@evledraar.gmail.com/; So I
> don't know for sure, maybe there's things I missed there.
> 
> I think since Junio picked up the "base" version of this and it looks
> like we're going that way first that's not something we need to discuss
> now if you'd like to punt it, but I'd really like to get that cleared up
> post-base topic.
> 
> In brief summary:
> 
> I'm 100% with you on hooks being driven by config, that they aren't is
> in hindsight a historical wart. Ditto the parallel execution etc. (which
> I'd suggested in an earlier iteration). That's all great.
> 
> Where you lose me is the need for having "git hook" be an administrative
> interface for it, particularly (as noted in the linked E-Mail) since the
> need for that over simply using "git config", or a trivial "git config"
> wrapper seems to be fallout from other arbitrary design choices.
> 
> I.e. that all the config for a hook needing to be discovered by a
> two-pass iteration over the config space (or keeping state), as opposed
> to a "hookcfg.<name>.*" (or whatever) prefix.
> 
> Maybe that makes sense in the eventual end-state, your series has the
> equivalent of "WIP, more will be added later" around that "git hook"
> command; but not having the full overview of that I think we can make
> simpler inroads into getting us all of the practical featureset we want,
> without regretting our choices in command & config interfaces later.
> 
> > Anyway, I do hear also that you don't have interest in driving this
> > subset to completion, and that's fine. Correct me if I'm wrong.
> 
> I submitted a v3 of this (which I forgot to label as such in the
> subject) at
> https://lore.kernel.org/git/cover-00.27-0000000000-20210617T101216Z-avarab@gmail.com/;
> given the timing our E-Mails may have crossed.
> 
> But no, I will drive this subset to completion. What I meant with the
> "run with it" comment and the earlier reply on v1 of my "base" version
> here: https://lore.kernel.org/git/87y2bs7gyc.fsf@evledraar.gmail.com/
> 
> ... is that I'd be happier if I managed to just convince you that the
> more piecemeal approach is better, and something you'd want to pick up &
> drive going forward.
> 
> I.e. it's still >95% your code, just re-arranged and split into subsets
> of your patches. I really did not mean to "steal" it, it's just
> something I hacked up one day to see if the more incremental approach
> I'd been suggesting (and felt you were either ignoring or were too busy
> to address) was something that could work.

Ok. Thanks for clarifying.

Yes, I do like this direction, and I'm pleased you were able to chop it
up in a way where partial submission made sense - I struggled with that,
myself. Yes, I am excited that you want to drive this series :) :) and
will be happy to rebase on top of it.

I'll have a look at the range-diff for v3 this week. Thanks.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 00/31] minimal restart of "config-based-hooks"
  2021-06-02  7:56         ` Ævar Arnfjörð Bjarmason
  2021-06-02  9:39           ` Ævar Arnfjörð Bjarmason
@ 2021-06-25 18:14           ` Felipe Contreras
  1 sibling, 0 replies; 479+ messages in thread
From: Felipe Contreras @ 2021-06-25 18:14 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason, Emily Shaffer
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan

Ævar Arnfjörð Bjarmason wrote:

> I'm a bit surprised at what seems to be some hostility or annoyance that
> I submitted this as a set of patches. That's ultimately something that
> saves everyone involved time (well, except me by coming up with said
> patches). To borrow some words:
> 
>     "Talk is cheap. Show me the code." ― Linus Torvalds.

Words I live by.

> If I give you feedback suggesting that maybe we should reorganize this
> thing to split out refactorings from behavior changes I'm asking you to
> do extra work. Ultimately neither I, you nor anyone else can really know
> if such a proposed effort is going to be better until it happens.

Indeed. That's why a lot of time instead of simply replying to a mail
with an idea, I actually attempt to code the idea, and only then send te
reply.

I would say 90% of the time what I originally thought changes once I
actually try to implement it.

> It's can be really hard to see how/where to split things when you're the
> author of the code. It's really hard in the "theory of mind" sense of
> things to explain an idea to someone who doesn't have the information
> you have.

More like impossible.

No comic simply writes an act and goes to an auditorium like Carnegie
Hall to simply present it knowing full well how people are going to
react to it.

You never really know how other people are going to react, so it's
better to not make assumptions, just try and find out.

> I think I've made a good argument above for why this takes you a step
> forward, not backwards, I'm hoping despite this initial reply that
> you'll come to agree on that.

Not to mention that the goal is not to land Emily's patches, the goal is
to improve the code while minimizing the potential breakage. For that we
need as many eyes as possible, and in my opinion your reorganization
patches totally help in that regard.

The fact that this makes it easier to land Emily's patches is an added
benefit.

> In any case, writing code is hard, but splitting it up like I've done
> here is rather easily done. It took me about a day with waiting for
> "rebase -i --exec='make test'" equivalent, and that's from being mostly
> unfamiliar with the code in question beforehand.

It's easy if you are an expert at rebasing, which not many people are.

But you can't become an expert if you don't do it, over and over. So
it's usually better to not think too much about it and simply do it.

Cheers.

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 479+ messages in thread

* RE: [PATCH v2 00/30] Minimal restart of "config-based-hooks"
  2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
                           ` (31 preceding siblings ...)
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
@ 2021-06-25 18:22         ` Felipe Contreras
  32 siblings, 0 replies; 479+ messages in thread
From: Felipe Contreras @ 2021-06-25 18:22 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason, git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Ævar Arnfjörð Bjarmason wrote:
> I proposed splitting Emily's "hook config" topic[1] into at least a
> topic that retains all current behavior of the codebase, and merely
> refactors existing behavior to new APIs, and then doing behavior
> changes later.

This is good but can you add an introduction that is more than "Emily's
hook config topic"? What is this patch series doing? (assume that
Emily's patches don't exist)

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 479+ messages in thread

* RE: [PATCH v2 13/30] read-cache: convert post-index-change hook to use config
  2021-06-14 10:33         ` [PATCH v2 13/30] read-cache: convert post-index-change hook to use config Ævar Arnfjörð Bjarmason
@ 2021-06-25 18:32           ` Felipe Contreras
  0 siblings, 0 replies; 479+ messages in thread
From: Felipe Contreras @ 2021-06-25 18:32 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason, git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Ævar Arnfjörð Bjarmason wrote:
> From: Emily Shaffer <emilyshaffer@google.com>
> 
> By using hook.h instead of run-command.h to run, post-index-change hooks
> can now be specified in the config in addition to the hookdir.
> post-index-change is not run anywhere besides in read-cache.c.
> 
> This removes the last direct user of run_hook_ve(), so we can make the
> function static now. It'll be removed entirely soon.

This is mixing two logically independent changes. For the same reason I
suggested to split the run_hook_le() removal I think this should be
split too.

Not a big deal though, especially since the splitted patch would be very
small.

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 479+ messages in thread

* RE: [PATCH v2 15/30] run-command: remove old run_hook_{le,ve}() hook API
  2021-06-14 10:33         ` [PATCH v2 15/30] run-command: remove old run_hook_{le,ve}() hook API Ævar Arnfjörð Bjarmason
@ 2021-06-25 18:34           ` Felipe Contreras
  0 siblings, 0 replies; 479+ messages in thread
From: Felipe Contreras @ 2021-06-25 18:34 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason, git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Ævar Arnfjörð Bjarmason wrote:
> From: Emily Shaffer <emilyshaffer@google.com>
> 
> The new hook.h library has replaced all run-command.h hook-related
> functionality. So let's delete this dead code.

Ahhh, much less mental load of the reviewer ;)

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 01/30] hook: add 'run' subcommand
  2021-06-18 22:13               ` Emily Shaffer
  2021-06-20 19:30                 ` Ævar Arnfjörð Bjarmason
@ 2021-06-25 19:02                 ` Felipe Contreras
  1 sibling, 0 replies; 479+ messages in thread
From: Felipe Contreras @ 2021-06-25 19:02 UTC (permalink / raw)
  To: Emily Shaffer, Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

Emily Shaffer wrote:
> On Tue, Jun 15, 2021 at 11:36:26AM +0200, Ævar Arnfjörð Bjarmason wrote:
> > > Overall, I think I like the direction your reroll is going - I've needed
> > > some time to process it. Hopefully I'll be able to get through all or
> > > most of the series this week, but there's a lot going on here, too. I'll
> > > do what I can. Thanks for the help.
> > 
> > Yeah, will reply to any qusetions etc; and as noted above my initial
> > goal here was "hey, what about this approach", so if you wanted to pick
> > this up & run with it...
> > 
> > This particular version of the series is at github.com/avar/git.git's
> > es-avar/config-based-hooks-3 b.t.w.
> 
> Have finished scanning through the rest of the series, and I think I
> understand your goal a little better - you are not saying "let me take
> over and drive this part of the feature set", which is what I thought
> initially. Instead, you seem to be saying "let's chop it up this way
> instead".

Indeed. In particular Ævar's chopping allowed me to visualize what the
patches were trying to do and it was much easier to review. Step by
step. I don't know about others, but I think it's similar.

> I don't dislike the reorganization, but I do still wonder whether it's
> a setback to the progress the original series had made. I guess it is
> hard to know - I had thought the original series was pretty much ready
> to go in, therefore making "what if we ordered it this way" moot. But it
> seems that you disagree.
> 
> Anyway, I do hear also that you don't have interest in driving this
> subset to completion, and that's fine. Correct me if I'm wrong.

In an open source project nobody "owns" a set of patches, we can all
work on them collaboratively.

> I'll keep thinking on this over the weekend. Thanks for the suggestion.

My suggestion is to not think about it too much. Just find what would be
the next logical step to do on top of Ævar's base and simply do it
(cherry-pick or rebase).

By simply trying it out you would get a much better idea of how the
series could progress to the end-goal you have in mind.

Cheers.

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 01/30] hook: add 'run' subcommand
  2021-06-14 21:33           ` Emily Shaffer
  2021-06-15  9:36             ` Ævar Arnfjörð Bjarmason
@ 2021-06-25 19:08             ` Felipe Contreras
  1 sibling, 0 replies; 479+ messages in thread
From: Felipe Contreras @ 2021-06-25 19:08 UTC (permalink / raw)
  To: Emily Shaffer, Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

Emily Shaffer wrote:
> On Mon, Jun 14, 2021 at 12:32:50PM +0200, Ævar Arnfjörð Bjarmason wrote:
> > 
> > In order to enable hooks to be run as an external process, by a
> > standalone Git command, or by tools which wrap Git, provide an external
> > means to run all configured hook commands for a given hook event.
> 
> From what it says on the box, I'm slightly worried about this patch
> doing too much at once, but let's see... (I think this is also a common
> thing you and I disagree on - how much work to do per commit - so feel
> free to ignore me ;) )

From my cursory look this is the only big patch, and I trust the reason
Ævar made it so big is that he couldn't figure out a way to chop it into
smaller pieces that would somehow work.

> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> 
> Thanks for including attribution - I appreciate it.

It's not really an attribution, since he used your code he is pretty
much obligated to put your s-o-b, see Developer Certificate of Origin

https://en.wikipedia.org/wiki/Developer_Certificate_of_Origin

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 01/30] hook: add 'run' subcommand
  2021-06-22  0:00                   ` Emily Shaffer
@ 2021-06-29  1:12                     ` Junio C Hamano
  0 siblings, 0 replies; 479+ messages in thread
From: Junio C Hamano @ 2021-06-29  1:12 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: Ævar Arnfjörð Bjarmason, git, Jeff King,
	Taylor Blau, Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee

Emily Shaffer <emilyshaffer@google.com> writes:

> Ok. Thanks for clarifying.
>
> Yes, I do like this direction, and I'm pleased you were able to chop it
> up in a way where partial submission made sense - I struggled with that,
> myself. Yes, I am excited that you want to drive this series :) :) and
> will be happy to rebase on top of it.
>
> I'll have a look at the range-diff for v3 this week. Thanks.

Thanks for working together so well.  Very pleased to see
contributors and reviewers working together constructively.


^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 07/27] git hook run: add an --ignore-missing flag
  2021-06-17 10:22           ` [PATCH 07/27] git hook run: add an --ignore-missing flag Ævar Arnfjörð Bjarmason
@ 2021-07-02 23:47             ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-07-02 23:47 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

On Thu, Jun 17, 2021 at 12:22:41PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> For certain one-shot hooks we'd like to optimistically run them, and
> not complain if they don't exist. This will be used by send-email in a
> subsequent commit.
> 
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>  Documentation/git-hook.txt | 10 +++++++++-
>  builtin/hook.c             |  8 +++++++-
>  t/t1800-hook.sh            |  7 ++++++-
>  3 files changed, 22 insertions(+), 3 deletions(-)
> 
> diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
> index 660d6a992a..097fb9de63 100644
> --- a/Documentation/git-hook.txt
> +++ b/Documentation/git-hook.txt
> @@ -8,7 +8,7 @@ git-hook - run git hooks
>  SYNOPSIS
>  --------
>  [verse]
> -'git hook' run <hook-name> [-- <hook-args>]
> +'git hook' run [--ignore-missing] <hook-name> [-- <hook-args>]
>  
>  DESCRIPTION
>  -----------
> @@ -29,6 +29,14 @@ optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
>  arguments (if any) differ by hook name, see linkgit:githooks[5] for
>  what those are.
>  
> +OPTIONS
> +-------
> +
> +--ignore-missing::
> +	Ignore any missing hook by quietly returning zero. Used for
> +	tools that want to do a blind one-shot run of a hook that may
> +	or may not be present.
> +
>  SEE ALSO
>  --------
>  linkgit:githooks[5]
> diff --git a/builtin/hook.c b/builtin/hook.c
> index 7714d31ef1..47e0de7bbc 100644
> --- a/builtin/hook.c
> +++ b/builtin/hook.c
> @@ -22,10 +22,13 @@ static int run(int argc, const char **argv, const char *prefix)
>  	int i;
>  	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
>  	int rc = 0;
> +	int ignore_missing = 0;
>  	const char *hook_name;
>  	const char *hook_path;
>  
>  	struct option run_options[] = {
> +		OPT_BOOL(0, "ignore-missing", &ignore_missing,
> +			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
>  		OPT_END(),
>  	};
>  
> @@ -49,11 +52,14 @@ static int run(int argc, const char **argv, const char *prefix)
>  	/*
>  	 * We are not using run_hooks() because we'd like to detect
>  	 * missing hooks. Let's find it ourselves and call
> -	 * run_found_hooks() instead.
> +	 * run_found_hooks() instead...
>  	 */
>  	hook_name = argv[0];
>  	hook_path = find_hook(hook_name);
>  	if (!hook_path) {
> +		/* ... act like run_hooks() under --ignore-missing */
> +		if (ignore_missing)
> +			return 0;
>  		error("cannot find a hook named %s", hook_name);
>  		return 1;
>  	}

Have started to rebase the configgy stuff on top of this series, and the
final implementation here is kind of confusing to me. The difference
between run_hooks() and run_found_hooks() is the behavior when there's
no hook to run; but you're checking for missing hooks before you even
let them try to run here in builtin/hook.c. So is it really necessary to
have the two methods? I think I'm missing something?

> diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
> index ecd517b162..542e551628 100755
> --- a/t/t1800-hook.sh
> +++ b/t/t1800-hook.sh
> @@ -23,7 +23,12 @@ test_expect_success 'git hook run: nonexistent hook' '
>  	test_cmp stderr.expect stderr.actual
>  '
>  
> -test_expect_success 'git hook run: basic' '
> +test_expect_success 'git hook run: nonexistent hook with --ignore-missing' '
> +	git hook run --ignore-missing does-not-exist 2>stderr.actual &&
> +	test_must_be_empty stderr.actual
> +'
> +
> +test_expect_success 'git hook run -- basic' '
>  	write_script .git/hooks/test-hook <<-EOF &&
>  	echo Test hook
>  	EOF
> -- 
> 2.32.0.576.g59759b6ca7d
> 

^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH 0/9] config-based hooks restarted
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (26 preceding siblings ...)
  2021-06-17 10:23           ` [PATCH 27/27] hooks: fix a TOCTOU in "did we run a hook?" heuristic Ævar Arnfjörð Bjarmason
@ 2021-07-15 23:25           ` Emily Shaffer
  2021-07-15 23:25             ` [PATCH 1/9] hook: run a list of hooks instead Emily Shaffer
                               ` (8 more replies)
  2021-07-28 20:39           ` [PATCH 00/27] Base for "config-based-hooks" Emily Shaffer
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
  29 siblings, 9 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-07-15 23:25 UTC (permalink / raw)
  To: git
  Cc: Emily Shaffer, Junio C Hamano, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m. carlson, Josh Steadmon,
	Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

The work formerly found at
https://lore.kernel.org/git/20210527000856.695702-1-emilyshaffer@google.com
has been rebased on top of
https://lore.kernel.org/git/cover-00.27-0000000000-20210617T101216Z-avarab@gmail.com
(ab/config-based-hooks-base).

Some features have been dropped from v9 of the larger config-based-hooks
series, and many patches have been replaced by Ævar's new base.

This series no longer introduces the 'hook.runHooksDir' config - I think
it is OK to introduce that later and quibble about it in isolation from
the rest of this work.

I also would encourage folks interested in reviewing this to not spend
too much time looking through it without first leaving a review on
Ævar's series which this is based on; this series won't go anywhere
without that one. :) (And yes, I owe a review on the more recent version
of that series, too.)

Note: I am worried that somewhere around patch 6 a flaky test may have
been added - I saw flaky failures on the osx-clang and linux-musl tests
when I ran CI on GitHub. I *think* this was due to missing a field when
initializing 'struct hook' for addition to the hook linked list, and I
*think* I have fixed that, but if something jumps out to anyone in the
code or in the test, I'd really appreciate the hint. A passing CI run
for the tip of this series is at
https://github.com/nasamuffin/git/actions/runs/1035559898.

 - Emily

Emily Shaffer (9):
  hook: run a list of hooks instead
  hook: allow parallel hook execution
  hook: introduce "git hook list"
  hook: treat hookdir hook specially
  hook: allow running non-native hooks
  hook: include hooks from the config
  hook: allow out-of-repo 'git hook' invocations
  hook: teach 'hookcmd' config to alias hook scripts
  hook: implement hookcmd.<name>.skip

 Documentation/config/hook.txt |  22 +++
 Documentation/git-hook.txt    |  97 ++++++++++++-
 builtin/am.c                  |  12 +-
 builtin/checkout.c            |   3 +-
 builtin/clone.c               |   3 +-
 builtin/gc.c                  |   3 +-
 builtin/hook.c                |  58 +++++++-
 builtin/merge.c               |   3 +-
 builtin/rebase.c              |   3 +-
 builtin/receive-pack.c        |  14 +-
 builtin/worktree.c            |   3 +-
 commit.c                      |   4 +-
 git.c                         |   2 +-
 hook.c                        | 251 +++++++++++++++++++++++++++++++---
 hook.h                        |  44 ++++--
 read-cache.c                  |   3 +-
 refs.c                        |   3 +-
 reset.c                       |   4 +-
 sequencer.c                   |   7 +-
 t/t1360-config-based-hooks.sh | 227 ++++++++++++++++++++++++++++++
 transport.c                   |   4 +-
 21 files changed, 711 insertions(+), 59 deletions(-)
 create mode 100644 Documentation/config/hook.txt
 create mode 100755 t/t1360-config-based-hooks.sh

-- 
2.32.0.402.g57bb445576-goog


^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH 1/9] hook: run a list of hooks instead
  2021-07-15 23:25           ` [PATCH 0/9] config-based hooks restarted Emily Shaffer
@ 2021-07-15 23:25             ` Emily Shaffer
  2021-07-15 23:25             ` [PATCH 2/9] hook: allow parallel hook execution Emily Shaffer
                               ` (7 subsequent siblings)
  8 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-07-15 23:25 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

To prepare for multihook support, teach hook.[hc] to take a list of
hooks at run_hooks and run_found_hooks. Right now the list is always one
entry, but in the future we will allow users to supply more than one
executable for a single hook event.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 builtin/hook.c |  9 +++---
 hook.c         | 85 ++++++++++++++++++++++++++++++++++++++++----------
 hook.h         | 15 ++++++++-
 3 files changed, 87 insertions(+), 22 deletions(-)

diff --git a/builtin/hook.c b/builtin/hook.c
index 169a8dd08f..a41ff36da9 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -25,7 +25,7 @@ static int run(int argc, const char **argv, const char *prefix)
 	int rc = 0;
 	int ignore_missing = 0;
 	const char *hook_name;
-	const char *hook_path;
+	struct list_head *hooks;
 
 	struct option run_options[] = {
 		OPT_BOOL(0, "ignore-missing", &ignore_missing,
@@ -58,15 +58,16 @@ static int run(int argc, const char **argv, const char *prefix)
 	 * run_found_hooks() instead...
 	 */
 	hook_name = argv[0];
-	hook_path = find_hook(hook_name);
-	if (!hook_path) {
+	hooks = hook_list(hook_name);
+	if (list_empty(hooks)) {
 		/* ... act like run_hooks() under --ignore-missing */
 		if (ignore_missing)
 			return 0;
 		error("cannot find a hook named %s", hook_name);
 		return 1;
 	}
-	rc = run_found_hooks(hook_name, hook_path, &opt);
+
+	rc = run_found_hooks(hook_name, hooks, &opt);
 
 	run_hooks_opt_clear(&opt);
 
diff --git a/hook.c b/hook.c
index 31e822bf51..c1dac6982f 100644
--- a/hook.c
+++ b/hook.c
@@ -4,6 +4,28 @@
 #include "hook-list.h"
 #include "config.h"
 
+static void free_hook(struct hook *ptr)
+{
+	if (ptr) {
+		free(ptr->feed_pipe_cb_data);
+	}
+	free(ptr);
+}
+
+static void remove_hook(struct list_head *to_remove)
+{
+	struct hook *hook_to_remove = list_entry(to_remove, struct hook, list);
+	list_del(to_remove);
+	free_hook(hook_to_remove);
+}
+
+void clear_hook_list(struct list_head *head)
+{
+	struct list_head *pos, *tmp;
+	list_for_each_safe(pos, tmp, head)
+		remove_hook(pos);
+}
+
 static int known_hook(const char *name)
 {
 	const char **p;
@@ -71,6 +93,30 @@ int hook_exists(const char *name)
 	return !!find_hook(name);
 }
 
+struct list_head* hook_list(const char* hookname)
+{
+	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
+
+	INIT_LIST_HEAD(hook_head);
+
+	if (!hookname)
+		return NULL;
+
+	if (have_git_dir()) {
+		const char *hook_path = find_hook(hookname);
+
+		/* Add the hook from the hookdir */
+		if (hook_path) {
+			struct hook *to_add = xmalloc(sizeof(*to_add));
+			to_add->hook_path = hook_path;
+			to_add->feed_pipe_cb_data = NULL;
+			list_add_tail(&to_add->list, hook_head);
+		}
+	}
+
+	return hook_head;
+}
+
 void run_hooks_opt_clear(struct run_hooks_opt *o)
 {
 	strvec_clear(&o->env);
@@ -108,6 +154,8 @@ static int pick_next_hook(struct child_process *cp,
 	struct hook_cb_data *hook_cb = pp_cb;
 	struct hook *run_me = hook_cb->run_me;
 
+	if (!run_me)
+		return 0;
 
 	/* reopen the file for stdin; run_command closes it. */
 	if (hook_cb->options->path_to_stdin) {
@@ -126,7 +174,10 @@ static int pick_next_hook(struct child_process *cp,
 	cp->dir = hook_cb->options->dir;
 
 	/* add command */
-	strvec_push(&cp->args, run_me->hook_path);
+	if (hook_cb->options->absolute_path)
+		strvec_push(&cp->args, absolute_path(run_me->hook_path));
+	else
+		strvec_push(&cp->args, run_me->hook_path);
 
 	/*
 	 * add passed-in argv, without expanding - let the user get back
@@ -137,6 +188,13 @@ static int pick_next_hook(struct child_process *cp,
 	/* Provide context for errors if necessary */
 	*pp_task_cb = run_me;
 
+	/* Get the next entry ready */
+	if (hook_cb->run_me->list.next == hook_cb->head)
+		hook_cb->run_me = NULL;
+	else
+		hook_cb->run_me = list_entry(hook_cb->run_me->list.next,
+					     struct hook, list);
+
 	return 1;
 }
 
@@ -170,24 +228,17 @@ static int notify_hook_finished(int result,
 	return 1;
 }
 
-int run_found_hooks(const char *hook_name, const char *hook_path,
+int run_found_hooks(const char *hook_name, struct list_head *hooks,
 		    struct run_hooks_opt *options)
 {
-	struct strbuf abs_path = STRBUF_INIT;
-	struct hook my_hook = {
-		.hook_path = hook_path,
-	};
 	struct hook_cb_data cb_data = {
 		.rc = 0,
 		.hook_name = hook_name,
 		.options = options,
 		.invoked_hook = options->invoked_hook,
 	};
-	if (options->absolute_path) {
-		strbuf_add_absolute_path(&abs_path, hook_path);
-		my_hook.hook_path = abs_path.buf;
-	}
-	cb_data.run_me = &my_hook;
+
+	cb_data.run_me = list_first_entry(hooks, struct hook, list);
 
 	if (options->jobs != 1)
 		BUG("we do not handle %d or any other != 1 job number yet", options->jobs);
@@ -201,15 +252,13 @@ int run_found_hooks(const char *hook_name, const char *hook_path,
 				   &cb_data,
 				   "hook",
 				   hook_name);
-	if (options->absolute_path)
-		strbuf_release(&abs_path);
 
 	return cb_data.rc;
 }
 
 int run_hooks(const char *hook_name, struct run_hooks_opt *options)
 {
-	const char *hook_path;
+	struct list_head *hooks;
 	int ret;
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
@@ -217,15 +266,17 @@ int run_hooks(const char *hook_name, struct run_hooks_opt *options)
 	if (options->path_to_stdin && options->feed_pipe)
 		BUG("choose only one method to populate stdin");
 
-	hook_path = find_hook(hook_name);
+	hooks = hook_list(hook_name);
 
 	/*
 	 * If you need to act on a missing hook, use run_found_hooks()
 	 * instead
 	 */
-	if (!hook_path)
+	if (list_empty(hooks))
 		return 0;
 
-	ret = run_found_hooks(hook_name, hook_path, options);
+	ret = run_found_hooks(hook_name, hooks, options);
+
+	clear_hook_list(hooks);
 	return ret;
 }
diff --git a/hook.h b/hook.h
index 9d9171672d..b97237931b 100644
--- a/hook.h
+++ b/hook.h
@@ -3,6 +3,7 @@
 #include "strbuf.h"
 #include "strvec.h"
 #include "run-command.h"
+#include "list.h"
 
 /*
  * Returns the path to the hook file, or NULL if the hook is missing
@@ -17,6 +18,7 @@ const char *find_hook(const char *name);
 int hook_exists(const char *hookname);
 
 struct hook {
+	struct list_head list;
 	/* The path to the hook */
 	const char *hook_path;
 
@@ -27,6 +29,12 @@ struct hook {
 	void *feed_pipe_cb_data;
 };
 
+/*
+ * Provides a linked list of 'struct hook' detailing commands which should run
+ * in response to the 'hookname' event, in execution order.
+ */
+struct list_head* hook_list(const char *hookname);
+
 struct run_hooks_opt
 {
 	/* Environment vars to be set for each hook */
@@ -103,6 +111,7 @@ struct hook_cb_data {
 	/* rc reflects the cumulative failure state */
 	int rc;
 	const char *hook_name;
+	struct list_head *head;
 	struct hook *run_me;
 	struct run_hooks_opt *options;
 	int *invoked_hook;
@@ -120,6 +129,10 @@ int run_hooks(const char *hook_name, struct run_hooks_opt *options);
  * Takes an already resolved hook and runs it. Internally the simpler
  * run_hooks() will call this.
  */
-int run_found_hooks(const char *hookname, const char *hook_path,
+int run_found_hooks(const char *hookname, struct list_head *hooks,
 		    struct run_hooks_opt *options);
+
+/* Empties the list at 'head', calling 'free_hook()' on each entry */
+void clear_hook_list(struct list_head *head);
+
 #endif
-- 
2.32.0.402.g57bb445576-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 2/9] hook: allow parallel hook execution
  2021-07-15 23:25           ` [PATCH 0/9] config-based hooks restarted Emily Shaffer
  2021-07-15 23:25             ` [PATCH 1/9] hook: run a list of hooks instead Emily Shaffer
@ 2021-07-15 23:25             ` Emily Shaffer
  2021-07-16  8:36               ` Ævar Arnfjörð Bjarmason
  2021-07-15 23:25             ` [PATCH 3/9] hook: introduce "git hook list" Emily Shaffer
                               ` (6 subsequent siblings)
  8 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-07-15 23:25 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

In many cases, there's no reason not to allow hooks to execute in
parallel. run_processes_parallel() is well-suited - it's a task queue
that runs its housekeeping in series, which means users don't
need to worry about thread safety on their callback data. True
multithreaded execution with the async_* functions isn't necessary here.
Synchronous hook execution can be achieved by only allowing 1 job to run
at a time.

Teach run_hooks() to use that function for simple hooks which don't
require stdin or capture of stderr.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/config/hook.txt |  4 ++++
 Documentation/git-hook.txt    | 17 ++++++++++++++++-
 builtin/am.c                  | 12 ++++++++----
 builtin/checkout.c            |  3 ++-
 builtin/clone.c               |  3 ++-
 builtin/gc.c                  |  3 ++-
 builtin/hook.c                |  6 +++++-
 builtin/merge.c               |  3 ++-
 builtin/rebase.c              |  3 ++-
 builtin/receive-pack.c        | 14 ++++++++++----
 builtin/worktree.c            |  3 ++-
 commit.c                      |  4 +++-
 hook.c                        | 32 +++++++++++++++++++++++++++++---
 hook.h                        | 11 +++++------
 read-cache.c                  |  3 ++-
 refs.c                        |  3 ++-
 reset.c                       |  4 +++-
 sequencer.c                   |  7 +++++--
 transport.c                   |  4 +++-
 19 files changed, 107 insertions(+), 32 deletions(-)
 create mode 100644 Documentation/config/hook.txt

diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
new file mode 100644
index 0000000000..96d3d6572c
--- /dev/null
+++ b/Documentation/config/hook.txt
@@ -0,0 +1,4 @@
+hook.jobs::
+	Specifies how many hooks can be run simultaneously during parallelized
+	hook execution. If unspecified, defaults to the number of processors on
+	the current system.
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index fa68c1f391..8bf82b5dd4 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -8,7 +8,8 @@ git-hook - run git hooks
 SYNOPSIS
 --------
 [verse]
-'git hook' run [--to-stdin=<path>] [--ignore-missing] <hook-name> [-- <hook-args>]
+'git hook' run [--to-stdin=<path>] [--ignore-missing] [(-j|--jobs) <n>]
+	<hook-name> [-- <hook-args>]
 
 DESCRIPTION
 -----------
@@ -42,6 +43,20 @@ OPTIONS
 	tools that want to do a blind one-shot run of a hook that may
 	or may not be present.
 
+-j::
+--jobs::
+	Only valid for `run`.
++
+Specify how many hooks to run simultaneously. If this flag is not specified, use
+the value of the `hook.jobs` config. If the config is not specified, use the
+number of CPUs on the current system. Some hooks may be ineligible for
+parallelization: for example, 'commit-msg' intends hooks modify the commit
+message body and cannot be parallelized.
+
+CONFIGURATION
+-------------
+include::config/hook.txt[]
+
 SEE ALSO
 --------
 linkgit:githooks[5]
diff --git a/builtin/am.c b/builtin/am.c
index 6e4f9c8036..bdad38142a 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -445,8 +445,9 @@ static void am_destroy(const struct am_state *state)
 static int run_applypatch_msg_hook(struct am_state *state)
 {
 	int ret;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
 
+	run_hooks_opt_init_sync(&opt);
 	assert(state->msg);
 	strvec_push(&opt.args, am_path(state, "final-commit"));
 	ret = run_hooks("applypatch-msg", &opt);
@@ -467,9 +468,10 @@ static int run_applypatch_msg_hook(struct am_state *state)
  */
 static int run_post_rewrite_hook(const struct am_state *state)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
 	int ret;
 
+	run_hooks_opt_init_async(&opt);
 	strvec_push(&opt.args, "rebase");
 	opt.path_to_stdin = am_path(state, "rewritten");
 
@@ -1602,9 +1604,10 @@ static void do_commit(const struct am_state *state)
 	struct commit_list *parents = NULL;
 	const char *reflog_msg, *author, *committer = NULL;
 	struct strbuf sb = STRBUF_INIT;
-	struct run_hooks_opt hook_opt_pre = RUN_HOOKS_OPT_INIT;
-	struct run_hooks_opt hook_opt_post = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt_pre;
+	struct run_hooks_opt hook_opt_post;
 
+	run_hooks_opt_init_async(&hook_opt_pre);
 	if (run_hooks("pre-applypatch", &hook_opt_pre)) {
 		run_hooks_opt_clear(&hook_opt_pre);
 		exit(1);
@@ -1659,6 +1662,7 @@ static void do_commit(const struct am_state *state)
 		fclose(fp);
 	}
 
+	run_hooks_opt_init_async(&hook_opt_post);
 	run_hooks("post-applypatch", &hook_opt_post);
 
 	run_hooks_opt_clear(&hook_opt_pre);
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 6205ace09f..be4450a433 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -107,9 +107,10 @@ struct branch_info {
 static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit,
 			      int changed)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
 	int rc;
 
+	run_hooks_opt_init_sync(&opt);
 	/* "new_commit" can be NULL when checking out from the index before
 	   a commit exists. */
 	strvec_pushl(&opt.args,
diff --git a/builtin/clone.c b/builtin/clone.c
index de57a3119b..87cfbf60e5 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -776,7 +776,7 @@ static int checkout(int submodule_progress)
 	struct tree *tree;
 	struct tree_desc t;
 	int err = 0;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt;
 
 	if (option_no_checkout)
 		return 0;
@@ -822,6 +822,7 @@ static int checkout(int submodule_progress)
 	if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
 		die(_("unable to write new index file"));
 
+	run_hooks_opt_init_sync(&hook_opt);
 	strvec_pushl(&hook_opt.args, oid_to_hex(null_oid()), oid_to_hex(&oid), "1", NULL);
 	err |= run_hooks("post-checkout", &hook_opt);
 	run_hooks_opt_clear(&hook_opt);
diff --git a/builtin/gc.c b/builtin/gc.c
index a12641a691..16890b097c 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -349,7 +349,7 @@ static void add_repack_incremental_option(void)
 
 static int need_to_gc(void)
 {
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt;
 
 	/*
 	 * Setting gc.auto to 0 or negative can disable the
@@ -397,6 +397,7 @@ static int need_to_gc(void)
 	else
 		return 0;
 
+	run_hooks_opt_init_async(&hook_opt);
 	if (run_hooks("pre-auto-gc", &hook_opt)) {
 		run_hooks_opt_clear(&hook_opt);
 		return 0;
diff --git a/builtin/hook.c b/builtin/hook.c
index a41ff36da9..d196d8498c 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -21,7 +21,7 @@ static const char * const builtin_hook_run_usage[] = {
 static int run(int argc, const char **argv, const char *prefix)
 {
 	int i;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
 	int rc = 0;
 	int ignore_missing = 0;
 	const char *hook_name;
@@ -32,9 +32,13 @@ static int run(int argc, const char **argv, const char *prefix)
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
 		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
 			   N_("file to read into hooks' stdin")),
+		OPT_INTEGER('j', "jobs", &opt.jobs,
+			    N_("run up to <n> hooks simultaneously")),
 		OPT_END(),
 	};
 
+	run_hooks_opt_init_async(&opt);
+
 	argc = parse_options(argc, argv, prefix, run_options,
 			     builtin_hook_run_usage,
 			     PARSE_OPT_KEEP_UNKNOWN | PARSE_OPT_KEEP_DASHDASH);
diff --git a/builtin/merge.c b/builtin/merge.c
index 0425c9bf2b..67c2eba053 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -448,7 +448,7 @@ static void finish(struct commit *head_commit,
 		   const struct object_id *new_head, const char *msg)
 {
 	struct strbuf reflog_message = STRBUF_INIT;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
 	const struct object_id *head = &head_commit->object.oid;
 
 	if (!msg)
@@ -490,6 +490,7 @@ static void finish(struct commit *head_commit,
 	}
 
 	/* Run a post-merge hook */
+	run_hooks_opt_init_async(&opt);
 	strvec_push(&opt.args, squash ? "1" : "0");
 	run_hooks("post-merge", &opt);
 	run_hooks_opt_clear(&opt);
diff --git a/builtin/rebase.c b/builtin/rebase.c
index 2081f6fa8d..fe9f144cad 100644
--- a/builtin/rebase.c
+++ b/builtin/rebase.c
@@ -1314,7 +1314,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	char *squash_onto_name = NULL;
 	int reschedule_failed_exec = -1;
 	int allow_preemptive_ff = 1;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt;
 	struct option builtin_rebase_options[] = {
 		OPT_STRING(0, "onto", &options.onto_name,
 			   N_("revision"),
@@ -2024,6 +2024,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	}
 
 	/* If a hook exists, give it a chance to interrupt*/
+	run_hooks_opt_init_async(&hook_opt);
 	strvec_pushl(&hook_opt.args, options.upstream_arg, argc ? argv[0] : NULL, NULL);
 	if (!ok_to_skip_pre_rebase &&
 	    run_hooks("pre-rebase", &hook_opt)) {
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index cd658f41d5..e4726eb211 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -910,7 +910,7 @@ static int run_receive_hook(struct command *commands,
 			    int skip_broken,
 			    const struct string_list *push_options)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
 	struct receive_hook_feed_context ctx;
 	int rc;
 	struct command *iter = commands;
@@ -922,6 +922,7 @@ static int run_receive_hook(struct command *commands,
 		return 0;
 
 	/* pre-receive hooks should run in series as the hook updates refs */
+	run_hooks_opt_init_async(&opt);
 	if (!strcmp(hook_name, "pre-receive"))
 		opt.jobs = 1;
 
@@ -956,9 +957,10 @@ static int run_receive_hook(struct command *commands,
 
 static int run_update_hook(struct command *cmd)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
 	int code;
 
+	run_hooks_opt_init_async(&opt);
 	strvec_pushl(&opt.args,
 		     cmd->ref_name,
 		     oid_to_hex(&cmd->old_oid),
@@ -1443,7 +1445,10 @@ static const char *push_to_checkout(unsigned char *hash,
 				    struct strvec *env,
 				    const char *work_tree)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
+
+
+	run_hooks_opt_init_sync(&opt);
 	opt.invoked_hook = invoked_hook;
 
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
@@ -1642,8 +1647,9 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 static void run_update_post_hook(struct command *commands)
 {
 	struct command *cmd;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
 
+	run_hooks_opt_init_async(&opt);
 	for (cmd = commands; cmd; cmd = cmd->next) {
 		if (cmd->error_string || cmd->did_not_exist)
 			continue;
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 2ad26a76f4..5a2c9d1039 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -382,8 +382,9 @@ static int add_worktree(const char *path, const char *refname,
 	 * is_junk is cleared, but do return appropriate code when hook fails.
 	 */
 	if (!ret && opts->checkout) {
-		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		struct run_hooks_opt opt;
 
+		run_hooks_opt_init_sync(&opt);
 		strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
 		strvec_pushl(&opt.args,
 			     oid_to_hex(null_oid()),
diff --git a/commit.c b/commit.c
index cf62ebceae..58ee2c81bb 100644
--- a/commit.c
+++ b/commit.c
@@ -1700,10 +1700,12 @@ int run_commit_hook(int editor_is_used, const char *index_file,
 		    int *invoked_hook,
 		    const char *name, ...)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
 	va_list args;
 	const char *arg;
 	int ret;
+
+	run_hooks_opt_init_sync(&opt);
 	strvec_pushf(&opt.env, "GIT_INDEX_FILE=%s", index_file);
 
 	/*
diff --git a/hook.c b/hook.c
index c1dac6982f..935751fa6c 100644
--- a/hook.c
+++ b/hook.c
@@ -88,6 +88,14 @@ const char *find_hook(const char *name)
 	return path.buf;
 }
 
+int configured_hook_jobs(void)
+{
+	int n = online_cpus();
+	git_config_get_int("hook.jobs", &n);
+
+	return n;
+}
+
 int hook_exists(const char *name)
 {
 	return !!find_hook(name);
@@ -117,6 +125,26 @@ struct list_head* hook_list(const char* hookname)
 	return hook_head;
 }
 
+void run_hooks_opt_init_sync(struct run_hooks_opt *o)
+{
+	strvec_init(&o->env);
+	strvec_init(&o->args);
+	o->path_to_stdin = NULL;
+	o->jobs = 1;
+	o->dir = NULL;
+	o->feed_pipe = NULL;
+	o->feed_pipe_ctx = NULL;
+	o->consume_sideband = NULL;
+	o->invoked_hook = NULL;
+	o->absolute_path = 0;
+}
+
+void run_hooks_opt_init_async(struct run_hooks_opt *o)
+{
+	run_hooks_opt_init_sync(o);
+	o->jobs = configured_hook_jobs();
+}
+
 void run_hooks_opt_clear(struct run_hooks_opt *o)
 {
 	strvec_clear(&o->env);
@@ -238,11 +266,9 @@ int run_found_hooks(const char *hook_name, struct list_head *hooks,
 		.invoked_hook = options->invoked_hook,
 	};
 
+	cb_data.head = hooks;
 	cb_data.run_me = list_first_entry(hooks, struct hook, list);
 
-	if (options->jobs != 1)
-		BUG("we do not handle %d or any other != 1 job number yet", options->jobs);
-
 	run_processes_parallel_tr2(options->jobs,
 				   pick_next_hook,
 				   notify_start_failure,
diff --git a/hook.h b/hook.h
index b97237931b..586ddf40bb 100644
--- a/hook.h
+++ b/hook.h
@@ -35,6 +35,9 @@ struct hook {
  */
 struct list_head* hook_list(const char *hookname);
 
+/* Provides the number of threads to use for parallel hook execution. */
+int configured_hook_jobs(void);
+
 struct run_hooks_opt
 {
 	/* Environment vars to be set for each hook */
@@ -93,12 +96,6 @@ struct run_hooks_opt
 	int *invoked_hook;
 };
 
-#define RUN_HOOKS_OPT_INIT { \
-	.jobs = 1, \
-	.env = STRVEC_INIT, \
-	.args = STRVEC_INIT, \
-}
-
 /*
  * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
  * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'.
@@ -117,6 +114,8 @@ struct hook_cb_data {
 	int *invoked_hook;
 };
 
+void run_hooks_opt_init_sync(struct run_hooks_opt *o);
+void run_hooks_opt_init_async(struct run_hooks_opt *o);
 void run_hooks_opt_clear(struct run_hooks_opt *o);
 
 /*
diff --git a/read-cache.c b/read-cache.c
index f801313cc9..e8cbbc6ef2 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -3063,7 +3063,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 {
 	int ret;
 	int was_full = !istate->sparse_index;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt;
 
 	ret = convert_to_sparse(istate);
 
@@ -3092,6 +3092,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 	else
 		ret = close_lock_file_gently(lock);
 
+	run_hooks_opt_init_async(&hook_opt);
 	strvec_pushl(&hook_opt.args,
 		     istate->updated_workdir ? "1" : "0",
 		     istate->updated_skipworktree ? "1" : "0",
diff --git a/refs.c b/refs.c
index 1149e7e7dc..61d0bb2579 100644
--- a/refs.c
+++ b/refs.c
@@ -2063,7 +2063,7 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 				const char *state)
 {
 	struct strbuf buf = STRBUF_INIT;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
 	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int ret = 0, i;
 	char o[GIT_MAX_HEXSZ + 1], n[GIT_MAX_HEXSZ + 1];
@@ -2071,6 +2071,7 @@ static int run_transaction_hook(struct ref_transaction *transaction,
 	if (!hook_exists("reference-transaction"))
 		return ret;
 
+	run_hooks_opt_init_async(&opt);
 	strvec_push(&opt.args, state);
 
 	for (i = 0; i < transaction->nr; i++) {
diff --git a/reset.c b/reset.c
index e6af33b901..48d45f5b79 100644
--- a/reset.c
+++ b/reset.c
@@ -128,7 +128,9 @@ int reset_head(struct repository *r, struct object_id *oid, const char *action,
 					    reflog_head);
 	}
 	if (run_hook) {
-		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		struct run_hooks_opt opt;
+
+		run_hooks_opt_init_sync(&opt);
 		strvec_pushl(&opt.args,
 			     oid_to_hex(orig ? orig : null_oid()),
 			     oid_to_hex(oid),
diff --git a/sequencer.c b/sequencer.c
index 2440b9dccd..17b93242a7 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1148,11 +1148,13 @@ int update_head_with_reflog(const struct commit *old_head,
 static int run_rewrite_hook(const struct object_id *oldoid,
 			    const struct object_id *newoid)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
 	struct strbuf tmp = STRBUF_INIT;
 	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int code;
 
+	run_hooks_opt_init_async(&opt);
+
 	strvec_push(&opt.args, "amend");
 
 	strbuf_addf(&tmp,
@@ -4524,7 +4526,7 @@ static int pick_commits(struct repository *r,
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
 			struct child_process notes_cp = CHILD_PROCESS_INIT;
-			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+			struct run_hooks_opt hook_opt;
 
 			notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY);
 			notes_cp.git_cmd = 1;
@@ -4534,6 +4536,7 @@ static int pick_commits(struct repository *r,
 			/* we don't care if this copying failed */
 			run_command(&notes_cp);
 
+			run_hooks_opt_init_async(&hook_opt);
 			hook_opt.path_to_stdin = rebase_path_rewritten_list();
 			strvec_push(&hook_opt.args, "rebase");
 			run_hooks("post-rewrite", &hook_opt);
diff --git a/transport.c b/transport.c
index 9969ed2cdd..3381d24225 100644
--- a/transport.c
+++ b/transport.c
@@ -1200,11 +1200,13 @@ static int run_pre_push_hook(struct transport *transport,
 			     struct ref *remote_refs)
 {
 	int ret = 0;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt;
 	struct strbuf tmp = STRBUF_INIT;
 	struct ref *r;
 	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 
+	run_hooks_opt_init_async(&opt);
+
 	strvec_push(&opt.args, transport->remote->name);
 	strvec_push(&opt.args, transport->url);
 
-- 
2.32.0.402.g57bb445576-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 3/9] hook: introduce "git hook list"
  2021-07-15 23:25           ` [PATCH 0/9] config-based hooks restarted Emily Shaffer
  2021-07-15 23:25             ` [PATCH 1/9] hook: run a list of hooks instead Emily Shaffer
  2021-07-15 23:25             ` [PATCH 2/9] hook: allow parallel hook execution Emily Shaffer
@ 2021-07-15 23:25             ` Emily Shaffer
  2021-07-16  8:52               ` Ævar Arnfjörð Bjarmason
  2021-07-15 23:25             ` [PATCH 4/9] hook: treat hookdir hook specially Emily Shaffer
                               ` (5 subsequent siblings)
  8 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-07-15 23:25 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

If more than one hook will be run, it may be useful to see a list of
which hooks should be run. At very least, it will be useful for us to
test the semantics of multihooks ourselves.

For now, only list the hooks which will run in the order they will run
in; later, it might be useful to include more information like where the
hooks were configured and whether or not they will run.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 builtin/hook.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 hook.c         | 18 ++++++++----------
 2 files changed, 51 insertions(+), 10 deletions(-)

diff --git a/builtin/hook.c b/builtin/hook.c
index d196d8498c..8340c8c9a8 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -8,6 +8,7 @@
 
 static const char * const builtin_hook_usage[] = {
 	N_("git hook <command> [...]"),
+	N_("git hook list <hookname>"),
 	N_("git hook run [<args>] <hook-name> [-- <hook-args>]"),
 	NULL
 };
@@ -18,6 +19,46 @@ static const char * const builtin_hook_run_usage[] = {
 	NULL
 };
 
+static int list(int argc, const char **argv, const char *prefix)
+{
+	struct list_head *head, *pos;
+	const char *hookname = NULL;
+	struct strbuf hookdir_annotation = STRBUF_INIT;
+
+	struct option list_options[] = {
+		OPT_END(),
+	};
+
+	argc = parse_options(argc, argv, prefix, list_options,
+			     builtin_hook_usage, 0);
+
+	if (argc < 1) {
+		usage_msg_opt(_("You must specify a hook event name to list."),
+			      builtin_hook_usage, list_options);
+	}
+
+	hookname = argv[0];
+
+	head = hook_list(hookname);
+
+	if (list_empty(head)) {
+		printf(_("no commands configured for hook '%s'\n"),
+		       hookname);
+		return 0;
+	}
+
+	list_for_each(pos, head) {
+		struct hook *item = list_entry(pos, struct hook, list);
+		item = list_entry(pos, struct hook, list);
+		if (item)
+			printf("%s\n", item->hook_path);
+	}
+
+	clear_hook_list(head);
+	strbuf_release(&hookdir_annotation);
+
+	return 0;
+}
 static int run(int argc, const char **argv, const char *prefix)
 {
 	int i;
@@ -88,6 +129,8 @@ int cmd_hook(int argc, const char **argv, const char *prefix)
 	if (!argc)
 		usage_with_options(builtin_hook_usage, builtin_hook_options);
 
+	if (!strcmp(argv[0], "list"))
+		return list(argc, argv, prefix);
 	if (!strcmp(argv[0], "run"))
 		return run(argc, argv, prefix);
 	else
diff --git a/hook.c b/hook.c
index 935751fa6c..19138a8290 100644
--- a/hook.c
+++ b/hook.c
@@ -104,22 +104,20 @@ int hook_exists(const char *name)
 struct list_head* hook_list(const char* hookname)
 {
 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
+	const char *hook_path = find_hook(hookname);
+
 
 	INIT_LIST_HEAD(hook_head);
 
 	if (!hookname)
 		return NULL;
 
-	if (have_git_dir()) {
-		const char *hook_path = find_hook(hookname);
-
-		/* Add the hook from the hookdir */
-		if (hook_path) {
-			struct hook *to_add = xmalloc(sizeof(*to_add));
-			to_add->hook_path = hook_path;
-			to_add->feed_pipe_cb_data = NULL;
-			list_add_tail(&to_add->list, hook_head);
-		}
+	/* Add the hook from the hookdir */
+	if (hook_path) {
+		struct hook *to_add = xmalloc(sizeof(*to_add));
+		to_add->hook_path = hook_path;
+		to_add->feed_pipe_cb_data = NULL;
+		list_add_tail(&to_add->list, hook_head);
 	}
 
 	return hook_head;
-- 
2.32.0.402.g57bb445576-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 4/9] hook: treat hookdir hook specially
  2021-07-15 23:25           ` [PATCH 0/9] config-based hooks restarted Emily Shaffer
                               ` (2 preceding siblings ...)
  2021-07-15 23:25             ` [PATCH 3/9] hook: introduce "git hook list" Emily Shaffer
@ 2021-07-15 23:25             ` Emily Shaffer
  2021-07-16  8:58               ` Ævar Arnfjörð Bjarmason
  2021-07-15 23:25             ` [PATCH 5/9] hook: allow running non-native hooks Emily Shaffer
                               ` (4 subsequent siblings)
  8 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-07-15 23:25 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Soon, we will allow users to specify hooks using the config. These
config-specified hooks may require different child_process options than
hook executables in the gitdir. So, let's differentiate between hooks
coming from the gitdir and hooks coming from the config.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 hook.c | 3 ++-
 hook.h | 2 ++
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/hook.c b/hook.c
index 19138a8290..3a588cb055 100644
--- a/hook.c
+++ b/hook.c
@@ -117,6 +117,7 @@ struct list_head* hook_list(const char* hookname)
 		struct hook *to_add = xmalloc(sizeof(*to_add));
 		to_add->hook_path = hook_path;
 		to_add->feed_pipe_cb_data = NULL;
+		to_add->from_hookdir = 1;
 		list_add_tail(&to_add->list, hook_head);
 	}
 
@@ -200,7 +201,7 @@ static int pick_next_hook(struct child_process *cp,
 	cp->dir = hook_cb->options->dir;
 
 	/* add command */
-	if (hook_cb->options->absolute_path)
+	if (run_me->from_hookdir && hook_cb->options->absolute_path)
 		strvec_push(&cp->args, absolute_path(run_me->hook_path));
 	else
 		strvec_push(&cp->args, run_me->hook_path);
diff --git a/hook.h b/hook.h
index 586ddf40bb..60389cd8cd 100644
--- a/hook.h
+++ b/hook.h
@@ -22,6 +22,8 @@ struct hook {
 	/* The path to the hook */
 	const char *hook_path;
 
+	unsigned from_hookdir : 1;
+
 	/*
 	 * Use this to keep state for your feed_pipe_fn if you are using
 	 * run_hooks_opt.feed_pipe. Otherwise, do not touch it.
-- 
2.32.0.402.g57bb445576-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 5/9] hook: allow running non-native hooks
  2021-07-15 23:25           ` [PATCH 0/9] config-based hooks restarted Emily Shaffer
                               ` (3 preceding siblings ...)
  2021-07-15 23:25             ` [PATCH 4/9] hook: treat hookdir hook specially Emily Shaffer
@ 2021-07-15 23:25             ` Emily Shaffer
  2021-07-15 23:26             ` [PATCH 6/9] hook: include hooks from the config Emily Shaffer
                               ` (3 subsequent siblings)
  8 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-07-15 23:25 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

As the hook architecture and 'git hook run' become more featureful, we
may find wrappers wanting to use the hook architecture to run their own
hooks, thereby getting nice things like parallelism and idiomatic Git
configuration for free. Enable this by letting 'git hook run' bypass the
known_hooks() check.

We do still want to keep known_hooks() around, though - by die()ing when
an internal Git call asks for run_hooks("my-new-hook"), we can remind
Git developers to update Documentation/githooks.txt with their new hook,
which in turn helps Git users discover this new hook.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 builtin/hook.c |  4 ++--
 hook.c         | 32 ++++++++++++++++++++++++++++----
 hook.h         | 16 +++++++++++++++-
 3 files changed, 45 insertions(+), 7 deletions(-)

diff --git a/builtin/hook.c b/builtin/hook.c
index 8340c8c9a8..b08f9c9c4f 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -39,7 +39,7 @@ static int list(int argc, const char **argv, const char *prefix)
 
 	hookname = argv[0];
 
-	head = hook_list(hookname);
+	head = hook_list(hookname, 1);
 
 	if (list_empty(head)) {
 		printf(_("no commands configured for hook '%s'\n"),
@@ -103,7 +103,7 @@ static int run(int argc, const char **argv, const char *prefix)
 	 * run_found_hooks() instead...
 	 */
 	hook_name = argv[0];
-	hooks = hook_list(hook_name);
+	hooks = hook_list(hook_name, 1);
 	if (list_empty(hooks)) {
 		/* ... act like run_hooks() under --ignore-missing */
 		if (ignore_missing)
diff --git a/hook.c b/hook.c
index 3a588cb055..e1cf035022 100644
--- a/hook.c
+++ b/hook.c
@@ -52,12 +52,21 @@ static int known_hook(const char *name)
 
 const char *find_hook(const char *name)
 {
-	static struct strbuf path = STRBUF_INIT;
+	const char *hook_path;
 
 	if (!known_hook(name))
 		die(_("the hook '%s' is not known to git, should be in hook-list.h via githooks(5)"),
 		    name);
 
+	hook_path = find_hook_gently(name);
+
+	return hook_path;
+}
+
+const char *find_hook_gently(const char *name)
+{
+	static struct strbuf path = STRBUF_INIT;
+
 	strbuf_reset(&path);
 	strbuf_git_path(&path, "hooks/%s", name);
 	if (access(path.buf, X_OK) < 0) {
@@ -101,10 +110,16 @@ int hook_exists(const char *name)
 	return !!find_hook(name);
 }
 
-struct list_head* hook_list(const char* hookname)
+struct hook_config_cb
+{
+	struct strbuf *hook_key;
+	struct list_head *list;
+};
+
+struct list_head* hook_list(const char* hookname, int allow_unknown)
 {
 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
-	const char *hook_path = find_hook(hookname);
+	const char *hook_path;
 
 
 	INIT_LIST_HEAD(hook_head);
@@ -112,6 +127,11 @@ struct list_head* hook_list(const char* hookname)
 	if (!hookname)
 		return NULL;
 
+	if (allow_unknown)
+		hook_path = find_hook_gently(hookname);
+	else
+		hook_path = find_hook(hookname);
+
 	/* Add the hook from the hookdir */
 	if (hook_path) {
 		struct hook *to_add = xmalloc(sizeof(*to_add));
@@ -291,7 +311,11 @@ int run_hooks(const char *hook_name, struct run_hooks_opt *options)
 	if (options->path_to_stdin && options->feed_pipe)
 		BUG("choose only one method to populate stdin");
 
-	hooks = hook_list(hook_name);
+	/*
+	 * 'git hooks run <hookname>' uses run_found_hooks, so we don't need to
+	 * allow unknown hooknames here.
+	 */
+	hooks = hook_list(hook_name, 0);
 
 	/*
 	 * If you need to act on a missing hook, use run_found_hooks()
diff --git a/hook.h b/hook.h
index 60389cd8cd..2559232880 100644
--- a/hook.h
+++ b/hook.h
@@ -9,8 +9,16 @@
  * Returns the path to the hook file, or NULL if the hook is missing
  * or disabled. Note that this points to static storage that will be
  * overwritten by further calls to find_hook and run_hook_*.
+ *
+ * If the hook is not a native hook (e.g. present in Documentation/githooks.txt)
+ * find_hook() will die(). find_hook_gently() does not consult the native hook
+ * list and will check for any hook name in the hooks directory regardless of
+ * whether it is known. find_hook() should be used by internal calls to hooks,
+ * and find_hook_gently() should only be used when the hookname was provided by
+ * the user, such as by 'git hook run my-wrapper-hook'.
  */
 const char *find_hook(const char *name);
+const char *find_hook_gently(const char *name);
 
 /*
  * A boolean version of find_hook()
@@ -34,8 +42,14 @@ struct hook {
 /*
  * Provides a linked list of 'struct hook' detailing commands which should run
  * in response to the 'hookname' event, in execution order.
+ *
+ * If allow_unknown is unset, hooks will be checked against the hook list
+ * generated from Documentation/githooks.txt. Otherwise, any hook name will be
+ * allowed. allow_unknown should only be set when the hook name is provided by
+ * the user; internal calls to hook_list should make sure the hook they are
+ * invoking is present in Documentation/githooks.txt.
  */
-struct list_head* hook_list(const char *hookname);
+struct list_head* hook_list(const char *hookname, int allow_unknown);
 
 /* Provides the number of threads to use for parallel hook execution. */
 int configured_hook_jobs(void);
-- 
2.32.0.402.g57bb445576-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 6/9] hook: include hooks from the config
  2021-07-15 23:25           ` [PATCH 0/9] config-based hooks restarted Emily Shaffer
                               ` (4 preceding siblings ...)
  2021-07-15 23:25             ` [PATCH 5/9] hook: allow running non-native hooks Emily Shaffer
@ 2021-07-15 23:26             ` Emily Shaffer
  2021-07-16  9:01               ` Ævar Arnfjörð Bjarmason
  2021-07-15 23:26             ` [PATCH 7/9] hook: allow out-of-repo 'git hook' invocations Emily Shaffer
                               ` (2 subsequent siblings)
  8 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-07-15 23:26 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach the hook.[hc] library to parse configs to populare the list of
hooks to run for a given event.

Multiple commands can be specified for a given hook by providing
multiple "hook.<hookname>.command = <path-to-hook>" lines. Hooks will be
run in config order.

For example:

  $ git config --list | grep ^hook
  hook.pre-commit.command=~/bar.sh

  $ git hook run
  # Runs ~/bar.sh
  # Runs .git/hooks/pre-commit

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/config/hook.txt |   5 ++
 Documentation/git-hook.txt    |  16 +++-
 builtin/hook.c                |   2 +-
 hook.c                        | 109 ++++++++++++++++++++---
 hook.h                        |   2 +-
 t/t1360-config-based-hooks.sh | 159 ++++++++++++++++++++++++++++++++++
 6 files changed, 278 insertions(+), 15 deletions(-)
 create mode 100755 t/t1360-config-based-hooks.sh

diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
index 96d3d6572c..a97b980cca 100644
--- a/Documentation/config/hook.txt
+++ b/Documentation/config/hook.txt
@@ -1,3 +1,8 @@
+hook.<command>.command::
+	A command to execute during the <command> hook event. This can be an
+	executable on your device, a oneliner for your shell, or the name of a
+	hookcmd. See linkgit:git-hook[1].
+
 hook.jobs::
 	Specifies how many hooks can be run simultaneously during parallelized
 	hook execution. If unspecified, defaults to the number of processors on
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 8bf82b5dd4..8e2ab724e8 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -18,12 +18,24 @@ This command is an interface to git hooks (see linkgit:githooks[5]).
 Currently it only provides a convenience wrapper for running hooks for
 use by git itself. In the future it might gain other functionality.
 
+This command parses the default configuration files for sections like `hook
+"<hook name>"`. `hook` is used to describe the commands which will be run during
+a particular hook event; commands are run in the order Git encounters them
+during the configuration parse (see linkgit:git-config[1]).
+
+In general, when instructions suggest adding a script to
+`.git/hooks/<something>`, you can specify it in the config instead by running
+`git config --add hook.<something>.command <path-to-script>` - this way you can
+share the script between multiple repos. That is, `cp ~/my-script.sh
+~/project/.git/hooks/pre-commit` would become `git config --add
+hook.pre-commit.command ~/my-script.sh`.
+
 SUBCOMMANDS
 -----------
 
 run::
-	Run the `<hook-name>` hook. See linkgit:githooks[5] for
-	the hook names we support.
+	Runs hooks configured for `<hook-name>`, in the order they are
+	discovered during the config parse.
 +
 Any positional arguments to the hook should be passed after an
 optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
diff --git a/builtin/hook.c b/builtin/hook.c
index b08f9c9c4f..c54b0a4d13 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -51,7 +51,7 @@ static int list(int argc, const char **argv, const char *prefix)
 		struct hook *item = list_entry(pos, struct hook, list);
 		item = list_entry(pos, struct hook, list);
 		if (item)
-			printf("%s\n", item->hook_path);
+			printf("%s\n", item->command);
 	}
 
 	clear_hook_list(head);
diff --git a/hook.c b/hook.c
index e1cf035022..ed90edcad7 100644
--- a/hook.c
+++ b/hook.c
@@ -12,6 +12,51 @@ static void free_hook(struct hook *ptr)
 	free(ptr);
 }
 
+/*
+ * Walks the linked list at 'head' to check if any hook running 'command'
+ * already exists. Returns a pointer to that hook if so, otherwise returns NULL.
+ */
+static struct hook * find_hook_by_command(struct list_head *head, const char *command)
+{
+	struct list_head *pos = NULL, *tmp = NULL;
+	struct hook *found = NULL;
+
+	list_for_each_safe(pos, tmp, head) {
+		struct hook *it = list_entry(pos, struct hook, list);
+		if (!strcmp(it->command, command)) {
+		    list_del(pos);
+		    found = it;
+		    break;
+		}
+	}
+	return found;
+}
+
+/*
+ * Adds a hook if it's not already in the list, or moves it to the tail of the
+ * list if it was already there.
+ * Returns a handle to the hook in case more modification is needed. Do not free
+ * the returned handle.
+ */
+static struct hook * append_or_move_hook(struct list_head *head, const char *command)
+{
+	/* check if the hook is already in the list */
+	struct hook *to_add = find_hook_by_command(head, command);
+
+	if (!to_add) {
+		/* adding a new hook, not moving an old one */
+		to_add = xmalloc(sizeof(*to_add));
+		to_add->command = command;
+		to_add->feed_pipe_cb_data = NULL;
+		/* This gets overwritten in hook_list() for hookdir hooks. */
+		to_add->from_hookdir = 0;
+	}
+
+	list_add_tail(&to_add->list, head);
+
+	return to_add;
+}
+
 static void remove_hook(struct list_head *to_remove)
 {
 	struct hook *hook_to_remove = list_entry(to_remove, struct hook, list);
@@ -116,30 +161,69 @@ struct hook_config_cb
 	struct list_head *list;
 };
 
+/*
+ * Callback for git_config which adds configured hooks to a hook list.
+ * Hooks can be configured by specifying hook.<name>.command, for example,
+ * hook.pre-commit.command = echo "pre-commit hook!"
+ */
+static int hook_config_lookup(const char *key, const char *value, void *cb_data)
+{
+	struct hook_config_cb *data = cb_data;
+	const char *hook_key = data->hook_key->buf;
+	struct list_head *head = data->list;
+
+	if (!strcmp(key, hook_key)) {
+		const char *command = value;
+		struct strbuf hookcmd_name = STRBUF_INIT;
+
+
+		if (!command) {
+			strbuf_release(&hookcmd_name);
+			BUG("git_config_get_value overwrote a string it shouldn't have");
+		}
+
+		/* TODO: implement skipping hooks */
+
+		/* TODO: immplement hook aliases */
+
+		/*
+		 * TODO: implement an option-getting callback, e.g.
+		 *   get configs by pattern hookcmd.$value.*
+		 *   for each key+value, do_callback(key, value, cb_data)
+		 */
+		append_or_move_hook(head, command);
+
+		strbuf_release(&hookcmd_name);
+	}
+
+	return 0;
+}
+
 struct list_head* hook_list(const char* hookname, int allow_unknown)
 {
 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
 	const char *hook_path;
-
+	struct strbuf hook_key = STRBUF_INIT;
+	struct hook_config_cb cb_data = { &hook_key, hook_head };
 
 	INIT_LIST_HEAD(hook_head);
 
 	if (!hookname)
 		return NULL;
 
+	/* Add the hooks from the config, e.g. hook.pre-commit.command */
+	strbuf_addf(&hook_key, "hook.%s.command", hookname);
+	git_config(hook_config_lookup, &cb_data);
+
+
 	if (allow_unknown)
 		hook_path = find_hook_gently(hookname);
 	else
 		hook_path = find_hook(hookname);
 
 	/* Add the hook from the hookdir */
-	if (hook_path) {
-		struct hook *to_add = xmalloc(sizeof(*to_add));
-		to_add->hook_path = hook_path;
-		to_add->feed_pipe_cb_data = NULL;
-		to_add->from_hookdir = 1;
-		list_add_tail(&to_add->list, hook_head);
-	}
+	if (hook_path)
+		append_or_move_hook(hook_head, hook_path)->from_hookdir = 1;
 
 	return hook_head;
 }
@@ -220,11 +304,14 @@ static int pick_next_hook(struct child_process *cp,
 	cp->trace2_hook_name = hook_cb->hook_name;
 	cp->dir = hook_cb->options->dir;
 
+	/* to enable oneliners, let config-specified hooks run in shell */
+	cp->use_shell = !run_me->from_hookdir;
+
 	/* add command */
 	if (run_me->from_hookdir && hook_cb->options->absolute_path)
-		strvec_push(&cp->args, absolute_path(run_me->hook_path));
+		strvec_push(&cp->args, absolute_path(run_me->command));
 	else
-		strvec_push(&cp->args, run_me->hook_path);
+		strvec_push(&cp->args, run_me->command);
 
 	/*
 	 * add passed-in argv, without expanding - let the user get back
@@ -255,7 +342,7 @@ static int notify_start_failure(struct strbuf *out,
 	hook_cb->rc |= 1;
 
 	strbuf_addf(out, _("Couldn't start hook '%s'\n"),
-		    attempted->hook_path);
+		    attempted->command);
 
 	return 1;
 }
diff --git a/hook.h b/hook.h
index 2559232880..e8cd6b7c67 100644
--- a/hook.h
+++ b/hook.h
@@ -28,7 +28,7 @@ int hook_exists(const char *hookname);
 struct hook {
 	struct list_head list;
 	/* The path to the hook */
-	const char *hook_path;
+	const char *command;
 
 	unsigned from_hookdir : 1;
 
diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh
new file mode 100755
index 0000000000..12fca516ec
--- /dev/null
+++ b/t/t1360-config-based-hooks.sh
@@ -0,0 +1,159 @@
+#!/bin/bash
+
+test_description='config-managed multihooks, including git-hook command'
+
+. ./test-lib.sh
+
+ROOT=
+if test_have_prereq MINGW
+then
+	# In Git for Windows, Unix-like paths work only in shell scripts;
+	# `git.exe`, however, will prefix them with the pseudo root directory
+	# (of the Unix shell). Let's accommodate for that.
+	ROOT="$(cd / && pwd)"
+fi
+
+setup_hooks () {
+	test_config hook.pre-commit.command "/path/ghi" --add
+	test_config_global hook.pre-commit.command "/path/def" --add
+}
+
+setup_hookdir () {
+	mkdir .git/hooks
+	write_script .git/hooks/pre-commit <<-EOF
+	echo \"Legacy Hook\"
+	EOF
+	test_when_finished rm -rf .git/hooks
+}
+
+test_expect_success 'git hook rejects commands without a mode' '
+	test_must_fail git hook pre-commit
+'
+
+test_expect_success 'git hook rejects commands without a hookname' '
+	test_must_fail git hook list
+'
+
+test_expect_success 'git hook list orders by config order' '
+	setup_hooks &&
+
+	cat >expected <<-EOF &&
+	$ROOT/path/def
+	$ROOT/path/ghi
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list reorders on duplicate commands' '
+	setup_hooks &&
+
+	test_config hook.pre-commit.command "/path/def" --add &&
+
+	cat >expected <<-EOF &&
+	$ROOT/path/ghi
+	$ROOT/path/def
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list shows hooks from the hookdir' '
+	setup_hookdir &&
+
+	cat >expected <<-EOF &&
+	.git/hooks/pre-commit
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+
+test_expect_success 'inline hook definitions execute oneliners' '
+	test_config hook.pre-commit.command "echo \"Hello World\"" &&
+
+	echo "Hello World" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'inline hook definitions resolve paths' '
+	write_script sample-hook.sh <<-EOF &&
+	echo \"Sample Hook\"
+	EOF
+
+	test_when_finished "rm sample-hook.sh" &&
+
+	test_config hook.pre-commit.command "\"$(pwd)/sample-hook.sh\"" &&
+
+	echo \"Sample Hook\" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook run can pass args' '
+	write_script sample-hook.sh <<-\EOF &&
+	echo $1
+	echo $2
+	EOF
+
+	test_config hook.pre-commit.command "\"$(pwd)/sample-hook.sh\"" &&
+
+	cat >expected <<-EOF &&
+	arg1
+	arg2
+	EOF
+
+	git hook run pre-commit -- arg1 arg2 2>actual &&
+
+	test_cmp expected actual
+'
+
+test_expect_success 'hookdir hook included in git hook run' '
+	setup_hookdir &&
+
+	echo \"Legacy Hook\" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'out-of-repo runs excluded' '
+	setup_hooks &&
+
+	nongit test_must_fail git hook run pre-commit
+'
+
+test_expect_success 'stdin to multiple hooks' '
+	git config --add hook.test.command "xargs -P1 -I% echo a%" &&
+	git config --add hook.test.command "xargs -P1 -I% echo b%" &&
+	test_when_finished "test_unconfig hook.test.command" &&
+
+	cat >input <<-EOF &&
+	1
+	2
+	3
+	EOF
+
+	cat >expected <<-EOF &&
+	a1
+	a2
+	a3
+	b1
+	b2
+	b3
+	EOF
+
+	git hook run --to-stdin=input test 2>actual &&
+	test_cmp expected actual
+'
+
+test_done
-- 
2.32.0.402.g57bb445576-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 7/9] hook: allow out-of-repo 'git hook' invocations
  2021-07-15 23:25           ` [PATCH 0/9] config-based hooks restarted Emily Shaffer
                               ` (5 preceding siblings ...)
  2021-07-15 23:26             ` [PATCH 6/9] hook: include hooks from the config Emily Shaffer
@ 2021-07-15 23:26             ` Emily Shaffer
  2021-07-16  8:33               ` Ævar Arnfjörð Bjarmason
  2021-07-15 23:26             ` [PATCH 8/9] hook: teach 'hookcmd' config to alias hook scripts Emily Shaffer
  2021-07-15 23:26             ` [PATCH 9/9] hook: implement hookcmd.<name>.skip Emily Shaffer
  8 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-07-15 23:26 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Since hooks can now be supplied via the config, and a config can be
present without a gitdir via the global and system configs, we can start
to allow 'git hook run' to occur without a gitdir. This enables us to do
things like run sendemail-validate hooks when running 'git send-email'
from a nongit directory.

It still doesn't make sense to look for hooks in the hookdir in nongit
repos, though, as there is no hookdir.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

Notes:
    For hookdir hooks, do we want to run them in nongit dir when core.hooksPath
    is set? For example, if someone set core.hooksPath in their global config and
    then ran 'git hook run sendemail-validate' in a nongit dir?

 git.c                         |  2 +-
 hook.c                        | 18 ++++++++++--------
 t/t1360-config-based-hooks.sh | 13 +++++++++++++
 3 files changed, 24 insertions(+), 9 deletions(-)

diff --git a/git.c b/git.c
index 540909c391..39988ee3b0 100644
--- a/git.c
+++ b/git.c
@@ -538,7 +538,7 @@ static struct cmd_struct commands[] = {
 	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
 	{ "hash-object", cmd_hash_object },
 	{ "help", cmd_help },
-	{ "hook", cmd_hook, RUN_SETUP },
+	{ "hook", cmd_hook, RUN_SETUP_GENTLY },
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "init", cmd_init_db },
 	{ "init-db", cmd_init_db },
diff --git a/hook.c b/hook.c
index ed90edcad7..b08b876d5d 100644
--- a/hook.c
+++ b/hook.c
@@ -202,7 +202,6 @@ static int hook_config_lookup(const char *key, const char *value, void *cb_data)
 struct list_head* hook_list(const char* hookname, int allow_unknown)
 {
 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
-	const char *hook_path;
 	struct strbuf hook_key = STRBUF_INIT;
 	struct hook_config_cb cb_data = { &hook_key, hook_head };
 
@@ -216,14 +215,17 @@ struct list_head* hook_list(const char* hookname, int allow_unknown)
 	git_config(hook_config_lookup, &cb_data);
 
 
-	if (allow_unknown)
-		hook_path = find_hook_gently(hookname);
-	else
-		hook_path = find_hook(hookname);
+	if (have_git_dir()) {
+		const char *hook_path;
+		if (allow_unknown)
+			hook_path = find_hook_gently(hookname);
+		else
+			hook_path = find_hook(hookname);
 
-	/* Add the hook from the hookdir */
-	if (hook_path)
-		append_or_move_hook(hook_head, hook_path)->from_hookdir = 1;
+		/* Add the hook from the hookdir */
+		if (hook_path)
+			append_or_move_hook(hook_head, hook_path)->from_hookdir = 1;
+	}
 
 	return hook_head;
 }
diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh
index 12fca516ec..e4a7b06ad1 100755
--- a/t/t1360-config-based-hooks.sh
+++ b/t/t1360-config-based-hooks.sh
@@ -34,6 +34,19 @@ test_expect_success 'git hook rejects commands without a hookname' '
 	test_must_fail git hook list
 '
 
+test_expect_success 'git hook runs outside of a repo' '
+	setup_hooks &&
+
+	cat >expected <<-EOF &&
+	$ROOT/path/def
+	EOF
+
+	nongit git config --list --global &&
+
+	nongit git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
 test_expect_success 'git hook list orders by config order' '
 	setup_hooks &&
 
-- 
2.32.0.402.g57bb445576-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 8/9] hook: teach 'hookcmd' config to alias hook scripts
  2021-07-15 23:25           ` [PATCH 0/9] config-based hooks restarted Emily Shaffer
                               ` (6 preceding siblings ...)
  2021-07-15 23:26             ` [PATCH 7/9] hook: allow out-of-repo 'git hook' invocations Emily Shaffer
@ 2021-07-15 23:26             ` Emily Shaffer
  2021-07-16  9:13               ` Ævar Arnfjörð Bjarmason
  2021-07-15 23:26             ` [PATCH 9/9] hook: implement hookcmd.<name>.skip Emily Shaffer
  8 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-07-15 23:26 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

To enable fine-grained options which apply to a single hook executable,
and to make it easier for a single executable to be run on multiple hook
events, teach "hookcmd.<alias>.config". These can be configured as
follows:

  [hookcmd.my-linter]
    command = ~/my-linter.sh
  [hook.pre-commit]
    command = my-linter

During the config parse, we can attempt to dereference the
'hook.pre-commit.command' string 'my-linter' and check if it matches any
hookcmd names; if so, we can run the command associated with that
hookcmd alias instead.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/config/hook.txt |  5 +++++
 Documentation/git-hook.txt    | 42 +++++++++++++++++++++++++++++++----
 hook.c                        |  9 +++++++-
 t/t1360-config-based-hooks.sh | 19 ++++++++++++++++
 4 files changed, 70 insertions(+), 5 deletions(-)

diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
index a97b980cca..5b35170664 100644
--- a/Documentation/config/hook.txt
+++ b/Documentation/config/hook.txt
@@ -3,6 +3,11 @@ hook.<command>.command::
 	executable on your device, a oneliner for your shell, or the name of a
 	hookcmd. See linkgit:git-hook[1].
 
+hookcmd.<name>.command::
+	A command to execute during a hook for which <name> has been specified
+	as a command. This can be an executable on your device or a oneliner for
+	your shell. See linkgit:git-hook[1].
+
 hook.jobs::
 	Specifies how many hooks can be run simultaneously during parallelized
 	hook execution. If unspecified, defaults to the number of processors on
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 8e2ab724e8..1a4d22fd90 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -18,10 +18,44 @@ This command is an interface to git hooks (see linkgit:githooks[5]).
 Currently it only provides a convenience wrapper for running hooks for
 use by git itself. In the future it might gain other functionality.
 
-This command parses the default configuration files for sections like `hook
-"<hook name>"`. `hook` is used to describe the commands which will be run during
-a particular hook event; commands are run in the order Git encounters them
-during the configuration parse (see linkgit:git-config[1]).
+This command parses the default configuration files for sections `hook` and
+`hookcmd`. `hook` is used to describe the commands which will be run during a
+particular hook event; commands are run in the order Git encounters them during
+the configuration parse (see linkgit:git-config[1]). `hookcmd` is used to
+describe attributes of a specific command. If additional attributes don't need
+to be specified, a command to run can be specified directly in the `hook`
+section; if a `hookcmd` by that name isn't found, Git will attempt to run the
+provided value directly. For example:
+
+Global config
+----
+  [hook "post-commit"]
+    command = "linter"
+    command = "~/typocheck.sh"
+
+  [hookcmd "linter"]
+    command = "/bin/linter --c"
+----
+
+Local config
+----
+  [hook "prepare-commit-msg"]
+    command = "linter"
+  [hook "post-commit"]
+    command = "python ~/run-test-suite.py"
+----
+
+With these configs, you'd then run post-commit hooks in this order:
+
+  /bin/linter --c
+  ~/typocheck.sh
+  python ~/run-test-suite.py
+  .git/hooks/post-commit (if present)
+
+and prepare-commit-msg hooks in this order:
+
+  /bin/linter --c
+  .git/hooks/prepare-commit-msg (if present)
 
 In general, when instructions suggest adding a script to
 `.git/hooks/<something>`, you can specify it in the config instead by running
diff --git a/hook.c b/hook.c
index b08b876d5d..21904d90f6 100644
--- a/hook.c
+++ b/hook.c
@@ -184,7 +184,14 @@ static int hook_config_lookup(const char *key, const char *value, void *cb_data)
 
 		/* TODO: implement skipping hooks */
 
-		/* TODO: immplement hook aliases */
+		/*
+		 * Check if a hookcmd with that name exists. If it doesn't,
+		 * 'git_config_get_value()' is documented not to touch &command,
+		 * so we don't need to do anything.
+		 */
+		strbuf_reset(&hookcmd_name);
+		strbuf_addf(&hookcmd_name, "hookcmd.%s.command", command);
+		git_config_get_value(hookcmd_name.buf, &command);
 
 		/*
 		 * TODO: implement an option-getting callback, e.g.
diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh
index e4a7b06ad1..50ee824f05 100755
--- a/t/t1360-config-based-hooks.sh
+++ b/t/t1360-config-based-hooks.sh
@@ -18,6 +18,11 @@ setup_hooks () {
 	test_config_global hook.pre-commit.command "/path/def" --add
 }
 
+setup_hookcmd () {
+	test_config hook.pre-commit.command "abc" --add
+	test_config_global hookcmd.abc.command "/path/abc" --add
+}
+
 setup_hookdir () {
 	mkdir .git/hooks
 	write_script .git/hooks/pre-commit <<-EOF
@@ -59,6 +64,20 @@ test_expect_success 'git hook list orders by config order' '
 	test_cmp expected actual
 '
 
+test_expect_success 'git hook list dereferences a hookcmd' '
+	setup_hooks &&
+	setup_hookcmd &&
+
+	cat >expected <<-EOF &&
+	$ROOT/path/def
+	$ROOT/path/ghi
+	$ROOT/path/abc
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
 test_expect_success 'git hook list reorders on duplicate commands' '
 	setup_hooks &&
 
-- 
2.32.0.402.g57bb445576-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH 9/9] hook: implement hookcmd.<name>.skip
  2021-07-15 23:25           ` [PATCH 0/9] config-based hooks restarted Emily Shaffer
                               ` (7 preceding siblings ...)
  2021-07-15 23:26             ` [PATCH 8/9] hook: teach 'hookcmd' config to alias hook scripts Emily Shaffer
@ 2021-07-15 23:26             ` Emily Shaffer
  8 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-07-15 23:26 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

If a user wants a specific repo to skip execution of a hook which is set
at a global or system level, they will be able to do so by specifying
'skip' in their repo config:

~/.gitconfig
  [hook.pre-commit]
    command = skippable-oneliner
    command = skippable-hookcmd

  [hookcmd.skippable-hookcmd]
    command = foo.sh

$GIT_DIR/.git/config
  [hookcmd.skippable-oneliner]
    skip = true
  [hookcmd.skippable-hookcmd]
    skip = true

Later it may make sense to add an option like
"hookcmd.<name>.<hook-event>-skip" - but for simplicity, let's start
with a universal skip setting like this.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/config/hook.txt |  8 ++++++++
 Documentation/git-hook.txt    | 30 +++++++++++++++++++++++++++++
 hook.c                        | 31 +++++++++++++++++++++---------
 t/t1360-config-based-hooks.sh | 36 +++++++++++++++++++++++++++++++++++
 4 files changed, 96 insertions(+), 9 deletions(-)

diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
index 5b35170664..6b3776f06f 100644
--- a/Documentation/config/hook.txt
+++ b/Documentation/config/hook.txt
@@ -8,6 +8,14 @@ hookcmd.<name>.command::
 	as a command. This can be an executable on your device or a oneliner for
 	your shell. See linkgit:git-hook[1].
 
+hookcmd.<name>.skip::
+	Specify this boolean to remove a command from earlier in the execution
+	order. Useful if you want to make a single repo an exception to hook
+	configured at the system or global scope. If there is no hookcmd
+	specified for the command you want to skip, you can use the value of
+	`hook.<command>.command` as <name> as a shortcut. The "skip" setting
+	must be specified after the "hook.<command>.command" to have an effect.
+
 hook.jobs::
 	Specifies how many hooks can be run simultaneously during parallelized
 	hook execution. If unspecified, defaults to the number of processors on
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 1a4d22fd90..fcd13da4ff 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -57,6 +57,36 @@ and prepare-commit-msg hooks in this order:
   /bin/linter --c
   .git/hooks/prepare-commit-msg (if present)
 
+If there is a command you wish to run in most cases but have one or two
+exceptional repos where it should be skipped, you can specify
+`hookcmd.<name>.skip`, for example:
+
+System config
+----
+  [hook "post-commit"]
+    command = check-for-secrets
+
+  [hookcmd "check-for-secrets"]
+    command = /bin/secret-checker --aggressive
+----
+
+Local config
+----
+  [hookcmd "check-for-secrets"]
+    skip = true
+  # This works for inlined hook commands, too:
+  [hookcmd "~/typocheck.sh"]
+    skip = true
+----
+
+After these configs are added, and including the earlier example configs, the
+hook list becomes:
+
+post-commit:
+  /bin/linter --c
+  python ~/run-test-suite.py
+  .git/hooks/post-commit (if present)
+
 In general, when instructions suggest adding a script to
 `.git/hooks/<something>`, you can specify it in the config instead by running
 `git config --add hook.<something>.command <path-to-script>` - this way you can
diff --git a/hook.c b/hook.c
index 21904d90f6..5faa1690e4 100644
--- a/hook.c
+++ b/hook.c
@@ -18,6 +18,7 @@ static void free_hook(struct hook *ptr)
  */
 static struct hook * find_hook_by_command(struct list_head *head, const char *command)
 {
+	/* check if the hook is already in the list */
 	struct list_head *pos = NULL, *tmp = NULL;
 	struct hook *found = NULL;
 
@@ -40,7 +41,6 @@ static struct hook * find_hook_by_command(struct list_head *head, const char *co
  */
 static struct hook * append_or_move_hook(struct list_head *head, const char *command)
 {
-	/* check if the hook is already in the list */
 	struct hook *to_add = find_hook_by_command(head, command);
 
 	if (!to_add) {
@@ -175,14 +175,15 @@ static int hook_config_lookup(const char *key, const char *value, void *cb_data)
 	if (!strcmp(key, hook_key)) {
 		const char *command = value;
 		struct strbuf hookcmd_name = STRBUF_INIT;
+		int skip = 0;
 
-
-		if (!command) {
-			strbuf_release(&hookcmd_name);
-			BUG("git_config_get_value overwrote a string it shouldn't have");
-		}
-
-		/* TODO: implement skipping hooks */
+		/*
+		 * Check if we're removing that hook instead. Hookcmds are
+		 * removed by name, and inlined hooks are removed by command
+		 * content.
+		 */
+		strbuf_addf(&hookcmd_name, "hookcmd.%s.skip", command);
+		git_config_get_bool(hookcmd_name.buf, &skip);
 
 		/*
 		 * Check if a hookcmd with that name exists. If it doesn't,
@@ -193,12 +194,24 @@ static int hook_config_lookup(const char *key, const char *value, void *cb_data)
 		strbuf_addf(&hookcmd_name, "hookcmd.%s.command", command);
 		git_config_get_value(hookcmd_name.buf, &command);
 
+		if (!command) {
+			strbuf_release(&hookcmd_name);
+			BUG("git_config_get_value overwrote a string it shouldn't have");
+		}
+
 		/*
 		 * TODO: implement an option-getting callback, e.g.
 		 *   get configs by pattern hookcmd.$value.*
 		 *   for each key+value, do_callback(key, value, cb_data)
 		 */
-		append_or_move_hook(head, command);
+
+		if (skip) {
+			struct hook *to_remove = find_hook_by_command(head, command);
+			if (to_remove)
+				remove_hook(&(to_remove->list));
+		} else {
+			append_or_move_hook(head, command);
+		}
 
 		strbuf_release(&hookcmd_name);
 	}
diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh
index 50ee824f05..30dc7b6054 100755
--- a/t/t1360-config-based-hooks.sh
+++ b/t/t1360-config-based-hooks.sh
@@ -103,6 +103,42 @@ test_expect_success 'git hook list shows hooks from the hookdir' '
 	test_cmp expected actual
 '
 
+test_expect_success 'git hook list removes skipped hookcmd' '
+	setup_hookcmd &&
+	test_config hookcmd.abc.skip "true" --add &&
+
+	cat >expected <<-EOF &&
+	no commands configured for hook '\''pre-commit'\''
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list ignores skip referring to unused hookcmd' '
+	test_config hookcmd.abc.command "/path/abc" --add &&
+	test_config hookcmd.abc.skip "true" --add &&
+
+	cat >expected <<-EOF &&
+	no commands configured for hook '\''pre-commit'\''
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list removes skipped inlined hook' '
+	setup_hooks &&
+	test_config hookcmd."$ROOT/path/ghi".skip "true" --add &&
+
+	cat >expected <<-EOF &&
+	$ROOT/path/def
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
 
 test_expect_success 'inline hook definitions execute oneliners' '
 	test_config hook.pre-commit.command "echo \"Hello World\"" &&
-- 
2.32.0.402.g57bb445576-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* Re: [PATCH 7/9] hook: allow out-of-repo 'git hook' invocations
  2021-07-15 23:26             ` [PATCH 7/9] hook: allow out-of-repo 'git hook' invocations Emily Shaffer
@ 2021-07-16  8:33               ` Ævar Arnfjörð Bjarmason
  2021-07-22 23:07                 ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-16  8:33 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Jul 15 2021, Emily Shaffer wrote:

> Since hooks can now be supplied via the config, and a config can be
> present without a gitdir via the global and system configs, we can start
> to allow 'git hook run' to occur without a gitdir. This enables us to do
> things like run sendemail-validate hooks when running 'git send-email'
> from a nongit directory.
>
> It still doesn't make sense to look for hooks in the hookdir in nongit
> repos, though, as there is no hookdir.

Hrm, I haven't tested but re the discussion we had about
RUN_SETUP_GENTLY on my re-rolled base topic is this really just a
regression in my changes there?

I.e. I assumed we could do RUN_SETUP for the bug-for-bug compatibility
step, but send-email runs out of repo hooks and we just didn't have
tests for it, or am I missing something?

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 2/9] hook: allow parallel hook execution
  2021-07-15 23:25             ` [PATCH 2/9] hook: allow parallel hook execution Emily Shaffer
@ 2021-07-16  8:36               ` Ævar Arnfjörð Bjarmason
  2021-07-22 21:12                 ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-16  8:36 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Jul 15 2021, Emily Shaffer wrote:

> In many cases, there's no reason not to allow hooks to execute in
> parallel. run_processes_parallel() is well-suited - it's a task queue
> that runs its housekeeping in series, which means users don't
> need to worry about thread safety on their callback data. True
> multithreaded execution with the async_* functions isn't necessary here.
> Synchronous hook execution can be achieved by only allowing 1 job to run
> at a time.
>
> Teach run_hooks() to use that function for simple hooks which don't
> require stdin or capture of stderr.

This doesn't mention...

>  	int ret;
> -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
> +	struct run_hooks_opt opt;
>  
> +	run_hooks_opt_init_sync(&opt);


...why we need to bring the s/macro/func/ init pattern, back, but looking ahead...

> +int configured_hook_jobs(void)a
> +{
> +	int n = online_cpus();
> +	git_config_get_int("hook.jobs", &n);
> +
> +	return n;
> +}
> +
>  int hook_exists(const char *name)
>  {
>  	return !!find_hook(name);
> @@ -117,6 +125,26 @@ struct list_head* hook_list(const char* hookname)
>  	return hook_head;
>  }
>  
> +void run_hooks_opt_init_sync(struct run_hooks_opt *o)
> +{
> +	strvec_init(&o->env);
> +	strvec_init(&o->args);
> +	o->path_to_stdin = NULL;
> +	o->jobs = 1;
> +	o->dir = NULL;
> +	o->feed_pipe = NULL;
> +	o->feed_pipe_ctx = NULL;
> +	o->consume_sideband = NULL;
> +	o->invoked_hook = NULL;
> +	o->absolute_path = 0;
> +}
> +
> +void run_hooks_opt_init_async(struct run_hooks_opt *o)
> +{
> +	run_hooks_opt_init_sync(o);
> +	o->jobs = configured_hook_jobs();
> +}

...okey, so it's because you brought back the "call jobs function" in
one of the init functions.

I had a comment in a previous round, I found
https://lore.kernel.org/git/87lf7bzbrk.fsf@evledraar.gmail.com/, but I
think there was a later one where I commented on the "jobs" field
specifically.

Anyway, it seems much easier to me to just keep the simpler macro init
and then:

> -	if (options->jobs != 1)
> -		BUG("we do not handle %d or any other != 1 job number yet", options->jobs);
> -
>  	run_processes_parallel_tr2(options->jobs,
>  				   pick_next_hook,
>  				   notify_start_failure,

There's this one place where we use the "jobs" parameter, just do
something like this there:
        
        int configured_hook_jobs(void)
        {
                static int jobs;
                if (!jobs)
                    return jobs;
                if (git_config_get_int("hook.jobs", &jobs))
                    jobs = online_cpus();
                return jobs;
        }

I.e. you also needlessly call online_cpus() when we're about to override
it in the config. The git_config_get_int()'s return value indicates
whether you need to do that. Then just:

    int jobs = options->jobs ? options->jobs : configured_hook_jobs();
    run_processes_parallel_tr2(jobs, [...]);

Or some such, i.e. we can defer getting the job number away from startup
to when we actually need to start those jobs, and your whole use of a
function init pattern came down to doing that really early.

As an aside if you /do/ need to do init-via-function my 5726a6b4012 (*.c
*_init(): define in terms of corresponding *_INIT macro, 2021-07-01) in
"next" shows a much nicer way to do that. I.e. you'd just do:

    void run_hooks_opt_init_sync(struct run_hooks_opt *o)
    {
         struct run_hooks_opt blank = RUN_HOOKS_OPT_INIT;
         memcpy(o, &blank, sizeof(*o));
    }

    void run_hooks_opt_init_async(struct run_hooks_opt *o)
    {
        run_hooks_opt_init_sync(o);
        o->jobs = configured_hook_jobs();
    }

In some cases we do actually need to do init via functions, but can init
a large option via the macro, which IMO is nicer to read, but here I
think we don't need the functions at all per the above.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 3/9] hook: introduce "git hook list"
  2021-07-15 23:25             ` [PATCH 3/9] hook: introduce "git hook list" Emily Shaffer
@ 2021-07-16  8:52               ` Ævar Arnfjörð Bjarmason
  2021-07-22 22:18                 ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-16  8:52 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Jul 15 2021, Emily Shaffer wrote:

>  static const char * const builtin_hook_usage[] = {
>  	N_("git hook <command> [...]"),
> +	N_("git hook list <hookname>"),
>  	N_("git hook run [<args>] <hook-name> [-- <hook-args>]"),
>  	NULL

Uses <hook-name> already, let's use that too. I can't remember if it's
something I changed myself, or if your original version used both and I
picked one for consistency, or...

Anyway, I can re-roll the base topic or whatever, but let's have the end
result use one or the other.

> +	if (argc < 1) {
> +		usage_msg_opt(_("You must specify a hook event name to list."),
> +			      builtin_hook_usage, list_options);
> +	}

{} braces not needed.


> +	if (!strcmp(argv[0], "list"))
> +		return list(argc, argv, prefix);
>  	if (!strcmp(argv[0], "run"))

This should be "else if" now.

(Doesn't matter for code execution, just IMO readability, but I'll leave
that to you ... :)

>  		return run(argc, argv, prefix);
>  	else
> diff --git a/hook.c b/hook.c
> index 935751fa6c..19138a8290 100644
> --- a/hook.c
> +++ b/hook.c
> @@ -104,22 +104,20 @@ int hook_exists(const char *name)
>  struct list_head* hook_list(const char* hookname)
>  {
>  	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
> +	const char *hook_path = find_hook(hookname);
> +
>  
>  	INIT_LIST_HEAD(hook_head);
>  
>  	if (!hookname)
>  		return NULL;
>  
> -	if (have_git_dir()) {
> -		const char *hook_path = find_hook(hookname);
> -
> -		/* Add the hook from the hookdir */
> -		if (hook_path) {
> -			struct hook *to_add = xmalloc(sizeof(*to_add));
> -			to_add->hook_path = hook_path;
> -			to_add->feed_pipe_cb_data = NULL;
> -			list_add_tail(&to_add->list, hook_head);
> -		}
> +	/* Add the hook from the hookdir */
> +	if (hook_path) {
> +		struct hook *to_add = xmalloc(sizeof(*to_add));
> +		to_add->hook_path = hook_path;
> +		to_add->feed_pipe_cb_data = NULL;
> +		list_add_tail(&to_add->list, hook_head);

Maybe we should have a INIT for "struct hook" too? This also needlessly
leaves behind an un-free'd hook struct in a way that it wouldn't if we
just had this on the stack, no?

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 4/9] hook: treat hookdir hook specially
  2021-07-15 23:25             ` [PATCH 4/9] hook: treat hookdir hook specially Emily Shaffer
@ 2021-07-16  8:58               ` Ævar Arnfjörð Bjarmason
  2021-07-22 22:24                 ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-16  8:58 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Jul 15 2021, Emily Shaffer wrote:

> Soon, we will allow users to specify hooks using the config. These
> config-specified hooks may require different child_process options than
> hook executables in the gitdir. So, let's differentiate between hooks
> coming from the gitdir and hooks coming from the config.
>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  hook.c | 3 ++-
>  hook.h | 2 ++
>  2 files changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/hook.c b/hook.c
> index 19138a8290..3a588cb055 100644
> --- a/hook.c
> +++ b/hook.c
> @@ -117,6 +117,7 @@ struct list_head* hook_list(const char* hookname)
>  		struct hook *to_add = xmalloc(sizeof(*to_add));
>  		to_add->hook_path = hook_path;
>  		to_add->feed_pipe_cb_data = NULL;
> +		to_add->from_hookdir = 1;
>  		list_add_tail(&to_add->list, hook_head);
>  	}
>  
> @@ -200,7 +201,7 @@ static int pick_next_hook(struct child_process *cp,
>  	cp->dir = hook_cb->options->dir;
>  
>  	/* add command */
> -	if (hook_cb->options->absolute_path)
> +	if (run_me->from_hookdir && hook_cb->options->absolute_path)
>  		strvec_push(&cp->args, absolute_path(run_me->hook_path));
>  	else
>  		strvec_push(&cp->args, run_me->hook_path);
> diff --git a/hook.h b/hook.h
> index 586ddf40bb..60389cd8cd 100644
> --- a/hook.h
> +++ b/hook.h
> @@ -22,6 +22,8 @@ struct hook {
>  	/* The path to the hook */
>  	const char *hook_path;
>  
> +	unsigned from_hookdir : 1;
> +
>  	/*
>  	 * Use this to keep state for your feed_pipe_fn if you are using
>  	 * run_hooks_opt.feed_pipe. Otherwise, do not touch it.

The "from_hookdir" looks like it isn't used until 6/9, and maybe the
absolute_path change too? In any case this seems like a carried-forward
rebase of
https://lore.kernel.org/git/20210311021037.3001235-5-emilyshaffer@google.com/
or some version thereof.

At this point I tihnk it would be way better to squash this and other
such changes that basically add a field to a struct that isn't used yet
into whatever commit use/need them.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 6/9] hook: include hooks from the config
  2021-07-15 23:26             ` [PATCH 6/9] hook: include hooks from the config Emily Shaffer
@ 2021-07-16  9:01               ` Ævar Arnfjörð Bjarmason
  2021-07-22 22:51                 ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-16  9:01 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Jul 15 2021, Emily Shaffer wrote:

> +static struct hook * find_hook_by_command(struct list_head *head, const char *command)

nit: "*find[...]" not "* find[...]", also let's wrap the long line.

> +{
> +	struct list_head *pos = NULL, *tmp = NULL;
> +	struct hook *found = NULL;
> +
> +	list_for_each_safe(pos, tmp, head) {
> +		struct hook *it = list_entry(pos, struct hook, list);
> +		if (!strcmp(it->command, command)) {
> +		    list_del(pos);
> +		    found = it;
> +		    break;

Indented with spaces.

Also is there some subtlety in the list macro here or can we just
"s/break/return it/" and skip the break/return pattern?

> +static struct hook * append_or_move_hook(struct list_head *head, const char *command)

Same whitespace nits.

> +	if (!to_add) {
> +		/* adding a new hook, not moving an old one */
> +		to_add = xmalloc(sizeof(*to_add));
> +		to_add->command = command;
> +		to_add->feed_pipe_cb_data = NULL;
> +		/* This gets overwritten in hook_list() for hookdir hooks. */
> +		to_add->from_hookdir = 0;

I commented on init verbosity elsewhere, i.e. we could do some things
via macros, but in this case just having an "init" helper make sense,
but we have at least two places copying the same init of all fields,
should just be hook_init_hook() or whatever it'll be called. Maybe with
a second "from hookdir" param?

> +	if (!strcmp(key, hook_key)) {
> +		const char *command = value;
> +		struct strbuf hookcmd_name = STRBUF_INIT;
> +
> +

Nit: 3x\n, not 2x\n

> +		if (!command) {
> +			strbuf_release(&hookcmd_name);

You don't need to strbuf_release() things that you haven't done anything
except init'd, but also...

> +			BUG("git_config_get_value overwrote a string it shouldn't have");

...even if that were the case and it called malloc() memory diligence
when we're calling BUG() is probably going overboard, and I say that as
someone who'll tend to go overboard with it :)

> +		}
> +
> +		/* TODO: implement skipping hooks */
> +
> +		/* TODO: immplement hook aliases */
> +
> +		/*
> +		 * TODO: implement an option-getting callback, e.g.
> +		 *   get configs by pattern hookcmd.$value.*
> +		 *   for each key+value, do_callback(key, value, cb_data)
> +		 */

I think we should drop the TODO and just let the commit message /
comments speak to what we actually implement, and subsequent patches can
add more features.

> -
> +	struct strbuf hook_key = STRBUF_INIT;
> +	struct hook_config_cb cb_data = { &hook_key, hook_head };

Let's use designated initializers.

>  
>  	INIT_LIST_HEAD(hook_head);
>  
>  	if (!hookname)
>  		return NULL;
>  
> +	/* Add the hooks from the config, e.g. hook.pre-commit.command */
> +	strbuf_addf(&hook_key, "hook.%s.command", hookname);
> +	git_config(hook_config_lookup, &cb_data);
> +
> +

Another 3x\n

> +	/* to enable oneliners, let config-specified hooks run in shell */
> +	cp->use_shell = !run_me->from_hookdir;

I've lost track at this point, but doesn't that mean we're going to use
a shell when we run our own do-not-need-a-shell hooks ourselves?

Isn't isatty() more appropriate here, or actually even interactively why
is the shell needed (maybe this is answered elswhere...).

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 8/9] hook: teach 'hookcmd' config to alias hook scripts
  2021-07-15 23:26             ` [PATCH 8/9] hook: teach 'hookcmd' config to alias hook scripts Emily Shaffer
@ 2021-07-16  9:13               ` Ævar Arnfjörð Bjarmason
  2021-07-22 23:31                 ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-16  9:13 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Jul 15 2021, Emily Shaffer wrote:

> To enable fine-grained options which apply to a single hook executable,
> and to make it easier for a single executable to be run on multiple hook
> events, teach "hookcmd.<alias>.config". These can be configured as
> follows:
> [...]
> diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
> index a97b980cca..5b35170664 100644
> --- a/Documentation/config/hook.txt
> +++ b/Documentation/config/hook.txt
> @@ -3,6 +3,11 @@ hook.<command>.command::
>  	executable on your device, a oneliner for your shell, or the name of a
>  	hookcmd. See linkgit:git-hook[1].
>  
> +hookcmd.<name>.command::
> +	A command to execute during a hook for which <name> has been specified
> +	as a command. This can be an executable on your device or a oneliner for
> +	your shell. See linkgit:git-hook[1].
> +
> [...]
> +Global config
> +----
> +  [hook "post-commit"]
> +    command = "linter"
> +    command = "~/typocheck.sh"
> +
> +  [hookcmd "linter"]
> +    command = "/bin/linter --c"
> +----
> +
> +Local config
> +----
> +  [hook "prepare-commit-msg"]
> +    command = "linter"
> +  [hook "post-commit"]
> +    command = "python ~/run-test-suite.py"
> +----
> +
> +With these configs, you'd then run post-commit hooks in this order:
> +
> +  /bin/linter --c
> +  ~/typocheck.sh
> +  python ~/run-test-suite.py
> +  .git/hooks/post-commit (if present)
> +
> +and prepare-commit-msg hooks in this order:
> +
> +  /bin/linter --c
> +  .git/hooks/prepare-commit-msg (if present)
>  

I still have outstanding feedback on the fundamental design
here. I.e. why is this not:

    hook.<name>.event = post-commit
    hook.<name>.command = <path>

See:

https://lore.kernel.org/git/87mtv8fww3.fsf@evledraar.gmail.com/

As noted there I don't see why not, and more complexity results from the
design choice of doing it the way you're proposing. I.e. we can't
discover hooks based on config prefixes, and we end up sticking full FS
paths in config keys.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 2/9] hook: allow parallel hook execution
  2021-07-16  8:36               ` Ævar Arnfjörð Bjarmason
@ 2021-07-22 21:12                 ` Emily Shaffer
  2021-07-23  9:30                   ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-07-22 21:12 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Fri, Jul 16, 2021 at 10:36:10AM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Thu, Jul 15 2021, Emily Shaffer wrote:
> 
> > In many cases, there's no reason not to allow hooks to execute in
> > parallel. run_processes_parallel() is well-suited - it's a task queue
> > that runs its housekeeping in series, which means users don't
> > need to worry about thread safety on their callback data. True
> > multithreaded execution with the async_* functions isn't necessary here.
> > Synchronous hook execution can be achieved by only allowing 1 job to run
> > at a time.
> >
> > Teach run_hooks() to use that function for simple hooks which don't
> > require stdin or capture of stderr.
> 
> This doesn't mention...
> 
> >  	int ret;
> > -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
> > +	struct run_hooks_opt opt;
> >  
> > +	run_hooks_opt_init_sync(&opt);
> 
> 
> ...why we need to bring the s/macro/func/ init pattern, back, but looking ahead...
> 
> > +int configured_hook_jobs(void)a
> > +{
> > +	int n = online_cpus();
> > +	git_config_get_int("hook.jobs", &n);
> > +
> > +	return n;
> > +}
> > +
> >  int hook_exists(const char *name)
> >  {
> >  	return !!find_hook(name);
> > @@ -117,6 +125,26 @@ struct list_head* hook_list(const char* hookname)
> >  	return hook_head;
> >  }
> >  
> > +void run_hooks_opt_init_sync(struct run_hooks_opt *o)
> > +{
> > +	strvec_init(&o->env);
> > +	strvec_init(&o->args);
> > +	o->path_to_stdin = NULL;
> > +	o->jobs = 1;
> > +	o->dir = NULL;
> > +	o->feed_pipe = NULL;
> > +	o->feed_pipe_ctx = NULL;
> > +	o->consume_sideband = NULL;
> > +	o->invoked_hook = NULL;
> > +	o->absolute_path = 0;
> > +}
> > +
> > +void run_hooks_opt_init_async(struct run_hooks_opt *o)
> > +{
> > +	run_hooks_opt_init_sync(o);
> > +	o->jobs = configured_hook_jobs();
> > +}
> 
> ...okey, so it's because you brought back the "call jobs function" in
> one of the init functions.
> 
> I had a comment in a previous round, I found
> https://lore.kernel.org/git/87lf7bzbrk.fsf@evledraar.gmail.com/, but I
> think there was a later one where I commented on the "jobs" field
> specifically.
> 
> Anyway, it seems much easier to me to just keep the simpler macro init
> and then:
> 
> > -	if (options->jobs != 1)
> > -		BUG("we do not handle %d or any other != 1 job number yet", options->jobs);
> > -
> >  	run_processes_parallel_tr2(options->jobs,
> >  				   pick_next_hook,
> >  				   notify_start_failure,
> 
> There's this one place where we use the "jobs" parameter, just do
> something like this there:
>         
>         int configured_hook_jobs(void)
>         {
>                 static int jobs;
>                 if (!jobs)
>                     return jobs;
>                 if (git_config_get_int("hook.jobs", &jobs))
>                     jobs = online_cpus();
>                 return jobs;
>         }
> 
> I.e. you also needlessly call online_cpus() when we're about to override
> it in the config. The git_config_get_int()'s return value indicates
> whether you need to do that. Then just:
> 
>     int jobs = options->jobs ? options->jobs : configured_hook_jobs();
>     run_processes_parallel_tr2(jobs, [...]);

Ahh, and then let RUN_HOOKS_OPT_INIT_ASYNC set jobs to 0 ("go look it
up"). Yeah, that makes sense.

Shout if somehow you meant to leave just one initializer macro;
otherwise, I'll do it this way - with RUN_HOOKS_OPT_INIT_ASYNC and
RUN_HOOKS_OPT_INIT_SYNC. I think it's valuable for hook callers to make
it very plain at the callsite whether they're parallelizable or not, and
I think

 struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 opt.jobs = 0;

doesn't make that as obvious.

> 
> Or some such, i.e. we can defer getting the job number away from startup
> to when we actually need to start those jobs, and your whole use of a
> function init pattern came down to doing that really early.
> 
> As an aside if you /do/ need to do init-via-function my 5726a6b4012 (*.c
> *_init(): define in terms of corresponding *_INIT macro, 2021-07-01) in
> "next" shows a much nicer way to do that. I.e. you'd just do:
> 
>     void run_hooks_opt_init_sync(struct run_hooks_opt *o)
>     {
>          struct run_hooks_opt blank = RUN_HOOKS_OPT_INIT;
>          memcpy(o, &blank, sizeof(*o));
>     }
> 
>     void run_hooks_opt_init_async(struct run_hooks_opt *o)
>     {
>         run_hooks_opt_init_sync(o);
>         o->jobs = configured_hook_jobs();
>     }
> 
> In some cases we do actually need to do init via functions, but can init
> a large option via the macro, which IMO is nicer to read, but here I
> think we don't need the functions at all per the above.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 02/27] gc: use hook library for pre-auto-gc hook
  2021-06-17 10:22           ` [PATCH 02/27] gc: use hook library for pre-auto-gc hook Ævar Arnfjörð Bjarmason
@ 2021-07-22 21:58             ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-07-22 21:58 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

On Thu, Jun 17, 2021 at 12:22:36PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> Using the hook.h library instead of the run-command.h library to run
> pre-auto-gc means that those hooks can be set up in config files, as
> well as in the hookdir. pre-auto-gc is called only from builtin/gc.c.

This one is a good example of an issue Junio pointed out to me in an
earlier iteration - since the strvecs in run_hooks_opt need to be
cleared, you need to be careful with exiting without running
run_hooks_opt_clear(). need_to_gc() can exit before running the hook in
some cases.

 - Emily

> 
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>  builtin/gc.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/builtin/gc.c b/builtin/gc.c
> index f05d2f0a1a..a12641a691 100644
> --- a/builtin/gc.c
> +++ b/builtin/gc.c
> @@ -32,6 +32,7 @@
>  #include "remote.h"
>  #include "object-store.h"
>  #include "exec-cmd.h"
> +#include "hook.h"
>  
>  #define FAILED_RUN "failed to run %s"
>  
> @@ -348,6 +349,8 @@ static void add_repack_incremental_option(void)
>  
>  static int need_to_gc(void)
>  {
> +	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
> +
>  	/*
>  	 * Setting gc.auto to 0 or negative can disable the
>  	 * automatic gc.
> @@ -394,8 +397,11 @@ static int need_to_gc(void)
>  	else
>  		return 0;
>  
> -	if (run_hook_le(NULL, "pre-auto-gc", NULL))
> +	if (run_hooks("pre-auto-gc", &hook_opt)) {
> +		run_hooks_opt_clear(&hook_opt);
>  		return 0;
> +	}
> +	run_hooks_opt_clear(&hook_opt);
>  	return 1;
>  }
>  
> -- 
> 2.32.0.576.g59759b6ca7d
> 

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 3/9] hook: introduce "git hook list"
  2021-07-16  8:52               ` Ævar Arnfjörð Bjarmason
@ 2021-07-22 22:18                 ` Emily Shaffer
  2021-07-23  9:29                   ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-07-22 22:18 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Fri, Jul 16, 2021 at 10:52:27AM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Thu, Jul 15 2021, Emily Shaffer wrote:
> 
> >  static const char * const builtin_hook_usage[] = {
> >  	N_("git hook <command> [...]"),
> > +	N_("git hook list <hookname>"),
> >  	N_("git hook run [<args>] <hook-name> [-- <hook-args>]"),
> >  	NULL
> 
> Uses <hook-name> already, let's use that too. I can't remember if it's
> something I changed myself, or if your original version used both and I
> picked one for consistency, or...
> 
> Anyway, I can re-roll the base topic or whatever, but let's have the end
> result use one or the other.

'hook-name' is fine, I'll use that. Thanks for pointing it out.

> 
> > +	if (argc < 1) {
> > +		usage_msg_opt(_("You must specify a hook event name to list."),
> > +			      builtin_hook_usage, list_options);
> > +	}
> 
> {} braces not needed.
ACK
> 
> 
> > +	if (!strcmp(argv[0], "list"))
> > +		return list(argc, argv, prefix);
> >  	if (!strcmp(argv[0], "run"))
> 
> This should be "else if" now.
> 
> (Doesn't matter for code execution, just IMO readability, but I'll leave
> that to you ... :)

Eh, I think it's easier to read in the strcmps line up, so I will leave
it this way ;)

> 
> >  		return run(argc, argv, prefix);
> >  	else
> > diff --git a/hook.c b/hook.c
> > index 935751fa6c..19138a8290 100644
> > --- a/hook.c
> > +++ b/hook.c
> > @@ -104,22 +104,20 @@ int hook_exists(const char *name)
> >  struct list_head* hook_list(const char* hookname)
> >  {
> >  	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
> > +	const char *hook_path = find_hook(hookname);
> > +
> >  
> >  	INIT_LIST_HEAD(hook_head);
> >  
> >  	if (!hookname)
> >  		return NULL;
> >  
> > -	if (have_git_dir()) {
> > -		const char *hook_path = find_hook(hookname);
> > -
> > -		/* Add the hook from the hookdir */
> > -		if (hook_path) {
> > -			struct hook *to_add = xmalloc(sizeof(*to_add));
> > -			to_add->hook_path = hook_path;
> > -			to_add->feed_pipe_cb_data = NULL;
> > -			list_add_tail(&to_add->list, hook_head);
> > -		}
> > +	/* Add the hook from the hookdir */
> > +	if (hook_path) {
> > +		struct hook *to_add = xmalloc(sizeof(*to_add));
> > +		to_add->hook_path = hook_path;
> > +		to_add->feed_pipe_cb_data = NULL;
> > +		list_add_tail(&to_add->list, hook_head);
> 
> Maybe we should have a INIT for "struct hook" too? This also needlessly
> leaves behind an un-free'd hook struct in a way that it wouldn't if we
> just had this on the stack, no?

I can clean it up here, but I don't think we need an initializer for
struct hook. This code chunk gets moved into one of the list
manipulators (append_or_move() or something) after the config is
introduced.

I don't think it leaves an unfreed hook laying around, does it?
list_add_tail() uses the malloc'd pointer, and we free() in
clear_hook_list(). What am I missing?

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 4/9] hook: treat hookdir hook specially
  2021-07-16  8:58               ` Ævar Arnfjörð Bjarmason
@ 2021-07-22 22:24                 ` Emily Shaffer
  2021-07-23  9:26                   ` Ævar Arnfjörð Bjarmason
  2021-07-23 17:33                   ` Felipe Contreras
  0 siblings, 2 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-07-22 22:24 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Fri, Jul 16, 2021 at 10:58:34AM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Thu, Jul 15 2021, Emily Shaffer wrote:
> 
> > Soon, we will allow users to specify hooks using the config. These
> > config-specified hooks may require different child_process options than
> > hook executables in the gitdir. So, let's differentiate between hooks
> > coming from the gitdir and hooks coming from the config.
> >
> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > ---
> >  hook.c | 3 ++-
> >  hook.h | 2 ++
> >  2 files changed, 4 insertions(+), 1 deletion(-)
> >
> > diff --git a/hook.c b/hook.c
> > index 19138a8290..3a588cb055 100644
> > --- a/hook.c
> > +++ b/hook.c
> > @@ -117,6 +117,7 @@ struct list_head* hook_list(const char* hookname)
> >  		struct hook *to_add = xmalloc(sizeof(*to_add));
> >  		to_add->hook_path = hook_path;
> >  		to_add->feed_pipe_cb_data = NULL;
> > +		to_add->from_hookdir = 1;
> >  		list_add_tail(&to_add->list, hook_head);
> >  	}
> >  
> > @@ -200,7 +201,7 @@ static int pick_next_hook(struct child_process *cp,
> >  	cp->dir = hook_cb->options->dir;
> >  
> >  	/* add command */
> > -	if (hook_cb->options->absolute_path)
> > +	if (run_me->from_hookdir && hook_cb->options->absolute_path)
> >  		strvec_push(&cp->args, absolute_path(run_me->hook_path));
> >  	else
> >  		strvec_push(&cp->args, run_me->hook_path);
> > diff --git a/hook.h b/hook.h
> > index 586ddf40bb..60389cd8cd 100644
> > --- a/hook.h
> > +++ b/hook.h
> > @@ -22,6 +22,8 @@ struct hook {
> >  	/* The path to the hook */
> >  	const char *hook_path;
> >  
> > +	unsigned from_hookdir : 1;
> > +
> >  	/*
> >  	 * Use this to keep state for your feed_pipe_fn if you are using
> >  	 * run_hooks_opt.feed_pipe. Otherwise, do not touch it.
> 
> The "from_hookdir" looks like it isn't used until 6/9, and maybe the
> absolute_path change too? In any case this seems like a carried-forward
> rebase of
> https://lore.kernel.org/git/20210311021037.3001235-5-emilyshaffer@google.com/
> or some version thereof.
> 
> At this point I tihnk it would be way better to squash this and other
> such changes that basically add a field to a struct that isn't used yet
> into whatever commit use/need them.

I think at this point we run into you and me having different
patch-storytelling styles - which probably is what led to the big topic
restart in the first place ;)

I'd prefer to see the "start using config too" patch stay as small as it
can; that's why I ended up with a handful of minor setup commits and
then one "and now here's config" commit.

Even if it's different from how you would tell it - is it wrong? (And if
it is, that's fine, and I'll change it, but I don't think it is - that's
why I structured the series this way.)

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 6/9] hook: include hooks from the config
  2021-07-16  9:01               ` Ævar Arnfjörð Bjarmason
@ 2021-07-22 22:51                 ` Emily Shaffer
  2021-07-23  9:22                   ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-07-22 22:51 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Fri, Jul 16, 2021 at 11:01:24AM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Thu, Jul 15 2021, Emily Shaffer wrote:
> 
> > +static struct hook * find_hook_by_command(struct list_head *head, const char *command)
> 
> nit: "*find[...]" not "* find[...]", also let's wrap the long line.
ACK
> 
> > +{
> > +	struct list_head *pos = NULL, *tmp = NULL;
> > +	struct hook *found = NULL;
> > +
> > +	list_for_each_safe(pos, tmp, head) {
> > +		struct hook *it = list_entry(pos, struct hook, list);
> > +		if (!strcmp(it->command, command)) {
> > +		    list_del(pos);
> > +		    found = it;
> > +		    break;
> 
> Indented with spaces.

I don't know how I even did this. *facepalm*

> 
> Also is there some subtlety in the list macro here or can we just
> "s/break/return it/" and skip the break/return pattern?

I guess it's probably fine, but we'd need the final return anyway
("otherwise returns NULL"). IMO one return is more readable than two
returns, so I'd rather leave this.
> 
> > +static struct hook * append_or_move_hook(struct list_head *head, const char *command)
> 
> Same whitespace nits.
ACK
> 
> > +	if (!to_add) {
> > +		/* adding a new hook, not moving an old one */
> > +		to_add = xmalloc(sizeof(*to_add));
> > +		to_add->command = command;
> > +		to_add->feed_pipe_cb_data = NULL;
> > +		/* This gets overwritten in hook_list() for hookdir hooks. */
> > +		to_add->from_hookdir = 0;
> 
> I commented on init verbosity elsewhere, i.e. we could do some things
> via macros, but in this case just having an "init" helper make sense,
> but we have at least two places copying the same init of all fields,
> should just be hook_init_hook() or whatever it'll be called. Maybe with
> a second "from hookdir" param?

Hm, where is the second place where we init everything? I think with
this commit we remove anywhere we're putting together a 'struct hook' manually
except during this helper? Hooks from hookdir are initted by
'append_or_move_hook()'ing them to the end of the list and modifying the
from_hookdir field, and builtin/hook.c just calls hook_list() (and some
list.h helpers to find an entry).

> 
> > +	if (!strcmp(key, hook_key)) {
> > +		const char *command = value;
> > +		struct strbuf hookcmd_name = STRBUF_INIT;
> > +
> > +
> 
> Nit: 3x\n, not 2x\n
> 
> > +		if (!command) {
> > +			strbuf_release(&hookcmd_name);
> 
> You don't need to strbuf_release() things that you haven't done anything
> except init'd, but also...
> 
> > +			BUG("git_config_get_value overwrote a string it shouldn't have");
> 
> ...even if that were the case and it called malloc() memory diligence
> when we're calling BUG() is probably going overboard, and I say that as
> someone who'll tend to go overboard with it :)

:) ok.

> 
> > +		}
> > +
> > +		/* TODO: implement skipping hooks */
> > +
> > +		/* TODO: immplement hook aliases */
> > +
> > +		/*
> > +		 * TODO: implement an option-getting callback, e.g.
> > +		 *   get configs by pattern hookcmd.$value.*
> > +		 *   for each key+value, do_callback(key, value, cb_data)
> > +		 */
> 
> I think we should drop the TODO and just let the commit message /
> comments speak to what we actually implement, and subsequent patches can
> add more features.

Ok, sure.

> 
> > -
> > +	struct strbuf hook_key = STRBUF_INIT;
> > +	struct hook_config_cb cb_data = { &hook_key, hook_head };
> 
> Let's use designated initializers.
ACK
> 
> >  
> >  	INIT_LIST_HEAD(hook_head);
> >  
> >  	if (!hookname)
> >  		return NULL;
> >  
> > +	/* Add the hooks from the config, e.g. hook.pre-commit.command */
> > +	strbuf_addf(&hook_key, "hook.%s.command", hookname);
> > +	git_config(hook_config_lookup, &cb_data);
> > +
> > +
> 
> Another 3x\n
ACK
> 
> > +	/* to enable oneliners, let config-specified hooks run in shell */
> > +	cp->use_shell = !run_me->from_hookdir;
> 
> I've lost track at this point, but doesn't that mean we're going to use
> a shell when we run our own do-not-need-a-shell hooks ourselves?
> 
> Isn't isatty() more appropriate here, or actually even interactively why
> is the shell needed (maybe this is answered elswhere...).

use_shell means "conditionally guess whether I need to wrap this thing
in `sh -c`" - it doesn't have anything to do with TTY or not. So we need
this for something like `hook.post-commit.command = echo "made a
commit"`. In this case the entire argv[0] will be the oneliner, which
you will need use_shell set for. If we *do* just do something simple,
like `hook.post-commit.command = /bin/mail`, even though
use_shell is marked, the child_process runner will notice that there's
no reason to wrap in 'sh -c' and so will just run the /bin/mail
executable directly.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 7/9] hook: allow out-of-repo 'git hook' invocations
  2021-07-16  8:33               ` Ævar Arnfjörð Bjarmason
@ 2021-07-22 23:07                 ` Emily Shaffer
  2021-07-23  9:18                   ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-07-22 23:07 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Fri, Jul 16, 2021 at 10:33:25AM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Thu, Jul 15 2021, Emily Shaffer wrote:
> 
> > Since hooks can now be supplied via the config, and a config can be
> > present without a gitdir via the global and system configs, we can start
> > to allow 'git hook run' to occur without a gitdir. This enables us to do
> > things like run sendemail-validate hooks when running 'git send-email'
> > from a nongit directory.
> >
> > It still doesn't make sense to look for hooks in the hookdir in nongit
> > repos, though, as there is no hookdir.
> 
> Hrm, I haven't tested but re the discussion we had about
> RUN_SETUP_GENTLY on my re-rolled base topic is this really just a
> regression in my changes there?
> 
> I.e. I assumed we could do RUN_SETUP for the bug-for-bug compatibility
> step, but send-email runs out of repo hooks and we just didn't have
> tests for it, or am I missing something?

I'm not sure. I could see a case for you including RUN_SETUP_GENTLY on
your series and adding a test for sendemail-validate + core.hooksPath in
global config. I think I also don't have support for that case here,
actually....

Anyway, it looks like right now git-send-email.perl:validate_patch()
doesn't bother if it's out-of-repo, so this wouldn't have worked before (and
still won't work even after this change). So either I can add a patch to
my series to allow that, or you can modify your patch converting
sendemail-validate to 'git hook run' to drop the 'if $repo' line and
teach your series to behave correctly with nongit+hooksPath. It looks
like in my earlier attempt at the series, I did drop that check and run
'git hook run' no matter what kind of directory we're in.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 8/9] hook: teach 'hookcmd' config to alias hook scripts
  2021-07-16  9:13               ` Ævar Arnfjörð Bjarmason
@ 2021-07-22 23:31                 ` Emily Shaffer
  2021-07-23  7:41                   ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-07-22 23:31 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Fri, Jul 16, 2021 at 11:13:42AM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Thu, Jul 15 2021, Emily Shaffer wrote:
> 
> > To enable fine-grained options which apply to a single hook executable,
> > and to make it easier for a single executable to be run on multiple hook
> > events, teach "hookcmd.<alias>.config". These can be configured as
> > follows:
> > [...]
> > diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
> > index a97b980cca..5b35170664 100644
> > --- a/Documentation/config/hook.txt
> > +++ b/Documentation/config/hook.txt
> > @@ -3,6 +3,11 @@ hook.<command>.command::
> >  	executable on your device, a oneliner for your shell, or the name of a
> >  	hookcmd. See linkgit:git-hook[1].
> >  
> > +hookcmd.<name>.command::
> > +	A command to execute during a hook for which <name> has been specified
> > +	as a command. This can be an executable on your device or a oneliner for
> > +	your shell. See linkgit:git-hook[1].
> > +
> > [...]
> > +Global config
> > +----
> > +  [hook "post-commit"]
> > +    command = "linter"
> > +    command = "~/typocheck.sh"
> > +
> > +  [hookcmd "linter"]
> > +    command = "/bin/linter --c"
> > +----
> > +
> > +Local config
> > +----
> > +  [hook "prepare-commit-msg"]
> > +    command = "linter"
> > +  [hook "post-commit"]
> > +    command = "python ~/run-test-suite.py"
> > +----
> > +
> > +With these configs, you'd then run post-commit hooks in this order:
> > +
> > +  /bin/linter --c
> > +  ~/typocheck.sh
> > +  python ~/run-test-suite.py
> > +  .git/hooks/post-commit (if present)
> > +
> > +and prepare-commit-msg hooks in this order:
> > +
> > +  /bin/linter --c
> > +  .git/hooks/prepare-commit-msg (if present)
> >  
> 
> I still have outstanding feedback on the fundamental design
> here. I.e. why is this not:
> 
>     hook.<name>.event = post-commit
>     hook.<name>.command = <path>
> 
> See:
> 
> https://lore.kernel.org/git/87mtv8fww3.fsf@evledraar.gmail.com/
> 
> As noted there I don't see why not, and more complexity results from the
> design choice of doing it the way you're proposing. I.e. we can't
> discover hooks based on config prefixes, and we end up sticking full FS
> paths in config keys.

Interesting. My gut says that it would make it harder to neatly write a
config with the same hook running at the very beginning of one event and
the very end of another, but I'm not sure whether that's a case to
optimize for.

I would also need twice as many lines to run a script I wrote as a hook
- that is, the base case which is probably all most people will need. So
with your proposal I *must* name every hook I want to add. Instead of
"hook.pre-commit.command = ~/check-for-debug-strings" I need to
configure "hook.check-debug-strings.event = pre-commit" and
"hook.check-debug-strings.command = ~/check-for-debug-strings". That's a
little frustrating, and if I never want to skip that check or move it
later or something, it will be extra overhead for no gain for me.

I do agree that your approach bans the regrettably awkward
"hookcmd.~/check-for-debug-strings.skip = true", but I'm not sure
whether or not it's worth it.

To help us visualize the change, here's some common and complicated
tasks and how they look with either schema (let mine be A and yours be
B):

#1 - Add a oneliner (not executing a script)
A:
[hook "post-commit"]
	command = echo post commit
B:
[hook "oneliner"]
	event = post-commit
	command = echo post commit

#2 - Execute a script
A:
[hook "post-commit"]
	command = ~/my-post-commit-hook
B:
[hook "script"]
	event = post-commit
	command = ~/my-post-commit-hook

#3 - Run a handful of scripts in a specific order on one event
A:
[hook "post-commit"]
	command = ~/my-post-commit-1
	command = ~/my-post-commit-2
	command = ~/my-post-commit-3
B:
[hook "script 1"]
	event = post-commit
	command = ~/my-post-commit-1
[hook "script 2"]
	event = post-commit
	command = ~/my-post-commit-2
[hook "script 3"]
	event = post-commit
	command = ~/my-post-commit-3

#4 - Skip a script configured more globally
A:
(original config)
[hook "post-commit"]
	command = /corp/stuff/corp-post-commit
(local config)
[hookcmd "/corp/stuff/corp-post-commit"]
	skip = true
OR,
(original config)
[hookcmd "corp-pc"]
	command = /corp/stuff/corp-post-commit
[hook "post-commit"]
	command = corp-pc
(local config)
[hookcmd "corp-pc"]
	skip = true
B:
(original config)
[hook "corp-pc"]
	event = post-commit
	command = /corp/stuff/corp-post-commit
(local config)
[hook "corp-pc"]
	skip = true

#5 - Execute one script for multiple events
A:
[hookcmd "reusable-hook"]
	command = /long/path/reusable-hook
[hook "post-commit"]
	command = reusable-hook
[hook "pre-commit"]
	command = reusable-hook
[hook "prepare-commit-msg"]
	command = reusable-hook
B:
[hook "reusable-hook"]
	command = /long/path/reusable-hook
	event = post-commit
	event = pre-commit
	event = prepare-commit-msg

#6 - Execute the same script early for one event and late for another
event
A:
(global)
[hookcmd "reusable-hook"]
	command = /long/path/reusable-hook
[hook "pre-commit"]
	command = reusable-hook
(local)
[hook "post-commit"]
	command = other
	command = hooks
	command = reusable-hook

B:
(global)
[hook "reusable-hook"]
	command = /long/path/reusable-hook
	event = pre-commit
(local)
[hook "other"]
	event = post-commit
	command = other
[hook "hooks"]
	event = post-commit
	command = hooks
[hook "reusable-hook"]
	event = reusable-hook


I'm not going to comment on which one I like more yet - I think I will
study this for a while and let others weigh in on their preferences too.
I tried to order the cases from most common to less common. Please feel
free to chime in with more use cases that you think would be handy which
I forgot :)

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 8/9] hook: teach 'hookcmd' config to alias hook scripts
  2021-07-22 23:31                 ` Emily Shaffer
@ 2021-07-23  7:41                   ` Ævar Arnfjörð Bjarmason
  2021-08-04 20:38                     ` Emily Shaffer
  2021-08-04 21:49                     ` Jonathan Tan
  0 siblings, 2 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-23  7:41 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Jul 22 2021, Emily Shaffer wrote:

> On Fri, Jul 16, 2021 at 11:13:42AM +0200, Ævar Arnfjörð Bjarmason wrote:
>> 
>> 
>> On Thu, Jul 15 2021, Emily Shaffer wrote:
>> 
>> > To enable fine-grained options which apply to a single hook executable,
>> > and to make it easier for a single executable to be run on multiple hook
>> > events, teach "hookcmd.<alias>.config". These can be configured as
>> > follows:
>> > [...]
>> > diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
>> > index a97b980cca..5b35170664 100644
>> > --- a/Documentation/config/hook.txt
>> > +++ b/Documentation/config/hook.txt
>> > @@ -3,6 +3,11 @@ hook.<command>.command::
>> >  	executable on your device, a oneliner for your shell, or the name of a
>> >  	hookcmd. See linkgit:git-hook[1].
>> >  
>> > +hookcmd.<name>.command::
>> > +	A command to execute during a hook for which <name> has been specified
>> > +	as a command. This can be an executable on your device or a oneliner for
>> > +	your shell. See linkgit:git-hook[1].
>> > +
>> > [...]
>> > +Global config
>> > +----
>> > +  [hook "post-commit"]
>> > +    command = "linter"
>> > +    command = "~/typocheck.sh"
>> > +
>> > +  [hookcmd "linter"]
>> > +    command = "/bin/linter --c"
>> > +----
>> > +
>> > +Local config
>> > +----
>> > +  [hook "prepare-commit-msg"]
>> > +    command = "linter"
>> > +  [hook "post-commit"]
>> > +    command = "python ~/run-test-suite.py"
>> > +----
>> > +
>> > +With these configs, you'd then run post-commit hooks in this order:
>> > +
>> > +  /bin/linter --c
>> > +  ~/typocheck.sh
>> > +  python ~/run-test-suite.py
>> > +  .git/hooks/post-commit (if present)
>> > +
>> > +and prepare-commit-msg hooks in this order:
>> > +
>> > +  /bin/linter --c
>> > +  .git/hooks/prepare-commit-msg (if present)
>> >  
>> 
>> I still have outstanding feedback on the fundamental design
>> here. I.e. why is this not:
>> 
>>     hook.<name>.event = post-commit
>>     hook.<name>.command = <path>
>> 
>> See:
>> 
>> https://lore.kernel.org/git/87mtv8fww3.fsf@evledraar.gmail.com/
>> 
>> As noted there I don't see why not, and more complexity results from the
>> design choice of doing it the way you're proposing. I.e. we can't
>> discover hooks based on config prefixes, and we end up sticking full FS
>> paths in config keys.
>
> Interesting. My gut says that it would make it harder to neatly write a
> config with the same hook running at the very beginning of one event and
> the very end of another, but I'm not sure whether that's a case to
> optimize for.
>
> I would also need twice as many lines to run a script I wrote as a hook
> - that is, the base case which is probably all most people will need. So
> with your proposal I *must* name every hook I want to add. Instead of
> "hook.pre-commit.command = ~/check-for-debug-strings" I need to
> configure "hook.check-debug-strings.event = pre-commit" and
> "hook.check-debug-strings.command = ~/check-for-debug-strings". That's a
> little frustrating, and if I never want to skip that check or move it
> later or something, it will be extra overhead for no gain for me.

The gain is that "git hook list" becomes a trivial "git config
-show-origin --show-scope --get-regexp" wrapper.

So the series either doesn't need "git hook list" or such a thing
becomes much less complex, especially given the proposed addition of
other features in the area like "git hook edit", i.e. (quoting the
linked E-Mail):

    As just one example; surely "git config edit <name>" would need to
    run around and find config files to edit, then open them in a loop
    for you, no?

    Which we'd eventually want for "git config" in general with an
    --edit-regexp option or whatever, which brings us (well, at least
    me) back to "then let's just add it to git-config?".

> I do agree that your approach bans the regrettably awkward
> "hookcmd.~/check-for-debug-strings.skip = true", but I'm not sure
> whether or not it's worth it.

That design choice also means that you can't expand the path using "git
config --get --type=path.

We do have that with the "includeIf" construct, but if we can avoid it
we should, it makes it play nicer with other assumptions and features of
the config system.

As noted in the follow-up reply while we don't case normalize the LeVeL"
part of "ThReE.LeVeL.KeY" that's tolower(), which we know isn't a 1=1
mapping on some
FS's. https://lore.kernel.org/git/87y2ebo16v.fsf@evledraar.gmail.com/

> To help us visualize the change, here's some common and complicated
> tasks and how they look with either schema (let mine be A and yours be
> B):

Before diving into that, I'll just say I don't care about the trivial
specifics of how this is done, i.e. the bikeshedding of what the config
keys etc. are named.

Just in (as noted above) design choices here forcing avoidable
complexity in other areas.

> #1 - Add a oneliner (not executing a script)
> A:
> [hook "post-commit"]
> 	command = echo post commit
> B:
> [hook "oneliner"]
> 	event = post-commit
> 	command = echo post commit
> #2 - Execute a script
> A:
> [hook "post-commit"]
> 	command = ~/my-post-commit-hook
> B:
> [hook "script"]
> 	event = post-commit
> 	command = ~/my-post-commit-hook

...

> #3 - Run a handful of scripts in a specific order on one event
> A:
> [hook "post-commit"]
> 	command = ~/my-post-commit-1
> 	command = ~/my-post-commit-2
> 	command = ~/my-post-commit-3
> B:
> [hook "script 1"]
> 	event = post-commit
> 	command = ~/my-post-commit-1
> [hook "script 2"]
> 	event = post-commit
> 	command = ~/my-post-commit-2
> [hook "script 3"]
> 	event = post-commit
> 	command = ~/my-post-commit-3

To reply to all the above, yes, your suggestion comes out ahead in being
less verbose.

But isn't the real difference not in the differing prefixes, i.e. hook.*
and hookcmd.* (A) v.s. always hook.* (B, which is what I'm mainly
focusing on, i.e. what requires the added complexity.

But that in that in your proposed way of doing it it's:

    hook.<event-or-name>.*

V.s. my suggestion of:

    hook.<name>.*

And thus whenever you have a <event-or-name> that just happens to be a
built-in hook listed in githooks(5) we in (A) implicitly expand config
like:

    hook.post-commit.command = echo foo

To:

    hook.post-commit.command = echo hi
    hook.post-commit.type    = post-commit

But not knowing about "foo" we don't expand:

    hook.foo.command = echo foo

To:

    hook.foo.command = echo hi
    hook.foo.type    = foo # This would be an error, or ignored.

But rather leave "dangling" for the user to later supply the "*.event"
themselves, i.e.:

    hook.foo.command = echo hi
    hook.foo.event = post-commit

And means that you presumably need to detect this case and error about
it, but my proposed model does not:

    hook.post-commit.command = echo hi
    # User error: "*.type" and <event-or-name>" does not match for
    # "hook.*.command"
    hook.post-commit.type    = pre-commit

And furthermore, that if someone in your model were to do:

    hook.verify-commit.command = echo hi

It's "dangling" today, but if a later version of git learns about a
"verify-commit" hook we'll start running it unexpectedly.

Because your design conflates the slot for known hook type names and
user-supplied names.

On balance I think it's better just to always supply two lines per-hook,
but whether we have this proposed shorthand or not is mostly orthogonal
to everything I mentioned in
https://lore.kernel.org/git/87mtv8fww3.fsf@evledraar.gmail.com/

I.e. my proposed version could also have it, but thinking about it I
think it's not worth it, we should always use <name>, not
<event-or-name> for the reasons noted if you'll read ahead...

> #4 - Skip a script configured more globally
> A:
> (original config)
> [hook "post-commit"]
> 	command = /corp/stuff/corp-post-commit
> (local config)
> [hookcmd "/corp/stuff/corp-post-commit"]
> 	skip = true
> OR,
> (original config)
> [hookcmd "corp-pc"]
> 	command = /corp/stuff/corp-post-commit
> [hook "post-commit"]
> 	command = corp-pc
> (local config)
> [hookcmd "corp-pc"]
> 	skip = true
> B:
> (original config)
> [hook "corp-pc"]
> 	event = post-commit
> 	command = /corp/stuff/corp-post-commit
> (local config)
> [hook "corp-pc"]
> 	skip = true

...which are among other things that no, my proposed version doesn't
have "skip" at all, see point #3 at
https://lore.kernel.org/git/87mtv8fww3.fsf@evledraar.gmail.com/

I.e. I think the "skip" is a thing that falls out of a complexity of
your design that I'm proposing to do away with.

That complexity being that you use <event-or-name> and I use <name>, and
you want to in turn support N number of "*.command" for any
"hook.<event-or-name>".

I'm suggesting we use the typical "last key wins" semantics, if you want
multiple commands for a given hook type you'll just supply multiple
"hook.<name>" sections with the same "hook.*.event = type" in all.

The way to skip hooks in my proposal is:

    hook.<name>.command = true

Or whatever noop command you want on the RHS. In practice we wouldn't
invoke "true", just like we don't invoke "cat" for "PAGER=cat".

But unlike "*.skip" that doesn't require complexity in implementation or
user understanding, it just falls naturally out of the rest of the
model.

> #5 - Execute one script for multiple events
> A:
> [hookcmd "reusable-hook"]
> 	command = /long/path/reusable-hook
> [hook "post-commit"]
> 	command = reusable-hook
> [hook "pre-commit"]
> 	command = reusable-hook
> [hook "prepare-commit-msg"]
> 	command = reusable-hook
> B:
> [hook "reusable-hook"]
> 	command = /long/path/reusable-hook
> 	event = post-commit
> 	event = pre-commit
> 	event = prepare-commit-msg

It's been so long since I wrote the original E-Mail that I'm having to
skim my own summary there & come up with things as I go along, so maybe
this conflicts with something I said earlier.

But no, I don't think I meant that *.event should be multi-value, or
that that magic is worthwhile. I.e. I think we just want:

    [hook "not-a-reusable-section-1"]
        command = /long/path/reusable-hook
        event = post-commit
    [hook "not-a-reusable-section-2"]
        command = /long/path/reusable-hook
        event = pre-commit
    [hook "not-a-reusable-section-3"]
        command = /long/path/reusable-hook
        event = prepare-commit-msg

I.e. is wanting to use the same command for N different hooks really
that common a use-case to avoid a little verbosity, and quite a lot of
complexity?

How would such a hook even be implemented?

We don't have anything in your hook execution model (and I don't
remember you adding this) which passes down a "you are this type of
hook" to hooks.

It's now implicit in that the hook invoked at .git/hooks/post-commit or
.git/hooks/pre-commit can check its own basename, but it won't be with
configurable hooks.

We could start passing down a GIT_HOOK_TYPE in the environment or
whatever, but I think the simpler case of just having the user do it is
better.

I'm assuming that mainly because people have wanted a
"/long/path/reusable-hook" router script because we have not supported
executing N hooks, or supported concurrency. Once we do that the
complexity of such routing scripts (mostly, not everyone's needs will be
met) will be replaced by a little bit of config.

I don't see why it's worth it to micro-optimize for lines in that config
at the cost of increased config complexity.

For example, how do you "skip" an "event" type? You have it for
"*.command"? So let's say in your model (A) I have system config like:

    [hook "my-reusable"]
    command = /some/commit-msg-hook
    event = prepare-commit-msg

Now I want to skip that, easy enough, in my local config:

    [hook "my-reusable"]
    skip = true

Or if I wanted to also invoke it for commit-msg events, in my local
config (which piggy-backs on the system one):

    # Now runs for both prepare-commit-msg and commit-message
    [hook "my-reusable"]
    event = commit-msg

Or, if I want to skip the command name and substitute my own:

    [hook "my-reusable"]
    skip = true
    command = /my/commit-msg-hook

I'm just assuming you mean that to work, as discussed above I think all
this is leading us down the road of too much complexity.

But anyway, now I want to instead not run it as a prepare-commit-msg
hook, but a commit-msg hook.

As far as I can tell you haven't specified that, but I think something
consistent with your model would be

    [hook "my-reusable"]
    event = skip
    event = commit-msg

Or rather, because that conflates "skip" with a hook event type name:

    [hook "my-reusable"]
    skipEvent = true
    event = commit-msg

I.e. the "skip" you have now really means "skipCommand", so we'll need a
skipSomeName to skip other "hook.*.someName".

As noted above I think all of this is too complex, let's just do the
system config like this (exact same as the above):

    [hook "my-reusable"]
    command = /some/commit-msg-hook
    event = prepare-commit-msg

To skip it, in my local config:

    [hook "my-reusable"]
    command = true

And to re-use /some/commit-msg-hook without having to re-include it in
your local config the answer is that we don't support that level of
cleverness. Just do this:

    # First skip the system hook
    [hook "my-reusable"]
    command = true

    # Set up another hook, just copy/pasting the /some/commit-msg-hook command
    [hook "my-local"]
    command = /some/commit-msg-hook
    event = commit-msg

I.e. I think these cases of someone having say a /etc/gitconfig hook on
their system not under their control that they want to not run, *but*
not as the "event" there, but as another event type is too specific a
use-case for us to worry about.

> #6 - Execute the same script early for one event and late for another
> event
> A:
> (global)
> [hookcmd "reusable-hook"]
> 	command = /long/path/reusable-hook
> [hook "pre-commit"]
> 	command = reusable-hook
> (local)
> [hook "post-commit"]
> 	command = other
> 	command = hooks
> 	command = reusable-hook

Even with I think it's fair to say deep knowledge of your proposal at
this point I still needed to read this a few times to see if that:

    command = reusable-hook

Is referring to:

    [hookcmd "reusable-hook"]

I.e. is it going to run:

    command = /long/path/reusable-hook

Or is it just re-specifying /long/path/reusable-hook but relying on a
$PATH lookup?

Having reasoned through that I think the answer is the former. But that
also means that in your model:

    [hookcmd "rm -rf /"]
    command = echo this will not actually destroy your data
    [hook "pre-commit"]
    command = rm -rf /

Is going to run that friendly "echo" command, since "command = rm -rf /"
just refers to the "rm -rf /" <name>, not <command>, right the "hookcmd"
line is removed, at which point we'll stop treating it as a <name> and
run it as a <command>?

In practice I think users are unlikely to use actively harmful names
like that.

I'm just making the point that I should not need to know about previous
config to see if a "hook.pre-commit.command = rm -rf /" is harmless or
not, or need to carefully squint to see if the "reusable-hook" is
referring to a section name or command name.

Or am I just confused at this point?

> B:
> (global)
> [hook "reusable-hook"]
> 	command = /long/path/reusable-hook
> 	event = pre-commit
> (local)
> [hook "other"]
> 	event = post-commit
> 	command = other
> [hook "hooks"]
> 	event = post-commit
> 	command = hooks
> [hook "reusable-hook"]
> 	event = reusable-hook

No, that's not what I'm proposing. I.e. "*.event" is reserved for
built-in events listed in githooks(5) that git itself knows about.

If you say "event = reusable-hook" that's going to be one of ignored,
warned or errored about.

(Probably "ignored" is the best thing to support multi-version setups,
but that's not hook-specific, just how we treat unknown config in
general)

My version of this would be the same as noted with "we don't support
that level of cleverness[...]" above.

I.e. you'd not re-use that "/long/path/reusable-hook", you can skip it
through "command = true", then just copy the relevnt part into your new
config. So:

        (global)
	[hook "reusable-hook"]
		command = /long/path/reusable-hook
		event = pre-commit

	(local)
	[hook "reusable-hook"]
		command = true # skip it

        # The "hooks" name is arbitrary, "my-hooks" or whatever would be
        # clearer, but just going with your example...

	[hook "hooks"]
		event = post-commit
		command = hooks
        # Not very reusable then...   
	[hook "reusable-hook"]
                command = /long/path/reusable-hook
		event = pre-commit


> [...]Please feel free to chime in with more use cases that you think would
> be handy which I forgot :)

I couldn't find this at a quick glance but I think we also had a
disussion at some point about hooks controlling parallelism. AFAICT your
current implementation just has global:

    hook.jobs=N

And we then blacklist certain hooks to always have hook.jobs=1, e.g. the
commit-msg hook that needs an implicit "lock" on that file (or rather,
we think that's the most common use-case).

I think my version of always having hook.<name>.{event,command} be one
value is also better in that case, i.e. we'd then:

    [hook "myhook"]
    command = some-command
    event = pre-receive
    parallel = true # the default

    [hook "myhook2"]
    command = some-command2
    event = pre-receive
    parallel = true # the default

    [hook "myhook3"]
    command = some-unconcurrent-command
    event = pre-receive
    parallel = false # I'm not OK with concurrency

To go along with something like:

    hook
	jobs = 8

Or:

    [hookEvent "pre-receive"]
    jobs = 4

But if you have N numer of "command" in a section it gets murky, does
"parallel = false" then apply to the whole section, but sections can
have one or more values. So we'd need both a
"hookEvent.pre-receive.jobs=N" and per-section config to
control/suppress parallelism?

I haven't thought about it deeply, but have a hunch that having sections
be a 1=1 mapping with a specific command instead of either 1=1 or 1=many
is going to make that easier to implement and understand.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 7/9] hook: allow out-of-repo 'git hook' invocations
  2021-07-22 23:07                 ` Emily Shaffer
@ 2021-07-23  9:18                   ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-23  9:18 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Jul 22 2021, Emily Shaffer wrote:

> On Fri, Jul 16, 2021 at 10:33:25AM +0200, Ævar Arnfjörð Bjarmason wrote:
>> 
>> 
>> On Thu, Jul 15 2021, Emily Shaffer wrote:
>> 
>> > Since hooks can now be supplied via the config, and a config can be
>> > present without a gitdir via the global and system configs, we can start
>> > to allow 'git hook run' to occur without a gitdir. This enables us to do
>> > things like run sendemail-validate hooks when running 'git send-email'
>> > from a nongit directory.
>> >
>> > It still doesn't make sense to look for hooks in the hookdir in nongit
>> > repos, though, as there is no hookdir.
>> 
>> Hrm, I haven't tested but re the discussion we had about
>> RUN_SETUP_GENTLY on my re-rolled base topic is this really just a
>> regression in my changes there?
>> 
>> I.e. I assumed we could do RUN_SETUP for the bug-for-bug compatibility
>> step, but send-email runs out of repo hooks and we just didn't have
>> tests for it, or am I missing something?
>
> I'm not sure. I could see a case for you including RUN_SETUP_GENTLY on
> your series and adding a test for sendemail-validate + core.hooksPath in
> global config. I think I also don't have support for that case here,
> actually....

The narrow goal of the base topic is to be a bug-for-bug compatible
version of what we have now on "master", just via a dispatch
command/API.

So yeah, that git-send-email.perl behavior looks bizarre, but let's fix
it in a separate set of behavior changing patches on top, no?

> Anyway, it looks like right now git-send-email.perl:validate_patch()
> doesn't bother if it's out-of-repo, so this wouldn't have worked before (and
> still won't work even after this change). So either I can add a patch to
> my series to allow that, or you can modify your patch converting
> sendemail-validate to 'git hook run' to drop the 'if $repo' line and
> teach your series to behave correctly with nongit+hooksPath. It looks
> like in my earlier attempt at the series, I did drop that check and run
> 'git hook run' no matter what kind of directory we're in.

I think this was one of the things that either needed a test change in
your series, or I saw tests for changes to existing but untested
behavior, I think this was the latter. 

I completely agree that the behavior is weird, probably undesired and
should be changed. I'm just saying that we should split up refactorings
& behavior changes. 

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 6/9] hook: include hooks from the config
  2021-07-22 22:51                 ` Emily Shaffer
@ 2021-07-23  9:22                   ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-23  9:22 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Jul 22 2021, Emily Shaffer wrote:

> On Fri, Jul 16, 2021 at 11:01:24AM +0200, Ævar Arnfjörð Bjarmason wrote:
>> 
>> 
>> On Thu, Jul 15 2021, Emily Shaffer wrote:
>> 
>> > +static struct hook * find_hook_by_command(struct list_head *head, const char *command)
>> 
>> nit: "*find[...]" not "* find[...]", also let's wrap the long line.
> ACK
>> 
>> > +{
>> > +	struct list_head *pos = NULL, *tmp = NULL;
>> > +	struct hook *found = NULL;
>> > +
>> > +	list_for_each_safe(pos, tmp, head) {
>> > +		struct hook *it = list_entry(pos, struct hook, list);
>> > +		if (!strcmp(it->command, command)) {
>> > +		    list_del(pos);
>> > +		    found = it;
>> > +		    break;
>> 
>> Indented with spaces.
>
> I don't know how I even did this. *facepalm*
>
>> 
>> Also is there some subtlety in the list macro here or can we just
>> "s/break/return it/" and skip the break/return pattern?
>
> I guess it's probably fine, but we'd need the final return anyway
> ("otherwise returns NULL"). IMO one return is more readable than two
> returns, so I'd rather leave this.

Sure makes sense. I'd tend to go for two returns, but let's not split
hairs on personal style.

>> 
>> > +static struct hook * append_or_move_hook(struct list_head *head, const char *command)
>> 
>> Same whitespace nits.
> ACK
>> 
>> > +	if (!to_add) {
>> > +		/* adding a new hook, not moving an old one */
>> > +		to_add = xmalloc(sizeof(*to_add));
>> > +		to_add->command = command;
>> > +		to_add->feed_pipe_cb_data = NULL;
>> > +		/* This gets overwritten in hook_list() for hookdir hooks. */
>> > +		to_add->from_hookdir = 0;
>> 
>> I commented on init verbosity elsewhere, i.e. we could do some things
>> via macros, but in this case just having an "init" helper make sense,
>> but we have at least two places copying the same init of all fields,
>> should just be hook_init_hook() or whatever it'll be called. Maybe with
>> a second "from hookdir" param?
>
> Hm, where is the second place where we init everything? I think with
> this commit we remove anywhere we're putting together a 'struct hook' manually
> except during this helper? Hooks from hookdir are initted by
> 'append_or_move_hook()'ing them to the end of the list and modifying the
> from_hookdir field, and builtin/hook.c just calls hook_list() (and some
> list.h helpers to find an entry).

Looking again I think I just misread this then, thanks.

>> > +	/* to enable oneliners, let config-specified hooks run in shell */
>> > +	cp->use_shell = !run_me->from_hookdir;
>> 
>> I've lost track at this point, but doesn't that mean we're going to use
>> a shell when we run our own do-not-need-a-shell hooks ourselves?
>> 
>> Isn't isatty() more appropriate here, or actually even interactively why
>> is the shell needed (maybe this is answered elswhere...).
>
> use_shell means "conditionally guess whether I need to wrap this thing
> in `sh -c`" - it doesn't have anything to do with TTY or not. So we need
> this for something like `hook.post-commit.command = echo "made a
> commit"`. In this case the entire argv[0] will be the oneliner, which
> you will need use_shell set for. If we *do* just do something simple,
> like `hook.post-commit.command = /bin/mail`, even though
> use_shell is marked, the child_process runner will notice that there's
> no reason to wrap in 'sh -c' and so will just run the /bin/mail
> executable directly.

Ah, I missed that. Makes sense.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 4/9] hook: treat hookdir hook specially
  2021-07-22 22:24                 ` Emily Shaffer
@ 2021-07-23  9:26                   ` Ævar Arnfjörð Bjarmason
  2021-07-23 17:33                   ` Felipe Contreras
  1 sibling, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-23  9:26 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Jul 22 2021, Emily Shaffer wrote:

> On Fri, Jul 16, 2021 at 10:58:34AM +0200, Ævar Arnfjörð Bjarmason wrote:
>> 
>> 
>> On Thu, Jul 15 2021, Emily Shaffer wrote:
>> 
>> > Soon, we will allow users to specify hooks using the config. These
>> > config-specified hooks may require different child_process options than
>> > hook executables in the gitdir. So, let's differentiate between hooks
>> > coming from the gitdir and hooks coming from the config.
>> >
>> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
>> > ---
>> >  hook.c | 3 ++-
>> >  hook.h | 2 ++
>> >  2 files changed, 4 insertions(+), 1 deletion(-)
>> >
>> > diff --git a/hook.c b/hook.c
>> > index 19138a8290..3a588cb055 100644
>> > --- a/hook.c
>> > +++ b/hook.c
>> > @@ -117,6 +117,7 @@ struct list_head* hook_list(const char* hookname)
>> >  		struct hook *to_add = xmalloc(sizeof(*to_add));
>> >  		to_add->hook_path = hook_path;
>> >  		to_add->feed_pipe_cb_data = NULL;
>> > +		to_add->from_hookdir = 1;
>> >  		list_add_tail(&to_add->list, hook_head);
>> >  	}
>> >  
>> > @@ -200,7 +201,7 @@ static int pick_next_hook(struct child_process *cp,
>> >  	cp->dir = hook_cb->options->dir;
>> >  
>> >  	/* add command */
>> > -	if (hook_cb->options->absolute_path)
>> > +	if (run_me->from_hookdir && hook_cb->options->absolute_path)
>> >  		strvec_push(&cp->args, absolute_path(run_me->hook_path));
>> >  	else
>> >  		strvec_push(&cp->args, run_me->hook_path);
>> > diff --git a/hook.h b/hook.h
>> > index 586ddf40bb..60389cd8cd 100644
>> > --- a/hook.h
>> > +++ b/hook.h
>> > @@ -22,6 +22,8 @@ struct hook {
>> >  	/* The path to the hook */
>> >  	const char *hook_path;
>> >  
>> > +	unsigned from_hookdir : 1;
>> > +
>> >  	/*
>> >  	 * Use this to keep state for your feed_pipe_fn if you are using
>> >  	 * run_hooks_opt.feed_pipe. Otherwise, do not touch it.
>> 
>> The "from_hookdir" looks like it isn't used until 6/9, and maybe the
>> absolute_path change too? In any case this seems like a carried-forward
>> rebase of
>> https://lore.kernel.org/git/20210311021037.3001235-5-emilyshaffer@google.com/
>> or some version thereof.
>> 
>> At this point I tihnk it would be way better to squash this and other
>> such changes that basically add a field to a struct that isn't used yet
>> into whatever commit use/need them.
>
> I think at this point we run into you and me having different
> patch-storytelling styles - which probably is what led to the big topic
> restart in the first place ;)
>
> I'd prefer to see the "start using config too" patch stay as small as it
> can; that's why I ended up with a handful of minor setup commits and
> then one "and now here's config" commit.
>
> Even if it's different from how you would tell it - is it wrong? (And if
> it is, that's fine, and I'll change it, but I don't think it is - that's
> why I structured the series this way.)

It's not wrong, if you'd like it that way sure.

The only reason I nudged you about it was that I assumed you'd perhaps
mostly rebased these one-at-a-time on top of the base topic, before the
base topic the patches were much larger.

There is some arbitrary cut-off-limit where it makes sense to split the
addition of code to be used later with the actual use, and that limit is
a matter of taste.

I thought perhaps given the base topic some of these patches should be
squshed given their new size, but if you've looked at them and think
it's fine as-is let's leave it at that.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 3/9] hook: introduce "git hook list"
  2021-07-22 22:18                 ` Emily Shaffer
@ 2021-07-23  9:29                   ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-23  9:29 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Jul 22 2021, Emily Shaffer wrote:

> On Fri, Jul 16, 2021 at 10:52:27AM +0200, Ævar Arnfjörð Bjarmason wrote:
>> 
>> 
>> On Thu, Jul 15 2021, Emily Shaffer wrote:
>> 
>> >  static const char * const builtin_hook_usage[] = {
>> >  	N_("git hook <command> [...]"),
>> > +	N_("git hook list <hookname>"),
>> >  	N_("git hook run [<args>] <hook-name> [-- <hook-args>]"),
>> >  	NULL
>> 
>> Uses <hook-name> already, let's use that too. I can't remember if it's
>> something I changed myself, or if your original version used both and I
>> picked one for consistency, or...
>> 
>> Anyway, I can re-roll the base topic or whatever, but let's have the end
>> result use one or the other.
>
> 'hook-name' is fine, I'll use that. Thanks for pointing it out.
>
>> 
>> > +	if (argc < 1) {
>> > +		usage_msg_opt(_("You must specify a hook event name to list."),
>> > +			      builtin_hook_usage, list_options);
>> > +	}
>> 
>> {} braces not needed.
> ACK
>> 
>> 
>> > +	if (!strcmp(argv[0], "list"))
>> > +		return list(argc, argv, prefix);
>> >  	if (!strcmp(argv[0], "run"))
>> 
>> This should be "else if" now.
>> 
>> (Doesn't matter for code execution, just IMO readability, but I'll leave
>> that to you ... :)
>
> Eh, I think it's easier to read in the strcmps line up, so I will leave
> it this way ;)

*nod*

>> 
>> >  		return run(argc, argv, prefix);
>> >  	else
>> > diff --git a/hook.c b/hook.c
>> > index 935751fa6c..19138a8290 100644
>> > --- a/hook.c
>> > +++ b/hook.c
>> > @@ -104,22 +104,20 @@ int hook_exists(const char *name)
>> >  struct list_head* hook_list(const char* hookname)
>> >  {
>> >  	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
>> > +	const char *hook_path = find_hook(hookname);
>> > +
>> >  
>> >  	INIT_LIST_HEAD(hook_head);
>> >  
>> >  	if (!hookname)
>> >  		return NULL;
>> >  
>> > -	if (have_git_dir()) {
>> > -		const char *hook_path = find_hook(hookname);
>> > -
>> > -		/* Add the hook from the hookdir */
>> > -		if (hook_path) {
>> > -			struct hook *to_add = xmalloc(sizeof(*to_add));
>> > -			to_add->hook_path = hook_path;
>> > -			to_add->feed_pipe_cb_data = NULL;
>> > -			list_add_tail(&to_add->list, hook_head);
>> > -		}
>> > +	/* Add the hook from the hookdir */
>> > +	if (hook_path) {
>> > +		struct hook *to_add = xmalloc(sizeof(*to_add));
>> > +		to_add->hook_path = hook_path;
>> > +		to_add->feed_pipe_cb_data = NULL;
>> > +		list_add_tail(&to_add->list, hook_head);
>> 
>> Maybe we should have a INIT for "struct hook" too? This also needlessly
>> leaves behind an un-free'd hook struct in a way that it wouldn't if we
>> just had this on the stack, no?
>
> I can clean it up here, but I don't think we need an initializer for
> struct hook. This code chunk gets moved into one of the list
> manipulators (append_or_move() or something) after the config is
> introduced.
>
> I don't think it leaves an unfreed hook laying around, does it?
> list_add_tail() uses the malloc'd pointer, and we free() in
> clear_hook_list(). What am I missing?

I don't think you're missing anything. I replied on that "struct hook"
in another E-Mail, i.e. I think I just misread parts of this. Thanks.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 2/9] hook: allow parallel hook execution
  2021-07-22 21:12                 ` Emily Shaffer
@ 2021-07-23  9:30                   ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-23  9:30 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Jul 22 2021, Emily Shaffer wrote:

> On Fri, Jul 16, 2021 at 10:36:10AM +0200, Ævar Arnfjörð Bjarmason wrote:
>> 
>> 
>> On Thu, Jul 15 2021, Emily Shaffer wrote:
>> 
>> > In many cases, there's no reason not to allow hooks to execute in
>> > parallel. run_processes_parallel() is well-suited - it's a task queue
>> > that runs its housekeeping in series, which means users don't
>> > need to worry about thread safety on their callback data. True
>> > multithreaded execution with the async_* functions isn't necessary here.
>> > Synchronous hook execution can be achieved by only allowing 1 job to run
>> > at a time.
>> >
>> > Teach run_hooks() to use that function for simple hooks which don't
>> > require stdin or capture of stderr.
>> 
>> This doesn't mention...
>> 
>> >  	int ret;
>> > -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
>> > +	struct run_hooks_opt opt;
>> >  
>> > +	run_hooks_opt_init_sync(&opt);
>> 
>> 
>> ...why we need to bring the s/macro/func/ init pattern, back, but looking ahead...
>> 
>> > +int configured_hook_jobs(void)a
>> > +{
>> > +	int n = online_cpus();
>> > +	git_config_get_int("hook.jobs", &n);
>> > +
>> > +	return n;
>> > +}
>> > +
>> >  int hook_exists(const char *name)
>> >  {
>> >  	return !!find_hook(name);
>> > @@ -117,6 +125,26 @@ struct list_head* hook_list(const char* hookname)
>> >  	return hook_head;
>> >  }
>> >  
>> > +void run_hooks_opt_init_sync(struct run_hooks_opt *o)
>> > +{
>> > +	strvec_init(&o->env);
>> > +	strvec_init(&o->args);
>> > +	o->path_to_stdin = NULL;
>> > +	o->jobs = 1;
>> > +	o->dir = NULL;
>> > +	o->feed_pipe = NULL;
>> > +	o->feed_pipe_ctx = NULL;
>> > +	o->consume_sideband = NULL;
>> > +	o->invoked_hook = NULL;
>> > +	o->absolute_path = 0;
>> > +}
>> > +
>> > +void run_hooks_opt_init_async(struct run_hooks_opt *o)
>> > +{
>> > +	run_hooks_opt_init_sync(o);
>> > +	o->jobs = configured_hook_jobs();
>> > +}
>> 
>> ...okey, so it's because you brought back the "call jobs function" in
>> one of the init functions.
>> 
>> I had a comment in a previous round, I found
>> https://lore.kernel.org/git/87lf7bzbrk.fsf@evledraar.gmail.com/, but I
>> think there was a later one where I commented on the "jobs" field
>> specifically.
>> 
>> Anyway, it seems much easier to me to just keep the simpler macro init
>> and then:
>> 
>> > -	if (options->jobs != 1)
>> > -		BUG("we do not handle %d or any other != 1 job number yet", options->jobs);
>> > -
>> >  	run_processes_parallel_tr2(options->jobs,
>> >  				   pick_next_hook,
>> >  				   notify_start_failure,
>> 
>> There's this one place where we use the "jobs" parameter, just do
>> something like this there:
>>         
>>         int configured_hook_jobs(void)
>>         {
>>                 static int jobs;
>>                 if (!jobs)
>>                     return jobs;
>>                 if (git_config_get_int("hook.jobs", &jobs))
>>                     jobs = online_cpus();
>>                 return jobs;
>>         }
>> 
>> I.e. you also needlessly call online_cpus() when we're about to override
>> it in the config. The git_config_get_int()'s return value indicates
>> whether you need to do that. Then just:
>> 
>>     int jobs = options->jobs ? options->jobs : configured_hook_jobs();
>>     run_processes_parallel_tr2(jobs, [...]);
>
> Ahh, and then let RUN_HOOKS_OPT_INIT_ASYNC set jobs to 0 ("go look it
> up"). Yeah, that makes sense.
>
> Shout if somehow you meant to leave just one initializer macro;
> otherwise, I'll do it this way - with RUN_HOOKS_OPT_INIT_ASYNC and
> RUN_HOOKS_OPT_INIT_SYNC. I think it's valuable for hook callers to make
> it very plain at the callsite whether they're parallelizable or not, and
> I think
>
>  struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
>  opt.jobs = 0;
>
> doesn't make that as obvious.

Yes agreed, sorry about the ambiguity, I meant we should have two init
macros, just like e.g. STRING_LIST_INIT_NODUP and STRING_LIST_INIT_DUP.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 4/9] hook: treat hookdir hook specially
  2021-07-22 22:24                 ` Emily Shaffer
  2021-07-23  9:26                   ` Ævar Arnfjörð Bjarmason
@ 2021-07-23 17:33                   ` Felipe Contreras
  2021-07-23 18:22                     ` Eric Sunshine
  1 sibling, 1 reply; 479+ messages in thread
From: Felipe Contreras @ 2021-07-23 17:33 UTC (permalink / raw)
  To: Emily Shaffer, Ævar Arnfjörð Bjarmason; +Cc: git

Hello Emily,

Emily Shaffer wrote:
> On Fri, Jul 16, 2021 at 10:58:34AM +0200, Ævar Arnfjörð Bjarmason wrote:
> > On Thu, Jul 15 2021, Emily Shaffer wrote:
> > 
> > > Soon, we will allow users to specify hooks using the config. These
> > > config-specified hooks may require different child_process options than
> > > hook executables in the gitdir. So, let's differentiate between hooks
> > > coming from the gitdir and hooks coming from the config.
> > >
> > > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > > ---
> > >  hook.c | 3 ++-
> > >  hook.h | 2 ++
> > >  2 files changed, 4 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/hook.c b/hook.c
> > > index 19138a8290..3a588cb055 100644
> > > --- a/hook.c
> > > +++ b/hook.c
> > > @@ -117,6 +117,7 @@ struct list_head* hook_list(const char* hookname)
> > >  		struct hook *to_add = xmalloc(sizeof(*to_add));
> > >  		to_add->hook_path = hook_path;
> > >  		to_add->feed_pipe_cb_data = NULL;
> > > +		to_add->from_hookdir = 1;
> > >  		list_add_tail(&to_add->list, hook_head);
> > >  	}
> > >  
> > > @@ -200,7 +201,7 @@ static int pick_next_hook(struct child_process *cp,
> > >  	cp->dir = hook_cb->options->dir;
> > >  
> > >  	/* add command */
> > > -	if (hook_cb->options->absolute_path)
> > > +	if (run_me->from_hookdir && hook_cb->options->absolute_path)
> > >  		strvec_push(&cp->args, absolute_path(run_me->hook_path));
> > >  	else
> > >  		strvec_push(&cp->args, run_me->hook_path);
> > > diff --git a/hook.h b/hook.h
> > > index 586ddf40bb..60389cd8cd 100644
> > > --- a/hook.h
> > > +++ b/hook.h
> > > @@ -22,6 +22,8 @@ struct hook {
> > >  	/* The path to the hook */
> > >  	const char *hook_path;
> > >  
> > > +	unsigned from_hookdir : 1;
> > > +
> > >  	/*
> > >  	 * Use this to keep state for your feed_pipe_fn if you are using
> > >  	 * run_hooks_opt.feed_pipe. Otherwise, do not touch it.
> > 
> > The "from_hookdir" looks like it isn't used until 6/9, and maybe the
> > absolute_path change too? In any case this seems like a carried-forward
> > rebase of
> > https://lore.kernel.org/git/20210311021037.3001235-5-emilyshaffer@google.com/
> > or some version thereof.
> > 
> > At this point I tihnk it would be way better to squash this and other
> > such changes that basically add a field to a struct that isn't used yet
> > into whatever commit use/need them.
> 
> I think at this point we run into you and me having different
> patch-storytelling styles - which probably is what led to the big topic
> restart in the first place ;)

Yes, but as a reader of the story I prefer not to have to read the
entire thing in order to understand it. I prefer each page to tell a
small story.

> I'd prefer to see the "start using config too" patch stay as small as it
> can; that's why I ended up with a handful of minor setup commits and
> then one "and now here's config" commit.

I'm in favor of small patches too, but not to the point where the patch
is not useful by itself.

This is where where patch storytelling and actual storytelling differ:
we don't need Chekhov's guns.

> Even if it's different from how you would tell it - is it wrong? (And if
> it is, that's fine, and I'll change it, but I don't think it is - that's
> why I structured the series this way.)

It's not wrong.

But one of the strongest advantages of open source is that you can have
many more reviewers than in closed source, and following Linus's law:
"given enough eyeballs, all bugs are shallow", that's a great thing.

But in order to fully take advantage of that you need to write patches
in a way that armchair reviewers can simply take a cursory look at the
patch and figure out that's obviously correct.

Putting my armchair reviewer hat I cannot do that for this particular
patch, I would need to do more work to make sense of it, and while I'm
writing this message to explains that, others will simply skip it, and
that's a lost opportunity.

Cheers.

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 4/9] hook: treat hookdir hook specially
  2021-07-23 17:33                   ` Felipe Contreras
@ 2021-07-23 18:22                     ` Eric Sunshine
  2021-07-23 20:02                       ` Felipe Contreras
  0 siblings, 1 reply; 479+ messages in thread
From: Eric Sunshine @ 2021-07-23 18:22 UTC (permalink / raw)
  To: Felipe Contreras
  Cc: Emily Shaffer, Ævar Arnfjörð Bjarmason, Git List

On Fri, Jul 23, 2021 at 1:34 PM Felipe Contreras
<felipe.contreras@gmail.com> wrote:
> Emily Shaffer wrote:
> > On Fri, Jul 16, 2021 at 10:58:34AM +0200, Ævar Arnfjörð Bjarmason wrote:
> > > At this point I tihnk it would be way better to squash this and other
> > > such changes that basically add a field to a struct that isn't used yet
> > > into whatever commit use/need them.
> >
> > I think at this point we run into you and me having different
> > patch-storytelling styles - which probably is what led to the big topic
> > restart in the first place ;)
>
> Yes, but as a reader of the story I prefer not to have to read the
> entire thing in order to understand it. I prefer each page to tell a
> small story.
>
> Putting my armchair reviewer hat I cannot do that for this particular
> patch, I would need to do more work to make sense of it, and while I'm
> writing this message to explains that, others will simply skip it, and
> that's a lost opportunity.

Implicit in what Felipe and Ævar are saying is that a well-structured
patch series asks the reviewer to keep only one or two details in mind
after reading a patch in order to understand the next patch in the
series, and that the reviewer shouldn't be expected to keep a large
set of details in mind over several patches. Unlike the author of the
patches who can keep all the details in mind at once and understands
the series in its entirety, reviewers (usually) don't have such
luxury[1]. So, it's important to hand-hold reviewers as much as
possible by not asking them to remember a lot of details between
patches and by ensuring that the details which they must remember only
need to be remembered for a very short time. This is why it is
helpful, for instance, to bundle documentation and test updates in the
patch with changes to code, so the reviewer can see at a glance that
the changes to documentation and tests match the changes to the code,
rather than delaying documentation and test updates until later in the
series.

[1]: If you've ever read a novel in which the author has multiple
story lines running and switches between story lines infrequently,
such that when the author switches back to story-line "A", which you
last saw 100 pages ago, and you can't remember what was going in "A"
or even who the minor characters are anymore, so that you have to go
back and reread 10 or 20 pages from the previous time you saw "A",
then that's representative of the difficulty reviewers can experience
when reading a patch series, except with a patch series, the cognitive
load is already quite high. (Very nice run-on sentence I just wrote.)

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 4/9] hook: treat hookdir hook specially
  2021-07-23 18:22                     ` Eric Sunshine
@ 2021-07-23 20:02                       ` Felipe Contreras
  0 siblings, 0 replies; 479+ messages in thread
From: Felipe Contreras @ 2021-07-23 20:02 UTC (permalink / raw)
  To: Eric Sunshine, Felipe Contreras
  Cc: Emily Shaffer, Ævar Arnfjörð Bjarmason, Git List

Eric Sunshine wrote:
> On Fri, Jul 23, 2021 at 1:34 PM Felipe Contreras
> <felipe.contreras@gmail.com> wrote:
> > Emily Shaffer wrote:
> > > On Fri, Jul 16, 2021 at 10:58:34AM +0200, Ævar Arnfjörð Bjarmason wrote:
> > > > At this point I tihnk it would be way better to squash this and other
> > > > such changes that basically add a field to a struct that isn't used yet
> > > > into whatever commit use/need them.
> > >
> > > I think at this point we run into you and me having different
> > > patch-storytelling styles - which probably is what led to the big topic
> > > restart in the first place ;)
> >
> > Yes, but as a reader of the story I prefer not to have to read the
> > entire thing in order to understand it. I prefer each page to tell a
> > small story.
> >
> > Putting my armchair reviewer hat I cannot do that for this particular
> > patch, I would need to do more work to make sense of it, and while I'm
> > writing this message to explains that, others will simply skip it, and
> > that's a lost opportunity.
> 
> Implicit in what Felipe and Ævar are saying is that a well-structured
> patch series asks the reviewer to keep only one or two details in mind
> after reading a patch in order to understand the next patch in the
> series, and that the reviewer shouldn't be expected to keep a large
> set of details in mind over several patches. Unlike the author of the
> patches who can keep all the details in mind at once and understands
> the series in its entirety, reviewers (usually) don't have such
> luxury[1]. So, it's important to hand-hold reviewers as much as
> possible by not asking them to remember a lot of details between
> patches and by ensuring that the details which they must remember only
> need to be remembered for a very short time. This is why it is
> helpful, for instance, to bundle documentation and test updates in the
> patch with changes to code, so the reviewer can see at a glance that
> the changes to documentation and tests match the changes to the code,
> rather than delaying documentation and test updates until later in the
> series.

Another important point is that while as a patch author it's natural to
push back against possibly unnecessary changes because it would require
considerably more work, there's a reason why writers sometimes chose to
rewrite entire chapters, and it's because every effort to improve the
text as an author would be translated into less effort for the
potentially millions of readers.

It's a multiplicative effect.

Sure, programmers don't have millions of readers, but I think it makes
sense to do 2x the effort as a patch author to receive 4x the review
(at least).

Cheers.

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 00/27] Base for "config-based-hooks"
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (27 preceding siblings ...)
  2021-07-15 23:25           ` [PATCH 0/9] config-based hooks restarted Emily Shaffer
@ 2021-07-28 20:39           ` Emily Shaffer
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
  29 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-07-28 20:39 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

On Thu, Jun 17, 2021 at 12:22:34PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> This v3 of the "Base for 'config-based-hooks'" topic is >95% a
> slimmed-down versio nof Emily Schaffer's work to introduce a mechanism
(Tiniest of nits - I only have a 'c' in my middle name ;) )
> to drive hooks via config.
> 
> This topic doesn't do that, but moves all hook execution (C libary and
> Perl|Python script) to either the new hook.[ch] library code, or the
> "git hook run" utility.
> 
> See previous iterations for more details:
> 
>  v0 (Emily's): http://lore.kernel.org/git/20210527000856.695702-1-emilyshaffer@google.com
>  v1: https://lore.kernel.org/git/cover-00.31-00000000000-20210528T110515Z-avarab@gmail.com/
>  v2: http://lore.kernel.org/git/cover-00.30-00000000000-20210614T101920Z-avarab@gmail.com
> 
> This series gained two new dependencies since v2, my just-submitted
> preparatory topics:
> 
>     https://lore.kernel.org/git/cover-0.3-0000000000-20210617T095827Z-avarab@gmail.com/
>     https://lore.kernel.org/git/cover-0.3-0000000000-20210617T100239Z-avarab@gmail.com/
> 
> And hopefully addreses all the feedback on v2, mainly/entirely from
> Emily at https://lore.kernel.org/git/YMfLO9CT+iIDR3OA@google.com
> 
> I'd normally have waited longer for a v3, but as discussed in the
> small hook.[ch] topic that precedes this one building the new
> hook-list.h had an error on Windows CI due to a missing CMake change,
> that's now fixed.
> 
> Other changes can be seen in the range-diff, all rather trivial fixes
> like comment fixes, trimming off "argc/argv" before we pass things to
> "git hook run", removing a redundant test setup etc.
> 
> I also added a GIT_TEST_FAKE_HOOKS=true for use in the test suite to
> make us support a "test-hook" and a "does-not-exist" hook, we'd
> previously accept those outside the test environment.

I initially wasn't sure this is necessary, and I'm still not entirely
sure - it makes more sense to me to see test-hook and does-not-exist use
essentially
https://lore.kernel.org/git/20210715232603.3415111-8-emilyshaffer@google.com.

Would it make sense for you to absorb that much - "when we're 'git hook
run'ing, don't validate hookname" - into your series to avoid this
test-fake-hooks thing, and also teach the test-helper to use (or not
use) the allow_unknown flag?

[...]

> Range-diff:
>  1:  447d349c73 !  1:  cf4b06bfdf hook: add 'run' subcommand
>     @@ Documentation/git-hook.txt (new)
>     @@ Documentation/githooks.txt: and "0" meaning they were not.
>     -@@ Makefile: LIB_OBJS += hash-lookup.o
>     - LIB_OBJS += hashmap.o
>     - LIB_OBJS += help.o
>     - LIB_OBJS += hex.o
>     -+LIB_OBJS += hook.o
>     - LIB_OBJS += ident.o
>     - LIB_OBJS += json-writer.o
>     - LIB_OBJS += kwset.o

Ah, no longer adding hook.o to the Makefile because it's been added in
one of your splintered-off dependency topics. Ok.

>     @@ builtin/hook.c (new)
>      +#include "strvec.h"
>      +
>      +static const char * const builtin_hook_usage[] = {
>     ++	N_("git hook <command> [...]"),
>     ++	N_("git hook run <hook-name> [-- <hook-args>]"),
>     ++	NULL
>     ++};
>     ++
>     ++static const char * const builtin_hook_run_usage[] = {
>      +	N_("git hook run <hook-name> [-- <hook-args>]"),
>      +	NULL
>      +};
Cool, so now we have a separate 'git hook<enter>' usage and 'git hook
run<enter>' usage.

I was going to complain about the strings being duplicated, but it
appears that other commands that use this pattern - like 'git remote' -
do it the same way as you're doing here. So seems like that's a problem
to ponder later, unrelated to this topic ;)

>     @@ builtin/hook.c (new)
>      +	};
>      +
>      +	argc = parse_options(argc, argv, prefix, run_options,
>     -+			     builtin_hook_usage, PARSE_OPT_KEEP_UNKNOWN | PARSE_OPT_KEEP_DASHDASH);
>     ++			     builtin_hook_run_usage,
>     ++			     PARSE_OPT_KEEP_UNKNOWN | PARSE_OPT_KEEP_DASHDASH);
>      +
>     -+	if (argc > 2) {
>     -+		if (strcmp(argv[2], "--") &&
>     -+		    strcmp(argv[2], "--end-of-options"))
>     ++	if (argc > 1) {
>     ++		if (strcmp(argv[1], "--") &&
>     ++		    strcmp(argv[1], "--end-of-options"))
>      +			/* Having a -- for "run" is mandatory */
>      +			usage_with_options(builtin_hook_usage, run_options);
>      +		/* Add our arguments, start after -- */
>     -+		for (i = 3 ; i < argc; i++)
>     ++		for (i = 2 ; i < argc; i++)

Nice, so the implementation of run_hooks() has one less stray arg to
worry about.

>      +			strvec_push(&opt.args, argv[i]);
>      +	}
>      +
>      +	/* Need to take into account core.hooksPath */
>      +	git_config(git_default_config, NULL);
>      +
>     -+	hook_name = argv[1];
>     ++	/*
>     ++	 * We are not using run_hooks() because we'd like to detect
>     ++	 * missing hooks. Let's find it ourselves and call
>     ++	 * run_found_hooks() instead.
>     ++	 */
>     ++	hook_name = argv[0];

And a more explicit comment. Thanks.

>      +	hook_path = find_hook(hook_name);
>      +	if (!hook_path) {
>      +		error("cannot find a hook named %s", hook_name);
>     @@ builtin/hook.c (new)
>      +	struct option builtin_hook_options[] = {
>      +		OPT_END(),
>      +	};
>     ++	argc = parse_options(argc, argv, NULL, builtin_hook_options,
>     ++			     builtin_hook_usage, PARSE_OPT_STOP_AT_NON_OPTION);
>     ++	if (!argc)
>     ++		usage_with_options(builtin_hook_usage, builtin_hook_options);

Nice - parse_options() removes the 'hook' part of the invocation for us,
so we don't need to carry it around into the subcommands. Thanks.

>      +
>     -+	if (!strcmp(argv[1], "run"))
>     ++	if (!strcmp(argv[0], "run"))
>      +		return run(argc, argv, prefix);
>     -+	usage_with_options(builtin_hook_usage, builtin_hook_options);
>     -+	return 1;
>     ++	else
>     ++		usage_with_options(builtin_hook_usage, builtin_hook_options);
>      +}

Ah, and here we lose the awkward "return something even though
usage_with_options() exits". Ok.

>     + ## hook.c ##
>     + static int known_hook(const char *name)
>     + {
>     + 	const char **p;
>     + 	size_t len = strlen(name);
>     ++	static int test_hooks_ok = -1;
>     ++

Setting aside whether or not I think it's a good idea, this seems
correct - we either allow test hooks for the entire lifetime of the
process (e.g. from the test runner) or enforce hookname validation for
the entire lifetime of the process (e.g. 'git hook run' or normal Git
operation). So it makes sense to cache it in a static. Ok.

>     + 	for (p = hook_name_list; *p; p++) {
>     + 		const char *hook = *p;
>     + 
>     +@@ hook.c: static int known_hook(const char *name)
>     + 			return 1;
>     + 	}
>     + 
>     ++	if (test_hooks_ok == -1)
>     ++		test_hooks_ok = git_env_bool("GIT_TEST_FAKE_HOOKS", 0);
>     ++
>     ++	if (test_hooks_ok &&
>     ++	    (!strcmp(name, "test-hook") ||
>     ++	     !strcmp(name, "does-not-exist")))
>     ++		return 1;
>     ++

As I mentioned above, this still seems too restrictive to me, but I
don't need to rehash it here.

>     @@ hook.c (new)
>      +	struct hook_cb_data *hook_cb = pp_cb;
>      +	struct hook *run_me = hook_cb->run_me;
>      +
>     -+	if (!run_me)
>     -+		BUG("did we not return 1 in notify_hook_finished?");
>     -+

I think the logic here is not quite correct, and I patched it in my
series on top when I enabled multiple hooks.

My understanding of the contract with run_processes_parallel() is this:
 - When you're done assigning tasks, return '0' from pick_next_hook()
 - If you want to stop all task processing because of one task's exit
   state, return '1' from notify_hook_finished()

But you instead are unconditionally returning '1' from
'notify_hook_finished()', and not handling "no more tasks to assign" in
pick_next_hook(). I think it is more correct to return 0 from
'notify_hook_finished()' - "this hook ran correctly, or if it ran
poorly, at least we don't want to prevent anybody else from running, as
that's not our problem". And then in your pick_next_hook(), since you
know you only want to run one hook and you aren't using a list yet, at
the end after you prepare the child_process, you can set hook_cb->run_me
= NULL - "there is no next hook to pick" - and then use "if (!run_me)"
to return 0 to tell run_processes_parallel() that there are no more
tasks to assign.

I  mention it because without noticing your changed flow in this patch,
I wrote a fairly pernicious bug in my rebase on top. By leaving
'notify_hook_finished()' to return 1, but enabling parallelism,
multihooks *appeared* to work - because all the tasks were assigned and
started when running in parallel, even though the first one to finish
said "wait don't start any more tasks please". But when running multiple
hooks with j=1, only the first hook would run, even though the list was
populated correctly - and stepping through pick_next_hook() didn't
indicate why.

I *think* you did it this way to avoid calling pick_next_hook() again with
nothing to iterate over, but since run_processes_parallel() is written
for lists of tasks, I don't think that kludge is the "correct" approach.

>     +@@ hook.h: const char *find_hook(const char *name);
>     -+	/* Number of threads to parallelize across */
>     ++	/*
>     ++	 * Number of threads to parallelize across, currently a stub,
>     ++	 * we use the parallel API for future-proofing, but we always
>     ++	 * have one hook of a given name, so this is always an
>     ++	 * implicit 1 for now.
>     ++	 */
>      +	int jobs;

Makes me wonder whether you really even want to leave it in here. I
guess it makes your call to run_processes_parallel() cleaner. But it'd
also be handy to swap '1' for 'options->jobs' in
https://lore.kernel.org/git/20210715232603.3415111-3-emilyshaffer@google.com
instead, if you'd prefer.

>      +};
>      +
>     @@ hook.h (new)
>      +	.args = STRVEC_INIT, \
>      +}
>      +
>     -+/*
>     -+ * Callback provided to feed_pipe_fn and consume_sideband_fn.
>     -+ */

Why delete the comment? What was it hurting? ;)

>     @@ t/t1800-hook.sh (new)
>     ++test_expect_success 'git hook run: nonexistent hook' '

Ah, thanks for '%s/--/:/'. I know it was a nit, but that was bugging me :)

>      +test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
>      +	mkdir my-hooks &&
>     -+	write_script my-hooks/test-hook <<-EOF &&
>     -+	echo Hook ran >>actual
>     ++	write_script my-hooks/test-hook <<-\EOF &&
>     ++	echo Hook ran $1 >>actual
>      +	EOF
>      +
>      +	cat >expect <<-\EOF &&
>      +	Test hook
>     -+	Hook ran
>     -+	Hook ran
>     -+	Hook ran
>     -+	Hook ran
>     ++	Hook ran one
>     ++	Hook ran two
>     ++	Hook ran three
>     ++	Hook ran four
>      +	EOF
>      +
>      +	# Test various ways of specifying the path. See also
>      +	# t1350-config-hooks-path.sh
>      +	>actual &&
>     -+	git hook run test-hook 2>>actual &&
>     -+	git -c core.hooksPath=my-hooks hook run test-hook 2>>actual &&
>     -+	git -c core.hooksPath=my-hooks/ hook run test-hook 2>>actual &&
>     -+	git -c core.hooksPath="$PWD/my-hooks" hook run test-hook 2>>actual &&
>     -+	git -c core.hooksPath="$PWD/my-hooks/" hook run test-hook 2>>actual &&
>     ++	git hook run test-hook -- ignored 2>>actual &&
>     ++	git -c core.hooksPath=my-hooks hook run test-hook -- one 2>>actual &&
>     ++	git -c core.hooksPath=my-hooks/ hook run test-hook -- two 2>>actual &&
>     ++	git -c core.hooksPath="$PWD/my-hooks" hook run test-hook -- three 2>>actual &&
>     ++	git -c core.hooksPath="$PWD/my-hooks/" hook run test-hook -- four 2>>actual &&

Thanks - this will be much easier to debug when someone (me, probably)
breaks one of these by accident ;)

>      +	test_cmp expect actual
>      +'
>      +
>     -+test_expect_success 'set up a pre-commit hook in core.hooksPath' '
>     -+	>actual &&
>     -+	mkdir -p .git/custom-hooks .git/hooks &&
>     -+	write_script .git/custom-hooks/pre-commit <<-\EOF &&
>     -+	echo CUSTOM >>actual
>     -+	EOF
>     -+	write_script .git/hooks/pre-commit <<-\EOF
>     -+	echo NORMAL >>actual
>     -+	EOF
>     -+'

Hm, is this removed because we feel like the test-hook test above gives
us enough coverage? Ok.

>  9:  8d8b2d2645 !  7:  12347d901b git hook run: add an --ignore-missing flag
>       ## t/t1800-hook.sh ##
>     +-test_expect_success 'git hook run: basic' '
>     ++test_expect_success 'git hook run: nonexistent hook with --ignore-missing' '
>      +	git hook run --ignore-missing does-not-exist 2>stderr.actual &&
>      +	test_must_be_empty stderr.actual
>      +'
>      +
>     - test_expect_success 'git hook run -- basic' '
>     ++test_expect_success 'git hook run -- basic' '

Ah, I see the range-diff caught s/--/:/ earlier but that step was missed
in patch 9, which made this bit of the diff a little confused.

> 11:  aa970a8175 !  9:  246a82b55b git-p4: use 'git hook' to run hooks
>     @@ Commit message
>          Python, we can directly call 'git hook run'. We emulate the existence
>          check with the --ignore-missing flag.
>      
>     +    As this is the last hook execution in git.git to not go through "git
>     +    hook run" or the hook.[ch] library we can now be absolutely sure that
>     +    our assertion in hook.c that only hooks known by the generated (from
>     +    githooks(5)) hook-list.h are permitted.
>     +

\o/

> 17:  c4f60db606 ! 15:  b7c0ee9719 hook: support passing stdin to hooks
>       ## builtin/hook.c ##
>       static const char * const builtin_hook_usage[] = {
>     + 	N_("git hook <command> [...]"),
>      -	N_("git hook run <hook-name> [-- <hook-args>]"),
>     ++	N_("git hook run [<args>] <hook-name> [-- <hook-args>]"),
>     + 	NULL
>     + };
>     + 
>     + static const char * const builtin_hook_run_usage[] = {
>     + 	N_("git hook run <hook-name> [-- <hook-args>]"),
>      +	N_("git hook run [--to-stdin=<path>] <hook-name> [-- <hook-args>]"),

Nice, and here the strings in builtin_hook_usage and
builtin_hook_run_usage start to differ, which makes my earlier concern
entirely moot. Thanks.

--

Aside from the questions about a) this test hook envvar and b) specific
mechanics of terminating run_processes_parallel(), this series looks
good to me.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH v4 00/36] Run hooks via "git run hook" & hook library
  2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
                             ` (28 preceding siblings ...)
  2021-07-28 20:39           ` [PATCH 00/27] Base for "config-based-hooks" Emily Shaffer
@ 2021-08-03 19:38           ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 01/36] Makefile: mark "check" target as .PHONY Ævar Arnfjörð Bjarmason
                               ` (39 more replies)
  29 siblings, 40 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

This is a v4 re-roll of the "Base for "config-based-hooks" topic.

It's grown from 27 to 36 patches mainly because I re-folded the two
topics it depended on into it, i.e.:

    https://lore.kernel.org/git/cover-0.3-0000000000-20210629T190137Z-avarab@gmail.com/
    https://lore.kernel.org/git/cover-0.3-0000000000-20210629T183325Z-avarab@gmail.com/

I think those have long since reached a level of stability suitable
for being merged down, but since Junio didn't pick them up
separately[1][2] there wasn't much point in keeping them split-up.

This re-roll mostly addresses Emily's comments on the v3:
https://lore.kernel.org/git/YQHAasrmcbdiCDQF@google.com/

I.e.:

 * I did not change the part where I die on hooks that aren't known to
   git itself. I think it makes sense to keep this topic purely for
   bug-for-bug compatibility with existing behavior, and to flip the
   RUN_SETUP_GENTLY flag in her follow-up feature topic.

 * The (ab)use of the run_processes_parallel() API should be gone. I
   just misunderstood how it worked.

 * The v3 would segfault on a plain "git hook run". I've changed
   builtin/hook.c to use the same pattern for subcommands as
   builtin/commit-graph.c et al. This also makes the usage output more
   consistent.

 * The stub "jobs" member of the struct is now gone, and it's
   hardcoded to 1. Emily's feature topic can add it back (per her
   suggestion).

Other updates:

 * In the base topic the s/Signed-off-by/Reviewed-by/g from René
   change that Junio applied locally has been folded in.

 * Almost all the callers were just "one-shot" callers, I introduced a
   new run_hooks_oneshot() function for those, which gets rid of the
   verbosity around memory management, see e.g. the "builtin/gc.c" in
   the range-diff below. That run_hooks_oneshot() can also take a NULL
   set of options.

 * We'd leak memory from "my_hook.feed_pipe_cb_data" in hook.c, we now
   free() it.

 * Some s/STRING_LIST_INIT_DUP/STRING_LIST_INIT_NODUP &
   strbuf_detach()/ changes in transport.c and reference-transaction
   to avoid needless verbosity around memory management.

 * A new 'hook.c users: use "hook_exists()" insted of "find_hook()"'
   patch in what was previously one of the base topics, makes
   subsequent changes smaller.

1. https://lore.kernel.org/git/87sg00qfbp.fsf@evledraar.gmail.com/
2. https://lore.kernel.org/git/87a6mevkrx.fsf@evledraar.gmail.com/

Emily Shaffer (26):
  hook.c: add a hook_exists() wrapper and use it in bugreport.c
  hook: add 'run' subcommand
  gc: use hook library for pre-auto-gc hook
  rebase: convert pre-rebase to use hook.h
  am: convert applypatch to use hook.h
  hooks: convert 'post-checkout' hook to hook library
  merge: convert post-merge to use hook.h
  send-email: use 'git hook run' for 'sendemail-validate'
  git-p4: use 'git hook' to run hooks
  commit: convert {pre-commit,prepare-commit-msg} hook to hook.h
  read-cache: convert post-index-change to use hook.h
  receive-pack: convert push-to-checkout hook to hook.h
  run-command: remove old run_hook_{le,ve}() hook API
  run-command: allow stdin for run_processes_parallel
  hook: support passing stdin to hooks
  am: convert 'post-rewrite' hook to hook.h
  run-command: add stdin callback for parallelization
  hook: provide stdin by string_list or callback
  hook: convert 'post-rewrite' hook in sequencer.c to hook.h
  transport: convert pre-push hook to hook.h
  reference-transaction: use hook.h to run hooks
  run-command: allow capturing of collated output
  hooks: allow callers to capture output
  receive-pack: convert 'update' hook to hook.h
  post-update: use hook.h library
  receive-pack: convert receive hooks to hook.h

Ævar Arnfjörð Bjarmason (10):
  Makefile: mark "check" target as .PHONY
  Makefile: stop hardcoding {command,config}-list.h
  Makefile: remove an out-of-date comment
  hook.[ch]: move find_hook() to this new library
  hook.c users: use "hook_exists()" insted of "find_hook()"
  hook-list.h: add a generated list of hooks, like config-list.h
  git hook run: add an --ignore-missing flag
  hook tests: test for exact "pre-push" hook input
  hook tests: use a modern style for "pre-push" tests
  hooks: fix a TOCTOU in "did we run a hook?" heuristic

 .gitignore                          |   2 +
 Documentation/git-hook.txt          |  51 +++++
 Documentation/githooks.txt          |   4 +
 Makefile                            |  26 ++-
 builtin.h                           |   1 +
 builtin/am.c                        |  29 +--
 builtin/bugreport.c                 |  46 +----
 builtin/checkout.c                  |  14 +-
 builtin/clone.c                     |   6 +-
 builtin/commit.c                    |  19 +-
 builtin/fetch.c                     |   1 +
 builtin/gc.c                        |   3 +-
 builtin/hook.c                      |  92 +++++++++
 builtin/merge.c                     |  21 +-
 builtin/rebase.c                    |   6 +-
 builtin/receive-pack.c              | 285 +++++++++++++---------------
 builtin/submodule--helper.c         |   2 +-
 builtin/worktree.c                  |  29 ++-
 command-list.txt                    |   1 +
 commit.c                            |  16 +-
 commit.h                            |   3 +-
 compat/vcbuild/README               |   2 +-
 config.mak.uname                    |   6 +-
 contrib/buildsystems/CMakeLists.txt |   7 +
 generate-hooklist.sh                |  18 ++
 git-p4.py                           |  72 +------
 git-send-email.perl                 |  20 +-
 git.c                               |   1 +
 hook.c                              | 246 ++++++++++++++++++++++++
 hook.h                              | 126 ++++++++++++
 read-cache.c                        |  11 +-
 refs.c                              |  41 ++--
 reset.c                             |  14 +-
 run-command.c                       | 157 +++++++--------
 run-command.h                       |  55 +++---
 sequencer.c                         |  86 ++++-----
 submodule.c                         |   1 +
 t/helper/test-run-command.c         |  46 ++++-
 t/t0061-run-command.sh              |  37 ++++
 t/t1800-hook.sh                     | 156 +++++++++++++++
 t/t5571-pre-push-hook.sh            |  94 +++++----
 t/t9001-send-email.sh               |   4 +-
 transport.c                         |  57 ++----
 43 files changed, 1303 insertions(+), 611 deletions(-)
 create mode 100644 Documentation/git-hook.txt
 create mode 100644 builtin/hook.c
 create mode 100755 generate-hooklist.sh
 create mode 100644 hook.c
 create mode 100644 hook.h
 create mode 100755 t/t1800-hook.sh

Range-diff against v3:
 1:  27c94247f87 =  1:  81fe1ed90d5 Makefile: mark "check" target as .PHONY
 2:  6e164edb0b0 !  2:  0f749530777 Makefile: stop hardcoding {command,config}-list.h
    @@ Commit message
         added in 029bac01a8 (Makefile: add {program,xdiff,test,git,fuzz}-objs
         & objects targets, 2021-02-23).
     
    +    A subsequent commit will add a new generated hook-list.h. By doing
    +    this refactoring we'll only need to add the new file to the
    +    GENERATED_H variable, not EXCEPT_HDRS, the vcbuild/README etc.
    +
         I have not tested the Windows-specific change in config.mak.uname
         being made here, but we use other variables from the Makefile in the
         same block, and the GENERATED_H is fully defined before we include
         config.mak.uname.
     
         Hardcoding command-list.h there seems to have been a case of
    -    copy/paste programming in dce7d29551 (msvc: support building Git using
    -    MS Visual C++, 2019-06-25). The config-list.h was added later in
    -    709df95b78 (help: move list_config_help to builtin/help, 2020-04-16).
    +    copy/paste programming in 976aaedca0 (msvc: add a Makefile target to
    +    pre-generate the Visual Studio solution, 2019-07-29). The
    +    config-list.h was added later in 709df95b78 (help: move
    +    list_config_help to builtin/help, 2020-04-16).
     
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
 3:  ddae86802e2 !  3:  644b31fe281 Makefile: remove an out-of-date comment
    @@ Commit message
         The rest of it was also somewhere between inaccurate and outdated,
         since as of b8ba629264 (Makefile: fold MISC_H into LIB_H, 2012-06-20)
         it's not followed by a list of header files, that got moved earlier in
    -    the file into LIB_H in b8ba629264 (Makefile: fold MISC_H into LIB_H,
    -    2012-06-20).
    +    the file into LIB_H in 60d24dd255 (Makefile: fold XDIFF_H and VCSSVN_H
    +    into LIB_H, 2012-07-06).
     
         Let's just remove it entirely, to the extent that we have anything
         useful to say here the comment on the
    @@ Makefile: ifneq ($(dep_files_present),)
     -# Dependencies on automatically generated headers such as command-list.h
     -# should _not_ be included here, since they are necessary even when
     -# building an object for the first time.
    - 
    +-
      $(OBJECTS): $(LIB_H) $(GENERATED_H)
      endif
    + 
 4:  58c37e4f06e =  4:  89c4d44b0c3 hook.[ch]: move find_hook() to this new library
 5:  0cf7e078ef4 =  5:  3514e0c0251 hook.c: add a hook_exists() wrapper and use it in bugreport.c
 -:  ----------- >  6:  d5ef40f77dc hook.c users: use "hook_exists()" insted of "find_hook()"
 6:  f343fc7ae66 !  7:  4cfd72722c1 hook-list.h: add a generated list of hooks, like config-list.h
    @@ Commit message
          - 976aaedca0 (msvc: add a Makefile target to pre-generate the Visual
            Studio solution, 2019-07-29)
     
    +    The LC_ALL=C is needed because at least in my locale the dash ("-") is
    +    ignored for the purposes of sorting, which results in a different
    +    order. I'm not aware of anything in git that has a hard dependency on
    +    the order, but e.g. the bugreport output would end up using whatever
    +    locale was in effect when git was compiled.
    +
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    +    Helped-by: René Scharfe <l.s.r@web.de>
     
      ## .gitignore ##
     @@
    @@ Makefile: command-list.h: $(wildcard Documentation/git*.txt)
      		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
      		command-list.txt >$@+ && mv $@+ $@
      
    -+hook-list.h: generate-hooklist.sh
    -+hook-list.h: Documentation/githooks.txt
    ++hook-list.h: generate-hooklist.sh Documentation/githooks.txt
     +	$(QUIET_GEN)$(SHELL_PATH) ./generate-hooklist.sh \
     +		>$@+ && mv $@+ $@
     +
    @@ contrib/buildsystems/CMakeLists.txt: if(NOT EXISTS ${CMAKE_BINARY_DIR}/config-li
      ## generate-hooklist.sh (new) ##
     @@
     +#!/bin/sh
    ++#
    ++# Usage: ./generate-hooklist.sh >hook-list.h
     +
    -+echo "/* Automatically generated by generate-hooklist.sh */"
    ++cat <<EOF
    ++/* Automatically generated by generate-hooklist.sh */
     +
    -+print_hook_list () {
    -+	cat <<EOF
     +static const char *hook_name_list[] = {
     +EOF
    -+	perl -ne '
    -+		chomp;
    -+		@l[$.] = $_;
    -+		push @h => $l[$. - 1] if /^~~~+$/s;
    -+		END {
    -+			print qq[\t"$_",\n] for sort @h;
    -+		}
    -+	' <Documentation/githooks.txt
    -+	cat <<EOF
    ++
    ++sed -n -e '/^~~~~*$/ {x; s/^.*$/	"&",/; p;}; x' \
    ++	<Documentation/githooks.txt |
    ++	LC_ALL=C sort
    ++
    ++cat <<EOF
     +	NULL,
     +};
     +EOF
    -+}
    -+
    -+echo
    -+print_hook_list
     
      ## hook.c ##
     @@
 7:  cf4b06bfdf8 !  8:  7cb4a4cb69e hook: add 'run' subcommand
    @@ Commit message
         let's start with the bare minimum required to support our simplest
         hooks.
     
    +    In terms of implementation the usage_with_options() and "goto usage"
    +    pattern here mirrors that of
    +    builtin/{commit-graph,multi-pack-index}.c.
    +
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
    @@ builtin/hook.c (new)
     +#include "strbuf.h"
     +#include "strvec.h"
     +
    ++#define BUILTIN_HOOK_RUN_USAGE \
    ++	N_("git hook run <hook-name> [-- <hook-args>]")
    ++
     +static const char * const builtin_hook_usage[] = {
    -+	N_("git hook <command> [...]"),
    -+	N_("git hook run <hook-name> [-- <hook-args>]"),
    ++	BUILTIN_HOOK_RUN_USAGE,
     +	NULL
     +};
     +
     +static const char * const builtin_hook_run_usage[] = {
    -+	N_("git hook run <hook-name> [-- <hook-args>]"),
    ++	BUILTIN_HOOK_RUN_USAGE,
     +	NULL
     +};
     +
    @@ builtin/hook.c (new)
     +{
     +	int i;
     +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	int rc = 0;
     +	const char *hook_name;
     +	const char *hook_path;
    -+
     +	struct option run_options[] = {
     +		OPT_END(),
     +	};
    ++	int ret;
     +
     +	argc = parse_options(argc, argv, prefix, run_options,
     +			     builtin_hook_run_usage,
    -+			     PARSE_OPT_KEEP_UNKNOWN | PARSE_OPT_KEEP_DASHDASH);
    -+
    -+	if (argc > 1) {
    -+		if (strcmp(argv[1], "--") &&
    -+		    strcmp(argv[1], "--end-of-options"))
    -+			/* Having a -- for "run" is mandatory */
    -+			usage_with_options(builtin_hook_usage, run_options);
    -+		/* Add our arguments, start after -- */
    -+		for (i = 2 ; i < argc; i++)
    -+			strvec_push(&opt.args, argv[i]);
    -+	}
    ++			     PARSE_OPT_KEEP_DASHDASH);
     +
    -+	/* Need to take into account core.hooksPath */
    -+	git_config(git_default_config, NULL);
    ++	if (!argc)
    ++		goto usage;
     +
     +	/*
    -+	 * We are not using run_hooks() because we'd like to detect
    -+	 * missing hooks. Let's find it ourselves and call
    -+	 * run_found_hooks() instead.
    ++	 * Having a -- for "run" when providing <hook-args> is
    ++	 * mandatory.
     +	 */
    ++	if (argc > 1 && strcmp(argv[1], "--") &&
    ++	    strcmp(argv[1], "--end-of-options"))
    ++		goto usage;
    ++
    ++	/* Add our arguments, start after -- */
    ++	for (i = 2 ; i < argc; i++)
    ++		strvec_push(&opt.args, argv[i]);
    ++
    ++	/* Need to take into account core.hooksPath */
    ++	git_config(git_default_config, NULL);
    ++
     +	hook_name = argv[0];
     +	hook_path = find_hook(hook_name);
     +	if (!hook_path) {
     +		error("cannot find a hook named %s", hook_name);
     +		return 1;
     +	}
    -+	rc = run_found_hooks(hook_name, hook_path, &opt);
     +
    ++	ret = run_hooks(hook_name, hook_path, &opt);
     +	run_hooks_opt_clear(&opt);
    -+
    -+	return rc;
    ++	return ret;
    ++usage:
    ++	usage_with_options(builtin_hook_run_usage, run_options);
     +}
     +
     +int cmd_hook(int argc, const char **argv, const char *prefix)
    @@ builtin/hook.c (new)
     +	struct option builtin_hook_options[] = {
     +		OPT_END(),
     +	};
    ++
     +	argc = parse_options(argc, argv, NULL, builtin_hook_options,
     +			     builtin_hook_usage, PARSE_OPT_STOP_AT_NON_OPTION);
     +	if (!argc)
    -+		usage_with_options(builtin_hook_usage, builtin_hook_options);
    ++		goto usage;
     +
     +	if (!strcmp(argv[0], "run"))
     +		return run(argc, argv, prefix);
    -+	else
    -+		usage_with_options(builtin_hook_usage, builtin_hook_options);
    ++
    ++usage:
    ++	usage_with_options(builtin_hook_usage, builtin_hook_options);
     +}
     
      ## command-list.txt ##
    @@ hook.c: int hook_exists(const char *name)
     +	struct hook_cb_data *hook_cb = pp_cb;
     +	struct hook *run_me = hook_cb->run_me;
     +
    ++	if (!run_me)
    ++		return 0;
    ++
     +	cp->no_stdin = 1;
     +	cp->env = hook_cb->options->env.v;
     +	cp->stdout_to_stderr = 1;
    @@ hook.c: int hook_exists(const char *name)
     +	/* Provide context for errors if necessary */
     +	*pp_task_cb = run_me;
     +
    ++	/*
    ++	 * This pick_next_hook() will be called again, we're only
    ++	 * running one hook, so indicate that no more work will be
    ++	 * done.
    ++	 */
    ++	hook_cb->run_me = NULL;
    ++
     +	return 1;
     +}
     +
    @@ hook.c: int hook_exists(const char *name)
     +
     +	hook_cb->rc |= result;
     +
    -+	return 1;
    ++	return 0;
     +}
     +
    -+int run_found_hooks(const char *hook_name, const char *hook_path,
    -+		    struct run_hooks_opt *options)
    ++int run_hooks(const char *hook_name, const char *hook_path,
    ++	      struct run_hooks_opt *options)
     +{
     +	struct hook my_hook = {
     +		.hook_path = hook_path,
    @@ hook.c: int hook_exists(const char *name)
     +		.hook_name = hook_name,
     +		.options = options,
     +	};
    -+	cb_data.run_me = &my_hook;
    ++	int jobs = 1;
    ++
    ++	if (!options)
    ++		BUG("a struct run_hooks_opt must be provided to run_hooks");
     +
    -+	if (options->jobs != 1)
    -+		BUG("we do not handle %d or any other != 1 job number yet", options->jobs);
    ++	cb_data.run_me = &my_hook;
     +
    -+	run_processes_parallel_tr2(options->jobs,
    ++	run_processes_parallel_tr2(jobs,
     +				   pick_next_hook,
     +				   notify_start_failure,
     +				   notify_hook_finished,
    @@ hook.c: int hook_exists(const char *name)
     +				   hook_name);
     +
     +	return cb_data.rc;
    -+}
    -+
    -+int run_hooks(const char *hook_name, struct run_hooks_opt *options)
    -+{
    -+	const char *hook_path;
    -+	int ret;
    -+	if (!options)
    -+		BUG("a struct run_hooks_opt must be provided to run_hooks");
    -+
    -+	hook_path = find_hook(hook_name);
    -+
    -+	/*
    -+	 * If you need to act on a missing hook, use run_found_hooks()
    -+	 * instead
    -+	 */
    -+	if (!hook_path)
    -+		return 0;
    -+
    -+	ret = run_found_hooks(hook_name, hook_path, options);
    -+	return ret;
     +}
     
      ## hook.h ##
    @@ hook.h: const char *find_hook(const char *name);
     +
     +	/* Args to be passed to each hook */
     +	struct strvec args;
    -+
    -+	/*
    -+	 * Number of threads to parallelize across, currently a stub,
    -+	 * we use the parallel API for future-proofing, but we always
    -+	 * have one hook of a given name, so this is always an
    -+	 * implicit 1 for now.
    -+	 */
    -+	int jobs;
     +};
     +
     +#define RUN_HOOKS_OPT_INIT { \
    -+	.jobs = 1, \
     +	.env = STRVEC_INIT, \
     +	.args = STRVEC_INIT, \
     +}
     +
    ++/*
    ++ * Callback provided to feed_pipe_fn and consume_sideband_fn.
    ++ */
     +struct hook_cb_data {
     +	/* rc reflects the cumulative failure state */
     +	int rc;
    @@ hook.h: const char *find_hook(const char *name);
     +
     +void run_hooks_opt_clear(struct run_hooks_opt *o);
     +
    -+/*
    -+ * Calls find_hook(hookname) and runs the hooks (if any) with
    -+ * run_found_hooks().
    -+ */
    -+int run_hooks(const char *hook_name, struct run_hooks_opt *options);
    -+
    -+/*
    -+ * Takes an already resolved hook and runs it. Internally the simpler
    -+ * run_hooks() will call this.
    ++/**
    ++ * Takes an already resolved hook found via find_hook() and runs
    ++ * it. Does not call run_hooks_opt_clear() for you.
     + */
    -+int run_found_hooks(const char *hookname, const char *hook_path,
    -+		    struct run_hooks_opt *options);
    ++int run_hooks(const char *hookname, const char *hook_path,
    ++	      struct run_hooks_opt *options);
      #endif
     
      ## t/t1800-hook.sh (new) ##
    @@ t/t1800-hook.sh (new)
     +
     +test_expect_success 'git hook usage' '
     +	test_expect_code 129 git hook &&
    -+	test_expect_code 129 git hook -h &&
    -+	test_expect_code 129 git hook run -h
    ++	test_expect_code 129 git hook run &&
    ++	test_expect_code 129 git hook run -h &&
    ++	test_expect_code 129 git hook run --unknown 2>err &&
    ++	grep "unknown option" err
     +'
     +
     +test_expect_success 'setup GIT_TEST_FAKE_HOOKS=true to permit "test-hook" and "does-not-exist" names"' '
 8:  7209f73f281 !  9:  2b8500aa675 gc: use hook library for pre-auto-gc hook
    @@ Metadata
      ## Commit message ##
         gc: use hook library for pre-auto-gc hook
     
    -    Using the hook.h library instead of the run-command.h library to run
    -    pre-auto-gc means that those hooks can be set up in config files, as
    -    well as in the hookdir. pre-auto-gc is called only from builtin/gc.c.
    +    Move the pre-auto-gc hook away from run-command.h to and over to the
    +    new hook.h library.
    +
    +    To do this introduce a simple run_hooks_oneshot() wrapper, we'll be
    +    using it extensively for these simple cases of wanting to run a single
    +    hook under a given name, and having it free the memory we allocate for
    +    us.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    @@ builtin/gc.c
      
      #define FAILED_RUN "failed to run %s"
      
    -@@ builtin/gc.c: static void add_repack_incremental_option(void)
    - 
    - static int need_to_gc(void)
    - {
    -+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
    -+
    - 	/*
    - 	 * Setting gc.auto to 0 or negative can disable the
    - 	 * automatic gc.
     @@ builtin/gc.c: static int need_to_gc(void)
      	else
      		return 0;
      
     -	if (run_hook_le(NULL, "pre-auto-gc", NULL))
    -+	if (run_hooks("pre-auto-gc", &hook_opt)) {
    -+		run_hooks_opt_clear(&hook_opt);
    ++	if (run_hooks_oneshot("pre-auto-gc", NULL))
      		return 0;
    -+	}
    -+	run_hooks_opt_clear(&hook_opt);
      	return 1;
      }
    +
    + ## hook.c ##
    +@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
      
    + 	return cb_data.rc;
    + }
    ++
    ++int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
    ++{
    ++	const char *hook_path;
    ++	int ret;
    ++	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
    ++
    ++	if (!options)
    ++		options = &hook_opt_scratch;
    ++
    ++	hook_path = find_hook(hook_name);
    ++	if (!hook_path) {
    ++		ret = 0;
    ++		goto cleanup;
    ++	}
    ++
    ++	ret = run_hooks(hook_name, hook_path, options);
    ++cleanup:
    ++	run_hooks_opt_clear(options);
    ++	return ret;
    ++}
    +
    + ## hook.h ##
    +@@ hook.h: void run_hooks_opt_clear(struct run_hooks_opt *o);
    + /**
    +  * Takes an already resolved hook found via find_hook() and runs
    +  * it. Does not call run_hooks_opt_clear() for you.
    ++ *
    ++ * See run_hooks_oneshot() for the simpler one-shot API.
    +  */
    + int run_hooks(const char *hookname, const char *hook_path,
    + 	      struct run_hooks_opt *options);
    ++
    ++/**
    ++ * Calls find_hook() on your "hook_name" and runs the hooks (if any)
    ++ * with run_hooks().
    ++ *
    ++ * If "options" is provided calls run_hooks_opt_clear() on it for
    ++ * you. If "options" is NULL a scratch one will be provided for you
    ++ * before calling run_hooks().
    ++ */
    ++int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
    ++
    + #endif
 9:  e9a1e7cf61e ! 10:  3ee55d2c10f rebase: teach pre-rebase to use hook.h
    @@ Metadata
     Author: Emily Shaffer <emilyshaffer@google.com>
     
      ## Commit message ##
    -    rebase: teach pre-rebase to use hook.h
    +    rebase: convert pre-rebase to use hook.h
     
         Move the pre-rebase hook away from run-command.h to and over to the
         new hook.h library.
    @@ builtin/rebase.c: int cmd_rebase(int argc, const char **argv, const char *prefix
      	if (!ok_to_skip_pre_rebase &&
     -	    run_hook_le(NULL, "pre-rebase", options.upstream_arg,
     -			argc ? argv[0] : NULL, NULL))
    -+	    run_hooks("pre-rebase", &hook_opt)) {
    -+		run_hooks_opt_clear(&hook_opt);
    ++	    run_hooks_oneshot("pre-rebase", &hook_opt))
      		die(_("The pre-rebase hook refused to rebase."));
    -+	}
    -+	run_hooks_opt_clear(&hook_opt);
      
      	if (options.flags & REBASE_DIFFSTAT) {
    - 		struct diff_options opts;
10:  1d087269303 ! 11:  050f20d14f0 am: convert applypatch hooks to use config
    @@ Metadata
     Author: Emily Shaffer <emilyshaffer@google.com>
     
      ## Commit message ##
    -    am: convert applypatch hooks to use config
    +    am: convert applypatch to use hook.h
     
         Teach pre-applypatch, post-applypatch, and applypatch-msg to use the
         hook.h library instead of the run-command.h library.
    @@ builtin/am.c: static void am_destroy(const struct am_state *state)
      	assert(state->msg);
     -	ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL);
     +	strvec_push(&opt.args, am_path(state, "final-commit"));
    -+	ret = run_hooks("applypatch-msg", &opt);
    -+	run_hooks_opt_clear(&opt);
    ++	ret = run_hooks_oneshot("applypatch-msg", &opt);
      
      	if (!ret) {
      		FREE_AND_NULL(state->msg);
     @@ builtin/am.c: static void do_commit(const struct am_state *state)
    - 	struct commit_list *parents = NULL;
      	const char *reflog_msg, *author, *committer = NULL;
      	struct strbuf sb = STRBUF_INIT;
    -+	struct run_hooks_opt hook_opt_pre = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt hook_opt_post = RUN_HOOKS_OPT_INIT;
      
     -	if (run_hook_le(NULL, "pre-applypatch", NULL))
    -+	if (run_hooks("pre-applypatch", &hook_opt_pre)) {
    -+		run_hooks_opt_clear(&hook_opt_pre);
    ++	if (run_hooks_oneshot("pre-applypatch", NULL))
      		exit(1);
    -+	}
      
      	if (write_cache_as_tree(&tree, 0, NULL))
    - 		die(_("git write-tree failed to write a tree"));
     @@ builtin/am.c: static void do_commit(const struct am_state *state)
      		fclose(fp);
      	}
      
     -	run_hook_le(NULL, "post-applypatch", NULL);
    -+	run_hooks("post-applypatch", &hook_opt_post);
    ++	run_hooks_oneshot("post-applypatch", NULL);
      
    -+	run_hooks_opt_clear(&hook_opt_pre);
    -+	run_hooks_opt_clear(&hook_opt_post);
      	strbuf_release(&sb);
      }
    - 
11:  32eec5dc2f0 ! 12:  ac875d284da hooks: convert 'post-checkout' hook to hook library
    @@ builtin/checkout.c: struct branch_info {
     -			   oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()),
     -			   changed ? "1" : "0", NULL);
     +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	int rc;
     +
      	/* "new_commit" can be NULL when checking out from the index before
      	   a commit exists. */
    @@ builtin/checkout.c: struct branch_info {
     +		     oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()),
     +		     changed ? "1" : "0",
     +		     NULL);
    -+	rc = run_hooks("post-checkout", &opt);
    -+	run_hooks_opt_clear(&opt);
    -+	return rc;
    ++	return run_hooks_oneshot("post-checkout", &opt);
      }
      
      static int update_some(const struct object_id *oid, struct strbuf *base,
    @@ builtin/clone.c: static int checkout(int submodule_progress)
     -	err |= run_hook_le(NULL, "post-checkout", oid_to_hex(null_oid()),
     -			   oid_to_hex(&oid), "1", NULL);
     +	strvec_pushl(&hook_opt.args, oid_to_hex(null_oid()), oid_to_hex(&oid), "1", NULL);
    -+	err |= run_hooks("post-checkout", &hook_opt);
    -+	run_hooks_opt_clear(&hook_opt);
    ++	err |= run_hooks_oneshot("post-checkout", &hook_opt);
      
      	if (!err && (option_recurse_submodules.nr > 0)) {
      		struct strvec args = STRVEC_INIT;
    @@ builtin/worktree.c: static int add_worktree(const char *path, const char *refnam
     +		opt.dir = path;
     +		opt.absolute_path = 1;
     +
    -+		ret = run_hooks("post-checkout", &opt);
    -+
    -+		run_hooks_opt_clear(&opt);
    ++		ret = run_hooks_oneshot("post-checkout", &opt);
      	}
      
      	strvec_clear(&child_env);
    @@ hook.c: static int pick_next_hook(struct child_process *cp,
      	/* add command */
      	strvec_push(&cp->args, run_me->hook_path);
     @@ hook.c: static int notify_hook_finished(int result,
    - int run_found_hooks(const char *hook_name, const char *hook_path,
    - 		    struct run_hooks_opt *options)
    + int run_hooks(const char *hook_name, const char *hook_path,
    + 	      struct run_hooks_opt *options)
      {
     +	struct strbuf abs_path = STRBUF_INIT;
      	struct hook my_hook = {
      		.hook_path = hook_path,
      	};
    -@@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
    - 		.hook_name = hook_name,
    - 		.options = options,
    - 	};
    +@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
    + 	if (!options)
    + 		BUG("a struct run_hooks_opt must be provided to run_hooks");
    + 
     +	if (options->absolute_path) {
     +		strbuf_add_absolute_path(&abs_path, hook_path);
     +		my_hook.hook_path = abs_path.buf;
     +	}
      	cb_data.run_me = &my_hook;
      
    - 	if (options->jobs != 1)
    -@@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
    - 				   &cb_data,
    + 	run_processes_parallel_tr2(jobs,
    +@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
      				   "hook",
      				   hook_name);
    + 
     +	if (options->absolute_path)
     +		strbuf_release(&abs_path);
    - 
    ++
      	return cb_data.rc;
      }
    + 
     
      ## hook.h ##
     @@ hook.h: struct run_hooks_opt
    - 	 * implicit 1 for now.
    - 	 */
    - 	int jobs;
    + 
    + 	/* Args to be passed to each hook */
    + 	struct strvec args;
     +
     +	/* Resolve and run the "absolute_path(hook)" instead of
     +	 * "hook". Used for "git worktree" hooks
    @@ hook.h: struct run_hooks_opt
     +
     +	/* Path to initial working directory for subprocess */
     +	const char *dir;
    -+
      };
      
      #define RUN_HOOKS_OPT_INIT { \
     
      ## read-cache.c ##
     @@
    - #include "progress.h"
      #include "sparse-index.h"
      #include "csum-file.h"
    + #include "promisor-remote.h"
     +#include "hook.h"
      
      /* Mask for the name length in ce_flags in the on-disk index */
    @@ reset.c: int reset_head(struct repository *r, struct object_id *oid, const char
     +			     oid_to_hex(oid),
     +			     "1",
     +			     NULL);
    -+		run_hooks("post-checkout", &opt);
    -+		run_hooks_opt_clear(&opt);
    ++		run_hooks_oneshot("post-checkout", &opt);
     +	}
      
      leave_reset_head:
12:  e9fa3f67593 ! 13:  69763bc2255 merge: use config-based hooks for post-merge hook
    @@ Metadata
     Author: Emily Shaffer <emilyshaffer@google.com>
     
      ## Commit message ##
    -    merge: use config-based hooks for post-merge hook
    +    merge: convert post-merge to use hook.h
     
    -    Teach post-merge to use the hook.h library instead of the run-command.h
    -    library to run hooks. This means that post-merge hooks can come from the
    -    config as well as from the hookdir. post-merge is invoked only from
    -    builtin/merge.c.
    +    Teach post-merge to use the hook.h library instead of the
    +    run-command.h library to run hooks.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    @@ builtin/merge.c: static void finish(struct commit *head_commit,
      	/* Run a post-merge hook */
     -	run_hook_le(NULL, "post-merge", squash ? "1" : "0", NULL);
     +	strvec_push(&opt.args, squash ? "1" : "0");
    -+	run_hooks("post-merge", &opt);
    -+	run_hooks_opt_clear(&opt);
    ++	run_hooks_oneshot("post-merge", &opt);
      
      	apply_autostash(git_path_merge_autostash(the_repository));
      	strbuf_release(&reflog_message);
    -@@ builtin/merge.c: static void prepare_to_commit(struct commit_list *remoteheads)
    - 	 * and write it out as a tree.  We must do this before we invoke
    - 	 * the editor and after we invoke run_status above.
    - 	 */
    --	if (find_hook("pre-merge-commit"))
    -+	if (hook_exists("pre-merge-commit"))
    - 		discard_cache();
    - 	read_cache_from(index_file);
    - 	strbuf_addbuf(&msg, &merge_msg);
13:  12347d901bb ! 14:  2ca1ca1b8e4 git hook run: add an --ignore-missing flag
    @@ Documentation/git-hook.txt: optional `--` (or `--end-of-options`, see linkgit:gi
      linkgit:githooks[5]
     
      ## builtin/hook.c ##
    +@@
    + #include "strvec.h"
    + 
    + #define BUILTIN_HOOK_RUN_USAGE \
    +-	N_("git hook run <hook-name> [-- <hook-args>]")
    ++	N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")
    + 
    + static const char * const builtin_hook_usage[] = {
    + 	BUILTIN_HOOK_RUN_USAGE,
     @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
    + {
      	int i;
      	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    - 	int rc = 0;
     +	int ignore_missing = 0;
      	const char *hook_name;
      	const char *hook_path;
    - 
      	struct option run_options[] = {
     +		OPT_BOOL(0, "ignore-missing", &ignore_missing,
     +			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
      		OPT_END(),
      	};
    - 
    + 	int ret;
     @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
    - 	/*
    - 	 * We are not using run_hooks() because we'd like to detect
    - 	 * missing hooks. Let's find it ourselves and call
    --	 * run_found_hooks() instead.
    -+	 * run_found_hooks() instead...
    - 	 */
    + 	git_config(git_default_config, NULL);
    + 
      	hook_name = argv[0];
    ++	if (ignore_missing)
    ++		return run_hooks_oneshot(hook_name, &opt);
      	hook_path = find_hook(hook_name);
      	if (!hook_path) {
    -+		/* ... act like run_hooks() under --ignore-missing */
    -+		if (ignore_missing)
    -+			return 0;
      		error("cannot find a hook named %s", hook_name);
    - 		return 1;
    - 	}
     
      ## t/t1800-hook.sh ##
     @@ t/t1800-hook.sh: test_expect_success 'git hook run: nonexistent hook' '
      	test_cmp stderr.expect stderr.actual
      '
      
    --test_expect_success 'git hook run: basic' '
     +test_expect_success 'git hook run: nonexistent hook with --ignore-missing' '
     +	git hook run --ignore-missing does-not-exist 2>stderr.actual &&
     +	test_must_be_empty stderr.actual
     +'
     +
    -+test_expect_success 'git hook run -- basic' '
    + test_expect_success 'git hook run: basic' '
      	write_script .git/hooks/test-hook <<-EOF &&
      	echo Test hook
    - 	EOF
14:  71d209b4077 ! 15:  5b66b04bec7 send-email: use 'git hook run' for 'sendemail-validate'
    @@ git-send-email.perl: sub validate_patch {
      	if ($repo) {
     +		my $hook_name = 'sendemail-validate';
      		my $hooks_path = $repo->command_oneline('rev-parse', '--git-path', 'hooks');
    --		my $validate_hook = catfile($hooks_path,
    + 		require File::Spec;
    +-		my $validate_hook = File::Spec->catfile($hooks_path,
     -					    'sendemail-validate');
    -+		my $validate_hook = catfile($hooks_path, $hook_name);
    ++		my $validate_hook = File::Spec->catfile($hooks_path, $hook_name);
      		my $hook_error;
      		if (-x $validate_hook) {
    - 			my $target = abs_path($fn);
    + 			require Cwd;
     @@ git-send-email.perl: sub validate_patch {
      			chdir($repo->wc_path() or $repo->repo_path())
      				or die("chdir: $!");
15:  246a82b55b2 = 16:  14a37a43db2 git-p4: use 'git hook' to run hooks
16:  e3f8482d803 ! 17:  ad5d0e0e7de commit: use hook.h to execute hooks
    @@ Metadata
     Author: Emily Shaffer <emilyshaffer@google.com>
     
      ## Commit message ##
    -    commit: use hook.h to execute hooks
    +    commit: convert {pre-commit,prepare-commit-msg} hook to hook.h
     
    -    Teach run_commit_hook() to call hook.h instead of run-command.h. This
    -    covers 'pre-commit', 'commit-msg', and
    -    'prepare-commit-msg'.
    -
    -    Additionally, ask the hook library - not run-command - whether any
    -    hooks will be run, as it's possible hooks may exist in the config but
    -    not the hookdir.
    -
    -    Because all but 'post-commit' hooks are expected to make some state
    -    change, force all but 'post-commit' hook to run in series. 'post-commit'
    -    "is meant primarily for notification, and cannot affect the outcome of
    -    `git commit`," so it is fine to run in parallel.
    +    Move these hooks hook away from run-command.h to and over to the new
    +    hook.h library.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
    - ## builtin/commit.c ##
    -@@ builtin/commit.c: static int prepare_to_commit(const char *index_file, const char *prefix,
    - 		return 0;
    - 	}
    - 
    --	if (!no_verify && find_hook("pre-commit")) {
    -+	if (!no_verify && hook_exists("pre-commit")) {
    - 		/*
    - 		 * Re-read the index as pre-commit hook could have updated it,
    - 		 * and write it out as a tree.  We must do this before we invoke
    -
      ## commit.c ##
     @@
      #include "commit-reach.h"
    @@ commit.c: size_t ignore_non_trailer(const char *buf, size_t len)
     -	struct strvec hook_env = STRVEC_INIT;
     +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
      	va_list args;
    +-	int ret;
     +	const char *arg;
    - 	int ret;
    --
    + 
     -	strvec_pushf(&hook_env, "GIT_INDEX_FILE=%s", index_file);
     +	strvec_pushf(&opt.env, "GIT_INDEX_FILE=%s", index_file);
      
    @@ commit.c: size_t ignore_non_trailer(const char *buf, size_t len)
     +		strvec_push(&opt.args, arg);
      	va_end(args);
     -	strvec_clear(&hook_env);
    -+
    -+	ret = run_hooks(name, &opt);
    -+	run_hooks_opt_clear(&opt);
      
    - 	return ret;
    +-	return ret;
    ++	return run_hooks_oneshot(name, &opt);
      }
    -
    - ## sequencer.c ##
    -@@ sequencer.c: static int try_to_commit(struct repository *r,
    - 		}
    - 	}
    - 
    --	if (find_hook("prepare-commit-msg")) {
    -+	if (hook_exists("prepare-commit-msg")) {
    - 		res = run_prepare_commit_msg_hook(r, msg, hook_commit);
    - 		if (res)
    - 			goto out;
17:  6ed61071c5e ! 18:  3d3a33e2674 read-cache: convert post-index-change hook to use config
    @@ Metadata
     Author: Emily Shaffer <emilyshaffer@google.com>
     
      ## Commit message ##
    -    read-cache: convert post-index-change hook to use config
    +    read-cache: convert post-index-change to use hook.h
     
    -    By using hook.h instead of run-command.h to run, post-index-change hooks
    -    can now be specified in the config in addition to the hookdir.
    -    post-index-change is not run anywhere besides in read-cache.c.
    +    Move the post-index-change hook away from run-command.h to and over to
    +    the new hook.h library.
     
         This removes the last direct user of run_hook_ve(), so we can make the
         function static now. It'll be removed entirely soon.
    @@ read-cache.c: static int do_write_locked_index(struct index_state *istate, struc
     +		     istate->updated_workdir ? "1" : "0",
     +		     istate->updated_skipworktree ? "1" : "0",
     +		     NULL);
    -+	run_hooks("post-index-change", &hook_opt);
    -+	run_hooks_opt_clear(&hook_opt);
    ++	run_hooks_oneshot("post-index-change", &hook_opt);
     +
      	istate->updated_workdir = 0;
      	istate->updated_skipworktree = 0;
18:  e4ef3f4548a ! 19:  893f8666301 receive-pack: convert push-to-checkout hook to hook.h
    @@ Metadata
      ## Commit message ##
         receive-pack: convert push-to-checkout hook to hook.h
     
    -    By using hook.h instead of run-command.h to invoke push-to-checkout,
    -    hooks can now be specified in the config as well as in the hookdir.
    -    push-to-checkout is not called anywhere but in builtin/receive-pack.c.
    +    Move the push-to-checkout hook away from run-command.h to and over to
    +    the new hook.h library.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    @@ builtin/receive-pack.c: static const char *push_to_checkout(unsigned char *hash,
     -			hash_to_hex(hash), NULL))
     +	strvec_pushv(&opt.env, env->v);
     +	strvec_push(&opt.args, hash_to_hex(hash));
    -+	if (run_hooks(push_to_checkout_hook, &opt)) {
    -+		run_hooks_opt_clear(&opt);
    ++	if (run_hooks_oneshot(push_to_checkout_hook, &opt))
      		return "push-to-checkout hook declined";
    --	else
    -+	} else {
    -+		run_hooks_opt_clear(&opt);
    - 		return NULL;
    -+	}
    - }
    - 
    - static const char *update_worktree(unsigned char *sha1, const struct worktree *worktree)
    -@@ builtin/receive-pack.c: static const char *update_worktree(unsigned char *sha1, const struct worktree *w
    - 
    - 	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
    - 
    --	if (!find_hook(push_to_checkout_hook))
    -+	if (!hook_exists(push_to_checkout_hook))
    - 		retval = push_to_deploy(sha1, &env, work_tree);
      	else
    - 		retval = push_to_checkout(sha1, &env, work_tree);
    + 		return NULL;
19:  e3dda367ec9 = 20:  070433deba5 run-command: remove old run_hook_{le,ve}() hook API
20:  477d75bf579 = 21:  1028e0c1667 run-command: allow stdin for run_processes_parallel
21:  b7c0ee9719a ! 22:  639e59e9ed0 hook: support passing stdin to hooks
    @@ Documentation/git-hook.txt: what those are.
     
      ## builtin/hook.c ##
     @@
    + #include "strvec.h"
      
    - static const char * const builtin_hook_usage[] = {
    - 	N_("git hook <command> [...]"),
    --	N_("git hook run <hook-name> [-- <hook-args>]"),
    -+	N_("git hook run [<args>] <hook-name> [-- <hook-args>]"),
    - 	NULL
    - };
    - 
    - static const char * const builtin_hook_run_usage[] = {
    - 	N_("git hook run <hook-name> [-- <hook-args>]"),
    -+	N_("git hook run [--to-stdin=<path>] <hook-name> [-- <hook-args>]"),
    - 	NULL
    - };
    + #define BUILTIN_HOOK_RUN_USAGE \
    +-	N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")
    ++	N_("git hook run [--ignore-missing] [--to-stdin=<path>] <hook-name> [-- <hook-args>]")
      
    + static const char * const builtin_hook_usage[] = {
    + 	BUILTIN_HOOK_RUN_USAGE,
     @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
      	struct option run_options[] = {
      		OPT_BOOL(0, "ignore-missing", &ignore_missing,
    @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
     +			   N_("file to read into hooks' stdin")),
      		OPT_END(),
      	};
    - 
    + 	int ret;
     
      ## hook.c ##
     @@ hook.c: static int pick_next_hook(struct child_process *cp,
    - 	struct hook_cb_data *hook_cb = pp_cb;
    - 	struct hook *run_me = hook_cb->run_me;
    + 	if (!run_me)
    + 		return 0;
      
     -	cp->no_stdin = 1;
    -+
     +	/* reopen the file for stdin; run_command closes it. */
     +	if (hook_cb->options->path_to_stdin) {
     +		cp->no_stdin = 0;
    @@ hook.c: static int pick_next_hook(struct child_process *cp,
     
      ## hook.h ##
     @@ hook.h: struct run_hooks_opt
    + 
      	/* Path to initial working directory for subprocess */
      	const char *dir;
    - 
    ++
     +	/* Path to file which should be piped to stdin for each hook */
     +	const char *path_to_stdin;
      };
22:  4035069a98c ! 23:  7d1925cca48 am: convert 'post-rewrite' hook to hook.h
    @@ builtin/am.c: static int run_applypatch_msg_hook(struct am_state *state)
      {
     -	struct child_process cp = CHILD_PROCESS_INIT;
     -	const char *hook = find_hook("post-rewrite");
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    - 	int ret;
    - 
    +-	int ret;
    +-
     -	if (!hook)
     -		return 0;
     -
    @@ builtin/am.c: static int run_applypatch_msg_hook(struct am_state *state)
     -	cp.in = xopen(am_path(state, "rewritten"), O_RDONLY);
     -	cp.stdout_to_stderr = 1;
     -	cp.trace2_hook_name = "post-rewrite";
    -+	strvec_push(&opt.args, "rebase");
    -+	opt.path_to_stdin = am_path(state, "rewritten");
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
      
     -	ret = run_command(&cp);
    -+	ret = run_hooks("post-rewrite", &opt);
    ++	strvec_push(&opt.args, "rebase");
    ++	opt.path_to_stdin = am_path(state, "rewritten");
      
     -	close(cp.in);
    -+	run_hooks_opt_clear(&opt);
    - 	return ret;
    +-	return ret;
    ++	return run_hooks_oneshot("post-rewrite", &opt);
      }
      
    + /**
23:  c847a19581a ! 24:  0c24221b522 run-command: add stdin callback for parallelization
    @@ builtin/submodule--helper.c: static int update_submodules(struct submodule_updat
      
     
      ## hook.c ##
    -@@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
    - 	run_processes_parallel_tr2(options->jobs,
    +@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
    + 	run_processes_parallel_tr2(jobs,
      				   pick_next_hook,
      				   notify_start_failure,
     +				   NULL,
    @@ run-command.h: typedef int (*task_finished_fn)(int result,
     +			       feed_pipe_fn, task_finished_fn, void *pp_cb,
      			       const char *tr2_category, const char *tr2_label);
      
    - #endif
    + /**
     
      ## submodule.c ##
     @@ submodule.c: int fetch_populated_submodules(struct repository *r,
24:  da46c859c1c ! 25:  05d1085f7eb hook: provide stdin by string_list or callback
    @@ hook.c: static int pick_next_hook(struct child_process *cp,
      	} else {
      		cp->no_stdin = 1;
      	}
    -@@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
    - 	run_processes_parallel_tr2(options->jobs,
    +@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
    + 	run_processes_parallel_tr2(jobs,
      				   pick_next_hook,
      				   notify_start_failure,
     -				   NULL,
    @@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
      				   notify_hook_finished,
      				   &cb_data,
      				   "hook",
    -@@ hook.c: int run_hooks(const char *hook_name, struct run_hooks_opt *options)
    +@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
    + 
    + 	if (options->absolute_path)
    + 		strbuf_release(&abs_path);
    ++	free(my_hook.feed_pipe_cb_data);
    + 
    + 	return cb_data.rc;
    + }
    +@@ hook.c: int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
      	if (!options)
    - 		BUG("a struct run_hooks_opt must be provided to run_hooks");
    + 		options = &hook_opt_scratch;
      
     +	if (options->path_to_stdin && options->feed_pipe)
     +		BUG("choose only one method to populate stdin");
     +
      	hook_path = find_hook(hook_name);
    - 
    - 	/*
    + 	if (!hook_path) {
    + 		ret = 0;
     
      ## hook.h ##
     @@ hook.h: int hook_exists(const char *hookname);
    @@ hook.h: struct run_hooks_opt
     + */
     +int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb);
     +
    - struct hook_cb_data {
    - 	/* rc reflects the cumulative failure state */
    - 	int rc;
    + /*
    +  * Callback provided to feed_pipe_fn and consume_sideband_fn.
    +  */
25:  7343be28ef4 ! 26:  4b7175af2e5 hook: convert 'post-rewrite' hook in sequencer.c to hook.h
    @@ sequencer.c: int update_head_with_reflog(const struct commit *old_head,
     +	opt.feed_pipe = pipe_from_string_list;
     +	opt.feed_pipe_ctx = &to_stdin;
     +
    -+	code = run_hooks("post-rewrite", &opt);
    ++	code = run_hooks_oneshot("post-rewrite", &opt);
     +
    -+	run_hooks_opt_clear(&opt);
     +	strbuf_release(&tmp);
     +	string_list_clear(&to_stdin, 0);
     +	return code;
    @@ sequencer.c: static int pick_commits(struct repository *r,
     +
     +			hook_opt.path_to_stdin = rebase_path_rewritten_list();
     +			strvec_push(&hook_opt.args, "rebase");
    -+			run_hooks("post-rewrite", &hook_opt);
    -+			run_hooks_opt_clear(&hook_opt);
    ++			run_hooks_oneshot("post-rewrite", &hook_opt);
      		}
      		apply_autostash(rebase_path_autostash());
      
26:  85bf13a0835 ! 27:  3f24e056410 transport: convert pre-push hook to use config
    @@ Metadata
     Author: Emily Shaffer <emilyshaffer@google.com>
     
      ## Commit message ##
    -    transport: convert pre-push hook to use config
    +    transport: convert pre-push hook to hook.h
     
    -    By using the hook.h:run_hooks API, pre-push hooks can be specified in
    -    the config as well as in the hookdir.
    +    Move the pre-push hook away from run-command.h to and over to the new
    +    hook.h library.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    @@ transport.c: static void die_with_unpushed_submodules(struct string_list *needs_
     -	int ret = 0, x;
     +	int ret = 0;
     +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct strbuf tmp = STRBUF_INIT;
      	struct ref *r;
     -	struct child_process proc = CHILD_PROCESS_INIT;
     -	struct strbuf buf;
    @@ transport.c: static void die_with_unpushed_submodules(struct string_list *needs_
     -		finish_command(&proc);
     -		return -1;
     -	}
    --
    --	sigchain_push(SIGPIPE, SIG_IGN);
    -+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
    ++	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
      
    +-	sigchain_push(SIGPIPE, SIG_IGN);
    +-
     -	strbuf_init(&buf, 256);
     +	strvec_push(&opt.args, transport->remote->name);
     +	strvec_push(&opt.args, transport->url);
      
      	for (r = remote_refs; r; r = r->next) {
    ++		struct strbuf buf = STRBUF_INIT;
    ++
      		if (!r->peer_ref) continue;
    -@@ transport.c: static int run_pre_push_hook(struct transport *transport,
    + 		if (r->status == REF_STATUS_REJECT_NONFASTFORWARD) continue;
    + 		if (r->status == REF_STATUS_REJECT_STALE) continue;
      		if (r->status == REF_STATUS_REJECT_REMOTE_UPDATED) continue;
      		if (r->status == REF_STATUS_UPTODATE) continue;
      
     -		strbuf_reset(&buf);
     -		strbuf_addf( &buf, "%s %s %s %s\n",
    -+		strbuf_reset(&tmp);
    -+		strbuf_addf(&tmp, "%s %s %s %s",
    ++		strbuf_addf(&buf, "%s %s %s %s",
      			 r->peer_ref->name, oid_to_hex(&r->new_oid),
      			 r->name, oid_to_hex(&r->old_oid));
     -
    @@ transport.c: static int run_pre_push_hook(struct transport *transport,
     -				ret = -1;
     -			break;
     -		}
    -+		string_list_append(&to_stdin, tmp.buf);
    ++		string_list_append(&to_stdin, strbuf_detach(&buf, NULL));
      	}
      
     -	strbuf_release(&buf);
    @@ transport.c: static int run_pre_push_hook(struct transport *transport,
     -	x = finish_command(&proc);
     -	if (!ret)
     -		ret = x;
    -+	ret = run_hooks("pre-push", &opt);
    -+	run_hooks_opt_clear(&opt);
    -+	strbuf_release(&tmp);
    ++	ret = run_hooks_oneshot("pre-push", &opt);
    ++	to_stdin.strdup_strings = 1;
     +	string_list_clear(&to_stdin, 0);
      
      	return ret;
 -:  ----------- > 28:  ecf75f33233 hook tests: test for exact "pre-push" hook input
 -:  ----------- > 29:  2c961be94b4 hook tests: use a modern style for "pre-push" tests
27:  331014bad17 ! 30:  1ce456f9d9d reference-transaction: use hook.h to run hooks
    @@ refs.c: int ref_update_reject_duplicates(struct string_list *refnames,
      				const char *state)
      {
     -	struct child_process proc = CHILD_PROCESS_INIT;
    - 	struct strbuf buf = STRBUF_INIT;
    +-	struct strbuf buf = STRBUF_INIT;
     -	const char *hook;
     +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
    ++	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
      	int ret = 0, i;
    -+	char o[GIT_MAX_HEXSZ + 1], n[GIT_MAX_HEXSZ + 1];
      
     -	hook = find_hook("reference-transaction");
     -	if (!hook)
    -+	if (!hook_exists("reference-transaction"))
    - 		return ret;
    - 
    +-		return ret;
    +-
     -	strvec_pushl(&proc.args, hook, state, NULL);
     -	proc.in = -1;
     -	proc.stdout_to_stderr = 1;
    @@ refs.c: int ref_update_reject_duplicates(struct string_list *refnames,
     -
     -	ret = start_command(&proc);
     -	if (ret)
    --		return ret;
    --
    ++	if (!hook_exists("reference-transaction"))
    + 		return ret;
    + 
     -	sigchain_push(SIGPIPE, SIG_IGN);
     +	strvec_push(&opt.args, state);
      
      	for (i = 0; i < transaction->nr; i++) {
      		struct ref_update *update = transaction->updates[i];
    -+		oid_to_hex_r(o, &update->old_oid);
    -+		oid_to_hex_r(n, &update->new_oid);
    ++		struct strbuf buf = STRBUF_INIT;
      
    - 		strbuf_reset(&buf);
    +-		strbuf_reset(&buf);
     -		strbuf_addf(&buf, "%s %s %s\n",
    --			    oid_to_hex(&update->old_oid),
    --			    oid_to_hex(&update->new_oid),
    --			    update->refname);
    ++		strbuf_addf(&buf, "%s %s %s",
    + 			    oid_to_hex(&update->old_oid),
    + 			    oid_to_hex(&update->new_oid),
    + 			    update->refname);
     -
     -		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
     -			if (errno != EPIPE)
     -				ret = -1;
     -			break;
     -		}
    -+		strbuf_addf(&buf, "%s %s %s", o, n, update->refname);
    -+		string_list_append(&to_stdin, buf.buf);
    ++		string_list_append(&to_stdin, strbuf_detach(&buf, NULL));
      	}
      
     -	close(proc.in);
     -	sigchain_pop(SIGPIPE);
    +-	strbuf_release(&buf);
     +	opt.feed_pipe = pipe_from_string_list;
     +	opt.feed_pipe_ctx = &to_stdin;
     +
    -+	ret = run_hooks("reference-transaction", &opt);
    -+	run_hooks_opt_clear(&opt);
    - 	strbuf_release(&buf);
    ++	ret = run_hooks_oneshot("reference-transaction", &opt);
    ++	to_stdin.strdup_strings = 1;
     +	string_list_clear(&to_stdin, 0);
      
     -	ret |= finish_command(&proc);
28:  f7f56d0a3bb ! 31:  6e5f1f5bd3a run-command: allow capturing of collated output
    @@ builtin/submodule--helper.c: static int update_submodules(struct submodule_updat
      
     
      ## hook.c ##
    -@@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
    +@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
      				   pick_next_hook,
      				   notify_start_failure,
      				   options->feed_pipe,
    @@ run-command.h: int run_processes_parallel(int n,
     +			       task_finished_fn, void *pp_cb,
      			       const char *tr2_category, const char *tr2_label);
      
    - #endif
    + /**
     
      ## submodule.c ##
     @@ submodule.c: int fetch_populated_submodules(struct repository *r,
29:  7f7fcc06885 ! 32:  0b6e9c6d07a hooks: allow callers to capture output
    @@ Commit message
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## hook.c ##
    -@@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
    +@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
      				   pick_next_hook,
      				   notify_start_failure,
      				   options->feed_pipe,
    @@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
      				   notify_hook_finished,
      				   &cb_data,
      				   "hook",
    + 				   hook_name);
    + 
    ++
    + 	if (options->absolute_path)
    + 		strbuf_release(&abs_path);
    + 	free(my_hook.feed_pipe_cb_data);
     
      ## hook.h ##
     @@ hook.h: struct run_hooks_opt
30:  e74d49e5593 ! 33:  dcf63634338 receive-pack: convert 'update' hook to hook.h
    @@ builtin/receive-pack.c: static int run_receive_hook(struct command *commands,
     +static int run_update_hook(struct command *cmd)
     +{
     +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	int code;
     +
     +	strvec_pushl(&opt.args,
     +		     cmd->ref_name,
    @@ builtin/receive-pack.c: static int run_receive_hook(struct command *commands,
     -	return finish_command(&proc);
     +		opt.consume_sideband = hook_output_to_sideband;
     +
    -+	code = run_hooks("update", &opt);
    -+	run_hooks_opt_clear(&opt);
    -+	return code;
    ++	return run_hooks_oneshot("update", &opt);
      }
      
      static struct command *find_command_by_refname(struct command *list,
31:  0bdc4878ac8 ! 34:  f352a485e59 post-update: use hook.h library
    @@ builtin/receive-pack.c: static const char *update(struct command *cmd, struct sh
     -			copy_to_sideband(proc.err, -1, NULL);
     -		finish_command(&proc);
     -	}
    -+	run_hooks("post-update", &opt);
    -+	run_hooks_opt_clear(&opt);
    ++	run_hooks_oneshot("post-update", &opt);
      }
      
      static void check_aliased_update_internal(struct command *cmd,
32:  db70b59b3bd ! 35:  ceef2f3e804 receive-pack: convert receive hooks to hook.h
    @@ builtin/receive-pack.c: static void hook_output_to_sideband(struct strbuf *outpu
     +{
     +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
     +	struct receive_hook_feed_context ctx;
    -+	int rc;
     +	struct command *iter = commands;
     +
     +	/* if there are no valid commands, don't invoke the hook at all. */
    @@ builtin/receive-pack.c: static void hook_output_to_sideband(struct strbuf *outpu
     +	if (!iter)
     +		return 0;
     +
    -+	/* pre-receive hooks should run in series as the hook updates refs */
    -+	if (!strcmp(hook_name, "pre-receive"))
    -+		opt.jobs = 1;
    -+
     +	if (push_options) {
     +		int i;
     +		for (i = 0; i < push_options->nr; i++)
    @@ builtin/receive-pack.c: static void hook_output_to_sideband(struct strbuf *outpu
     +	opt.feed_pipe = feed_receive_hook_cb;
     +	opt.feed_pipe_ctx = &ctx;
     +
    -+	rc = run_hooks(hook_name, &opt);
    -+	run_hooks_opt_clear(&opt);
    -+	return rc;
    ++	return run_hooks_oneshot(hook_name, &opt);
     +}
     +
      static int run_update_hook(struct command *cmd)
33:  d86fedf041c ! 36:  b71d7628b40 hooks: fix a TOCTOU in "did we run a hook?" heuristic
    @@ hook.c: static int notify_hook_finished(int result,
     +	if (hook_cb->invoked_hook)
     +		*hook_cb->invoked_hook = 1;
     +
    - 	return 1;
    + 	return 0;
      }
      
    -@@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
    +@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
      		.rc = 0,
      		.hook_name = hook_name,
      		.options = options,
     +		.invoked_hook = options->invoked_hook,
      	};
    - 	if (options->absolute_path) {
    - 		strbuf_add_absolute_path(&abs_path, hook_path);
    + 	int jobs = 1;
    + 
     
      ## hook.h ##
     @@ hook.h: struct run_hooks_opt
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH v4 01/36] Makefile: mark "check" target as .PHONY
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-20  0:04               ` Emily Shaffer
  2021-08-03 19:38             ` [PATCH v4 02/36] Makefile: stop hardcoding {command,config}-list.h Ævar Arnfjörð Bjarmason
                               ` (38 subsequent siblings)
  39 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Fix a bug in 44c9e8594e (Fix up header file dependencies and add
sparse checking rules, 2005-07-03), we never marked the phony "check"
target as such.

Perhaps we should just remove it, since as of a combination of
912f9980d2 (Makefile: help people who run 'make check' by mistake,
2008-11-11) 0bcd9ae85d (sparse: Fix errors due to missing
target-specific variables, 2011-04-21) we've been suggesting the user
run "make sparse" directly.

But under that mode it still does something, as well as directing the
user to run "make test" under non-sparse. So let's punt that and
narrowly fix the PHONY bug.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Makefile b/Makefile
index c6f6246bf63..2ff038069e8 100644
--- a/Makefile
+++ b/Makefile
@@ -2931,6 +2931,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
+.PHONY: check
 check: config-list.h command-list.h
 	@if sparse; \
 	then \
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 02/36] Makefile: stop hardcoding {command,config}-list.h
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 01/36] Makefile: mark "check" target as .PHONY Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-20  0:03               ` Emily Shaffer
  2021-08-03 19:38             ` [PATCH v4 03/36] Makefile: remove an out-of-date comment Ævar Arnfjörð Bjarmason
                               ` (37 subsequent siblings)
  39 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Change various places that hardcode the names of these two files to
refer to either $(GENERATED_H), or to a new generated-hdrs
target. That target is consistent with the *-objs targets I recently
added in 029bac01a8 (Makefile: add {program,xdiff,test,git,fuzz}-objs
& objects targets, 2021-02-23).

A subsequent commit will add a new generated hook-list.h. By doing
this refactoring we'll only need to add the new file to the
GENERATED_H variable, not EXCEPT_HDRS, the vcbuild/README etc.

I have not tested the Windows-specific change in config.mak.uname
being made here, but we use other variables from the Makefile in the
same block, and the GENERATED_H is fully defined before we include
config.mak.uname.

Hardcoding command-list.h there seems to have been a case of
copy/paste programming in 976aaedca0 (msvc: add a Makefile target to
pre-generate the Visual Studio solution, 2019-07-29). The
config-list.h was added later in 709df95b78 (help: move
list_config_help to builtin/help, 2020-04-16).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile              | 6 ++++--
 compat/vcbuild/README | 2 +-
 config.mak.uname      | 6 +++---
 3 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/Makefile b/Makefile
index 2ff038069e8..89bf0dd7332 100644
--- a/Makefile
+++ b/Makefile
@@ -823,6 +823,8 @@ XDIFF_LIB = xdiff/lib.a
 
 GENERATED_H += command-list.h
 GENERATED_H += config-list.h
+.PHONY: generated-hdrs
+generated-hdrs: $(GENERATED_H)
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2909,7 +2911,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
+EXCEPT_HDRS := $(GENERATED_H) unicode-width.h compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
 endif
@@ -2932,7 +2934,7 @@ style:
 	git clang-format --style file --diff --extensions c,h
 
 .PHONY: check
-check: config-list.h command-list.h
+check: $(GENERATED_H)
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
diff --git a/compat/vcbuild/README b/compat/vcbuild/README
index 51fb083dbbe..29ec1d0f104 100644
--- a/compat/vcbuild/README
+++ b/compat/vcbuild/README
@@ -92,7 +92,7 @@ The Steps of Build Git with VS2008
    the git operations.
 
 3. Inside Git's directory run the command:
-       make command-list.h config-list.h
+       make generated-hdrs
    to generate the header file needed to compile git.
 
 4. Then either build Git with the GNU Make Makefile in the Git projects
diff --git a/config.mak.uname b/config.mak.uname
index 69413fb3dc0..9988378160b 100644
--- a/config.mak.uname
+++ b/config.mak.uname
@@ -732,9 +732,9 @@ vcxproj:
 	 echo '</Project>') >git-remote-http/LinkOrCopyRemoteHttp.targets
 	git add -f git/LinkOrCopyBuiltins.targets git-remote-http/LinkOrCopyRemoteHttp.targets
 
-	# Add command-list.h and config-list.h
-	$(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 config-list.h command-list.h
-	git add -f config-list.h command-list.h
+	# Add generated headers
+	$(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 $(GENERATED_H)
+	git add -f $(GENERATED_H)
 
 	# Add scripts
 	rm -f perl/perl.mak
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 03/36] Makefile: remove an out-of-date comment
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 01/36] Makefile: mark "check" target as .PHONY Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 02/36] Makefile: stop hardcoding {command,config}-list.h Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-20  0:05               ` Emily Shaffer
  2021-08-03 19:38             ` [PATCH v4 04/36] hook.[ch]: move find_hook() to this new library Ævar Arnfjörð Bjarmason
                               ` (36 subsequent siblings)
  39 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

This comment added in dfea575017 (Makefile: lazily compute header
dependencies, 2010-01-26) has been out of date since
92b88eba9f (Makefile: use `git ls-files` to list header files, if
possible, 2019-03-04), when we did exactly what it tells us not to do
and added $(GENERATED_H) to $(OBJECTS) dependencies.

The rest of it was also somewhere between inaccurate and outdated,
since as of b8ba629264 (Makefile: fold MISC_H into LIB_H, 2012-06-20)
it's not followed by a list of header files, that got moved earlier in
the file into LIB_H in 60d24dd255 (Makefile: fold XDIFF_H and VCSSVN_H
into LIB_H, 2012-07-06).

Let's just remove it entirely, to the extent that we have anything
useful to say here the comment on the
"USE_COMPUTED_HEADER_DEPENDENCIES" variable a few lines above this
change does the job for us.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/Makefile b/Makefile
index 89bf0dd7332..0a540dcd34e 100644
--- a/Makefile
+++ b/Makefile
@@ -2519,13 +2519,6 @@ ifneq ($(dep_files_present),)
 include $(dep_files_present)
 endif
 else
-# Dependencies on header files, for platforms that do not support
-# the gcc -MMD option.
-#
-# Dependencies on automatically generated headers such as command-list.h
-# should _not_ be included here, since they are necessary even when
-# building an object for the first time.
-
 $(OBJECTS): $(LIB_H) $(GENERATED_H)
 endif
 
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 04/36] hook.[ch]: move find_hook() to this new library
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (2 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 03/36] Makefile: remove an out-of-date comment Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-20  0:08               ` Emily Shaffer
  2021-08-03 19:38             ` [PATCH v4 05/36] hook.c: add a hook_exists() wrapper and use it in bugreport.c Ævar Arnfjörð Bjarmason
                               ` (35 subsequent siblings)
  39 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Move the find_hook() function from run-command.c to a new hook.c
library. This change establishes a stub library that's pretty
pointless right now, but will see much wider use with Emily Shaffer's
upcoming "configuration-based hooks" series.

Eventually all the hook related code will live in hook.[ch]. Let's
start that process by moving the simple find_hook() function over
as-is.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile               |  1 +
 builtin/am.c           |  1 +
 builtin/bugreport.c    |  2 +-
 builtin/commit.c       |  1 +
 builtin/merge.c        |  1 +
 builtin/receive-pack.c |  1 +
 builtin/worktree.c     |  1 +
 hook.c                 | 37 +++++++++++++++++++++++++++++++++++++
 hook.h                 | 11 +++++++++++
 refs.c                 |  1 +
 run-command.c          | 35 +----------------------------------
 run-command.h          |  7 -------
 sequencer.c            |  1 +
 transport.c            |  1 +
 14 files changed, 59 insertions(+), 42 deletions(-)
 create mode 100644 hook.c
 create mode 100644 hook.h

diff --git a/Makefile b/Makefile
index 0a540dcd34e..bc258886904 100644
--- a/Makefile
+++ b/Makefile
@@ -910,6 +910,7 @@ LIB_OBJS += hash-lookup.o
 LIB_OBJS += hashmap.o
 LIB_OBJS += help.o
 LIB_OBJS += hex.o
+LIB_OBJS += hook.o
 LIB_OBJS += ident.o
 LIB_OBJS += json-writer.o
 LIB_OBJS += kwset.o
diff --git a/builtin/am.c b/builtin/am.c
index 0c2ad96b70e..c603f3cebdf 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -11,6 +11,7 @@
 #include "parse-options.h"
 #include "dir.h"
 #include "run-command.h"
+#include "hook.h"
 #include "quote.h"
 #include "tempfile.h"
 #include "lockfile.h"
diff --git a/builtin/bugreport.c b/builtin/bugreport.c
index 9915a5841de..596f079a7f9 100644
--- a/builtin/bugreport.c
+++ b/builtin/bugreport.c
@@ -3,7 +3,7 @@
 #include "strbuf.h"
 #include "help.h"
 #include "compat/compiler.h"
-#include "run-command.h"
+#include "hook.h"
 
 
 static void get_system_info(struct strbuf *sys_info)
diff --git a/builtin/commit.c b/builtin/commit.c
index 7436262aae2..51b07ee02ea 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -19,6 +19,7 @@
 #include "revision.h"
 #include "wt-status.h"
 #include "run-command.h"
+#include "hook.h"
 #include "refs.h"
 #include "log-tree.h"
 #include "strbuf.h"
diff --git a/builtin/merge.c b/builtin/merge.c
index a8a843b1f54..be98d66b0a8 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -13,6 +13,7 @@
 #include "builtin.h"
 #include "lockfile.h"
 #include "run-command.h"
+#include "hook.h"
 #include "diff.h"
 #include "diff-merges.h"
 #include "refs.h"
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 2d1f97e1ca7..97aebdc15bd 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -7,6 +7,7 @@
 #include "pkt-line.h"
 #include "sideband.h"
 #include "run-command.h"
+#include "hook.h"
 #include "exec-cmd.h"
 #include "commit.h"
 #include "object.h"
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 0d0a80da61f..d22ece93e1a 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -8,6 +8,7 @@
 #include "branch.h"
 #include "refs.h"
 #include "run-command.h"
+#include "hook.h"
 #include "sigchain.h"
 #include "submodule.h"
 #include "utf8.h"
diff --git a/hook.c b/hook.c
new file mode 100644
index 00000000000..c4dbef1d0ef
--- /dev/null
+++ b/hook.c
@@ -0,0 +1,37 @@
+#include "cache.h"
+#include "hook.h"
+#include "run-command.h"
+
+const char *find_hook(const char *name)
+{
+	static struct strbuf path = STRBUF_INIT;
+
+	strbuf_reset(&path);
+	strbuf_git_path(&path, "hooks/%s", name);
+	if (access(path.buf, X_OK) < 0) {
+		int err = errno;
+
+#ifdef STRIP_EXTENSION
+		strbuf_addstr(&path, STRIP_EXTENSION);
+		if (access(path.buf, X_OK) >= 0)
+			return path.buf;
+		if (errno == EACCES)
+			err = errno;
+#endif
+
+		if (err == EACCES && advice_ignored_hook) {
+			static struct string_list advise_given = STRING_LIST_INIT_DUP;
+
+			if (!string_list_lookup(&advise_given, name)) {
+				string_list_insert(&advise_given, name);
+				advise(_("The '%s' hook was ignored because "
+					 "it's not set as executable.\n"
+					 "You can disable this warning with "
+					 "`git config advice.ignoredHook false`."),
+				       path.buf);
+			}
+		}
+		return NULL;
+	}
+	return path.buf;
+}
diff --git a/hook.h b/hook.h
new file mode 100644
index 00000000000..68624f16059
--- /dev/null
+++ b/hook.h
@@ -0,0 +1,11 @@
+#ifndef HOOK_H
+#define HOOK_H
+
+/*
+ * Returns the path to the hook file, or NULL if the hook is missing
+ * or disabled. Note that this points to static storage that will be
+ * overwritten by further calls to find_hook and run_hook_*.
+ */
+const char *find_hook(const char *name);
+
+#endif
diff --git a/refs.c b/refs.c
index 8b9f7c3a80a..6211692eaae 100644
--- a/refs.c
+++ b/refs.c
@@ -10,6 +10,7 @@
 #include "refs.h"
 #include "refs/refs-internal.h"
 #include "run-command.h"
+#include "hook.h"
 #include "object-store.h"
 #include "object.h"
 #include "tag.h"
diff --git a/run-command.c b/run-command.c
index f72e72cce73..352f5be1646 100644
--- a/run-command.c
+++ b/run-command.c
@@ -8,6 +8,7 @@
 #include "string-list.h"
 #include "quote.h"
 #include "config.h"
+#include "hook.h"
 
 void child_process_init(struct child_process *child)
 {
@@ -1319,40 +1320,6 @@ int async_with_fork(void)
 #endif
 }
 
-const char *find_hook(const char *name)
-{
-	static struct strbuf path = STRBUF_INIT;
-
-	strbuf_reset(&path);
-	strbuf_git_path(&path, "hooks/%s", name);
-	if (access(path.buf, X_OK) < 0) {
-		int err = errno;
-
-#ifdef STRIP_EXTENSION
-		strbuf_addstr(&path, STRIP_EXTENSION);
-		if (access(path.buf, X_OK) >= 0)
-			return path.buf;
-		if (errno == EACCES)
-			err = errno;
-#endif
-
-		if (err == EACCES && advice_ignored_hook) {
-			static struct string_list advise_given = STRING_LIST_INIT_DUP;
-
-			if (!string_list_lookup(&advise_given, name)) {
-				string_list_insert(&advise_given, name);
-				advise(_("The '%s' hook was ignored because "
-					 "it's not set as executable.\n"
-					 "You can disable this warning with "
-					 "`git config advice.ignoredHook false`."),
-				       path.buf);
-			}
-		}
-		return NULL;
-	}
-	return path.buf;
-}
-
 int run_hook_ve(const char *const *env, const char *name, va_list args)
 {
 	struct child_process hook = CHILD_PROCESS_INIT;
diff --git a/run-command.h b/run-command.h
index af1296769f9..f76b740f927 100644
--- a/run-command.h
+++ b/run-command.h
@@ -204,13 +204,6 @@ int finish_command_in_signal(struct child_process *);
  */
 int run_command(struct child_process *);
 
-/*
- * Returns the path to the hook file, or NULL if the hook is missing
- * or disabled. Note that this points to static storage that will be
- * overwritten by further calls to find_hook and run_hook_*.
- */
-const char *find_hook(const char *name);
-
 /**
  * Run a hook.
  * The first argument is a pathname to an index file, or NULL
diff --git a/sequencer.c b/sequencer.c
index 7f07cd00f3f..ea4199d65a4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -8,6 +8,7 @@
 #include "sequencer.h"
 #include "tag.h"
 #include "run-command.h"
+#include "hook.h"
 #include "exec-cmd.h"
 #include "utf8.h"
 #include "cache-tree.h"
diff --git a/transport.c b/transport.c
index 17e9629710a..77e196f75f5 100644
--- a/transport.c
+++ b/transport.c
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "transport.h"
 #include "run-command.h"
+#include "hook.h"
 #include "pkt-line.h"
 #include "fetch-pack.h"
 #include "remote.h"
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 05/36] hook.c: add a hook_exists() wrapper and use it in bugreport.c
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (3 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 04/36] hook.[ch]: move find_hook() to this new library Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-20  0:09               ` Emily Shaffer
  2021-08-03 19:38             ` [PATCH v4 06/36] hook.c users: use "hook_exists()" insted of "find_hook()" Ævar Arnfjörð Bjarmason
                               ` (34 subsequent siblings)
  39 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Add a boolean version of the find_hook() function for those callers
who are only interested in checking whether the hook exists, not what
the path to it is.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/bugreport.c | 2 +-
 hook.c              | 5 +++++
 hook.h              | 5 +++++
 3 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/builtin/bugreport.c b/builtin/bugreport.c
index 596f079a7f9..941c8d5e270 100644
--- a/builtin/bugreport.c
+++ b/builtin/bugreport.c
@@ -82,7 +82,7 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 
 	for (i = 0; i < ARRAY_SIZE(hook); i++)
-		if (find_hook(hook[i]))
+		if (hook_exists(hook[i]))
 			strbuf_addf(hook_info, "%s\n", hook[i]);
 }
 
diff --git a/hook.c b/hook.c
index c4dbef1d0ef..97cd799a320 100644
--- a/hook.c
+++ b/hook.c
@@ -35,3 +35,8 @@ const char *find_hook(const char *name)
 	}
 	return path.buf;
 }
+
+int hook_exists(const char *name)
+{
+	return !!find_hook(name);
+}
diff --git a/hook.h b/hook.h
index 68624f16059..4c547ac15e5 100644
--- a/hook.h
+++ b/hook.h
@@ -8,4 +8,9 @@
  */
 const char *find_hook(const char *name);
 
+/*
+ * A boolean version of find_hook()
+ */
+int hook_exists(const char *hookname);
+
 #endif
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 06/36] hook.c users: use "hook_exists()" insted of "find_hook()"
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (4 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 05/36] hook.c: add a hook_exists() wrapper and use it in bugreport.c Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-20  0:10               ` Emily Shaffer
  2021-08-03 19:38             ` [PATCH v4 07/36] hook-list.h: add a generated list of hooks, like config-list.h Ævar Arnfjörð Bjarmason
                               ` (33 subsequent siblings)
  39 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Use the new hook_exists() function instead of find_hook() where the
latter was called in boolean contexts. This make subsequent changes in
a series where we further refactor the hook API clearer, as we won't
conflate wanting to get the path of the hook with checking for its
existence.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/commit.c       | 2 +-
 builtin/merge.c        | 2 +-
 builtin/receive-pack.c | 2 +-
 sequencer.c            | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index 51b07ee02ea..aa3c741efa9 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1052,7 +1052,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (!no_verify && find_hook("pre-commit")) {
+	if (!no_verify && hook_exists("pre-commit")) {
 		/*
 		 * Re-read the index as pre-commit hook could have updated it,
 		 * and write it out as a tree.  We must do this before we invoke
diff --git a/builtin/merge.c b/builtin/merge.c
index be98d66b0a8..03f244dd5a0 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -849,7 +849,7 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 	 * and write it out as a tree.  We must do this before we invoke
 	 * the editor and after we invoke run_status above.
 	 */
-	if (find_hook("pre-merge-commit"))
+	if (hook_exists("pre-merge-commit"))
 		discard_cache();
 	read_cache_from(index_file);
 	strbuf_addbuf(&msg, &merge_msg);
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 97aebdc15bd..91fa799b66e 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1464,7 +1464,7 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 
 	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
 
-	if (!find_hook(push_to_checkout_hook))
+	if (!hook_exists(push_to_checkout_hook))
 		retval = push_to_deploy(sha1, &env, work_tree);
 	else
 		retval = push_to_checkout(sha1, &env, work_tree);
diff --git a/sequencer.c b/sequencer.c
index ea4199d65a4..9aac08c1545 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1446,7 +1446,7 @@ static int try_to_commit(struct repository *r,
 		}
 	}
 
-	if (find_hook("prepare-commit-msg")) {
+	if (hook_exists("prepare-commit-msg")) {
 		res = run_prepare_commit_msg_hook(r, msg, hook_commit);
 		if (res)
 			goto out;
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 07/36] hook-list.h: add a generated list of hooks, like config-list.h
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (5 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 06/36] hook.c users: use "hook_exists()" insted of "find_hook()" Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 08/36] hook: add 'run' subcommand Ævar Arnfjörð Bjarmason
                               ` (32 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason, René Scharfe

Make githooks(5) the source of truth for what hooks git supports, and
die hooks we don't know about in find_hook(). This ensures that the
documentation and the C code's idea about existing hooks doesn't
diverge.

We still have Perl and Python code running its own hooks, but that'll
be addressed by Emily Shaffer's upcoming "git hook run" command.

This resolves a long-standing TODO item in bugreport.c of there being
no centralized listing of hooks, and fixes a bug with the bugreport
listing only knowing about 1/4 of the p4 hooks. It didn't know about
the recent "reference-transaction" hook either.

I have not been able to directly test the CMake change being made
here. Since 4c2c38e800 (ci: modification of main.yml to use cmake for
vs-build job, 2020-06-26) some of the Windows CI has a hard dependency
on CMake, this change works there, and is to my eyes an obviously
correct use of a pattern established in previous CMake changes,
namely:

 - 061c2240b1 (Introduce CMake support for configuring Git,
    2020-06-12)
 - 709df95b78 (help: move list_config_help to builtin/help,
    2020-04-16)
 - 976aaedca0 (msvc: add a Makefile target to pre-generate the Visual
   Studio solution, 2019-07-29)

The LC_ALL=C is needed because at least in my locale the dash ("-") is
ignored for the purposes of sorting, which results in a different
order. I'm not aware of anything in git that has a hard dependency on
the order, but e.g. the bugreport output would end up using whatever
locale was in effect when git was compiled.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Helped-by: René Scharfe <l.s.r@web.de>
---
 .gitignore                          |  1 +
 Makefile                            | 10 ++++++-
 builtin/bugreport.c                 | 44 ++++++-----------------------
 contrib/buildsystems/CMakeLists.txt |  7 +++++
 generate-hooklist.sh                | 18 ++++++++++++
 hook.c                              | 19 +++++++++++++
 6 files changed, 62 insertions(+), 37 deletions(-)
 create mode 100755 generate-hooklist.sh

diff --git a/.gitignore b/.gitignore
index 311841f9bed..6be9de41ae8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -190,6 +190,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/hook-list.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Makefile b/Makefile
index bc258886904..1740b99db00 100644
--- a/Makefile
+++ b/Makefile
@@ -823,6 +823,8 @@ XDIFF_LIB = xdiff/lib.a
 
 GENERATED_H += command-list.h
 GENERATED_H += config-list.h
+GENERATED_H += hook-list.h
+
 .PHONY: generated-hdrs
 generated-hdrs: $(GENERATED_H)
 
@@ -2226,7 +2228,9 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
+hook.sp hook.s hook.o: hook-list.h
+
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h hook-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2259,6 +2263,10 @@ command-list.h: $(wildcard Documentation/git*.txt)
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+hook-list.h: generate-hooklist.sh Documentation/githooks.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-hooklist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
diff --git a/builtin/bugreport.c b/builtin/bugreport.c
index 941c8d5e270..a7a1fcb8a7a 100644
--- a/builtin/bugreport.c
+++ b/builtin/bugreport.c
@@ -4,6 +4,7 @@
 #include "help.h"
 #include "compat/compiler.h"
 #include "hook.h"
+#include "hook-list.h"
 
 
 static void get_system_info(struct strbuf *sys_info)
@@ -41,39 +42,7 @@ static void get_system_info(struct strbuf *sys_info)
 
 static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 {
-	/*
-	 * NEEDSWORK: Doesn't look like there is a list of all possible hooks;
-	 * so below is a transcription of `git help hooks`. Later, this should
-	 * be replaced with some programmatically generated list (generated from
-	 * doc or else taken from some library which tells us about all the
-	 * hooks)
-	 */
-	static const char *hook[] = {
-		"applypatch-msg",
-		"pre-applypatch",
-		"post-applypatch",
-		"pre-commit",
-		"pre-merge-commit",
-		"prepare-commit-msg",
-		"commit-msg",
-		"post-commit",
-		"pre-rebase",
-		"post-checkout",
-		"post-merge",
-		"pre-push",
-		"pre-receive",
-		"update",
-		"post-receive",
-		"post-update",
-		"push-to-checkout",
-		"pre-auto-gc",
-		"post-rewrite",
-		"sendemail-validate",
-		"fsmonitor-watchman",
-		"p4-pre-submit",
-		"post-index-change",
-	};
-	int i;
+	const char **p;
 
 	if (nongit) {
 		strbuf_addstr(hook_info,
@@ -81,9 +50,12 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 		return;
 	}
 
-	for (i = 0; i < ARRAY_SIZE(hook); i++)
-		if (hook_exists(hook[i]))
-			strbuf_addf(hook_info, "%s\n", hook[i]);
+	for (p = hook_name_list; *p; p++) {
+		const char *hook = *p;
+
+		if (hook_exists(hook))
+			strbuf_addf(hook_info, "%s\n", hook);
+	}
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt
index 171b4124afe..fd1399c440f 100644
--- a/contrib/buildsystems/CMakeLists.txt
+++ b/contrib/buildsystems/CMakeLists.txt
@@ -624,6 +624,13 @@ if(NOT EXISTS ${CMAKE_BINARY_DIR}/config-list.h)
 			OUTPUT_FILE ${CMAKE_BINARY_DIR}/config-list.h)
 endif()
 
+if(NOT EXISTS ${CMAKE_BINARY_DIR}/hook-list.h)
+	message("Generating hook-list.h")
+	execute_process(COMMAND ${SH_EXE} ${CMAKE_SOURCE_DIR}/generate-hooklist.sh
+			WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
+			OUTPUT_FILE ${CMAKE_BINARY_DIR}/hook-list.h)
+endif()
+
 include_directories(${CMAKE_BINARY_DIR})
 
 #build
diff --git a/generate-hooklist.sh b/generate-hooklist.sh
new file mode 100755
index 00000000000..6d4e56d1a31
--- /dev/null
+++ b/generate-hooklist.sh
@@ -0,0 +1,18 @@
+#!/bin/sh
+#
+# Usage: ./generate-hooklist.sh >hook-list.h
+
+cat <<EOF
+/* Automatically generated by generate-hooklist.sh */
+
+static const char *hook_name_list[] = {
+EOF
+
+sed -n -e '/^~~~~*$/ {x; s/^.*$/	"&",/; p;}; x' \
+	<Documentation/githooks.txt |
+	LC_ALL=C sort
+
+cat <<EOF
+	NULL,
+};
+EOF
diff --git a/hook.c b/hook.c
index 97cd799a320..1f1db1ec9bf 100644
--- a/hook.c
+++ b/hook.c
@@ -1,11 +1,30 @@
 #include "cache.h"
 #include "hook.h"
 #include "run-command.h"
+#include "hook-list.h"
+
+static int known_hook(const char *name)
+{
+	const char **p;
+	size_t len = strlen(name);
+	for (p = hook_name_list; *p; p++) {
+		const char *hook = *p;
+
+		if (!strncmp(name, hook, len) && hook[len] == '\0')
+			return 1;
+	}
+
+	return 0;
+}
 
 const char *find_hook(const char *name)
 {
 	static struct strbuf path = STRBUF_INIT;
 
+	if (!known_hook(name))
+		die(_("the hook '%s' is not known to git, should be in hook-list.h via githooks(5)"),
+		    name);
+
 	strbuf_reset(&path);
 	strbuf_git_path(&path, "hooks/%s", name);
 	if (access(path.buf, X_OK) < 0) {
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 08/36] hook: add 'run' subcommand
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (6 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 07/36] hook-list.h: add a generated list of hooks, like config-list.h Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-04 10:15               ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 09/36] gc: use hook library for pre-auto-gc hook Ævar Arnfjörð Bjarmason
                               ` (31 subsequent siblings)
  39 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

In order to enable hooks to be run as an external process, by a
standalone Git command, or by tools which wrap Git, provide an external
means to run all configured hook commands for a given hook event.

Most of our hooks require more complex functionality than this, but
let's start with the bare minimum required to support our simplest
hooks.

In terms of implementation the usage_with_options() and "goto usage"
pattern here mirrors that of
builtin/{commit-graph,multi-pack-index}.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .gitignore                 |   1 +
 Documentation/git-hook.txt |  38 +++++++++++
 Documentation/githooks.txt |   4 ++
 Makefile                   |   1 +
 builtin.h                  |   1 +
 builtin/hook.c             |  85 ++++++++++++++++++++++++
 command-list.txt           |   1 +
 git.c                      |   1 +
 hook.c                     | 111 +++++++++++++++++++++++++++++++
 hook.h                     |  41 ++++++++++++
 t/t1800-hook.sh            | 133 +++++++++++++++++++++++++++++++++++++
 11 files changed, 417 insertions(+)
 create mode 100644 Documentation/git-hook.txt
 create mode 100644 builtin/hook.c
 create mode 100755 t/t1800-hook.sh

diff --git a/.gitignore b/.gitignore
index 6be9de41ae8..66189ca3cdc 100644
--- a/.gitignore
+++ b/.gitignore
@@ -77,6 +77,7 @@
 /git-grep
 /git-hash-object
 /git-help
+/git-hook
 /git-http-backend
 /git-http-fetch
 /git-http-push
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
new file mode 100644
index 00000000000..660d6a992a0
--- /dev/null
+++ b/Documentation/git-hook.txt
@@ -0,0 +1,38 @@
+git-hook(1)
+===========
+
+NAME
+----
+git-hook - run git hooks
+
+SYNOPSIS
+--------
+[verse]
+'git hook' run <hook-name> [-- <hook-args>]
+
+DESCRIPTION
+-----------
+
+This command is an interface to git hooks (see linkgit:githooks[5]).
+Currently it only provides a convenience wrapper for running hooks for
+use by git itself. In the future it might gain other functionality.
+
+SUBCOMMANDS
+-----------
+
+run::
+	Run the `<hook-name>` hook. See linkgit:githooks[5] for
+	the hook names we support.
++
+Any positional arguments to the hook should be passed after an
+optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
+arguments (if any) differ by hook name, see linkgit:githooks[5] for
+what those are.
+
+SEE ALSO
+--------
+linkgit:githooks[5]
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index b51959ff941..a16e62bc8c8 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -698,6 +698,10 @@ and "0" meaning they were not.
 Only one parameter should be set to "1" when the hook runs.  The hook
 running passing "1", "1" should not be possible.
 
+SEE ALSO
+--------
+linkgit:git-hook[1]
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index 1740b99db00..e68ca41507f 100644
--- a/Makefile
+++ b/Makefile
@@ -1114,6 +1114,7 @@ BUILTIN_OBJS += builtin/get-tar-commit-id.o
 BUILTIN_OBJS += builtin/grep.o
 BUILTIN_OBJS += builtin/hash-object.o
 BUILTIN_OBJS += builtin/help.o
+BUILTIN_OBJS += builtin/hook.o
 BUILTIN_OBJS += builtin/index-pack.o
 BUILTIN_OBJS += builtin/init-db.o
 BUILTIN_OBJS += builtin/interpret-trailers.o
diff --git a/builtin.h b/builtin.h
index 16ecd5586f0..91740c15149 100644
--- a/builtin.h
+++ b/builtin.h
@@ -164,6 +164,7 @@ int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix);
 int cmd_grep(int argc, const char **argv, const char *prefix);
 int cmd_hash_object(int argc, const char **argv, const char *prefix);
 int cmd_help(int argc, const char **argv, const char *prefix);
+int cmd_hook(int argc, const char **argv, const char *prefix);
 int cmd_index_pack(int argc, const char **argv, const char *prefix);
 int cmd_init_db(int argc, const char **argv, const char *prefix);
 int cmd_interpret_trailers(int argc, const char **argv, const char *prefix);
diff --git a/builtin/hook.c b/builtin/hook.c
new file mode 100644
index 00000000000..41dd15550cf
--- /dev/null
+++ b/builtin/hook.c
@@ -0,0 +1,85 @@
+#include "cache.h"
+#include "builtin.h"
+#include "config.h"
+#include "hook.h"
+#include "parse-options.h"
+#include "strbuf.h"
+#include "strvec.h"
+
+#define BUILTIN_HOOK_RUN_USAGE \
+	N_("git hook run <hook-name> [-- <hook-args>]")
+
+static const char * const builtin_hook_usage[] = {
+	BUILTIN_HOOK_RUN_USAGE,
+	NULL
+};
+
+static const char * const builtin_hook_run_usage[] = {
+	BUILTIN_HOOK_RUN_USAGE,
+	NULL
+};
+
+static int run(int argc, const char **argv, const char *prefix)
+{
+	int i;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	const char *hook_name;
+	const char *hook_path;
+	struct option run_options[] = {
+		OPT_END(),
+	};
+	int ret;
+
+	argc = parse_options(argc, argv, prefix, run_options,
+			     builtin_hook_run_usage,
+			     PARSE_OPT_KEEP_DASHDASH);
+
+	if (!argc)
+		goto usage;
+
+	/*
+	 * Having a -- for "run" when providing <hook-args> is
+	 * mandatory.
+	 */
+	if (argc > 1 && strcmp(argv[1], "--") &&
+	    strcmp(argv[1], "--end-of-options"))
+		goto usage;
+
+	/* Add our arguments, start after -- */
+	for (i = 2 ; i < argc; i++)
+		strvec_push(&opt.args, argv[i]);
+
+	/* Need to take into account core.hooksPath */
+	git_config(git_default_config, NULL);
+
+	hook_name = argv[0];
+	hook_path = find_hook(hook_name);
+	if (!hook_path) {
+		error("cannot find a hook named %s", hook_name);
+		return 1;
+	}
+
+	ret = run_hooks(hook_name, hook_path, &opt);
+	run_hooks_opt_clear(&opt);
+	return ret;
+usage:
+	usage_with_options(builtin_hook_run_usage, run_options);
+}
+
+int cmd_hook(int argc, const char **argv, const char *prefix)
+{
+	struct option builtin_hook_options[] = {
+		OPT_END(),
+	};
+
+	argc = parse_options(argc, argv, NULL, builtin_hook_options,
+			     builtin_hook_usage, PARSE_OPT_STOP_AT_NON_OPTION);
+	if (!argc)
+		goto usage;
+
+	if (!strcmp(argv[0], "run"))
+		return run(argc, argv, prefix);
+
+usage:
+	usage_with_options(builtin_hook_usage, builtin_hook_options);
+}
diff --git a/command-list.txt b/command-list.txt
index a289f09ed6f..9ccd8e5aebe 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -103,6 +103,7 @@ git-grep                                mainporcelain           info
 git-gui                                 mainporcelain
 git-hash-object                         plumbingmanipulators
 git-help                                ancillaryinterrogators          complete
+git-hook                                mainporcelain
 git-http-backend                        synchingrepositories
 git-http-fetch                          synchelpers
 git-http-push                           synchelpers
diff --git a/git.c b/git.c
index 18bed9a9964..540909c391f 100644
--- a/git.c
+++ b/git.c
@@ -538,6 +538,7 @@ static struct cmd_struct commands[] = {
 	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
 	{ "hash-object", cmd_hash_object },
 	{ "help", cmd_help },
+	{ "hook", cmd_hook, RUN_SETUP },
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "init", cmd_init_db },
 	{ "init-db", cmd_init_db },
diff --git a/hook.c b/hook.c
index 1f1db1ec9bf..c8ff3975043 100644
--- a/hook.c
+++ b/hook.c
@@ -2,11 +2,14 @@
 #include "hook.h"
 #include "run-command.h"
 #include "hook-list.h"
+#include "config.h"
 
 static int known_hook(const char *name)
 {
 	const char **p;
 	size_t len = strlen(name);
+	static int test_hooks_ok = -1;
+
 	for (p = hook_name_list; *p; p++) {
 		const char *hook = *p;
 
@@ -14,6 +17,14 @@ static int known_hook(const char *name)
 			return 1;
 	}
 
+	if (test_hooks_ok == -1)
+		test_hooks_ok = git_env_bool("GIT_TEST_FAKE_HOOKS", 0);
+
+	if (test_hooks_ok &&
+	    (!strcmp(name, "test-hook") ||
+	     !strcmp(name, "does-not-exist")))
+		return 1;
+
 	return 0;
 }
 
@@ -59,3 +70,103 @@ int hook_exists(const char *name)
 {
 	return !!find_hook(name);
 }
+
+void run_hooks_opt_clear(struct run_hooks_opt *o)
+{
+	strvec_clear(&o->env);
+	strvec_clear(&o->args);
+}
+
+static int pick_next_hook(struct child_process *cp,
+			  struct strbuf *out,
+			  void *pp_cb,
+			  void **pp_task_cb)
+{
+	struct hook_cb_data *hook_cb = pp_cb;
+	struct hook *run_me = hook_cb->run_me;
+
+	if (!run_me)
+		return 0;
+
+	cp->no_stdin = 1;
+	cp->env = hook_cb->options->env.v;
+	cp->stdout_to_stderr = 1;
+	cp->trace2_hook_name = hook_cb->hook_name;
+
+	/* add command */
+	strvec_push(&cp->args, run_me->hook_path);
+
+	/*
+	 * add passed-in argv, without expanding - let the user get back
+	 * exactly what they put in
+	 */
+	strvec_pushv(&cp->args, hook_cb->options->args.v);
+
+	/* Provide context for errors if necessary */
+	*pp_task_cb = run_me;
+
+	/*
+	 * This pick_next_hook() will be called again, we're only
+	 * running one hook, so indicate that no more work will be
+	 * done.
+	 */
+	hook_cb->run_me = NULL;
+
+	return 1;
+}
+
+static int notify_start_failure(struct strbuf *out,
+				void *pp_cb,
+				void *pp_task_cp)
+{
+	struct hook_cb_data *hook_cb = pp_cb;
+	struct hook *attempted = pp_task_cp;
+
+	hook_cb->rc |= 1;
+
+	strbuf_addf(out, _("Couldn't start hook '%s'\n"),
+		    attempted->hook_path);
+
+	return 1;
+}
+
+static int notify_hook_finished(int result,
+				struct strbuf *out,
+				void *pp_cb,
+				void *pp_task_cb)
+{
+	struct hook_cb_data *hook_cb = pp_cb;
+
+	hook_cb->rc |= result;
+
+	return 0;
+}
+
+int run_hooks(const char *hook_name, const char *hook_path,
+	      struct run_hooks_opt *options)
+{
+	struct hook my_hook = {
+		.hook_path = hook_path,
+	};
+	struct hook_cb_data cb_data = {
+		.rc = 0,
+		.hook_name = hook_name,
+		.options = options,
+	};
+	int jobs = 1;
+
+	if (!options)
+		BUG("a struct run_hooks_opt must be provided to run_hooks");
+
+	cb_data.run_me = &my_hook;
+
+	run_processes_parallel_tr2(jobs,
+				   pick_next_hook,
+				   notify_start_failure,
+				   notify_hook_finished,
+				   &cb_data,
+				   "hook",
+				   hook_name);
+
+	return cb_data.rc;
+}
diff --git a/hook.h b/hook.h
index 4c547ac15e5..361984c69d4 100644
--- a/hook.h
+++ b/hook.h
@@ -1,5 +1,8 @@
 #ifndef HOOK_H
 #define HOOK_H
+#include "strbuf.h"
+#include "strvec.h"
+#include "run-command.h"
 
 /*
  * Returns the path to the hook file, or NULL if the hook is missing
@@ -13,4 +16,42 @@ const char *find_hook(const char *name);
  */
 int hook_exists(const char *hookname);
 
+struct hook {
+	/* The path to the hook */
+	const char *hook_path;
+};
+
+struct run_hooks_opt
+{
+	/* Environment vars to be set for each hook */
+	struct strvec env;
+
+	/* Args to be passed to each hook */
+	struct strvec args;
+};
+
+#define RUN_HOOKS_OPT_INIT { \
+	.env = STRVEC_INIT, \
+	.args = STRVEC_INIT, \
+}
+
+/*
+ * Callback provided to feed_pipe_fn and consume_sideband_fn.
+ */
+struct hook_cb_data {
+	/* rc reflects the cumulative failure state */
+	int rc;
+	const char *hook_name;
+	struct hook *run_me;
+	struct run_hooks_opt *options;
+};
+
+void run_hooks_opt_clear(struct run_hooks_opt *o);
+
+/**
+ * Takes an already resolved hook found via find_hook() and runs
+ * it. Does not call run_hooks_opt_clear() for you.
+ */
+int run_hooks(const char *hookname, const char *hook_path,
+	      struct run_hooks_opt *options);
 #endif
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
new file mode 100755
index 00000000000..644df0a583c
--- /dev/null
+++ b/t/t1800-hook.sh
@@ -0,0 +1,133 @@
+#!/bin/bash
+
+test_description='git-hook command'
+
+. ./test-lib.sh
+
+test_expect_success 'git hook usage' '
+	test_expect_code 129 git hook &&
+	test_expect_code 129 git hook run &&
+	test_expect_code 129 git hook run -h &&
+	test_expect_code 129 git hook run --unknown 2>err &&
+	grep "unknown option" err
+'
+
+test_expect_success 'setup GIT_TEST_FAKE_HOOKS=true to permit "test-hook" and "does-not-exist" names"' '
+	GIT_TEST_FAKE_HOOKS=true &&
+	export GIT_TEST_FAKE_HOOKS
+'
+
+test_expect_success 'git hook run: nonexistent hook' '
+	cat >stderr.expect <<-\EOF &&
+	error: cannot find a hook named test-hook
+	EOF
+	test_expect_code 1 git hook run test-hook 2>stderr.actual &&
+	test_cmp stderr.expect stderr.actual
+'
+
+test_expect_success 'git hook run: basic' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo Test hook
+	EOF
+
+	cat >expect <<-\EOF &&
+	Test hook
+	EOF
+	git hook run test-hook 2>actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git hook run: stdout and stderr both write to our stderr' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo >&1 Will end up on stderr
+	echo >&2 Will end up on stderr
+	EOF
+
+	cat >stderr.expect <<-\EOF &&
+	Will end up on stderr
+	Will end up on stderr
+	EOF
+	git hook run test-hook >stdout.actual 2>stderr.actual &&
+	test_cmp stderr.expect stderr.actual &&
+	test_must_be_empty stdout.actual
+'
+
+test_expect_success 'git hook run: exit codes are passed along' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	exit 1
+	EOF
+
+	test_expect_code 1 git hook run test-hook &&
+
+	write_script .git/hooks/test-hook <<-EOF &&
+	exit 2
+	EOF
+
+	test_expect_code 2 git hook run test-hook &&
+
+	write_script .git/hooks/test-hook <<-EOF &&
+	exit 128
+	EOF
+
+	test_expect_code 128 git hook run test-hook &&
+
+	write_script .git/hooks/test-hook <<-EOF &&
+	exit 129
+	EOF
+
+	test_expect_code 129 git hook run test-hook
+'
+
+test_expect_success 'git hook run arg u ments without -- is not allowed' '
+	test_expect_code 129 git hook run test-hook arg u ments
+'
+
+test_expect_success 'git hook run -- pass arguments' '
+	write_script .git/hooks/test-hook <<-\EOF &&
+	echo $1
+	echo $2
+	EOF
+
+	cat >expect <<-EOF &&
+	arg
+	u ments
+	EOF
+
+	git hook run test-hook -- arg "u ments" 2>actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git hook run -- out-of-repo runs excluded' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo Test hook
+	EOF
+
+	nongit test_must_fail git hook run test-hook
+'
+
+test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
+	mkdir my-hooks &&
+	write_script my-hooks/test-hook <<-\EOF &&
+	echo Hook ran $1 >>actual
+	EOF
+
+	cat >expect <<-\EOF &&
+	Test hook
+	Hook ran one
+	Hook ran two
+	Hook ran three
+	Hook ran four
+	EOF
+
+	# Test various ways of specifying the path. See also
+	# t1350-config-hooks-path.sh
+	>actual &&
+	git hook run test-hook -- ignored 2>>actual &&
+	git -c core.hooksPath=my-hooks hook run test-hook -- one 2>>actual &&
+	git -c core.hooksPath=my-hooks/ hook run test-hook -- two 2>>actual &&
+	git -c core.hooksPath="$PWD/my-hooks" hook run test-hook -- three 2>>actual &&
+	git -c core.hooksPath="$PWD/my-hooks/" hook run test-hook -- four 2>>actual &&
+	test_cmp expect actual
+'
+
+test_done
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 09/36] gc: use hook library for pre-auto-gc hook
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (7 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 08/36] hook: add 'run' subcommand Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 10/36] rebase: convert pre-rebase to use hook.h Ævar Arnfjörð Bjarmason
                               ` (30 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Move the pre-auto-gc hook away from run-command.h to and over to the
new hook.h library.

To do this introduce a simple run_hooks_oneshot() wrapper, we'll be
using it extensively for these simple cases of wanting to run a single
hook under a given name, and having it free the memory we allocate for
us.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/gc.c |  3 ++-
 hook.c       | 21 +++++++++++++++++++++
 hook.h       | 13 +++++++++++++
 3 files changed, 36 insertions(+), 1 deletion(-)

diff --git a/builtin/gc.c b/builtin/gc.c
index f05d2f0a1ac..2f74cf394d3 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -32,6 +32,7 @@
 #include "remote.h"
 #include "object-store.h"
 #include "exec-cmd.h"
+#include "hook.h"
 
 #define FAILED_RUN "failed to run %s"
 
@@ -394,7 +395,7 @@ static int need_to_gc(void)
 	else
 		return 0;
 
-	if (run_hook_le(NULL, "pre-auto-gc", NULL))
+	if (run_hooks_oneshot("pre-auto-gc", NULL))
 		return 0;
 	return 1;
 }
diff --git a/hook.c b/hook.c
index c8ff3975043..981a9bf46e6 100644
--- a/hook.c
+++ b/hook.c
@@ -170,3 +170,24 @@ int run_hooks(const char *hook_name, const char *hook_path,
 
 	return cb_data.rc;
 }
+
+int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
+{
+	const char *hook_path;
+	int ret;
+	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
+
+	if (!options)
+		options = &hook_opt_scratch;
+
+	hook_path = find_hook(hook_name);
+	if (!hook_path) {
+		ret = 0;
+		goto cleanup;
+	}
+
+	ret = run_hooks(hook_name, hook_path, options);
+cleanup:
+	run_hooks_opt_clear(options);
+	return ret;
+}
diff --git a/hook.h b/hook.h
index 361984c69d4..2201ab9d568 100644
--- a/hook.h
+++ b/hook.h
@@ -51,7 +51,20 @@ void run_hooks_opt_clear(struct run_hooks_opt *o);
 /**
  * Takes an already resolved hook found via find_hook() and runs
  * it. Does not call run_hooks_opt_clear() for you.
+ *
+ * See run_hooks_oneshot() for the simpler one-shot API.
  */
 int run_hooks(const char *hookname, const char *hook_path,
 	      struct run_hooks_opt *options);
+
+/**
+ * Calls find_hook() on your "hook_name" and runs the hooks (if any)
+ * with run_hooks().
+ *
+ * If "options" is provided calls run_hooks_opt_clear() on it for
+ * you. If "options" is NULL a scratch one will be provided for you
+ * before calling run_hooks().
+ */
+int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
+
 #endif
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 10/36] rebase: convert pre-rebase to use hook.h
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (8 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 09/36] gc: use hook library for pre-auto-gc hook Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 11/36] am: convert applypatch " Ævar Arnfjörð Bjarmason
                               ` (29 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Move the pre-rebase hook away from run-command.h to and over to the
new hook.h library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/rebase.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/builtin/rebase.c b/builtin/rebase.c
index 12f093121d9..e7c668c99b1 100644
--- a/builtin/rebase.c
+++ b/builtin/rebase.c
@@ -28,6 +28,7 @@
 #include "sequencer.h"
 #include "rebase-interactive.h"
 #include "reset.h"
+#include "hook.h"
 
 #define DEFAULT_REFLOG_ACTION "rebase"
 
@@ -1313,6 +1314,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	char *squash_onto_name = NULL;
 	int reschedule_failed_exec = -1;
 	int allow_preemptive_ff = 1;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
 	struct option builtin_rebase_options[] = {
 		OPT_STRING(0, "onto", &options.onto_name,
 			   N_("revision"),
@@ -2022,9 +2024,9 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	}
 
 	/* If a hook exists, give it a chance to interrupt*/
+	strvec_pushl(&hook_opt.args, options.upstream_arg, argc ? argv[0] : NULL, NULL);
 	if (!ok_to_skip_pre_rebase &&
-	    run_hook_le(NULL, "pre-rebase", options.upstream_arg,
-			argc ? argv[0] : NULL, NULL))
+	    run_hooks_oneshot("pre-rebase", &hook_opt))
 		die(_("The pre-rebase hook refused to rebase."));
 
 	if (options.flags & REBASE_DIFFSTAT) {
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 11/36] am: convert applypatch to use hook.h
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (9 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 10/36] rebase: convert pre-rebase to use hook.h Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 12/36] hooks: convert 'post-checkout' hook to hook library Ævar Arnfjörð Bjarmason
                               ` (28 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Teach pre-applypatch, post-applypatch, and applypatch-msg to use the
hook.h library instead of the run-command.h library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/am.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index c603f3cebdf..e444b18b64a 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -446,9 +446,11 @@ static void am_destroy(const struct am_state *state)
 static int run_applypatch_msg_hook(struct am_state *state)
 {
 	int ret;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 
 	assert(state->msg);
-	ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL);
+	strvec_push(&opt.args, am_path(state, "final-commit"));
+	ret = run_hooks_oneshot("applypatch-msg", &opt);
 
 	if (!ret) {
 		FREE_AND_NULL(state->msg);
@@ -1609,7 +1611,7 @@ static void do_commit(const struct am_state *state)
 	const char *reflog_msg, *author, *committer = NULL;
 	struct strbuf sb = STRBUF_INIT;
 
-	if (run_hook_le(NULL, "pre-applypatch", NULL))
+	if (run_hooks_oneshot("pre-applypatch", NULL))
 		exit(1);
 
 	if (write_cache_as_tree(&tree, 0, NULL))
@@ -1661,7 +1663,7 @@ static void do_commit(const struct am_state *state)
 		fclose(fp);
 	}
 
-	run_hook_le(NULL, "post-applypatch", NULL);
+	run_hooks_oneshot("post-applypatch", NULL);
 
 	strbuf_release(&sb);
 }
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 12/36] hooks: convert 'post-checkout' hook to hook library
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (10 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 11/36] am: convert applypatch " Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 13/36] merge: convert post-merge to use hook.h Ævar Arnfjörð Bjarmason
                               ` (27 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Move the running of the 'post-checkout' hook away from run-command.h
to the new hook.h library. For "worktree" this requires a change to it
to run the hooks from a given directory.

We could strictly speaking skip the "absolute_path" flag and just
check if "dir" is specified, but let's split them up for clarity, as
well as for any future user who'd like to set "dir" but not implicitly
change the argument to an absolute path.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/checkout.c | 14 +++++++++-----
 builtin/clone.c    |  6 ++++--
 builtin/worktree.c | 28 ++++++++++++----------------
 hook.c             |  9 +++++++++
 hook.h             |  8 ++++++++
 read-cache.c       |  1 +
 reset.c            | 14 ++++++++++----
 7 files changed, 53 insertions(+), 27 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index f4cd7747d35..6d69b4c0113 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "diff.h"
 #include "dir.h"
+#include "hook.h"
 #include "ll-merge.h"
 #include "lockfile.h"
 #include "merge-recursive.h"
@@ -106,13 +107,16 @@ struct branch_info {
 static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit,
 			      int changed)
 {
-	return run_hook_le(NULL, "post-checkout",
-			   oid_to_hex(old_commit ? &old_commit->object.oid : null_oid()),
-			   oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()),
-			   changed ? "1" : "0", NULL);
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
 	/* "new_commit" can be NULL when checking out from the index before
 	   a commit exists. */
-
+	strvec_pushl(&opt.args,
+		     oid_to_hex(old_commit ? &old_commit->object.oid : null_oid()),
+		     oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()),
+		     changed ? "1" : "0",
+		     NULL);
+	return run_hooks_oneshot("post-checkout", &opt);
 }
 
 static int update_some(const struct object_id *oid, struct strbuf *base,
diff --git a/builtin/clone.c b/builtin/clone.c
index 66fe66679c8..27fc05ee511 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -32,6 +32,7 @@
 #include "connected.h"
 #include "packfile.h"
 #include "list-objects-filter-options.h"
+#include "hook.h"
 
 /*
  * Overall FIXMEs:
@@ -775,6 +776,7 @@ static int checkout(int submodule_progress)
 	struct tree *tree;
 	struct tree_desc t;
 	int err = 0;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
 
 	if (option_no_checkout)
 		return 0;
@@ -820,8 +822,8 @@ static int checkout(int submodule_progress)
 	if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
 		die(_("unable to write new index file"));
 
-	err |= run_hook_le(NULL, "post-checkout", oid_to_hex(null_oid()),
-			   oid_to_hex(&oid), "1", NULL);
+	strvec_pushl(&hook_opt.args, oid_to_hex(null_oid()), oid_to_hex(&oid), "1", NULL);
+	err |= run_hooks_oneshot("post-checkout", &hook_opt);
 
 	if (!err && (option_recurse_submodules.nr > 0)) {
 		struct strvec args = STRVEC_INIT;
diff --git a/builtin/worktree.c b/builtin/worktree.c
index d22ece93e1a..330867c19bf 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -382,22 +382,18 @@ static int add_worktree(const char *path, const char *refname,
 	 * is_junk is cleared, but do return appropriate code when hook fails.
 	 */
 	if (!ret && opts->checkout) {
-		const char *hook = find_hook("post-checkout");
-		if (hook) {
-			const char *env[] = { "GIT_DIR", "GIT_WORK_TREE", NULL };
-			cp.git_cmd = 0;
-			cp.no_stdin = 1;
-			cp.stdout_to_stderr = 1;
-			cp.dir = path;
-			cp.env = env;
-			cp.argv = NULL;
-			cp.trace2_hook_name = "post-checkout";
-			strvec_pushl(&cp.args, absolute_path(hook),
-				     oid_to_hex(null_oid()),
-				     oid_to_hex(&commit->object.oid),
-				     "1", NULL);
-			ret = run_command(&cp);
-		}
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
+		strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
+		strvec_pushl(&opt.args,
+			     oid_to_hex(null_oid()),
+			     oid_to_hex(&commit->object.oid),
+			     "1",
+			     NULL);
+		opt.dir = path;
+		opt.absolute_path = 1;
+
+		ret = run_hooks_oneshot("post-checkout", &opt);
 	}
 
 	strvec_clear(&child_env);
diff --git a/hook.c b/hook.c
index 981a9bf46e6..5ecf79c8397 100644
--- a/hook.c
+++ b/hook.c
@@ -92,6 +92,7 @@ static int pick_next_hook(struct child_process *cp,
 	cp->env = hook_cb->options->env.v;
 	cp->stdout_to_stderr = 1;
 	cp->trace2_hook_name = hook_cb->hook_name;
+	cp->dir = hook_cb->options->dir;
 
 	/* add command */
 	strvec_push(&cp->args, run_me->hook_path);
@@ -145,6 +146,7 @@ static int notify_hook_finished(int result,
 int run_hooks(const char *hook_name, const char *hook_path,
 	      struct run_hooks_opt *options)
 {
+	struct strbuf abs_path = STRBUF_INIT;
 	struct hook my_hook = {
 		.hook_path = hook_path,
 	};
@@ -158,6 +160,10 @@ int run_hooks(const char *hook_name, const char *hook_path,
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
 
+	if (options->absolute_path) {
+		strbuf_add_absolute_path(&abs_path, hook_path);
+		my_hook.hook_path = abs_path.buf;
+	}
 	cb_data.run_me = &my_hook;
 
 	run_processes_parallel_tr2(jobs,
@@ -168,6 +174,9 @@ int run_hooks(const char *hook_name, const char *hook_path,
 				   "hook",
 				   hook_name);
 
+	if (options->absolute_path)
+		strbuf_release(&abs_path);
+
 	return cb_data.rc;
 }
 
diff --git a/hook.h b/hook.h
index 2201ab9d568..53ea3a9649e 100644
--- a/hook.h
+++ b/hook.h
@@ -28,6 +28,14 @@ struct run_hooks_opt
 
 	/* Args to be passed to each hook */
 	struct strvec args;
+
+	/* Resolve and run the "absolute_path(hook)" instead of
+	 * "hook". Used for "git worktree" hooks
+	 */
+	int absolute_path;
+
+	/* Path to initial working directory for subprocess */
+	const char *dir;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
diff --git a/read-cache.c b/read-cache.c
index 99a174b91e6..c9e2b013972 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -28,6 +28,7 @@
 #include "sparse-index.h"
 #include "csum-file.h"
 #include "promisor-remote.h"
+#include "hook.h"
 
 /* Mask for the name length in ce_flags in the on-disk index */
 
diff --git a/reset.c b/reset.c
index 4bea758053b..6499bc5127d 100644
--- a/reset.c
+++ b/reset.c
@@ -7,6 +7,7 @@
 #include "tree-walk.h"
 #include "tree.h"
 #include "unpack-trees.h"
+#include "hook.h"
 
 int reset_head(struct repository *r, struct object_id *oid, const char *action,
 	       const char *switch_to_branch, unsigned flags,
@@ -126,10 +127,15 @@ int reset_head(struct repository *r, struct object_id *oid, const char *action,
 			ret = create_symref("HEAD", switch_to_branch,
 					    reflog_head);
 	}
-	if (run_hook)
-		run_hook_le(NULL, "post-checkout",
-			    oid_to_hex(orig ? orig : null_oid()),
-			    oid_to_hex(oid), "1", NULL);
+	if (run_hook) {
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		strvec_pushl(&opt.args,
+			     oid_to_hex(orig ? orig : null_oid()),
+			     oid_to_hex(oid),
+			     "1",
+			     NULL);
+		run_hooks_oneshot("post-checkout", &opt);
+	}
 
 leave_reset_head:
 	strbuf_release(&msg);
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 13/36] merge: convert post-merge to use hook.h
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (11 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 12/36] hooks: convert 'post-checkout' hook to hook library Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 14/36] git hook run: add an --ignore-missing flag Ævar Arnfjörð Bjarmason
                               ` (26 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Teach post-merge to use the hook.h library instead of the
run-command.h library to run hooks.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/merge.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/builtin/merge.c b/builtin/merge.c
index 03f244dd5a0..4965df2ac29 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -448,6 +448,7 @@ static void finish(struct commit *head_commit,
 		   const struct object_id *new_head, const char *msg)
 {
 	struct strbuf reflog_message = STRBUF_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	const struct object_id *head = &head_commit->object.oid;
 
 	if (!msg)
@@ -489,7 +490,8 @@ static void finish(struct commit *head_commit,
 	}
 
 	/* Run a post-merge hook */
-	run_hook_le(NULL, "post-merge", squash ? "1" : "0", NULL);
+	strvec_push(&opt.args, squash ? "1" : "0");
+	run_hooks_oneshot("post-merge", &opt);
 
 	apply_autostash(git_path_merge_autostash(the_repository));
 	strbuf_release(&reflog_message);
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 14/36] git hook run: add an --ignore-missing flag
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (12 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 13/36] merge: convert post-merge to use hook.h Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 15/36] send-email: use 'git hook run' for 'sendemail-validate' Ævar Arnfjörð Bjarmason
                               ` (25 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

For certain one-shot hooks we'd like to optimistically run them, and
not complain if they don't exist. This will be used by send-email in a
subsequent commit.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-hook.txt | 10 +++++++++-
 builtin/hook.c             |  7 ++++++-
 t/t1800-hook.sh            |  5 +++++
 3 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 660d6a992a0..097fb9de63b 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -8,7 +8,7 @@ git-hook - run git hooks
 SYNOPSIS
 --------
 [verse]
-'git hook' run <hook-name> [-- <hook-args>]
+'git hook' run [--ignore-missing] <hook-name> [-- <hook-args>]
 
 DESCRIPTION
 -----------
@@ -29,6 +29,14 @@ optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
 arguments (if any) differ by hook name, see linkgit:githooks[5] for
 what those are.
 
+OPTIONS
+-------
+
+--ignore-missing::
+	Ignore any missing hook by quietly returning zero. Used for
+	tools that want to do a blind one-shot run of a hook that may
+	or may not be present.
+
 SEE ALSO
 --------
 linkgit:githooks[5]
diff --git a/builtin/hook.c b/builtin/hook.c
index 41dd15550cf..f33db9953c7 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -7,7 +7,7 @@
 #include "strvec.h"
 
 #define BUILTIN_HOOK_RUN_USAGE \
-	N_("git hook run <hook-name> [-- <hook-args>]")
+	N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")
 
 static const char * const builtin_hook_usage[] = {
 	BUILTIN_HOOK_RUN_USAGE,
@@ -23,9 +23,12 @@ static int run(int argc, const char **argv, const char *prefix)
 {
 	int i;
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	int ignore_missing = 0;
 	const char *hook_name;
 	const char *hook_path;
 	struct option run_options[] = {
+		OPT_BOOL(0, "ignore-missing", &ignore_missing,
+			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
 		OPT_END(),
 	};
 	int ret;
@@ -53,6 +56,8 @@ static int run(int argc, const char **argv, const char *prefix)
 	git_config(git_default_config, NULL);
 
 	hook_name = argv[0];
+	if (ignore_missing)
+		return run_hooks_oneshot(hook_name, &opt);
 	hook_path = find_hook(hook_name);
 	if (!hook_path) {
 		error("cannot find a hook named %s", hook_name);
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
index 644df0a583c..49df5a2cdfb 100755
--- a/t/t1800-hook.sh
+++ b/t/t1800-hook.sh
@@ -25,6 +25,11 @@ test_expect_success 'git hook run: nonexistent hook' '
 	test_cmp stderr.expect stderr.actual
 '
 
+test_expect_success 'git hook run: nonexistent hook with --ignore-missing' '
+	git hook run --ignore-missing does-not-exist 2>stderr.actual &&
+	test_must_be_empty stderr.actual
+'
+
 test_expect_success 'git hook run: basic' '
 	write_script .git/hooks/test-hook <<-EOF &&
 	echo Test hook
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 15/36] send-email: use 'git hook run' for 'sendemail-validate'
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (13 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 14/36] git hook run: add an --ignore-missing flag Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 16/36] git-p4: use 'git hook' to run hooks Ævar Arnfjörð Bjarmason
                               ` (24 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Change the "sendmail-validate" hook to be run via the "git hook run"
wrapper instead of via a direct invocation.

This is the smallest possibly change to get "send-email" using "git
hook run". We still check the hook itself with "-x", and set a
"GIT_DIR" variable, both of which are asserted by our tests. We'll
need to get rid of this special behavior if we start running N hooks,
but for now let's be as close to bug-for-bug compatible as possible.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-send-email.perl   | 20 ++++++++++++--------
 t/t9001-send-email.sh |  4 ++--
 2 files changed, 14 insertions(+), 10 deletions(-)

diff --git a/git-send-email.perl b/git-send-email.perl
index e65d969d0bb..126850d974b 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -195,13 +195,13 @@ sub format_2822_time {
 my $editor;
 
 sub system_or_msg {
-	my ($args, $msg) = @_;
+	my ($args, $msg, $cmd_name) = @_;
 	system(@$args);
 	my $signalled = $? & 127;
 	my $exit_code = $? >> 8;
 	return unless $signalled or $exit_code;
 
-	my @sprintf_args = ($args->[0], $exit_code);
+	my @sprintf_args = ($cmd_name ? $cmd_name : $args->[0], $exit_code);
 	if (defined $msg) {
 		# Quiet the 'redundant' warning category, except we
 		# need to support down to Perl 5.8, so we can't do a
@@ -2031,10 +2031,10 @@ sub validate_patch {
 	my ($fn, $xfer_encoding) = @_;
 
 	if ($repo) {
+		my $hook_name = 'sendemail-validate';
 		my $hooks_path = $repo->command_oneline('rev-parse', '--git-path', 'hooks');
 		require File::Spec;
-		my $validate_hook = File::Spec->catfile($hooks_path,
-					    'sendemail-validate');
+		my $validate_hook = File::Spec->catfile($hooks_path, $hook_name);
 		my $hook_error;
 		if (-x $validate_hook) {
 			require Cwd;
@@ -2044,13 +2044,17 @@ sub validate_patch {
 			chdir($repo->wc_path() or $repo->repo_path())
 				or die("chdir: $!");
 			local $ENV{"GIT_DIR"} = $repo->repo_path();
-			$hook_error = system_or_msg([$validate_hook, $target]);
+			my @validate_hook = ("git", "hook", "run", "--ignore-missing", $hook_name, "--", $target);
+			$hook_error = system_or_msg(\@validate_hook, undef,
+						       "git hook run $hook_name -- <patch>");
 			chdir($cwd_save) or die("chdir: $!");
 		}
 		if ($hook_error) {
-			die sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
-				       "%s\n" .
-				       "warning: no patches were sent\n"), $fn, $hook_error);
+			$hook_error = sprintf(__("fatal: %s: rejected by %s hook\n" .
+						 $hook_error . "\n" .
+						 "warning: no patches were sent\n"),
+					      $fn, $hook_name);
+			die $hook_error;
 		}
 	}
 
diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index 57fc10e7f82..9ec7d75f0ff 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -539,7 +539,7 @@ test_expect_success $PREREQ "--validate respects relative core.hooksPath path" '
 	test_path_is_file my-hooks.ran &&
 	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
-	fatal: command '"'"'my-hooks/sendemail-validate'"'"' died with exit code 1
+	fatal: command '"'"'git hook run sendemail-validate -- <patch>'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
@@ -558,7 +558,7 @@ test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" '
 	test_path_is_file my-hooks.ran &&
 	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
-	fatal: command '"'"'$hooks_path/sendemail-validate'"'"' died with exit code 1
+	fatal: command '"'"'git hook run sendemail-validate -- <patch>'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 16/36] git-p4: use 'git hook' to run hooks
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (14 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 15/36] send-email: use 'git hook run' for 'sendemail-validate' Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 17/36] commit: convert {pre-commit,prepare-commit-msg} hook to hook.h Ævar Arnfjörð Bjarmason
                               ` (23 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Instead of duplicating the behavior of run-command.h:run_hook_le() in
Python, we can directly call 'git hook run'. We emulate the existence
check with the --ignore-missing flag.

As this is the last hook execution in git.git to not go through "git
hook run" or the hook.[ch] library we can now be absolutely sure that
our assertion in hook.c that only hooks known by the generated (from
githooks(5)) hook-list.h are permitted.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-p4.py | 72 ++++++-------------------------------------------------
 1 file changed, 7 insertions(+), 65 deletions(-)

diff --git a/git-p4.py b/git-p4.py
index 2b4500226aa..1f24cbf0bca 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -207,71 +207,13 @@ def decode_path(path):
         return path
 
 def run_git_hook(cmd, param=[]):
-    """Execute a hook if the hook exists."""
-    if verbose:
-        sys.stderr.write("Looking for hook: %s\n" % cmd)
-        sys.stderr.flush()
-
-    hooks_path = gitConfig("core.hooksPath")
-    if len(hooks_path) <= 0:
-        hooks_path = os.path.join(os.environ["GIT_DIR"], "hooks")
-
-    if not isinstance(param, list):
-        param=[param]
-
-    # resolve hook file name, OS depdenent
-    hook_file = os.path.join(hooks_path, cmd)
-    if platform.system() == 'Windows':
-        if not os.path.isfile(hook_file):
-            # look for the file with an extension
-            files = glob.glob(hook_file + ".*")
-            if not files:
-                return True
-            files.sort()
-            hook_file = files.pop()
-            while hook_file.upper().endswith(".SAMPLE"):
-                # The file is a sample hook. We don't want it
-                if len(files) > 0:
-                    hook_file = files.pop()
-                else:
-                    return True
-
-    if not os.path.isfile(hook_file) or not os.access(hook_file, os.X_OK):
-        return True
-
-    return run_hook_command(hook_file, param) == 0
-
-def run_hook_command(cmd, param):
-    """Executes a git hook command
-       cmd = the command line file to be executed. This can be
-       a file that is run by OS association.
-
-       param = a list of parameters to pass to the cmd command
-
-       On windows, the extension is checked to see if it should
-       be run with the Git for Windows Bash shell.  If there
-       is no file extension, the file is deemed a bash shell
-       and will be handed off to sh.exe. Otherwise, Windows
-       will be called with the shell to handle the file assocation.
-
-       For non Windows operating systems, the file is called
-       as an executable.
-    """
-    cli = [cmd] + param
-    use_shell = False
-    if platform.system() == 'Windows':
-        (root,ext) = os.path.splitext(cmd)
-        if ext == "":
-            exe_path = os.environ.get("EXEPATH")
-            if exe_path is None:
-                exe_path = ""
-            else:
-                exe_path = os.path.join(exe_path, "bin")
-            cli = [os.path.join(exe_path, "SH.EXE")] + cli
-        else:
-            use_shell = True
-    return subprocess.call(cli, shell=use_shell)
-
+    """args are specified with -a <arg> -a <arg> -a <arg>"""
+    args = ['git', 'hook', 'run', '--ignore-missing', cmd]
+    if param:
+        args.append("--")
+        for p in param:
+            args.append(p)
+    return subprocess.call(args) == 0
 
 def write_pipe(c, stdin):
     if verbose:
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 17/36] commit: convert {pre-commit,prepare-commit-msg} hook to hook.h
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (15 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 16/36] git-p4: use 'git hook' to run hooks Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 18/36] read-cache: convert post-index-change to use hook.h Ævar Arnfjörð Bjarmason
                               ` (22 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Move these hooks hook away from run-command.h to and over to the new
hook.h library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 commit.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/commit.c b/commit.c
index 143f472c0f2..63d7943a86d 100644
--- a/commit.c
+++ b/commit.c
@@ -21,6 +21,7 @@
 #include "commit-reach.h"
 #include "run-command.h"
 #include "shallow.h"
+#include "hook.h"
 
 static struct commit_extra_header *read_commit_extra_header_lines(const char *buf, size_t len, const char **);
 
@@ -1698,22 +1699,22 @@ size_t ignore_non_trailer(const char *buf, size_t len)
 int run_commit_hook(int editor_is_used, const char *index_file,
 		    const char *name, ...)
 {
-	struct strvec hook_env = STRVEC_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	va_list args;
-	int ret;
+	const char *arg;
 
-	strvec_pushf(&hook_env, "GIT_INDEX_FILE=%s", index_file);
+	strvec_pushf(&opt.env, "GIT_INDEX_FILE=%s", index_file);
 
 	/*
 	 * Let the hook know that no editor will be launched.
 	 */
 	if (!editor_is_used)
-		strvec_push(&hook_env, "GIT_EDITOR=:");
+		strvec_push(&opt.env, "GIT_EDITOR=:");
 
 	va_start(args, name);
-	ret = run_hook_ve(hook_env.v, name, args);
+	while ((arg = va_arg(args, const char *)))
+		strvec_push(&opt.args, arg);
 	va_end(args);
-	strvec_clear(&hook_env);
 
-	return ret;
+	return run_hooks_oneshot(name, &opt);
 }
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 18/36] read-cache: convert post-index-change to use hook.h
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (16 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 17/36] commit: convert {pre-commit,prepare-commit-msg} hook to hook.h Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 19/36] receive-pack: convert push-to-checkout hook to hook.h Ævar Arnfjörð Bjarmason
                               ` (21 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Move the post-index-change hook away from run-command.h to and over to
the new hook.h library.

This removes the last direct user of run_hook_ve(), so we can make the
function static now. It'll be removed entirely soon.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 read-cache.c  | 10 +++++++---
 run-command.c |  2 +-
 run-command.h |  1 -
 3 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/read-cache.c b/read-cache.c
index c9e2b013972..90099ca14df 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -3068,6 +3068,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 {
 	int ret;
 	int was_full = !istate->sparse_index;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
 
 	ret = convert_to_sparse(istate);
 
@@ -3096,9 +3097,12 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 	else
 		ret = close_lock_file_gently(lock);
 
-	run_hook_le(NULL, "post-index-change",
-			istate->updated_workdir ? "1" : "0",
-			istate->updated_skipworktree ? "1" : "0", NULL);
+	strvec_pushl(&hook_opt.args,
+		     istate->updated_workdir ? "1" : "0",
+		     istate->updated_skipworktree ? "1" : "0",
+		     NULL);
+	run_hooks_oneshot("post-index-change", &hook_opt);
+
 	istate->updated_workdir = 0;
 	istate->updated_skipworktree = 0;
 
diff --git a/run-command.c b/run-command.c
index 352f5be1646..b4341ba1c7b 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1320,7 +1320,7 @@ int async_with_fork(void)
 #endif
 }
 
-int run_hook_ve(const char *const *env, const char *name, va_list args)
+static int run_hook_ve(const char *const *env, const char *name, va_list args)
 {
 	struct child_process hook = CHILD_PROCESS_INIT;
 	const char *p;
diff --git a/run-command.h b/run-command.h
index f76b740f927..7a867d41217 100644
--- a/run-command.h
+++ b/run-command.h
@@ -219,7 +219,6 @@ int run_command(struct child_process *);
  */
 LAST_ARG_MUST_BE_NULL
 int run_hook_le(const char *const *env, const char *name, ...);
-int run_hook_ve(const char *const *env, const char *name, va_list args);
 
 /*
  * Trigger an auto-gc
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 19/36] receive-pack: convert push-to-checkout hook to hook.h
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (17 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 18/36] read-cache: convert post-index-change to use hook.h Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 20/36] run-command: remove old run_hook_{le,ve}() hook API Ævar Arnfjörð Bjarmason
                               ` (20 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Move the push-to-checkout hook away from run-command.h to and over to
the new hook.h library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/receive-pack.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 91fa799b66e..a7d03bbc7d3 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1435,9 +1435,12 @@ static const char *push_to_checkout(unsigned char *hash,
 				    struct strvec *env,
 				    const char *work_tree)
 {
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
-	if (run_hook_le(env->v, push_to_checkout_hook,
-			hash_to_hex(hash), NULL))
+	strvec_pushv(&opt.env, env->v);
+	strvec_push(&opt.args, hash_to_hex(hash));
+	if (run_hooks_oneshot(push_to_checkout_hook, &opt))
 		return "push-to-checkout hook declined";
 	else
 		return NULL;
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 20/36] run-command: remove old run_hook_{le,ve}() hook API
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (18 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 19/36] receive-pack: convert push-to-checkout hook to hook.h Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 21/36] run-command: allow stdin for run_processes_parallel Ævar Arnfjörð Bjarmason
                               ` (19 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

The new hook.h library has replaced all run-command.h hook-related
functionality. So let's delete this dead code.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 run-command.c | 32 --------------------------------
 run-command.h | 16 ----------------
 2 files changed, 48 deletions(-)

diff --git a/run-command.c b/run-command.c
index b4341ba1c7b..1399243de8a 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1320,38 +1320,6 @@ int async_with_fork(void)
 #endif
 }
 
-static int run_hook_ve(const char *const *env, const char *name, va_list args)
-{
-	struct child_process hook = CHILD_PROCESS_INIT;
-	const char *p;
-
-	p = find_hook(name);
-	if (!p)
-		return 0;
-
-	strvec_push(&hook.args, p);
-	while ((p = va_arg(args, const char *)))
-		strvec_push(&hook.args, p);
-	hook.env = env;
-	hook.no_stdin = 1;
-	hook.stdout_to_stderr = 1;
-	hook.trace2_hook_name = name;
-
-	return run_command(&hook);
-}
-
-int run_hook_le(const char *const *env, const char *name, ...)
-{
-	va_list args;
-	int ret;
-
-	va_start(args, name);
-	ret = run_hook_ve(env, name, args);
-	va_end(args);
-
-	return ret;
-}
-
 struct io_pump {
 	/* initialized by caller */
 	int fd;
diff --git a/run-command.h b/run-command.h
index 7a867d41217..cfb6887e4ae 100644
--- a/run-command.h
+++ b/run-command.h
@@ -204,22 +204,6 @@ int finish_command_in_signal(struct child_process *);
  */
 int run_command(struct child_process *);
 
-/**
- * Run a hook.
- * The first argument is a pathname to an index file, or NULL
- * if the hook uses the default index file or no index is needed.
- * The second argument is the name of the hook.
- * The further arguments correspond to the hook arguments.
- * The last argument has to be NULL to terminate the arguments list.
- * If the hook does not exist or is not executable, the return
- * value will be zero.
- * If it is executable, the hook will be executed and the exit
- * status of the hook is returned.
- * On execution, .stdout_to_stderr and .no_stdin will be set.
- */
-LAST_ARG_MUST_BE_NULL
-int run_hook_le(const char *const *env, const char *name, ...);
-
 /*
  * Trigger an auto-gc
  */
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 21/36] run-command: allow stdin for run_processes_parallel
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (19 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 20/36] run-command: remove old run_hook_{le,ve}() hook API Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 22/36] hook: support passing stdin to hooks Ævar Arnfjörð Bjarmason
                               ` (18 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

While it makes sense not to inherit stdin from the parent process to
avoid deadlocking, it's not necessary to completely ban stdin to
children. An informed user should be able to configure stdin safely. By
setting `some_child.process.no_stdin=1` before calling `get_next_task()`
we provide a reasonable default behavior but enable users to set up
stdin streaming for themselves during the callback.

`some_child.process.stdout_to_stderr`, however, remains unmodifiable by
`get_next_task()` - the rest of the run_processes_parallel() API depends
on child output in stderr.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 run-command.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/run-command.c b/run-command.c
index 1399243de8a..482ee2d76c6 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1627,6 +1627,14 @@ static int pp_start_one(struct parallel_processes *pp)
 	if (i == pp->max_processes)
 		BUG("bookkeeping is hard");
 
+	/*
+	 * By default, do not inherit stdin from the parent process - otherwise,
+	 * all children would share stdin! Users may overwrite this to provide
+	 * something to the child's stdin by having their 'get_next_task'
+	 * callback assign 0 to .no_stdin and an appropriate integer to .in.
+	 */
+	pp->children[i].process.no_stdin = 1;
+
 	code = pp->get_next_task(&pp->children[i].process,
 				 &pp->children[i].err,
 				 pp->data,
@@ -1638,7 +1646,6 @@ static int pp_start_one(struct parallel_processes *pp)
 	}
 	pp->children[i].process.err = -1;
 	pp->children[i].process.stdout_to_stderr = 1;
-	pp->children[i].process.no_stdin = 1;
 
 	if (start_command(&pp->children[i].process)) {
 		code = pp->start_failure(&pp->children[i].err,
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 22/36] hook: support passing stdin to hooks
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (20 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 21/36] run-command: allow stdin for run_processes_parallel Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 23/36] am: convert 'post-rewrite' hook to hook.h Ævar Arnfjörð Bjarmason
                               ` (17 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Some hooks (such as post-rewrite) need to take input via stdin.
Previously, callers provided stdin to hooks by setting
run-command.h:child_process.in, which takes a FD. Callers would open the
file in question themselves before calling run-command(). However, since
we will now need to seek to the front of the file and read it again for
every hook which runs, hook.h:run_command() takes a path and handles FD
management itself. Since this file is opened for read only, it should
not prevent later parallel execution support.

On the frontend, this is supported by asking for a file path, rather
than by reading stdin. Reading directly from stdin would involve caching
the entire stdin (to memory or to disk) and reading it back from the
beginning to each hook. We'd want to support cases like insufficient
memory or storage for the file. While this may prove useful later, for
now the path of least resistance is to just ask the user to make this
interim file themselves.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-hook.txt |  7 ++++++-
 builtin/hook.c             |  4 +++-
 hook.c                     |  8 +++++++-
 hook.h                     |  3 +++
 t/t1800-hook.sh            | 18 ++++++++++++++++++
 5 files changed, 37 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 097fb9de63b..fa68c1f3912 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -8,7 +8,7 @@ git-hook - run git hooks
 SYNOPSIS
 --------
 [verse]
-'git hook' run [--ignore-missing] <hook-name> [-- <hook-args>]
+'git hook' run [--to-stdin=<path>] [--ignore-missing] <hook-name> [-- <hook-args>]
 
 DESCRIPTION
 -----------
@@ -32,6 +32,11 @@ what those are.
 OPTIONS
 -------
 
+--to-stdin::
+	For "run"; Specify a file which will be streamed into the
+	hook's stdin. The hook will receive the entire file from
+	beginning to EOF.
+
 --ignore-missing::
 	Ignore any missing hook by quietly returning zero. Used for
 	tools that want to do a blind one-shot run of a hook that may
diff --git a/builtin/hook.c b/builtin/hook.c
index f33db9953c7..27dce6a2f0e 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -7,7 +7,7 @@
 #include "strvec.h"
 
 #define BUILTIN_HOOK_RUN_USAGE \
-	N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")
+	N_("git hook run [--ignore-missing] [--to-stdin=<path>] <hook-name> [-- <hook-args>]")
 
 static const char * const builtin_hook_usage[] = {
 	BUILTIN_HOOK_RUN_USAGE,
@@ -29,6 +29,8 @@ static int run(int argc, const char **argv, const char *prefix)
 	struct option run_options[] = {
 		OPT_BOOL(0, "ignore-missing", &ignore_missing,
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
+		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
+			   N_("file to read into hooks' stdin")),
 		OPT_END(),
 	};
 	int ret;
diff --git a/hook.c b/hook.c
index 5ecf79c8397..63c9a60921f 100644
--- a/hook.c
+++ b/hook.c
@@ -88,7 +88,13 @@ static int pick_next_hook(struct child_process *cp,
 	if (!run_me)
 		return 0;
 
-	cp->no_stdin = 1;
+	/* reopen the file for stdin; run_command closes it. */
+	if (hook_cb->options->path_to_stdin) {
+		cp->no_stdin = 0;
+		cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY);
+	} else {
+		cp->no_stdin = 1;
+	}
 	cp->env = hook_cb->options->env.v;
 	cp->stdout_to_stderr = 1;
 	cp->trace2_hook_name = hook_cb->hook_name;
diff --git a/hook.h b/hook.h
index 53ea3a9649e..cd6a68a3b50 100644
--- a/hook.h
+++ b/hook.h
@@ -36,6 +36,9 @@ struct run_hooks_opt
 
 	/* Path to initial working directory for subprocess */
 	const char *dir;
+
+	/* Path to file which should be piped to stdin for each hook */
+	const char *path_to_stdin;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
index 49df5a2cdfb..217db848b3f 100755
--- a/t/t1800-hook.sh
+++ b/t/t1800-hook.sh
@@ -135,4 +135,22 @@ test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
 	test_cmp expect actual
 '
 
+test_expect_success 'stdin to hooks' '
+	write_script .git/hooks/test-hook <<-\EOF &&
+	echo BEGIN stdin
+	cat
+	echo END stdin
+	EOF
+
+	cat >expect <<-EOF &&
+	BEGIN stdin
+	hello
+	END stdin
+	EOF
+
+	echo hello >input &&
+	git hook run --to-stdin=input test-hook 2>actual &&
+	test_cmp expect actual
+'
+
 test_done
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 23/36] am: convert 'post-rewrite' hook to hook.h
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (21 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 22/36] hook: support passing stdin to hooks Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 24/36] run-command: add stdin callback for parallelization Ævar Arnfjörð Bjarmason
                               ` (16 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/am.c | 20 ++++----------------
 1 file changed, 4 insertions(+), 16 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index e444b18b64a..9e3d4d9ab44 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -467,24 +467,12 @@ static int run_applypatch_msg_hook(struct am_state *state)
  */
 static int run_post_rewrite_hook(const struct am_state *state)
 {
-	struct child_process cp = CHILD_PROCESS_INIT;
-	const char *hook = find_hook("post-rewrite");
-	int ret;
-
-	if (!hook)
-		return 0;
-
-	strvec_push(&cp.args, hook);
-	strvec_push(&cp.args, "rebase");
-
-	cp.in = xopen(am_path(state, "rewritten"), O_RDONLY);
-	cp.stdout_to_stderr = 1;
-	cp.trace2_hook_name = "post-rewrite";
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 
-	ret = run_command(&cp);
+	strvec_push(&opt.args, "rebase");
+	opt.path_to_stdin = am_path(state, "rewritten");
 
-	close(cp.in);
-	return ret;
+	return run_hooks_oneshot("post-rewrite", &opt);
 }
 
 /**
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 24/36] run-command: add stdin callback for parallelization
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (22 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 23/36] am: convert 'post-rewrite' hook to hook.h Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 25/36] hook: provide stdin by string_list or callback Ævar Arnfjörð Bjarmason
                               ` (15 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

If a user of the run_processes_parallel() API wants to pipe a large
amount of information to stdin of each parallel command, that
information could exceed the buffer of the pipe allocated for that
process's stdin.  Generally this is solved by repeatedly writing to
child_process.in between calls to start_command() and finish_command();
run_processes_parallel() did not provide users an opportunity to access
child_process at that time.

Because the data might be extremely large (for example, a list of all
refs received during a push from a client) simply taking a string_list
or strbuf is not as scalable as using a callback; the rest of the
run_processes_parallel() API also uses callbacks, so making this feature
match the rest of the API reduces mental load on the user.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/fetch.c             |  1 +
 builtin/submodule--helper.c |  2 +-
 hook.c                      |  1 +
 run-command.c               | 54 +++++++++++++++++++++++++++++++++++--
 run-command.h               | 17 +++++++++++-
 submodule.c                 |  1 +
 t/helper/test-run-command.c | 31 ++++++++++++++++++---
 t/t0061-run-command.sh      | 30 +++++++++++++++++++++
 8 files changed, 129 insertions(+), 8 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index 25740c13df1..fef6e85d003 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1817,6 +1817,7 @@ static int fetch_multiple(struct string_list *list, int max_children)
 		result = run_processes_parallel_tr2(max_children,
 						    &fetch_next_remote,
 						    &fetch_failed_to_start,
+						    NULL,
 						    &fetch_finished,
 						    &state,
 						    "fetch", "parallel/fetch");
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index f73963ad67d..f42ded548bf 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2294,7 +2294,7 @@ static int update_submodules(struct submodule_update_clone *suc)
 	int i;
 
 	run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task,
-				   update_clone_start_failure,
+				   update_clone_start_failure, NULL,
 				   update_clone_task_finished, suc, "submodule",
 				   "parallel/update");
 
diff --git a/hook.c b/hook.c
index 63c9a60921f..a7462639d97 100644
--- a/hook.c
+++ b/hook.c
@@ -175,6 +175,7 @@ int run_hooks(const char *hook_name, const char *hook_path,
 	run_processes_parallel_tr2(jobs,
 				   pick_next_hook,
 				   notify_start_failure,
+				   NULL,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
diff --git a/run-command.c b/run-command.c
index 482ee2d76c6..f1616858d18 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1492,6 +1492,7 @@ struct parallel_processes {
 
 	get_next_task_fn get_next_task;
 	start_failure_fn start_failure;
+	feed_pipe_fn feed_pipe;
 	task_finished_fn task_finished;
 
 	struct {
@@ -1519,6 +1520,13 @@ static int default_start_failure(struct strbuf *out,
 	return 0;
 }
 
+static int default_feed_pipe(struct strbuf *pipe,
+			     void *pp_cb,
+			     void *pp_task_cb)
+{
+	return 1;
+}
+
 static int default_task_finished(int result,
 				 struct strbuf *out,
 				 void *pp_cb,
@@ -1549,6 +1557,7 @@ static void pp_init(struct parallel_processes *pp,
 		    int n,
 		    get_next_task_fn get_next_task,
 		    start_failure_fn start_failure,
+		    feed_pipe_fn feed_pipe,
 		    task_finished_fn task_finished,
 		    void *data)
 {
@@ -1567,6 +1576,7 @@ static void pp_init(struct parallel_processes *pp,
 	pp->get_next_task = get_next_task;
 
 	pp->start_failure = start_failure ? start_failure : default_start_failure;
+	pp->feed_pipe = feed_pipe ? feed_pipe : default_feed_pipe;
 	pp->task_finished = task_finished ? task_finished : default_task_finished;
 
 	pp->nr_processes = 0;
@@ -1664,6 +1674,37 @@ static int pp_start_one(struct parallel_processes *pp)
 	return 0;
 }
 
+static void pp_buffer_stdin(struct parallel_processes *pp)
+{
+	int i;
+	struct strbuf sb = STRBUF_INIT;
+
+	/* Buffer stdin for each pipe. */
+	for (i = 0; i < pp->max_processes; i++) {
+		if (pp->children[i].state == GIT_CP_WORKING &&
+		    pp->children[i].process.in > 0) {
+			int done;
+			strbuf_reset(&sb);
+			done = pp->feed_pipe(&sb, pp->data,
+					      pp->children[i].data);
+			if (sb.len) {
+				if (write_in_full(pp->children[i].process.in,
+					      sb.buf, sb.len) < 0) {
+					if (errno != EPIPE)
+						die_errno("write");
+					done = 1;
+				}
+			}
+			if (done) {
+				close(pp->children[i].process.in);
+				pp->children[i].process.in = 0;
+			}
+		}
+	}
+
+	strbuf_release(&sb);
+}
+
 static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout)
 {
 	int i;
@@ -1728,6 +1769,7 @@ static int pp_collect_finished(struct parallel_processes *pp)
 		pp->nr_processes--;
 		pp->children[i].state = GIT_CP_FREE;
 		pp->pfd[i].fd = -1;
+		pp->children[i].process.in = 0;
 		child_process_init(&pp->children[i].process);
 
 		if (i != pp->output_owner) {
@@ -1761,6 +1803,7 @@ static int pp_collect_finished(struct parallel_processes *pp)
 int run_processes_parallel(int n,
 			   get_next_task_fn get_next_task,
 			   start_failure_fn start_failure,
+			   feed_pipe_fn feed_pipe,
 			   task_finished_fn task_finished,
 			   void *pp_cb)
 {
@@ -1769,7 +1812,9 @@ int run_processes_parallel(int n,
 	int spawn_cap = 4;
 	struct parallel_processes pp;
 
-	pp_init(&pp, n, get_next_task, start_failure, task_finished, pp_cb);
+	sigchain_push(SIGPIPE, SIG_IGN);
+
+	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, task_finished, pp_cb);
 	while (1) {
 		for (i = 0;
 		    i < spawn_cap && !pp.shutdown &&
@@ -1786,6 +1831,7 @@ int run_processes_parallel(int n,
 		}
 		if (!pp.nr_processes)
 			break;
+		pp_buffer_stdin(&pp);
 		pp_buffer_stderr(&pp, output_timeout);
 		pp_output(&pp);
 		code = pp_collect_finished(&pp);
@@ -1797,11 +1843,15 @@ int run_processes_parallel(int n,
 	}
 
 	pp_cleanup(&pp);
+
+	sigchain_pop(SIGPIPE);
+
 	return 0;
 }
 
 int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 			       start_failure_fn start_failure,
+			       feed_pipe_fn feed_pipe,
 			       task_finished_fn task_finished, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label)
 {
@@ -1811,7 +1861,7 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 				   ((n < 1) ? online_cpus() : n));
 
 	result = run_processes_parallel(n, get_next_task, start_failure,
-					task_finished, pp_cb);
+					feed_pipe, task_finished, pp_cb);
 
 	trace2_region_leave(tr2_category, tr2_label, NULL);
 
diff --git a/run-command.h b/run-command.h
index cfb6887e4ae..80d394664ae 100644
--- a/run-command.h
+++ b/run-command.h
@@ -422,6 +422,20 @@ typedef int (*start_failure_fn)(struct strbuf *out,
 				void *pp_cb,
 				void *pp_task_cb);
 
+/**
+ * This callback is called repeatedly on every child process who requests
+ * start_command() to create a pipe by setting child_process.in < 0.
+ *
+ * pp_cb is the callback cookie as passed into run_processes_parallel, and
+ * pp_task_cb is the callback cookie as passed into get_next_task_fn.
+ * The contents of 'send' will be read into the pipe and passed to the pipe.
+ *
+ * Return nonzero to close the pipe.
+ */
+typedef int (*feed_pipe_fn)(struct strbuf *pipe,
+			    void *pp_cb,
+			    void *pp_task_cb);
+
 /**
  * This callback is called on every child process that finished processing.
  *
@@ -456,10 +470,11 @@ typedef int (*task_finished_fn)(int result,
 int run_processes_parallel(int n,
 			   get_next_task_fn,
 			   start_failure_fn,
+			   feed_pipe_fn,
 			   task_finished_fn,
 			   void *pp_cb);
 int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn,
-			       task_finished_fn, void *pp_cb,
+			       feed_pipe_fn, task_finished_fn, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label);
 
 /**
diff --git a/submodule.c b/submodule.c
index 8e611fe1dbf..db1700a502d 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1632,6 +1632,7 @@ int fetch_populated_submodules(struct repository *r,
 	run_processes_parallel_tr2(max_parallel_jobs,
 				   get_next_submodule,
 				   fetch_start_failure,
+				   NULL,
 				   fetch_finish,
 				   &spf,
 				   "submodule", "parallel/fetch");
diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c
index 7ae03dc7123..9348184d303 100644
--- a/t/helper/test-run-command.c
+++ b/t/helper/test-run-command.c
@@ -32,8 +32,13 @@ static int parallel_next(struct child_process *cp,
 		return 0;
 
 	strvec_pushv(&cp->args, d->argv);
+	cp->in = d->in;
+	cp->no_stdin = d->no_stdin;
 	strbuf_addstr(err, "preloaded output of a child\n");
 	number_callbacks++;
+
+	*task_cb = xmalloc(sizeof(int));
+	*(int*)(*task_cb) = 2;
 	return 1;
 }
 
@@ -55,6 +60,17 @@ static int task_finished(int result,
 	return 1;
 }
 
+static int test_stdin(struct strbuf *pipe, void *cb, void *task_cb)
+{
+	int *lines_remaining = task_cb;
+
+	if (*lines_remaining)
+		strbuf_addf(pipe, "sample stdin %d\n", --(*lines_remaining));
+
+	return !(*lines_remaining);
+}
+
+
 struct testsuite {
 	struct string_list tests, failed;
 	int next;
@@ -185,7 +201,7 @@ static int testsuite(int argc, const char **argv)
 		suite.tests.nr, max_jobs);
 
 	ret = run_processes_parallel(max_jobs, next_test, test_failed,
-				     test_finished, &suite);
+				     test_stdin, test_finished, &suite);
 
 	if (suite.failed.nr > 0) {
 		ret = 1;
@@ -413,15 +429,22 @@ int cmd__run_command(int argc, const char **argv)
 
 	if (!strcmp(argv[1], "run-command-parallel"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, &proc));
+					    NULL, NULL, NULL, &proc));
 
 	if (!strcmp(argv[1], "run-command-abort"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, task_finished, &proc));
+					    NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-no-jobs"))
 		exit(run_processes_parallel(jobs, no_job,
-					    NULL, task_finished, &proc));
+					    NULL, NULL, task_finished, &proc));
+
+	if (!strcmp(argv[1], "run-command-stdin")) {
+		proc.in = -1;
+		proc.no_stdin = 0;
+		exit (run_processes_parallel(jobs, parallel_next, NULL,
+					     test_stdin, NULL, &proc));
+	}
 
 	fprintf(stderr, "check usage\n");
 	return 1;
diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh
index 7d599675e35..87759482ad1 100755
--- a/t/t0061-run-command.sh
+++ b/t/t0061-run-command.sh
@@ -143,6 +143,36 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai
 	test_cmp expect actual
 '
 
+cat >expect <<-EOF
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+EOF
+
+test_expect_success 'run_command listens to stdin' '
+	write_script stdin-script <<-\EOF &&
+	echo "listening for stdin:"
+	while read line; do
+		echo "$line"
+	done
+	EOF
+	test-tool run-command run-command-stdin 2 ./stdin-script 2>actual &&
+	test_cmp expect actual
+'
+
 cat >expect <<-EOF
 preloaded output of a child
 asking for a quick stop
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 25/36] hook: provide stdin by string_list or callback
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (23 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 24/36] run-command: add stdin callback for parallelization Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 26/36] hook: convert 'post-rewrite' hook in sequencer.c to hook.h Ævar Arnfjörð Bjarmason
                               ` (14 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

In cases where a hook requires only a small amount of information via
stdin, it should be simple for users to provide a string_list alone. But
in more complicated cases where the stdin is too large to hold in
memory, let's instead provide a callback the users can populate line
after line.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 hook.c | 33 ++++++++++++++++++++++++++++++++-
 hook.h | 27 +++++++++++++++++++++++++++
 2 files changed, 59 insertions(+), 1 deletion(-)

diff --git a/hook.c b/hook.c
index a7462639d97..44e73827800 100644
--- a/hook.c
+++ b/hook.c
@@ -77,6 +77,29 @@ void run_hooks_opt_clear(struct run_hooks_opt *o)
 	strvec_clear(&o->args);
 }
 
+int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb)
+{
+	int *item_idx;
+	struct hook *ctx = pp_task_cb;
+	struct hook_cb_data *hook_cb = pp_cb;
+	struct string_list *to_pipe = hook_cb->options->feed_pipe_ctx;
+
+	/* Bootstrap the state manager if necessary. */
+	if (!ctx->feed_pipe_cb_data) {
+		ctx->feed_pipe_cb_data = xmalloc(sizeof(unsigned int));
+		*(int*)ctx->feed_pipe_cb_data = 0;
+	}
+
+	item_idx = ctx->feed_pipe_cb_data;
+
+	if (*item_idx < to_pipe->nr) {
+		strbuf_addf(pipe, "%s\n", to_pipe->items[*item_idx].string);
+		(*item_idx)++;
+		return 0;
+	}
+	return 1;
+}
+
 static int pick_next_hook(struct child_process *cp,
 			  struct strbuf *out,
 			  void *pp_cb,
@@ -92,6 +115,10 @@ static int pick_next_hook(struct child_process *cp,
 	if (hook_cb->options->path_to_stdin) {
 		cp->no_stdin = 0;
 		cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY);
+	} else if (hook_cb->options->feed_pipe) {
+		/* ask for start_command() to make a pipe for us */
+		cp->in = -1;
+		cp->no_stdin = 0;
 	} else {
 		cp->no_stdin = 1;
 	}
@@ -175,7 +202,7 @@ int run_hooks(const char *hook_name, const char *hook_path,
 	run_processes_parallel_tr2(jobs,
 				   pick_next_hook,
 				   notify_start_failure,
-				   NULL,
+				   options->feed_pipe,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
@@ -183,6 +210,7 @@ int run_hooks(const char *hook_name, const char *hook_path,
 
 	if (options->absolute_path)
 		strbuf_release(&abs_path);
+	free(my_hook.feed_pipe_cb_data);
 
 	return cb_data.rc;
 }
@@ -196,6 +224,9 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 	if (!options)
 		options = &hook_opt_scratch;
 
+	if (options->path_to_stdin && options->feed_pipe)
+		BUG("choose only one method to populate stdin");
+
 	hook_path = find_hook(hook_name);
 	if (!hook_path) {
 		ret = 0;
diff --git a/hook.h b/hook.h
index cd6a68a3b50..b55f283f90b 100644
--- a/hook.h
+++ b/hook.h
@@ -19,6 +19,12 @@ int hook_exists(const char *hookname);
 struct hook {
 	/* The path to the hook */
 	const char *hook_path;
+
+	/*
+	 * Use this to keep state for your feed_pipe_fn if you are using
+	 * run_hooks_opt.feed_pipe. Otherwise, do not touch it.
+	 */
+	void *feed_pipe_cb_data;
 };
 
 struct run_hooks_opt
@@ -39,6 +45,19 @@ struct run_hooks_opt
 
 	/* Path to file which should be piped to stdin for each hook */
 	const char *path_to_stdin;
+
+	/*
+	 * Callback and state pointer to ask for more content to pipe to stdin.
+	 * Will be called repeatedly, for each hook. See
+	 * hook.c:pipe_from_stdin() for an example. Keep per-hook state in
+	 * hook.feed_pipe_cb_data (per process). Keep initialization context in
+	 * feed_pipe_ctx (shared by all processes).
+	 *
+	 * See 'pipe_from_string_list()' for info about how to specify a
+	 * string_list as the stdin input instead of writing your own handler.
+	 */
+	feed_pipe_fn feed_pipe;
+	void *feed_pipe_ctx;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
@@ -46,6 +65,14 @@ struct run_hooks_opt
 	.args = STRVEC_INIT, \
 }
 
+/*
+ * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
+ * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'.
+ * This will pipe each string in the list to stdin, separated by newlines.  (Do
+ * not inject your own newlines.)
+ */
+int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb);
+
 /*
  * Callback provided to feed_pipe_fn and consume_sideband_fn.
  */
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 26/36] hook: convert 'post-rewrite' hook in sequencer.c to hook.h
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (24 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 25/36] hook: provide stdin by string_list or callback Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 27/36] transport: convert pre-push hook " Ævar Arnfjörð Bjarmason
                               ` (13 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

By using 'hook.h' for 'post-rewrite', we simplify hook invocations by
not needing to put together our own 'struct child_process'.

The signal handling that's being removed by this commit now takes
place in run-command.h:run_processes_parallel(), so it is OK to remove
them here.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 sequencer.c | 79 ++++++++++++++++++++++-------------------------------
 1 file changed, 32 insertions(+), 47 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 9aac08c1545..77f809c00e4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -35,6 +35,7 @@
 #include "commit-reach.h"
 #include "rebase-interactive.h"
 #include "reset.h"
+#include "string-list.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -1147,33 +1148,27 @@ int update_head_with_reflog(const struct commit *old_head,
 static int run_rewrite_hook(const struct object_id *oldoid,
 			    const struct object_id *newoid)
 {
-	struct child_process proc = CHILD_PROCESS_INIT;
-	const char *argv[3];
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct strbuf tmp = STRBUF_INIT;
+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int code;
-	struct strbuf sb = STRBUF_INIT;
 
-	argv[0] = find_hook("post-rewrite");
-	if (!argv[0])
-		return 0;
+	strvec_push(&opt.args, "amend");
 
-	argv[1] = "amend";
-	argv[2] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = "post-rewrite";
-
-	code = start_command(&proc);
-	if (code)
-		return code;
-	strbuf_addf(&sb, "%s %s\n", oid_to_hex(oldoid), oid_to_hex(newoid));
-	sigchain_push(SIGPIPE, SIG_IGN);
-	write_in_full(proc.in, sb.buf, sb.len);
-	close(proc.in);
-	strbuf_release(&sb);
-	sigchain_pop(SIGPIPE);
-	return finish_command(&proc);
+	strbuf_addf(&tmp,
+		    "%s %s",
+		    oid_to_hex(oldoid),
+		    oid_to_hex(newoid));
+	string_list_append(&to_stdin, tmp.buf);
+
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
+
+	code = run_hooks_oneshot("post-rewrite", &opt);
+
+	strbuf_release(&tmp);
+	string_list_clear(&to_stdin, 0);
+	return code;
 }
 
 void commit_post_rewrite(struct repository *r,
@@ -4526,30 +4521,20 @@ static int pick_commits(struct repository *r,
 		flush_rewritten_pending();
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
-			struct child_process child = CHILD_PROCESS_INIT;
-			const char *post_rewrite_hook =
-				find_hook("post-rewrite");
-
-			child.in = open(rebase_path_rewritten_list(), O_RDONLY);
-			child.git_cmd = 1;
-			strvec_push(&child.args, "notes");
-			strvec_push(&child.args, "copy");
-			strvec_push(&child.args, "--for-rewrite=rebase");
+			struct child_process notes_cp = CHILD_PROCESS_INIT;
+			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+
+			notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY);
+			notes_cp.git_cmd = 1;
+			strvec_push(&notes_cp.args, "notes");
+			strvec_push(&notes_cp.args, "copy");
+			strvec_push(&notes_cp.args, "--for-rewrite=rebase");
 			/* we don't care if this copying failed */
-			run_command(&child);
-
-			if (post_rewrite_hook) {
-				struct child_process hook = CHILD_PROCESS_INIT;
-
-				hook.in = open(rebase_path_rewritten_list(),
-					O_RDONLY);
-				hook.stdout_to_stderr = 1;
-				hook.trace2_hook_name = "post-rewrite";
-				strvec_push(&hook.args, post_rewrite_hook);
-				strvec_push(&hook.args, "rebase");
-				/* we don't care if this hook failed */
-				run_command(&hook);
-			}
+			run_command(&notes_cp);
+
+			hook_opt.path_to_stdin = rebase_path_rewritten_list();
+			strvec_push(&hook_opt.args, "rebase");
+			run_hooks_oneshot("post-rewrite", &hook_opt);
 		}
 		apply_autostash(rebase_path_autostash());
 
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 27/36] transport: convert pre-push hook to hook.h
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (25 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 26/36] hook: convert 'post-rewrite' hook in sequencer.c to hook.h Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 28/36] hook tests: test for exact "pre-push" hook input Ævar Arnfjörð Bjarmason
                               ` (12 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Move the pre-push hook away from run-command.h to and over to the new
hook.h library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 transport.c | 56 ++++++++++++++---------------------------------------
 1 file changed, 14 insertions(+), 42 deletions(-)

diff --git a/transport.c b/transport.c
index 77e196f75f5..4ca8fc0391d 100644
--- a/transport.c
+++ b/transport.c
@@ -1203,63 +1203,35 @@ static void die_with_unpushed_submodules(struct string_list *needs_pushing)
 static int run_pre_push_hook(struct transport *transport,
 			     struct ref *remote_refs)
 {
-	int ret = 0, x;
+	int ret = 0;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	struct ref *r;
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct strbuf buf;
-	const char *argv[4];
-
-	if (!(argv[0] = find_hook("pre-push")))
-		return 0;
-
-	argv[1] = transport->remote->name;
-	argv[2] = transport->url;
-	argv[3] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.trace2_hook_name = "pre-push";
-
-	if (start_command(&proc)) {
-		finish_command(&proc);
-		return -1;
-	}
+	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
 
-	sigchain_push(SIGPIPE, SIG_IGN);
-
-	strbuf_init(&buf, 256);
+	strvec_push(&opt.args, transport->remote->name);
+	strvec_push(&opt.args, transport->url);
 
 	for (r = remote_refs; r; r = r->next) {
+		struct strbuf buf = STRBUF_INIT;
+
 		if (!r->peer_ref) continue;
 		if (r->status == REF_STATUS_REJECT_NONFASTFORWARD) continue;
 		if (r->status == REF_STATUS_REJECT_STALE) continue;
 		if (r->status == REF_STATUS_REJECT_REMOTE_UPDATED) continue;
 		if (r->status == REF_STATUS_UPTODATE) continue;
 
-		strbuf_reset(&buf);
-		strbuf_addf( &buf, "%s %s %s %s\n",
+		strbuf_addf(&buf, "%s %s %s %s",
 			 r->peer_ref->name, oid_to_hex(&r->new_oid),
 			 r->name, oid_to_hex(&r->old_oid));
-
-		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
-			/* We do not mind if a hook does not read all refs. */
-			if (errno != EPIPE)
-				ret = -1;
-			break;
-		}
+		string_list_append(&to_stdin, strbuf_detach(&buf, NULL));
 	}
 
-	strbuf_release(&buf);
-
-	x = close(proc.in);
-	if (!ret)
-		ret = x;
-
-	sigchain_pop(SIGPIPE);
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
 
-	x = finish_command(&proc);
-	if (!ret)
-		ret = x;
+	ret = run_hooks_oneshot("pre-push", &opt);
+	to_stdin.strdup_strings = 1;
+	string_list_clear(&to_stdin, 0);
 
 	return ret;
 }
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 28/36] hook tests: test for exact "pre-push" hook input
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (26 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 27/36] transport: convert pre-push hook " Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-20  0:16               ` Emily Shaffer
  2021-08-03 19:38             ` [PATCH v4 29/36] hook tests: use a modern style for "pre-push" tests Ævar Arnfjörð Bjarmason
                               ` (11 subsequent siblings)
  39 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Extend the tests added in ec55559f937 (push: Add support for pre-push
hooks, 2013-01-13) to exhaustively test for the exact input we're
expecting. This helps a parallel series that's refactoring how the
hook is called, to e.g. make sure that we don't miss a trailing
newline.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5571-pre-push-hook.sh | 23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)

diff --git a/t/t5571-pre-push-hook.sh b/t/t5571-pre-push-hook.sh
index ad8d5804f7b..d2857a6fbc0 100755
--- a/t/t5571-pre-push-hook.sh
+++ b/t/t5571-pre-push-hook.sh
@@ -11,7 +11,7 @@ HOOKDIR="$(git rev-parse --git-dir)/hooks"
 HOOK="$HOOKDIR/pre-push"
 mkdir -p "$HOOKDIR"
 write_script "$HOOK" <<EOF
-cat >/dev/null
+cat >actual
 exit 0
 EOF
 
@@ -20,10 +20,16 @@ test_expect_success 'setup' '
 	git init --bare repo1 &&
 	git remote add parent1 repo1 &&
 	test_commit one &&
-	git push parent1 HEAD:foreign
+	cat >expect <<-EOF &&
+	HEAD $(git rev-parse HEAD) refs/heads/foreign $(test_oid zero)
+	EOF
+
+	test_when_finished "rm actual" &&
+	git push parent1 HEAD:foreign &&
+	test_cmp expect actual
 '
 write_script "$HOOK" <<EOF
-cat >/dev/null
+cat >actual
 exit 1
 EOF
 
@@ -32,11 +38,18 @@ export COMMIT1
 
 test_expect_success 'push with failing hook' '
 	test_commit two &&
-	test_must_fail git push parent1 HEAD
+	cat >expect <<-EOF &&
+	HEAD $(git rev-parse HEAD) refs/heads/main $(test_oid zero)
+	EOF
+
+	test_when_finished "rm actual" &&
+	test_must_fail git push parent1 HEAD &&
+	test_cmp expect actual
 '
 
 test_expect_success '--no-verify bypasses hook' '
-	git push --no-verify parent1 HEAD
+	git push --no-verify parent1 HEAD &&
+	test_path_is_missing actual
 '
 
 COMMIT2="$(git rev-parse HEAD)"
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 29/36] hook tests: use a modern style for "pre-push" tests
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (27 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 28/36] hook tests: test for exact "pre-push" hook input Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-20  0:18               ` Emily Shaffer
  2021-08-03 19:38             ` [PATCH v4 30/36] reference-transaction: use hook.h to run hooks Ævar Arnfjörð Bjarmason
                               ` (10 subsequent siblings)
  39 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Indent the here-docs and use "test_cmp" instead of "diff" in tests
added in ec55559f937 (push: Add support for pre-push hooks,
2013-01-13). Let's also use the more typical "expect" instead of
"expected" to be consistent with the rest of the test file.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5571-pre-push-hook.sh | 71 ++++++++++++++++++++--------------------
 1 file changed, 35 insertions(+), 36 deletions(-)

diff --git a/t/t5571-pre-push-hook.sh b/t/t5571-pre-push-hook.sh
index d2857a6fbc0..6d0d5b854ea 100755
--- a/t/t5571-pre-push-hook.sh
+++ b/t/t5571-pre-push-hook.sh
@@ -61,15 +61,15 @@ echo "$2" >>actual
 cat >>actual
 EOF
 
-cat >expected <<EOF
-parent1
-repo1
-refs/heads/main $COMMIT2 refs/heads/foreign $COMMIT1
-EOF
-
 test_expect_success 'push with hook' '
+	cat >expected <<-EOF &&
+	parent1
+	repo1
+	refs/heads/main $COMMIT2 refs/heads/foreign $COMMIT1
+	EOF
+
 	git push parent1 main:foreign &&
-	diff expected actual
+	test_cmp expected actual
 '
 
 test_expect_success 'add a branch' '
@@ -80,49 +80,48 @@ test_expect_success 'add a branch' '
 COMMIT3="$(git rev-parse HEAD)"
 export COMMIT3
 
-cat >expected <<EOF
-parent1
-repo1
-refs/heads/other $COMMIT3 refs/heads/foreign $COMMIT2
-EOF
-
 test_expect_success 'push to default' '
+	cat >expect <<-EOF &&
+	parent1
+	repo1
+	refs/heads/other $COMMIT3 refs/heads/foreign $COMMIT2
+	EOF
 	git push &&
-	diff expected actual
+	test_cmp expect actual
 '
 
-cat >expected <<EOF
-parent1
-repo1
-refs/tags/one $COMMIT1 refs/tags/tag1 $ZERO_OID
-HEAD~ $COMMIT2 refs/heads/prev $ZERO_OID
-EOF
-
 test_expect_success 'push non-branches' '
+	cat >expect <<-EOF &&
+	parent1
+	repo1
+	refs/tags/one $COMMIT1 refs/tags/tag1 $ZERO_OID
+	HEAD~ $COMMIT2 refs/heads/prev $ZERO_OID
+	EOF
+
 	git push parent1 one:tag1 HEAD~:refs/heads/prev &&
-	diff expected actual
+	test_cmp expect actual
 '
 
-cat >expected <<EOF
-parent1
-repo1
-(delete) $ZERO_OID refs/heads/prev $COMMIT2
-EOF
-
 test_expect_success 'push delete' '
+	cat >expect <<-EOF &&
+	parent1
+	repo1
+	(delete) $ZERO_OID refs/heads/prev $COMMIT2
+	EOF
+
 	git push parent1 :prev &&
-	diff expected actual
+	test_cmp expect actual
 '
 
-cat >expected <<EOF
-repo1
-repo1
-HEAD $COMMIT3 refs/heads/other $ZERO_OID
-EOF
-
 test_expect_success 'push to URL' '
+	cat >expect <<-EOF &&
+	repo1
+	repo1
+	HEAD $COMMIT3 refs/heads/other $ZERO_OID
+	EOF
+
 	git push repo1 HEAD &&
-	diff expected actual
+	test_cmp expect actual
 '
 
 test_expect_success 'set up many-ref tests' '
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 30/36] reference-transaction: use hook.h to run hooks
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (28 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 29/36] hook tests: use a modern style for "pre-push" tests Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 31/36] run-command: allow capturing of collated output Ævar Arnfjörð Bjarmason
                               ` (9 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c | 40 +++++++++++++---------------------------
 1 file changed, 13 insertions(+), 27 deletions(-)

diff --git a/refs.c b/refs.c
index 6211692eaae..73d4a939267 100644
--- a/refs.c
+++ b/refs.c
@@ -2062,47 +2062,33 @@ int ref_update_reject_duplicates(struct string_list *refnames,
 static int run_transaction_hook(struct ref_transaction *transaction,
 				const char *state)
 {
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct strbuf buf = STRBUF_INIT;
-	const char *hook;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
 	int ret = 0, i;
 
-	hook = find_hook("reference-transaction");
-	if (!hook)
-		return ret;
-
-	strvec_pushl(&proc.args, hook, state, NULL);
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = "reference-transaction";
-
-	ret = start_command(&proc);
-	if (ret)
+	if (!hook_exists("reference-transaction"))
 		return ret;
 
-	sigchain_push(SIGPIPE, SIG_IGN);
+	strvec_push(&opt.args, state);
 
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		struct strbuf buf = STRBUF_INIT;
 
-		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
+		strbuf_addf(&buf, "%s %s %s",
 			    oid_to_hex(&update->old_oid),
 			    oid_to_hex(&update->new_oid),
 			    update->refname);
-
-		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
-			if (errno != EPIPE)
-				ret = -1;
-			break;
-		}
+		string_list_append(&to_stdin, strbuf_detach(&buf, NULL));
 	}
 
-	close(proc.in);
-	sigchain_pop(SIGPIPE);
-	strbuf_release(&buf);
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
+
+	ret = run_hooks_oneshot("reference-transaction", &opt);
+	to_stdin.strdup_strings = 1;
+	string_list_clear(&to_stdin, 0);
 
-	ret |= finish_command(&proc);
 	return ret;
 }
 
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 31/36] run-command: allow capturing of collated output
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (29 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 30/36] reference-transaction: use hook.h to run hooks Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 32/36] hooks: allow callers to capture output Ævar Arnfjörð Bjarmason
                               ` (8 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Some callers, for example server-side hooks which wish to relay hook
output to clients across a transport, want to capture what would
normally print to stderr and do something else with it. Allow that via a
callback.

By calling the callback regardless of whether there's output available,
we allow clients to send e.g. a keepalive if necessary.

Because we expose a strbuf, not a fd or FILE*, there's no need to create
a temporary pipe or similar - we can just skip the print to stderr and
instead hand it to the caller.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/fetch.c             |  2 +-
 builtin/submodule--helper.c |  2 +-
 hook.c                      |  1 +
 run-command.c               | 33 +++++++++++++++++++++++++--------
 run-command.h               | 18 +++++++++++++++++-
 submodule.c                 |  2 +-
 t/helper/test-run-command.c | 25 ++++++++++++++++++++-----
 t/t0061-run-command.sh      |  7 +++++++
 8 files changed, 73 insertions(+), 17 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index fef6e85d003..de14df5085c 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1817,7 +1817,7 @@ static int fetch_multiple(struct string_list *list, int max_children)
 		result = run_processes_parallel_tr2(max_children,
 						    &fetch_next_remote,
 						    &fetch_failed_to_start,
-						    NULL,
+						    NULL, NULL,
 						    &fetch_finished,
 						    &state,
 						    "fetch", "parallel/fetch");
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index f42ded548bf..e5e5a8bcfb5 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2294,7 +2294,7 @@ static int update_submodules(struct submodule_update_clone *suc)
 	int i;
 
 	run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task,
-				   update_clone_start_failure, NULL,
+				   update_clone_start_failure, NULL, NULL,
 				   update_clone_task_finished, suc, "submodule",
 				   "parallel/update");
 
diff --git a/hook.c b/hook.c
index 44e73827800..9ae3007cdd3 100644
--- a/hook.c
+++ b/hook.c
@@ -203,6 +203,7 @@ int run_hooks(const char *hook_name, const char *hook_path,
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
+				   NULL,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
diff --git a/run-command.c b/run-command.c
index f1616858d18..aacc336f951 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1493,6 +1493,7 @@ struct parallel_processes {
 	get_next_task_fn get_next_task;
 	start_failure_fn start_failure;
 	feed_pipe_fn feed_pipe;
+	consume_sideband_fn consume_sideband;
 	task_finished_fn task_finished;
 
 	struct {
@@ -1558,6 +1559,7 @@ static void pp_init(struct parallel_processes *pp,
 		    get_next_task_fn get_next_task,
 		    start_failure_fn start_failure,
 		    feed_pipe_fn feed_pipe,
+		    consume_sideband_fn consume_sideband,
 		    task_finished_fn task_finished,
 		    void *data)
 {
@@ -1578,6 +1580,7 @@ static void pp_init(struct parallel_processes *pp,
 	pp->start_failure = start_failure ? start_failure : default_start_failure;
 	pp->feed_pipe = feed_pipe ? feed_pipe : default_feed_pipe;
 	pp->task_finished = task_finished ? task_finished : default_task_finished;
+	pp->consume_sideband = consume_sideband;
 
 	pp->nr_processes = 0;
 	pp->output_owner = 0;
@@ -1614,7 +1617,10 @@ static void pp_cleanup(struct parallel_processes *pp)
 	 * When get_next_task added messages to the buffer in its last
 	 * iteration, the buffered output is non empty.
 	 */
-	strbuf_write(&pp->buffered_output, stderr);
+	if (pp->consume_sideband)
+		pp->consume_sideband(&pp->buffered_output, pp->data);
+	else
+		strbuf_write(&pp->buffered_output, stderr);
 	strbuf_release(&pp->buffered_output);
 
 	sigchain_pop_common();
@@ -1735,9 +1741,13 @@ static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout)
 static void pp_output(struct parallel_processes *pp)
 {
 	int i = pp->output_owner;
+
 	if (pp->children[i].state == GIT_CP_WORKING &&
 	    pp->children[i].err.len) {
-		strbuf_write(&pp->children[i].err, stderr);
+		if (pp->consume_sideband)
+			pp->consume_sideband(&pp->children[i].err, pp->data);
+		else
+			strbuf_write(&pp->children[i].err, stderr);
 		strbuf_reset(&pp->children[i].err);
 	}
 }
@@ -1776,11 +1786,15 @@ static int pp_collect_finished(struct parallel_processes *pp)
 			strbuf_addbuf(&pp->buffered_output, &pp->children[i].err);
 			strbuf_reset(&pp->children[i].err);
 		} else {
-			strbuf_write(&pp->children[i].err, stderr);
+			/* Output errors, then all other finished child processes */
+			if (pp->consume_sideband) {
+				pp->consume_sideband(&pp->children[i].err, pp->data);
+				pp->consume_sideband(&pp->buffered_output, pp->data);
+			} else {
+				strbuf_write(&pp->children[i].err, stderr);
+				strbuf_write(&pp->buffered_output, stderr);
+			}
 			strbuf_reset(&pp->children[i].err);
-
-			/* Output all other finished child processes */
-			strbuf_write(&pp->buffered_output, stderr);
 			strbuf_reset(&pp->buffered_output);
 
 			/*
@@ -1804,6 +1818,7 @@ int run_processes_parallel(int n,
 			   get_next_task_fn get_next_task,
 			   start_failure_fn start_failure,
 			   feed_pipe_fn feed_pipe,
+			   consume_sideband_fn consume_sideband,
 			   task_finished_fn task_finished,
 			   void *pp_cb)
 {
@@ -1814,7 +1829,7 @@ int run_processes_parallel(int n,
 
 	sigchain_push(SIGPIPE, SIG_IGN);
 
-	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, task_finished, pp_cb);
+	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, consume_sideband, task_finished, pp_cb);
 	while (1) {
 		for (i = 0;
 		    i < spawn_cap && !pp.shutdown &&
@@ -1852,6 +1867,7 @@ int run_processes_parallel(int n,
 int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 			       start_failure_fn start_failure,
 			       feed_pipe_fn feed_pipe,
+			       consume_sideband_fn consume_sideband,
 			       task_finished_fn task_finished, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label)
 {
@@ -1861,7 +1877,8 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 				   ((n < 1) ? online_cpus() : n));
 
 	result = run_processes_parallel(n, get_next_task, start_failure,
-					feed_pipe, task_finished, pp_cb);
+					feed_pipe, consume_sideband,
+					task_finished, pp_cb);
 
 	trace2_region_leave(tr2_category, tr2_label, NULL);
 
diff --git a/run-command.h b/run-command.h
index 80d394664ae..e321d23bbd2 100644
--- a/run-command.h
+++ b/run-command.h
@@ -436,6 +436,20 @@ typedef int (*feed_pipe_fn)(struct strbuf *pipe,
 			    void *pp_cb,
 			    void *pp_task_cb);
 
+/**
+ * If this callback is provided, instead of collating process output to stderr,
+ * they will be collated into a new pipe. consume_sideband_fn will be called
+ * repeatedly. When output is available on that pipe, it will be contained in
+ * 'output'. But it will be called with an empty 'output' too, to allow for
+ * keepalives or similar operations if necessary.
+ *
+ * pp_cb is the callback cookie as passed into run_processes_parallel.
+ *
+ * Since this callback is provided with the collated output, no task cookie is
+ * provided.
+ */
+typedef void (*consume_sideband_fn)(struct strbuf *output, void *pp_cb);
+
 /**
  * This callback is called on every child process that finished processing.
  *
@@ -471,10 +485,12 @@ int run_processes_parallel(int n,
 			   get_next_task_fn,
 			   start_failure_fn,
 			   feed_pipe_fn,
+			   consume_sideband_fn,
 			   task_finished_fn,
 			   void *pp_cb);
 int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn,
-			       feed_pipe_fn, task_finished_fn, void *pp_cb,
+			       feed_pipe_fn, consume_sideband_fn,
+			       task_finished_fn, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label);
 
 /**
diff --git a/submodule.c b/submodule.c
index db1700a502d..32364d8bd56 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1632,7 +1632,7 @@ int fetch_populated_submodules(struct repository *r,
 	run_processes_parallel_tr2(max_parallel_jobs,
 				   get_next_submodule,
 				   fetch_start_failure,
-				   NULL,
+				   NULL, NULL,
 				   fetch_finish,
 				   &spf,
 				   "submodule", "parallel/fetch");
diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c
index 9348184d303..d53db6d11c4 100644
--- a/t/helper/test-run-command.c
+++ b/t/helper/test-run-command.c
@@ -51,6 +51,16 @@ static int no_job(struct child_process *cp,
 	return 0;
 }
 
+static void test_consume_sideband(struct strbuf *output, void *cb)
+{
+	FILE *sideband;
+
+	sideband = fopen("./sideband", "a");
+
+	strbuf_write(output, sideband);
+	fclose(sideband);
+}
+
 static int task_finished(int result,
 			 struct strbuf *err,
 			 void *pp_cb,
@@ -201,7 +211,7 @@ static int testsuite(int argc, const char **argv)
 		suite.tests.nr, max_jobs);
 
 	ret = run_processes_parallel(max_jobs, next_test, test_failed,
-				     test_stdin, test_finished, &suite);
+				     test_stdin, NULL, test_finished, &suite);
 
 	if (suite.failed.nr > 0) {
 		ret = 1;
@@ -429,23 +439,28 @@ int cmd__run_command(int argc, const char **argv)
 
 	if (!strcmp(argv[1], "run-command-parallel"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, NULL, &proc));
+					    NULL, NULL, NULL, NULL, &proc));
 
 	if (!strcmp(argv[1], "run-command-abort"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, task_finished, &proc));
+					    NULL, NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-no-jobs"))
 		exit(run_processes_parallel(jobs, no_job,
-					    NULL, NULL, task_finished, &proc));
+					    NULL, NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-stdin")) {
 		proc.in = -1;
 		proc.no_stdin = 0;
 		exit (run_processes_parallel(jobs, parallel_next, NULL,
-					     test_stdin, NULL, &proc));
+					     test_stdin, NULL, NULL, &proc));
 	}
 
+	if (!strcmp(argv[1], "run-command-sideband"))
+		exit(run_processes_parallel(jobs, parallel_next, NULL, NULL,
+					    test_consume_sideband, NULL,
+					    &proc));
+
 	fprintf(stderr, "check usage\n");
 	return 1;
 }
diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh
index 87759482ad1..e99f6c7f445 100755
--- a/t/t0061-run-command.sh
+++ b/t/t0061-run-command.sh
@@ -143,6 +143,13 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai
 	test_cmp expect actual
 '
 
+test_expect_success 'run_command can divert output' '
+	test_when_finished rm sideband &&
+	test-tool run-command run-command-sideband 3 sh -c "printf \"%s\n%s\n\" Hello World" 2>actual &&
+	test_must_be_empty actual &&
+	test_cmp expect sideband
+'
+
 cat >expect <<-EOF
 preloaded output of a child
 listening for stdin:
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 32/36] hooks: allow callers to capture output
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (30 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 31/36] run-command: allow capturing of collated output Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:38             ` [PATCH v4 33/36] receive-pack: convert 'update' hook to hook.h Ævar Arnfjörð Bjarmason
                               ` (7 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Some server-side hooks will require capturing output to send over
sideband instead of printing directly to stderr. Expose that capability.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 hook.c | 3 ++-
 hook.h | 8 ++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/hook.c b/hook.c
index 9ae3007cdd3..efed5f73b4d 100644
--- a/hook.c
+++ b/hook.c
@@ -203,12 +203,13 @@ int run_hooks(const char *hook_name, const char *hook_path,
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
-				   NULL,
+				   options->consume_sideband,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
 				   hook_name);
 
+
 	if (options->absolute_path)
 		strbuf_release(&abs_path);
 	free(my_hook.feed_pipe_cb_data);
diff --git a/hook.h b/hook.h
index b55f283f90b..37a9690c2ca 100644
--- a/hook.h
+++ b/hook.h
@@ -58,6 +58,14 @@ struct run_hooks_opt
 	 */
 	feed_pipe_fn feed_pipe;
 	void *feed_pipe_ctx;
+
+	/*
+	 * Populate this to capture output and prevent it from being printed to
+	 * stderr. This will be passed directly through to
+	 * run_command:run_parallel_processes(). See t/helper/test-run-command.c
+	 * for an example.
+	 */
+	consume_sideband_fn consume_sideband;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 33/36] receive-pack: convert 'update' hook to hook.h
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (31 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 32/36] hooks: allow callers to capture output Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:38             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:39             ` [PATCH v4 34/36] post-update: use hook.h library Ævar Arnfjörð Bjarmason
                               ` (6 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:38 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

This makes use of the new sideband API in hook.h added in the
preceding commit.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/receive-pack.c | 62 ++++++++++++++++++++++++++++--------------
 1 file changed, 41 insertions(+), 21 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index a7d03bbc7d3..31ce4ece4e7 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -937,33 +937,53 @@ static int run_receive_hook(struct command *commands,
 	return status;
 }
 
-static int run_update_hook(struct command *cmd)
+static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
 {
-	const char *argv[5];
-	struct child_process proc = CHILD_PROCESS_INIT;
-	int code;
+	int keepalive_active = 0;
 
-	argv[0] = find_hook("update");
-	if (!argv[0])
-		return 0;
+	if (keepalive_in_sec <= 0)
+		use_keepalive = KEEPALIVE_NEVER;
+	if (use_keepalive == KEEPALIVE_ALWAYS)
+		keepalive_active = 1;
 
-	argv[1] = cmd->ref_name;
-	argv[2] = oid_to_hex(&cmd->old_oid);
-	argv[3] = oid_to_hex(&cmd->new_oid);
-	argv[4] = NULL;
+	/* send a keepalive if there is no data to write */
+	if (keepalive_active && !output->len) {
+		static const char buf[] = "0005\1";
+		write_or_die(1, buf, sizeof(buf) - 1);
+		return;
+	}
 
-	proc.no_stdin = 1;
-	proc.stdout_to_stderr = 1;
-	proc.err = use_sideband ? -1 : 0;
-	proc.argv = argv;
-	proc.trace2_hook_name = "update";
+	if (use_keepalive == KEEPALIVE_AFTER_NUL && !keepalive_active) {
+		const char *first_null = memchr(output->buf, '\0', output->len);
+		if (first_null) {
+			/* The null bit is excluded. */
+			size_t before_null = first_null - output->buf;
+			size_t after_null = output->len - (before_null + 1);
+			keepalive_active = 1;
+			send_sideband(1, 2, output->buf, before_null, use_sideband);
+			send_sideband(1, 2, first_null + 1, after_null, use_sideband);
+
+			return;
+		}
+	}
+
+	send_sideband(1, 2, output->buf, output->len, use_sideband);
+}
+
+static int run_update_hook(struct command *cmd)
+{
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
+	strvec_pushl(&opt.args,
+		     cmd->ref_name,
+		     oid_to_hex(&cmd->old_oid),
+		     oid_to_hex(&cmd->new_oid),
+		     NULL);
 
-	code = start_command(&proc);
-	if (code)
-		return code;
 	if (use_sideband)
-		copy_to_sideband(proc.err, -1, NULL);
-	return finish_command(&proc);
+		opt.consume_sideband = hook_output_to_sideband;
+
+	return run_hooks_oneshot("update", &opt);
 }
 
 static struct command *find_command_by_refname(struct command *list,
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 34/36] post-update: use hook.h library
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (32 preceding siblings ...)
  2021-08-03 19:38             ` [PATCH v4 33/36] receive-pack: convert 'update' hook to hook.h Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:39             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:39             ` [PATCH v4 35/36] receive-pack: convert receive hooks to hook.h Ævar Arnfjörð Bjarmason
                               ` (5 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:39 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/receive-pack.c | 25 ++++++-------------------
 1 file changed, 6 insertions(+), 19 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 31ce4ece4e7..26e302aab85 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1650,33 +1650,20 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 static void run_update_post_hook(struct command *commands)
 {
 	struct command *cmd;
-	struct child_process proc = CHILD_PROCESS_INIT;
-	const char *hook;
-
-	hook = find_hook("post-update");
-	if (!hook)
-		return;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 
 	for (cmd = commands; cmd; cmd = cmd->next) {
 		if (cmd->error_string || cmd->did_not_exist)
 			continue;
-		if (!proc.args.nr)
-			strvec_push(&proc.args, hook);
-		strvec_push(&proc.args, cmd->ref_name);
+		strvec_push(&opt.args, cmd->ref_name);
 	}
-	if (!proc.args.nr)
+	if (!opt.args.nr)
 		return;
 
-	proc.no_stdin = 1;
-	proc.stdout_to_stderr = 1;
-	proc.err = use_sideband ? -1 : 0;
-	proc.trace2_hook_name = "post-update";
+	if (use_sideband)
+		opt.consume_sideband = hook_output_to_sideband;
 
-	if (!start_command(&proc)) {
-		if (use_sideband)
-			copy_to_sideband(proc.err, -1, NULL);
-		finish_command(&proc);
-	}
+	run_hooks_oneshot("post-update", &opt);
 }
 
 static void check_aliased_update_internal(struct command *cmd,
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 35/36] receive-pack: convert receive hooks to hook.h
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (33 preceding siblings ...)
  2021-08-03 19:39             ` [PATCH v4 34/36] post-update: use hook.h library Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:39             ` Ævar Arnfjörð Bjarmason
  2021-08-03 19:39             ` [PATCH v4 36/36] hooks: fix a TOCTOU in "did we run a hook?" heuristic Ævar Arnfjörð Bjarmason
                               ` (4 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:39 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/receive-pack.c | 190 ++++++++++++++++++-----------------------
 1 file changed, 83 insertions(+), 107 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 26e302aab85..c3984680d7f 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -747,7 +747,7 @@ static int check_cert_push_options(const struct string_list *push_options)
 	return retval;
 }
 
-static void prepare_push_cert_sha1(struct child_process *proc)
+static void prepare_push_cert_sha1(struct run_hooks_opt *opt)
 {
 	static int already_done;
 
@@ -771,110 +771,42 @@ static void prepare_push_cert_sha1(struct child_process *proc)
 		nonce_status = check_nonce(push_cert.buf, bogs);
 	}
 	if (!is_null_oid(&push_cert_oid)) {
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT=%s",
 			     oid_to_hex(&push_cert_oid));
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_SIGNER=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_SIGNER=%s",
 			     sigcheck.signer ? sigcheck.signer : "");
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_KEY=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_KEY=%s",
 			     sigcheck.key ? sigcheck.key : "");
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_STATUS=%c",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_STATUS=%c",
 			     sigcheck.result);
 		if (push_cert_nonce) {
-			strvec_pushf(&proc->env_array,
+			strvec_pushf(&opt->env,
 				     "GIT_PUSH_CERT_NONCE=%s",
 				     push_cert_nonce);
-			strvec_pushf(&proc->env_array,
+			strvec_pushf(&opt->env,
 				     "GIT_PUSH_CERT_NONCE_STATUS=%s",
 				     nonce_status);
 			if (nonce_status == NONCE_SLOP)
-				strvec_pushf(&proc->env_array,
+				strvec_pushf(&opt->env,
 					     "GIT_PUSH_CERT_NONCE_SLOP=%ld",
 					     nonce_stamp_slop);
 		}
 	}
 }
 
+struct receive_hook_feed_context {
+	struct command *cmd;
+	int skip_broken;
+};
+
 struct receive_hook_feed_state {
 	struct command *cmd;
 	struct ref_push_report *report;
 	int skip_broken;
 	struct strbuf buf;
-	const struct string_list *push_options;
 };
 
-typedef int (*feed_fn)(void *, const char **, size_t *);
-static int run_and_feed_hook(const char *hook_name, feed_fn feed,
-			     struct receive_hook_feed_state *feed_state)
-{
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct async muxer;
-	const char *argv[2];
-	int code;
-
-	argv[0] = find_hook(hook_name);
-	if (!argv[0])
-		return 0;
-
-	argv[1] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = hook_name;
-
-	if (feed_state->push_options) {
-		int i;
-		for (i = 0; i < feed_state->push_options->nr; i++)
-			strvec_pushf(&proc.env_array,
-				     "GIT_PUSH_OPTION_%d=%s", i,
-				     feed_state->push_options->items[i].string);
-		strvec_pushf(&proc.env_array, "GIT_PUSH_OPTION_COUNT=%d",
-			     feed_state->push_options->nr);
-	} else
-		strvec_pushf(&proc.env_array, "GIT_PUSH_OPTION_COUNT");
-
-	if (tmp_objdir)
-		strvec_pushv(&proc.env_array, tmp_objdir_env(tmp_objdir));
-
-	if (use_sideband) {
-		memset(&muxer, 0, sizeof(muxer));
-		muxer.proc = copy_to_sideband;
-		muxer.in = -1;
-		code = start_async(&muxer);
-		if (code)
-			return code;
-		proc.err = muxer.in;
-	}
-
-	prepare_push_cert_sha1(&proc);
-
-	code = start_command(&proc);
-	if (code) {
-		if (use_sideband)
-			finish_async(&muxer);
-		return code;
-	}
-
-	sigchain_push(SIGPIPE, SIG_IGN);
-
-	while (1) {
-		const char *buf;
-		size_t n;
-		if (feed(feed_state, &buf, &n))
-			break;
-		if (write_in_full(proc.in, buf, n) < 0)
-			break;
-	}
-	close(proc.in);
-	if (use_sideband)
-		finish_async(&muxer);
-
-	sigchain_pop(SIGPIPE);
-
-	return finish_command(&proc);
-}
-
-static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
+static int feed_receive_hook(void *state_)
 {
 	struct receive_hook_feed_state *state = state_;
 	struct command *cmd = state->cmd;
@@ -883,9 +815,7 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
 	       state->skip_broken && (cmd->error_string || cmd->did_not_exist))
 		cmd = cmd->next;
 	if (!cmd)
-		return -1; /* EOF */
-	if (!bufp)
-		return 0; /* OK, can feed something. */
+		return 1; /* EOF - close the pipe*/
 	strbuf_reset(&state->buf);
 	if (!state->report)
 		state->report = cmd->report;
@@ -909,32 +839,36 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
 			    cmd->ref_name);
 		state->cmd = cmd->next;
 	}
-	if (bufp) {
-		*bufp = state->buf.buf;
-		*sizep = state->buf.len;
-	}
 	return 0;
 }
 
-static int run_receive_hook(struct command *commands,
-			    const char *hook_name,
-			    int skip_broken,
-			    const struct string_list *push_options)
+static int feed_receive_hook_cb(struct strbuf *pipe, void *pp_cb, void *pp_task_cb)
 {
-	struct receive_hook_feed_state state;
-	int status;
-
-	strbuf_init(&state.buf, 0);
-	state.cmd = commands;
-	state.skip_broken = skip_broken;
-	state.report = NULL;
-	if (feed_receive_hook(&state, NULL, NULL))
-		return 0;
-	state.cmd = commands;
-	state.push_options = push_options;
-	status = run_and_feed_hook(hook_name, feed_receive_hook, &state);
-	strbuf_release(&state.buf);
-	return status;
+	struct hook *hook = pp_task_cb;
+	struct receive_hook_feed_state *feed_state = hook->feed_pipe_cb_data;
+	int rc;
+
+	/* first-time setup */
+	if (!feed_state) {
+		struct hook_cb_data *hook_cb = pp_cb;
+		struct run_hooks_opt *opt = hook_cb->options;
+		struct receive_hook_feed_context *ctx = opt->feed_pipe_ctx;
+		if (!ctx)
+			BUG("run_hooks_opt.feed_pipe_ctx required for receive hook");
+
+		feed_state = xmalloc(sizeof(struct receive_hook_feed_state));
+		strbuf_init(&feed_state->buf, 0);
+		feed_state->cmd = ctx->cmd;
+		feed_state->skip_broken = ctx->skip_broken;
+		feed_state->report = NULL;
+
+		hook->feed_pipe_cb_data = feed_state;
+	}
+
+	rc = feed_receive_hook(feed_state);
+	if (!rc)
+		strbuf_addbuf(pipe, &feed_state->buf);
+	return rc;
 }
 
 static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
@@ -970,6 +904,48 @@ static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
 	send_sideband(1, 2, output->buf, output->len, use_sideband);
 }
 
+static int run_receive_hook(struct command *commands,
+			    const char *hook_name,
+			    int skip_broken,
+			    const struct string_list *push_options)
+{
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct receive_hook_feed_context ctx;
+	struct command *iter = commands;
+
+	/* if there are no valid commands, don't invoke the hook at all. */
+	while (iter && skip_broken && (iter->error_string || iter->did_not_exist))
+		iter = iter->next;
+	if (!iter)
+		return 0;
+
+	if (push_options) {
+		int i;
+		for (i = 0; i < push_options->nr; i++)
+			strvec_pushf(&opt.env, "GIT_PUSH_OPTION_%d=%s", i,
+				     push_options->items[i].string);
+		strvec_pushf(&opt.env, "GIT_PUSH_OPTION_COUNT=%d", push_options->nr);
+	} else
+		strvec_push(&opt.env, "GIT_PUSH_OPTION_COUNT");
+
+	if (tmp_objdir)
+		strvec_pushv(&opt.env, tmp_objdir_env(tmp_objdir));
+
+	prepare_push_cert_sha1(&opt);
+
+	/* set up sideband printer */
+	if (use_sideband)
+		opt.consume_sideband = hook_output_to_sideband;
+
+	/* set up stdin callback */
+	ctx.cmd = commands;
+	ctx.skip_broken = skip_broken;
+	opt.feed_pipe = feed_receive_hook_cb;
+	opt.feed_pipe_ctx = &ctx;
+
+	return run_hooks_oneshot(hook_name, &opt);
+}
+
 static int run_update_hook(struct command *cmd)
 {
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 36/36] hooks: fix a TOCTOU in "did we run a hook?" heuristic
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (34 preceding siblings ...)
  2021-08-03 19:39             ` [PATCH v4 35/36] receive-pack: convert receive hooks to hook.h Ævar Arnfjörð Bjarmason
@ 2021-08-03 19:39             ` Ævar Arnfjörð Bjarmason
  2021-08-12  0:42             ` [PATCH v2 0/6] config-based hooks restarted Emily Shaffer
                               ` (3 subsequent siblings)
  39 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-03 19:39 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Fix a Time-of-check to time-of-use (TOCTOU) race in code added in
680ee550d72 (commit: skip discarding the index if there is no
pre-commit hook, 2017-08-14).

We can fix the race passing around information about whether or not we
ran the hook in question, instead of running hook_exists() after the
fact to check if the hook in question exists. This problem has been
noted on-list when 680ee550d72 was discussed[1], but had not been
fixed.

In addition to fixing this for the pre-commit hook as suggested there
I'm also fixing this for the pre-merge-commit hook. See
6098817fd7f (git-merge: honor pre-merge-commit hook, 2019-08-07) for
the introduction of its previous behavior.

Let's also change this for the push-to-checkout hook. Now instead of
checking if the hook exists and either doing a push to checkout or a
push to deploy we'll always attempt a push to checkout. If the hook
doesn't exist we'll fall back on push to deploy. The same behavior as
before, without the TOCTOU race. See 0855331941b (receive-pack:
support push-to-checkout hook, 2014-12-01) for the introduction of the
previous behavior.

This leaves uses of hook_exists() in two places that matter. The
"reference-transaction" check in refs.c, see 67541597670 (refs:
implement reference transaction hook, 2020-06-19), and the
prepare-commit-msg hook, see 66618a50f9c (sequencer: run
'prepare-commit-msg' hook, 2018-01-24).

In both of those cases we're saving ourselves CPU time by not
preparing data for the hook that we'll then do nothing with if we
don't have the hook. So using this "invoked_hook" pattern doesn't make
sense in those cases.

More importantly, in those cases the worst we'll do is miss that we
"should" run the hook because a new hook appeared, whereas in the
pre-commit and pre-merge-commit cases we'll skip an important
discard_cache() on the bases of our faulty guess.

I do think none of these races really matter in practice. It would be
some one-off issue as a hook was added or removed. I did think it was
stupid that we didn't pass a "did this run?" flag instead of doing
this guessing at a distance though, so now we're not guessing anymore.

1. https://lore.kernel.org/git/20170810191613.kpmhzg4seyxy3cpq@sigill.intra.peff.net/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/commit.c       | 18 +++++++++++-------
 builtin/merge.c        | 16 ++++++++++------
 builtin/receive-pack.c |  8 +++++---
 commit.c               |  1 +
 commit.h               |  3 ++-
 hook.c                 |  4 ++++
 hook.h                 | 10 ++++++++++
 sequencer.c            |  4 ++--
 8 files changed, 45 insertions(+), 19 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index aa3c741efa9..0b7642e8324 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -725,11 +725,13 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	int clean_message_contents = (cleanup_mode != COMMIT_MSG_CLEANUP_NONE);
 	int old_display_comment_prefix;
 	int merge_contains_scissors = 0;
+	int invoked_hook = 0;
 
 	/* This checks and barfs if author is badly specified */
 	determine_author_info(author_ident);
 
-	if (!no_verify && run_commit_hook(use_editor, index_file, "pre-commit", NULL))
+	if (!no_verify && run_commit_hook(use_editor, index_file, &invoked_hook,
+					  "pre-commit", NULL))
 		return 0;
 
 	if (squash_message) {
@@ -1052,10 +1054,10 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (!no_verify && hook_exists("pre-commit")) {
+	if (!no_verify && invoked_hook) {
 		/*
-		 * Re-read the index as pre-commit hook could have updated it,
-		 * and write it out as a tree.  We must do this before we invoke
+		 * Re-read the index as the pre-commit-commit hook was invoked
+		 * and could have updated it. We must do this before we invoke
 		 * the editor and after we invoke run_status above.
 		 */
 		discard_cache();
@@ -1067,7 +1069,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (run_commit_hook(use_editor, index_file, "prepare-commit-msg",
+	if (run_commit_hook(use_editor, index_file, NULL, "prepare-commit-msg",
 			    git_path_commit_editmsg(), hook_arg1, hook_arg2, NULL))
 		return 0;
 
@@ -1084,7 +1086,8 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	}
 
 	if (!no_verify &&
-	    run_commit_hook(use_editor, index_file, "commit-msg", git_path_commit_editmsg(), NULL)) {
+	    run_commit_hook(use_editor, index_file, NULL, "commit-msg",
+			    git_path_commit_editmsg(), NULL)) {
 		return 0;
 	}
 
@@ -1840,7 +1843,8 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 
 	repo_rerere(the_repository, 0);
 	run_auto_maintenance(quiet);
-	run_commit_hook(use_editor, get_index_file(), "post-commit", NULL);
+	run_commit_hook(use_editor, get_index_file(), NULL, "post-commit",
+			NULL);
 	if (amend && !no_post_rewrite) {
 		commit_post_rewrite(the_repository, current_head, &oid);
 	}
diff --git a/builtin/merge.c b/builtin/merge.c
index 4965df2ac29..9bd4a2532c3 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -843,15 +843,18 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 {
 	struct strbuf msg = STRBUF_INIT;
 	const char *index_file = get_index_file();
+	int invoked_hook = 0;
 
-	if (!no_verify && run_commit_hook(0 < option_edit, index_file, "pre-merge-commit", NULL))
+	if (!no_verify && run_commit_hook(0 < option_edit, index_file,
+					  &invoked_hook, "pre-merge-commit",
+					  NULL))
 		abort_commit(remoteheads, NULL);
 	/*
-	 * Re-read the index as pre-merge-commit hook could have updated it,
-	 * and write it out as a tree.  We must do this before we invoke
+	 * Re-read the index as the pre-merge-commit hook was invoked
+	 * and could have updated it. We must do this before we invoke
 	 * the editor and after we invoke run_status above.
 	 */
-	if (hook_exists("pre-merge-commit"))
+	if (invoked_hook)
 		discard_cache();
 	read_cache_from(index_file);
 	strbuf_addbuf(&msg, &merge_msg);
@@ -872,7 +875,8 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 		append_signoff(&msg, ignore_non_trailer(msg.buf, msg.len), 0);
 	write_merge_heads(remoteheads);
 	write_file_buf(git_path_merge_msg(the_repository), msg.buf, msg.len);
-	if (run_commit_hook(0 < option_edit, get_index_file(), "prepare-commit-msg",
+	if (run_commit_hook(0 < option_edit, get_index_file(), NULL,
+			    "prepare-commit-msg",
 			    git_path_merge_msg(the_repository), "merge", NULL))
 		abort_commit(remoteheads, NULL);
 	if (0 < option_edit) {
@@ -881,7 +885,7 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 	}
 
 	if (!no_verify && run_commit_hook(0 < option_edit, get_index_file(),
-					  "commit-msg",
+					  NULL, "commit-msg",
 					  git_path_merge_msg(the_repository), NULL))
 		abort_commit(remoteheads, NULL);
 
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index c3984680d7f..ebec6f3bb10 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1428,10 +1428,12 @@ static const char *push_to_deploy(unsigned char *sha1,
 static const char *push_to_checkout_hook = "push-to-checkout";
 
 static const char *push_to_checkout(unsigned char *hash,
+				    int *invoked_hook,
 				    struct strvec *env,
 				    const char *work_tree)
 {
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	opt.invoked_hook = invoked_hook;
 
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
 	strvec_pushv(&opt.env, env->v);
@@ -1446,6 +1448,7 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 {
 	const char *retval, *work_tree, *git_dir = NULL;
 	struct strvec env = STRVEC_INIT;
+	int invoked_hook = 0;
 
 	if (worktree && worktree->path)
 		work_tree = worktree->path;
@@ -1463,10 +1466,9 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 
 	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
 
-	if (!hook_exists(push_to_checkout_hook))
+	retval = push_to_checkout(sha1, &invoked_hook, &env, work_tree);
+	if (!invoked_hook)
 		retval = push_to_deploy(sha1, &env, work_tree);
-	else
-		retval = push_to_checkout(sha1, &env, work_tree);
 
 	strvec_clear(&env);
 	return retval;
diff --git a/commit.c b/commit.c
index 63d7943a86d..842e47beae2 100644
--- a/commit.c
+++ b/commit.c
@@ -1697,6 +1697,7 @@ size_t ignore_non_trailer(const char *buf, size_t len)
 }
 
 int run_commit_hook(int editor_is_used, const char *index_file,
+		    int *invoked_hook,
 		    const char *name, ...)
 {
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
diff --git a/commit.h b/commit.h
index df42eb434f3..b5a542993c6 100644
--- a/commit.h
+++ b/commit.h
@@ -363,7 +363,8 @@ int compare_commits_by_commit_date(const void *a_, const void *b_, void *unused)
 int compare_commits_by_gen_then_commit_date(const void *a_, const void *b_, void *unused);
 
 LAST_ARG_MUST_BE_NULL
-int run_commit_hook(int editor_is_used, const char *index_file, const char *name, ...);
+int run_commit_hook(int editor_is_used, const char *index_file,
+		    int *invoked_hook, const char *name, ...);
 
 /* Sign a commit or tag buffer, storing the result in a header. */
 int sign_with_header(struct strbuf *buf, const char *keyid);
diff --git a/hook.c b/hook.c
index efed5f73b4d..ee20b2e3658 100644
--- a/hook.c
+++ b/hook.c
@@ -173,6 +173,9 @@ static int notify_hook_finished(int result,
 
 	hook_cb->rc |= result;
 
+	if (hook_cb->invoked_hook)
+		*hook_cb->invoked_hook = 1;
+
 	return 0;
 }
 
@@ -187,6 +190,7 @@ int run_hooks(const char *hook_name, const char *hook_path,
 		.rc = 0,
 		.hook_name = hook_name,
 		.options = options,
+		.invoked_hook = options->invoked_hook,
 	};
 	int jobs = 1;
 
diff --git a/hook.h b/hook.h
index 37a9690c2ca..58dfbf474c9 100644
--- a/hook.h
+++ b/hook.h
@@ -66,6 +66,15 @@ struct run_hooks_opt
 	 * for an example.
 	 */
 	consume_sideband_fn consume_sideband;
+
+	/*
+	 * A pointer which if provided will be set to 1 or 0 depending
+	 * on if a hook was invoked (i.e. existed), regardless of
+	 * whether or not that was successful. Used for avoiding
+	 * TOCTOU races in code that would otherwise call hook_exist()
+	 * after a "maybe hook run" to see if a hook was invoked.
+	 */
+	int *invoked_hook;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
@@ -90,6 +99,7 @@ struct hook_cb_data {
 	const char *hook_name;
 	struct hook *run_me;
 	struct run_hooks_opt *options;
+	int *invoked_hook;
 };
 
 void run_hooks_opt_clear(struct run_hooks_opt *o);
diff --git a/sequencer.c b/sequencer.c
index 77f809c00e4..f451e23d0c1 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1203,7 +1203,7 @@ static int run_prepare_commit_msg_hook(struct repository *r,
 	} else {
 		arg1 = "message";
 	}
-	if (run_commit_hook(0, r->index_file, "prepare-commit-msg", name,
+	if (run_commit_hook(0, r->index_file, NULL, "prepare-commit-msg", name,
 			    arg1, arg2, NULL))
 		ret = error(_("'prepare-commit-msg' hook failed"));
 
@@ -1533,7 +1533,7 @@ static int try_to_commit(struct repository *r,
 		goto out;
 	}
 
-	run_commit_hook(0, r->index_file, "post-commit", NULL);
+	run_commit_hook(0, r->index_file, NULL, "post-commit", NULL);
 	if (flags & AMEND_MSG)
 		commit_post_rewrite(r, current_head, oid);
 
-- 
2.33.0.rc0.595.ge31e012651d


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* Re: [PATCH v4 08/36] hook: add 'run' subcommand
  2021-08-03 19:38             ` [PATCH v4 08/36] hook: add 'run' subcommand Ævar Arnfjörð Bjarmason
@ 2021-08-04 10:15               ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-04 10:15 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason


On Tue, Aug 03 2021, Ævar Arnfjörð Bjarmason wrote:

> From: Emily Shaffer <emilyshaffer@google.com>
> [...]
> diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
> new file mode 100755
> index 00000000000..644df0a583c
> --- /dev/null
> +++ b/t/t1800-hook.sh
> @@ -0,0 +1,133 @@
> +#!/bin/bash
> +

It appears that since at least May 2020 and across N re-rolls of this
topic nobody (including me) has noticed that this should be #!/bin/sh,
not #!/bin/bash.

I've got it fixed up locally, but thought it was not worth another
36-patch re-roll pending further feedback.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 8/9] hook: teach 'hookcmd' config to alias hook scripts
  2021-07-23  7:41                   ` Ævar Arnfjörð Bjarmason
@ 2021-08-04 20:38                     ` Emily Shaffer
  2021-08-05  0:17                       ` Ævar Arnfjörð Bjarmason
  2021-08-04 21:49                     ` Jonathan Tan
  1 sibling, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-08-04 20:38 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Fri, Jul 23, 2021 at 09:41:35AM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Thu, Jul 22 2021, Emily Shaffer wrote:
> 
> > On Fri, Jul 16, 2021 at 11:13:42AM +0200, Ævar Arnfjörð Bjarmason wrote:
> >> 
> >> 
> >> On Thu, Jul 15 2021, Emily Shaffer wrote:
> >> 
> >> > To enable fine-grained options which apply to a single hook executable,
> >> > and to make it easier for a single executable to be run on multiple hook
> >> > events, teach "hookcmd.<alias>.config". These can be configured as
> >> > follows:
> >> > [...]
> >> > diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
> >> > index a97b980cca..5b35170664 100644
> >> > --- a/Documentation/config/hook.txt
> >> > +++ b/Documentation/config/hook.txt
> >> > @@ -3,6 +3,11 @@ hook.<command>.command::
> >> >  	executable on your device, a oneliner for your shell, or the name of a
> >> >  	hookcmd. See linkgit:git-hook[1].
> >> >  
> >> > +hookcmd.<name>.command::
> >> > +	A command to execute during a hook for which <name> has been specified
> >> > +	as a command. This can be an executable on your device or a oneliner for
> >> > +	your shell. See linkgit:git-hook[1].
> >> > +
> >> > [...]
> >> > +Global config
> >> > +----
> >> > +  [hook "post-commit"]
> >> > +    command = "linter"
> >> > +    command = "~/typocheck.sh"
> >> > +
> >> > +  [hookcmd "linter"]
> >> > +    command = "/bin/linter --c"
> >> > +----
> >> > +
> >> > +Local config
> >> > +----
> >> > +  [hook "prepare-commit-msg"]
> >> > +    command = "linter"
> >> > +  [hook "post-commit"]
> >> > +    command = "python ~/run-test-suite.py"
> >> > +----
> >> > +
> >> > +With these configs, you'd then run post-commit hooks in this order:
> >> > +
> >> > +  /bin/linter --c
> >> > +  ~/typocheck.sh
> >> > +  python ~/run-test-suite.py
> >> > +  .git/hooks/post-commit (if present)
> >> > +
> >> > +and prepare-commit-msg hooks in this order:
> >> > +
> >> > +  /bin/linter --c
> >> > +  .git/hooks/prepare-commit-msg (if present)
> >> >  
> >> 
> >> I still have outstanding feedback on the fundamental design
> >> here. I.e. why is this not:
> >> 
> >>     hook.<name>.event = post-commit
> >>     hook.<name>.command = <path>
> >> 
> >> See:
> >> 
> >> https://lore.kernel.org/git/87mtv8fww3.fsf@evledraar.gmail.com/
> >> 
> >> As noted there I don't see why not, and more complexity results from the
> >> design choice of doing it the way you're proposing. I.e. we can't
> >> discover hooks based on config prefixes, and we end up sticking full FS
> >> paths in config keys.
> >
> > Interesting. My gut says that it would make it harder to neatly write a
> > config with the same hook running at the very beginning of one event and
> > the very end of another, but I'm not sure whether that's a case to
> > optimize for.
> >
> > I would also need twice as many lines to run a script I wrote as a hook
> > - that is, the base case which is probably all most people will need. So
> > with your proposal I *must* name every hook I want to add. Instead of
> > "hook.pre-commit.command = ~/check-for-debug-strings" I need to
> > configure "hook.check-debug-strings.event = pre-commit" and
> > "hook.check-debug-strings.command = ~/check-for-debug-strings". That's a
> > little frustrating, and if I never want to skip that check or move it
> > later or something, it will be extra overhead for no gain for me.
> 
> The gain is that "git hook list" becomes a trivial "git config
> -show-origin --show-scope --get-regexp" wrapper.

This isn't a very compelling reason to me, if it makes the user
experience worse. (I'm not convinced that it does - just saying I
disagree with this reasoning.)

> 
> So the series either doesn't need "git hook list" or such a thing
> becomes much less complex, especially given the proposed addition of
> other features in the area like "git hook edit", i.e. (quoting the
> linked E-Mail):

I still think "git hook list" is useful to have for end-users, compared
to remembering the appropriate "git config" invocation. It futureproofs
us if we do want to change the execution ordering to something besides
config order, it's easier to remember/more discoverable, etc.

>     As just one example; surely "git config edit <name>" would need to
>     run around and find config files to edit, then open them in a loop
>     for you, no?
> 
>     Which we'd eventually want for "git config" in general with an
>     --edit-regexp option or whatever, which brings us (well, at least
>     me) back to "then let's just add it to git-config?".
> 
> > I do agree that your approach bans the regrettably awkward
> > "hookcmd.~/check-for-debug-strings.skip = true", but I'm not sure
> > whether or not it's worth it.
> 
> That design choice also means that you can't expand the path using "git
> config --get --type=path.

Yeah, this is a pretty handy point.

> 
> We do have that with the "includeIf" construct, but if we can avoid it
> we should, it makes it play nicer with other assumptions and features of
> the config system.
> 
> As noted in the follow-up reply while we don't case normalize the LeVeL"
> part of "ThReE.LeVeL.KeY" that's tolower(), which we know isn't a 1=1
> mapping on some
> FS's. https://lore.kernel.org/git/87y2ebo16v.fsf@evledraar.gmail.com/

This, too.

> 
> > To help us visualize the change, here's some common and complicated
> > tasks and how they look with either schema (let mine be A and yours be
> > B):
> 
> Before diving into that, I'll just say I don't care about the trivial
> specifics of how this is done, i.e. the bikeshedding of what the config
> keys etc. are named.
> 
> Just in (as noted above) design choices here forcing avoidable
> complexity in other areas.
> 
> > #1 - Add a oneliner (not executing a script)
> > A:
> > [hook "post-commit"]
> > 	command = echo post commit
> > B:
> > [hook "oneliner"]
> > 	event = post-commit
> > 	command = echo post commit
> > #2 - Execute a script
> > A:
> > [hook "post-commit"]
> > 	command = ~/my-post-commit-hook
> > B:
> > [hook "script"]
> > 	event = post-commit
> > 	command = ~/my-post-commit-hook
> 
> ...
> 
> > #3 - Run a handful of scripts in a specific order on one event
> > A:
> > [hook "post-commit"]
> > 	command = ~/my-post-commit-1
> > 	command = ~/my-post-commit-2
> > 	command = ~/my-post-commit-3
> > B:
> > [hook "script 1"]
> > 	event = post-commit
> > 	command = ~/my-post-commit-1
> > [hook "script 2"]
> > 	event = post-commit
> > 	command = ~/my-post-commit-2
> > [hook "script 3"]
> > 	event = post-commit
> > 	command = ~/my-post-commit-3
> 
> To reply to all the above, yes, your suggestion comes out ahead in being
> less verbose.
> 
> But isn't the real difference not in the differing prefixes, i.e. hook.*
> and hookcmd.* (A) v.s. always hook.* (B, which is what I'm mainly
> focusing on, i.e. what requires the added complexity.
> 
> But that in that in your proposed way of doing it it's:
> 
>     hook.<event-or-name>.*
> 
> V.s. my suggestion of:
> 
>     hook.<name>.*
> 

No, I think that's a misunderstanding of proposal A. My proposal A is
always hook.<event>.*, and hookcmd.<name>.*, and the way that I read
your proposal B is that it's always hook.<name>.*.

> And thus whenever you have a <event-or-name> that just happens to be a
> built-in hook listed in githooks(5) we in (A) implicitly expand config
> like:
> 
>     hook.post-commit.command = echo foo
> 
> 
>     hook.post-commit.command = echo hi
>     hook.post-commit.type    = post-commit

This is where I'm becoming confused. I *think* you're saying that with
your proposal B, if the subsection happens to be the name of a built-in
hook, we can skip the .event field. But it also reads like you're saying
this is how my proposal A works, which isn't the case, because the
subsection is always assumed to be the name of the hook event we're
trying to run.

> 
> But not knowing about "foo" we don't expand:
> 
>     hook.foo.command = echo foo
> 
> 
>     hook.foo.command = echo hi
>     hook.foo.type    = foo # This would be an error, or ignored.
> 
> But rather leave "dangling" for the user to later supply the "*.event"
> themselves, i.e.:
> 
>     hook.foo.command = echo hi
>     hook.foo.event = post-commit

If you're talking about proposal B, then no, this isn't how it works.
This config would imply that someone else (like a wrapper) can run "echo
hi" by asking for "git hook run foo". But I'm not sure exactly that
that's what you mean...

> 
> And means that you presumably need to detect this case and error about
> it, but my proposed model does not:
> 
>     hook.post-commit.command = echo hi
>     # User error: "*.type" and <event-or-name>" does not match for
>     # "hook.*.command"
>     hook.post-commit.type    = pre-commit
> 
> And furthermore, that if someone in your model were to do:
> 
>     hook.verify-commit.command = echo hi
> 
> It's "dangling" today, but if a later version of git learns about a
> "verify-commit" hook we'll start running it unexpectedly.

No, that's in fact as designed, with my model B. The user configured
"echo hi" to run on "verify-commit" events; if those events are
initially used by some wrapper, but later we decide they're a great idea
and absorb the verify-commit event into native Git, then this is working
as intended. I think your argument is based on a misunderstanding of the
design.

Could it be that the way I provided the examples (my schema after A: and
your schema after B:) made it hard to parse? Sorry about that if so.

> 
> Because your design conflates the slot for known hook type names and
> user-supplied names.
> 
> On balance I think it's better just to always supply two lines per-hook,
> but whether we have this proposed shorthand or not is mostly orthogonal
> to everything I mentioned in
> https://lore.kernel.org/git/87mtv8fww3.fsf@evledraar.gmail.com/
> 
> I.e. my proposed version could also have it, but thinking about it I
> think it's not worth it, we should always use <name>, not
> <event-or-name> for the reasons noted if you'll read ahead...
> 
> > #4 - Skip a script configured more globally
> > A:
> > (original config)
> > [hook "post-commit"]
> > 	command = /corp/stuff/corp-post-commit
> > (local config)
> > [hookcmd "/corp/stuff/corp-post-commit"]
> > 	skip = true
> > OR,
> > (original config)
> > [hookcmd "corp-pc"]
> > 	command = /corp/stuff/corp-post-commit
> > [hook "post-commit"]
> > 	command = corp-pc
> > (local config)
> > [hookcmd "corp-pc"]
> > 	skip = true
> > B:
> > (original config)
> > [hook "corp-pc"]
> > 	event = post-commit
> > 	command = /corp/stuff/corp-post-commit
> > (local config)
> > [hook "corp-pc"]
> > 	skip = true
> 
> ...which are among other things that no, my proposed version doesn't
> have "skip" at all, see point #3 at
> https://lore.kernel.org/git/87mtv8fww3.fsf@evledraar.gmail.com/
> 
> I.e. I think the "skip" is a thing that falls out of a complexity of
> your design that I'm proposing to do away with.
> 
> That complexity being that you use <event-or-name> and I use <name>, and
> you want to in turn support N number of "*.command" for any
> "hook.<event-or-name>".
> 
> I'm suggesting we use the typical "last key wins" semantics, if you want
> multiple commands for a given hook type you'll just supply multiple
> "hook.<name>" sections with the same "hook.*.event = type" in all.
> 
> The way to skip hooks in my proposal is:
> 
>     hook.<name>.command = true

I see - instead of fiddly reorganizing the list, you're just changing
what the specific hook name is redirecting to.

Then, instead of storing the resolved command in the 'struct hook', we
can store the name, and always dereference the name right before
execution, using last-one-wins semantics.

The only thing I don't like about this is that it's a little bit
confusing to the user. I personally would rather have more complicated
code and less complicated user experience, so I like 'skip' more because
it's readable and because you don't need to guess "oh, it'll just
execute /bin/true which is a noop". That's easy to understand for
someone familiar with Unix scripting (and therefore for we Git
contributors), but confusing for everybody else.

Maybe we can take the essential spirit of this implementation but make
it a little bit simpler. In pseudocode:

populate_list(hookname):
  # look for everybody who says "hook.*.event = $hookname"
  hooks = get_configs_matching(hook.*.event, hookname)
  # the "*" bit is the bit we dereference later
  for hook in hooks:
    hook_list.add({.name = hook.key.subsection})

run_hooks(hook_list):
  for hook in hook_list:
    # should we even run?
    if (get_last_config("hook." + hook.name + ".skip"))
      continue

    # dereference the hook name and find out the command
    cmd = get_last_config("hook." + hook.name + ".command")
    run(cmd)

The price for this improved readability is an extra config lookup in
each case. But the code is still readable, definitely moreso than
rewalking the config list over and over as the implementation stands
now. I personally think it's worth it.

(One could also imagine a more readable placeholder than /bin/true in
the "hook.<name>.command" field, but then we get to decide just how hard
we want to avoid colliding with legitimate user hook names. Is .command
= skip enough? is .command = RESERVED_NAME_SKIP_THIS_HOOK enough? Is
something unrunnable like .command = "skip me" enough, although it could
conceivably collide with a determined user?)

> 
> Or whatever noop command you want on the RHS. In practice we wouldn't
> invoke "true", just like we don't invoke "cat" for "PAGER=cat".
> 
> But unlike "*.skip" that doesn't require complexity in implementation or
> user understanding, it just falls naturally out of the rest of the
> model.
> 
> > #5 - Execute one script for multiple events
> > A:
> > [hookcmd "reusable-hook"]
> > 	command = /long/path/reusable-hook
> > [hook "post-commit"]
> > 	command = reusable-hook
> > [hook "pre-commit"]
> > 	command = reusable-hook
> > [hook "prepare-commit-msg"]
> > 	command = reusable-hook
> > B:
> > [hook "reusable-hook"]
> > 	command = /long/path/reusable-hook
> > 	event = post-commit
> > 	event = pre-commit
> > 	event = prepare-commit-msg
> 
> It's been so long since I wrote the original E-Mail that I'm having to
> skim my own summary there & come up with things as I go along, so maybe
> this conflicts with something I said earlier.
> 
> But no, I don't think I meant that *.event should be multi-value, or
> that that magic is worthwhile. I.e. I think we just want:
> 
>     [hook "not-a-reusable-section-1"]
>         command = /long/path/reusable-hook
>         event = post-commit
>     [hook "not-a-reusable-section-2"]
>         command = /long/path/reusable-hook
>         event = pre-commit
>     [hook "not-a-reusable-section-3"]
>         command = /long/path/reusable-hook
>         event = prepare-commit-msg
> 
> I.e. is wanting to use the same command for N different hooks really
> that common a use-case to avoid a little verbosity, and quite a lot of
> complexity?
> 
> How would such a hook even be implemented?
> 
> We don't have anything in your hook execution model (and I don't
> remember you adding this) which passes down a "you are this type of
> hook" to hooks.

No, but it's something I'm interested in passing as an environment
variable. I didn't add it to this series because it seemed to me to
distract from the core feature. We would like to add it to simplify our
invocations of https://github.com/awslabs/git-secrets, so it's on my
radar.

And even without this, just today I wanted to configure a hook to really
make sure I had config.mak set up in all my Git worktrees (since you
pointed out I pushed code that fails -Wall.... ;) ) and set it up to run
on both pre-commit and post-checkout. So yes, it is feasible now, for
very stupid hooks.

> 
> It's now implicit in that the hook invoked at .git/hooks/post-commit or
> .git/hooks/pre-commit can check its own basename, but it won't be with
> configurable hooks.
> 
> We could start passing down a GIT_HOOK_TYPE in the environment or
> whatever, but I think the simpler case of just having the user do it is
> better.
> 
> I'm assuming that mainly because people have wanted a
> "/long/path/reusable-hook" router script because we have not supported
> executing N hooks, or supported concurrency. Once we do that the
> complexity of such routing scripts (mostly, not everyone's needs will be
> met) will be replaced by a little bit of config.

The above linked git-secrets is an example; the config is irritating and
doesn't really need to be there, when we could just teach the script
itself how to check GIT_HOOK_TYPE envvar.

> 
> I don't see why it's worth it to micro-optimize for lines in that config
> at the cost of increased config complexity.
> 
> For example, how do you "skip" an "event" type? You have it for
> "*.command"? So let's say in your model (A) I have system config like:
(The following isn't how my model A works, because it refers always to
"hook.<event>", as I mentioned above. But I also covered this same topic
in detail earlier in this email, so I'm going to snip this section.)

> > #6 - Execute the same script early for one event and late for another
> > event
> > A:
> > (global)
> > [hookcmd "reusable-hook"]
> > 	command = /long/path/reusable-hook
> > [hook "pre-commit"]
> > 	command = reusable-hook
> > (local)
> > [hook "post-commit"]
> > 	command = other
> > 	command = hooks
> > 	command = reusable-hook
> 
> Even with I think it's fair to say deep knowledge of your proposal at
> this point I still needed to read this a few times to see if that:
> 
>     command = reusable-hook
> 
> Is referring to:
> 
>     [hookcmd "reusable-hook"]
> 
> I.e. is it going to run:
> 
>     command = /long/path/reusable-hook
> 
> Or is it just re-specifying /long/path/reusable-hook but relying on a
> $PATH lookup?
> 
> Having reasoned through that I think the answer is the former. But that
> also means that in your model:
> 
>     [hookcmd "rm -rf /"]
>     command = echo this will not actually destroy your data
>     [hook "pre-commit"]
>     command = rm -rf /
> 
> Is going to run that friendly "echo" command, since "command = rm -rf /"
> just refers to the "rm -rf /" <name>, not <command>, right the "hookcmd"
> line is removed, at which point we'll stop treating it as a <name> and
> run it as a <command>?
> 
> In practice I think users are unlikely to use actively harmful names
> like that.
> 
> I'm just making the point that I should not need to know about previous
> config to see if a "hook.pre-commit.command = rm -rf /" is harmless or
> not, or need to carefully squint to see if the "reusable-hook" is
> referring to a section name or command name.
> 
> Or am I just confused at this point?

Nah, that's an accurate understanding of how it would work with the
current implementation. Sure, it's a good point.

> 
> > B:
> > (global)
> > [hook "reusable-hook"]
> > 	command = /long/path/reusable-hook
> > 	event = pre-commit
> > (local)
> > [hook "other"]
> > 	event = post-commit
> > 	command = other
> > [hook "hooks"]
> > 	event = post-commit
> > 	command = hooks
> > [hook "reusable-hook"]
> > 	event = reusable-hook

Hm, I think I mistyped when I wrote "event = reusable-hook" here, and
meant to write "event = post-commit" to indicate that we would move
"reusable-hook" to the bottom of the execution list for "post-commit".

I'll snip most of the rebuttal about my typo.

> 
>         (global)
> 	[hook "reusable-hook"]
> 		command = /long/path/reusable-hook
> 		event = pre-commit
> 
> 	(local)
> 	[hook "reusable-hook"]
> 		command = true # skip it
> 
>         # The "hooks" name is arbitrary, "my-hooks" or whatever would be
>         # clearer, but just going with your example...
> 
> 	[hook "hooks"]
> 		event = post-commit
> 		command = hooks
>         # Not very reusable then...   
> 	[hook "reusable-hook"]
>                 command = /long/path/reusable-hook
> 		event = pre-commit
> 
> 
> > [...]Please feel free to chime in with more use cases that you think would
> > be handy which I forgot :)
> 
> I couldn't find this at a quick glance but I think we also had a
> disussion at some point about hooks controlling parallelism. AFAICT your
> current implementation just has global:
> 
>     hook.jobs=N
> 
> And we then blacklist certain hooks to always have hook.jobs=1, e.g. the
> commit-msg hook that needs an implicit "lock" on that file (or rather,
> we think that's the most common use-case).
> 
> I think my version of always having hook.<name>.{event,command} be one
> value is also better in that case, i.e. we'd then:
> 
>     [hook "myhook"]
>     command = some-command
>     event = pre-receive
>     parallel = true # the default
> 
>     [hook "myhook2"]
>     command = some-command2
>     event = pre-receive
>     parallel = true # the default
> 
>     [hook "myhook3"]
>     command = some-unconcurrent-command
>     event = pre-receive
>     parallel = false # I'm not OK with concurrency

I am not sure what it means for a single executable to write "parallel =
true" - it is a single executable.

Ok, that is me being facetious - I think you are saying we can AND
together all of the 'hook.<thing-with-event-we-care-about>.parallel' to
decide whether or not to run in parallel.

I would rather not discuss this now, for this series, because regardless
of which config schema we use today, we can figure out "parallel unless
we really don't want it" later on. It is too complex to discuss in the
context of "hey, we should also configure hooks somewhere else". Let's
leave it for future work.

> I haven't thought about it deeply, but have a hunch that having sections
> be a 1=1 mapping with a specific command instead of either 1=1 or 1=many
> is going to make that easier to implement and understand.

Sure, probably. But it can be added later to either schema... ;)


Anyway, I did a quick strawpoll with my colleagues (Jonathan Nieder,
Jonathan Tan, and Josh Steadmon) and they all like your syntax more. I
am ambivalent between the two, personally.

So what I will do, hopefully today, maybe not, is the following:

 - no more hookcmd. (and there was much rejoicing)
 - Hooks are specified with "[hook "name-of-hook"]"
 - I do see value in having an explicit .skip field rather than mapping
   .command to a noop, so "hook.name-of-hook.skip" as described above.
   Of course the method you described will work regardless, since its
   mechanism is based on the inherent result of executing /bin/true.
 - I do see value in allowing "hook.name-of-hook.event" to be defined
   repeatably, as described above, so I will include that.

And left for later work:
 - teaching HOOK_EVENT_NAME=post-commit (etc)
 - figuring out what the heck we want to do with allowing hooks to
   describe whether they allow parallelism

Unfortunately I've dropped the design doc from this series, since nobody
seemed interested in having it checked in, but I'll try to rework the
help doc to make the schema more clear.

Thanks, I think it was useful to hash this out.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 8/9] hook: teach 'hookcmd' config to alias hook scripts
  2021-07-23  7:41                   ` Ævar Arnfjörð Bjarmason
  2021-08-04 20:38                     ` Emily Shaffer
@ 2021-08-04 21:49                     ` Jonathan Tan
  1 sibling, 0 replies; 479+ messages in thread
From: Jonathan Tan @ 2021-08-04 21:49 UTC (permalink / raw)
  To: avarab; +Cc: emilyshaffer, git, Jonathan Tan

> > I would also need twice as many lines to run a script I wrote as a hook
> > - that is, the base case which is probably all most people will need. So
> > with your proposal I *must* name every hook I want to add. Instead of
> > "hook.pre-commit.command = ~/check-for-debug-strings" I need to
> > configure "hook.check-debug-strings.event = pre-commit" and
> > "hook.check-debug-strings.command = ~/check-for-debug-strings". That's a
> > little frustrating, and if I never want to skip that check or move it
> > later or something, it will be extra overhead for no gain for me.
> 
> The gain is that "git hook list" becomes a trivial "git config
> -show-origin --show-scope --get-regexp" wrapper.

I think that in both schemes, "git hook list" is not difficult to
implement.

> > I do agree that your approach bans the regrettably awkward
> > "hookcmd.~/check-for-debug-strings.skip = true", but I'm not sure
> > whether or not it's worth it.
> 
> That design choice also means that you can't expand the path using "git
> config --get --type=path.

I'm not sure what the use of this is, especially when the more important
position is in "hook.pre-commit.command" (which can be expanded, I
believe).

> As noted in the follow-up reply while we don't case normalize the LeVeL"
> part of "ThReE.LeVeL.KeY" that's tolower(), which we know isn't a 1=1
> mapping on some
> FS's. https://lore.kernel.org/git/87y2ebo16v.fsf@evledraar.gmail.com/

We're comparing strings in the config file, though, not strings against
what's on the filesystem.

> To reply to all the above, yes, your suggestion comes out ahead in being
> less verbose.
> 
> But isn't the real difference not in the differing prefixes, i.e. hook.*
> and hookcmd.* (A) v.s. always hook.* (B, which is what I'm mainly
> focusing on, i.e. what requires the added complexity.

OK, this is a good point - it may be confusing for the user to remember
the difference between "hook" and "hookcmd", and your scheme eliminates
that.

> But that in that in your proposed way of doing it it's:
> 
>     hook.<event-or-name>.*
> 
> V.s. my suggestion of:
> 
>     hook.<name>.*

[snip discussion of how hook.<event-or-name> is bad]

I think you were thinking that someone would say "let's unify 'hook' and
'hookcmd' then" in response to you saying "'hook' vs 'hookcmd' is bad",
and you're arguing against this new point. I don't think anyone is
saying that, though.

> That complexity being that you use <event-or-name> and I use <name>, and
> you want to in turn support N number of "*.command" for any
> "hook.<event-or-name>".

I don't know if anyone is wanting to support this.

> The way to skip hooks in my proposal is:
> 
>     hook.<name>.command = true
> 
> Or whatever noop command you want on the RHS. In practice we wouldn't
> invoke "true", just like we don't invoke "cat" for "PAGER=cat".
> 
> But unlike "*.skip" that doesn't require complexity in implementation or
> user understanding, it just falls naturally out of the rest of the
> model.

This is a good point.

> Even with I think it's fair to say deep knowledge of your proposal at
> this point I still needed to read this a few times to see if that:
> 
>     command = reusable-hook
> 
> Is referring to:
> 
>     [hookcmd "reusable-hook"]
> 
> I.e. is it going to run:
> 
>     command = /long/path/reusable-hook
> 
> Or is it just re-specifying /long/path/reusable-hook but relying on a
> $PATH lookup?

This is true - seeing "hook.post-commit.command = reusable-hook", we
need to look at the rest of the config to see how it is interpreted.

Going back to the central idea, though, I think that the main advantage
of the scheme Emily proposed is the ability to write, more concisely:

  [hook.pre-commit]
  command = /path/to/command-1
  command = /path/to/command-2

instead of

  [hook.command-1]
  event = pre-commit
  command = /path/to/command-1
  [hook.command-2]
  event = pre-commit
  command = /path/to/command-2

But Ævar's scheme gives us the advantage that if need to do anything
more complicated (even merely slightly more complicated - for example,
skipping a hook or overriding the command of a hook), it would be hard
to write (and even to just figure it out) in Emily's scheme. In Ævar's
scheme, just following the standard "last config wins" rule (and knowing
about /usr/bin/true) can already do a lot. For this reason, I think it's
worth considering Ævar's scheme - writing 3 lines instead of 2 is not
much more difficult to teach and to do.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 8/9] hook: teach 'hookcmd' config to alias hook scripts
  2021-08-04 20:38                     ` Emily Shaffer
@ 2021-08-05  0:17                       ` Ævar Arnfjörð Bjarmason
  2021-08-05 21:45                         ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-05  0:17 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Wed, Aug 04 2021, Emily Shaffer wrote:

> On Fri, Jul 23, 2021 at 09:41:35AM +0200, Ævar Arnfjörð Bjarmason wrote:
>> 
>> 
>> On Thu, Jul 22 2021, Emily Shaffer wrote:
>> 
>> > On Fri, Jul 16, 2021 at 11:13:42AM +0200, Ævar Arnfjörð Bjarmason wrote:
>> >> 
>> >> 
>> >> On Thu, Jul 15 2021, Emily Shaffer wrote:
>> >> 
>> >> > To enable fine-grained options which apply to a single hook executable,
>> >> > and to make it easier for a single executable to be run on multiple hook
>> >> > events, teach "hookcmd.<alias>.config". These can be configured as
>> >> > follows:
>> >> > [...]
>> >> > diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
>> >> > index a97b980cca..5b35170664 100644
>> >> > --- a/Documentation/config/hook.txt
>> >> > +++ b/Documentation/config/hook.txt
>> >> > @@ -3,6 +3,11 @@ hook.<command>.command::
>> >> >  	executable on your device, a oneliner for your shell, or the name of a
>> >> >  	hookcmd. See linkgit:git-hook[1].
>> >> >  
>> >> > +hookcmd.<name>.command::
>> >> > +	A command to execute during a hook for which <name> has been specified
>> >> > +	as a command. This can be an executable on your device or a oneliner for
>> >> > +	your shell. See linkgit:git-hook[1].
>> >> > +
>> >> > [...]
>> >> > +Global config
>> >> > +----
>> >> > +  [hook "post-commit"]
>> >> > +    command = "linter"
>> >> > +    command = "~/typocheck.sh"
>> >> > +
>> >> > +  [hookcmd "linter"]
>> >> > +    command = "/bin/linter --c"
>> >> > +----
>> >> > +
>> >> > +Local config
>> >> > +----
>> >> > +  [hook "prepare-commit-msg"]
>> >> > +    command = "linter"
>> >> > +  [hook "post-commit"]
>> >> > +    command = "python ~/run-test-suite.py"
>> >> > +----
>> >> > +
>> >> > +With these configs, you'd then run post-commit hooks in this order:
>> >> > +
>> >> > +  /bin/linter --c
>> >> > +  ~/typocheck.sh
>> >> > +  python ~/run-test-suite.py
>> >> > +  .git/hooks/post-commit (if present)
>> >> > +
>> >> > +and prepare-commit-msg hooks in this order:
>> >> > +
>> >> > +  /bin/linter --c
>> >> > +  .git/hooks/prepare-commit-msg (if present)
>> >> >  
>> >> 
>> >> I still have outstanding feedback on the fundamental design
>> >> here. I.e. why is this not:
>> >> 
>> >>     hook.<name>.event = post-commit
>> >>     hook.<name>.command = <path>
>> >> 
>> >> See:
>> >> 
>> >> https://lore.kernel.org/git/87mtv8fww3.fsf@evledraar.gmail.com/
>> >> 
>> >> As noted there I don't see why not, and more complexity results from the
>> >> design choice of doing it the way you're proposing. I.e. we can't
>> >> discover hooks based on config prefixes, and we end up sticking full FS
>> >> paths in config keys.
>> >
>> > Interesting. My gut says that it would make it harder to neatly write a
>> > config with the same hook running at the very beginning of one event and
>> > the very end of another, but I'm not sure whether that's a case to
>> > optimize for.
>> >
>> > I would also need twice as many lines to run a script I wrote as a hook
>> > - that is, the base case which is probably all most people will need. So
>> > with your proposal I *must* name every hook I want to add. Instead of
>> > "hook.pre-commit.command = ~/check-for-debug-strings" I need to
>> > configure "hook.check-debug-strings.event = pre-commit" and
>> > "hook.check-debug-strings.command = ~/check-for-debug-strings". That's a
>> > little frustrating, and if I never want to skip that check or move it
>> > later or something, it will be extra overhead for no gain for me.
>> 
>> The gain is that "git hook list" becomes a trivial "git config
>> -show-origin --show-scope --get-regexp" wrapper.
>
> This isn't a very compelling reason to me, if it makes the user
> experience worse. (I'm not convinced that it does - just saying I
> disagree with this reasoning.)

...(see below)...

>> 
>> So the series either doesn't need "git hook list" or such a thing
>> becomes much less complex, especially given the proposed addition of
>> other features in the area like "git hook edit", i.e. (quoting the
>> linked E-Mail):
>
> I still think "git hook list" is useful to have for end-users, compared
> to remembering the appropriate "git config" invocation. It futureproofs
> us if we do want to change the execution ordering to something besides
> config order, it's easier to remember/more discoverable, etc.

To clarify: I'm not against there being a "git hook list", I think we
should have it.

It's useful UI. I.e. we have "run", it makes sense to have the same
mechanism spew out its idea of what hooks are where, especially since
that's not entirely contained in the config, but a union of config and
GIT_DIR/hooks (and then there's core.hooksPath...).

So I'm very much for it being there.

I've just been commenting on the relative complexity of the config
schema, which in one way surfaces as not being able to do the config
part of the "git hook list" as a simple shell-out (or equivalent library
invocation) of "git config" under the hood, but also means that users
need to mentally track a more complex format when e.g. manually editing
config...

>> [...]
>> But isn't the real difference not in the differing prefixes, i.e. hook.*
>> and hookcmd.* (A) v.s. always hook.* (B, which is what I'm mainly
>> focusing on, i.e. what requires the added complexity.
>> 
>> But that in that in your proposed way of doing it it's:
>> 
>>     hook.<event-or-name>.*
>> 
>> V.s. my suggestion of:
>> 
>>     hook.<name>.*
>> 
>
> No, I think that's a misunderstanding of proposal A. My proposal A is
> always hook.<event>.*, and hookcmd.<name>.*, and the way that I read
> your proposal B is that it's always hook.<name>.*.

Ah, thanks.

>> And thus whenever you have a <event-or-name> that just happens to be a
>> built-in hook listed in githooks(5) we in (A) implicitly expand config
>> like:
>> 
>>     hook.post-commit.command = echo foo
>> 
>> 
>>     hook.post-commit.command = echo hi
>>     hook.post-commit.type    = post-commit
>
> This is where I'm becoming confused. I *think* you're saying that with
> your proposal B, if the subsection happens to be the name of a built-in
> hook, we can skip the .event field. But it also reads like you're saying
> this is how my proposal A works, which isn't the case, because the
> subsection is always assumed to be the name of the hook event we're
> trying to run.

I *think* this is best addressed below, moving on...

>> 
>> But not knowing about "foo" we don't expand:
>> 
>>     hook.foo.command = echo foo
>> 
>> 
>>     hook.foo.command = echo hi
>>     hook.foo.type    = foo # This would be an error, or ignored.
>> 
>> But rather leave "dangling" for the user to later supply the "*.event"
>> themselves, i.e.:
>> 
>>     hook.foo.command = echo hi
>>     hook.foo.event = post-commit
>
> If you're talking about proposal B, then no, this isn't how it works.
> This config would imply that someone else (like a wrapper) can run "echo
> hi" by asking for "git hook run foo". But I'm not sure exactly that
> that's what you mean...

...and on (maybe)....

>> 
>> And means that you presumably need to detect this case and error about
>> it, but my proposed model does not:
>> 
>>     hook.post-commit.command = echo hi
>>     # User error: "*.type" and <event-or-name>" does not match for
>>     # "hook.*.command"
>>     hook.post-commit.type    = pre-commit
>> 
>> And furthermore, that if someone in your model were to do:
>> 
>>     hook.verify-commit.command = echo hi
>> 
>> It's "dangling" today, but if a later version of git learns about a
>> "verify-commit" hook we'll start running it unexpectedly.
>
> No, that's in fact as designed, with my model B. The user configured
> "echo hi" to run on "verify-commit" events; if those events are
> initially used by some wrapper, but later we decide they're a great idea
> and absorb the verify-commit event into native Git, then this is working
> as intended. I think your argument is based on a misunderstanding of the
> design.
>
> Could it be that the way I provided the examples (my schema after A: and
> your schema after B:) made it hard to parse? Sorry about that if so.

Aren't you assuming that users who specify a verify-commit will be happy
because git's usurping of that will 1=1 match what they were using
"verify-commit" for.

I'm pointing out that we can't know that, and since you want to make
"git hook run" a general thing that runs any <name> of script you've
configured, and not just what's in githooks(5) that it becomes very
likely that if we add a new hook with some obvious name that we'll
either break things for users, or subtly change behavior.

Which isn't just theoretical, e.g. I tend to run something like a "git
log --check @{u}.." before I run git-send-email, with this configurable
hook mechanism having a "git hook run sendemail-check" would be a way I
might expose that in my own ~/.gitconfig.

But if git-send-email learns a "sendemail-check" and the behavior
doesn't exactly match mine; E.g. maybe it similar to pre-auto-gc expects
me to return a status code to ask me if I want to abort on a failed
--check, but mine expects a revision range to run "log --check".

In practice that's a non-issue with the current hook mechanism,
i.e. nobody's sticking a script into .git/hooks/my-custom-name and
expecting it to do anything useful (and if they are, they have only
themselves to blame).

Whereas we'd now be actively inviting users to squat on the same
namespace we ourselves will need for future hooks.

>> 
>> Because your design conflates the slot for known hook type names and
>> user-supplied names.
>> 
>> On balance I think it's better just to always supply two lines per-hook,
>> but whether we have this proposed shorthand or not is mostly orthogonal
>> to everything I mentioned in
>> https://lore.kernel.org/git/87mtv8fww3.fsf@evledraar.gmail.com/
>> 
>> I.e. my proposed version could also have it, but thinking about it I
>> think it's not worth it, we should always use <name>, not
>> <event-or-name> for the reasons noted if you'll read ahead...
>> 
>> > #4 - Skip a script configured more globally
>> > A:
>> > (original config)
>> > [hook "post-commit"]
>> > 	command = /corp/stuff/corp-post-commit
>> > (local config)
>> > [hookcmd "/corp/stuff/corp-post-commit"]
>> > 	skip = true
>> > OR,
>> > (original config)
>> > [hookcmd "corp-pc"]
>> > 	command = /corp/stuff/corp-post-commit
>> > [hook "post-commit"]
>> > 	command = corp-pc
>> > (local config)
>> > [hookcmd "corp-pc"]
>> > 	skip = true
>> > B:
>> > (original config)
>> > [hook "corp-pc"]
>> > 	event = post-commit
>> > 	command = /corp/stuff/corp-post-commit
>> > (local config)
>> > [hook "corp-pc"]
>> > 	skip = true
>> 
>> ...which are among other things that no, my proposed version doesn't
>> have "skip" at all, see point #3 at
>> https://lore.kernel.org/git/87mtv8fww3.fsf@evledraar.gmail.com/
>> 
>> I.e. I think the "skip" is a thing that falls out of a complexity of
>> your design that I'm proposing to do away with.
>> 
>> That complexity being that you use <event-or-name> and I use <name>, and
>> you want to in turn support N number of "*.command" for any
>> "hook.<event-or-name>".
>> 
>> I'm suggesting we use the typical "last key wins" semantics, if you want
>> multiple commands for a given hook type you'll just supply multiple
>> "hook.<name>" sections with the same "hook.*.event = type" in all.
>> 
>> The way to skip hooks in my proposal is:
>> 
>>     hook.<name>.command = true
>
> I see - instead of fiddly reorganizing the list, you're just changing
> what the specific hook name is redirecting to.
>
> Then, instead of storing the resolved command in the 'struct hook', we
> can store the name, and always dereference the name right before
> execution, using last-one-wins semantics.
>
> The only thing I don't like about this is that it's a little bit
> confusing to the user. I personally would rather have more complicated
> code and less complicated user experience, so I like 'skip' more because
> it's readable and because you don't need to guess "oh, it'll just
> execute /bin/true which is a noop". That's easy to understand for
> someone familiar with Unix scripting (and therefore for we Git
> contributors), but confusing for everybody else.
>
> Maybe we can take the essential spirit of this implementation but make
> it a little bit simpler. In pseudocode:
>
> populate_list(hookname):
>   # look for everybody who says "hook.*.event = $hookname"
>   hooks = get_configs_matching(hook.*.event, hookname)
>   # the "*" bit is the bit we dereference later
>   for hook in hooks:
>     hook_list.add({.name = hook.key.subsection})
>
> run_hooks(hook_list):
>   for hook in hook_list:
>     # should we even run?
>     if (get_last_config("hook." + hook.name + ".skip"))
>       continue
>
>     # dereference the hook name and find out the command
>     cmd = get_last_config("hook." + hook.name + ".command")
>     run(cmd)
>
> The price for this improved readability is an extra config lookup in
> each case. But the code is still readable, definitely moreso than
> rewalking the config list over and over as the implementation stands
> now. I personally think it's worth it.

I think the less complicated user experience is for our config system to
work as consistently as possible with other config. If we define special
rules and what's effectively special syntax with "skip" etc. we're
asking users to read how that works, and keep that all in their head
just to deal with that one area of our config.

> (One could also imagine a more readable placeholder than /bin/true in
> the "hook.<name>.command" field, but then we get to decide just how hard
> we want to avoid colliding with legitimate user hook names. Is .command
> = skip enough? is .command = RESERVED_NAME_SKIP_THIS_HOOK enough? Is
> something unrunnable like .command = "skip me" enough, although it could
> conceivably collide with a determined user?)

All of that's something you'll need to explain in detail to users, which
all seems way more complex than a simple:

    To skip a previously defined hook insert a noop-command, any will
    do, but setting it to "true" (usually /bin/true) is a handy
    convention for doing nothing.

I.e. by keeping the config field as doing one thing only you avoid any
such collisions etc.

>> 
>> Or whatever noop command you want on the RHS. In practice we wouldn't
>> invoke "true", just like we don't invoke "cat" for "PAGER=cat".
>> 
>> But unlike "*.skip" that doesn't require complexity in implementation or
>> user understanding, it just falls naturally out of the rest of the
>> model.
>> 
>> > #5 - Execute one script for multiple events
>> > A:
>> > [hookcmd "reusable-hook"]
>> > 	command = /long/path/reusable-hook
>> > [hook "post-commit"]
>> > 	command = reusable-hook
>> > [hook "pre-commit"]
>> > 	command = reusable-hook
>> > [hook "prepare-commit-msg"]
>> > 	command = reusable-hook
>> > B:
>> > [hook "reusable-hook"]
>> > 	command = /long/path/reusable-hook
>> > 	event = post-commit
>> > 	event = pre-commit
>> > 	event = prepare-commit-msg
>> 
>> It's been so long since I wrote the original E-Mail that I'm having to
>> skim my own summary there & come up with things as I go along, so maybe
>> this conflicts with something I said earlier.
>> 
>> But no, I don't think I meant that *.event should be multi-value, or
>> that that magic is worthwhile. I.e. I think we just want:
>> 
>>     [hook "not-a-reusable-section-1"]
>>         command = /long/path/reusable-hook
>>         event = post-commit
>>     [hook "not-a-reusable-section-2"]
>>         command = /long/path/reusable-hook
>>         event = pre-commit
>>     [hook "not-a-reusable-section-3"]
>>         command = /long/path/reusable-hook
>>         event = prepare-commit-msg
>> 
>> I.e. is wanting to use the same command for N different hooks really
>> that common a use-case to avoid a little verbosity, and quite a lot of
>> complexity?
>> 
>> How would such a hook even be implemented?
>> 
>> We don't have anything in your hook execution model (and I don't
>> remember you adding this) which passes down a "you are this type of
>> hook" to hooks.
>
> No, but it's something I'm interested in passing as an environment
> variable. I didn't add it to this series because it seemed to me to
> distract from the core feature. We would like to add it to simplify our
> invocations of https://github.com/awslabs/git-secrets, so it's on my
> radar.

Having such an env var as part of the initial series seems like a
sensible thing to have.

> And even without this, just today I wanted to configure a hook to really
> make sure I had config.mak set up in all my Git worktrees (since you
> pointed out I pushed code that fails -Wall.... ;) ) and set it up to run
> on both pre-commit and post-checkout. So yes, it is feasible now, for
> very stupid hooks.

Aside: It's nice to use DEVELOPER=1 in config.mak (or as make param)
when hacking on git, it'll set -Wall -Werror and other nice warning
flags.

> [...]
>> > [...]Please feel free to chime in with more use cases that you think would
>> > be handy which I forgot :)
>> 
>> I couldn't find this at a quick glance but I think we also had a
>> disussion at some point about hooks controlling parallelism. AFAICT your
>> current implementation just has global:
>> 
>>     hook.jobs=N
>> 
>> And we then blacklist certain hooks to always have hook.jobs=1, e.g. the
>> commit-msg hook that needs an implicit "lock" on that file (or rather,
>> we think that's the most common use-case).
>> 
>> I think my version of always having hook.<name>.{event,command} be one
>> value is also better in that case, i.e. we'd then:
>> 
>>     [hook "myhook"]
>>     command = some-command
>>     event = pre-receive
>>     parallel = true # the default
>> 
>>     [hook "myhook2"]
>>     command = some-command2
>>     event = pre-receive
>>     parallel = true # the default
>> 
>>     [hook "myhook3"]
>>     command = some-unconcurrent-command
>>     event = pre-receive
>>     parallel = false # I'm not OK with concurrency
>
> I am not sure what it means for a single executable to write "parallel =
> true" - it is a single executable.
>
> Ok, that is me being facetious - I think you are saying we can AND
> together all of the 'hook.<thing-with-event-we-care-about>.parallel' to
> decide whether or not to run in parallel.

Right, the case (whatever the config mechanism) wanting to use several
off-the-shelf hooks and accomplish through git some version of this:

    parallel -j8 pre-receive-parallel-*.sh &&
    parallel -j1 pre-receive-non-parallel-*.sh

I.e. since we have N scripts for the "pre-receive" type, and we're
expecting to say whether on not parallelism is OK or not, it seems like
a natural thing we'll want to declare that differently for some of those
than for others.

> I would rather not discuss this now, for this series, because regardless
> of which config schema we use today, we can figure out "parallel unless
> we really don't want it" later on. It is too complex to discuss in the
> context of "hey, we should also configure hooks somewhere else". Let's
> leave it for future work.

The point is that no, we really can't figure it out as easily later on
regardless of the config schema.

Because with 1=many you can't have 1=many.someAttribute=XYZ without that
*.someAttribute=XYZ declaring something for all of 1=many, whereas if
it's 1=1 then 1=1.someAttribute.XYZ obviously applies only to that 1=1.

>> I haven't thought about it deeply, but have a hunch that having sections
>> be a 1=1 mapping with a specific command instead of either 1=1 or 1=many
>> is going to make that easier to implement and understand.
>
> Sure, probably. But it can be added later to either schema... ;)
>
>
> Anyway, I did a quick strawpoll with my colleagues (Jonathan Nieder,
> Jonathan Tan, and Josh Steadmon) and they all like your syntax more. I
> am ambivalent between the two, personally.
>
> So what I will do, hopefully today, maybe not, is the following:
>
>  - no more hookcmd. (and there was much rejoicing)
>  - Hooks are specified with "[hook "name-of-hook"]"

Thanks, I look forward to checking that out. As should be obvious from
my misunderstanding of some of your config proposal (I snippet out some
of those from the reply) I'm still not entirely clear on even what the
current proposed behavior is, hopefully something simpler will be easier
to grok ... :)

>  - I do see value in having an explicit .skip field rather than mapping
>    .command to a noop, so "hook.name-of-hook.skip" as described above.
>    Of course the method you described will work regardless, since its
>    mechanism is based on the inherent result of executing /bin/true.

I think we've mainly focused on the theoretical aspect of this, but FWIW
I'm still entirely unclear on what this feature is even aimed for.

All of the rest of our config does not have an explicit "skip" for
anything, just last-set-wins. In terms of a real-world use-case wouldn't
a user just edit or comment out the config earlier in ~/.gitconfig, and
not "skip" it at the end with "git config [...] --add"?

I suspect that the use-case is some Googly centrally managed
/etc/gitconfig, but that's just speculation...

>  - I do see value in allowing "hook.name-of-hook.event" to be defined
>    repeatably, as described above, so I will include that.
> [...]
>  - figuring out what the heck we want to do with allowing hooks to
>    describe whether they allow parallelism

Which just to check if we're on the same page, needs to be figured out
because of the complexity of that "defined repeatably", no?

> Unfortunately I've dropped the design doc from this series, since nobody
> seemed interested in having it checked in, but I'll try to rework the
> help doc to make the schema more clear.

FWIW I wasn't opposed to it per-se, but remembered commenting on it in
earlier rounds, and it being some combination of docs that should really
be in manpages (i.e. describing current actual behavior), more general
rational and "why is it like this" (which correctly belongs in design
docs), and musings about hypothetical future features (e.g. "git hook
edit" and the like)'.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 8/9] hook: teach 'hookcmd' config to alias hook scripts
  2021-08-05  0:17                       ` Ævar Arnfjörð Bjarmason
@ 2021-08-05 21:45                         ` Emily Shaffer
  2021-08-05 22:26                           ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-08-05 21:45 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Thu, Aug 05, 2021 at 02:17:29AM +0200, Ævar Arnfjörð Bjarmason wrote:
> >> The gain is that "git hook list" becomes a trivial "git config
> >> -show-origin --show-scope --get-regexp" wrapper.
> >
> > This isn't a very compelling reason to me, if it makes the user
> > experience worse. (I'm not convinced that it does - just saying I
> > disagree with this reasoning.)
> 
> ...(see below)...
> 
> >> 
> >> So the series either doesn't need "git hook list" or such a thing
> >> becomes much less complex, especially given the proposed addition of
> >> other features in the area like "git hook edit", i.e. (quoting the
> >> linked E-Mail):
> >
> > I still think "git hook list" is useful to have for end-users, compared
> > to remembering the appropriate "git config" invocation. It futureproofs
> > us if we do want to change the execution ordering to something besides
> > config order, it's easier to remember/more discoverable, etc.
> 
> To clarify: I'm not against there being a "git hook list", I think we
> should have it.
> 
> It's useful UI. I.e. we have "run", it makes sense to have the same
> mechanism spew out its idea of what hooks are where, especially since
> that's not entirely contained in the config, but a union of config and
> GIT_DIR/hooks (and then there's core.hooksPath...).
> 
> So I'm very much for it being there.
> 
> I've just been commenting on the relative complexity of the config
> schema, which in one way surfaces as not being able to do the config
> part of the "git hook list" as a simple shell-out (or equivalent library
> invocation) of "git config" under the hood, but also means that users
> need to mentally track a more complex format when e.g. manually editing
> config...

Sure, makes sense.

> >> 
> >> And means that you presumably need to detect this case and error about
> >> it, but my proposed model does not:
> >> 
> >>     hook.post-commit.command = echo hi
> >>     # User error: "*.type" and <event-or-name>" does not match for
> >>     # "hook.*.command"
> >>     hook.post-commit.type    = pre-commit
> >> 
> >> And furthermore, that if someone in your model were to do:
> >> 
> >>     hook.verify-commit.command = echo hi
> >> 
> >> It's "dangling" today, but if a later version of git learns about a
> >> "verify-commit" hook we'll start running it unexpectedly.
> >
> > No, that's in fact as designed, with my model B. The user configured
> > "echo hi" to run on "verify-commit" events; if those events are
> > initially used by some wrapper, but later we decide they're a great idea
> > and absorb the verify-commit event into native Git, then this is working
> > as intended. I think your argument is based on a misunderstanding of the
> > design.
> >
> > Could it be that the way I provided the examples (my schema after A: and
> > your schema after B:) made it hard to parse? Sorry about that if so.
> 
> Aren't you assuming that users who specify a verify-commit will be happy
> because git's usurping of that will 1=1 match what they were using
> "verify-commit" for.
> 
> I'm pointing out that we can't know that, and since you want to make
> "git hook run" a general thing that runs any <name> of script you've
> configured, and not just what's in githooks(5) that it becomes very
> likely that if we add a new hook with some obvious name that we'll
> either break things for users, or subtly change behavior.
> 
> Which isn't just theoretical, e.g. I tend to run something like a "git
> log --check @{u}.." before I run git-send-email, with this configurable
> hook mechanism having a "git hook run sendemail-check" would be a way I
> might expose that in my own ~/.gitconfig.
> 
> But if git-send-email learns a "sendemail-check" and the behavior
> doesn't exactly match mine; E.g. maybe it similar to pre-auto-gc expects
> me to return a status code to ask me if I want to abort on a failed
> --check, but mine expects a revision range to run "log --check".
> 
> In practice that's a non-issue with the current hook mechanism,
> i.e. nobody's sticking a script into .git/hooks/my-custom-name and
> expecting it to do anything useful (and if they are, they have only
> themselves to blame).
> 
> Whereas we'd now be actively inviting users to squat on the same
> namespace we ourselves will need for future hooks.

Yeah, this is a good point. Seems worth a note in the 'git hook run'
doc, making a point that "you can use this for your wrapper to run
specific hooks, but be careful about namespace collisions". We're a lot
less likely to add a hook named "repotool-verify-commit" than we are to
add a hook named "verify-commit", for example.

I think it's enough to warn about future namespace collisions and make
an "at your own risk" note.

> > Maybe we can take the essential spirit of this implementation but make
> > it a little bit simpler. In pseudocode:
> >
> > populate_list(hookname):
> >   # look for everybody who says "hook.*.event = $hookname"
> >   hooks = get_configs_matching(hook.*.event, hookname)
> >   # the "*" bit is the bit we dereference later
> >   for hook in hooks:
> >     hook_list.add({.name = hook.key.subsection})
> >
> > run_hooks(hook_list):
> >   for hook in hook_list:
> >     # should we even run?
> >     if (get_last_config("hook." + hook.name + ".skip"))
> >       continue
> >
> >     # dereference the hook name and find out the command
> >     cmd = get_last_config("hook." + hook.name + ".command")
> >     run(cmd)
> >
> > The price for this improved readability is an extra config lookup in
> > each case. But the code is still readable, definitely moreso than
> > rewalking the config list over and over as the implementation stands
> > now. I personally think it's worth it.
> 
> I think the less complicated user experience is for our config system to
> work as consistently as possible with other config. If we define special
> rules and what's effectively special syntax with "skip" etc. we're
> asking users to read how that works, and keep that all in their head
> just to deal with that one area of our config.
> 
> > (One could also imagine a more readable placeholder than /bin/true in
> > the "hook.<name>.command" field, but then we get to decide just how hard
> > we want to avoid colliding with legitimate user hook names. Is .command
> > = skip enough? is .command = RESERVED_NAME_SKIP_THIS_HOOK enough? Is
> > something unrunnable like .command = "skip me" enough, although it could
> > conceivably collide with a determined user?)
> 
> All of that's something you'll need to explain in detail to users, which
> all seems way more complex than a simple:
> 
>     To skip a previously defined hook insert a noop-command, any will
>     do, but setting it to "true" (usually /bin/true) is a handy
>     convention for doing nothing.
> 
> I.e. by keeping the config field as doing one thing only you avoid any
> such collisions etc.

I disagree fundamentally that "find and run a noop command like
/bin/true" is simpler to average users than "skip it by setting a
config". Like I said below, by including "skip" both approaches will
work.

> > No, but it's something I'm interested in passing as an environment
> > variable. I didn't add it to this series because it seemed to me to
> > distract from the core feature. We would like to add it to simplify our
> > invocations of https://github.com/awslabs/git-secrets, so it's on my
> > radar.
> 
> Having such an env var as part of the initial series seems like a
> sensible thing to have.

Eh. To me, it feels like feature creep. It also is something we could
add today to the existing hook mechanism (even if it's a little
pointless since you can basename, like you say), so it feels orthogonal.
I would prefer not to add it in this series.

> 
> > And even without this, just today I wanted to configure a hook to really
> > make sure I had config.mak set up in all my Git worktrees (since you
> > pointed out I pushed code that fails -Wall.... ;) ) and set it up to run
> > on both pre-commit and post-checkout. So yes, it is feasible now, for
> > very stupid hooks.
> 
> Aside: It's nice to use DEVELOPER=1 in config.mak (or as make param)
> when hacking on git, it'll set -Wall -Werror and other nice warning
> flags.

This is literally what I was saying I wrote the hook to do, and is also
a tip I included when I wrote MyFirstContribution.txt.

> > I am not sure what it means for a single executable to write "parallel =
> > true" - it is a single executable.
> >
> > Ok, that is me being facetious - I think you are saying we can AND
> > together all of the 'hook.<thing-with-event-we-care-about>.parallel' to
> > decide whether or not to run in parallel.
> 
> Right, the case (whatever the config mechanism) wanting to use several
> off-the-shelf hooks and accomplish through git some version of this:
> 
>     parallel -j8 pre-receive-parallel-*.sh &&
>     parallel -j1 pre-receive-non-parallel-*.sh
> 
> I.e. since we have N scripts for the "pre-receive" type, and we're
> expecting to say whether on not parallelism is OK or not, it seems like
> a natural thing we'll want to declare that differently for some of those
> than for others.
> 
> > I would rather not discuss this now, for this series, because regardless
> > of which config schema we use today, we can figure out "parallel unless
> > we really don't want it" later on. It is too complex to discuss in the
> > context of "hey, we should also configure hooks somewhere else". Let's
> > leave it for future work.
> 
> The point is that no, we really can't figure it out as easily later on
> regardless of the config schema.
> 
> Because with 1=many you can't have 1=many.someAttribute=XYZ without that
> *.someAttribute=XYZ declaring something for all of 1=many, whereas if
> it's 1=1 then 1=1.someAttribute.XYZ obviously applies only to that 1=1.

I think this is moot, since we are moving to "all config hooks have a
name", but my plan previously was to let this be set on a hookcmd.
Essentially, your suggestion is to make every hook a hookcmd. My point
was that it's easy to extend [object which represents an executable] in
the config to include "always run me in series" or "run me in series for
this specific event" regardless. That is, one could imagine, discarding
entirely the hookcmd junk and going with the schema I sketched in my
last email (which lands somewhere between yours and mine):

[hook "linter"]
  command = ~/linter.sh
  event = pre-commit
  parallel = false

or...

[hook "linter"]
  command = ~/linter.sh
  event = pre-commit
  event = commit-msg

[hook "linter.commit-msg"]
  parallel = false

Or even...

[hook "linter"]
  command = ~/linter.sh
  event = pre-commit
  event = commit-msg
  parallel = commit-msg

The possibilities go on, as far as configuration goes.

To me, the harder part of this problem is actually implementing the
execution. We had some discussions at length early on in the
config-based hook series about ways to do this kind of complex "some
stuff needs synchronous execution and some stuff doesn't, in the same
event" and decided that it mostly resolved to "you ain't gonna need it"
principle. So I would prefer to discuss this when we find out we do
actually need it.

> 
> >> I haven't thought about it deeply, but have a hunch that having sections
> >> be a 1=1 mapping with a specific command instead of either 1=1 or 1=many
> >> is going to make that easier to implement and understand.
> >
> > Sure, probably. But it can be added later to either schema... ;)
> >
> >
> > Anyway, I did a quick strawpoll with my colleagues (Jonathan Nieder,
> > Jonathan Tan, and Josh Steadmon) and they all like your syntax more. I
> > am ambivalent between the two, personally.
> >
> > So what I will do, hopefully today, maybe not, is the following:
> >
> >  - no more hookcmd. (and there was much rejoicing)
> >  - Hooks are specified with "[hook "name-of-hook"]"
> 
> Thanks, I look forward to checking that out. As should be obvious from
> my misunderstanding of some of your config proposal (I snippet out some
> of those from the reply) I'm still not entirely clear on even what the
> current proposed behavior is, hopefully something simpler will be easier
> to grok ... :)

Fingers crossed ;)

> 
> >  - I do see value in having an explicit .skip field rather than mapping
> >    .command to a noop, so "hook.name-of-hook.skip" as described above.
> >    Of course the method you described will work regardless, since its
> >    mechanism is based on the inherent result of executing /bin/true.
> 
> I think we've mainly focused on the theoretical aspect of this, but FWIW
> I'm still entirely unclear on what this feature is even aimed for.
> 
> All of the rest of our config does not have an explicit "skip" for
> anything, just last-set-wins. In terms of a real-world use-case wouldn't
> a user just edit or comment out the config earlier in ~/.gitconfig, and
> not "skip" it at the end with "git config [...] --add"?
> 
> I suspect that the use-case is some Googly centrally managed
> /etc/gitconfig, but that's just speculation...

Yep, this is exactly why. We've talked often on-list about how we ship
and configure Git for Googlers, but the upshot is "we pack up 'next' and
also ship an '/etc/gitconfig'".

But I can also think of one really basic scenario when I'd want to skip
a hook in one repo without just commenting out my ~/.gitconfig: the
Gerrit Change-Id hook.

Gerrit requires all commit messages to contain this Change-Id: abc123
footer. It adds the footer by way of a commit-msg hook. That hook works
the same regardless of what your Gerrit remote is, so you can run the
same script on any project that uses Gerrit for code review. If, as I have in
the past, the vast majority of my projects use Gerrit, but I have one
project which does not, then I would love to configure the Gerrit
Change-Id hook globally and un-configure it for my one non-Gerrit
project.

(At that time, I maintained a subsystem in a project based on Yocto, so
I needed to regularly contribute to 5-10 projects, all but one of which
used Gerrit. The one non-Gerrit one used a mailing list. I also had a
hobby project and my dotfiles, neither of which used Gerrit. This is not
an uncommon use case.)

> 
> >  - I do see value in allowing "hook.name-of-hook.event" to be defined
> >    repeatably, as described above, so I will include that.
> > [...]
> >  - figuring out what the heck we want to do with allowing hooks to
> >    describe whether they allow parallelism
> 
> Which just to check if we're on the same page, needs to be figured out
> because of the complexity of that "defined repeatably", no?

As I wrote above, I think this falls under YAGNI, to be honest, and I
don't think we've painted ourselves into a corner if we do end up
needing it. I think the semantical complexity is a harder problem than
the syntactical complexity, and I'd rather just punt it ;)

> 
> > Unfortunately I've dropped the design doc from this series, since nobody
> > seemed interested in having it checked in, but I'll try to rework the
> > help doc to make the schema more clear.
> 
> FWIW I wasn't opposed to it per-se, but remembered commenting on it in
> earlier rounds, and it being some combination of docs that should really
> be in manpages (i.e. describing current actual behavior), more general
> rational and "why is it like this" (which correctly belongs in design
> docs), and musings about hypothetical future features (e.g. "git hook
> edit" and the like)'.

Yeah, that's pretty typical of design docs at places I've worked now and
in the past. I don't have a big problem with not checking it in - those
kinds of docs are most valuable during the early design stage, and so I
think that doc has now served its purpose.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 8/9] hook: teach 'hookcmd' config to alias hook scripts
  2021-08-05 21:45                         ` Emily Shaffer
@ 2021-08-05 22:26                           ` Ævar Arnfjörð Bjarmason
  2021-08-06 20:18                             ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-05 22:26 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Aug 05 2021, Emily Shaffer wrote:

> On Thu, Aug 05, 2021 at 02:17:29AM +0200, Ævar Arnfjörð Bjarmason wrote:
>> [...]
>> > No, that's in fact as designed, with my model B. The user configured
>> > "echo hi" to run on "verify-commit" events; if those events are
>> > initially used by some wrapper, but later we decide they're a great idea
>> > and absorb the verify-commit event into native Git, then this is working
>> > as intended. I think your argument is based on a misunderstanding of the
>> > design.
>> >
>> > Could it be that the way I provided the examples (my schema after A: and
>> > your schema after B:) made it hard to parse? Sorry about that if so.
>> 
>> Aren't you assuming that users who specify a verify-commit will be happy
>> because git's usurping of that will 1=1 match what they were using
>> "verify-commit" for.
>> 
>> I'm pointing out that we can't know that, and since you want to make
>> "git hook run" a general thing that runs any <name> of script you've
>> configured, and not just what's in githooks(5) that it becomes very
>> likely that if we add a new hook with some obvious name that we'll
>> either break things for users, or subtly change behavior.
>> 
>> Which isn't just theoretical, e.g. I tend to run something like a "git
>> log --check @{u}.." before I run git-send-email, with this configurable
>> hook mechanism having a "git hook run sendemail-check" would be a way I
>> might expose that in my own ~/.gitconfig.
>> 
>> But if git-send-email learns a "sendemail-check" and the behavior
>> doesn't exactly match mine; E.g. maybe it similar to pre-auto-gc expects
>> me to return a status code to ask me if I want to abort on a failed
>> --check, but mine expects a revision range to run "log --check".
>> 
>> In practice that's a non-issue with the current hook mechanism,
>> i.e. nobody's sticking a script into .git/hooks/my-custom-name and
>> expecting it to do anything useful (and if they are, they have only
>> themselves to blame).
>> 
>> Whereas we'd now be actively inviting users to squat on the same
>> namespace we ourselves will need for future hooks.
>
> Yeah, this is a good point. Seems worth a note in the 'git hook run'
> doc, making a point that "you can use this for your wrapper to run
> specific hooks, but be careful about namespace collisions". We're a lot
> less likely to add a hook named "repotool-verify-commit" than we are to
> add a hook named "verify-commit", for example.
>
> I think it's enough to warn about future namespace collisions and make
> an "at your own risk" note.

I might have lost track at this point, but later examples in this E-Mail
you show don't seem to require such a note.

I.e. it's only an issue if we conflate a semantically meaningful slot
like "pre-commit" in the config with one that can also have the meaning
of simply defining an arbitrary user-decided name.

There's no such collision if the config uses
e.g. hook.mycheck.event=pre-commit & hook.mycheck.command=mycmd, as
opposed to hook.pre-commit.command=mycmd.

On the specifics of that example: I don't really care about the
bikeshedding of the config key naming specifics, just the semantics of
not putting user defined names and hook type names in the same slot if
we can avoid it.

>> > No, but it's something I'm interested in passing as an environment
>> > variable. I didn't add it to this series because it seemed to me to
>> > distract from the core feature. We would like to add it to simplify our
>> > invocations of https://github.com/awslabs/git-secrets, so it's on my
>> > radar.
>> 
>> Having such an env var as part of the initial series seems like a
>> sensible thing to have.
>
> Eh. To me, it feels like feature creep. It also is something we could
> add today to the existing hook mechanism (even if it's a little
> pointless since you can basename, like you say), so it feels orthogonal.
> I would prefer not to add it in this series.

Sure, I guess you can add two hook sections to replace e.g. your
{pre,post}-receive hook (which are commonly routed to the same script
with file-based hooks). Having a single setenv() seems easy enough, and
I'd bet a way more common use-case than wanting to skip earlier defined
hooks...

>> > I am not sure what it means for a single executable to write "parallel =
>> > true" - it is a single executable.
>> >
>> > Ok, that is me being facetious - I think you are saying we can AND
>> > together all of the 'hook.<thing-with-event-we-care-about>.parallel' to
>> > decide whether or not to run in parallel.
>> 
>> Right, the case (whatever the config mechanism) wanting to use several
>> off-the-shelf hooks and accomplish through git some version of this:
>> 
>>     parallel -j8 pre-receive-parallel-*.sh &&
>>     parallel -j1 pre-receive-non-parallel-*.sh
>> 
>> I.e. since we have N scripts for the "pre-receive" type, and we're
>> expecting to say whether on not parallelism is OK or not, it seems like
>> a natural thing we'll want to declare that differently for some of those
>> than for others.
>> 
>> > I would rather not discuss this now, for this series, because regardless
>> > of which config schema we use today, we can figure out "parallel unless
>> > we really don't want it" later on. It is too complex to discuss in the
>> > context of "hey, we should also configure hooks somewhere else". Let's
>> > leave it for future work.
>> 
>> The point is that no, we really can't figure it out as easily later on
>> regardless of the config schema.
>> 
>> Because with 1=many you can't have 1=many.someAttribute=XYZ without that
>> *.someAttribute=XYZ declaring something for all of 1=many, whereas if
>> it's 1=1 then 1=1.someAttribute.XYZ obviously applies only to that 1=1.
>
> I think this is moot, since we are moving to "all config hooks have a
> name", but my plan previously was to let this be set on a hookcmd.
> Essentially, your suggestion is to make every hook a hookcmd. My point
> was that it's easy to extend [object which represents an executable] in
> the config to include "always run me in series" or "run me in series for
> this specific event" regardless. That is, one could imagine, discarding
> entirely the hookcmd junk and going with the schema I sketched in my
> last email (which lands somewhere between yours and mine):

Just to be clear, I don't have any concrete suggestion in mind right now
(actually as I write this I can only vaguely recall what I suggested
before).

What I have been suggesting is not any specific implementation, but that
we have a bias for the simple over the complex for an initial
implementation.

Complexity can always be added later, whereas coming up with a config
schema that's irregular compared to other existing config in git is
something we might regret sooner than later.

> [hook "linter"]
>   command = ~/linter.sh
>   event = pre-commit
>   parallel = false
>
> or...
>
> [hook "linter"]
>   command = ~/linter.sh
>   event = pre-commit
>   event = commit-msg
>
> [hook "linter.commit-msg"]
>   parallel = false
>
> Or even...
>
> [hook "linter"]
>   command = ~/linter.sh
>   event = pre-commit
>   event = commit-msg
>   parallel = commit-msg
>
> The possibilities go on, as far as configuration goes.
>
> To me, the harder part of this problem is actually implementing the
> execution. We had some discussions at length early on in the
> config-based hook series about ways to do this kind of complex "some
> stuff needs synchronous execution and some stuff doesn't, in the same
> event" and decided that it mostly resolved to "you ain't gonna need it"
> principle. So I would prefer to discuss this when we find out we do
> actually need it.

What I was mainly going for with "we really can't figure it out as
easily later" above was not that this tweaking of "jobs" or parallelism
was essential per-hook.

But that it was a handy shorthand for a config attribute you might want
to define for hooks, and having what are effectively groups of hooks,
with N "command" or "event" in one section might make things more
complex once you'd want to define optional attributes for one of those
commands or events.

>> [...]
>> >  - I do see value in having an explicit .skip field rather than mapping
>> >    .command to a noop, so "hook.name-of-hook.skip" as described above.
>> >    Of course the method you described will work regardless, since its
>> >    mechanism is based on the inherent result of executing /bin/true.
>> 
>> I think we've mainly focused on the theoretical aspect of this, but FWIW
>> I'm still entirely unclear on what this feature is even aimed for.
>> 
>> All of the rest of our config does not have an explicit "skip" for
>> anything, just last-set-wins. In terms of a real-world use-case wouldn't
>> a user just edit or comment out the config earlier in ~/.gitconfig, and
>> not "skip" it at the end with "git config [...] --add"?
>> 
>> I suspect that the use-case is some Googly centrally managed
>> /etc/gitconfig, but that's just speculation...
>
> Yep, this is exactly why. We've talked often on-list about how we ship
> and configure Git for Googlers, but the upshot is "we pack up 'next' and
> also ship an '/etc/gitconfig'".
>
> But I can also think of one really basic scenario when I'd want to skip
> a hook in one repo without just commenting out my ~/.gitconfig: the
> Gerrit Change-Id hook.
>
> Gerrit requires all commit messages to contain this Change-Id: abc123
> footer. It adds the footer by way of a commit-msg hook. That hook works
> the same regardless of what your Gerrit remote is, so you can run the
> same script on any project that uses Gerrit for code review. If, as I have in
> the past, the vast majority of my projects use Gerrit, but I have one
> project which does not, then I would love to configure the Gerrit
> Change-Id hook globally and un-configure it for my one non-Gerrit
> project.
>
> (At that time, I maintained a subsystem in a project based on Yocto, so
> I needed to regularly contribute to 5-10 projects, all but one of which
> used Gerrit. The one non-Gerrit one used a mailing list. I also had a
> hobby project and my dotfiles, neither of which used Gerrit. This is not
> an uncommon use case.)

> I disagree fundamentally that "find and run a noop command like
> /bin/true" is simpler to average users than "skip it by setting a
> config". Like I said below, by including "skip" both approaches will
> work.

In reply to this, and moving things around a bit in the reply:

>> All of that's something you'll need to explain in detail to users, which
>> all seems way more complex than a simple:
>> 
>>     To skip a previously defined hook insert a noop-command, any will
>>     do, but setting it to "true" (usually /bin/true) is a handy
>>     convention for doing nothing.
>> 
>> I.e. by keeping the config field as doing one thing only you avoid any
>> such collisions etc.
>
> I disagree fundamentally that "find and run a noop command like
> /bin/true" is simpler to average users than "skip it by setting a
> config". Like I said below, by including "skip" both approaches will
> work.

To clarify, I haven't been advocating for that "skip = true" convention
because I think it's a sensible thing per-se, but that I think this
use-case is something that an individual configurable feature in git
doesn't need stateful syntax to deal with.

We have any number of multi-value and single-value config within git. I
just don't see why on balance hooks need a special syntax to skip
earlier set config for hooks specifically.

E.g. this gerrit example would also be true of someone in a corporate
setting using git-send-email, and wanting a list of sendemail.cc on all
but their dotfiles project, or one other non-work project.

Does that mean we need a sendemail.skipCC and special handling for it in
git-send-email.perl? No, I think we'd generally advice users to just put
those projects under ~/work or whatever, and then use config includes to
set config for that group of projects based on the path:

    [includeIf "gitdir:~/work/"]
        path = ~/.gitconfig.d/work

Or, if a hook is really so special that it's needed everywhere define it
in /etc/gitconfig, and then just make the hook itself do:

    if test "$(git config --bool googleHooks.disableOurGlobalHook)" = "true"
    then
        exit 0
    fi

Which is pretty much (with the hook.* config prefix) how we've adviced
users to do this since approximately forever with the sample hooks we
ship.

The advantage of using includes in that way is e.g. that you can easily
see how your hook came to be configured with:

    git config --list --show-origin

I.e. that (by convention) it comes via a conditionally included
~/.gitconfig.d/gerrit file. If it's a multi-value like sendemail.cc the
semantics are also clear, e.g. you can get all values we'll use with
"git config --get-all".

Whereas choosing to implement this with something that *looks* like a
config keyword, but really isn't is just confusing.

We need to explain in one way how users might arrange for the likes of
sendemail.cc to be defined for some, but not all of their repos, and
explain it differently when it comes to hooks. There's inherent value in
that explanation being the same for both.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH 8/9] hook: teach 'hookcmd' config to alias hook scripts
  2021-08-05 22:26                           ` Ævar Arnfjörð Bjarmason
@ 2021-08-06 20:18                             ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-06 20:18 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Fri, Aug 06, 2021 at 12:26:02AM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Thu, Aug 05 2021, Emily Shaffer wrote:
> 
> > On Thu, Aug 05, 2021 at 02:17:29AM +0200, Ævar Arnfjörð Bjarmason wrote:
> >> [...]
> >> > No, that's in fact as designed, with my model B. The user configured
> >> > "echo hi" to run on "verify-commit" events; if those events are
> >> > initially used by some wrapper, but later we decide they're a great idea
> >> > and absorb the verify-commit event into native Git, then this is working
> >> > as intended. I think your argument is based on a misunderstanding of the
> >> > design.
> >> >
> >> > Could it be that the way I provided the examples (my schema after A: and
> >> > your schema after B:) made it hard to parse? Sorry about that if so.
> >> 
> >> Aren't you assuming that users who specify a verify-commit will be happy
> >> because git's usurping of that will 1=1 match what they were using
> >> "verify-commit" for.
> >> 
> >> I'm pointing out that we can't know that, and since you want to make
> >> "git hook run" a general thing that runs any <name> of script you've
> >> configured, and not just what's in githooks(5) that it becomes very
> >> likely that if we add a new hook with some obvious name that we'll
> >> either break things for users, or subtly change behavior.
> >> 
> >> Which isn't just theoretical, e.g. I tend to run something like a "git
> >> log --check @{u}.." before I run git-send-email, with this configurable
> >> hook mechanism having a "git hook run sendemail-check" would be a way I
> >> might expose that in my own ~/.gitconfig.
> >> 
> >> But if git-send-email learns a "sendemail-check" and the behavior
> >> doesn't exactly match mine; E.g. maybe it similar to pre-auto-gc expects
> >> me to return a status code to ask me if I want to abort on a failed
> >> --check, but mine expects a revision range to run "log --check".
> >> 
> >> In practice that's a non-issue with the current hook mechanism,
> >> i.e. nobody's sticking a script into .git/hooks/my-custom-name and
> >> expecting it to do anything useful (and if they are, they have only
> >> themselves to blame).
> >> 
> >> Whereas we'd now be actively inviting users to squat on the same
> >> namespace we ourselves will need for future hooks.
> >
> > Yeah, this is a good point. Seems worth a note in the 'git hook run'
> > doc, making a point that "you can use this for your wrapper to run
> > specific hooks, but be careful about namespace collisions". We're a lot
> > less likely to add a hook named "repotool-verify-commit" than we are to
> > add a hook named "verify-commit", for example.
> >
> > I think it's enough to warn about future namespace collisions and make
> > an "at your own risk" note.
> 
> I might have lost track at this point, but later examples in this E-Mail
> you show don't seem to require such a note.
> 
> I.e. it's only an issue if we conflate a semantically meaningful slot
> like "pre-commit" in the config with one that can also have the meaning
> of simply defining an arbitrary user-decided name.

No, I don't think that's the case. The examples I used later in the mail
don't have anything to do with this "wrapper wants to use 'git hooks
run'" case.

Let's imagine a current commonly-used wrapper, 'repo'
(https://gerrit.googlesource.com/git-repo/+/refs/heads/master/README.md).
This wrapper is used to manage multiple Git repos into something
semantically similar to submodules.

Let's say that 'repo' decides that instead of rolling its own hook
system, it would like to use Git's native hook system. Let's say that it
wants to call user-defined hooks any time someone has just completed
running 'repo sync'. It could ask users to define a config like so:

  [hook "myrepohook"]
    event = post-sync
    command = ~/some/downloaded/myrepohook --with --args

Then 'repo' itself can, at the end of its 'repo sync' implementation,
call 'git hook run post-sync' (instead of using some other hook
specification schema).

But Git itself has a 'git submodule sync' command, and let's imagine
that we decided we want to run some user-defined hook at the end of 'git
submodule sync'. Naturally, we can call it 'post-sync', and then we will
accidentally invoke the hooks which someone configured for 'repo' - as
you pointed out would be a concern with my design, this will *also* be
a concern with your design.

This specific use case is what I think we can get around with some "at
your own risk" documentation, because we are much less likely to collide
with someone asking their users to configure like so:

  [hook "myrepohook"]
    event = repo-tool-post-sync
    command = ~/some/downloaded/myrepohook --with-args

> >> > No, but it's something I'm interested in passing as an environment
> >> > variable. I didn't add it to this series because it seemed to me to
> >> > distract from the core feature. We would like to add it to simplify our
> >> > invocations of https://github.com/awslabs/git-secrets, so it's on my
> >> > radar.
> >> 
> >> Having such an env var as part of the initial series seems like a
> >> sensible thing to have.
> >
> > Eh. To me, it feels like feature creep. It also is something we could
> > add today to the existing hook mechanism (even if it's a little
> > pointless since you can basename, like you say), so it feels orthogonal.
> > I would prefer not to add it in this series.
> 
> Sure, I guess you can add two hook sections to replace e.g. your
> {pre,post}-receive hook (which are commonly routed to the same script
> with file-based hooks). Having a single setenv() seems easy enough, and
> I'd bet a way more common use-case than wanting to skip earlier defined
> hooks...

More than anything else, I think this comment convinced me that the skip
config also falls under YAGNI and I can drop it (and we can add it later
if "hook.myhook.command = true" isn't clear enough).

I snipped the rest because I don't disagree or have anything further to
say. Look for a reroll this weekend, we hope.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH v2 0/6] config-based hooks restarted
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (35 preceding siblings ...)
  2021-08-03 19:39             ` [PATCH v4 36/36] hooks: fix a TOCTOU in "did we run a hook?" heuristic Ævar Arnfjörð Bjarmason
@ 2021-08-12  0:42             ` Emily Shaffer
  2021-08-12  0:42               ` [PATCH v2 1/6] hook: run a list of hooks instead Emily Shaffer
                                 ` (7 more replies)
  2021-08-19  0:17             ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Emily Shaffer
                               ` (2 subsequent siblings)
  39 siblings, 8 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-12  0:42 UTC (permalink / raw)
  To: git
  Cc: Emily Shaffer, Junio C Hamano, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m. carlson, Josh Steadmon,
	Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

This is the config-based hooks topic rebased onto v4 of Ævar's
branch[1]. There is a happy CI build of it on GitHub[2].

The topic overall adds the ability to set up hooks by modifying the
config, in addition to placing specially named hooks into the hookdir.
This enables users to specify multiple hooks for a given event, and so
this topic also fleshes out the use of the run_processes_parallel() API
which is now introduced in Ævar's reordering of prior patches.

Patches 1-4 make some minor changes to prepare Ævar's series to handle
more than one hook at a time. With the exception of patch 4, there
should be no behavior change for existing hooks.

Patch 2 is opinionated about which hooks should and shouldn't be allowed
to run in parallel; if you care about a specific hook, please take a
look there.

Patch 5 is the motivating feature - it begins to parse the config
looking for hooks.

Patch 6 takes advantage of the decoupling of hooks and GITDIR to allow
out-of-repo hook runs, which would only run hooks specified in the
system or global config. This mainly targets 'sendemail-validate', but
'git-sendemail.perl' still explicitly disallows out-of-repo hook
execution on that hook for now. (Maybe that change should be added in
this series? Or maybe patch 6 belongs with that kind of change?)

Since v1:

The largest change is that the config schema is different, following the
discussion from [3] and on. 'hookcmd' has gone away entirely, and all
config-specified hooks need a user-provided name associated with them.

I have also dropped the 'skip' config from this series (also discussed
in [3]) as users can reset their 'hook.myhook.command' to 'true' or some
other noop in a later config to effectively disable any hook. The 'skip'
feature may be added later or may not, depending on if users ask us for
it.

The 'from_hookdir' field has been removed entirely. When storing hooks
by name instead of by command, it makes sense enough to store the one
from the hookdir in a special way - NULL name - because it does not have
a name to specify. This way we can use that field as an indicator that
the hook came from the hookdir, and we also do not need to reserve a
name to use for hookdir hooks (as opposed to setting hookdir hooks to
have name = "FROM_HOOKDIR" or something).

The tests have been moved into t1800-hook.sh and simplified.

The 'struct run_hooks_opt' initializer has been moved to macros (as
opposed to functions), and .jobs=0 indicates that we should look up the
configured job count or nproc at hook run time. Ævar asked me to change
these macros to RUN_HOOKS_OPT_INIT and RUN_HOOKS_OPT_INIT_SYNC or
something to simplify the merge, but I did not do that - I removed all
instances of RUN_HOOKS_OPT_INIT and now everybody uses
RUN_HOOKS_OPT_INIT_SYNC or RUN_HOOKS_OPT_INIT_ASYNC. I think it's
important for contributors to be able to tell at a glance whether the
hook event expects resource contention that would prevent parallelism. I
also think it makes the review easier (if longer) for folks who are
reviewing trying to decide whether the parallelism is appropriate for
each hook event. I do think that run_hooks_oneshot() also is a step
backwards in this regard, but I didn't get a chance to say so on the
commit that introduces it yet.

Thanks in advance, all.

 - Emily

1: https://lore.kernel.org/git/cover-v4-00.36-00000000000-20210803T191505Z-avarab%40gmail.com
2: https://github.com/nasamuffin/git/actions/runs/1122126800 (which
points to an earlier successul run before I messed with the commit
messages)
3: https://lore.kernel.org/git/87fswey5wd.fsf%40evledraar.gmail.com

Emily Shaffer (6):
  hook: run a list of hooks instead
  hook: allow parallel hook execution
  hook: introduce "git hook list"
  hook: allow running non-native hooks
  hook: include hooks from the config
  hook: allow out-of-repo 'git hook' invocations

 Documentation/config/hook.txt |   9 ++
 Documentation/git-hook.txt    |  48 +++++-
 builtin/am.c                  |   4 +-
 builtin/checkout.c            |   2 +-
 builtin/clone.c               |   2 +-
 builtin/hook.c                |  68 +++++++-
 builtin/merge.c               |   2 +-
 builtin/rebase.c              |   2 +-
 builtin/receive-pack.c        |   9 +-
 builtin/worktree.c            |   2 +-
 commit.c                      |   2 +-
 git.c                         |   2 +-
 hook.c                        | 289 +++++++++++++++++++++++++++++-----
 hook.h                        |  61 +++++--
 read-cache.c                  |   2 +-
 refs.c                        |   2 +-
 reset.c                       |   3 +-
 sequencer.c                   |   4 +-
 t/t1800-hook.sh               | 161 ++++++++++++++++++-
 transport.c                   |   2 +-
 20 files changed, 596 insertions(+), 80 deletions(-)
 create mode 100644 Documentation/config/hook.txt

Range-diff against v1:
1:  c4b95fb08a < -:  ---------- hook: treat hookdir hook specially
-:  ---------- > 1:  5177e8ba2c hook: run a list of hooks instead
-:  ---------- > 2:  eda439cd57 hook: allow parallel hook execution
-:  ---------- > 3:  cdfe3b6e16 hook: introduce "git hook list"
2:  e6a56ac674 = 4:  eb4e03e22b hook: allow running non-native hooks
3:  32ad49ea9b ! 5:  2c8e874158 hook: include hooks from the config
    @@ Commit message
         hooks to run for a given event.
     
         Multiple commands can be specified for a given hook by providing
    -    multiple "hook.<hookname>.command = <path-to-hook>" lines. Hooks will be
    -    run in config order.
    +    multiple "hook.<friendly-name>.command = <path-to-hook>" and
    +    "hook.<friendly-name>.event = <hook-event>" lines. Hooks will be run in
    +    config order of the "hook.<name>.event" lines.
     
         For example:
     
           $ git config --list | grep ^hook
    -      hook.pre-commit.command=~/bar.sh
    +      hook.bar.command=~/bar.sh
    +      hook.bar.event=pre-commit
     
           $ git hook run
           # Runs ~/bar.sh
    @@ hook.c: const char *find_hook_gently(const char *name)
     -		struct hook *to_add = xmalloc(sizeof(*to_add));
     -		to_add->hook_path = hook_path;
     -		to_add->feed_pipe_cb_data = NULL;
    --		to_add->from_hookdir = 1;
     -		list_add_tail(&to_add->list, hook_head);
     -	}
     +	/* Add the hook from the hookdir. The placeholder makes it easier to
    @@ hook.c: static int pick_next_hook(struct child_process *cp,
     +	cp->use_shell = !!run_me->name;
     +
      	/* add command */
    --	if (run_me->from_hookdir && hook_cb->options->absolute_path)
    +-	if (hook_cb->options->absolute_path)
     -		strvec_push(&cp->args, absolute_path(run_me->hook_path));
     -	else
     -		strvec_push(&cp->args, run_me->hook_path);
    @@ hook.h: int hook_exists(const char *hookname);
      	struct list_head list;
     -	/* The path to the hook */
     -	const char *hook_path;
    --
    --	unsigned from_hookdir : 1;
     +	/*
     +	 * The friendly name of the hook. NULL indicates the hook is from the
     +	 * hookdir.
4:  91e54185b3 = 6:  3216e51b6b hook: allow out-of-repo 'git hook' invocations
-- 
2.32.0.605.g8dce9f2422-goog


^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH v2 1/6] hook: run a list of hooks instead
  2021-08-12  0:42             ` [PATCH v2 0/6] config-based hooks restarted Emily Shaffer
@ 2021-08-12  0:42               ` Emily Shaffer
  2021-08-12 17:25                 ` Junio C Hamano
  2021-08-12  0:42               ` [PATCH v2 2/6] hook: allow parallel hook execution Emily Shaffer
                                 ` (6 subsequent siblings)
  7 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-08-12  0:42 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

To prepare for multihook support, teach hook.[hc] to take a list of
hooks at run_hooks and run_found_hooks. Right now the list is always one
entry, but in the future we will allow users to supply more than one
executable for a single hook event.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 builtin/hook.c |  14 ++++---
 hook.c         | 103 +++++++++++++++++++++++++++++++++++--------------
 hook.h         |  16 +++++++-
 3 files changed, 96 insertions(+), 37 deletions(-)

diff --git a/builtin/hook.c b/builtin/hook.c
index 5eb7cf73a4..4d39c9e75e 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -25,7 +25,8 @@ static int run(int argc, const char **argv, const char *prefix)
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	int ignore_missing = 0;
 	const char *hook_name;
-	const char *hook_path;
+	struct list_head *hooks;
+
 	struct option run_options[] = {
 		OPT_BOOL(0, "ignore-missing", &ignore_missing,
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
@@ -58,15 +59,16 @@ static int run(int argc, const char **argv, const char *prefix)
 	git_config(git_default_config, NULL);
 
 	hook_name = argv[0];
-	if (ignore_missing)
-		return run_hooks_oneshot(hook_name, &opt);
-	hook_path = find_hook(hook_name);
-	if (!hook_path) {
+	hooks = hook_list(hook_name);
+	if (list_empty(hooks)) {
+		/* ... act like run_hooks_oneshot() under --ignore-missing */
+		if (ignore_missing)
+			return 0;
 		error("cannot find a hook named %s", hook_name);
 		return 1;
 	}
 
-	ret = run_hooks(hook_name, hook_path, &opt);
+	ret = run_hooks(hook_name, hooks, &opt);
 	run_hooks_opt_clear(&opt);
 	return ret;
 usage:
diff --git a/hook.c b/hook.c
index ee20b2e365..80e150548c 100644
--- a/hook.c
+++ b/hook.c
@@ -4,6 +4,28 @@
 #include "hook-list.h"
 #include "config.h"
 
+static void free_hook(struct hook *ptr)
+{
+	if (ptr) {
+		free(ptr->feed_pipe_cb_data);
+	}
+	free(ptr);
+}
+
+static void remove_hook(struct list_head *to_remove)
+{
+	struct hook *hook_to_remove = list_entry(to_remove, struct hook, list);
+	list_del(to_remove);
+	free_hook(hook_to_remove);
+}
+
+void clear_hook_list(struct list_head *head)
+{
+	struct list_head *pos, *tmp;
+	list_for_each_safe(pos, tmp, head)
+		remove_hook(pos);
+}
+
 static int known_hook(const char *name)
 {
 	const char **p;
@@ -71,6 +93,30 @@ int hook_exists(const char *name)
 	return !!find_hook(name);
 }
 
+struct list_head* hook_list(const char* hookname)
+{
+	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
+
+	INIT_LIST_HEAD(hook_head);
+
+	if (!hookname)
+		return NULL;
+
+	if (have_git_dir()) {
+		const char *hook_path = find_hook(hookname);
+
+		/* Add the hook from the hookdir */
+		if (hook_path) {
+			struct hook *to_add = xmalloc(sizeof(*to_add));
+			to_add->hook_path = hook_path;
+			to_add->feed_pipe_cb_data = NULL;
+			list_add_tail(&to_add->list, hook_head);
+		}
+	}
+
+	return hook_head;
+}
+
 void run_hooks_opt_clear(struct run_hooks_opt *o)
 {
 	strvec_clear(&o->env);
@@ -128,7 +174,10 @@ static int pick_next_hook(struct child_process *cp,
 	cp->dir = hook_cb->options->dir;
 
 	/* add command */
-	strvec_push(&cp->args, run_me->hook_path);
+	if (hook_cb->options->absolute_path)
+		strvec_push(&cp->args, absolute_path(run_me->hook_path));
+	else
+		strvec_push(&cp->args, run_me->hook_path);
 
 	/*
 	 * add passed-in argv, without expanding - let the user get back
@@ -139,12 +188,12 @@ static int pick_next_hook(struct child_process *cp,
 	/* Provide context for errors if necessary */
 	*pp_task_cb = run_me;
 
-	/*
-	 * This pick_next_hook() will be called again, we're only
-	 * running one hook, so indicate that no more work will be
-	 * done.
-	 */
-	hook_cb->run_me = NULL;
+	/* Get the next entry ready */
+	if (hook_cb->run_me->list.next == hook_cb->head)
+		hook_cb->run_me = NULL;
+	else
+		hook_cb->run_me = list_entry(hook_cb->run_me->list.next,
+					     struct hook, list);
 
 	return 1;
 }
@@ -179,13 +228,9 @@ static int notify_hook_finished(int result,
 	return 0;
 }
 
-int run_hooks(const char *hook_name, const char *hook_path,
-	      struct run_hooks_opt *options)
+int run_hooks(const char *hook_name, struct list_head *hooks,
+		    struct run_hooks_opt *options)
 {
-	struct strbuf abs_path = STRBUF_INIT;
-	struct hook my_hook = {
-		.hook_path = hook_path,
-	};
 	struct hook_cb_data cb_data = {
 		.rc = 0,
 		.hook_name = hook_name,
@@ -197,11 +242,9 @@ int run_hooks(const char *hook_name, const char *hook_path,
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
 
-	if (options->absolute_path) {
-		strbuf_add_absolute_path(&abs_path, hook_path);
-		my_hook.hook_path = abs_path.buf;
-	}
-	cb_data.run_me = &my_hook;
+
+	cb_data.head = hooks;
+	cb_data.run_me = list_first_entry(hooks, struct hook, list);
 
 	run_processes_parallel_tr2(jobs,
 				   pick_next_hook,
@@ -213,18 +256,15 @@ int run_hooks(const char *hook_name, const char *hook_path,
 				   "hook",
 				   hook_name);
 
-
-	if (options->absolute_path)
-		strbuf_release(&abs_path);
-	free(my_hook.feed_pipe_cb_data);
+	clear_hook_list(hooks);
 
 	return cb_data.rc;
 }
 
 int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 {
-	const char *hook_path;
-	int ret;
+	struct list_head *hooks;
+	int ret = 0;
 	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
 
 	if (!options)
@@ -233,14 +273,19 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 	if (options->path_to_stdin && options->feed_pipe)
 		BUG("choose only one method to populate stdin");
 
-	hook_path = find_hook(hook_name);
-	if (!hook_path) {
-		ret = 0;
+	hooks = hook_list(hook_name);
+
+	/*
+	 * If you need to act on a missing hook, use run_found_hooks()
+	 * instead
+	 */
+	if (list_empty(hooks))
 		goto cleanup;
-	}
 
-	ret = run_hooks(hook_name, hook_path, options);
+	ret = run_hooks(hook_name, hooks, options);
+
 cleanup:
 	run_hooks_opt_clear(options);
+	clear_hook_list(hooks);
 	return ret;
 }
diff --git a/hook.h b/hook.h
index 58dfbf474c..7705e6a529 100644
--- a/hook.h
+++ b/hook.h
@@ -3,6 +3,7 @@
 #include "strbuf.h"
 #include "strvec.h"
 #include "run-command.h"
+#include "list.h"
 
 /*
  * Returns the path to the hook file, or NULL if the hook is missing
@@ -17,6 +18,7 @@ const char *find_hook(const char *name);
 int hook_exists(const char *hookname);
 
 struct hook {
+	struct list_head list;
 	/* The path to the hook */
 	const char *hook_path;
 
@@ -27,6 +29,12 @@ struct hook {
 	void *feed_pipe_cb_data;
 };
 
+/*
+ * Provides a linked list of 'struct hook' detailing commands which should run
+ * in response to the 'hookname' event, in execution order.
+ */
+struct list_head* hook_list(const char *hookname);
+
 struct run_hooks_opt
 {
 	/* Environment vars to be set for each hook */
@@ -97,6 +105,7 @@ struct hook_cb_data {
 	/* rc reflects the cumulative failure state */
 	int rc;
 	const char *hook_name;
+	struct list_head *head;
 	struct hook *run_me;
 	struct run_hooks_opt *options;
 	int *invoked_hook;
@@ -110,8 +119,8 @@ void run_hooks_opt_clear(struct run_hooks_opt *o);
  *
  * See run_hooks_oneshot() for the simpler one-shot API.
  */
-int run_hooks(const char *hookname, const char *hook_path,
-	      struct run_hooks_opt *options);
+int run_hooks(const char *hookname, struct list_head *hooks,
+		    struct run_hooks_opt *options);
 
 /**
  * Calls find_hook() on your "hook_name" and runs the hooks (if any)
@@ -123,4 +132,7 @@ int run_hooks(const char *hookname, const char *hook_path,
  */
 int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
 
+/* Empties the list at 'head', calling 'free_hook()' on each entry */
+void clear_hook_list(struct list_head *head);
+
 #endif
-- 
2.32.0.605.g8dce9f2422-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 2/6] hook: allow parallel hook execution
  2021-08-12  0:42             ` [PATCH v2 0/6] config-based hooks restarted Emily Shaffer
  2021-08-12  0:42               ` [PATCH v2 1/6] hook: run a list of hooks instead Emily Shaffer
@ 2021-08-12  0:42               ` Emily Shaffer
  2021-08-12 17:51                 ` Junio C Hamano
  2021-08-12  0:42               ` [PATCH v2 3/6] hook: introduce "git hook list" Emily Shaffer
                                 ` (5 subsequent siblings)
  7 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-08-12  0:42 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer, Ævar Arnfjörð Bjarmason

In many cases, there's no reason not to allow hooks to execute in
parallel. run_processes_parallel() is well-suited - it's a task queue
that runs its housekeeping in series, which means users don't
need to worry about thread safety on their callback data. True
multithreaded execution with the async_* functions isn't necessary here.
Synchronous hook execution can be achieved by only allowing 1 job to run
at a time.

Teach run_hooks() to use that function for simple hooks which don't
require stdin or capture of stderr.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/config/hook.txt |  4 ++++
 Documentation/git-hook.txt    | 17 ++++++++++++++++-
 builtin/am.c                  |  4 ++--
 builtin/checkout.c            |  2 +-
 builtin/clone.c               |  2 +-
 builtin/hook.c                |  4 +++-
 builtin/merge.c               |  2 +-
 builtin/rebase.c              |  2 +-
 builtin/receive-pack.c        |  9 +++++----
 builtin/worktree.c            |  2 +-
 commit.c                      |  2 +-
 hook.c                        | 36 +++++++++++++++++++++++++++++++----
 hook.h                        | 24 ++++++++++++++++++-----
 read-cache.c                  |  2 +-
 refs.c                        |  2 +-
 reset.c                       |  3 ++-
 sequencer.c                   |  4 ++--
 transport.c                   |  2 +-
 18 files changed, 94 insertions(+), 29 deletions(-)
 create mode 100644 Documentation/config/hook.txt

diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
new file mode 100644
index 0000000000..96d3d6572c
--- /dev/null
+++ b/Documentation/config/hook.txt
@@ -0,0 +1,4 @@
+hook.jobs::
+	Specifies how many hooks can be run simultaneously during parallelized
+	hook execution. If unspecified, defaults to the number of processors on
+	the current system.
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index fa68c1f391..8bf82b5dd4 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -8,7 +8,8 @@ git-hook - run git hooks
 SYNOPSIS
 --------
 [verse]
-'git hook' run [--to-stdin=<path>] [--ignore-missing] <hook-name> [-- <hook-args>]
+'git hook' run [--to-stdin=<path>] [--ignore-missing] [(-j|--jobs) <n>]
+	<hook-name> [-- <hook-args>]
 
 DESCRIPTION
 -----------
@@ -42,6 +43,20 @@ OPTIONS
 	tools that want to do a blind one-shot run of a hook that may
 	or may not be present.
 
+-j::
+--jobs::
+	Only valid for `run`.
++
+Specify how many hooks to run simultaneously. If this flag is not specified, use
+the value of the `hook.jobs` config. If the config is not specified, use the
+number of CPUs on the current system. Some hooks may be ineligible for
+parallelization: for example, 'commit-msg' intends hooks modify the commit
+message body and cannot be parallelized.
+
+CONFIGURATION
+-------------
+include::config/hook.txt[]
+
 SEE ALSO
 --------
 linkgit:githooks[5]
diff --git a/builtin/am.c b/builtin/am.c
index 9e3d4d9ab4..c24a27d6a1 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -446,7 +446,7 @@ static void am_destroy(const struct am_state *state)
 static int run_applypatch_msg_hook(struct am_state *state)
 {
 	int ret;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
 
 	assert(state->msg);
 	strvec_push(&opt.args, am_path(state, "final-commit"));
@@ -467,7 +467,7 @@ static int run_applypatch_msg_hook(struct am_state *state)
  */
 static int run_post_rewrite_hook(const struct am_state *state)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
 
 	strvec_push(&opt.args, "rebase");
 	opt.path_to_stdin = am_path(state, "rewritten");
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 6d69b4c011..27166c0bb8 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -107,7 +107,7 @@ struct branch_info {
 static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit,
 			      int changed)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
 
 	/* "new_commit" can be NULL when checking out from the index before
 	   a commit exists. */
diff --git a/builtin/clone.c b/builtin/clone.c
index 27fc05ee51..599e7a7936 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -776,7 +776,7 @@ static int checkout(int submodule_progress)
 	struct tree *tree;
 	struct tree_desc t;
 	int err = 0;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_SYNC;
 
 	if (option_no_checkout)
 		return 0;
diff --git a/builtin/hook.c b/builtin/hook.c
index 4d39c9e75e..12c9126032 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -22,7 +22,7 @@ static const char * const builtin_hook_run_usage[] = {
 static int run(int argc, const char **argv, const char *prefix)
 {
 	int i;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
 	int ignore_missing = 0;
 	const char *hook_name;
 	struct list_head *hooks;
@@ -32,6 +32,8 @@ static int run(int argc, const char **argv, const char *prefix)
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
 		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
 			   N_("file to read into hooks' stdin")),
+		OPT_INTEGER('j', "jobs", &opt.jobs,
+			    N_("run up to <n> hooks simultaneously")),
 		OPT_END(),
 	};
 	int ret;
diff --git a/builtin/merge.c b/builtin/merge.c
index 9bd4a2532c..c749c382c3 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -448,7 +448,7 @@ static void finish(struct commit *head_commit,
 		   const struct object_id *new_head, const char *msg)
 {
 	struct strbuf reflog_message = STRBUF_INIT;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
 	const struct object_id *head = &head_commit->object.oid;
 
 	if (!msg)
diff --git a/builtin/rebase.c b/builtin/rebase.c
index e7c668c99b..fecf248ed9 100644
--- a/builtin/rebase.c
+++ b/builtin/rebase.c
@@ -1314,7 +1314,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	char *squash_onto_name = NULL;
 	int reschedule_failed_exec = -1;
 	int allow_preemptive_ff = 1;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_ASYNC;
 	struct option builtin_rebase_options[] = {
 		OPT_STRING(0, "onto", &options.onto_name,
 			   N_("revision"),
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index ebec6f3bb1..b32dcc9000 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -909,7 +909,7 @@ static int run_receive_hook(struct command *commands,
 			    int skip_broken,
 			    const struct string_list *push_options)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
 	struct receive_hook_feed_context ctx;
 	struct command *iter = commands;
 
@@ -948,7 +948,7 @@ static int run_receive_hook(struct command *commands,
 
 static int run_update_hook(struct command *cmd)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
 
 	strvec_pushl(&opt.args,
 		     cmd->ref_name,
@@ -1432,7 +1432,8 @@ static const char *push_to_checkout(unsigned char *hash,
 				    struct strvec *env,
 				    const char *work_tree)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
+
 	opt.invoked_hook = invoked_hook;
 
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
@@ -1628,7 +1629,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 static void run_update_post_hook(struct command *commands)
 {
 	struct command *cmd;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
 
 	for (cmd = commands; cmd; cmd = cmd->next) {
 		if (cmd->error_string || cmd->did_not_exist)
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 330867c19b..efead564c1 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -382,7 +382,7 @@ static int add_worktree(const char *path, const char *refname,
 	 * is_junk is cleared, but do return appropriate code when hook fails.
 	 */
 	if (!ret && opts->checkout) {
-		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
 
 		strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
 		strvec_pushl(&opt.args,
diff --git a/commit.c b/commit.c
index 842e47beae..0e6e5a5b27 100644
--- a/commit.c
+++ b/commit.c
@@ -1700,7 +1700,7 @@ int run_commit_hook(int editor_is_used, const char *index_file,
 		    int *invoked_hook,
 		    const char *name, ...)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
 	va_list args;
 	const char *arg;
 
diff --git a/hook.c b/hook.c
index 80e150548c..37f682c6d8 100644
--- a/hook.c
+++ b/hook.c
@@ -228,6 +228,28 @@ static int notify_hook_finished(int result,
 	return 0;
 }
 
+/*
+ * Determines how many jobs to use after we know we want to parallelize. First
+ * priority is the config 'hook.jobs' and second priority is the number of CPUs.
+ */
+static int configured_hook_jobs(void)
+{
+	/*
+	 * The config and the CPU count probably won't change during the process
+	 * lifetime, so cache the result in case we invoke multiple hooks during
+	 * one process.
+	 */
+	static int jobs = 0;
+	if (jobs)
+		return jobs;
+
+	if (git_config_get_int("hook.jobs", &jobs))
+		/* if the config isn't set, fall back to CPU count. */
+		jobs = online_cpus();
+
+	return jobs;
+}
+
 int run_hooks(const char *hook_name, struct list_head *hooks,
 		    struct run_hooks_opt *options)
 {
@@ -237,16 +259,18 @@ int run_hooks(const char *hook_name, struct list_head *hooks,
 		.options = options,
 		.invoked_hook = options->invoked_hook,
 	};
-	int jobs = 1;
 
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
 
-
 	cb_data.head = hooks;
 	cb_data.run_me = list_first_entry(hooks, struct hook, list);
 
-	run_processes_parallel_tr2(jobs,
+	/* INIT_ASYNC sets jobs to 0, so go look up how many to use. */
+	if (!options->jobs)
+		options->jobs = configured_hook_jobs();
+
+	run_processes_parallel_tr2(options->jobs,
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
@@ -265,7 +289,11 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 {
 	struct list_head *hooks;
 	int ret = 0;
-	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
+	/*
+	 * Turn on parallelism by default. Hooks which don't want it should
+	 * specify their options accordingly.
+	 */
+	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT_ASYNC;
 
 	if (!options)
 		options = &hook_opt_scratch;
diff --git a/hook.h b/hook.h
index 7705e6a529..4f90228a0c 100644
--- a/hook.h
+++ b/hook.h
@@ -43,6 +43,13 @@ struct run_hooks_opt
 	/* Args to be passed to each hook */
 	struct strvec args;
 
+	/*
+	 * Number of threads to parallelize across. Set to 0 to use the
+	 * 'hook.jobs' config or, if that config is unset, the number of cores
+	 * on the system.
+	 */
+	int jobs;
+
 	/* Resolve and run the "absolute_path(hook)" instead of
 	 * "hook". Used for "git worktree" hooks
 	 */
@@ -85,11 +92,6 @@ struct run_hooks_opt
 	int *invoked_hook;
 };
 
-#define RUN_HOOKS_OPT_INIT { \
-	.env = STRVEC_INIT, \
-	.args = STRVEC_INIT, \
-}
-
 /*
  * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
  * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'.
@@ -111,6 +113,18 @@ struct hook_cb_data {
 	int *invoked_hook;
 };
 
+#define RUN_HOOKS_OPT_INIT_SYNC { \
+	.jobs = 1, \
+	.env = STRVEC_INIT, \
+	.args = STRVEC_INIT, \
+}
+
+#define RUN_HOOKS_OPT_INIT_ASYNC { \
+	.jobs = 0, \
+	.env = STRVEC_INIT, \
+	.args = STRVEC_INIT, \
+}
+
 void run_hooks_opt_clear(struct run_hooks_opt *o);
 
 /**
diff --git a/read-cache.c b/read-cache.c
index 90099ca14d..fd2bc67667 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -3068,7 +3068,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 {
 	int ret;
 	int was_full = !istate->sparse_index;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_ASYNC;
 
 	ret = convert_to_sparse(istate);
 
diff --git a/refs.c b/refs.c
index 73d4a93926..305a075746 100644
--- a/refs.c
+++ b/refs.c
@@ -2062,7 +2062,7 @@ int ref_update_reject_duplicates(struct string_list *refnames,
 static int run_transaction_hook(struct ref_transaction *transaction,
 				const char *state)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
 	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
 	int ret = 0, i;
 
diff --git a/reset.c b/reset.c
index 6499bc5127..b93fe6a783 100644
--- a/reset.c
+++ b/reset.c
@@ -128,7 +128,8 @@ int reset_head(struct repository *r, struct object_id *oid, const char *action,
 					    reflog_head);
 	}
 	if (run_hook) {
-		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
+
 		strvec_pushl(&opt.args,
 			     oid_to_hex(orig ? orig : null_oid()),
 			     oid_to_hex(oid),
diff --git a/sequencer.c b/sequencer.c
index f451e23d0c..f13a0cbfbe 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1148,7 +1148,7 @@ int update_head_with_reflog(const struct commit *old_head,
 static int run_rewrite_hook(const struct object_id *oldoid,
 			    const struct object_id *newoid)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
 	struct strbuf tmp = STRBUF_INIT;
 	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int code;
@@ -4522,7 +4522,7 @@ static int pick_commits(struct repository *r,
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
 			struct child_process notes_cp = CHILD_PROCESS_INIT;
-			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_ASYNC;
 
 			notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY);
 			notes_cp.git_cmd = 1;
diff --git a/transport.c b/transport.c
index 4ca8fc0391..abf8785885 100644
--- a/transport.c
+++ b/transport.c
@@ -1204,7 +1204,7 @@ static int run_pre_push_hook(struct transport *transport,
 			     struct ref *remote_refs)
 {
 	int ret = 0;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
 	struct ref *r;
 	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
 
-- 
2.32.0.605.g8dce9f2422-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 3/6] hook: introduce "git hook list"
  2021-08-12  0:42             ` [PATCH v2 0/6] config-based hooks restarted Emily Shaffer
  2021-08-12  0:42               ` [PATCH v2 1/6] hook: run a list of hooks instead Emily Shaffer
  2021-08-12  0:42               ` [PATCH v2 2/6] hook: allow parallel hook execution Emily Shaffer
@ 2021-08-12  0:42               ` Emily Shaffer
  2021-08-12 18:59                 ` Junio C Hamano
  2021-08-12  0:42               ` [PATCH v2 4/6] hook: allow running non-native hooks Emily Shaffer
                                 ` (4 subsequent siblings)
  7 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-08-12  0:42 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

If more than one hook will be run, it may be useful to see a list of
which hooks should be run. At very least, it will be useful for us to
test the semantics of multihooks ourselves.

For now, only list the hooks which will run in the order they will run
in; later, it might be useful to include more information like where the
hooks were configured and whether or not they will run.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 builtin/hook.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
 hook.c         | 18 ++++++++----------
 2 files changed, 57 insertions(+), 10 deletions(-)

diff --git a/builtin/hook.c b/builtin/hook.c
index 12c9126032..c36b05376c 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -8,8 +8,11 @@
 
 #define BUILTIN_HOOK_RUN_USAGE \
 	N_("git hook run [--to-stdin=<path>] <hook-name> [-- <hook-args>]")
+#define BUILTIN_HOOK_LIST_USAGE \
+	N_("git hook list <hook-name>")
 
 static const char * const builtin_hook_usage[] = {
+	BUILTIN_HOOK_LIST_USAGE,
 	BUILTIN_HOOK_RUN_USAGE,
 	NULL
 };
@@ -19,6 +22,50 @@ static const char * const builtin_hook_run_usage[] = {
 	NULL
 };
 
+static const char *const builtin_hook_list_usage[] = {
+	BUILTIN_HOOK_LIST_USAGE,
+	NULL
+};
+
+static int list(int argc, const char **argv, const char *prefix)
+{
+	struct list_head *head, *pos;
+	const char *hookname = NULL;
+	struct strbuf hookdir_annotation = STRBUF_INIT;
+
+	struct option list_options[] = {
+		OPT_END(),
+	};
+
+	argc = parse_options(argc, argv, prefix, list_options,
+			     builtin_hook_list_usage, 0);
+
+	if (argc < 1)
+		usage_msg_opt(_("You must specify a hook event name to list."),
+			      builtin_hook_list_usage, list_options);
+
+	hookname = argv[0];
+
+	head = hook_list(hookname);
+
+	if (list_empty(head)) {
+		printf(_("no commands configured for hook '%s'\n"),
+		       hookname);
+		return 0;
+	}
+
+	list_for_each(pos, head) {
+		struct hook *item = list_entry(pos, struct hook, list);
+		item = list_entry(pos, struct hook, list);
+		if (item)
+			printf("%s\n", item->hook_path);
+	}
+
+	clear_hook_list(head);
+	strbuf_release(&hookdir_annotation);
+
+	return 0;
+}
 static int run(int argc, const char **argv, const char *prefix)
 {
 	int i;
@@ -88,6 +135,8 @@ int cmd_hook(int argc, const char **argv, const char *prefix)
 	if (!argc)
 		goto usage;
 
+	if (!strcmp(argv[0], "list"))
+		return list(argc, argv, prefix);
 	if (!strcmp(argv[0], "run"))
 		return run(argc, argv, prefix);
 
diff --git a/hook.c b/hook.c
index 37f682c6d8..2714b63473 100644
--- a/hook.c
+++ b/hook.c
@@ -96,22 +96,20 @@ int hook_exists(const char *name)
 struct list_head* hook_list(const char* hookname)
 {
 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
+	const char *hook_path = find_hook(hookname);
+
 
 	INIT_LIST_HEAD(hook_head);
 
 	if (!hookname)
 		return NULL;
 
-	if (have_git_dir()) {
-		const char *hook_path = find_hook(hookname);
-
-		/* Add the hook from the hookdir */
-		if (hook_path) {
-			struct hook *to_add = xmalloc(sizeof(*to_add));
-			to_add->hook_path = hook_path;
-			to_add->feed_pipe_cb_data = NULL;
-			list_add_tail(&to_add->list, hook_head);
-		}
+	/* Add the hook from the hookdir */
+	if (hook_path) {
+		struct hook *to_add = xmalloc(sizeof(*to_add));
+		to_add->hook_path = hook_path;
+		to_add->feed_pipe_cb_data = NULL;
+		list_add_tail(&to_add->list, hook_head);
 	}
 
 	return hook_head;
-- 
2.32.0.605.g8dce9f2422-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 4/6] hook: allow running non-native hooks
  2021-08-12  0:42             ` [PATCH v2 0/6] config-based hooks restarted Emily Shaffer
                                 ` (2 preceding siblings ...)
  2021-08-12  0:42               ` [PATCH v2 3/6] hook: introduce "git hook list" Emily Shaffer
@ 2021-08-12  0:42               ` Emily Shaffer
  2021-08-12 19:08                 ` Junio C Hamano
  2021-08-12  0:42               ` [PATCH v2 5/6] hook: include hooks from the config Emily Shaffer
                                 ` (3 subsequent siblings)
  7 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-08-12  0:42 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

As the hook architecture and 'git hook run' become more featureful, we
may find wrappers wanting to use the hook architecture to run their own
hooks, thereby getting nice things like parallelism and idiomatic Git
configuration for free. Enable this by letting 'git hook run' bypass the
known_hooks() check.

We do still want to keep known_hooks() around, though - by die()ing when
an internal Git call asks for run_hooks("my-new-hook"), we can remind
Git developers to update Documentation/githooks.txt with their new hook,
which in turn helps Git users discover this new hook.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-hook.txt |  8 ++++++++
 builtin/hook.c             |  4 ++--
 hook.c                     | 32 ++++++++++++++++++++++++++++----
 hook.h                     | 16 +++++++++++++++-
 4 files changed, 53 insertions(+), 7 deletions(-)

diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 8bf82b5dd4..11a8b87c60 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -18,6 +18,14 @@ This command is an interface to git hooks (see linkgit:githooks[5]).
 Currently it only provides a convenience wrapper for running hooks for
 use by git itself. In the future it might gain other functionality.
 
+It's possible to use this command to refer to hooks which are not native to Git,
+for example if a wrapper around Git wishes to expose hooks into its own
+operation in a way which is already familiar to Git users. However, wrappers
+invoking such hooks should be careful to name their hook events something which
+Git is unlikely to use for a native hook later on. For example, Git is much less
+likely to create a `mytool-validate-commit` hook than it is to create a
+`validate-commit` hook.
+
 SUBCOMMANDS
 -----------
 
diff --git a/builtin/hook.c b/builtin/hook.c
index c36b05376c..3aa65dd791 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -46,7 +46,7 @@ static int list(int argc, const char **argv, const char *prefix)
 
 	hookname = argv[0];
 
-	head = hook_list(hookname);
+	head = hook_list(hookname, 1);
 
 	if (list_empty(head)) {
 		printf(_("no commands configured for hook '%s'\n"),
@@ -108,7 +108,7 @@ static int run(int argc, const char **argv, const char *prefix)
 	git_config(git_default_config, NULL);
 
 	hook_name = argv[0];
-	hooks = hook_list(hook_name);
+	hooks = hook_list(hook_name, 1);
 	if (list_empty(hooks)) {
 		/* ... act like run_hooks_oneshot() under --ignore-missing */
 		if (ignore_missing)
diff --git a/hook.c b/hook.c
index 2714b63473..e5acd02a50 100644
--- a/hook.c
+++ b/hook.c
@@ -52,12 +52,21 @@ static int known_hook(const char *name)
 
 const char *find_hook(const char *name)
 {
-	static struct strbuf path = STRBUF_INIT;
+	const char *hook_path;
 
 	if (!known_hook(name))
 		die(_("the hook '%s' is not known to git, should be in hook-list.h via githooks(5)"),
 		    name);
 
+	hook_path = find_hook_gently(name);
+
+	return hook_path;
+}
+
+const char *find_hook_gently(const char *name)
+{
+	static struct strbuf path = STRBUF_INIT;
+
 	strbuf_reset(&path);
 	strbuf_git_path(&path, "hooks/%s", name);
 	if (access(path.buf, X_OK) < 0) {
@@ -93,10 +102,16 @@ int hook_exists(const char *name)
 	return !!find_hook(name);
 }
 
-struct list_head* hook_list(const char* hookname)
+struct hook_config_cb
+{
+	struct strbuf *hook_key;
+	struct list_head *list;
+};
+
+struct list_head* hook_list(const char* hookname, int allow_unknown)
 {
 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
-	const char *hook_path = find_hook(hookname);
+	const char *hook_path;
 
 
 	INIT_LIST_HEAD(hook_head);
@@ -104,6 +119,11 @@ struct list_head* hook_list(const char* hookname)
 	if (!hookname)
 		return NULL;
 
+	if (allow_unknown)
+		hook_path = find_hook_gently(hookname);
+	else
+		hook_path = find_hook(hookname);
+
 	/* Add the hook from the hookdir */
 	if (hook_path) {
 		struct hook *to_add = xmalloc(sizeof(*to_add));
@@ -299,7 +319,11 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 	if (options->path_to_stdin && options->feed_pipe)
 		BUG("choose only one method to populate stdin");
 
-	hooks = hook_list(hook_name);
+	/*
+	 * 'git hooks run <hookname>' uses run_found_hooks, so we don't need to
+	 * allow unknown hooknames here.
+	 */
+	hooks = hook_list(hook_name, 0);
 
 	/*
 	 * If you need to act on a missing hook, use run_found_hooks()
diff --git a/hook.h b/hook.h
index 4f90228a0c..ffa96c6e4d 100644
--- a/hook.h
+++ b/hook.h
@@ -9,8 +9,16 @@
  * Returns the path to the hook file, or NULL if the hook is missing
  * or disabled. Note that this points to static storage that will be
  * overwritten by further calls to find_hook and run_hook_*.
+ *
+ * If the hook is not a native hook (e.g. present in Documentation/githooks.txt)
+ * find_hook() will die(). find_hook_gently() does not consult the native hook
+ * list and will check for any hook name in the hooks directory regardless of
+ * whether it is known. find_hook() should be used by internal calls to hooks,
+ * and find_hook_gently() should only be used when the hookname was provided by
+ * the user, such as by 'git hook run my-wrapper-hook'.
  */
 const char *find_hook(const char *name);
+const char *find_hook_gently(const char *name);
 
 /*
  * A boolean version of find_hook()
@@ -32,8 +40,14 @@ struct hook {
 /*
  * Provides a linked list of 'struct hook' detailing commands which should run
  * in response to the 'hookname' event, in execution order.
+ *
+ * If allow_unknown is unset, hooks will be checked against the hook list
+ * generated from Documentation/githooks.txt. Otherwise, any hook name will be
+ * allowed. allow_unknown should only be set when the hook name is provided by
+ * the user; internal calls to hook_list should make sure the hook they are
+ * invoking is present in Documentation/githooks.txt.
  */
-struct list_head* hook_list(const char *hookname);
+struct list_head* hook_list(const char *hookname, int allow_unknown);
 
 struct run_hooks_opt
 {
-- 
2.32.0.605.g8dce9f2422-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 5/6] hook: include hooks from the config
  2021-08-12  0:42             ` [PATCH v2 0/6] config-based hooks restarted Emily Shaffer
                                 ` (3 preceding siblings ...)
  2021-08-12  0:42               ` [PATCH v2 4/6] hook: allow running non-native hooks Emily Shaffer
@ 2021-08-12  0:42               ` Emily Shaffer
  2021-08-12 20:48                 ` Junio C Hamano
  2021-08-12  0:42               ` [PATCH v2 6/6] hook: allow out-of-repo 'git hook' invocations Emily Shaffer
                                 ` (2 subsequent siblings)
  7 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-08-12  0:42 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach the hook.[hc] library to parse configs to populare the list of
hooks to run for a given event.

Multiple commands can be specified for a given hook by providing
multiple "hook.<friendly-name>.command = <path-to-hook>" and
"hook.<friendly-name>.event = <hook-event>" lines. Hooks will be run in
config order of the "hook.<name>.event" lines.

For example:

  $ git config --list | grep ^hook
  hook.bar.command=~/bar.sh
  hook.bar.event=pre-commit

  $ git hook run
  # Runs ~/bar.sh
  # Runs .git/hooks/pre-commit

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/config/hook.txt |   5 +
 Documentation/git-hook.txt    |  23 ++++-
 builtin/hook.c                |   5 +-
 hook.c                        | 166 +++++++++++++++++++++++++++++-----
 hook.h                        |   7 +-
 t/t1800-hook.sh               | 141 ++++++++++++++++++++++++++++-
 6 files changed, 318 insertions(+), 29 deletions(-)

diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
index 96d3d6572c..a97b980cca 100644
--- a/Documentation/config/hook.txt
+++ b/Documentation/config/hook.txt
@@ -1,3 +1,8 @@
+hook.<command>.command::
+	A command to execute during the <command> hook event. This can be an
+	executable on your device, a oneliner for your shell, or the name of a
+	hookcmd. See linkgit:git-hook[1].
+
 hook.jobs::
 	Specifies how many hooks can be run simultaneously during parallelized
 	hook execution. If unspecified, defaults to the number of processors on
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 11a8b87c60..c610ed9583 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -26,12 +26,31 @@ Git is unlikely to use for a native hook later on. For example, Git is much less
 likely to create a `mytool-validate-commit` hook than it is to create a
 `validate-commit` hook.
 
+This command parses the default configuration files for pairs of configs like
+so:
+
+  [hook "linter"]
+    event = pre-commit
+    command = ~/bin/linter --c
+
+Conmmands are run in the order Git encounters their associated
+`hook.<name>.event` configs during the configuration parse (see
+linkgit:git-config[1]).
+
+In general, when instructions suggest adding a script to
+`.git/hooks/<hook-event>`, you can specify it in the config instead by running
+`git config --add hook.<some-name>.command <path-to-script> && git config --add
+hook.<some-name>.event <hook-event>` - this way you can share the script between
+multiple repos. That is, `cp ~/my-script.sh ~/project/.git/hooks/pre-commit`
+would become `git config --add hook.my-script.command ~/my-script.sh && git
+config --add hook.my-script.event pre-commit`.
+
 SUBCOMMANDS
 -----------
 
 run::
-	Run the `<hook-name>` hook. See linkgit:githooks[5] for
-	the hook names we support.
+	Runs hooks configured for `<hook-name>`, in the order they are
+	discovered during the config parse.
 +
 Any positional arguments to the hook should be passed after an
 optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
diff --git a/builtin/hook.c b/builtin/hook.c
index 3aa65dd791..ea49dc4ef6 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -49,7 +49,7 @@ static int list(int argc, const char **argv, const char *prefix)
 	head = hook_list(hookname, 1);
 
 	if (list_empty(head)) {
-		printf(_("no commands configured for hook '%s'\n"),
+		printf(_("no hooks configured for event '%s'\n"),
 		       hookname);
 		return 0;
 	}
@@ -58,7 +58,8 @@ static int list(int argc, const char **argv, const char *prefix)
 		struct hook *item = list_entry(pos, struct hook, list);
 		item = list_entry(pos, struct hook, list);
 		if (item)
-			printf("%s\n", item->hook_path);
+			printf("%s\n", item->name ? item->name
+						  : _("hook from hookdir"));
 	}
 
 	clear_hook_list(head);
diff --git a/hook.c b/hook.c
index e5acd02a50..51ada266bc 100644
--- a/hook.c
+++ b/hook.c
@@ -12,6 +12,50 @@ static void free_hook(struct hook *ptr)
 	free(ptr);
 }
 
+/*
+ * Walks the linked list at 'head' to check if any hook named 'name'
+ * already exists. Returns a pointer to that hook if so, otherwise returns NULL.
+ */
+static struct hook *find_hook_by_name(struct list_head *head,
+					 const char *name)
+{
+	struct list_head *pos = NULL, *tmp = NULL;
+	struct hook *found = NULL;
+
+	list_for_each_safe(pos, tmp, head) {
+		struct hook *it = list_entry(pos, struct hook, list);
+		if (!strcmp(it->name, name)) {
+			list_del(pos);
+			found = it;
+			break;
+		}
+	}
+	return found;
+}
+
+/*
+ * Adds a hook if it's not already in the list, or moves it to the tail of the
+ * list if it was already there. name == NULL indicates it's from the hookdir;
+ * just append it in this case.
+ */
+static void append_or_move_hook(struct list_head *head, const char *name)
+{
+	struct hook *to_add = NULL;
+
+	/* if it's not from hookdir, check if the hook is already in the list */
+	if (name)
+		to_add = find_hook_by_name(head, name);
+
+	if (!to_add) {
+		/* adding a new hook, not moving an old one */
+		to_add = xmalloc(sizeof(*to_add));
+		to_add->name = name;
+		to_add->feed_pipe_cb_data = NULL;
+	}
+
+	list_add_tail(&to_add->list, head);
+}
+
 static void remove_hook(struct list_head *to_remove)
 {
 	struct hook *hook_to_remove = list_entry(to_remove, struct hook, list);
@@ -99,38 +143,80 @@ const char *find_hook_gently(const char *name)
 
 int hook_exists(const char *name)
 {
-	return !!find_hook(name);
+	return !list_empty(hook_list(name, 0));
 }
 
 struct hook_config_cb
 {
-	struct strbuf *hook_key;
+	const char *hook_event;
 	struct list_head *list;
 };
 
-struct list_head* hook_list(const char* hookname, int allow_unknown)
+/*
+ * Callback for git_config which adds configured hooks to a hook list.  Hooks
+ * can be configured by specifying both hook.<friend-name>.command = <path> and
+ * hook.<friendly-name>.event = <hook-event>.
+ */
+static int hook_config_lookup(const char *key, const char *value, void *cb_data)
+{
+	struct hook_config_cb *data = cb_data;
+	const char *subsection, *parsed_key;
+	size_t subsection_len = 0;
+	struct strbuf subsection_cpy = STRBUF_INIT;
+
+	/*
+	 * Don't bother doing the expensive parse if there's no
+	 * chance that the config matches 'hook.myhook.event = hook_event'.
+	 */
+	if (!value || strcmp(value, data->hook_event))
+		return 0;
+
+	/* Looking for "hook.friendlyname.event = hook_event" */
+	if (parse_config_key(key,
+			    "hook",
+			    &subsection,
+			    &subsection_len,
+			    &parsed_key) ||
+	    strcmp(parsed_key, "event"))
+		return 0;
+
+	/*
+	 * 'subsection' is a pointer to the internals of 'key', which we don't
+	 * own the memory for. Copy it away to the hook list.
+	 */
+	strbuf_add(&subsection_cpy, subsection, subsection_len);
+
+	append_or_move_hook(data->list, strbuf_detach(&subsection_cpy, NULL));
+
+
+	return 0;
+}
+
+struct list_head* hook_list(const char *hookname, int allow_unknown)
 {
 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
-	const char *hook_path;
+	struct hook_config_cb cb_data = {
+		.hook_event = hookname,
+		.list = hook_head,
+	};
 
+	if (!allow_unknown && !known_hook(hookname))
+		die(_("Don't recognize hook event '%s'. "
+		      "Is it documented in 'githooks.txt'?"),
+		      hookname);
 
 	INIT_LIST_HEAD(hook_head);
 
 	if (!hookname)
 		return NULL;
 
-	if (allow_unknown)
-		hook_path = find_hook_gently(hookname);
-	else
-		hook_path = find_hook(hookname);
+	/* Add the hooks from the config, e.g. hook.myhook.event = pre-commit */
+	git_config(hook_config_lookup, &cb_data);
 
-	/* Add the hook from the hookdir */
-	if (hook_path) {
-		struct hook *to_add = xmalloc(sizeof(*to_add));
-		to_add->hook_path = hook_path;
-		to_add->feed_pipe_cb_data = NULL;
-		list_add_tail(&to_add->list, hook_head);
-	}
+	/* Add the hook from the hookdir. The placeholder makes it easier to
+	 * allocate work in pick_next_hook. */
+	if (find_hook_gently(hookname))
+		append_or_move_hook(hook_head, NULL);
 
 	return hook_head;
 }
@@ -191,11 +277,43 @@ static int pick_next_hook(struct child_process *cp,
 	cp->trace2_hook_name = hook_cb->hook_name;
 	cp->dir = hook_cb->options->dir;
 
+	/* to enable oneliners, let config-specified hooks run in shell.
+	 * config-specified hooks have a name. */
+	cp->use_shell = !!run_me->name;
+
 	/* add command */
-	if (hook_cb->options->absolute_path)
-		strvec_push(&cp->args, absolute_path(run_me->hook_path));
-	else
-		strvec_push(&cp->args, run_me->hook_path);
+	if (run_me->name) {
+		/* ...from config */
+		struct strbuf cmd_key = STRBUF_INIT;
+		char *command = NULL;
+
+		strbuf_addf(&cmd_key, "hook.%s.command", run_me->name);
+		if (git_config_get_string(cmd_key.buf, &command)) {
+			/* TODO test me! */
+			die(_("'hook.%s.command' must be configured "
+			      "or 'hook.%s.event' must be removed; aborting.\n"),
+			    run_me->name, run_me->name);
+		}
+
+		strvec_push(&cp->args, command);
+	} else {
+		/* ...from hookdir. */
+		const char *hook_path = NULL;
+		/*
+		 *
+		 * At this point we are already running, so don't validate
+		 * whether the hook name is known or not.
+		 */
+		hook_path = find_hook_gently(hook_cb->hook_name);
+		if (!hook_path)
+			BUG("hookdir hook in hook list but no hookdir hook present in filesystem");
+
+		if (hook_cb->options->absolute_path)
+			hook_path = absolute_path(hook_path);
+
+		strvec_push(&cp->args, hook_path);
+	}
+
 
 	/*
 	 * add passed-in argv, without expanding - let the user get back
@@ -225,8 +343,11 @@ static int notify_start_failure(struct strbuf *out,
 
 	hook_cb->rc |= 1;
 
-	strbuf_addf(out, _("Couldn't start hook '%s'\n"),
-		    attempted->hook_path);
+	if (attempted->name)
+		strbuf_addf(out, _("Couldn't start hook '%s'\n"),
+		    attempted->name);
+	else
+		strbuf_addstr(out, _("Couldn't start hook from hooks directory\n"));
 
 	return 1;
 }
@@ -320,7 +441,8 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 		BUG("choose only one method to populate stdin");
 
 	/*
-	 * 'git hooks run <hookname>' uses run_found_hooks, so we don't need to
+	 * 'git hooks run <hookname>' uses run_found_hooks, and we want to make
+	 * sure internal callers are using known hooks, so we don't need to
 	 * allow unknown hooknames here.
 	 */
 	hooks = hook_list(hook_name, 0);
diff --git a/hook.h b/hook.h
index ffa96c6e4d..a6263864b9 100644
--- a/hook.h
+++ b/hook.h
@@ -27,8 +27,11 @@ int hook_exists(const char *hookname);
 
 struct hook {
 	struct list_head list;
-	/* The path to the hook */
-	const char *hook_path;
+	/*
+	 * The friendly name of the hook. NULL indicates the hook is from the
+	 * hookdir.
+	 */
+	const char *name;
 
 	/*
 	 * Use this to keep state for your feed_pipe_fn if you are using
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
index 217db848b3..ef2432f53a 100755
--- a/t/t1800-hook.sh
+++ b/t/t1800-hook.sh
@@ -1,13 +1,29 @@
 #!/bin/bash
 
-test_description='git-hook command'
+test_description='git-hook command and config-managed multihooks'
 
 . ./test-lib.sh
 
+setup_hooks () {
+	test_config hook.ghi.event pre-commit --add
+	test_config hook.ghi.command "/path/ghi" --add
+	test_config_global hook.def.event pre-commit --add
+	test_config_global hook.def.command "/path/def" --add
+}
+
+setup_hookdir () {
+	mkdir .git/hooks
+	write_script .git/hooks/pre-commit <<-EOF
+	echo \"Legacy Hook\"
+	EOF
+	test_when_finished rm -rf .git/hooks
+}
+
 test_expect_success 'git hook usage' '
 	test_expect_code 129 git hook &&
 	test_expect_code 129 git hook run &&
 	test_expect_code 129 git hook run -h &&
+	test_expect_code 129 git hook list -h &&
 	test_expect_code 129 git hook run --unknown 2>err &&
 	grep "unknown option" err
 '
@@ -153,4 +169,127 @@ test_expect_success 'stdin to hooks' '
 	test_cmp expect actual
 '
 
+test_expect_success 'git hook list orders by config order' '
+	setup_hooks &&
+
+	cat >expected <<-EOF &&
+	def
+	ghi
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list reorders on duplicate event declarations' '
+	setup_hooks &&
+
+	# 'def' is usually configured globally; move it to the end by
+	# configuring it locally.
+	test_config hook.def.event "pre-commit" --add &&
+
+	cat >expected <<-EOF &&
+	ghi
+	def
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list shows hooks from the hookdir' '
+	setup_hookdir &&
+
+	cat >expected <<-EOF &&
+	hook from hookdir
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'inline hook definitions execute oneliners' '
+	test_config hook.oneliner.event "pre-commit" &&
+	test_config hook.oneliner.command "echo \"Hello World\"" &&
+
+	echo "Hello World" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'inline hook definitions resolve paths' '
+	write_script sample-hook.sh <<-EOF &&
+	echo \"Sample Hook\"
+	EOF
+
+	test_when_finished "rm sample-hook.sh" &&
+
+	test_config hook.sample-hook.event pre-commit &&
+	test_config hook.sample-hook.command "\"$(pwd)/sample-hook.sh\"" &&
+
+	echo \"Sample Hook\" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'hookdir hook included in git hook run' '
+	setup_hookdir &&
+
+	echo \"Legacy Hook\" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'stdin to multiple hooks' '
+	test_config hook.stdin-a.event "test-hook" --add &&
+	test_config hook.stdin-a.command "xargs -P1 -I% echo a%" --add &&
+	test_config hook.stdin-b.event "test-hook" --add &&
+	test_config hook.stdin-b.command "xargs -P1 -I% echo b%" --add &&
+
+	cat >input <<-EOF &&
+	1
+	2
+	3
+	EOF
+
+	cat >expected <<-EOF &&
+	a1
+	a2
+	a3
+	b1
+	b2
+	b3
+	EOF
+
+	git hook run --to-stdin=input test-hook 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'multiple hooks in series' '
+	test_config hook.series-1.event "test-hook" &&
+	test_config hook.series-1.command "echo 1" --add &&
+	test_config hook.series-2.event "test-hook" &&
+	test_config hook.series-2.command "echo 2" --add &&
+	mkdir .git/hooks &&
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo 3
+	EOF
+
+	cat >expected <<-EOF &&
+	1
+	2
+	3
+	EOF
+
+	git hook run -j1 test-hook 2>actual &&
+	test_cmp expected actual &&
+
+	rm -rf .git/hooks
+'
 test_done
-- 
2.32.0.605.g8dce9f2422-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v2 6/6] hook: allow out-of-repo 'git hook' invocations
  2021-08-12  0:42             ` [PATCH v2 0/6] config-based hooks restarted Emily Shaffer
                                 ` (4 preceding siblings ...)
  2021-08-12  0:42               ` [PATCH v2 5/6] hook: include hooks from the config Emily Shaffer
@ 2021-08-12  0:42               ` Emily Shaffer
  2021-08-12  4:47               ` [PATCH v2 0/6] config-based hooks restarted Junio C Hamano
  2021-08-19  3:34               ` [PATCH v3 " Emily Shaffer
  7 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-12  0:42 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Since hooks can now be supplied via the config, and a config can be
present without a gitdir via the global and system configs, we can start
to allow 'git hook run' to occur without a gitdir. This enables us to do
things like run sendemail-validate hooks when running 'git send-email'
from a nongit directory.

It still doesn't make sense to look for hooks in the hookdir in nongit
repos, though, as there is no hookdir.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

Notes:
    For hookdir hooks, do we want to run them in nongit dir when core.hooksPath
    is set? For example, if someone set core.hooksPath in their global config and
    then ran 'git hook run sendemail-validate' in a nongit dir?

 git.c           |  2 +-
 hook.c          |  2 +-
 t/t1800-hook.sh | 20 +++++++++++++++-----
 3 files changed, 17 insertions(+), 7 deletions(-)

diff --git a/git.c b/git.c
index 540909c391..39988ee3b0 100644
--- a/git.c
+++ b/git.c
@@ -538,7 +538,7 @@ static struct cmd_struct commands[] = {
 	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
 	{ "hash-object", cmd_hash_object },
 	{ "help", cmd_help },
-	{ "hook", cmd_hook, RUN_SETUP },
+	{ "hook", cmd_hook, RUN_SETUP_GENTLY },
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "init", cmd_init_db },
 	{ "init-db", cmd_init_db },
diff --git a/hook.c b/hook.c
index 51ada266bc..87d57f4118 100644
--- a/hook.c
+++ b/hook.c
@@ -215,7 +215,7 @@ struct list_head* hook_list(const char *hookname, int allow_unknown)
 
 	/* Add the hook from the hookdir. The placeholder makes it easier to
 	 * allocate work in pick_next_hook. */
-	if (find_hook_gently(hookname))
+	if (have_git_dir() && find_hook_gently(hookname))
 		append_or_move_hook(hook_head, NULL);
 
 	return hook_head;
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
index ef2432f53a..a7e45c0d16 100755
--- a/t/t1800-hook.sh
+++ b/t/t1800-hook.sh
@@ -118,15 +118,25 @@ test_expect_success 'git hook run -- pass arguments' '
 	test_cmp expect actual
 '
 
-test_expect_success 'git hook run -- out-of-repo runs excluded' '
-	write_script .git/hooks/test-hook <<-EOF &&
-	echo Test hook
-	EOF
+test_expect_success 'git hook run: out-of-repo runs execute global hooks' '
+	test_config_global hook.global-hook.event test-hook --add &&
+	test_config_global hook.global-hook.command "echo no repo no problems" --add &&
+
+	echo "global-hook" >expect &&
+	nongit git hook list test-hook >actual &&
+	test_cmp expect actual &&
+
+	echo "no repo no problems" >expect &&
 
-	nongit test_must_fail git hook run test-hook
+	nongit git hook run test-hook 2>actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo Test hook
+	EOF
+
 	mkdir my-hooks &&
 	write_script my-hooks/test-hook <<-\EOF &&
 	echo Hook ran $1 >>actual
-- 
2.32.0.605.g8dce9f2422-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 0/6] config-based hooks restarted
  2021-08-12  0:42             ` [PATCH v2 0/6] config-based hooks restarted Emily Shaffer
                                 ` (5 preceding siblings ...)
  2021-08-12  0:42               ` [PATCH v2 6/6] hook: allow out-of-repo 'git hook' invocations Emily Shaffer
@ 2021-08-12  4:47               ` Junio C Hamano
  2021-08-12  5:02                 ` Junio C Hamano
  2021-08-19  3:34               ` [PATCH v3 " Emily Shaffer
  7 siblings, 1 reply; 479+ messages in thread
From: Junio C Hamano @ 2021-08-12  4:47 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: git, Jeff King, Taylor Blau, Felipe Contreras, Eric Sunshine,
	brian m. carlson, Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Emily Shaffer <emilyshaffer@google.com> writes:

> This is the config-based hooks topic rebased onto v4 of Ævar's
> branch[1].

I have [1] in my tree, but these patches do not seem to apply
cleanly; I see a failure in "git hooks list" step.

Should I perhaps try merging [1] to one of the v2.33-rc and then
queue these on top?

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 0/6] config-based hooks restarted
  2021-08-12  4:47               ` [PATCH v2 0/6] config-based hooks restarted Junio C Hamano
@ 2021-08-12  5:02                 ` Junio C Hamano
  2021-08-16 22:31                   ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Junio C Hamano @ 2021-08-12  5:02 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: git, Jeff King, Taylor Blau, Felipe Contreras, Eric Sunshine,
	brian m. carlson, Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Junio C Hamano <gitster@pobox.com> writes:

> Emily Shaffer <emilyshaffer@google.com> writes:
>
>> This is the config-based hooks topic rebased onto v4 of Ævar's
>> branch[1].
>
> I have [1] in my tree, but these patches do not seem to apply
> cleanly; I see a failure in "git hooks list" step.
>
> Should I perhaps try merging [1] to one of the v2.33-rc and then
> queue these on top?

Ah, I figured it out.  Your 3/6 seems to be stale wrt 4787177b
(hook: support passing stdin to hooks, 2021-08-03) that came from
https://lore.kernel.org/git/patch-v4-22.36-639e59e9ed0-20210803T191505Z-avarab@gmail.com/

That patch did this to the early part of builtin/hook.c:

        diff --git a/builtin/hook.c b/builtin/hook.c
        index f33db9953c..27dce6a2f0 100644
        --- a/builtin/hook.c
        +++ b/builtin/hook.c
        @@ -7,7 +7,7 @@
         #include "strvec.h"

         #define BUILTIN_HOOK_RUN_USAGE \
        -	N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")
        +	N_("git hook run [--ignore-missing] [--to-stdin=<path>] <hook-name> [-- <hook-args>]")

         static const char * const builtin_hook_usage[] = {
                BUILTIN_HOOK_RUN_USAGE,


but your [v2 3/6] here expects something entirely different.

        diff --git a/builtin/hook.c b/builtin/hook.c
        index 12c9126032..c36b05376c 100644
        --- a/builtin/hook.c
        +++ b/builtin/hook.c
        @@ -8,8 +8,11 @@

         #define BUILTIN_HOOK_RUN_USAGE \
                N_("git hook run [--to-stdin=<path>] <hook-name> [-- <hook-args>]")
        +#define BUILTIN_HOOK_LIST_USAGE \
        +	N_("git hook list <hook-name>")

I've wiggled the patch in, as there wasn't any other funny
inconsistency like this one, but please double check the result
after I push it out perhaps tomorrow morning.

Thanks.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 1/6] hook: run a list of hooks instead
  2021-08-12  0:42               ` [PATCH v2 1/6] hook: run a list of hooks instead Emily Shaffer
@ 2021-08-12 17:25                 ` Junio C Hamano
  2021-08-16 23:38                   ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Junio C Hamano @ 2021-08-12 17:25 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> To prepare for multihook support, teach hook.[hc] to take a list of
> hooks at run_hooks and run_found_hooks. Right now the list is always one
> entry, but in the future we will allow users to supply more than one
> executable for a single hook event.
>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  builtin/hook.c |  14 ++++---
>  hook.c         | 103 +++++++++++++++++++++++++++++++++++--------------
>  hook.h         |  16 +++++++-
>  3 files changed, 96 insertions(+), 37 deletions(-)
>
> diff --git a/builtin/hook.c b/builtin/hook.c
> index 5eb7cf73a4..4d39c9e75e 100644
> --- a/builtin/hook.c
> +++ b/builtin/hook.c
> @@ -25,7 +25,8 @@ static int run(int argc, const char **argv, const char *prefix)
>  	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
>  	int ignore_missing = 0;
>  	const char *hook_name;
> -	const char *hook_path;
> +	struct list_head *hooks;
> +

Natural.  We used to use the path to the hook because we were
expecting only one. We now use the name to find a list of hooks.

All the caller sees is just list_head without any direct visibility
into it, which feels like a great abstraction.  Presumably everything
will go through the API functions taking this opaque "list of hooks"
thing (or "the first one in the list" if the API function does not
iterate over it, perhaps?).

>  	struct option run_options[] = {
>  		OPT_BOOL(0, "ignore-missing", &ignore_missing,
>  			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
> @@ -58,15 +59,16 @@ static int run(int argc, const char **argv, const char *prefix)
>  	git_config(git_default_config, NULL);
>  
>  	hook_name = argv[0];
> -	if (ignore_missing)
> -		return run_hooks_oneshot(hook_name, &opt);
> -	hook_path = find_hook(hook_name);
> -	if (!hook_path) {
> +	hooks = hook_list(hook_name);

OK.

> +	if (list_empty(hooks)) {
> +		/* ... act like run_hooks_oneshot() under --ignore-missing */
> +		if (ignore_missing)
> +			return 0;

OK.

>  		error("cannot find a hook named %s", hook_name);
>  		return 1;
>  	}

> diff --git a/hook.c b/hook.c
> index ee20b2e365..80e150548c 100644
> --- a/hook.c
> +++ b/hook.c
> @@ -4,6 +4,28 @@
>  #include "hook-list.h"
>  #include "config.h"
>  
> +static void free_hook(struct hook *ptr)
> +{
> +	if (ptr) {
> +		free(ptr->feed_pipe_cb_data);
> +	}

Lose the extra {}, as we do not do more than the above free() even
at the end of the series?

> +	free(ptr);
> +}
> +
> +static void remove_hook(struct list_head *to_remove)
> +{
> +	struct hook *hook_to_remove = list_entry(to_remove, struct hook, list);
> +	list_del(to_remove);
> +	free_hook(hook_to_remove);
> +}
> +
> +void clear_hook_list(struct list_head *head)
> +{
> +	struct list_head *pos, *tmp;
> +	list_for_each_safe(pos, tmp, head)
> +		remove_hook(pos);
> +}

OK.

> +struct list_head* hook_list(const char* hookname)

Shift both of the asterisks; our asterisks do not stick to the type
but to the identifier.

> +{
> +	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
> +
> +	INIT_LIST_HEAD(hook_head);
> +
> +	if (!hookname)
> +		return NULL;

Checking for invalid hookname first would avoid leaking hook_head,
no?  The caller of hook_list() we saw earlier immediately calls
list_empty() which will segfault.  The caller need to be tightened,
but I wonder if it would be a programmer's error to pass a NULL
hookname to this function.  If so, this can simply be a BUG() and
the earlier allocation and initialization of hook_head can stay
where they are.  Otherwise, the caller should see if this returns a
NULL.

> +	if (have_git_dir()) {
> +		const char *hook_path = find_hook(hookname);
> +
> +		/* Add the hook from the hookdir */
> +		if (hook_path) {
> +			struct hook *to_add = xmalloc(sizeof(*to_add));
> +			to_add->hook_path = hook_path;
> +			to_add->feed_pipe_cb_data = NULL;
> +			list_add_tail(&to_add->list, hook_head);
> +		}
> +	}

Calling this function to grab a list of hooks when we are not in any
repository is not an error but just we get "there is nothing to
run".  Does the design give us a more useful behaviour, compared to
alternatives like "you have to be in a repository or calling this
function is an error"?

Not an objection wrapped in a rhetorical question, but a genuine
question.  "It would help this and that usecase" would be an ideal
answer, "We could do either way, but we happened to have written the
code this way first, and at the end of the series we did not see any
practical downsides" would also be a great answer.

> +	return hook_head;
> +}
> +

> +/*
> + * Provides a linked list of 'struct hook' detailing commands which should run
> + * in response to the 'hookname' event, in execution order.
> + */
> +struct list_head* hook_list(const char *hookname);

struct list_head *hook_list(const char *hookname);

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 2/6] hook: allow parallel hook execution
  2021-08-12  0:42               ` [PATCH v2 2/6] hook: allow parallel hook execution Emily Shaffer
@ 2021-08-12 17:51                 ` Junio C Hamano
  2021-08-16 23:59                   ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Junio C Hamano @ 2021-08-12 17:51 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git, Ævar Arnfjörð Bjarmason

Emily Shaffer <emilyshaffer@google.com> writes:

> In many cases, there's no reason not to allow hooks to execute in
> parallel. run_processes_parallel() is well-suited - it's a task queue
> that runs its housekeeping in series, which means users don't
> need to worry about thread safety on their callback data. True
> multithreaded execution with the async_* functions isn't necessary here.
> Synchronous hook execution can be achieved by only allowing 1 job to run
> at a time.
>
> Teach run_hooks() to use that function for simple hooks which don't
> require stdin or capture of stderr.

Since run_hooks() has been using run_processes_parallel_tr2()
already in the ab/config-based-hooks-base topic, the above
description puzzled me quite a bit.  I think you meant to say with
this step you update the callers to pass either .jobs=1 or .jobs=0
in the hooks_opt, so that hooks _can_ run in parallel (as opposed
to using hardcoded jobs=1 in run_hooks() when calling the underlying
run_processes_parallel_tr2() function).

I do not think SYNC and ASYNC are great adjectives for what you are
doing in this step.  Wouldn't ASYNC mean "the caller tells the hooks
to run, continues without waiting for them to finish"?  In order to
let more than one hooks to run at the same time, the caller has to
continue without waiting for the first one it started so that it can
start the second one before the first one finishes (otherwise these
two hooks will not run at the same time), but ASYNC implys a lot
more than that.  After starting all of these hooks, the caller may
continue doing things that are not even related to these spawned
hooks if a hook is run asynchronously.

But I do not think that is what is happening here.  After all,
run_processes_parallel_tr2() will not return until all of the
subprocesses are culled.

Perhaps the distinction you are trying to express is easier to
convey to readers if you instead used the verb TO RUN with the
adverb SERIALLY vs IN PARALLEL (as opposed to SYNCHRONOUSLY vs
ASYNCHRONOUSLY)?

> diff --git a/builtin/hook.c b/builtin/hook.c
> index 4d39c9e75e..12c9126032 100644
> --- a/builtin/hook.c
> +++ b/builtin/hook.c
> @@ -22,7 +22,7 @@ static const char * const builtin_hook_run_usage[] = {
>  static int run(int argc, const char **argv, const char *prefix)
>  {
>  	int i;
> -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
> +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
>  	int ignore_missing = 0;
>  	const char *hook_name;
>  	struct list_head *hooks;
> @@ -32,6 +32,8 @@ static int run(int argc, const char **argv, const char *prefix)
>  			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
>  		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
>  			   N_("file to read into hooks' stdin")),
> +		OPT_INTEGER('j', "jobs", &opt.jobs,
> +			    N_("run up to <n> hooks simultaneously")),
>  		OPT_END(),
>  	};
>  	int ret;

OK, so by default "git hook" runs 1 at a time but we can instruct to
run the optimal number of jobs via "git hook -j0" etc.

> diff --git a/hook.c b/hook.c
> index 80e150548c..37f682c6d8 100644
> --- a/hook.c
> +++ b/hook.c
> @@ -228,6 +228,28 @@ static int notify_hook_finished(int result,
>  	return 0;
>  }
>  
> +/*
> + * Determines how many jobs to use after we know we want to parallelize. First
> + * priority is the config 'hook.jobs' and second priority is the number of CPUs.
> + */
> +static int configured_hook_jobs(void)
> +{
> +	/*
> +	 * The config and the CPU count probably won't change during the process
> +	 * lifetime, so cache the result in case we invoke multiple hooks during
> +	 * one process.
> +	 */
> +	static int jobs = 0;
> +	if (jobs)
> +		return jobs;
> +
> +	if (git_config_get_int("hook.jobs", &jobs))
> +		/* if the config isn't set, fall back to CPU count. */
> +		jobs = online_cpus();
> +
> +	return jobs;
> +}

Not a suggestion to make this improvement as a part of this series,
but there already are more than several codepaths doing essentially
the same thing as above, taking their own configuration or
environment variables and falling back to online_cpus().

It may be worth introducing a shared helper function so that this
can become

	static int configured_hook_jobs(void)
	{
		static int jobs; /* no need for "= 0" */

		if (!jobs)
			jobs = default_parallelism("hook.jobs", NULL);
		return jobs;
	}

where the new helper takes the configuration and environment
variable names.

Thanks.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 3/6] hook: introduce "git hook list"
  2021-08-12  0:42               ` [PATCH v2 3/6] hook: introduce "git hook list" Emily Shaffer
@ 2021-08-12 18:59                 ` Junio C Hamano
  2021-08-17  0:36                   ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Junio C Hamano @ 2021-08-12 18:59 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> +static int list(int argc, const char **argv, const char *prefix)
> +{
> +	struct list_head *head, *pos;
> +	const char *hookname = NULL;
> +	struct strbuf hookdir_annotation = STRBUF_INIT;
> +
> +	struct option list_options[] = {
> +		OPT_END(),
> +	};
> +
> +	argc = parse_options(argc, argv, prefix, list_options,
> +			     builtin_hook_list_usage, 0);
> +
> +	if (argc < 1)
> +		usage_msg_opt(_("You must specify a hook event name to list."),
> +			      builtin_hook_list_usage, list_options);
> +
> +	hookname = argv[0];
> +
> +	head = hook_list(hookname);
> +
> +	if (list_empty(head)) {

The same "can't hook_list() signal an error by returning NULL?"
comment applies here.

	head = hook_list(hookname);
	if (!head)
		die(_("no such hook '%s'"), hookname);

or something?

> +		printf(_("no commands configured for hook '%s'\n"),
> +		       hookname);
> +		return 0;

If it is a normally expected state that there is no hook for the
given name, signalling success by returning 0 here may be sensible,
but then the message should at least go to the standard error stream
to leave the standard output empty, so that a caller can reasonably
do something like

	for path in $(git hooks list "$1")
	do
		ls -l "$path"
	done

If we really want to show such a message, perhaps

	if (list_empty(head)) {
        	if (!quiet)
			warning(_("no commands configured"));
		return 0;
	}

The normal display just shows the path without saying "command %s
will run for hook %s"; the warning probably should do the same.

Having said that, if it truly is a normal and expected state that no
hook is defined for the given name, I actually think there should be
no message.

> +	}
> +
> +	list_for_each(pos, head) {
> +		struct hook *item = list_entry(pos, struct hook, list);
> +		item = list_entry(pos, struct hook, list);
> +		if (item)
> +			printf("%s\n", item->hook_path);
> +	}

> diff --git a/hook.c b/hook.c
> index 37f682c6d8..2714b63473 100644
> --- a/hook.c
> +++ b/hook.c
> @@ -96,22 +96,20 @@ int hook_exists(const char *name)
>  struct list_head* hook_list(const char* hookname)
>  {
>  	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
> +	const char *hook_path = find_hook(hookname);
> +
>  
>  	INIT_LIST_HEAD(hook_head);
>  
>  	if (!hookname)
>  		return NULL;
>  
> -	if (have_git_dir()) {
> -		const char *hook_path = find_hook(hookname);
> -
> -		/* Add the hook from the hookdir */
> -		if (hook_path) {
> -			struct hook *to_add = xmalloc(sizeof(*to_add));
> -			to_add->hook_path = hook_path;
> -			to_add->feed_pipe_cb_data = NULL;
> -			list_add_tail(&to_add->list, hook_head);
> -		}
> +	/* Add the hook from the hookdir */
> +	if (hook_path) {
> +		struct hook *to_add = xmalloc(sizeof(*to_add));
> +		to_add->hook_path = hook_path;
> +		to_add->feed_pipe_cb_data = NULL;
> +		list_add_tail(&to_add->list, hook_head);
>  	}

I do not think this belongs to the step to add "list" command.  The
log message does not explain or justify why have-git-dir goes away,
either.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 4/6] hook: allow running non-native hooks
  2021-08-12  0:42               ` [PATCH v2 4/6] hook: allow running non-native hooks Emily Shaffer
@ 2021-08-12 19:08                 ` Junio C Hamano
  2021-08-18 20:51                   ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Junio C Hamano @ 2021-08-12 19:08 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> diff --git a/builtin/hook.c b/builtin/hook.c
> index c36b05376c..3aa65dd791 100644
> --- a/builtin/hook.c
> +++ b/builtin/hook.c
> @@ -46,7 +46,7 @@ static int list(int argc, const char **argv, const char *prefix)
>  
>  	hookname = argv[0];
>  
> -	head = hook_list(hookname);
> +	head = hook_list(hookname, 1);
>  
>  	if (list_empty(head)) {
>  		printf(_("no commands configured for hook '%s'\n"),
> @@ -108,7 +108,7 @@ static int run(int argc, const char **argv, const char *prefix)
>  	git_config(git_default_config, NULL);
>  
>  	hook_name = argv[0];
> -	hooks = hook_list(hook_name);
> +	hooks = hook_list(hook_name, 1);
>  	if (list_empty(hooks)) {
>  		/* ... act like run_hooks_oneshot() under --ignore-missing */
>  		if (ignore_missing)

This is minor, as I expect that the callers of hook_list() will
always confined in builtin/hook.c, but it probably is easier to read
if you gave two functions, just like you have the pair of helpers
find_hook() and find_hook_gently(), as the literal "1" forces the
readers to remember if that means "die if not found", or "ok if it
is a bogus name".

In addition, it may make more sense to keep hook_list() signal
failure by returning NULL and leave the dying to the caller.
In-code callers (as opposed to "git hook run" that can throw any
random string that came from the user at the API) will never throw a
bogus name unless there is a bug, and they'll have to check for an
error return from hook_list() anyway and the error message they
would give may have to be different from the one that is given
against a hook name randomly thrown at us by the user.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 5/6] hook: include hooks from the config
  2021-08-12  0:42               ` [PATCH v2 5/6] hook: include hooks from the config Emily Shaffer
@ 2021-08-12 20:48                 ` Junio C Hamano
  2021-08-19  0:09                   ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Junio C Hamano @ 2021-08-12 20:48 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> Teach the hook.[hc] library to parse configs to populare the list of
> hooks to run for a given event.
>
> Multiple commands can be specified for a given hook by providing
> multiple "hook.<friendly-name>.command = <path-to-hook>" and
> "hook.<friendly-name>.event = <hook-event>" lines. Hooks will be run in
> config order of the "hook.<name>.event" lines.
>
> For example:
>
>   $ git config --list | grep ^hook
>   hook.bar.command=~/bar.sh
>   hook.bar.event=pre-commit

Your answer might be "read the design doc", but it is unclear to me
why "bar" (friendly-name) is needed in this picture at all.  Is it
because you may want to fire more than one command for pre-commit
event?  IOW,

	[hook "bar"]
		command = bar1.sh
		command = bar2.sh
		event = pre-commit

is easier to manage with an extra level of redirection?  I doubt it
as 

	[hook "pre-commit"]
		command = bar1.sh
		command = bar2.sh

would be equally expressive and shorter.  Or would it help use case
for multiple "friendly-name" to refer to the same "event", e.g.

	[hook "xyzzy"]
		event = pre-commit
		command = xyzzy1

	[hook "frotz"]
		event = pre-commit
                command = frotz1
                command = frotz2

or something?  I am not sure if this gives us useful extra
flexibility, and if so, the extra flexibility helps us more than it
confuses us.

And moving the "event" to the second level in the configuration
hierarchy, getting rid of "friendly-name" from the design, would not
make this example unworkable, either:

>   $ git hook run
>   # Runs ~/bar.sh
>   # Runs .git/hooks/pre-commit

Again, this is not an objection wrapped in a rhetorical question.
It just is that I do not see how the extra level of redirection
helps us.

> diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
> index 96d3d6572c..a97b980cca 100644
> --- a/Documentation/config/hook.txt
> +++ b/Documentation/config/hook.txt
> @@ -1,3 +1,8 @@
> +hook.<command>.command::
> +	A command to execute during the <command> hook event. This can be an
> +	executable on your device, a oneliner for your shell, or the name of a
> +	hookcmd. See linkgit:git-hook[1].

Please make sure you use the terminology consistently.  If the
second level is "friendly name", hook.<name>.command should be
described, instead of hook.<command>.command.

Also, to help those who are familiar with the current Git from their
use in the past 10 years or so, giving an example name from the
current system may help, e.g. when describing hook.<name>.event,
you may want to say the values are things like "pre-commit",
"receive", etc.

> +This command parses the default configuration files for pairs of configs like
> +so:
> +
> +  [hook "linter"]
> +    event = pre-commit
> +    command = ~/bin/linter --c

The above addition of .command should also have hook.<name>.event
next to it, no?

> +Conmmands are run in the order Git encounters their associated

"Conmmands -> Commands", I would think.

> +`hook.<name>.event` configs during the configuration parse (see
> +linkgit:git-config[1]).

Here you use <name>, which should be matched by the description in
the first hunk of the patch to this file.

> +In general, when instructions suggest adding a script to
> +`.git/hooks/<hook-event>`, you can specify it in the config instead by running
> +`git config --add hook.<some-name>.command <path-to-script> && git config --add
> +hook.<some-name>.event <hook-event>` - this way you can share the script between
> +multiple repos. That is, `cp ~/my-script.sh ~/project/.git/hooks/pre-commit`
> +would become `git config --add hook.my-script.command ~/my-script.sh && git
> +config --add hook.my-script.event pre-commit`.

One repository may use a friendly name "xyzzy" while the other may
use "frotz" to group the hooks that trigger upon "pre-commit" event,
but unless one of the repositories change the friendly name to
match, they cannot share these configurations, no?  It seems that an
extra level of indirection is hindering sharing, rather than
helping.

> diff --git a/builtin/hook.c b/builtin/hook.c
> index 3aa65dd791..ea49dc4ef6 100644
> --- a/builtin/hook.c
> +++ b/builtin/hook.c
> @@ -49,7 +49,7 @@ static int list(int argc, const char **argv, const char *prefix)
>  	head = hook_list(hookname, 1);
>  
>  	if (list_empty(head)) {
> -		printf(_("no commands configured for hook '%s'\n"),
> +		printf(_("no hooks configured for event '%s'\n"),
>  		       hookname);
> ...
> @@ -58,7 +58,8 @@ static int list(int argc, const char **argv, const char *prefix)
>  		struct hook *item = list_entry(pos, struct hook, list);
>  		item = list_entry(pos, struct hook, list);
>  		if (item)
> -			printf("%s\n", item->hook_path);
> +			printf("%s\n", item->name ? item->name
> +						  : _("hook from hookdir"));
>  	}

I won't comment on this part as my comments on earlier patches would
probably have butchered the preimage already for this change to
survive intact ;-)

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 0/6] config-based hooks restarted
  2021-08-12  5:02                 ` Junio C Hamano
@ 2021-08-16 22:31                   ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-16 22:31 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Jeff King, Taylor Blau, Felipe Contreras, Eric Sunshine,
	brian m. carlson, Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

On Wed, Aug 11, 2021 at 10:02:53PM -0700, Junio C Hamano wrote:
> 
> Junio C Hamano <gitster@pobox.com> writes:
> 
> > Emily Shaffer <emilyshaffer@google.com> writes:
> >
> >> This is the config-based hooks topic rebased onto v4 of Ævar's
> >> branch[1].
> >
> > I have [1] in my tree, but these patches do not seem to apply
> > cleanly; I see a failure in "git hooks list" step.
> >
> > Should I perhaps try merging [1] to one of the v2.33-rc and then
> > queue these on top?
> 
> Ah, I figured it out.  Your 3/6 seems to be stale wrt 4787177b
> (hook: support passing stdin to hooks, 2021-08-03) that came from
> https://lore.kernel.org/git/patch-v4-22.36-639e59e9ed0-20210803T191505Z-avarab@gmail.com/

Ah, I am sorry!

> That patch did this to the early part of builtin/hook.c:
> 
>         diff --git a/builtin/hook.c b/builtin/hook.c
>         index f33db9953c..27dce6a2f0 100644
>         --- a/builtin/hook.c
>         +++ b/builtin/hook.c
>         @@ -7,7 +7,7 @@
>          #include "strvec.h"
> 
>          #define BUILTIN_HOOK_RUN_USAGE \
>         -	N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")
>         +	N_("git hook run [--ignore-missing] [--to-stdin=<path>] <hook-name> [-- <hook-args>]")
> 
>          static const char * const builtin_hook_usage[] = {
>                 BUILTIN_HOOK_RUN_USAGE,
> 
> 
> but your [v2 3/6] here expects something entirely different.
> 
>         diff --git a/builtin/hook.c b/builtin/hook.c
>         index 12c9126032..c36b05376c 100644
>         --- a/builtin/hook.c
>         +++ b/builtin/hook.c
>         @@ -8,8 +8,11 @@
> 
>          #define BUILTIN_HOOK_RUN_USAGE \
>                 N_("git hook run [--to-stdin=<path>] <hook-name> [-- <hook-args>]")
>         +#define BUILTIN_HOOK_LIST_USAGE \
>         +	N_("git hook list <hook-name>")
> 
> I've wiggled the patch in, as there wasn't any other funny
> inconsistency like this one, but please double check the result
> after I push it out perhaps tomorrow morning.

I'll take a look and update local accordingly. Sorry for the trouble.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 1/6] hook: run a list of hooks instead
  2021-08-12 17:25                 ` Junio C Hamano
@ 2021-08-16 23:38                   ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-16 23:38 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Thu, Aug 12, 2021 at 10:25:03AM -0700, Junio C Hamano wrote:
> 
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > To prepare for multihook support, teach hook.[hc] to take a list of
> > hooks at run_hooks and run_found_hooks. Right now the list is always one
> > entry, but in the future we will allow users to supply more than one
> > executable for a single hook event.
> >
> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > ---
> >  builtin/hook.c |  14 ++++---
> >  hook.c         | 103 +++++++++++++++++++++++++++++++++++--------------
> >  hook.h         |  16 +++++++-
> >  3 files changed, 96 insertions(+), 37 deletions(-)
> >
> > diff --git a/builtin/hook.c b/builtin/hook.c
> > index 5eb7cf73a4..4d39c9e75e 100644
> > --- a/builtin/hook.c
> > +++ b/builtin/hook.c
> > @@ -25,7 +25,8 @@ static int run(int argc, const char **argv, const char *prefix)
> >  	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
> >  	int ignore_missing = 0;
> >  	const char *hook_name;
> > -	const char *hook_path;
> > +	struct list_head *hooks;
> > +
> 
> Natural.  We used to use the path to the hook because we were
> expecting only one. We now use the name to find a list of hooks.
> 
> All the caller sees is just list_head without any direct visibility
> into it, which feels like a great abstraction.  Presumably everything
> will go through the API functions taking this opaque "list of hooks"
> thing (or "the first one in the list" if the API function does not
> iterate over it, perhaps?).

Hum, I guess that in a later patch builtin/hook.c does learn to take
apart the list_head into a 'struct hook' to print the output of 'git
hook list'. I haven't read your review of that patch yet though.

> > diff --git a/hook.c b/hook.c
> > index ee20b2e365..80e150548c 100644
> > --- a/hook.c
> > +++ b/hook.c
> > @@ -4,6 +4,28 @@
> >  #include "hook-list.h"
> >  #include "config.h"
> >  
> > +static void free_hook(struct hook *ptr)
> > +{
> > +	if (ptr) {
> > +		free(ptr->feed_pipe_cb_data);
> > +	}
> 
> Lose the extra {}, as we do not do more than the above free() even
> at the end of the series?

ACK

> 
> > +struct list_head* hook_list(const char* hookname)
> 
> Shift both of the asterisks; our asterisks do not stick to the type
> but to the identifier.

ACK

> 
> > +{
> > +	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
> > +
> > +	INIT_LIST_HEAD(hook_head);
> > +
> > +	if (!hookname)
> > +		return NULL;
> 
> Checking for invalid hookname first would avoid leaking hook_head,
> no?  The caller of hook_list() we saw earlier immediately calls
> list_empty() which will segfault.  The caller need to be tightened,
> but I wonder if it would be a programmer's error to pass a NULL
> hookname to this function.  If so, this can simply be a BUG() and
> the earlier allocation and initialization of hook_head can stay
> where they are.  Otherwise, the caller should see if this returns a
> NULL.

Ah, good point. I think this makes sense to BUG().

> 
> > +	if (have_git_dir()) {
> > +		const char *hook_path = find_hook(hookname);
> > +
> > +		/* Add the hook from the hookdir */
> > +		if (hook_path) {
> > +			struct hook *to_add = xmalloc(sizeof(*to_add));
> > +			to_add->hook_path = hook_path;
> > +			to_add->feed_pipe_cb_data = NULL;
> > +			list_add_tail(&to_add->list, hook_head);
> > +		}
> > +	}
> 
> Calling this function to grab a list of hooks when we are not in any
> repository is not an error but just we get "there is nothing to
> run".  Does the design give us a more useful behaviour, compared to
> alternatives like "you have to be in a repository or calling this
> function is an error"?

Later we enable calling hook_list without a gitdir, in patch 6
(https://lore.kernel.org/git/20210812004258.74318-7-emilyshaffer%40google.com).
So maybe the behavior as it is now is premature?

But leaving the hook list empty means we can behave gracefully if
anybody is calling a hook without necessarily being sure they have
gitdir. I would need to audit callsites to check if they are checking
whether they have one or not. I think in most cases they probably are
checking that.

> 
> Not an objection wrapped in a rhetorical question, but a genuine
> question.  "It would help this and that usecase" would be an ideal
> answer, "We could do either way, but we happened to have written the
> code this way first, and at the end of the series we did not see any
> practical downsides" would also be a great answer.

The main use case, today, for letting this work nicely even outside of
a gitdir, is sendemail-validate hook. But mostly, I was thinking that
when we're freed from dependence on .git/hooks/, there's no reason to
disallow out-of-repo hooks in case someone wants to add a new one in the
future - for example to 'git maintenance' daemon runs.

> 
> > +	return hook_head;
> > +}
> > +
> 
> > +/*
> > + * Provides a linked list of 'struct hook' detailing commands which should run
> > + * in response to the 'hookname' event, in execution order.
> > + */
> > +struct list_head* hook_list(const char *hookname);
> 
> struct list_head *hook_list(const char *hookname);
ACK

Thanks for the thoughts.
 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 2/6] hook: allow parallel hook execution
  2021-08-12 17:51                 ` Junio C Hamano
@ 2021-08-16 23:59                   ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-16 23:59 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Ævar Arnfjörð Bjarmason

On Thu, Aug 12, 2021 at 10:51:01AM -0700, Junio C Hamano wrote:
> 
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > In many cases, there's no reason not to allow hooks to execute in
> > parallel. run_processes_parallel() is well-suited - it's a task queue
> > that runs its housekeeping in series, which means users don't
> > need to worry about thread safety on their callback data. True
> > multithreaded execution with the async_* functions isn't necessary here.
> > Synchronous hook execution can be achieved by only allowing 1 job to run
> > at a time.
> >
> > Teach run_hooks() to use that function for simple hooks which don't
> > require stdin or capture of stderr.
> 
> Since run_hooks() has been using run_processes_parallel_tr2()
> already in the ab/config-based-hooks-base topic, the above
> description puzzled me quite a bit.  I think you meant to say with
> this step you update the callers to pass either .jobs=1 or .jobs=0
> in the hooks_opt, so that hooks _can_ run in parallel (as opposed
> to using hardcoded jobs=1 in run_hooks() when calling the underlying
> run_processes_parallel_tr2() function).

I think you are right, and this commit message is from a much older
version of the series. Ok; I will reword it. I think it can be much
simpler.

> 
> I do not think SYNC and ASYNC are great adjectives for what you are
> doing in this step.  Wouldn't ASYNC mean "the caller tells the hooks
> to run, continues without waiting for them to finish"?  In order to
> let more than one hooks to run at the same time, the caller has to
> continue without waiting for the first one it started so that it can
> start the second one before the first one finishes (otherwise these
> two hooks will not run at the same time), but ASYNC implys a lot
> more than that.  After starting all of these hooks, the caller may
> continue doing things that are not even related to these spawned
> hooks if a hook is run asynchronously.
> 
> But I do not think that is what is happening here.  After all,
> run_processes_parallel_tr2() will not return until all of the
> subprocesses are culled.
> 
> Perhaps the distinction you are trying to express is easier to
> convey to readers if you instead used the verb TO RUN with the
> adverb SERIALLY vs IN PARALLEL (as opposed to SYNCHRONOUSLY vs
> ASYNCHRONOUSLY)?

Good point. I think you are right and I'll change the init macros to
SERIAL/PARALLEL instead.

> 
> > diff --git a/builtin/hook.c b/builtin/hook.c
> > index 4d39c9e75e..12c9126032 100644
> > --- a/builtin/hook.c
> > +++ b/builtin/hook.c
> > @@ -22,7 +22,7 @@ static const char * const builtin_hook_run_usage[] = {
> >  static int run(int argc, const char **argv, const char *prefix)
> >  {
> >  	int i;
> > -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
> > +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
> >  	int ignore_missing = 0;
> >  	const char *hook_name;
> >  	struct list_head *hooks;
> > @@ -32,6 +32,8 @@ static int run(int argc, const char **argv, const char *prefix)
> >  			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
> >  		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
> >  			   N_("file to read into hooks' stdin")),
> > +		OPT_INTEGER('j', "jobs", &opt.jobs,
> > +			    N_("run up to <n> hooks simultaneously")),
> >  		OPT_END(),
> >  	};
> >  	int ret;
> 
> OK, so by default "git hook" runs 1 at a time but we can instruct to
> run the optimal number of jobs via "git hook -j0" etc.

Is it surprising for an explicitly provided "-j0" to still be overridden
by a config if one was provided? That is, "-j0" doesn't mean
"-j$(nproc)" as it does in many other Unixy tools, here. It means
"-j(hook.jobs OR $(nproc))". I guess you could also say that "if you
configured hook.jobs, it is because you decided nproc was less optimal
than the value you put into hook.jobs" - so maybe it is fine?

> 
> > diff --git a/hook.c b/hook.c
> > index 80e150548c..37f682c6d8 100644
> > --- a/hook.c
> > +++ b/hook.c
> > @@ -228,6 +228,28 @@ static int notify_hook_finished(int result,
> >  	return 0;
> >  }
> >  
> > +/*
> > + * Determines how many jobs to use after we know we want to parallelize. First
> > + * priority is the config 'hook.jobs' and second priority is the number of CPUs.
> > + */
> > +static int configured_hook_jobs(void)
> > +{
> > +	/*
> > +	 * The config and the CPU count probably won't change during the process
> > +	 * lifetime, so cache the result in case we invoke multiple hooks during
> > +	 * one process.
> > +	 */
> > +	static int jobs = 0;
> > +	if (jobs)
> > +		return jobs;
> > +
> > +	if (git_config_get_int("hook.jobs", &jobs))
> > +		/* if the config isn't set, fall back to CPU count. */
> > +		jobs = online_cpus();
> > +
> > +	return jobs;
> > +}
> 
> Not a suggestion to make this improvement as a part of this series,
> but there already are more than several codepaths doing essentially
> the same thing as above, taking their own configuration or
> environment variables and falling back to online_cpus().
> 
> It may be worth introducing a shared helper function so that this
> can become
> 
> 	static int configured_hook_jobs(void)
> 	{
> 		static int jobs; /* no need for "= 0" */
> 
> 		if (!jobs)
> 			jobs = default_parallelism("hook.jobs", NULL);
> 		return jobs;
> 	}
> 
> where the new helper takes the configuration and environment
> variable names.

Sounds handy.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 3/6] hook: introduce "git hook list"
  2021-08-12 18:59                 ` Junio C Hamano
@ 2021-08-17  0:36                   ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-17  0:36 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Thu, Aug 12, 2021 at 11:59:51AM -0700, Junio C Hamano wrote:
> 
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > +static int list(int argc, const char **argv, const char *prefix)
> > +{
> > +	struct list_head *head, *pos;
> > +	const char *hookname = NULL;
> > +	struct strbuf hookdir_annotation = STRBUF_INIT;
> > +
> > +	struct option list_options[] = {
> > +		OPT_END(),
> > +	};
> > +
> > +	argc = parse_options(argc, argv, prefix, list_options,
> > +			     builtin_hook_list_usage, 0);
> > +
> > +	if (argc < 1)
> > +		usage_msg_opt(_("You must specify a hook event name to list."),
> > +			      builtin_hook_list_usage, list_options);
> > +
> > +	hookname = argv[0];
> > +
> > +	head = hook_list(hookname);
> > +
> > +	if (list_empty(head)) {
> 
> The same "can't hook_list() signal an error by returning NULL?"
> comment applies here.
> 
> 	head = hook_list(hookname);
> 	if (!head)
> 		die(_("no such hook '%s'"), hookname);
> 
> or something?
> 
> > +		printf(_("no commands configured for hook '%s'\n"),
> > +		       hookname);
> > +		return 0;
> 
> If it is a normally expected state that there is no hook for the
> given name, signalling success by returning 0 here may be sensible,
> but then the message should at least go to the standard error stream
> to leave the standard output empty, so that a caller can reasonably
> do something like
> 
> 	for path in $(git hooks list "$1")
> 	do
> 		ls -l "$path"
> 	done
> 
> If we really want to show such a message, perhaps
> 
> 	if (list_empty(head)) {
>         	if (!quiet)
> 			warning(_("no commands configured"));
> 		return 0;
> 	}
> 
> The normal display just shows the path without saying "command %s
> will run for hook %s"; the warning probably should do the same.
> 
> Having said that, if it truly is a normal and expected state that no
> hook is defined for the given name, I actually think there should be
> no message.

Ah, I think you are saying "either return an error code and be chatty if you
want, or return an empty list and a success code, but pick one". Makes
sense to me.

No message + well-defined return code sounds fine. I'll do that.

> 
> > +	}
> > +
> > +	list_for_each(pos, head) {
> > +		struct hook *item = list_entry(pos, struct hook, list);
> > +		item = list_entry(pos, struct hook, list);
> > +		if (item)
> > +			printf("%s\n", item->hook_path);
> > +	}
> 
> > diff --git a/hook.c b/hook.c
> > index 37f682c6d8..2714b63473 100644
> > --- a/hook.c
> > +++ b/hook.c
> > @@ -96,22 +96,20 @@ int hook_exists(const char *name)
> >  struct list_head* hook_list(const char* hookname)
> >  {
> >  	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
> > +	const char *hook_path = find_hook(hookname);
> > +
> >  
> >  	INIT_LIST_HEAD(hook_head);
> >  
> >  	if (!hookname)
> >  		return NULL;
> >  
> > -	if (have_git_dir()) {
> > -		const char *hook_path = find_hook(hookname);
> > -
> > -		/* Add the hook from the hookdir */
> > -		if (hook_path) {
> > -			struct hook *to_add = xmalloc(sizeof(*to_add));
> > -			to_add->hook_path = hook_path;
> > -			to_add->feed_pipe_cb_data = NULL;
> > -			list_add_tail(&to_add->list, hook_head);
> > -		}
> > +	/* Add the hook from the hookdir */
> > +	if (hook_path) {
> > +		struct hook *to_add = xmalloc(sizeof(*to_add));
> > +		to_add->hook_path = hook_path;
> > +		to_add->feed_pipe_cb_data = NULL;
> > +		list_add_tail(&to_add->list, hook_head);
> >  	}
> 
> I do not think this belongs to the step to add "list" command.  The
> log message does not explain or justify why have-git-dir goes away,
> either.

Ah, sure.


It seems like I also didn't update the documentation for 'git hook'
command during this commit. Will fix that as well.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 4/6] hook: allow running non-native hooks
  2021-08-12 19:08                 ` Junio C Hamano
@ 2021-08-18 20:51                   ` Emily Shaffer
  2021-08-18 21:14                     ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-08-18 20:51 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Thu, Aug 12, 2021 at 12:08:10PM -0700, Junio C Hamano wrote:
> 
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > diff --git a/builtin/hook.c b/builtin/hook.c
> > index c36b05376c..3aa65dd791 100644
> > --- a/builtin/hook.c
> > +++ b/builtin/hook.c
> > @@ -46,7 +46,7 @@ static int list(int argc, const char **argv, const char *prefix)
> >  
> >  	hookname = argv[0];
> >  
> > -	head = hook_list(hookname);
> > +	head = hook_list(hookname, 1);
> >  
> >  	if (list_empty(head)) {
> >  		printf(_("no commands configured for hook '%s'\n"),
> > @@ -108,7 +108,7 @@ static int run(int argc, const char **argv, const char *prefix)
> >  	git_config(git_default_config, NULL);
> >  
> >  	hook_name = argv[0];
> > -	hooks = hook_list(hook_name);
> > +	hooks = hook_list(hook_name, 1);
> >  	if (list_empty(hooks)) {
> >  		/* ... act like run_hooks_oneshot() under --ignore-missing */
> >  		if (ignore_missing)
> 
> This is minor, as I expect that the callers of hook_list() will
> always confined in builtin/hook.c, but it probably is easier to read
> if you gave two functions, just like you have the pair of helpers
> find_hook() and find_hook_gently(), as the literal "1" forces the
> readers to remember if that means "die if not found", or "ok if it
> is a bogus name".

Yes, I see what you mean. Ok. I have been wanting to change the naming
anyways - most functions in hook.h are verb-y ("find hook", "run hooks",
so on) but hook_list stands out as the only noun-y function.

So I considered changing it to "list_hooks" and "list_hooks_gently", to align
with find_hook(_gently)....

> 
> In addition, it may make more sense to keep hook_list() signal
> failure by returning NULL and leave the dying to the caller.
> In-code callers (as opposed to "git hook run" that can throw any
> random string that came from the user at the API) will never throw a
> bogus name unless there is a bug, and they'll have to check for an
> error return from hook_list() anyway and the error message they
> would give may have to be different from the one that is given
> against a hook name randomly thrown at us by the user.

Sure, that makes sense enough... but then I wonder if it would be better
to let the caller check whether the name is allowed at all, first,
separately from the hook_list() call.

On the one hand, having hook_list() do the validation of the hook name
makes it harder for a hook doing something very unusual to neglect to
add documentation. (I'm thinking of, for example, a hook doing something
equally weird to the proc-receive hook, which cannot use the hook
library because it needs to be able to do this weird two-way
communication thing.
(https://lore.kernel.org/git/20210527000856.695702-31-emilyshaffer%40google.com))
It would be pretty bad for a hook which is already complicated to also
forget to include documentation.

On the other hand, as it is now - builtin/hook.c hardcodes "I don't care
if the hook is unknown" and hook.c hardcodes "reject if the hook is
unknown" and nobody else calls hook_list at all - it isn't so bad to
bail early, before even calling hook_list() in the first place, if the
hook is unknown.

I also think that approach would make a callsite easier to understand
than checking for null from hook_list().

  const char *hookname = "my-new-hook";

  /* Here it's pretty clear what the reason for the error was... */
  if (!known_hook(hookname))
    BUG("is hook '%s' in Documentation/githooks.txt?", hookname);

  hooks = hook_list(hookname);
  ...

vs.

  const char *hookname = "my-new-hook";
  hooks = hook_list(hookname);
  /*
   * But here, I have to go and look at the hook_list() source to
   * understand why null 'hooks' means I missed some doc step.
   */
  if (!hookname)
    BUG("is hook '%s' in Documentation/githooks.txt?", hookname);
  ...

Maybe others disagree with me, but I would guess the first example is
more easily understandable to someone unfamiliar with the hook code. So
I think I will go with that approach, and include some notice in the doc
comment over hook_list().

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 4/6] hook: allow running non-native hooks
  2021-08-18 20:51                   ` Emily Shaffer
@ 2021-08-18 21:14                     ` Emily Shaffer
  2021-08-18 21:24                       ` Junio C Hamano
  0 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-08-18 21:14 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Wed, Aug 18, 2021 at 01:51:58PM -0700, Emily Shaffer wrote:
> 
> On Thu, Aug 12, 2021 at 12:08:10PM -0700, Junio C Hamano wrote:
> > 
> > Emily Shaffer <emilyshaffer@google.com> writes:
> > 
> > > diff --git a/builtin/hook.c b/builtin/hook.c
> > > index c36b05376c..3aa65dd791 100644
> > > --- a/builtin/hook.c
> > > +++ b/builtin/hook.c
> > > @@ -46,7 +46,7 @@ static int list(int argc, const char **argv, const char *prefix)
> > >  
> > >  	hookname = argv[0];
> > >  
> > > -	head = hook_list(hookname);
> > > +	head = hook_list(hookname, 1);
> > >  
> > >  	if (list_empty(head)) {
> > >  		printf(_("no commands configured for hook '%s'\n"),
> > > @@ -108,7 +108,7 @@ static int run(int argc, const char **argv, const char *prefix)
> > >  	git_config(git_default_config, NULL);
> > >  
> > >  	hook_name = argv[0];
> > > -	hooks = hook_list(hook_name);
> > > +	hooks = hook_list(hook_name, 1);
> > >  	if (list_empty(hooks)) {
> > >  		/* ... act like run_hooks_oneshot() under --ignore-missing */
> > >  		if (ignore_missing)
> > 
> > This is minor, as I expect that the callers of hook_list() will
> > always confined in builtin/hook.c, but it probably is easier to read
> > if you gave two functions, just like you have the pair of helpers
> > find_hook() and find_hook_gently(), as the literal "1" forces the
> > readers to remember if that means "die if not found", or "ok if it
> > is a bogus name".
> 
> Yes, I see what you mean. Ok. I have been wanting to change the naming
> anyways - most functions in hook.h are verb-y ("find hook", "run hooks",
> so on) but hook_list stands out as the only noun-y function.
> 
> So I considered changing it to "list_hooks" and "list_hooks_gently", to align
> with find_hook(_gently)....
> 
> > 
> > In addition, it may make more sense to keep hook_list() signal
> > failure by returning NULL and leave the dying to the caller.
> > In-code callers (as opposed to "git hook run" that can throw any
> > random string that came from the user at the API) will never throw a
> > bogus name unless there is a bug, and they'll have to check for an
> > error return from hook_list() anyway and the error message they
> > would give may have to be different from the one that is given
> > against a hook name randomly thrown at us by the user.
> 
> Sure, that makes sense enough... but then I wonder if it would be better
> to let the caller check whether the name is allowed at all, first,
> separately from the hook_list() call.
> 
> On the one hand, having hook_list() do the validation of the hook name
> makes it harder for a hook doing something very unusual to neglect to
> add documentation. (I'm thinking of, for example, a hook doing something
> equally weird to the proc-receive hook, which cannot use the hook
> library because it needs to be able to do this weird two-way
> communication thing.
> (https://lore.kernel.org/git/20210527000856.695702-31-emilyshaffer%40google.com))
> It would be pretty bad for a hook which is already complicated to also
> forget to include documentation.
> 
> On the other hand, as it is now - builtin/hook.c hardcodes "I don't care
> if the hook is unknown" and hook.c hardcodes "reject if the hook is
> unknown" and nobody else calls hook_list at all - it isn't so bad to
> bail early, before even calling hook_list() in the first place, if the
> hook is unknown.
> 
> I also think that approach would make a callsite easier to understand
> than checking for null from hook_list().
> 
>   const char *hookname = "my-new-hook";
> 
>   /* Here it's pretty clear what the reason for the error was... */
>   if (!known_hook(hookname))
>     BUG("is hook '%s' in Documentation/githooks.txt?", hookname);
> 
>   hooks = hook_list(hookname);
>   ...
> 
> vs.
> 
>   const char *hookname = "my-new-hook";
>   hooks = hook_list(hookname);
>   /*
>    * But here, I have to go and look at the hook_list() source to
>    * understand why null 'hooks' means I missed some doc step.
>    */
>   if (!hookname)
>     BUG("is hook '%s' in Documentation/githooks.txt?", hookname);
>   ...
> 
> Maybe others disagree with me, but I would guess the first example is
> more easily understandable to someone unfamiliar with the hook code. So
> I think I will go with that approach, and include some notice in the doc
> comment over hook_list().

Hm. Now that I sit to type it, I guess putting the onus on the
strange-new-hook caller to also type "if known_hook()" is about the same
as just expecting the strange-new-hook caller to know they are supposed
to document their hook. Plus, known_hook() is static right now.

I think it still makes sense to BUG() instead of error() or die() in
'list_hooks()' (non-gently) - the failure of that call is a developer
error, either in not having documented their hook correctly or in
calling 'list_hooks()' instead of 'list_hooks_gently()' when they meant
the latter. So I will not take the NULL return approach.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 4/6] hook: allow running non-native hooks
  2021-08-18 21:14                     ` Emily Shaffer
@ 2021-08-18 21:24                       ` Junio C Hamano
  0 siblings, 0 replies; 479+ messages in thread
From: Junio C Hamano @ 2021-08-18 21:24 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

>> Yes, I see what you mean. Ok. I have been wanting to change the naming
>> anyways - most functions in hook.h are verb-y ("find hook", "run hooks",
>> so on) but hook_list stands out as the only noun-y function.
>> 
>> So I considered changing it to "list_hooks" and "list_hooks_gently", to align
>> with find_hook(_gently)....

I do not claim I am goot at naming (or better than you at it
anyway), but list-hooks sounds to me like it is calling printf()
to show the hooks to the user, not computing a list of hooks and
returning it to the caller.

>> I also think that approach would make a callsite easier to understand
>> than checking for null from hook_list().
>> 
>>   const char *hookname = "my-new-hook";
>> 
>>   /* Here it's pretty clear what the reason for the error was... */
>>   if (!known_hook(hookname))
>>     BUG("is hook '%s' in Documentation/githooks.txt?", hookname);

Yes.  The callsite becomes easier to understand, and it separates
the responsibility between the helper to respond to "please give me
list of defined hooks" and its caller that may react to the returned
list with "ok, among these hooks, this does not look kosher for such
and such reason, so I'd die/warn/error" much cleaner.



^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v2 5/6] hook: include hooks from the config
  2021-08-12 20:48                 ` Junio C Hamano
@ 2021-08-19  0:09                   ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-19  0:09 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Thu, Aug 12, 2021 at 01:48:00PM -0700, Junio C Hamano wrote:
> 
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > Teach the hook.[hc] library to parse configs to populare the list of
> > hooks to run for a given event.
> >
> > Multiple commands can be specified for a given hook by providing
> > multiple "hook.<friendly-name>.command = <path-to-hook>" and
> > "hook.<friendly-name>.event = <hook-event>" lines. Hooks will be run in
> > config order of the "hook.<name>.event" lines.
> >
> > For example:
> >
> >   $ git config --list | grep ^hook
> >   hook.bar.command=~/bar.sh
> >   hook.bar.event=pre-commit
> 
> Your answer might be "read the design doc", but it is unclear to me
> why "bar" (friendly-name) is needed in this picture at all.  Is it
> because you may want to fire more than one command for pre-commit
> event?  IOW,
> 
> 	[hook "bar"]
> 		command = bar1.sh
> 		command = bar2.sh
> 		event = pre-commit
> 
> is easier to manage with an extra level of redirection?  I doubt it
> as 
> 
> 	[hook "pre-commit"]
> 		command = bar1.sh
> 		command = bar2.sh
> 
> would be equally expressive and shorter.  Or would it help use case
> for multiple "friendly-name" to refer to the same "event", e.g.
> 
> 	[hook "xyzzy"]
> 		event = pre-commit
> 		command = xyzzy1
> 
> 	[hook "frotz"]
> 		event = pre-commit
>                 command = frotz1
>                 command = frotz2
> 
> or something?  I am not sure if this gives us useful extra
> flexibility, and if so, the extra flexibility helps us more than it
> confuses us.
> 
> And moving the "event" to the second level in the configuration
> hierarchy, getting rid of "friendly-name" from the design, would not
> make this example unworkable, either:
> 
> >   $ git hook run
> >   # Runs ~/bar.sh
> >   # Runs .git/hooks/pre-commit
> 
> Again, this is not an objection wrapped in a rhetorical question.
> It just is that I do not see how the extra level of redirection
> helps us.

Please have a look at
https://lore.kernel.org/git/87fswey5wd.fsf%40evledraar.gmail.com and
replies, where Ævar and I discussed the schema change at length. I know
it is a lot of back and forth but I think it is useful to understand why
I ended up changing the schema this way.

> 
> > diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
> > index 96d3d6572c..a97b980cca 100644
> > --- a/Documentation/config/hook.txt
> > +++ b/Documentation/config/hook.txt
> > @@ -1,3 +1,8 @@
> > +hook.<command>.command::
> > +	A command to execute during the <command> hook event. This can be an
> > +	executable on your device, a oneliner for your shell, or the name of a
> > +	hookcmd. See linkgit:git-hook[1].
> 
> Please make sure you use the terminology consistently.  If the
> second level is "friendly name", hook.<name>.command should be
> described, instead of hook.<command>.command.

Thanks, this is an oversight. Will update the config/hook.txt doc in
next reroll.

> 
> Also, to help those who are familiar with the current Git from their
> use in the past 10 years or so, giving an example name from the
> current system may help, e.g. when describing hook.<name>.event,
> you may want to say the values are things like "pre-commit",
> "receive", etc.

Sure.

> 
> > +This command parses the default configuration files for pairs of configs like
> > +so:
> > +
> > +  [hook "linter"]
> > +    event = pre-commit
> > +    command = ~/bin/linter --c
> 
> The above addition of .command should also have hook.<name>.event
> next to it, no?

I don't understand the question. Doesn't this config snippet equate to
"""
  hook.linter.event=pre-commit
  hook.linter.command=~/bin/linter --c
"""
? So in this case, '<name>' is 'linter', as that's not a native Git hook.

> 
> > +Conmmands are run in the order Git encounters their associated
> 
> "Conmmands -> Commands", I would think.
ACK
> 
> > +`hook.<name>.event` configs during the configuration parse (see
> > +linkgit:git-config[1]).
> 
> Here you use <name>, which should be matched by the description in
> the first hunk of the patch to this file.
Yep.
> 
> > +In general, when instructions suggest adding a script to
> > +`.git/hooks/<hook-event>`, you can specify it in the config instead by running
> > +`git config --add hook.<some-name>.command <path-to-script> && git config --add
> > +hook.<some-name>.event <hook-event>` - this way you can share the script between
> > +multiple repos. That is, `cp ~/my-script.sh ~/project/.git/hooks/pre-commit`
> > +would become `git config --add hook.my-script.command ~/my-script.sh && git
> > +config --add hook.my-script.event pre-commit`.
> 
> One repository may use a friendly name "xyzzy" while the other may
> use "frotz" to group the hooks that trigger upon "pre-commit" event,
> but unless one of the repositories change the friendly name to
> match, they cannot share these configurations, no?  It seems that an
> extra level of indirection is hindering sharing, rather than
> helping.

Ah, I think this means the documentation isn't sufficient, if you are
asking that. Instead of explaining it in ephemeral email, I think it is
better for me to explain it in documentation reroll, and for you to then
tell me how you interpret it. I expect to send the reroll before I go
home today, since I didn't receive comments from anybody besides you so
far.

Thanks very much for the feedback on the doc - this is very useful.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v4 00/36] Run hooks via "git run hook" & hook library
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (36 preceding siblings ...)
  2021-08-12  0:42             ` [PATCH v2 0/6] config-based hooks restarted Emily Shaffer
@ 2021-08-19  0:17             ` Emily Shaffer
  2021-08-19 23:40             ` Emily Shaffer
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
  39 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-19  0:17 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

On Tue, Aug 03, 2021 at 09:38:26PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> This is a v4 re-roll of the "Base for "config-based-hooks" topic.

I performed the rebase of my series on top of this one, and did not find
very much objectionable there, but do not consider this email a thorough
review of the interdiff. I expect to have time for that later in the
week.

> Other updates:
> 
>  * In the base topic the s/Signed-off-by/Reviewed-by/g from René
>    change that Junio applied locally has been folded in.
> 
>  * Almost all the callers were just "one-shot" callers, I introduced a
>    new run_hooks_oneshot() function for those, which gets rid of the
>    verbosity around memory management, see e.g. the "builtin/gc.c" in
>    the range-diff below. That run_hooks_oneshot() can also take a NULL
>    set of options.

I am not so wild about this, to be honest, only because after my patch
2, it is hard to understand whether or not a hook is running in parallel
or in series. It is extra code-reader overhead to know that
run_hooks_oneshot() assumes that a hook can be parallelized, and I think
that is unfortunate.

But that is a pretty minor complaint, and I do like the lower
complexity, and I agree the defaults are "good enough" for most hook
events. So I guess I'm asking for ideas on how to make it more obvious
what this oneshot call does in terms of parallelism.

Thanks, and I hope to get to a fuller review of the interdiff soon.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH v3 0/6] config-based hooks restarted
  2021-08-12  0:42             ` [PATCH v2 0/6] config-based hooks restarted Emily Shaffer
                                 ` (6 preceding siblings ...)
  2021-08-12  4:47               ` [PATCH v2 0/6] config-based hooks restarted Junio C Hamano
@ 2021-08-19  3:34               ` Emily Shaffer
  2021-08-19  3:34                 ` [PATCH v3 1/6] hook: run a list of hooks instead Emily Shaffer
                                   ` (7 more replies)
  7 siblings, 8 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-19  3:34 UTC (permalink / raw)
  To: git
  Cc: Emily Shaffer, Junio C Hamano, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m. carlson, Josh Steadmon,
	Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

This is the config-based hooks topic rebased onto v4 of Ævar's
branch[1]. There is a happy CI build of it on GitHub[2].

The topic overall adds the ability to set up hooks by modifying the
config, in addition to placing specially named hooks into the hookdir.
This enables users to specify multiple hooks for a given event, and so
this topic also fleshes out the use of the run_processes_parallel() API
which is now introduced in Ævar's reordering of prior patches.

Patches 1-4 make some minor changes to prepare Ævar's series to handle
more than one hook at a time. With the exception of patch 4, there
should be no behavior change for existing hooks.

Patch 2 is opinionated about which hooks should and shouldn't be allowed
to run in parallel; if you care about a specific hook, please take a
look there.

Patch 5 is the motivating feature - it begins to parse the config
looking for hooks.

Patch 6 takes advantage of the decoupling of hooks and GITDIR to allow
out-of-repo hook runs, which would only run hooks specified in the
system or global config. This mainly targets 'sendemail-validate', but
'git-sendemail.perl' still explicitly disallows out-of-repo hook
execution on that hook for now. (Maybe that change should be added in
this series? Or maybe patch 6 belongs with that kind of change?)

Since v2, I have addressed comments left by Junio - thanks for the
review. Mostly these were small nits, but two somewhat larger changes
came out:

 - 'hook_list' becomes 'list_hooks' and 'list_hooks_gently'. It might
   still be better to name it something else, and it might still be
   better to decouple the "is this a known hook" check from the "give me
   all the hooks" function; comments welcome.
 - The documentation in patch 5, which starts using the config, has been
   expanded to hopefully become more clear. I am especially interested
   in feedback in this doc change, as it's the main place users should
   be able to learn how to use the new feature.

Everything else should be pretty minor.

Right now I'm trying to focus on this series first and foremost, hence
sending two rerolls based on the same version of Ævar's base restart.
I'll try to perform a code review on Ævar's latest tomorrow.

Thanks!
 - Emily

1: https://lore.kernel.org/git/cover-v4-00.36-00000000000-20210803T191505Z-avarab%40gmail.com
2: https://github.com/nasamuffin/git/actions/runs/1145128560

Emily Shaffer (6):
  hook: run a list of hooks instead
  hook: allow parallel hook execution
  hook: introduce "git hook list"
  hook: allow running non-native hooks
  hook: include hooks from the config
  hook: allow out-of-repo 'git hook' invocations

 Documentation/config/hook.txt |  22 +++
 Documentation/git-hook.txt    |  87 +++++++++-
 builtin/am.c                  |   4 +-
 builtin/checkout.c            |   2 +-
 builtin/clone.c               |   2 +-
 builtin/hook.c                |  65 +++++++-
 builtin/merge.c               |   2 +-
 builtin/rebase.c              |   2 +-
 builtin/receive-pack.c        |   9 +-
 builtin/worktree.c            |   2 +-
 commit.c                      |   2 +-
 git.c                         |   2 +-
 hook.c                        | 293 +++++++++++++++++++++++++++++-----
 hook.h                        |  61 +++++--
 read-cache.c                  |   2 +-
 refs.c                        |   2 +-
 reset.c                       |   3 +-
 sequencer.c                   |   4 +-
 t/t1800-hook.sh               | 161 ++++++++++++++++++-
 transport.c                   |   2 +-
 20 files changed, 648 insertions(+), 81 deletions(-)
 create mode 100644 Documentation/config/hook.txt

Range-diff against v2:
 1:  81fe1ed90d <  -:  ---------- Makefile: mark "check" target as .PHONY
 2:  b32abc81fb <  -:  ---------- Makefile: stop hardcoding {command,config}-list.h
 3:  fa46bd1154 <  -:  ---------- Makefile: remove an out-of-date comment
 4:  715fd63089 <  -:  ---------- hook.[ch]: move find_hook() to this new library
 5:  ac6c018d27 <  -:  ---------- hook.c: add a hook_exists() wrapper and use it in bugreport.c
 6:  53619d87a2 <  -:  ---------- hook.c users: use "hook_exists()" insted of "find_hook()"
 7:  39dbb89620 <  -:  ---------- hook-list.h: add a generated list of hooks, like config-list.h
 8:  7d2c1cec74 <  -:  ---------- hook: add 'run' subcommand
 9:  3fea1cdaf8 <  -:  ---------- gc: use hook library for pre-auto-gc hook
10:  c86a9c6f95 <  -:  ---------- rebase: convert pre-rebase to use hook.h
11:  31dff1f274 <  -:  ---------- am: convert applypatch to use hook.h
12:  a2e27d745c <  -:  ---------- hooks: convert 'post-checkout' hook to hook library
13:  1a71fa8b75 <  -:  ---------- merge: convert post-merge to use hook.h
14:  95e19fa468 <  -:  ---------- git hook run: add an --ignore-missing flag
15:  371f581e3b <  -:  ---------- send-email: use 'git hook run' for 'sendemail-validate'
16:  99306a4bd3 <  -:  ---------- git-p4: use 'git hook' to run hooks
17:  59d1a563ca <  -:  ---------- commit: convert {pre-commit,prepare-commit-msg} hook to hook.h
18:  7c8de04707 <  -:  ---------- read-cache: convert post-index-change to use hook.h
19:  a2e7176609 <  -:  ---------- receive-pack: convert push-to-checkout hook to hook.h
20:  f03ef24df7 <  -:  ---------- run-command: remove old run_hook_{le,ve}() hook API
21:  18c1dfe677 <  -:  ---------- run-command: allow stdin for run_processes_parallel
22:  eff713fc19 <  -:  ---------- hook: support passing stdin to hooks
23:  2cbec65834 <  -:  ---------- am: convert 'post-rewrite' hook to hook.h
24:  5d34d18bee <  -:  ---------- run-command: add stdin callback for parallelization
25:  b989d910ed <  -:  ---------- hook: provide stdin by string_list or callback
26:  ce6929493b <  -:  ---------- hook: convert 'post-rewrite' hook in sequencer.c to hook.h
27:  13affda962 <  -:  ---------- transport: convert pre-push hook to hook.h
28:  c74720029a <  -:  ---------- hook tests: test for exact "pre-push" hook input
29:  1252a251bb <  -:  ---------- hook tests: use a modern style for "pre-push" tests
30:  3f2488fea6 <  -:  ---------- reference-transaction: use hook.h to run hooks
31:  2e78c47ead <  -:  ---------- run-command: allow capturing of collated output
32:  95132acbb0 <  -:  ---------- hooks: allow callers to capture output
33:  f2510ca910 <  -:  ---------- receive-pack: convert 'update' hook to hook.h
34:  8686308864 <  -:  ---------- post-update: use hook.h library
35:  b10f5142a0 <  -:  ---------- receive-pack: convert receive hooks to hook.h
36:  5c29c932c3 <  -:  ---------- hooks: fix a TOCTOU in "did we run a hook?" heuristic
37:  5177e8ba2c !  1:  6d6400329c hook: run a list of hooks instead
    @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
     -		return run_hooks_oneshot(hook_name, &opt);
     -	hook_path = find_hook(hook_name);
     -	if (!hook_path) {
    -+	hooks = hook_list(hook_name);
    ++	hooks = list_hooks(hook_name);
     +	if (list_empty(hooks)) {
     +		/* ... act like run_hooks_oneshot() under --ignore-missing */
     +		if (ignore_missing)
    @@ hook.c
      
     +static void free_hook(struct hook *ptr)
     +{
    -+	if (ptr) {
    ++	if (ptr)
     +		free(ptr->feed_pipe_cb_data);
    -+	}
     +	free(ptr);
     +}
     +
    @@ hook.c
      static int known_hook(const char *name)
      {
      	const char **p;
    -@@ hook.c: int hook_exists(const char *name)
    - 	return !!find_hook(name);
    - }
    +@@ hook.c: const char *find_hook(const char *name)
      
    -+struct list_head* hook_list(const char* hookname)
    + int hook_exists(const char *name)
    + {
    +-	return !!find_hook(name);
    ++	return !list_empty(list_hooks(name));
    ++}
    ++
    ++struct list_head *list_hooks(const char *hookname)
     +{
     +	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
     +
     +	INIT_LIST_HEAD(hook_head);
     +
     +	if (!hookname)
    -+		return NULL;
    ++		BUG("null hookname was provided to hook_list()!");
     +
     +	if (have_git_dir()) {
     +		const char *hook_path = find_hook(hookname);
    @@ hook.c: int hook_exists(const char *name)
     +	}
     +
     +	return hook_head;
    -+}
    -+
    + }
    + 
      void run_hooks_opt_clear(struct run_hooks_opt *o)
    - {
    - 	strvec_clear(&o->env);
     @@ hook.c: static int pick_next_hook(struct child_process *cp,
      	cp->dir = hook_cb->options->dir;
      
    @@ hook.c: int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *optio
     -	hook_path = find_hook(hook_name);
     -	if (!hook_path) {
     -		ret = 0;
    -+	hooks = hook_list(hook_name);
    ++	hooks = list_hooks(hook_name);
     +
     +	/*
     +	 * If you need to act on a missing hook, use run_found_hooks()
    @@ hook.h: struct hook {
     + * Provides a linked list of 'struct hook' detailing commands which should run
     + * in response to the 'hookname' event, in execution order.
     + */
    -+struct list_head* hook_list(const char *hookname);
    ++struct list_head *list_hooks(const char *hookname);
     +
      struct run_hooks_opt
      {
38:  eda439cd57 !  2:  dfb995ce4d hook: allow parallel hook execution
    @@ Commit message
         hook: allow parallel hook execution
     
         In many cases, there's no reason not to allow hooks to execute in
    -    parallel. run_processes_parallel() is well-suited - it's a task queue
    -    that runs its housekeeping in series, which means users don't
    -    need to worry about thread safety on their callback data. True
    -    multithreaded execution with the async_* functions isn't necessary here.
    -    Synchronous hook execution can be achieved by only allowing 1 job to run
    -    at a time.
    +    parallel, if more than one was provided. hook.c already calls
    +    run_processes_parallel(), so all we need to do is allow the job count we
    +    hand to run_processes_parallel() to be greater than 1.
     
    -    Teach run_hooks() to use that function for simple hooks which don't
    -    require stdin or capture of stderr.
    +    If users have specified no alternative, we can use the processor count
    +    from online_cpus() to run an efficient number of tasks at once. However,
    +    users can also override this number if they want by configuring
    +    'hook.jobs'.
    +
    +    To avoid looking up 'hook.jobs' in cases where we don't end up with any
    +    hooks to run anyways, teach the hook runner commands to notice if
    +    .jobs==0 and do a config or online_cpus() lookup if so, when we already
    +    know we have jobs to run.
    +
    +    Serial execution can still be performed by setting .jobs == 1.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
         Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    @@ builtin/am.c: static void am_destroy(const struct am_state *state)
      {
      	int ret;
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
      
      	assert(state->msg);
      	strvec_push(&opt.args, am_path(state, "final-commit"));
    @@ builtin/am.c: static int run_applypatch_msg_hook(struct am_state *state)
      static int run_post_rewrite_hook(const struct am_state *state)
      {
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      
      	strvec_push(&opt.args, "rebase");
      	opt.path_to_stdin = am_path(state, "rewritten");
    @@ builtin/checkout.c: struct branch_info {
      			      int changed)
      {
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
      
      	/* "new_commit" can be NULL when checking out from the index before
      	   a commit exists. */
    @@ builtin/clone.c: static int checkout(int submodule_progress)
      	struct tree_desc t;
      	int err = 0;
     -	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_SYNC;
    ++	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_SERIAL;
      
      	if (option_no_checkout)
      		return 0;
    @@ builtin/hook.c: static const char * const builtin_hook_run_usage[] = {
      {
      	int i;
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
      	int ignore_missing = 0;
      	const char *hook_name;
      	struct list_head *hooks;
    @@ builtin/merge.c: static void finish(struct commit *head_commit,
      {
      	struct strbuf reflog_message = STRBUF_INIT;
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      	const struct object_id *head = &head_commit->object.oid;
      
      	if (!msg)
    @@ builtin/rebase.c: int cmd_rebase(int argc, const char **argv, const char *prefix
      	int reschedule_failed_exec = -1;
      	int allow_preemptive_ff = 1;
     -	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      	struct option builtin_rebase_options[] = {
      		OPT_STRING(0, "onto", &options.onto_name,
      			   N_("revision"),
    @@ builtin/receive-pack.c: static int run_receive_hook(struct command *commands,
      			    const struct string_list *push_options)
      {
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      	struct receive_hook_feed_context ctx;
      	struct command *iter = commands;
      
    @@ builtin/receive-pack.c: static int run_receive_hook(struct command *commands,
      static int run_update_hook(struct command *cmd)
      {
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      
      	strvec_pushl(&opt.args,
      		     cmd->ref_name,
    @@ builtin/receive-pack.c: static const char *push_to_checkout(unsigned char *hash,
      				    const char *work_tree)
      {
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
     +
      	opt.invoked_hook = invoked_hook;
      
    @@ builtin/receive-pack.c: static const char *update(struct command *cmd, struct sh
      {
      	struct command *cmd;
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      
      	for (cmd = commands; cmd; cmd = cmd->next) {
      		if (cmd->error_string || cmd->did_not_exist)
    @@ builtin/worktree.c: static int add_worktree(const char *path, const char *refnam
      	 */
      	if (!ret && opts->checkout) {
     -		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
    ++		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
      
      		strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
      		strvec_pushl(&opt.args,
    @@ commit.c: int run_commit_hook(int editor_is_used, const char *index_file,
      		    const char *name, ...)
      {
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
      	va_list args;
      	const char *arg;
      
    @@ hook.c: int run_hooks(const char *hook_name, struct list_head *hooks,
      	cb_data.run_me = list_first_entry(hooks, struct hook, list);
      
     -	run_processes_parallel_tr2(jobs,
    -+	/* INIT_ASYNC sets jobs to 0, so go look up how many to use. */
    ++	/* INIT_PARALLEL sets jobs to 0, so go look up how many to use. */
     +	if (!options->jobs)
     +		options->jobs = configured_hook_jobs();
     +
    @@ hook.c: int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *optio
     +	 * Turn on parallelism by default. Hooks which don't want it should
     +	 * specify their options accordingly.
     +	 */
    -+	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT_PARALLEL;
      
      	if (!options)
      		options = &hook_opt_scratch;
    @@ hook.h: struct hook_cb_data {
      	int *invoked_hook;
      };
      
    -+#define RUN_HOOKS_OPT_INIT_SYNC { \
    ++#define RUN_HOOKS_OPT_INIT_SERIAL { \
     +	.jobs = 1, \
     +	.env = STRVEC_INIT, \
     +	.args = STRVEC_INIT, \
     +}
     +
    -+#define RUN_HOOKS_OPT_INIT_ASYNC { \
    ++#define RUN_HOOKS_OPT_INIT_PARALLEL { \
     +	.jobs = 0, \
     +	.env = STRVEC_INIT, \
     +	.args = STRVEC_INIT, \
    @@ read-cache.c: static int do_write_locked_index(struct index_state *istate, struc
      	int ret;
      	int was_full = !istate->sparse_index;
     -	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      
      	ret = convert_to_sparse(istate);
      
    @@ refs.c: int ref_update_reject_duplicates(struct string_list *refnames,
      				const char *state)
      {
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
      	int ret = 0, i;
      
    @@ reset.c: int reset_head(struct repository *r, struct object_id *oid, const char
      	}
      	if (run_hook) {
     -		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
     +
      		strvec_pushl(&opt.args,
      			     oid_to_hex(orig ? orig : null_oid()),
    @@ sequencer.c: int update_head_with_reflog(const struct commit *old_head,
      			    const struct object_id *newoid)
      {
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      	struct strbuf tmp = STRBUF_INIT;
      	struct string_list to_stdin = STRING_LIST_INIT_DUP;
      	int code;
    @@ sequencer.c: static int pick_commits(struct repository *r,
      				st.st_size > 0) {
      			struct child_process notes_cp = CHILD_PROCESS_INIT;
     -			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
    -+			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      
      			notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY);
      			notes_cp.git_cmd = 1;
    @@ transport.c: static int run_pre_push_hook(struct transport *transport,
      {
      	int ret = 0;
     -	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
    -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC;
    ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
      	struct ref *r;
      	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
      
39:  cdfe3b6e16 !  3:  c8a04306e9 hook: introduce "git hook list"
    @@ Commit message
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
     
    + ## Documentation/git-hook.txt ##
    +@@ Documentation/git-hook.txt: SYNOPSIS
    + [verse]
    + 'git hook' run [--to-stdin=<path>] [--ignore-missing] [(-j|--jobs) <n>]
    + 	<hook-name> [-- <hook-args>]
    ++'git hook' list <hook-name>
    + 
    + DESCRIPTION
    + -----------
    +@@ Documentation/git-hook.txt: optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
    + arguments (if any) differ by hook name, see linkgit:githooks[5] for
    + what those are.
    + 
    ++list::
    ++	Print a list of hooks which will be run on `<hook-name>` event. If no
    ++	hooks are configured for that event, print nothing and return 1.
    ++
    + OPTIONS
    + -------
    + 
    +
      ## builtin/hook.c ##
     @@
      
      #define BUILTIN_HOOK_RUN_USAGE \
    - 	N_("git hook run [--to-stdin=<path>] <hook-name> [-- <hook-args>]")
    + 	N_("git hook run [--ignore-missing] [--to-stdin=<path>] <hook-name> [-- <hook-args>]")
     +#define BUILTIN_HOOK_LIST_USAGE \
     +	N_("git hook list <hook-name>")
      
    @@ builtin/hook.c: static const char * const builtin_hook_run_usage[] = {
     +
     +	head = hook_list(hookname);
     +
    -+	if (list_empty(head)) {
    -+		printf(_("no commands configured for hook '%s'\n"),
    -+		       hookname);
    -+		return 0;
    -+	}
    ++	if (list_empty(head))
    ++		return 1;
     +
     +	list_for_each(pos, head) {
     +		struct hook *item = list_entry(pos, struct hook, list);
    @@ builtin/hook.c: int cmd_hook(int argc, const char **argv, const char *prefix)
      
     
      ## hook.c ##
    -@@ hook.c: int hook_exists(const char *name)
    - struct list_head* hook_list(const char* hookname)
    +@@ hook.c: struct list_head *list_hooks(const char *hookname)
      {
      	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
    -+	const char *hook_path = find_hook(hookname);
    -+
      
    ++
      	INIT_LIST_HEAD(hook_head);
      
      	if (!hookname)
    - 		return NULL;
    +@@ hook.c: struct list_head *list_hooks(const char *hookname)
      
    --	if (have_git_dir()) {
    --		const char *hook_path = find_hook(hookname);
    + 	if (have_git_dir()) {
    + 		const char *hook_path = find_hook(hookname);
     -
     -		/* Add the hook from the hookdir */
    --		if (hook_path) {
    --			struct hook *to_add = xmalloc(sizeof(*to_add));
    --			to_add->hook_path = hook_path;
    --			to_add->feed_pipe_cb_data = NULL;
    --			list_add_tail(&to_add->list, hook_head);
    --		}
    -+	/* Add the hook from the hookdir */
    -+	if (hook_path) {
    -+		struct hook *to_add = xmalloc(sizeof(*to_add));
    -+		to_add->hook_path = hook_path;
    -+		to_add->feed_pipe_cb_data = NULL;
    -+		list_add_tail(&to_add->list, hook_head);
    - 	}
    - 
    - 	return hook_head;
    + 		if (hook_path) {
    + 			struct hook *to_add = xmalloc(sizeof(*to_add));
    + 			to_add->hook_path = hook_path;
40:  eb4e03e22b !  4:  af14116d0f hook: allow running non-native hooks
    @@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
      	hookname = argv[0];
      
     -	head = hook_list(hookname);
    -+	head = hook_list(hookname, 1);
    ++	head = list_hooks_gently(hookname);
      
    - 	if (list_empty(head)) {
    - 		printf(_("no commands configured for hook '%s'\n"),
    + 	if (list_empty(head))
    + 		return 1;
     @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
      	git_config(git_default_config, NULL);
      
      	hook_name = argv[0];
    --	hooks = hook_list(hook_name);
    -+	hooks = hook_list(hook_name, 1);
    +-	hooks = list_hooks(hook_name);
    ++	hooks = list_hooks_gently(hook_name);
      	if (list_empty(hooks)) {
      		/* ... act like run_hooks_oneshot() under --ignore-missing */
      		if (ignore_missing)
    @@ hook.c: static int known_hook(const char *name)
     +	const char *hook_path;
      
      	if (!known_hook(name))
    - 		die(_("the hook '%s' is not known to git, should be in hook-list.h via githooks(5)"),
    +-		die(_("the hook '%s' is not known to git, should be in hook-list.h via githooks(5)"),
    ++		BUG(_("the hook '%s' is not known to git, should be in hook-list.h via githooks(5)"),
      		    name);
      
     +	hook_path = find_hook_gently(name);
    @@ hook.c: static int known_hook(const char *name)
      	strbuf_git_path(&path, "hooks/%s", name);
      	if (access(path.buf, X_OK) < 0) {
     @@ hook.c: int hook_exists(const char *name)
    - 	return !!find_hook(name);
    + 	return !list_empty(list_hooks(name));
      }
      
    --struct list_head* hook_list(const char* hookname)
     +struct hook_config_cb
     +{
     +	struct strbuf *hook_key;
     +	struct list_head *list;
     +};
     +
    -+struct list_head* hook_list(const char* hookname, int allow_unknown)
    + struct list_head *list_hooks(const char *hookname)
      {
    - 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
    --	const char *hook_path = find_hook(hookname);
    -+	const char *hook_path;
    +-	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
    ++	if (!known_hook(hookname))
    ++		BUG("Don't recognize hook event '%s'! "
    ++		    "Is it documented in Documentation/githooks.txt?",
    ++		    hookname);
    ++	return list_hooks_gently(hookname);
    ++}
      
    ++struct list_head *list_hooks_gently(const char *hookname)
    ++{
    ++	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
      
      	INIT_LIST_HEAD(hook_head);
    -@@ hook.c: struct list_head* hook_list(const char* hookname)
    - 	if (!hookname)
    - 		return NULL;
    - 
    -+	if (allow_unknown)
    -+		hook_path = find_hook_gently(hookname);
    -+	else
    -+		hook_path = find_hook(hookname);
    -+
    - 	/* Add the hook from the hookdir */
    - 	if (hook_path) {
    - 		struct hook *to_add = xmalloc(sizeof(*to_add));
    + 
    +@@ hook.c: struct list_head *list_hooks(const char *hookname)
    + 		BUG("null hookname was provided to hook_list()!");
    + 
    + 	if (have_git_dir()) {
    +-		const char *hook_path = find_hook(hookname);
    ++		const char *hook_path = find_hook_gently(hookname);
    + 		if (hook_path) {
    + 			struct hook *to_add = xmalloc(sizeof(*to_add));
    + 			to_add->hook_path = hook_path;
     @@ hook.c: int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
      	if (options->path_to_stdin && options->feed_pipe)
      		BUG("choose only one method to populate stdin");
      
    --	hooks = hook_list(hook_name);
     +	/*
    -+	 * 'git hooks run <hookname>' uses run_found_hooks, so we don't need to
    ++	 * 'git hooks run <hookname>' uses run_hooks, so we don't need to
     +	 * allow unknown hooknames here.
     +	 */
    -+	hooks = hook_list(hook_name, 0);
    + 	hooks = list_hooks(hook_name);
      
      	/*
    - 	 * If you need to act on a missing hook, use run_found_hooks()
     
      ## hook.h ##
     @@
    @@ hook.h
       * overwritten by further calls to find_hook and run_hook_*.
     + *
     + * If the hook is not a native hook (e.g. present in Documentation/githooks.txt)
    -+ * find_hook() will die(). find_hook_gently() does not consult the native hook
    ++ * find_hook() will BUG(). find_hook_gently() does not consult the native hook
     + * list and will check for any hook name in the hooks directory regardless of
     + * whether it is known. find_hook() should be used by internal calls to hooks,
     + * and find_hook_gently() should only be used when the hookname was provided by
    @@ hook.h: struct hook {
       * Provides a linked list of 'struct hook' detailing commands which should run
       * in response to the 'hookname' event, in execution order.
     + *
    -+ * If allow_unknown is unset, hooks will be checked against the hook list
    -+ * generated from Documentation/githooks.txt. Otherwise, any hook name will be
    -+ * allowed. allow_unknown should only be set when the hook name is provided by
    -+ * the user; internal calls to hook_list should make sure the hook they are
    -+ * invoking is present in Documentation/githooks.txt.
    ++ * list_hooks() checks the provided hookname against Documentation/githooks.txt
    ++ * and BUG()s if it is not found.  list_hooks_gently() allows any hookname. The
    ++ * latter should only be used when the hook name is provided by the user, and
    ++ * the former should be used by internal callers.
       */
    --struct list_head* hook_list(const char *hookname);
    -+struct list_head* hook_list(const char *hookname, int allow_unknown);
    + struct list_head *list_hooks(const char *hookname);
    ++struct list_head *list_hooks_gently(const char *hookname);
      
      struct run_hooks_opt
      {
41:  2c8e874158 !  5:  2bbb179962 hook: include hooks from the config
    @@ Commit message
     
      ## Documentation/config/hook.txt ##
     @@
    -+hook.<command>.command::
    -+	A command to execute during the <command> hook event. This can be an
    -+	executable on your device, a oneliner for your shell, or the name of a
    -+	hookcmd. See linkgit:git-hook[1].
    ++hook.<name>.command::
    ++	A command to execute whenever `hook.<name>` is invoked. `<name>` should
    ++	be a unique "friendly" name which you can use to identify this hook
    ++	command. (You can specify when to invoke this command with
    ++	`hook.<name>.event`.) The value can be an executable on your device or a
    ++	oneliner for your shell. If more than one value is specified for the
    ++	same `<name>`, the last value parsed will be the only command executed.
    ++	See linkgit:git-hook[1].
    ++
    ++hook.<name>.event::
    ++	The hook events which should invoke `hook.<name>`. `<name>` should be a
    ++	unique "friendly" name which you can use to identify this hook. The
    ++	value should be the name of a hook event, like "pre-commit" or "update".
    ++	(See linkgit:githooks[5] for a complete list of hooks Git knows about.)
    ++	On the specified event, the associated `hook.<name>.command` will be
    ++	executed. More than one event can be specified if you wish for
    ++	`hook.<name>` to execute on multiple events. See linkgit:git-hook[1].
     +
      hook.jobs::
      	Specifies how many hooks can be run simultaneously during parallelized
    @@ Documentation/git-hook.txt: Git is unlikely to use for a native hook later on. F
     +    event = pre-commit
     +    command = ~/bin/linter --c
     +
    -+Conmmands are run in the order Git encounters their associated
    ++In this example, `[hook "linter"]` represents one script - `~/bin/linter --c` -
    ++which can be shared by many repos, and even by many hook events, if appropriate.
    ++
    ++Commands are run in the order Git encounters their associated
     +`hook.<name>.event` configs during the configuration parse (see
    -+linkgit:git-config[1]).
    ++linkgit:git-config[1]). Although multiple `hook.linter.event` configs can be
    ++added, only one `hook.linter.command` event is valid - Git uses "last-one-wins"
    ++to determine which command to run.
    ++
    ++So if you wanted your linter to run when you commit as well as when you push,
    ++you would configure it like so:
    ++
    ++  [hook "linter"]
    ++    event = pre-commit
    ++    event = pre-push
    ++    command = ~/bin/linter --c
    ++
    ++With this config, `~/bin/linter --c` would be run by Git before a commit is
    ++generated (during `pre-commit`) as well as before a push is performed (during
    ++`pre-push`).
    ++
    ++And if you wanted to run your linter as well as a secret-leak detector during
    ++only the "pre-commit" hook event, you would configure it instead like so:
    ++
    ++  [hook "linter"]
    ++    event = pre-commit
    ++    command = ~/bin/linter --c
    ++  [hook "no-leaks"]
    ++    event = pre-commit
    ++    command = ~/bin/leak-detector
    ++
    ++With this config, before a commit is generated (during `pre-commit`), Git would
    ++first start `~/bin/linter --c` and second start `~/bin/leak-detector`. It would
    ++evaluate the output of each when deciding whether to proceed with the commit.
    ++
    ++For a full list of hook events which you can set your `hook.<name>.event` to,
    ++and how hooks are invoked during those events, see linkgit:githooks[5].
     +
     +In general, when instructions suggest adding a script to
     +`.git/hooks/<hook-event>`, you can specify it in the config instead by running
    @@ Documentation/git-hook.txt: Git is unlikely to use for a native hook later on. F
      optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
     
      ## builtin/hook.c ##
    -@@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
    - 	head = hook_list(hookname, 1);
    - 
    - 	if (list_empty(head)) {
    --		printf(_("no commands configured for hook '%s'\n"),
    -+		printf(_("no hooks configured for event '%s'\n"),
    - 		       hookname);
    - 		return 0;
    - 	}
     @@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
      		struct hook *item = list_entry(pos, struct hook, list);
      		item = list_entry(pos, struct hook, list);
    @@ hook.c: static void free_hook(struct hook *ptr)
      static void remove_hook(struct list_head *to_remove)
      {
      	struct hook *hook_to_remove = list_entry(to_remove, struct hook, list);
    -@@ hook.c: const char *find_hook_gently(const char *name)
    - 
    - int hook_exists(const char *name)
    - {
    --	return !!find_hook(name);
    -+	return !list_empty(hook_list(name, 0));
    - }
    +@@ hook.c: int hook_exists(const char *name)
      
      struct hook_config_cb
      {
    @@ hook.c: const char *find_hook_gently(const char *name)
      	struct list_head *list;
      };
      
    --struct list_head* hook_list(const char* hookname, int allow_unknown)
     +/*
     + * Callback for git_config which adds configured hooks to a hook list.  Hooks
     + * can be configured by specifying both hook.<friend-name>.command = <path> and
    @@ hook.c: const char *find_hook_gently(const char *name)
     +	return 0;
     +}
     +
    -+struct list_head* hook_list(const char *hookname, int allow_unknown)
    + struct list_head *list_hooks(const char *hookname)
    + {
    + 	if (!known_hook(hookname))
    +@@ hook.c: struct list_head *list_hooks(const char *hookname)
    + struct list_head *list_hooks_gently(const char *hookname)
      {
      	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
    --	const char *hook_path;
     +	struct hook_config_cb cb_data = {
     +		.hook_event = hookname,
     +		.list = hook_head,
     +	};
      
    -+	if (!allow_unknown && !known_hook(hookname))
    -+		die(_("Don't recognize hook event '%s'. "
    -+		      "Is it documented in 'githooks.txt'?"),
    -+		      hookname);
    - 
      	INIT_LIST_HEAD(hook_head);
      
      	if (!hookname)
    - 		return NULL;
    + 		BUG("null hookname was provided to hook_list()!");
      
    --	if (allow_unknown)
    --		hook_path = find_hook_gently(hookname);
    --	else
    --		hook_path = find_hook(hookname);
    +-	if (have_git_dir()) {
    +-		const char *hook_path = find_hook_gently(hookname);
    +-		if (hook_path) {
    +-			struct hook *to_add = xmalloc(sizeof(*to_add));
    +-			to_add->hook_path = hook_path;
    +-			to_add->feed_pipe_cb_data = NULL;
    +-			list_add_tail(&to_add->list, hook_head);
    +-		}
    +-	}
     +	/* Add the hooks from the config, e.g. hook.myhook.event = pre-commit */
     +	git_config(hook_config_lookup, &cb_data);
    - 
    --	/* Add the hook from the hookdir */
    --	if (hook_path) {
    --		struct hook *to_add = xmalloc(sizeof(*to_add));
    --		to_add->hook_path = hook_path;
    --		to_add->feed_pipe_cb_data = NULL;
    --		list_add_tail(&to_add->list, hook_head);
    --	}
    ++
     +	/* Add the hook from the hookdir. The placeholder makes it easier to
     +	 * allocate work in pick_next_hook. */
     +	if (find_hook_gently(hookname))
    @@ hook.c: static int notify_start_failure(struct strbuf *out,
      
      	return 1;
      }
    -@@ hook.c: int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
    - 		BUG("choose only one method to populate stdin");
    - 
    - 	/*
    --	 * 'git hooks run <hookname>' uses run_found_hooks, so we don't need to
    -+	 * 'git hooks run <hookname>' uses run_found_hooks, and we want to make
    -+	 * sure internal callers are using known hooks, so we don't need to
    - 	 * allow unknown hooknames here.
    - 	 */
    - 	hooks = hook_list(hook_name, 0);
     
      ## hook.h ##
     @@ hook.h: int hook_exists(const char *hookname);
42:  3216e51b6b !  6:  30ffe98601 hook: allow out-of-repo 'git hook' invocations
    @@ git.c: static struct cmd_struct commands[] = {
      	{ "init-db", cmd_init_db },
     
      ## hook.c ##
    -@@ hook.c: struct list_head* hook_list(const char *hookname, int allow_unknown)
    +@@ hook.c: struct list_head *list_hooks_gently(const char *hookname)
      
      	/* Add the hook from the hookdir. The placeholder makes it easier to
      	 * allocate work in pick_next_hook. */
-- 
2.33.0.rc2.250.ged5fa647cd-goog


^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH v3 1/6] hook: run a list of hooks instead
  2021-08-19  3:34               ` [PATCH v3 " Emily Shaffer
@ 2021-08-19  3:34                 ` Emily Shaffer
  2021-08-24 14:56                   ` Ævar Arnfjörð Bjarmason
  2021-08-19  3:34                 ` [PATCH v3 2/6] hook: allow parallel hook execution Emily Shaffer
                                   ` (6 subsequent siblings)
  7 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-08-19  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

To prepare for multihook support, teach hook.[hc] to take a list of
hooks at run_hooks and run_found_hooks. Right now the list is always one
entry, but in the future we will allow users to supply more than one
executable for a single hook event.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 builtin/hook.c |  14 ++++---
 hook.c         | 104 +++++++++++++++++++++++++++++++++++--------------
 hook.h         |  16 +++++++-
 3 files changed, 96 insertions(+), 38 deletions(-)

diff --git a/builtin/hook.c b/builtin/hook.c
index 27dce6a2f0..e18357ba1f 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -25,7 +25,8 @@ static int run(int argc, const char **argv, const char *prefix)
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	int ignore_missing = 0;
 	const char *hook_name;
-	const char *hook_path;
+	struct list_head *hooks;
+
 	struct option run_options[] = {
 		OPT_BOOL(0, "ignore-missing", &ignore_missing,
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
@@ -58,15 +59,16 @@ static int run(int argc, const char **argv, const char *prefix)
 	git_config(git_default_config, NULL);
 
 	hook_name = argv[0];
-	if (ignore_missing)
-		return run_hooks_oneshot(hook_name, &opt);
-	hook_path = find_hook(hook_name);
-	if (!hook_path) {
+	hooks = list_hooks(hook_name);
+	if (list_empty(hooks)) {
+		/* ... act like run_hooks_oneshot() under --ignore-missing */
+		if (ignore_missing)
+			return 0;
 		error("cannot find a hook named %s", hook_name);
 		return 1;
 	}
 
-	ret = run_hooks(hook_name, hook_path, &opt);
+	ret = run_hooks(hook_name, hooks, &opt);
 	run_hooks_opt_clear(&opt);
 	return ret;
 usage:
diff --git a/hook.c b/hook.c
index ee20b2e365..333cfd633a 100644
--- a/hook.c
+++ b/hook.c
@@ -4,6 +4,27 @@
 #include "hook-list.h"
 #include "config.h"
 
+static void free_hook(struct hook *ptr)
+{
+	if (ptr)
+		free(ptr->feed_pipe_cb_data);
+	free(ptr);
+}
+
+static void remove_hook(struct list_head *to_remove)
+{
+	struct hook *hook_to_remove = list_entry(to_remove, struct hook, list);
+	list_del(to_remove);
+	free_hook(hook_to_remove);
+}
+
+void clear_hook_list(struct list_head *head)
+{
+	struct list_head *pos, *tmp;
+	list_for_each_safe(pos, tmp, head)
+		remove_hook(pos);
+}
+
 static int known_hook(const char *name)
 {
 	const char **p;
@@ -68,7 +89,31 @@ const char *find_hook(const char *name)
 
 int hook_exists(const char *name)
 {
-	return !!find_hook(name);
+	return !list_empty(list_hooks(name));
+}
+
+struct list_head *list_hooks(const char *hookname)
+{
+	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
+
+	INIT_LIST_HEAD(hook_head);
+
+	if (!hookname)
+		BUG("null hookname was provided to hook_list()!");
+
+	if (have_git_dir()) {
+		const char *hook_path = find_hook(hookname);
+
+		/* Add the hook from the hookdir */
+		if (hook_path) {
+			struct hook *to_add = xmalloc(sizeof(*to_add));
+			to_add->hook_path = hook_path;
+			to_add->feed_pipe_cb_data = NULL;
+			list_add_tail(&to_add->list, hook_head);
+		}
+	}
+
+	return hook_head;
 }
 
 void run_hooks_opt_clear(struct run_hooks_opt *o)
@@ -128,7 +173,10 @@ static int pick_next_hook(struct child_process *cp,
 	cp->dir = hook_cb->options->dir;
 
 	/* add command */
-	strvec_push(&cp->args, run_me->hook_path);
+	if (hook_cb->options->absolute_path)
+		strvec_push(&cp->args, absolute_path(run_me->hook_path));
+	else
+		strvec_push(&cp->args, run_me->hook_path);
 
 	/*
 	 * add passed-in argv, without expanding - let the user get back
@@ -139,12 +187,12 @@ static int pick_next_hook(struct child_process *cp,
 	/* Provide context for errors if necessary */
 	*pp_task_cb = run_me;
 
-	/*
-	 * This pick_next_hook() will be called again, we're only
-	 * running one hook, so indicate that no more work will be
-	 * done.
-	 */
-	hook_cb->run_me = NULL;
+	/* Get the next entry ready */
+	if (hook_cb->run_me->list.next == hook_cb->head)
+		hook_cb->run_me = NULL;
+	else
+		hook_cb->run_me = list_entry(hook_cb->run_me->list.next,
+					     struct hook, list);
 
 	return 1;
 }
@@ -179,13 +227,9 @@ static int notify_hook_finished(int result,
 	return 0;
 }
 
-int run_hooks(const char *hook_name, const char *hook_path,
-	      struct run_hooks_opt *options)
+int run_hooks(const char *hook_name, struct list_head *hooks,
+		    struct run_hooks_opt *options)
 {
-	struct strbuf abs_path = STRBUF_INIT;
-	struct hook my_hook = {
-		.hook_path = hook_path,
-	};
 	struct hook_cb_data cb_data = {
 		.rc = 0,
 		.hook_name = hook_name,
@@ -197,11 +241,9 @@ int run_hooks(const char *hook_name, const char *hook_path,
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
 
-	if (options->absolute_path) {
-		strbuf_add_absolute_path(&abs_path, hook_path);
-		my_hook.hook_path = abs_path.buf;
-	}
-	cb_data.run_me = &my_hook;
+
+	cb_data.head = hooks;
+	cb_data.run_me = list_first_entry(hooks, struct hook, list);
 
 	run_processes_parallel_tr2(jobs,
 				   pick_next_hook,
@@ -213,18 +255,15 @@ int run_hooks(const char *hook_name, const char *hook_path,
 				   "hook",
 				   hook_name);
 
-
-	if (options->absolute_path)
-		strbuf_release(&abs_path);
-	free(my_hook.feed_pipe_cb_data);
+	clear_hook_list(hooks);
 
 	return cb_data.rc;
 }
 
 int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 {
-	const char *hook_path;
-	int ret;
+	struct list_head *hooks;
+	int ret = 0;
 	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
 
 	if (!options)
@@ -233,14 +272,19 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 	if (options->path_to_stdin && options->feed_pipe)
 		BUG("choose only one method to populate stdin");
 
-	hook_path = find_hook(hook_name);
-	if (!hook_path) {
-		ret = 0;
+	hooks = list_hooks(hook_name);
+
+	/*
+	 * If you need to act on a missing hook, use run_found_hooks()
+	 * instead
+	 */
+	if (list_empty(hooks))
 		goto cleanup;
-	}
 
-	ret = run_hooks(hook_name, hook_path, options);
+	ret = run_hooks(hook_name, hooks, options);
+
 cleanup:
 	run_hooks_opt_clear(options);
+	clear_hook_list(hooks);
 	return ret;
 }
diff --git a/hook.h b/hook.h
index 58dfbf474c..12b56616bb 100644
--- a/hook.h
+++ b/hook.h
@@ -3,6 +3,7 @@
 #include "strbuf.h"
 #include "strvec.h"
 #include "run-command.h"
+#include "list.h"
 
 /*
  * Returns the path to the hook file, or NULL if the hook is missing
@@ -17,6 +18,7 @@ const char *find_hook(const char *name);
 int hook_exists(const char *hookname);
 
 struct hook {
+	struct list_head list;
 	/* The path to the hook */
 	const char *hook_path;
 
@@ -27,6 +29,12 @@ struct hook {
 	void *feed_pipe_cb_data;
 };
 
+/*
+ * Provides a linked list of 'struct hook' detailing commands which should run
+ * in response to the 'hookname' event, in execution order.
+ */
+struct list_head *list_hooks(const char *hookname);
+
 struct run_hooks_opt
 {
 	/* Environment vars to be set for each hook */
@@ -97,6 +105,7 @@ struct hook_cb_data {
 	/* rc reflects the cumulative failure state */
 	int rc;
 	const char *hook_name;
+	struct list_head *head;
 	struct hook *run_me;
 	struct run_hooks_opt *options;
 	int *invoked_hook;
@@ -110,8 +119,8 @@ void run_hooks_opt_clear(struct run_hooks_opt *o);
  *
  * See run_hooks_oneshot() for the simpler one-shot API.
  */
-int run_hooks(const char *hookname, const char *hook_path,
-	      struct run_hooks_opt *options);
+int run_hooks(const char *hookname, struct list_head *hooks,
+		    struct run_hooks_opt *options);
 
 /**
  * Calls find_hook() on your "hook_name" and runs the hooks (if any)
@@ -123,4 +132,7 @@ int run_hooks(const char *hookname, const char *hook_path,
  */
 int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
 
+/* Empties the list at 'head', calling 'free_hook()' on each entry */
+void clear_hook_list(struct list_head *head);
+
 #endif
-- 
2.33.0.rc2.250.ged5fa647cd-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v3 2/6] hook: allow parallel hook execution
  2021-08-19  3:34               ` [PATCH v3 " Emily Shaffer
  2021-08-19  3:34                 ` [PATCH v3 1/6] hook: run a list of hooks instead Emily Shaffer
@ 2021-08-19  3:34                 ` Emily Shaffer
  2021-08-24 15:01                   ` Ævar Arnfjörð Bjarmason
  2021-08-19  3:34                 ` [PATCH v3 3/6] hook: introduce "git hook list" Emily Shaffer
                                   ` (5 subsequent siblings)
  7 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-08-19  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer, Ævar Arnfjörð Bjarmason

In many cases, there's no reason not to allow hooks to execute in
parallel, if more than one was provided. hook.c already calls
run_processes_parallel(), so all we need to do is allow the job count we
hand to run_processes_parallel() to be greater than 1.

If users have specified no alternative, we can use the processor count
from online_cpus() to run an efficient number of tasks at once. However,
users can also override this number if they want by configuring
'hook.jobs'.

To avoid looking up 'hook.jobs' in cases where we don't end up with any
hooks to run anyways, teach the hook runner commands to notice if
.jobs==0 and do a config or online_cpus() lookup if so, when we already
know we have jobs to run.

Serial execution can still be performed by setting .jobs == 1.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/config/hook.txt |  4 ++++
 Documentation/git-hook.txt    | 17 ++++++++++++++++-
 builtin/am.c                  |  4 ++--
 builtin/checkout.c            |  2 +-
 builtin/clone.c               |  2 +-
 builtin/hook.c                |  4 +++-
 builtin/merge.c               |  2 +-
 builtin/rebase.c              |  2 +-
 builtin/receive-pack.c        |  9 +++++----
 builtin/worktree.c            |  2 +-
 commit.c                      |  2 +-
 hook.c                        | 36 +++++++++++++++++++++++++++++++----
 hook.h                        | 24 ++++++++++++++++++-----
 read-cache.c                  |  2 +-
 refs.c                        |  2 +-
 reset.c                       |  3 ++-
 sequencer.c                   |  4 ++--
 transport.c                   |  2 +-
 18 files changed, 94 insertions(+), 29 deletions(-)
 create mode 100644 Documentation/config/hook.txt

diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
new file mode 100644
index 0000000000..96d3d6572c
--- /dev/null
+++ b/Documentation/config/hook.txt
@@ -0,0 +1,4 @@
+hook.jobs::
+	Specifies how many hooks can be run simultaneously during parallelized
+	hook execution. If unspecified, defaults to the number of processors on
+	the current system.
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index fa68c1f391..8bf82b5dd4 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -8,7 +8,8 @@ git-hook - run git hooks
 SYNOPSIS
 --------
 [verse]
-'git hook' run [--to-stdin=<path>] [--ignore-missing] <hook-name> [-- <hook-args>]
+'git hook' run [--to-stdin=<path>] [--ignore-missing] [(-j|--jobs) <n>]
+	<hook-name> [-- <hook-args>]
 
 DESCRIPTION
 -----------
@@ -42,6 +43,20 @@ OPTIONS
 	tools that want to do a blind one-shot run of a hook that may
 	or may not be present.
 
+-j::
+--jobs::
+	Only valid for `run`.
++
+Specify how many hooks to run simultaneously. If this flag is not specified, use
+the value of the `hook.jobs` config. If the config is not specified, use the
+number of CPUs on the current system. Some hooks may be ineligible for
+parallelization: for example, 'commit-msg' intends hooks modify the commit
+message body and cannot be parallelized.
+
+CONFIGURATION
+-------------
+include::config/hook.txt[]
+
 SEE ALSO
 --------
 linkgit:githooks[5]
diff --git a/builtin/am.c b/builtin/am.c
index 9e3d4d9ab4..c7ffc7eafc 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -446,7 +446,7 @@ static void am_destroy(const struct am_state *state)
 static int run_applypatch_msg_hook(struct am_state *state)
 {
 	int ret;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
 
 	assert(state->msg);
 	strvec_push(&opt.args, am_path(state, "final-commit"));
@@ -467,7 +467,7 @@ static int run_applypatch_msg_hook(struct am_state *state)
  */
 static int run_post_rewrite_hook(const struct am_state *state)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 	strvec_push(&opt.args, "rebase");
 	opt.path_to_stdin = am_path(state, "rewritten");
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 6d69b4c011..2cb5c67f64 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -107,7 +107,7 @@ struct branch_info {
 static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit,
 			      int changed)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
 
 	/* "new_commit" can be NULL when checking out from the index before
 	   a commit exists. */
diff --git a/builtin/clone.c b/builtin/clone.c
index 27fc05ee51..986c3b1932 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -776,7 +776,7 @@ static int checkout(int submodule_progress)
 	struct tree *tree;
 	struct tree_desc t;
 	int err = 0;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_SERIAL;
 
 	if (option_no_checkout)
 		return 0;
diff --git a/builtin/hook.c b/builtin/hook.c
index e18357ba1f..4dd3617876 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -22,7 +22,7 @@ static const char * const builtin_hook_run_usage[] = {
 static int run(int argc, const char **argv, const char *prefix)
 {
 	int i;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
 	int ignore_missing = 0;
 	const char *hook_name;
 	struct list_head *hooks;
@@ -32,6 +32,8 @@ static int run(int argc, const char **argv, const char *prefix)
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
 		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
 			   N_("file to read into hooks' stdin")),
+		OPT_INTEGER('j', "jobs", &opt.jobs,
+			    N_("run up to <n> hooks simultaneously")),
 		OPT_END(),
 	};
 	int ret;
diff --git a/builtin/merge.c b/builtin/merge.c
index 9bd4a2532c..81d7ebbbf6 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -448,7 +448,7 @@ static void finish(struct commit *head_commit,
 		   const struct object_id *new_head, const char *msg)
 {
 	struct strbuf reflog_message = STRBUF_INIT;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	const struct object_id *head = &head_commit->object.oid;
 
 	if (!msg)
diff --git a/builtin/rebase.c b/builtin/rebase.c
index e7c668c99b..ad118c983c 100644
--- a/builtin/rebase.c
+++ b/builtin/rebase.c
@@ -1314,7 +1314,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	char *squash_onto_name = NULL;
 	int reschedule_failed_exec = -1;
 	int allow_preemptive_ff = 1;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	struct option builtin_rebase_options[] = {
 		OPT_STRING(0, "onto", &options.onto_name,
 			   N_("revision"),
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index ebec6f3bb1..7460124b74 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -909,7 +909,7 @@ static int run_receive_hook(struct command *commands,
 			    int skip_broken,
 			    const struct string_list *push_options)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	struct receive_hook_feed_context ctx;
 	struct command *iter = commands;
 
@@ -948,7 +948,7 @@ static int run_receive_hook(struct command *commands,
 
 static int run_update_hook(struct command *cmd)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 	strvec_pushl(&opt.args,
 		     cmd->ref_name,
@@ -1432,7 +1432,8 @@ static const char *push_to_checkout(unsigned char *hash,
 				    struct strvec *env,
 				    const char *work_tree)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
+
 	opt.invoked_hook = invoked_hook;
 
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
@@ -1628,7 +1629,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 static void run_update_post_hook(struct command *commands)
 {
 	struct command *cmd;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 	for (cmd = commands; cmd; cmd = cmd->next) {
 		if (cmd->error_string || cmd->did_not_exist)
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 330867c19b..3090506790 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -382,7 +382,7 @@ static int add_worktree(const char *path, const char *refname,
 	 * is_junk is cleared, but do return appropriate code when hook fails.
 	 */
 	if (!ret && opts->checkout) {
-		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
 
 		strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
 		strvec_pushl(&opt.args,
diff --git a/commit.c b/commit.c
index 842e47beae..a38bd04752 100644
--- a/commit.c
+++ b/commit.c
@@ -1700,7 +1700,7 @@ int run_commit_hook(int editor_is_used, const char *index_file,
 		    int *invoked_hook,
 		    const char *name, ...)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
 	va_list args;
 	const char *arg;
 
diff --git a/hook.c b/hook.c
index 333cfd633a..b8420353fa 100644
--- a/hook.c
+++ b/hook.c
@@ -227,6 +227,28 @@ static int notify_hook_finished(int result,
 	return 0;
 }
 
+/*
+ * Determines how many jobs to use after we know we want to parallelize. First
+ * priority is the config 'hook.jobs' and second priority is the number of CPUs.
+ */
+static int configured_hook_jobs(void)
+{
+	/*
+	 * The config and the CPU count probably won't change during the process
+	 * lifetime, so cache the result in case we invoke multiple hooks during
+	 * one process.
+	 */
+	static int jobs = 0;
+	if (jobs)
+		return jobs;
+
+	if (git_config_get_int("hook.jobs", &jobs))
+		/* if the config isn't set, fall back to CPU count. */
+		jobs = online_cpus();
+
+	return jobs;
+}
+
 int run_hooks(const char *hook_name, struct list_head *hooks,
 		    struct run_hooks_opt *options)
 {
@@ -236,16 +258,18 @@ int run_hooks(const char *hook_name, struct list_head *hooks,
 		.options = options,
 		.invoked_hook = options->invoked_hook,
 	};
-	int jobs = 1;
 
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
 
-
 	cb_data.head = hooks;
 	cb_data.run_me = list_first_entry(hooks, struct hook, list);
 
-	run_processes_parallel_tr2(jobs,
+	/* INIT_PARALLEL sets jobs to 0, so go look up how many to use. */
+	if (!options->jobs)
+		options->jobs = configured_hook_jobs();
+
+	run_processes_parallel_tr2(options->jobs,
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
@@ -264,7 +288,11 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 {
 	struct list_head *hooks;
 	int ret = 0;
-	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
+	/*
+	 * Turn on parallelism by default. Hooks which don't want it should
+	 * specify their options accordingly.
+	 */
+	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 	if (!options)
 		options = &hook_opt_scratch;
diff --git a/hook.h b/hook.h
index 12b56616bb..cd3123d290 100644
--- a/hook.h
+++ b/hook.h
@@ -43,6 +43,13 @@ struct run_hooks_opt
 	/* Args to be passed to each hook */
 	struct strvec args;
 
+	/*
+	 * Number of threads to parallelize across. Set to 0 to use the
+	 * 'hook.jobs' config or, if that config is unset, the number of cores
+	 * on the system.
+	 */
+	int jobs;
+
 	/* Resolve and run the "absolute_path(hook)" instead of
 	 * "hook". Used for "git worktree" hooks
 	 */
@@ -85,11 +92,6 @@ struct run_hooks_opt
 	int *invoked_hook;
 };
 
-#define RUN_HOOKS_OPT_INIT { \
-	.env = STRVEC_INIT, \
-	.args = STRVEC_INIT, \
-}
-
 /*
  * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
  * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'.
@@ -111,6 +113,18 @@ struct hook_cb_data {
 	int *invoked_hook;
 };
 
+#define RUN_HOOKS_OPT_INIT_SERIAL { \
+	.jobs = 1, \
+	.env = STRVEC_INIT, \
+	.args = STRVEC_INIT, \
+}
+
+#define RUN_HOOKS_OPT_INIT_PARALLEL { \
+	.jobs = 0, \
+	.env = STRVEC_INIT, \
+	.args = STRVEC_INIT, \
+}
+
 void run_hooks_opt_clear(struct run_hooks_opt *o);
 
 /**
diff --git a/read-cache.c b/read-cache.c
index 90099ca14d..f157e62531 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -3068,7 +3068,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 {
 	int ret;
 	int was_full = !istate->sparse_index;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 	ret = convert_to_sparse(istate);
 
diff --git a/refs.c b/refs.c
index 73d4a93926..5543b8cdab 100644
--- a/refs.c
+++ b/refs.c
@@ -2062,7 +2062,7 @@ int ref_update_reject_duplicates(struct string_list *refnames,
 static int run_transaction_hook(struct ref_transaction *transaction,
 				const char *state)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
 	int ret = 0, i;
 
diff --git a/reset.c b/reset.c
index 6499bc5127..1941adb771 100644
--- a/reset.c
+++ b/reset.c
@@ -128,7 +128,8 @@ int reset_head(struct repository *r, struct object_id *oid, const char *action,
 					    reflog_head);
 	}
 	if (run_hook) {
-		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
+
 		strvec_pushl(&opt.args,
 			     oid_to_hex(orig ? orig : null_oid()),
 			     oid_to_hex(oid),
diff --git a/sequencer.c b/sequencer.c
index f451e23d0c..30fbe79b8a 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1148,7 +1148,7 @@ int update_head_with_reflog(const struct commit *old_head,
 static int run_rewrite_hook(const struct object_id *oldoid,
 			    const struct object_id *newoid)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	struct strbuf tmp = STRBUF_INIT;
 	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int code;
@@ -4522,7 +4522,7 @@ static int pick_commits(struct repository *r,
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
 			struct child_process notes_cp = CHILD_PROCESS_INIT;
-			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 			notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY);
 			notes_cp.git_cmd = 1;
diff --git a/transport.c b/transport.c
index 4ca8fc0391..33da71a108 100644
--- a/transport.c
+++ b/transport.c
@@ -1204,7 +1204,7 @@ static int run_pre_push_hook(struct transport *transport,
 			     struct ref *remote_refs)
 {
 	int ret = 0;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	struct ref *r;
 	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
 
-- 
2.33.0.rc2.250.ged5fa647cd-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v3 3/6] hook: introduce "git hook list"
  2021-08-19  3:34               ` [PATCH v3 " Emily Shaffer
  2021-08-19  3:34                 ` [PATCH v3 1/6] hook: run a list of hooks instead Emily Shaffer
  2021-08-19  3:34                 ` [PATCH v3 2/6] hook: allow parallel hook execution Emily Shaffer
@ 2021-08-19  3:34                 ` Emily Shaffer
  2021-08-24 15:08                   ` Ævar Arnfjörð Bjarmason
  2021-08-24 15:53                   ` Ævar Arnfjörð Bjarmason
  2021-08-19  3:34                 ` [PATCH v3 4/6] hook: allow running non-native hooks Emily Shaffer
                                   ` (4 subsequent siblings)
  7 siblings, 2 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-19  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

If more than one hook will be run, it may be useful to see a list of
which hooks should be run. At very least, it will be useful for us to
test the semantics of multihooks ourselves.

For now, only list the hooks which will run in the order they will run
in; later, it might be useful to include more information like where the
hooks were configured and whether or not they will run.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-hook.txt |  5 +++++
 builtin/hook.c             | 46 ++++++++++++++++++++++++++++++++++++++
 hook.c                     |  3 +--
 3 files changed, 52 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 8bf82b5dd4..701ada9fc0 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -10,6 +10,7 @@ SYNOPSIS
 [verse]
 'git hook' run [--to-stdin=<path>] [--ignore-missing] [(-j|--jobs) <n>]
 	<hook-name> [-- <hook-args>]
+'git hook' list <hook-name>
 
 DESCRIPTION
 -----------
@@ -30,6 +31,10 @@ optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
 arguments (if any) differ by hook name, see linkgit:githooks[5] for
 what those are.
 
+list::
+	Print a list of hooks which will be run on `<hook-name>` event. If no
+	hooks are configured for that event, print nothing and return 1.
+
 OPTIONS
 -------
 
diff --git a/builtin/hook.c b/builtin/hook.c
index 4dd3617876..d21f303eca 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -8,8 +8,11 @@
 
 #define BUILTIN_HOOK_RUN_USAGE \
 	N_("git hook run [--ignore-missing] [--to-stdin=<path>] <hook-name> [-- <hook-args>]")
+#define BUILTIN_HOOK_LIST_USAGE \
+	N_("git hook list <hook-name>")
 
 static const char * const builtin_hook_usage[] = {
+	BUILTIN_HOOK_LIST_USAGE,
 	BUILTIN_HOOK_RUN_USAGE,
 	NULL
 };
@@ -19,6 +22,47 @@ static const char * const builtin_hook_run_usage[] = {
 	NULL
 };
 
+static const char *const builtin_hook_list_usage[] = {
+	BUILTIN_HOOK_LIST_USAGE,
+	NULL
+};
+
+static int list(int argc, const char **argv, const char *prefix)
+{
+	struct list_head *head, *pos;
+	const char *hookname = NULL;
+	struct strbuf hookdir_annotation = STRBUF_INIT;
+
+	struct option list_options[] = {
+		OPT_END(),
+	};
+
+	argc = parse_options(argc, argv, prefix, list_options,
+			     builtin_hook_list_usage, 0);
+
+	if (argc < 1)
+		usage_msg_opt(_("You must specify a hook event name to list."),
+			      builtin_hook_list_usage, list_options);
+
+	hookname = argv[0];
+
+	head = hook_list(hookname);
+
+	if (list_empty(head))
+		return 1;
+
+	list_for_each(pos, head) {
+		struct hook *item = list_entry(pos, struct hook, list);
+		item = list_entry(pos, struct hook, list);
+		if (item)
+			printf("%s\n", item->hook_path);
+	}
+
+	clear_hook_list(head);
+	strbuf_release(&hookdir_annotation);
+
+	return 0;
+}
 static int run(int argc, const char **argv, const char *prefix)
 {
 	int i;
@@ -88,6 +132,8 @@ int cmd_hook(int argc, const char **argv, const char *prefix)
 	if (!argc)
 		goto usage;
 
+	if (!strcmp(argv[0], "list"))
+		return list(argc, argv, prefix);
 	if (!strcmp(argv[0], "run"))
 		return run(argc, argv, prefix);
 
diff --git a/hook.c b/hook.c
index b8420353fa..b1ea372e15 100644
--- a/hook.c
+++ b/hook.c
@@ -96,6 +96,7 @@ struct list_head *list_hooks(const char *hookname)
 {
 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
 
+
 	INIT_LIST_HEAD(hook_head);
 
 	if (!hookname)
@@ -103,8 +104,6 @@ struct list_head *list_hooks(const char *hookname)
 
 	if (have_git_dir()) {
 		const char *hook_path = find_hook(hookname);
-
-		/* Add the hook from the hookdir */
 		if (hook_path) {
 			struct hook *to_add = xmalloc(sizeof(*to_add));
 			to_add->hook_path = hook_path;
-- 
2.33.0.rc2.250.ged5fa647cd-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v3 4/6] hook: allow running non-native hooks
  2021-08-19  3:34               ` [PATCH v3 " Emily Shaffer
                                   ` (2 preceding siblings ...)
  2021-08-19  3:34                 ` [PATCH v3 3/6] hook: introduce "git hook list" Emily Shaffer
@ 2021-08-19  3:34                 ` Emily Shaffer
  2021-08-24 15:55                   ` Ævar Arnfjörð Bjarmason
  2021-08-19  3:34                 ` [PATCH v3 5/6] hook: include hooks from the config Emily Shaffer
                                   ` (3 subsequent siblings)
  7 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-08-19  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

As the hook architecture and 'git hook run' become more featureful, we
may find wrappers wanting to use the hook architecture to run their own
hooks, thereby getting nice things like parallelism and idiomatic Git
configuration for free. Enable this by letting 'git hook run' bypass the
known_hooks() check.

We do still want to keep known_hooks() around, though - by die()ing when
an internal Git call asks for run_hooks("my-new-hook"), we can remind
Git developers to update Documentation/githooks.txt with their new hook,
which in turn helps Git users discover this new hook.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-hook.txt |  8 ++++++++
 builtin/hook.c             |  4 ++--
 hook.c                     | 35 +++++++++++++++++++++++++++++++----
 hook.h                     | 14 ++++++++++++++
 4 files changed, 55 insertions(+), 6 deletions(-)

diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 701ada9fc0..d1db084e4f 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -19,6 +19,14 @@ This command is an interface to git hooks (see linkgit:githooks[5]).
 Currently it only provides a convenience wrapper for running hooks for
 use by git itself. In the future it might gain other functionality.
 
+It's possible to use this command to refer to hooks which are not native to Git,
+for example if a wrapper around Git wishes to expose hooks into its own
+operation in a way which is already familiar to Git users. However, wrappers
+invoking such hooks should be careful to name their hook events something which
+Git is unlikely to use for a native hook later on. For example, Git is much less
+likely to create a `mytool-validate-commit` hook than it is to create a
+`validate-commit` hook.
+
 SUBCOMMANDS
 -----------
 
diff --git a/builtin/hook.c b/builtin/hook.c
index d21f303eca..80397d39f5 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -46,7 +46,7 @@ static int list(int argc, const char **argv, const char *prefix)
 
 	hookname = argv[0];
 
-	head = hook_list(hookname);
+	head = list_hooks_gently(hookname);
 
 	if (list_empty(head))
 		return 1;
@@ -105,7 +105,7 @@ static int run(int argc, const char **argv, const char *prefix)
 	git_config(git_default_config, NULL);
 
 	hook_name = argv[0];
-	hooks = list_hooks(hook_name);
+	hooks = list_hooks_gently(hook_name);
 	if (list_empty(hooks)) {
 		/* ... act like run_hooks_oneshot() under --ignore-missing */
 		if (ignore_missing)
diff --git a/hook.c b/hook.c
index b1ea372e15..ab1e86ddcf 100644
--- a/hook.c
+++ b/hook.c
@@ -51,12 +51,21 @@ static int known_hook(const char *name)
 
 const char *find_hook(const char *name)
 {
-	static struct strbuf path = STRBUF_INIT;
+	const char *hook_path;
 
 	if (!known_hook(name))
-		die(_("the hook '%s' is not known to git, should be in hook-list.h via githooks(5)"),
+		BUG(_("the hook '%s' is not known to git, should be in hook-list.h via githooks(5)"),
 		    name);
 
+	hook_path = find_hook_gently(name);
+
+	return hook_path;
+}
+
+const char *find_hook_gently(const char *name)
+{
+	static struct strbuf path = STRBUF_INIT;
+
 	strbuf_reset(&path);
 	strbuf_git_path(&path, "hooks/%s", name);
 	if (access(path.buf, X_OK) < 0) {
@@ -92,10 +101,24 @@ int hook_exists(const char *name)
 	return !list_empty(list_hooks(name));
 }
 
+struct hook_config_cb
+{
+	struct strbuf *hook_key;
+	struct list_head *list;
+};
+
 struct list_head *list_hooks(const char *hookname)
 {
-	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
+	if (!known_hook(hookname))
+		BUG("Don't recognize hook event '%s'! "
+		    "Is it documented in Documentation/githooks.txt?",
+		    hookname);
+	return list_hooks_gently(hookname);
+}
 
+struct list_head *list_hooks_gently(const char *hookname)
+{
+	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
 
 	INIT_LIST_HEAD(hook_head);
 
@@ -103,7 +126,7 @@ struct list_head *list_hooks(const char *hookname)
 		BUG("null hookname was provided to hook_list()!");
 
 	if (have_git_dir()) {
-		const char *hook_path = find_hook(hookname);
+		const char *hook_path = find_hook_gently(hookname);
 		if (hook_path) {
 			struct hook *to_add = xmalloc(sizeof(*to_add));
 			to_add->hook_path = hook_path;
@@ -299,6 +322,10 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 	if (options->path_to_stdin && options->feed_pipe)
 		BUG("choose only one method to populate stdin");
 
+	/*
+	 * 'git hooks run <hookname>' uses run_hooks, so we don't need to
+	 * allow unknown hooknames here.
+	 */
 	hooks = list_hooks(hook_name);
 
 	/*
diff --git a/hook.h b/hook.h
index cd3123d290..6b7b2d14d2 100644
--- a/hook.h
+++ b/hook.h
@@ -9,8 +9,16 @@
  * Returns the path to the hook file, or NULL if the hook is missing
  * or disabled. Note that this points to static storage that will be
  * overwritten by further calls to find_hook and run_hook_*.
+ *
+ * If the hook is not a native hook (e.g. present in Documentation/githooks.txt)
+ * find_hook() will BUG(). find_hook_gently() does not consult the native hook
+ * list and will check for any hook name in the hooks directory regardless of
+ * whether it is known. find_hook() should be used by internal calls to hooks,
+ * and find_hook_gently() should only be used when the hookname was provided by
+ * the user, such as by 'git hook run my-wrapper-hook'.
  */
 const char *find_hook(const char *name);
+const char *find_hook_gently(const char *name);
 
 /*
  * A boolean version of find_hook()
@@ -32,8 +40,14 @@ struct hook {
 /*
  * Provides a linked list of 'struct hook' detailing commands which should run
  * in response to the 'hookname' event, in execution order.
+ *
+ * list_hooks() checks the provided hookname against Documentation/githooks.txt
+ * and BUG()s if it is not found.  list_hooks_gently() allows any hookname. The
+ * latter should only be used when the hook name is provided by the user, and
+ * the former should be used by internal callers.
  */
 struct list_head *list_hooks(const char *hookname);
+struct list_head *list_hooks_gently(const char *hookname);
 
 struct run_hooks_opt
 {
-- 
2.33.0.rc2.250.ged5fa647cd-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v3 5/6] hook: include hooks from the config
  2021-08-19  3:34               ` [PATCH v3 " Emily Shaffer
                                   ` (3 preceding siblings ...)
  2021-08-19  3:34                 ` [PATCH v3 4/6] hook: allow running non-native hooks Emily Shaffer
@ 2021-08-19  3:34                 ` Emily Shaffer
  2021-08-19 22:39                   ` Junio C Hamano
  2021-08-24 19:30                   ` Ævar Arnfjörð Bjarmason
  2021-08-19  3:34                 ` [PATCH v3 6/6] hook: allow out-of-repo 'git hook' invocations Emily Shaffer
                                   ` (2 subsequent siblings)
  7 siblings, 2 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-19  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach the hook.[hc] library to parse configs to populare the list of
hooks to run for a given event.

Multiple commands can be specified for a given hook by providing
multiple "hook.<friendly-name>.command = <path-to-hook>" and
"hook.<friendly-name>.event = <hook-event>" lines. Hooks will be run in
config order of the "hook.<name>.event" lines.

For example:

  $ git config --list | grep ^hook
  hook.bar.command=~/bar.sh
  hook.bar.event=pre-commit

  $ git hook run
  # Runs ~/bar.sh
  # Runs .git/hooks/pre-commit

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/config/hook.txt |  18 ++++
 Documentation/git-hook.txt    |  57 ++++++++++++-
 builtin/hook.c                |   3 +-
 hook.c                        | 153 ++++++++++++++++++++++++++++++----
 hook.h                        |   7 +-
 t/t1800-hook.sh               | 141 ++++++++++++++++++++++++++++++-
 6 files changed, 357 insertions(+), 22 deletions(-)

diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
index 96d3d6572c..c394756328 100644
--- a/Documentation/config/hook.txt
+++ b/Documentation/config/hook.txt
@@ -1,3 +1,21 @@
+hook.<name>.command::
+	A command to execute whenever `hook.<name>` is invoked. `<name>` should
+	be a unique "friendly" name which you can use to identify this hook
+	command. (You can specify when to invoke this command with
+	`hook.<name>.event`.) The value can be an executable on your device or a
+	oneliner for your shell. If more than one value is specified for the
+	same `<name>`, the last value parsed will be the only command executed.
+	See linkgit:git-hook[1].
+
+hook.<name>.event::
+	The hook events which should invoke `hook.<name>`. `<name>` should be a
+	unique "friendly" name which you can use to identify this hook. The
+	value should be the name of a hook event, like "pre-commit" or "update".
+	(See linkgit:githooks[5] for a complete list of hooks Git knows about.)
+	On the specified event, the associated `hook.<name>.command` will be
+	executed. More than one event can be specified if you wish for
+	`hook.<name>` to execute on multiple events. See linkgit:git-hook[1].
+
 hook.jobs::
 	Specifies how many hooks can be run simultaneously during parallelized
 	hook execution. If unspecified, defaults to the number of processors on
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index d1db084e4f..9c6cbdc2eb 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -27,12 +27,65 @@ Git is unlikely to use for a native hook later on. For example, Git is much less
 likely to create a `mytool-validate-commit` hook than it is to create a
 `validate-commit` hook.
 
+This command parses the default configuration files for pairs of configs like
+so:
+
+  [hook "linter"]
+    event = pre-commit
+    command = ~/bin/linter --c
+
+In this example, `[hook "linter"]` represents one script - `~/bin/linter --c` -
+which can be shared by many repos, and even by many hook events, if appropriate.
+
+Commands are run in the order Git encounters their associated
+`hook.<name>.event` configs during the configuration parse (see
+linkgit:git-config[1]). Although multiple `hook.linter.event` configs can be
+added, only one `hook.linter.command` event is valid - Git uses "last-one-wins"
+to determine which command to run.
+
+So if you wanted your linter to run when you commit as well as when you push,
+you would configure it like so:
+
+  [hook "linter"]
+    event = pre-commit
+    event = pre-push
+    command = ~/bin/linter --c
+
+With this config, `~/bin/linter --c` would be run by Git before a commit is
+generated (during `pre-commit`) as well as before a push is performed (during
+`pre-push`).
+
+And if you wanted to run your linter as well as a secret-leak detector during
+only the "pre-commit" hook event, you would configure it instead like so:
+
+  [hook "linter"]
+    event = pre-commit
+    command = ~/bin/linter --c
+  [hook "no-leaks"]
+    event = pre-commit
+    command = ~/bin/leak-detector
+
+With this config, before a commit is generated (during `pre-commit`), Git would
+first start `~/bin/linter --c` and second start `~/bin/leak-detector`. It would
+evaluate the output of each when deciding whether to proceed with the commit.
+
+For a full list of hook events which you can set your `hook.<name>.event` to,
+and how hooks are invoked during those events, see linkgit:githooks[5].
+
+In general, when instructions suggest adding a script to
+`.git/hooks/<hook-event>`, you can specify it in the config instead by running
+`git config --add hook.<some-name>.command <path-to-script> && git config --add
+hook.<some-name>.event <hook-event>` - this way you can share the script between
+multiple repos. That is, `cp ~/my-script.sh ~/project/.git/hooks/pre-commit`
+would become `git config --add hook.my-script.command ~/my-script.sh && git
+config --add hook.my-script.event pre-commit`.
+
 SUBCOMMANDS
 -----------
 
 run::
-	Run the `<hook-name>` hook. See linkgit:githooks[5] for
-	the hook names we support.
+	Runs hooks configured for `<hook-name>`, in the order they are
+	discovered during the config parse.
 +
 Any positional arguments to the hook should be passed after an
 optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
diff --git a/builtin/hook.c b/builtin/hook.c
index 80397d39f5..50233366a8 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -55,7 +55,8 @@ static int list(int argc, const char **argv, const char *prefix)
 		struct hook *item = list_entry(pos, struct hook, list);
 		item = list_entry(pos, struct hook, list);
 		if (item)
-			printf("%s\n", item->hook_path);
+			printf("%s\n", item->name ? item->name
+						  : _("hook from hookdir"));
 	}
 
 	clear_hook_list(head);
diff --git a/hook.c b/hook.c
index ab1e86ddcf..581d87cbbd 100644
--- a/hook.c
+++ b/hook.c
@@ -11,6 +11,50 @@ static void free_hook(struct hook *ptr)
 	free(ptr);
 }
 
+/*
+ * Walks the linked list at 'head' to check if any hook named 'name'
+ * already exists. Returns a pointer to that hook if so, otherwise returns NULL.
+ */
+static struct hook *find_hook_by_name(struct list_head *head,
+					 const char *name)
+{
+	struct list_head *pos = NULL, *tmp = NULL;
+	struct hook *found = NULL;
+
+	list_for_each_safe(pos, tmp, head) {
+		struct hook *it = list_entry(pos, struct hook, list);
+		if (!strcmp(it->name, name)) {
+			list_del(pos);
+			found = it;
+			break;
+		}
+	}
+	return found;
+}
+
+/*
+ * Adds a hook if it's not already in the list, or moves it to the tail of the
+ * list if it was already there. name == NULL indicates it's from the hookdir;
+ * just append it in this case.
+ */
+static void append_or_move_hook(struct list_head *head, const char *name)
+{
+	struct hook *to_add = NULL;
+
+	/* if it's not from hookdir, check if the hook is already in the list */
+	if (name)
+		to_add = find_hook_by_name(head, name);
+
+	if (!to_add) {
+		/* adding a new hook, not moving an old one */
+		to_add = xmalloc(sizeof(*to_add));
+		to_add->name = name;
+		to_add->feed_pipe_cb_data = NULL;
+	}
+
+	list_add_tail(&to_add->list, head);
+}
+
 static void remove_hook(struct list_head *to_remove)
 {
 	struct hook *hook_to_remove = list_entry(to_remove, struct hook, list);
@@ -103,10 +147,50 @@ int hook_exists(const char *name)
 
 struct hook_config_cb
 {
-	struct strbuf *hook_key;
+	const char *hook_event;
 	struct list_head *list;
 };
 
+/*
+ * Callback for git_config which adds configured hooks to a hook list.  Hooks
+ * can be configured by specifying both hook.<friend-name>.command = <path> and
+ * hook.<friendly-name>.event = <hook-event>.
+ */
+static int hook_config_lookup(const char *key, const char *value, void *cb_data)
+{
+	struct hook_config_cb *data = cb_data;
+	const char *subsection, *parsed_key;
+	size_t subsection_len = 0;
+	struct strbuf subsection_cpy = STRBUF_INIT;
+
+	/*
+	 * Don't bother doing the expensive parse if there's no
+	 * chance that the config matches 'hook.myhook.event = hook_event'.
+	 */
+	if (!value || strcmp(value, data->hook_event))
+		return 0;
+
+	/* Looking for "hook.friendlyname.event = hook_event" */
+	if (parse_config_key(key,
+			    "hook",
+			    &subsection,
+			    &subsection_len,
+			    &parsed_key) ||
+	    strcmp(parsed_key, "event"))
+		return 0;
+
+	/*
+	 * 'subsection' is a pointer to the internals of 'key', which we don't
+	 * own the memory for. Copy it away to the hook list.
+	 */
+	strbuf_add(&subsection_cpy, subsection, subsection_len);
+
+	append_or_move_hook(data->list, strbuf_detach(&subsection_cpy, NULL));
+
+
+	return 0;
+}
+
 struct list_head *list_hooks(const char *hookname)
 {
 	if (!known_hook(hookname))
@@ -119,21 +203,23 @@ struct list_head *list_hooks(const char *hookname)
 struct list_head *list_hooks_gently(const char *hookname)
 {
 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
+	struct hook_config_cb cb_data = {
+		.hook_event = hookname,
+		.list = hook_head,
+	};
 
 	INIT_LIST_HEAD(hook_head);
 
 	if (!hookname)
 		BUG("null hookname was provided to hook_list()!");
 
-	if (have_git_dir()) {
-		const char *hook_path = find_hook_gently(hookname);
-		if (hook_path) {
-			struct hook *to_add = xmalloc(sizeof(*to_add));
-			to_add->hook_path = hook_path;
-			to_add->feed_pipe_cb_data = NULL;
-			list_add_tail(&to_add->list, hook_head);
-		}
-	}
+	/* Add the hooks from the config, e.g. hook.myhook.event = pre-commit */
+	git_config(hook_config_lookup, &cb_data);
+
+	/* Add the hook from the hookdir. The placeholder makes it easier to
+	 * allocate work in pick_next_hook. */
+	if (find_hook_gently(hookname))
+		append_or_move_hook(hook_head, NULL);
 
 	return hook_head;
 }
@@ -194,11 +280,43 @@ static int pick_next_hook(struct child_process *cp,
 	cp->trace2_hook_name = hook_cb->hook_name;
 	cp->dir = hook_cb->options->dir;
 
+	/* to enable oneliners, let config-specified hooks run in shell.
+	 * config-specified hooks have a name. */
+	cp->use_shell = !!run_me->name;
+
 	/* add command */
-	if (hook_cb->options->absolute_path)
-		strvec_push(&cp->args, absolute_path(run_me->hook_path));
-	else
-		strvec_push(&cp->args, run_me->hook_path);
+	if (run_me->name) {
+		/* ...from config */
+		struct strbuf cmd_key = STRBUF_INIT;
+		char *command = NULL;
+
+		strbuf_addf(&cmd_key, "hook.%s.command", run_me->name);
+		if (git_config_get_string(cmd_key.buf, &command)) {
+			/* TODO test me! */
+			die(_("'hook.%s.command' must be configured "
+			      "or 'hook.%s.event' must be removed; aborting.\n"),
+			    run_me->name, run_me->name);
+		}
+
+		strvec_push(&cp->args, command);
+	} else {
+		/* ...from hookdir. */
+		const char *hook_path = NULL;
+		/*
+		 *
+		 * At this point we are already running, so don't validate
+		 * whether the hook name is known or not.
+		 */
+		hook_path = find_hook_gently(hook_cb->hook_name);
+		if (!hook_path)
+			BUG("hookdir hook in hook list but no hookdir hook present in filesystem");
+
+		if (hook_cb->options->absolute_path)
+			hook_path = absolute_path(hook_path);
+
+		strvec_push(&cp->args, hook_path);
+	}
+
 
 	/*
 	 * add passed-in argv, without expanding - let the user get back
@@ -228,8 +346,11 @@ static int notify_start_failure(struct strbuf *out,
 
 	hook_cb->rc |= 1;
 
-	strbuf_addf(out, _("Couldn't start hook '%s'\n"),
-		    attempted->hook_path);
+	if (attempted->name)
+		strbuf_addf(out, _("Couldn't start hook '%s'\n"),
+		    attempted->name);
+	else
+		strbuf_addstr(out, _("Couldn't start hook from hooks directory\n"));
 
 	return 1;
 }
diff --git a/hook.h b/hook.h
index 6b7b2d14d2..621bd2cde1 100644
--- a/hook.h
+++ b/hook.h
@@ -27,8 +27,11 @@ int hook_exists(const char *hookname);
 
 struct hook {
 	struct list_head list;
-	/* The path to the hook */
-	const char *hook_path;
+	/*
+	 * The friendly name of the hook. NULL indicates the hook is from the
+	 * hookdir.
+	 */
+	const char *name;
 
 	/*
 	 * Use this to keep state for your feed_pipe_fn if you are using
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
index 217db848b3..ef2432f53a 100755
--- a/t/t1800-hook.sh
+++ b/t/t1800-hook.sh
@@ -1,13 +1,29 @@
 #!/bin/bash
 
-test_description='git-hook command'
+test_description='git-hook command and config-managed multihooks'
 
 . ./test-lib.sh
 
+setup_hooks () {
+	test_config hook.ghi.event pre-commit --add
+	test_config hook.ghi.command "/path/ghi" --add
+	test_config_global hook.def.event pre-commit --add
+	test_config_global hook.def.command "/path/def" --add
+}
+
+setup_hookdir () {
+	mkdir .git/hooks
+	write_script .git/hooks/pre-commit <<-EOF
+	echo \"Legacy Hook\"
+	EOF
+	test_when_finished rm -rf .git/hooks
+}
+
 test_expect_success 'git hook usage' '
 	test_expect_code 129 git hook &&
 	test_expect_code 129 git hook run &&
 	test_expect_code 129 git hook run -h &&
+	test_expect_code 129 git hook list -h &&
 	test_expect_code 129 git hook run --unknown 2>err &&
 	grep "unknown option" err
 '
@@ -153,4 +169,127 @@ test_expect_success 'stdin to hooks' '
 	test_cmp expect actual
 '
 
+test_expect_success 'git hook list orders by config order' '
+	setup_hooks &&
+
+	cat >expected <<-EOF &&
+	def
+	ghi
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list reorders on duplicate event declarations' '
+	setup_hooks &&
+
+	# 'def' is usually configured globally; move it to the end by
+	# configuring it locally.
+	test_config hook.def.event "pre-commit" --add &&
+
+	cat >expected <<-EOF &&
+	ghi
+	def
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list shows hooks from the hookdir' '
+	setup_hookdir &&
+
+	cat >expected <<-EOF &&
+	hook from hookdir
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'inline hook definitions execute oneliners' '
+	test_config hook.oneliner.event "pre-commit" &&
+	test_config hook.oneliner.command "echo \"Hello World\"" &&
+
+	echo "Hello World" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'inline hook definitions resolve paths' '
+	write_script sample-hook.sh <<-EOF &&
+	echo \"Sample Hook\"
+	EOF
+
+	test_when_finished "rm sample-hook.sh" &&
+
+	test_config hook.sample-hook.event pre-commit &&
+	test_config hook.sample-hook.command "\"$(pwd)/sample-hook.sh\"" &&
+
+	echo \"Sample Hook\" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'hookdir hook included in git hook run' '
+	setup_hookdir &&
+
+	echo \"Legacy Hook\" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'stdin to multiple hooks' '
+	test_config hook.stdin-a.event "test-hook" --add &&
+	test_config hook.stdin-a.command "xargs -P1 -I% echo a%" --add &&
+	test_config hook.stdin-b.event "test-hook" --add &&
+	test_config hook.stdin-b.command "xargs -P1 -I% echo b%" --add &&
+
+	cat >input <<-EOF &&
+	1
+	2
+	3
+	EOF
+
+	cat >expected <<-EOF &&
+	a1
+	a2
+	a3
+	b1
+	b2
+	b3
+	EOF
+
+	git hook run --to-stdin=input test-hook 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'multiple hooks in series' '
+	test_config hook.series-1.event "test-hook" &&
+	test_config hook.series-1.command "echo 1" --add &&
+	test_config hook.series-2.event "test-hook" &&
+	test_config hook.series-2.command "echo 2" --add &&
+	mkdir .git/hooks &&
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo 3
+	EOF
+
+	cat >expected <<-EOF &&
+	1
+	2
+	3
+	EOF
+
+	git hook run -j1 test-hook 2>actual &&
+	test_cmp expected actual &&
+
+	rm -rf .git/hooks
+'
 test_done
-- 
2.33.0.rc2.250.ged5fa647cd-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v3 6/6] hook: allow out-of-repo 'git hook' invocations
  2021-08-19  3:34               ` [PATCH v3 " Emily Shaffer
                                   ` (4 preceding siblings ...)
  2021-08-19  3:34                 ` [PATCH v3 5/6] hook: include hooks from the config Emily Shaffer
@ 2021-08-19  3:34                 ` Emily Shaffer
  2021-08-24 20:12                   ` Ævar Arnfjörð Bjarmason
  2021-08-24 20:29                 ` [PATCH v3 0/6] config-based hooks restarted Ævar Arnfjörð Bjarmason
  2021-09-09 12:41                 ` [PATCH v4 0/5] " Ævar Arnfjörð Bjarmason
  7 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-08-19  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Since hooks can now be supplied via the config, and a config can be
present without a gitdir via the global and system configs, we can start
to allow 'git hook run' to occur without a gitdir. This enables us to do
things like run sendemail-validate hooks when running 'git send-email'
from a nongit directory.

It still doesn't make sense to look for hooks in the hookdir in nongit
repos, though, as there is no hookdir.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---

Notes:
    For hookdir hooks, do we want to run them in nongit dir when core.hooksPath
    is set? For example, if someone set core.hooksPath in their global config and
    then ran 'git hook run sendemail-validate' in a nongit dir?

 git.c           |  2 +-
 hook.c          |  2 +-
 t/t1800-hook.sh | 20 +++++++++++++++-----
 3 files changed, 17 insertions(+), 7 deletions(-)

diff --git a/git.c b/git.c
index 540909c391..39988ee3b0 100644
--- a/git.c
+++ b/git.c
@@ -538,7 +538,7 @@ static struct cmd_struct commands[] = {
 	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
 	{ "hash-object", cmd_hash_object },
 	{ "help", cmd_help },
-	{ "hook", cmd_hook, RUN_SETUP },
+	{ "hook", cmd_hook, RUN_SETUP_GENTLY },
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "init", cmd_init_db },
 	{ "init-db", cmd_init_db },
diff --git a/hook.c b/hook.c
index 581d87cbbd..2e08156546 100644
--- a/hook.c
+++ b/hook.c
@@ -218,7 +218,7 @@ struct list_head *list_hooks_gently(const char *hookname)
 
 	/* Add the hook from the hookdir. The placeholder makes it easier to
 	 * allocate work in pick_next_hook. */
-	if (find_hook_gently(hookname))
+	if (have_git_dir() && find_hook_gently(hookname))
 		append_or_move_hook(hook_head, NULL);
 
 	return hook_head;
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
index ef2432f53a..a7e45c0d16 100755
--- a/t/t1800-hook.sh
+++ b/t/t1800-hook.sh
@@ -118,15 +118,25 @@ test_expect_success 'git hook run -- pass arguments' '
 	test_cmp expect actual
 '
 
-test_expect_success 'git hook run -- out-of-repo runs excluded' '
-	write_script .git/hooks/test-hook <<-EOF &&
-	echo Test hook
-	EOF
+test_expect_success 'git hook run: out-of-repo runs execute global hooks' '
+	test_config_global hook.global-hook.event test-hook --add &&
+	test_config_global hook.global-hook.command "echo no repo no problems" --add &&
+
+	echo "global-hook" >expect &&
+	nongit git hook list test-hook >actual &&
+	test_cmp expect actual &&
+
+	echo "no repo no problems" >expect &&
 
-	nongit test_must_fail git hook run test-hook
+	nongit git hook run test-hook 2>actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo Test hook
+	EOF
+
 	mkdir my-hooks &&
 	write_script my-hooks/test-hook <<-\EOF &&
 	echo Hook ran $1 >>actual
-- 
2.33.0.rc2.250.ged5fa647cd-goog


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 5/6] hook: include hooks from the config
  2021-08-19  3:34                 ` [PATCH v3 5/6] hook: include hooks from the config Emily Shaffer
@ 2021-08-19 22:39                   ` Junio C Hamano
  2021-08-19 23:43                     ` Emily Shaffer
  2021-08-24 19:30                   ` Ævar Arnfjörð Bjarmason
  1 sibling, 1 reply; 479+ messages in thread
From: Junio C Hamano @ 2021-08-19 22:39 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
> index 96d3d6572c..c394756328 100644
> --- a/Documentation/config/hook.txt
> +++ b/Documentation/config/hook.txt
> @@ -1,3 +1,21 @@
> +hook.<name>.command::
> +	A command to execute whenever `hook.<name>` is invoked. `<name>` should
> +	be a unique "friendly" name which you can use to identify this hook
> +	command. (You can specify when to invoke this command with
> +	`hook.<name>.event`.) The value can be an executable on your device or a
> +	oneliner for your shell. If more than one value is specified for the
> +	same `<name>`, the last value parsed will be the only command executed.
> +	See linkgit:git-hook[1].
> +
> +hook.<name>.event::
> +	The hook events which should invoke `hook.<name>`. `<name>` should be a
> +	unique "friendly" name which you can use to identify this hook. The
> +	value should be the name of a hook event, like "pre-commit" or "update".
> +	(See linkgit:githooks[5] for a complete list of hooks Git knows about.)
> +	On the specified event, the associated `hook.<name>.command` will be
> +	executed. More than one event can be specified if you wish for
> +	`hook.<name>` to execute on multiple events. See linkgit:git-hook[1].

Looking much better.  It now gives enough information to readers to
understand (if not enough to convince that it is a good idea) why an
indirection with "friendly name" between the event and command is
there.  In short, <name> names the command to be run and without
indirection, you'd end up having to write:

    [hook "check-whitespace && spellcheck-log-message"]
	event = pre-commit
    [hook "check-whitespace && spellcheck-log-message"]
	event = another-hookable-event

which may give the same expressiveness (and may even be workable if
the configuration were machine generated) but it is typo-prone, and
a single typo, or even an insignificant whitespace change in the
command, would destroy the grouping of "this command fires upon
these events".

It becomes much less typo prone with the indirection, i.e.

    [hook "logcheck"]
	command = check-whitespace && spellcheck-log-message

    [hook "logcheck"]
	event = pre-commit

    [hook "logcheck"]
	event = another-hookable-event

using the "friendly name", especially if these entries are spread
across different configuration files.

My original question was primarily because I thought the
second-level <name> corresponded to <event>.  If that were the case,
it can trivially be made simpler without making it typo-prone, i.e.

    [hook "pre-commit"]
	command = check-whitespace && spellcheck-log-message

    [hook "another-hookable-event"]
	command = check-whitespace && spellcheck-log-message

since the name of the event would be much shorter than the command
line.  But since we are not grouping per hookable event (to apply
the "last one wins" rule to determine which single command is the
one that gets to run).

Thanks.



^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v4 00/36] Run hooks via "git run hook" & hook library
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (37 preceding siblings ...)
  2021-08-19  0:17             ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Emily Shaffer
@ 2021-08-19 23:40             ` Emily Shaffer
  2021-09-02  7:21               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
  39 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-08-19 23:40 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

On Tue, Aug 03, 2021 at 09:38:26PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> This is a v4 re-roll of the "Base for "config-based-hooks" topic.
>  * Almost all the callers were just "one-shot" callers, I introduced a
>    new run_hooks_oneshot() function for those, which gets rid of the
>    verbosity around memory management, see e.g. the "builtin/gc.c" in
>    the range-diff below. That run_hooks_oneshot() can also take a NULL
>    set of options.

I discussed this elswhere, but just to make doubly sure that comment
doesn't get lost, here's a link to my mail:
https://lore.kernel.org/git/YR2jLdYQA5CVzX5h%40google.com

> Range-diff against v3:
>  1:  27c94247f87 =  1:  81fe1ed90d5 Makefile: mark "check" target as .PHONY
>  2:  6e164edb0b0 !  2:  0f749530777 Makefile: stop hardcoding {command,config}-list.h
>  3:  ddae86802e2 !  3:  644b31fe281 Makefile: remove an out-of-date comment
>  4:  58c37e4f06e =  4:  89c4d44b0c3 hook.[ch]: move find_hook() to this new library
>  5:  0cf7e078ef4 =  5:  3514e0c0251 hook.c: add a hook_exists() wrapper and use it in bugreport.c
>  -:  ----------- >  6:  d5ef40f77dc hook.c users: use "hook_exists()" insted of "find_hook()"
>  6:  f343fc7ae66 !  7:  4cfd72722c1 hook-list.h: add a generated list of hooks, like config-list.h

Since these 6 were broken out and then brought back into this series,
I'll review them individually downthread.

>  7:  cf4b06bfdf8 !  8:  7cb4a4cb69e hook: add 'run' subcommand
>     @@ builtin/hook.c (new)
>      +#include "strbuf.h"
>      +#include "strvec.h"
>      +
>     ++#define BUILTIN_HOOK_RUN_USAGE \
>     ++	N_("git hook run <hook-name> [-- <hook-args>]")
>     ++

Nice, now we avoid string duplication here...

>      +static const char * const builtin_hook_usage[] = {
>     -+	N_("git hook <command> [...]"),

...and there is no point including this vague thing that has a more
specific description right after. Ok.

>     -+	N_("git hook run <hook-name> [-- <hook-args>]"),
>     ++	BUILTIN_HOOK_RUN_USAGE,
>      +	NULL
>      +};
[...]
>     @@ builtin/hook.c (new)
>      +{
>      +	int i;
>      +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
>     -+	int rc = 0;
>      +	const char *hook_name;
>      +	const char *hook_path;
>     -+
>      +	struct option run_options[] = {
>      +		OPT_END(),
>      +	};
>     ++	int ret;
>      +
>      +	argc = parse_options(argc, argv, prefix, run_options,
>      +			     builtin_hook_run_usage,
>     -+			     PARSE_OPT_KEEP_UNKNOWN | PARSE_OPT_KEEP_DASHDASH);
>     -+
>     -+	if (argc > 1) {
>     -+		if (strcmp(argv[1], "--") &&
>     -+		    strcmp(argv[1], "--end-of-options"))
>     -+			/* Having a -- for "run" is mandatory */
>     -+			usage_with_options(builtin_hook_usage, run_options);
>     -+		/* Add our arguments, start after -- */
>     -+		for (i = 2 ; i < argc; i++)
>     -+			strvec_push(&opt.args, argv[i]);
>     -+	}
>     ++			     PARSE_OPT_KEEP_DASHDASH);

Nice - this code is being broken up and moved later.

>      +
>     -+	/* Need to take into account core.hooksPath */
>     -+	git_config(git_default_config, NULL);
>     ++	if (!argc)
>     ++		goto usage;
>      +
>      +	/*
>     -+	 * We are not using run_hooks() because we'd like to detect
>     -+	 * missing hooks. Let's find it ourselves and call
>     -+	 * run_found_hooks() instead.

This comment disappears entirely. I am not too terribly upset about it,
since this behavior changes in my series anyway, but it seems like it
might have been an accident?

>     ++	 * Having a -- for "run" when providing <hook-args> is
>     ++	 * mandatory.
>      +	 */
>     ++	if (argc > 1 && strcmp(argv[1], "--") &&
>     ++	    strcmp(argv[1], "--end-of-options"))
>     ++		goto usage;
>     ++
>     ++	/* Add our arguments, start after -- */
>     ++	for (i = 2 ; i < argc; i++)
>     ++		strvec_push(&opt.args, argv[i]);
>     ++
>     ++	/* Need to take into account core.hooksPath */
>     ++	git_config(git_default_config, NULL);
>     ++
>      +	hook_name = argv[0];
>      +	hook_path = find_hook(hook_name);
>      +	if (!hook_path) {
>      +		error("cannot find a hook named %s", hook_name);
>      +		return 1;
>      +	}
>     -+	rc = run_found_hooks(hook_name, hook_path, &opt);
>      +
>     ++	ret = run_hooks(hook_name, hook_path, &opt);
>      +	run_hooks_opt_clear(&opt);
>     -+
>     -+	return rc;
>     ++	return ret;
>     ++usage:
>     ++	usage_with_options(builtin_hook_run_usage, run_options);
>      +}
>      +
>      +int cmd_hook(int argc, const char **argv, const char *prefix)
>     @@ builtin/hook.c (new)
>      +	struct option builtin_hook_options[] = {
>      +		OPT_END(),
>      +	};
>     ++
>      +	argc = parse_options(argc, argv, NULL, builtin_hook_options,
>      +			     builtin_hook_usage, PARSE_OPT_STOP_AT_NON_OPTION);
>      +	if (!argc)
>     -+		usage_with_options(builtin_hook_usage, builtin_hook_options);
>     ++		goto usage;
>      +
>      +	if (!strcmp(argv[0], "run"))
>      +		return run(argc, argv, prefix);
>     -+	else
>     -+		usage_with_options(builtin_hook_usage, builtin_hook_options);
>     ++
>     ++usage:
>     ++	usage_with_options(builtin_hook_usage, builtin_hook_options);
>      +}

The goto pattern seems readable enough elsewhere, though.

>       ## command-list.txt ##
>     @@ hook.c: int hook_exists(const char *name)
>      +	struct hook_cb_data *hook_cb = pp_cb;
>      +	struct hook *run_me = hook_cb->run_me;
>      +
>     ++	if (!run_me)
>     ++		return 0;
>     ++
Now we protect ourselves from the repeated calls to pick_next_hook().
Good.
>      +	cp->no_stdin = 1;
>      +	cp->env = hook_cb->options->env.v;
>      +	cp->stdout_to_stderr = 1;
>     @@ hook.c: int hook_exists(const char *name)
>      +	/* Provide context for errors if necessary */
>      +	*pp_task_cb = run_me;
>      +
>     ++	/*
>     ++	 * This pick_next_hook() will be called again, we're only
>     ++	 * running one hook, so indicate that no more work will be
>     ++	 * done.
>     ++	 */
>     ++	hook_cb->run_me = NULL;
>     ++
And clearly explaining the caller's behavior here. Sure.
>      +	return 1;
>      +}
>      +
>     @@ hook.c: int hook_exists(const char *name)
>      +
>      +	hook_cb->rc |= result;
>      +
>     -+	return 1;
>     ++	return 0;
And finally, report "everything is fine" on task finished. Thanks.
>      +}
>      +
>     -+int run_found_hooks(const char *hook_name, const char *hook_path,
>     -+		    struct run_hooks_opt *options)
>     ++int run_hooks(const char *hook_name, const char *hook_path,
>     ++	      struct run_hooks_opt *options)

Ok. With the rename, we only have "verbose way to call it" and "speedy
way to call it". No more "run_found_hooks()". Thanks, I think I like
this better - I found "run_found_hooks()" to be a little ambiguous.

>     @@ hook.c: int hook_exists(const char *name)
>      +				   hook_name);
>      +
>      +	return cb_data.rc;
>     -+}
>     -+
>     -+int run_hooks(const char *hook_name, struct run_hooks_opt *options)
>     -+{
>     -+	const char *hook_path;
>     -+	int ret;
>     -+	if (!options)
>     -+		BUG("a struct run_hooks_opt must be provided to run_hooks");
>     -+
>     -+	hook_path = find_hook(hook_name);
>     -+
>     -+	/*
>     -+	 * If you need to act on a missing hook, use run_found_hooks()
>     -+	 * instead
>     -+	 */
>     -+	if (!hook_path)
>     -+		return 0;
>     -+
>     -+	ret = run_found_hooks(hook_name, hook_path, options);
>     -+	return ret;

Ok - so we wait for another patch to introduce run_hooks_oneshot. Fine.

>      +}
>      
>       ## hook.h ##
>     @@ hook.h: const char *find_hook(const char *name);
>      +
>      +	/* Args to be passed to each hook */
>      +	struct strvec args;
>     -+
>     -+	/*
>     -+	 * Number of threads to parallelize across, currently a stub,
>     -+	 * we use the parallel API for future-proofing, but we always
>     -+	 * have one hook of a given name, so this is always an
>     -+	 * implicit 1 for now.
>     -+	 */
>     -+	int jobs;

Thanks. I think this makes more sense than the weird "futureproofing"
state it was in before.

>       ## t/t1800-hook.sh (new) ##
>     @@ t/t1800-hook.sh (new)
>      +
>      +test_expect_success 'git hook usage' '
>      +	test_expect_code 129 git hook &&
>     -+	test_expect_code 129 git hook -h &&
>     -+	test_expect_code 129 git hook run -h
>     ++	test_expect_code 129 git hook run &&
>     ++	test_expect_code 129 git hook run -h &&
>     ++	test_expect_code 129 git hook run --unknown 2>err &&
>     ++	grep "unknown option" err
>      +'

Nice. I was able to use this test myself when I added 'git hook list',
too.

The changes to this patch look good to me.

>  8:  7209f73f281 !  9:  2b8500aa675 gc: use hook library for pre-auto-gc hook
>     @@ Metadata
>       ## Commit message ##
>          gc: use hook library for pre-auto-gc hook
>      
>     -    Using the hook.h library instead of the run-command.h library to run
>     -    pre-auto-gc means that those hooks can be set up in config files, as
>     -    well as in the hookdir. pre-auto-gc is called only from builtin/gc.c.
>     +    Move the pre-auto-gc hook away from run-command.h to and over to the
>     +    new hook.h library.
>     +
>     +    To do this introduce a simple run_hooks_oneshot() wrapper, we'll be
>     +    using it extensively for these simple cases of wanting to run a single
>     +    hook under a given name, and having it free the memory we allocate for
>     +    us.

Cool, so we have an excuse for introducing the oneshot wrapper right
away. Thanks.

>     -@@ builtin/gc.c: static void add_repack_incremental_option(void)
>     - 
>     - static int need_to_gc(void)
>     - {
>     -+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
>     -+
>     - 	/*
>     - 	 * Setting gc.auto to 0 or negative can disable the
>     - 	 * automatic gc.
>      @@ builtin/gc.c: static int need_to_gc(void)
>       	else
>       		return 0;
>       
>      -	if (run_hook_le(NULL, "pre-auto-gc", NULL))
>     -+	if (run_hooks("pre-auto-gc", &hook_opt)) {
>     -+		run_hooks_opt_clear(&hook_opt);
>     ++	if (run_hooks_oneshot("pre-auto-gc", NULL))
>       		return 0;
>     -+	}
>     -+	run_hooks_opt_clear(&hook_opt);
>       	return 1;
>       }

Nice - the callsite looks much tidier now.

>     +
>     + ## hook.c ##
>     +@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
>       
>     + 	return cb_data.rc;
>     + }
>     ++
>     ++int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
>     ++{
>     ++	const char *hook_path;
>     ++	int ret;
>     ++	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
>     ++
>     ++	if (!options)
>     ++		options = &hook_opt_scratch;

Ah, it still takes an options (optionally, ha ha) because run_hooks()
would require the caller to provide the hook list/hook path. Ok. Seems
fine, and I like that the '_oneshot' naming change makes it less
surprising that run_hooks_opt_clear() will get called for you.

>     ++
>     ++	hook_path = find_hook(hook_name);
>     ++	if (!hook_path) {
>     ++		ret = 0;
>     ++		goto cleanup;
>     ++	}
>     ++
>     ++	ret = run_hooks(hook_name, hook_path, options);
>     ++cleanup:
>     ++	run_hooks_opt_clear(options);
>     ++	return ret;
>     ++}
>     +
>     + ## hook.h ##
>     +@@ hook.h: void run_hooks_opt_clear(struct run_hooks_opt *o);
>     + /**
>     +  * Takes an already resolved hook found via find_hook() and runs
>     +  * it. Does not call run_hooks_opt_clear() for you.
>     ++ *
>     ++ * See run_hooks_oneshot() for the simpler one-shot API.
>     +  */
>     + int run_hooks(const char *hookname, const char *hook_path,
>     + 	      struct run_hooks_opt *options);
>     ++
>     ++/**
>     ++ * Calls find_hook() on your "hook_name" and runs the hooks (if any)
>     ++ * with run_hooks().
>     ++ *
>     ++ * If "options" is provided calls run_hooks_opt_clear() on it for
>     ++ * you. If "options" is NULL a scratch one will be provided for you
>     ++ * before calling run_hooks().

"A scratch one will be provided for you" doesn't sound quite right -
it's not exposed to the caller at all, but the comment sounds like
you're handing the caller this scratch struct. Maybe it's better to say
"the default options from RUN_HOOKS_OPT_INIT will be used"?

As a bonus, if you directly reference the initter macro, then it will
automatically become clear to folks reading the documentation what the
expected parallelism is on their hook, as the parallelism (_SERIES or
_PARALLEL) is included on the name of the macro in my change later.

>     ++ */
>     ++int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
>     ++
>     + #endif

>  9:  e9a1e7cf61e ! 10:  3ee55d2c10f rebase: teach pre-rebase to use hook.h
>     @@ Metadata
>      Author: Emily Shaffer <emilyshaffer@google.com>
>      
>       ## Commit message ##
>     -    rebase: teach pre-rebase to use hook.h
>     +    rebase: convert pre-rebase to use hook.h
>      
>          Move the pre-rebase hook away from run-command.h to and over to the
>          new hook.h library.
>     @@ builtin/rebase.c: int cmd_rebase(int argc, const char **argv, const char *prefix
>       	if (!ok_to_skip_pre_rebase &&
>      -	    run_hook_le(NULL, "pre-rebase", options.upstream_arg,
>      -			argc ? argv[0] : NULL, NULL))
>     -+	    run_hooks("pre-rebase", &hook_opt)) {
>     -+		run_hooks_opt_clear(&hook_opt);
>     ++	    run_hooks_oneshot("pre-rebase", &hook_opt))

Ok, it just uses the oneshot call instead. Looks good.
> 10:  1d087269303 ! 11:  050f20d14f0 am: convert applypatch hooks to use config
> 11:  32eec5dc2f0 ! 12:  ac875d284da hooks: convert 'post-checkout' hook to hook library
> 12:  e9fa3f67593 ! 13:  69763bc2255 merge: use config-based hooks for post-merge hook

The above all just use the oneshot instead, and have some minor tweaks
to commit messages to lose references to the config. Looks good.

> 13:  12347d901bb ! 14:  2ca1ca1b8e4 git hook run: add an --ignore-missing flag
>     @@ Documentation/git-hook.txt: optional `--` (or `--end-of-options`, see linkgit:gi
>       linkgit:githooks[5]
>      
>       ## builtin/hook.c ##
>     +@@
>     + #include "strvec.h"
>     + 
>     + #define BUILTIN_HOOK_RUN_USAGE \
>     +-	N_("git hook run <hook-name> [-- <hook-args>]")
>     ++	N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")

Ok. We update the usage in the macro here instead. Nice.

>      @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
>     - 	/*
>     - 	 * We are not using run_hooks() because we'd like to detect
>     - 	 * missing hooks. Let's find it ourselves and call
>     --	 * run_found_hooks() instead.
>     -+	 * run_found_hooks() instead...
>     - 	 */
>     + 	git_config(git_default_config, NULL);
>     + 
>       	hook_name = argv[0];
>     ++	if (ignore_missing)
>     ++		return run_hooks_oneshot(hook_name, &opt);
>       	hook_path = find_hook(hook_name);
>       	if (!hook_path) {
>     -+		/* ... act like run_hooks() under --ignore-missing */
>     -+		if (ignore_missing)
>     -+			return 0;
>       		error("cannot find a hook named %s", hook_name);
>     - 		return 1;
>     - 	}

Hm. I actually think this was clearer before. I *think* that is because
you and I have different opinions on the clarity of multiple returns :)

But I think it is a little confusing to say "ok, we will call an
entirely different entry point in case of this one thing which we will
have to check anyways".

> 14:  71d209b4077 ! 15:  5b66b04bec7 send-email: use 'git hook run' for 'sendemail-validate'
>     @@ git-send-email.perl: sub validate_patch {
>       	if ($repo) {
>      +		my $hook_name = 'sendemail-validate';
>       		my $hooks_path = $repo->command_oneline('rev-parse', '--git-path', 'hooks');
>     --		my $validate_hook = catfile($hooks_path,
>     + 		require File::Spec;
>     +-		my $validate_hook = File::Spec->catfile($hooks_path,
>      -					    'sendemail-validate');
>     -+		my $validate_hook = catfile($hooks_path, $hook_name);
>     ++		my $validate_hook = File::Spec->catfile($hooks_path, $hook_name);

Ok, we are adapting to some other change around File::Spec that happened
to git-send-email.perl elsewhere. The change to this patch looks good to
me.

>       		my $hook_error;
>       		if (-x $validate_hook) {
>     - 			my $target = abs_path($fn);
>     + 			require Cwd;
>      @@ git-send-email.perl: sub validate_patch {
>       			chdir($repo->wc_path() or $repo->repo_path())
>       				or die("chdir: $!");
> 15:  246a82b55b2 = 16:  14a37a43db2 git-p4: use 'git hook' to run hooks
> 16:  e3f8482d803 ! 17:  ad5d0e0e7de commit: use hook.h to execute hooks
>     @@ Metadata
>      Author: Emily Shaffer <emilyshaffer@google.com>
>      
>       ## Commit message ##
>     -    commit: use hook.h to execute hooks
>     +    commit: convert {pre-commit,prepare-commit-msg} hook to hook.h
>      
>     -    Teach run_commit_hook() to call hook.h instead of run-command.h. This
>     -    covers 'pre-commit', 'commit-msg', and
>     -    'prepare-commit-msg'.
>     -
>     -    Additionally, ask the hook library - not run-command - whether any
>     -    hooks will be run, as it's possible hooks may exist in the config but
>     -    not the hookdir.
>     -
>     -    Because all but 'post-commit' hooks are expected to make some state
>     -    change, force all but 'post-commit' hook to run in series. 'post-commit'
>     -    "is meant primarily for notification, and cannot affect the outcome of
>     -    `git commit`," so it is fine to run in parallel.

I am a little bummed that the patch reordering effectively removes the
justification for parallel/serial execution for each hook from 'git
blame'. Oh well.

>     +    Move these hooks hook away from run-command.h to and over to the new
>     +    hook.h library.
>      
>          Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
>          Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>      
>       ## commit.c ##

Otherwise the main change here is dealing with other improvements
earlier in the series, like 'hook_exists()' and 'run_hooks_oneshot()'.
Looks fine.

> 17:  6ed61071c5e ! 18:  3d3a33e2674 read-cache: convert post-index-change hook to use config
>     @@ Metadata
>      Author: Emily Shaffer <emilyshaffer@google.com>
>      
>       ## Commit message ##
>     -    read-cache: convert post-index-change hook to use config
>     +    read-cache: convert post-index-change to use hook.h
>      
>     -    By using hook.h instead of run-command.h to run, post-index-change hooks
>     -    can now be specified in the config in addition to the hookdir.
>     -    post-index-change is not run anywhere besides in read-cache.c.
>     +    Move the post-index-change hook away from run-command.h to and over to
>     +    the new hook.h library.

I do not think it is necessary to drop the mention about
'post-index-change' being run anywhere else, but that's just a nit, not
worth fixing :)

>      
>          This removes the last direct user of run_hook_ve(), so we can make the
>          function static now. It'll be removed entirely soon.
>     @@ read-cache.c: static int do_write_locked_index(struct index_state *istate, struc
>      +		     istate->updated_workdir ? "1" : "0",
>      +		     istate->updated_skipworktree ? "1" : "0",
>      +		     NULL);
>     -+	run_hooks("post-index-change", &hook_opt);
>     -+	run_hooks_opt_clear(&hook_opt);
>     ++	run_hooks_oneshot("post-index-change", &hook_opt);
>      +

Mechanical change to using run_hooks_oneshot(). OK.

>       	istate->updated_workdir = 0;
>       	istate->updated_skipworktree = 0;
> 18:  e4ef3f4548a ! 19:  893f8666301 receive-pack: convert push-to-checkout hook to hook.h
>     @@ Metadata
>       ## Commit message ##
>          receive-pack: convert push-to-checkout hook to hook.h
>      
>     -    By using hook.h instead of run-command.h to invoke push-to-checkout,
>     -    hooks can now be specified in the config as well as in the hookdir.
>     -    push-to-checkout is not called anywhere but in builtin/receive-pack.c.
>     +    Move the push-to-checkout hook away from run-command.h to and over to
>     +    the new hook.h library.

You made a comment in the commit before about the final instance of
run_hook_ve; do you want to make a similar one here about run_hook_le?

The range-diff here is mechanical so looks fine to me.

> 19:  e3dda367ec9 = 20:  070433deba5 run-command: remove old run_hook_{le,ve}() hook API
> 20:  477d75bf579 = 21:  1028e0c1667 run-command: allow stdin for run_processes_parallel
> 21:  b7c0ee9719a ! 22:  639e59e9ed0 hook: support passing stdin to hooks
>     @@ Documentation/git-hook.txt: what those are.
>      
>       ## builtin/hook.c ##
>      @@
>     + #include "strvec.h"
>       
>     - static const char * const builtin_hook_usage[] = {
>     - 	N_("git hook <command> [...]"),
>     --	N_("git hook run <hook-name> [-- <hook-args>]"),
>     -+	N_("git hook run [<args>] <hook-name> [-- <hook-args>]"),
>     - 	NULL
>     - };
>     - 
>     - static const char * const builtin_hook_run_usage[] = {
>     - 	N_("git hook run <hook-name> [-- <hook-args>]"),
>     -+	N_("git hook run [--to-stdin=<path>] <hook-name> [-- <hook-args>]"),
>     - 	NULL
>     - };
>     + #define BUILTIN_HOOK_RUN_USAGE \
>     +-	N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")
>     ++	N_("git hook run [--ignore-missing] [--to-stdin=<path>] <hook-name> [-- <hook-args>]")
>       
Just coping with the macro instead of the duplicated strings. Ok.

The range-diff looks fine to me.

> 22:  4035069a98c ! 23:  7d1925cca48 am: convert 'post-rewrite' hook to hook.h
>     @@ builtin/am.c: static int run_applypatch_msg_hook(struct am_state *state)
>       {
>      -	struct child_process cp = CHILD_PROCESS_INIT;
>      -	const char *hook = find_hook("post-rewrite");
>     -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
>     - 	int ret;
>     - 
>     +-	int ret;
>     +-
>      -	if (!hook)
>      -		return 0;
>      -
>     @@ builtin/am.c: static int run_applypatch_msg_hook(struct am_state *state)
>      -	cp.in = xopen(am_path(state, "rewritten"), O_RDONLY);
>      -	cp.stdout_to_stderr = 1;
>      -	cp.trace2_hook_name = "post-rewrite";
>     -+	strvec_push(&opt.args, "rebase");
>     -+	opt.path_to_stdin = am_path(state, "rewritten");
>     ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
>       
>      -	ret = run_command(&cp);
>     -+	ret = run_hooks("post-rewrite", &opt);
>     ++	strvec_push(&opt.args, "rebase");
>     ++	opt.path_to_stdin = am_path(state, "rewritten");
>       
>      -	close(cp.in);
>     -+	run_hooks_opt_clear(&opt);
>     - 	return ret;
>     +-	return ret;
>     ++	return run_hooks_oneshot("post-rewrite", &opt);
>       }
>       
>     + /**

Man, the range-diff did a terrible job of summarizing this :)

The patch itself looks great, though, with the switch to using
run_hooks_oneshot. Thanks.

> 24:  da46c859c1c ! 25:  05d1085f7eb hook: provide stdin by string_list or callback
>     @@ hook.c: static int pick_next_hook(struct child_process *cp,
>       	} else {
>       		cp->no_stdin = 1;
>       	}
>     -@@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
>     - 	run_processes_parallel_tr2(options->jobs,
>     +@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
>     + 	run_processes_parallel_tr2(jobs,
>       				   pick_next_hook,
>       				   notify_start_failure,
>      -				   NULL,
>     @@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
>       				   notify_hook_finished,
>       				   &cb_data,
>       				   "hook",
>     -@@ hook.c: int run_hooks(const char *hook_name, struct run_hooks_opt *options)
>     +@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
>     + 
>     + 	if (options->absolute_path)
>     + 		strbuf_release(&abs_path);
>     ++	free(my_hook.feed_pipe_cb_data);

Nice catch. Thanks.

> 25:  7343be28ef4 ! 26:  4b7175af2e5 hook: convert 'post-rewrite' hook in sequencer.c to hook.h
> 26:  85bf13a0835 ! 27:  3f24e056410 transport: convert pre-push hook to use config
>     @@ Metadata
>      Author: Emily Shaffer <emilyshaffer@google.com>
>      
>       ## Commit message ##
>     -    transport: convert pre-push hook to use config
>     +    transport: convert pre-push hook to hook.h
>      
>     -    By using the hook.h:run_hooks API, pre-push hooks can be specified in
>     -    the config as well as in the hookdir.
>     +    Move the pre-push hook away from run-command.h to and over to the new
>     +    hook.h library.
>      
>          Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
>          Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>     @@ transport.c: static void die_with_unpushed_submodules(struct string_list *needs_
>      -	int ret = 0, x;
>      +	int ret = 0;
>      +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
>     -+	struct strbuf tmp = STRBUF_INIT;
>       	struct ref *r;
>      -	struct child_process proc = CHILD_PROCESS_INIT;
>      -	struct strbuf buf;
>     @@ transport.c: static void die_with_unpushed_submodules(struct string_list *needs_
>      -		finish_command(&proc);
>      -		return -1;
>      -	}
>     --
>     --	sigchain_push(SIGPIPE, SIG_IGN);
>     -+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
>     ++	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
>       
>     +-	sigchain_push(SIGPIPE, SIG_IGN);
>     +-
>      -	strbuf_init(&buf, 256);
>      +	strvec_push(&opt.args, transport->remote->name);
>      +	strvec_push(&opt.args, transport->url);
>       
>       	for (r = remote_refs; r; r = r->next) {
>     ++		struct strbuf buf = STRBUF_INIT;

Ah. It is not being freed because its lifetime is being managed by
to_stdin instead. Ok, cool!

>     ++
>       		if (!r->peer_ref) continue;
>     -@@ transport.c: static int run_pre_push_hook(struct transport *transport,
>     + 		if (r->status == REF_STATUS_REJECT_NONFASTFORWARD) continue;
>     + 		if (r->status == REF_STATUS_REJECT_STALE) continue;
>       		if (r->status == REF_STATUS_REJECT_REMOTE_UPDATED) continue;
>       		if (r->status == REF_STATUS_UPTODATE) continue;
>       
>      -		strbuf_reset(&buf);
>      -		strbuf_addf( &buf, "%s %s %s %s\n",
>     -+		strbuf_reset(&tmp);
>     -+		strbuf_addf(&tmp, "%s %s %s %s",
>     ++		strbuf_addf(&buf, "%s %s %s %s",
>       			 r->peer_ref->name, oid_to_hex(&r->new_oid),
>       			 r->name, oid_to_hex(&r->old_oid));
>      -
>     @@ transport.c: static int run_pre_push_hook(struct transport *transport,
>      -				ret = -1;
>      -			break;
>      -		}
>     -+		string_list_append(&to_stdin, tmp.buf);
>     ++		string_list_append(&to_stdin, strbuf_detach(&buf, NULL));
>       	}
>       
>      -	strbuf_release(&buf);
>     @@ transport.c: static int run_pre_push_hook(struct transport *transport,
>      -	x = finish_command(&proc);
>      -	if (!ret)
>      -		ret = x;
>     -+	ret = run_hooks("pre-push", &opt);
>     -+	run_hooks_opt_clear(&opt);
>     -+	strbuf_release(&tmp);
>     ++	ret = run_hooks_oneshot("pre-push", &opt);
>     ++	to_stdin.strdup_strings = 1;

And this is a typical change to the oneshot function. Cool.

>      +	string_list_clear(&to_stdin, 0);
>       
>       	return ret;
>  -:  ----------- > 28:  ecf75f33233 hook tests: test for exact "pre-push" hook input
>  -:  ----------- > 29:  2c961be94b4 hook tests: use a modern style for "pre-push" tests

I'll take a look at these separately.

> 27:  331014bad17 ! 30:  1ce456f9d9d reference-transaction: use hook.h to run hooks
>     @@ refs.c: int ref_update_reject_duplicates(struct string_list *refnames,
>       				const char *state)
>       {
>      -	struct child_process proc = CHILD_PROCESS_INIT;
>     - 	struct strbuf buf = STRBUF_INIT;
>     +-	struct strbuf buf = STRBUF_INIT;
>      -	const char *hook;
>      +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
>     -+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
>     ++	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
>       	int ret = 0, i;
>     -+	char o[GIT_MAX_HEXSZ + 1], n[GIT_MAX_HEXSZ + 1];
>       
>      -	hook = find_hook("reference-transaction");
>      -	if (!hook)
>     -+	if (!hook_exists("reference-transaction"))
>     - 		return ret;
>     - 
>     +-		return ret;
>     +-
>      -	strvec_pushl(&proc.args, hook, state, NULL);
>      -	proc.in = -1;
>      -	proc.stdout_to_stderr = 1;
>     @@ refs.c: int ref_update_reject_duplicates(struct string_list *refnames,
>      -
>      -	ret = start_command(&proc);
>      -	if (ret)
>     --		return ret;
>     --
>     ++	if (!hook_exists("reference-transaction"))
>     + 		return ret;
>     + 
>      -	sigchain_push(SIGPIPE, SIG_IGN);
>      +	strvec_push(&opt.args, state);
>       
>       	for (i = 0; i < transaction->nr; i++) {
>       		struct ref_update *update = transaction->updates[i];
>     -+		oid_to_hex_r(o, &update->old_oid);
>     -+		oid_to_hex_r(n, &update->new_oid);
>     ++		struct strbuf buf = STRBUF_INIT;

Ah, this is doing the same thing as with 'pre-push'. Cool.

>       
>     - 		strbuf_reset(&buf);
>     +-		strbuf_reset(&buf);
>      -		strbuf_addf(&buf, "%s %s %s\n",
>     --			    oid_to_hex(&update->old_oid),
>     --			    oid_to_hex(&update->new_oid),
>     --			    update->refname);
>     ++		strbuf_addf(&buf, "%s %s %s",
>     + 			    oid_to_hex(&update->old_oid),
>     + 			    oid_to_hex(&update->new_oid),
>     + 			    update->refname);
>      -
>      -		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
>      -			if (errno != EPIPE)
>      -				ret = -1;
>      -			break;
>      -		}
>     -+		strbuf_addf(&buf, "%s %s %s", o, n, update->refname);
>     -+		string_list_append(&to_stdin, buf.buf);
>     ++		string_list_append(&to_stdin, strbuf_detach(&buf, NULL));
>       	}
>       
>      -	close(proc.in);
>      -	sigchain_pop(SIGPIPE);
>     +-	strbuf_release(&buf);
>      +	opt.feed_pipe = pipe_from_string_list;
>      +	opt.feed_pipe_ctx = &to_stdin;
>      +
>     -+	ret = run_hooks("reference-transaction", &opt);
>     -+	run_hooks_opt_clear(&opt);
>     - 	strbuf_release(&buf);
>     ++	ret = run_hooks_oneshot("reference-transaction", &opt);
>     ++	to_stdin.strdup_strings = 1;

And turning on strdup_strings here means we will free them when we call
string_list_clear. Ok.

>      +	string_list_clear(&to_stdin, 0);
>       
>      -	ret |= finish_command(&proc);
> 32:  db70b59b3bd ! 35:  ceef2f3e804 receive-pack: convert receive hooks to hook.h
>     @@ builtin/receive-pack.c: static void hook_output_to_sideband(struct strbuf *outpu
>      +{
>      +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
>      +	struct receive_hook_feed_context ctx;
>     -+	int rc;
>      +	struct command *iter = commands;
>      +
>      +	/* if there are no valid commands, don't invoke the hook at all. */
>     @@ builtin/receive-pack.c: static void hook_output_to_sideband(struct strbuf *outpu
>      +	if (!iter)
>      +		return 0;
>      +
>     -+	/* pre-receive hooks should run in series as the hook updates refs */
>     -+	if (!strcmp(hook_name, "pre-receive"))
>     -+		opt.jobs = 1;
>     -+

Hm, interesting. I'll note this as a case to watch for when I add the
.jobs option back in.


Thanks. I'll take a look at the newly added/copied over patches next.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 5/6] hook: include hooks from the config
  2021-08-19 22:39                   ` Junio C Hamano
@ 2021-08-19 23:43                     ` Emily Shaffer
  2021-08-19 23:48                       ` Junio C Hamano
  0 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-08-19 23:43 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Thu, Aug 19, 2021 at 03:39:06PM -0700, Junio C Hamano wrote:
> 
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
> > index 96d3d6572c..c394756328 100644
> > --- a/Documentation/config/hook.txt
> > +++ b/Documentation/config/hook.txt
> > @@ -1,3 +1,21 @@
> > +hook.<name>.command::
> > +	A command to execute whenever `hook.<name>` is invoked. `<name>` should
> > +	be a unique "friendly" name which you can use to identify this hook
> > +	command. (You can specify when to invoke this command with
> > +	`hook.<name>.event`.) The value can be an executable on your device or a
> > +	oneliner for your shell. If more than one value is specified for the
> > +	same `<name>`, the last value parsed will be the only command executed.
> > +	See linkgit:git-hook[1].
> > +
> > +hook.<name>.event::
> > +	The hook events which should invoke `hook.<name>`. `<name>` should be a
> > +	unique "friendly" name which you can use to identify this hook. The
> > +	value should be the name of a hook event, like "pre-commit" or "update".
> > +	(See linkgit:githooks[5] for a complete list of hooks Git knows about.)
> > +	On the specified event, the associated `hook.<name>.command` will be
> > +	executed. More than one event can be specified if you wish for
> > +	`hook.<name>` to execute on multiple events. See linkgit:git-hook[1].
> 
> Looking much better.  It now gives enough information to readers to
> understand (if not enough to convince that it is a good idea) why an
> indirection with "friendly name" between the event and command is
> there.  In short, <name> names the command to be run and without
> indirection, you'd end up having to write:
> 
>     [hook "check-whitespace && spellcheck-log-message"]
> 	event = pre-commit
>     [hook "check-whitespace && spellcheck-log-message"]
> 	event = another-hookable-event
> 
> which may give the same expressiveness (and may even be workable if
> the configuration were machine generated) but it is typo-prone, and
> a single typo, or even an insignificant whitespace change in the
> command, would destroy the grouping of "this command fires upon
> these events".
> 
> It becomes much less typo prone with the indirection, i.e.
> 
>     [hook "logcheck"]
> 	command = check-whitespace && spellcheck-log-message
> 
>     [hook "logcheck"]
> 	event = pre-commit
> 
>     [hook "logcheck"]
> 	event = another-hookable-event
> 
> using the "friendly name", especially if these entries are spread
> across different configuration files.
> 
> My original question was primarily because I thought the
> second-level <name> corresponded to <event>.  If that were the case,
> it can trivially be made simpler without making it typo-prone, i.e.
> 
>     [hook "pre-commit"]
> 	command = check-whitespace && spellcheck-log-message
> 
>     [hook "another-hookable-event"]
> 	command = check-whitespace && spellcheck-log-message
> 
> since the name of the event would be much shorter than the command
> line.  But since we are not grouping per hookable event (to apply
> the "last one wins" rule to determine which single command is the
> one that gets to run).
> 

To be clear, the config schema did work the way you describe until this
revision. Ævar suggested the change and it seemed like a good idea to me
(and the rest of the Google folks) so I changed between v2 and v3 of the
restart.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 5/6] hook: include hooks from the config
  2021-08-19 23:43                     ` Emily Shaffer
@ 2021-08-19 23:48                       ` Junio C Hamano
  0 siblings, 0 replies; 479+ messages in thread
From: Junio C Hamano @ 2021-08-19 23:48 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> On Thu, Aug 19, 2021 at 03:39:06PM -0700, Junio C Hamano wrote:
>> 
>> My original question was primarily because I thought the
>> second-level <name> corresponded to <event>.  If that were the case,
>> it can trivially be made simpler without making it typo-prone, i.e.
>> 
>>     [hook "pre-commit"]
>> 	command = check-whitespace && spellcheck-log-message
>> 
>>     [hook "another-hookable-event"]
>> 	command = check-whitespace && spellcheck-log-message
>> 
>> since the name of the event would be much shorter than the command
>> line.  But since we are not grouping per hookable event (to apply
>> the "last one wins" rule to determine which single command is the
>> one that gets to run).
>> 
>
> To be clear, the config schema did work the way you describe until this
> revision. Ævar suggested the change and it seemed like a good idea to me
> (and the rest of the Google folks) so I changed between v2 and v3 of the
> restart.

To be clear, you do not want to take the above as my suggestion to
go back to the previous one, since that is not what I meant.  As
long as you and others are happy with what you folks ended up with,
i.e. the current one that uses <name> that is a short-hand for
<command>, that is what matters.

Thanks.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v4 02/36] Makefile: stop hardcoding {command,config}-list.h
  2021-08-03 19:38             ` [PATCH v4 02/36] Makefile: stop hardcoding {command,config}-list.h Ævar Arnfjörð Bjarmason
@ 2021-08-20  0:03               ` Emily Shaffer
  2021-08-24 14:22                 ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-08-20  0:03 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

On Tue, Aug 03, 2021 at 09:38:28PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> Change various places that hardcode the names of these two files to
> refer to either $(GENERATED_H), or to a new generated-hdrs
> target. That target is consistent with the *-objs targets I recently
> added in 029bac01a8 (Makefile: add {program,xdiff,test,git,fuzz}-objs
> & objects targets, 2021-02-23).
> 
> A subsequent commit will add a new generated hook-list.h. By doing
> this refactoring we'll only need to add the new file to the
> GENERATED_H variable, not EXCEPT_HDRS, the vcbuild/README etc.
> 
> I have not tested the Windows-specific change in config.mak.uname
> being made here, but we use other variables from the Makefile in the
> same block, and the GENERATED_H is fully defined before we include
> config.mak.uname.

Is it not something you can get coverage for via, for example, the
GitHub Actions CI suite? I wonder if that means we want some test to
check that these generated lists came together correctly?

Otherwise the diff looks straightforward.
> 
> Hardcoding command-list.h there seems to have been a case of
> copy/paste programming in 976aaedca0 (msvc: add a Makefile target to
> pre-generate the Visual Studio solution, 2019-07-29). The
> config-list.h was added later in 709df95b78 (help: move
> list_config_help to builtin/help, 2020-04-16).
> 
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Emily Shaffer <emilyshaffer@google.com>

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v4 01/36] Makefile: mark "check" target as .PHONY
  2021-08-03 19:38             ` [PATCH v4 01/36] Makefile: mark "check" target as .PHONY Ævar Arnfjörð Bjarmason
@ 2021-08-20  0:04               ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-20  0:04 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

On Tue, Aug 03, 2021 at 09:38:27PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> Fix a bug in 44c9e8594e (Fix up header file dependencies and add
> sparse checking rules, 2005-07-03), we never marked the phony "check"
> target as such.

A simple enough change, and it gave me an excuse to go and look up what
.PHONY actually does. Thanks ;)

> 
> Perhaps we should just remove it, since as of a combination of
> 912f9980d2 (Makefile: help people who run 'make check' by mistake,
> 2008-11-11) 0bcd9ae85d (sparse: Fix errors due to missing
> target-specific variables, 2011-04-21) we've been suggesting the user
> run "make sparse" directly.
> 
> But under that mode it still does something, as well as directing the
> user to run "make test" under non-sparse. So let's punt that and
> narrowly fix the PHONY bug.
> 
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  Makefile | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/Makefile b/Makefile
> index c6f6246bf63..2ff038069e8 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -2931,6 +2931,7 @@ hdr-check: $(HCO)
>  style:
>  	git clang-format --style file --diff --extensions c,h
>  
> +.PHONY: check
>  check: config-list.h command-list.h
>  	@if sparse; \
>  	then \
> -- 
> 2.33.0.rc0.595.ge31e012651d
> 

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v4 03/36] Makefile: remove an out-of-date comment
  2021-08-03 19:38             ` [PATCH v4 03/36] Makefile: remove an out-of-date comment Ævar Arnfjörð Bjarmason
@ 2021-08-20  0:05               ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-20  0:05 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

On Tue, Aug 03, 2021 at 09:38:29PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> This comment added in dfea575017 (Makefile: lazily compute header
> dependencies, 2010-01-26) has been out of date since
> 92b88eba9f (Makefile: use `git ls-files` to list header files, if
> possible, 2019-03-04), when we did exactly what it tells us not to do
> and added $(GENERATED_H) to $(OBJECTS) dependencies.
> 
> The rest of it was also somewhere between inaccurate and outdated,
> since as of b8ba629264 (Makefile: fold MISC_H into LIB_H, 2012-06-20)
> it's not followed by a list of header files, that got moved earlier in
> the file into LIB_H in 60d24dd255 (Makefile: fold XDIFF_H and VCSSVN_H
> into LIB_H, 2012-07-06).
> 
> Let's just remove it entirely, to the extent that we have anything
> useful to say here the comment on the
> "USE_COMPUTED_HEADER_DEPENDENCIES" variable a few lines above this
> change does the job for us.
> 
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  Makefile | 7 -------
>  1 file changed, 7 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 89bf0dd7332..0a540dcd34e 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -2519,13 +2519,6 @@ ifneq ($(dep_files_present),)
>  include $(dep_files_present)
>  endif
>  else
> -# Dependencies on header files, for platforms that do not support
> -# the gcc -MMD option.
> -#
> -# Dependencies on automatically generated headers such as command-list.h
> -# should _not_ be included here, since they are necessary even when
> -# building an object for the first time.
> -
>  $(OBJECTS): $(LIB_H) $(GENERATED_H)
>  endif
>  
> -- 
> 2.33.0.rc0.595.ge31e012651d
> 

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v4 04/36] hook.[ch]: move find_hook() to this new library
  2021-08-03 19:38             ` [PATCH v4 04/36] hook.[ch]: move find_hook() to this new library Ævar Arnfjörð Bjarmason
@ 2021-08-20  0:08               ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-20  0:08 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

On Tue, Aug 03, 2021 at 09:38:30PM +0200, Ævar Arnfjörð Bjarmason wrote:

The subject is now a little confusing (and I guess it would have been
before, too). Can we remove "this new library" and replace it with
something that makes sense on its own?

> 
> Move the find_hook() function from run-command.c to a new hook.c
> library. This change establishes a stub library that's pretty
> pointless right now, but will see much wider use with Emily Shaffer's
> upcoming "configuration-based hooks" series.
> 
> Eventually all the hook related code will live in hook.[ch]. Let's
> start that process by moving the simple find_hook() function over
> as-is.
> 
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

It doesn't make sense for me to add a Reviewed-by line here, I guess.
But it seems to me to be fine to do this early on instead of at the very
end.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v4 05/36] hook.c: add a hook_exists() wrapper and use it in bugreport.c
  2021-08-03 19:38             ` [PATCH v4 05/36] hook.c: add a hook_exists() wrapper and use it in bugreport.c Ævar Arnfjörð Bjarmason
@ 2021-08-20  0:09               ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-20  0:09 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

On Tue, Aug 03, 2021 at 09:38:31PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> Add a boolean version of the find_hook() function for those callers
> who are only interested in checking whether the hook exists, not what
> the path to it is.

Seems fine.

> 
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>  builtin/bugreport.c | 2 +-
>  hook.c              | 5 +++++
>  hook.h              | 5 +++++
>  3 files changed, 11 insertions(+), 1 deletion(-)
> 
> diff --git a/builtin/bugreport.c b/builtin/bugreport.c
> index 596f079a7f9..941c8d5e270 100644
> --- a/builtin/bugreport.c
> +++ b/builtin/bugreport.c
> @@ -82,7 +82,7 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
>  	}
>  
>  	for (i = 0; i < ARRAY_SIZE(hook); i++)
> -		if (find_hook(hook[i]))
> +		if (hook_exists(hook[i]))
>  			strbuf_addf(hook_info, "%s\n", hook[i]);
>  }
>  
> diff --git a/hook.c b/hook.c
> index c4dbef1d0ef..97cd799a320 100644
> --- a/hook.c
> +++ b/hook.c
> @@ -35,3 +35,8 @@ const char *find_hook(const char *name)
>  	}
>  	return path.buf;
>  }
> +
> +int hook_exists(const char *name)
> +{
> +	return !!find_hook(name);

Later on I'll change this to refer to the output of list_hooks()
instead.

> +}
> diff --git a/hook.h b/hook.h
> index 68624f16059..4c547ac15e5 100644
> --- a/hook.h
> +++ b/hook.h
> @@ -8,4 +8,9 @@
>   */
>  const char *find_hook(const char *name);
>  
> +/*
> + * A boolean version of find_hook()
> + */
> +int hook_exists(const char *hookname);
> +
>  #endif
> -- 
> 2.33.0.rc0.595.ge31e012651d
> 

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v4 06/36] hook.c users: use "hook_exists()" insted of "find_hook()"
  2021-08-03 19:38             ` [PATCH v4 06/36] hook.c users: use "hook_exists()" insted of "find_hook()" Ævar Arnfjörð Bjarmason
@ 2021-08-20  0:10               ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-20  0:10 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

On Tue, Aug 03, 2021 at 09:38:32PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> Use the new hook_exists() function instead of find_hook() where the
> latter was called in boolean contexts. This make subsequent changes in
> a series where we further refactor the hook API clearer, as we won't
> conflate wanting to get the path of the hook with checking for its
> existence.
> 

None of these callsites capture the return string from find_hook(), so
this looks fine.

> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  builtin/commit.c       | 2 +-
>  builtin/merge.c        | 2 +-
>  builtin/receive-pack.c | 2 +-
>  sequencer.c            | 2 +-
>  4 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/builtin/commit.c b/builtin/commit.c
> index 51b07ee02ea..aa3c741efa9 100644
> --- a/builtin/commit.c
> +++ b/builtin/commit.c
> @@ -1052,7 +1052,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
>  		return 0;
>  	}
>  
> -	if (!no_verify && find_hook("pre-commit")) {
> +	if (!no_verify && hook_exists("pre-commit")) {
>  		/*
>  		 * Re-read the index as pre-commit hook could have updated it,
>  		 * and write it out as a tree.  We must do this before we invoke
> diff --git a/builtin/merge.c b/builtin/merge.c
> index be98d66b0a8..03f244dd5a0 100644
> --- a/builtin/merge.c
> +++ b/builtin/merge.c
> @@ -849,7 +849,7 @@ static void prepare_to_commit(struct commit_list *remoteheads)
>  	 * and write it out as a tree.  We must do this before we invoke
>  	 * the editor and after we invoke run_status above.
>  	 */
> -	if (find_hook("pre-merge-commit"))
> +	if (hook_exists("pre-merge-commit"))
>  		discard_cache();
>  	read_cache_from(index_file);
>  	strbuf_addbuf(&msg, &merge_msg);
> diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
> index 97aebdc15bd..91fa799b66e 100644
> --- a/builtin/receive-pack.c
> +++ b/builtin/receive-pack.c
> @@ -1464,7 +1464,7 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
>  
>  	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
>  
> -	if (!find_hook(push_to_checkout_hook))
> +	if (!hook_exists(push_to_checkout_hook))
>  		retval = push_to_deploy(sha1, &env, work_tree);
>  	else
>  		retval = push_to_checkout(sha1, &env, work_tree);
> diff --git a/sequencer.c b/sequencer.c
> index ea4199d65a4..9aac08c1545 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -1446,7 +1446,7 @@ static int try_to_commit(struct repository *r,
>  		}
>  	}
>  
> -	if (find_hook("prepare-commit-msg")) {
> +	if (hook_exists("prepare-commit-msg")) {
>  		res = run_prepare_commit_msg_hook(r, msg, hook_commit);
>  		if (res)
>  			goto out;
> -- 
> 2.33.0.rc0.595.ge31e012651d
> 

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v4 28/36] hook tests: test for exact "pre-push" hook input
  2021-08-03 19:38             ` [PATCH v4 28/36] hook tests: test for exact "pre-push" hook input Ævar Arnfjörð Bjarmason
@ 2021-08-20  0:16               ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-20  0:16 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

On Tue, Aug 03, 2021 at 09:38:54PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> Extend the tests added in ec55559f937 (push: Add support for pre-push
> hooks, 2013-01-13) to exhaustively test for the exact input we're
> expecting. This helps a parallel series that's refactoring how the
> hook is called, to e.g. make sure that we don't miss a trailing
> newline

The reference to "a parallel series" I think didn't belong in the
commit-msg in the first place, and doesn't make sense now (because this
is in said series, right?).

> 
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>  t/t5571-pre-push-hook.sh | 23 ++++++++++++++++++-----
>  1 file changed, 18 insertions(+), 5 deletions(-)
> 
> diff --git a/t/t5571-pre-push-hook.sh b/t/t5571-pre-push-hook.sh
> index ad8d5804f7b..d2857a6fbc0 100755
> --- a/t/t5571-pre-push-hook.sh
> +++ b/t/t5571-pre-push-hook.sh
> @@ -11,7 +11,7 @@ HOOKDIR="$(git rev-parse --git-dir)/hooks"
>  HOOK="$HOOKDIR/pre-push"
>  mkdir -p "$HOOKDIR"
>  write_script "$HOOK" <<EOF
> -cat >/dev/null
> +cat >actual
>  exit 0
>  EOF
>  
> @@ -20,10 +20,16 @@ test_expect_success 'setup' '
>  	git init --bare repo1 &&
>  	git remote add parent1 repo1 &&
>  	test_commit one &&
> -	git push parent1 HEAD:foreign
> +	cat >expect <<-EOF &&
> +	HEAD $(git rev-parse HEAD) refs/heads/foreign $(test_oid zero)
> +	EOF
> +
> +	test_when_finished "rm actual" &&
> +	git push parent1 HEAD:foreign &&
> +	test_cmp expect actual
>  '
>  write_script "$HOOK" <<EOF
> -cat >/dev/null
> +cat >actual

Hm, I am not sure I like this. It upsets the usual convention of what
'actual' means ("output I captured from a 'git' command"). It is
nitpicky, but I would be happier to see it cat to some other filename,
like 'received-input'.

>  exit 1
>  EOF
>  
> @@ -32,11 +38,18 @@ export COMMIT1
>  
>  test_expect_success 'push with failing hook' '
>  	test_commit two &&
> -	test_must_fail git push parent1 HEAD
> +	cat >expect <<-EOF &&
> +	HEAD $(git rev-parse HEAD) refs/heads/main $(test_oid zero)
> +	EOF
> +
> +	test_when_finished "rm actual" &&
> +	test_must_fail git push parent1 HEAD &&
> +	test_cmp expect actual
>  '
>  
>  test_expect_success '--no-verify bypasses hook' '
> -	git push --no-verify parent1 HEAD
> +	git push --no-verify parent1 HEAD &&
> +	test_path_is_missing actual

Other than the naming nit, though, the test changes look reasonable to
me to ensure we don't goof up the stdin support in hook.c. Thanks.

With (or, I guess, without) changes,
Reviewed-by: Emily Shaffer <emilyshaffer@google.com>

>  '
>  
>  COMMIT2="$(git rev-parse HEAD)"
> -- 
> 2.33.0.rc0.595.ge31e012651d
> 

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v4 29/36] hook tests: use a modern style for "pre-push" tests
  2021-08-03 19:38             ` [PATCH v4 29/36] hook tests: use a modern style for "pre-push" tests Ævar Arnfjörð Bjarmason
@ 2021-08-20  0:18               ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-20  0:18 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

On Tue, Aug 03, 2021 at 09:38:55PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> Indent the here-docs and use "test_cmp" instead of "diff" in tests
> added in ec55559f937 (push: Add support for pre-push hooks,
> 2013-01-13). Let's also use the more typical "expect" instead of
> "expected" to be consistent with the rest of the test file.
> 
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>  t/t5571-pre-push-hook.sh | 71 ++++++++++++++++++++--------------------
>  1 file changed, 35 insertions(+), 36 deletions(-)
> 
> diff --git a/t/t5571-pre-push-hook.sh b/t/t5571-pre-push-hook.sh
> index d2857a6fbc0..6d0d5b854ea 100755
> --- a/t/t5571-pre-push-hook.sh
> +++ b/t/t5571-pre-push-hook.sh
> @@ -61,15 +61,15 @@ echo "$2" >>actual
>  cat >>actual
>  EOF
>  
> -cat >expected <<EOF
> -parent1
> -repo1
> -refs/heads/main $COMMIT2 refs/heads/foreign $COMMIT1
> -EOF
> -
>  test_expect_success 'push with hook' '
> +	cat >expected <<-EOF &&
> +	parent1
> +	repo1
> +	refs/heads/main $COMMIT2 refs/heads/foreign $COMMIT1
> +	EOF
> +
>  	git push parent1 main:foreign &&
> -	diff expected actual
> +	test_cmp expected actual

Your commit message tells me you're getting rid of 'expected', but here
it is. Should that just be 'expect'?

Otherwise it looks fine.

Reviewed-by: Emily Shaffer <emilyshaffer@google.com>

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v4 02/36] Makefile: stop hardcoding {command,config}-list.h
  2021-08-20  0:03               ` Emily Shaffer
@ 2021-08-24 14:22                 ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-24 14:22 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee


On Thu, Aug 19 2021, Emily Shaffer wrote:

> On Tue, Aug 03, 2021 at 09:38:28PM +0200, Ævar Arnfjörð Bjarmason wrote:
>> 
>> Change various places that hardcode the names of these two files to
>> refer to either $(GENERATED_H), or to a new generated-hdrs
>> target. That target is consistent with the *-objs targets I recently
>> added in 029bac01a8 (Makefile: add {program,xdiff,test,git,fuzz}-objs
>> & objects targets, 2021-02-23).
>> 
>> A subsequent commit will add a new generated hook-list.h. By doing
>> this refactoring we'll only need to add the new file to the
>> GENERATED_H variable, not EXCEPT_HDRS, the vcbuild/README etc.
>> 
>> I have not tested the Windows-specific change in config.mak.uname
>> being made here, but we use other variables from the Makefile in the
>> same block, and the GENERATED_H is fully defined before we include
>> config.mak.uname.
>
> Is it not something you can get coverage for via, for example, the
> GitHub Actions CI suite? I wonder if that means we want some test to
> check that these generated lists came together correctly?

I'll amend this. Yes it's being tested via the CI, and it's the CI that
failed before. I should have said "I have not tested this directly",
i.e. I don't have a local Windows machine, but pushing it to the CI
build that uses this works...

> Otherwise the diff looks straightforward.
>> 
>> Hardcoding command-list.h there seems to have been a case of
>> copy/paste programming in 976aaedca0 (msvc: add a Makefile target to
>> pre-generate the Visual Studio solution, 2019-07-29). The
>> config-list.h was added later in 709df95b78 (help: move
>> list_config_help to builtin/help, 2020-04-16).
>> 
>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> Reviewed-by: Emily Shaffer <emilyshaffer@google.com>


^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 1/6] hook: run a list of hooks instead
  2021-08-19  3:34                 ` [PATCH v3 1/6] hook: run a list of hooks instead Emily Shaffer
@ 2021-08-24 14:56                   ` Ævar Arnfjörð Bjarmason
  2021-08-26 21:16                     ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-24 14:56 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Wed, Aug 18 2021, Emily Shaffer wrote:

> @@ -25,7 +25,8 @@ static int run(int argc, const char **argv, const char *prefix)
>  	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
>  	int ignore_missing = 0;
>  	const char *hook_name;
> -	const char *hook_path;
> +	struct list_head *hooks;
> +
>  	struct option run_options[] = {
>  		OPT_BOOL(0, "ignore-missing", &ignore_missing,
>  			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),

In general in this patch series there's a bunch of little whitespace
changes like that along with other changes. I think it's probably best
if I just absorb that in the "base" topic instead of doing them
here. E.g. in this case we could also add a line between "struct option"
and the rest.

I don't mind either way, but the whitespace churn makes for distracting
reading...

> @@ -58,15 +59,16 @@ static int run(int argc, const char **argv, const char *prefix)
>  	git_config(git_default_config, NULL);
>  
>  	hook_name = argv[0];
> -	if (ignore_missing)
> -		return run_hooks_oneshot(hook_name, &opt);
> -	hook_path = find_hook(hook_name);
> -	if (!hook_path) {
> +	hooks = list_hooks(hook_name);
> +	if (list_empty(hooks)) {
> +		/* ... act like run_hooks_oneshot() under --ignore-missing */
> +		if (ignore_missing)
> +			return 0;
>  		error("cannot find a hook named %s", hook_name);
>  		return 1;
>  	}
>  
> -	ret = run_hooks(hook_name, hook_path, &opt);
> +	ret = run_hooks(hook_name, hooks, &opt);
>  	run_hooks_opt_clear(&opt);
>  	return ret;

This memory management is a bit inconsistent. So here we list_hooks(),
pass it to run_hooks(), which clears it for us, OK...

> -int run_hooks(const char *hook_name, const char *hook_path,
> -	      struct run_hooks_opt *options)
> +int run_hooks(const char *hook_name, struct list_head *hooks,
> +		    struct run_hooks_opt *options)
>  {
> -	struct strbuf abs_path = STRBUF_INIT;
> -	struct hook my_hook = {
> -		.hook_path = hook_path,
> -	};
>  	struct hook_cb_data cb_data = {
>  		.rc = 0,
>  		.hook_name = hook_name,
> @@ -197,11 +241,9 @@ int run_hooks(const char *hook_name, const char *hook_path,
>  	if (!options)
>  		BUG("a struct run_hooks_opt must be provided to run_hooks");
>  
> -	if (options->absolute_path) {
> -		strbuf_add_absolute_path(&abs_path, hook_path);
> -		my_hook.hook_path = abs_path.buf;
> -	}
> -	cb_data.run_me = &my_hook;
> +
> +	cb_data.head = hooks;
> +	cb_data.run_me = list_first_entry(hooks, struct hook, list);
>  
>  	run_processes_parallel_tr2(jobs,
>  				   pick_next_hook,
> @@ -213,18 +255,15 @@ int run_hooks(const char *hook_name, const char *hook_path,
>  				   "hook",
>  				   hook_name);
>  
> -
> -	if (options->absolute_path)
> -		strbuf_release(&abs_path);
> -	free(my_hook.feed_pipe_cb_data);
> +	clear_hook_list(hooks);
>  
>  	return cb_data.rc;
>  }

Which we can see here will call clear_hook_list(), so far so good, but then...

>  int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
>  {
> -	const char *hook_path;
> -	int ret;
> +	struct list_head *hooks;
> +	int ret = 0;
>  	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
>  
>  	if (!options)
> @@ -233,14 +272,19 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
>  	if (options->path_to_stdin && options->feed_pipe)
>  		BUG("choose only one method to populate stdin");
>  
> -	hook_path = find_hook(hook_name);
> -	if (!hook_path) {
> -		ret = 0;
> +	hooks = list_hooks(hook_name);
> +
> +	/*
> +	 * If you need to act on a missing hook, use run_found_hooks()
> +	 * instead
> +	 */
> +	if (list_empty(hooks))
>  		goto cleanup;
> -	}
>  
> -	ret = run_hooks(hook_name, hook_path, options);
> +	ret = run_hooks(hook_name, hooks, options);
> +
>  cleanup:
>  	run_hooks_opt_clear(options);
> +	clear_hook_list(hooks);

...the oneshot command also does clear_hook_list(), after calling
run_hooks() which cleared it already.  That looks like a mistake,
i.e. we should always trust run_hooks() to clear it, no?

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 2/6] hook: allow parallel hook execution
  2021-08-19  3:34                 ` [PATCH v3 2/6] hook: allow parallel hook execution Emily Shaffer
@ 2021-08-24 15:01                   ` Ævar Arnfjörð Bjarmason
  2021-08-24 16:13                     ` Eric Sunshine
  2021-08-26 22:36                     ` Emily Shaffer
  0 siblings, 2 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-24 15:01 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Wed, Aug 18 2021, Emily Shaffer wrote:

> -'git hook' run [--to-stdin=<path>] [--ignore-missing] <hook-name> [-- <hook-args>]
> +'git hook' run [--to-stdin=<path>] [--ignore-missing] [(-j|--jobs) <n>]
> +	<hook-name> [-- <hook-args>]

As an aside I wondered if it shouldn't be [[-j|--jobs] <n>], but grepped
around and found that (x|y|z) means a mandatory pick of x, y or z, but
[x|y|z] means that, plus possibly picking none, I think.

So this is fine, just something I wondered about...

> +-j::
> +--jobs::
> +	Only valid for `run`.
> ++
> +Specify how many hooks to run simultaneously. If this flag is not specified, use
> +the value of the `hook.jobs` config. If the config is not specified, use the

s/use the value/uses the value/

Also we usually say "of the XYZ config, see linkgit:git-config[1]", or
something to that effect when we mention config variables. Perhaps we
should do the same here.

> +number of CPUs on the current system. Some hooks may be ineligible for
> +parallelization: for example, 'commit-msg' intends hooks modify the commit
> +message body and cannot be parallelized.

Not something that *needs* to happen in this series, but I wonder if we
shouldn't have per-type config here too, so users could force it even
for those hook types if they want.

> -#define RUN_HOOKS_OPT_INIT { \
> -	.env = STRVEC_INIT, \
> -	.args = STRVEC_INIT, \
> -}
> -
>  /*
>   * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
>   * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'.
> @@ -111,6 +113,18 @@ struct hook_cb_data {
>  	int *invoked_hook;
>  };
>  
> +#define RUN_HOOKS_OPT_INIT_SERIAL { \
> +	.jobs = 1, \
> +	.env = STRVEC_INIT, \
> +	.args = STRVEC_INIT, \
> +}
> +
> +#define RUN_HOOKS_OPT_INIT_PARALLEL { \
> +	.jobs = 0, \
> +	.env = STRVEC_INIT, \
> +	.args = STRVEC_INIT, \
> +}
> +

Ditto earlier comments about whitespace churn, i.e. I can just move this
around in the base topic, so the diff here is the change/addition, not
also moving things around.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 3/6] hook: introduce "git hook list"
  2021-08-19  3:34                 ` [PATCH v3 3/6] hook: introduce "git hook list" Emily Shaffer
@ 2021-08-24 15:08                   ` Ævar Arnfjörð Bjarmason
  2021-08-26 21:43                     ` Emily Shaffer
  2021-08-24 15:53                   ` Ævar Arnfjörð Bjarmason
  1 sibling, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-24 15:08 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Wed, Aug 18 2021, Emily Shaffer wrote:

> +static int list(int argc, const char **argv, const char *prefix)
> +{
> +	struct list_head *head, *pos;
> +	const char *hookname = NULL;
> +	struct strbuf hookdir_annotation = STRBUF_INIT;
> +
> +	struct option list_options[] = {
> +		OPT_END(),
> +	};
> +
> +	argc = parse_options(argc, argv, prefix, list_options,
> +			     builtin_hook_list_usage, 0);
> +
> +	if (argc < 1)
> +		usage_msg_opt(_("You must specify a hook event name to list."),
> +			      builtin_hook_list_usage, list_options);

Untested, but aren't we silently ignoring:

    git hook list pre-receive some extra gar bage here

I.e. shouldn't this be an "argc != 1" check?

> +
> +	hookname = argv[0];
> +
> +	head = hook_list(hookname);
> +
> +	if (list_empty(head))
> +		return 1;
> +
> +	list_for_each(pos, head) {
> +		struct hook *item = list_entry(pos, struct hook, list);
> +		item = list_entry(pos, struct hook, list);
> +		if (item)
> +			printf("%s\n", item->hook_path);

Nit/suggestion: use puts(x) instead of printf("%s\n", x), but that's
also a bikeshedding/style preference, so ignore if you disagree...

> +	}
> +
> +	clear_hook_list(head);

Nit/API suggestion: Maybe s/list_for_each/list_for_each_safe/ and
remove_hook() in the loop would make more sense for this one-shot caller
than iterating over the list twice?

Anyway, currently remove_hook() is static, and it's probably good to not
peek behind the curtain here, so on second thought clear_hook_list() is
probably best...

> +	strbuf_release(&hookdir_annotation);

This function did nothing with hookdir_annotation. Looks like leftover
WIP code, but maybe it's used in (and should be moved to) a later
commit, let's keep reading...

> [...]
>  	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
>  
> +
>  	INIT_LIST_HEAD(hook_head);

..ditto...
>  
>  	if (!hookname)
> @@ -103,8 +104,6 @@ struct list_head *list_hooks(const char *hookname)
>  
>  	if (have_git_dir()) {
>  		const char *hook_path = find_hook(hookname);
> -

... earlier notes about whitespace churn...

> -		/* Add the hook from the hookdir */
>  		if (hook_path) {
>  			struct hook *to_add = xmalloc(sizeof(*to_add));
>  			to_add->hook_path = hook_path;


^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 3/6] hook: introduce "git hook list"
  2021-08-19  3:34                 ` [PATCH v3 3/6] hook: introduce "git hook list" Emily Shaffer
  2021-08-24 15:08                   ` Ævar Arnfjörð Bjarmason
@ 2021-08-24 15:53                   ` Ævar Arnfjörð Bjarmason
  2021-08-26 22:38                     ` Emily Shaffer
  1 sibling, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-24 15:53 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Wed, Aug 18 2021, Emily Shaffer wrote:

> +	head = hook_list(hookname);

This doesn't compile, as hook_list() is added in a later
commit. Something something earlier suggestion of "git rebase -i --exec
'make test'" :)

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 4/6] hook: allow running non-native hooks
  2021-08-19  3:34                 ` [PATCH v3 4/6] hook: allow running non-native hooks Emily Shaffer
@ 2021-08-24 15:55                   ` Ævar Arnfjörð Bjarmason
  2021-08-26 22:50                     ` Emily Shaffer
  0 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-24 15:55 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Wed, Aug 18 2021, Emily Shaffer wrote:

> As the hook architecture and 'git hook run' become more featureful, we
> may find wrappers wanting to use the hook architecture to run their own
> hooks, thereby getting nice things like parallelism and idiomatic Git
> configuration for free. Enable this by letting 'git hook run' bypass the
> known_hooks() check.
>
> We do still want to keep known_hooks() around, though - by die()ing when
> an internal Git call asks for run_hooks("my-new-hook"), we can remind
> Git developers to update Documentation/githooks.txt with their new hook,
> which in turn helps Git users discover this new hook.
>
> [...]
>
> +It's possible to use this command to refer to hooks which are not native to Git,
> +for example if a wrapper around Git wishes to expose hooks into its own
> +operation in a way which is already familiar to Git users. However, wrappers
> +invoking such hooks should be careful to name their hook events something which
> +Git is unlikely to use for a native hook later on. For example, Git is much less
> +likely to create a `mytool-validate-commit` hook than it is to create a
> +`validate-commit` hook.
> +
>  SUBCOMMANDS
>  -----------

The goal here makes sense, but...

> diff --git a/builtin/hook.c b/builtin/hook.c
> index d21f303eca..80397d39f5 100644
> --- a/builtin/hook.c
> +++ b/builtin/hook.c
> @@ -46,7 +46,7 @@ static int list(int argc, const char **argv, const char *prefix)
>  
>  	hookname = argv[0];
>  
> -	head = hook_list(hookname);
> +	head = list_hooks_gently(hookname);
>  
>  	if (list_empty(head))
>  		return 1;
> @@ -105,7 +105,7 @@ static int run(int argc, const char **argv, const char *prefix)
>  	git_config(git_default_config, NULL);
>  
>  	hook_name = argv[0];
> -	hooks = list_hooks(hook_name);
> +	hooks = list_hooks_gently(hook_name);
>  	if (list_empty(hooks)) {
>  		/* ... act like run_hooks_oneshot() under --ignore-missing */
>  		if (ignore_missing)

This introduces a bug v.s. the previous state, e.g. before:

    $ git hook run --ignore-missing foobar
    fatal: the hook 'foobar' is not known to git, should be in hook-list.h via githooks(5)

But after we'll silently ignore it. I.e. we've conflated
--ignore-missing with a new and hypothetical (and this is now a synonym
of) --ignore-missing-and-allow-unknown-hook-names.

So we've conflated the user's one-shot "foobar" script with wanting to
catch a typo in e.g. git-send-email.perl.

Also instead of the user's typos being caught with a die (here using
your BUG(...) version):

    $ git hook list pre-recive
    BUG: hook.c:115: Don't recognize hook event 'pre-recive'! Is it documented in Documentation/githooks.txt?
    Aborted

We'll now silently return 1, so indistinguishabl from typing it properly
as pre-receive.

All that being said I think it's arguable that if we're going to allow
"git hook run blahblah" that the die() in the base topic in my
"hook-list.h: add a generated list of hooks, like config-list.h" is more
trouble than it's worth.

I.e. do we really need to be concerned about new hooks being added and
someone forgetting a githooks.txt update, or a typo in the git.git code
that nobody notices? Probably not.

But I think the change here is clearly broken vis-a-vis the stated goals
of its commit message as it stands, i.e. "[...]we do still want to keep
known_hooks() around, though[...]". Should we fix it by adding a new
internal-only flag to the command, or just saying we shouldn't have the
behavior at all? What do you think.

Aside from that, this change seems to be untested, I tried making this
non-gentle for testing, and all tests still passed. I.e. we don't have
any tests for running such a hook like mytool-validate-commit, but
should as part of this change.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 2/6] hook: allow parallel hook execution
  2021-08-24 15:01                   ` Ævar Arnfjörð Bjarmason
@ 2021-08-24 16:13                     ` Eric Sunshine
  2021-08-26 22:36                     ` Emily Shaffer
  1 sibling, 0 replies; 479+ messages in thread
From: Eric Sunshine @ 2021-08-24 16:13 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: Emily Shaffer, Git List

On Tue, Aug 24, 2021 at 11:06 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> On Wed, Aug 18 2021, Emily Shaffer wrote:
> > +'git hook' run [--to-stdin=<path>] [--ignore-missing] [(-j|--jobs) <n>]
> > +     <hook-name> [-- <hook-args>]
>
> As an aside I wondered if it shouldn't be [[-j|--jobs] <n>], but grepped
> around and found that (x|y|z) means a mandatory pick of x, y or z, but
> [x|y|z] means that, plus possibly picking none, I think.
>
> So this is fine, just something I wondered about...

Indeed, the text is fine as-is.

> > +Specify how many hooks to run simultaneously. If this flag is not specified, use
> > +the value of the `hook.jobs` config. If the config is not specified, use the
>
> s/use the value/uses the value/

As a native English speaker, the text as written (even if a bit
stilted) sounds better than the suggested replacement. A small tweak
might improve it slightly:

    If this flag is not specified, the value of the `hooks.job`
    configuration is used instead (see linkgit:git-config[1]).

but is not at all worth a re-roll or a lot of discussion.

> Also we usually say "of the XYZ config, see linkgit:git-config[1]", or
> something to that effect when we mention config variables. Perhaps we
> should do the same here.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 5/6] hook: include hooks from the config
  2021-08-19  3:34                 ` [PATCH v3 5/6] hook: include hooks from the config Emily Shaffer
  2021-08-19 22:39                   ` Junio C Hamano
@ 2021-08-24 19:30                   ` Ævar Arnfjörð Bjarmason
  2021-08-31 19:05                     ` Emily Shaffer
  1 sibling, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-24 19:30 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Wed, Aug 18 2021, Emily Shaffer wrote:

> Teach the hook.[hc] library to parse configs to populare the list of
> hooks to run for a given event.

s/populare/populate/

> Multiple commands can be specified for a given hook by providing
> multiple "hook.<friendly-name>.command = <path-to-hook>" and
> "hook.<friendly-name>.event = <hook-event>" lines. Hooks will be run in
> config order of the "hook.<name>.event" lines.

The "will be run in order" probably needs some amending to account for
--jobs, no?

> For example:
>
>   $ git config --list | grep ^hook
>   hook.bar.command=~/bar.sh
>   hook.bar.event=pre-commit

Perhaps the example should use:

    git config --get-regexp '^hook\.'

>   $ git hook run
>   # Runs ~/bar.sh
>   # Runs .git/hooks/pre-commit

And this "# Runs" is not actual output by git, but just an explanation
for what happens, better to reword it somehow so it doesn't give that
impression.

But the example also seems to be broken, surely it should be "git hook
run bar", not "git hook run"? Reading ahead, but presumably no-arg
doesn't run all known hooks...

> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  Documentation/config/hook.txt |  18 ++++
>  Documentation/git-hook.txt    |  57 ++++++++++++-
>  builtin/hook.c                |   3 +-
>  hook.c                        | 153 ++++++++++++++++++++++++++++++----
>  hook.h                        |   7 +-
>  t/t1800-hook.sh               | 141 ++++++++++++++++++++++++++++++-
>  6 files changed, 357 insertions(+), 22 deletions(-)
>
> diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
> index 96d3d6572c..c394756328 100644
> --- a/Documentation/config/hook.txt
> +++ b/Documentation/config/hook.txt
> @@ -1,3 +1,21 @@
> +hook.<name>.command::
> +	A command to execute whenever `hook.<name>` is invoked. `<name>` should
> +	be a unique "friendly" name which you can use to identify this hook
> +	command. (You can specify when to invoke this command with
> +	`hook.<name>.event`.) The value can be an executable on your device or a
> +	oneliner for your shell. If more than one value is specified for the
> +	same `<name>`, the last value parsed will be the only command executed.
> +	See linkgit:git-hook[1].

Hrm, so here we say "If more than one value is specified for ... the
last value parsed will be the only command executed", but in the commit
message it's:

    Multiple commands can be specified for a given hook by providing
    multiple "hook.<friendly-name>.command = <path-to-hook>" and
    "hook.<friendly-name>.event = <hook-event>" lines.

As we've discussed earlier I've got a preference for the former, but
just reading this commit message & doc the for the first time I still
don't know what you went for, looks like one or the other needs
updating. I'm intentionally not reading ahead as I review this.

Saying that it's a "unique name", but also discussing what happens if
it's not unique in the sense that we have multiple "hook.<name>.*" is a
bit confusing. I think I know what you're going for, perhaps something
like this would be better to describe it:

    For a "hook.<name>.{command,event}" hook entry you'll need to pick a
    "<name>" that's not shared with any other hook, if you do normal
    single-value config semantics apply and git will use the last
    provided value.

Or something...

> +hook.<name>.event::
> +	The hook events which should invoke `hook.<name>`. `<name>` should be a
> +	unique "friendly" name which you can use to identify this hook. The
p> +	value should be the name of a hook event, like "pre-commit" or "update".
> +	(See linkgit:githooks[5] for a complete list of hooks Git knows about.)
> +	On the specified event, the associated `hook.<name>.command` will be
> +	executed. More than one event can be specified if you wish for
> +	`hook.<name>` to execute on multiple events. See linkgit:git-hook[1].
> +
>  hook.jobs::
>  	Specifies how many hooks can be run simultaneously during parallelized
>  	hook execution. If unspecified, defaults to the number of processors on
> diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
> index d1db084e4f..9c6cbdc2eb 100644
> --- a/Documentation/git-hook.txt
> +++ b/Documentation/git-hook.txt
> @@ -27,12 +27,65 @@ Git is unlikely to use for a native hook later on. For example, Git is much less
>  likely to create a `mytool-validate-commit` hook than it is to create a
>  `validate-commit` hook.
>  
> +This command parses the default configuration files for pairs of configs like
> +so:
> +
> +  [hook "linter"]
> +    event = pre-commit
> +    command = ~/bin/linter --c
> +
> +In this example, `[hook "linter"]` represents one script - `~/bin/linter --c` -
> +which can be shared by many repos, and even by many hook events, if appropriate.

OK, so now it seems like "hook.<name>.command" is 1=1 for "hook.<name>"
and "command", and "hook.<name>.event" is 1=many, maybe that's correct,
but reading on...

> +Commands are run in the order Git encounters their associated
> +`hook.<name>.event` configs during the configuration parse (see
> +linkgit:git-config[1]). Although multiple `hook.linter.event` configs can be
> +added, only one `hook.linter.command` event is valid - Git uses "last-one-wins"
> +to determine which command to run.

...ah, and confirmed here...

> +So if you wanted your linter to run when you commit as well as when you push,
> +you would configure it like so:
> +
> +  [hook "linter"]
> +    event = pre-commit
> +    event = pre-push
> +    command = ~/bin/linter --c
> +
> +With this config, `~/bin/linter --c` would be run by Git before a commit is
> +generated (during `pre-commit`) as well as before a push is performed (during
> +`pre-push`).

Aside: I know we're discussing a presumably imaginary linter, but it's a
bit jarring to see "--" for a one-letter short-option, perhaps "-c" or
"--check" for the example...

> +And if you wanted to run your linter as well as a secret-leak detector during
> +only the "pre-commit" hook event, you would configure it instead like so:
> +
> +  [hook "linter"]
> +    event = pre-commit
> +    command = ~/bin/linter --c
> +  [hook "no-leaks"]
> +    event = pre-commit
> +    command = ~/bin/leak-detector

I think these examples would flow a bit more naturally if we started by
discussing how to setup one configured hook, then two unrelated hooks
(perhaps a "commit-msg" and "pre-commit" hook), and then finally the
cases where a given "hook.<name>.command" has multiple
"hook.<name>.event" entries. My assumption in suggesting that is that
that's, respectively, the most common to less common use cases.

> +With this config, before a commit is generated (during `pre-commit`), Git would
> +first start `~/bin/linter --c` and second start `~/bin/leak-detector`. It would
> +evaluate the output of each when deciding whether to proceed with the commit.
> +
> +For a full list of hook events which you can set your `hook.<name>.event` to,
> +and how hooks are invoked during those events, see linkgit:githooks[5].

Let's discuss what happens with unknown values for `hook.<name>.event`
here, i.e. just "are ignored".

[I'll discuss my opinions on the new and revised config schema in
another mail, just commenting on the patch here in terms of its stated
goals].

> +In general, when instructions suggest adding a script to
> +`.git/hooks/<hook-event>`, you can specify it in the config instead by running
> +`git config --add hook.<some-name>.command <path-to-script> && git config --add
> +hook.<some-name>.event <hook-event>` - this way you can share the script between
> +multiple repos. That is, `cp ~/my-script.sh ~/project/.git/hooks/pre-commit`
> +would become `git config --add hook.my-script.command ~/my-script.sh && git
> +config --add hook.my-script.event pre-commit`.

I think this would benefit a lot from being split up into code example
prose, so:

    You can [...]

    ----
    git config -add hook.<some-name>.command <path-to-script> &&
    git config -add hook.<some-name>.event <hook-event>
    ----

It's more lines, but I think a lot more readable.

I think the part of this that says "You can share the script between
multiple repos" could really use some elaboration.

I.e. if the user is following an example where they'd otherwise edit
.git/hooks/ and then just run "git config", they'll add it to
.git/config.

So then it won't be shared between multiple repos, what I think this
paragraph is trying to go for is that instead of say:

    ln -s ~/my-script.sh .git/hooks/pre-commit

You can instead do, with configurable hooks:

    git config hook.my-pre-commit.command ~/my-script.sh
    git config --add hook.my-pre-commit.event pre-commit

Notice how I omitted the --add for the first one. It's also confusing if
we're documenting "*.command" as "last one wins" to use --add with it,
in an example that's discussing how to add it to some local repo, no?

Or is this example really trying to discuss what would happen if you:

    cat >~/my-script.sh
    ...
    ^C
    chmod +x ~/my-script.sh
    git config --global hook.my-pre-commit.command ~/my-script.sh
    git config hook.my-pre-commit.event pre-commit

That's not clear to me, in any case, I think we'd do well to lead with
the former, and then discuss the latter. I.e. "here's what you'd symlink
before, now it's in config like this", and then discuss how you'd set
"global" or perhaps included config, which isn't possible with the
.git/hooks/<script> mechanism.

>  SUBCOMMANDS
>  -----------
>  
>  run::
> -	Run the `<hook-name>` hook. See linkgit:githooks[5] for
> -	the hook names we support.
> +	Runs hooks configured for `<hook-name>`, in the order they are
> +	discovered during the config parse.

This and any other things that discuss how hooks are run in detail
should really talk about the thing running .git/hooks/<name> *and* any
configured hooks, and on the subject of --jobs and ordering we should
also talk about what the order of config v.s. .git/hooks/<name> is.

> +	/* to enable oneliners, let config-specified hooks run in shell.
> +	 * config-specified hooks have a name. */

nit: usual style is multi-line comments like:

    /*
     * some text[...]
     * some more...
     */

Not:

    /* text here right away[...]
     * some more ... */


> +	cp->use_shell = !!run_me->name;
> +
>  	/* add command */
> -	if (hook_cb->options->absolute_path)
> -		strvec_push(&cp->args, absolute_path(run_me->hook_path));
> -	else
> -		strvec_push(&cp->args, run_me->hook_path);
> +	if (run_me->name) {
> +		/* ...from config */
> +		struct strbuf cmd_key = STRBUF_INIT;
> +		char *command = NULL;
> +
> +		strbuf_addf(&cmd_key, "hook.%s.command", run_me->name);

Missing strbuf_release() for this later?

> +		if (git_config_get_string(cmd_key.buf, &command)) {
> +			/* TODO test me! */

...seems easy enough to just have a test for..., i.e. an *.event entry
with no *.command.

> +			die(_("'hook.%s.command' must be configured "
> +			      "or 'hook.%s.event' must be removed; aborting.\n"),
> +			    run_me->name, run_me->name);
> +		}
> +
> +		strvec_push(&cp->args, command);
> +	} else {
> +		/* ...from hookdir. */
> +		const char *hook_path = NULL;
> +		/*
> +		 *

Nit: Too few \n before the text in a comment earlier, too many here :)

> +		 * At this point we are already running, so don't validate
> +		 * whether the hook name is known or not.

...because it was done earlier somewhere, or...?

> +		 */
> +		hook_path = find_hook_gently(hook_cb->hook_name);
> +		if (!hook_path)
> +			BUG("hookdir hook in hook list but no hookdir hook present in filesystem");
> +
> +		if (hook_cb->options->absolute_path)
> +			hook_path = absolute_path(hook_path);
> +
> +		strvec_push(&cp->args, hook_path);
> +	}
> +
>  
>  	/*
>  	 * add passed-in argv, without expanding - let the user get back
> @@ -228,8 +346,11 @@ static int notify_start_failure(struct strbuf *out,
>  
>  	hook_cb->rc |= 1;
>  
> -	strbuf_addf(out, _("Couldn't start hook '%s'\n"),
> -		    attempted->hook_path);
> +	if (attempted->name)
> +		strbuf_addf(out, _("Couldn't start hook '%s'\n"),
> +		    attempted->name);
> +	else
> +		strbuf_addstr(out, _("Couldn't start hook from hooks directory\n"));
>  
>  	return 1;
>  }
> diff --git a/hook.h b/hook.h
> index 6b7b2d14d2..621bd2cde1 100644
> --- a/hook.h
> +++ b/hook.h
> @@ -27,8 +27,11 @@ int hook_exists(const char *hookname);
>  
>  struct hook {
>  	struct list_head list;
> -	/* The path to the hook */
> -	const char *hook_path;
> +	/*
> +	 * The friendly name of the hook. NULL indicates the hook is from the
> +	 * hookdir.
> +	 */
> +	const char *name;
>  
>  	/*
>  	 * Use this to keep state for your feed_pipe_fn if you are using
> diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
> index 217db848b3..ef2432f53a 100755
> --- a/t/t1800-hook.sh
> +++ b/t/t1800-hook.sh
> @@ -1,13 +1,29 @@
>  #!/bin/bash
>  
> -test_description='git-hook command'
> +test_description='git-hook command and config-managed multihooks'
>  
>  . ./test-lib.sh
>  
> +setup_hooks () {
> +	test_config hook.ghi.event pre-commit --add
> +	test_config hook.ghi.command "/path/ghi" --add
> +	test_config_global hook.def.event pre-commit --add
> +	test_config_global hook.def.command "/path/def" --add

Isn't --add redundant here? Seems no test is stressing multi-value
hook.{ghi,def}.* from a quick glance...

> +}
> +
> +setup_hookdir () {
> +	mkdir .git/hooks
> +	write_script .git/hooks/pre-commit <<-EOF
> +	echo \"Legacy Hook\"
> +	EOF
> +	test_when_finished rm -rf .git/hooks
> +}
> +
>  test_expect_success 'git hook usage' '
>  	test_expect_code 129 git hook &&
>  	test_expect_code 129 git hook run &&
>  	test_expect_code 129 git hook run -h &&
> +	test_expect_code 129 git hook list -h &&

Doesn't this belong in a previous commit that added "git hook list", not
here?

>  	test_expect_code 129 git hook run --unknown 2>err &&
>  	grep "unknown option" err
>  '
> @@ -153,4 +169,127 @@ test_expect_success 'stdin to hooks' '
>  	test_cmp expect actual
>  '
>  
> +test_expect_success 'git hook list orders by config order' '
> +	setup_hooks &&
> +
> +	cat >expected <<-EOF &&
> +	def
> +	ghi
> +	EOF
> +
> +	git hook list pre-commit >actual &&
> +	test_cmp expected actual
> +'
> +
> +test_expect_success 'git hook list reorders on duplicate event declarations' '
> +	setup_hooks &&
> +
> +	# 'def' is usually configured globally; move it to the end by
> +	# configuring it locally.
> +	test_config hook.def.event "pre-commit" --add &&

Ah, well the --add belongs here, but not needed in setup_hooks, right?

> +
> +	cat >expected <<-EOF &&
> +	ghi
> +	def
> +	EOF
> +
> +	git hook list pre-commit >actual &&
> +	test_cmp expected actual
> +'
> +
> +test_expect_success 'git hook list shows hooks from the hookdir' '
> +	setup_hookdir &&
> +
> +	cat >expected <<-EOF &&
> +	hook from hookdir
> +	EOF
> +
> +	git hook list pre-commit >actual &&
> +	test_cmp expected actual
> +'
> +
> +test_expect_success 'inline hook definitions execute oneliners' '
> +	test_config hook.oneliner.event "pre-commit" &&
> +	test_config hook.oneliner.command "echo \"Hello World\"" &&
> +
> +	echo "Hello World" >expected &&
> +
> +	# hooks are run with stdout_to_stderr = 1
> +	git hook run pre-commit 2>actual &&
> +	test_cmp expected actual
> +'
> +
> +test_expect_success 'inline hook definitions resolve paths' '
> +	write_script sample-hook.sh <<-EOF &&
> +	echo \"Sample Hook\"
> +	EOF
> +
> +	test_when_finished "rm sample-hook.sh" &&
> +
> +	test_config hook.sample-hook.event pre-commit &&
> +	test_config hook.sample-hook.command "\"$(pwd)/sample-hook.sh\"" &&
> +
> +	echo \"Sample Hook\" >expected &&
> +
> +	# hooks are run with stdout_to_stderr = 1
> +	git hook run pre-commit 2>actual &&
> +	test_cmp expected actual
> +'
> +
> +test_expect_success 'hookdir hook included in git hook run' '
> +	setup_hookdir &&
> +
> +	echo \"Legacy Hook\" >expected &&
> +
> +	# hooks are run with stdout_to_stderr = 1
> +	git hook run pre-commit 2>actual &&
> +	test_cmp expected actual
> +'
> +
> +test_expect_success 'stdin to multiple hooks' '
> +	test_config hook.stdin-a.event "test-hook" --add &&
> +	test_config hook.stdin-a.command "xargs -P1 -I% echo a%" --add &&
> +	test_config hook.stdin-b.event "test-hook" --add &&
> +	test_config hook.stdin-b.command "xargs -P1 -I% echo b%" --add &&
> +
> +	cat >input <<-EOF &&
> +	1
> +	2
> +	3
> +	EOF
> +
> +	cat >expected <<-EOF &&
> +	a1
> +	a2
> +	a3
> +	b1
> +	b2
> +	b3
> +	EOF

For any here-docs without variables, use <<-\EOF, note the backslash.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 6/6] hook: allow out-of-repo 'git hook' invocations
  2021-08-19  3:34                 ` [PATCH v3 6/6] hook: allow out-of-repo 'git hook' invocations Emily Shaffer
@ 2021-08-24 20:12                   ` Ævar Arnfjörð Bjarmason
  2021-08-24 20:38                     ` Randall S. Becker
  2021-08-31 21:09                     ` Emily Shaffer
  0 siblings, 2 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-24 20:12 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Wed, Aug 18 2021, Emily Shaffer wrote:

> Since hooks can now be supplied via the config, and a config can be
> present without a gitdir via the global and system configs, we can start
> to allow 'git hook run' to occur without a gitdir. This enables us to do
> things like run sendemail-validate hooks when running 'git send-email'
> from a nongit directory.

Sensible goal. Perhaps we should note in an earlier commit when
config-based hooks are introduced something like:

    Even though we've added config-based hooks, they currently only work
    if we can find a .git directory, even though certain commands such
    as "git send-email" (or only that command?) can be run outside of a
    git directory. A subsequent commit will address that edge-case.

> [...]
> Notes:
>     For hookdir hooks, do we want to run them in nongit dir when core.hooksPath
>     is set? For example, if someone set core.hooksPath in their global config and
>     then ran 'git hook run sendemail-validate' in a nongit dir?
> [...]
>  git.c           |  2 +-
>  hook.c          |  2 +-
>  t/t1800-hook.sh | 20 +++++++++++++++-----
>  3 files changed, 17 insertions(+), 7 deletions(-)
>
> diff --git a/git.c b/git.c
> index 540909c391..39988ee3b0 100644
> --- a/git.c
> +++ b/git.c
> @@ -538,7 +538,7 @@ static struct cmd_struct commands[] = {
>  	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
>  	{ "hash-object", cmd_hash_object },
>  	{ "help", cmd_help },
> -	{ "hook", cmd_hook, RUN_SETUP },
> +	{ "hook", cmd_hook, RUN_SETUP_GENTLY },
>  	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
>  	{ "init", cmd_init_db },
>  	{ "init-db", cmd_init_db },
> diff --git a/hook.c b/hook.c
> index 581d87cbbd..2e08156546 100644
> --- a/hook.c
> +++ b/hook.c
> @@ -218,7 +218,7 @@ struct list_head *list_hooks_gently(const char *hookname)
>  
>  	/* Add the hook from the hookdir. The placeholder makes it easier to
>  	 * allocate work in pick_next_hook. */
> -	if (find_hook_gently(hookname))
> +	if (have_git_dir() && find_hook_gently(hookname))
>  		append_or_move_hook(hook_head, NULL);
>  
>  	return hook_head;
> diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
> index ef2432f53a..a7e45c0d16 100755
> --- a/t/t1800-hook.sh
> +++ b/t/t1800-hook.sh
> @@ -118,15 +118,25 @@ test_expect_success 'git hook run -- pass arguments' '
>  	test_cmp expect actual
>  '
>  
> -test_expect_success 'git hook run -- out-of-repo runs excluded' '
> -	write_script .git/hooks/test-hook <<-EOF &&
> -	echo Test hook
> -	EOF
> +test_expect_success 'git hook run: out-of-repo runs execute global hooks' '
> +	test_config_global hook.global-hook.event test-hook --add &&
> +	test_config_global hook.global-hook.command "echo no repo no problems" --add &&
> +
> +	echo "global-hook" >expect &&
> +	nongit git hook list test-hook >actual &&
> +	test_cmp expect actual &&
> +
> +	echo "no repo no problems" >expect &&
>  
> -	nongit test_must_fail git hook run test-hook
> +	nongit git hook run test-hook 2>actual &&
> +	test_cmp expect actual
>  '
>  
>  test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
> +	write_script .git/hooks/test-hook <<-EOF &&
> +	echo Test hook
> +	EOF
> +
>  	mkdir my-hooks &&
>  	write_script my-hooks/test-hook <<-\EOF &&
>  	echo Hook ran $1 >>actual

If the only user of this is git-send-email, let's have tests for this in
t/t9001-send-email.sh. That should also address your "Notes" above,
i.e. let's just test it with core.hooksPath and see what the interaction
looks like.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 0/6] config-based hooks restarted
  2021-08-19  3:34               ` [PATCH v3 " Emily Shaffer
                                   ` (5 preceding siblings ...)
  2021-08-19  3:34                 ` [PATCH v3 6/6] hook: allow out-of-repo 'git hook' invocations Emily Shaffer
@ 2021-08-24 20:29                 ` Ævar Arnfjörð Bjarmason
  2021-09-02 22:01                   ` Emily Shaffer
  2021-09-09 12:41                 ` [PATCH v4 0/5] " Ævar Arnfjörð Bjarmason
  7 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-24 20:29 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m. carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee


On Wed, Aug 18 2021, Emily Shaffer wrote:

> This is the config-based hooks topic rebased onto v4 of Ævar's
> branch[1]. There is a happy CI build of it on GitHub[2].
> [...]
> Right now I'm trying to focus on this series first and foremost, hence
> sending two rerolls based on the same version of Ævar's base restart.
> I'll try to perform a code review on Ævar's latest tomorrow.
> [...]

First thank for the review on the base topic at
https://lore.kernel.org/git/YR2jLdYQA5CVzX5h@google.com/ &
https://lore.kernel.org/git/YR7r3h1AG4Zyn7x7@google.com/ and related.

I'll re-roll it soon pending your feedback on comments I left on this
series that pertain to it. I.e. there's some things that I think are
better just fixed up in the base topic (from trivial things like
whitespace changes, to some behavior changes), makes for easier reading
rather than going back & forth between the two.

I've read this whole series through, as promised in
https://lore.kernel.org/git/87lf4qeh86.fsf@evledraar.gmail.com/:

    [I'll discuss my opinions on the new and revised config schema in
    another mail, just commenting on the patch here in terms of its stated
    goals].

I.e. here's some general comments, numbered for ease of reference:

0)

I think this is in much much better shape vis-as-vis the simplified
config schema that's now being proposed re our discussion starting
around https://lore.kernel.org/git/87bl6ttmlv.fsf@evledraar.gmail.com/

I.e. the main complexity of the "skip" mechanism is gone, and also the
conflation of hook names with hook commands (the "rm -rf /" as a
<name-or-cmd> discussed in the above).

So before going any further, I'll just say that I wouldn't object much
to this design going in as-is. What I'm about to mention here below is
much closer to bikeshedding in my mind than "this is really to complex
to go in-tree", which was my opinion of the config schema before.

1)

On the current config design: First, before going into any detail on
that, I think whatever anyone's opinion is on that that the
design-focused patches as they stand could really use more a more
extended discussion of the design.

I.e. talk about the previously considered config schema, why it evolved
into its current form. The trade-offs involved, and why the patch
proposed to implement the schema it's implementing over another earlier
or alternate design.

I.e. https://lore.kernel.org/git/20210819033450.3382652-6-emilyshaffer@google.com
is two very short paragraphs. We won't be able to summarize all our
month-long discussion on the config design in one commit message, but I
think at least discussing it somewhat / linking to relevant on-list
discussions would make future source spelunking easier.

2)

So that out of the way, a comment on the current config design, which
should be read in the context of what I noted in #0. I.e. I'm *much*
happier with this version.

That being said I'm still not convinced that the simple 1=1 mapping of
"hook.<name>.command" and its "value" should be followed by the 1=many
mapping of "hook.<name>.event" and its "value".

I.e. we're making the trade-off of saving the user from typing out or
specifying:
    
    [hook "my-pre-commit"]
    command = ~/hooks/pre-commit-or-push
    event = pre-commit
    [hook "my-pre-push"]
    command = ~/hooks/pre-commit-or-push
    event = pre-push

And instead being able to do:

    [hook "my-pre-commit-or-push"]
    command = ~/hooks/pre-commit-or-push
    event = pre-commit
    event = pre-push

So for the very common case, saving two config lines. "Two" because as
we discussed[1] as there's currently no GIT_HOOK_TYPE env var. So this
form will work pretty much only for that case.

I.e. unlike with .git/hook/<name> the hook run via config can't
determine what <hook-type> it's being run at, so as it stands this is
only useful for those hooks listed in githooks(5) where someone would
want to do the exact same thing for one or more <hook-name> names. You
can't use it as a general routing mechanism for any hook type as it
stands.

I *think* that's only these two, perhaps "update" and "pre-receive",
with the hook seeing if it consume stdin/has arguments to disambiguate
the two.

But even with a GIT_HOOK_TYPE passed the trade-off, as discussed in [1],
and downthread in [2], is that by having it 1=many we're closing the
door on any future hook.<name>.<whatever>. I.e. config that would change
the behavior of that hook, but you'd want to change it in another way
for at least one of the N event types.

Well, "closing the door" as in if you'd want that you'd have to split up
the section from the "my-pre-commit-or-push" example above to the
"my-pre-commit" and "my-pre-push" example.

But again, on the "is the complexity worth it" we're then having to
explain to users that they can do it one way if the want no config other
than hook.<name>.{command,event*}, but another if they have another key
in that namespace.

You've said that you wanted to add something like a GIT_HOOK_TYPE
environment variable. Fair enough, and I guess we could add it in a
re-roll of this series. I'm mainly commenting on the end-state of *this*
series in particular. I.e. I think it leaves the user & implementation
with a config schema that still seems to be needlessly complex for the
very limited benefits that complexity brings us in what you're able to
do with it now.

But some of that goes back to the comments I had on 5/6[3], i.e. I'm
willing to be convinced, but I think that the current commit message &
added docs aren't really selling the idea of why it's worth it.

3) 

As an extension to my comments on 5/6[3], I think this whole notion of
"git hook run <name>" as invoked by a user of git is just more confusing
the more I think about it.

I.e. 5/6[4] is apparently seeking to implement a way to just make that
facility a general way for users to run some command on their system to
do whatever, instead of say using /usr/bin/parallel or a shell alias.

But then we also use that command as our own dispatch mechanism for our
own known hooks, except mostly not, since we mostly use the C API
ourselves directly.

It's particularly confusing that if you say run "git hook run
pre-auto-gc" as a user to test your hook you'll have that hook run in
the same way that git-gc(1) would run it. So someone developing a hook
might think they can use "git hook run" for testing it.

But if you do the same with say "git hook run pre-receive" or anything
else that feeds arguments or stdin (e.g. "update", or "pre-receive"),
you'll have your hook happily being run by git, but in a way that's not
at all how such a hook will be run when it's run by git "for real".

So I wonder if we shouldn't just have the thing die() if you try to run
any hook that's in githooks(5) itself, except for sendemail-validate and
the p4 hooks, since we need to run those ourselves.

Or have those use an internal-only "git hook--helper", and start out
with "git hook" just supporting "git hook list", and then later on have
"git hook run" (or perhaps "git hook run-configured"?) be an entry point
for this facility of running some arbitrary script that's not a "real"
hook.

I don't know, maybe I'm the only one that finds this confusing...

1. https://lore.kernel.org/git/87bl6ttmlv.fsf@evledraar.gmail.com/
2. https://lore.kernel.org/git/877dh0n1b3.fsf@evledraar.gmail.com/
3. https://lore.kernel.org/git/87lf4qeh86.fsf@evledraar.gmail.com/
4. https://lore.kernel.org/git/20210819033450.3382652-6-emilyshaffer@google.com/

^ permalink raw reply	[flat|nested] 479+ messages in thread

* RE: [PATCH v3 6/6] hook: allow out-of-repo 'git hook' invocations
  2021-08-24 20:12                   ` Ævar Arnfjörð Bjarmason
@ 2021-08-24 20:38                     ` Randall S. Becker
  2021-08-24 22:45                       ` Ævar Arnfjörð Bjarmason
  2021-08-31 21:09                     ` Emily Shaffer
  1 sibling, 1 reply; 479+ messages in thread
From: Randall S. Becker @ 2021-08-24 20:38 UTC (permalink / raw)
  To: 'Ævar Arnfjörð Bjarmason',
	'Emily Shaffer'
  Cc: git

On August 24, 2021 4:12 PM, Ævar Arnfjörð Bjarmason wrote:
>On Wed, Aug 18 2021, Emily Shaffer wrote:
>
>> Since hooks can now be supplied via the config, and a config can be
>> present without a gitdir via the global and system configs, we can
>> start to allow 'git hook run' to occur without a gitdir. This enables
>> us to do things like run sendemail-validate hooks when running 'git send-email'
>> from a nongit directory.
>
>Sensible goal. Perhaps we should note in an earlier commit when config-based hooks are introduced something like:

To clarify the requirements here, if running without a gitdir (and thus without a repository?) how will front-ends know what to supply? Will this just be "some shell script" that runs?
>
>    Even though we've added config-based hooks, they currently only work
>    if we can find a .git directory, even though certain commands such
>    as "git send-email" (or only that command?) can be run outside of a
>    git directory. A subsequent commit will address that edge-case.

So we cannot assume anything about the repository, correct? Similar to running git version but not git status?

>> [...]
>> Notes:
>>     For hookdir hooks, do we want to run them in nongit dir when core.hooksPath
>>     is set? For example, if someone set core.hooksPath in their global config and
>>     then ran 'git hook run sendemail-validate' in a nongit dir?

So this is complete consent to run outside of git? I wonder whether there needs to be an attribute associated with the hook that enables this edge capability. That way we can validate whether the hook should be run or not (from front-end scripts).


^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 6/6] hook: allow out-of-repo 'git hook' invocations
  2021-08-24 20:38                     ` Randall S. Becker
@ 2021-08-24 22:45                       ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-24 22:45 UTC (permalink / raw)
  To: Randall S. Becker; +Cc: 'Emily Shaffer', git


On Tue, Aug 24 2021, Randall S. Becker wrote:

> On August 24, 2021 4:12 PM, Ævar Arnfjörð Bjarmason wrote:
>>On Wed, Aug 18 2021, Emily Shaffer wrote:
>>
>>> Since hooks can now be supplied via the config, and a config can be
>>> present without a gitdir via the global and system configs, we can
>>> start to allow 'git hook run' to occur without a gitdir. This enables
>>> us to do things like run sendemail-validate hooks when running 'git send-email'
>>> from a nongit directory.
>>
>>Sensible goal. Perhaps we should note in an earlier commit when config-based hooks are introduced something like:
>
> To clarify the requirements here, if running without a gitdir (and
> thus without a repository?) how will front-ends know what to supply?
> Will this just be "some shell script" that runs?

Emily's also aiming to have "git hook run" running "some shell script",
but in this case we're talking about any git program that runs hooks,
but doesn't require a repo. I think the only one is git-send-email's
sendemail-validate hook.

>>
>>    Even though we've added config-based hooks, they currently only work
>>    if we can find a .git directory, even though certain commands such
>>    as "git send-email" (or only that command?) can be run outside of a
>>    git directory. A subsequent commit will address that edge-case.
>
> So we cannot assume anything about the repository, correct? Similar to running git version but not git status?

Right, with RUN_SETUP_GENTLY we may not have a repo at all, so like
commands such as "git bundle", "git config", "git ls-remote" etc.

>>> [...]
>>> Notes:
>>>     For hookdir hooks, do we want to run them in nongit dir when core.hooksPath
>>>     is set? For example, if someone set core.hooksPath in their global config and
>>>     then ran 'git hook run sendemail-validate' in a nongit dir?
>
> So this is complete consent to run outside of git? I wonder whether
> there needs to be an attribute associated with the hook that enables
> this edge capability. That way we can validate whether the hook should
> be run or not (from front-end scripts).

If I understand you correctly you're pointing out that anyone with a
sendemail-validate script could previously have assumed a repo, but not
anymore. So e.g. someone who always has other config the hook requires
when they run it in repository might unexpectedly have that hook fail if
they naïvely set the config for that sendemail-validate hook via "git
config --global".

I think that's a good point, and one I hadn't really considered. I think
it's probably best to just document this edge case for the one (or few?)
hooks that have this caveat than to not support it. I.e. it seems useful
to have a sendemail-validate without a repo (just to parse/validate the
about-to-be-sent-email), that we needed a repo before was really only an
emergent effect.

Also, I think it's not new with this config-based hook mechanism, but
something that I introduced in back when I added core.hooksPath (but I
don't think I realized I was adding this edge case at the time).

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 1/6] hook: run a list of hooks instead
  2021-08-24 14:56                   ` Ævar Arnfjörð Bjarmason
@ 2021-08-26 21:16                     ` Emily Shaffer
  2021-08-27 11:15                       ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-08-26 21:16 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Tue, Aug 24, 2021 at 04:56:10PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Wed, Aug 18 2021, Emily Shaffer wrote:
> 
> > @@ -25,7 +25,8 @@ static int run(int argc, const char **argv, const char *prefix)
> >  	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
> >  	int ignore_missing = 0;
> >  	const char *hook_name;
> > -	const char *hook_path;
> > +	struct list_head *hooks;
> > +
> >  	struct option run_options[] = {
> >  		OPT_BOOL(0, "ignore-missing", &ignore_missing,
> >  			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
> 
> In general in this patch series there's a bunch of little whitespace
> changes like that along with other changes. I think it's probably best
> if I just absorb that in the "base" topic instead of doing them
> here. E.g. in this case we could also add a line between "struct option"
> and the rest.
> 
> I don't mind either way, but the whitespace churn makes for distracting
> reading...

Ah, hm. I don't know if in this specific case it's necessary for me to
even have this whitespace change, since 'run_options' is still a struct
declaration. I'll just drop this one, but in general whichever
whitespace bits you like from this topic, feel free to absorb. Will make
a note to scan through the diff when I rebase onto your next reroll
checking for spurious whitespace changes.

> 
> > @@ -58,15 +59,16 @@ static int run(int argc, const char **argv, const char *prefix)
> >  	git_config(git_default_config, NULL);
> >  
> >  	hook_name = argv[0];
> > -	if (ignore_missing)
> > -		return run_hooks_oneshot(hook_name, &opt);
> > -	hook_path = find_hook(hook_name);
> > -	if (!hook_path) {
> > +	hooks = list_hooks(hook_name);
> > +	if (list_empty(hooks)) {
> > +		/* ... act like run_hooks_oneshot() under --ignore-missing */
> > +		if (ignore_missing)
> > +			return 0;
> >  		error("cannot find a hook named %s", hook_name);
> >  		return 1;
> >  	}
> >  
> > -	ret = run_hooks(hook_name, hook_path, &opt);
> > +	ret = run_hooks(hook_name, hooks, &opt);
> >  	run_hooks_opt_clear(&opt);
> >  	return ret;
> 
> This memory management is a bit inconsistent. So here we list_hooks(),
> pass it to run_hooks(), which clears it for us, OK...
> 
> > -int run_hooks(const char *hook_name, const char *hook_path,
> > -	      struct run_hooks_opt *options)
> > +int run_hooks(const char *hook_name, struct list_head *hooks,
> > +		    struct run_hooks_opt *options)
> >  {
> > -	struct strbuf abs_path = STRBUF_INIT;
> > -	struct hook my_hook = {
> > -		.hook_path = hook_path,
> > -	};
> >  	struct hook_cb_data cb_data = {
> >  		.rc = 0,
> >  		.hook_name = hook_name,
> > @@ -197,11 +241,9 @@ int run_hooks(const char *hook_name, const char *hook_path,
> >  	if (!options)
> >  		BUG("a struct run_hooks_opt must be provided to run_hooks");
> >  
> > -	if (options->absolute_path) {
> > -		strbuf_add_absolute_path(&abs_path, hook_path);
> > -		my_hook.hook_path = abs_path.buf;
> > -	}
> > -	cb_data.run_me = &my_hook;
> > +
> > +	cb_data.head = hooks;
> > +	cb_data.run_me = list_first_entry(hooks, struct hook, list);
> >  
> >  	run_processes_parallel_tr2(jobs,
> >  				   pick_next_hook,
> > @@ -213,18 +255,15 @@ int run_hooks(const char *hook_name, const char *hook_path,
> >  				   "hook",
> >  				   hook_name);
> >  
> > -
> > -	if (options->absolute_path)
> > -		strbuf_release(&abs_path);
> > -	free(my_hook.feed_pipe_cb_data);
> > +	clear_hook_list(hooks);
> >  
> >  	return cb_data.rc;
> >  }
> 
> Which we can see here will call clear_hook_list(), so far so good, but then...
> 
> >  int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
> >  {
> > -	const char *hook_path;
> > -	int ret;
> > +	struct list_head *hooks;
> > +	int ret = 0;
> >  	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
> >  
> >  	if (!options)
> > @@ -233,14 +272,19 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
> >  	if (options->path_to_stdin && options->feed_pipe)
> >  		BUG("choose only one method to populate stdin");
> >  
> > -	hook_path = find_hook(hook_name);
> > -	if (!hook_path) {
> > -		ret = 0;
> > +	hooks = list_hooks(hook_name);
> > +
> > +	/*
> > +	 * If you need to act on a missing hook, use run_found_hooks()
> > +	 * instead
> > +	 */
> > +	if (list_empty(hooks))
> >  		goto cleanup;
> > -	}
> >  
> > -	ret = run_hooks(hook_name, hook_path, options);
> > +	ret = run_hooks(hook_name, hooks, options);
> > +
> >  cleanup:
> >  	run_hooks_opt_clear(options);
> > +	clear_hook_list(hooks);
> 
> ...the oneshot command also does clear_hook_list(), after calling
> run_hooks() which cleared it already.  That looks like a mistake,
> i.e. we should always trust run_hooks() to clear it, no?

Ah, good catch. I will update the comment on run_hooks() and fix
_oneshot() :)

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 3/6] hook: introduce "git hook list"
  2021-08-24 15:08                   ` Ævar Arnfjörð Bjarmason
@ 2021-08-26 21:43                     ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-26 21:43 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Tue, Aug 24, 2021 at 05:08:25PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Wed, Aug 18 2021, Emily Shaffer wrote:
> 
> > +static int list(int argc, const char **argv, const char *prefix)
> > +{
> > +	struct list_head *head, *pos;
> > +	const char *hookname = NULL;
> > +	struct strbuf hookdir_annotation = STRBUF_INIT;
> > +
> > +	struct option list_options[] = {
> > +		OPT_END(),
> > +	};
> > +
> > +	argc = parse_options(argc, argv, prefix, list_options,
> > +			     builtin_hook_list_usage, 0);
> > +
> > +	if (argc < 1)
> > +		usage_msg_opt(_("You must specify a hook event name to list."),
> > +			      builtin_hook_list_usage, list_options);
> 
> Untested, but aren't we silently ignoring:
> 
>     git hook list pre-receive some extra gar bage here
> 
> I.e. shouldn't this be an "argc != 1" check?

Yeah, I think you are right. Will switch.

> 
> > +
> > +	hookname = argv[0];
> > +
> > +	head = hook_list(hookname);
> > +
> > +	if (list_empty(head))
> > +		return 1;
> > +
> > +	list_for_each(pos, head) {
> > +		struct hook *item = list_entry(pos, struct hook, list);
> > +		item = list_entry(pos, struct hook, list);
> > +		if (item)
> > +			printf("%s\n", item->hook_path);
> 
> Nit/suggestion: use puts(x) instead of printf("%s\n", x), but that's
> also a bikeshedding/style preference, so ignore if you disagree...

I was curious, because today I learned about puts() ;), so I checked
(sorry for escape gore):

  $ gg puts\( | wc -l
  217
  $ gg "printf(\"%s\(\\\\n\)\?\"" | wc -l
  96

So looks like it is indeed more idiomatic by about 2x to just use
puts(). Will switch.

> 
> > +	}
> > +
> > +	clear_hook_list(head);
> 
> Nit/API suggestion: Maybe s/list_for_each/list_for_each_safe/ and
> remove_hook() in the loop would make more sense for this one-shot caller
> than iterating over the list twice?
> 
> Anyway, currently remove_hook() is static, and it's probably good to not
> peek behind the curtain here, so on second thought clear_hook_list() is
> probably best...

Sounds like you talked yourself out of it before I could. Noop ;)

> 
> > +	strbuf_release(&hookdir_annotation);
> 
> This function did nothing with hookdir_annotation. Looks like leftover
> WIP code, but maybe it's used in (and should be moved to) a later
> commit, let's keep reading...

Ah, leftover WIP code indeed. Will drop it. I do think, though, that
"hook from hookdir" is an ugly thing to say in list(), so any better
suggestions welcome.

> 
> > [...]
> >  	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
> >  
> > +
> >  	INIT_LIST_HEAD(hook_head);
> 
> ..ditto...
ACK

> >  
> >  	if (!hookname)
> > @@ -103,8 +104,6 @@ struct list_head *list_hooks(const char *hookname)
> >  
> >  	if (have_git_dir()) {
> >  		const char *hook_path = find_hook(hookname);
> > -
> 
> ... earlier notes about whitespace churn...
> 
> > -		/* Add the hook from the hookdir */
> >  		if (hook_path) {
> >  			struct hook *to_add = xmalloc(sizeof(*to_add));
> >  			to_add->hook_path = hook_path;
> 

Thanks.
 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 2/6] hook: allow parallel hook execution
  2021-08-24 15:01                   ` Ævar Arnfjörð Bjarmason
  2021-08-24 16:13                     ` Eric Sunshine
@ 2021-08-26 22:36                     ` Emily Shaffer
  1 sibling, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-26 22:36 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Tue, Aug 24, 2021 at 05:01:09PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Wed, Aug 18 2021, Emily Shaffer wrote:
> 
> > -'git hook' run [--to-stdin=<path>] [--ignore-missing] <hook-name> [-- <hook-args>]
> > +'git hook' run [--to-stdin=<path>] [--ignore-missing] [(-j|--jobs) <n>]
> > +	<hook-name> [-- <hook-args>]
> 
> As an aside I wondered if it shouldn't be [[-j|--jobs] <n>], but grepped
> around and found that (x|y|z) means a mandatory pick of x, y or z, but
> [x|y|z] means that, plus possibly picking none, I think.
> 
> So this is fine, just something I wondered about...

Hm, I guess I hadn't see [x|y|z] (or paid attention to it). Anyway, yes,
I think [(-j|--jobs) <n>] is probably most correct here: "entirely
optionally you can provide a jobs arg; but if you do it must start with
-j or --jobs and it must contain a number". I guess you could also say
[-j <n> | --jobs <n>] but that might leave the reader to consider
whether <n> has a different meaning in each? Anyway, doesn't matter
really, and thanks for completely nerd-sniping me ;)

> 
> > +-j::
> > +--jobs::
> > +	Only valid for `run`.
> > ++
> > +Specify how many hooks to run simultaneously. If this flag is not specified, use
> > +the value of the `hook.jobs` config. If the config is not specified, use the
> 
> s/use the value/uses the value/
> 
> Also we usually say "of the XYZ config, see linkgit:git-config[1]", or
> something to that effect when we mention config variables. Perhaps we
> should do the same here.

Done, thanks.

> 
> > +number of CPUs on the current system. Some hooks may be ineligible for
> > +parallelization: for example, 'commit-msg' intends hooks modify the commit
> > +message body and cannot be parallelized.
> 
> Not something that *needs* to happen in this series, but I wonder if we
> shouldn't have per-type config here too, so users could force it even
> for those hook types if they want.

Yeah, I'm happy to implement that later when someone complains... ;)

> 
> > -#define RUN_HOOKS_OPT_INIT { \
> > -	.env = STRVEC_INIT, \
> > -	.args = STRVEC_INIT, \
> > -}
> > -
> >  /*
> >   * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
> >   * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'.
> > @@ -111,6 +113,18 @@ struct hook_cb_data {
> >  	int *invoked_hook;
> >  };
> >  
> > +#define RUN_HOOKS_OPT_INIT_SERIAL { \
> > +	.jobs = 1, \
> > +	.env = STRVEC_INIT, \
> > +	.args = STRVEC_INIT, \
> > +}
> > +
> > +#define RUN_HOOKS_OPT_INIT_PARALLEL { \
> > +	.jobs = 0, \
> > +	.env = STRVEC_INIT, \
> > +	.args = STRVEC_INIT, \
> > +}
> > +
> 
> Ditto earlier comments about whitespace churn, i.e. I can just move this
> around in the base topic, so the diff here is the change/addition, not
> also moving things around.

Nah, I think I just wasn't paying much attention to where it goes. It
makes more sense to put this by the 'struct run_hooks_opt' decl, I'll
move mine instead.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 3/6] hook: introduce "git hook list"
  2021-08-24 15:53                   ` Ævar Arnfjörð Bjarmason
@ 2021-08-26 22:38                     ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-26 22:38 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Tue, Aug 24, 2021 at 05:53:48PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Wed, Aug 18 2021, Emily Shaffer wrote:
> 
> > +	head = hook_list(hookname);
> 
> This doesn't compile, as hook_list() is added in a later
> commit. Something something earlier suggestion of "git rebase -i --exec
> 'make test'" :)


Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaargh. Thanks.

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 4/6] hook: allow running non-native hooks
  2021-08-24 15:55                   ` Ævar Arnfjörð Bjarmason
@ 2021-08-26 22:50                     ` Emily Shaffer
  2021-08-27  0:22                       ` Junio C Hamano
  0 siblings, 1 reply; 479+ messages in thread
From: Emily Shaffer @ 2021-08-26 22:50 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Tue, Aug 24, 2021 at 05:55:13PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Wed, Aug 18 2021, Emily Shaffer wrote:
> 
> > As the hook architecture and 'git hook run' become more featureful, we
> > may find wrappers wanting to use the hook architecture to run their own
> > hooks, thereby getting nice things like parallelism and idiomatic Git
> > configuration for free. Enable this by letting 'git hook run' bypass the
> > known_hooks() check.
> >
> > We do still want to keep known_hooks() around, though - by die()ing when
> > an internal Git call asks for run_hooks("my-new-hook"), we can remind
> > Git developers to update Documentation/githooks.txt with their new hook,
> > which in turn helps Git users discover this new hook.
> >
> > [...]
> >
> > +It's possible to use this command to refer to hooks which are not native to Git,
> > +for example if a wrapper around Git wishes to expose hooks into its own
> > +operation in a way which is already familiar to Git users. However, wrappers
> > +invoking such hooks should be careful to name their hook events something which
> > +Git is unlikely to use for a native hook later on. For example, Git is much less
> > +likely to create a `mytool-validate-commit` hook than it is to create a
> > +`validate-commit` hook.
> > +
> >  SUBCOMMANDS
> >  -----------
> 
> The goal here makes sense, but...
> 
> > diff --git a/builtin/hook.c b/builtin/hook.c
> > index d21f303eca..80397d39f5 100644
> > --- a/builtin/hook.c
> > +++ b/builtin/hook.c
> > @@ -46,7 +46,7 @@ static int list(int argc, const char **argv, const char *prefix)
> >  
> >  	hookname = argv[0];
> >  
> > -	head = hook_list(hookname);
> > +	head = list_hooks_gently(hookname);
> >  
> >  	if (list_empty(head))
> >  		return 1;
> > @@ -105,7 +105,7 @@ static int run(int argc, const char **argv, const char *prefix)
> >  	git_config(git_default_config, NULL);
> >  
> >  	hook_name = argv[0];
> > -	hooks = list_hooks(hook_name);
> > +	hooks = list_hooks_gently(hook_name);
> >  	if (list_empty(hooks)) {
> >  		/* ... act like run_hooks_oneshot() under --ignore-missing */
> >  		if (ignore_missing)
> 
> This introduces a bug v.s. the previous state, e.g. before:
> 
>     $ git hook run --ignore-missing foobar
>     fatal: the hook 'foobar' is not known to git, should be in hook-list.h via githooks(5)
> 
> But after we'll silently ignore it. I.e. we've conflated
> --ignore-missing with a new and hypothetical (and this is now a synonym
> of) --ignore-missing-and-allow-unknown-hook-names.
> 
> So we've conflated the user's one-shot "foobar" script with wanting to
> catch a typo in e.g. git-send-email.perl.
> 
> Also instead of the user's typos being caught with a die (here using
> your BUG(...) version):
> 
>     $ git hook list pre-recive
>     BUG: hook.c:115: Don't recognize hook event 'pre-recive'! Is it documented in Documentation/githooks.txt?
>     Aborted
> 
> We'll now silently return 1, so indistinguishabl from typing it properly
> as pre-receive.
> 
> All that being said I think it's arguable that if we're going to allow
> "git hook run blahblah" that the die() in the base topic in my
> "hook-list.h: add a generated list of hooks, like config-list.h" is more
> trouble than it's worth.
> 
> I.e. do we really need to be concerned about new hooks being added and
> someone forgetting a githooks.txt update, or a typo in the git.git code
> that nobody notices? Probably not.
> 
> But I think the change here is clearly broken vis-a-vis the stated goals
> of its commit message as it stands, i.e. "[...]we do still want to keep
> known_hooks() around, though[...]". Should we fix it by adding a new
> internal-only flag to the command, or just saying we shouldn't have the
> behavior at all? What do you think.

I think it's A) pretty important to make it easy for users to run
whatever not-necessarily-git-native hook they want, and B) useful for
script Git commands to take advantage of the typo check. So, I'll add a
`--enforce-known-hookname` (or maybe a better named one, this isn't my
strong suit) and switch git-send-email and friends to use it. Like we
discussed off-list, I think it's a good idea to drop the envvar for
exceptional test names from the codebase entirely.

> 
> Aside from that, this change seems to be untested, I tried making this
> non-gentle for testing, and all tests still passed. I.e. we don't have
> any tests for running such a hook like mytool-validate-commit, but
> should as part of this change.

Sure.


Actually, I was in the middle of typing about how I wouldn't change your
'test-hook' and so on tests, and it occurs to me that it might actually
be a better fit for your series to add this --reject-unknown (or
whatever) flag, instead of the envvar magics. So I'll hold off on making
any changes unless I hear from you.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 4/6] hook: allow running non-native hooks
  2021-08-26 22:50                     ` Emily Shaffer
@ 2021-08-27  0:22                       ` Junio C Hamano
  0 siblings, 0 replies; 479+ messages in thread
From: Junio C Hamano @ 2021-08-27  0:22 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Ævar Arnfjörð Bjarmason, git

Emily Shaffer <emilyshaffer@google.com> writes:

> I think it's A) pretty important to make it easy for users to run
> whatever not-necessarily-git-native hook they want, and B) useful for
> script Git commands to take advantage of the typo check. So, I'll add a
> `--enforce-known-hookname` (or maybe a better named one, this isn't my
> strong suit) and switch git-send-email and friends to use it.

I somehow feel this is backwards.  

Once you write the invocation of "git hook run <hookname>" into your
script and tested it, there is no further need for typo checking.

What's the use case you are trying to help with typo checking?  When
a script takes a hookname from the user and runs "git hook run $1",
then passing --this-must-be-a-known-hook option that errors out when
the named hook does not exist and unrecognised (there is no need to
error out if a hook with unusual name the user gave us does
exist---the user asked us to run it, so we just can run it) might
make sense.  But I am somehow not getting the sense that it is the
expected use case you are worried about.

If the reason why you are making the typo-checking an opt-in feature
is because you want to allow users to run "git hook run" with
minimum typing, I suspect that you may be optimizing for a wrong
case.  Interactive users are the ones that benefit from
typo-checking the most, and if they are interactive (as opposed to
being a script), they are flexible enough not to say "git hook run
foobar" when they know foobar does not exist and they know foobar is
not a generally accepted hook, no?  So, I think it makes more sense
to by default allow a hook with a recognizable name to be missing,
but complain when a randomly named hook is missing, and to have an
option that permits a hook to be unrecognised and missing.




^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 1/6] hook: run a list of hooks instead
  2021-08-26 21:16                     ` Emily Shaffer
@ 2021-08-27 11:15                       ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-08-27 11:15 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git


On Thu, Aug 26 2021, Emily Shaffer wrote:

> On Tue, Aug 24, 2021 at 04:56:10PM +0200, Ævar Arnfjörð Bjarmason wrote:
>> 
>> 
>> On Wed, Aug 18 2021, Emily Shaffer wrote:
>> 
>> > @@ -25,7 +25,8 @@ static int run(int argc, const char **argv, const char *prefix)
>> >  	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
>> >  	int ignore_missing = 0;
>> >  	const char *hook_name;
>> > -	const char *hook_path;
>> > +	struct list_head *hooks;
>> > +
>> >  	struct option run_options[] = {
>> >  		OPT_BOOL(0, "ignore-missing", &ignore_missing,
>> >  			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
>> 
>> In general in this patch series there's a bunch of little whitespace
>> changes like that along with other changes. I think it's probably best
>> if I just absorb that in the "base" topic instead of doing them
>> here. E.g. in this case we could also add a line between "struct option"
>> and the rest.
>> 
>> I don't mind either way, but the whitespace churn makes for distracting
>> reading...
>
> Ah, hm. I don't know if in this specific case it's necessary for me to
> even have this whitespace change, since 'run_options' is still a struct
> declaration. I'll just drop this one, but in general whichever
> whitespace bits you like from this topic, feel free to absorb. Will make
> a note to scan through the diff when I rebase onto your next reroll
> checking for spurious whitespace changes.
>
>> 
>> > @@ -58,15 +59,16 @@ static int run(int argc, const char **argv, const char *prefix)
>> >  	git_config(git_default_config, NULL);
>> >  
>> >  	hook_name = argv[0];
>> > -	if (ignore_missing)
>> > -		return run_hooks_oneshot(hook_name, &opt);
>> > -	hook_path = find_hook(hook_name);
>> > -	if (!hook_path) {
>> > +	hooks = list_hooks(hook_name);
>> > +	if (list_empty(hooks)) {
>> > +		/* ... act like run_hooks_oneshot() under --ignore-missing */
>> > +		if (ignore_missing)
>> > +			return 0;
>> >  		error("cannot find a hook named %s", hook_name);
>> >  		return 1;
>> >  	}
>> >  
>> > -	ret = run_hooks(hook_name, hook_path, &opt);
>> > +	ret = run_hooks(hook_name, hooks, &opt);
>> >  	run_hooks_opt_clear(&opt);
>> >  	return ret;
>> 
>> This memory management is a bit inconsistent. So here we list_hooks(),
>> pass it to run_hooks(), which clears it for us, OK...
>> 
>> > -int run_hooks(const char *hook_name, const char *hook_path,
>> > -	      struct run_hooks_opt *options)
>> > +int run_hooks(const char *hook_name, struct list_head *hooks,
>> > +		    struct run_hooks_opt *options)
>> >  {
>> > -	struct strbuf abs_path = STRBUF_INIT;
>> > -	struct hook my_hook = {
>> > -		.hook_path = hook_path,
>> > -	};
>> >  	struct hook_cb_data cb_data = {
>> >  		.rc = 0,
>> >  		.hook_name = hook_name,
>> > @@ -197,11 +241,9 @@ int run_hooks(const char *hook_name, const char *hook_path,
>> >  	if (!options)
>> >  		BUG("a struct run_hooks_opt must be provided to run_hooks");
>> >  
>> > -	if (options->absolute_path) {
>> > -		strbuf_add_absolute_path(&abs_path, hook_path);
>> > -		my_hook.hook_path = abs_path.buf;
>> > -	}
>> > -	cb_data.run_me = &my_hook;
>> > +
>> > +	cb_data.head = hooks;
>> > +	cb_data.run_me = list_first_entry(hooks, struct hook, list);
>> >  
>> >  	run_processes_parallel_tr2(jobs,
>> >  				   pick_next_hook,
>> > @@ -213,18 +255,15 @@ int run_hooks(const char *hook_name, const char *hook_path,
>> >  				   "hook",
>> >  				   hook_name);
>> >  
>> > -
>> > -	if (options->absolute_path)
>> > -		strbuf_release(&abs_path);
>> > -	free(my_hook.feed_pipe_cb_data);
>> > +	clear_hook_list(hooks);
>> >  
>> >  	return cb_data.rc;
>> >  }
>> 
>> Which we can see here will call clear_hook_list(), so far so good, but then...
>> 
>> >  int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
>> >  {
>> > -	const char *hook_path;
>> > -	int ret;
>> > +	struct list_head *hooks;
>> > +	int ret = 0;
>> >  	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
>> >  
>> >  	if (!options)
>> > @@ -233,14 +272,19 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
>> >  	if (options->path_to_stdin && options->feed_pipe)
>> >  		BUG("choose only one method to populate stdin");
>> >  
>> > -	hook_path = find_hook(hook_name);
>> > -	if (!hook_path) {
>> > -		ret = 0;
>> > +	hooks = list_hooks(hook_name);
>> > +
>> > +	/*
>> > +	 * If you need to act on a missing hook, use run_found_hooks()
>> > +	 * instead
>> > +	 */
>> > +	if (list_empty(hooks))
>> >  		goto cleanup;
>> > -	}
>> >  
>> > -	ret = run_hooks(hook_name, hook_path, options);
>> > +	ret = run_hooks(hook_name, hooks, options);
>> > +
>> >  cleanup:
>> >  	run_hooks_opt_clear(options);
>> > +	clear_hook_list(hooks);
>> 
>> ...the oneshot command also does clear_hook_list(), after calling
>> run_hooks() which cleared it already.  That looks like a mistake,
>> i.e. we should always trust run_hooks() to clear it, no?
>
> Ah, good catch. I will update the comment on run_hooks() and fix
> _oneshot() :)
>
>  - Emily

I found a further memory issue with this, on "seen" running e.g. t0001
when compiled with SANITIZE=leak is broken by this series.

It's because in clear_hook_list() you clear the list of hooks, but
forget to clear the malloc'd container, so a missing free() fixes it. As
in the POC patch at the end of this mail.

But e.g. "git hook list <name>" is still broken, easy enough to fix,
just also needs fixing of the list_hooks_gently() callsites to e.g. this:
    
    diff --git a/builtin/hook.c b/builtin/hook.c
    index 50233366a8..2cd1831075 100644
    --- a/builtin/hook.c
    +++ b/builtin/hook.c
    @@ -48,8 +48,10 @@ static int list(int argc, const char **argv, const char *prefix)
     
     	head = list_hooks_gently(hookname);
     
    -	if (list_empty(head))
    +	if (list_empty(head)) {
    +		clear_hook_list(head);
     		return 1;
    +	}
     
     	list_for_each(pos, head) {
     		struct hook *item = list_entry(pos, struct hook, list);

Although going to that length to make the SANITIZE=leak run clean is
arguably overdoing it...

diff --git a/hook.c b/hook.c
index 23af86b9ea..e6e1e4173a 100644
--- a/hook.c
+++ b/hook.c
@@ -67,6 +67,7 @@ void clear_hook_list(struct list_head *head)
 	struct list_head *pos, *tmp;
 	list_for_each_safe(pos, tmp, head)
 		remove_hook(pos);
+	free(head);
 }
 
 static int known_hook(const char *name)
@@ -142,7 +143,16 @@ const char *find_hook_gently(const char *name)
 
 int hook_exists(const char *name)
 {
-	return !list_empty(list_hooks(name));
+	struct list_head *hooks;
+	int exists;
+
+	hooks = list_hooks(name);
+
+	exists = !list_empty(hooks);
+
+	clear_hook_list(hooks);
+
+	return exists;
 }
 
 struct hook_config_cb

^ permalink raw reply related	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 5/6] hook: include hooks from the config
  2021-08-24 19:30                   ` Ævar Arnfjörð Bjarmason
@ 2021-08-31 19:05                     ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-31 19:05 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Tue, Aug 24, 2021 at 09:30:25PM +0200, Ævar Arnfjörð Bjarmason wrote:

Disclaimer: I was writing a pretty involved reply to this and my tmux
session crashed, so hopefully I can recall it well enough. Sorry if
anything is confusing :)

> 
> On Wed, Aug 18 2021, Emily Shaffer wrote:
> 
> > Teach the hook.[hc] library to parse configs to populare the list of
> > hooks to run for a given event.
> 
> s/populare/populate/
ack

> 
> > Multiple commands can be specified for a given hook by providing
> > multiple "hook.<friendly-name>.command = <path-to-hook>" and
> > "hook.<friendly-name>.event = <hook-event>" lines. Hooks will be run in
> > config order of the "hook.<name>.event" lines.
> 
> The "will be run in order" probably needs some amending to account for
> --jobs, no?

I changed it to "started in order", and tacked on "(but may be run in
parallel)".

> 
> > For example:
> >
> >   $ git config --list | grep ^hook
> >   hook.bar.command=~/bar.sh
> >   hook.bar.event=pre-commit
> 
> Perhaps the example should use:
> 
>     git config --get-regexp '^hook\.'

Sure, I better not inflict my crappy habits on readers ;)

> 
> >   $ git hook run
> >   # Runs ~/bar.sh
> >   # Runs .git/hooks/pre-commit
> 
> And this "# Runs" is not actual output by git, but just an explanation
> for what happens, better to reword it somehow so it doesn't give that
> impression.
> 
> But the example also seems to be broken, surely it should be "git hook
> run bar", not "git hook run"? Reading ahead, but presumably no-arg
> doesn't run all known hooks...

Ah, even the suggestion was wrong - it should be `git hook run
pre-commit`. Zzzz. :)

> 
> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > ---
> >  Documentation/config/hook.txt |  18 ++++
> >  Documentation/git-hook.txt    |  57 ++++++++++++-
> >  builtin/hook.c                |   3 +-
> >  hook.c                        | 153 ++++++++++++++++++++++++++++++----
> >  hook.h                        |   7 +-
> >  t/t1800-hook.sh               | 141 ++++++++++++++++++++++++++++++-
> >  6 files changed, 357 insertions(+), 22 deletions(-)
> >
> > diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
> > index 96d3d6572c..c394756328 100644
> > --- a/Documentation/config/hook.txt
> > +++ b/Documentation/config/hook.txt
> > @@ -1,3 +1,21 @@
> > +hook.<name>.command::
> > +	A command to execute whenever `hook.<name>` is invoked. `<name>` should
> > +	be a unique "friendly" name which you can use to identify this hook
> > +	command. (You can specify when to invoke this command with
> > +	`hook.<name>.event`.) The value can be an executable on your device or a
> > +	oneliner for your shell. If more than one value is specified for the
> > +	same `<name>`, the last value parsed will be the only command executed.
> > +	See linkgit:git-hook[1].
> 
> Hrm, so here we say "If more than one value is specified for ... the
> last value parsed will be the only command executed", but in the commit
> message it's:
> 
>     Multiple commands can be specified for a given hook by providing
>     multiple "hook.<friendly-name>.command = <path-to-hook>" and
>     "hook.<friendly-name>.event = <hook-event>" lines.
> 
> As we've discussed earlier I've got a preference for the former, but
> just reading this commit message & doc the for the first time I still
> don't know what you went for, looks like one or the other needs
> updating. I'm intentionally not reading ahead as I review this.
> 
> Saying that it's a "unique name", but also discussing what happens if
> it's not unique in the sense that we have multiple "hook.<name>.*" is a
> bit confusing. I think I know what you're going for, perhaps something
> like this would be better to describe it:
> 
>     For a "hook.<name>.{command,event}" hook entry you'll need to pick a
>     "<name>" that's not shared with any other hook, if you do normal
>     single-value config semantics apply and git will use the last
>     provided value.
> 
> Or something...

Yeah, I ended up reworking this a lot. I think the manpage needs some
structuring work, to be honest - I wrote a lot about how to use 'git
hook run' to add hooks to your wrapper around Git, for example, and
stuck it in a section.

While I'm waiting for your next reroll, I'm planning to send the doc to
a tech writer we've got on loan internally and see if they've got any
tips for us here. Hopefully they can help us out :)

I'm going to snip the rest of the doc comments, because while I know I
took action on them last week, I'm not sure I recall what I changed;
and I'm hoping to get a third party to take a look. I did read them all, I
promise :)

> > +	/* to enable oneliners, let config-specified hooks run in shell.
> > +	 * config-specified hooks have a name. */
> 
> nit: usual style is multi-line comments like:
> 
>     /*
>      * some text[...]
>      * some more...
>      */
> 
> Not:
> 
>     /* text here right away[...]
>      * some more ... */

ACK. By the way, anybody have tips for making Vim gracefully transition
from

 /* happily typing a single line comment

to

 /*
  * happily typing a single line comment that
  * became super long

Wonder if anything is working quite well for anybody, because I mess
this up all the time ;)

I guess I could just check for this kind of thing at pre-commit time.
Maybe with a hook. ;) ;)

> 
> > +	cp->use_shell = !!run_me->name;
> > +
> >  	/* add command */
> > -	if (hook_cb->options->absolute_path)
> > -		strvec_push(&cp->args, absolute_path(run_me->hook_path));
> > -	else
> > -		strvec_push(&cp->args, run_me->hook_path);
> > +	if (run_me->name) {
> > +		/* ...from config */
> > +		struct strbuf cmd_key = STRBUF_INIT;
> > +		char *command = NULL;
> > +
> > +		strbuf_addf(&cmd_key, "hook.%s.command", run_me->name);
> 
> Missing strbuf_release() for this later?

Yep, thanks.

> 
> > +		if (git_config_get_string(cmd_key.buf, &command)) {
> > +			/* TODO test me! */
> 
> ...seems easy enough to just have a test for..., i.e. an *.event entry
> with no *.command.

Yeah, this TODO was probably for myself, so I wouldn't push the reroll
without writing the test. That worked really well....

> 
> > +			die(_("'hook.%s.command' must be configured "
> > +			      "or 'hook.%s.event' must be removed; aborting.\n"),
> > +			    run_me->name, run_me->name);
> > +		}
> > +
> > +		strvec_push(&cp->args, command);
> > +	} else {
> > +		/* ...from hookdir. */
> > +		const char *hook_path = NULL;
> > +		/*
> > +		 *
> 
> Nit: Too few \n before the text in a comment earlier, too many here :)
*facepalm*

> 
> > +		 * At this point we are already running, so don't validate
> > +		 * whether the hook name is known or not.
> 
> ...because it was done earlier somewhere, or...?

Yeah, that validation occurs in list_hooks() instead. I'll make the
comment more clear.

> 
> > +		 */
> > +		hook_path = find_hook_gently(hook_cb->hook_name);
> > +		if (!hook_path)
> > +			BUG("hookdir hook in hook list but no hookdir hook present in filesystem");
> > +
> > +		if (hook_cb->options->absolute_path)
> > +			hook_path = absolute_path(hook_path);
> > +
> > +		strvec_push(&cp->args, hook_path);
> > +	}
> > +
> >  
> >  	/*
> >  	 * add passed-in argv, without expanding - let the user get back
> > @@ -228,8 +346,11 @@ static int notify_start_failure(struct strbuf *out,
> >  
> >  	hook_cb->rc |= 1;
> >  
> > -	strbuf_addf(out, _("Couldn't start hook '%s'\n"),
> > -		    attempted->hook_path);
> > +	if (attempted->name)
> > +		strbuf_addf(out, _("Couldn't start hook '%s'\n"),
> > +		    attempted->name);
> > +	else
> > +		strbuf_addstr(out, _("Couldn't start hook from hooks directory\n"));
> >  
> >  	return 1;
> >  }
> > diff --git a/hook.h b/hook.h
> > index 6b7b2d14d2..621bd2cde1 100644
> > --- a/hook.h
> > +++ b/hook.h
> > @@ -27,8 +27,11 @@ int hook_exists(const char *hookname);
> >  
> >  struct hook {
> >  	struct list_head list;
> > -	/* The path to the hook */
> > -	const char *hook_path;
> > +	/*
> > +	 * The friendly name of the hook. NULL indicates the hook is from the
> > +	 * hookdir.
> > +	 */
> > +	const char *name;
> >  
> >  	/*
> >  	 * Use this to keep state for your feed_pipe_fn if you are using
> > diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
> > index 217db848b3..ef2432f53a 100755
> > --- a/t/t1800-hook.sh
> > +++ b/t/t1800-hook.sh
> > @@ -1,13 +1,29 @@
> >  #!/bin/bash
> >  
> > -test_description='git-hook command'
> > +test_description='git-hook command and config-managed multihooks'
> >  
> >  . ./test-lib.sh
> >  
> > +setup_hooks () {
> > +	test_config hook.ghi.event pre-commit --add
> > +	test_config hook.ghi.command "/path/ghi" --add
> > +	test_config_global hook.def.event pre-commit --add
> > +	test_config_global hook.def.command "/path/def" --add
> 
> Isn't --add redundant here? Seems no test is stressing multi-value
> hook.{ghi,def}.* from a quick glance...

Sounds like a failing in the test suite ;)

Will remove --add from the command configs, and will stick in test for a
hook to run in multiple places.

I went back and forth over whether to add the extra .event config in the
setup vs. in a specific test, and I decided that by adding it in the
setup, we get some implicit "this is fine" assurance, as well as the one
explicit test (which I added just now) that the hook shows up in both
places.

> 
> > +}
> > +
> > +setup_hookdir () {
> > +	mkdir .git/hooks
> > +	write_script .git/hooks/pre-commit <<-EOF
> > +	echo \"Legacy Hook\"
> > +	EOF
> > +	test_when_finished rm -rf .git/hooks
> > +}
> > +
> >  test_expect_success 'git hook usage' '
> >  	test_expect_code 129 git hook &&
> >  	test_expect_code 129 git hook run &&
> >  	test_expect_code 129 git hook run -h &&
> > +	test_expect_code 129 git hook list -h &&
> 
> Doesn't this belong in a previous commit that added "git hook list", not
> here?

Yes, nice. Thanks.

> 
> >  	test_expect_code 129 git hook run --unknown 2>err &&
> >  	grep "unknown option" err
> >  '
> > @@ -153,4 +169,127 @@ test_expect_success 'stdin to hooks' '
> >  	test_cmp expect actual
> >  '
> >  
> > +test_expect_success 'git hook list orders by config order' '
> > +	setup_hooks &&
> > +
> > +	cat >expected <<-EOF &&
> > +	def
> > +	ghi
> > +	EOF
> > +
> > +	git hook list pre-commit >actual &&
> > +	test_cmp expected actual
> > +'
> > +
> > +test_expect_success 'git hook list reorders on duplicate event declarations' '
> > +	setup_hooks &&
> > +
> > +	# 'def' is usually configured globally; move it to the end by
> > +	# configuring it locally.
> > +	test_config hook.def.event "pre-commit" --add &&
> 
> Ah, well the --add belongs here, but not needed in setup_hooks, right?

Addressed above.

> > +
> > +	cat >input <<-EOF &&
> > +	1
> > +	2
> > +	3
> > +	EOF
> > +
> > +	cat >expected <<-EOF &&
> > +	a1
> > +	a2
> > +	a3
> > +	b1
> > +	b2
> > +	b3
> > +	EOF
> 
> For any here-docs without variables, use <<-\EOF, note the backslash.
ACK


Thanks.
 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 6/6] hook: allow out-of-repo 'git hook' invocations
  2021-08-24 20:12                   ` Ævar Arnfjörð Bjarmason
  2021-08-24 20:38                     ` Randall S. Becker
@ 2021-08-31 21:09                     ` Emily Shaffer
  1 sibling, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-08-31 21:09 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: git

On Tue, Aug 24, 2021 at 10:12:04PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Wed, Aug 18 2021, Emily Shaffer wrote:
> 
> > Since hooks can now be supplied via the config, and a config can be
> > present without a gitdir via the global and system configs, we can start
> > to allow 'git hook run' to occur without a gitdir. This enables us to do
> > things like run sendemail-validate hooks when running 'git send-email'
> > from a nongit directory.
> 
> Sensible goal. Perhaps we should note in an earlier commit when
> config-based hooks are introduced something like:
> 
>     Even though we've added config-based hooks, they currently only work
>     if we can find a .git directory, even though certain commands such
>     as "git send-email" (or only that command?) can be run outside of a
>     git directory. A subsequent commit will address that edge-case.

Hmmm. I personally do not like "a subsequent commit adds this" in commit
messages. But I'm having trouble expressing why :) To me, "I know about
me and everything that came before me" is fine and "I wish in the future
x would happen" is fine, for commit messages, but "x will happen in the
future" feels a little wrong.

Anyway, I think it is a little distracting to include that message in
the earlier commit, since the "doesn't work outside of a repo" aspect of
hooks is not changing at all.

> 
> > [...]
> > Notes:
> >     For hookdir hooks, do we want to run them in nongit dir when core.hooksPath
> >     is set? For example, if someone set core.hooksPath in their global config and
> >     then ran 'git hook run sendemail-validate' in a nongit dir?
> > [...]
> >  git.c           |  2 +-
> >  hook.c          |  2 +-
> >  t/t1800-hook.sh | 20 +++++++++++++++-----
> >  3 files changed, 17 insertions(+), 7 deletions(-)
> >
> > diff --git a/git.c b/git.c
> > index 540909c391..39988ee3b0 100644
> > --- a/git.c
> > +++ b/git.c
> > @@ -538,7 +538,7 @@ static struct cmd_struct commands[] = {
> >  	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
> >  	{ "hash-object", cmd_hash_object },
> >  	{ "help", cmd_help },
> > -	{ "hook", cmd_hook, RUN_SETUP },
> > +	{ "hook", cmd_hook, RUN_SETUP_GENTLY },
> >  	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
> >  	{ "init", cmd_init_db },
> >  	{ "init-db", cmd_init_db },
> > diff --git a/hook.c b/hook.c
> > index 581d87cbbd..2e08156546 100644
> > --- a/hook.c
> > +++ b/hook.c
> > @@ -218,7 +218,7 @@ struct list_head *list_hooks_gently(const char *hookname)
> >  
> >  	/* Add the hook from the hookdir. The placeholder makes it easier to
> >  	 * allocate work in pick_next_hook. */
> > -	if (find_hook_gently(hookname))
> > +	if (have_git_dir() && find_hook_gently(hookname))
> >  		append_or_move_hook(hook_head, NULL);
> >  
> >  	return hook_head;
> > diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
> > index ef2432f53a..a7e45c0d16 100755
> > --- a/t/t1800-hook.sh
> > +++ b/t/t1800-hook.sh
> > @@ -118,15 +118,25 @@ test_expect_success 'git hook run -- pass arguments' '
> >  	test_cmp expect actual
> >  '
> >  
> > -test_expect_success 'git hook run -- out-of-repo runs excluded' '
> > -	write_script .git/hooks/test-hook <<-EOF &&
> > -	echo Test hook
> > -	EOF
> > +test_expect_success 'git hook run: out-of-repo runs execute global hooks' '
> > +	test_config_global hook.global-hook.event test-hook --add &&
> > +	test_config_global hook.global-hook.command "echo no repo no problems" --add &&
> > +
> > +	echo "global-hook" >expect &&
> > +	nongit git hook list test-hook >actual &&
> > +	test_cmp expect actual &&
> > +
> > +	echo "no repo no problems" >expect &&
> >  
> > -	nongit test_must_fail git hook run test-hook
> > +	nongit git hook run test-hook 2>actual &&
> > +	test_cmp expect actual
> >  '
> >  
> >  test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
> > +	write_script .git/hooks/test-hook <<-EOF &&
> > +	echo Test hook
> > +	EOF
> > +
> >  	mkdir my-hooks &&
> >  	write_script my-hooks/test-hook <<-\EOF &&
> >  	echo Hook ran $1 >>actual
> 
> If the only user of this is git-send-email, let's have tests for this in
> t/t9001-send-email.sh. That should also address your "Notes" above,
> i.e. let's just test it with core.hooksPath and see what the interaction
> looks like.

Yeah, I think it is probably fine to do allllll the git-send-email
fixups in this one commit, actually. I think your point is correct that
right now the only command which can benefit is git-send-email. It might
also be reasonable to drop this change from this series, but I won't
bother doing so unless it looks like this one commit is holding up the
rest of the series.

I could see, though, a later world where we might want something like a
"pre-clone" or "post-clone"(-pre-checkout) hook, now that we don't need
to have a .gitdir in order to run any hooks. Will make a very brief
"later it might be nice if..." addition to the commit message.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: [PATCH v4 00/36] Run hooks via "git run hook" & hook library
  2021-08-19 23:40             ` Emily Shaffer
@ 2021-09-02  7:21               ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02  7:21 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m . carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee


On Thu, Aug 19 2021, Emily Shaffer wrote:

I'm submitting a re-roll of this soon, just some notes.

> On Tue, Aug 03, 2021 at 09:38:26PM +0200, Ævar Arnfjörð Bjarmason wrote:
>> 
>> This is a v4 re-roll of the "Base for "config-based-hooks" topic.
>>  * Almost all the callers were just "one-shot" callers, I introduced a
>>    new run_hooks_oneshot() function for those, which gets rid of the
>>    verbosity around memory management, see e.g. the "builtin/gc.c" in
>>    the range-diff below. That run_hooks_oneshot() can also take a NULL
>>    set of options.
>
> I discussed this elswhere, but just to make doubly sure that comment
> doesn't get lost, here's a link to my mail:
> https://lore.kernel.org/git/YR2jLdYQA5CVzX5h%40google.com
>
>> Range-diff against v3:
>>  1:  27c94247f87 =  1:  81fe1ed90d5 Makefile: mark "check" target as .PHONY
>>  2:  6e164edb0b0 !  2:  0f749530777 Makefile: stop hardcoding {command,config}-list.h
>>  3:  ddae86802e2 !  3:  644b31fe281 Makefile: remove an out-of-date comment
>>  4:  58c37e4f06e =  4:  89c4d44b0c3 hook.[ch]: move find_hook() to this new library
>>  5:  0cf7e078ef4 =  5:  3514e0c0251 hook.c: add a hook_exists() wrapper and use it in bugreport.c
>>  -:  ----------- >  6:  d5ef40f77dc hook.c users: use "hook_exists()" insted of "find_hook()"
>>  6:  f343fc7ae66 !  7:  4cfd72722c1 hook-list.h: add a generated list of hooks, like config-list.h
>
> Since these 6 were broken out and then brought back into this series,
> I'll review them individually downthread.
>
>>  7:  cf4b06bfdf8 !  8:  7cb4a4cb69e hook: add 'run' subcommand
>>     @@ builtin/hook.c (new)
>>      +#include "strbuf.h"
>>      +#include "strvec.h"
>>      +
>>     ++#define BUILTIN_HOOK_RUN_USAGE \
>>     ++	N_("git hook run <hook-name> [-- <hook-args>]")
>>     ++
>
> Nice, now we avoid string duplication here...
>
>>      +static const char * const builtin_hook_usage[] = {
>>     -+	N_("git hook <command> [...]"),
>
> ...and there is no point including this vague thing that has a more
> specific description right after. Ok.
>
>>     -+	N_("git hook run <hook-name> [-- <hook-args>]"),
>>     ++	BUILTIN_HOOK_RUN_USAGE,
>>      +	NULL
>>      +};
> [...]
>>     @@ builtin/hook.c (new)
>>      +{
>>      +	int i;
>>      +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
>>     -+	int rc = 0;
>>      +	const char *hook_name;
>>      +	const char *hook_path;
>>     -+
>>      +	struct option run_options[] = {
>>      +		OPT_END(),
>>      +	};
>>     ++	int ret;
>>      +
>>      +	argc = parse_options(argc, argv, prefix, run_options,
>>      +			     builtin_hook_run_usage,
>>     -+			     PARSE_OPT_KEEP_UNKNOWN | PARSE_OPT_KEEP_DASHDASH);
>>     -+
>>     -+	if (argc > 1) {
>>     -+		if (strcmp(argv[1], "--") &&
>>     -+		    strcmp(argv[1], "--end-of-options"))
>>     -+			/* Having a -- for "run" is mandatory */
>>     -+			usage_with_options(builtin_hook_usage, run_options);
>>     -+		/* Add our arguments, start after -- */
>>     -+		for (i = 2 ; i < argc; i++)
>>     -+			strvec_push(&opt.args, argv[i]);
>>     -+	}
>>     ++			     PARSE_OPT_KEEP_DASHDASH);
>
> Nice - this code is being broken up and moved later.
>
>>      +
>>     -+	/* Need to take into account core.hooksPath */
>>     -+	git_config(git_default_config, NULL);
>>     ++	if (!argc)
>>     ++		goto usage;
>>      +
>>      +	/*
>>     -+	 * We are not using run_hooks() because we'd like to detect
>>     -+	 * missing hooks. Let's find it ourselves and call
>>     -+	 * run_found_hooks() instead.
>
> This comment disappears entirely. I am not too terribly upset about it,
> since this behavior changes in my series anyway, but it seems like it
> might have been an accident?

I removed it intentionally, I thought it was clear enough from the
control flow, but added it back in. Thanks.

>>     ++	 * Having a -- for "run" when providing <hook-args> is
>>     ++	 * mandatory.
>>      +	 */
>>     ++	if (argc > 1 && strcmp(argv[1], "--") &&
>>     ++	    strcmp(argv[1], "--end-of-options"))
>>     ++		goto usage;
>>     ++
>>     ++	/* Add our arguments, start after -- */
>>     ++	for (i = 2 ; i < argc; i++)
>>     ++		strvec_push(&opt.args, argv[i]);
>>     ++
>>     ++	/* Need to take into account core.hooksPath */
>>     ++	git_config(git_default_config, NULL);
>>     ++
>>      +	hook_name = argv[0];
>>      +	hook_path = find_hook(hook_name);
>>      +	if (!hook_path) {
>>      +		error("cannot find a hook named %s", hook_name);
>>      +		return 1;
>>      +	}
>>     -+	rc = run_found_hooks(hook_name, hook_path, &opt);
>>      +
>>     ++	ret = run_hooks(hook_name, hook_path, &opt);
>>      +	run_hooks_opt_clear(&opt);
>>     -+
>>     -+	return rc;
>>     ++	return ret;
>>     ++usage:
>>     ++	usage_with_options(builtin_hook_run_usage, run_options);
>>      +}
>>      +
>>      +int cmd_hook(int argc, const char **argv, const char *prefix)
>>     @@ builtin/hook.c (new)
>>      +	struct option builtin_hook_options[] = {
>>      +		OPT_END(),
>>      +	};
>>     ++
>>      +	argc = parse_options(argc, argv, NULL, builtin_hook_options,
>>      +			     builtin_hook_usage, PARSE_OPT_STOP_AT_NON_OPTION);
>>      +	if (!argc)
>>     -+		usage_with_options(builtin_hook_usage, builtin_hook_options);
>>     ++		goto usage;
>>      +
>>      +	if (!strcmp(argv[0], "run"))
>>      +		return run(argc, argv, prefix);
>>     -+	else
>>     -+		usage_with_options(builtin_hook_usage, builtin_hook_options);
>>     ++
>>     ++usage:
>>     ++	usage_with_options(builtin_hook_usage, builtin_hook_options);
>>      +}
>
> The goto pattern seems readable enough elsewhere, though.
>
>>       ## command-list.txt ##
>>     @@ hook.c: int hook_exists(const char *name)
>>      +	struct hook_cb_data *hook_cb = pp_cb;
>>      +	struct hook *run_me = hook_cb->run_me;
>>      +
>>     ++	if (!run_me)
>>     ++		return 0;
>>     ++
> Now we protect ourselves from the repeated calls to pick_next_hook().
> Good.
>>      +	cp->no_stdin = 1;
>>      +	cp->env = hook_cb->options->env.v;
>>      +	cp->stdout_to_stderr = 1;
>>     @@ hook.c: int hook_exists(const char *name)
>>      +	/* Provide context for errors if necessary */
>>      +	*pp_task_cb = run_me;
>>      +
>>     ++	/*
>>     ++	 * This pick_next_hook() will be called again, we're only
>>     ++	 * running one hook, so indicate that no more work will be
>>     ++	 * done.
>>     ++	 */
>>     ++	hook_cb->run_me = NULL;
>>     ++
> And clearly explaining the caller's behavior here. Sure.
>>      +	return 1;
>>      +}
>>      +
>>     @@ hook.c: int hook_exists(const char *name)
>>      +
>>      +	hook_cb->rc |= result;
>>      +
>>     -+	return 1;
>>     ++	return 0;
> And finally, report "everything is fine" on task finished. Thanks.
>>      +}
>>      +
>>     -+int run_found_hooks(const char *hook_name, const char *hook_path,
>>     -+		    struct run_hooks_opt *options)
>>     ++int run_hooks(const char *hook_name, const char *hook_path,
>>     ++	      struct run_hooks_opt *options)
>
> Ok. With the rename, we only have "verbose way to call it" and "speedy
> way to call it". No more "run_found_hooks()". Thanks, I think I like
> this better - I found "run_found_hooks()" to be a little ambiguous.
>
>>     @@ hook.c: int hook_exists(const char *name)
>>      +				   hook_name);
>>      +
>>      +	return cb_data.rc;
>>     -+}
>>     -+
>>     -+int run_hooks(const char *hook_name, struct run_hooks_opt *options)
>>     -+{
>>     -+	const char *hook_path;
>>     -+	int ret;
>>     -+	if (!options)
>>     -+		BUG("a struct run_hooks_opt must be provided to run_hooks");
>>     -+
>>     -+	hook_path = find_hook(hook_name);
>>     -+
>>     -+	/*
>>     -+	 * If you need to act on a missing hook, use run_found_hooks()
>>     -+	 * instead
>>     -+	 */
>>     -+	if (!hook_path)
>>     -+		return 0;
>>     -+
>>     -+	ret = run_found_hooks(hook_name, hook_path, options);
>>     -+	return ret;
>
> Ok - so we wait for another patch to introduce run_hooks_oneshot. Fine.
>
>>      +}
>>      
>>       ## hook.h ##
>>     @@ hook.h: const char *find_hook(const char *name);
>>      +
>>      +	/* Args to be passed to each hook */
>>      +	struct strvec args;
>>     -+
>>     -+	/*
>>     -+	 * Number of threads to parallelize across, currently a stub,
>>     -+	 * we use the parallel API for future-proofing, but we always
>>     -+	 * have one hook of a given name, so this is always an
>>     -+	 * implicit 1 for now.
>>     -+	 */
>>     -+	int jobs;
>
> Thanks. I think this makes more sense than the weird "futureproofing"
> state it was in before.
>
>>       ## t/t1800-hook.sh (new) ##
>>     @@ t/t1800-hook.sh (new)
>>      +
>>      +test_expect_success 'git hook usage' '
>>      +	test_expect_code 129 git hook &&
>>     -+	test_expect_code 129 git hook -h &&
>>     -+	test_expect_code 129 git hook run -h
>>     ++	test_expect_code 129 git hook run &&
>>     ++	test_expect_code 129 git hook run -h &&
>>     ++	test_expect_code 129 git hook run --unknown 2>err &&
>>     ++	grep "unknown option" err
>>      +'
>
> Nice. I was able to use this test myself when I added 'git hook list',
> too.
>
> The changes to this patch look good to me.
>
>>  8:  7209f73f281 !  9:  2b8500aa675 gc: use hook library for pre-auto-gc hook
>>     @@ Metadata
>>       ## Commit message ##
>>          gc: use hook library for pre-auto-gc hook
>>      
>>     -    Using the hook.h library instead of the run-command.h library to run
>>     -    pre-auto-gc means that those hooks can be set up in config files, as
>>     -    well as in the hookdir. pre-auto-gc is called only from builtin/gc.c.
>>     +    Move the pre-auto-gc hook away from run-command.h to and over to the
>>     +    new hook.h library.
>>     +
>>     +    To do this introduce a simple run_hooks_oneshot() wrapper, we'll be
>>     +    using it extensively for these simple cases of wanting to run a single
>>     +    hook under a given name, and having it free the memory we allocate for
>>     +    us.
>
> Cool, so we have an excuse for introducing the oneshot wrapper right
> away. Thanks.
>
>>     -@@ builtin/gc.c: static void add_repack_incremental_option(void)
>>     - 
>>     - static int need_to_gc(void)
>>     - {
>>     -+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
>>     -+
>>     - 	/*
>>     - 	 * Setting gc.auto to 0 or negative can disable the
>>     - 	 * automatic gc.
>>      @@ builtin/gc.c: static int need_to_gc(void)
>>       	else
>>       		return 0;
>>       
>>      -	if (run_hook_le(NULL, "pre-auto-gc", NULL))
>>     -+	if (run_hooks("pre-auto-gc", &hook_opt)) {
>>     -+		run_hooks_opt_clear(&hook_opt);
>>     ++	if (run_hooks_oneshot("pre-auto-gc", NULL))
>>       		return 0;
>>     -+	}
>>     -+	run_hooks_opt_clear(&hook_opt);
>>       	return 1;
>>       }
>
> Nice - the callsite looks much tidier now.
>
>>     +
>>     + ## hook.c ##
>>     +@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
>>       
>>     + 	return cb_data.rc;
>>     + }
>>     ++
>>     ++int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
>>     ++{
>>     ++	const char *hook_path;
>>     ++	int ret;
>>     ++	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
>>     ++
>>     ++	if (!options)
>>     ++		options = &hook_opt_scratch;
>
> Ah, it still takes an options (optionally, ha ha) because run_hooks()
> would require the caller to provide the hook list/hook path. Ok. Seems
> fine, and I like that the '_oneshot' naming change makes it less
> surprising that run_hooks_opt_clear() will get called for you.
>
>>     ++
>>     ++	hook_path = find_hook(hook_name);
>>     ++	if (!hook_path) {
>>     ++		ret = 0;
>>     ++		goto cleanup;
>>     ++	}
>>     ++
>>     ++	ret = run_hooks(hook_name, hook_path, options);
>>     ++cleanup:
>>     ++	run_hooks_opt_clear(options);
>>     ++	return ret;
>>     ++}
>>     +
>>     + ## hook.h ##
>>     +@@ hook.h: void run_hooks_opt_clear(struct run_hooks_opt *o);
>>     + /**
>>     +  * Takes an already resolved hook found via find_hook() and runs
>>     +  * it. Does not call run_hooks_opt_clear() for you.
>>     ++ *
>>     ++ * See run_hooks_oneshot() for the simpler one-shot API.
>>     +  */
>>     + int run_hooks(const char *hookname, const char *hook_path,
>>     + 	      struct run_hooks_opt *options);
>>     ++
>>     ++/**
>>     ++ * Calls find_hook() on your "hook_name" and runs the hooks (if any)
>>     ++ * with run_hooks().
>>     ++ *
>>     ++ * If "options" is provided calls run_hooks_opt_clear() on it for
>>     ++ * you. If "options" is NULL a scratch one will be provided for you
>>     ++ * before calling run_hooks().
>
> "A scratch one will be provided for you" doesn't sound quite right -
> it's not exposed to the caller at all, but the comment sounds like
> you're handing the caller this scratch struct. Maybe it's better to say
> "the default options from RUN_HOOKS_OPT_INIT will be used"?
>
> As a bonus, if you directly reference the initter macro, then it will
> automatically become clear to folks reading the documentation what the
> expected parallelism is on their hook, as the parallelism (_SERIES or
> _PARALLEL) is included on the name of the macro in my change later.

*Nod*, did that.

>>     ++ */
>>     ++int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
>>     ++
>>     + #endif
>
>>  9:  e9a1e7cf61e ! 10:  3ee55d2c10f rebase: teach pre-rebase to use hook.h
>>     @@ Metadata
>>      Author: Emily Shaffer <emilyshaffer@google.com>
>>      
>>       ## Commit message ##
>>     -    rebase: teach pre-rebase to use hook.h
>>     +    rebase: convert pre-rebase to use hook.h
>>      
>>          Move the pre-rebase hook away from run-command.h to and over to the
>>          new hook.h library.
>>     @@ builtin/rebase.c: int cmd_rebase(int argc, const char **argv, const char *prefix
>>       	if (!ok_to_skip_pre_rebase &&
>>      -	    run_hook_le(NULL, "pre-rebase", options.upstream_arg,
>>      -			argc ? argv[0] : NULL, NULL))
>>     -+	    run_hooks("pre-rebase", &hook_opt)) {
>>     -+		run_hooks_opt_clear(&hook_opt);
>>     ++	    run_hooks_oneshot("pre-rebase", &hook_opt))
>
> Ok, it just uses the oneshot call instead. Looks good.
>> 10:  1d087269303 ! 11:  050f20d14f0 am: convert applypatch hooks to use config
>> 11:  32eec5dc2f0 ! 12:  ac875d284da hooks: convert 'post-checkout' hook to hook library
>> 12:  e9fa3f67593 ! 13:  69763bc2255 merge: use config-based hooks for post-merge hook
>
> The above all just use the oneshot instead, and have some minor tweaks
> to commit messages to lose references to the config. Looks good.
>
>> 13:  12347d901bb ! 14:  2ca1ca1b8e4 git hook run: add an --ignore-missing flag
>>     @@ Documentation/git-hook.txt: optional `--` (or `--end-of-options`, see linkgit:gi
>>       linkgit:githooks[5]
>>      
>>       ## builtin/hook.c ##
>>     +@@
>>     + #include "strvec.h"
>>     + 
>>     + #define BUILTIN_HOOK_RUN_USAGE \
>>     +-	N_("git hook run <hook-name> [-- <hook-args>]")
>>     ++	N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")
>
> Ok. We update the usage in the macro here instead. Nice.
>
>>      @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
>>     - 	/*
>>     - 	 * We are not using run_hooks() because we'd like to detect
>>     - 	 * missing hooks. Let's find it ourselves and call
>>     --	 * run_found_hooks() instead.
>>     -+	 * run_found_hooks() instead...
>>     - 	 */
>>     + 	git_config(git_default_config, NULL);
>>     + 
>>       	hook_name = argv[0];
>>     ++	if (ignore_missing)
>>     ++		return run_hooks_oneshot(hook_name, &opt);
>>       	hook_path = find_hook(hook_name);
>>       	if (!hook_path) {
>>     -+		/* ... act like run_hooks() under --ignore-missing */
>>     -+		if (ignore_missing)
>>     -+			return 0;
>>       		error("cannot find a hook named %s", hook_name);
>>     - 		return 1;
>>     - 	}
>
> Hm. I actually think this was clearer before. I *think* that is because
> you and I have different opinions on the clarity of multiple returns :)
>
> But I think it is a little confusing to say "ok, we will call an
> entirely different entry point in case of this one thing which we will
> have to check anyways".

Tried to reword this per the above..

>> 14:  71d209b4077 ! 15:  5b66b04bec7 send-email: use 'git hook run' for 'sendemail-validate'
>>     @@ git-send-email.perl: sub validate_patch {
>>       	if ($repo) {
>>      +		my $hook_name = 'sendemail-validate';
>>       		my $hooks_path = $repo->command_oneline('rev-parse', '--git-path', 'hooks');
>>     --		my $validate_hook = catfile($hooks_path,
>>     + 		require File::Spec;
>>     +-		my $validate_hook = File::Spec->catfile($hooks_path,
>>      -					    'sendemail-validate');
>>     -+		my $validate_hook = catfile($hooks_path, $hook_name);
>>     ++		my $validate_hook = File::Spec->catfile($hooks_path, $hook_name);
>
> Ok, we are adapting to some other change around File::Spec that happened
> to git-send-email.perl elsewhere. The change to this patch looks good to
> me.
>
>>       		my $hook_error;
>>       		if (-x $validate_hook) {
>>     - 			my $target = abs_path($fn);
>>     + 			require Cwd;
>>      @@ git-send-email.perl: sub validate_patch {
>>       			chdir($repo->wc_path() or $repo->repo_path())
>>       				or die("chdir: $!");
>> 15:  246a82b55b2 = 16:  14a37a43db2 git-p4: use 'git hook' to run hooks
>> 16:  e3f8482d803 ! 17:  ad5d0e0e7de commit: use hook.h to execute hooks
>>     @@ Metadata
>>      Author: Emily Shaffer <emilyshaffer@google.com>
>>      
>>       ## Commit message ##
>>     -    commit: use hook.h to execute hooks
>>     +    commit: convert {pre-commit,prepare-commit-msg} hook to hook.h
>>      
>>     -    Teach run_commit_hook() to call hook.h instead of run-command.h. This
>>     -    covers 'pre-commit', 'commit-msg', and
>>     -    'prepare-commit-msg'.
>>     -
>>     -    Additionally, ask the hook library - not run-command - whether any
>>     -    hooks will be run, as it's possible hooks may exist in the config but
>>     -    not the hookdir.
>>     -
>>     -    Because all but 'post-commit' hooks are expected to make some state
>>     -    change, force all but 'post-commit' hook to run in series. 'post-commit'
>>     -    "is meant primarily for notification, and cannot affect the outcome of
>>     -    `git commit`," so it is fine to run in parallel.

I tried to address all of the above.

> I am a little bummed that the patch reordering effectively removes the
> justification for parallel/serial execution for each hook from 'git
> blame'. Oh well.

But it'll be there if you add it to your commit to migrate them to the
parallel macros, and that'll be easier to find with "blame" than having
it tangled up in the previous "do all the hook things" changes.

So a note to reword/rebase that into your series...

>>     +    Move these hooks hook away from run-command.h to and over to the new
>>     +    hook.h library.
>>      
>>          Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
>>          Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>>      
>>       ## commit.c ##
>
> Otherwise the main change here is dealing with other improvements
> earlier in the series, like 'hook_exists()' and 'run_hooks_oneshot()'.
> Looks fine.
>
>> 17:  6ed61071c5e ! 18:  3d3a33e2674 read-cache: convert post-index-change hook to use config
>>     @@ Metadata
>>      Author: Emily Shaffer <emilyshaffer@google.com>
>>      
>>       ## Commit message ##
>>     -    read-cache: convert post-index-change hook to use config
>>     +    read-cache: convert post-index-change to use hook.h
>>      
>>     -    By using hook.h instead of run-command.h to run, post-index-change hooks
>>     -    can now be specified in the config in addition to the hookdir.
>>     -    post-index-change is not run anywhere besides in read-cache.c.
>>     +    Move the post-index-change hook away from run-command.h to and over to
>>     +    the new hook.h library.
>
> I do not think it is necessary to drop the mention about
> 'post-index-change' being run anywhere else, but that's just a nit, not
> worth fixing :)

I removed this in rewording/removing now-false mentions of "this does
config".

I didn't add the post-index-change mentiond back in, I figured to do it
consistently I'd need to start with pre-auto-gc saying it's only handled
in builtin/gc.c & all the way down the line...

>>      
>>          This removes the last direct user of run_hook_ve(), so we can make the
>>          function static now. It'll be removed entirely soon.
>>     @@ read-cache.c: static int do_write_locked_index(struct index_state *istate, struc
>>      +		     istate->updated_workdir ? "1" : "0",
>>      +		     istate->updated_skipworktree ? "1" : "0",
>>      +		     NULL);
>>     -+	run_hooks("post-index-change", &hook_opt);
>>     -+	run_hooks_opt_clear(&hook_opt);
>>     ++	run_hooks_oneshot("post-index-change", &hook_opt);
>>      +
>
> Mechanical change to using run_hooks_oneshot(). OK.
>
>>       	istate->updated_workdir = 0;
>>       	istate->updated_skipworktree = 0;
>> 18:  e4ef3f4548a ! 19:  893f8666301 receive-pack: convert push-to-checkout hook to hook.h
>>     @@ Metadata
>>       ## Commit message ##
>>          receive-pack: convert push-to-checkout hook to hook.h
>>      
>>     -    By using hook.h instead of run-command.h to invoke push-to-checkout,
>>     -    hooks can now be specified in the config as well as in the hookdir.
>>     -    push-to-checkout is not called anywhere but in builtin/receive-pack.c.
>>     +    Move the push-to-checkout hook away from run-command.h to and over to
>>     +    the new hook.h library.
>
> You made a comment in the commit before about the final instance of
> run_hook_ve; do you want to make a similar one here about run_hook_le?

Willd.o

> The range-diff here is mechanical so looks fine to me.
>
>> 19:  e3dda367ec9 = 20:  070433deba5 run-command: remove old run_hook_{le,ve}() hook API
>> 20:  477d75bf579 = 21:  1028e0c1667 run-command: allow stdin for run_processes_parallel
>> 21:  b7c0ee9719a ! 22:  639e59e9ed0 hook: support passing stdin to hooks
>>     @@ Documentation/git-hook.txt: what those are.
>>      
>>       ## builtin/hook.c ##
>>      @@
>>     + #include "strvec.h"
>>       
>>     - static const char * const builtin_hook_usage[] = {
>>     - 	N_("git hook <command> [...]"),
>>     --	N_("git hook run <hook-name> [-- <hook-args>]"),
>>     -+	N_("git hook run [<args>] <hook-name> [-- <hook-args>]"),
>>     - 	NULL
>>     - };
>>     - 
>>     - static const char * const builtin_hook_run_usage[] = {
>>     - 	N_("git hook run <hook-name> [-- <hook-args>]"),
>>     -+	N_("git hook run [--to-stdin=<path>] <hook-name> [-- <hook-args>]"),
>>     - 	NULL
>>     - };
>>     + #define BUILTIN_HOOK_RUN_USAGE \
>>     +-	N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")
>>     ++	N_("git hook run [--ignore-missing] [--to-stdin=<path>] <hook-name> [-- <hook-args>]")
>>       
> Just coping with the macro instead of the duplicated strings. Ok.
>
> The range-diff looks fine to me.
>
>> 22:  4035069a98c ! 23:  7d1925cca48 am: convert 'post-rewrite' hook to hook.h
>>     @@ builtin/am.c: static int run_applypatch_msg_hook(struct am_state *state)
>>       {
>>      -	struct child_process cp = CHILD_PROCESS_INIT;
>>      -	const char *hook = find_hook("post-rewrite");
>>     -+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
>>     - 	int ret;
>>     - 
>>     +-	int ret;
>>     +-
>>      -	if (!hook)
>>      -		return 0;
>>      -
>>     @@ builtin/am.c: static int run_applypatch_msg_hook(struct am_state *state)
>>      -	cp.in = xopen(am_path(state, "rewritten"), O_RDONLY);
>>      -	cp.stdout_to_stderr = 1;
>>      -	cp.trace2_hook_name = "post-rewrite";
>>     -+	strvec_push(&opt.args, "rebase");
>>     -+	opt.path_to_stdin = am_path(state, "rewritten");
>>     ++	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
>>       
>>      -	ret = run_command(&cp);
>>     -+	ret = run_hooks("post-rewrite", &opt);
>>     ++	strvec_push(&opt.args, "rebase");
>>     ++	opt.path_to_stdin = am_path(state, "rewritten");
>>       
>>      -	close(cp.in);
>>     -+	run_hooks_opt_clear(&opt);
>>     - 	return ret;
>>     +-	return ret;
>>     ++	return run_hooks_oneshot("post-rewrite", &opt);
>>       }
>>       
>>     + /**
>
> Man, the range-diff did a terrible job of summarizing this :)
>
> The patch itself looks great, though, with the switch to using
> run_hooks_oneshot. Thanks.
>
>> 24:  da46c859c1c ! 25:  05d1085f7eb hook: provide stdin by string_list or callback
>>     @@ hook.c: static int pick_next_hook(struct child_process *cp,
>>       	} else {
>>       		cp->no_stdin = 1;
>>       	}
>>     -@@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
>>     - 	run_processes_parallel_tr2(options->jobs,
>>     +@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
>>     + 	run_processes_parallel_tr2(jobs,
>>       				   pick_next_hook,
>>       				   notify_start_failure,
>>      -				   NULL,
>>     @@ hook.c: int run_found_hooks(const char *hook_name, const char *hook_path,
>>       				   notify_hook_finished,
>>       				   &cb_data,
>>       				   "hook",
>>     -@@ hook.c: int run_hooks(const char *hook_name, struct run_hooks_opt *options)
>>     +@@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
>>     + 
>>     + 	if (options->absolute_path)
>>     + 		strbuf_release(&abs_path);
>>     ++	free(my_hook.feed_pipe_cb_data);
>
> Nice catch. Thanks.
>
>> 25:  7343be28ef4 ! 26:  4b7175af2e5 hook: convert 'post-rewrite' hook in sequencer.c to hook.h
>> 26:  85bf13a0835 ! 27:  3f24e056410 transport: convert pre-push hook to use config
>>     @@ Metadata
>>      Author: Emily Shaffer <emilyshaffer@google.com>
>>      
>>       ## Commit message ##
>>     -    transport: convert pre-push hook to use config
>>     +    transport: convert pre-push hook to hook.h
>>      
>>     -    By using the hook.h:run_hooks API, pre-push hooks can be specified in
>>     -    the config as well as in the hookdir.
>>     +    Move the pre-push hook away from run-command.h to and over to the new
>>     +    hook.h library.
>>      
>>          Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
>>          Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>>     @@ transport.c: static void die_with_unpushed_submodules(struct string_list *needs_
>>      -	int ret = 0, x;
>>      +	int ret = 0;
>>      +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
>>     -+	struct strbuf tmp = STRBUF_INIT;
>>       	struct ref *r;
>>      -	struct child_process proc = CHILD_PROCESS_INIT;
>>      -	struct strbuf buf;
>>     @@ transport.c: static void die_with_unpushed_submodules(struct string_list *needs_
>>      -		finish_command(&proc);
>>      -		return -1;
>>      -	}
>>     --
>>     --	sigchain_push(SIGPIPE, SIG_IGN);
>>     -+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
>>     ++	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
>>       
>>     +-	sigchain_push(SIGPIPE, SIG_IGN);
>>     +-
>>      -	strbuf_init(&buf, 256);
>>      +	strvec_push(&opt.args, transport->remote->name);
>>      +	strvec_push(&opt.args, transport->url);
>>       
>>       	for (r = remote_refs; r; r = r->next) {
>>     ++		struct strbuf buf = STRBUF_INIT;
>
> Ah. It is not being freed because its lifetime is being managed by
> to_stdin instead. Ok, cool!
>
>>     ++
>>       		if (!r->peer_ref) continue;
>>     -@@ transport.c: static int run_pre_push_hook(struct transport *transport,
>>     + 		if (r->status == REF_STATUS_REJECT_NONFASTFORWARD) continue;
>>     + 		if (r->status == REF_STATUS_REJECT_STALE) continue;
>>       		if (r->status == REF_STATUS_REJECT_REMOTE_UPDATED) continue;
>>       		if (r->status == REF_STATUS_UPTODATE) continue;
>>       
>>      -		strbuf_reset(&buf);
>>      -		strbuf_addf( &buf, "%s %s %s %s\n",
>>     -+		strbuf_reset(&tmp);
>>     -+		strbuf_addf(&tmp, "%s %s %s %s",
>>     ++		strbuf_addf(&buf, "%s %s %s %s",
>>       			 r->peer_ref->name, oid_to_hex(&r->new_oid),
>>       			 r->name, oid_to_hex(&r->old_oid));
>>      -
>>     @@ transport.c: static int run_pre_push_hook(struct transport *transport,
>>      -				ret = -1;
>>      -			break;
>>      -		}
>>     -+		string_list_append(&to_stdin, tmp.buf);
>>     ++		string_list_append(&to_stdin, strbuf_detach(&buf, NULL));
>>       	}
>>       
>>      -	strbuf_release(&buf);
>>     @@ transport.c: static int run_pre_push_hook(struct transport *transport,
>>      -	x = finish_command(&proc);
>>      -	if (!ret)
>>      -		ret = x;
>>     -+	ret = run_hooks("pre-push", &opt);
>>     -+	run_hooks_opt_clear(&opt);
>>     -+	strbuf_release(&tmp);
>>     ++	ret = run_hooks_oneshot("pre-push", &opt);
>>     ++	to_stdin.strdup_strings = 1;
>
> And this is a typical change to the oneshot function. Cool.
>
>>      +	string_list_clear(&to_stdin, 0);
>>       
>>       	return ret;
>>  -:  ----------- > 28:  ecf75f33233 hook tests: test for exact "pre-push" hook input
>>  -:  ----------- > 29:  2c961be94b4 hook tests: use a modern style for "pre-push" tests
>
> I'll take a look at these separately.
>
>> 27:  331014bad17 ! 30:  1ce456f9d9d reference-transaction: use hook.h to run hooks
>>     @@ refs.c: int ref_update_reject_duplicates(struct string_list *refnames,
>>       				const char *state)
>>       {
>>      -	struct child_process proc = CHILD_PROCESS_INIT;
>>     - 	struct strbuf buf = STRBUF_INIT;
>>     +-	struct strbuf buf = STRBUF_INIT;
>>      -	const char *hook;
>>      +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
>>     -+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
>>     ++	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
>>       	int ret = 0, i;
>>     -+	char o[GIT_MAX_HEXSZ + 1], n[GIT_MAX_HEXSZ + 1];
>>       
>>      -	hook = find_hook("reference-transaction");
>>      -	if (!hook)
>>     -+	if (!hook_exists("reference-transaction"))
>>     - 		return ret;
>>     - 
>>     +-		return ret;
>>     +-
>>      -	strvec_pushl(&proc.args, hook, state, NULL);
>>      -	proc.in = -1;
>>      -	proc.stdout_to_stderr = 1;
>>     @@ refs.c: int ref_update_reject_duplicates(struct string_list *refnames,
>>      -
>>      -	ret = start_command(&proc);
>>      -	if (ret)
>>     --		return ret;
>>     --
>>     ++	if (!hook_exists("reference-transaction"))
>>     + 		return ret;
>>     + 
>>      -	sigchain_push(SIGPIPE, SIG_IGN);
>>      +	strvec_push(&opt.args, state);
>>       
>>       	for (i = 0; i < transaction->nr; i++) {
>>       		struct ref_update *update = transaction->updates[i];
>>     -+		oid_to_hex_r(o, &update->old_oid);
>>     -+		oid_to_hex_r(n, &update->new_oid);
>>     ++		struct strbuf buf = STRBUF_INIT;
>
> Ah, this is doing the same thing as with 'pre-push'. Cool.
>
>>       
>>     - 		strbuf_reset(&buf);
>>     +-		strbuf_reset(&buf);
>>      -		strbuf_addf(&buf, "%s %s %s\n",
>>     --			    oid_to_hex(&update->old_oid),
>>     --			    oid_to_hex(&update->new_oid),
>>     --			    update->refname);
>>     ++		strbuf_addf(&buf, "%s %s %s",
>>     + 			    oid_to_hex(&update->old_oid),
>>     + 			    oid_to_hex(&update->new_oid),
>>     + 			    update->refname);
>>      -
>>      -		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
>>      -			if (errno != EPIPE)
>>      -				ret = -1;
>>      -			break;
>>      -		}
>>     -+		strbuf_addf(&buf, "%s %s %s", o, n, update->refname);
>>     -+		string_list_append(&to_stdin, buf.buf);
>>     ++		string_list_append(&to_stdin, strbuf_detach(&buf, NULL));
>>       	}
>>       
>>      -	close(proc.in);
>>      -	sigchain_pop(SIGPIPE);
>>     +-	strbuf_release(&buf);
>>      +	opt.feed_pipe = pipe_from_string_list;
>>      +	opt.feed_pipe_ctx = &to_stdin;
>>      +
>>     -+	ret = run_hooks("reference-transaction", &opt);
>>     -+	run_hooks_opt_clear(&opt);
>>     - 	strbuf_release(&buf);
>>     ++	ret = run_hooks_oneshot("reference-transaction", &opt);
>>     ++	to_stdin.strdup_strings = 1;
>
> And turning on strdup_strings here means we will free them when we call
> string_list_clear. Ok.
>
>>      +	string_list_clear(&to_stdin, 0);
>>       
>>      -	ret |= finish_command(&proc);
>> 32:  db70b59b3bd ! 35:  ceef2f3e804 receive-pack: convert receive hooks to hook.h
>>     @@ builtin/receive-pack.c: static void hook_output_to_sideband(struct strbuf *outpu
>>      +{
>>      +	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
>>      +	struct receive_hook_feed_context ctx;
>>     -+	int rc;
>>      +	struct command *iter = commands;
>>      +
>>      +	/* if there are no valid commands, don't invoke the hook at all. */
>>     @@ builtin/receive-pack.c: static void hook_output_to_sideband(struct strbuf *outpu
>>      +	if (!iter)
>>      +		return 0;
>>      +
>>     -+	/* pre-receive hooks should run in series as the hook updates refs */
>>     -+	if (!strcmp(hook_name, "pre-receive"))
>>     -+		opt.jobs = 1;
>>     -+
>
> Hm, interesting. I'll note this as a case to watch for when I add the
> .jobs option back in.
>
>
> Thanks. I'll take a look at the newly added/copied over patches next.
>
>  - Emily


^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH v5 00/36] Run hooks via "git run hook" & hook library
  2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
                               ` (38 preceding siblings ...)
  2021-08-19 23:40             ` Emily Shaffer
@ 2021-09-02 13:11             ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 01/36] Makefile: mark "check" target as .PHONY Ævar Arnfjörð Bjarmason
                                 ` (35 more replies)
  39 siblings, 36 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

This topic refactors our hook execution to use a new hook.[ch] API and
rew "git hook run" tool in the case of the Perl & Python scripts that
execute hooks. It should contain no functional changes in how hooks
are run.

It's mainly authored by Emily Schaffer & is followed-up by her series
to make hooks configurable via normal "git config": [1]

See [2] for the v4 of this. A range-diff follows, but some of the main
changes:

 * The part where we die() if we're asked about a hook not listed in
   githooks(5) is gone. This makes things in Emily's follow-on topic
   much easier.

 * Various whitespace/small refactoring changes designed to make
   Emily's follow-on topic smaller.

 * Changed "/*" to "/**" comments in hook.h as appropriate (API
   comments).

 * I also re-arranged some functions in the header file to make those
   API comments easier to follow. I.e. we start with structs, and then
   various functions grouped by their respective functionality.

 * Various small commit message / comment rewording etc., some in
   response to Emily's feedback, others that I found myself.

[The rest here is a summary of how this topic interacts with a re-roll
of Emily's topic coming after this, feel free to skip it if only
reading this topic]:

I have my own version of Emily's series rebased on top of this with
some significant changes / fixes, which I mainly rebased/hacked up to
validate that all the changes here made sense.

That version is an extensive edit not of her v3 (which I believe maps
to her own 30ffe98601e) but the cf1f8e34a34 tip I found at the tip of
her "config-based-hooks-restart" a couple of days ago)[3], which
already incorporated some of my own feedback. That version is at my
avar-nasamuffin/config-based-hooks-restart-3[4].

Emily: You should be able to run something like this to get a sensible
range-diff of it v.s. what you have now (probably with
s/nasamuffin/origin/):

    git range-diff \
    gitster/ab/config-based-hooks-base..nasamuffin/config-based-hooks-restart \
    avar/es-avar/config-based-hooks-6..avar/avar-nasamuffin/config-based-hooks-restart-3

There's a few miscellaneous fixes in there, e.g. it passes all commits
with SANITIZE=leak now, and the "hook: allow running non-native hooks"
commit is entirely gone except for the doc change, as it wasn't needed
with the changes I've made in this v5 (we'd already allow running
non-native hooks). Likewise all the fn() and fn_gently() function
split wasn't needed anymore, as it's all "gentle" now. It has a
trivial CI failure[6] (coccinelle nitpick), but other than that passes
all tests.

1. https://lore.kernel.org/git/20210819033450.3382652-1-emilyshaffer@google.com/
2. https://lore.kernel.org/git/cover-v4-00.36-00000000000-20210803T191505Z-avarab@gmail.com/
3. https://github.com/nasamuffin/git/tree/config-based-hooks-restart
4. https://github.com/avar/git/tree/avar-nasamuffin/config-based-hooks-restart-3
5. https://github.com/avar/git/runs/3493829211
6. https://github.com/avar/git/runs/3493829211

Emily Shaffer (26):
  hook.c: add a hook_exists() wrapper and use it in bugreport.c
  hook: add 'run' subcommand
  gc: use hook library for pre-auto-gc hook
  rebase: convert pre-rebase to use hook.h
  am: convert applypatch to use hook.h
  hooks: convert 'post-checkout' hook to hook library
  merge: convert post-merge to use hook.h
  send-email: use 'git hook run' for 'sendemail-validate'
  git-p4: use 'git hook' to run hooks
  commit: convert {pre-commit,prepare-commit-msg} hook to hook.h
  read-cache: convert post-index-change to use hook.h
  receive-pack: convert push-to-checkout hook to hook.h
  run-command: remove old run_hook_{le,ve}() hook API
  run-command: allow stdin for run_processes_parallel
  hook: support passing stdin to hooks
  am: convert 'post-rewrite' hook to hook.h
  run-command: add stdin callback for parallelization
  hook: provide stdin by string_list or callback
  hook: convert 'post-rewrite' hook in sequencer.c to hook.h
  transport: convert pre-push hook to hook.h
  reference-transaction: use hook.h to run hooks
  run-command: allow capturing of collated output
  hooks: allow callers to capture output
  receive-pack: convert 'update' hook to hook.h
  post-update: use hook.h library
  receive-pack: convert receive hooks to hook.h

Ævar Arnfjörð Bjarmason (10):
  Makefile: mark "check" target as .PHONY
  Makefile: stop hardcoding {command,config}-list.h
  Makefile: remove an out-of-date comment
  hook.[ch]: move find_hook() from run-command.c to hook.c
  hook.c users: use "hook_exists()" instead of "find_hook()"
  hook-list.h: add a generated list of hooks, like config-list.h
  git hook run: add an --ignore-missing flag
  hook tests: test for exact "pre-push" hook input
  hook tests: use a modern style for "pre-push" tests
  hooks: fix a TOCTOU in "did we run a hook?" heuristic

 .gitignore                          |   2 +
 Documentation/git-hook.txt          |  51 +++++
 Documentation/githooks.txt          |   4 +
 Makefile                            |  26 ++-
 builtin.h                           |   1 +
 builtin/am.c                        |  29 +--
 builtin/bugreport.c                 |  46 +----
 builtin/checkout.c                  |  14 +-
 builtin/clone.c                     |   6 +-
 builtin/commit.c                    |  19 +-
 builtin/fetch.c                     |   1 +
 builtin/gc.c                        |   3 +-
 builtin/hook.c                      |  98 ++++++++++
 builtin/merge.c                     |  21 +-
 builtin/rebase.c                    |   6 +-
 builtin/receive-pack.c              | 285 +++++++++++++---------------
 builtin/submodule--helper.c         |   2 +-
 builtin/worktree.c                  |  29 ++-
 command-list.txt                    |   1 +
 commit.c                            |  16 +-
 commit.h                            |   3 +-
 compat/vcbuild/README               |   2 +-
 config.mak.uname                    |   6 +-
 contrib/buildsystems/CMakeLists.txt |   7 +
 generate-hooklist.sh                |  18 ++
 git-p4.py                           |  72 +------
 git-send-email.perl                 |  20 +-
 git.c                               |   1 +
 hook.c                              | 219 +++++++++++++++++++++
 hook.h                              | 127 +++++++++++++
 read-cache.c                        |  11 +-
 refs.c                              |  41 ++--
 reset.c                             |  14 +-
 run-command.c                       | 157 +++++++--------
 run-command.h                       |  55 +++---
 sequencer.c                         |  86 ++++-----
 submodule.c                         |   1 +
 t/helper/test-run-command.c         |  46 ++++-
 t/t0061-run-command.sh              |  37 ++++
 t/t1800-hook.sh                     | 151 +++++++++++++++
 t/t5571-pre-push-hook.sh            |  94 +++++----
 t/t9001-send-email.sh               |   4 +-
 transport.c                         |  57 ++----
 43 files changed, 1278 insertions(+), 611 deletions(-)
 create mode 100644 Documentation/git-hook.txt
 create mode 100644 builtin/hook.c
 create mode 100755 generate-hooklist.sh
 create mode 100644 hook.c
 create mode 100644 hook.h
 create mode 100755 t/t1800-hook.sh

Range-diff against v4:
 1:  81fe1ed90d5 =  1:  ac419613fdc Makefile: mark "check" target as .PHONY
 2:  0f749530777 !  2:  a161b7f0a5c Makefile: stop hardcoding {command,config}-list.h
    @@ Commit message
         this refactoring we'll only need to add the new file to the
         GENERATED_H variable, not EXCEPT_HDRS, the vcbuild/README etc.
     
    -    I have not tested the Windows-specific change in config.mak.uname
    -    being made here, but we use other variables from the Makefile in the
    -    same block, and the GENERATED_H is fully defined before we include
    -    config.mak.uname.
    -
         Hardcoding command-list.h there seems to have been a case of
         copy/paste programming in 976aaedca0 (msvc: add a Makefile target to
         pre-generate the Visual Studio solution, 2019-07-29). The
 3:  644b31fe281 =  3:  ffef1d3257e Makefile: remove an out-of-date comment
 4:  89c4d44b0c3 !  4:  545e16c6f04 hook.[ch]: move find_hook() to this new library
    @@ Metadata
     Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## Commit message ##
    -    hook.[ch]: move find_hook() to this new library
    +    hook.[ch]: move find_hook() from run-command.c to hook.c
     
         Move the find_hook() function from run-command.c to a new hook.c
         library. This change establishes a stub library that's pretty
    @@ hook.h (new)
     +#ifndef HOOK_H
     +#define HOOK_H
     +
    -+/*
    ++/**
     + * Returns the path to the hook file, or NULL if the hook is missing
     + * or disabled. Note that this points to static storage that will be
    -+ * overwritten by further calls to find_hook and run_hook_*.
    ++ * overwritten by further calls to find_hook().
     + */
     +const char *find_hook(const char *name);
     +
 5:  3514e0c0251 !  5:  a9bc4519e9a hook.c: add a hook_exists() wrapper and use it in bugreport.c
    @@ hook.h
       */
      const char *find_hook(const char *name);
      
    -+/*
    ++/**
     + * A boolean version of find_hook()
     + */
     +int hook_exists(const char *hookname);
 6:  d5ef40f77dc !  6:  e99ec2e6f8f hook.c users: use "hook_exists()" insted of "find_hook()"
    @@ Metadata
     Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## Commit message ##
    -    hook.c users: use "hook_exists()" insted of "find_hook()"
    +    hook.c users: use "hook_exists()" instead of "find_hook()"
     
         Use the new hook_exists() function instead of find_hook() where the
         latter was called in boolean contexts. This make subsequent changes in
 7:  4cfd72722c1 !  7:  2ffb2332c8a hook-list.h: add a generated list of hooks, like config-list.h
    @@ Commit message
         hook-list.h: add a generated list of hooks, like config-list.h
     
         Make githooks(5) the source of truth for what hooks git supports, and
    -    die hooks we don't know about in find_hook(). This ensures that the
    -    documentation and the C code's idea about existing hooks doesn't
    -    diverge.
    +    punt out early on hooks we don't know about in find_hook(). This
    +    ensures that the documentation and the C code's idea about existing
    +    hooks doesn't diverge.
     
         We still have Perl and Python code running its own hooks, but that'll
         be addressed by Emily Shaffer's upcoming "git hook run" command.
    @@ Commit message
         listing only knowing about 1/4 of the p4 hooks. It didn't know about
         the recent "reference-transaction" hook either.
     
    +    We could make the find_hook() function die() or BUG() out if the new
    +    known_hook() returned 0, but let's make it return NULL just as it does
    +    when it can't find a hook of a known type. Making it die() is overly
    +    anal, and unlikely to be what we need in catching stupid typos in the
    +    name of some new hook hardcoded in git.git's sources. By making this
    +    be tolerant of unknown hook names, changes in a later series to make
    +    "git hook run" run arbitrary user-configured hook names will be easier
    +    to implement.
    +
         I have not been able to directly test the CMake change being made
         here. Since 4c2c38e800 (ci: modification of main.yml to use cmake for
         vs-build job, 2020-06-26) some of the Windows CI has a hard dependency
    @@ generate-hooklist.sh (new)
     +	NULL,
     +};
     +EOF
    -
    - ## hook.c ##
    -@@
    - #include "cache.h"
    - #include "hook.h"
    - #include "run-command.h"
    -+#include "hook-list.h"
    -+
    -+static int known_hook(const char *name)
    -+{
    -+	const char **p;
    -+	size_t len = strlen(name);
    -+	for (p = hook_name_list; *p; p++) {
    -+		const char *hook = *p;
    -+
    -+		if (!strncmp(name, hook, len) && hook[len] == '\0')
    -+			return 1;
    -+	}
    -+
    -+	return 0;
    -+}
    - 
    - const char *find_hook(const char *name)
    - {
    - 	static struct strbuf path = STRBUF_INIT;
    - 
    -+	if (!known_hook(name))
    -+		die(_("the hook '%s' is not known to git, should be in hook-list.h via githooks(5)"),
    -+		    name);
    -+
    - 	strbuf_reset(&path);
    - 	strbuf_git_path(&path, "hooks/%s", name);
    - 	if (access(path.buf, X_OK) < 0) {
 8:  7cb4a4cb69e !  8:  72dd1010f5b hook: add 'run' subcommand
    @@ builtin/hook.c (new)
     +	/* Need to take into account core.hooksPath */
     +	git_config(git_default_config, NULL);
     +
    ++	/*
    ++	 * We are not using a plain run_hooks() because we'd like to
    ++	 * detect missing hooks. Let's find it ourselves and call
    ++	 * run_hooks() instead.
    ++	 */
     +	hook_name = argv[0];
     +	hook_path = find_hook(hook_name);
     +	if (!hook_path) {
    @@ git.c: static struct cmd_struct commands[] = {
     
      ## hook.c ##
     @@
    + #include "cache.h"
      #include "hook.h"
      #include "run-command.h"
    - #include "hook-list.h"
     +#include "config.h"
      
    - static int known_hook(const char *name)
    + const char *find_hook(const char *name)
      {
    - 	const char **p;
    - 	size_t len = strlen(name);
    -+	static int test_hooks_ok = -1;
    -+
    - 	for (p = hook_name_list; *p; p++) {
    - 		const char *hook = *p;
    - 
    -@@ hook.c: static int known_hook(const char *name)
    - 			return 1;
    - 	}
    - 
    -+	if (test_hooks_ok == -1)
    -+		test_hooks_ok = git_env_bool("GIT_TEST_FAKE_HOOKS", 0);
    -+
    -+	if (test_hooks_ok &&
    -+	    (!strcmp(name, "test-hook") ||
    -+	     !strcmp(name, "does-not-exist")))
    -+		return 1;
    -+
    - 	return 0;
    - }
    - 
     @@ hook.c: int hook_exists(const char *name)
      {
      	return !!find_hook(name);
    @@ hook.h
     @@
      #ifndef HOOK_H
      #define HOOK_H
    -+#include "strbuf.h"
     +#include "strvec.h"
    -+#include "run-command.h"
    - 
    - /*
    -  * Returns the path to the hook file, or NULL if the hook is missing
    -@@ hook.h: const char *find_hook(const char *name);
    -  */
    - int hook_exists(const char *hookname);
    - 
    ++
     +struct hook {
     +	/* The path to the hook */
     +	const char *hook_path;
    @@ hook.h: const char *find_hook(const char *name);
     +	.args = STRVEC_INIT, \
     +}
     +
    -+/*
    -+ * Callback provided to feed_pipe_fn and consume_sideband_fn.
    -+ */
     +struct hook_cb_data {
     +	/* rc reflects the cumulative failure state */
     +	int rc;
    @@ hook.h: const char *find_hook(const char *name);
     +	struct hook *run_me;
     +	struct run_hooks_opt *options;
     +};
    -+
    + 
    + /**
    +  * Returns the path to the hook file, or NULL if the hook is missing
    +@@ hook.h: const char *find_hook(const char *name);
    +  */
    + int hook_exists(const char *hookname);
    + 
    ++/**
    ++ * Clear data from an initialized "struct run_hooks-opt".
    ++ */
     +void run_hooks_opt_clear(struct run_hooks_opt *o);
     +
     +/**
    @@ hook.h: const char *find_hook(const char *name);
     
      ## t/t1800-hook.sh (new) ##
     @@
    -+#!/bin/bash
    ++#!/bin/sh
     +
     +test_description='git-hook command'
     +
    @@ t/t1800-hook.sh (new)
     +	grep "unknown option" err
     +'
     +
    -+test_expect_success 'setup GIT_TEST_FAKE_HOOKS=true to permit "test-hook" and "does-not-exist" names"' '
    -+	GIT_TEST_FAKE_HOOKS=true &&
    -+	export GIT_TEST_FAKE_HOOKS
    -+'
    -+
     +test_expect_success 'git hook run: nonexistent hook' '
     +	cat >stderr.expect <<-\EOF &&
     +	error: cannot find a hook named test-hook
 9:  2b8500aa675 !  9:  821cc9bf11e gc: use hook library for pre-auto-gc hook
    @@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
     +	}
     +
     +	ret = run_hooks(hook_name, hook_path, options);
    ++
     +cleanup:
     +	run_hooks_opt_clear(options);
    ++
     +	return ret;
     +}
     
    @@ hook.h: void run_hooks_opt_clear(struct run_hooks_opt *o);
     + * with run_hooks().
     + *
     + * If "options" is provided calls run_hooks_opt_clear() on it for
    -+ * you. If "options" is NULL a scratch one will be provided for you
    -+ * before calling run_hooks().
    ++ * you. If "options" is NULL the default options from
    ++ * RUN_HOOKS_OPT_INIT will be used.
     + */
     +int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
     +
10:  3ee55d2c10f = 10:  d71c90254ea rebase: convert pre-rebase to use hook.h
11:  050f20d14f0 = 11:  ea3af2ccc4d am: convert applypatch to use hook.h
12:  ac875d284da ! 12:  fed0b52f88f hooks: convert 'post-checkout' hook to hook library
    @@ hook.h: struct run_hooks_opt
      	/* Args to be passed to each hook */
      	struct strvec args;
     +
    -+	/* Resolve and run the "absolute_path(hook)" instead of
    ++	/*
    ++	 * Resolve and run the "absolute_path(hook)" instead of
     +	 * "hook". Used for "git worktree" hooks
     +	 */
     +	int absolute_path;
13:  69763bc2255 = 13:  53d8721a0e3 merge: convert post-merge to use hook.h
14:  2ca1ca1b8e4 ! 14:  d60827a2856 git hook run: add an --ignore-missing flag
    @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
      	};
      	int ret;
     @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
    - 	git_config(git_default_config, NULL);
    - 
    + 	/*
    + 	 * We are not using a plain run_hooks() because we'd like to
    + 	 * detect missing hooks. Let's find it ourselves and call
    +-	 * run_hooks() instead.
    ++	 * run_hooks() instead...
    + 	 */
      	hook_name = argv[0];
     +	if (ignore_missing)
    ++		/* ... act like a plain run_hooks() under --ignore-missing */
     +		return run_hooks_oneshot(hook_name, &opt);
      	hook_path = find_hook(hook_name);
      	if (!hook_path) {
15:  5b66b04bec7 = 15:  d4976a0821f send-email: use 'git hook run' for 'sendemail-validate'
16:  14a37a43db2 = 16:  99f3dcd1945 git-p4: use 'git hook' to run hooks
17:  ad5d0e0e7de = 17:  509761454e6 commit: convert {pre-commit,prepare-commit-msg} hook to hook.h
18:  3d3a33e2674 = 18:  e2c94d95427 read-cache: convert post-index-change to use hook.h
19:  893f8666301 ! 19:  fa7d0d24ea2 receive-pack: convert push-to-checkout hook to hook.h
    @@ Commit message
         Move the push-to-checkout hook away from run-command.h to and over to
         the new hook.h library.
     
    +    This removes the last direct user of run_hook_le(), so we could remove
    +    that function now, but let's leave that to a follow-up cleanup commit.
    +
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
20:  070433deba5 = 20:  428bb5a6792 run-command: remove old run_hook_{le,ve}() hook API
21:  1028e0c1667 = 21:  994f6ad8602 run-command: allow stdin for run_processes_parallel
22:  639e59e9ed0 = 22:  3ccc654a664 hook: support passing stdin to hooks
23:  7d1925cca48 = 23:  f548e3d15e7 am: convert 'post-rewrite' hook to hook.h
24:  0c24221b522 = 24:  bb119fa7cc0 run-command: add stdin callback for parallelization
25:  05d1085f7eb ! 25:  2439f7752b8 hook: provide stdin by string_list or callback
    @@ hook.c: int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *optio
      		ret = 0;
     
      ## hook.h ##
    -@@ hook.h: int hook_exists(const char *hookname);
    +@@
    + #ifndef HOOK_H
    + #define HOOK_H
    ++#include "strbuf.h"
    + #include "strvec.h"
    ++#include "run-command.h"
    + 
      struct hook {
      	/* The path to the hook */
      	const char *hook_path;
    @@ hook.h: struct run_hooks_opt
      };
      
      #define RUN_HOOKS_OPT_INIT { \
    -@@ hook.h: struct run_hooks_opt
    - 	.args = STRVEC_INIT, \
    - }
    +@@ hook.h: int run_hooks(const char *hookname, const char *hook_path,
    +  */
    + int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
      
    -+/*
    ++/**
     + * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
    -+ * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'.
    ++ * string_list and set 'run_hooks_opt.feed_pipe' to pipe_from_string_list().
     + * This will pipe each string in the list to stdin, separated by newlines.  (Do
     + * not inject your own newlines.)
     + */
     +int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb);
     +
    - /*
    -  * Callback provided to feed_pipe_fn and consume_sideband_fn.
    -  */
    + #endif
26:  4b7175af2e5 = 26:  48a380b3a91 hook: convert 'post-rewrite' hook in sequencer.c to hook.h
27:  3f24e056410 = 27:  af6b9292aaa transport: convert pre-push hook to hook.h
28:  ecf75f33233 ! 28:  957691f0b6d hook tests: test for exact "pre-push" hook input
    @@ Commit message
     
         Extend the tests added in ec55559f937 (push: Add support for pre-push
         hooks, 2013-01-13) to exhaustively test for the exact input we're
    -    expecting. This helps a parallel series that's refactoring how the
    -    hook is called, to e.g. make sure that we don't miss a trailing
    -    newline.
    +    expecting. This ensures that we e.g. don't miss a trailing newline.
    +
    +    Appending to a file called "actual" is the established convention in
    +    this test for hooks, see the rest of the tests added in
    +    ec55559f937 (push: Add support for pre-push hooks, 2013-01-13). Let's
    +    follow that convention here.
     
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
29:  2c961be94b4 ! 29:  88fe2621549 hook tests: use a modern style for "pre-push" tests
    @@ t/t5571-pre-push-hook.sh: echo "$2" >>actual
     -EOF
     -
      test_expect_success 'push with hook' '
    -+	cat >expected <<-EOF &&
    ++	cat >expect <<-EOF &&
     +	parent1
     +	repo1
     +	refs/heads/main $COMMIT2 refs/heads/foreign $COMMIT1
    @@ t/t5571-pre-push-hook.sh: echo "$2" >>actual
     +
      	git push parent1 main:foreign &&
     -	diff expected actual
    -+	test_cmp expected actual
    ++	test_cmp expect actual
      '
      
      test_expect_success 'add a branch' '
30:  1ce456f9d9d = 30:  1d905e81779 reference-transaction: use hook.h to run hooks
31:  6e5f1f5bd3a = 31:  fac56a9d8af run-command: allow capturing of collated output
32:  0b6e9c6d07a = 32:  7d185cdf9d1 hooks: allow callers to capture output
33:  dcf63634338 = 33:  c8150e1239f receive-pack: convert 'update' hook to hook.h
34:  f352a485e59 = 34:  a20ad847c14 post-update: use hook.h library
35:  ceef2f3e804 = 35:  79c380be6ed receive-pack: convert receive hooks to hook.h
36:  b71d7628b40 ! 36:  fe056098534 hooks: fix a TOCTOU in "did we run a hook?" heuristic
    @@ hook.h: struct hook_cb_data {
     +	int *invoked_hook;
      };
      
    - void run_hooks_opt_clear(struct run_hooks_opt *o);
    + /**
     
      ## sequencer.c ##
     @@ sequencer.c: static int run_prepare_commit_msg_hook(struct repository *r,
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH v5 01/36] Makefile: mark "check" target as .PHONY
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 02/36] Makefile: stop hardcoding {command,config}-list.h Ævar Arnfjörð Bjarmason
                                 ` (34 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Fix a bug in 44c9e8594e (Fix up header file dependencies and add
sparse checking rules, 2005-07-03), we never marked the phony "check"
target as such.

Perhaps we should just remove it, since as of a combination of
912f9980d2 (Makefile: help people who run 'make check' by mistake,
2008-11-11) 0bcd9ae85d (sparse: Fix errors due to missing
target-specific variables, 2011-04-21) we've been suggesting the user
run "make sparse" directly.

But under that mode it still does something, as well as directing the
user to run "make test" under non-sparse. So let's punt that and
narrowly fix the PHONY bug.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Makefile b/Makefile
index d1feab008fc..5ec23e6b59a 100644
--- a/Makefile
+++ b/Makefile
@@ -2941,6 +2941,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
+.PHONY: check
 check: config-list.h command-list.h
 	@if sparse; \
 	then \
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 02/36] Makefile: stop hardcoding {command,config}-list.h
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 01/36] Makefile: mark "check" target as .PHONY Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 03/36] Makefile: remove an out-of-date comment Ævar Arnfjörð Bjarmason
                                 ` (33 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Change various places that hardcode the names of these two files to
refer to either $(GENERATED_H), or to a new generated-hdrs
target. That target is consistent with the *-objs targets I recently
added in 029bac01a8 (Makefile: add {program,xdiff,test,git,fuzz}-objs
& objects targets, 2021-02-23).

A subsequent commit will add a new generated hook-list.h. By doing
this refactoring we'll only need to add the new file to the
GENERATED_H variable, not EXCEPT_HDRS, the vcbuild/README etc.

Hardcoding command-list.h there seems to have been a case of
copy/paste programming in 976aaedca0 (msvc: add a Makefile target to
pre-generate the Visual Studio solution, 2019-07-29). The
config-list.h was added later in 709df95b78 (help: move
list_config_help to builtin/help, 2020-04-16).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile              | 6 ++++--
 compat/vcbuild/README | 2 +-
 config.mak.uname      | 6 +++---
 3 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/Makefile b/Makefile
index 5ec23e6b59a..4f1f13b5f0a 100644
--- a/Makefile
+++ b/Makefile
@@ -824,6 +824,8 @@ XDIFF_LIB = xdiff/lib.a
 
 GENERATED_H += command-list.h
 GENERATED_H += config-list.h
+.PHONY: generated-hdrs
+generated-hdrs: $(GENERATED_H)
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2919,7 +2921,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
+EXCEPT_HDRS := $(GENERATED_H) unicode-width.h compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
 endif
@@ -2942,7 +2944,7 @@ style:
 	git clang-format --style file --diff --extensions c,h
 
 .PHONY: check
-check: config-list.h command-list.h
+check: $(GENERATED_H)
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
diff --git a/compat/vcbuild/README b/compat/vcbuild/README
index 51fb083dbbe..29ec1d0f104 100644
--- a/compat/vcbuild/README
+++ b/compat/vcbuild/README
@@ -92,7 +92,7 @@ The Steps of Build Git with VS2008
    the git operations.
 
 3. Inside Git's directory run the command:
-       make command-list.h config-list.h
+       make generated-hdrs
    to generate the header file needed to compile git.
 
 4. Then either build Git with the GNU Make Makefile in the Git projects
diff --git a/config.mak.uname b/config.mak.uname
index 76516aaa9a5..8aac06eb094 100644
--- a/config.mak.uname
+++ b/config.mak.uname
@@ -735,9 +735,9 @@ vcxproj:
 	 echo '</Project>') >git-remote-http/LinkOrCopyRemoteHttp.targets
 	git add -f git/LinkOrCopyBuiltins.targets git-remote-http/LinkOrCopyRemoteHttp.targets
 
-	# Add command-list.h and config-list.h
-	$(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 config-list.h command-list.h
-	git add -f config-list.h command-list.h
+	# Add generated headers
+	$(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 $(GENERATED_H)
+	git add -f $(GENERATED_H)
 
 	# Add scripts
 	rm -f perl/perl.mak
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 03/36] Makefile: remove an out-of-date comment
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 01/36] Makefile: mark "check" target as .PHONY Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 02/36] Makefile: stop hardcoding {command,config}-list.h Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 04/36] hook.[ch]: move find_hook() from run-command.c to hook.c Ævar Arnfjörð Bjarmason
                                 ` (32 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

This comment added in dfea575017 (Makefile: lazily compute header
dependencies, 2010-01-26) has been out of date since
92b88eba9f (Makefile: use `git ls-files` to list header files, if
possible, 2019-03-04), when we did exactly what it tells us not to do
and added $(GENERATED_H) to $(OBJECTS) dependencies.

The rest of it was also somewhere between inaccurate and outdated,
since as of b8ba629264 (Makefile: fold MISC_H into LIB_H, 2012-06-20)
it's not followed by a list of header files, that got moved earlier in
the file into LIB_H in 60d24dd255 (Makefile: fold XDIFF_H and VCSSVN_H
into LIB_H, 2012-07-06).

Let's just remove it entirely, to the extent that we have anything
useful to say here the comment on the
"USE_COMPUTED_HEADER_DEPENDENCIES" variable a few lines above this
change does the job for us.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/Makefile b/Makefile
index 4f1f13b5f0a..53c67f4a726 100644
--- a/Makefile
+++ b/Makefile
@@ -2524,13 +2524,6 @@ ifneq ($(dep_files_present),)
 include $(dep_files_present)
 endif
 else
-# Dependencies on header files, for platforms that do not support
-# the gcc -MMD option.
-#
-# Dependencies on automatically generated headers such as command-list.h
-# should _not_ be included here, since they are necessary even when
-# building an object for the first time.
-
 $(OBJECTS): $(LIB_H) $(GENERATED_H)
 endif
 
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 04/36] hook.[ch]: move find_hook() from run-command.c to hook.c
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (2 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 03/36] Makefile: remove an out-of-date comment Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 05/36] hook.c: add a hook_exists() wrapper and use it in bugreport.c Ævar Arnfjörð Bjarmason
                                 ` (31 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Move the find_hook() function from run-command.c to a new hook.c
library. This change establishes a stub library that's pretty
pointless right now, but will see much wider use with Emily Shaffer's
upcoming "configuration-based hooks" series.

Eventually all the hook related code will live in hook.[ch]. Let's
start that process by moving the simple find_hook() function over
as-is.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile               |  1 +
 builtin/am.c           |  1 +
 builtin/bugreport.c    |  2 +-
 builtin/commit.c       |  1 +
 builtin/merge.c        |  1 +
 builtin/receive-pack.c |  1 +
 builtin/worktree.c     |  1 +
 hook.c                 | 37 +++++++++++++++++++++++++++++++++++++
 hook.h                 | 11 +++++++++++
 refs.c                 |  1 +
 run-command.c          | 35 +----------------------------------
 run-command.h          |  7 -------
 sequencer.c            |  1 +
 transport.c            |  1 +
 14 files changed, 59 insertions(+), 42 deletions(-)
 create mode 100644 hook.c
 create mode 100644 hook.h

diff --git a/Makefile b/Makefile
index 53c67f4a726..ae256672780 100644
--- a/Makefile
+++ b/Makefile
@@ -911,6 +911,7 @@ LIB_OBJS += hash-lookup.o
 LIB_OBJS += hashmap.o
 LIB_OBJS += help.o
 LIB_OBJS += hex.o
+LIB_OBJS += hook.o
 LIB_OBJS += ident.o
 LIB_OBJS += json-writer.o
 LIB_OBJS += kwset.o
diff --git a/builtin/am.c b/builtin/am.c
index 0c2ad96b70e..c603f3cebdf 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -11,6 +11,7 @@
 #include "parse-options.h"
 #include "dir.h"
 #include "run-command.h"
+#include "hook.h"
 #include "quote.h"
 #include "tempfile.h"
 #include "lockfile.h"
diff --git a/builtin/bugreport.c b/builtin/bugreport.c
index 9915a5841de..596f079a7f9 100644
--- a/builtin/bugreport.c
+++ b/builtin/bugreport.c
@@ -3,7 +3,7 @@
 #include "strbuf.h"
 #include "help.h"
 #include "compat/compiler.h"
-#include "run-command.h"
+#include "hook.h"
 
 
 static void get_system_info(struct strbuf *sys_info)
diff --git a/builtin/commit.c b/builtin/commit.c
index 243c626307c..26c8fad6366 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -19,6 +19,7 @@
 #include "revision.h"
 #include "wt-status.h"
 #include "run-command.h"
+#include "hook.h"
 #include "refs.h"
 #include "log-tree.h"
 #include "strbuf.h"
diff --git a/builtin/merge.c b/builtin/merge.c
index febb0c99c93..2fc78f54e43 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -13,6 +13,7 @@
 #include "builtin.h"
 #include "lockfile.h"
 #include "run-command.h"
+#include "hook.h"
 #include "diff.h"
 #include "diff-merges.h"
 #include "refs.h"
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 2d1f97e1ca7..97aebdc15bd 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -7,6 +7,7 @@
 #include "pkt-line.h"
 #include "sideband.h"
 #include "run-command.h"
+#include "hook.h"
 #include "exec-cmd.h"
 #include "commit.h"
 #include "object.h"
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 0d0a80da61f..d22ece93e1a 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -8,6 +8,7 @@
 #include "branch.h"
 #include "refs.h"
 #include "run-command.h"
+#include "hook.h"
 #include "sigchain.h"
 #include "submodule.h"
 #include "utf8.h"
diff --git a/hook.c b/hook.c
new file mode 100644
index 00000000000..c4dbef1d0ef
--- /dev/null
+++ b/hook.c
@@ -0,0 +1,37 @@
+#include "cache.h"
+#include "hook.h"
+#include "run-command.h"
+
+const char *find_hook(const char *name)
+{
+	static struct strbuf path = STRBUF_INIT;
+
+	strbuf_reset(&path);
+	strbuf_git_path(&path, "hooks/%s", name);
+	if (access(path.buf, X_OK) < 0) {
+		int err = errno;
+
+#ifdef STRIP_EXTENSION
+		strbuf_addstr(&path, STRIP_EXTENSION);
+		if (access(path.buf, X_OK) >= 0)
+			return path.buf;
+		if (errno == EACCES)
+			err = errno;
+#endif
+
+		if (err == EACCES && advice_ignored_hook) {
+			static struct string_list advise_given = STRING_LIST_INIT_DUP;
+
+			if (!string_list_lookup(&advise_given, name)) {
+				string_list_insert(&advise_given, name);
+				advise(_("The '%s' hook was ignored because "
+					 "it's not set as executable.\n"
+					 "You can disable this warning with "
+					 "`git config advice.ignoredHook false`."),
+				       path.buf);
+			}
+		}
+		return NULL;
+	}
+	return path.buf;
+}
diff --git a/hook.h b/hook.h
new file mode 100644
index 00000000000..56a7b3c9164
--- /dev/null
+++ b/hook.h
@@ -0,0 +1,11 @@
+#ifndef HOOK_H
+#define HOOK_H
+
+/**
+ * Returns the path to the hook file, or NULL if the hook is missing
+ * or disabled. Note that this points to static storage that will be
+ * overwritten by further calls to find_hook().
+ */
+const char *find_hook(const char *name);
+
+#endif
diff --git a/refs.c b/refs.c
index 8b9f7c3a80a..6211692eaae 100644
--- a/refs.c
+++ b/refs.c
@@ -10,6 +10,7 @@
 #include "refs.h"
 #include "refs/refs-internal.h"
 #include "run-command.h"
+#include "hook.h"
 #include "object-store.h"
 #include "object.h"
 #include "tag.h"
diff --git a/run-command.c b/run-command.c
index f72e72cce73..352f5be1646 100644
--- a/run-command.c
+++ b/run-command.c
@@ -8,6 +8,7 @@
 #include "string-list.h"
 #include "quote.h"
 #include "config.h"
+#include "hook.h"
 
 void child_process_init(struct child_process *child)
 {
@@ -1319,40 +1320,6 @@ int async_with_fork(void)
 #endif
 }
 
-const char *find_hook(const char *name)
-{
-	static struct strbuf path = STRBUF_INIT;
-
-	strbuf_reset(&path);
-	strbuf_git_path(&path, "hooks/%s", name);
-	if (access(path.buf, X_OK) < 0) {
-		int err = errno;
-
-#ifdef STRIP_EXTENSION
-		strbuf_addstr(&path, STRIP_EXTENSION);
-		if (access(path.buf, X_OK) >= 0)
-			return path.buf;
-		if (errno == EACCES)
-			err = errno;
-#endif
-
-		if (err == EACCES && advice_ignored_hook) {
-			static struct string_list advise_given = STRING_LIST_INIT_DUP;
-
-			if (!string_list_lookup(&advise_given, name)) {
-				string_list_insert(&advise_given, name);
-				advise(_("The '%s' hook was ignored because "
-					 "it's not set as executable.\n"
-					 "You can disable this warning with "
-					 "`git config advice.ignoredHook false`."),
-				       path.buf);
-			}
-		}
-		return NULL;
-	}
-	return path.buf;
-}
-
 int run_hook_ve(const char *const *env, const char *name, va_list args)
 {
 	struct child_process hook = CHILD_PROCESS_INIT;
diff --git a/run-command.h b/run-command.h
index af1296769f9..f76b740f927 100644
--- a/run-command.h
+++ b/run-command.h
@@ -204,13 +204,6 @@ int finish_command_in_signal(struct child_process *);
  */
 int run_command(struct child_process *);
 
-/*
- * Returns the path to the hook file, or NULL if the hook is missing
- * or disabled. Note that this points to static storage that will be
- * overwritten by further calls to find_hook and run_hook_*.
- */
-const char *find_hook(const char *name);
-
 /**
  * Run a hook.
  * The first argument is a pathname to an index file, or NULL
diff --git a/sequencer.c b/sequencer.c
index 1ceb4c0d7fd..3214f02f393 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -8,6 +8,7 @@
 #include "sequencer.h"
 #include "tag.h"
 #include "run-command.h"
+#include "hook.h"
 #include "exec-cmd.h"
 #include "utf8.h"
 #include "cache-tree.h"
diff --git a/transport.c b/transport.c
index 17e9629710a..77e196f75f5 100644
--- a/transport.c
+++ b/transport.c
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "transport.h"
 #include "run-command.h"
+#include "hook.h"
 #include "pkt-line.h"
 #include "fetch-pack.h"
 #include "remote.h"
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 05/36] hook.c: add a hook_exists() wrapper and use it in bugreport.c
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (3 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 04/36] hook.[ch]: move find_hook() from run-command.c to hook.c Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 06/36] hook.c users: use "hook_exists()" instead of "find_hook()" Ævar Arnfjörð Bjarmason
                                 ` (30 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Add a boolean version of the find_hook() function for those callers
who are only interested in checking whether the hook exists, not what
the path to it is.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/bugreport.c | 2 +-
 hook.c              | 5 +++++
 hook.h              | 5 +++++
 3 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/builtin/bugreport.c b/builtin/bugreport.c
index 596f079a7f9..941c8d5e270 100644
--- a/builtin/bugreport.c
+++ b/builtin/bugreport.c
@@ -82,7 +82,7 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 
 	for (i = 0; i < ARRAY_SIZE(hook); i++)
-		if (find_hook(hook[i]))
+		if (hook_exists(hook[i]))
 			strbuf_addf(hook_info, "%s\n", hook[i]);
 }
 
diff --git a/hook.c b/hook.c
index c4dbef1d0ef..97cd799a320 100644
--- a/hook.c
+++ b/hook.c
@@ -35,3 +35,8 @@ const char *find_hook(const char *name)
 	}
 	return path.buf;
 }
+
+int hook_exists(const char *name)
+{
+	return !!find_hook(name);
+}
diff --git a/hook.h b/hook.h
index 56a7b3c9164..368754b1201 100644
--- a/hook.h
+++ b/hook.h
@@ -8,4 +8,9 @@
  */
 const char *find_hook(const char *name);
 
+/**
+ * A boolean version of find_hook()
+ */
+int hook_exists(const char *hookname);
+
 #endif
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 06/36] hook.c users: use "hook_exists()" instead of "find_hook()"
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (4 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 05/36] hook.c: add a hook_exists() wrapper and use it in bugreport.c Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 07/36] hook-list.h: add a generated list of hooks, like config-list.h Ævar Arnfjörð Bjarmason
                                 ` (29 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Use the new hook_exists() function instead of find_hook() where the
latter was called in boolean contexts. This make subsequent changes in
a series where we further refactor the hook API clearer, as we won't
conflate wanting to get the path of the hook with checking for its
existence.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/commit.c       | 2 +-
 builtin/merge.c        | 2 +-
 builtin/receive-pack.c | 2 +-
 sequencer.c            | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index 26c8fad6366..f6ca9d04c78 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1052,7 +1052,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (!no_verify && find_hook("pre-commit")) {
+	if (!no_verify && hook_exists("pre-commit")) {
 		/*
 		 * Re-read the index as pre-commit hook could have updated it,
 		 * and write it out as a tree.  We must do this before we invoke
diff --git a/builtin/merge.c b/builtin/merge.c
index 2fc78f54e43..d75a438298b 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -850,7 +850,7 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 	 * and write it out as a tree.  We must do this before we invoke
 	 * the editor and after we invoke run_status above.
 	 */
-	if (find_hook("pre-merge-commit"))
+	if (hook_exists("pre-merge-commit"))
 		discard_cache();
 	read_cache_from(index_file);
 	strbuf_addbuf(&msg, &merge_msg);
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 97aebdc15bd..91fa799b66e 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1464,7 +1464,7 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 
 	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
 
-	if (!find_hook(push_to_checkout_hook))
+	if (!hook_exists(push_to_checkout_hook))
 		retval = push_to_deploy(sha1, &env, work_tree);
 	else
 		retval = push_to_checkout(sha1, &env, work_tree);
diff --git a/sequencer.c b/sequencer.c
index 3214f02f393..8999edf8d7b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1446,7 +1446,7 @@ static int try_to_commit(struct repository *r,
 		}
 	}
 
-	if (find_hook("prepare-commit-msg")) {
+	if (hook_exists("prepare-commit-msg")) {
 		res = run_prepare_commit_msg_hook(r, msg, hook_commit);
 		if (res)
 			goto out;
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 07/36] hook-list.h: add a generated list of hooks, like config-list.h
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (5 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 06/36] hook.c users: use "hook_exists()" instead of "find_hook()" Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 08/36] hook: add 'run' subcommand Ævar Arnfjörð Bjarmason
                                 ` (28 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason, René Scharfe

Make githooks(5) the source of truth for what hooks git supports, and
punt out early on hooks we don't know about in find_hook(). This
ensures that the documentation and the C code's idea about existing
hooks doesn't diverge.

We still have Perl and Python code running its own hooks, but that'll
be addressed by Emily Shaffer's upcoming "git hook run" command.

This resolves a long-standing TODO item in bugreport.c of there being
no centralized listing of hooks, and fixes a bug with the bugreport
listing only knowing about 1/4 of the p4 hooks. It didn't know about
the recent "reference-transaction" hook either.

We could make the find_hook() function die() or BUG() out if the new
known_hook() returned 0, but let's make it return NULL just as it does
when it can't find a hook of a known type. Making it die() is overly
anal, and unlikely to be what we need in catching stupid typos in the
name of some new hook hardcoded in git.git's sources. By making this
be tolerant of unknown hook names, changes in a later series to make
"git hook run" run arbitrary user-configured hook names will be easier
to implement.

I have not been able to directly test the CMake change being made
here. Since 4c2c38e800 (ci: modification of main.yml to use cmake for
vs-build job, 2020-06-26) some of the Windows CI has a hard dependency
on CMake, this change works there, and is to my eyes an obviously
correct use of a pattern established in previous CMake changes,
namely:

 - 061c2240b1 (Introduce CMake support for configuring Git,
    2020-06-12)
 - 709df95b78 (help: move list_config_help to builtin/help,
    2020-04-16)
 - 976aaedca0 (msvc: add a Makefile target to pre-generate the Visual
   Studio solution, 2019-07-29)

The LC_ALL=C is needed because at least in my locale the dash ("-") is
ignored for the purposes of sorting, which results in a different
order. I'm not aware of anything in git that has a hard dependency on
the order, but e.g. the bugreport output would end up using whatever
locale was in effect when git was compiled.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Helped-by: René Scharfe <l.s.r@web.de>
---
 .gitignore                          |  1 +
 Makefile                            | 10 ++++++-
 builtin/bugreport.c                 | 44 ++++++-----------------------
 contrib/buildsystems/CMakeLists.txt |  7 +++++
 generate-hooklist.sh                | 18 ++++++++++++
 5 files changed, 43 insertions(+), 37 deletions(-)
 create mode 100755 generate-hooklist.sh

diff --git a/.gitignore b/.gitignore
index 311841f9bed..6be9de41ae8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -190,6 +190,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/hook-list.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Makefile b/Makefile
index ae256672780..ed06b4e9634 100644
--- a/Makefile
+++ b/Makefile
@@ -824,6 +824,8 @@ XDIFF_LIB = xdiff/lib.a
 
 GENERATED_H += command-list.h
 GENERATED_H += config-list.h
+GENERATED_H += hook-list.h
+
 .PHONY: generated-hdrs
 generated-hdrs: $(GENERATED_H)
 
@@ -2231,7 +2233,9 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
+hook.sp hook.s hook.o: hook-list.h
+
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h hook-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2264,6 +2268,10 @@ command-list.h: $(wildcard Documentation/git*.txt)
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+hook-list.h: generate-hooklist.sh Documentation/githooks.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-hooklist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
diff --git a/builtin/bugreport.c b/builtin/bugreport.c
index 941c8d5e270..a7a1fcb8a7a 100644
--- a/builtin/bugreport.c
+++ b/builtin/bugreport.c
@@ -4,6 +4,7 @@
 #include "help.h"
 #include "compat/compiler.h"
 #include "hook.h"
+#include "hook-list.h"
 
 
 static void get_system_info(struct strbuf *sys_info)
@@ -41,39 +42,7 @@ static void get_system_info(struct strbuf *sys_info)
 
 static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 {
-	/*
-	 * NEEDSWORK: Doesn't look like there is a list of all possible hooks;
-	 * so below is a transcription of `git help hooks`. Later, this should
-	 * be replaced with some programmatically generated list (generated from
-	 * doc or else taken from some library which tells us about all the
-	 * hooks)
-	 */
-	static const char *hook[] = {
-		"applypatch-msg",
-		"pre-applypatch",
-		"post-applypatch",
-		"pre-commit",
-		"pre-merge-commit",
-		"prepare-commit-msg",
-		"commit-msg",
-		"post-commit",
-		"pre-rebase",
-		"post-checkout",
-		"post-merge",
-		"pre-push",
-		"pre-receive",
-		"update",
-		"post-receive",
-		"post-update",
-		"push-to-checkout",
-		"pre-auto-gc",
-		"post-rewrite",
-		"sendemail-validate",
-		"fsmonitor-watchman",
-		"p4-pre-submit",
-		"post-index-change",
-	};
-	int i;
+	const char **p;
 
 	if (nongit) {
 		strbuf_addstr(hook_info,
@@ -81,9 +50,12 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 		return;
 	}
 
-	for (i = 0; i < ARRAY_SIZE(hook); i++)
-		if (hook_exists(hook[i]))
-			strbuf_addf(hook_info, "%s\n", hook[i]);
+	for (p = hook_name_list; *p; p++) {
+		const char *hook = *p;
+
+		if (hook_exists(hook))
+			strbuf_addf(hook_info, "%s\n", hook);
+	}
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt
index 171b4124afe..fd1399c440f 100644
--- a/contrib/buildsystems/CMakeLists.txt
+++ b/contrib/buildsystems/CMakeLists.txt
@@ -624,6 +624,13 @@ if(NOT EXISTS ${CMAKE_BINARY_DIR}/config-list.h)
 			OUTPUT_FILE ${CMAKE_BINARY_DIR}/config-list.h)
 endif()
 
+if(NOT EXISTS ${CMAKE_BINARY_DIR}/hook-list.h)
+	message("Generating hook-list.h")
+	execute_process(COMMAND ${SH_EXE} ${CMAKE_SOURCE_DIR}/generate-hooklist.sh
+			WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
+			OUTPUT_FILE ${CMAKE_BINARY_DIR}/hook-list.h)
+endif()
+
 include_directories(${CMAKE_BINARY_DIR})
 
 #build
diff --git a/generate-hooklist.sh b/generate-hooklist.sh
new file mode 100755
index 00000000000..6d4e56d1a31
--- /dev/null
+++ b/generate-hooklist.sh
@@ -0,0 +1,18 @@
+#!/bin/sh
+#
+# Usage: ./generate-hooklist.sh >hook-list.h
+
+cat <<EOF
+/* Automatically generated by generate-hooklist.sh */
+
+static const char *hook_name_list[] = {
+EOF
+
+sed -n -e '/^~~~~*$/ {x; s/^.*$/	"&",/; p;}; x' \
+	<Documentation/githooks.txt |
+	LC_ALL=C sort
+
+cat <<EOF
+	NULL,
+};
+EOF
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 08/36] hook: add 'run' subcommand
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (6 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 07/36] hook-list.h: add a generated list of hooks, like config-list.h Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 09/36] gc: use hook library for pre-auto-gc hook Ævar Arnfjörð Bjarmason
                                 ` (27 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

In order to enable hooks to be run as an external process, by a
standalone Git command, or by tools which wrap Git, provide an external
means to run all configured hook commands for a given hook event.

Most of our hooks require more complex functionality than this, but
let's start with the bare minimum required to support our simplest
hooks.

In terms of implementation the usage_with_options() and "goto usage"
pattern here mirrors that of
builtin/{commit-graph,multi-pack-index}.c.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .gitignore                 |   1 +
 Documentation/git-hook.txt |  38 +++++++++++
 Documentation/githooks.txt |   4 ++
 Makefile                   |   1 +
 builtin.h                  |   1 +
 builtin/hook.c             |  90 ++++++++++++++++++++++++++
 command-list.txt           |   1 +
 git.c                      |   1 +
 hook.c                     | 101 +++++++++++++++++++++++++++++
 hook.h                     |  39 +++++++++++
 t/t1800-hook.sh            | 128 +++++++++++++++++++++++++++++++++++++
 11 files changed, 405 insertions(+)
 create mode 100644 Documentation/git-hook.txt
 create mode 100644 builtin/hook.c
 create mode 100755 t/t1800-hook.sh

diff --git a/.gitignore b/.gitignore
index 6be9de41ae8..66189ca3cdc 100644
--- a/.gitignore
+++ b/.gitignore
@@ -77,6 +77,7 @@
 /git-grep
 /git-hash-object
 /git-help
+/git-hook
 /git-http-backend
 /git-http-fetch
 /git-http-push
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
new file mode 100644
index 00000000000..660d6a992a0
--- /dev/null
+++ b/Documentation/git-hook.txt
@@ -0,0 +1,38 @@
+git-hook(1)
+===========
+
+NAME
+----
+git-hook - run git hooks
+
+SYNOPSIS
+--------
+[verse]
+'git hook' run <hook-name> [-- <hook-args>]
+
+DESCRIPTION
+-----------
+
+This command is an interface to git hooks (see linkgit:githooks[5]).
+Currently it only provides a convenience wrapper for running hooks for
+use by git itself. In the future it might gain other functionality.
+
+SUBCOMMANDS
+-----------
+
+run::
+	Run the `<hook-name>` hook. See linkgit:githooks[5] for
+	the hook names we support.
++
+Any positional arguments to the hook should be passed after an
+optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
+arguments (if any) differ by hook name, see linkgit:githooks[5] for
+what those are.
+
+SEE ALSO
+--------
+linkgit:githooks[5]
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index b51959ff941..a16e62bc8c8 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -698,6 +698,10 @@ and "0" meaning they were not.
 Only one parameter should be set to "1" when the hook runs.  The hook
 running passing "1", "1" should not be possible.
 
+SEE ALSO
+--------
+linkgit:git-hook[1]
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index ed06b4e9634..bbd416511a3 100644
--- a/Makefile
+++ b/Makefile
@@ -1115,6 +1115,7 @@ BUILTIN_OBJS += builtin/get-tar-commit-id.o
 BUILTIN_OBJS += builtin/grep.o
 BUILTIN_OBJS += builtin/hash-object.o
 BUILTIN_OBJS += builtin/help.o
+BUILTIN_OBJS += builtin/hook.o
 BUILTIN_OBJS += builtin/index-pack.o
 BUILTIN_OBJS += builtin/init-db.o
 BUILTIN_OBJS += builtin/interpret-trailers.o
diff --git a/builtin.h b/builtin.h
index 16ecd5586f0..91740c15149 100644
--- a/builtin.h
+++ b/builtin.h
@@ -164,6 +164,7 @@ int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix);
 int cmd_grep(int argc, const char **argv, const char *prefix);
 int cmd_hash_object(int argc, const char **argv, const char *prefix);
 int cmd_help(int argc, const char **argv, const char *prefix);
+int cmd_hook(int argc, const char **argv, const char *prefix);
 int cmd_index_pack(int argc, const char **argv, const char *prefix);
 int cmd_init_db(int argc, const char **argv, const char *prefix);
 int cmd_interpret_trailers(int argc, const char **argv, const char *prefix);
diff --git a/builtin/hook.c b/builtin/hook.c
new file mode 100644
index 00000000000..012a2973b38
--- /dev/null
+++ b/builtin/hook.c
@@ -0,0 +1,90 @@
+#include "cache.h"
+#include "builtin.h"
+#include "config.h"
+#include "hook.h"
+#include "parse-options.h"
+#include "strbuf.h"
+#include "strvec.h"
+
+#define BUILTIN_HOOK_RUN_USAGE \
+	N_("git hook run <hook-name> [-- <hook-args>]")
+
+static const char * const builtin_hook_usage[] = {
+	BUILTIN_HOOK_RUN_USAGE,
+	NULL
+};
+
+static const char * const builtin_hook_run_usage[] = {
+	BUILTIN_HOOK_RUN_USAGE,
+	NULL
+};
+
+static int run(int argc, const char **argv, const char *prefix)
+{
+	int i;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	const char *hook_name;
+	const char *hook_path;
+	struct option run_options[] = {
+		OPT_END(),
+	};
+	int ret;
+
+	argc = parse_options(argc, argv, prefix, run_options,
+			     builtin_hook_run_usage,
+			     PARSE_OPT_KEEP_DASHDASH);
+
+	if (!argc)
+		goto usage;
+
+	/*
+	 * Having a -- for "run" when providing <hook-args> is
+	 * mandatory.
+	 */
+	if (argc > 1 && strcmp(argv[1], "--") &&
+	    strcmp(argv[1], "--end-of-options"))
+		goto usage;
+
+	/* Add our arguments, start after -- */
+	for (i = 2 ; i < argc; i++)
+		strvec_push(&opt.args, argv[i]);
+
+	/* Need to take into account core.hooksPath */
+	git_config(git_default_config, NULL);
+
+	/*
+	 * We are not using a plain run_hooks() because we'd like to
+	 * detect missing hooks. Let's find it ourselves and call
+	 * run_hooks() instead.
+	 */
+	hook_name = argv[0];
+	hook_path = find_hook(hook_name);
+	if (!hook_path) {
+		error("cannot find a hook named %s", hook_name);
+		return 1;
+	}
+
+	ret = run_hooks(hook_name, hook_path, &opt);
+	run_hooks_opt_clear(&opt);
+	return ret;
+usage:
+	usage_with_options(builtin_hook_run_usage, run_options);
+}
+
+int cmd_hook(int argc, const char **argv, const char *prefix)
+{
+	struct option builtin_hook_options[] = {
+		OPT_END(),
+	};
+
+	argc = parse_options(argc, argv, NULL, builtin_hook_options,
+			     builtin_hook_usage, PARSE_OPT_STOP_AT_NON_OPTION);
+	if (!argc)
+		goto usage;
+
+	if (!strcmp(argv[0], "run"))
+		return run(argc, argv, prefix);
+
+usage:
+	usage_with_options(builtin_hook_usage, builtin_hook_options);
+}
diff --git a/command-list.txt b/command-list.txt
index a289f09ed6f..9ccd8e5aebe 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -103,6 +103,7 @@ git-grep                                mainporcelain           info
 git-gui                                 mainporcelain
 git-hash-object                         plumbingmanipulators
 git-help                                ancillaryinterrogators          complete
+git-hook                                mainporcelain
 git-http-backend                        synchingrepositories
 git-http-fetch                          synchelpers
 git-http-push                           synchelpers
diff --git a/git.c b/git.c
index 18bed9a9964..540909c391f 100644
--- a/git.c
+++ b/git.c
@@ -538,6 +538,7 @@ static struct cmd_struct commands[] = {
 	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
 	{ "hash-object", cmd_hash_object },
 	{ "help", cmd_help },
+	{ "hook", cmd_hook, RUN_SETUP },
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "init", cmd_init_db },
 	{ "init-db", cmd_init_db },
diff --git a/hook.c b/hook.c
index 97cd799a320..dd67801c962 100644
--- a/hook.c
+++ b/hook.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "hook.h"
 #include "run-command.h"
+#include "config.h"
 
 const char *find_hook(const char *name)
 {
@@ -40,3 +41,103 @@ int hook_exists(const char *name)
 {
 	return !!find_hook(name);
 }
+
+void run_hooks_opt_clear(struct run_hooks_opt *o)
+{
+	strvec_clear(&o->env);
+	strvec_clear(&o->args);
+}
+
+static int pick_next_hook(struct child_process *cp,
+			  struct strbuf *out,
+			  void *pp_cb,
+			  void **pp_task_cb)
+{
+	struct hook_cb_data *hook_cb = pp_cb;
+	struct hook *run_me = hook_cb->run_me;
+
+	if (!run_me)
+		return 0;
+
+	cp->no_stdin = 1;
+	cp->env = hook_cb->options->env.v;
+	cp->stdout_to_stderr = 1;
+	cp->trace2_hook_name = hook_cb->hook_name;
+
+	/* add command */
+	strvec_push(&cp->args, run_me->hook_path);
+
+	/*
+	 * add passed-in argv, without expanding - let the user get back
+	 * exactly what they put in
+	 */
+	strvec_pushv(&cp->args, hook_cb->options->args.v);
+
+	/* Provide context for errors if necessary */
+	*pp_task_cb = run_me;
+
+	/*
+	 * This pick_next_hook() will be called again, we're only
+	 * running one hook, so indicate that no more work will be
+	 * done.
+	 */
+	hook_cb->run_me = NULL;
+
+	return 1;
+}
+
+static int notify_start_failure(struct strbuf *out,
+				void *pp_cb,
+				void *pp_task_cp)
+{
+	struct hook_cb_data *hook_cb = pp_cb;
+	struct hook *attempted = pp_task_cp;
+
+	hook_cb->rc |= 1;
+
+	strbuf_addf(out, _("Couldn't start hook '%s'\n"),
+		    attempted->hook_path);
+
+	return 1;
+}
+
+static int notify_hook_finished(int result,
+				struct strbuf *out,
+				void *pp_cb,
+				void *pp_task_cb)
+{
+	struct hook_cb_data *hook_cb = pp_cb;
+
+	hook_cb->rc |= result;
+
+	return 0;
+}
+
+int run_hooks(const char *hook_name, const char *hook_path,
+	      struct run_hooks_opt *options)
+{
+	struct hook my_hook = {
+		.hook_path = hook_path,
+	};
+	struct hook_cb_data cb_data = {
+		.rc = 0,
+		.hook_name = hook_name,
+		.options = options,
+	};
+	int jobs = 1;
+
+	if (!options)
+		BUG("a struct run_hooks_opt must be provided to run_hooks");
+
+	cb_data.run_me = &my_hook;
+
+	run_processes_parallel_tr2(jobs,
+				   pick_next_hook,
+				   notify_start_failure,
+				   notify_hook_finished,
+				   &cb_data,
+				   "hook",
+				   hook_name);
+
+	return cb_data.rc;
+}
diff --git a/hook.h b/hook.h
index 368754b1201..d5bc1dc74c9 100644
--- a/hook.h
+++ b/hook.h
@@ -1,5 +1,33 @@
 #ifndef HOOK_H
 #define HOOK_H
+#include "strvec.h"
+
+struct hook {
+	/* The path to the hook */
+	const char *hook_path;
+};
+
+struct run_hooks_opt
+{
+	/* Environment vars to be set for each hook */
+	struct strvec env;
+
+	/* Args to be passed to each hook */
+	struct strvec args;
+};
+
+#define RUN_HOOKS_OPT_INIT { \
+	.env = STRVEC_INIT, \
+	.args = STRVEC_INIT, \
+}
+
+struct hook_cb_data {
+	/* rc reflects the cumulative failure state */
+	int rc;
+	const char *hook_name;
+	struct hook *run_me;
+	struct run_hooks_opt *options;
+};
 
 /**
  * Returns the path to the hook file, or NULL if the hook is missing
@@ -13,4 +41,15 @@ const char *find_hook(const char *name);
  */
 int hook_exists(const char *hookname);
 
+/**
+ * Clear data from an initialized "struct run_hooks-opt".
+ */
+void run_hooks_opt_clear(struct run_hooks_opt *o);
+
+/**
+ * Takes an already resolved hook found via find_hook() and runs
+ * it. Does not call run_hooks_opt_clear() for you.
+ */
+int run_hooks(const char *hookname, const char *hook_path,
+	      struct run_hooks_opt *options);
 #endif
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
new file mode 100755
index 00000000000..d0a3796f28e
--- /dev/null
+++ b/t/t1800-hook.sh
@@ -0,0 +1,128 @@
+#!/bin/sh
+
+test_description='git-hook command'
+
+. ./test-lib.sh
+
+test_expect_success 'git hook usage' '
+	test_expect_code 129 git hook &&
+	test_expect_code 129 git hook run &&
+	test_expect_code 129 git hook run -h &&
+	test_expect_code 129 git hook run --unknown 2>err &&
+	grep "unknown option" err
+'
+
+test_expect_success 'git hook run: nonexistent hook' '
+	cat >stderr.expect <<-\EOF &&
+	error: cannot find a hook named test-hook
+	EOF
+	test_expect_code 1 git hook run test-hook 2>stderr.actual &&
+	test_cmp stderr.expect stderr.actual
+'
+
+test_expect_success 'git hook run: basic' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo Test hook
+	EOF
+
+	cat >expect <<-\EOF &&
+	Test hook
+	EOF
+	git hook run test-hook 2>actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git hook run: stdout and stderr both write to our stderr' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo >&1 Will end up on stderr
+	echo >&2 Will end up on stderr
+	EOF
+
+	cat >stderr.expect <<-\EOF &&
+	Will end up on stderr
+	Will end up on stderr
+	EOF
+	git hook run test-hook >stdout.actual 2>stderr.actual &&
+	test_cmp stderr.expect stderr.actual &&
+	test_must_be_empty stdout.actual
+'
+
+test_expect_success 'git hook run: exit codes are passed along' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	exit 1
+	EOF
+
+	test_expect_code 1 git hook run test-hook &&
+
+	write_script .git/hooks/test-hook <<-EOF &&
+	exit 2
+	EOF
+
+	test_expect_code 2 git hook run test-hook &&
+
+	write_script .git/hooks/test-hook <<-EOF &&
+	exit 128
+	EOF
+
+	test_expect_code 128 git hook run test-hook &&
+
+	write_script .git/hooks/test-hook <<-EOF &&
+	exit 129
+	EOF
+
+	test_expect_code 129 git hook run test-hook
+'
+
+test_expect_success 'git hook run arg u ments without -- is not allowed' '
+	test_expect_code 129 git hook run test-hook arg u ments
+'
+
+test_expect_success 'git hook run -- pass arguments' '
+	write_script .git/hooks/test-hook <<-\EOF &&
+	echo $1
+	echo $2
+	EOF
+
+	cat >expect <<-EOF &&
+	arg
+	u ments
+	EOF
+
+	git hook run test-hook -- arg "u ments" 2>actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git hook run -- out-of-repo runs excluded' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo Test hook
+	EOF
+
+	nongit test_must_fail git hook run test-hook
+'
+
+test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
+	mkdir my-hooks &&
+	write_script my-hooks/test-hook <<-\EOF &&
+	echo Hook ran $1 >>actual
+	EOF
+
+	cat >expect <<-\EOF &&
+	Test hook
+	Hook ran one
+	Hook ran two
+	Hook ran three
+	Hook ran four
+	EOF
+
+	# Test various ways of specifying the path. See also
+	# t1350-config-hooks-path.sh
+	>actual &&
+	git hook run test-hook -- ignored 2>>actual &&
+	git -c core.hooksPath=my-hooks hook run test-hook -- one 2>>actual &&
+	git -c core.hooksPath=my-hooks/ hook run test-hook -- two 2>>actual &&
+	git -c core.hooksPath="$PWD/my-hooks" hook run test-hook -- three 2>>actual &&
+	git -c core.hooksPath="$PWD/my-hooks/" hook run test-hook -- four 2>>actual &&
+	test_cmp expect actual
+'
+
+test_done
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 09/36] gc: use hook library for pre-auto-gc hook
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (7 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 08/36] hook: add 'run' subcommand Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 10/36] rebase: convert pre-rebase to use hook.h Ævar Arnfjörð Bjarmason
                                 ` (26 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Move the pre-auto-gc hook away from run-command.h to and over to the
new hook.h library.

To do this introduce a simple run_hooks_oneshot() wrapper, we'll be
using it extensively for these simple cases of wanting to run a single
hook under a given name, and having it free the memory we allocate for
us.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/gc.c |  3 ++-
 hook.c       | 23 +++++++++++++++++++++++
 hook.h       | 13 +++++++++++++
 3 files changed, 38 insertions(+), 1 deletion(-)

diff --git a/builtin/gc.c b/builtin/gc.c
index 6ce5ca45126..c91a0786836 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -32,6 +32,7 @@
 #include "remote.h"
 #include "object-store.h"
 #include "exec-cmd.h"
+#include "hook.h"
 
 #define FAILED_RUN "failed to run %s"
 
@@ -394,7 +395,7 @@ static int need_to_gc(void)
 	else
 		return 0;
 
-	if (run_hook_le(NULL, "pre-auto-gc", NULL))
+	if (run_hooks_oneshot("pre-auto-gc", NULL))
 		return 0;
 	return 1;
 }
diff --git a/hook.c b/hook.c
index dd67801c962..1776895289e 100644
--- a/hook.c
+++ b/hook.c
@@ -141,3 +141,26 @@ int run_hooks(const char *hook_name, const char *hook_path,
 
 	return cb_data.rc;
 }
+
+int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
+{
+	const char *hook_path;
+	int ret;
+	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
+
+	if (!options)
+		options = &hook_opt_scratch;
+
+	hook_path = find_hook(hook_name);
+	if (!hook_path) {
+		ret = 0;
+		goto cleanup;
+	}
+
+	ret = run_hooks(hook_name, hook_path, options);
+
+cleanup:
+	run_hooks_opt_clear(options);
+
+	return ret;
+}
diff --git a/hook.h b/hook.h
index d5bc1dc74c9..b7a106a6e2b 100644
--- a/hook.h
+++ b/hook.h
@@ -49,7 +49,20 @@ void run_hooks_opt_clear(struct run_hooks_opt *o);
 /**
  * Takes an already resolved hook found via find_hook() and runs
  * it. Does not call run_hooks_opt_clear() for you.
+ *
+ * See run_hooks_oneshot() for the simpler one-shot API.
  */
 int run_hooks(const char *hookname, const char *hook_path,
 	      struct run_hooks_opt *options);
+
+/**
+ * Calls find_hook() on your "hook_name" and runs the hooks (if any)
+ * with run_hooks().
+ *
+ * If "options" is provided calls run_hooks_opt_clear() on it for
+ * you. If "options" is NULL the default options from
+ * RUN_HOOKS_OPT_INIT will be used.
+ */
+int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
+
 #endif
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 10/36] rebase: convert pre-rebase to use hook.h
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (8 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 09/36] gc: use hook library for pre-auto-gc hook Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 11/36] am: convert applypatch " Ævar Arnfjörð Bjarmason
                                 ` (25 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Move the pre-rebase hook away from run-command.h to and over to the
new hook.h library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/rebase.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/builtin/rebase.c b/builtin/rebase.c
index c284a7ace19..ee68a1df492 100644
--- a/builtin/rebase.c
+++ b/builtin/rebase.c
@@ -28,6 +28,7 @@
 #include "sequencer.h"
 #include "rebase-interactive.h"
 #include "reset.h"
+#include "hook.h"
 
 #define DEFAULT_REFLOG_ACTION "rebase"
 
@@ -1313,6 +1314,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	char *squash_onto_name = NULL;
 	int reschedule_failed_exec = -1;
 	int allow_preemptive_ff = 1;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
 	struct option builtin_rebase_options[] = {
 		OPT_STRING(0, "onto", &options.onto_name,
 			   N_("revision"),
@@ -2022,9 +2024,9 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	}
 
 	/* If a hook exists, give it a chance to interrupt*/
+	strvec_pushl(&hook_opt.args, options.upstream_arg, argc ? argv[0] : NULL, NULL);
 	if (!ok_to_skip_pre_rebase &&
-	    run_hook_le(NULL, "pre-rebase", options.upstream_arg,
-			argc ? argv[0] : NULL, NULL))
+	    run_hooks_oneshot("pre-rebase", &hook_opt))
 		die(_("The pre-rebase hook refused to rebase."));
 
 	if (options.flags & REBASE_DIFFSTAT) {
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 11/36] am: convert applypatch to use hook.h
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (9 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 10/36] rebase: convert pre-rebase to use hook.h Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 12/36] hooks: convert 'post-checkout' hook to hook library Ævar Arnfjörð Bjarmason
                                 ` (24 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Teach pre-applypatch, post-applypatch, and applypatch-msg to use the
hook.h library instead of the run-command.h library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/am.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index c603f3cebdf..e444b18b64a 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -446,9 +446,11 @@ static void am_destroy(const struct am_state *state)
 static int run_applypatch_msg_hook(struct am_state *state)
 {
 	int ret;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 
 	assert(state->msg);
-	ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL);
+	strvec_push(&opt.args, am_path(state, "final-commit"));
+	ret = run_hooks_oneshot("applypatch-msg", &opt);
 
 	if (!ret) {
 		FREE_AND_NULL(state->msg);
@@ -1609,7 +1611,7 @@ static void do_commit(const struct am_state *state)
 	const char *reflog_msg, *author, *committer = NULL;
 	struct strbuf sb = STRBUF_INIT;
 
-	if (run_hook_le(NULL, "pre-applypatch", NULL))
+	if (run_hooks_oneshot("pre-applypatch", NULL))
 		exit(1);
 
 	if (write_cache_as_tree(&tree, 0, NULL))
@@ -1661,7 +1663,7 @@ static void do_commit(const struct am_state *state)
 		fclose(fp);
 	}
 
-	run_hook_le(NULL, "post-applypatch", NULL);
+	run_hooks_oneshot("post-applypatch", NULL);
 
 	strbuf_release(&sb);
 }
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 12/36] hooks: convert 'post-checkout' hook to hook library
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (10 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 11/36] am: convert applypatch " Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 13/36] merge: convert post-merge to use hook.h Ævar Arnfjörð Bjarmason
                                 ` (23 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Move the running of the 'post-checkout' hook away from run-command.h
to the new hook.h library. For "worktree" this requires a change to it
to run the hooks from a given directory.

We could strictly speaking skip the "absolute_path" flag and just
check if "dir" is specified, but let's split them up for clarity, as
well as for any future user who'd like to set "dir" but not implicitly
change the argument to an absolute path.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/checkout.c | 14 +++++++++-----
 builtin/clone.c    |  6 ++++--
 builtin/worktree.c | 28 ++++++++++++----------------
 hook.c             |  9 +++++++++
 hook.h             |  9 +++++++++
 read-cache.c       |  1 +
 reset.c            | 14 ++++++++++----
 7 files changed, 54 insertions(+), 27 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index b5d477919a7..863b02a7d7c 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "diff.h"
 #include "dir.h"
+#include "hook.h"
 #include "ll-merge.h"
 #include "lockfile.h"
 #include "merge-recursive.h"
@@ -106,13 +107,16 @@ struct branch_info {
 static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit,
 			      int changed)
 {
-	return run_hook_le(NULL, "post-checkout",
-			   oid_to_hex(old_commit ? &old_commit->object.oid : null_oid()),
-			   oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()),
-			   changed ? "1" : "0", NULL);
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
 	/* "new_commit" can be NULL when checking out from the index before
 	   a commit exists. */
-
+	strvec_pushl(&opt.args,
+		     oid_to_hex(old_commit ? &old_commit->object.oid : null_oid()),
+		     oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()),
+		     changed ? "1" : "0",
+		     NULL);
+	return run_hooks_oneshot("post-checkout", &opt);
 }
 
 static int update_some(const struct object_id *oid, struct strbuf *base,
diff --git a/builtin/clone.c b/builtin/clone.c
index 66fe66679c8..27fc05ee511 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -32,6 +32,7 @@
 #include "connected.h"
 #include "packfile.h"
 #include "list-objects-filter-options.h"
+#include "hook.h"
 
 /*
  * Overall FIXMEs:
@@ -775,6 +776,7 @@ static int checkout(int submodule_progress)
 	struct tree *tree;
 	struct tree_desc t;
 	int err = 0;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
 
 	if (option_no_checkout)
 		return 0;
@@ -820,8 +822,8 @@ static int checkout(int submodule_progress)
 	if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
 		die(_("unable to write new index file"));
 
-	err |= run_hook_le(NULL, "post-checkout", oid_to_hex(null_oid()),
-			   oid_to_hex(&oid), "1", NULL);
+	strvec_pushl(&hook_opt.args, oid_to_hex(null_oid()), oid_to_hex(&oid), "1", NULL);
+	err |= run_hooks_oneshot("post-checkout", &hook_opt);
 
 	if (!err && (option_recurse_submodules.nr > 0)) {
 		struct strvec args = STRVEC_INIT;
diff --git a/builtin/worktree.c b/builtin/worktree.c
index d22ece93e1a..330867c19bf 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -382,22 +382,18 @@ static int add_worktree(const char *path, const char *refname,
 	 * is_junk is cleared, but do return appropriate code when hook fails.
 	 */
 	if (!ret && opts->checkout) {
-		const char *hook = find_hook("post-checkout");
-		if (hook) {
-			const char *env[] = { "GIT_DIR", "GIT_WORK_TREE", NULL };
-			cp.git_cmd = 0;
-			cp.no_stdin = 1;
-			cp.stdout_to_stderr = 1;
-			cp.dir = path;
-			cp.env = env;
-			cp.argv = NULL;
-			cp.trace2_hook_name = "post-checkout";
-			strvec_pushl(&cp.args, absolute_path(hook),
-				     oid_to_hex(null_oid()),
-				     oid_to_hex(&commit->object.oid),
-				     "1", NULL);
-			ret = run_command(&cp);
-		}
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
+		strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
+		strvec_pushl(&opt.args,
+			     oid_to_hex(null_oid()),
+			     oid_to_hex(&commit->object.oid),
+			     "1",
+			     NULL);
+		opt.dir = path;
+		opt.absolute_path = 1;
+
+		ret = run_hooks_oneshot("post-checkout", &opt);
 	}
 
 	strvec_clear(&child_env);
diff --git a/hook.c b/hook.c
index 1776895289e..8a923e52c1b 100644
--- a/hook.c
+++ b/hook.c
@@ -63,6 +63,7 @@ static int pick_next_hook(struct child_process *cp,
 	cp->env = hook_cb->options->env.v;
 	cp->stdout_to_stderr = 1;
 	cp->trace2_hook_name = hook_cb->hook_name;
+	cp->dir = hook_cb->options->dir;
 
 	/* add command */
 	strvec_push(&cp->args, run_me->hook_path);
@@ -116,6 +117,7 @@ static int notify_hook_finished(int result,
 int run_hooks(const char *hook_name, const char *hook_path,
 	      struct run_hooks_opt *options)
 {
+	struct strbuf abs_path = STRBUF_INIT;
 	struct hook my_hook = {
 		.hook_path = hook_path,
 	};
@@ -129,6 +131,10 @@ int run_hooks(const char *hook_name, const char *hook_path,
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
 
+	if (options->absolute_path) {
+		strbuf_add_absolute_path(&abs_path, hook_path);
+		my_hook.hook_path = abs_path.buf;
+	}
 	cb_data.run_me = &my_hook;
 
 	run_processes_parallel_tr2(jobs,
@@ -139,6 +145,9 @@ int run_hooks(const char *hook_name, const char *hook_path,
 				   "hook",
 				   hook_name);
 
+	if (options->absolute_path)
+		strbuf_release(&abs_path);
+
 	return cb_data.rc;
 }
 
diff --git a/hook.h b/hook.h
index b7a106a6e2b..3edd937e198 100644
--- a/hook.h
+++ b/hook.h
@@ -14,6 +14,15 @@ struct run_hooks_opt
 
 	/* Args to be passed to each hook */
 	struct strvec args;
+
+	/*
+	 * Resolve and run the "absolute_path(hook)" instead of
+	 * "hook". Used for "git worktree" hooks
+	 */
+	int absolute_path;
+
+	/* Path to initial working directory for subprocess */
+	const char *dir;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
diff --git a/read-cache.c b/read-cache.c
index 9048ef9e905..7a9ffea0885 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -28,6 +28,7 @@
 #include "sparse-index.h"
 #include "csum-file.h"
 #include "promisor-remote.h"
+#include "hook.h"
 
 /* Mask for the name length in ce_flags in the on-disk index */
 
diff --git a/reset.c b/reset.c
index 79310ae071b..1237ced8a58 100644
--- a/reset.c
+++ b/reset.c
@@ -7,6 +7,7 @@
 #include "tree-walk.h"
 #include "tree.h"
 #include "unpack-trees.h"
+#include "hook.h"
 
 int reset_head(struct repository *r, struct object_id *oid, const char *action,
 	       const char *switch_to_branch, unsigned flags,
@@ -125,10 +126,15 @@ int reset_head(struct repository *r, struct object_id *oid, const char *action,
 			ret = create_symref("HEAD", switch_to_branch,
 					    reflog_head);
 	}
-	if (run_hook)
-		run_hook_le(NULL, "post-checkout",
-			    oid_to_hex(orig ? orig : null_oid()),
-			    oid_to_hex(oid), "1", NULL);
+	if (run_hook) {
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		strvec_pushl(&opt.args,
+			     oid_to_hex(orig ? orig : null_oid()),
+			     oid_to_hex(oid),
+			     "1",
+			     NULL);
+		run_hooks_oneshot("post-checkout", &opt);
+	}
 
 leave_reset_head:
 	strbuf_release(&msg);
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 13/36] merge: convert post-merge to use hook.h
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (11 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 12/36] hooks: convert 'post-checkout' hook to hook library Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 14/36] git hook run: add an --ignore-missing flag Ævar Arnfjörð Bjarmason
                                 ` (22 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Teach post-merge to use the hook.h library instead of the
run-command.h library to run hooks.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/merge.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/builtin/merge.c b/builtin/merge.c
index d75a438298b..ca9b3ba4827 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -448,6 +448,7 @@ static void finish(struct commit *head_commit,
 		   const struct object_id *new_head, const char *msg)
 {
 	struct strbuf reflog_message = STRBUF_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	const struct object_id *head = &head_commit->object.oid;
 
 	if (!msg)
@@ -489,7 +490,8 @@ static void finish(struct commit *head_commit,
 	}
 
 	/* Run a post-merge hook */
-	run_hook_le(NULL, "post-merge", squash ? "1" : "0", NULL);
+	strvec_push(&opt.args, squash ? "1" : "0");
+	run_hooks_oneshot("post-merge", &opt);
 
 	apply_autostash(git_path_merge_autostash(the_repository));
 	strbuf_release(&reflog_message);
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 14/36] git hook run: add an --ignore-missing flag
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (12 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 13/36] merge: convert post-merge to use hook.h Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 15/36] send-email: use 'git hook run' for 'sendemail-validate' Ævar Arnfjörð Bjarmason
                                 ` (21 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

For certain one-shot hooks we'd like to optimistically run them, and
not complain if they don't exist. This will be used by send-email in a
subsequent commit.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-hook.txt | 10 +++++++++-
 builtin/hook.c             | 10 ++++++++--
 t/t1800-hook.sh            |  5 +++++
 3 files changed, 22 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 660d6a992a0..097fb9de63b 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -8,7 +8,7 @@ git-hook - run git hooks
 SYNOPSIS
 --------
 [verse]
-'git hook' run <hook-name> [-- <hook-args>]
+'git hook' run [--ignore-missing] <hook-name> [-- <hook-args>]
 
 DESCRIPTION
 -----------
@@ -29,6 +29,14 @@ optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
 arguments (if any) differ by hook name, see linkgit:githooks[5] for
 what those are.
 
+OPTIONS
+-------
+
+--ignore-missing::
+	Ignore any missing hook by quietly returning zero. Used for
+	tools that want to do a blind one-shot run of a hook that may
+	or may not be present.
+
 SEE ALSO
 --------
 linkgit:githooks[5]
diff --git a/builtin/hook.c b/builtin/hook.c
index 012a2973b38..76d49e672f4 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -7,7 +7,7 @@
 #include "strvec.h"
 
 #define BUILTIN_HOOK_RUN_USAGE \
-	N_("git hook run <hook-name> [-- <hook-args>]")
+	N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")
 
 static const char * const builtin_hook_usage[] = {
 	BUILTIN_HOOK_RUN_USAGE,
@@ -23,9 +23,12 @@ static int run(int argc, const char **argv, const char *prefix)
 {
 	int i;
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	int ignore_missing = 0;
 	const char *hook_name;
 	const char *hook_path;
 	struct option run_options[] = {
+		OPT_BOOL(0, "ignore-missing", &ignore_missing,
+			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
 		OPT_END(),
 	};
 	int ret;
@@ -55,9 +58,12 @@ static int run(int argc, const char **argv, const char *prefix)
 	/*
 	 * We are not using a plain run_hooks() because we'd like to
 	 * detect missing hooks. Let's find it ourselves and call
-	 * run_hooks() instead.
+	 * run_hooks() instead...
 	 */
 	hook_name = argv[0];
+	if (ignore_missing)
+		/* ... act like a plain run_hooks() under --ignore-missing */
+		return run_hooks_oneshot(hook_name, &opt);
 	hook_path = find_hook(hook_name);
 	if (!hook_path) {
 		error("cannot find a hook named %s", hook_name);
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
index d0a3796f28e..feb95b8fc8d 100755
--- a/t/t1800-hook.sh
+++ b/t/t1800-hook.sh
@@ -20,6 +20,11 @@ test_expect_success 'git hook run: nonexistent hook' '
 	test_cmp stderr.expect stderr.actual
 '
 
+test_expect_success 'git hook run: nonexistent hook with --ignore-missing' '
+	git hook run --ignore-missing does-not-exist 2>stderr.actual &&
+	test_must_be_empty stderr.actual
+'
+
 test_expect_success 'git hook run: basic' '
 	write_script .git/hooks/test-hook <<-EOF &&
 	echo Test hook
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 15/36] send-email: use 'git hook run' for 'sendemail-validate'
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (13 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 14/36] git hook run: add an --ignore-missing flag Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 16/36] git-p4: use 'git hook' to run hooks Ævar Arnfjörð Bjarmason
                                 ` (20 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Change the "sendmail-validate" hook to be run via the "git hook run"
wrapper instead of via a direct invocation.

This is the smallest possibly change to get "send-email" using "git
hook run". We still check the hook itself with "-x", and set a
"GIT_DIR" variable, both of which are asserted by our tests. We'll
need to get rid of this special behavior if we start running N hooks,
but for now let's be as close to bug-for-bug compatible as possible.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-send-email.perl   | 20 ++++++++++++--------
 t/t9001-send-email.sh |  4 ++--
 2 files changed, 14 insertions(+), 10 deletions(-)

diff --git a/git-send-email.perl b/git-send-email.perl
index e65d969d0bb..126850d974b 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -195,13 +195,13 @@ sub format_2822_time {
 my $editor;
 
 sub system_or_msg {
-	my ($args, $msg) = @_;
+	my ($args, $msg, $cmd_name) = @_;
 	system(@$args);
 	my $signalled = $? & 127;
 	my $exit_code = $? >> 8;
 	return unless $signalled or $exit_code;
 
-	my @sprintf_args = ($args->[0], $exit_code);
+	my @sprintf_args = ($cmd_name ? $cmd_name : $args->[0], $exit_code);
 	if (defined $msg) {
 		# Quiet the 'redundant' warning category, except we
 		# need to support down to Perl 5.8, so we can't do a
@@ -2031,10 +2031,10 @@ sub validate_patch {
 	my ($fn, $xfer_encoding) = @_;
 
 	if ($repo) {
+		my $hook_name = 'sendemail-validate';
 		my $hooks_path = $repo->command_oneline('rev-parse', '--git-path', 'hooks');
 		require File::Spec;
-		my $validate_hook = File::Spec->catfile($hooks_path,
-					    'sendemail-validate');
+		my $validate_hook = File::Spec->catfile($hooks_path, $hook_name);
 		my $hook_error;
 		if (-x $validate_hook) {
 			require Cwd;
@@ -2044,13 +2044,17 @@ sub validate_patch {
 			chdir($repo->wc_path() or $repo->repo_path())
 				or die("chdir: $!");
 			local $ENV{"GIT_DIR"} = $repo->repo_path();
-			$hook_error = system_or_msg([$validate_hook, $target]);
+			my @validate_hook = ("git", "hook", "run", "--ignore-missing", $hook_name, "--", $target);
+			$hook_error = system_or_msg(\@validate_hook, undef,
+						       "git hook run $hook_name -- <patch>");
 			chdir($cwd_save) or die("chdir: $!");
 		}
 		if ($hook_error) {
-			die sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" .
-				       "%s\n" .
-				       "warning: no patches were sent\n"), $fn, $hook_error);
+			$hook_error = sprintf(__("fatal: %s: rejected by %s hook\n" .
+						 $hook_error . "\n" .
+						 "warning: no patches were sent\n"),
+					      $fn, $hook_name);
+			die $hook_error;
 		}
 	}
 
diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index 57fc10e7f82..9ec7d75f0ff 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -539,7 +539,7 @@ test_expect_success $PREREQ "--validate respects relative core.hooksPath path" '
 	test_path_is_file my-hooks.ran &&
 	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
-	fatal: command '"'"'my-hooks/sendemail-validate'"'"' died with exit code 1
+	fatal: command '"'"'git hook run sendemail-validate -- <patch>'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
@@ -558,7 +558,7 @@ test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" '
 	test_path_is_file my-hooks.ran &&
 	cat >expect <<-EOF &&
 	fatal: longline.patch: rejected by sendemail-validate hook
-	fatal: command '"'"'$hooks_path/sendemail-validate'"'"' died with exit code 1
+	fatal: command '"'"'git hook run sendemail-validate -- <patch>'"'"' died with exit code 1
 	warning: no patches were sent
 	EOF
 	test_cmp expect actual
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 16/36] git-p4: use 'git hook' to run hooks
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (14 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 15/36] send-email: use 'git hook run' for 'sendemail-validate' Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 17/36] commit: convert {pre-commit,prepare-commit-msg} hook to hook.h Ævar Arnfjörð Bjarmason
                                 ` (19 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Instead of duplicating the behavior of run-command.h:run_hook_le() in
Python, we can directly call 'git hook run'. We emulate the existence
check with the --ignore-missing flag.

As this is the last hook execution in git.git to not go through "git
hook run" or the hook.[ch] library we can now be absolutely sure that
our assertion in hook.c that only hooks known by the generated (from
githooks(5)) hook-list.h are permitted.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-p4.py | 72 ++++++-------------------------------------------------
 1 file changed, 7 insertions(+), 65 deletions(-)

diff --git a/git-p4.py b/git-p4.py
index 2b4500226aa..1f24cbf0bca 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -207,71 +207,13 @@ def decode_path(path):
         return path
 
 def run_git_hook(cmd, param=[]):
-    """Execute a hook if the hook exists."""
-    if verbose:
-        sys.stderr.write("Looking for hook: %s\n" % cmd)
-        sys.stderr.flush()
-
-    hooks_path = gitConfig("core.hooksPath")
-    if len(hooks_path) <= 0:
-        hooks_path = os.path.join(os.environ["GIT_DIR"], "hooks")
-
-    if not isinstance(param, list):
-        param=[param]
-
-    # resolve hook file name, OS depdenent
-    hook_file = os.path.join(hooks_path, cmd)
-    if platform.system() == 'Windows':
-        if not os.path.isfile(hook_file):
-            # look for the file with an extension
-            files = glob.glob(hook_file + ".*")
-            if not files:
-                return True
-            files.sort()
-            hook_file = files.pop()
-            while hook_file.upper().endswith(".SAMPLE"):
-                # The file is a sample hook. We don't want it
-                if len(files) > 0:
-                    hook_file = files.pop()
-                else:
-                    return True
-
-    if not os.path.isfile(hook_file) or not os.access(hook_file, os.X_OK):
-        return True
-
-    return run_hook_command(hook_file, param) == 0
-
-def run_hook_command(cmd, param):
-    """Executes a git hook command
-       cmd = the command line file to be executed. This can be
-       a file that is run by OS association.
-
-       param = a list of parameters to pass to the cmd command
-
-       On windows, the extension is checked to see if it should
-       be run with the Git for Windows Bash shell.  If there
-       is no file extension, the file is deemed a bash shell
-       and will be handed off to sh.exe. Otherwise, Windows
-       will be called with the shell to handle the file assocation.
-
-       For non Windows operating systems, the file is called
-       as an executable.
-    """
-    cli = [cmd] + param
-    use_shell = False
-    if platform.system() == 'Windows':
-        (root,ext) = os.path.splitext(cmd)
-        if ext == "":
-            exe_path = os.environ.get("EXEPATH")
-            if exe_path is None:
-                exe_path = ""
-            else:
-                exe_path = os.path.join(exe_path, "bin")
-            cli = [os.path.join(exe_path, "SH.EXE")] + cli
-        else:
-            use_shell = True
-    return subprocess.call(cli, shell=use_shell)
-
+    """args are specified with -a <arg> -a <arg> -a <arg>"""
+    args = ['git', 'hook', 'run', '--ignore-missing', cmd]
+    if param:
+        args.append("--")
+        for p in param:
+            args.append(p)
+    return subprocess.call(args) == 0
 
 def write_pipe(c, stdin):
     if verbose:
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 17/36] commit: convert {pre-commit,prepare-commit-msg} hook to hook.h
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (15 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 16/36] git-p4: use 'git hook' to run hooks Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 18/36] read-cache: convert post-index-change to use hook.h Ævar Arnfjörð Bjarmason
                                 ` (18 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Move these hooks hook away from run-command.h to and over to the new
hook.h library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 commit.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/commit.c b/commit.c
index 143f472c0f2..63d7943a86d 100644
--- a/commit.c
+++ b/commit.c
@@ -21,6 +21,7 @@
 #include "commit-reach.h"
 #include "run-command.h"
 #include "shallow.h"
+#include "hook.h"
 
 static struct commit_extra_header *read_commit_extra_header_lines(const char *buf, size_t len, const char **);
 
@@ -1698,22 +1699,22 @@ size_t ignore_non_trailer(const char *buf, size_t len)
 int run_commit_hook(int editor_is_used, const char *index_file,
 		    const char *name, ...)
 {
-	struct strvec hook_env = STRVEC_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	va_list args;
-	int ret;
+	const char *arg;
 
-	strvec_pushf(&hook_env, "GIT_INDEX_FILE=%s", index_file);
+	strvec_pushf(&opt.env, "GIT_INDEX_FILE=%s", index_file);
 
 	/*
 	 * Let the hook know that no editor will be launched.
 	 */
 	if (!editor_is_used)
-		strvec_push(&hook_env, "GIT_EDITOR=:");
+		strvec_push(&opt.env, "GIT_EDITOR=:");
 
 	va_start(args, name);
-	ret = run_hook_ve(hook_env.v, name, args);
+	while ((arg = va_arg(args, const char *)))
+		strvec_push(&opt.args, arg);
 	va_end(args);
-	strvec_clear(&hook_env);
 
-	return ret;
+	return run_hooks_oneshot(name, &opt);
 }
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 18/36] read-cache: convert post-index-change to use hook.h
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (16 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 17/36] commit: convert {pre-commit,prepare-commit-msg} hook to hook.h Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 19/36] receive-pack: convert push-to-checkout hook to hook.h Ævar Arnfjörð Bjarmason
                                 ` (17 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Move the post-index-change hook away from run-command.h to and over to
the new hook.h library.

This removes the last direct user of run_hook_ve(), so we can make the
function static now. It'll be removed entirely soon.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 read-cache.c  | 10 +++++++---
 run-command.c |  2 +-
 run-command.h |  1 -
 3 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/read-cache.c b/read-cache.c
index 7a9ffea0885..875f6c1dea5 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -3069,6 +3069,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 {
 	int ret;
 	int was_full = !istate->sparse_index;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
 
 	ret = convert_to_sparse(istate);
 
@@ -3097,9 +3098,12 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 	else
 		ret = close_lock_file_gently(lock);
 
-	run_hook_le(NULL, "post-index-change",
-			istate->updated_workdir ? "1" : "0",
-			istate->updated_skipworktree ? "1" : "0", NULL);
+	strvec_pushl(&hook_opt.args,
+		     istate->updated_workdir ? "1" : "0",
+		     istate->updated_skipworktree ? "1" : "0",
+		     NULL);
+	run_hooks_oneshot("post-index-change", &hook_opt);
+
 	istate->updated_workdir = 0;
 	istate->updated_skipworktree = 0;
 
diff --git a/run-command.c b/run-command.c
index 352f5be1646..b4341ba1c7b 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1320,7 +1320,7 @@ int async_with_fork(void)
 #endif
 }
 
-int run_hook_ve(const char *const *env, const char *name, va_list args)
+static int run_hook_ve(const char *const *env, const char *name, va_list args)
 {
 	struct child_process hook = CHILD_PROCESS_INIT;
 	const char *p;
diff --git a/run-command.h b/run-command.h
index f76b740f927..7a867d41217 100644
--- a/run-command.h
+++ b/run-command.h
@@ -219,7 +219,6 @@ int run_command(struct child_process *);
  */
 LAST_ARG_MUST_BE_NULL
 int run_hook_le(const char *const *env, const char *name, ...);
-int run_hook_ve(const char *const *env, const char *name, va_list args);
 
 /*
  * Trigger an auto-gc
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 19/36] receive-pack: convert push-to-checkout hook to hook.h
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (17 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 18/36] read-cache: convert post-index-change to use hook.h Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 20/36] run-command: remove old run_hook_{le,ve}() hook API Ævar Arnfjörð Bjarmason
                                 ` (16 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Move the push-to-checkout hook away from run-command.h to and over to
the new hook.h library.

This removes the last direct user of run_hook_le(), so we could remove
that function now, but let's leave that to a follow-up cleanup commit.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/receive-pack.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 91fa799b66e..a7d03bbc7d3 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1435,9 +1435,12 @@ static const char *push_to_checkout(unsigned char *hash,
 				    struct strvec *env,
 				    const char *work_tree)
 {
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
-	if (run_hook_le(env->v, push_to_checkout_hook,
-			hash_to_hex(hash), NULL))
+	strvec_pushv(&opt.env, env->v);
+	strvec_push(&opt.args, hash_to_hex(hash));
+	if (run_hooks_oneshot(push_to_checkout_hook, &opt))
 		return "push-to-checkout hook declined";
 	else
 		return NULL;
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 20/36] run-command: remove old run_hook_{le,ve}() hook API
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (18 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 19/36] receive-pack: convert push-to-checkout hook to hook.h Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 21/36] run-command: allow stdin for run_processes_parallel Ævar Arnfjörð Bjarmason
                                 ` (15 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

The new hook.h library has replaced all run-command.h hook-related
functionality. So let's delete this dead code.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 run-command.c | 32 --------------------------------
 run-command.h | 16 ----------------
 2 files changed, 48 deletions(-)

diff --git a/run-command.c b/run-command.c
index b4341ba1c7b..1399243de8a 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1320,38 +1320,6 @@ int async_with_fork(void)
 #endif
 }
 
-static int run_hook_ve(const char *const *env, const char *name, va_list args)
-{
-	struct child_process hook = CHILD_PROCESS_INIT;
-	const char *p;
-
-	p = find_hook(name);
-	if (!p)
-		return 0;
-
-	strvec_push(&hook.args, p);
-	while ((p = va_arg(args, const char *)))
-		strvec_push(&hook.args, p);
-	hook.env = env;
-	hook.no_stdin = 1;
-	hook.stdout_to_stderr = 1;
-	hook.trace2_hook_name = name;
-
-	return run_command(&hook);
-}
-
-int run_hook_le(const char *const *env, const char *name, ...)
-{
-	va_list args;
-	int ret;
-
-	va_start(args, name);
-	ret = run_hook_ve(env, name, args);
-	va_end(args);
-
-	return ret;
-}
-
 struct io_pump {
 	/* initialized by caller */
 	int fd;
diff --git a/run-command.h b/run-command.h
index 7a867d41217..cfb6887e4ae 100644
--- a/run-command.h
+++ b/run-command.h
@@ -204,22 +204,6 @@ int finish_command_in_signal(struct child_process *);
  */
 int run_command(struct child_process *);
 
-/**
- * Run a hook.
- * The first argument is a pathname to an index file, or NULL
- * if the hook uses the default index file or no index is needed.
- * The second argument is the name of the hook.
- * The further arguments correspond to the hook arguments.
- * The last argument has to be NULL to terminate the arguments list.
- * If the hook does not exist or is not executable, the return
- * value will be zero.
- * If it is executable, the hook will be executed and the exit
- * status of the hook is returned.
- * On execution, .stdout_to_stderr and .no_stdin will be set.
- */
-LAST_ARG_MUST_BE_NULL
-int run_hook_le(const char *const *env, const char *name, ...);
-
 /*
  * Trigger an auto-gc
  */
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 21/36] run-command: allow stdin for run_processes_parallel
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (19 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 20/36] run-command: remove old run_hook_{le,ve}() hook API Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 22/36] hook: support passing stdin to hooks Ævar Arnfjörð Bjarmason
                                 ` (14 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

While it makes sense not to inherit stdin from the parent process to
avoid deadlocking, it's not necessary to completely ban stdin to
children. An informed user should be able to configure stdin safely. By
setting `some_child.process.no_stdin=1` before calling `get_next_task()`
we provide a reasonable default behavior but enable users to set up
stdin streaming for themselves during the callback.

`some_child.process.stdout_to_stderr`, however, remains unmodifiable by
`get_next_task()` - the rest of the run_processes_parallel() API depends
on child output in stderr.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 run-command.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/run-command.c b/run-command.c
index 1399243de8a..482ee2d76c6 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1627,6 +1627,14 @@ static int pp_start_one(struct parallel_processes *pp)
 	if (i == pp->max_processes)
 		BUG("bookkeeping is hard");
 
+	/*
+	 * By default, do not inherit stdin from the parent process - otherwise,
+	 * all children would share stdin! Users may overwrite this to provide
+	 * something to the child's stdin by having their 'get_next_task'
+	 * callback assign 0 to .no_stdin and an appropriate integer to .in.
+	 */
+	pp->children[i].process.no_stdin = 1;
+
 	code = pp->get_next_task(&pp->children[i].process,
 				 &pp->children[i].err,
 				 pp->data,
@@ -1638,7 +1646,6 @@ static int pp_start_one(struct parallel_processes *pp)
 	}
 	pp->children[i].process.err = -1;
 	pp->children[i].process.stdout_to_stderr = 1;
-	pp->children[i].process.no_stdin = 1;
 
 	if (start_command(&pp->children[i].process)) {
 		code = pp->start_failure(&pp->children[i].err,
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 22/36] hook: support passing stdin to hooks
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (20 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 21/36] run-command: allow stdin for run_processes_parallel Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 23/36] am: convert 'post-rewrite' hook to hook.h Ævar Arnfjörð Bjarmason
                                 ` (13 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Some hooks (such as post-rewrite) need to take input via stdin.
Previously, callers provided stdin to hooks by setting
run-command.h:child_process.in, which takes a FD. Callers would open the
file in question themselves before calling run-command(). However, since
we will now need to seek to the front of the file and read it again for
every hook which runs, hook.h:run_command() takes a path and handles FD
management itself. Since this file is opened for read only, it should
not prevent later parallel execution support.

On the frontend, this is supported by asking for a file path, rather
than by reading stdin. Reading directly from stdin would involve caching
the entire stdin (to memory or to disk) and reading it back from the
beginning to each hook. We'd want to support cases like insufficient
memory or storage for the file. While this may prove useful later, for
now the path of least resistance is to just ask the user to make this
interim file themselves.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-hook.txt |  7 ++++++-
 builtin/hook.c             |  4 +++-
 hook.c                     |  8 +++++++-
 hook.h                     |  3 +++
 t/t1800-hook.sh            | 18 ++++++++++++++++++
 5 files changed, 37 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 097fb9de63b..fa68c1f3912 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -8,7 +8,7 @@ git-hook - run git hooks
 SYNOPSIS
 --------
 [verse]
-'git hook' run [--ignore-missing] <hook-name> [-- <hook-args>]
+'git hook' run [--to-stdin=<path>] [--ignore-missing] <hook-name> [-- <hook-args>]
 
 DESCRIPTION
 -----------
@@ -32,6 +32,11 @@ what those are.
 OPTIONS
 -------
 
+--to-stdin::
+	For "run"; Specify a file which will be streamed into the
+	hook's stdin. The hook will receive the entire file from
+	beginning to EOF.
+
 --ignore-missing::
 	Ignore any missing hook by quietly returning zero. Used for
 	tools that want to do a blind one-shot run of a hook that may
diff --git a/builtin/hook.c b/builtin/hook.c
index 76d49e672f4..fae69068201 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -7,7 +7,7 @@
 #include "strvec.h"
 
 #define BUILTIN_HOOK_RUN_USAGE \
-	N_("git hook run [--ignore-missing] <hook-name> [-- <hook-args>]")
+	N_("git hook run [--ignore-missing] [--to-stdin=<path>] <hook-name> [-- <hook-args>]")
 
 static const char * const builtin_hook_usage[] = {
 	BUILTIN_HOOK_RUN_USAGE,
@@ -29,6 +29,8 @@ static int run(int argc, const char **argv, const char *prefix)
 	struct option run_options[] = {
 		OPT_BOOL(0, "ignore-missing", &ignore_missing,
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
+		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
+			   N_("file to read into hooks' stdin")),
 		OPT_END(),
 	};
 	int ret;
diff --git a/hook.c b/hook.c
index 8a923e52c1b..d156b0dc800 100644
--- a/hook.c
+++ b/hook.c
@@ -59,7 +59,13 @@ static int pick_next_hook(struct child_process *cp,
 	if (!run_me)
 		return 0;
 
-	cp->no_stdin = 1;
+	/* reopen the file for stdin; run_command closes it. */
+	if (hook_cb->options->path_to_stdin) {
+		cp->no_stdin = 0;
+		cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY);
+	} else {
+		cp->no_stdin = 1;
+	}
 	cp->env = hook_cb->options->env.v;
 	cp->stdout_to_stderr = 1;
 	cp->trace2_hook_name = hook_cb->hook_name;
diff --git a/hook.h b/hook.h
index 3edd937e198..253725e197b 100644
--- a/hook.h
+++ b/hook.h
@@ -23,6 +23,9 @@ struct run_hooks_opt
 
 	/* Path to initial working directory for subprocess */
 	const char *dir;
+
+	/* Path to file which should be piped to stdin for each hook */
+	const char *path_to_stdin;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
index feb95b8fc8d..6431b19e392 100755
--- a/t/t1800-hook.sh
+++ b/t/t1800-hook.sh
@@ -130,4 +130,22 @@ test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
 	test_cmp expect actual
 '
 
+test_expect_success 'stdin to hooks' '
+	write_script .git/hooks/test-hook <<-\EOF &&
+	echo BEGIN stdin
+	cat
+	echo END stdin
+	EOF
+
+	cat >expect <<-EOF &&
+	BEGIN stdin
+	hello
+	END stdin
+	EOF
+
+	echo hello >input &&
+	git hook run --to-stdin=input test-hook 2>actual &&
+	test_cmp expect actual
+'
+
 test_done
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 23/36] am: convert 'post-rewrite' hook to hook.h
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (21 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 22/36] hook: support passing stdin to hooks Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 24/36] run-command: add stdin callback for parallelization Ævar Arnfjörð Bjarmason
                                 ` (12 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/am.c | 20 ++++----------------
 1 file changed, 4 insertions(+), 16 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index e444b18b64a..9e3d4d9ab44 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -467,24 +467,12 @@ static int run_applypatch_msg_hook(struct am_state *state)
  */
 static int run_post_rewrite_hook(const struct am_state *state)
 {
-	struct child_process cp = CHILD_PROCESS_INIT;
-	const char *hook = find_hook("post-rewrite");
-	int ret;
-
-	if (!hook)
-		return 0;
-
-	strvec_push(&cp.args, hook);
-	strvec_push(&cp.args, "rebase");
-
-	cp.in = xopen(am_path(state, "rewritten"), O_RDONLY);
-	cp.stdout_to_stderr = 1;
-	cp.trace2_hook_name = "post-rewrite";
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 
-	ret = run_command(&cp);
+	strvec_push(&opt.args, "rebase");
+	opt.path_to_stdin = am_path(state, "rewritten");
 
-	close(cp.in);
-	return ret;
+	return run_hooks_oneshot("post-rewrite", &opt);
 }
 
 /**
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 24/36] run-command: add stdin callback for parallelization
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (22 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 23/36] am: convert 'post-rewrite' hook to hook.h Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-10-06 11:03                 ` ab/config-based-hooks-N status (was Re: [PATCH v5 24/36] run-command: add stdin callback for parallelization) Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 25/36] hook: provide stdin by string_list or callback Ævar Arnfjörð Bjarmason
                                 ` (11 subsequent siblings)
  35 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

If a user of the run_processes_parallel() API wants to pipe a large
amount of information to stdin of each parallel command, that
information could exceed the buffer of the pipe allocated for that
process's stdin.  Generally this is solved by repeatedly writing to
child_process.in between calls to start_command() and finish_command();
run_processes_parallel() did not provide users an opportunity to access
child_process at that time.

Because the data might be extremely large (for example, a list of all
refs received during a push from a client) simply taking a string_list
or strbuf is not as scalable as using a callback; the rest of the
run_processes_parallel() API also uses callbacks, so making this feature
match the rest of the API reduces mental load on the user.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/fetch.c             |  1 +
 builtin/submodule--helper.c |  2 +-
 hook.c                      |  1 +
 run-command.c               | 54 +++++++++++++++++++++++++++++++++++--
 run-command.h               | 17 +++++++++++-
 submodule.c                 |  1 +
 t/helper/test-run-command.c | 31 ++++++++++++++++++---
 t/t0061-run-command.sh      | 30 +++++++++++++++++++++
 8 files changed, 129 insertions(+), 8 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index e064687dbdc..b18ada08842 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1819,6 +1819,7 @@ static int fetch_multiple(struct string_list *list, int max_children)
 		result = run_processes_parallel_tr2(max_children,
 						    &fetch_next_remote,
 						    &fetch_failed_to_start,
+						    NULL,
 						    &fetch_finished,
 						    &state,
 						    "fetch", "parallel/fetch");
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index ef2776a9e45..0b73739c160 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2311,7 +2311,7 @@ static int update_submodules(struct submodule_update_clone *suc)
 	int i;
 
 	run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task,
-				   update_clone_start_failure,
+				   update_clone_start_failure, NULL,
 				   update_clone_task_finished, suc, "submodule",
 				   "parallel/update");
 
diff --git a/hook.c b/hook.c
index d156b0dc800..45983d08aed 100644
--- a/hook.c
+++ b/hook.c
@@ -146,6 +146,7 @@ int run_hooks(const char *hook_name, const char *hook_path,
 	run_processes_parallel_tr2(jobs,
 				   pick_next_hook,
 				   notify_start_failure,
+				   NULL,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
diff --git a/run-command.c b/run-command.c
index 482ee2d76c6..f1616858d18 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1492,6 +1492,7 @@ struct parallel_processes {
 
 	get_next_task_fn get_next_task;
 	start_failure_fn start_failure;
+	feed_pipe_fn feed_pipe;
 	task_finished_fn task_finished;
 
 	struct {
@@ -1519,6 +1520,13 @@ static int default_start_failure(struct strbuf *out,
 	return 0;
 }
 
+static int default_feed_pipe(struct strbuf *pipe,
+			     void *pp_cb,
+			     void *pp_task_cb)
+{
+	return 1;
+}
+
 static int default_task_finished(int result,
 				 struct strbuf *out,
 				 void *pp_cb,
@@ -1549,6 +1557,7 @@ static void pp_init(struct parallel_processes *pp,
 		    int n,
 		    get_next_task_fn get_next_task,
 		    start_failure_fn start_failure,
+		    feed_pipe_fn feed_pipe,
 		    task_finished_fn task_finished,
 		    void *data)
 {
@@ -1567,6 +1576,7 @@ static void pp_init(struct parallel_processes *pp,
 	pp->get_next_task = get_next_task;
 
 	pp->start_failure = start_failure ? start_failure : default_start_failure;
+	pp->feed_pipe = feed_pipe ? feed_pipe : default_feed_pipe;
 	pp->task_finished = task_finished ? task_finished : default_task_finished;
 
 	pp->nr_processes = 0;
@@ -1664,6 +1674,37 @@ static int pp_start_one(struct parallel_processes *pp)
 	return 0;
 }
 
+static void pp_buffer_stdin(struct parallel_processes *pp)
+{
+	int i;
+	struct strbuf sb = STRBUF_INIT;
+
+	/* Buffer stdin for each pipe. */
+	for (i = 0; i < pp->max_processes; i++) {
+		if (pp->children[i].state == GIT_CP_WORKING &&
+		    pp->children[i].process.in > 0) {
+			int done;
+			strbuf_reset(&sb);
+			done = pp->feed_pipe(&sb, pp->data,
+					      pp->children[i].data);
+			if (sb.len) {
+				if (write_in_full(pp->children[i].process.in,
+					      sb.buf, sb.len) < 0) {
+					if (errno != EPIPE)
+						die_errno("write");
+					done = 1;
+				}
+			}
+			if (done) {
+				close(pp->children[i].process.in);
+				pp->children[i].process.in = 0;
+			}
+		}
+	}
+
+	strbuf_release(&sb);
+}
+
 static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout)
 {
 	int i;
@@ -1728,6 +1769,7 @@ static int pp_collect_finished(struct parallel_processes *pp)
 		pp->nr_processes--;
 		pp->children[i].state = GIT_CP_FREE;
 		pp->pfd[i].fd = -1;
+		pp->children[i].process.in = 0;
 		child_process_init(&pp->children[i].process);
 
 		if (i != pp->output_owner) {
@@ -1761,6 +1803,7 @@ static int pp_collect_finished(struct parallel_processes *pp)
 int run_processes_parallel(int n,
 			   get_next_task_fn get_next_task,
 			   start_failure_fn start_failure,
+			   feed_pipe_fn feed_pipe,
 			   task_finished_fn task_finished,
 			   void *pp_cb)
 {
@@ -1769,7 +1812,9 @@ int run_processes_parallel(int n,
 	int spawn_cap = 4;
 	struct parallel_processes pp;
 
-	pp_init(&pp, n, get_next_task, start_failure, task_finished, pp_cb);
+	sigchain_push(SIGPIPE, SIG_IGN);
+
+	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, task_finished, pp_cb);
 	while (1) {
 		for (i = 0;
 		    i < spawn_cap && !pp.shutdown &&
@@ -1786,6 +1831,7 @@ int run_processes_parallel(int n,
 		}
 		if (!pp.nr_processes)
 			break;
+		pp_buffer_stdin(&pp);
 		pp_buffer_stderr(&pp, output_timeout);
 		pp_output(&pp);
 		code = pp_collect_finished(&pp);
@@ -1797,11 +1843,15 @@ int run_processes_parallel(int n,
 	}
 
 	pp_cleanup(&pp);
+
+	sigchain_pop(SIGPIPE);
+
 	return 0;
 }
 
 int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 			       start_failure_fn start_failure,
+			       feed_pipe_fn feed_pipe,
 			       task_finished_fn task_finished, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label)
 {
@@ -1811,7 +1861,7 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 				   ((n < 1) ? online_cpus() : n));
 
 	result = run_processes_parallel(n, get_next_task, start_failure,
-					task_finished, pp_cb);
+					feed_pipe, task_finished, pp_cb);
 
 	trace2_region_leave(tr2_category, tr2_label, NULL);
 
diff --git a/run-command.h b/run-command.h
index cfb6887e4ae..80d394664ae 100644
--- a/run-command.h
+++ b/run-command.h
@@ -422,6 +422,20 @@ typedef int (*start_failure_fn)(struct strbuf *out,
 				void *pp_cb,
 				void *pp_task_cb);
 
+/**
+ * This callback is called repeatedly on every child process who requests
+ * start_command() to create a pipe by setting child_process.in < 0.
+ *
+ * pp_cb is the callback cookie as passed into run_processes_parallel, and
+ * pp_task_cb is the callback cookie as passed into get_next_task_fn.
+ * The contents of 'send' will be read into the pipe and passed to the pipe.
+ *
+ * Return nonzero to close the pipe.
+ */
+typedef int (*feed_pipe_fn)(struct strbuf *pipe,
+			    void *pp_cb,
+			    void *pp_task_cb);
+
 /**
  * This callback is called on every child process that finished processing.
  *
@@ -456,10 +470,11 @@ typedef int (*task_finished_fn)(int result,
 int run_processes_parallel(int n,
 			   get_next_task_fn,
 			   start_failure_fn,
+			   feed_pipe_fn,
 			   task_finished_fn,
 			   void *pp_cb);
 int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn,
-			       task_finished_fn, void *pp_cb,
+			       feed_pipe_fn, task_finished_fn, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label);
 
 /**
diff --git a/submodule.c b/submodule.c
index 8e611fe1dbf..db1700a502d 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1632,6 +1632,7 @@ int fetch_populated_submodules(struct repository *r,
 	run_processes_parallel_tr2(max_parallel_jobs,
 				   get_next_submodule,
 				   fetch_start_failure,
+				   NULL,
 				   fetch_finish,
 				   &spf,
 				   "submodule", "parallel/fetch");
diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c
index 7ae03dc7123..9348184d303 100644
--- a/t/helper/test-run-command.c
+++ b/t/helper/test-run-command.c
@@ -32,8 +32,13 @@ static int parallel_next(struct child_process *cp,
 		return 0;
 
 	strvec_pushv(&cp->args, d->argv);
+	cp->in = d->in;
+	cp->no_stdin = d->no_stdin;
 	strbuf_addstr(err, "preloaded output of a child\n");
 	number_callbacks++;
+
+	*task_cb = xmalloc(sizeof(int));
+	*(int*)(*task_cb) = 2;
 	return 1;
 }
 
@@ -55,6 +60,17 @@ static int task_finished(int result,
 	return 1;
 }
 
+static int test_stdin(struct strbuf *pipe, void *cb, void *task_cb)
+{
+	int *lines_remaining = task_cb;
+
+	if (*lines_remaining)
+		strbuf_addf(pipe, "sample stdin %d\n", --(*lines_remaining));
+
+	return !(*lines_remaining);
+}
+
+
 struct testsuite {
 	struct string_list tests, failed;
 	int next;
@@ -185,7 +201,7 @@ static int testsuite(int argc, const char **argv)
 		suite.tests.nr, max_jobs);
 
 	ret = run_processes_parallel(max_jobs, next_test, test_failed,
-				     test_finished, &suite);
+				     test_stdin, test_finished, &suite);
 
 	if (suite.failed.nr > 0) {
 		ret = 1;
@@ -413,15 +429,22 @@ int cmd__run_command(int argc, const char **argv)
 
 	if (!strcmp(argv[1], "run-command-parallel"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, &proc));
+					    NULL, NULL, NULL, &proc));
 
 	if (!strcmp(argv[1], "run-command-abort"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, task_finished, &proc));
+					    NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-no-jobs"))
 		exit(run_processes_parallel(jobs, no_job,
-					    NULL, task_finished, &proc));
+					    NULL, NULL, task_finished, &proc));
+
+	if (!strcmp(argv[1], "run-command-stdin")) {
+		proc.in = -1;
+		proc.no_stdin = 0;
+		exit (run_processes_parallel(jobs, parallel_next, NULL,
+					     test_stdin, NULL, &proc));
+	}
 
 	fprintf(stderr, "check usage\n");
 	return 1;
diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh
index 7d599675e35..87759482ad1 100755
--- a/t/t0061-run-command.sh
+++ b/t/t0061-run-command.sh
@@ -143,6 +143,36 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai
 	test_cmp expect actual
 '
 
+cat >expect <<-EOF
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+preloaded output of a child
+listening for stdin:
+sample stdin 1
+sample stdin 0
+EOF
+
+test_expect_success 'run_command listens to stdin' '
+	write_script stdin-script <<-\EOF &&
+	echo "listening for stdin:"
+	while read line; do
+		echo "$line"
+	done
+	EOF
+	test-tool run-command run-command-stdin 2 ./stdin-script 2>actual &&
+	test_cmp expect actual
+'
+
 cat >expect <<-EOF
 preloaded output of a child
 asking for a quick stop
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 25/36] hook: provide stdin by string_list or callback
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (23 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 24/36] run-command: add stdin callback for parallelization Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 26/36] hook: convert 'post-rewrite' hook in sequencer.c to hook.h Ævar Arnfjörð Bjarmason
                                 ` (10 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

In cases where a hook requires only a small amount of information via
stdin, it should be simple for users to provide a string_list alone. But
in more complicated cases where the stdin is too large to hold in
memory, let's instead provide a callback the users can populate line
after line.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 hook.c | 33 ++++++++++++++++++++++++++++++++-
 hook.h | 29 +++++++++++++++++++++++++++++
 2 files changed, 61 insertions(+), 1 deletion(-)

diff --git a/hook.c b/hook.c
index 45983d08aed..7a6ef527443 100644
--- a/hook.c
+++ b/hook.c
@@ -48,6 +48,29 @@ void run_hooks_opt_clear(struct run_hooks_opt *o)
 	strvec_clear(&o->args);
 }
 
+int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb)
+{
+	int *item_idx;
+	struct hook *ctx = pp_task_cb;
+	struct hook_cb_data *hook_cb = pp_cb;
+	struct string_list *to_pipe = hook_cb->options->feed_pipe_ctx;
+
+	/* Bootstrap the state manager if necessary. */
+	if (!ctx->feed_pipe_cb_data) {
+		ctx->feed_pipe_cb_data = xmalloc(sizeof(unsigned int));
+		*(int*)ctx->feed_pipe_cb_data = 0;
+	}
+
+	item_idx = ctx->feed_pipe_cb_data;
+
+	if (*item_idx < to_pipe->nr) {
+		strbuf_addf(pipe, "%s\n", to_pipe->items[*item_idx].string);
+		(*item_idx)++;
+		return 0;
+	}
+	return 1;
+}
+
 static int pick_next_hook(struct child_process *cp,
 			  struct strbuf *out,
 			  void *pp_cb,
@@ -63,6 +86,10 @@ static int pick_next_hook(struct child_process *cp,
 	if (hook_cb->options->path_to_stdin) {
 		cp->no_stdin = 0;
 		cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY);
+	} else if (hook_cb->options->feed_pipe) {
+		/* ask for start_command() to make a pipe for us */
+		cp->in = -1;
+		cp->no_stdin = 0;
 	} else {
 		cp->no_stdin = 1;
 	}
@@ -146,7 +173,7 @@ int run_hooks(const char *hook_name, const char *hook_path,
 	run_processes_parallel_tr2(jobs,
 				   pick_next_hook,
 				   notify_start_failure,
-				   NULL,
+				   options->feed_pipe,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
@@ -154,6 +181,7 @@ int run_hooks(const char *hook_name, const char *hook_path,
 
 	if (options->absolute_path)
 		strbuf_release(&abs_path);
+	free(my_hook.feed_pipe_cb_data);
 
 	return cb_data.rc;
 }
@@ -167,6 +195,9 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 	if (!options)
 		options = &hook_opt_scratch;
 
+	if (options->path_to_stdin && options->feed_pipe)
+		BUG("choose only one method to populate stdin");
+
 	hook_path = find_hook(hook_name);
 	if (!hook_path) {
 		ret = 0;
diff --git a/hook.h b/hook.h
index 253725e197b..4e26b8d658d 100644
--- a/hook.h
+++ b/hook.h
@@ -1,10 +1,18 @@
 #ifndef HOOK_H
 #define HOOK_H
+#include "strbuf.h"
 #include "strvec.h"
+#include "run-command.h"
 
 struct hook {
 	/* The path to the hook */
 	const char *hook_path;
+
+	/*
+	 * Use this to keep state for your feed_pipe_fn if you are using
+	 * run_hooks_opt.feed_pipe. Otherwise, do not touch it.
+	 */
+	void *feed_pipe_cb_data;
 };
 
 struct run_hooks_opt
@@ -26,6 +34,19 @@ struct run_hooks_opt
 
 	/* Path to file which should be piped to stdin for each hook */
 	const char *path_to_stdin;
+
+	/*
+	 * Callback and state pointer to ask for more content to pipe to stdin.
+	 * Will be called repeatedly, for each hook. See
+	 * hook.c:pipe_from_stdin() for an example. Keep per-hook state in
+	 * hook.feed_pipe_cb_data (per process). Keep initialization context in
+	 * feed_pipe_ctx (shared by all processes).
+	 *
+	 * See 'pipe_from_string_list()' for info about how to specify a
+	 * string_list as the stdin input instead of writing your own handler.
+	 */
+	feed_pipe_fn feed_pipe;
+	void *feed_pipe_ctx;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
@@ -77,4 +98,12 @@ int run_hooks(const char *hookname, const char *hook_path,
  */
 int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
 
+/**
+ * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
+ * string_list and set 'run_hooks_opt.feed_pipe' to pipe_from_string_list().
+ * This will pipe each string in the list to stdin, separated by newlines.  (Do
+ * not inject your own newlines.)
+ */
+int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb);
+
 #endif
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 26/36] hook: convert 'post-rewrite' hook in sequencer.c to hook.h
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (24 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 25/36] hook: provide stdin by string_list or callback Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 27/36] transport: convert pre-push hook " Ævar Arnfjörð Bjarmason
                                 ` (9 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

By using 'hook.h' for 'post-rewrite', we simplify hook invocations by
not needing to put together our own 'struct child_process'.

The signal handling that's being removed by this commit now takes
place in run-command.h:run_processes_parallel(), so it is OK to remove
them here.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 sequencer.c | 79 ++++++++++++++++++++++-------------------------------
 1 file changed, 32 insertions(+), 47 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 8999edf8d7b..549b583b277 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -35,6 +35,7 @@
 #include "commit-reach.h"
 #include "rebase-interactive.h"
 #include "reset.h"
+#include "string-list.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -1147,33 +1148,27 @@ int update_head_with_reflog(const struct commit *old_head,
 static int run_rewrite_hook(const struct object_id *oldoid,
 			    const struct object_id *newoid)
 {
-	struct child_process proc = CHILD_PROCESS_INIT;
-	const char *argv[3];
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct strbuf tmp = STRBUF_INIT;
+	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int code;
-	struct strbuf sb = STRBUF_INIT;
 
-	argv[0] = find_hook("post-rewrite");
-	if (!argv[0])
-		return 0;
+	strvec_push(&opt.args, "amend");
 
-	argv[1] = "amend";
-	argv[2] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = "post-rewrite";
-
-	code = start_command(&proc);
-	if (code)
-		return code;
-	strbuf_addf(&sb, "%s %s\n", oid_to_hex(oldoid), oid_to_hex(newoid));
-	sigchain_push(SIGPIPE, SIG_IGN);
-	write_in_full(proc.in, sb.buf, sb.len);
-	close(proc.in);
-	strbuf_release(&sb);
-	sigchain_pop(SIGPIPE);
-	return finish_command(&proc);
+	strbuf_addf(&tmp,
+		    "%s %s",
+		    oid_to_hex(oldoid),
+		    oid_to_hex(newoid));
+	string_list_append(&to_stdin, tmp.buf);
+
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
+
+	code = run_hooks_oneshot("post-rewrite", &opt);
+
+	strbuf_release(&tmp);
+	string_list_clear(&to_stdin, 0);
+	return code;
 }
 
 void commit_post_rewrite(struct repository *r,
@@ -4526,30 +4521,20 @@ static int pick_commits(struct repository *r,
 		flush_rewritten_pending();
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
-			struct child_process child = CHILD_PROCESS_INIT;
-			const char *post_rewrite_hook =
-				find_hook("post-rewrite");
-
-			child.in = open(rebase_path_rewritten_list(), O_RDONLY);
-			child.git_cmd = 1;
-			strvec_push(&child.args, "notes");
-			strvec_push(&child.args, "copy");
-			strvec_push(&child.args, "--for-rewrite=rebase");
+			struct child_process notes_cp = CHILD_PROCESS_INIT;
+			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+
+			notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY);
+			notes_cp.git_cmd = 1;
+			strvec_push(&notes_cp.args, "notes");
+			strvec_push(&notes_cp.args, "copy");
+			strvec_push(&notes_cp.args, "--for-rewrite=rebase");
 			/* we don't care if this copying failed */
-			run_command(&child);
-
-			if (post_rewrite_hook) {
-				struct child_process hook = CHILD_PROCESS_INIT;
-
-				hook.in = open(rebase_path_rewritten_list(),
-					O_RDONLY);
-				hook.stdout_to_stderr = 1;
-				hook.trace2_hook_name = "post-rewrite";
-				strvec_push(&hook.args, post_rewrite_hook);
-				strvec_push(&hook.args, "rebase");
-				/* we don't care if this hook failed */
-				run_command(&hook);
-			}
+			run_command(&notes_cp);
+
+			hook_opt.path_to_stdin = rebase_path_rewritten_list();
+			strvec_push(&hook_opt.args, "rebase");
+			run_hooks_oneshot("post-rewrite", &hook_opt);
 		}
 		apply_autostash(rebase_path_autostash());
 
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 27/36] transport: convert pre-push hook to hook.h
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (25 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 26/36] hook: convert 'post-rewrite' hook in sequencer.c to hook.h Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 28/36] hook tests: test for exact "pre-push" hook input Ævar Arnfjörð Bjarmason
                                 ` (8 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Move the pre-push hook away from run-command.h to and over to the new
hook.h library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 transport.c | 56 ++++++++++++++---------------------------------------
 1 file changed, 14 insertions(+), 42 deletions(-)

diff --git a/transport.c b/transport.c
index 77e196f75f5..4ca8fc0391d 100644
--- a/transport.c
+++ b/transport.c
@@ -1203,63 +1203,35 @@ static void die_with_unpushed_submodules(struct string_list *needs_pushing)
 static int run_pre_push_hook(struct transport *transport,
 			     struct ref *remote_refs)
 {
-	int ret = 0, x;
+	int ret = 0;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	struct ref *r;
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct strbuf buf;
-	const char *argv[4];
-
-	if (!(argv[0] = find_hook("pre-push")))
-		return 0;
-
-	argv[1] = transport->remote->name;
-	argv[2] = transport->url;
-	argv[3] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.trace2_hook_name = "pre-push";
-
-	if (start_command(&proc)) {
-		finish_command(&proc);
-		return -1;
-	}
+	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
 
-	sigchain_push(SIGPIPE, SIG_IGN);
-
-	strbuf_init(&buf, 256);
+	strvec_push(&opt.args, transport->remote->name);
+	strvec_push(&opt.args, transport->url);
 
 	for (r = remote_refs; r; r = r->next) {
+		struct strbuf buf = STRBUF_INIT;
+
 		if (!r->peer_ref) continue;
 		if (r->status == REF_STATUS_REJECT_NONFASTFORWARD) continue;
 		if (r->status == REF_STATUS_REJECT_STALE) continue;
 		if (r->status == REF_STATUS_REJECT_REMOTE_UPDATED) continue;
 		if (r->status == REF_STATUS_UPTODATE) continue;
 
-		strbuf_reset(&buf);
-		strbuf_addf( &buf, "%s %s %s %s\n",
+		strbuf_addf(&buf, "%s %s %s %s",
 			 r->peer_ref->name, oid_to_hex(&r->new_oid),
 			 r->name, oid_to_hex(&r->old_oid));
-
-		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
-			/* We do not mind if a hook does not read all refs. */
-			if (errno != EPIPE)
-				ret = -1;
-			break;
-		}
+		string_list_append(&to_stdin, strbuf_detach(&buf, NULL));
 	}
 
-	strbuf_release(&buf);
-
-	x = close(proc.in);
-	if (!ret)
-		ret = x;
-
-	sigchain_pop(SIGPIPE);
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
 
-	x = finish_command(&proc);
-	if (!ret)
-		ret = x;
+	ret = run_hooks_oneshot("pre-push", &opt);
+	to_stdin.strdup_strings = 1;
+	string_list_clear(&to_stdin, 0);
 
 	return ret;
 }
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 28/36] hook tests: test for exact "pre-push" hook input
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (26 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 27/36] transport: convert pre-push hook " Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 29/36] hook tests: use a modern style for "pre-push" tests Ævar Arnfjörð Bjarmason
                                 ` (7 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Extend the tests added in ec55559f937 (push: Add support for pre-push
hooks, 2013-01-13) to exhaustively test for the exact input we're
expecting. This ensures that we e.g. don't miss a trailing newline.

Appending to a file called "actual" is the established convention in
this test for hooks, see the rest of the tests added in
ec55559f937 (push: Add support for pre-push hooks, 2013-01-13). Let's
follow that convention here.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5571-pre-push-hook.sh | 23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)

diff --git a/t/t5571-pre-push-hook.sh b/t/t5571-pre-push-hook.sh
index ad8d5804f7b..d2857a6fbc0 100755
--- a/t/t5571-pre-push-hook.sh
+++ b/t/t5571-pre-push-hook.sh
@@ -11,7 +11,7 @@ HOOKDIR="$(git rev-parse --git-dir)/hooks"
 HOOK="$HOOKDIR/pre-push"
 mkdir -p "$HOOKDIR"
 write_script "$HOOK" <<EOF
-cat >/dev/null
+cat >actual
 exit 0
 EOF
 
@@ -20,10 +20,16 @@ test_expect_success 'setup' '
 	git init --bare repo1 &&
 	git remote add parent1 repo1 &&
 	test_commit one &&
-	git push parent1 HEAD:foreign
+	cat >expect <<-EOF &&
+	HEAD $(git rev-parse HEAD) refs/heads/foreign $(test_oid zero)
+	EOF
+
+	test_when_finished "rm actual" &&
+	git push parent1 HEAD:foreign &&
+	test_cmp expect actual
 '
 write_script "$HOOK" <<EOF
-cat >/dev/null
+cat >actual
 exit 1
 EOF
 
@@ -32,11 +38,18 @@ export COMMIT1
 
 test_expect_success 'push with failing hook' '
 	test_commit two &&
-	test_must_fail git push parent1 HEAD
+	cat >expect <<-EOF &&
+	HEAD $(git rev-parse HEAD) refs/heads/main $(test_oid zero)
+	EOF
+
+	test_when_finished "rm actual" &&
+	test_must_fail git push parent1 HEAD &&
+	test_cmp expect actual
 '
 
 test_expect_success '--no-verify bypasses hook' '
-	git push --no-verify parent1 HEAD
+	git push --no-verify parent1 HEAD &&
+	test_path_is_missing actual
 '
 
 COMMIT2="$(git rev-parse HEAD)"
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 29/36] hook tests: use a modern style for "pre-push" tests
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (27 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 28/36] hook tests: test for exact "pre-push" hook input Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 30/36] reference-transaction: use hook.h to run hooks Ævar Arnfjörð Bjarmason
                                 ` (6 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Indent the here-docs and use "test_cmp" instead of "diff" in tests
added in ec55559f937 (push: Add support for pre-push hooks,
2013-01-13). Let's also use the more typical "expect" instead of
"expected" to be consistent with the rest of the test file.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5571-pre-push-hook.sh | 71 ++++++++++++++++++++--------------------
 1 file changed, 35 insertions(+), 36 deletions(-)

diff --git a/t/t5571-pre-push-hook.sh b/t/t5571-pre-push-hook.sh
index d2857a6fbc0..cc1a3e23668 100755
--- a/t/t5571-pre-push-hook.sh
+++ b/t/t5571-pre-push-hook.sh
@@ -61,15 +61,15 @@ echo "$2" >>actual
 cat >>actual
 EOF
 
-cat >expected <<EOF
-parent1
-repo1
-refs/heads/main $COMMIT2 refs/heads/foreign $COMMIT1
-EOF
-
 test_expect_success 'push with hook' '
+	cat >expect <<-EOF &&
+	parent1
+	repo1
+	refs/heads/main $COMMIT2 refs/heads/foreign $COMMIT1
+	EOF
+
 	git push parent1 main:foreign &&
-	diff expected actual
+	test_cmp expect actual
 '
 
 test_expect_success 'add a branch' '
@@ -80,49 +80,48 @@ test_expect_success 'add a branch' '
 COMMIT3="$(git rev-parse HEAD)"
 export COMMIT3
 
-cat >expected <<EOF
-parent1
-repo1
-refs/heads/other $COMMIT3 refs/heads/foreign $COMMIT2
-EOF
-
 test_expect_success 'push to default' '
+	cat >expect <<-EOF &&
+	parent1
+	repo1
+	refs/heads/other $COMMIT3 refs/heads/foreign $COMMIT2
+	EOF
 	git push &&
-	diff expected actual
+	test_cmp expect actual
 '
 
-cat >expected <<EOF
-parent1
-repo1
-refs/tags/one $COMMIT1 refs/tags/tag1 $ZERO_OID
-HEAD~ $COMMIT2 refs/heads/prev $ZERO_OID
-EOF
-
 test_expect_success 'push non-branches' '
+	cat >expect <<-EOF &&
+	parent1
+	repo1
+	refs/tags/one $COMMIT1 refs/tags/tag1 $ZERO_OID
+	HEAD~ $COMMIT2 refs/heads/prev $ZERO_OID
+	EOF
+
 	git push parent1 one:tag1 HEAD~:refs/heads/prev &&
-	diff expected actual
+	test_cmp expect actual
 '
 
-cat >expected <<EOF
-parent1
-repo1
-(delete) $ZERO_OID refs/heads/prev $COMMIT2
-EOF
-
 test_expect_success 'push delete' '
+	cat >expect <<-EOF &&
+	parent1
+	repo1
+	(delete) $ZERO_OID refs/heads/prev $COMMIT2
+	EOF
+
 	git push parent1 :prev &&
-	diff expected actual
+	test_cmp expect actual
 '
 
-cat >expected <<EOF
-repo1
-repo1
-HEAD $COMMIT3 refs/heads/other $ZERO_OID
-EOF
-
 test_expect_success 'push to URL' '
+	cat >expect <<-EOF &&
+	repo1
+	repo1
+	HEAD $COMMIT3 refs/heads/other $ZERO_OID
+	EOF
+
 	git push repo1 HEAD &&
-	diff expected actual
+	test_cmp expect actual
 '
 
 test_expect_success 'set up many-ref tests' '
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 30/36] reference-transaction: use hook.h to run hooks
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (28 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 29/36] hook tests: use a modern style for "pre-push" tests Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 31/36] run-command: allow capturing of collated output Ævar Arnfjörð Bjarmason
                                 ` (5 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 refs.c | 40 +++++++++++++---------------------------
 1 file changed, 13 insertions(+), 27 deletions(-)

diff --git a/refs.c b/refs.c
index 6211692eaae..73d4a939267 100644
--- a/refs.c
+++ b/refs.c
@@ -2062,47 +2062,33 @@ int ref_update_reject_duplicates(struct string_list *refnames,
 static int run_transaction_hook(struct ref_transaction *transaction,
 				const char *state)
 {
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct strbuf buf = STRBUF_INIT;
-	const char *hook;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
 	int ret = 0, i;
 
-	hook = find_hook("reference-transaction");
-	if (!hook)
-		return ret;
-
-	strvec_pushl(&proc.args, hook, state, NULL);
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = "reference-transaction";
-
-	ret = start_command(&proc);
-	if (ret)
+	if (!hook_exists("reference-transaction"))
 		return ret;
 
-	sigchain_push(SIGPIPE, SIG_IGN);
+	strvec_push(&opt.args, state);
 
 	for (i = 0; i < transaction->nr; i++) {
 		struct ref_update *update = transaction->updates[i];
+		struct strbuf buf = STRBUF_INIT;
 
-		strbuf_reset(&buf);
-		strbuf_addf(&buf, "%s %s %s\n",
+		strbuf_addf(&buf, "%s %s %s",
 			    oid_to_hex(&update->old_oid),
 			    oid_to_hex(&update->new_oid),
 			    update->refname);
-
-		if (write_in_full(proc.in, buf.buf, buf.len) < 0) {
-			if (errno != EPIPE)
-				ret = -1;
-			break;
-		}
+		string_list_append(&to_stdin, strbuf_detach(&buf, NULL));
 	}
 
-	close(proc.in);
-	sigchain_pop(SIGPIPE);
-	strbuf_release(&buf);
+	opt.feed_pipe = pipe_from_string_list;
+	opt.feed_pipe_ctx = &to_stdin;
+
+	ret = run_hooks_oneshot("reference-transaction", &opt);
+	to_stdin.strdup_strings = 1;
+	string_list_clear(&to_stdin, 0);
 
-	ret |= finish_command(&proc);
 	return ret;
 }
 
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 31/36] run-command: allow capturing of collated output
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (29 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 30/36] reference-transaction: use hook.h to run hooks Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 32/36] hooks: allow callers to capture output Ævar Arnfjörð Bjarmason
                                 ` (4 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Some callers, for example server-side hooks which wish to relay hook
output to clients across a transport, want to capture what would
normally print to stderr and do something else with it. Allow that via a
callback.

By calling the callback regardless of whether there's output available,
we allow clients to send e.g. a keepalive if necessary.

Because we expose a strbuf, not a fd or FILE*, there's no need to create
a temporary pipe or similar - we can just skip the print to stderr and
instead hand it to the caller.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/fetch.c             |  2 +-
 builtin/submodule--helper.c |  2 +-
 hook.c                      |  1 +
 run-command.c               | 33 +++++++++++++++++++++++++--------
 run-command.h               | 18 +++++++++++++++++-
 submodule.c                 |  2 +-
 t/helper/test-run-command.c | 25 ++++++++++++++++++++-----
 t/t0061-run-command.sh      |  7 +++++++
 8 files changed, 73 insertions(+), 17 deletions(-)

diff --git a/builtin/fetch.c b/builtin/fetch.c
index b18ada08842..27b63ee5200 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1819,7 +1819,7 @@ static int fetch_multiple(struct string_list *list, int max_children)
 		result = run_processes_parallel_tr2(max_children,
 						    &fetch_next_remote,
 						    &fetch_failed_to_start,
-						    NULL,
+						    NULL, NULL,
 						    &fetch_finished,
 						    &state,
 						    "fetch", "parallel/fetch");
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 0b73739c160..fbceab08b0d 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -2311,7 +2311,7 @@ static int update_submodules(struct submodule_update_clone *suc)
 	int i;
 
 	run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task,
-				   update_clone_start_failure, NULL,
+				   update_clone_start_failure, NULL, NULL,
 				   update_clone_task_finished, suc, "submodule",
 				   "parallel/update");
 
diff --git a/hook.c b/hook.c
index 7a6ef527443..169287ad152 100644
--- a/hook.c
+++ b/hook.c
@@ -174,6 +174,7 @@ int run_hooks(const char *hook_name, const char *hook_path,
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
+				   NULL,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
diff --git a/run-command.c b/run-command.c
index f1616858d18..aacc336f951 100644
--- a/run-command.c
+++ b/run-command.c
@@ -1493,6 +1493,7 @@ struct parallel_processes {
 	get_next_task_fn get_next_task;
 	start_failure_fn start_failure;
 	feed_pipe_fn feed_pipe;
+	consume_sideband_fn consume_sideband;
 	task_finished_fn task_finished;
 
 	struct {
@@ -1558,6 +1559,7 @@ static void pp_init(struct parallel_processes *pp,
 		    get_next_task_fn get_next_task,
 		    start_failure_fn start_failure,
 		    feed_pipe_fn feed_pipe,
+		    consume_sideband_fn consume_sideband,
 		    task_finished_fn task_finished,
 		    void *data)
 {
@@ -1578,6 +1580,7 @@ static void pp_init(struct parallel_processes *pp,
 	pp->start_failure = start_failure ? start_failure : default_start_failure;
 	pp->feed_pipe = feed_pipe ? feed_pipe : default_feed_pipe;
 	pp->task_finished = task_finished ? task_finished : default_task_finished;
+	pp->consume_sideband = consume_sideband;
 
 	pp->nr_processes = 0;
 	pp->output_owner = 0;
@@ -1614,7 +1617,10 @@ static void pp_cleanup(struct parallel_processes *pp)
 	 * When get_next_task added messages to the buffer in its last
 	 * iteration, the buffered output is non empty.
 	 */
-	strbuf_write(&pp->buffered_output, stderr);
+	if (pp->consume_sideband)
+		pp->consume_sideband(&pp->buffered_output, pp->data);
+	else
+		strbuf_write(&pp->buffered_output, stderr);
 	strbuf_release(&pp->buffered_output);
 
 	sigchain_pop_common();
@@ -1735,9 +1741,13 @@ static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout)
 static void pp_output(struct parallel_processes *pp)
 {
 	int i = pp->output_owner;
+
 	if (pp->children[i].state == GIT_CP_WORKING &&
 	    pp->children[i].err.len) {
-		strbuf_write(&pp->children[i].err, stderr);
+		if (pp->consume_sideband)
+			pp->consume_sideband(&pp->children[i].err, pp->data);
+		else
+			strbuf_write(&pp->children[i].err, stderr);
 		strbuf_reset(&pp->children[i].err);
 	}
 }
@@ -1776,11 +1786,15 @@ static int pp_collect_finished(struct parallel_processes *pp)
 			strbuf_addbuf(&pp->buffered_output, &pp->children[i].err);
 			strbuf_reset(&pp->children[i].err);
 		} else {
-			strbuf_write(&pp->children[i].err, stderr);
+			/* Output errors, then all other finished child processes */
+			if (pp->consume_sideband) {
+				pp->consume_sideband(&pp->children[i].err, pp->data);
+				pp->consume_sideband(&pp->buffered_output, pp->data);
+			} else {
+				strbuf_write(&pp->children[i].err, stderr);
+				strbuf_write(&pp->buffered_output, stderr);
+			}
 			strbuf_reset(&pp->children[i].err);
-
-			/* Output all other finished child processes */
-			strbuf_write(&pp->buffered_output, stderr);
 			strbuf_reset(&pp->buffered_output);
 
 			/*
@@ -1804,6 +1818,7 @@ int run_processes_parallel(int n,
 			   get_next_task_fn get_next_task,
 			   start_failure_fn start_failure,
 			   feed_pipe_fn feed_pipe,
+			   consume_sideband_fn consume_sideband,
 			   task_finished_fn task_finished,
 			   void *pp_cb)
 {
@@ -1814,7 +1829,7 @@ int run_processes_parallel(int n,
 
 	sigchain_push(SIGPIPE, SIG_IGN);
 
-	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, task_finished, pp_cb);
+	pp_init(&pp, n, get_next_task, start_failure, feed_pipe, consume_sideband, task_finished, pp_cb);
 	while (1) {
 		for (i = 0;
 		    i < spawn_cap && !pp.shutdown &&
@@ -1852,6 +1867,7 @@ int run_processes_parallel(int n,
 int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 			       start_failure_fn start_failure,
 			       feed_pipe_fn feed_pipe,
+			       consume_sideband_fn consume_sideband,
 			       task_finished_fn task_finished, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label)
 {
@@ -1861,7 +1877,8 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
 				   ((n < 1) ? online_cpus() : n));
 
 	result = run_processes_parallel(n, get_next_task, start_failure,
-					feed_pipe, task_finished, pp_cb);
+					feed_pipe, consume_sideband,
+					task_finished, pp_cb);
 
 	trace2_region_leave(tr2_category, tr2_label, NULL);
 
diff --git a/run-command.h b/run-command.h
index 80d394664ae..e321d23bbd2 100644
--- a/run-command.h
+++ b/run-command.h
@@ -436,6 +436,20 @@ typedef int (*feed_pipe_fn)(struct strbuf *pipe,
 			    void *pp_cb,
 			    void *pp_task_cb);
 
+/**
+ * If this callback is provided, instead of collating process output to stderr,
+ * they will be collated into a new pipe. consume_sideband_fn will be called
+ * repeatedly. When output is available on that pipe, it will be contained in
+ * 'output'. But it will be called with an empty 'output' too, to allow for
+ * keepalives or similar operations if necessary.
+ *
+ * pp_cb is the callback cookie as passed into run_processes_parallel.
+ *
+ * Since this callback is provided with the collated output, no task cookie is
+ * provided.
+ */
+typedef void (*consume_sideband_fn)(struct strbuf *output, void *pp_cb);
+
 /**
  * This callback is called on every child process that finished processing.
  *
@@ -471,10 +485,12 @@ int run_processes_parallel(int n,
 			   get_next_task_fn,
 			   start_failure_fn,
 			   feed_pipe_fn,
+			   consume_sideband_fn,
 			   task_finished_fn,
 			   void *pp_cb);
 int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn,
-			       feed_pipe_fn, task_finished_fn, void *pp_cb,
+			       feed_pipe_fn, consume_sideband_fn,
+			       task_finished_fn, void *pp_cb,
 			       const char *tr2_category, const char *tr2_label);
 
 /**
diff --git a/submodule.c b/submodule.c
index db1700a502d..32364d8bd56 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1632,7 +1632,7 @@ int fetch_populated_submodules(struct repository *r,
 	run_processes_parallel_tr2(max_parallel_jobs,
 				   get_next_submodule,
 				   fetch_start_failure,
-				   NULL,
+				   NULL, NULL,
 				   fetch_finish,
 				   &spf,
 				   "submodule", "parallel/fetch");
diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c
index 9348184d303..d53db6d11c4 100644
--- a/t/helper/test-run-command.c
+++ b/t/helper/test-run-command.c
@@ -51,6 +51,16 @@ static int no_job(struct child_process *cp,
 	return 0;
 }
 
+static void test_consume_sideband(struct strbuf *output, void *cb)
+{
+	FILE *sideband;
+
+	sideband = fopen("./sideband", "a");
+
+	strbuf_write(output, sideband);
+	fclose(sideband);
+}
+
 static int task_finished(int result,
 			 struct strbuf *err,
 			 void *pp_cb,
@@ -201,7 +211,7 @@ static int testsuite(int argc, const char **argv)
 		suite.tests.nr, max_jobs);
 
 	ret = run_processes_parallel(max_jobs, next_test, test_failed,
-				     test_stdin, test_finished, &suite);
+				     test_stdin, NULL, test_finished, &suite);
 
 	if (suite.failed.nr > 0) {
 		ret = 1;
@@ -429,23 +439,28 @@ int cmd__run_command(int argc, const char **argv)
 
 	if (!strcmp(argv[1], "run-command-parallel"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, NULL, &proc));
+					    NULL, NULL, NULL, NULL, &proc));
 
 	if (!strcmp(argv[1], "run-command-abort"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, task_finished, &proc));
+					    NULL, NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-no-jobs"))
 		exit(run_processes_parallel(jobs, no_job,
-					    NULL, NULL, task_finished, &proc));
+					    NULL, NULL, NULL, task_finished, &proc));
 
 	if (!strcmp(argv[1], "run-command-stdin")) {
 		proc.in = -1;
 		proc.no_stdin = 0;
 		exit (run_processes_parallel(jobs, parallel_next, NULL,
-					     test_stdin, NULL, &proc));
+					     test_stdin, NULL, NULL, &proc));
 	}
 
+	if (!strcmp(argv[1], "run-command-sideband"))
+		exit(run_processes_parallel(jobs, parallel_next, NULL, NULL,
+					    test_consume_sideband, NULL,
+					    &proc));
+
 	fprintf(stderr, "check usage\n");
 	return 1;
 }
diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh
index 87759482ad1..e99f6c7f445 100755
--- a/t/t0061-run-command.sh
+++ b/t/t0061-run-command.sh
@@ -143,6 +143,13 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai
 	test_cmp expect actual
 '
 
+test_expect_success 'run_command can divert output' '
+	test_when_finished rm sideband &&
+	test-tool run-command run-command-sideband 3 sh -c "printf \"%s\n%s\n\" Hello World" 2>actual &&
+	test_must_be_empty actual &&
+	test_cmp expect sideband
+'
+
 cat >expect <<-EOF
 preloaded output of a child
 listening for stdin:
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 32/36] hooks: allow callers to capture output
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (30 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 31/36] run-command: allow capturing of collated output Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 33/36] receive-pack: convert 'update' hook to hook.h Ævar Arnfjörð Bjarmason
                                 ` (3 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Some server-side hooks will require capturing output to send over
sideband instead of printing directly to stderr. Expose that capability.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 hook.c | 3 ++-
 hook.h | 8 ++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/hook.c b/hook.c
index 169287ad152..7f93fe40dba 100644
--- a/hook.c
+++ b/hook.c
@@ -174,12 +174,13 @@ int run_hooks(const char *hook_name, const char *hook_path,
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
-				   NULL,
+				   options->consume_sideband,
 				   notify_hook_finished,
 				   &cb_data,
 				   "hook",
 				   hook_name);
 
+
 	if (options->absolute_path)
 		strbuf_release(&abs_path);
 	free(my_hook.feed_pipe_cb_data);
diff --git a/hook.h b/hook.h
index 4e26b8d658d..a85e3e6981b 100644
--- a/hook.h
+++ b/hook.h
@@ -47,6 +47,14 @@ struct run_hooks_opt
 	 */
 	feed_pipe_fn feed_pipe;
 	void *feed_pipe_ctx;
+
+	/*
+	 * Populate this to capture output and prevent it from being printed to
+	 * stderr. This will be passed directly through to
+	 * run_command:run_parallel_processes(). See t/helper/test-run-command.c
+	 * for an example.
+	 */
+	consume_sideband_fn consume_sideband;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 33/36] receive-pack: convert 'update' hook to hook.h
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (31 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 32/36] hooks: allow callers to capture output Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 34/36] post-update: use hook.h library Ævar Arnfjörð Bjarmason
                                 ` (2 subsequent siblings)
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

This makes use of the new sideband API in hook.h added in the
preceding commit.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/receive-pack.c | 62 ++++++++++++++++++++++++++++--------------
 1 file changed, 41 insertions(+), 21 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index a7d03bbc7d3..31ce4ece4e7 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -937,33 +937,53 @@ static int run_receive_hook(struct command *commands,
 	return status;
 }
 
-static int run_update_hook(struct command *cmd)
+static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
 {
-	const char *argv[5];
-	struct child_process proc = CHILD_PROCESS_INIT;
-	int code;
+	int keepalive_active = 0;
 
-	argv[0] = find_hook("update");
-	if (!argv[0])
-		return 0;
+	if (keepalive_in_sec <= 0)
+		use_keepalive = KEEPALIVE_NEVER;
+	if (use_keepalive == KEEPALIVE_ALWAYS)
+		keepalive_active = 1;
 
-	argv[1] = cmd->ref_name;
-	argv[2] = oid_to_hex(&cmd->old_oid);
-	argv[3] = oid_to_hex(&cmd->new_oid);
-	argv[4] = NULL;
+	/* send a keepalive if there is no data to write */
+	if (keepalive_active && !output->len) {
+		static const char buf[] = "0005\1";
+		write_or_die(1, buf, sizeof(buf) - 1);
+		return;
+	}
 
-	proc.no_stdin = 1;
-	proc.stdout_to_stderr = 1;
-	proc.err = use_sideband ? -1 : 0;
-	proc.argv = argv;
-	proc.trace2_hook_name = "update";
+	if (use_keepalive == KEEPALIVE_AFTER_NUL && !keepalive_active) {
+		const char *first_null = memchr(output->buf, '\0', output->len);
+		if (first_null) {
+			/* The null bit is excluded. */
+			size_t before_null = first_null - output->buf;
+			size_t after_null = output->len - (before_null + 1);
+			keepalive_active = 1;
+			send_sideband(1, 2, output->buf, before_null, use_sideband);
+			send_sideband(1, 2, first_null + 1, after_null, use_sideband);
+
+			return;
+		}
+	}
+
+	send_sideband(1, 2, output->buf, output->len, use_sideband);
+}
+
+static int run_update_hook(struct command *cmd)
+{
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+
+	strvec_pushl(&opt.args,
+		     cmd->ref_name,
+		     oid_to_hex(&cmd->old_oid),
+		     oid_to_hex(&cmd->new_oid),
+		     NULL);
 
-	code = start_command(&proc);
-	if (code)
-		return code;
 	if (use_sideband)
-		copy_to_sideband(proc.err, -1, NULL);
-	return finish_command(&proc);
+		opt.consume_sideband = hook_output_to_sideband;
+
+	return run_hooks_oneshot("update", &opt);
 }
 
 static struct command *find_command_by_refname(struct command *list,
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 34/36] post-update: use hook.h library
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (32 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 33/36] receive-pack: convert 'update' hook to hook.h Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 35/36] receive-pack: convert receive hooks to hook.h Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 36/36] hooks: fix a TOCTOU in "did we run a hook?" heuristic Ævar Arnfjörð Bjarmason
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/receive-pack.c | 25 ++++++-------------------
 1 file changed, 6 insertions(+), 19 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 31ce4ece4e7..26e302aab85 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1650,33 +1650,20 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 static void run_update_post_hook(struct command *commands)
 {
 	struct command *cmd;
-	struct child_process proc = CHILD_PROCESS_INIT;
-	const char *hook;
-
-	hook = find_hook("post-update");
-	if (!hook)
-		return;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 
 	for (cmd = commands; cmd; cmd = cmd->next) {
 		if (cmd->error_string || cmd->did_not_exist)
 			continue;
-		if (!proc.args.nr)
-			strvec_push(&proc.args, hook);
-		strvec_push(&proc.args, cmd->ref_name);
+		strvec_push(&opt.args, cmd->ref_name);
 	}
-	if (!proc.args.nr)
+	if (!opt.args.nr)
 		return;
 
-	proc.no_stdin = 1;
-	proc.stdout_to_stderr = 1;
-	proc.err = use_sideband ? -1 : 0;
-	proc.trace2_hook_name = "post-update";
+	if (use_sideband)
+		opt.consume_sideband = hook_output_to_sideband;
 
-	if (!start_command(&proc)) {
-		if (use_sideband)
-			copy_to_sideband(proc.err, -1, NULL);
-		finish_command(&proc);
-	}
+	run_hooks_oneshot("post-update", &opt);
 }
 
 static void check_aliased_update_internal(struct command *cmd,
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 35/36] receive-pack: convert receive hooks to hook.h
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (33 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 34/36] post-update: use hook.h library Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  2021-09-02 13:11               ` [PATCH v5 36/36] hooks: fix a TOCTOU in "did we run a hook?" heuristic Ævar Arnfjörð Bjarmason
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/receive-pack.c | 190 ++++++++++++++++++-----------------------
 1 file changed, 83 insertions(+), 107 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 26e302aab85..c3984680d7f 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -747,7 +747,7 @@ static int check_cert_push_options(const struct string_list *push_options)
 	return retval;
 }
 
-static void prepare_push_cert_sha1(struct child_process *proc)
+static void prepare_push_cert_sha1(struct run_hooks_opt *opt)
 {
 	static int already_done;
 
@@ -771,110 +771,42 @@ static void prepare_push_cert_sha1(struct child_process *proc)
 		nonce_status = check_nonce(push_cert.buf, bogs);
 	}
 	if (!is_null_oid(&push_cert_oid)) {
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT=%s",
 			     oid_to_hex(&push_cert_oid));
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_SIGNER=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_SIGNER=%s",
 			     sigcheck.signer ? sigcheck.signer : "");
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_KEY=%s",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_KEY=%s",
 			     sigcheck.key ? sigcheck.key : "");
-		strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_STATUS=%c",
+		strvec_pushf(&opt->env, "GIT_PUSH_CERT_STATUS=%c",
 			     sigcheck.result);
 		if (push_cert_nonce) {
-			strvec_pushf(&proc->env_array,
+			strvec_pushf(&opt->env,
 				     "GIT_PUSH_CERT_NONCE=%s",
 				     push_cert_nonce);
-			strvec_pushf(&proc->env_array,
+			strvec_pushf(&opt->env,
 				     "GIT_PUSH_CERT_NONCE_STATUS=%s",
 				     nonce_status);
 			if (nonce_status == NONCE_SLOP)
-				strvec_pushf(&proc->env_array,
+				strvec_pushf(&opt->env,
 					     "GIT_PUSH_CERT_NONCE_SLOP=%ld",
 					     nonce_stamp_slop);
 		}
 	}
 }
 
+struct receive_hook_feed_context {
+	struct command *cmd;
+	int skip_broken;
+};
+
 struct receive_hook_feed_state {
 	struct command *cmd;
 	struct ref_push_report *report;
 	int skip_broken;
 	struct strbuf buf;
-	const struct string_list *push_options;
 };
 
-typedef int (*feed_fn)(void *, const char **, size_t *);
-static int run_and_feed_hook(const char *hook_name, feed_fn feed,
-			     struct receive_hook_feed_state *feed_state)
-{
-	struct child_process proc = CHILD_PROCESS_INIT;
-	struct async muxer;
-	const char *argv[2];
-	int code;
-
-	argv[0] = find_hook(hook_name);
-	if (!argv[0])
-		return 0;
-
-	argv[1] = NULL;
-
-	proc.argv = argv;
-	proc.in = -1;
-	proc.stdout_to_stderr = 1;
-	proc.trace2_hook_name = hook_name;
-
-	if (feed_state->push_options) {
-		int i;
-		for (i = 0; i < feed_state->push_options->nr; i++)
-			strvec_pushf(&proc.env_array,
-				     "GIT_PUSH_OPTION_%d=%s", i,
-				     feed_state->push_options->items[i].string);
-		strvec_pushf(&proc.env_array, "GIT_PUSH_OPTION_COUNT=%d",
-			     feed_state->push_options->nr);
-	} else
-		strvec_pushf(&proc.env_array, "GIT_PUSH_OPTION_COUNT");
-
-	if (tmp_objdir)
-		strvec_pushv(&proc.env_array, tmp_objdir_env(tmp_objdir));
-
-	if (use_sideband) {
-		memset(&muxer, 0, sizeof(muxer));
-		muxer.proc = copy_to_sideband;
-		muxer.in = -1;
-		code = start_async(&muxer);
-		if (code)
-			return code;
-		proc.err = muxer.in;
-	}
-
-	prepare_push_cert_sha1(&proc);
-
-	code = start_command(&proc);
-	if (code) {
-		if (use_sideband)
-			finish_async(&muxer);
-		return code;
-	}
-
-	sigchain_push(SIGPIPE, SIG_IGN);
-
-	while (1) {
-		const char *buf;
-		size_t n;
-		if (feed(feed_state, &buf, &n))
-			break;
-		if (write_in_full(proc.in, buf, n) < 0)
-			break;
-	}
-	close(proc.in);
-	if (use_sideband)
-		finish_async(&muxer);
-
-	sigchain_pop(SIGPIPE);
-
-	return finish_command(&proc);
-}
-
-static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
+static int feed_receive_hook(void *state_)
 {
 	struct receive_hook_feed_state *state = state_;
 	struct command *cmd = state->cmd;
@@ -883,9 +815,7 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
 	       state->skip_broken && (cmd->error_string || cmd->did_not_exist))
 		cmd = cmd->next;
 	if (!cmd)
-		return -1; /* EOF */
-	if (!bufp)
-		return 0; /* OK, can feed something. */
+		return 1; /* EOF - close the pipe*/
 	strbuf_reset(&state->buf);
 	if (!state->report)
 		state->report = cmd->report;
@@ -909,32 +839,36 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep)
 			    cmd->ref_name);
 		state->cmd = cmd->next;
 	}
-	if (bufp) {
-		*bufp = state->buf.buf;
-		*sizep = state->buf.len;
-	}
 	return 0;
 }
 
-static int run_receive_hook(struct command *commands,
-			    const char *hook_name,
-			    int skip_broken,
-			    const struct string_list *push_options)
+static int feed_receive_hook_cb(struct strbuf *pipe, void *pp_cb, void *pp_task_cb)
 {
-	struct receive_hook_feed_state state;
-	int status;
-
-	strbuf_init(&state.buf, 0);
-	state.cmd = commands;
-	state.skip_broken = skip_broken;
-	state.report = NULL;
-	if (feed_receive_hook(&state, NULL, NULL))
-		return 0;
-	state.cmd = commands;
-	state.push_options = push_options;
-	status = run_and_feed_hook(hook_name, feed_receive_hook, &state);
-	strbuf_release(&state.buf);
-	return status;
+	struct hook *hook = pp_task_cb;
+	struct receive_hook_feed_state *feed_state = hook->feed_pipe_cb_data;
+	int rc;
+
+	/* first-time setup */
+	if (!feed_state) {
+		struct hook_cb_data *hook_cb = pp_cb;
+		struct run_hooks_opt *opt = hook_cb->options;
+		struct receive_hook_feed_context *ctx = opt->feed_pipe_ctx;
+		if (!ctx)
+			BUG("run_hooks_opt.feed_pipe_ctx required for receive hook");
+
+		feed_state = xmalloc(sizeof(struct receive_hook_feed_state));
+		strbuf_init(&feed_state->buf, 0);
+		feed_state->cmd = ctx->cmd;
+		feed_state->skip_broken = ctx->skip_broken;
+		feed_state->report = NULL;
+
+		hook->feed_pipe_cb_data = feed_state;
+	}
+
+	rc = feed_receive_hook(feed_state);
+	if (!rc)
+		strbuf_addbuf(pipe, &feed_state->buf);
+	return rc;
 }
 
 static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
@@ -970,6 +904,48 @@ static void hook_output_to_sideband(struct strbuf *output, void *cb_data)
 	send_sideband(1, 2, output->buf, output->len, use_sideband);
 }
 
+static int run_receive_hook(struct command *commands,
+			    const char *hook_name,
+			    int skip_broken,
+			    const struct string_list *push_options)
+{
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct receive_hook_feed_context ctx;
+	struct command *iter = commands;
+
+	/* if there are no valid commands, don't invoke the hook at all. */
+	while (iter && skip_broken && (iter->error_string || iter->did_not_exist))
+		iter = iter->next;
+	if (!iter)
+		return 0;
+
+	if (push_options) {
+		int i;
+		for (i = 0; i < push_options->nr; i++)
+			strvec_pushf(&opt.env, "GIT_PUSH_OPTION_%d=%s", i,
+				     push_options->items[i].string);
+		strvec_pushf(&opt.env, "GIT_PUSH_OPTION_COUNT=%d", push_options->nr);
+	} else
+		strvec_push(&opt.env, "GIT_PUSH_OPTION_COUNT");
+
+	if (tmp_objdir)
+		strvec_pushv(&opt.env, tmp_objdir_env(tmp_objdir));
+
+	prepare_push_cert_sha1(&opt);
+
+	/* set up sideband printer */
+	if (use_sideband)
+		opt.consume_sideband = hook_output_to_sideband;
+
+	/* set up stdin callback */
+	ctx.cmd = commands;
+	ctx.skip_broken = skip_broken;
+	opt.feed_pipe = feed_receive_hook_cb;
+	opt.feed_pipe_ctx = &ctx;
+
+	return run_hooks_oneshot(hook_name, &opt);
+}
+
 static int run_update_hook(struct command *cmd)
 {
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v5 36/36] hooks: fix a TOCTOU in "did we run a hook?" heuristic
  2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
                                 ` (34 preceding siblings ...)
  2021-09-02 13:11               ` [PATCH v5 35/36] receive-pack: convert receive hooks to hook.h Ævar Arnfjörð Bjarmason
@ 2021-09-02 13:11               ` Ævar Arnfjörð Bjarmason
  35 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-02 13:11 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

Fix a Time-of-check to time-of-use (TOCTOU) race in code added in
680ee550d72 (commit: skip discarding the index if there is no
pre-commit hook, 2017-08-14).

We can fix the race passing around information about whether or not we
ran the hook in question, instead of running hook_exists() after the
fact to check if the hook in question exists. This problem has been
noted on-list when 680ee550d72 was discussed[1], but had not been
fixed.

In addition to fixing this for the pre-commit hook as suggested there
I'm also fixing this for the pre-merge-commit hook. See
6098817fd7f (git-merge: honor pre-merge-commit hook, 2019-08-07) for
the introduction of its previous behavior.

Let's also change this for the push-to-checkout hook. Now instead of
checking if the hook exists and either doing a push to checkout or a
push to deploy we'll always attempt a push to checkout. If the hook
doesn't exist we'll fall back on push to deploy. The same behavior as
before, without the TOCTOU race. See 0855331941b (receive-pack:
support push-to-checkout hook, 2014-12-01) for the introduction of the
previous behavior.

This leaves uses of hook_exists() in two places that matter. The
"reference-transaction" check in refs.c, see 67541597670 (refs:
implement reference transaction hook, 2020-06-19), and the
prepare-commit-msg hook, see 66618a50f9c (sequencer: run
'prepare-commit-msg' hook, 2018-01-24).

In both of those cases we're saving ourselves CPU time by not
preparing data for the hook that we'll then do nothing with if we
don't have the hook. So using this "invoked_hook" pattern doesn't make
sense in those cases.

More importantly, in those cases the worst we'll do is miss that we
"should" run the hook because a new hook appeared, whereas in the
pre-commit and pre-merge-commit cases we'll skip an important
discard_cache() on the bases of our faulty guess.

I do think none of these races really matter in practice. It would be
some one-off issue as a hook was added or removed. I did think it was
stupid that we didn't pass a "did this run?" flag instead of doing
this guessing at a distance though, so now we're not guessing anymore.

1. https://lore.kernel.org/git/20170810191613.kpmhzg4seyxy3cpq@sigill.intra.peff.net/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/commit.c       | 18 +++++++++++-------
 builtin/merge.c        | 16 ++++++++++------
 builtin/receive-pack.c |  8 +++++---
 commit.c               |  1 +
 commit.h               |  3 ++-
 hook.c                 |  4 ++++
 hook.h                 | 10 ++++++++++
 sequencer.c            |  4 ++--
 8 files changed, 45 insertions(+), 19 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index f6ca9d04c78..986146e3205 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -725,11 +725,13 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	int clean_message_contents = (cleanup_mode != COMMIT_MSG_CLEANUP_NONE);
 	int old_display_comment_prefix;
 	int merge_contains_scissors = 0;
+	int invoked_hook = 0;
 
 	/* This checks and barfs if author is badly specified */
 	determine_author_info(author_ident);
 
-	if (!no_verify && run_commit_hook(use_editor, index_file, "pre-commit", NULL))
+	if (!no_verify && run_commit_hook(use_editor, index_file, &invoked_hook,
+					  "pre-commit", NULL))
 		return 0;
 
 	if (squash_message) {
@@ -1052,10 +1054,10 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (!no_verify && hook_exists("pre-commit")) {
+	if (!no_verify && invoked_hook) {
 		/*
-		 * Re-read the index as pre-commit hook could have updated it,
-		 * and write it out as a tree.  We must do this before we invoke
+		 * Re-read the index as the pre-commit-commit hook was invoked
+		 * and could have updated it. We must do this before we invoke
 		 * the editor and after we invoke run_status above.
 		 */
 		discard_cache();
@@ -1067,7 +1069,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (run_commit_hook(use_editor, index_file, "prepare-commit-msg",
+	if (run_commit_hook(use_editor, index_file, NULL, "prepare-commit-msg",
 			    git_path_commit_editmsg(), hook_arg1, hook_arg2, NULL))
 		return 0;
 
@@ -1084,7 +1086,8 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	}
 
 	if (!no_verify &&
-	    run_commit_hook(use_editor, index_file, "commit-msg", git_path_commit_editmsg(), NULL)) {
+	    run_commit_hook(use_editor, index_file, NULL, "commit-msg",
+			    git_path_commit_editmsg(), NULL)) {
 		return 0;
 	}
 
@@ -1843,7 +1846,8 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 
 	repo_rerere(the_repository, 0);
 	run_auto_maintenance(quiet);
-	run_commit_hook(use_editor, get_index_file(), "post-commit", NULL);
+	run_commit_hook(use_editor, get_index_file(), NULL, "post-commit",
+			NULL);
 	if (amend && !no_post_rewrite) {
 		commit_post_rewrite(the_repository, current_head, &oid);
 	}
diff --git a/builtin/merge.c b/builtin/merge.c
index ca9b3ba4827..f215f264cc8 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -844,15 +844,18 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 {
 	struct strbuf msg = STRBUF_INIT;
 	const char *index_file = get_index_file();
+	int invoked_hook = 0;
 
-	if (!no_verify && run_commit_hook(0 < option_edit, index_file, "pre-merge-commit", NULL))
+	if (!no_verify && run_commit_hook(0 < option_edit, index_file,
+					  &invoked_hook, "pre-merge-commit",
+					  NULL))
 		abort_commit(remoteheads, NULL);
 	/*
-	 * Re-read the index as pre-merge-commit hook could have updated it,
-	 * and write it out as a tree.  We must do this before we invoke
+	 * Re-read the index as the pre-merge-commit hook was invoked
+	 * and could have updated it. We must do this before we invoke
 	 * the editor and after we invoke run_status above.
 	 */
-	if (hook_exists("pre-merge-commit"))
+	if (invoked_hook)
 		discard_cache();
 	read_cache_from(index_file);
 	strbuf_addbuf(&msg, &merge_msg);
@@ -875,7 +878,8 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 		append_signoff(&msg, ignore_non_trailer(msg.buf, msg.len), 0);
 	write_merge_heads(remoteheads);
 	write_file_buf(git_path_merge_msg(the_repository), msg.buf, msg.len);
-	if (run_commit_hook(0 < option_edit, get_index_file(), "prepare-commit-msg",
+	if (run_commit_hook(0 < option_edit, get_index_file(), NULL,
+			    "prepare-commit-msg",
 			    git_path_merge_msg(the_repository), "merge", NULL))
 		abort_commit(remoteheads, NULL);
 	if (0 < option_edit) {
@@ -884,7 +888,7 @@ static void prepare_to_commit(struct commit_list *remoteheads)
 	}
 
 	if (!no_verify && run_commit_hook(0 < option_edit, get_index_file(),
-					  "commit-msg",
+					  NULL, "commit-msg",
 					  git_path_merge_msg(the_repository), NULL))
 		abort_commit(remoteheads, NULL);
 
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index c3984680d7f..ebec6f3bb10 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1428,10 +1428,12 @@ static const char *push_to_deploy(unsigned char *sha1,
 static const char *push_to_checkout_hook = "push-to-checkout";
 
 static const char *push_to_checkout(unsigned char *hash,
+				    int *invoked_hook,
 				    struct strvec *env,
 				    const char *work_tree)
 {
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	opt.invoked_hook = invoked_hook;
 
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
 	strvec_pushv(&opt.env, env->v);
@@ -1446,6 +1448,7 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 {
 	const char *retval, *work_tree, *git_dir = NULL;
 	struct strvec env = STRVEC_INIT;
+	int invoked_hook = 0;
 
 	if (worktree && worktree->path)
 		work_tree = worktree->path;
@@ -1463,10 +1466,9 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w
 
 	strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir));
 
-	if (!hook_exists(push_to_checkout_hook))
+	retval = push_to_checkout(sha1, &invoked_hook, &env, work_tree);
+	if (!invoked_hook)
 		retval = push_to_deploy(sha1, &env, work_tree);
-	else
-		retval = push_to_checkout(sha1, &env, work_tree);
 
 	strvec_clear(&env);
 	return retval;
diff --git a/commit.c b/commit.c
index 63d7943a86d..842e47beae2 100644
--- a/commit.c
+++ b/commit.c
@@ -1697,6 +1697,7 @@ size_t ignore_non_trailer(const char *buf, size_t len)
 }
 
 int run_commit_hook(int editor_is_used, const char *index_file,
+		    int *invoked_hook,
 		    const char *name, ...)
 {
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
diff --git a/commit.h b/commit.h
index df42eb434f3..b5a542993c6 100644
--- a/commit.h
+++ b/commit.h
@@ -363,7 +363,8 @@ int compare_commits_by_commit_date(const void *a_, const void *b_, void *unused)
 int compare_commits_by_gen_then_commit_date(const void *a_, const void *b_, void *unused);
 
 LAST_ARG_MUST_BE_NULL
-int run_commit_hook(int editor_is_used, const char *index_file, const char *name, ...);
+int run_commit_hook(int editor_is_used, const char *index_file,
+		    int *invoked_hook, const char *name, ...);
 
 /* Sign a commit or tag buffer, storing the result in a header. */
 int sign_with_header(struct strbuf *buf, const char *keyid);
diff --git a/hook.c b/hook.c
index 7f93fe40dba..d045379ade8 100644
--- a/hook.c
+++ b/hook.c
@@ -144,6 +144,9 @@ static int notify_hook_finished(int result,
 
 	hook_cb->rc |= result;
 
+	if (hook_cb->invoked_hook)
+		*hook_cb->invoked_hook = 1;
+
 	return 0;
 }
 
@@ -158,6 +161,7 @@ int run_hooks(const char *hook_name, const char *hook_path,
 		.rc = 0,
 		.hook_name = hook_name,
 		.options = options,
+		.invoked_hook = options->invoked_hook,
 	};
 	int jobs = 1;
 
diff --git a/hook.h b/hook.h
index a85e3e6981b..f6dac75f1cc 100644
--- a/hook.h
+++ b/hook.h
@@ -55,6 +55,15 @@ struct run_hooks_opt
 	 * for an example.
 	 */
 	consume_sideband_fn consume_sideband;
+
+	/*
+	 * A pointer which if provided will be set to 1 or 0 depending
+	 * on if a hook was invoked (i.e. existed), regardless of
+	 * whether or not that was successful. Used for avoiding
+	 * TOCTOU races in code that would otherwise call hook_exist()
+	 * after a "maybe hook run" to see if a hook was invoked.
+	 */
+	int *invoked_hook;
 };
 
 #define RUN_HOOKS_OPT_INIT { \
@@ -68,6 +77,7 @@ struct hook_cb_data {
 	const char *hook_name;
 	struct hook *run_me;
 	struct run_hooks_opt *options;
+	int *invoked_hook;
 };
 
 /**
diff --git a/sequencer.c b/sequencer.c
index 549b583b277..db8044ab47d 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1203,7 +1203,7 @@ static int run_prepare_commit_msg_hook(struct repository *r,
 	} else {
 		arg1 = "message";
 	}
-	if (run_commit_hook(0, r->index_file, "prepare-commit-msg", name,
+	if (run_commit_hook(0, r->index_file, NULL, "prepare-commit-msg", name,
 			    arg1, arg2, NULL))
 		ret = error(_("'prepare-commit-msg' hook failed"));
 
@@ -1533,7 +1533,7 @@ static int try_to_commit(struct repository *r,
 		goto out;
 	}
 
-	run_commit_hook(0, r->index_file, "post-commit", NULL);
+	run_commit_hook(0, r->index_file, NULL, "post-commit", NULL);
 	if (flags & AMEND_MSG)
 		commit_post_rewrite(r, current_head, oid);
 
-- 
2.33.0.816.g1ba32acadee


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* Re: [PATCH v3 0/6] config-based hooks restarted
  2021-08-24 20:29                 ` [PATCH v3 0/6] config-based hooks restarted Ævar Arnfjörð Bjarmason
@ 2021-09-02 22:01                   ` Emily Shaffer
  0 siblings, 0 replies; 479+ messages in thread
From: Emily Shaffer @ 2021-09-02 22:01 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Taylor Blau, Felipe Contreras,
	Eric Sunshine, brian m. carlson, Josh Steadmon, Jonathan Tan,
	Derrick Stolee

On Tue, Aug 24, 2021 at 10:29:33PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> 
> On Wed, Aug 18 2021, Emily Shaffer wrote:
> 0)
> 
> I think this is in much much better shape vis-as-vis the simplified
> config schema that's now being proposed re our discussion starting
> around https://lore.kernel.org/git/87bl6ttmlv.fsf@evledraar.gmail.com/
> 
> I.e. the main complexity of the "skip" mechanism is gone, and also the
> conflation of hook names with hook commands (the "rm -rf /" as a
> <name-or-cmd> discussed in the above).
> 
> So before going any further, I'll just say that I wouldn't object much
> to this design going in as-is. What I'm about to mention here below is
> much closer to bikeshedding in my mind than "this is really to complex
> to go in-tree", which was my opinion of the config schema before.
> 
> 1)
> 
> On the current config design: First, before going into any detail on
> that, I think whatever anyone's opinion is on that that the
> design-focused patches as they stand could really use more a more
> extended discussion of the design.
> 
> I.e. talk about the previously considered config schema, why it evolved
> into its current form. The trade-offs involved, and why the patch
> proposed to implement the schema it's implementing over another earlier
> or alternate design.
> 
> I.e. https://lore.kernel.org/git/20210819033450.3382652-6-emilyshaffer@google.com
> is two very short paragraphs. We won't be able to summarize all our
> month-long discussion on the config design in one commit message, but I
> think at least discussing it somewhat / linking to relevant on-list
> discussions would make future source spelunking easier.

Hum. I don't think that it's necessary to summarize the whole discussion
in the commit message, but I think it's worth it to describe the
rationale ("we like this design because it makes x good practice easy
and y bug hard") and link out to the discussion for parties who are
interested in reading more.

> 
> 2)
> 
> So that out of the way, a comment on the current config design, which
> should be read in the context of what I noted in #0. I.e. I'm *much*
> happier with this version.
> 
> That being said I'm still not convinced that the simple 1=1 mapping of
> "hook.<name>.command" and its "value" should be followed by the 1=many
> mapping of "hook.<name>.event" and its "value".
> 
> I.e. we're making the trade-off of saving the user from typing out or
> specifying:
>     
>     [hook "my-pre-commit"]
>     command = ~/hooks/pre-commit-or-push
>     event = pre-commit
>     [hook "my-pre-push"]
>     command = ~/hooks/pre-commit-or-push
>     event = pre-push
> 
> And instead being able to do:
> 
>     [hook "my-pre-commit-or-push"]
>     command = ~/hooks/pre-commit-or-push
>     event = pre-commit
>     event = pre-push
> 
> So for the very common case, saving two config lines. "Two" because as
> we discussed[1] as there's currently no GIT_HOOK_TYPE env var. So this
> form will work pretty much only for that case.
> 
> I.e. unlike with .git/hook/<name> the hook run via config can't
> determine what <hook-type> it's being run at, so as it stands this is
> only useful for those hooks listed in githooks(5) where someone would
> want to do the exact same thing for one or more <hook-name> names. You
> can't use it as a general routing mechanism for any hook type as it
> stands.
> 
> I *think* that's only these two, perhaps "update" and "pre-receive",
> with the hook seeing if it consume stdin/has arguments to disambiguate
> the two.
> 
> But even with a GIT_HOOK_TYPE passed the trade-off, as discussed in [1],
> and downthread in [2], is that by having it 1=many we're closing the
> door on any future hook.<name>.<whatever>. I.e. config that would change
> the behavior of that hook, but you'd want to change it in another way
> for at least one of the N event types.

I'm not really sure that's the case, to be honest. Even with the config
scheme as is in this iteration, you can still define it the way you're
describing with no problem, and in fact it makes it easier for users to
apply some special config to all-but-one invocation.

Let's say for example the "git-secrets" hook, which we do have defined
for at least 3 different events at Google today, and a hypothetical
"parallelize me" config; for the sake of argument let's presume that
this is the far future and we've added a GIT_HOOK_TYPE envvar:

[hook "git-secrets"]
  command = /bin/git-secrets
  event = pre-commit
  event = pre-merge-commit
  parallelizable = true

[hook "git-secrets-mutexed"]
  command = /bin/git-secrets
  event = prepare-commit-msg
  event = commit-msg
  parallelizable = false

If we don't allow "hook.myhookname.event" to be multiply configurable,
then the user gets this really tedious task of defining every single
"hook.git-secrets-$hookevent.parallelizable" config.

> 
> Well, "closing the door" as in if you'd want that you'd have to split up
> the section from the "my-pre-commit-or-push" example above to the
> "my-pre-commit" and "my-pre-push" example.
> 
> But again, on the "is the complexity worth it" we're then having to
> explain to users that they can do it one way if the want no config other
> than hook.<name>.{command,event*}, but another if they have another key
> in that namespace.

I am not so worried about "this will be hard to explain, so we should
not do it" - I think we can make the documentation useful with enough
effort and expertise. (And yes, I'm feeling optimistic because I have an
actual technical writer taking a look at the manpage right now.)

> 
> You've said that you wanted to add something like a GIT_HOOK_TYPE
> environment variable. Fair enough, and I guess we could add it in a
> re-roll of this series. I'm mainly commenting on the end-state of *this*
> series in particular. I.e. I think it leaves the user & implementation
> with a config schema that still seems to be needlessly complex for the
> very limited benefits that complexity brings us in what you're able to
> do with it now.

I think that limiting ourselves in the way you're describing will make
it more difficult to bring additional benefits later. It is certainly
possible and valid to write your configuration the way you are
describing, without changing this schema.

> 
> But some of that goes back to the comments I had on 5/6[3], i.e. I'm
> willing to be convinced, but I think that the current commit message &
> added docs aren't really selling the idea of why it's worth it.

Ok. I think your point is not "the schema is still wrong" as much as it
is "the documentation could be much better", and I agree.

> 
> 3) 
> 
> As an extension to my comments on 5/6[3], I think this whole notion of
> "git hook run <name>" as invoked by a user of git is just more confusing
> the more I think about it.
> 
> I.e. 5/6[4] is apparently seeking to implement a way to just make that
> facility a general way for users to run some command on their system to
> do whatever, instead of say using /usr/bin/parallel or a shell alias.
> 
> But then we also use that command as our own dispatch mechanism for our
> own known hooks, except mostly not, since we mostly use the C API
> ourselves directly.
> 
> It's particularly confusing that if you say run "git hook run
> pre-auto-gc" as a user to test your hook you'll have that hook run in
> the same way that git-gc(1) would run it. So someone developing a hook
> might think they can use "git hook run" for testing it.
> 
> But if you do the same with say "git hook run pre-receive" or anything
> else that feeds arguments or stdin (e.g. "update", or "pre-receive"),
> you'll have your hook happily being run by git, but in a way that's not
> at all how such a hook will be run when it's run by git "for real".
> 
> So I wonder if we shouldn't just have the thing die() if you try to run
> any hook that's in githooks(5) itself, except for sendemail-validate and
> the p4 hooks, since we need to run those ourselves.

I think this again falls to a documentation issue. I would love to have
a tighter loop when developing a hook that takes weird mid-merge
arguments or whatever; making it easier to test the hook in isolation
with known inputs sounds like a good thing to me.

Personally, "I ran 'git hook run pre-receive' without being in the
middle of a receive operation and it didn't behave like it was in the
middle of a receive operation!" doesn't sound all that surprising to me.

> 
> Or have those use an internal-only "git hook--helper", and start out
> with "git hook" just supporting "git hook list", and then later on have
> "git hook run" (or perhaps "git hook run-configured"?) be an entry point
> for this facility of running some arbitrary script that's not a "real"
> hook.
> 
> I don't know, maybe I'm the only one that finds this confusing...
> 
> 1. https://lore.kernel.org/git/87bl6ttmlv.fsf@evledraar.gmail.com/
> 2. https://lore.kernel.org/git/877dh0n1b3.fsf@evledraar.gmail.com/
> 3. https://lore.kernel.org/git/87lf4qeh86.fsf@evledraar.gmail.com/
> 4. https://lore.kernel.org/git/20210819033450.3382652-6-emilyshaffer@google.com/

Besides "make the docs more obvious", I don't think there is anything in
this mail that I want to act on. I am very happy with the config schema
as it is, as well as the behavior of 'git hook run'.


As I mentioned here and somewhere else in the review, I am in the
process of getting feedback on the manpage from a tech writer this week,
so do not expect a reroll from me until at least next week. I saw your
reroll today and I'll try and look at it (or at least the interdiff)
tomorrow or Monday.

 - Emily

^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH v4 0/5] config-based hooks restarted
  2021-08-19  3:34               ` [PATCH v3 " Emily Shaffer
                                   ` (6 preceding siblings ...)
  2021-08-24 20:29                 ` [PATCH v3 0/6] config-based hooks restarted Ævar Arnfjörð Bjarmason
@ 2021-09-09 12:41                 ` Ævar Arnfjörð Bjarmason
  2021-09-09 12:41                   ` [PATCH v4 1/5] hook: run a list of hooks instead Ævar Arnfjörð Bjarmason
                                     ` (4 more replies)
  7 siblings, 5 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-09 12:41 UTC (permalink / raw)
  To: git
  Cc: Emily Shaffer, Junio C Hamano, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

This is a re-roll of Emily's es/config-based-hooks topic that goes on
top of the v5 of my ab/config-based-hooks-base, and which doesn't have
errors under SANITIZE=leak.

Emily: Sorry, I've got no intention to steal this one too, hopefully
you can get around to your own re-roll.

But per my
https://lore.kernel.org/git/87sfyfgtfh.fsf@evledraar.gmail.com/ the
lack of this re-roll is is currently blocking the pick-up of my
re-rolled v5 of ab/config-based-hooks-base at [1], as well as causing
a failure in "seen" when combined with my ab/sanitize-leak-ci (and
hn/reftable, but that's another issue...).

Junio: So hopefully you can pick up the v5[1] of the base topic now &
this preliminary v4 of es/config-based-hooks.

The range-diff below is against Emily's 30ffe98601e, i.e. her v3 at
[2].

This version is based on Emily's preliminary cf1f8e34a34
(nasamuffin/config-based-hooks-restart), which appeared to be her
August 31 rebasing addressing of many outstanding points in the v3
series.

My own changes on top of that were twofold: Adjustments to changes in
the base topic (many done to make the overall diff/changes here
smaller), and memory leak fixes to get this to pass under
SANITIZE=leak, there's various other minor but not-notable changes
here and there, see the range-diff.

1. https://lore.kernel.org/git/cover-v5-00.36-00000000000-20210902T125110Z-avarab@gmail.com/
2. https://lore.kernel.org/git/20210819033450.3382652-1-emilyshaffer@google.com/

Emily Shaffer (5):
  hook: run a list of hooks instead
  hook: allow parallel hook execution
  hook: introduce "git hook list"
  hook: include hooks from the config
  hook: allow out-of-repo 'git hook' invocations

 Documentation/config.txt      |   2 +
 Documentation/config/hook.txt |  22 +++
 Documentation/git-hook.txt    | 157 ++++++++++++++++++-
 builtin/am.c                  |   4 +-
 builtin/checkout.c            |   2 +-
 builtin/clone.c               |   2 +-
 builtin/hook.c                |  71 ++++++++-
 builtin/merge.c               |   2 +-
 builtin/rebase.c              |   2 +-
 builtin/receive-pack.c        |   9 +-
 builtin/worktree.c            |   2 +-
 commit.c                      |   2 +-
 git.c                         |   2 +-
 hook.c                        | 277 +++++++++++++++++++++++++++++-----
 hook.h                        |  45 +++++-
 read-cache.c                  |   2 +-
 refs.c                        |   2 +-
 reset.c                       |   3 +-
 sequencer.c                   |   4 +-
 t/t1800-hook.sh               | 194 +++++++++++++++++++++++-
 transport.c                   |   2 +-
 21 files changed, 734 insertions(+), 74 deletions(-)
 create mode 100644 Documentation/config/hook.txt

Range-diff against v3:
1:  6d6400329cd ! 1:  2f0cac14965 hook: run a list of hooks instead
    @@ Commit message
         executable for a single hook event.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## builtin/hook.c ##
     @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
    @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
      	const char *hook_name;
     -	const char *hook_path;
     +	struct list_head *hooks;
    -+
      	struct option run_options[] = {
      		OPT_BOOL(0, "ignore-missing", &ignore_missing,
      			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
     @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
    - 	git_config(git_default_config, NULL);
    - 
    + 	 * run_hooks() instead...
    + 	 */
      	hook_name = argv[0];
     -	if (ignore_missing)
    ++	hooks = list_hooks(hook_name);
    ++	if (list_empty(hooks)) {
    ++		clear_hook_list(hooks);
    ++
    + 		/* ... act like a plain run_hooks() under --ignore-missing */
     -		return run_hooks_oneshot(hook_name, &opt);
     -	hook_path = find_hook(hook_name);
     -	if (!hook_path) {
    -+	hooks = list_hooks(hook_name);
    -+	if (list_empty(hooks)) {
    -+		/* ... act like run_hooks_oneshot() under --ignore-missing */
     +		if (ignore_missing)
     +			return 0;
      		error("cannot find a hook named %s", hook_name);
    @@ builtin/hook.c: static int run(int argc, const char **argv, const char *prefix)
     
      ## hook.c ##
     @@
    - #include "hook-list.h"
    + #include "run-command.h"
      #include "config.h"
      
     +static void free_hook(struct hook *ptr)
     +{
    -+	if (ptr)
    -+		free(ptr->feed_pipe_cb_data);
    ++	if (!ptr)
    ++		return;
    ++
    ++	free(ptr->feed_pipe_cb_data);
     +	free(ptr);
     +}
     +
    @@ hook.c
     +	struct list_head *pos, *tmp;
     +	list_for_each_safe(pos, tmp, head)
     +		remove_hook(pos);
    ++	free(head);
     +}
     +
    - static int known_hook(const char *name)
    + const char *find_hook(const char *name)
      {
    - 	const char **p;
    + 	static struct strbuf path = STRBUF_INIT;
     @@ hook.c: const char *find_hook(const char *name)
      
      int hook_exists(const char *name)
      {
     -	return !!find_hook(name);
    -+	return !list_empty(list_hooks(name));
    ++	struct list_head *hooks;
    ++	int exists;
    ++
    ++	hooks = list_hooks(name);
    ++	exists = !list_empty(hooks);
    ++	clear_hook_list(hooks);
    ++
    ++	return exists;
     +}
     +
     +struct list_head *list_hooks(const char *hookname)
    @@ hook.c: static int notify_hook_finished(int result,
      }
      
     -int run_hooks(const char *hook_name, const char *hook_path,
    --	      struct run_hooks_opt *options)
     +int run_hooks(const char *hook_name, struct list_head *hooks,
    -+		    struct run_hooks_opt *options)
    + 	      struct run_hooks_opt *options)
      {
     -	struct strbuf abs_path = STRBUF_INIT;
     -	struct hook my_hook = {
    @@ hook.c: int run_hooks(const char *hook_name, const char *hook_path,
     -		my_hook.hook_path = abs_path.buf;
     -	}
     -	cb_data.run_me = &my_hook;
    -+
     +	cb_data.head = hooks;
     +	cb_data.run_me = list_first_entry(hooks, struct hook, list);
      
    @@ hook.c: int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *optio
      
     -	ret = run_hooks(hook_name, hook_path, options);
     +	ret = run_hooks(hook_name, hooks, options);
    -+
    + 
      cleanup:
      	run_hooks_opt_clear(options);
    -+	clear_hook_list(hooks);
    - 	return ret;
    - }
     
      ## hook.h ##
     @@
    @@ hook.h
      #include "run-command.h"
     +#include "list.h"
      
    - /*
    -  * Returns the path to the hook file, or NULL if the hook is missing
    -@@ hook.h: const char *find_hook(const char *name);
    - int hook_exists(const char *hookname);
    - 
      struct hook {
     +	struct list_head list;
      	/* The path to the hook */
      	const char *hook_path;
      
    -@@ hook.h: struct hook {
    - 	void *feed_pipe_cb_data;
    - };
    - 
    -+/*
    -+ * Provides a linked list of 'struct hook' detailing commands which should run
    -+ * in response to the 'hookname' event, in execution order.
    -+ */
    -+struct list_head *list_hooks(const char *hookname);
    -+
    - struct run_hooks_opt
    - {
    - 	/* Environment vars to be set for each hook */
     @@ hook.h: struct hook_cb_data {
      	/* rc reflects the cumulative failure state */
      	int rc;
    @@ hook.h: struct hook_cb_data {
      	struct hook *run_me;
      	struct run_hooks_opt *options;
      	int *invoked_hook;
    +@@ hook.h: struct hook_cb_data {
    + const char *find_hook(const char *name);
    + 
    + /**
    +- * A boolean version of find_hook()
    ++ * Provides a linked list of 'struct hook' detailing commands which should run
    ++ * in response to the 'hookname' event, in execution order.
    ++ */
    ++struct list_head *list_hooks(const char *hookname);
    ++
    ++/**
    ++ * A boolean version of list_hooks()
    +  */
    + int hook_exists(const char *hookname);
    + 
     @@ hook.h: void run_hooks_opt_clear(struct run_hooks_opt *o);
    + 
    + /**
    +  * Takes an already resolved hook found via find_hook() and runs
    +- * it. Does not call run_hooks_opt_clear() for you.
    ++ * it. Does not call run_hooks_opt_clear() for you, but does call
    ++ * clear_hook_list().
       *
       * See run_hooks_oneshot() for the simpler one-shot API.
       */
     -int run_hooks(const char *hookname, const char *hook_path,
    --	      struct run_hooks_opt *options);
     +int run_hooks(const char *hookname, struct list_head *hooks,
    -+		    struct run_hooks_opt *options);
    - 
    - /**
    -  * Calls find_hook() on your "hook_name" and runs the hooks (if any)
    -@@ hook.h: int run_hooks(const char *hookname, const char *hook_path,
    -  */
    - int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options);
    + 	      struct run_hooks_opt *options);
      
    -+/* Empties the list at 'head', calling 'free_hook()' on each entry */
    ++/**
    ++ * Empties the list at 'head', calling 'free_hook()' on each
    ++ * entry. Called implicitly by run_hooks() (and run_hooks_oneshot()).
    ++ */
     +void clear_hook_list(struct list_head *head);
     +
    - #endif
    + /**
    +  * Calls find_hook() on your "hook_name" and runs the hooks (if any)
    +  * with run_hooks().
2:  dfb995ce4d4 ! 2:  b03e70c805e hook: allow parallel hook execution
    @@ Commit message
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
         Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
    + ## Documentation/config.txt ##
    +@@ Documentation/config.txt: include::config/guitool.txt[]
    + 
    + include::config/help.txt[]
    + 
    ++include::config/hook.txt[]
    ++
    + include::config/http.txt[]
    + 
    + include::config/i18n.txt[]
    +
      ## Documentation/config/hook.txt (new) ##
     @@
     +hook.jobs::
    @@ Documentation/git-hook.txt: OPTIONS
     +--jobs::
     +	Only valid for `run`.
     ++
    -+Specify how many hooks to run simultaneously. If this flag is not specified, use
    -+the value of the `hook.jobs` config. If the config is not specified, use the
    -+number of CPUs on the current system. Some hooks may be ineligible for
    -+parallelization: for example, 'commit-msg' intends hooks modify the commit
    -+message body and cannot be parallelized.
    ++Specify how many hooks to run simultaneously. If this flag is not specified,
    ++uses the value of the `hook.jobs` config, see linkgit:git-config[1]. If the
    ++config is not specified, uses the number of CPUs on the current system. Some
    ++hooks may be ineligible for parallelization: for example, 'commit-msg' intends
    ++hooks modify the commit message body and cannot be parallelized.
     +
     +CONFIGURATION
     +-------------
    @@ hook.c: static int notify_hook_finished(int result,
     +}
     +
      int run_hooks(const char *hook_name, struct list_head *hooks,
    - 		    struct run_hooks_opt *options)
    + 	      struct run_hooks_opt *options)
      {
     @@ hook.c: int run_hooks(const char *hook_name, struct list_head *hooks,
      		.options = options,
    @@ hook.c: int run_hooks(const char *hook_name, struct list_head *hooks,
      
      	if (!options)
      		BUG("a struct run_hooks_opt must be provided to run_hooks");
    - 
    --
    +@@ hook.c: int run_hooks(const char *hook_name, struct list_head *hooks,
      	cb_data.head = hooks;
      	cb_data.run_me = list_first_entry(hooks, struct hook, list);
      
    @@ hook.h: struct run_hooks_opt
     +	 */
     +	int jobs;
     +
    - 	/* Resolve and run the "absolute_path(hook)" instead of
    + 	/*
    + 	 * Resolve and run the "absolute_path(hook)" instead of
      	 * "hook". Used for "git worktree" hooks
    - 	 */
     @@ hook.h: struct run_hooks_opt
      	int *invoked_hook;
      };
      
     -#define RUN_HOOKS_OPT_INIT { \
    --	.env = STRVEC_INIT, \
    --	.args = STRVEC_INIT, \
    --}
    --
    - /*
    -  * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the
    -  * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'.
    -@@ hook.h: struct hook_cb_data {
    - 	int *invoked_hook;
    - };
    - 
     +#define RUN_HOOKS_OPT_INIT_SERIAL { \
     +	.jobs = 1, \
     +	.env = STRVEC_INIT, \
    @@ hook.h: struct hook_cb_data {
     +
     +#define RUN_HOOKS_OPT_INIT_PARALLEL { \
     +	.jobs = 0, \
    -+	.env = STRVEC_INIT, \
    -+	.args = STRVEC_INIT, \
    -+}
    -+
    - void run_hooks_opt_clear(struct run_hooks_opt *o);
    - 
    - /**
    + 	.env = STRVEC_INIT, \
    + 	.args = STRVEC_INIT, \
    + }
     
      ## read-cache.c ##
     @@ read-cache.c: static int do_write_locked_index(struct index_state *istate, struct lock_file *l
3:  c8a04306e90 ! 3:  3e647b8dba7 hook: introduce "git hook list"
    @@ Commit message
         hooks were configured and whether or not they will run.
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
    +    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## Documentation/git-hook.txt ##
     @@ Documentation/git-hook.txt: SYNOPSIS
    @@ builtin/hook.c: static const char * const builtin_hook_run_usage[] = {
     +
     +static int list(int argc, const char **argv, const char *prefix)
     +{
    -+	struct list_head *head, *pos;
    ++	struct list_head *hooks;
    ++	struct list_head *pos;
     +	const char *hookname = NULL;
    -+	struct strbuf hookdir_annotation = STRBUF_INIT;
    -+
     +	struct option list_options[] = {
     +		OPT_END(),
     +	};
    ++	int ret = 0;
     +
     +	argc = parse_options(argc, argv, prefix, list_options,
     +			     builtin_hook_list_usage, 0);
     +
    -+	if (argc < 1)
    ++	/*
    ++	 * The only unnamed argument provided should be the hook-name; if we add
    ++	 * arguments later they probably should be caught by parse_options.
    ++	 */
    ++	if (argc != 1)
     +		usage_msg_opt(_("You must specify a hook event name to list."),
     +			      builtin_hook_list_usage, list_options);
     +
     +	hookname = argv[0];
     +
    -+	head = hook_list(hookname);
    ++	hooks = list_hooks(hookname);
     +
    -+	if (list_empty(head))
    -+		return 1;
    ++	if (list_empty(hooks)) {
    ++		ret = 1;
    ++		goto cleanup;
    ++	}
     +
    -+	list_for_each(pos, head) {
    ++	list_for_each(pos, hooks) {
     +		struct hook *item = list_entry(pos, struct hook, list);
     +		item = list_entry(pos, struct hook, list);
     +		if (item)
     +			printf("%s\n", item->hook_path);
     +	}
     +
    -+	clear_hook_list(head);
    -+	strbuf_release(&hookdir_annotation);
    ++cleanup:
    ++	clear_hook_list(hooks);
     +
    -+	return 0;
    ++	return ret;
     +}
      static int run(int argc, const char **argv, const char *prefix)
      {
    @@ builtin/hook.c: int cmd_hook(int argc, const char **argv, const char *prefix)
      		return run(argc, argv, prefix);
      
     
    - ## hook.c ##
    -@@ hook.c: struct list_head *list_hooks(const char *hookname)
    - {
    - 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
    + ## t/t1800-hook.sh ##
    +@@ t/t1800-hook.sh: test_expect_success 'git hook usage' '
    + 	test_expect_code 129 git hook run &&
    + 	test_expect_code 129 git hook run -h &&
    + 	test_expect_code 129 git hook run --unknown 2>err &&
    ++	test_expect_code 129 git hook list &&
    ++	test_expect_code 129 git hook list -h &&
    + 	grep "unknown option" err
    + '
      
    -+
    - 	INIT_LIST_HEAD(hook_head);
    +@@ t/t1800-hook.sh: test_expect_success 'git hook run -- pass arguments' '
    + 	test_cmp expect actual
    + '
      
    - 	if (!hookname)
    -@@ hook.c: struct list_head *list_hooks(const char *hookname)
    - 
    - 	if (have_git_dir()) {
    - 		const char *hook_path = find_hook(hookname);
    --
    --		/* Add the hook from the hookdir */
    - 		if (hook_path) {
    - 			struct hook *to_add = xmalloc(sizeof(*to_add));
    - 			to_add->hook_path = hook_path;
    ++test_expect_success 'git hook list: does-not-exist hook' '
    ++	test_expect_code 1 git hook list does-not-exist
    ++'
    ++
    ++test_expect_success 'git hook list: existing hook' '
    ++	cat >expect <<-\EOF &&
    ++	.git/hooks/test-hook
    ++	EOF
    ++	git hook list test-hook >actual &&
    ++	test_cmp expect actual
    ++'
    ++
    + test_expect_success 'git hook run -- out-of-repo runs excluded' '
    + 	write_script .git/hooks/test-hook <<-EOF &&
    + 	echo Test hook
4:  af14116d0fa < -:  ----------- hook: allow running non-native hooks
5:  2bbb179962e ! 4:  d0f5b30fb27 hook: include hooks from the config
    @@ Metadata
      ## Commit message ##
         hook: include hooks from the config
     
    -    Teach the hook.[hc] library to parse configs to populare the list of
    +    Teach the hook.[hc] library to parse configs to populate the list of
         hooks to run for a given event.
     
         Multiple commands can be specified for a given hook by providing
         multiple "hook.<friendly-name>.command = <path-to-hook>" and
    -    "hook.<friendly-name>.event = <hook-event>" lines. Hooks will be run in
    -    config order of the "hook.<name>.event" lines.
    +    "hook.<friendly-name>.event = <hook-event>" lines. Hooks will be started
    +    in config order of the "hook.<name>.event" lines (but may run in
    +    parallel).
     
         For example:
     
    -      $ git config --list | grep ^hook
    +      $ git config --get-regexp "^hook\."
           hook.bar.command=~/bar.sh
           hook.bar.event=pre-commit
     
    -      $ git hook run
    -      # Runs ~/bar.sh
    -      # Runs .git/hooks/pre-commit
    +      # Will run ~/bar.sh, then .git/hooks/pre-commit
    +      $ git hook run pre-commit
     
         Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
     
    @@ Documentation/config/hook.txt
      	hook execution. If unspecified, defaults to the number of processors on
     
      ## Documentation/git-hook.txt ##
    -@@ Documentation/git-hook.txt: Git is unlikely to use for a native hook later on. For example, Git is much less
    - likely to create a `mytool-validate-commit` hook than it is to create a
    - `validate-commit` hook.
    +@@ Documentation/git-hook.txt: This command is an interface to git hooks (see linkgit:githooks[5]).
    + Currently it only provides a convenience wrapper for running hooks for
    + use by git itself. In the future it might gain other functionality.
      
    -+This command parses the default configuration files for pairs of configs like
    ++It's possible to use this command to refer to hooks which are not native to Git,
    ++for example if a wrapper around Git wishes to expose hooks into its own
    ++operation in a way which is already familiar to Git users. However, wrappers
    ++invoking such hooks should be careful to name their hook events something which
    ++Git is unlikely to use for a native hook later on. For example, Git is much less
    ++likely to create a `mytool-validate-commit` hook than it is to create a
    ++`validate-commit` hook.
    ++
    ++This command parses the default configuration files for sets of configs like
     +so:
     +
     +  [hook "linter"]
     +    event = pre-commit
    -+    command = ~/bin/linter --c
    ++    command = ~/bin/linter --cpp20
     +
    -+In this example, `[hook "linter"]` represents one script - `~/bin/linter --c` -
    -+which can be shared by many repos, and even by many hook events, if appropriate.
    ++In this example, `[hook "linter"]` represents one script - `~/bin/linter
    ++--cpp20` - which can be shared by many repos, and even by many hook events, if
    ++appropriate.
    ++
    ++To add an unrelated hook which runs on a different event, for example a
    ++spell-checker for your commit messages, you would write a configuration like so:
    ++
    ++  [hook "linter"]
    ++    event = pre-commit
    ++    command = ~/bin/linter --cpp20
    ++  [hook "spellcheck"]
    ++    event = commit-msg
    ++    command = ~/bin/spellchecker
    ++
    ++With this config, when you run 'git commit', first `~/bin/linter --cpp20` will
    ++have a chance to check your files to be committed (during the `pre-commit` hook
    ++event`), and then `~/bin/spellchecker` will have a chance to check your commit
    ++message (during the `commit-msg` hook event).
     +
     +Commands are run in the order Git encounters their associated
     +`hook.<name>.event` configs during the configuration parse (see
    @@ Documentation/git-hook.txt: Git is unlikely to use for a native hook later on. F
     +  [hook "linter"]
     +    event = pre-commit
     +    event = pre-push
    -+    command = ~/bin/linter --c
    ++    command = ~/bin/linter --cpp20
     +
    -+With this config, `~/bin/linter --c` would be run by Git before a commit is
    ++With this config, `~/bin/linter --cpp20` would be run by Git before a commit is
     +generated (during `pre-commit`) as well as before a push is performed (during
     +`pre-push`).
     +
    @@ Documentation/git-hook.txt: Git is unlikely to use for a native hook later on. F
     +
     +  [hook "linter"]
     +    event = pre-commit
    -+    command = ~/bin/linter --c
    ++    command = ~/bin/linter --cpp20
     +  [hook "no-leaks"]
     +    event = pre-commit
     +    command = ~/bin/leak-detector
     +
     +With this config, before a commit is generated (during `pre-commit`), Git would
    -+first start `~/bin/linter --c` and second start `~/bin/leak-detector`. It would
    -+evaluate the output of each when deciding whether to proceed with the commit.
    ++first start `~/bin/linter --cpp20` and second start `~/bin/leak-detector`. It
    ++would evaluate the output of each when deciding whether to proceed with the
    ++commit.
     +
     +For a full list of hook events which you can set your `hook.<name>.event` to,
     +and how hooks are invoked during those events, see linkgit:githooks[5].
     +
    ++Git will ignore any `hook.<name>.event` that specifies an event it doesn't
    ++recognize. This is intended so that tools which wrap Git can use the hook
    ++infrastructure to run their own hooks; see <<WRAPPERS>> for more guidance.
    ++
     +In general, when instructions suggest adding a script to
    -+`.git/hooks/<hook-event>`, you can specify it in the config instead by running
    -+`git config --add hook.<some-name>.command <path-to-script> && git config --add
    -+hook.<some-name>.event <hook-event>` - this way you can share the script between
    -+multiple repos. That is, `cp ~/my-script.sh ~/project/.git/hooks/pre-commit`
    -+would become `git config --add hook.my-script.command ~/my-script.sh && git
    -+config --add hook.my-script.event pre-commit`.
    ++`.git/hooks/<hook-event>`, you can specify it in the config instead by running:
    ++
    ++----
    ++git config hook.<some-name>.command <path-to-script>
    ++git config --add hook.<some-name>.event <hook-event>
    ++----
    ++
    ++This way you can share the script between multiple repos. That is, `cp
    ++~/my-script.sh ~/project/.git/hooks/pre-commit` would become:
    ++
    ++----
    ++git config hook.my-script.command ~/my-script.sh
    ++git config --add hook.my-script.event pre-commit
    ++----
     +
      SUBCOMMANDS
      -----------
    @@ Documentation/git-hook.txt: Git is unlikely to use for a native hook later on. F
      +
      Any positional arguments to the hook should be passed after an
      optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
    +@@ Documentation/git-hook.txt: config is not specified, uses the number of CPUs on the current system. Some
    + hooks may be ineligible for parallelization: for example, 'commit-msg' intends
    + hooks modify the commit message body and cannot be parallelized.
    + 
    ++[[WRAPPERS]]
    ++WRAPPERS
    ++--------
    ++
    ++`git hook run` has been designed to make it easy for tools which wrap Git to
    ++configure and execute hooks using the Git hook infrastructure. It is possible to
    ++provide arguments, environment variables (TODO this is missing from reroll TODO),
    ++and stdin via the command line, as well as specifying parallel or series
    ++execution if the user has provided multiple hooks.
    ++
    ++Assuming your wrapper wants to support a hook named "mywrapper-start-tests", you
    ++can have your users specify their hooks like so:
    ++
    ++  [hook "setup-test-dashboard"]
    ++    event = mywrapper-start-tests
    ++    command = ~/mywrapper/setup-dashboard.py --tap
    ++
    ++Then, in your 'mywrapper' tool, you can invoke any users' configured hooks by
    ++running:
    ++
    ++----
    ++git hook run mywrapper-start-tests \
    ++  # providing something to stdin
    ++  --stdin some-tempfile-123 \
    ++  # setting an env var (TODO THIS IS MISSING TODO)
    ++  --env MYWRAPPER_EXECUTION_MODE=foo \
    ++  # execute hooks in serial
    ++  --jobs 1 \
    ++  # plus some arguments of your own...
    ++  -- \
    ++  --testname bar \
    ++  baz
    ++----
    ++
    ++Take care to name your wrapper's hook events in a way which is unlikely to
    ++overlap with Git's native hooks (see linkgit:githooks[5]) - a hook event named
    ++`mywrappertool-validate-commit` is much less likely to be added to native Git
    ++than a hook event named `validate-commit`. If Git begins to use a hook event
    ++named the same thing as your wrapper hook, it may invoke your users' hooks in
    ++unintended and unsupported ways.
    ++
    + CONFIGURATION
    + -------------
    + include::config/hook.txt[]
     
      ## builtin/hook.c ##
     @@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
    @@ builtin/hook.c: static int list(int argc, const char **argv, const char *prefix)
     +						  : _("hook from hookdir"));
      	}
      
    - 	clear_hook_list(head);
    + cleanup:
     
      ## hook.c ##
     @@ hook.c: static void free_hook(struct hook *ptr)
    + 	if (!ptr)
    + 		return;
    + 
    ++	free(ptr->name);
    + 	free(ptr->feed_pipe_cb_data);
      	free(ptr);
      }
      
    @@ hook.c: static void free_hook(struct hook *ptr)
     +
     +	if (!to_add) {
     +		/* adding a new hook, not moving an old one */
    -+		to_add = xmalloc(sizeof(*to_add));
    -+		to_add->name = name;
    -+		to_add->feed_pipe_cb_data = NULL;
    ++		to_add = xcalloc(1, sizeof(*to_add));
    ++		to_add->name = xstrdup_or_null(name);
     +	}
     +
     +	list_add_tail(&to_add->list, head);
    @@ hook.c: static void free_hook(struct hook *ptr)
      {
      	struct hook *hook_to_remove = list_entry(to_remove, struct hook, list);
     @@ hook.c: int hook_exists(const char *name)
    + 	return exists;
    + }
      
    - struct hook_config_cb
    - {
    --	struct strbuf *hook_key;
    ++struct hook_config_cb
    ++{
     +	const char *hook_event;
    - 	struct list_head *list;
    - };
    - 
    ++	struct list_head *list;
    ++};
    ++
     +/*
     + * Callback for git_config which adds configured hooks to a hook list.  Hooks
     + * can be configured by specifying both hook.<friend-name>.command = <path> and
    @@ hook.c: int hook_exists(const char *name)
     +	 */
     +	strbuf_add(&subsection_cpy, subsection, subsection_len);
     +
    -+	append_or_move_hook(data->list, strbuf_detach(&subsection_cpy, NULL));
    -+
    ++	append_or_move_hook(data->list, subsection_cpy.buf);
    ++	strbuf_release(&subsection_cpy);
     +
     +	return 0;
     +}
     +
      struct list_head *list_hooks(const char *hookname)
    - {
    - 	if (!known_hook(hookname))
    -@@ hook.c: struct list_head *list_hooks(const char *hookname)
    - struct list_head *list_hooks_gently(const char *hookname)
      {
      	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
     +	struct hook_config_cb cb_data = {
    @@ hook.c: struct list_head *list_hooks(const char *hookname)
      		BUG("null hookname was provided to hook_list()!");
      
     -	if (have_git_dir()) {
    --		const char *hook_path = find_hook_gently(hookname);
    +-		const char *hook_path = find_hook(hookname);
    ++	/* Add the hooks from the config, e.g. hook.myhook.event = pre-commit */
    ++	git_config(hook_config_lookup, &cb_data);
    + 
    +-		/* Add the hook from the hookdir */
     -		if (hook_path) {
     -			struct hook *to_add = xmalloc(sizeof(*to_add));
     -			to_add->hook_path = hook_path;
    @@ hook.c: struct list_head *list_hooks(const char *hookname)
     -			list_add_tail(&to_add->list, hook_head);
     -		}
     -	}
    -+	/* Add the hooks from the config, e.g. hook.myhook.event = pre-commit */
    -+	git_config(hook_config_lookup, &cb_data);
    -+
     +	/* Add the hook from the hookdir. The placeholder makes it easier to
     +	 * allocate work in pick_next_hook. */
    -+	if (find_hook_gently(hookname))
    ++	if (find_hook(hookname))
     +		append_or_move_hook(hook_head, NULL);
      
      	return hook_head;
    @@ hook.c: static int pick_next_hook(struct child_process *cp,
      	cp->trace2_hook_name = hook_cb->hook_name;
      	cp->dir = hook_cb->options->dir;
      
    -+	/* to enable oneliners, let config-specified hooks run in shell.
    -+	 * config-specified hooks have a name. */
    ++	/*
    ++	 * to enable oneliners, let config-specified hooks run in shell.
    ++	 * config-specified hooks have a name.
    ++	 */
     +	cp->use_shell = !!run_me->name;
     +
      	/* add command */
    @@ hook.c: static int pick_next_hook(struct child_process *cp,
     +		}
     +
     +		strvec_push(&cp->args, command);
    ++		free(command);
    ++		strbuf_release(&cmd_key);
     +	} else {
     +		/* ...from hookdir. */
     +		const char *hook_path = NULL;
     +		/*
    -+		 *
     +		 * At this point we are already running, so don't validate
    -+		 * whether the hook name is known or not.
    ++		 * whether the hook name is known or not. Validation was
    ++		 * performed earlier in list_hooks().
     +		 */
    -+		hook_path = find_hook_gently(hook_cb->hook_name);
    ++		hook_path = find_hook(hook_cb->hook_name);
     +		if (!hook_path)
     +			BUG("hookdir hook in hook list but no hookdir hook present in filesystem");
     +
    @@ hook.c: static int notify_start_failure(struct strbuf *out,
      }
     
      ## hook.h ##
    -@@ hook.h: int hook_exists(const char *hookname);
    +@@
      
      struct hook {
      	struct list_head list;
    @@ hook.h: int hook_exists(const char *hookname);
     +	 * The friendly name of the hook. NULL indicates the hook is from the
     +	 * hookdir.
     +	 */
    -+	const char *name;
    ++	char *name;
      
      	/*
      	 * Use this to keep state for your feed_pipe_fn if you are using
     
      ## t/t1800-hook.sh ##
     @@
    - #!/bin/bash
    + #!/bin/sh
      
     -test_description='git-hook command'
     +test_description='git-hook command and config-managed multihooks'
    @@ t/t1800-hook.sh
      . ./test-lib.sh
      
     +setup_hooks () {
    ++	test_config hook.ghi.command "/path/ghi"
     +	test_config hook.ghi.event pre-commit --add
    -+	test_config hook.ghi.command "/path/ghi" --add
    ++	test_config hook.ghi.event test-hook --add
    ++	test_config_global hook.def.command "/path/def"
     +	test_config_global hook.def.event pre-commit --add
    -+	test_config_global hook.def.command "/path/def" --add
     +}
     +
     +setup_hookdir () {
    @@ t/t1800-hook.sh
      	test_expect_code 129 git hook run -h &&
     +	test_expect_code 129 git hook list -h &&
      	test_expect_code 129 git hook run --unknown 2>err &&
    - 	grep "unknown option" err
    - '
    + 	test_expect_code 129 git hook list &&
    + 	test_expect_code 129 git hook list -h &&
    +@@ t/t1800-hook.sh: test_expect_success 'git hook list: does-not-exist hook' '
    + 
    + test_expect_success 'git hook list: existing hook' '
    + 	cat >expect <<-\EOF &&
    +-	.git/hooks/test-hook
    ++	hook from hookdir
    + 	EOF
    + 	git hook list test-hook >actual &&
    + 	test_cmp expect actual
     @@ t/t1800-hook.sh: test_expect_success 'stdin to hooks' '
      	test_cmp expect actual
      '
    @@ t/t1800-hook.sh: test_expect_success 'stdin to hooks' '
     +test_expect_success 'git hook list orders by config order' '
     +	setup_hooks &&
     +
    -+	cat >expected <<-EOF &&
    ++	cat >expected <<-\EOF &&
     +	def
     +	ghi
     +	EOF
    @@ t/t1800-hook.sh: test_expect_success 'stdin to hooks' '
     +	# configuring it locally.
     +	test_config hook.def.event "pre-commit" --add &&
     +
    -+	cat >expected <<-EOF &&
    ++	cat >expected <<-\EOF &&
     +	ghi
     +	def
     +	EOF
    @@ t/t1800-hook.sh: test_expect_success 'stdin to hooks' '
     +	test_cmp expected actual
     +'
     +
    ++test_expect_success 'hook can be configured for multiple events' '
    ++	setup_hooks &&
    ++
    ++	# 'ghi' should be included in both 'pre-commit' and 'test-hook'
    ++	git hook list pre-commit >actual &&
    ++	grep "ghi" actual &&
    ++	git hook list test-hook >actual &&
    ++	grep "ghi" actual
    ++'
    ++
     +test_expect_success 'git hook list shows hooks from the hookdir' '
     +	setup_hookdir &&
     +
    -+	cat >expected <<-EOF &&
    ++	cat >expected <<-\EOF &&
     +	hook from hookdir
     +	EOF
     +
    @@ t/t1800-hook.sh: test_expect_success 'stdin to hooks' '
     +'
     +
     +test_expect_success 'inline hook definitions resolve paths' '
    -+	write_script sample-hook.sh <<-EOF &&
    ++	write_script sample-hook.sh <<-\EOF &&
     +	echo \"Sample Hook\"
     +	EOF
     +
    @@ t/t1800-hook.sh: test_expect_success 'stdin to hooks' '
     +	test_config hook.stdin-b.event "test-hook" --add &&
     +	test_config hook.stdin-b.command "xargs -P1 -I% echo b%" --add &&
     +
    -+	cat >input <<-EOF &&
    ++	cat >input <<-\EOF &&
     +	1
     +	2
     +	3
     +	EOF
     +
    -+	cat >expected <<-EOF &&
    ++	cat >expected <<-\EOF &&
     +	a1
     +	a2
     +	a3
    @@ t/t1800-hook.sh: test_expect_success 'stdin to hooks' '
     +	echo 3
     +	EOF
     +
    -+	cat >expected <<-EOF &&
    ++	cat >expected <<-\EOF &&
     +	1
     +	2
     +	3
    @@ t/t1800-hook.sh: test_expect_success 'stdin to hooks' '
     +
     +	rm -rf .git/hooks
     +'
    ++
    ++test_expect_success 'rejects hooks with no commands configured' '
    ++	test_config hook.broken.event "test-hook" &&
    ++
    ++	echo broken >expected &&
    ++	git hook list test-hook >actual &&
    ++	test_cmp expected actual &&
    ++	test_must_fail git hook run test-hook
    ++'
    ++
      test_done
6:  30ffe98601e ! 5:  5d5e9726fd8 hook: allow out-of-repo 'git hook' invocations
    @@ git.c: static struct cmd_struct commands[] = {
      	{ "init-db", cmd_init_db },
     
      ## hook.c ##
    -@@ hook.c: struct list_head *list_hooks_gently(const char *hookname)
    +@@ hook.c: struct list_head *list_hooks(const char *hookname)
      
      	/* Add the hook from the hookdir. The placeholder makes it easier to
      	 * allocate work in pick_next_hook. */
    --	if (find_hook_gently(hookname))
    -+	if (have_git_dir() && find_hook_gently(hookname))
    +-	if (find_hook(hookname))
    ++	if (have_git_dir() && find_hook(hookname))
      		append_or_move_hook(hook_head, NULL);
      
      	return hook_head;
     
      ## t/t1800-hook.sh ##
    -@@ t/t1800-hook.sh: test_expect_success 'git hook run -- pass arguments' '
    +@@ t/t1800-hook.sh: test_expect_success 'git hook list: existing hook' '
      	test_cmp expect actual
      '
      
-- 
2.33.0.867.g88ec4638586


^ permalink raw reply	[flat|nested] 479+ messages in thread

* [PATCH v4 1/5] hook: run a list of hooks instead
  2021-09-09 12:41                 ` [PATCH v4 0/5] " Ævar Arnfjörð Bjarmason
@ 2021-09-09 12:41                   ` Ævar Arnfjörð Bjarmason
  2021-09-09 12:59                     ` [PATCH v4] fixup! " Ævar Arnfjörð Bjarmason
  2021-09-09 12:42                   ` [PATCH v4 2/5] hook: allow parallel hook execution Ævar Arnfjörð Bjarmason
                                     ` (3 subsequent siblings)
  4 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-09 12:41 UTC (permalink / raw)
  To: git
  Cc: Emily Shaffer, Junio C Hamano, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

To prepare for multihook support, teach hook.[hc] to take a list of
hooks at run_hooks and run_found_hooks. Right now the list is always one
entry, but in the future we will allow users to supply more than one
executable for a single hook event.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/hook.c |  14 ++++---
 hook.c         | 109 ++++++++++++++++++++++++++++++++++++-------------
 hook.h         |  22 ++++++++--
 3 files changed, 107 insertions(+), 38 deletions(-)

diff --git a/builtin/hook.c b/builtin/hook.c
index fae69068201..398980523aa 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -25,7 +25,7 @@ static int run(int argc, const char **argv, const char *prefix)
 	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
 	int ignore_missing = 0;
 	const char *hook_name;
-	const char *hook_path;
+	struct list_head *hooks;
 	struct option run_options[] = {
 		OPT_BOOL(0, "ignore-missing", &ignore_missing,
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
@@ -63,16 +63,18 @@ static int run(int argc, const char **argv, const char *prefix)
 	 * run_hooks() instead...
 	 */
 	hook_name = argv[0];
-	if (ignore_missing)
+	hooks = list_hooks(hook_name);
+	if (list_empty(hooks)) {
+		clear_hook_list(hooks);
+
 		/* ... act like a plain run_hooks() under --ignore-missing */
-		return run_hooks_oneshot(hook_name, &opt);
-	hook_path = find_hook(hook_name);
-	if (!hook_path) {
+		if (ignore_missing)
+			return 0;
 		error("cannot find a hook named %s", hook_name);
 		return 1;
 	}
 
-	ret = run_hooks(hook_name, hook_path, &opt);
+	ret = run_hooks(hook_name, hooks, &opt);
 	run_hooks_opt_clear(&opt);
 	return ret;
 usage:
diff --git a/hook.c b/hook.c
index d045379ade8..2b2c16a9095 100644
--- a/hook.c
+++ b/hook.c
@@ -3,6 +3,30 @@
 #include "run-command.h"
 #include "config.h"
 
+static void free_hook(struct hook *ptr)
+{
+	if (!ptr)
+		return;
+
+	free(ptr->feed_pipe_cb_data);
+	free(ptr);
+}
+
+static void remove_hook(struct list_head *to_remove)
+{
+	struct hook *hook_to_remove = list_entry(to_remove, struct hook, list);
+	list_del(to_remove);
+	free_hook(hook_to_remove);
+}
+
+void clear_hook_list(struct list_head *head)
+{
+	struct list_head *pos, *tmp;
+	list_for_each_safe(pos, tmp, head)
+		remove_hook(pos);
+	free(head);
+}
+
 const char *find_hook(const char *name)
 {
 	static struct strbuf path = STRBUF_INIT;
@@ -39,7 +63,38 @@ const char *find_hook(const char *name)
 
 int hook_exists(const char *name)
 {
-	return !!find_hook(name);
+	struct list_head *hooks;
+	int exists;
+
+	hooks = list_hooks(name);
+	exists = !list_empty(hooks);
+	clear_hook_list(hooks);
+
+	return exists;
+}
+
+struct list_head *list_hooks(const char *hookname)
+{
+	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
+
+	INIT_LIST_HEAD(hook_head);
+
+	if (!hookname)
+		BUG("null hookname was provided to hook_list()!");
+
+	if (have_git_dir()) {
+		const char *hook_path = find_hook(hookname);
+
+		/* Add the hook from the hookdir */
+		if (hook_path) {
+			struct hook *to_add = xmalloc(sizeof(*to_add));
+			to_add->hook_path = hook_path;
+			to_add->feed_pipe_cb_data = NULL;
+			list_add_tail(&to_add->list, hook_head);
+		}
+	}
+
+	return hook_head;
 }
 
 void run_hooks_opt_clear(struct run_hooks_opt *o)
@@ -99,7 +154,10 @@ static int pick_next_hook(struct child_process *cp,
 	cp->dir = hook_cb->options->dir;
 
 	/* add command */
-	strvec_push(&cp->args, run_me->hook_path);
+	if (hook_cb->options->absolute_path)
+		strvec_push(&cp->args, absolute_path(run_me->hook_path));
+	else
+		strvec_push(&cp->args, run_me->hook_path);
 
 	/*
 	 * add passed-in argv, without expanding - let the user get back
@@ -110,12 +168,12 @@ static int pick_next_hook(struct child_process *cp,
 	/* Provide context for errors if necessary */
 	*pp_task_cb = run_me;
 
-	/*
-	 * This pick_next_hook() will be called again, we're only
-	 * running one hook, so indicate that no more work will be
-	 * done.
-	 */
-	hook_cb->run_me = NULL;
+	/* Get the next entry ready */
+	if (hook_cb->run_me->list.next == hook_cb->head)
+		hook_cb->run_me = NULL;
+	else
+		hook_cb->run_me = list_entry(hook_cb->run_me->list.next,
+					     struct hook, list);
 
 	return 1;
 }
@@ -150,13 +208,9 @@ static int notify_hook_finished(int result,
 	return 0;
 }
 
-int run_hooks(const char *hook_name, const char *hook_path,
+int run_hooks(const char *hook_name, struct list_head *hooks,
 	      struct run_hooks_opt *options)
 {
-	struct strbuf abs_path = STRBUF_INIT;
-	struct hook my_hook = {
-		.hook_path = hook_path,
-	};
 	struct hook_cb_data cb_data = {
 		.rc = 0,
 		.hook_name = hook_name,
@@ -168,11 +222,8 @@ int run_hooks(const char *hook_name, const char *hook_path,
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
 
-	if (options->absolute_path) {
-		strbuf_add_absolute_path(&abs_path, hook_path);
-		my_hook.hook_path = abs_path.buf;
-	}
-	cb_data.run_me = &my_hook;
+	cb_data.head = hooks;
+	cb_data.run_me = list_first_entry(hooks, struct hook, list);
 
 	run_processes_parallel_tr2(jobs,
 				   pick_next_hook,
@@ -184,18 +235,15 @@ int run_hooks(const char *hook_name, const char *hook_path,
 				   "hook",
 				   hook_name);
 
-
-	if (options->absolute_path)
-		strbuf_release(&abs_path);
-	free(my_hook.feed_pipe_cb_data);
+	clear_hook_list(hooks);
 
 	return cb_data.rc;
 }
 
 int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 {
-	const char *hook_path;
-	int ret;
+	struct list_head *hooks;
+	int ret = 0;
 	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
 
 	if (!options)
@@ -204,13 +252,16 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 	if (options->path_to_stdin && options->feed_pipe)
 		BUG("choose only one method to populate stdin");
 
-	hook_path = find_hook(hook_name);
-	if (!hook_path) {
-		ret = 0;
+	hooks = list_hooks(hook_name);
+
+	/*
+	 * If you need to act on a missing hook, use run_found_hooks()
+	 * instead
+	 */
+	if (list_empty(hooks))
 		goto cleanup;
-	}
 
-	ret = run_hooks(hook_name, hook_path, options);
+	ret = run_hooks(hook_name, hooks, options);
 
 cleanup:
 	run_hooks_opt_clear(options);
diff --git a/hook.h b/hook.h
index f6dac75f1cc..49b4c335f86 100644
--- a/hook.h
+++ b/hook.h
@@ -3,8 +3,10 @@
 #include "strbuf.h"
 #include "strvec.h"
 #include "run-command.h"
+#include "list.h"
 
 struct hook {
+	struct list_head list;
 	/* The path to the hook */
 	const char *hook_path;
 
@@ -75,6 +77,7 @@ struct hook_cb_data {
 	/* rc reflects the cumulative failure state */
 	int rc;
 	const char *hook_name;
+	struct list_head *head;
 	struct hook *run_me;
 	struct run_hooks_opt *options;
 	int *invoked_hook;
@@ -88,7 +91,13 @@ struct hook_cb_data {
 const char *find_hook(const char *name);
 
 /**
- * A boolean version of find_hook()
+ * Provides a linked list of 'struct hook' detailing commands which should run
+ * in response to the 'hookname' event, in execution order.
+ */
+struct list_head *list_hooks(const char *hookname);
+
+/**
+ * A boolean version of list_hooks()
  */
 int hook_exists(const char *hookname);
 
@@ -99,13 +108,20 @@ void run_hooks_opt_clear(struct run_hooks_opt *o);
 
 /**
  * Takes an already resolved hook found via find_hook() and runs
- * it. Does not call run_hooks_opt_clear() for you.
+ * it. Does not call run_hooks_opt_clear() for you, but does call
+ * clear_hook_list().
  *
  * See run_hooks_oneshot() for the simpler one-shot API.
  */
-int run_hooks(const char *hookname, const char *hook_path,
+int run_hooks(const char *hookname, struct list_head *hooks,
 	      struct run_hooks_opt *options);
 
+/**
+ * Empties the list at 'head', calling 'free_hook()' on each
+ * entry. Called implicitly by run_hooks() (and run_hooks_oneshot()).
+ */
+void clear_hook_list(struct list_head *head);
+
 /**
  * Calls find_hook() on your "hook_name" and runs the hooks (if any)
  * with run_hooks().
-- 
2.33.0.867.g88ec4638586


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 2/5] hook: allow parallel hook execution
  2021-09-09 12:41                 ` [PATCH v4 0/5] " Ævar Arnfjörð Bjarmason
  2021-09-09 12:41                   ` [PATCH v4 1/5] hook: run a list of hooks instead Ævar Arnfjörð Bjarmason
@ 2021-09-09 12:42                   ` Ævar Arnfjörð Bjarmason
  2021-09-09 12:42                   ` [PATCH v4 3/5] hook: introduce "git hook list" Ævar Arnfjörð Bjarmason
                                     ` (2 subsequent siblings)
  4 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-09 12:42 UTC (permalink / raw)
  To: git
  Cc: Emily Shaffer, Junio C Hamano, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

In many cases, there's no reason not to allow hooks to execute in
parallel, if more than one was provided. hook.c already calls
run_processes_parallel(), so all we need to do is allow the job count we
hand to run_processes_parallel() to be greater than 1.

If users have specified no alternative, we can use the processor count
from online_cpus() to run an efficient number of tasks at once. However,
users can also override this number if they want by configuring
'hook.jobs'.

To avoid looking up 'hook.jobs' in cases where we don't end up with any
hooks to run anyways, teach the hook runner commands to notice if
.jobs==0 and do a config or online_cpus() lookup if so, when we already
know we have jobs to run.

Serial execution can still be performed by setting .jobs == 1.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/config.txt      |  2 ++
 Documentation/config/hook.txt |  4 ++++
 Documentation/git-hook.txt    | 17 ++++++++++++++++-
 builtin/am.c                  |  4 ++--
 builtin/checkout.c            |  2 +-
 builtin/clone.c               |  2 +-
 builtin/hook.c                |  4 +++-
 builtin/merge.c               |  2 +-
 builtin/rebase.c              |  2 +-
 builtin/receive-pack.c        |  9 +++++----
 builtin/worktree.c            |  2 +-
 commit.c                      |  2 +-
 hook.c                        | 35 ++++++++++++++++++++++++++++++++---
 hook.h                        | 16 +++++++++++++++-
 read-cache.c                  |  2 +-
 refs.c                        |  2 +-
 reset.c                       |  3 ++-
 sequencer.c                   |  4 ++--
 transport.c                   |  2 +-
 19 files changed, 92 insertions(+), 24 deletions(-)
 create mode 100644 Documentation/config/hook.txt

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 0c0e6b859f1..6fb218f649d 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -391,6 +391,8 @@ include::config/guitool.txt[]
 
 include::config/help.txt[]
 
+include::config/hook.txt[]
+
 include::config/http.txt[]
 
 include::config/i18n.txt[]
diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
new file mode 100644
index 00000000000..96d3d6572c1
--- /dev/null
+++ b/Documentation/config/hook.txt
@@ -0,0 +1,4 @@
+hook.jobs::
+	Specifies how many hooks can be run simultaneously during parallelized
+	hook execution. If unspecified, defaults to the number of processors on
+	the current system.
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index fa68c1f3912..79e82479ec6 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -8,7 +8,8 @@ git-hook - run git hooks
 SYNOPSIS
 --------
 [verse]
-'git hook' run [--to-stdin=<path>] [--ignore-missing] <hook-name> [-- <hook-args>]
+'git hook' run [--to-stdin=<path>] [--ignore-missing] [(-j|--jobs) <n>]
+	<hook-name> [-- <hook-args>]
 
 DESCRIPTION
 -----------
@@ -42,6 +43,20 @@ OPTIONS
 	tools that want to do a blind one-shot run of a hook that may
 	or may not be present.
 
+-j::
+--jobs::
+	Only valid for `run`.
++
+Specify how many hooks to run simultaneously. If this flag is not specified,
+uses the value of the `hook.jobs` config, see linkgit:git-config[1]. If the
+config is not specified, uses the number of CPUs on the current system. Some
+hooks may be ineligible for parallelization: for example, 'commit-msg' intends
+hooks modify the commit message body and cannot be parallelized.
+
+CONFIGURATION
+-------------
+include::config/hook.txt[]
+
 SEE ALSO
 --------
 linkgit:githooks[5]
diff --git a/builtin/am.c b/builtin/am.c
index 9e3d4d9ab44..c7ffc7eafc5 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -446,7 +446,7 @@ static void am_destroy(const struct am_state *state)
 static int run_applypatch_msg_hook(struct am_state *state)
 {
 	int ret;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
 
 	assert(state->msg);
 	strvec_push(&opt.args, am_path(state, "final-commit"));
@@ -467,7 +467,7 @@ static int run_applypatch_msg_hook(struct am_state *state)
  */
 static int run_post_rewrite_hook(const struct am_state *state)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 	strvec_push(&opt.args, "rebase");
 	opt.path_to_stdin = am_path(state, "rewritten");
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 863b02a7d7c..6b99d31c6ba 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -107,7 +107,7 @@ struct branch_info {
 static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit,
 			      int changed)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
 
 	/* "new_commit" can be NULL when checking out from the index before
 	   a commit exists. */
diff --git a/builtin/clone.c b/builtin/clone.c
index 27fc05ee511..986c3b1932a 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -776,7 +776,7 @@ static int checkout(int submodule_progress)
 	struct tree *tree;
 	struct tree_desc t;
 	int err = 0;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_SERIAL;
 
 	if (option_no_checkout)
 		return 0;
diff --git a/builtin/hook.c b/builtin/hook.c
index 398980523aa..9b6272cfd3b 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -22,7 +22,7 @@ static const char * const builtin_hook_run_usage[] = {
 static int run(int argc, const char **argv, const char *prefix)
 {
 	int i;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
 	int ignore_missing = 0;
 	const char *hook_name;
 	struct list_head *hooks;
@@ -31,6 +31,8 @@ static int run(int argc, const char **argv, const char *prefix)
 			 N_("exit quietly with a zero exit code if the requested hook cannot be found")),
 		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
 			   N_("file to read into hooks' stdin")),
+		OPT_INTEGER('j', "jobs", &opt.jobs,
+			    N_("run up to <n> hooks simultaneously")),
 		OPT_END(),
 	};
 	int ret;
diff --git a/builtin/merge.c b/builtin/merge.c
index f215f264cc8..c01eb535d6b 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -448,7 +448,7 @@ static void finish(struct commit *head_commit,
 		   const struct object_id *new_head, const char *msg)
 {
 	struct strbuf reflog_message = STRBUF_INIT;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	const struct object_id *head = &head_commit->object.oid;
 
 	if (!msg)
diff --git a/builtin/rebase.c b/builtin/rebase.c
index ee68a1df492..a4bbb9abb35 100644
--- a/builtin/rebase.c
+++ b/builtin/rebase.c
@@ -1314,7 +1314,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	char *squash_onto_name = NULL;
 	int reschedule_failed_exec = -1;
 	int allow_preemptive_ff = 1;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	struct option builtin_rebase_options[] = {
 		OPT_STRING(0, "onto", &options.onto_name,
 			   N_("revision"),
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index ebec6f3bb10..7460124b743 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -909,7 +909,7 @@ static int run_receive_hook(struct command *commands,
 			    int skip_broken,
 			    const struct string_list *push_options)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	struct receive_hook_feed_context ctx;
 	struct command *iter = commands;
 
@@ -948,7 +948,7 @@ static int run_receive_hook(struct command *commands,
 
 static int run_update_hook(struct command *cmd)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 	strvec_pushl(&opt.args,
 		     cmd->ref_name,
@@ -1432,7 +1432,8 @@ static const char *push_to_checkout(unsigned char *hash,
 				    struct strvec *env,
 				    const char *work_tree)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
+
 	opt.invoked_hook = invoked_hook;
 
 	strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree));
@@ -1628,7 +1629,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 static void run_update_post_hook(struct command *commands)
 {
 	struct command *cmd;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 	for (cmd = commands; cmd; cmd = cmd->next) {
 		if (cmd->error_string || cmd->did_not_exist)
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 330867c19bf..30905067906 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -382,7 +382,7 @@ static int add_worktree(const char *path, const char *refname,
 	 * is_junk is cleared, but do return appropriate code when hook fails.
 	 */
 	if (!ret && opts->checkout) {
-		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
 
 		strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL);
 		strvec_pushl(&opt.args,
diff --git a/commit.c b/commit.c
index 842e47beae2..a38bd047524 100644
--- a/commit.c
+++ b/commit.c
@@ -1700,7 +1700,7 @@ int run_commit_hook(int editor_is_used, const char *index_file,
 		    int *invoked_hook,
 		    const char *name, ...)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_SERIAL;
 	va_list args;
 	const char *arg;
 
diff --git a/hook.c b/hook.c
index 2b2c16a9095..600030c59ec 100644
--- a/hook.c
+++ b/hook.c
@@ -208,6 +208,28 @@ static int notify_hook_finished(int result,
 	return 0;
 }
 
+/*
+ * Determines how many jobs to use after we know we want to parallelize. First
+ * priority is the config 'hook.jobs' and second priority is the number of CPUs.
+ */
+static int configured_hook_jobs(void)
+{
+	/*
+	 * The config and the CPU count probably won't change during the process
+	 * lifetime, so cache the result in case we invoke multiple hooks during
+	 * one process.
+	 */
+	static int jobs = 0;
+	if (jobs)
+		return jobs;
+
+	if (git_config_get_int("hook.jobs", &jobs))
+		/* if the config isn't set, fall back to CPU count. */
+		jobs = online_cpus();
+
+	return jobs;
+}
+
 int run_hooks(const char *hook_name, struct list_head *hooks,
 	      struct run_hooks_opt *options)
 {
@@ -217,7 +239,6 @@ int run_hooks(const char *hook_name, struct list_head *hooks,
 		.options = options,
 		.invoked_hook = options->invoked_hook,
 	};
-	int jobs = 1;
 
 	if (!options)
 		BUG("a struct run_hooks_opt must be provided to run_hooks");
@@ -225,7 +246,11 @@ int run_hooks(const char *hook_name, struct list_head *hooks,
 	cb_data.head = hooks;
 	cb_data.run_me = list_first_entry(hooks, struct hook, list);
 
-	run_processes_parallel_tr2(jobs,
+	/* INIT_PARALLEL sets jobs to 0, so go look up how many to use. */
+	if (!options->jobs)
+		options->jobs = configured_hook_jobs();
+
+	run_processes_parallel_tr2(options->jobs,
 				   pick_next_hook,
 				   notify_start_failure,
 				   options->feed_pipe,
@@ -244,7 +269,11 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 {
 	struct list_head *hooks;
 	int ret = 0;
-	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT;
+	/*
+	 * Turn on parallelism by default. Hooks which don't want it should
+	 * specify their options accordingly.
+	 */
+	struct run_hooks_opt hook_opt_scratch = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 	if (!options)
 		options = &hook_opt_scratch;
diff --git a/hook.h b/hook.h
index 49b4c335f86..fe16ab35028 100644
--- a/hook.h
+++ b/hook.h
@@ -25,6 +25,13 @@ struct run_hooks_opt
 	/* Args to be passed to each hook */
 	struct strvec args;
 
+	/*
+	 * Number of threads to parallelize across. Set to 0 to use the
+	 * 'hook.jobs' config or, if that config is unset, the number of cores
+	 * on the system.
+	 */
+	int jobs;
+
 	/*
 	 * Resolve and run the "absolute_path(hook)" instead of
 	 * "hook". Used for "git worktree" hooks
@@ -68,7 +75,14 @@ struct run_hooks_opt
 	int *invoked_hook;
 };
 
-#define RUN_HOOKS_OPT_INIT { \
+#define RUN_HOOKS_OPT_INIT_SERIAL { \
+	.jobs = 1, \
+	.env = STRVEC_INIT, \
+	.args = STRVEC_INIT, \
+}
+
+#define RUN_HOOKS_OPT_INIT_PARALLEL { \
+	.jobs = 0, \
 	.env = STRVEC_INIT, \
 	.args = STRVEC_INIT, \
 }
diff --git a/read-cache.c b/read-cache.c
index 875f6c1dea5..98e9fb8b04e 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -3069,7 +3069,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l
 {
 	int ret;
 	int was_full = !istate->sparse_index;
-	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 	ret = convert_to_sparse(istate);
 
diff --git a/refs.c b/refs.c
index 73d4a939267..5543b8cdaba 100644
--- a/refs.c
+++ b/refs.c
@@ -2062,7 +2062,7 @@ int ref_update_reject_duplicates(struct string_list *refnames,
 static int run_transaction_hook(struct ref_transaction *transaction,
 				const char *state)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
 	int ret = 0, i;
 
diff --git a/reset.c b/reset.c
index 1237ced8a58..63a9c513409 100644
--- a/reset.c
+++ b/reset.c
@@ -127,7 +127,8 @@ int reset_head(struct repository *r, struct object_id *oid, const char *action,
 					    reflog_head);
 	}
 	if (run_hook) {
-		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+		struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
+
 		strvec_pushl(&opt.args,
 			     oid_to_hex(orig ? orig : null_oid()),
 			     oid_to_hex(oid),
diff --git a/sequencer.c b/sequencer.c
index db8044ab47d..979cd01c303 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1148,7 +1148,7 @@ int update_head_with_reflog(const struct commit *old_head,
 static int run_rewrite_hook(const struct object_id *oldoid,
 			    const struct object_id *newoid)
 {
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	struct strbuf tmp = STRBUF_INIT;
 	struct string_list to_stdin = STRING_LIST_INIT_DUP;
 	int code;
@@ -4522,7 +4522,7 @@ static int pick_commits(struct repository *r,
 		if (!stat(rebase_path_rewritten_list(), &st) &&
 				st.st_size > 0) {
 			struct child_process notes_cp = CHILD_PROCESS_INIT;
-			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT;
+			struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 
 			notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY);
 			notes_cp.git_cmd = 1;
diff --git a/transport.c b/transport.c
index 4ca8fc0391d..33da71a108b 100644
--- a/transport.c
+++ b/transport.c
@@ -1204,7 +1204,7 @@ static int run_pre_push_hook(struct transport *transport,
 			     struct ref *remote_refs)
 {
 	int ret = 0;
-	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT;
+	struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_PARALLEL;
 	struct ref *r;
 	struct string_list to_stdin = STRING_LIST_INIT_NODUP;
 
-- 
2.33.0.867.g88ec4638586


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 3/5] hook: introduce "git hook list"
  2021-09-09 12:41                 ` [PATCH v4 0/5] " Ævar Arnfjörð Bjarmason
  2021-09-09 12:41                   ` [PATCH v4 1/5] hook: run a list of hooks instead Ævar Arnfjörð Bjarmason
  2021-09-09 12:42                   ` [PATCH v4 2/5] hook: allow parallel hook execution Ævar Arnfjörð Bjarmason
@ 2021-09-09 12:42                   ` Ævar Arnfjörð Bjarmason
  2021-09-09 12:42                   ` [PATCH v4 4/5] hook: include hooks from the config Ævar Arnfjörð Bjarmason
  2021-09-09 12:42                   ` [PATCH v4 5/5] hook: allow out-of-repo 'git hook' invocations Ævar Arnfjörð Bjarmason
  4 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-09 12:42 UTC (permalink / raw)
  To: git
  Cc: Emily Shaffer, Junio C Hamano, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason

From: Emily Shaffer <emilyshaffer@google.com>

If more than one hook will be run, it may be useful to see a list of
which hooks should be run. At very least, it will be useful for us to
test the semantics of multihooks ourselves.

For now, only list the hooks which will run in the order they will run
in; later, it might be useful to include more information like where the
hooks were configured and whether or not they will run.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-hook.txt |  5 ++++
 builtin/hook.c             | 52 ++++++++++++++++++++++++++++++++++++++
 t/t1800-hook.sh            | 14 ++++++++++
 3 files changed, 71 insertions(+)

diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 79e82479ec6..88588b38143 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -10,6 +10,7 @@ SYNOPSIS
 [verse]
 'git hook' run [--to-stdin=<path>] [--ignore-missing] [(-j|--jobs) <n>]
 	<hook-name> [-- <hook-args>]
+'git hook' list <hook-name>
 
 DESCRIPTION
 -----------
@@ -30,6 +31,10 @@ optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
 arguments (if any) differ by hook name, see linkgit:githooks[5] for
 what those are.
 
+list::
+	Print a list of hooks which will be run on `<hook-name>` event. If no
+	hooks are configured for that event, print nothing and return 1.
+
 OPTIONS
 -------
 
diff --git a/builtin/hook.c b/builtin/hook.c
index 9b6272cfd3b..1e6b15d565a 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -8,8 +8,11 @@
 
 #define BUILTIN_HOOK_RUN_USAGE \
 	N_("git hook run [--ignore-missing] [--to-stdin=<path>] <hook-name> [-- <hook-args>]")
+#define BUILTIN_HOOK_LIST_USAGE \
+	N_("git hook list <hook-name>")
 
 static const char * const builtin_hook_usage[] = {
+	BUILTIN_HOOK_LIST_USAGE,
 	BUILTIN_HOOK_RUN_USAGE,
 	NULL
 };
@@ -19,6 +22,53 @@ static const char * const builtin_hook_run_usage[] = {
 	NULL
 };
 
+static const char *const builtin_hook_list_usage[] = {
+	BUILTIN_HOOK_LIST_USAGE,
+	NULL
+};
+
+static int list(int argc, const char **argv, const char *prefix)
+{
+	struct list_head *hooks;
+	struct list_head *pos;
+	const char *hookname = NULL;
+	struct option list_options[] = {
+		OPT_END(),
+	};
+	int ret = 0;
+
+	argc = parse_options(argc, argv, prefix, list_options,
+			     builtin_hook_list_usage, 0);
+
+	/*
+	 * The only unnamed argument provided should be the hook-name; if we add
+	 * arguments later they probably should be caught by parse_options.
+	 */
+	if (argc != 1)
+		usage_msg_opt(_("You must specify a hook event name to list."),
+			      builtin_hook_list_usage, list_options);
+
+	hookname = argv[0];
+
+	hooks = list_hooks(hookname);
+
+	if (list_empty(hooks)) {
+		ret = 1;
+		goto cleanup;
+	}
+
+	list_for_each(pos, hooks) {
+		struct hook *item = list_entry(pos, struct hook, list);
+		item = list_entry(pos, struct hook, list);
+		if (item)
+			printf("%s\n", item->hook_path);
+	}
+
+cleanup:
+	clear_hook_list(hooks);
+
+	return ret;
+}
 static int run(int argc, const char **argv, const char *prefix)
 {
 	int i;
@@ -94,6 +144,8 @@ int cmd_hook(int argc, const char **argv, const char *prefix)
 	if (!argc)
 		goto usage;
 
+	if (!strcmp(argv[0], "list"))
+		return list(argc, argv, prefix);
 	if (!strcmp(argv[0], "run"))
 		return run(argc, argv, prefix);
 
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
index 6431b19e392..7a1dae4e95e 100755
--- a/t/t1800-hook.sh
+++ b/t/t1800-hook.sh
@@ -9,6 +9,8 @@ test_expect_success 'git hook usage' '
 	test_expect_code 129 git hook run &&
 	test_expect_code 129 git hook run -h &&
 	test_expect_code 129 git hook run --unknown 2>err &&
+	test_expect_code 129 git hook list &&
+	test_expect_code 129 git hook list -h &&
 	grep "unknown option" err
 '
 
@@ -97,6 +99,18 @@ test_expect_success 'git hook run -- pass arguments' '
 	test_cmp expect actual
 '
 
+test_expect_success 'git hook list: does-not-exist hook' '
+	test_expect_code 1 git hook list does-not-exist
+'
+
+test_expect_success 'git hook list: existing hook' '
+	cat >expect <<-\EOF &&
+	.git/hooks/test-hook
+	EOF
+	git hook list test-hook >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'git hook run -- out-of-repo runs excluded' '
 	write_script .git/hooks/test-hook <<-EOF &&
 	echo Test hook
-- 
2.33.0.867.g88ec4638586


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 4/5] hook: include hooks from the config
  2021-09-09 12:41                 ` [PATCH v4 0/5] " Ævar Arnfjörð Bjarmason
                                     ` (2 preceding siblings ...)
  2021-09-09 12:42                   ` [PATCH v4 3/5] hook: introduce "git hook list" Ævar Arnfjörð Bjarmason
@ 2021-09-09 12:42                   ` Ævar Arnfjörð Bjarmason
  2021-09-09 12:42                   ` [PATCH v4 5/5] hook: allow out-of-repo 'git hook' invocations Ævar Arnfjörð Bjarmason
  4 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-09 12:42 UTC (permalink / raw)
  To: git
  Cc: Emily Shaffer, Junio C Hamano, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee

From: Emily Shaffer <emilyshaffer@google.com>

Teach the hook.[hc] library to parse configs to populate the list of
hooks to run for a given event.

Multiple commands can be specified for a given hook by providing
multiple "hook.<friendly-name>.command = <path-to-hook>" and
"hook.<friendly-name>.event = <hook-event>" lines. Hooks will be started
in config order of the "hook.<name>.event" lines (but may run in
parallel).

For example:

  $ git config --get-regexp "^hook\."
  hook.bar.command=~/bar.sh
  hook.bar.event=pre-commit

  # Will run ~/bar.sh, then .git/hooks/pre-commit
  $ git hook run pre-commit

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/config/hook.txt |  18 ++++
 Documentation/git-hook.txt    | 135 +++++++++++++++++++++++++++-
 builtin/hook.c                |   3 +-
 hook.c                        | 161 +++++++++++++++++++++++++++++----
 hook.h                        |   7 +-
 t/t1800-hook.sh               | 164 +++++++++++++++++++++++++++++++++-
 6 files changed, 465 insertions(+), 23 deletions(-)

diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt
index 96d3d6572c1..c3947563280 100644
--- a/Documentation/config/hook.txt
+++ b/Documentation/config/hook.txt
@@ -1,3 +1,21 @@
+hook.<name>.command::
+	A command to execute whenever `hook.<name>` is invoked. `<name>` should
+	be a unique "friendly" name which you can use to identify this hook
+	command. (You can specify when to invoke this command with
+	`hook.<name>.event`.) The value can be an executable on your device or a
+	oneliner for your shell. If more than one value is specified for the
+	same `<name>`, the last value parsed will be the only command executed.
+	See linkgit:git-hook[1].
+
+hook.<name>.event::
+	The hook events which should invoke `hook.<name>`. `<name>` should be a
+	unique "friendly" name which you can use to identify this hook. The
+	value should be the name of a hook event, like "pre-commit" or "update".
+	(See linkgit:githooks[5] for a complete list of hooks Git knows about.)
+	On the specified event, the associated `hook.<name>.command` will be
+	executed. More than one event can be specified if you wish for
+	`hook.<name>` to execute on multiple events. See linkgit:git-hook[1].
+
 hook.jobs::
 	Specifies how many hooks can be run simultaneously during parallelized
 	hook execution. If unspecified, defaults to the number of processors on
diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 88588b38143..51bda42fb83 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -19,12 +19,102 @@ This command is an interface to git hooks (see linkgit:githooks[5]).
 Currently it only provides a convenience wrapper for running hooks for
 use by git itself. In the future it might gain other functionality.
 
+It's possible to use this command to refer to hooks which are not native to Git,
+for example if a wrapper around Git wishes to expose hooks into its own
+operation in a way which is already familiar to Git users. However, wrappers
+invoking such hooks should be careful to name their hook events something which
+Git is unlikely to use for a native hook later on. For example, Git is much less
+likely to create a `mytool-validate-commit` hook than it is to create a
+`validate-commit` hook.
+
+This command parses the default configuration files for sets of configs like
+so:
+
+  [hook "linter"]
+    event = pre-commit
+    command = ~/bin/linter --cpp20
+
+In this example, `[hook "linter"]` represents one script - `~/bin/linter
+--cpp20` - which can be shared by many repos, and even by many hook events, if
+appropriate.
+
+To add an unrelated hook which runs on a different event, for example a
+spell-checker for your commit messages, you would write a configuration like so:
+
+  [hook "linter"]
+    event = pre-commit
+    command = ~/bin/linter --cpp20
+  [hook "spellcheck"]
+    event = commit-msg
+    command = ~/bin/spellchecker
+
+With this config, when you run 'git commit', first `~/bin/linter --cpp20` will
+have a chance to check your files to be committed (during the `pre-commit` hook
+event`), and then `~/bin/spellchecker` will have a chance to check your commit
+message (during the `commit-msg` hook event).
+
+Commands are run in the order Git encounters their associated
+`hook.<name>.event` configs during the configuration parse (see
+linkgit:git-config[1]). Although multiple `hook.linter.event` configs can be
+added, only one `hook.linter.command` event is valid - Git uses "last-one-wins"
+to determine which command to run.
+
+So if you wanted your linter to run when you commit as well as when you push,
+you would configure it like so:
+
+  [hook "linter"]
+    event = pre-commit
+    event = pre-push
+    command = ~/bin/linter --cpp20
+
+With this config, `~/bin/linter --cpp20` would be run by Git before a commit is
+generated (during `pre-commit`) as well as before a push is performed (during
+`pre-push`).
+
+And if you wanted to run your linter as well as a secret-leak detector during
+only the "pre-commit" hook event, you would configure it instead like so:
+
+  [hook "linter"]
+    event = pre-commit
+    command = ~/bin/linter --cpp20
+  [hook "no-leaks"]
+    event = pre-commit
+    command = ~/bin/leak-detector
+
+With this config, before a commit is generated (during `pre-commit`), Git would
+first start `~/bin/linter --cpp20` and second start `~/bin/leak-detector`. It
+would evaluate the output of each when deciding whether to proceed with the
+commit.
+
+For a full list of hook events which you can set your `hook.<name>.event` to,
+and how hooks are invoked during those events, see linkgit:githooks[5].
+
+Git will ignore any `hook.<name>.event` that specifies an event it doesn't
+recognize. This is intended so that tools which wrap Git can use the hook
+infrastructure to run their own hooks; see <<WRAPPERS>> for more guidance.
+
+In general, when instructions suggest adding a script to
+`.git/hooks/<hook-event>`, you can specify it in the config instead by running:
+
+----
+git config hook.<some-name>.command <path-to-script>
+git config --add hook.<some-name>.event <hook-event>
+----
+
+This way you can share the script between multiple repos. That is, `cp
+~/my-script.sh ~/project/.git/hooks/pre-commit` would become:
+
+----
+git config hook.my-script.command ~/my-script.sh
+git config --add hook.my-script.event pre-commit
+----
+
 SUBCOMMANDS
 -----------
 
 run::
-	Run the `<hook-name>` hook. See linkgit:githooks[5] for
-	the hook names we support.
+	Runs hooks configured for `<hook-name>`, in the order they are
+	discovered during the config parse.
 +
 Any positional arguments to the hook should be passed after an
 optional `--` (or `--end-of-options`, see linkgit:gitcli[7]). The
@@ -58,6 +148,47 @@ config is not specified, uses the number of CPUs on the current system. Some
 hooks may be ineligible for parallelization: for example, 'commit-msg' intends
 hooks modify the commit message body and cannot be parallelized.
 
+[[WRAPPERS]]
+WRAPPERS
+--------
+
+`git hook run` has been designed to make it easy for tools which wrap Git to
+configure and execute hooks using the Git hook infrastructure. It is possible to
+provide arguments, environment variables (TODO this is missing from reroll TODO),
+and stdin via the command line, as well as specifying parallel or series
+execution if the user has provided multiple hooks.
+
+Assuming your wrapper wants to support a hook named "mywrapper-start-tests", you
+can have your users specify their hooks like so:
+
+  [hook "setup-test-dashboard"]
+    event = mywrapper-start-tests
+    command = ~/mywrapper/setup-dashboard.py --tap
+
+Then, in your 'mywrapper' tool, you can invoke any users' configured hooks by
+running:
+
+----
+git hook run mywrapper-start-tests \
+  # providing something to stdin
+  --stdin some-tempfile-123 \
+  # setting an env var (TODO THIS IS MISSING TODO)
+  --env MYWRAPPER_EXECUTION_MODE=foo \
+  # execute hooks in serial
+  --jobs 1 \
+  # plus some arguments of your own...
+  -- \
+  --testname bar \
+  baz
+----
+
+Take care to name your wrapper's hook events in a way which is unlikely to
+overlap with Git's native hooks (see linkgit:githooks[5]) - a hook event named
+`mywrappertool-validate-commit` is much less likely to be added to native Git
+than a hook event named `validate-commit`. If Git begins to use a hook event
+named the same thing as your wrapper hook, it may invoke your users' hooks in
+unintended and unsupported ways.
+
 CONFIGURATION
 -------------
 include::config/hook.txt[]
diff --git a/builtin/hook.c b/builtin/hook.c
index 1e6b15d565a..bb8fdde6bad 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -61,7 +61,8 @@ static int list(int argc, const char **argv, const char *prefix)
 		struct hook *item = list_entry(pos, struct hook, list);
 		item = list_entry(pos, struct hook, list);
 		if (item)
-			printf("%s\n", item->hook_path);
+			printf("%s\n", item->name ? item->name
+						  : _("hook from hookdir"));
 	}
 
 cleanup:
diff --git a/hook.c b/hook.c
index 600030c59ec..b825fa7c7ae 100644
--- a/hook.c
+++ b/hook.c
@@ -8,10 +8,54 @@ static void free_hook(struct hook *ptr)
 	if (!ptr)
 		return;
 
+	free(ptr->name);
 	free(ptr->feed_pipe_cb_data);
 	free(ptr);
 }
 
+/*
+ * Walks the linked list at 'head' to check if any hook named 'name'
+ * already exists. Returns a pointer to that hook if so, otherwise returns NULL.
+ */
+static struct hook *find_hook_by_name(struct list_head *head,
+					 const char *name)
+{
+	struct list_head *pos = NULL, *tmp = NULL;
+	struct hook *found = NULL;
+
+	list_for_each_safe(pos, tmp, head) {
+		struct hook *it = list_entry(pos, struct hook, list);
+		if (!strcmp(it->name, name)) {
+			list_del(pos);
+			found = it;
+			break;
+		}
+	}
+	return found;
+}
+
+/*
+ * Adds a hook if it's not already in the list, or moves it to the tail of the
+ * list if it was already there. name == NULL indicates it's from the hookdir;
+ * just append it in this case.
+ */
+static void append_or_move_hook(struct list_head *head, const char *name)
+{
+	struct hook *to_add = NULL;
+
+	/* if it's not from hookdir, check if the hook is already in the list */
+	if (name)
+		to_add = find_hook_by_name(head, name);
+
+	if (!to_add) {
+		/* adding a new hook, not moving an old one */
+		to_add = xcalloc(1, sizeof(*to_add));
+		to_add->name = xstrdup_or_null(name);
+	}
+
+	list_add_tail(&to_add->list, head);
+}
+
 static void remove_hook(struct list_head *to_remove)
 {
 	struct hook *hook_to_remove = list_entry(to_remove, struct hook, list);
@@ -73,26 +117,72 @@ int hook_exists(const char *name)
 	return exists;
 }
 
+struct hook_config_cb
+{
+	const char *hook_event;
+	struct list_head *list;
+};
+
+/*
+ * Callback for git_config which adds configured hooks to a hook list.  Hooks
+ * can be configured by specifying both hook.<friend-name>.command = <path> and
+ * hook.<friendly-name>.event = <hook-event>.
+ */
+static int hook_config_lookup(const char *key, const char *value, void *cb_data)
+{
+	struct hook_config_cb *data = cb_data;
+	const char *subsection, *parsed_key;
+	size_t subsection_len = 0;
+	struct strbuf subsection_cpy = STRBUF_INIT;
+
+	/*
+	 * Don't bother doing the expensive parse if there's no
+	 * chance that the config matches 'hook.myhook.event = hook_event'.
+	 */
+	if (!value || strcmp(value, data->hook_event))
+		return 0;
+
+	/* Looking for "hook.friendlyname.event = hook_event" */
+	if (parse_config_key(key,
+			    "hook",
+			    &subsection,
+			    &subsection_len,
+			    &parsed_key) ||
+	    strcmp(parsed_key, "event"))
+		return 0;
+
+	/*
+	 * 'subsection' is a pointer to the internals of 'key', which we don't
+	 * own the memory for. Copy it away to the hook list.
+	 */
+	strbuf_add(&subsection_cpy, subsection, subsection_len);
+
+	append_or_move_hook(data->list, subsection_cpy.buf);
+	strbuf_release(&subsection_cpy);
+
+	return 0;
+}
+
 struct list_head *list_hooks(const char *hookname)
 {
 	struct list_head *hook_head = xmalloc(sizeof(struct list_head));
+	struct hook_config_cb cb_data = {
+		.hook_event = hookname,
+		.list = hook_head,
+	};
 
 	INIT_LIST_HEAD(hook_head);
 
 	if (!hookname)
 		BUG("null hookname was provided to hook_list()!");
 
-	if (have_git_dir()) {
-		const char *hook_path = find_hook(hookname);
+	/* Add the hooks from the config, e.g. hook.myhook.event = pre-commit */
+	git_config(hook_config_lookup, &cb_data);
 
-		/* Add the hook from the hookdir */
-		if (hook_path) {
-			struct hook *to_add = xmalloc(sizeof(*to_add));
-			to_add->hook_path = hook_path;
-			to_add->feed_pipe_cb_data = NULL;
-			list_add_tail(&to_add->list, hook_head);
-		}
-	}
+	/* Add the hook from the hookdir. The placeholder makes it easier to
+	 * allocate work in pick_next_hook. */
+	if (find_hook(hookname))
+		append_or_move_hook(hook_head, NULL);
 
 	return hook_head;
 }
@@ -153,11 +243,47 @@ static int pick_next_hook(struct child_process *cp,
 	cp->trace2_hook_name = hook_cb->hook_name;
 	cp->dir = hook_cb->options->dir;
 
+	/*
+	 * to enable oneliners, let config-specified hooks run in shell.
+	 * config-specified hooks have a name.
+	 */
+	cp->use_shell = !!run_me->name;
+
 	/* add command */
-	if (hook_cb->options->absolute_path)
-		strvec_push(&cp->args, absolute_path(run_me->hook_path));
-	else
-		strvec_push(&cp->args, run_me->hook_path);
+	if (run_me->name) {
+		/* ...from config */
+		struct strbuf cmd_key = STRBUF_INIT;
+		char *command = NULL;
+
+		strbuf_addf(&cmd_key, "hook.%s.command", run_me->name);
+		if (git_config_get_string(cmd_key.buf, &command)) {
+			/* TODO test me! */
+			die(_("'hook.%s.command' must be configured "
+			      "or 'hook.%s.event' must be removed; aborting.\n"),
+			    run_me->name, run_me->name);
+		}
+
+		strvec_push(&cp->args, command);
+		free(command);
+		strbuf_release(&cmd_key);
+	} else {
+		/* ...from hookdir. */
+		const char *hook_path = NULL;
+		/*
+		 * At this point we are already running, so don't validate
+		 * whether the hook name is known or not. Validation was
+		 * performed earlier in list_hooks().
+		 */
+		hook_path = find_hook(hook_cb->hook_name);
+		if (!hook_path)
+			BUG("hookdir hook in hook list but no hookdir hook present in filesystem");
+
+		if (hook_cb->options->absolute_path)
+			hook_path = absolute_path(hook_path);
+
+		strvec_push(&cp->args, hook_path);
+	}
+
 
 	/*
 	 * add passed-in argv, without expanding - let the user get back
@@ -187,8 +313,11 @@ static int notify_start_failure(struct strbuf *out,
 
 	hook_cb->rc |= 1;
 
-	strbuf_addf(out, _("Couldn't start hook '%s'\n"),
-		    attempted->hook_path);
+	if (attempted->name)
+		strbuf_addf(out, _("Couldn't start hook '%s'\n"),
+		    attempted->name);
+	else
+		strbuf_addstr(out, _("Couldn't start hook from hooks directory\n"));
 
 	return 1;
 }
diff --git a/hook.h b/hook.h
index fe16ab35028..4b728991089 100644
--- a/hook.h
+++ b/hook.h
@@ -7,8 +7,11 @@
 
 struct hook {
 	struct list_head list;
-	/* The path to the hook */
-	const char *hook_path;
+	/*
+	 * The friendly name of the hook. NULL indicates the hook is from the
+	 * hookdir.
+	 */
+	char *name;
 
 	/*
 	 * Use this to keep state for your feed_pipe_fn if you are using
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
index 7a1dae4e95e..68e7ff7de7e 100755
--- a/t/t1800-hook.sh
+++ b/t/t1800-hook.sh
@@ -1,13 +1,30 @@
 #!/bin/sh
 
-test_description='git-hook command'
+test_description='git-hook command and config-managed multihooks'
 
 . ./test-lib.sh
 
+setup_hooks () {
+	test_config hook.ghi.command "/path/ghi"
+	test_config hook.ghi.event pre-commit --add
+	test_config hook.ghi.event test-hook --add
+	test_config_global hook.def.command "/path/def"
+	test_config_global hook.def.event pre-commit --add
+}
+
+setup_hookdir () {
+	mkdir .git/hooks
+	write_script .git/hooks/pre-commit <<-EOF
+	echo \"Legacy Hook\"
+	EOF
+	test_when_finished rm -rf .git/hooks
+}
+
 test_expect_success 'git hook usage' '
 	test_expect_code 129 git hook &&
 	test_expect_code 129 git hook run &&
 	test_expect_code 129 git hook run -h &&
+	test_expect_code 129 git hook list -h &&
 	test_expect_code 129 git hook run --unknown 2>err &&
 	test_expect_code 129 git hook list &&
 	test_expect_code 129 git hook list -h &&
@@ -105,7 +122,7 @@ test_expect_success 'git hook list: does-not-exist hook' '
 
 test_expect_success 'git hook list: existing hook' '
 	cat >expect <<-\EOF &&
-	.git/hooks/test-hook
+	hook from hookdir
 	EOF
 	git hook list test-hook >actual &&
 	test_cmp expect actual
@@ -162,4 +179,147 @@ test_expect_success 'stdin to hooks' '
 	test_cmp expect actual
 '
 
+test_expect_success 'git hook list orders by config order' '
+	setup_hooks &&
+
+	cat >expected <<-\EOF &&
+	def
+	ghi
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'git hook list reorders on duplicate event declarations' '
+	setup_hooks &&
+
+	# 'def' is usually configured globally; move it to the end by
+	# configuring it locally.
+	test_config hook.def.event "pre-commit" --add &&
+
+	cat >expected <<-\EOF &&
+	ghi
+	def
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'hook can be configured for multiple events' '
+	setup_hooks &&
+
+	# 'ghi' should be included in both 'pre-commit' and 'test-hook'
+	git hook list pre-commit >actual &&
+	grep "ghi" actual &&
+	git hook list test-hook >actual &&
+	grep "ghi" actual
+'
+
+test_expect_success 'git hook list shows hooks from the hookdir' '
+	setup_hookdir &&
+
+	cat >expected <<-\EOF &&
+	hook from hookdir
+	EOF
+
+	git hook list pre-commit >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'inline hook definitions execute oneliners' '
+	test_config hook.oneliner.event "pre-commit" &&
+	test_config hook.oneliner.command "echo \"Hello World\"" &&
+
+	echo "Hello World" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'inline hook definitions resolve paths' '
+	write_script sample-hook.sh <<-\EOF &&
+	echo \"Sample Hook\"
+	EOF
+
+	test_when_finished "rm sample-hook.sh" &&
+
+	test_config hook.sample-hook.event pre-commit &&
+	test_config hook.sample-hook.command "\"$(pwd)/sample-hook.sh\"" &&
+
+	echo \"Sample Hook\" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'hookdir hook included in git hook run' '
+	setup_hookdir &&
+
+	echo \"Legacy Hook\" >expected &&
+
+	# hooks are run with stdout_to_stderr = 1
+	git hook run pre-commit 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'stdin to multiple hooks' '
+	test_config hook.stdin-a.event "test-hook" --add &&
+	test_config hook.stdin-a.command "xargs -P1 -I% echo a%" --add &&
+	test_config hook.stdin-b.event "test-hook" --add &&
+	test_config hook.stdin-b.command "xargs -P1 -I% echo b%" --add &&
+
+	cat >input <<-\EOF &&
+	1
+	2
+	3
+	EOF
+
+	cat >expected <<-\EOF &&
+	a1
+	a2
+	a3
+	b1
+	b2
+	b3
+	EOF
+
+	git hook run --to-stdin=input test-hook 2>actual &&
+	test_cmp expected actual
+'
+
+test_expect_success 'multiple hooks in series' '
+	test_config hook.series-1.event "test-hook" &&
+	test_config hook.series-1.command "echo 1" --add &&
+	test_config hook.series-2.event "test-hook" &&
+	test_config hook.series-2.command "echo 2" --add &&
+	mkdir .git/hooks &&
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo 3
+	EOF
+
+	cat >expected <<-\EOF &&
+	1
+	2
+	3
+	EOF
+
+	git hook run -j1 test-hook 2>actual &&
+	test_cmp expected actual &&
+
+	rm -rf .git/hooks
+'
+
+test_expect_success 'rejects hooks with no commands configured' '
+	test_config hook.broken.event "test-hook" &&
+
+	echo broken >expected &&
+	git hook list test-hook >actual &&
+	test_cmp expected actual &&
+	test_must_fail git hook run test-hook
+'
+
 test_done
-- 
2.33.0.867.g88ec4638586


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4 5/5] hook: allow out-of-repo 'git hook' invocations
  2021-09-09 12:41                 ` [PATCH v4 0/5] " Ævar Arnfjörð Bjarmason
                                     ` (3 preceding siblings ...)
  2021-09-09 12:42                   ` [PATCH v4 4/5] hook: include hooks from the config Ævar Arnfjörð Bjarmason
@ 2021-09-09 12:42                   ` Ævar Arnfjörð Bjarmason
  4 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-09 12:42 UTC (permalink / raw)
  To: git
  Cc: Emily Shaffer, Junio C Hamano, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee

From: Emily Shaffer <emilyshaffer@google.com>

Since hooks can now be supplied via the config, and a config can be
present without a gitdir via the global and system configs, we can start
to allow 'git hook run' to occur without a gitdir. This enables us to do
things like run sendemail-validate hooks when running 'git send-email'
from a nongit directory.

It still doesn't make sense to look for hooks in the hookdir in nongit
repos, though, as there is no hookdir.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 git.c           |  2 +-
 hook.c          |  2 +-
 t/t1800-hook.sh | 20 +++++++++++++++-----
 3 files changed, 17 insertions(+), 7 deletions(-)

diff --git a/git.c b/git.c
index 540909c391f..39988ee3b02 100644
--- a/git.c
+++ b/git.c
@@ -538,7 +538,7 @@ static struct cmd_struct commands[] = {
 	{ "grep", cmd_grep, RUN_SETUP_GENTLY },
 	{ "hash-object", cmd_hash_object },
 	{ "help", cmd_help },
-	{ "hook", cmd_hook, RUN_SETUP },
+	{ "hook", cmd_hook, RUN_SETUP_GENTLY },
 	{ "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "init", cmd_init_db },
 	{ "init-db", cmd_init_db },
diff --git a/hook.c b/hook.c
index b825fa7c7ae..dc3033cb4c7 100644
--- a/hook.c
+++ b/hook.c
@@ -181,7 +181,7 @@ struct list_head *list_hooks(const char *hookname)
 
 	/* Add the hook from the hookdir. The placeholder makes it easier to
 	 * allocate work in pick_next_hook. */
-	if (find_hook(hookname))
+	if (have_git_dir() && find_hook(hookname))
 		append_or_move_hook(hook_head, NULL);
 
 	return hook_head;
diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
index 68e7ff7de7e..6b6ba30e88e 100755
--- a/t/t1800-hook.sh
+++ b/t/t1800-hook.sh
@@ -128,15 +128,25 @@ test_expect_success 'git hook list: existing hook' '
 	test_cmp expect actual
 '
 
-test_expect_success 'git hook run -- out-of-repo runs excluded' '
-	write_script .git/hooks/test-hook <<-EOF &&
-	echo Test hook
-	EOF
+test_expect_success 'git hook run: out-of-repo runs execute global hooks' '
+	test_config_global hook.global-hook.event test-hook --add &&
+	test_config_global hook.global-hook.command "echo no repo no problems" --add &&
+
+	echo "global-hook" >expect &&
+	nongit git hook list test-hook >actual &&
+	test_cmp expect actual &&
+
+	echo "no repo no problems" >expect &&
 
-	nongit test_must_fail git hook run test-hook
+	nongit git hook run test-hook 2>actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
+	write_script .git/hooks/test-hook <<-EOF &&
+	echo Test hook
+	EOF
+
 	mkdir my-hooks &&
 	write_script my-hooks/test-hook <<-\EOF &&
 	echo Hook ran $1 >>actual
-- 
2.33.0.867.g88ec4638586


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* [PATCH v4] fixup! hook: run a list of hooks instead
  2021-09-09 12:41                   ` [PATCH v4 1/5] hook: run a list of hooks instead Ævar Arnfjörð Bjarmason
@ 2021-09-09 12:59                     ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-09 12:59 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer, Junio C Hamano, Ævar Arnfjörð Bjarmason

---

And of course just as I sent out
<cover-v4-0.5-00000000000-20210909T122802Z-avarab@gmail.com> I noticed
that there was one last memory leak left, it just wasn't stressed by
t1800-hook.sh, but some of the test of the tests, including
t0000-basic.sh!

That test coverage should be improved, but in the meantime Junio:
Here's a fixup to the just-sent you can hopefully squash into the
1/5. Sorry about the screwup.

 hook.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/hook.c b/hook.c
index dc3033cb4c7..4b4205bce6c 100644
--- a/hook.c
+++ b/hook.c
@@ -416,8 +416,10 @@ int run_hooks_oneshot(const char *hook_name, struct run_hooks_opt *options)
 	 * If you need to act on a missing hook, use run_found_hooks()
 	 * instead
 	 */
-	if (list_empty(hooks))
+	if (list_empty(hooks)) {
+		clear_hook_list(hooks);
 		goto cleanup;
+	}
 
 	ret = run_hooks(hook_name, hooks, options);
 
-- 
2.33.0.867.g88ec4638586


^ permalink raw reply related	[flat|nested] 479+ messages in thread

* ab/config-based-hooks-N status (was Re: [PATCH v5 24/36] run-command: add stdin callback for parallelization)
  2021-09-02 13:11               ` [PATCH v5 24/36] run-command: add stdin callback for parallelization Ævar Arnfjörð Bjarmason
@ 2021-10-06 11:03                 ` Ævar Arnfjörð Bjarmason
  2021-10-12 12:59                   ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-10-06 11:03 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason, git


On Thu, Sep 02 2021, Ævar Arnfjörð Bjarmason wrote:

Emily, there's a...:

> diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c
> index 7ae03dc7123..9348184d303 100644
> --- a/t/helper/test-run-command.c
> +++ b/t/helper/test-run-command.c
> @@ -32,8 +32,13 @@ static int parallel_next(struct child_process *cp,
>  		return 0;
>  
>  	strvec_pushv(&cp->args, d->argv);
> +	cp->in = d->in;
> +	cp->no_stdin = d->no_stdin;
>  	strbuf_addstr(err, "preloaded output of a child\n");
>  	number_callbacks++;
> +
> +	*task_cb = xmalloc(sizeof(int));
> +	*(int*)(*task_cb) = 2;
>  	return 1;
>  }

Probably trivial to solve failure here in t0061-run-command.sh if you
compile with SANITIZE=leak. This failed in combination with my[1] (but
for anyone reading along, this patch has been ejected from "seen" a
while ago).

More generally: The equivalent of 01-07/36 of this series is being
merged into "next". As described in a plan to submit this topic
incrementally I was hoping to do 08-20/36 next, i.e. up to "run-command:
remove old run_hook_{le,ve}() hook API". See [2] for that plan.

You've been inactive on-list recently, it would be nice to time this so
that by the time it gets to 21-36/36 (which I was planning to split in
two per [2]) that you'd have time to review/help with outstanding issues
etc, for eventually re-submitting your "config based hooks" on top once
this all lands.

1. https://lore.kernel.org/git/patch-02.10-9a8804e1d9a-20211006T094705Z-avarab@gmail.com/
2. https://lore.kernel.org/git/875yut8nns.fsf@evledraar.gmail.com/

^ permalink raw reply	[flat|nested] 479+ messages in thread

* Re: ab/config-based-hooks-N status (was Re: [PATCH v5 24/36] run-command: add stdin callback for parallelization)
  2021-10-06 11:03                 ` ab/config-based-hooks-N status (was Re: [PATCH v5 24/36] run-command: add stdin callback for parallelization) Ævar Arnfjörð Bjarmason
@ 2021-10-12 12:59                   ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 479+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-10-12 12:59 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: Junio C Hamano, Emily Shaffer, Jeff King, Taylor Blau,
	Felipe Contreras, Eric Sunshine, brian m . carlson,
	Josh Steadmon, Jonathan Tan, Derrick Stolee,
	Ævar Arnfjörð Bjarmason, git


On Wed, Oct 06 2021, Ævar Arnfjörð Bjarmason wrote:

> On Thu, Sep 02 2021, Ævar Arnfjörð Bjarmason wrote:
>
> Emily, there's a...:
>
>> diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c
>> index 7ae03dc7123..9348184d303 100644
>> --- a/t/helper/test-run-command.c
>> +++ b/t/helper/test-run-command.c
>> @@ -32,8 +32,13 @@ static int parallel_next(struct child_process *cp,
>>  		return 0;
>>  
>>  	strvec_pushv(&cp->args, d->argv);
>> +	cp->in = d->in;
>> +	cp->no_stdin = d->no_stdin;
>>  	strbuf_addstr(err, "preloaded output of a child\n");
>>  	number_callbacks++;
>> +
>> +	*task_cb = xmalloc(sizeof(int));
>> +	*(int*)(*task_cb) = 2;
>>  	return 1;
>>  }
>
> Probably trivial to solve failure here in t0061-run-command.sh if you
> compile with SANITIZE=leak. This failed in combination with my[1] (but
> for anyone reading along, this patch has been ejected from "seen" a
> while ago).
>
> More generally: The equivalent of 01-07/36 of this series is being
> merged into "next". As described in a plan to submit this topic
> incrementally I was hoping to do 08-20/36 next, i.e. up to "run-command:
> remove old run_hook_{le,ve}() hook API". See [2] for that plan.
>
> You've been inactive on-list recently, it would be nice to time this so
> that by the time it gets to 21-36/36 (which I was planning to split in
> two per [2]) that you'd have time to review/help with outstanding issues
> etc, for eventually re-submitting your "config based hooks" on top once
> this all lands.
>
> 1. https://lore.kernel.org/git/patch-02.10-9a8804e1d9a-20211006T094705Z-avarab@gmail.com/
> 2. https://lore.kernel.org/git/875yut8nns.fsf@evledraar.gmail.com/

Since I had a reason to look at this again, this fixes it. I've squashed
it into my "base" branch, but it won't be in the next batch I submit (I
have the cut-off point before this commit):

diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c
index fa25bcbbc0d..e9b4214d163 100644
--- a/t/helper/test-run-command.c
+++ b/t/helper/test-run-command.c
@@ -61,11 +61,21 @@ static void test_consume_sideband(struct strbuf *output, void *cb)
 	fclose(sideband);
 }
 
+static int task_free(int result,
+		     struct strbuf *err,
+		     void *pp_cb,
+		     void *pp_task_cb)
+{
+	free(pp_task_cb);
+	return 0;
+}
+
 static int task_finished(int result,
 			 struct strbuf *err,
 			 void *pp_cb,
 			 void *pp_task_cb)
 {
+	task_free(0, NULL, NULL, pp_task_cb);
 	strbuf_addstr(err, "asking for a quick stop\n");
 	return 1;
 }
@@ -438,7 +448,7 @@ int cmd__run_command(int argc, const char **argv)
 
 	if (!strcmp(argv[1], "run-command-parallel"))
 		exit(run_processes_parallel(jobs, parallel_next,
-					    NULL, NULL, NULL, NULL, &proc));
+					    NULL, NULL, NULL, task_free, &proc));
 
 	if (!strcmp(argv[1], "run-command-abort"))
 		exit(run_processes_parallel(jobs, parallel_next,
@@ -452,12 +462,12 @@ int cmd__run_command(int argc, const char **argv)
 		proc.in = -1;
 		proc.no_stdin = 0;
 		exit (run_processes_parallel(jobs, parallel_next, NULL,
-					     test_stdin, NULL, NULL, &proc));
+					     test_stdin, NULL, task_free, &proc));
 	}
 
 	if (!strcmp(argv[1], "run-command-sideband"))
 		exit(run_processes_parallel(jobs, parallel_next, NULL, NULL,
-					    test_consume_sideband, NULL,
+					    test_consume_sideband, task_free,
 					    &proc));
 
 	fprintf(stderr, "check usage\n");

^ permalink raw reply related	[flat|nested] 479+ messages in thread

end of thread, other threads:[~2021-10-12 13:00 UTC | newest]

Thread overview: 479+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-11  2:10 [PATCH v8 00/37] config-based hooks Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 01/37] doc: propose hooks managed by the config Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 02/37] hook: scaffolding for git-hook subcommand Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 03/37] hook: add list command Emily Shaffer
2021-03-12  8:20   ` Ævar Arnfjörð Bjarmason
2021-03-24 17:31     ` Emily Shaffer
2021-03-25 12:36       ` Ævar Arnfjörð Bjarmason
2021-03-11  2:10 ` [PATCH v8 04/37] hook: include hookdir hook in list Emily Shaffer
2021-03-12  8:30   ` Ævar Arnfjörð Bjarmason
2021-03-24 17:56     ` Emily Shaffer
2021-03-24 19:11       ` Junio C Hamano
2021-03-24 19:23         ` Eric Sunshine
2021-03-24 20:07           ` Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 05/37] hook: teach hook.runHookDir Emily Shaffer
2021-03-12  8:33   ` Ævar Arnfjörð Bjarmason
2021-03-24 18:46     ` Emily Shaffer
2021-03-24 22:38       ` Ævar Arnfjörð Bjarmason
2021-03-11  2:10 ` [PATCH v8 06/37] hook: implement hookcmd.<name>.skip Emily Shaffer
2021-03-12  8:49   ` Ævar Arnfjörð Bjarmason
2021-03-11  2:10 ` [PATCH v8 07/37] parse-options: parse into strvec Emily Shaffer
2021-03-12  8:50   ` Ævar Arnfjörð Bjarmason
2021-03-24 20:34     ` Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 08/37] hook: add 'run' subcommand Emily Shaffer
2021-03-12  8:54   ` Ævar Arnfjörð Bjarmason
2021-03-24 21:29     ` Emily Shaffer
2021-03-12 10:22   ` Junio C Hamano
2021-03-11  2:10 ` [PATCH v8 09/37] hook: introduce hook_exists() Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 10/37] hook: support passing stdin to hooks Emily Shaffer
2021-03-12  9:00   ` Ævar Arnfjörð Bjarmason
2021-03-12 10:22   ` Junio C Hamano
2021-03-11  2:10 ` [PATCH v8 11/37] run-command: allow stdin for run_processes_parallel Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 12/37] hook: allow parallel hook execution Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 13/37] hook: allow specifying working directory for hooks Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 14/37] run-command: add stdin callback for parallelization Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 15/37] hook: provide stdin by string_list or callback Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 16/37] run-command: allow capturing of collated output Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 17/37] hooks: allow callers to capture output Emily Shaffer
2021-03-12  9:08   ` Ævar Arnfjörð Bjarmason
2021-03-24 21:54     ` Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 18/37] commit: use config-based hooks Emily Shaffer
2021-03-12 10:22   ` Junio C Hamano
2021-03-11  2:10 ` [PATCH v8 19/37] am: convert applypatch hooks to use config Emily Shaffer
2021-03-12 10:23   ` Junio C Hamano
2021-03-29 23:39     ` Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 20/37] merge: use config-based hooks for post-merge hook Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 21/37] gc: use hook library for pre-auto-gc hook Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 22/37] rebase: teach pre-rebase to use hook.h Emily Shaffer
2021-03-12 10:24   ` Junio C Hamano
2021-03-11  2:10 ` [PATCH v8 23/37] read-cache: convert post-index-change hook to use config Emily Shaffer
2021-03-12 10:22   ` Junio C Hamano
2021-03-29 23:56     ` Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 24/37] receive-pack: convert push-to-checkout hook to hook.h Emily Shaffer
2021-03-12 10:24   ` Junio C Hamano
2021-03-29 23:59     ` Emily Shaffer
2021-03-30  0:10       ` Junio C Hamano
2021-03-11  2:10 ` [PATCH v8 25/37] git-p4: use 'git hook' to run hooks Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 26/37] hooks: convert 'post-checkout' hook to hook library Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 27/37] hook: convert 'post-rewrite' hook to config Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 28/37] transport: convert pre-push hook to use config Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 29/37] reference-transaction: look for hooks in config Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 30/37] receive-pack: convert 'update' hook to hook.h Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 31/37] proc-receive: acquire hook list from hook.h Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 32/37] post-update: use hook.h library Emily Shaffer
2021-03-12  9:14   ` Ævar Arnfjörð Bjarmason
2021-03-30  0:01     ` Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 33/37] receive-pack: convert receive hooks to hook.h Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 34/37] bugreport: use hook_exists instead of find_hook Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 35/37] git-send-email: use 'git hook run' for 'sendemail-validate' Emily Shaffer
2021-03-12  9:21   ` Ævar Arnfjörð Bjarmason
2021-03-30  0:03     ` Emily Shaffer
2021-03-31 21:47     ` Emily Shaffer
2021-03-31 22:06       ` Junio C Hamano
2021-04-01 18:08         ` Emily Shaffer
2021-04-01 18:55           ` Junio C Hamano
2021-04-02 11:34       ` [PATCH 0/2] git-send-email: refactor duplicate $? checks into a function Ævar Arnfjörð Bjarmason
2021-04-02 11:34         ` [PATCH 1/2] git-send-email: replace "map" in void context with "for" Ævar Arnfjörð Bjarmason
2021-04-02 21:31           ` Junio C Hamano
2021-04-02 21:37             ` Emily Shaffer
2021-04-02 11:34         ` [PATCH 2/2] git-send-email: refactor duplicate $? checks into a function Ævar Arnfjörð Bjarmason
2021-04-02 21:36           ` Junio C Hamano
2021-04-04  9:19         ` [PATCH v2 0/4] refactor duplicate $? checks into a function + improve errors Ævar Arnfjörð Bjarmason
2021-04-04  9:19           ` [PATCH v2 1/4] git-send-email: replace "map" in void context with "for" Ævar Arnfjörð Bjarmason
2021-04-04  9:19           ` [PATCH v2 2/4] git-send-email: refactor duplicate $? checks into a function Ævar Arnfjörð Bjarmason
2021-04-05 19:11             ` Junio C Hamano
2021-04-05 23:47             ` Junio C Hamano
2021-04-08 22:43               ` Junio C Hamano
2021-04-08 22:46                 ` Junio C Hamano
2021-04-08 23:54                   ` Ævar Arnfjörð Bjarmason
2021-04-09  0:08                     ` Junio C Hamano
2021-05-03 20:30                       ` Emily Shaffer
2021-04-04  9:19           ` [PATCH v2 3/4] git-send-email: test full --validate output Ævar Arnfjörð Bjarmason
2021-04-04  9:19           ` [PATCH v2 4/4] git-send-email: improve --validate error output Ævar Arnfjörð Bjarmason
2021-04-05 19:14             ` Junio C Hamano
2021-04-06 14:00           ` [PATCH v3 0/3] refactor duplicate $? checks into a function + improve errors Ævar Arnfjörð Bjarmason
2021-04-06 14:00             ` [PATCH v3 1/3] git-send-email: test full --validate output Ævar Arnfjörð Bjarmason
2021-04-06 14:00             ` [PATCH v3 2/3] git-send-email: refactor duplicate $? checks into a function Ævar Arnfjörð Bjarmason
2021-04-06 14:00             ` [PATCH v3 3/3] git-send-email: improve --validate error output Ævar Arnfjörð Bjarmason
2021-04-06 20:33             ` [PATCH v3 0/3] refactor duplicate $? checks into a function + improve errors Junio C Hamano
2021-03-12 23:29   ` [PATCH v8 35/37] git-send-email: use 'git hook run' for 'sendemail-validate' Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 36/37] run-command: stop thinking about hooks Emily Shaffer
2021-03-12  9:23   ` Ævar Arnfjörð Bjarmason
2021-03-30  0:07     ` Emily Shaffer
2021-03-11  2:10 ` [PATCH v8 37/37] docs: unify githooks and git-hook manpages Emily Shaffer
2021-03-12  9:29   ` Ævar Arnfjörð Bjarmason
2021-03-30  0:10     ` Emily Shaffer
2021-04-07  2:36   ` Junio C Hamano
2021-04-08 20:20     ` Jeff Hostetler
2021-04-08 21:17       ` Junio C Hamano
2021-04-08 23:46     ` Emily Shaffer
2021-04-09  0:03       ` Junio C Hamano
2021-03-11 22:26 ` [PATCH v8 00/37] config-based hooks Junio C Hamano
2021-03-12 23:27   ` Emily Shaffer
2021-03-12  9:49 ` Ævar Arnfjörð Bjarmason
2021-03-17 18:41   ` Emily Shaffer
2021-03-17 19:16     ` Emily Shaffer
2021-03-12 11:13 ` Ævar Arnfjörð Bjarmason
2021-03-25 12:41   ` Ævar Arnfjörð Bjarmason
2021-05-27  0:08 ` [PATCH v9 00/37] propose " Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 01/37] doc: propose hooks managed by the config Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 02/37] hook: introduce git-hook subcommand Emily Shaffer
2021-05-27  2:18     ` Junio C Hamano
2021-05-27  0:08   ` [PATCH v9 03/37] hook: include hookdir hook in list Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 04/37] hook: teach hook.runHookDir Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 05/37] hook: implement hookcmd.<name>.skip Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 06/37] parse-options: parse into strvec Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 07/37] hook: add 'run' subcommand Emily Shaffer
2021-06-03  9:07     ` Ævar Arnfjörð Bjarmason
2021-06-03 22:29       ` Junio C Hamano
2021-05-27  0:08   ` [PATCH v9 08/37] hook: introduce hook_exists() Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 09/37] hook: support passing stdin to hooks Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 10/37] run-command: allow stdin for run_processes_parallel Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 11/37] hook: allow parallel hook execution Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 12/37] hook: allow specifying working directory for hooks Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 13/37] run-command: add stdin callback for parallelization Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 14/37] hook: provide stdin by string_list or callback Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 15/37] run-command: allow capturing of collated output Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 16/37] hooks: allow callers to capture output Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 17/37] commit: use config-based hooks Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 18/37] am: convert applypatch hooks to use config Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 19/37] merge: use config-based hooks for post-merge hook Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 20/37] gc: use hook library for pre-auto-gc hook Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 21/37] rebase: teach pre-rebase to use hook.h Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 22/37] read-cache: convert post-index-change hook to use config Emily Shaffer
2021-05-27 23:04     ` Ævar Arnfjörð Bjarmason
2021-05-28  1:09       ` Taylor Blau
2021-05-31 19:21       ` Felipe Contreras
2021-05-27  0:08   ` [PATCH v9 23/37] receive-pack: convert push-to-checkout hook to hook.h Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 24/37] git-p4: use 'git hook' to run hooks Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 25/37] hooks: convert 'post-checkout' hook to hook library Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 26/37] hook: convert 'post-rewrite' hook to config Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 27/37] transport: convert pre-push hook to use config Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 28/37] reference-transaction: look for hooks in config Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 29/37] receive-pack: convert 'update' hook to hook.h Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 30/37] proc-receive: acquire hook list from hook.h Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 31/37] post-update: use hook.h library Emily Shaffer
2021-06-14  9:09     ` Ævar Arnfjörð Bjarmason
2021-05-27  0:08   ` [PATCH v9 32/37] receive-pack: convert receive hooks to hook.h Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 33/37] bugreport: use hook_exists instead of find_hook Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 34/37] git-send-email: use 'git hook run' for 'sendemail-validate' Emily Shaffer
2021-05-27 11:56     ` Ævar Arnfjörð Bjarmason
2021-05-27  0:08   ` [PATCH v9 35/37] run-command: stop thinking about hooks Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 36/37] doc: clarify fsmonitor-watchman specification Emily Shaffer
2021-05-27  0:08   ` [PATCH v9 37/37] docs: link githooks and git-hook manpages Emily Shaffer
2021-06-03  9:18     ` Ævar Arnfjörð Bjarmason
2021-05-27 11:49   ` [PATCH v9 00/37] propose config-based hooks Ævar Arnfjörð Bjarmason
2021-05-27 13:36   ` Ævar Arnfjörð Bjarmason
2021-05-27 17:46     ` Felipe Contreras
2021-05-28 12:11     ` [PATCH 00/31] minimal restart of "config-based-hooks" Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 01/31] hooks tests: don't leave "actual" nonexisting on failure Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 02/31] gc tests: add a test for the "pre-auto-gc" hook Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 03/31] hook: add 'run' subcommand Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 04/31] run-command.h: move find_hook() to hook.h Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 05/31] hook.c: add a hook_exists() wrapper and use it in bugreport.c Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 06/31] gc: use hook library for pre-auto-gc hook Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 07/31] rebase: teach pre-rebase to use hook.h Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 08/31] am: convert applypatch hooks to use config Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 09/31] hooks: convert 'post-checkout' hook to hook library Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 10/31] merge: use config-based hooks for post-merge hook Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 11/31] git hook run: add an --ignore-missing flag Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 12/31] send-email: use 'git hook run' for 'sendemail-validate' Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 13/31] git-p4: use 'git hook' to run hooks Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 14/31] commit: use hook.h to execute hooks Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 15/31] read-cache: convert post-index-change hook to use config Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 16/31] receive-pack: convert push-to-checkout hook to hook.h Ævar Arnfjörð Bjarmason
2021-06-02  0:51         ` Felipe Contreras
2021-05-28 12:11       ` [PATCH 17/31] run-command: allow stdin for run_processes_parallel Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 18/31] hook: support passing stdin to hooks Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 19/31] am: convert 'post-rewrite' hook to hook.h Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 20/31] run-command: add stdin callback for parallelization Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 21/31] hook: provide stdin by string_list or callback Ævar Arnfjörð Bjarmason
2021-06-02  1:00         ` Felipe Contreras
2021-05-28 12:11       ` [PATCH 22/31] hook: convert 'post-rewrite' hook in sequencer.c to hook.h Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 23/31] transport: convert pre-push hook to use config Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 24/31] reference-transaction: use hook.h to run hooks Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 25/31] run-command: allow capturing of collated output Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 26/31] hooks: allow callers to capture output Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 27/31] receive-pack: convert 'update' hook to hook.h Ævar Arnfjörð Bjarmason
2021-06-02  1:04         ` Felipe Contreras
2021-05-28 12:11       ` [PATCH 28/31] post-update: use hook.h library Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 29/31] receive-pack: convert receive hooks to hook.h Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 30/31] hooks: fix a TOCTOU in "did we run a hook?" heuristic Ævar Arnfjörð Bjarmason
2021-05-28 12:48         ` Bagas Sanjaya
2021-05-28 14:11           ` Ævar Arnfjörð Bjarmason
2021-05-28 12:11       ` [PATCH 31/31] hook-list.h: add a generated list of hooks, like config-list.h Ævar Arnfjörð Bjarmason
2021-06-01  1:00         ` Ævar Arnfjörð Bjarmason
2021-06-01 18:14       ` [PATCH 00/31] minimal restart of "config-based-hooks" Emily Shaffer
2021-06-01 20:50         ` Derrick Stolee
2021-06-02  5:42           ` Felipe Contreras
2021-06-02  7:46             ` Ævar Arnfjörð Bjarmason
2021-06-02  9:34           ` Ævar Arnfjörð Bjarmason
2021-06-02  5:30         ` Felipe Contreras
2021-06-02  7:56         ` Ævar Arnfjörð Bjarmason
2021-06-02  9:39           ` Ævar Arnfjörð Bjarmason
2021-06-25 18:14           ` Felipe Contreras
2021-06-14 10:32       ` [PATCH v2 00/30] Minimal " Ævar Arnfjörð Bjarmason
2021-06-14 10:32         ` [PATCH v2 01/30] hook: add 'run' subcommand Ævar Arnfjörð Bjarmason
2021-06-14 21:33           ` Emily Shaffer
2021-06-15  9:36             ` Ævar Arnfjörð Bjarmason
2021-06-18 22:13               ` Emily Shaffer
2021-06-20 19:30                 ` Ævar Arnfjörð Bjarmason
2021-06-21  3:44                   ` Junio C Hamano
2021-06-22  0:00                   ` Emily Shaffer
2021-06-29  1:12                     ` Junio C Hamano
2021-06-25 19:02                 ` Felipe Contreras
2021-06-25 19:08             ` Felipe Contreras
2021-06-14 10:32         ` [PATCH v2 02/30] run-command.h: move find_hook() to hook.h Ævar Arnfjörð Bjarmason
2021-06-14 10:32         ` [PATCH v2 03/30] hook.c: add a hook_exists() wrapper and use it in bugreport.c Ævar Arnfjörð Bjarmason
2021-06-14 10:32         ` [PATCH v2 04/30] gc: use hook library for pre-auto-gc hook Ævar Arnfjörð Bjarmason
2021-06-14 23:57           ` Emily Shaffer
2021-06-14 10:32         ` [PATCH v2 05/30] rebase: teach pre-rebase to use hook.h Ævar Arnfjörð Bjarmason
2021-06-14 10:32         ` [PATCH v2 06/30] am: convert applypatch hooks to use config Ævar Arnfjörð Bjarmason
2021-06-14 10:32         ` [PATCH v2 07/30] hooks: convert 'post-checkout' hook to hook library Ævar Arnfjörð Bjarmason
2021-06-14 10:32         ` [PATCH v2 08/30] merge: use config-based hooks for post-merge hook Ævar Arnfjörð Bjarmason
2021-06-14 10:32         ` [PATCH v2 09/30] git hook run: add an --ignore-missing flag Ævar Arnfjörð Bjarmason
2021-06-14 10:32         ` [PATCH v2 10/30] send-email: use 'git hook run' for 'sendemail-validate' Ævar Arnfjörð Bjarmason
2021-06-14 10:33         ` [PATCH v2 11/30] git-p4: use 'git hook' to run hooks Ævar Arnfjörð Bjarmason
2021-06-14 10:33         ` [PATCH v2 12/30] commit: use hook.h to execute hooks Ævar Arnfjörð Bjarmason
2021-06-14 10:33         ` [PATCH v2 13/30] read-cache: convert post-index-change hook to use config Ævar Arnfjörð Bjarmason
2021-06-25 18:32           ` Felipe Contreras
2021-06-14 10:33         ` [PATCH v2 14/30] receive-pack: convert push-to-checkout hook to hook.h Ævar Arnfjörð Bjarmason
2021-06-14 10:33         ` [PATCH v2 15/30] run-command: remove old run_hook_{le,ve}() hook API Ævar Arnfjörð Bjarmason
2021-06-25 18:34           ` Felipe Contreras
2021-06-14 10:33         ` [PATCH v2 16/30] run-command: allow stdin for run_processes_parallel Ævar Arnfjörð Bjarmason
2021-06-14 10:33         ` [PATCH v2 17/30] hook: support passing stdin to hooks Ævar Arnfjörð Bjarmason
2021-06-14 10:33         ` [PATCH v2 18/30] am: convert 'post-rewrite' hook to hook.h Ævar Arnfjörð Bjarmason
2021-06-14 10:33         ` [PATCH v2 19/30] run-command: add stdin callback for parallelization Ævar Arnfjörð Bjarmason
2021-06-14 10:33         ` [PATCH v2 20/30] hook: provide stdin by string_list or callback Ævar Arnfjörð Bjarmason
2021-06-14 10:33         ` [PATCH v2 21/30] hook: convert 'post-rewrite' hook in sequencer.c to hook.h Ævar Arnfjörð Bjarmason
2021-06-14 10:33         ` [PATCH v2 22/30] transport: convert pre-push hook to use config Ævar Arnfjörð Bjarmason
2021-06-14 10:33         ` [PATCH v2 23/30] reference-transaction: use hook.h to run hooks Ævar Arnfjörð Bjarmason
2021-06-14 10:33         ` [PATCH v2 24/30] run-command: allow capturing of collated output Ævar Arnfjörð Bjarmason
2021-06-14 10:33         ` [PATCH v2 25/30] hooks: allow callers to capture output Ævar Arnfjörð Bjarmason
2021-06-14 10:33         ` [PATCH v2 26/30] receive-pack: convert 'update' hook to hook.h Ævar Arnfjörð Bjarmason
2021-06-14 10:33         ` [PATCH v2 27/30] post-update: use hook.h library Ævar Arnfjörð Bjarmason
2021-06-14 10:33         ` [PATCH v2 28/30] receive-pack: convert receive hooks to hook.h Ævar Arnfjörð Bjarmason
2021-06-14 10:33         ` [PATCH v2 29/30] hooks: fix a TOCTOU in "did we run a hook?" heuristic Ævar Arnfjörð Bjarmason
2021-06-14 10:33         ` [PATCH v2 30/30] hook-list.h: add a generated list of hooks, like config-list.h Ævar Arnfjörð Bjarmason
2021-06-15 10:02           ` Ævar Arnfjörð Bjarmason
2021-06-14 20:22         ` [PATCH v2 00/30] Minimal restart of "config-based-hooks" Emily Shaffer
2021-06-16  0:45           ` Junio C Hamano
2021-06-17 10:22         ` [PATCH 00/27] Base for "config-based-hooks" Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 01/27] hook: add 'run' subcommand Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 02/27] gc: use hook library for pre-auto-gc hook Ævar Arnfjörð Bjarmason
2021-07-22 21:58             ` Emily Shaffer
2021-06-17 10:22           ` [PATCH 03/27] rebase: teach pre-rebase to use hook.h Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 04/27] am: convert applypatch hooks to use config Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 05/27] hooks: convert 'post-checkout' hook to hook library Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 06/27] merge: use config-based hooks for post-merge hook Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 07/27] git hook run: add an --ignore-missing flag Ævar Arnfjörð Bjarmason
2021-07-02 23:47             ` Emily Shaffer
2021-06-17 10:22           ` [PATCH 08/27] send-email: use 'git hook run' for 'sendemail-validate' Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 09/27] git-p4: use 'git hook' to run hooks Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 10/27] commit: use hook.h to execute hooks Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 11/27] read-cache: convert post-index-change hook to use config Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 12/27] receive-pack: convert push-to-checkout hook to hook.h Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 13/27] run-command: remove old run_hook_{le,ve}() hook API Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 14/27] run-command: allow stdin for run_processes_parallel Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 15/27] hook: support passing stdin to hooks Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 16/27] am: convert 'post-rewrite' hook to hook.h Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 17/27] run-command: add stdin callback for parallelization Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 18/27] hook: provide stdin by string_list or callback Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 19/27] hook: convert 'post-rewrite' hook in sequencer.c to hook.h Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 20/27] transport: convert pre-push hook to use config Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 21/27] reference-transaction: use hook.h to run hooks Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 22/27] run-command: allow capturing of collated output Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 23/27] hooks: allow callers to capture output Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 24/27] receive-pack: convert 'update' hook to hook.h Ævar Arnfjörð Bjarmason
2021-06-17 10:22           ` [PATCH 25/27] post-update: use hook.h library Ævar Arnfjörð Bjarmason
2021-06-17 10:23           ` [PATCH 26/27] receive-pack: convert receive hooks to hook.h Ævar Arnfjörð Bjarmason
2021-06-17 10:23           ` [PATCH 27/27] hooks: fix a TOCTOU in "did we run a hook?" heuristic Ævar Arnfjörð Bjarmason
2021-06-18 22:09             ` Emily Shaffer
2021-07-15 23:25           ` [PATCH 0/9] config-based hooks restarted Emily Shaffer
2021-07-15 23:25             ` [PATCH 1/9] hook: run a list of hooks instead Emily Shaffer
2021-07-15 23:25             ` [PATCH 2/9] hook: allow parallel hook execution Emily Shaffer
2021-07-16  8:36               ` Ævar Arnfjörð Bjarmason
2021-07-22 21:12                 ` Emily Shaffer
2021-07-23  9:30                   ` Ævar Arnfjörð Bjarmason
2021-07-15 23:25             ` [PATCH 3/9] hook: introduce "git hook list" Emily Shaffer
2021-07-16  8:52               ` Ævar Arnfjörð Bjarmason
2021-07-22 22:18                 ` Emily Shaffer
2021-07-23  9:29                   ` Ævar Arnfjörð Bjarmason
2021-07-15 23:25             ` [PATCH 4/9] hook: treat hookdir hook specially Emily Shaffer
2021-07-16  8:58               ` Ævar Arnfjörð Bjarmason
2021-07-22 22:24                 ` Emily Shaffer
2021-07-23  9:26                   ` Ævar Arnfjörð Bjarmason
2021-07-23 17:33                   ` Felipe Contreras
2021-07-23 18:22                     ` Eric Sunshine
2021-07-23 20:02                       ` Felipe Contreras
2021-07-15 23:25             ` [PATCH 5/9] hook: allow running non-native hooks Emily Shaffer
2021-07-15 23:26             ` [PATCH 6/9] hook: include hooks from the config Emily Shaffer
2021-07-16  9:01               ` Ævar Arnfjörð Bjarmason
2021-07-22 22:51                 ` Emily Shaffer
2021-07-23  9:22                   ` Ævar Arnfjörð Bjarmason
2021-07-15 23:26             ` [PATCH 7/9] hook: allow out-of-repo 'git hook' invocations Emily Shaffer
2021-07-16  8:33               ` Ævar Arnfjörð Bjarmason
2021-07-22 23:07                 ` Emily Shaffer
2021-07-23  9:18                   ` Ævar Arnfjörð Bjarmason
2021-07-15 23:26             ` [PATCH 8/9] hook: teach 'hookcmd' config to alias hook scripts Emily Shaffer
2021-07-16  9:13               ` Ævar Arnfjörð Bjarmason
2021-07-22 23:31                 ` Emily Shaffer
2021-07-23  7:41                   ` Ævar Arnfjörð Bjarmason
2021-08-04 20:38                     ` Emily Shaffer
2021-08-05  0:17                       ` Ævar Arnfjörð Bjarmason
2021-08-05 21:45                         ` Emily Shaffer
2021-08-05 22:26                           ` Ævar Arnfjörð Bjarmason
2021-08-06 20:18                             ` Emily Shaffer
2021-08-04 21:49                     ` Jonathan Tan
2021-07-15 23:26             ` [PATCH 9/9] hook: implement hookcmd.<name>.skip Emily Shaffer
2021-07-28 20:39           ` [PATCH 00/27] Base for "config-based-hooks" Emily Shaffer
2021-08-03 19:38           ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 01/36] Makefile: mark "check" target as .PHONY Ævar Arnfjörð Bjarmason
2021-08-20  0:04               ` Emily Shaffer
2021-08-03 19:38             ` [PATCH v4 02/36] Makefile: stop hardcoding {command,config}-list.h Ævar Arnfjörð Bjarmason
2021-08-20  0:03               ` Emily Shaffer
2021-08-24 14:22                 ` Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 03/36] Makefile: remove an out-of-date comment Ævar Arnfjörð Bjarmason
2021-08-20  0:05               ` Emily Shaffer
2021-08-03 19:38             ` [PATCH v4 04/36] hook.[ch]: move find_hook() to this new library Ævar Arnfjörð Bjarmason
2021-08-20  0:08               ` Emily Shaffer
2021-08-03 19:38             ` [PATCH v4 05/36] hook.c: add a hook_exists() wrapper and use it in bugreport.c Ævar Arnfjörð Bjarmason
2021-08-20  0:09               ` Emily Shaffer
2021-08-03 19:38             ` [PATCH v4 06/36] hook.c users: use "hook_exists()" insted of "find_hook()" Ævar Arnfjörð Bjarmason
2021-08-20  0:10               ` Emily Shaffer
2021-08-03 19:38             ` [PATCH v4 07/36] hook-list.h: add a generated list of hooks, like config-list.h Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 08/36] hook: add 'run' subcommand Ævar Arnfjörð Bjarmason
2021-08-04 10:15               ` Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 09/36] gc: use hook library for pre-auto-gc hook Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 10/36] rebase: convert pre-rebase to use hook.h Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 11/36] am: convert applypatch " Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 12/36] hooks: convert 'post-checkout' hook to hook library Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 13/36] merge: convert post-merge to use hook.h Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 14/36] git hook run: add an --ignore-missing flag Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 15/36] send-email: use 'git hook run' for 'sendemail-validate' Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 16/36] git-p4: use 'git hook' to run hooks Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 17/36] commit: convert {pre-commit,prepare-commit-msg} hook to hook.h Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 18/36] read-cache: convert post-index-change to use hook.h Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 19/36] receive-pack: convert push-to-checkout hook to hook.h Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 20/36] run-command: remove old run_hook_{le,ve}() hook API Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 21/36] run-command: allow stdin for run_processes_parallel Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 22/36] hook: support passing stdin to hooks Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 23/36] am: convert 'post-rewrite' hook to hook.h Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 24/36] run-command: add stdin callback for parallelization Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 25/36] hook: provide stdin by string_list or callback Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 26/36] hook: convert 'post-rewrite' hook in sequencer.c to hook.h Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 27/36] transport: convert pre-push hook " Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 28/36] hook tests: test for exact "pre-push" hook input Ævar Arnfjörð Bjarmason
2021-08-20  0:16               ` Emily Shaffer
2021-08-03 19:38             ` [PATCH v4 29/36] hook tests: use a modern style for "pre-push" tests Ævar Arnfjörð Bjarmason
2021-08-20  0:18               ` Emily Shaffer
2021-08-03 19:38             ` [PATCH v4 30/36] reference-transaction: use hook.h to run hooks Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 31/36] run-command: allow capturing of collated output Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 32/36] hooks: allow callers to capture output Ævar Arnfjörð Bjarmason
2021-08-03 19:38             ` [PATCH v4 33/36] receive-pack: convert 'update' hook to hook.h Ævar Arnfjörð Bjarmason
2021-08-03 19:39             ` [PATCH v4 34/36] post-update: use hook.h library Ævar Arnfjörð Bjarmason
2021-08-03 19:39             ` [PATCH v4 35/36] receive-pack: convert receive hooks to hook.h Ævar Arnfjörð Bjarmason
2021-08-03 19:39             ` [PATCH v4 36/36] hooks: fix a TOCTOU in "did we run a hook?" heuristic Ævar Arnfjörð Bjarmason
2021-08-12  0:42             ` [PATCH v2 0/6] config-based hooks restarted Emily Shaffer
2021-08-12  0:42               ` [PATCH v2 1/6] hook: run a list of hooks instead Emily Shaffer
2021-08-12 17:25                 ` Junio C Hamano
2021-08-16 23:38                   ` Emily Shaffer
2021-08-12  0:42               ` [PATCH v2 2/6] hook: allow parallel hook execution Emily Shaffer
2021-08-12 17:51                 ` Junio C Hamano
2021-08-16 23:59                   ` Emily Shaffer
2021-08-12  0:42               ` [PATCH v2 3/6] hook: introduce "git hook list" Emily Shaffer
2021-08-12 18:59                 ` Junio C Hamano
2021-08-17  0:36                   ` Emily Shaffer
2021-08-12  0:42               ` [PATCH v2 4/6] hook: allow running non-native hooks Emily Shaffer
2021-08-12 19:08                 ` Junio C Hamano
2021-08-18 20:51                   ` Emily Shaffer
2021-08-18 21:14                     ` Emily Shaffer
2021-08-18 21:24                       ` Junio C Hamano
2021-08-12  0:42               ` [PATCH v2 5/6] hook: include hooks from the config Emily Shaffer
2021-08-12 20:48                 ` Junio C Hamano
2021-08-19  0:09                   ` Emily Shaffer
2021-08-12  0:42               ` [PATCH v2 6/6] hook: allow out-of-repo 'git hook' invocations Emily Shaffer
2021-08-12  4:47               ` [PATCH v2 0/6] config-based hooks restarted Junio C Hamano
2021-08-12  5:02                 ` Junio C Hamano
2021-08-16 22:31                   ` Emily Shaffer
2021-08-19  3:34               ` [PATCH v3 " Emily Shaffer
2021-08-19  3:34                 ` [PATCH v3 1/6] hook: run a list of hooks instead Emily Shaffer
2021-08-24 14:56                   ` Ævar Arnfjörð Bjarmason
2021-08-26 21:16                     ` Emily Shaffer
2021-08-27 11:15                       ` Ævar Arnfjörð Bjarmason
2021-08-19  3:34                 ` [PATCH v3 2/6] hook: allow parallel hook execution Emily Shaffer
2021-08-24 15:01                   ` Ævar Arnfjörð Bjarmason
2021-08-24 16:13                     ` Eric Sunshine
2021-08-26 22:36                     ` Emily Shaffer
2021-08-19  3:34                 ` [PATCH v3 3/6] hook: introduce "git hook list" Emily Shaffer
2021-08-24 15:08                   ` Ævar Arnfjörð Bjarmason
2021-08-26 21:43                     ` Emily Shaffer
2021-08-24 15:53                   ` Ævar Arnfjörð Bjarmason
2021-08-26 22:38                     ` Emily Shaffer
2021-08-19  3:34                 ` [PATCH v3 4/6] hook: allow running non-native hooks Emily Shaffer
2021-08-24 15:55                   ` Ævar Arnfjörð Bjarmason
2021-08-26 22:50                     ` Emily Shaffer
2021-08-27  0:22                       ` Junio C Hamano
2021-08-19  3:34                 ` [PATCH v3 5/6] hook: include hooks from the config Emily Shaffer
2021-08-19 22:39                   ` Junio C Hamano
2021-08-19 23:43                     ` Emily Shaffer
2021-08-19 23:48                       ` Junio C Hamano
2021-08-24 19:30                   ` Ævar Arnfjörð Bjarmason
2021-08-31 19:05                     ` Emily Shaffer
2021-08-19  3:34                 ` [PATCH v3 6/6] hook: allow out-of-repo 'git hook' invocations Emily Shaffer
2021-08-24 20:12                   ` Ævar Arnfjörð Bjarmason
2021-08-24 20:38                     ` Randall S. Becker
2021-08-24 22:45                       ` Ævar Arnfjörð Bjarmason
2021-08-31 21:09                     ` Emily Shaffer
2021-08-24 20:29                 ` [PATCH v3 0/6] config-based hooks restarted Ævar Arnfjörð Bjarmason
2021-09-02 22:01                   ` Emily Shaffer
2021-09-09 12:41                 ` [PATCH v4 0/5] " Ævar Arnfjörð Bjarmason
2021-09-09 12:41                   ` [PATCH v4 1/5] hook: run a list of hooks instead Ævar Arnfjörð Bjarmason
2021-09-09 12:59                     ` [PATCH v4] fixup! " Ævar Arnfjörð Bjarmason
2021-09-09 12:42                   ` [PATCH v4 2/5] hook: allow parallel hook execution Ævar Arnfjörð Bjarmason
2021-09-09 12:42                   ` [PATCH v4 3/5] hook: introduce "git hook list" Ævar Arnfjörð Bjarmason
2021-09-09 12:42                   ` [PATCH v4 4/5] hook: include hooks from the config Ævar Arnfjörð Bjarmason
2021-09-09 12:42                   ` [PATCH v4 5/5] hook: allow out-of-repo 'git hook' invocations Ævar Arnfjörð Bjarmason
2021-08-19  0:17             ` [PATCH v4 00/36] Run hooks via "git run hook" & hook library Emily Shaffer
2021-08-19 23:40             ` Emily Shaffer
2021-09-02  7:21               ` Ævar Arnfjörð Bjarmason
2021-09-02 13:11             ` [PATCH v5 " Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 01/36] Makefile: mark "check" target as .PHONY Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 02/36] Makefile: stop hardcoding {command,config}-list.h Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 03/36] Makefile: remove an out-of-date comment Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 04/36] hook.[ch]: move find_hook() from run-command.c to hook.c Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 05/36] hook.c: add a hook_exists() wrapper and use it in bugreport.c Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 06/36] hook.c users: use "hook_exists()" instead of "find_hook()" Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 07/36] hook-list.h: add a generated list of hooks, like config-list.h Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 08/36] hook: add 'run' subcommand Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 09/36] gc: use hook library for pre-auto-gc hook Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 10/36] rebase: convert pre-rebase to use hook.h Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 11/36] am: convert applypatch " Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 12/36] hooks: convert 'post-checkout' hook to hook library Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 13/36] merge: convert post-merge to use hook.h Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 14/36] git hook run: add an --ignore-missing flag Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 15/36] send-email: use 'git hook run' for 'sendemail-validate' Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 16/36] git-p4: use 'git hook' to run hooks Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 17/36] commit: convert {pre-commit,prepare-commit-msg} hook to hook.h Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 18/36] read-cache: convert post-index-change to use hook.h Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 19/36] receive-pack: convert push-to-checkout hook to hook.h Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 20/36] run-command: remove old run_hook_{le,ve}() hook API Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 21/36] run-command: allow stdin for run_processes_parallel Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 22/36] hook: support passing stdin to hooks Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 23/36] am: convert 'post-rewrite' hook to hook.h Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 24/36] run-command: add stdin callback for parallelization Ævar Arnfjörð Bjarmason
2021-10-06 11:03                 ` ab/config-based-hooks-N status (was Re: [PATCH v5 24/36] run-command: add stdin callback for parallelization) Ævar Arnfjörð Bjarmason
2021-10-12 12:59                   ` Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 25/36] hook: provide stdin by string_list or callback Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 26/36] hook: convert 'post-rewrite' hook in sequencer.c to hook.h Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 27/36] transport: convert pre-push hook " Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 28/36] hook tests: test for exact "pre-push" hook input Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 29/36] hook tests: use a modern style for "pre-push" tests Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 30/36] reference-transaction: use hook.h to run hooks Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 31/36] run-command: allow capturing of collated output Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 32/36] hooks: allow callers to capture output Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 33/36] receive-pack: convert 'update' hook to hook.h Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 34/36] post-update: use hook.h library Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 35/36] receive-pack: convert receive hooks to hook.h Ævar Arnfjörð Bjarmason
2021-09-02 13:11               ` [PATCH v5 36/36] hooks: fix a TOCTOU in "did we run a hook?" heuristic Ævar Arnfjörð Bjarmason
2021-06-25 18:22         ` [PATCH v2 00/30] Minimal restart of "config-based-hooks" Felipe Contreras

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).