[PATCH] builtin-commit: re-read file index before launching editor

[PATCH] builtin-commit: re-read file index before launching editor

[Samuel Yvon via GitGitGadget]

From: Samuel Yvon <samuelyvon9@gmail.com>

Changes made within a pre-commit hook are not reflected within
the editor (for instance, with `git commit --verbose`) because
the index is re-read after the editor has been used.

This has the consequence of not displaying the actual changes made
by the pre-commit hook, but committing them anyways.

While it is often argued that the hook's purpose is not to automatically
write content to the repository, it is acknowledged that using the
pre-commit to apply mechanical fixes on top of the existing changes
is a supported use case, along with verifying the content.

I think not seeing these mechanical fixes before commiting is incorrect.
A developer might expect the commit to look a certain way but if the
pre-commit goes wrong the invalid changes will go unnoticed until
committed.

Signed-off-by: Samuel Yvon <samuelyvon9@gmail.com>
---
    builtin-commit: Re-read file index before launching editor
    
    Changes made within a pre-commit hook are not reflected within the
    editor (for instance, with git commit --verbose) because the index is
    re-read after the editor has been used.
    
    This has the consequence of not displaying the actual changes made by
    the pre-commit hook, but committing them anyways.
    
    While it is often argued that the hook's purpose is not to automatically
    write content to the repository, it is acknowledged that using the
    pre-commit to apply mechanical fixes on top of the existing changes is a
    supported use case, along with verifying the content.
    
    I think not seeing these mechanical fixes before commiting is incorrect.
    A developer might expect the commit to look a certain way but if the
    pre-commit goes wrong the invalid changes will go unnoticed until
    committed.
    
    I had a small exchange in the Google Group before submitting this Pr.
    Here is a link for cross-referencing:
    https://groups.google.com/g/git-mentoring/c/FsP83I9mN6c
    
    As a side note, I do not know who manages the Github Repo but the
    following description threw me off a little bit:
    
    Git Source Code Mirror - This is a publish-only repository and all pull requests are ignored. 
    
    
    since after looking deeper it seems the PRs are not ignored.
    
    Signed-off-by: Samuel Yvon samuelyvon9@gmail.com

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1127%2FSamuelYvon%2Fmaint-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1127/SamuelYvon/maint-v1
Pull-Request: https://github.com/git/git/pull/1127

 builtin/commit.c | 21 +++++++++++----------
 1 file changed, 11 insertions(+), 10 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index 7c9b1e7be3a..e75b11d1c60 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -728,8 +728,17 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	/* This checks and barfs if author is badly specified */
 	determine_author_info(author_ident);
 
-	if (!no_verify && run_commit_hook(use_editor, index_file, "pre-commit", NULL))
-		return 0;
+	if (!no_verify && find_hook("pre-commit")) {
+		if(run_commit_hook(use_editor, index_file, "pre-commit", NULL))
+			return 0;
+
+		/*
+		 * Re-read the index as pre-commit hook could have updated it,
+		 * and write it out as a tree.
+		 */
+		discard_cache();
+		read_cache_from(index_file);
+	}
 
 	if (squash_message) {
 		/*
@@ -1051,14 +1060,6 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (!no_verify && find_hook("pre-commit")) {
-		/*
-		 * Re-read the index as pre-commit hook could have updated it,
-		 * and write it out as a tree.  We must do this before we invoke
-		 * the editor and after we invoke run_status above.
-		 */
-		discard_cache();
-	}
 	read_cache_from(index_file);
 
 	if (update_main_cache_tree(0)) {

base-commit: 5fbd2fc5997dfa4d4593a862fe729b1e7a89bcf8
-- 
gitgitgadget

Re: [PATCH] builtin-commit: re-read file index before launching editor

[Ævar Arnfjörð Bjarmason]

On Tue, Nov 09 2021, Samuel Yvon via GitGitGadget wrote:

> From: Samuel Yvon <samuelyvon9@gmail.com>
>
> Changes made within a pre-commit hook are not reflected within
> the editor (for instance, with `git commit --verbose`) because
> the index is re-read after the editor has been used.
>
> This has the consequence of not displaying the actual changes made
> by the pre-commit hook, but committing them anyways.
>
> While it is often argued that the hook's purpose is not to automatically
> write content to the repository, it is acknowledged that using the
> pre-commit to apply mechanical fixes on top of the existing changes
> is a supported use case, along with verifying the content.
>
> I think not seeing these mechanical fixes before commiting is incorrect.
> A developer might expect the commit to look a certain way but if the
> pre-commit goes wrong the invalid changes will go unnoticed until
> committed.
>
> Signed-off-by: Samuel Yvon <samuelyvon9@gmail.com>
> ---
>     builtin-commit: Re-read file index before launching editor
>     
>     Changes made within a pre-commit hook are not reflected within the
>     editor (for instance, with git commit --verbose) because the index is
>     re-read after the editor has been used.
>     
>     This has the consequence of not displaying the actual changes made by
>     the pre-commit hook, but committing them anyways.
>     
>     While it is often argued that the hook's purpose is not to automatically
>     write content to the repository, it is acknowledged that using the
>     pre-commit to apply mechanical fixes on top of the existing changes is a
>     supported use case, along with verifying the content.
>     
>     I think not seeing these mechanical fixes before commiting is incorrect.
>     A developer might expect the commit to look a certain way but if the
>     pre-commit goes wrong the invalid changes will go unnoticed until
>     committed.
>     
>     I had a small exchange in the Google Group before submitting this Pr.
>     Here is a link for cross-referencing:
>     https://groups.google.com/g/git-mentoring/c/FsP83I9mN6c
>     
>     As a side note, I do not know who manages the Github Repo but the
>     following description threw me off a little bit:
>     
>     Git Source Code Mirror - This is a publish-only repository and all pull requests are ignored. 
>     
>     
>     since after looking deeper it seems the PRs are not ignored.
>     
>     Signed-off-by: Samuel Yvon samuelyvon9@gmail.com
>
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1127%2FSamuelYvon%2Fmaint-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1127/SamuelYvon/maint-v1
> Pull-Request: https://github.com/git/git/pull/1127
>
>  builtin/commit.c | 21 +++++++++++----------
>  1 file changed, 11 insertions(+), 10 deletions(-)
>
> diff --git a/builtin/commit.c b/builtin/commit.c
> index 7c9b1e7be3a..e75b11d1c60 100644
> --- a/builtin/commit.c
> +++ b/builtin/commit.c
> @@ -728,8 +728,17 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
>  	/* This checks and barfs if author is badly specified */
>  	determine_author_info(author_ident);
>  
> -	if (!no_verify && run_commit_hook(use_editor, index_file, "pre-commit", NULL))
> -		return 0;
> +	if (!no_verify && find_hook("pre-commit")) {
> +		if(run_commit_hook(use_editor, index_file, "pre-commit", NULL))
> +			return 0;
> +
> +		/*
> +		 * Re-read the index as pre-commit hook could have updated it,
> +		 * and write it out as a tree.
> +		 */
> +		discard_cache();
> +		read_cache_from(index_file);
> +	}
>  
>  	if (squash_message) {
>  		/*
> @@ -1051,14 +1060,6 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
>  		return 0;
>  	}
>  
> -	if (!no_verify && find_hook("pre-commit")) {
> -		/*
> -		 * Re-read the index as pre-commit hook could have updated it,
> -		 * and write it out as a tree.  We must do this before we invoke
> -		 * the editor and after we invoke run_status above.
> -		 */
> -		discard_cache();
> -	}
>  	read_cache_from(index_file);
>  
>  	if (update_main_cache_tree(0)) {
>
> base-commit: 5fbd2fc5997dfa4d4593a862fe729b1e7a89bcf8

This hook logic is actively being changed these days, we're currently at
batch 2/3 of 3/3 (or was it 4/4? I forget) to rewrite all these hook
interfaces.

I haven't looked carefully at this, but I've got a patch that touches
this exact logic here that'll be in-flight before too long:
https://lore.kernel.org/git/patch-v5-36.36-fe056098534-20210902T125111Z-avarab@gmail.com/

Anyway, what you have here seems orthagonal, and it looks like these two
could be combined, but on top of my linked patch we wouldn't be racy
with the new behavior either.

But (and I've got to run) check out the commit message of that patch, perhaps it links to useful past discussions/commits.

I.e. a thing not covered in your commit message is a discussion of if
the current behavior was explicitly desired by anyone, or if we just
ended up with it by accident.

The code you're moving around has a comment which seems to suggest that
what you want *is* the desired behavior, i.e. we re-read it before
invoking the editor, so we should have the updated diff, but just don't?

Or maybe I'm misunderstanding it.

Re: [PATCH] builtin-commit: re-read file index before launching editor

[samuelyvon9]

From: Samuel Yvon <samuelyvon9@gmail.com>

> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
> The code you're moving around has a comment which seems to suggest that
> what you want *is* the desired behavior, i.e. we re-read it before
> invoking the editor, so we should have the updated diff, but just don't?

My understanding is that it was once the behaviour and has changed over time.
I am saying this based on
https://lore.kernel.org/git/xmqqk0yripca.fsf@gitster.c.googlers.com/t/#u

Specifically,

> Junio C Hamano <gitster@pobox.com> writes:
> Even before ec84bd00 (git-commit: Refactor creation of log message.,
> 2008-02-05), the code anticipated that pre-commit may touch the index
> and tried to cope with it.
> However, ec84bd00 moved the place where we re-read the on-disk index
> in the sequence, and updated a message that used to read:
> 
> -    /*
> -     * Re-read the index as pre-commit hook could have updated it,
> -     * and write it out as a tree.
> -     */
> 
> to:
> 
> +    /*
> +     * Re-read the index as pre-commit hook could have updated it,
> +     * and write it out as a tree.  We must do this before we invoke
> +     * the editor and after we invoke run_status above.
> +     */
> 
> Unfortunately there is no mention of the reason why we "must" here.

Looking at ec84bd00 (git-commit: Refactor creation of log message., 2008-02-05),
we can see that the editor is launched after the cache has been reset. The only
part that troubles me is the line in the comment that specifies that "we must do
this ... after we invoke run_status above". I have tested (with my limited knowledge
of the internals of git) and it seems to be of no consequence of flushing before
the call to run_status, but I might be missing something.

> The code you're moving around has a comment which seems to suggest that
> what you want *is* the desired behavior, i.e. we re-read it before
> invoking the editor, so we should have the updated diff, but just don't?

I think this is the case (based on the previously linked conversation).

Re: [PATCH] builtin-commit: re-read file index before launching editor

[Ævar Arnfjörð Bjarmason]

On Mon, Nov 08 2021, samuelyvon9@gmail.com wrote:

> From: Samuel Yvon <samuelyvon9@gmail.com>
>
>> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>> The code you're moving around has a comment which seems to suggest that
>> what you want *is* the desired behavior, i.e. we re-read it before
>> invoking the editor, so we should have the updated diff, but just don't?
>
> My understanding is that it was once the behaviour and has changed over time.
> I am saying this based on
> https://lore.kernel.org/git/xmqqk0yripca.fsf@gitster.c.googlers.com/t/#u
>
> Specifically,
>
>> Junio C Hamano <gitster@pobox.com> writes:
>> Even before ec84bd00 (git-commit: Refactor creation of log message.,
>> 2008-02-05), the code anticipated that pre-commit may touch the index
>> and tried to cope with it.
>> However, ec84bd00 moved the place where we re-read the on-disk index
>> in the sequence, and updated a message that used to read:
>> 
>> -    /*
>> -     * Re-read the index as pre-commit hook could have updated it,
>> -     * and write it out as a tree.
>> -     */
>> 
>> to:
>> 
>> +    /*
>> +     * Re-read the index as pre-commit hook could have updated it,
>> +     * and write it out as a tree.  We must do this before we invoke
>> +     * the editor and after we invoke run_status above.
>> +     */
>> 
>> Unfortunately there is no mention of the reason why we "must" here.
>
> Looking at ec84bd00 (git-commit: Refactor creation of log message., 2008-02-05),
> we can see that the editor is launched after the cache has been reset. The only
> part that troubles me is the line in the comment that specifies that "we must do
> this ... after we invoke run_status above". I have tested (with my limited knowledge
> of the internals of git) and it seems to be of no consequence of flushing before
> the call to run_status, but I might be missing something.
>
>> The code you're moving around has a comment which seems to suggest that
>> what you want *is* the desired behavior, i.e. we re-read it before
>> invoking the editor, so we should have the updated diff, but just don't?
>
> I think this is the case (based on the previously linked conversation).

*nod*, the implicit suggestion here being: Let's put more of that
summary into the commit message. It helps when looking up/discovering
older behavior.

The comment was first added in 2888605c649 (builtin-commit: fix
partial-commit support, 2007-11-18), and quite suspicuous in timing is
f5bbc3225c4 (Port git commit to C., 2007-11-08) where we moved from
git-commit.sh.

It's a bit of a pain to build git that old, but my hunch is that perhaps
this was tested with git-commit.sh, where the reading of the index would
be another process.

Re: [PATCH] builtin-commit: re-read file index before launching editor

[Samuel Yvon]

> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
> *nod*, the implicit suggestion here being: Let's put more of that
> summary into the commit message. It helps when looking up/discovering
> older behavior.

Sorry! I will prepare an amended patch later. Exploring the linked commits
has raised more questions on my end.

> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
> The comment was first added in 2888605c649 (builtin-commit: fix
>         partial-commit support, 2007-11-18), and quite suspicuous in timing is
> f5bbc3225c4 (Port git commit to C., 2007-11-08) where we moved from
> git-commit.sh.

Exploring these commits, I fear I might have been confused; the editor was
always launched after the reset (still today). The problem, as noted earlier,
is the fact that the run_status is ran before the cache reset, which
creates the outdated diff. I can't speculate to the intent of the
author, but exploring the code makes this comment:

> +    /*
> +     * Re-read the index as pre-commit hook could have updated it,
> +     * and write it out as a tree.  We must do this before we invoke
> +     * the editor and after we invoke run_status above.
> +     */

more confusing, because there is no point of invoking the editor after the reset
if the status is written before, since the editor won't show anything different.

On one hand, flushing then showing the editor seems to indicate that we want the
editor to be up-to-date, but because the status is prepared before the flushing,
maybe not?

While it seems the current behaviour has been the behaviour since the start,
I am inclined to continue pushing for this change. Unless I am missing something,
the comment is contradictory and we should be coherent with the idea of
accepting changes made within the pre-commit hook, as noted in 
https://lore.kernel.org/git/xmqqk0yripca.fsf@gitster.c.googlers.com/t/#u:

>> Junio C Hamano <gitster@pobox.com> writes:
>> Even before ec84bd00 (git-commit: Refactor creation of log message.,
>> 2008-02-05), the code anticipated that pre-commit may touch the index
>> and tried to cope with it.

Re: [PATCH] builtin-commit: re-read file index before launching editor

[Junio C Hamano]

Samuel Yvon <samuelyvon9@gmail.com> writes:

> On one hand, flushing then showing the editor seems to indicate that we want the
> editor to be up-to-date, but because the status is prepared before the flushing,
> maybe not?
>
> While it seems the current behaviour has been the behaviour since the start,
> I am inclined to continue pushing for this change. Unless I am missing something,
> the comment is contradictory and we should be coherent with the idea of
> accepting changes made within the pre-commit hook, as noted in 
> https://lore.kernel.org/git/xmqqk0yripca.fsf@gitster.c.googlers.com/t/#u:
>
>>> Junio C Hamano <gitster@pobox.com> writes:
>>> Even before ec84bd00 (git-commit: Refactor creation of log message.,
>>> 2008-02-05), the code anticipated that pre-commit may touch the index
>>> and tried to cope with it.

You seem to be quoting the thread over and over, but what you are
quoting is somewhat different from the concluding part of what I
said, which was:

    If I have to guess, I think the reason is because pre-commit
    automation is expected to be some sort of mechanical change and
    not part of the actual work that the end-user produced, it would
    become easier to perform the "final review" of "what have I done
    so far---does everything make sense?"  if such "extra" changes
    are excluded.

    So, in short, it is not "undefined", but rather it seems to be a
    designed behaviour that we are seeing.

I do not personally mind if we change the philosophy but because it
has been a longstanding designed behaviour, it may need a careful
transition plan.

Re: [PATCH] builtin-commit: re-read file index before launching editor

[Samuel Yvon]

Junio C Hamano <gitster@pobox.com> writes:

> You seem to be quoting the thread over and over, but what you are
> quoting is somewhat different from the concluding part of what I
> said, which was:
>> If I have to guess, I think the reason is because pre-commit
>> automation is expected to be some sort of mechanical change and
>> not part of the actual work that the end-user produced, it would
>> become easier to perform the "final review" of "what have I done
>> so far---does everything make sense?"  if such "extra" changes
>> are excluded.
> So, in short, it is not "undefined", but rather it seems to be a
> designed behaviour that we are seeing.

Apologies if you feel I misquoted you. I am confused by the comment in the
original code w.r.t the location of the cache reset, which (from my
understanding) is contradictory with the quoted conversation and so I
focused on those parts.

Junio C Hamano <gitster@pobox.com> writes:
> I do not personally mind if we change the philosophy but because it
> has been a longstanding designed behaviour, it may need a careful
> transition plan.

Out of curiosity, what would that involve? 

Re: [PATCH] builtin-commit: re-read file index before launching editor

[Junio C Hamano]

Samuel Yvon <samuelyvon9@gmail.com> writes:

>> I do not personally mind if we change the philosophy but because it
>> has been a longstanding designed behaviour, it may need a careful
>> transition plan.
>
> Out of curiosity, what would that involve? 

We need to make sure we do not break workflows of existing users who
relies on the current behaviour.  

As Git is widely used, it is very likely that some of the workflow
element existing users use depend on the current behaviour.  

Letting pre-commit do its thing and letting run_status() compute
"commitable" without its effect may be something they depend on.  It
is the same for any other behaviour we may be tempted to modify.  We
would be breaking Git for them by changing.

So, not breaking by not changing behaviour, if we can do so, would
be ideal.  Introducing a configuration option and hide the new and
different behaviour behind it, so that only the folks who agree to
take responsibility of adjusting to the new behaviour, would be a
good way to isolate the existing users from the change.

If we come up with a new behaviour that would make the world a
better place if adopted by everybody, we might be tempted to
eventually make it the _only_ behaviour available to everybody.  I
am not sure if it is the case here, but if it were, the transition
plan would become even more involved.  We'd need to start by warning
existing users (i.e. the code must detect a case where pre-commit
mucks with the index and having an extra "re-read index file" would
have made a difference---and tell the user that they must adjust to
the change of behaviour in the future), wait for a few years and
then flip the behaviour while removing the warning, or something
like that.

Description of github.com/git/git, was Re: [PATCH] builtin-commit: re-read file index before launching editor

[Johannes Schindelin]

Hi Samuel,

On Tue, 9 Nov 2021, Samuel Yvon via GitGitGadget wrote:

>     As a side note, I do not know who manages the Github Repo but the
>     following description threw me off a little bit:
>
>     Git Source Code Mirror - This is a publish-only repository and all pull requests are ignored.

I would be in favor of changing this description. How about:

	Git Source Code Mirror - Pull Requests can be submitted via GitGitGadget

Thoughts?

Ciao,
Dscho

Re: Description of github.com/git/git, was Re: [PATCH] builtin-commit: re-read file index before launching editor

[Samuel Yvon]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> I would be in favor of changing this description. How about:
> Git Source Code Mirror - Pull Requests can be submitted via GitGitGadget
> Thoughts?

I think it might be somewhat obscure for newcomers (What is GitGitGadget?).
I know Github supports links in project descriptions. Maybe

    Git Source Code Mirror - Please read https://gitgitgadget.github.io/ before submitting Pull Requests

This would have helped me figure out if submitting a PR was gonna end up nowhere or not.

Re: Description of github.com/git/git, was Re: [PATCH] builtin-commit: re-read file index before launching editor

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> I would be in favor of changing this description. How about:
>
> 	Git Source Code Mirror - Pull Requests can be submitted via GitGitGadget

Presense of GGG does not change the fact that the repository is
publish only.  But "all pull requests are ignored" is not correct
anymore.

    Git Source Code Mirror - This is a publish-only repository but
    pull requests can be turned into patches to the mailing list via
    GitGitGadget. Please follow Documentation/SubmittingPatches
    procedure for any of your improvements.

I didn't double-check but I presume SubmittingPatches these days
talk about GGG?

Re: Description of github.com/git/git, was Re: [PATCH] builtin-commit: re-read file index before launching editor

[Taylor Blau]

On Tue, Nov 09, 2021 at 11:03:45AM -0800, Junio C Hamano wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> > I would be in favor of changing this description. How about:
> >
> > 	Git Source Code Mirror - Pull Requests can be submitted via GitGitGadget
>
> Presense of GGG does not change the fact that the repository is
> publish only.  But "all pull requests are ignored" is not correct
> anymore.
>
>     Git Source Code Mirror - This is a publish-only repository but
>     pull requests can be turned into patches to the mailing list via
>     GitGitGadget. Please follow Documentation/SubmittingPatches
>     procedure for any of your improvements.
>
> I didn't double-check but I presume SubmittingPatches these days
> talk about GGG?

I don't think so. Documentation/MyFirstContribution.txt does as of
76644e3268 (documentation: add tutorial for first contribution,
2019-05-17), but the same treatment was not applied elsewhere.

I would not be sad to see a reference to GGG be made in
SubmittingPatches.

Thanks,
Taylor

Re: Description of github.com/git/git, was Re: [PATCH] builtin-commit: re-read file index before launching editor

[Samuel Yvon]

Junio C Hamano <gitster@pobox.com> writes:

> I didn't double-check but I presume SubmittingPatches these days
> talk about GGG?

It does not, but it links to Documentation/MyFirstContribution,
which does. 

Junio C Hamano <gitster@pobox.com> writes:

>     Git Source Code Mirror - This is a publish-only repository but
>     pull requests can be turned into patches to the mailing list via
>     GitGitGadget. Please follow Documentation/SubmittingPatches
>     procedure for any of your improvements.

I think this is descriptive enough so that newcomers can find their way.

Re: Description of github.com/git/git, was Re: [PATCH] builtin-commit: re-read file index before launching editor

[Johannes Schindelin]

Hi,

On Tue, 9 Nov 2021, Samuel Yvon wrote:

> Junio C Hamano <gitster@pobox.com> writes:
>
> >     Git Source Code Mirror - This is a publish-only repository but
> >     pull requests can be turned into patches to the mailing list via
> >     GitGitGadget. Please follow Documentation/SubmittingPatches
> >     procedure for any of your improvements.
>
> I think this is descriptive enough so that newcomers can find their way.

Excellent. Junio had made the adjustment, and I merely added a link to
GitGitGadget's high-level description.

Thank you, all,
Dscho

[PATCH v2] builtin-commit: re-read file index before run_status

[Samuel Yvon via GitGitGadget]

From: Samuel Yvon <samuelyvon9@gmail.com>

Changes made within a pre-commit hook are not reflected within
the editor (for instance, with `git commit --verbose`) because
the index is re-read after the status has been written.

This has the consequence of not displaying the actual changes made
by the pre-commit hook, but committing them anyways.

While it is often argued that the hook's purpose is not to automatically
write content to the repository, it is acknowledged that using the
pre-commit to apply mechanical fixes on top of the existing changes
is a supported use case, along with verifying the content.

I think not seeing these mechanical fixes before commiting is incorrect.
A developer might expect the commit to look a certain way but if the
pre-commit goes wrong the invalid changes will go unnoticed until
committed.

The comment suggesting that the cache must be reset after run_status
and before the editor being launched was added in ec84bd00,
(git-commit: Refactor creation of log message., 2008-02-05). It is
unclear why the run_status must be called *after* the cache reset.
However, calling run_status after the cache reset does not update
the status line to state of the current index in the case a
pre-commit hook is ran and changes files in the staging area.

Signed-off-by: Samuel Yvon <samuelyvon9@gmail.com>
---
    builtin-commit: re-read file index before run_status
    
    Changes made within a pre-commit hook are not reflected within the
    editor (for instance, with git commit --verbose) because the index is
    re-read after the editor has been used.
    
    This has the consequence of not displaying the actual changes made by
    the pre-commit hook, but committing them anyways.
    
    While it is often argued that the hook's purpose is not to automatically
    write content to the repository, it is acknowledged that using the
    pre-commit to apply mechanical fixes on top of the existing changes is a
    supported use case, along with verifying the content.
    
    I think not seeing these mechanical fixes before commiting is incorrect.
    A developer might expect the commit to look a certain way but if the
    pre-commit goes wrong the invalid changes will go unnoticed until
    committed.
    
    I had a small exchange in the Google Group before submitting this Pr.
    Here is a link for cross-referencing:
    https://groups.google.com/g/git-mentoring/c/FsP83I9mN6c
    
    As a side note, I do not know who manages the Github Repo but the
    following description threw me off a little bit:
    
    Git Source Code Mirror - This is a publish-only repository and all pull requests are ignored. 
    
    
    since after looking deeper it seems the PRs are not ignored.
    
    Changes since v1:
    
     * Edited the title to more accurately reflect the changes
     * More details in commit messages
    
    Signed-off-by: Samuel Yvon samuelyvon9@gmail.com

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1127%2FSamuelYvon%2Fmaint-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1127/SamuelYvon/maint-v2
Pull-Request: https://github.com/git/git/pull/1127

Range-diff vs v1:

 1:  271bcf89d1e ! 1:  f09244a0ba3 builtin-commit: re-read file index before launching editor
     @@ Metadata
      Author: Samuel Yvon <samuelyvon9@gmail.com>
      
       ## Commit message ##
     -    builtin-commit: re-read file index before launching editor
     +    builtin-commit: re-read file index before run_status
      
          Changes made within a pre-commit hook are not reflected within
          the editor (for instance, with `git commit --verbose`) because
     -    the index is re-read after the editor has been used.
     +    the index is re-read after the status has been written.
      
          This has the consequence of not displaying the actual changes made
          by the pre-commit hook, but committing them anyways.
     @@ Commit message
          pre-commit goes wrong the invalid changes will go unnoticed until
          committed.
      
     +    The comment suggesting that the cache must be reset after run_status
     +    and before the editor being launched was added in ec84bd00,
     +    (git-commit: Refactor creation of log message., 2008-02-05). It is
     +    unclear why the run_status must be called *after* the cache reset.
     +    However, calling run_status after the cache reset does not update
     +    the status line to state of the current index in the case a
     +    pre-commit hook is ran and changes files in the staging area.
     +
          Signed-off-by: Samuel Yvon <samuelyvon9@gmail.com>
      
       ## builtin/commit.c ##


 builtin/commit.c | 21 +++++++++++----------
 1 file changed, 11 insertions(+), 10 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index 7c9b1e7be3a..e75b11d1c60 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -728,8 +728,17 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 	/* This checks and barfs if author is badly specified */
 	determine_author_info(author_ident);
 
-	if (!no_verify && run_commit_hook(use_editor, index_file, "pre-commit", NULL))
-		return 0;
+	if (!no_verify && find_hook("pre-commit")) {
+		if(run_commit_hook(use_editor, index_file, "pre-commit", NULL))
+			return 0;
+
+		/*
+		 * Re-read the index as pre-commit hook could have updated it,
+		 * and write it out as a tree.
+		 */
+		discard_cache();
+		read_cache_from(index_file);
+	}
 
 	if (squash_message) {
 		/*
@@ -1051,14 +1060,6 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		return 0;
 	}
 
-	if (!no_verify && find_hook("pre-commit")) {
-		/*
-		 * Re-read the index as pre-commit hook could have updated it,
-		 * and write it out as a tree.  We must do this before we invoke
-		 * the editor and after we invoke run_status above.
-		 */
-		discard_cache();
-	}
 	read_cache_from(index_file);
 
 	if (update_main_cache_tree(0)) {

base-commit: 5fbd2fc5997dfa4d4593a862fe729b1e7a89bcf8
-- 
gitgitgadget

Re: [PATCH v2] builtin-commit: re-read file index before run_status

[Junio C Hamano]

"Samuel Yvon via GitGitGadget" <gitgitgadget@gmail.com> writes:

> The comment suggesting that the cache must be reset after run_status
> and before the editor being launched was added in ec84bd00,
> (git-commit: Refactor creation of log message., 2008-02-05). It is
> unclear why the run_status must be called *after* the cache reset.

An older thread cited earlier suspected that it is to reflect the
changes given to "git commit" proper, excluding whatever pre-commit
did, to the return value of run_status(), which becomes the value of
committable variable to answer "do we have anything to commit?".

And moving the call would affect both the contents of the status
buffer (i.e. the list of paths got changed starts including what
pre-commit did) and the "committable" bit by counting such a change
as a true change, avoiding the "no empty commit by default" check,
in a consistent way, hopefully.  I wonder if we have test to
demonstrate that, and if there isn't perhaps we would want to add
one.

> However, calling run_status after the cache reset does not update
> the status line to state of the current index in the case a
> pre-commit hook is ran and changes files in the staging area.

And if this change also affects the "committable" assignment in a
consistent way, it should probably want to be mentioned in this
paragraph, too.

I am not convinced by the claim that there is no need for careful
transition plans (yet), but I personally agree with the end state
(with the above suggested tweaks, that is).

Thanks for working on the topic.

Re: [PATCH v2] builtin-commit: re-read file index before run_status

[Samuel Yvon]

Apologies for the time I took to reply,

Junio C Hamano <gitster@pobox.com> writes:
> And moving the call would affect both the contents of the status
> buffer (i.e. the list of paths got changed starts including what
>         pre-commit did) and the "committable" bit by counting such a change
> as a true change, avoiding the "no empty commit by default" check,
> in a consistent way, hopefully.  I wonder if we have test to
> demonstrate that, and if there isn't perhaps we would want to add
> one.

So, just to make sure I understand well, the concern is that an empty commit
would trigger the commit routine, run the pre-commit hook, which may add files
(thus making it an non-empty commit) and then push 100% automatic changes to a
repo. I agree that this would be invalid behaviour and very odd, I will 
make sure no empty commit is allowed.

Junio C Hamano <gitster@pobox.com> writes:
> Samuel Yvon <samuelyvon9@gmail.com> writes:
> > However, calling run_status after the cache reset does not update
> > the status line to state of the current index in the case a
> > pre-commit hook is ran and changes files in the staging area.
> 
> And if this change also affects the "committable" assignment in a
> consistent way, it should probably want to be mentioned in this
> paragraph, too.

What do you mean by "commitable assignment"? 

> I am not convinced by the claim that there is no need for careful
> transition plans (yet), but I personally agree with the end state
> (with the above suggested tweaks, that is).

With the last message, I agree the safest option is probably to leave this
configurable for now and off by default.

So here's the next steps that I intend to take to get this merged in:

- Add a test for empty commit (if non-existent) and ensure the behaviour is the same
- Add a config option (or maybe a switch?) for migration purposes, with the default
  being the current behaviour.

Re: [PATCH v2] builtin-commit: re-read file index before run_status

[Junio C Hamano]

Samuel Yvon <samuelyvon9@gmail.com> writes:

>> And if this change also affects the "committable" assignment in a
>> consistent way, it should probably want to be mentioned in this
>> paragraph, too.
>
> What do you mean by "commitable assignment"? 

Assignment to the committable variable in the codepath you are
looking at, was what I meant.