Re: [PATCH v4 1/1] unpack-trees: skip stat on fsmonitor-valid files

[Junio C Hamano <gitster@pobox.com>]

"Utsav Shah via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Utsav Shah <utsav@dropbox.com>
>
> The index might be aware that a file hasn't modified via fsmonitor, but
> unpack-trees did not pay attention to it and checked via ie_match_stat
> which can be inefficient on certain filesystems. This significantly slows
> down commands that run oneway_merge, like checkout and reset --hard.
>
> This patch makes oneway_merge check whether a file is considered
> unchanged through fsmonitor and skips ie_match_stat on it. unpack-trees
> also now correctly copies over fsmonitor validity state from the source
> index. Finally, for correctness, we force a refresh of fsmonitor state in
> tweak_fsmonitor.

Instead of saying "also now correctly copies..." as if it started
working correctly by accident, be more assertive and actively make
it so ;-)

    Check if a file is unchanged by fsmonitor in oneway_merge(), and
    avoid unnecessary calls to ie_match_stat().  Copy the fsmonitor
    validity state from the source index to the destination index in
    unpack_trees().  Force a refresh of the fsmonitor state in
    tweak_fsmonitor(), which is called after the index file is read
    from the disk, for correctness.

perhaps.

> After this change, commands like stash (that use reset --hard
> internally) go from 8s or more to ~2s on a 250k file repository on a
> mac.

Good.

>
> Changes since the last version are:
> * The sanity checks around accessing the fsmonitor_dirty bitmap have
> been moved to another patch, which is in message id [1]
> * Unintended indentation changes in fsmonitor have been removed
> * A comment explaining what untracked->use_fsmonitor means has been
> re-added (it was dropped in the previous version)
> * A few "helped-by" entries have been added to the patch
>
> [1]: (xmqqzhh0d0ma.fsf@gitster-ct.c.googlers.com)

The above is for the cover letter or after the three-dash lines, and
not for the log message.

> Helped-by: Junio C Hamano <gitster@pobox.com>
> Helped-by: Kevin Willford <Kevin.Willford@microsoft.com>
> Signed-off-by: Utsav Shah <utsav@dropbox.com>
> ---
>  fsmonitor.c                 | 23 +++++++++++++++++------
>  t/t7519-status-fsmonitor.sh |  9 +++++++--
>  unpack-trees.c              |  6 +++++-
>  3 files changed, 29 insertions(+), 9 deletions(-)
>
> diff --git a/fsmonitor.c b/fsmonitor.c
> index 1f4aa1b150..0d270da80f 100644
> --- a/fsmonitor.c
> +++ b/fsmonitor.c
> @@ -189,13 +189,26 @@ void refresh_fsmonitor(struct index_state *istate)
>  		}
>  		if (bol < query_result.len)
>  			fsmonitor_refresh_callback(istate, buf + bol);
> +
> +		/* Now mark the untracked cache for fsmonitor usage */
> +		if (istate->untracked)
> +			istate->untracked->use_fsmonitor = 1;
>  	} else {
> +
> +		/* We only want to run the post index changed hook if we've actually changed entries, so keep track
> +		 * if we actually changed entries or not */

Multi-line comment style.

> +		int is_cache_changed = 0;
>  		/* Mark all entries invalid */
> -		for (i = 0; i < istate->cache_nr; i++)
> -			istate->cache[i]->ce_flags &= ~CE_FSMONITOR_VALID;
> +		for (i = 0; i < istate->cache_nr; i++) {

Lack of blank line between the last decl and the first stmt.
Probably the blank should go before "/* Mark all ...".

> @@ -257,9 +270,7 @@ void tweak_fsmonitor(struct index_state *istate)
>  				    (uintmax_t)istate->fsmonitor_dirty->bit_size, istate->cache_nr);
>  			ewah_each_bit(istate->fsmonitor_dirty, fsmonitor_ewah_callback, istate);
>  
> -			/* Now mark the untracked cache for fsmonitor usage */
> -			if (istate->untracked)
> -				istate->untracked->use_fsmonitor = 1;
> +			refresh_fsmonitor(istate);
>  		}
>  
>  		ewah_free(istate->fsmonitor_dirty);

Looks good.

Thanks.