From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: Ben Peart <peartben@gmail.com>
Cc: "Git Mailing List" <git@vger.kernel.org>,
"Junio C Hamano" <gitster@pobox.com>,
benpeart@microsoft.com,
"Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>,
"Johannes Schindelin" <johannes.schindelin@gmx.de>,
David.Turner@twosigma.com, "Jeff King" <peff@peff.net>
Subject: Re: [PATCH v2 6/6] fsmonitor: add a sample query-fsmonitor hook script for Watchman
Date: Thu, 25 May 2017 23:05:11 +0200 [thread overview]
Message-ID: <CACBZZX7AA5sbdqhY+W2v3HRSudkWv4cHM+EQUr-X34FGvLH2hw@mail.gmail.com> (raw)
In-Reply-To: <20170518201333.13088-7-benpeart@microsoft.com>
On Thu, May 18, 2017 at 10:13 PM, Ben Peart <peartben@gmail.com> wrote:
> This hook script integrates the new fsmonitor capabilities of git with
> the cross platform Watchman file watching service. To use the script:
>
> Download and install Watchman from https://facebook.github.io/watchman/
> and instruct Watchman to watch your working directory for changes
> ('watchman watch-project /usr/src/git').
>
> Rename the sample integration hook from query-fsmonitor.sample to
> query-fsmonitor.
>
> Configure git to use the extension ('git config core.fsmonitor true')
> and optionally turn on the untracked cache for optimal performance
> ('git config core.untrackedcache true').
>
> Signed-off-by: Ben Peart <benpeart@microsoft.com>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> templates/hooks--query-fsmonitor.sample | 27 +++++++++++++++++++++++++++
> 1 file changed, 27 insertions(+)
> create mode 100644 templates/hooks--query-fsmonitor.sample
>
> diff --git a/templates/hooks--query-fsmonitor.sample b/templates/hooks--query-fsmonitor.sample
> new file mode 100644
> index 0000000000..4bd22f21d8
> --- /dev/null
> +++ b/templates/hooks--query-fsmonitor.sample
> @@ -0,0 +1,27 @@
> +#!/bin/sh
> +#
> +# An example hook script to integrate Watchman
> +# (https://facebook.github.io/watchman/) with git to provide fast
> +# git status.
> +#
> +# The hook is passed a time_t formatted as a string and outputs to
> +# stdout all files that have been modified since the given time.
> +# Paths must be relative to the root of the working tree and
> +# separated by a single NUL.
> +#
> +# To enable this hook, rename this file to "query-fsmonitor"
> +
> +# Convert unix style paths to escaped Windows style paths
> +case "$(uname -s)" in
> +MINGW*|MSYS_NT*)
> + GIT_WORK_TREE="$(cygpath -aw "$PWD" | sed 's,\\,\\\\,g')"
> + ;;
> +*)
> + GIT_WORK_TREE="$PWD"
> + ;;
> +esac
> +
> +# Query Watchman for all the changes since the requested time
> +echo "[\"query\", \"$GIT_WORK_TREE\", {\"since\": $1, \"fields\":[\"name\"]}]" | \
> +watchman -j | \
> +perl -e 'use JSON::PP; my $o = JSON::PP->new->utf8->decode(join("", <>)); die "Watchman: $o->{'error'}.\nFalling back to scanning...\n" if defined($o->{"error"}); print(join("\0", @{$o->{"files"}}));'
I couldn't get watchman to work for me (built from source, keep
getting [1]), but I hacked up something you can hopefully test &
squash on top of this:
# Query Watchman for all the changes since the requested time
-echo "[\"query\", \"$GIT_WORK_TREE\", {\"since\": $time_t,
\"fields\":[\"name\"]}]" | \
-watchman -j | \
-perl -e 'use JSON::PP; my $o = JSON::PP->new->utf8->decode(join("",
<>)); die "Watchman: $o->{'error'}.\nFalling back to scanning...\n" if
defined($o-
+echo "[\"query\", \"$GIT_WORK_TREE\", {\"since\": $time_t,
\"fields\":[\"name\"]}]" |
+ watchman -j |
+ perl -0666 -e '
+ use strict;
+ use warnings;
+
+ my $stdin = <>;
+ die "Watchman: The watchman command returned no
output, error above?\n" if $stdin eq "";
+ die "Watchman: Invalid input: $stdin\n" unless $stdin =~ /^\{/;
+
+ my $json_pkg;
+ eval {
+ require JSON::XS;
+ $json_pkg = "JSON::XS";
+ 1;
+ } or do {
+ require JSON::PP;
+ $json_pkg = "JSON::PP";
+ };
+
+ my $o = $json_pkg->new->utf8->decode($stdin);
+ die "Watchman: $o->{error}.\nFalling back to scanning...\n"
+ if $o->{error};
+
+ local $, = "\0";
+ print @{$o->{files}};
+ '
Rationale:
* We use the much faster JSON::XS if it's installed.
* We use strict/warnings
* Micro optimization: Replace joining stdin with an equivalent -0666
invocation. See "perldoc perlrun".
* Micro optimization: No need to join up the possibly large list of
files into one big string, just set $, to \0 and stream out the
array.
* Error handling: My watchman is broken (so actually this isn't
tested), it only spews to stderr and exits. Handle that by checking
whether stdin is "".
Those changes are available at
https://github.com/avar/git/commits/avar/fsmonitor
1. watchman: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version
`CXXABI_1.3.11' not found (required by watchman)
next prev parent reply other threads:[~2017-05-25 21:05 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-05-18 20:13 [PATCH v2 0/6] Fast git status via a file system watcher Ben Peart
2017-05-18 20:13 ` [PATCH v2 1/6] bswap: add 64 bit endianness helper get_be64 Ben Peart
2017-05-18 20:13 ` [PATCH v2 2/6] dir: make lookup_untracked() available outside of dir.c Ben Peart
2017-05-18 20:13 ` [PATCH v2 3/6] fsmonitor: teach git to optionally utilize a file system monitor to speed up detecting new or changed files Ben Peart
2017-05-19 15:33 ` Ben Peart
2017-05-20 10:41 ` Junio C Hamano
2017-05-24 12:30 ` Christian Couder
2017-05-18 20:13 ` [PATCH v2 4/6] fsmonitor: add test cases for fsmonitor extension Ben Peart
2017-05-20 16:55 ` Torsten Bögershausen
2017-05-18 20:13 ` [PATCH v2 5/6] fsmonitor: add documentation for the " Ben Peart
2017-05-20 11:28 ` Junio C Hamano
2017-05-20 12:10 ` Ævar Arnfjörð Bjarmason
2017-05-22 16:18 ` Ben Peart
2017-05-22 17:28 ` Ævar Arnfjörð Bjarmason
2017-05-25 13:49 ` Ben Peart
2017-05-18 20:13 ` [PATCH v2 6/6] fsmonitor: add a sample query-fsmonitor hook script for Watchman Ben Peart
2017-05-24 13:12 ` Christian Couder
2017-05-26 9:47 ` Ævar Arnfjörð Bjarmason
2017-05-26 16:02 ` Ben Peart
2017-05-25 21:05 ` Ævar Arnfjörð Bjarmason [this message]
2017-05-24 10:54 ` [PATCH v2 0/6] Fast git status via a file system watcher Christian Couder
2017-05-25 13:55 ` Ben Peart
2017-05-27 6:57 ` Christian Couder
2017-05-30 18:05 ` Ben Peart
2017-05-30 20:33 ` Christian Couder
2017-05-30 23:11 ` Ben Peart
2017-05-31 7:37 ` Christian Couder
2017-05-31 7:59 ` Christian Couder
2017-05-31 13:37 ` Ben Peart
2017-05-31 14:10 ` Ævar Arnfjörð Bjarmason
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CACBZZX7AA5sbdqhY+W2v3HRSudkWv4cHM+EQUr-X34FGvLH2hw@mail.gmail.com \
--to=avarab@gmail.com \
--cc=David.Turner@twosigma.com \
--cc=benpeart@microsoft.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=johannes.schindelin@gmx.de \
--cc=pclouds@gmail.com \
--cc=peartben@gmail.com \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).