[PATCH 1/2] grep: do not use --index in the short usage output

[PATCH 1/2] grep: do not use --index in the short usage output

[Bert Wesarg]

Utilize the PARSE_OPT_NEGHELP option to show --no-index in the usage
generated by parse-options.

Signed-off-by: Bert Wesarg <bert.wesarg@googlemail.com>
---
 builtin/grep.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/builtin/grep.c b/builtin/grep.c
index 1c359c2..e0562b0 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -774,8 +774,9 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
 	struct option options[] = {
 		OPT_BOOLEAN(0, "cached", &cached,
 			"search in index instead of in the work tree"),
-		OPT_BOOLEAN(0, "index", &use_index,
-			"--no-index finds in contents not managed by git"),
+		{ OPTION_BOOLEAN, 0, "index", &use_index, NULL,
+			"finds in contents not managed by git",
+			PARSE_OPT_NOARG | PARSE_OPT_NEGHELP },
 		OPT_GROUP(""),
 		OPT_BOOLEAN('v', "invert-match", &opt.invert,
 			"show non-matching lines"),
-- 
1.7.6.789.gb4599

[PATCH 2/2] grep --no-index: don't use git standard exclusions

[Bert Wesarg]

The --no-index mode is intended to be used outside of a git repository, but
enabling the git standard exclusions outside a git repositories does not make
any sense. Especially if it is not possible to disable them.

Thus do not use the standard exclusions in --no-index mode.

Signed-off-by: Bert Wesarg <bert.wesarg@googlemail.com>
---

On Wed, Jul 20, 2011 at 22:57, Junio C Hamano <gitster@pobox.com> wrote:
>  - Since 3081623 (grep --no-index: allow use of "git grep" outside a git
>   repository, 2010-01-15) and 59332d1 (Resurrect "git grep --no-index",
>   2010-02-06), "grep --no-index" incorrectly paid attention to the
>   exclude patterns. We shouldn't have, and we'd fix that bug.

Fix this bug.

 builtin/grep.c  |    1 -
 t/t7810-grep.sh |    2 +-
 2 files changed, 1 insertions(+), 2 deletions(-)

diff --git a/builtin/grep.c b/builtin/grep.c
index e0562b0..127584e 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -643,7 +643,6 @@ static int grep_directory(struct grep_opt *opt, const struct pathspec *pathspec)
 	int i, hit = 0;
 
 	memset(&dir, 0, sizeof(dir));
-	setup_standard_excludes(&dir);
 
 	fill_directory(&dir, pathspec->raw);
 	for (i = 0; i < dir.nr; i++) {
diff --git a/t/t7810-grep.sh b/t/t7810-grep.sh
index 0d60016..4a05e79 100755
--- a/t/t7810-grep.sh
+++ b/t/t7810-grep.sh
@@ -554,7 +554,6 @@ test_expect_success 'outside of git repository' '
 	mkdir -p non/git/sub &&
 	echo hello >non/git/file1 &&
 	echo world >non/git/sub/file2 &&
-	echo ".*o*" >non/git/.gitignore &&
 	{
 		echo file1:hello &&
 		echo sub/file2:world
@@ -581,6 +580,7 @@ test_expect_success 'inside git repository but with --no-index' '
 	echo world >is/git/sub/file2 &&
 	echo ".*o*" >is/git/.gitignore &&
 	{
+		echo ".gitignore:.*o*" &&
 		echo file1:hello &&
 		echo sub/file2:world
 	} >is/expect.full &&
-- 
1.7.6.789.gb4599

Re: [PATCH 2/2] grep --no-index: don't use git standard exclusions

[Junio C Hamano]

Bert Wesarg <bert.wesarg@googlemail.com> writes:

> On Wed, Jul 20, 2011 at 22:57, Junio C Hamano <gitster@pobox.com> wrote:
>>  - Since 3081623 (grep --no-index: allow use of "git grep" outside a git
>>   repository, 2010-01-15) and 59332d1 (Resurrect "git grep --no-index",
>>   2010-02-06), "grep --no-index" incorrectly paid attention to the
>>   exclude patterns. We shouldn't have, and we'd fix that bug.
>
> Fix this bug.

On a busy list like this, it is brutal to withhold the better clues you
certainly had when you wrote this message that would help people to locate
the original message you are quoting, and instead forcing everybody to go
back 5000 messages in the archive to find it. E.g.

    http://article.gmane.org/gmane.comp.version-control.git/177548
    http://mid.gmane.org/7vzkk86577.fsf@alter.siamese.dyndns.org

Or perhaps have

    References: <7vzkk86577.fsf@alter.siamese.dyndns.org>

in the header.

As to the patch, I think this addresses only one fourth of the issue
identified in that discussion (it is a good starting point, though).

With this change, it would now make sense to teach --[no-]exclude-standard
to "git grep", and "--exclude-standard" is immediately useful when used
with "--no-index". When we add "git grep --untracked-too" (which lets us
search in the working tree), people can add "--no-exclude-standard" to the
command line to say "I want to find the needle even from an ignored file".

Thanks.

Re: [PATCH 2/2] grep --no-index: don't use git standard exclusions

[Bert Wesarg]

On Thu, Sep 15, 2011 at 21:44, Junio C Hamano <gitster@pobox.com> wrote:
> Bert Wesarg <bert.wesarg@googlemail.com> writes:
>
>> On Wed, Jul 20, 2011 at 22:57, Junio C Hamano <gitster@pobox.com> wrote:
>>>  - Since 3081623 (grep --no-index: allow use of "git grep" outside a git
>>>   repository, 2010-01-15) and 59332d1 (Resurrect "git grep --no-index",
>>>   2010-02-06), "grep --no-index" incorrectly paid attention to the
>>>   exclude patterns. We shouldn't have, and we'd fix that bug.
>>
>> Fix this bug.
>
> On a busy list like this, it is brutal to withhold the better clues you
> certainly had when you wrote this message that would help people to locate
> the original message you are quoting, and instead forcing everybody to go
> back 5000 messages in the archive to find it. E.g.
>
>    http://article.gmane.org/gmane.comp.version-control.git/177548
>    http://mid.gmane.org/7vzkk86577.fsf@alter.siamese.dyndns.org
>
> Or perhaps have
>
>    References: <7vzkk86577.fsf@alter.siamese.dyndns.org>
>
> in the header.

Sorry for this patch with insufficient references and context. I
realized it too late, and the time was short.

>
> As to the patch, I think this addresses only one fourth of the issue
> identified in that discussion (it is a good starting point, though).
>

I thought to split the bug fixing from the new features. I already
implemented --exclude-standard, including --exclude=<patter>,
--exclude-from=<file> and --exclude-per-directory=<file>. But it's not
ready because of missing tests and documentation. So I just spilled
this bug fix patch out and will now work on the next part.

> With this change, it would now make sense to teach --[no-]exclude-standard
> to "git grep", and "--exclude-standard" is immediately useful when used
> with "--no-index". When we add "git grep --untracked-too" (which lets us
> search in the working tree), people can add "--no-exclude-standard" to the
> command line to say "I want to find the needle even from an ignored file".

Would '--untracked-too' only be a synonym for '--no-index
--exclude-standard', i.e. the current behavior?

Bert

>
> Thanks.
>

Re* [PATCH 2/2] grep --no-index: don't use git standard exclusions

[Junio C Hamano]

Bert Wesarg <bert.wesarg@googlemail.com> writes:

> Would '--untracked-too' only be a synonym for '--no-index
> --exclude-standard', i.e. the current behavior?

That basically would be the idea. Perhaps something like this on top of
a9e6436 (grep --no-index: don't use git standard exclusions, 2011-09-15).

-- >8 --
Subject: [PATCH 1/2] grep: teach --untracked and --exclude options

In a working tree of a git managed repository, "grep --untracked" would
find the specified patterns from files in untracked files in addition to
its usual behaviour of finding them in the tracked files.

By default, when working with "--no-index" option, "grep" does not pay
attention to .gitignore mechanism. "grep --no-index --exclude" can be
used to tell the command to use .gitignore and stop reporting hits from
files that would be ignored. Also, when working without "--no-index",
"grep" honors .gitignore mechanism, and "grep --no-exclude" can be used
to tell the command to include hits from files that are ignored.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/git-grep.txt |   15 ++++++++++++++-
 builtin-grep.c             |   25 ++++++++++++++++++-------
 2 files changed, 32 insertions(+), 8 deletions(-)

diff --git a/Documentation/git-grep.txt b/Documentation/git-grep.txt
index e019e76..2ccfb90 100644
--- a/Documentation/git-grep.txt
+++ b/Documentation/git-grep.txt
@@ -9,7 +9,7 @@ git-grep - Print lines matching a pattern
 SYNOPSIS
 --------
 [verse]
-'git grep' [--cached]
+'git grep' [--cached] [--untracked] [--excludes]
 	   [-a | --text] [-I] [-i | --ignore-case] [-w | --word-regexp]
 	   [-v | --invert-match] [-h|-H] [--full-name]
 	   [-E | --extended-regexp] [-G | --basic-regexp]
@@ -36,6 +36,19 @@ OPTIONS
 	Instead of searching in the working tree files, check
 	the blobs registered in the index file.
 
+--untracked::
+	In addition to searching in the tracked files in the working
+	tree, search also in untracked files.
+
+--no-excludes::
+	Also search in ignored files by not honoring the `.gitignore`
+	mechanism. Only useful with `--untracked`.
+
+--excludes::
+	Do not pay attention to ignored files specified via the	`.gitignore`
+	mechanism.  Only useful when searching files in the current directory
+	with `--no-index`.
+
 -a::
 --text::
 	Process binary files as if they were text.
diff --git a/builtin-grep.c b/builtin-grep.c
index a10946d..c6cfdf8 100644
--- a/builtin-grep.c
+++ b/builtin-grep.c
@@ -646,12 +646,14 @@ static int grep_object(struct grep_opt *opt, const char **paths,
 	die("unable to grep from object of type %s", typename(obj->type));
 }
 
-static int grep_directory(struct grep_opt *opt, const char **paths)
+static int grep_directory(struct grep_opt *opt, const char **paths, int exc_std)
 {
 	struct dir_struct dir;
 	int i, hit = 0;
 
 	memset(&dir, 0, sizeof(dir));
+	if (exc_std)
+		setup_standard_excludes(&dir);
 
 	fill_directory(&dir, paths);
 	for (i = 0; i < dir.nr; i++) {
@@ -749,7 +751,7 @@ static int help_callback(const struct option *opt, const char *arg, int unset)
 int cmd_grep(int argc, const char **argv, const char *prefix)
 {
 	int hit = 0;
-	int cached = 0;
+	int cached = 0, untracked = 0, opt_exclude = -1;
 	int seen_dashdash = 0;
 	int external_grep_allowed__ignored;
 	struct grep_opt opt;
@@ -764,6 +766,10 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
 		{ OPTION_BOOLEAN, 0, "index", &use_index, NULL,
 			"finds in contents not managed by git",
 			PARSE_OPT_NOARG | PARSE_OPT_NEGHELP },
+		OPT_BOOLEAN(0, "untracked", &untracked,
+			"search in both tracked and untracked files"),
+		OPT_SET_INT(0, "exclude", &opt_exclude,
+			    "search also in ignored files", 1),
 		OPT_GROUP(""),
 		OPT_BOOLEAN('v', "invert-match", &opt.invert,
 			"show non-matching lines"),
@@ -950,18 +956,23 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
 		paths[1] = NULL;
 	}
 
-	if (!use_index) {
+	if (!use_index && (untracked || cached))
+		die("--cached or --untracked cannot be used with --no-index.");
+
+	if (!use_index || untracked) {
 		int hit;
-		if (cached)
-			die("--cached cannot be used with --no-index.");
+		int use_exclude = (opt_exclude < 0) ? use_index : !!opt_exclude;
 		if (list.nr)
-			die("--no-index cannot be used with revs.");
-		hit = grep_directory(&opt, paths);
+			die("--no-index or --untracked cannot be used with revs.");
+		hit = grep_directory(&opt, paths, use_exclude);
 		if (use_threads)
 			hit |= wait_all();
 		return !hit;
 	}
 
+	if (0 <= opt_exclude)
+		die("--exclude or --no-exclude cannot be used for tracked contents.");
+
 	if (!list.nr) {
 		int hit;
 		if (!cached)
-- 
1.7.7.rc3.4.g8d714

[PATCH 2/2] grep: --untracked and --exclude tests

[Junio C Hamano]

Test various combinations of these options and --no-index.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/t7002-grep.sh |   34 +++++++++++++++++++++++++++++++++-
 1 files changed, 33 insertions(+), 1 deletions(-)

diff --git a/t/t7002-grep.sh b/t/t7002-grep.sh
index 918d33f..2cbbdcc 100755
--- a/t/t7002-grep.sh
+++ b/t/t7002-grep.sh
@@ -455,6 +455,23 @@ test_expect_success 'outside of git repository' '
 		test_must_fail git grep o &&
 		git grep --no-index o >../../actual.sub &&
 		test_cmp ../../expect.sub ../../actual.sub
+	) &&
+
+	echo ".*o*" >non/git/.gitignore &&
+	(
+		GIT_CEILING_DIRECTORIES="$(pwd)/non/git" &&
+		export GIT_CEILING_DIRECTORIES &&
+		cd non/git &&
+		test_must_fail git grep o &&
+		git grep --no-index --exclude o >../actual.full &&
+		test_cmp ../expect.full ../actual.full &&
+
+		{
+			echo ".gitignore:.*o*"
+			cat ../expect.full
+		} >../expect.with.ignored &&
+		git grep --no-index --no-exclude o >../actual.full &&
+		test_cmp ../expect.with.ignored ../actual.full
 	)
 '
 
@@ -465,9 +482,12 @@ test_expect_success 'inside git repository but with --no-index' '
 	echo world >is/git/sub/file2 &&
 	echo ".*o*" >is/git/.gitignore &&
 	{
-		echo ".gitignore:.*o*" &&
 		echo file1:hello &&
 		echo sub/file2:world
+	} >is/expect.unignored &&
+	{
+		echo ".gitignore:.*o*" &&
+		cat is/expect.unignored
 	} >is/expect.full &&
 	: >is/expect.empty &&
 	echo file2:world >is/expect.sub
@@ -476,12 +496,24 @@ test_expect_success 'inside git repository but with --no-index' '
 		git init &&
 		test_must_fail git grep o >../actual.full &&
 		test_cmp ../expect.empty ../actual.full &&
+
+		git grep --untracked o >../actual.unignored &&
+		test_cmp ../expect.unignored ../actual.unignored &&
+
 		git grep --no-index o >../actual.full &&
 		test_cmp ../expect.full ../actual.full &&
+
+		git grep --no-index --exclude o >../actual.unignored &&
+		test_cmp ../expect.unignored ../actual.unignored &&
+
 		cd sub &&
 		test_must_fail git grep o >../../actual.sub &&
 		test_cmp ../../expect.empty ../../actual.sub &&
+
 		git grep --no-index o >../../actual.sub &&
+		test_cmp ../../expect.sub ../../actual.sub &&
+
+		git grep --untracked o >../../actual.sub &&
 		test_cmp ../../expect.sub ../../actual.sub
 	)
 '
-- 
1.7.7.rc3.4.g8d714

Re: Re* [PATCH 2/2] grep --no-index: don't use git standard exclusions

[Bert Wesarg]

On Wed, Sep 28, 2011 at 00:21, Junio C Hamano <gitster@pobox.com> wrote:
> Bert Wesarg <bert.wesarg@googlemail.com> writes:
>
>> Would '--untracked-too' only be a synonym for '--no-index
>> --exclude-standard', i.e. the current behavior?
>
> That basically would be the idea. Perhaps something like this on top of
> a9e6436 (grep --no-index: don't use git standard exclusions, 2011-09-15).
>
> -- >8 --
> Subject: [PATCH 1/2] grep: teach --untracked and --exclude options

I would still vote to name the option --exclude-standard, like it is
done in 'git ls-files'. Which also has a --exclude=<pattern> option.
And I think a --exclude=<pattern> option would be useful for 'git
grep', too.

Else:

Acked-by: Bert Wesarg <bert.wesarg@googlemail.com>

Thanks.

Bert

>
> In a working tree of a git managed repository, "grep --untracked" would
> find the specified patterns from files in untracked files in addition to
> its usual behaviour of finding them in the tracked files.
>
> By default, when working with "--no-index" option, "grep" does not pay
> attention to .gitignore mechanism. "grep --no-index --exclude" can be
> used to tell the command to use .gitignore and stop reporting hits from
> files that would be ignored. Also, when working without "--no-index",
> "grep" honors .gitignore mechanism, and "grep --no-exclude" can be used
> to tell the command to include hits from files that are ignored.
>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>  Documentation/git-grep.txt |   15 ++++++++++++++-
>  builtin-grep.c             |   25 ++++++++++++++++++-------
>  2 files changed, 32 insertions(+), 8 deletions(-)
>
> diff --git a/Documentation/git-grep.txt b/Documentation/git-grep.txt
> index e019e76..2ccfb90 100644
> --- a/Documentation/git-grep.txt
> +++ b/Documentation/git-grep.txt
> @@ -9,7 +9,7 @@ git-grep - Print lines matching a pattern
>  SYNOPSIS
>  --------
>  [verse]
> -'git grep' [--cached]
> +'git grep' [--cached] [--untracked] [--excludes]
>           [-a | --text] [-I] [-i | --ignore-case] [-w | --word-regexp]
>           [-v | --invert-match] [-h|-H] [--full-name]
>           [-E | --extended-regexp] [-G | --basic-regexp]
> @@ -36,6 +36,19 @@ OPTIONS
>        Instead of searching in the working tree files, check
>        the blobs registered in the index file.
>
> +--untracked::
> +       In addition to searching in the tracked files in the working
> +       tree, search also in untracked files.
> +
> +--no-excludes::
> +       Also search in ignored files by not honoring the `.gitignore`
> +       mechanism. Only useful with `--untracked`.
> +
> +--excludes::
> +       Do not pay attention to ignored files specified via the `.gitignore`
> +       mechanism.  Only useful when searching files in the current directory
> +       with `--no-index`.
> +
>  -a::
>  --text::
>        Process binary files as if they were text.
> diff --git a/builtin-grep.c b/builtin-grep.c
> index a10946d..c6cfdf8 100644
> --- a/builtin-grep.c
> +++ b/builtin-grep.c
> @@ -646,12 +646,14 @@ static int grep_object(struct grep_opt *opt, const char **paths,
>        die("unable to grep from object of type %s", typename(obj->type));
>  }
>
> -static int grep_directory(struct grep_opt *opt, const char **paths)
> +static int grep_directory(struct grep_opt *opt, const char **paths, int exc_std)
>  {
>        struct dir_struct dir;
>        int i, hit = 0;
>
>        memset(&dir, 0, sizeof(dir));
> +       if (exc_std)
> +               setup_standard_excludes(&dir);
>
>        fill_directory(&dir, paths);
>        for (i = 0; i < dir.nr; i++) {
> @@ -749,7 +751,7 @@ static int help_callback(const struct option *opt, const char *arg, int unset)
>  int cmd_grep(int argc, const char **argv, const char *prefix)
>  {
>        int hit = 0;
> -       int cached = 0;
> +       int cached = 0, untracked = 0, opt_exclude = -1;
>        int seen_dashdash = 0;
>        int external_grep_allowed__ignored;
>        struct grep_opt opt;
> @@ -764,6 +766,10 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
>                { OPTION_BOOLEAN, 0, "index", &use_index, NULL,
>                        "finds in contents not managed by git",
>                        PARSE_OPT_NOARG | PARSE_OPT_NEGHELP },
> +               OPT_BOOLEAN(0, "untracked", &untracked,
> +                       "search in both tracked and untracked files"),
> +               OPT_SET_INT(0, "exclude", &opt_exclude,
> +                           "search also in ignored files", 1),
>                OPT_GROUP(""),
>                OPT_BOOLEAN('v', "invert-match", &opt.invert,
>                        "show non-matching lines"),
> @@ -950,18 +956,23 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
>                paths[1] = NULL;
>        }
>
> -       if (!use_index) {
> +       if (!use_index && (untracked || cached))
> +               die("--cached or --untracked cannot be used with --no-index.");
> +
> +       if (!use_index || untracked) {
>                int hit;
> -               if (cached)
> -                       die("--cached cannot be used with --no-index.");
> +               int use_exclude = (opt_exclude < 0) ? use_index : !!opt_exclude;
>                if (list.nr)
> -                       die("--no-index cannot be used with revs.");
> -               hit = grep_directory(&opt, paths);
> +                       die("--no-index or --untracked cannot be used with revs.");
> +               hit = grep_directory(&opt, paths, use_exclude);
>                if (use_threads)
>                        hit |= wait_all();
>                return !hit;
>        }
>
> +       if (0 <= opt_exclude)
> +               die("--exclude or --no-exclude cannot be used for tracked contents.");
> +
>        if (!list.nr) {
>                int hit;
>                if (!cached)
> --
> 1.7.7.rc3.4.g8d714
>
>
>

Re: Re* [PATCH 2/2] grep --no-index: don't use git standard exclusions

[Junio C Hamano]

Bert Wesarg <bert.wesarg@googlemail.com> writes:

> I would still vote to name the option --exclude-standard, like it is
> done in 'git ls-files'.

Thanks for catching.