[PATCH v1 00/45] nd/parse-pathspec and :(glob) pathspec magic

[PATCH v1 00/45] nd/parse-pathspec and :(glob) pathspec magic

[Nguyễn Thái Ngọc Duy]

Probably not much to say. A big portion of this series is the
conversion to struct pathspec, which enables more use of pathspec
magic. :(glob) magic is added to verify that the conversion makes
sense.

Andrew Wong (1):
  setup.c: check that the pathspec magic ends with ")"

Nguyễn Thái Ngọc Duy (44):
  clean: remove unused variable "seen"
  Move struct pathspec and related functions to pathspec.[ch]
  pathspec: i18n-ize error strings in pathspec parsing code
  pathspec: add copy_pathspec
  Add parse_pathspec() that converts cmdline args to struct pathspec
  parse_pathspec: save original pathspec for reporting
  parse_pathspec: add PATHSPEC_PREFER_{CWD,FULL}
  Convert some get_pathspec() calls to parse_pathspec()
  parse_pathspec: a special flag for max_depth feature
  parse_pathspec: support stripping submodule trailing slashes
  parse_pathspec: support stripping/checking submodule paths
  parse_pathspec: support prefixing original patterns
  Guard against new pathspec magic in pathspec matching code
  clean: convert to use parse_pathspec
  commit: convert to use parse_pathspec
  status: convert to use parse_pathspec
  rerere: convert to use parse_pathspec
  checkout: convert to use parse_pathspec
  rm: convert to use parse_pathspec
  ls-files: convert to use parse_pathspec
  archive: convert to use parse_pathspec
  check-ignore: convert to use parse_pathspec
  add: convert to use parse_pathspec
  reset: convert to use parse_pathspec
  Convert read_cache_preload() to take struct pathspec
  Convert run_add_interactive to use struct pathspec
  Convert unmerge_cache to take struct pathspec
  checkout: convert read_tree_some to take struct pathspec
  Convert report_path_error to take struct pathspec
  Convert refresh_index to take struct pathspec
  Convert {read,fill}_directory to take struct pathspec
  Convert add_files_to_cache to take struct pathspec
  Convert common_prefix() to use struct pathspec
  Remove diff_tree_{setup,release}_paths
  Remove init_pathspec() in favor of parse_pathspec()
  Remove match_pathspec() in favor of match_pathspec_depth()
  tree-diff: remove the use of pathspec's raw[] in follow-rename codepath
  parse_pathspec: make sure the prefix part is wildcard-free
  parse_pathspec: preserve prefix length via PATHSPEC_PREFIX_ORIGIN
  Kill limit_pathspec_to_literal() as it's only used by parse_pathspec()
  pathspec: support :(literal) syntax for noglob pathspec
  pathspec: make --literal-pathspecs disable pathspec magic
  pathspec: support :(glob) syntax
  Rename field "raw" to "_raw" in struct pathspec

 Documentation/git.txt              |  23 +-
 Documentation/glossary-content.txt |  33 +++
 archive.c                          |  18 +-
 archive.h                          |   4 +-
 builtin/add.c                      | 156 ++++++--------
 builtin/blame.c                    |  14 +-
 builtin/check-ignore.c             |  34 +--
 builtin/checkout.c                 |  46 ++--
 builtin/clean.c                    |  24 +--
 builtin/commit.c                   |  37 ++--
 builtin/diff-files.c               |   2 +-
 builtin/diff-index.c               |   2 +-
 builtin/diff.c                     |   6 +-
 builtin/grep.c                     |  10 +-
 builtin/log.c                      |   2 +-
 builtin/ls-files.c                 |  75 +++----
 builtin/ls-tree.c                  |  13 +-
 builtin/mv.c                       |  13 +-
 builtin/rerere.c                   |   8 +-
 builtin/reset.c                    |  33 +--
 builtin/rm.c                       |  24 +--
 builtin/update-index.c             |   6 +-
 cache.h                            |  34 +--
 commit.h                           |   2 +-
 diff-lib.c                         |   3 +-
 diff.h                             |   3 +-
 dir.c                              | 261 +++++-----------------
 dir.h                              |  18 +-
 git.c                              |   8 +
 merge-recursive.c                  |   2 +-
 notes-merge.c                      |   4 +-
 path.c                             |  15 +-
 pathspec.c                         | 431 +++++++++++++++++++++++++++++++++----
 pathspec.h                         |  59 ++++-
 preload-index.c                    |  21 +-
 read-cache.c                       |   5 +-
 rerere.c                           |   7 +-
 rerere.h                           |   4 +-
 resolve-undo.c                     |   4 +-
 resolve-undo.h                     |   2 +-
 revision.c                         |  11 +-
 setup.c                            | 157 +-------------
 t/t0008-ignores.sh                 |   8 +-
 t/t6130-pathspec-noglob.sh         |  18 ++
 tree-diff.c                        |  48 +++--
 tree-walk.c                        |  21 +-
 tree.c                             |   4 +-
 tree.h                             |   2 +-
 wt-status.c                        |  18 +-
 wt-status.h                        |   2 +-
 50 files changed, 983 insertions(+), 772 deletions(-)

-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 01/45] setup.c: check that the pathspec magic ends with ")"

[Nguyễn Thái Ngọc Duy]

From: Andrew Wong <andrew.kw.w@gmail.com>

The previous code did not diagnose an incorrectly spelled ":(top"
as an error.

Signed-off-by: Andrew Wong <andrew.kw.w@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 setup.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/setup.c b/setup.c
index 1dee47e..d0cd784 100644
--- a/setup.c
+++ b/setup.c
@@ -223,8 +223,9 @@ static const char *prefix_pathspec(const char *prefix, int prefixlen, const char
 				die("Invalid pathspec magic '%.*s' in '%s'",
 				    (int) len, copyfrom, elt);
 		}
-		if (*copyfrom == ')')
-			copyfrom++;
+		if (*copyfrom != ')')
+			die("Missing ')' at the end of pathspec magic in '%s'", elt);
+		copyfrom++;
 	} else {
 		/* shorthand */
 		for (copyfrom = elt + 1;
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 02/45] clean: remove unused variable "seen"

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/clean.c | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/builtin/clean.c b/builtin/clean.c
index 04e396b..f955a40 100644
--- a/builtin/clean.c
+++ b/builtin/clean.c
@@ -155,7 +155,6 @@ int cmd_clean(int argc, const char **argv, const char *prefix)
 	struct string_list exclude_list = STRING_LIST_INIT_NODUP;
 	struct exclude_list *el;
 	const char *qname;
-	char *seen = NULL;
 	struct option options[] = {
 		OPT__QUIET(&quiet, N_("do not print names of files removed")),
 		OPT__DRY_RUN(&dry_run, N_("dry run")),
@@ -214,9 +213,6 @@ int cmd_clean(int argc, const char **argv, const char *prefix)
 
 	fill_directory(&dir, pathspec);
 
-	if (pathspec)
-		seen = xmalloc(argc > 0 ? argc : 1);
-
 	for (i = 0; i < dir.nr; i++) {
 		struct dir_entry *ent = dir.entries[i];
 		int len, pos;
@@ -250,11 +246,9 @@ int cmd_clean(int argc, const char **argv, const char *prefix)
 		if (lstat(ent->name, &st))
 			continue;
 
-		if (pathspec) {
-			memset(seen, 0, argc > 0 ? argc : 1);
+		if (pathspec)
 			matches = match_pathspec(pathspec, ent->name, len,
-						 0, seen);
-		}
+						 0, NULL);
 
 		if (S_ISDIR(st.st_mode)) {
 			strbuf_addstr(&directory, ent->name);
@@ -281,7 +275,6 @@ int cmd_clean(int argc, const char **argv, const char *prefix)
 			}
 		}
 	}
-	free(seen);
 
 	strbuf_release(&directory);
 	string_list_clear(&exclude_list, 0);
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 03/45] Move struct pathspec and related functions to pathspec.[ch]

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 archive.c              |   1 +
 builtin/grep.c         |   1 +
 builtin/ls-files.c     |   1 +
 builtin/ls-tree.c      |   1 +
 builtin/update-index.c |   1 +
 cache.h                |  22 +-------
 diff.h                 |   1 +
 dir.c                  |   1 +
 pathspec.c             | 149 +++++++++++++++++++++++++++++++++++++++++++++++++
 pathspec.h             |  21 +++++++
 preload-index.c        |   1 +
 setup.c                | 149 -------------------------------------------------
 tree-walk.c            |   1 +
 13 files changed, 181 insertions(+), 169 deletions(-)

diff --git a/archive.c b/archive.c
index 93e00bb..72c6b0f 100644
--- a/archive.c
+++ b/archive.c
@@ -5,6 +5,7 @@
 #include "archive.h"
 #include "parse-options.h"
 #include "unpack-trees.h"
+#include "pathspec.h"
 
 static char const * const archive_usage[] = {
 	N_("git archive [options] <tree-ish> [<path>...]"),
diff --git a/builtin/grep.c b/builtin/grep.c
index 8025964..3701c2e 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -17,6 +17,7 @@
 #include "grep.h"
 #include "quote.h"
 #include "dir.h"
+#include "pathspec.h"
 
 static char const * const grep_usage[] = {
 	N_("git grep [options] [-e] <pattern> [<rev>...] [[--] <path>...]"),
diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index 175e6e3..752570b 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -13,6 +13,7 @@
 #include "parse-options.h"
 #include "resolve-undo.h"
 #include "string-list.h"
+#include "pathspec.h"
 
 static int abbrev;
 static int show_deleted;
diff --git a/builtin/ls-tree.c b/builtin/ls-tree.c
index fb76e38..93fc3a0 100644
--- a/builtin/ls-tree.c
+++ b/builtin/ls-tree.c
@@ -10,6 +10,7 @@
 #include "quote.h"
 #include "builtin.h"
 #include "parse-options.h"
+#include "pathspec.h"
 
 static int line_termination = '\n';
 #define LS_RECURSIVE 1
diff --git a/builtin/update-index.c b/builtin/update-index.c
index ada1dff..ffae585 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -11,6 +11,7 @@
 #include "refs.h"
 #include "resolve-undo.h"
 #include "parse-options.h"
+#include "pathspec.h"
 
 /*
  * Default to not allowing changes to the list of files. The
diff --git a/cache.h b/cache.h
index e493563..5115c1b 100644
--- a/cache.h
+++ b/cache.h
@@ -182,6 +182,8 @@ struct cache_entry {
 #error "CE_EXTENDED_FLAGS out of range"
 #endif
 
+struct pathspec;
+
 /*
  * Copy the sha1 and stat state of a cache entry from one to
  * another. But we never change the name, or the hash state!
@@ -474,28 +476,8 @@ extern int index_name_is_other(const struct index_state *, const char *, int);
 extern int ie_match_stat(const struct index_state *, struct cache_entry *, struct stat *, unsigned int);
 extern int ie_modified(const struct index_state *, struct cache_entry *, struct stat *, unsigned int);
 
-#define PATHSPEC_ONESTAR 1	/* the pathspec pattern sastisfies GFNM_ONESTAR */
-
-struct pathspec {
-	const char **raw; /* get_pathspec() result, not freed by free_pathspec() */
-	int nr;
-	unsigned int has_wildcard:1;
-	unsigned int recursive:1;
-	int max_depth;
-	struct pathspec_item {
-		const char *match;
-		int len;
-		int nowildcard_len;
-		int flags;
-	} *items;
-};
-
-extern int init_pathspec(struct pathspec *, const char **);
-extern void free_pathspec(struct pathspec *);
 extern int ce_path_match(const struct cache_entry *ce, const struct pathspec *pathspec);
 
-extern int limit_pathspec_to_literal(void);
-
 #define HASH_WRITE_OBJECT 1
 #define HASH_FORMAT_CHECK 2
 extern int index_fd(unsigned char *sha1, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);
diff --git a/diff.h b/diff.h
index 78b4091..d1bc914 100644
--- a/diff.h
+++ b/diff.h
@@ -5,6 +5,7 @@
 #define DIFF_H
 
 #include "tree-walk.h"
+#include "pathspec.h"
 
 struct rev_info;
 struct diff_options;
diff --git a/dir.c b/dir.c
index 57394e4..97ad45b 100644
--- a/dir.c
+++ b/dir.c
@@ -11,6 +11,7 @@
 #include "dir.h"
 #include "refs.h"
 #include "wildmatch.h"
+#include "pathspec.h"
 
 struct path_simplify {
 	int len;
diff --git a/pathspec.c b/pathspec.c
index 284f397..aa3e4d0 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -99,3 +99,152 @@ void die_if_path_beyond_symlink(const char *path, const char *prefix)
 		die(_("'%s' is beyond a symbolic link"), path + len);
 	}
 }
+
+/*
+ * Magic pathspec
+ *
+ * NEEDSWORK: These need to be moved to dir.h or even to a new
+ * pathspec.h when we restructure get_pathspec() users to use the
+ * "struct pathspec" interface.
+ *
+ * Possible future magic semantics include stuff like:
+ *
+ *	{ PATHSPEC_NOGLOB, '!', "noglob" },
+ *	{ PATHSPEC_ICASE, '\0', "icase" },
+ *	{ PATHSPEC_RECURSIVE, '*', "recursive" },
+ *	{ PATHSPEC_REGEXP, '\0', "regexp" },
+ *
+ */
+#define PATHSPEC_FROMTOP    (1<<0)
+
+static struct pathspec_magic {
+	unsigned bit;
+	char mnemonic; /* this cannot be ':'! */
+	const char *name;
+} pathspec_magic[] = {
+	{ PATHSPEC_FROMTOP, '/', "top" },
+};
+
+/*
+ * Take an element of a pathspec and check for magic signatures.
+ * Append the result to the prefix.
+ *
+ * For now, we only parse the syntax and throw out anything other than
+ * "top" magic.
+ *
+ * NEEDSWORK: This needs to be rewritten when we start migrating
+ * get_pathspec() users to use the "struct pathspec" interface.  For
+ * example, a pathspec element may be marked as case-insensitive, but
+ * the prefix part must always match literally, and a single stupid
+ * string cannot express such a case.
+ */
+static const char *prefix_pathspec(const char *prefix, int prefixlen, const char *elt)
+{
+	unsigned magic = 0;
+	const char *copyfrom = elt;
+	int i;
+
+	if (elt[0] != ':') {
+		; /* nothing to do */
+	} else if (elt[1] == '(') {
+		/* longhand */
+		const char *nextat;
+		for (copyfrom = elt + 2;
+		     *copyfrom && *copyfrom != ')';
+		     copyfrom = nextat) {
+			size_t len = strcspn(copyfrom, ",)");
+			if (copyfrom[len] == ')')
+				nextat = copyfrom + len;
+			else
+				nextat = copyfrom + len + 1;
+			if (!len)
+				continue;
+			for (i = 0; i < ARRAY_SIZE(pathspec_magic); i++)
+				if (strlen(pathspec_magic[i].name) == len &&
+				    !strncmp(pathspec_magic[i].name, copyfrom, len)) {
+					magic |= pathspec_magic[i].bit;
+					break;
+				}
+			if (ARRAY_SIZE(pathspec_magic) <= i)
+				die("Invalid pathspec magic '%.*s' in '%s'",
+				    (int) len, copyfrom, elt);
+		}
+		if (*copyfrom != ')')
+			die("Missing ')' at the end of pathspec magic in '%s'", elt);
+		copyfrom++;
+	} else {
+		/* shorthand */
+		for (copyfrom = elt + 1;
+		     *copyfrom && *copyfrom != ':';
+		     copyfrom++) {
+			char ch = *copyfrom;
+
+			if (!is_pathspec_magic(ch))
+				break;
+			for (i = 0; i < ARRAY_SIZE(pathspec_magic); i++)
+				if (pathspec_magic[i].mnemonic == ch) {
+					magic |= pathspec_magic[i].bit;
+					break;
+				}
+			if (ARRAY_SIZE(pathspec_magic) <= i)
+				die("Unimplemented pathspec magic '%c' in '%s'",
+				    ch, elt);
+		}
+		if (*copyfrom == ':')
+			copyfrom++;
+	}
+
+	if (magic & PATHSPEC_FROMTOP)
+		return xstrdup(copyfrom);
+	else
+		return prefix_path(prefix, prefixlen, copyfrom);
+}
+
+/*
+ * N.B. get_pathspec() is deprecated in favor of the "struct pathspec"
+ * based interface - see pathspec_magic above.
+ *
+ * Arguments:
+ *  - prefix - a path relative to the root of the working tree
+ *  - pathspec - a list of paths underneath the prefix path
+ *
+ * Iterates over pathspec, prepending each path with prefix,
+ * and return the resulting list.
+ *
+ * If pathspec is empty, return a singleton list containing prefix.
+ *
+ * If pathspec and prefix are both empty, return an empty list.
+ *
+ * This is typically used by built-in commands such as add.c, in order
+ * to normalize argv arguments provided to the built-in into a list of
+ * paths to process, all relative to the root of the working tree.
+ */
+const char **get_pathspec(const char *prefix, const char **pathspec)
+{
+	const char *entry = *pathspec;
+	const char **src, **dst;
+	int prefixlen;
+
+	if (!prefix && !entry)
+		return NULL;
+
+	if (!entry) {
+		static const char *spec[2];
+		spec[0] = prefix;
+		spec[1] = NULL;
+		return spec;
+	}
+
+	/* Otherwise we have to re-write the entries.. */
+	src = pathspec;
+	dst = pathspec;
+	prefixlen = prefix ? strlen(prefix) : 0;
+	while (*src) {
+		*(dst++) = prefix_pathspec(prefix, prefixlen, *src);
+		src++;
+	}
+	*dst = NULL;
+	if (!*pathspec)
+		return NULL;
+	return pathspec;
+}
diff --git a/pathspec.h b/pathspec.h
index db0184a..7884068 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -1,6 +1,27 @@
 #ifndef PATHSPEC_H
 #define PATHSPEC_H
 
+#define PATHSPEC_ONESTAR 1	/* the pathspec pattern sastisfies GFNM_ONESTAR */
+
+struct pathspec {
+	const char **raw; /* get_pathspec() result, not freed by free_pathspec() */
+	int nr;
+	unsigned int has_wildcard:1;
+	unsigned int recursive:1;
+	int max_depth;
+	struct pathspec_item {
+		const char *match;
+		int len;
+		int nowildcard_len;
+		int flags;
+	} *items;
+};
+
+extern int init_pathspec(struct pathspec *, const char **);
+extern void free_pathspec(struct pathspec *);
+
+extern int limit_pathspec_to_literal(void);
+
 extern char *find_pathspecs_matching_against_index(const char **pathspec);
 extern void add_pathspec_matches_against_index(const char **pathspec, char *seen, int specs);
 extern const char *check_path_for_gitlink(const char *path);
diff --git a/preload-index.c b/preload-index.c
index 49cb08d..cddfffa 100644
--- a/preload-index.c
+++ b/preload-index.c
@@ -2,6 +2,7 @@
  * Copyright (C) 2008 Linus Torvalds
  */
 #include "cache.h"
+#include "pathspec.h"
 
 #ifdef NO_PTHREADS
 static void preload_index(struct index_state *index, const char **pathspec)
diff --git a/setup.c b/setup.c
index d0cd784..acdc634 100644
--- a/setup.c
+++ b/setup.c
@@ -155,155 +155,6 @@ void verify_non_filename(const char *prefix, const char *arg)
 }
 
 /*
- * Magic pathspec
- *
- * NEEDSWORK: These need to be moved to dir.h or even to a new
- * pathspec.h when we restructure get_pathspec() users to use the
- * "struct pathspec" interface.
- *
- * Possible future magic semantics include stuff like:
- *
- *	{ PATHSPEC_NOGLOB, '!', "noglob" },
- *	{ PATHSPEC_ICASE, '\0', "icase" },
- *	{ PATHSPEC_RECURSIVE, '*', "recursive" },
- *	{ PATHSPEC_REGEXP, '\0', "regexp" },
- *
- */
-#define PATHSPEC_FROMTOP    (1<<0)
-
-static struct pathspec_magic {
-	unsigned bit;
-	char mnemonic; /* this cannot be ':'! */
-	const char *name;
-} pathspec_magic[] = {
-	{ PATHSPEC_FROMTOP, '/', "top" },
-};
-
-/*
- * Take an element of a pathspec and check for magic signatures.
- * Append the result to the prefix.
- *
- * For now, we only parse the syntax and throw out anything other than
- * "top" magic.
- *
- * NEEDSWORK: This needs to be rewritten when we start migrating
- * get_pathspec() users to use the "struct pathspec" interface.  For
- * example, a pathspec element may be marked as case-insensitive, but
- * the prefix part must always match literally, and a single stupid
- * string cannot express such a case.
- */
-static const char *prefix_pathspec(const char *prefix, int prefixlen, const char *elt)
-{
-	unsigned magic = 0;
-	const char *copyfrom = elt;
-	int i;
-
-	if (elt[0] != ':') {
-		; /* nothing to do */
-	} else if (elt[1] == '(') {
-		/* longhand */
-		const char *nextat;
-		for (copyfrom = elt + 2;
-		     *copyfrom && *copyfrom != ')';
-		     copyfrom = nextat) {
-			size_t len = strcspn(copyfrom, ",)");
-			if (copyfrom[len] == ')')
-				nextat = copyfrom + len;
-			else
-				nextat = copyfrom + len + 1;
-			if (!len)
-				continue;
-			for (i = 0; i < ARRAY_SIZE(pathspec_magic); i++)
-				if (strlen(pathspec_magic[i].name) == len &&
-				    !strncmp(pathspec_magic[i].name, copyfrom, len)) {
-					magic |= pathspec_magic[i].bit;
-					break;
-				}
-			if (ARRAY_SIZE(pathspec_magic) <= i)
-				die("Invalid pathspec magic '%.*s' in '%s'",
-				    (int) len, copyfrom, elt);
-		}
-		if (*copyfrom != ')')
-			die("Missing ')' at the end of pathspec magic in '%s'", elt);
-		copyfrom++;
-	} else {
-		/* shorthand */
-		for (copyfrom = elt + 1;
-		     *copyfrom && *copyfrom != ':';
-		     copyfrom++) {
-			char ch = *copyfrom;
-
-			if (!is_pathspec_magic(ch))
-				break;
-			for (i = 0; i < ARRAY_SIZE(pathspec_magic); i++)
-				if (pathspec_magic[i].mnemonic == ch) {
-					magic |= pathspec_magic[i].bit;
-					break;
-				}
-			if (ARRAY_SIZE(pathspec_magic) <= i)
-				die("Unimplemented pathspec magic '%c' in '%s'",
-				    ch, elt);
-		}
-		if (*copyfrom == ':')
-			copyfrom++;
-	}
-
-	if (magic & PATHSPEC_FROMTOP)
-		return xstrdup(copyfrom);
-	else
-		return prefix_path(prefix, prefixlen, copyfrom);
-}
-
-/*
- * N.B. get_pathspec() is deprecated in favor of the "struct pathspec"
- * based interface - see pathspec_magic above.
- *
- * Arguments:
- *  - prefix - a path relative to the root of the working tree
- *  - pathspec - a list of paths underneath the prefix path
- *
- * Iterates over pathspec, prepending each path with prefix,
- * and return the resulting list.
- *
- * If pathspec is empty, return a singleton list containing prefix.
- *
- * If pathspec and prefix are both empty, return an empty list.
- *
- * This is typically used by built-in commands such as add.c, in order
- * to normalize argv arguments provided to the built-in into a list of
- * paths to process, all relative to the root of the working tree.
- */
-const char **get_pathspec(const char *prefix, const char **pathspec)
-{
-	const char *entry = *pathspec;
-	const char **src, **dst;
-	int prefixlen;
-
-	if (!prefix && !entry)
-		return NULL;
-
-	if (!entry) {
-		static const char *spec[2];
-		spec[0] = prefix;
-		spec[1] = NULL;
-		return spec;
-	}
-
-	/* Otherwise we have to re-write the entries.. */
-	src = pathspec;
-	dst = pathspec;
-	prefixlen = prefix ? strlen(prefix) : 0;
-	while (*src) {
-		*(dst++) = prefix_pathspec(prefix, prefixlen, *src);
-		src++;
-	}
-	*dst = NULL;
-	if (!*pathspec)
-		return NULL;
-	return pathspec;
-}
-
-/*
  * Test if it looks like we're at a git directory.
  * We want to see:
  *
diff --git a/tree-walk.c b/tree-walk.c
index 6e30ef9..72a9613 100644
--- a/tree-walk.c
+++ b/tree-walk.c
@@ -3,6 +3,7 @@
 #include "unpack-trees.h"
 #include "dir.h"
 #include "tree.h"
+#include "pathspec.h"
 
 static const char *get_mode(const char *str, unsigned int *modep)
 {
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 04/45] pathspec: i18n-ize error strings in pathspec parsing code

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 pathspec.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/pathspec.c b/pathspec.c
index aa3e4d0..94d64d2 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -166,11 +166,11 @@ static const char *prefix_pathspec(const char *prefix, int prefixlen, const char
 					break;
 				}
 			if (ARRAY_SIZE(pathspec_magic) <= i)
-				die("Invalid pathspec magic '%.*s' in '%s'",
+				die(_("Invalid pathspec magic '%.*s' in '%s'"),
 				    (int) len, copyfrom, elt);
 		}
 		if (*copyfrom != ')')
-			die("Missing ')' at the end of pathspec magic in '%s'", elt);
+			die(_("Missing ')' at the end of pathspec magic in '%s'"), elt);
 		copyfrom++;
 	} else {
 		/* shorthand */
@@ -187,7 +187,7 @@ static const char *prefix_pathspec(const char *prefix, int prefixlen, const char
 					break;
 				}
 			if (ARRAY_SIZE(pathspec_magic) <= i)
-				die("Unimplemented pathspec magic '%c' in '%s'",
+				die(_("Unimplemented pathspec magic '%c' in '%s'"),
 				    ch, elt);
 		}
 		if (*copyfrom == ':')
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 05/45] pathspec: add copy_pathspec

[Nguyễn Thái Ngọc Duy]

The function is made to use with free_pathspec() because a simple
struct assignment is not enough (free_pathspec wants to free "items"
pointer).

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/mv.c | 13 +++++++------
 pathspec.c   |  8 ++++++++
 pathspec.h   |  1 +
 3 files changed, 16 insertions(+), 6 deletions(-)

diff --git a/builtin/mv.c b/builtin/mv.c
index 034fec9..16ce99b 100644
--- a/builtin/mv.c
+++ b/builtin/mv.c
@@ -15,8 +15,9 @@ static const char * const builtin_mv_usage[] = {
 	NULL
 };
 
-static const char **copy_pathspec(const char *prefix, const char **pathspec,
-				  int count, int base_name)
+static const char **internal_copy_pathspec(const char *prefix,
+					   const char **pathspec,
+					   int count, int base_name)
 {
 	int i;
 	const char **result = xmalloc((count + 1) * sizeof(const char *));
@@ -81,17 +82,17 @@ int cmd_mv(int argc, const char **argv, const char *prefix)
 	if (read_cache() < 0)
 		die(_("index file corrupt"));
 
-	source = copy_pathspec(prefix, argv, argc, 0);
+	source = internal_copy_pathspec(prefix, argv, argc, 0);
 	modes = xcalloc(argc, sizeof(enum update_mode));
-	dest_path = copy_pathspec(prefix, argv + argc, 1, 0);
+	dest_path = internal_copy_pathspec(prefix, argv + argc, 1, 0);
 
 	if (dest_path[0][0] == '\0')
 		/* special case: "." was normalized to "" */
-		destination = copy_pathspec(dest_path[0], argv, argc, 1);
+		destination = internal_copy_pathspec(dest_path[0], argv, argc, 1);
 	else if (!lstat(dest_path[0], &st) &&
 			S_ISDIR(st.st_mode)) {
 		dest_path[0] = add_slash(dest_path[0]);
-		destination = copy_pathspec(dest_path[0], argv, argc, 1);
+		destination = internal_copy_pathspec(dest_path[0], argv, argc, 1);
 	} else {
 		if (argc != 1)
 			die("destination '%s' is not a directory", dest_path[0]);
diff --git a/pathspec.c b/pathspec.c
index 94d64d2..ab53b8a 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -248,3 +248,11 @@ const char **get_pathspec(const char *prefix, const char **pathspec)
 		return NULL;
 	return pathspec;
 }
+
+void copy_pathspec(struct pathspec *dst, const struct pathspec *src)
+{
+	*dst = *src;
+	dst->items = xmalloc(sizeof(struct pathspec_item) * dst->nr);
+	memcpy(dst->items, src->items,
+	       sizeof(struct pathspec_item) * dst->nr);
+}
diff --git a/pathspec.h b/pathspec.h
index 7884068..a621676 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -18,6 +18,7 @@ struct pathspec {
 };
 
 extern int init_pathspec(struct pathspec *, const char **);
+extern void copy_pathspec(struct pathspec *dst, const struct pathspec *src);
 extern void free_pathspec(struct pathspec *);
 
 extern int limit_pathspec_to_literal(void);
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 06/45] Add parse_pathspec() that converts cmdline args to struct pathspec

[Nguyễn Thái Ngọc Duy]

Currently to fill a struct pathspec, we do:

   const char **paths;
   paths = get_pathspec(prefix, argv);
   ...
   init_pathspec(&pathspec, paths);

"paths" can only carry bare strings, which loses information from
command line arguments such as pathspec magic or the prefix part's
length for each argument.

parse_pathspec() is introduced to combine the two calls into one. The
plan is gradually replace all get_pathspec() and init_pathspec() with
parse_pathspec(). get_pathspec() now becomes a thin wrapper of
parse_pathspec().

parse_pathspec() allows the caller to reject the pathspec magics that
it does not support. When a new pathspec magic is introduced, we can
enable it per command after making sure that all underlying code has no
problem with the new magic.

"flags" parameter is currently unused. But it would allow callers to
pass certain instructions to parse_pathspec, for example forcing
literal pathspec when no magic is used.

With the introduction of parse_pathspec, there are now two functions
that can initialize struct pathspec: init_pathspec and
parse_pathspec. Any semantic changes in struct pathspec must be
reflected in both functions. init_pathspec() will be phased out in
favor of parse_pathspec().

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 dir.c      |   4 +-
 dir.h      |   2 +
 pathspec.c | 167 +++++++++++++++++++++++++++++++++++++++++++++++--------------
 pathspec.h |  11 ++++
 4 files changed, 144 insertions(+), 40 deletions(-)

diff --git a/dir.c b/dir.c
index 97ad45b..a442467 100644
--- a/dir.c
+++ b/dir.c
@@ -338,7 +338,7 @@ int match_pathspec_depth(const struct pathspec *ps,
 /*
  * Return the length of the "simple" part of a path match limiter.
  */
-static int simple_length(const char *match)
+int simple_length(const char *match)
 {
 	int len = -1;
 
@@ -350,7 +350,7 @@ static int simple_length(const char *match)
 	}
 }
 
-static int no_wildcard(const char *string)
+int no_wildcard(const char *string)
 {
 	return string[simple_length(string)] == '\0';
 }
diff --git a/dir.h b/dir.h
index c3eb4b5..89427fd 100644
--- a/dir.h
+++ b/dir.h
@@ -125,6 +125,8 @@ struct dir_struct {
 #define MATCHED_RECURSIVELY 1
 #define MATCHED_FNMATCH 2
 #define MATCHED_EXACTLY 3
+extern int simple_length(const char *match);
+extern int no_wildcard(const char *string);
 extern char *common_prefix(const char **pathspec);
 extern int match_pathspec(const char **pathspec, const char *name, int namelen, int prefix, char *seen);
 extern int match_pathspec_depth(const struct pathspec *pathspec,
diff --git a/pathspec.c b/pathspec.c
index ab53b8a..bc9d9ac 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -103,10 +103,6 @@ void die_if_path_beyond_symlink(const char *path, const char *prefix)
 /*
  * Magic pathspec
  *
- * NEEDSWORK: These need to be moved to dir.h or even to a new
- * pathspec.h when we restructure get_pathspec() users to use the
- * "struct pathspec" interface.
- *
  * Possible future magic semantics include stuff like:
  *
  *	{ PATHSPEC_NOGLOB, '!', "noglob" },
@@ -115,7 +111,6 @@ void die_if_path_beyond_symlink(const char *path, const char *prefix)
  *	{ PATHSPEC_REGEXP, '\0', "regexp" },
  *
  */
-#define PATHSPEC_FROMTOP    (1<<0)
 
 static struct pathspec_magic {
 	unsigned bit;
@@ -127,7 +122,7 @@ static struct pathspec_magic {
 
 /*
  * Take an element of a pathspec and check for magic signatures.
- * Append the result to the prefix.
+ * Append the result to the prefix. Return the magic bitmap.
  *
  * For now, we only parse the syntax and throw out anything other than
  * "top" magic.
@@ -138,10 +133,15 @@ static struct pathspec_magic {
  * the prefix part must always match literally, and a single stupid
  * string cannot express such a case.
  */
-static const char *prefix_pathspec(const char *prefix, int prefixlen, const char *elt)
+static unsigned prefix_pathspec(struct pathspec_item *item,
+				unsigned *p_short_magic,
+				const char **raw, unsigned flags,
+				const char *prefix, int prefixlen,
+				const char *elt)
 {
-	unsigned magic = 0;
+	unsigned magic = 0, short_magic = 0;
 	const char *copyfrom = elt;
+	char *match;
 	int i;
 
 	if (elt[0] != ':') {
@@ -183,7 +183,7 @@ static const char *prefix_pathspec(const char *prefix, int prefixlen, const char
 				break;
 			for (i = 0; i < ARRAY_SIZE(pathspec_magic); i++)
 				if (pathspec_magic[i].mnemonic == ch) {
-					magic |= pathspec_magic[i].bit;
+					short_magic |= pathspec_magic[i].bit;
 					break;
 				}
 			if (ARRAY_SIZE(pathspec_magic) <= i)
@@ -194,15 +194,127 @@ static const char *prefix_pathspec(const char *prefix, int prefixlen, const char
 			copyfrom++;
 	}
 
+	magic |= short_magic;
+	*p_short_magic = short_magic;
+
 	if (magic & PATHSPEC_FROMTOP)
-		return xstrdup(copyfrom);
+		match = xstrdup(copyfrom);
 	else
-		return prefix_path(prefix, prefixlen, copyfrom);
+		match = prefix_path(prefix, prefixlen, copyfrom);
+	*raw = item->match = match;
+	item->len = strlen(item->match);
+	if (limit_pathspec_to_literal())
+		item->nowildcard_len = item->len;
+	else
+		item->nowildcard_len = simple_length(item->match);
+	item->flags = 0;
+	if (item->nowildcard_len < item->len &&
+	    item->match[item->nowildcard_len] == '*' &&
+	    no_wildcard(item->match + item->nowildcard_len + 1))
+		item->flags |= PATHSPEC_ONESTAR;
+	return magic;
+}
+
+static int pathspec_item_cmp(const void *a_, const void *b_)
+{
+	struct pathspec_item *a, *b;
+
+	a = (struct pathspec_item *)a_;
+	b = (struct pathspec_item *)b_;
+	return strcmp(a->match, b->match);
+}
+
+static void NORETURN unsupported_magic(const char *pattern,
+				       unsigned magic,
+				       unsigned short_magic)
+{
+	struct strbuf sb = STRBUF_INIT;
+	int i, n;
+	for (n = i = 0; i < ARRAY_SIZE(pathspec_magic); i++) {
+		const struct pathspec_magic *m = pathspec_magic + i;
+		if (!(magic & m->bit))
+			continue;
+		if (sb.len)
+			strbuf_addstr(&sb, " ");
+		if (short_magic & m->bit)
+			strbuf_addf(&sb, "'%c'", m->mnemonic);
+		else
+			strbuf_addf(&sb, "'%s'", m->name);
+		n++;
+	}
+	/*
+	 * We may want to substitue "this command" with a command
+	 * name. E.g. when add--interactive dies when running
+	 * "checkout -p"
+	 */
+	die(_("%s: pathspec magic not supported by this command: %s"),
+	    pattern, sb.buf);
+}
+
+/*
+ * Given command line arguments and a prefix, convert the input to
+ * pathspec. die() any magic in magic_mask is used.
+ */
+void parse_pathspec(struct pathspec *pathspec,
+		    unsigned magic_mask, unsigned flags,
+		    const char *prefix, const char **argv)
+{
+	struct pathspec_item *item;
+	const char *entry = argv ? *argv : NULL;
+	int i, n, prefixlen;
+
+	memset(pathspec, 0, sizeof(*pathspec));
+
+	/* No arguments, no prefix -> no pathspec */
+	if (!entry && !prefix)
+		return;
+
+	/* No arguments with prefix -> prefix pathspec */
+	if (!entry) {
+		static const char *raw[2];
+
+		pathspec->items = item = xmalloc(sizeof(*item));
+		item->match = prefix;
+		item->nowildcard_len = item->len = strlen(prefix);
+		raw[0] = prefix;
+		raw[1] = NULL;
+		pathspec->nr = 1;
+		pathspec->raw = raw;
+		return;
+	}
+
+	n = 0;
+	while (argv[n])
+		n++;
+
+	pathspec->nr = n;
+	pathspec->items = item = xmalloc(sizeof(*item) * n);
+	pathspec->raw = argv;
+	prefixlen = prefix ? strlen(prefix) : 0;
+
+	for (i = 0; i < n; i++) {
+		unsigned short_magic;
+		entry = argv[i];
+
+		item[i].magic = prefix_pathspec(item + i, &short_magic,
+						argv + i, flags,
+						prefix, prefixlen, entry);
+		if (item[i].magic & magic_mask)
+			unsupported_magic(entry,
+					  item[i].magic & magic_mask,
+					  short_magic);
+		if (item[i].nowildcard_len < item[i].len)
+			pathspec->has_wildcard = 1;
+		pathspec->magic |= item[i].magic;
+	}
+
+	qsort(pathspec->items, pathspec->nr,
+	      sizeof(struct pathspec_item), pathspec_item_cmp);
 }
 
 /*
  * N.B. get_pathspec() is deprecated in favor of the "struct pathspec"
- * based interface - see pathspec_magic above.
+ * based interface - see pathspec.c:parse_pathspec().
  *
  * Arguments:
  *  - prefix - a path relative to the root of the working tree
@@ -221,32 +333,11 @@ static const char *prefix_pathspec(const char *prefix, int prefixlen, const char
  */
 const char **get_pathspec(const char *prefix, const char **pathspec)
 {
-	const char *entry = *pathspec;
-	const char **src, **dst;
-	int prefixlen;
-
-	if (!prefix && !entry)
-		return NULL;
-
-	if (!entry) {
-		static const char *spec[2];
-		spec[0] = prefix;
-		spec[1] = NULL;
-		return spec;
-	}
-
-	/* Otherwise we have to re-write the entries.. */
-	src = pathspec;
-	dst = pathspec;
-	prefixlen = prefix ? strlen(prefix) : 0;
-	while (*src) {
-		*(dst++) = prefix_pathspec(prefix, prefixlen, *src);
-		src++;
-	}
-	*dst = NULL;
-	if (!*pathspec)
-		return NULL;
-	return pathspec;
+	struct pathspec ps;
+	parse_pathspec(&ps,
+		       PATHSPEC_ALL_MAGIC & ~PATHSPEC_FROMTOP,
+		       0, prefix, pathspec);
+	return ps.raw;
 }
 
 void copy_pathspec(struct pathspec *dst, const struct pathspec *src)
diff --git a/pathspec.h b/pathspec.h
index a621676..937ec91 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -1,6 +1,10 @@
 #ifndef PATHSPEC_H
 #define PATHSPEC_H
 
+/* Pathspec magic */
+#define PATHSPEC_FROMTOP	(1<<0)
+#define PATHSPEC_ALL_MAGIC PATHSPEC_FROMTOP
+
 #define PATHSPEC_ONESTAR 1	/* the pathspec pattern sastisfies GFNM_ONESTAR */
 
 struct pathspec {
@@ -8,9 +12,11 @@ struct pathspec {
 	int nr;
 	unsigned int has_wildcard:1;
 	unsigned int recursive:1;
+	unsigned magic;
 	int max_depth;
 	struct pathspec_item {
 		const char *match;
+		unsigned magic;
 		int len;
 		int nowildcard_len;
 		int flags;
@@ -18,6 +24,11 @@ struct pathspec {
 };
 
 extern int init_pathspec(struct pathspec *, const char **);
+extern void parse_pathspec(struct pathspec *pathspec,
+			   unsigned magic_mask,
+			   unsigned flags,
+			   const char *prefix,
+			   const char **args);
 extern void copy_pathspec(struct pathspec *dst, const struct pathspec *src);
 extern void free_pathspec(struct pathspec *);
 
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 07/45] parse_pathspec: save original pathspec for reporting

[Nguyễn Thái Ngọc Duy]

We usually use pathspec_item's match field for pathspec error
reporting. However "match" (or "raw") does not show the magic part,
which will play more important role later on. Preserve exact user
input for reporting.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 dir.c      | 1 +
 pathspec.c | 2 ++
 pathspec.h | 1 +
 3 files changed, 4 insertions(+)

diff --git a/dir.c b/dir.c
index a442467..8541ea7 100644
--- a/dir.c
+++ b/dir.c
@@ -1649,6 +1649,7 @@ int init_pathspec(struct pathspec *pathspec, const char **paths)
 		const char *path = paths[i];
 
 		item->match = path;
+		item->original = path;
 		item->len = strlen(path);
 		item->flags = 0;
 		if (limit_pathspec_to_literal()) {
diff --git a/pathspec.c b/pathspec.c
index bc9d9ac..2bda633 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -202,6 +202,7 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 	else
 		match = prefix_path(prefix, prefixlen, copyfrom);
 	*raw = item->match = match;
+	item->original = elt;
 	item->len = strlen(item->match);
 	if (limit_pathspec_to_literal())
 		item->nowildcard_len = item->len;
@@ -275,6 +276,7 @@ void parse_pathspec(struct pathspec *pathspec,
 
 		pathspec->items = item = xmalloc(sizeof(*item));
 		item->match = prefix;
+		item->original = prefix;
 		item->nowildcard_len = item->len = strlen(prefix);
 		raw[0] = prefix;
 		raw[1] = NULL;
diff --git a/pathspec.h b/pathspec.h
index 937ec91..cc5841b 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -16,6 +16,7 @@ struct pathspec {
 	int max_depth;
 	struct pathspec_item {
 		const char *match;
+		const char *original;
 		unsigned magic;
 		int len;
 		int nowildcard_len;
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 08/45] parse_pathspec: add PATHSPEC_PREFER_{CWD,FULL}

[Nguyễn Thái Ngọc Duy]

We have two ways of dealing with empty pathspec:

1. limit it to current prefix
2. match the entire working directory

Some commands go with #1, some #2. get_pathspec() and parse_pathspec()
only support #1. Make parse_pathspec() reject empty pathspec by
default. #1 and #2 can be specified via new flags. This makes it more
expressive about default behavior at command level.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 pathspec.c | 13 ++++++++++++-
 pathspec.h |  4 ++++
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/pathspec.c b/pathspec.c
index 2bda633..6dd944a 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -270,10 +270,20 @@ void parse_pathspec(struct pathspec *pathspec,
 	if (!entry && !prefix)
 		return;
 
+	if ((flags & PATHSPEC_PREFER_CWD) &&
+	    (flags & PATHSPEC_PREFER_FULL))
+		die("BUG: PATHSPEC_PREFER_CWD and PATHSPEC_PREFER_FULL are incompatible");
+
 	/* No arguments with prefix -> prefix pathspec */
 	if (!entry) {
 		static const char *raw[2];
 
+		if (flags & PATHSPEC_PREFER_FULL)
+			return;
+
+		if (!(flags & PATHSPEC_PREFER_CWD))
+			die("BUG: parse_pathspec cannot take no arguments in this case");
+
 		pathspec->items = item = xmalloc(sizeof(*item));
 		item->match = prefix;
 		item->original = prefix;
@@ -338,7 +348,8 @@ const char **get_pathspec(const char *prefix, const char **pathspec)
 	struct pathspec ps;
 	parse_pathspec(&ps,
 		       PATHSPEC_ALL_MAGIC & ~PATHSPEC_FROMTOP,
-		       0, prefix, pathspec);
+		       PATHSPEC_PREFER_CWD,
+		       prefix, pathspec);
 	return ps.raw;
 }
 
diff --git a/pathspec.h b/pathspec.h
index cc5841b..d630e8b 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -24,6 +24,10 @@ struct pathspec {
 	} *items;
 };
 
+/* parse_pathspec flags */
+#define PATHSPEC_PREFER_CWD (1<<0) /* No args means match cwd */
+#define PATHSPEC_PREFER_FULL (1<<1) /* No args means match everything */
+
 extern int init_pathspec(struct pathspec *, const char **);
 extern void parse_pathspec(struct pathspec *pathspec,
 			   unsigned magic_mask,
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 09/45] Convert some get_pathspec() calls to parse_pathspec()

[Nguyễn Thái Ngọc Duy]

These call sites follow the pattern:

   paths = get_pathspec(prefix, argv);
   init_pathspec(&pathspec, paths);

which can be converted into a single parse_pathspec() call.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/grep.c         |  6 +++---
 builtin/ls-tree.c      | 10 +++++++++-
 builtin/update-index.c |  5 +++--
 revision.c             |  4 ++--
 4 files changed, 17 insertions(+), 8 deletions(-)

diff --git a/builtin/grep.c b/builtin/grep.c
index 3701c2e..54b089e 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -631,7 +631,6 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
 	const char *show_in_pager = NULL, *default_pager = "dummy";
 	struct grep_opt opt;
 	struct object_array list = OBJECT_ARRAY_INIT;
-	const char **paths = NULL;
 	struct pathspec pathspec;
 	struct string_list path_list = STRING_LIST_INIT_NODUP;
 	int i;
@@ -860,8 +859,9 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
 			verify_filename(prefix, argv[j], j == i);
 	}
 
-	paths = get_pathspec(prefix, argv + i);
-	init_pathspec(&pathspec, paths);
+	parse_pathspec(&pathspec, 0,
+		       PATHSPEC_PREFER_CWD,
+		       prefix, argv + i);
 	pathspec.max_depth = opt.max_depth;
 	pathspec.recursive = 1;
 
diff --git a/builtin/ls-tree.c b/builtin/ls-tree.c
index 93fc3a0..bdb03f3 100644
--- a/builtin/ls-tree.c
+++ b/builtin/ls-tree.c
@@ -167,7 +167,15 @@ int cmd_ls_tree(int argc, const char **argv, const char *prefix)
 	if (get_sha1(argv[0], sha1))
 		die("Not a valid object name %s", argv[0]);
 
-	init_pathspec(&pathspec, get_pathspec(prefix, argv + 1));
+	/*
+	 * show_recursive() rolls its own matching code and is
+	 * generally ignorant of 'struct pathspec'. The magic mask
+	 * cannot be lifted until it is converted to use
+	 * match_pathspec_depth() or tree_entry_interesting()
+	 */
+	parse_pathspec(&pathspec, 0,
+		       PATHSPEC_PREFER_CWD,
+		       prefix, argv + 1);
 	for (i = 0; i < pathspec.nr; i++)
 		pathspec.items[i].nowildcard_len = pathspec.items[i].len;
 	pathspec.has_wildcard = 0;
diff --git a/builtin/update-index.c b/builtin/update-index.c
index ffae585..3c6905d 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -547,10 +547,11 @@ static int do_reupdate(int ac, const char **av,
 	 */
 	int pos;
 	int has_head = 1;
-	const char **paths = get_pathspec(prefix, av + 1);
 	struct pathspec pathspec;
 
-	init_pathspec(&pathspec, paths);
+	parse_pathspec(&pathspec, 0,
+		       PATHSPEC_PREFER_CWD,
+		       prefix, av + 1);
 
 	if (read_ref("HEAD", head_sha1))
 		/* If there is no HEAD, that means it is an initial
diff --git a/revision.c b/revision.c
index ef60205..e257614 100644
--- a/revision.c
+++ b/revision.c
@@ -1852,8 +1852,8 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s
 		 */
 		ALLOC_GROW(prune_data.path, prune_data.nr+1, prune_data.alloc);
 		prune_data.path[prune_data.nr++] = NULL;
-		init_pathspec(&revs->prune_data,
-			      get_pathspec(revs->prefix, prune_data.path));
+		parse_pathspec(&revs->prune_data, 0, 0,
+			       revs->prefix, prune_data.path);
 	}
 
 	if (revs->def == NULL)
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 10/45] parse_pathspec: a special flag for max_depth feature

[Nguyễn Thái Ngọc Duy]

match_pathspec_depth() and tree_entry_interesting() check max_depth
field in order to support "git grep --max-depth". The feature
activation is tied to "recursive" field, which led to some unwated
activation, e.g. 5c8eeb8 (diff-index: enable recursive pathspec
matching in unpack_trees - 2012-01-15).

This patch decouples the activation from "recursive" field, puts it in
"magic" field instead. This makes sure that only "git grep" can
activate this feature. And because parse_pathspec knows when the
feature is not used, it does not need to sort pathspec (required for
max_depth to work correctly). A small win for non-grep cases.

Even though a new magic flag is introduced, no magic syntax is. The
magic can be only enabled by parse_pathspec() caller. We might someday
want to support ":(maxdepth:10)src." It all depends on actual use
cases.

max_depth feature cannot be enabled via init_pathspec() anymore. But
that's ok because init_pathspec() is on its way to /dev/null.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/grep.c | 3 ++-
 diff-lib.c     | 1 -
 dir.c          | 8 ++++++--
 pathspec.c     | 8 ++++++--
 pathspec.h     | 6 +++++-
 tree-diff.c    | 1 -
 tree-walk.c    | 8 ++++++--
 7 files changed, 25 insertions(+), 10 deletions(-)

diff --git a/builtin/grep.c b/builtin/grep.c
index 54b089e..1c7397a 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -860,7 +860,8 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
 	}
 
 	parse_pathspec(&pathspec, 0,
-		       PATHSPEC_PREFER_CWD,
+		       PATHSPEC_PREFER_CWD |
+		       (opt.max_depth != -1 ? PATHSPEC_MAXDEPTH_VALID : 0),
 		       prefix, argv + i);
 	pathspec.max_depth = opt.max_depth;
 	pathspec.recursive = 1;
diff --git a/diff-lib.c b/diff-lib.c
index f35de0f..4729157 100644
--- a/diff-lib.c
+++ b/diff-lib.c
@@ -474,7 +474,6 @@ static int diff_cache(struct rev_info *revs,
 	opts.dst_index = NULL;
 	opts.pathspec = &revs->diffopt.pathspec;
 	opts.pathspec->recursive = 1;
-	opts.pathspec->max_depth = -1;
 
 	init_tree_desc(&t, tree->buffer, tree->size);
 	return unpack_trees(1, &t, &opts);
diff --git a/dir.c b/dir.c
index 8541ea7..1e9db41 100644
--- a/dir.c
+++ b/dir.c
@@ -298,7 +298,9 @@ int match_pathspec_depth(const struct pathspec *ps,
 	int i, retval = 0;
 
 	if (!ps->nr) {
-		if (!ps->recursive || ps->max_depth == -1)
+		if (!ps->recursive ||
+		    !(ps->magic & PATHSPEC_MAXDEPTH) ||
+		    ps->max_depth == -1)
 			return MATCHED_RECURSIVELY;
 
 		if (within_depth(name, namelen, 0, ps->max_depth))
@@ -315,7 +317,9 @@ int match_pathspec_depth(const struct pathspec *ps,
 		if (seen && seen[i] == MATCHED_EXACTLY)
 			continue;
 		how = match_pathspec_item(ps->items+i, prefix, name, namelen);
-		if (ps->recursive && ps->max_depth != -1 &&
+		if (ps->recursive &&
+		    (ps->magic & PATHSPEC_MAXDEPTH) &&
+		    ps->max_depth != -1 &&
 		    how && how != MATCHED_FNMATCH) {
 			int len = ps->items[i].len;
 			if (name[len] == '/')
diff --git a/pathspec.c b/pathspec.c
index 6dd944a..b2446c3 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -266,6 +266,9 @@ void parse_pathspec(struct pathspec *pathspec,
 
 	memset(pathspec, 0, sizeof(*pathspec));
 
+	if (flags & PATHSPEC_MAXDEPTH_VALID)
+		pathspec->magic |= PATHSPEC_MAXDEPTH;
+
 	/* No arguments, no prefix -> no pathspec */
 	if (!entry && !prefix)
 		return;
@@ -320,8 +323,9 @@ void parse_pathspec(struct pathspec *pathspec,
 		pathspec->magic |= item[i].magic;
 	}
 
-	qsort(pathspec->items, pathspec->nr,
-	      sizeof(struct pathspec_item), pathspec_item_cmp);
+	if (pathspec->magic & PATHSPEC_MAXDEPTH)
+		qsort(pathspec->items, pathspec->nr,
+		      sizeof(struct pathspec_item), pathspec_item_cmp);
 }
 
 /*
diff --git a/pathspec.h b/pathspec.h
index d630e8b..aa98597 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -3,7 +3,10 @@
 
 /* Pathspec magic */
 #define PATHSPEC_FROMTOP	(1<<0)
-#define PATHSPEC_ALL_MAGIC PATHSPEC_FROMTOP
+#define PATHSPEC_MAXDEPTH	(1<<1)
+#define PATHSPEC_ALL_MAGIC	  \
+	(PATHSPEC_FROMTOP	| \
+	 PATHSPEC_MAXDEPTH)
 
 #define PATHSPEC_ONESTAR 1	/* the pathspec pattern sastisfies GFNM_ONESTAR */
 
@@ -27,6 +30,7 @@ struct pathspec {
 /* parse_pathspec flags */
 #define PATHSPEC_PREFER_CWD (1<<0) /* No args means match cwd */
 #define PATHSPEC_PREFER_FULL (1<<1) /* No args means match everything */
+#define PATHSPEC_MAXDEPTH_VALID (1<<2) /* max_depth field is valid */
 
 extern int init_pathspec(struct pathspec *, const char **);
 extern void parse_pathspec(struct pathspec *pathspec,
diff --git a/tree-diff.c b/tree-diff.c
index ba01563..826512e 100644
--- a/tree-diff.c
+++ b/tree-diff.c
@@ -138,7 +138,6 @@ int diff_tree(struct tree_desc *t1, struct tree_desc *t2,
 
 	/* Enable recursion indefinitely */
 	opt->pathspec.recursive = DIFF_OPT_TST(opt, RECURSIVE);
-	opt->pathspec.max_depth = -1;
 
 	strbuf_init(&base, PATH_MAX);
 	strbuf_add(&base, base_str, baselen);
diff --git a/tree-walk.c b/tree-walk.c
index 72a9613..d399ca9 100644
--- a/tree-walk.c
+++ b/tree-walk.c
@@ -637,7 +637,9 @@ enum interesting tree_entry_interesting(const struct name_entry *entry,
 		entry_not_interesting : all_entries_not_interesting;
 
 	if (!ps->nr) {
-		if (!ps->recursive || ps->max_depth == -1)
+		if (!ps->recursive ||
+		    !(ps->magic & PATHSPEC_MAXDEPTH) ||
+		    ps->max_depth == -1)
 			return all_entries_interesting;
 		return within_depth(base->buf + base_offset, baselen,
 				    !!S_ISDIR(entry->mode),
@@ -658,7 +660,9 @@ enum interesting tree_entry_interesting(const struct name_entry *entry,
 			if (!match_dir_prefix(base_str, match, matchlen))
 				goto match_wildcards;
 
-			if (!ps->recursive || ps->max_depth == -1)
+			if (!ps->recursive ||
+			    !(ps->magic & PATHSPEC_MAXDEPTH) ||
+			    ps->max_depth == -1)
 				return all_entries_interesting;
 
 			return within_depth(base_str + matchlen + 1,
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 11/45] parse_pathspec: support stripping submodule trailing slashes

[Nguyễn Thái Ngọc Duy]

This flag is equivalent to builtin/ls-files.c:strip_trailing_slashes()
and is intended to replace that function when ls-files is converted to
use parse_pathspec.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 pathspec.c | 9 +++++++++
 pathspec.h | 1 +
 2 files changed, 10 insertions(+)

diff --git a/pathspec.c b/pathspec.c
index b2446c3..2da8bc9 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -204,6 +204,15 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 	*raw = item->match = match;
 	item->original = elt;
 	item->len = strlen(item->match);
+
+	if ((flags & PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP) &&
+	    (item->len >= 1 && item->match[item->len - 1] == '/') &&
+	    (i = cache_name_pos(item->match, item->len - 1)) >= 0 &&
+	    S_ISGITLINK(active_cache[i]->ce_mode)) {
+		item->len--;
+		match[item->len] = '\0';
+	}
+
 	if (limit_pathspec_to_literal())
 		item->nowildcard_len = item->len;
 	else
diff --git a/pathspec.h b/pathspec.h
index aa98597..f24b07b 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -31,6 +31,7 @@ struct pathspec {
 #define PATHSPEC_PREFER_CWD (1<<0) /* No args means match cwd */
 #define PATHSPEC_PREFER_FULL (1<<1) /* No args means match everything */
 #define PATHSPEC_MAXDEPTH_VALID (1<<2) /* max_depth field is valid */
+#define PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP (1<<3)
 
 extern int init_pathspec(struct pathspec *, const char **);
 extern void parse_pathspec(struct pathspec *pathspec,
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 12/45] parse_pathspec: support stripping/checking submodule paths

[Nguyễn Thái Ngọc Duy]

PATHSPEC_SYMLINK_LEADING_PATH and _STRIP_SUBMODULE_SLASH_EXPENSIVE are
respectively the alternate implementation of
pathspec.c:die_if_path_beyond_symlink() and
pathspec.c:check_path_for_gitlink(). They are intended to replace
those functions when builtin/add.c and builtin/check-ignore.c are
converted to use parse_pathspec.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 pathspec.c | 26 ++++++++++++++++++++++++++
 pathspec.h |  2 ++
 2 files changed, 28 insertions(+)

diff --git a/pathspec.c b/pathspec.c
index 2da8bc9..834db80 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -213,6 +213,26 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 		match[item->len] = '\0';
 	}
 
+	if (flags & PATHSPEC_STRIP_SUBMODULE_SLASH_EXPENSIVE)
+		for (i = 0; i < active_nr; i++) {
+			struct cache_entry *ce = active_cache[i];
+			int ce_len = ce_namelen(ce);
+
+			if (!S_ISGITLINK(ce->ce_mode))
+				continue;
+
+			if (item->len <= ce_len || match[ce_len] != '/' ||
+			    memcmp(ce->name, match, ce_len))
+				continue;
+			if (item->len == ce_len + 1) {
+				/* strip trailing slash */
+				item->len--;
+				match[item->len] = '\0';
+			} else
+				die (_("Pathspec '%s' is in submodule '%.*s'"),
+				     elt, ce_len, ce->name);
+		}
+
 	if (limit_pathspec_to_literal())
 		item->nowildcard_len = item->len;
 	else
@@ -327,6 +347,12 @@ void parse_pathspec(struct pathspec *pathspec,
 			unsupported_magic(entry,
 					  item[i].magic & magic_mask,
 					  short_magic);
+
+		if ((flags & PATHSPEC_SYMLINK_LEADING_PATH) &&
+		    has_symlink_leading_path(item[i].match, item[i].len)) {
+			die(_("pathspec '%s' is beyond a symbolic link"), entry);
+		}
+
 		if (item[i].nowildcard_len < item[i].len)
 			pathspec->has_wildcard = 1;
 		pathspec->magic |= item[i].magic;
diff --git a/pathspec.h b/pathspec.h
index f24b07b..6ffd09d 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -32,6 +32,8 @@ struct pathspec {
 #define PATHSPEC_PREFER_FULL (1<<1) /* No args means match everything */
 #define PATHSPEC_MAXDEPTH_VALID (1<<2) /* max_depth field is valid */
 #define PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP (1<<3)
+#define PATHSPEC_SYMLINK_LEADING_PATH (1<<4)
+#define PATHSPEC_STRIP_SUBMODULE_SLASH_EXPENSIVE (1<<5)
 
 extern int init_pathspec(struct pathspec *, const char **);
 extern void parse_pathspec(struct pathspec *pathspec,
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 13/45] parse_pathspec: support prefixing original patterns

[Nguyễn Thái Ngọc Duy]

This makes 'original' suitable for passing to an external command
because all pathspec magic is left in place, provided that the
external command understands pathspec. The prefixing is needed because
we usually launch a subcommand at worktree's top directory and the
subcommand can no longer calculate the prefix itself.

This slightly affects the original purpose of 'original'
(i.e. reporting). We should report without prefixing. So only turn
this flag on when you know you are about to pass the result straight
away to an external command.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 pathspec.c | 12 +++++++++++-
 pathspec.h |  1 +
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/pathspec.c b/pathspec.c
index 834db80..9a57c0c 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -202,7 +202,17 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 	else
 		match = prefix_path(prefix, prefixlen, copyfrom);
 	*raw = item->match = match;
-	item->original = elt;
+	/*
+	 * Prefix the pathspec (keep all magic) and put to
+	 * original. Useful for passing to another command.
+	 */
+	if (flags & PATHSPEC_PREFIX_ORIGIN) {
+		struct strbuf sb = STRBUF_INIT;
+		strbuf_add(&sb, elt, copyfrom - elt);
+		strbuf_addstr(&sb, match);
+		item->original = strbuf_detach(&sb, NULL);
+	} else
+		item->original = elt;
 	item->len = strlen(item->match);
 
 	if ((flags & PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP) &&
diff --git a/pathspec.h b/pathspec.h
index 6ffd09d..1cef9c6 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -34,6 +34,7 @@ struct pathspec {
 #define PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP (1<<3)
 #define PATHSPEC_SYMLINK_LEADING_PATH (1<<4)
 #define PATHSPEC_STRIP_SUBMODULE_SLASH_EXPENSIVE (1<<5)
+#define PATHSPEC_PREFIX_ORIGIN (1<<6)
 
 extern int init_pathspec(struct pathspec *, const char **);
 extern void parse_pathspec(struct pathspec *pathspec,
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 14/45] Guard against new pathspec magic in pathspec matching code

[Nguyễn Thái Ngọc Duy]

GUARD_PATHSPEC() marks pathspec-sensitive code (basically anything in
'struct pathspec' except fields "nr" and "original"). GUARD_PATHSPEC()
is not supposed to fail. The steps for a new pathspec magic or
optimization would be:

 - update parse_pathspec, add extra information to struct pathspec

 - grep GUARD_PATHSPEC() and update all relevant code (or note those
   that won't work with your new stuff). Update GUARD_PATHSPEC mask
   accordingly.

 - update parse_pathspec calls to disable new magic early at command
   parsing level. Make sure parse_pathspec() catches unsupported
   syntax, not until GUARD_PATHSPEC catches it.

 - add tests to verify supported/unsupported commands both work as
   expected.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/diff.c |  2 ++
 dir.c          |  2 ++
 pathspec.h     |  7 +++++++
 tree-diff.c    | 19 +++++++++++++++++++
 tree-walk.c    |  2 ++
 5 files changed, 32 insertions(+)

diff --git a/builtin/diff.c b/builtin/diff.c
index 8c2af6c..d237e0a 100644
--- a/builtin/diff.c
+++ b/builtin/diff.c
@@ -371,6 +371,8 @@ int cmd_diff(int argc, const char **argv, const char *prefix)
 		die(_("unhandled object '%s' given."), name);
 	}
 	if (rev.prune_data.nr) {
+		/* builtin_diff_b_f() */
+		GUARD_PATHSPEC(&rev.prune_data, PATHSPEC_FROMTOP);
 		if (!path)
 			path = rev.prune_data.items[0].match;
 		paths += rev.prune_data.nr;
diff --git a/dir.c b/dir.c
index 1e9db41..6094ba8 100644
--- a/dir.c
+++ b/dir.c
@@ -297,6 +297,8 @@ int match_pathspec_depth(const struct pathspec *ps,
 {
 	int i, retval = 0;
 
+	GUARD_PATHSPEC(ps, PATHSPEC_FROMTOP | PATHSPEC_MAXDEPTH);
+
 	if (!ps->nr) {
 		if (!ps->recursive ||
 		    !(ps->magic & PATHSPEC_MAXDEPTH) ||
diff --git a/pathspec.h b/pathspec.h
index 1cef9c6..ed5d3a6 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -27,6 +27,13 @@ struct pathspec {
 	} *items;
 };
 
+#define GUARD_PATHSPEC(ps, mask) \
+	do { \
+		if ((ps)->magic & ~(mask))	       \
+			die("BUG:%s:%d: unsupported magic %x",	\
+			    __FILE__, __LINE__, (ps)->magic & ~(mask)); \
+	} while (0)
+
 /* parse_pathspec flags */
 #define PATHSPEC_PREFER_CWD (1<<0) /* No args means match cwd */
 #define PATHSPEC_PREFER_FULL (1<<1) /* No args means match everything */
diff --git a/tree-diff.c b/tree-diff.c
index 826512e..5a87614 100644
--- a/tree-diff.c
+++ b/tree-diff.c
@@ -198,6 +198,25 @@ static void try_to_follow_renames(struct tree_desc *t1, struct tree_desc *t2, co
 	const char *paths[1];
 	int i;
 
+	/*
+	 * follow-rename code is very specific, we need exactly one
+	 * path. Magic that matches more than one path is not
+	 * supported.
+	 */
+	GUARD_PATHSPEC(&opt->pathspec, PATHSPEC_FROMTOP);
+#if 0
+	/*
+	 * We should reject wildcards as well. Unfortunately we
+	 * haven't got a reliable way to detect that 'foo\*bar' in
+	 * fact has no wildcards. nowildcard_len is merely a hint for
+	 * optimization. Let it slip for now until wildmatch is taught
+	 * about dry-run mode and returns wildcard info.
+	 */
+	if (opt->pathspec.has_wildcard)
+		die("BUG:%s:%d: wildcards are not supported",
+		    __FILE__, __LINE__);
+#endif
+
 	/* Remove the file creation entry from the diff queue, and remember it */
 	choice = q->queue[0];
 	q->nr = 0;
diff --git a/tree-walk.c b/tree-walk.c
index d399ca9..37b157e 100644
--- a/tree-walk.c
+++ b/tree-walk.c
@@ -636,6 +636,8 @@ enum interesting tree_entry_interesting(const struct name_entry *entry,
 	enum interesting never_interesting = ps->has_wildcard ?
 		entry_not_interesting : all_entries_not_interesting;
 
+	GUARD_PATHSPEC(ps, PATHSPEC_FROMTOP | PATHSPEC_MAXDEPTH);
+
 	if (!ps->nr) {
 		if (!ps->recursive ||
 		    !(ps->magic & PATHSPEC_MAXDEPTH) ||
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 15/45] clean: convert to use parse_pathspec

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/clean.c | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/builtin/clean.c b/builtin/clean.c
index f955a40..fdd4980 100644
--- a/builtin/clean.c
+++ b/builtin/clean.c
@@ -13,6 +13,7 @@
 #include "refs.h"
 #include "string-list.h"
 #include "quote.h"
+#include "pathspec.h"
 
 static int force = -1; /* unset */
 
@@ -150,7 +151,7 @@ int cmd_clean(int argc, const char **argv, const char *prefix)
 	int rm_flags = REMOVE_DIR_KEEP_NESTED_GIT;
 	struct strbuf directory = STRBUF_INIT;
 	struct dir_struct dir;
-	static const char **pathspec;
+	struct pathspec pathspec;
 	struct strbuf buf = STRBUF_INIT;
 	struct string_list exclude_list = STRING_LIST_INIT_NODUP;
 	struct exclude_list *el;
@@ -209,9 +210,11 @@ int cmd_clean(int argc, const char **argv, const char *prefix)
 	for (i = 0; i < exclude_list.nr; i++)
 		add_exclude(exclude_list.items[i].string, "", 0, el, -(i+1));
 
-	pathspec = get_pathspec(prefix, argv);
+	parse_pathspec(&pathspec, 0,
+		       PATHSPEC_PREFER_CWD,
+		       prefix, argv);
 
-	fill_directory(&dir, pathspec);
+	fill_directory(&dir, pathspec.raw);
 
 	for (i = 0; i < dir.nr; i++) {
 		struct dir_entry *ent = dir.entries[i];
@@ -246,9 +249,9 @@ int cmd_clean(int argc, const char **argv, const char *prefix)
 		if (lstat(ent->name, &st))
 			continue;
 
-		if (pathspec)
-			matches = match_pathspec(pathspec, ent->name, len,
-						 0, NULL);
+		if (pathspec.nr)
+			matches = match_pathspec_depth(&pathspec, ent->name,
+						       len, 0, NULL);
 
 		if (S_ISDIR(st.st_mode)) {
 			strbuf_addstr(&directory, ent->name);
@@ -262,7 +265,7 @@ int cmd_clean(int argc, const char **argv, const char *prefix)
 			}
 			strbuf_reset(&directory);
 		} else {
-			if (pathspec && !matches)
+			if (pathspec.nr && !matches)
 				continue;
 			res = dry_run ? 0 : unlink(ent->name);
 			if (res) {
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 16/45] commit: convert to use parse_pathspec

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/commit.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index 3348aa1..ba6731b 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -277,17 +277,17 @@ static char *prepare_index(int argc, const char **argv, const char *prefix,
 {
 	int fd;
 	struct string_list partial;
-	const char **pathspec = NULL;
+	struct pathspec pathspec;
 	char *old_index_env = NULL;
 	int refresh_flags = REFRESH_QUIET;
 
 	if (is_status)
 		refresh_flags |= REFRESH_UNMERGED;
+	parse_pathspec(&pathspec, 0,
+		       PATHSPEC_PREFER_FULL,
+		       prefix, argv);
 
-	if (*argv)
-		pathspec = get_pathspec(prefix, argv);
-
-	if (read_cache_preload(pathspec) < 0)
+	if (read_cache_preload(pathspec.raw) < 0)
 		die(_("index file corrupt"));
 
 	if (interactive) {
@@ -329,9 +329,9 @@ static char *prepare_index(int argc, const char **argv, const char *prefix,
 	 * (A) if all goes well, commit the real index;
 	 * (B) on failure, rollback the real index.
 	 */
-	if (all || (also && pathspec && *pathspec)) {
+	if (all || (also && pathspec.nr)) {
 		fd = hold_locked_index(&index_lock, 1);
-		add_files_to_cache(also ? prefix : NULL, pathspec, 0);
+		add_files_to_cache(also ? prefix : NULL, pathspec.raw, 0);
 		refresh_cache_or_die(refresh_flags);
 		update_main_cache_tree(WRITE_TREE_SILENT);
 		if (write_cache(fd, active_cache, active_nr) ||
@@ -350,7 +350,7 @@ static char *prepare_index(int argc, const char **argv, const char *prefix,
 	 * and create commit from the_index.
 	 * We still need to refresh the index here.
 	 */
-	if (!only && (!pathspec || !*pathspec)) {
+	if (!only && !pathspec.nr) {
 		fd = hold_locked_index(&index_lock, 1);
 		refresh_cache_or_die(refresh_flags);
 		if (active_cache_changed) {
@@ -395,7 +395,7 @@ static char *prepare_index(int argc, const char **argv, const char *prefix,
 
 	memset(&partial, 0, sizeof(partial));
 	partial.strdup_strings = 1;
-	if (list_paths(&partial, !current_head ? NULL : "HEAD", prefix, pathspec))
+	if (list_paths(&partial, !current_head ? NULL : "HEAD", prefix, pathspec.raw))
 		exit(1);
 
 	discard_cache();
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 17/45] status: convert to use parse_pathspec

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/commit.c |  9 +++++----
 wt-status.c      | 18 ++++++++----------
 wt-status.h      |  2 +-
 3 files changed, 14 insertions(+), 15 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index ba6731b..d8e6b28 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1195,11 +1195,12 @@ int cmd_status(int argc, const char **argv, const char *prefix)
 	handle_untracked_files_arg(&s);
 	if (show_ignored_in_status)
 		s.show_ignored_files = 1;
-	if (*argv)
-		s.pathspec = get_pathspec(prefix, argv);
+	parse_pathspec(&s.pathspec, 0,
+		       PATHSPEC_PREFER_FULL,
+		       prefix, argv);
 
-	read_cache_preload(s.pathspec);
-	refresh_index(&the_index, REFRESH_QUIET|REFRESH_UNMERGED, s.pathspec, NULL, NULL);
+	read_cache_preload(s.pathspec.raw);
+	refresh_index(&the_index, REFRESH_QUIET|REFRESH_UNMERGED, s.pathspec.raw, NULL, NULL);
 
 	fd = hold_locked_index(&index_lock, 0);
 	if (0 <= fd)
diff --git a/wt-status.c b/wt-status.c
index ef405d0..5dec9af 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "pathspec.h"
 #include "wt-status.h"
 #include "object.h"
 #include "dir.h"
@@ -434,7 +435,7 @@ static void wt_status_collect_changes_worktree(struct wt_status *s)
 	}
 	rev.diffopt.format_callback = wt_status_collect_changed_cb;
 	rev.diffopt.format_callback_data = s;
-	init_pathspec(&rev.prune_data, s->pathspec);
+	copy_pathspec(&rev.prune_data, &s->pathspec);
 	run_diff_files(&rev, 0);
 }
 
@@ -459,22 +460,20 @@ static void wt_status_collect_changes_index(struct wt_status *s)
 	rev.diffopt.detect_rename = 1;
 	rev.diffopt.rename_limit = 200;
 	rev.diffopt.break_opt = 0;
-	init_pathspec(&rev.prune_data, s->pathspec);
+	copy_pathspec(&rev.prune_data, &s->pathspec);
 	run_diff_index(&rev, 1);
 }
 
 static void wt_status_collect_changes_initial(struct wt_status *s)
 {
-	struct pathspec pathspec;
 	int i;
 
-	init_pathspec(&pathspec, s->pathspec);
 	for (i = 0; i < active_nr; i++) {
 		struct string_list_item *it;
 		struct wt_status_change_data *d;
 		struct cache_entry *ce = active_cache[i];
 
-		if (!ce_path_match(ce, &pathspec))
+		if (!ce_path_match(ce, &s->pathspec))
 			continue;
 		it = string_list_insert(&s->change, ce->name);
 		d = it->util;
@@ -489,7 +488,6 @@ static void wt_status_collect_changes_initial(struct wt_status *s)
 		else
 			d->index_status = DIFF_STATUS_ADDED;
 	}
-	free_pathspec(&pathspec);
 }
 
 static void wt_status_collect_untracked(struct wt_status *s)
@@ -505,11 +503,11 @@ static void wt_status_collect_untracked(struct wt_status *s)
 			DIR_SHOW_OTHER_DIRECTORIES | DIR_HIDE_EMPTY_DIRECTORIES;
 	setup_standard_excludes(&dir);
 
-	fill_directory(&dir, s->pathspec);
+	fill_directory(&dir, s->pathspec.raw);
 	for (i = 0; i < dir.nr; i++) {
 		struct dir_entry *ent = dir.entries[i];
 		if (cache_name_is_other(ent->name, ent->len) &&
-		    match_pathspec(s->pathspec, ent->name, ent->len, 0, NULL))
+		    match_pathspec_depth(&s->pathspec, ent->name, ent->len, 0, NULL))
 			string_list_insert(&s->untracked, ent->name);
 		free(ent);
 	}
@@ -519,11 +517,11 @@ static void wt_status_collect_untracked(struct wt_status *s)
 		dir.flags = DIR_SHOW_IGNORED;
 		if (s->show_untracked_files != SHOW_ALL_UNTRACKED_FILES)
 			dir.flags |= DIR_SHOW_OTHER_DIRECTORIES;
-		fill_directory(&dir, s->pathspec);
+		fill_directory(&dir, s->pathspec.raw);
 		for (i = 0; i < dir.nr; i++) {
 			struct dir_entry *ent = dir.entries[i];
 			if (cache_name_is_other(ent->name, ent->len) &&
-			    match_pathspec(s->pathspec, ent->name, ent->len, 0, NULL))
+			    match_pathspec_depth(&s->pathspec, ent->name, ent->len, 0, NULL))
 				string_list_insert(&s->ignored, ent->name);
 			free(ent);
 		}
diff --git a/wt-status.h b/wt-status.h
index 81e1dcf..bb01c23 100644
--- a/wt-status.h
+++ b/wt-status.h
@@ -44,7 +44,7 @@ struct wt_status {
 	int is_initial;
 	char *branch;
 	const char *reference;
-	const char **pathspec;
+	struct pathspec pathspec;
 	int verbose;
 	int amend;
 	enum commit_whence whence;
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 18/45] rerere: convert to use parse_pathspec

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/rerere.c | 8 +++++---
 rerere.c         | 9 +++++----
 rerere.h         | 4 +++-
 3 files changed, 13 insertions(+), 8 deletions(-)

diff --git a/builtin/rerere.c b/builtin/rerere.c
index dc1708e..4e51add 100644
--- a/builtin/rerere.c
+++ b/builtin/rerere.c
@@ -6,6 +6,7 @@
 #include "rerere.h"
 #include "xdiff/xdiff.h"
 #include "xdiff-interface.h"
+#include "pathspec.h"
 
 static const char * const rerere_usage[] = {
 	N_("git rerere [clear | forget path... | status | remaining | diff | gc]"),
@@ -68,11 +69,12 @@ int cmd_rerere(int argc, const char **argv, const char *prefix)
 		return rerere(flags);
 
 	if (!strcmp(argv[0], "forget")) {
-		const char **pathspec;
+		struct pathspec pathspec;
 		if (argc < 2)
 			warning("'git rerere forget' without paths is deprecated");
-		pathspec = get_pathspec(prefix, argv + 1);
-		return rerere_forget(pathspec);
+		parse_pathspec(&pathspec, 0, PATHSPEC_PREFER_CWD,
+			       prefix, argv + 1);
+		return rerere_forget(&pathspec);
 	}
 
 	fd = setup_rerere(&merge_rr, flags);
diff --git a/rerere.c b/rerere.c
index a6a5cd5..c52c1f2 100644
--- a/rerere.c
+++ b/rerere.c
@@ -6,6 +6,7 @@
 #include "resolve-undo.h"
 #include "ll-merge.h"
 #include "attr.h"
+#include "pathspec.h"
 
 #define RESOLVED 0
 #define PUNTED 1
@@ -655,7 +656,7 @@ static int rerere_forget_one_path(const char *path, struct string_list *rr)
 	return 0;
 }
 
-int rerere_forget(const char **pathspec)
+int rerere_forget(struct pathspec *pathspec)
 {
 	int i, fd;
 	struct string_list conflict = STRING_LIST_INIT_DUP;
@@ -666,12 +667,12 @@ int rerere_forget(const char **pathspec)
 
 	fd = setup_rerere(&merge_rr, RERERE_NOAUTOUPDATE);
 
-	unmerge_cache(pathspec);
+	unmerge_cache(pathspec->raw);
 	find_conflict(&conflict);
 	for (i = 0; i < conflict.nr; i++) {
 		struct string_list_item *it = &conflict.items[i];
-		if (!match_pathspec(pathspec, it->string, strlen(it->string),
-				    0, NULL))
+		if (!match_pathspec_depth(pathspec, it->string, strlen(it->string),
+					  0, NULL))
 			continue;
 		rerere_forget_one_path(it->string, &merge_rr);
 	}
diff --git a/rerere.h b/rerere.h
index 156d2aa..4aa06c9 100644
--- a/rerere.h
+++ b/rerere.h
@@ -3,6 +3,8 @@
 
 #include "string-list.h"
 
+struct pathspec;
+
 #define RERERE_AUTOUPDATE   01
 #define RERERE_NOAUTOUPDATE 02
 
@@ -16,7 +18,7 @@ extern void *RERERE_RESOLVED;
 extern int setup_rerere(struct string_list *, int);
 extern int rerere(int);
 extern const char *rerere_path(const char *hex, const char *file);
-extern int rerere_forget(const char **);
+extern int rerere_forget(struct pathspec *);
 extern int rerere_remaining(struct string_list *);
 extern void rerere_clear(struct string_list *);
 extern void rerere_gc(struct string_list *);
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 19/45] checkout: convert to use parse_pathspec

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/checkout.c | 40 ++++++++++++++++++++++++----------------
 1 file changed, 24 insertions(+), 16 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index a9c1b5a..8dc5f51 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -45,7 +45,7 @@ struct checkout_opts {
 
 	int branch_exists;
 	const char *prefix;
-	const char **pathspec;
+	struct pathspec pathspec;
 	struct tree *source_tree;
 };
 
@@ -256,39 +256,37 @@ static int checkout_paths(const struct checkout_opts *opts,
 
 	if (opts->patch_mode)
 		return run_add_interactive(revision, "--patch=checkout",
-					   opts->pathspec);
+					   opts->pathspec.raw);
 
 	lock_file = xcalloc(1, sizeof(struct lock_file));
 
 	newfd = hold_locked_index(lock_file, 1);
-	if (read_cache_preload(opts->pathspec) < 0)
+	if (read_cache_preload(opts->pathspec.raw) < 0)
 		return error(_("corrupt index file"));
 
 	if (opts->source_tree)
-		read_tree_some(opts->source_tree, opts->pathspec);
+		read_tree_some(opts->source_tree, opts->pathspec.raw);
 
-	for (pos = 0; opts->pathspec[pos]; pos++)
-		;
-	ps_matched = xcalloc(1, pos);
+	ps_matched = xcalloc(1, opts->pathspec.nr);
 
 	for (pos = 0; pos < active_nr; pos++) {
 		struct cache_entry *ce = active_cache[pos];
 		if (opts->source_tree && !(ce->ce_flags & CE_UPDATE))
 			continue;
-		match_pathspec(opts->pathspec, ce->name, ce_namelen(ce), 0, ps_matched);
+		match_pathspec_depth(&opts->pathspec, ce->name, ce_namelen(ce), 0, ps_matched);
 	}
 
-	if (report_path_error(ps_matched, opts->pathspec, opts->prefix))
+	if (report_path_error(ps_matched, opts->pathspec.raw, opts->prefix))
 		return 1;
 
 	/* "checkout -m path" to recreate conflicted state */
 	if (opts->merge)
-		unmerge_cache(opts->pathspec);
+		unmerge_cache(opts->pathspec.raw);
 
 	/* Any unmerged paths? */
 	for (pos = 0; pos < active_nr; pos++) {
 		struct cache_entry *ce = active_cache[pos];
-		if (match_pathspec(opts->pathspec, ce->name, ce_namelen(ce), 0, NULL)) {
+		if (match_pathspec_depth(&opts->pathspec, ce->name, ce_namelen(ce), 0, NULL)) {
 			if (!ce_stage(ce))
 				continue;
 			if (opts->force) {
@@ -315,7 +313,7 @@ static int checkout_paths(const struct checkout_opts *opts,
 		struct cache_entry *ce = active_cache[pos];
 		if (opts->source_tree && !(ce->ce_flags & CE_UPDATE))
 			continue;
-		if (match_pathspec(opts->pathspec, ce->name, ce_namelen(ce), 0, NULL)) {
+		if (match_pathspec_depth(&opts->pathspec, ce->name, ce_namelen(ce), 0, NULL)) {
 			if (!ce_stage(ce)) {
 				errs |= checkout_entry(ce, &state, NULL);
 				continue;
@@ -960,7 +958,7 @@ static int switch_unborn_to_new_branch(const struct checkout_opts *opts)
 static int checkout_branch(struct checkout_opts *opts,
 			   struct branch_info *new)
 {
-	if (opts->pathspec)
+	if (opts->pathspec.nr)
 		die(_("paths cannot be used with switching branches"));
 
 	if (opts->patch_mode)
@@ -1110,9 +1108,19 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 	}
 
 	if (argc) {
-		opts.pathspec = get_pathspec(prefix, argv);
+		/*
+		 * In patch mode (opts.patch_mode != 0), we pass the
+		 * pathspec to an external program, git-add--interactive.
+		 * Do not accept any kind of magic that that program
+		 * cannot handle. Magic mask is pretty safe to be
+		 * lifted for new magic when opts.patch_mode == 0.
+		 */
+		parse_pathspec(&opts.pathspec,
+			       opts.patch_mode == 0 ? 0 :
+			       (PATHSPEC_ALL_MAGIC & ~PATHSPEC_FROMTOP),
+			       0, prefix, argv);
 
-		if (!opts.pathspec)
+		if (!opts.pathspec.nr)
 			die(_("invalid path specification"));
 
 		/*
@@ -1144,7 +1152,7 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 		strbuf_release(&buf);
 	}
 
-	if (opts.patch_mode || opts.pathspec)
+	if (opts.patch_mode || opts.pathspec.nr)
 		return checkout_paths(&opts, new.name);
 	else
 		return checkout_branch(&opts, &new);
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 20/45] rm: convert to use parse_pathspec

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/rm.c | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/builtin/rm.c b/builtin/rm.c
index dabfcf6..eb1b745 100644
--- a/builtin/rm.c
+++ b/builtin/rm.c
@@ -10,6 +10,7 @@
 #include "tree-walk.h"
 #include "parse-options.h"
 #include "submodule.h"
+#include "pathspec.h"
 
 static const char * const builtin_rm_usage[] = {
 	N_("git rm [options] [--] <file>..."),
@@ -216,7 +217,7 @@ static struct option builtin_rm_options[] = {
 int cmd_rm(int argc, const char **argv, const char *prefix)
 {
 	int i, newfd;
-	const char **pathspec;
+	struct pathspec pathspec;
 	char *seen;
 
 	git_config(git_default_config, NULL);
@@ -249,31 +250,30 @@ int cmd_rm(int argc, const char **argv, const char *prefix)
 		}
 	}
 
-	pathspec = get_pathspec(prefix, argv);
-	refresh_index(&the_index, REFRESH_QUIET, pathspec, NULL, NULL);
+	parse_pathspec(&pathspec, 0, PATHSPEC_PREFER_CWD, prefix, argv);
+	refresh_index(&the_index, REFRESH_QUIET, pathspec.raw, NULL, NULL);
 
 	seen = NULL;
-	for (i = 0; pathspec[i] ; i++)
-		/* nothing */;
-	seen = xcalloc(i, 1);
+	seen = xcalloc(pathspec.nr, 1);
 
 	for (i = 0; i < active_nr; i++) {
 		struct cache_entry *ce = active_cache[i];
-		if (!match_pathspec(pathspec, ce->name, ce_namelen(ce), 0, seen))
+		if (!match_pathspec_depth(&pathspec, ce->name, ce_namelen(ce), 0, seen))
 			continue;
 		ALLOC_GROW(list.entry, list.nr + 1, list.alloc);
 		list.entry[list.nr].name = ce->name;
 		list.entry[list.nr++].is_submodule = S_ISGITLINK(ce->ce_mode);
 	}
 
-	if (pathspec) {
-		const char *match;
+	if (pathspec.nr) {
+		const char *original;
 		int seen_any = 0;
-		for (i = 0; (match = pathspec[i]) != NULL ; i++) {
+		for (i = 0; i < pathspec.nr; i++) {
+			original = pathspec.items[i].original;
 			if (!seen[i]) {
 				if (!ignore_unmatch) {
 					die(_("pathspec '%s' did not match any files"),
-					    match);
+					    original);
 				}
 			}
 			else {
@@ -281,7 +281,7 @@ int cmd_rm(int argc, const char **argv, const char *prefix)
 			}
 			if (!recursive && seen[i] == MATCHED_RECURSIVELY)
 				die(_("not removing '%s' recursively without -r"),
-				    *match ? match : ".");
+				    *original ? original : ".");
 		}
 
 		if (! seen_any)
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 21/45] ls-files: convert to use parse_pathspec

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/ls-files.c | 46 +++++++++++++---------------------------------
 1 file changed, 13 insertions(+), 33 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index 752570b..bd64829 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -31,7 +31,7 @@ static int debug_mode;
 static const char *prefix;
 static int max_prefix_len;
 static int prefix_len;
-static const char **pathspec;
+static struct pathspec pathspec;
 static int error_unmatch;
 static char *ps_matched;
 static const char *with_tree;
@@ -60,7 +60,7 @@ static void show_dir_entry(const char *tag, struct dir_entry *ent)
 	if (len >= ent->len)
 		die("git ls-files: internal error - directory entry not superset of prefix");
 
-	if (!match_pathspec(pathspec, ent->name, ent->len, len, ps_matched))
+	if (!match_pathspec_depth(&pathspec, ent->name, ent->len, len, ps_matched))
 		return;
 
 	fputs(tag, stdout);
@@ -135,7 +135,7 @@ static void show_ce_entry(const char *tag, struct cache_entry *ce)
 	if (len >= ce_namelen(ce))
 		die("git ls-files: internal error - cache entry not superset of prefix");
 
-	if (!match_pathspec(pathspec, ce->name, ce_namelen(ce), len, ps_matched))
+	if (!match_pathspec_depth(&pathspec, ce->name, ce_namelen(ce), len, ps_matched))
 		return;
 
 	if (tag && *tag && show_valid_bit &&
@@ -189,7 +189,7 @@ static void show_ru_info(void)
 		len = strlen(path);
 		if (len < max_prefix_len)
 			continue; /* outside of the prefix */
-		if (!match_pathspec(pathspec, path, len, max_prefix_len, ps_matched))
+		if (!match_pathspec_depth(&pathspec, path, len, max_prefix_len, ps_matched))
 			continue; /* uninterested */
 		for (i = 0; i < 3; i++) {
 			if (!ui->mode[i])
@@ -218,7 +218,7 @@ static void show_files(struct dir_struct *dir)
 
 	/* For cached/deleted files we don't need to even do the readdir */
 	if (show_others || show_killed) {
-		fill_directory(dir, pathspec);
+		fill_directory(dir, pathspec.raw);
 		if (show_others)
 			show_other_files(dir);
 		if (show_killed)
@@ -289,21 +289,6 @@ static void prune_cache(const char *prefix)
 	active_nr = last;
 }
 
-static void strip_trailing_slash_from_submodules(void)
-{
-	const char **p;
-
-	for (p = pathspec; *p != NULL; p++) {
-		int len = strlen(*p), pos;
-
-		if (len < 1 || (*p)[len - 1] != '/')
-			continue;
-		pos = cache_name_pos(*p, len - 1);
-		if (pos >= 0 && S_ISGITLINK(active_cache[pos]->ce_mode))
-			*p = xstrndup(*p, len - 1);
-	}
-}
-
 /*
  * Read the tree specified with --with-tree option
  * (typically, HEAD) into stage #1 and then
@@ -557,23 +542,18 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 	if (require_work_tree && !is_inside_work_tree())
 		setup_work_tree();
 
-	pathspec = get_pathspec(prefix, argv);
-
-	/* be nice with submodule paths ending in a slash */
-	if (pathspec)
-		strip_trailing_slash_from_submodules();
+	parse_pathspec(&pathspec, 0,
+		       PATHSPEC_PREFER_CWD |
+		       PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP,
+		       prefix, argv);
 
 	/* Find common prefix for all pathspec's */
-	max_prefix = common_prefix(pathspec);
+	max_prefix = common_prefix(pathspec.raw);
 	max_prefix_len = max_prefix ? strlen(max_prefix) : 0;
 
 	/* Treat unmatching pathspec elements as errors */
-	if (pathspec && error_unmatch) {
-		int num;
-		for (num = 0; pathspec[num]; num++)
-			;
-		ps_matched = xcalloc(1, num);
-	}
+	if (pathspec.nr && error_unmatch)
+		ps_matched = xcalloc(1, pathspec.nr);
 
 	if ((dir.flags & DIR_SHOW_IGNORED) && !exc_given)
 		die("ls-files --ignored needs some exclude pattern");
@@ -600,7 +580,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 
 	if (ps_matched) {
 		int bad;
-		bad = report_path_error(ps_matched, pathspec, prefix);
+		bad = report_path_error(ps_matched, pathspec.raw, prefix);
 		if (bad)
 			fprintf(stderr, "Did you forget to 'git add'?\n");
 
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 22/45] archive: convert to use parse_pathspec

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 archive.c | 19 +++++++++++--------
 archive.h |  4 +++-
 2 files changed, 14 insertions(+), 9 deletions(-)

diff --git a/archive.c b/archive.c
index 72c6b0f..95cd174 100644
--- a/archive.c
+++ b/archive.c
@@ -5,7 +5,6 @@
 #include "archive.h"
 #include "parse-options.h"
 #include "unpack-trees.h"
-#include "pathspec.h"
 
 static char const * const archive_usage[] = {
 	N_("git archive [options] <tree-ish> [<path>...]"),
@@ -152,7 +151,6 @@ int write_archive_entries(struct archiver_args *args,
 	struct archiver_context context;
 	struct unpack_trees_options opts;
 	struct tree_desc t;
-	struct pathspec pathspec;
 	int err;
 
 	if (args->baselen > 0 && args->base[args->baselen - 1] == '/') {
@@ -187,10 +185,8 @@ int write_archive_entries(struct archiver_args *args,
 		git_attr_set_direction(GIT_ATTR_INDEX, &the_index);
 	}
 
-	init_pathspec(&pathspec, args->pathspec);
-	err = read_tree_recursive(args->tree, "", 0, 0, &pathspec,
+	err = read_tree_recursive(args->tree, "", 0, 0, &args->pathspec,
 				  write_archive_entry, &context);
-	free_pathspec(&pathspec);
 	if (err == READ_TREE_RECURSIVE)
 		err = 0;
 	return err;
@@ -223,7 +219,7 @@ static int path_exists(struct tree *tree, const char *path)
 	struct pathspec pathspec;
 	int ret;
 
-	init_pathspec(&pathspec, paths);
+	parse_pathspec(&pathspec, 0, 0, "", paths);
 	ret = read_tree_recursive(tree, "", 0, 0, &pathspec, reject_entry, NULL);
 	free_pathspec(&pathspec);
 	return ret != 0;
@@ -232,11 +228,18 @@ static int path_exists(struct tree *tree, const char *path)
 static void parse_pathspec_arg(const char **pathspec,
 		struct archiver_args *ar_args)
 {
-	ar_args->pathspec = pathspec = get_pathspec("", pathspec);
+	/*
+	 * must be consistent with parse_pathspec in path_exists()
+	 * Also if pathspec patterns are dependent, we're in big
+	 * trouble as we test each one separately
+	 */
+	parse_pathspec(&ar_args->pathspec, 0,
+		       PATHSPEC_PREFER_FULL,
+		       "", pathspec);
 	if (pathspec) {
 		while (*pathspec) {
 			if (!path_exists(ar_args->tree, *pathspec))
-				die("path not found: %s", *pathspec);
+				die(_("pathspec '%s' did not match any files"), *pathspec);
 			pathspec++;
 		}
 	}
diff --git a/archive.h b/archive.h
index 895afcd..4a791e1 100644
--- a/archive.h
+++ b/archive.h
@@ -1,6 +1,8 @@
 #ifndef ARCHIVE_H
 #define ARCHIVE_H
 
+#include "pathspec.h"
+
 struct archiver_args {
 	const char *base;
 	size_t baselen;
@@ -8,7 +10,7 @@ struct archiver_args {
 	const unsigned char *commit_sha1;
 	const struct commit *commit;
 	time_t time;
-	const char **pathspec;
+	struct pathspec pathspec;
 	unsigned int verbose : 1;
 	unsigned int worktree_attributes : 1;
 	unsigned int convert : 1;
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 23/45] check-ignore: convert to use parse_pathspec

[Nguyễn Thái Ngọc Duy]

check-ignore (at least the test suite) seems to rely on the pattern
order. PATHSPEC_KEEP_ORDER is introduced to explictly express this.
The lack of PATHSPEC_MAXDEPTH_VALID is sufficient because it's the
only flag that reorders pathspecs, but it's less obvious that way.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/check-ignore.c | 34 +++++++++++++++++++++-------------
 pathspec.c             |  6 +++++-
 pathspec.h             |  1 +
 t/t0008-ignores.sh     |  8 ++++----
 4 files changed, 31 insertions(+), 18 deletions(-)

diff --git a/builtin/check-ignore.c b/builtin/check-ignore.c
index 0240f99..6e55f06 100644
--- a/builtin/check-ignore.c
+++ b/builtin/check-ignore.c
@@ -53,14 +53,14 @@ static void output_exclude(const char *path, struct exclude *exclude)
 	}
 }
 
-static int check_ignore(const char *prefix, const char **pathspec)
+static int check_ignore(int argc, const char **argv, const char *prefix)
 {
 	struct dir_struct dir;
-	const char *path, *full_path;
 	char *seen;
 	int num_ignored = 0, dtype = DT_UNKNOWN, i;
 	struct path_exclude_check check;
 	struct exclude *exclude;
+	struct pathspec pathspec;
 
 	/* read_cache() is only necessary so we can watch out for submodules. */
 	if (read_cache() < 0)
@@ -70,31 +70,39 @@ static int check_ignore(const char *prefix, const char **pathspec)
 	dir.flags |= DIR_COLLECT_IGNORED;
 	setup_standard_excludes(&dir);
 
-	if (!pathspec || !*pathspec) {
+	if (!argc) {
 		if (!quiet)
 			fprintf(stderr, "no pathspec given.\n");
 		return 0;
 	}
 
+	/*
+	 * check-ignore just needs paths. Magic beyond :/ is really
+	 * irrelevant.
+	 */
+	parse_pathspec(&pathspec,
+		       PATHSPEC_ALL_MAGIC & ~PATHSPEC_FROMTOP,
+		       PATHSPEC_SYMLINK_LEADING_PATH |
+		       PATHSPEC_STRIP_SUBMODULE_SLASH_EXPENSIVE |
+		       PATHSPEC_KEEP_ORDER,
+		       prefix, argv);
+
 	path_exclude_check_init(&check, &dir);
 	/*
 	 * look for pathspecs matching entries in the index, since these
 	 * should not be ignored, in order to be consistent with
 	 * 'git status', 'git add' etc.
 	 */
-	seen = find_pathspecs_matching_against_index(pathspec);
-	for (i = 0; pathspec[i]; i++) {
-		path = pathspec[i];
-		full_path = prefix_path(prefix, prefix
-					? strlen(prefix) : 0, path);
-		full_path = check_path_for_gitlink(full_path);
-		die_if_path_beyond_symlink(full_path, prefix);
+	seen = find_pathspecs_matching_against_index(pathspec.raw);
+	for (i = 0; i < pathspec.nr; i++) {
+		const char *full_path = pathspec.raw[i];
 		if (!seen[i]) {
 			exclude = last_exclude_matching_path(&check, full_path,
 							     -1, &dtype);
 			if (exclude) {
 				if (!quiet)
-					output_exclude(path, exclude);
+					output_exclude(pathspec.items[i].original,
+						       exclude);
 				num_ignored++;
 			}
 		}
@@ -129,7 +137,7 @@ static int check_ignore_stdin_paths(const char *prefix)
 	}
 	ALLOC_GROW(pathspec, nr + 1, alloc);
 	pathspec[nr] = NULL;
-	num_ignored = check_ignore(prefix, (const char **)pathspec);
+	num_ignored = check_ignore(nr, (const char **)pathspec, prefix);
 	maybe_flush_or_die(stdout, "attribute to stdout");
 	strbuf_release(&buf);
 	strbuf_release(&nbuf);
@@ -165,7 +173,7 @@ int cmd_check_ignore(int argc, const char **argv, const char *prefix)
 	if (stdin_paths) {
 		num_ignored = check_ignore_stdin_paths(prefix);
 	} else {
-		num_ignored = check_ignore(prefix, argv);
+		num_ignored = check_ignore(argc, argv, prefix);
 		maybe_flush_or_die(stdout, "ignore to stdout");
 	}
 
diff --git a/pathspec.c b/pathspec.c
index 9a57c0c..f531038 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -368,9 +368,13 @@ void parse_pathspec(struct pathspec *pathspec,
 		pathspec->magic |= item[i].magic;
 	}
 
-	if (pathspec->magic & PATHSPEC_MAXDEPTH)
+
+	if (pathspec->magic & PATHSPEC_MAXDEPTH) {
+		if (flags & PATHSPEC_KEEP_ORDER)
+			die("BUG: PATHSPEC_MAXDEPTH_VALID and PATHSPEC_KEEP_ORDER are incompatible");
 		qsort(pathspec->items, pathspec->nr,
 		      sizeof(struct pathspec_item), pathspec_item_cmp);
+	}
 }
 
 /*
diff --git a/pathspec.h b/pathspec.h
index ed5d3a6..44253c8 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -42,6 +42,7 @@ struct pathspec {
 #define PATHSPEC_SYMLINK_LEADING_PATH (1<<4)
 #define PATHSPEC_STRIP_SUBMODULE_SLASH_EXPENSIVE (1<<5)
 #define PATHSPEC_PREFIX_ORIGIN (1<<6)
+#define PATHSPEC_KEEP_ORDER (1<<7)
 
 extern int init_pathspec(struct pathspec *, const char **);
 extern void parse_pathspec(struct pathspec *pathspec,
diff --git a/t/t0008-ignores.sh b/t/t0008-ignores.sh
index 9c1bde1..c41f60b 100755
--- a/t/t0008-ignores.sh
+++ b/t/t0008-ignores.sh
@@ -397,7 +397,7 @@ test_expect_success_multi SYMLINKS 'symlink' '' '
 
 test_expect_success_multi SYMLINKS 'beyond a symlink' '' '
 	test_check_ignore "a/symlink/foo" 128 &&
-	test_stderr "fatal: '\''a/symlink/foo'\'' is beyond a symbolic link"
+	test_stderr "fatal: pathspec '\''a/symlink/foo'\'' is beyond a symbolic link"
 '
 
 test_expect_success_multi SYMLINKS 'beyond a symlink from subdirectory' '' '
@@ -405,7 +405,7 @@ test_expect_success_multi SYMLINKS 'beyond a symlink from subdirectory' '' '
 		cd a &&
 		test_check_ignore "symlink/foo" 128
 	) &&
-	test_stderr "fatal: '\''symlink/foo'\'' is beyond a symbolic link"
+	test_stderr "fatal: pathspec '\''symlink/foo'\'' is beyond a symbolic link"
 '
 
 ############################################################################
@@ -414,7 +414,7 @@ test_expect_success_multi SYMLINKS 'beyond a symlink from subdirectory' '' '
 
 test_expect_success_multi 'submodule' '' '
 	test_check_ignore "a/submodule/one" 128 &&
-	test_stderr "fatal: Path '\''a/submodule/one'\'' is in submodule '\''a/submodule'\''"
+	test_stderr "fatal: Pathspec '\''a/submodule/one'\'' is in submodule '\''a/submodule'\''"
 '
 
 test_expect_success_multi 'submodule from subdirectory' '' '
@@ -422,7 +422,7 @@ test_expect_success_multi 'submodule from subdirectory' '' '
 		cd a &&
 		test_check_ignore "submodule/one" 128
 	) &&
-	test_stderr "fatal: Path '\''a/submodule/one'\'' is in submodule '\''a/submodule'\''"
+	test_stderr "fatal: Pathspec '\''submodule/one'\'' is in submodule '\''a/submodule'\''"
 '
 
 ############################################################################
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 24/45] add: convert to use parse_pathspec

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/add.c | 100 +++++++++++++++++++++++++---------------------------------
 pathspec.c    |  43 -------------------------
 2 files changed, 43 insertions(+), 100 deletions(-)

diff --git a/builtin/add.c b/builtin/add.c
index ab1c9e8..ec6fbe3 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -121,21 +121,6 @@ static char *prune_directory(struct dir_struct *dir, const char **pathspec, int
 	return seen;
 }
 
-/*
- * Checks the index to see whether any path in pathspec refers to
- * something inside a submodule.  If so, dies with an error message.
- */
-static void treat_gitlinks(const char **pathspec)
-{
-	int i;
-
-	if (!pathspec || !*pathspec)
-		return;
-
-	for (i = 0; pathspec[i]; i++)
-		pathspec[i] = check_path_for_gitlink(pathspec[i]);
-}
-
 static void refresh(int verbose, const char **pathspec)
 {
 	char *seen;
@@ -153,25 +138,6 @@ static void refresh(int verbose, const char **pathspec)
         free(seen);
 }
 
-/*
- * Normalizes argv relative to prefix, via get_pathspec(), and then
- * runs die_if_path_beyond_symlink() on each path in the normalized
- * list.
- */
-static const char **validate_pathspec(const char **argv, const char *prefix)
-{
-	const char **pathspec = get_pathspec(prefix, argv);
-
-	if (pathspec) {
-		const char **p;
-		for (p = pathspec; *p; p++) {
-			die_if_path_beyond_symlink(*p, prefix);
-		}
-	}
-
-	return pathspec;
-}
-
 int run_add_interactive(const char *revision, const char *patch_mode,
 			const char **pathspec)
 {
@@ -203,17 +169,23 @@ int run_add_interactive(const char *revision, const char *patch_mode,
 
 int interactive_add(int argc, const char **argv, const char *prefix, int patch)
 {
-	const char **pathspec = NULL;
+	struct pathspec pathspec;
 
-	if (argc) {
-		pathspec = validate_pathspec(argv, prefix);
-		if (!pathspec)
-			return -1;
-	}
+	/*
+	 * git-add--interactive itself does not parse pathspec. It
+	 * simply passes the pathspec to other builtin commands. Let's
+	 * hope all of them support all magic, or we'll need to limit
+	 * the magic here. There is still a problem with prefix. But
+	 * that'll be worked on later on.
+	 */
+	parse_pathspec(&pathspec, PATHSPEC_ALL_MAGIC & ~PATHSPEC_FROMTOP,
+		       PATHSPEC_PREFER_FULL |
+		       PATHSPEC_SYMLINK_LEADING_PATH,
+		       prefix, argv);
 
 	return run_add_interactive(NULL,
 				   patch ? "--patch" : NULL,
-				   pathspec);
+				   pathspec.raw);
 }
 
 static int edit_patch(int argc, const char **argv, const char *prefix)
@@ -354,7 +326,7 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 {
 	int exit_status = 0;
 	int newfd;
-	const char **pathspec;
+	struct pathspec pathspec;
 	struct dir_struct dir;
 	int flags;
 	int add_new_files;
@@ -415,11 +387,19 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 		fprintf(stderr, _("Maybe you wanted to say 'git add .'?\n"));
 		return 0;
 	}
-	pathspec = validate_pathspec(argv, prefix);
 
 	if (read_cache() < 0)
 		die(_("index file corrupt"));
-	treat_gitlinks(pathspec);
+
+	/*
+	 * Check the "pathspec '%s' did not match any files" block
+	 * below before enabling new magic.
+	 */
+	parse_pathspec(&pathspec, 0,
+		       PATHSPEC_PREFER_FULL |
+		       PATHSPEC_SYMLINK_LEADING_PATH |
+		       PATHSPEC_STRIP_SUBMODULE_SLASH_EXPENSIVE,
+		       prefix, argv);
 
 	if (add_new_files) {
 		int baselen;
@@ -432,33 +412,39 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 		}
 
 		/* This picks up the paths that are not tracked */
-		baselen = fill_directory(&dir, pathspec);
-		if (pathspec)
-			seen = prune_directory(&dir, pathspec, baselen);
+		baselen = fill_directory(&dir, pathspec.raw);
+		if (pathspec.nr)
+			seen = prune_directory(&dir, pathspec.raw, baselen);
 	}
 
 	if (refresh_only) {
-		refresh(verbose, pathspec);
+		refresh(verbose, pathspec.raw);
 		goto finish;
 	}
 
-	if (pathspec) {
+	if (pathspec.nr) {
 		int i;
 		struct path_exclude_check check;
 
 		path_exclude_check_init(&check, &dir);
 		if (!seen)
-			seen = find_pathspecs_matching_against_index(pathspec);
-		for (i = 0; pathspec[i]; i++) {
-			if (!seen[i] && pathspec[i][0]
-			    && !file_exists(pathspec[i])) {
+			seen = find_pathspecs_matching_against_index(pathspec.raw);
+
+		/*
+		 * file_exists() assumes exact match
+		 */
+		GUARD_PATHSPEC(&pathspec, PATHSPEC_FROMTOP);
+
+		for (i = 0; pathspec.raw[i]; i++) {
+			if (!seen[i] && pathspec.raw[i][0]
+			    && !file_exists(pathspec.raw[i])) {
 				if (ignore_missing) {
 					int dtype = DT_UNKNOWN;
-					if (is_path_excluded(&check, pathspec[i], -1, &dtype))
-						dir_add_ignored(&dir, pathspec[i], strlen(pathspec[i]));
+					if (is_path_excluded(&check, pathspec.raw[i], -1, &dtype))
+						dir_add_ignored(&dir, pathspec.raw[i], strlen(pathspec.raw[i]));
 				} else
 					die(_("pathspec '%s' did not match any files"),
-					    pathspec[i]);
+					    pathspec.raw[i]);
 			}
 		}
 		free(seen);
@@ -467,7 +453,7 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 
 	plug_bulk_checkin();
 
-	exit_status |= add_files_to_cache(prefix, pathspec, flags);
+	exit_status |= add_files_to_cache(prefix, pathspec.raw, flags);
 
 	if (add_new_files)
 		exit_status |= add_files(&dir, flags);
diff --git a/pathspec.c b/pathspec.c
index f531038..0b658cb 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -58,49 +58,6 @@ char *find_pathspecs_matching_against_index(const char **pathspec)
 }
 
 /*
- * Check the index to see whether path refers to a submodule, or
- * something inside a submodule.  If the former, returns the path with
- * any trailing slash stripped.  If the latter, dies with an error
- * message.
- */
-const char *check_path_for_gitlink(const char *path)
-{
-	int i, path_len = strlen(path);
-	for (i = 0; i < active_nr; i++) {
-		struct cache_entry *ce = active_cache[i];
-		if (S_ISGITLINK(ce->ce_mode)) {
-			int ce_len = ce_namelen(ce);
-			if (path_len <= ce_len || path[ce_len] != '/' ||
-			    memcmp(ce->name, path, ce_len))
-				/* path does not refer to this
-				 * submodule or anything inside it */
-				continue;
-			if (path_len == ce_len + 1) {
-				/* path refers to submodule;
-				 * strip trailing slash */
-				return xstrndup(ce->name, ce_len);
-			} else {
-				die (_("Path '%s' is in submodule '%.*s'"),
-				     path, ce_len, ce->name);
-			}
-		}
-	}
-	return path;
-}
-
-/*
- * Dies if the given path refers to a file inside a symlinked
- * directory in the index.
- */
-void die_if_path_beyond_symlink(const char *path, const char *prefix)
-{
-	if (has_symlink_leading_path(path, strlen(path))) {
-		int len = prefix ? strlen(prefix) : 0;
-		die(_("'%s' is beyond a symbolic link"), path + len);
-	}
-}
-
-/*
  * Magic pathspec
  *
  * Possible future magic semantics include stuff like:
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 25/45] reset: convert to use parse_pathspec

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/reset.c | 26 ++++++++++++++++----------
 1 file changed, 16 insertions(+), 10 deletions(-)

diff --git a/builtin/reset.c b/builtin/reset.c
index 6032131..da7282e 100644
--- a/builtin/reset.c
+++ b/builtin/reset.c
@@ -171,7 +171,10 @@ static void die_if_unmerged_cache(int reset_type)
 
 }
 
-static const char **parse_args(const char **argv, const char *prefix, const char **rev_ret)
+static void parse_args(struct pathspec *pathspec,
+		       const char **argv, const char *prefix,
+		       int patch_mode,
+		       const char **rev_ret)
 {
 	const char *rev = "HEAD";
 	unsigned char unused[20];
@@ -213,7 +216,10 @@ static const char **parse_args(const char **argv, const char *prefix, const char
 		}
 	}
 	*rev_ret = rev;
-	return argv[0] ? get_pathspec(prefix, argv) : NULL;
+	parse_pathspec(pathspec,
+		       patch_mode ? PATHSPEC_ALL_MAGIC & ~PATHSPEC_FROMTOP : 0,
+		       PATHSPEC_PREFER_FULL,
+		       prefix, argv);
 }
 
 static int update_refs(const char *rev, const unsigned char *sha1)
@@ -243,7 +249,7 @@ int cmd_reset(int argc, const char **argv, const char *prefix)
 	int patch_mode = 0, unborn;
 	const char *rev;
 	unsigned char sha1[20];
-	const char **pathspec = NULL;
+	struct pathspec pathspec;
 	const struct option options[] = {
 		OPT__QUIET(&quiet, N_("be quiet, only report errors")),
 		OPT_SET_INT(0, "mixed", &reset_type,
@@ -263,13 +269,13 @@ int cmd_reset(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, git_reset_usage,
 						PARSE_OPT_KEEP_DASHDASH);
-	pathspec = parse_args(argv, prefix, &rev);
+	parse_args(&pathspec, argv, prefix, patch_mode, &rev);
 
 	unborn = !strcmp(rev, "HEAD") && get_sha1("HEAD", sha1);
 	if (unborn) {
 		/* reset on unborn branch: treat as reset to empty tree */
 		hashcpy(sha1, EMPTY_TREE_SHA1_BIN);
-	} else if (!pathspec) {
+	} else if (!pathspec.nr) {
 		struct commit *commit;
 		if (get_sha1_committish(rev, sha1))
 			die(_("Failed to resolve '%s' as a valid revision."), rev);
@@ -290,13 +296,13 @@ int cmd_reset(int argc, const char **argv, const char *prefix)
 	if (patch_mode) {
 		if (reset_type != NONE)
 			die(_("--patch is incompatible with --{hard,mixed,soft}"));
-		return run_add_interactive(sha1_to_hex(sha1), "--patch=reset", pathspec);
+		return run_add_interactive(sha1_to_hex(sha1), "--patch=reset", pathspec.raw);
 	}
 
 	/* git reset tree [--] paths... can be used to
 	 * load chosen paths from the tree into the index without
 	 * affecting the working tree nor HEAD. */
-	if (pathspec) {
+	if (pathspec.nr) {
 		if (reset_type == MIXED)
 			warning(_("--mixed with paths is deprecated; use 'git reset -- <paths>' instead."));
 		else if (reset_type != NONE)
@@ -323,7 +329,7 @@ int cmd_reset(int argc, const char **argv, const char *prefix)
 		struct lock_file *lock = xcalloc(1, sizeof(struct lock_file));
 		int newfd = hold_locked_index(lock, 1);
 		if (reset_type == MIXED) {
-			if (read_from_tree(pathspec, sha1))
+			if (read_from_tree(pathspec.raw, sha1))
 				return 1;
 		} else {
 			int err = reset_index(sha1, reset_type, quiet);
@@ -344,7 +350,7 @@ int cmd_reset(int argc, const char **argv, const char *prefix)
 			die(_("Could not write new index file."));
 	}
 
-	if (!pathspec && !unborn) {
+	if (!pathspec.nr && !unborn) {
 		/* Any resets without paths update HEAD to the head being
 		 * switched to, saving the previous head in ORIG_HEAD before. */
 		update_ref_status = update_refs(rev, sha1);
@@ -352,7 +358,7 @@ int cmd_reset(int argc, const char **argv, const char *prefix)
 		if (reset_type == HARD && !update_ref_status && !quiet)
 			print_new_head_line(lookup_commit_reference(sha1));
 	}
-	if (!pathspec)
+	if (!pathspec.nr)
 		remove_branch_state();
 
 	return update_ref_status;
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 26/45] Convert read_cache_preload() to take struct pathspec

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/checkout.c   |  2 +-
 builtin/commit.c     |  4 ++--
 builtin/diff-files.c |  2 +-
 builtin/diff-index.c |  2 +-
 builtin/diff.c       |  4 ++--
 cache.h              |  2 +-
 preload-index.c      | 20 +++++++++++---------
 7 files changed, 19 insertions(+), 17 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 8dc5f51..3c19cb4 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -261,7 +261,7 @@ static int checkout_paths(const struct checkout_opts *opts,
 	lock_file = xcalloc(1, sizeof(struct lock_file));
 
 	newfd = hold_locked_index(lock_file, 1);
-	if (read_cache_preload(opts->pathspec.raw) < 0)
+	if (read_cache_preload(&opts->pathspec) < 0)
 		return error(_("corrupt index file"));
 
 	if (opts->source_tree)
diff --git a/builtin/commit.c b/builtin/commit.c
index d8e6b28..564f359 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -287,7 +287,7 @@ static char *prepare_index(int argc, const char **argv, const char *prefix,
 		       PATHSPEC_PREFER_FULL,
 		       prefix, argv);
 
-	if (read_cache_preload(pathspec.raw) < 0)
+	if (read_cache_preload(&pathspec) < 0)
 		die(_("index file corrupt"));
 
 	if (interactive) {
@@ -1199,7 +1199,7 @@ int cmd_status(int argc, const char **argv, const char *prefix)
 		       PATHSPEC_PREFER_FULL,
 		       prefix, argv);
 
-	read_cache_preload(s.pathspec.raw);
+	read_cache_preload(&s.pathspec);
 	refresh_index(&the_index, REFRESH_QUIET|REFRESH_UNMERGED, s.pathspec.raw, NULL, NULL);
 
 	fd = hold_locked_index(&index_lock, 0);
diff --git a/builtin/diff-files.c b/builtin/diff-files.c
index 46085f8..9200069 100644
--- a/builtin/diff-files.c
+++ b/builtin/diff-files.c
@@ -61,7 +61,7 @@ int cmd_diff_files(int argc, const char **argv, const char *prefix)
 	    (rev.diffopt.output_format & DIFF_FORMAT_PATCH))
 		rev.combine_merges = rev.dense_combined_merges = 1;
 
-	if (read_cache_preload(rev.diffopt.pathspec.raw) < 0) {
+	if (read_cache_preload(&rev.diffopt.pathspec) < 0) {
 		perror("read_cache_preload");
 		return -1;
 	}
diff --git a/builtin/diff-index.c b/builtin/diff-index.c
index 1c737f7..ce15b23 100644
--- a/builtin/diff-index.c
+++ b/builtin/diff-index.c
@@ -43,7 +43,7 @@ int cmd_diff_index(int argc, const char **argv, const char *prefix)
 		usage(diff_cache_usage);
 	if (!cached) {
 		setup_work_tree();
-		if (read_cache_preload(rev.diffopt.pathspec.raw) < 0) {
+		if (read_cache_preload(&rev.diffopt.pathspec) < 0) {
 			perror("read_cache_preload");
 			return -1;
 		}
diff --git a/builtin/diff.c b/builtin/diff.c
index d237e0a..6b4e3f9 100644
--- a/builtin/diff.c
+++ b/builtin/diff.c
@@ -140,7 +140,7 @@ static int builtin_diff_index(struct rev_info *revs,
 		usage(builtin_diff_usage);
 	if (!cached) {
 		setup_work_tree();
-		if (read_cache_preload(revs->diffopt.pathspec.raw) < 0) {
+		if (read_cache_preload(&revs->diffopt.pathspec) < 0) {
 			perror("read_cache_preload");
 			return -1;
 		}
@@ -240,7 +240,7 @@ static int builtin_diff_files(struct rev_info *revs, int argc, const char **argv
 		revs->combine_merges = revs->dense_combined_merges = 1;
 
 	setup_work_tree();
-	if (read_cache_preload(revs->diffopt.pathspec.raw) < 0) {
+	if (read_cache_preload(&revs->diffopt.pathspec) < 0) {
 		perror("read_cache_preload");
 		return -1;
 	}
diff --git a/cache.h b/cache.h
index 5115c1b..cc90d7b 100644
--- a/cache.h
+++ b/cache.h
@@ -436,7 +436,7 @@ extern int init_db(const char *template_dir, unsigned int flags);
 
 /* Initialize and use the cache information */
 extern int read_index(struct index_state *);
-extern int read_index_preload(struct index_state *, const char **pathspec);
+extern int read_index_preload(struct index_state *, const struct pathspec *pathspec);
 extern int read_index_from(struct index_state *, const char *path);
 extern int is_index_unborn(struct index_state *);
 extern int read_index_unmerged(struct index_state *);
diff --git a/preload-index.c b/preload-index.c
index cddfffa..8c44ceb 100644
--- a/preload-index.c
+++ b/preload-index.c
@@ -5,7 +5,8 @@
 #include "pathspec.h"
 
 #ifdef NO_PTHREADS
-static void preload_index(struct index_state *index, const char **pathspec)
+static void preload_index(struct index_state *index,
+			  const struct pathspec *pathspec)
 {
 	; /* nothing */
 }
@@ -25,7 +26,7 @@ static void preload_index(struct index_state *index, const char **pathspec)
 struct thread_data {
 	pthread_t pthread;
 	struct index_state *index;
-	const char **pathspec;
+	struct pathspec pathspec;
 	int offset, nr;
 };
 
@@ -36,9 +37,7 @@ static void *preload_thread(void *_data)
 	struct index_state *index = p->index;
 	struct cache_entry **cep = index->cache + p->offset;
 	struct cache_def cache;
-	struct pathspec pathspec;
 
-	init_pathspec(&pathspec, p->pathspec);
 	memset(&cache, 0, sizeof(cache));
 	nr = p->nr;
 	if (nr + p->offset > index->cache_nr)
@@ -54,7 +53,7 @@ static void *preload_thread(void *_data)
 			continue;
 		if (ce_uptodate(ce))
 			continue;
-		if (!ce_path_match(ce, &pathspec))
+		if (!ce_path_match(ce, &p->pathspec))
 			continue;
 		if (threaded_has_symlink_leading_path(&cache, ce->name, ce_namelen(ce)))
 			continue;
@@ -64,11 +63,11 @@ static void *preload_thread(void *_data)
 			continue;
 		ce_mark_uptodate(ce);
 	} while (--nr > 0);
-	free_pathspec(&pathspec);
 	return NULL;
 }
 
-static void preload_index(struct index_state *index, const char **pathspec)
+static void preload_index(struct index_state *index,
+			  const struct pathspec *pathspec)
 {
 	int threads, i, work, offset;
 	struct thread_data data[MAX_PARALLEL];
@@ -83,10 +82,12 @@ static void preload_index(struct index_state *index, const char **pathspec)
 		threads = MAX_PARALLEL;
 	offset = 0;
 	work = DIV_ROUND_UP(index->cache_nr, threads);
+	memset(&data, 0, sizeof(data));
 	for (i = 0; i < threads; i++) {
 		struct thread_data *p = data+i;
 		p->index = index;
-		p->pathspec = pathspec;
+		if (pathspec)
+			copy_pathspec(&p->pathspec, pathspec);
 		p->offset = offset;
 		p->nr = work;
 		offset += work;
@@ -101,7 +102,8 @@ static void preload_index(struct index_state *index, const char **pathspec)
 }
 #endif
 
-int read_index_preload(struct index_state *index, const char **pathspec)
+int read_index_preload(struct index_state *index,
+		       const struct pathspec *pathspec)
 {
 	int retval = read_index(index);
 
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 27/45] Convert run_add_interactive to use struct pathspec

[Nguyễn Thái Ngọc Duy]

This passes the pathspec, more or less unmodified, to
git-add--interactive. The command itself does not process pathspec. It
simply passes the pathspec to other builtin commands. So if all those
commands support pathspec, we're good.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/add.c      | 26 ++++++++++----------------
 builtin/checkout.c |  9 ++++-----
 builtin/reset.c    |  8 ++++----
 commit.h           |  2 +-
 4 files changed, 19 insertions(+), 26 deletions(-)

diff --git a/builtin/add.c b/builtin/add.c
index ec6fbe3..2b20d7d 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -139,16 +139,12 @@ static void refresh(int verbose, const char **pathspec)
 }
 
 int run_add_interactive(const char *revision, const char *patch_mode,
-			const char **pathspec)
+			const struct pathspec *pathspec)
 {
-	int status, ac, pc = 0;
+	int status, ac, i;
 	const char **args;
 
-	if (pathspec)
-		while (pathspec[pc])
-			pc++;
-
-	args = xcalloc(sizeof(const char *), (pc + 5));
+	args = xcalloc(sizeof(const char *), (pathspec->nr + 6));
 	ac = 0;
 	args[ac++] = "add--interactive";
 	if (patch_mode)
@@ -156,11 +152,9 @@ int run_add_interactive(const char *revision, const char *patch_mode,
 	if (revision)
 		args[ac++] = revision;
 	args[ac++] = "--";
-	if (pc) {
-		memcpy(&(args[ac]), pathspec, sizeof(const char *) * pc);
-		ac += pc;
-	}
-	args[ac] = NULL;
+	for (i = 0; i < pathspec->nr; i++)
+		/* pass original pathspec, to be re-parsed */
+		args[ac++] = pathspec->items[i].original;
 
 	status = run_command_v_opt(args, RUN_GIT_CMD);
 	free(args);
@@ -175,17 +169,17 @@ int interactive_add(int argc, const char **argv, const char *prefix, int patch)
 	 * git-add--interactive itself does not parse pathspec. It
 	 * simply passes the pathspec to other builtin commands. Let's
 	 * hope all of them support all magic, or we'll need to limit
-	 * the magic here. There is still a problem with prefix. But
-	 * that'll be worked on later on.
+	 * the magic here.
 	 */
 	parse_pathspec(&pathspec, PATHSPEC_ALL_MAGIC & ~PATHSPEC_FROMTOP,
 		       PATHSPEC_PREFER_FULL |
-		       PATHSPEC_SYMLINK_LEADING_PATH,
+		       PATHSPEC_SYMLINK_LEADING_PATH |
+		       PATHSPEC_PREFIX_ORIGIN,
 		       prefix, argv);
 
 	return run_add_interactive(NULL,
 				   patch ? "--patch" : NULL,
-				   pathspec.raw);
+				   &pathspec);
 }
 
 static int edit_patch(int argc, const char **argv, const char *prefix)
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 3c19cb4..2ddff95 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -256,7 +256,7 @@ static int checkout_paths(const struct checkout_opts *opts,
 
 	if (opts->patch_mode)
 		return run_add_interactive(revision, "--patch=checkout",
-					   opts->pathspec.raw);
+					   &opts->pathspec);
 
 	lock_file = xcalloc(1, sizeof(struct lock_file));
 
@@ -1115,10 +1115,9 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 		 * cannot handle. Magic mask is pretty safe to be
 		 * lifted for new magic when opts.patch_mode == 0.
 		 */
-		parse_pathspec(&opts.pathspec,
-			       opts.patch_mode == 0 ? 0 :
-			       (PATHSPEC_ALL_MAGIC & ~PATHSPEC_FROMTOP),
-			       0, prefix, argv);
+		parse_pathspec(&opts.pathspec, 0,
+			       opts.patch_mode ? PATHSPEC_PREFIX_ORIGIN : 0,
+			       prefix, argv);
 
 		if (!opts.pathspec.nr)
 			die(_("invalid path specification"));
diff --git a/builtin/reset.c b/builtin/reset.c
index da7282e..7c6e8b6 100644
--- a/builtin/reset.c
+++ b/builtin/reset.c
@@ -216,9 +216,9 @@ static void parse_args(struct pathspec *pathspec,
 		}
 	}
 	*rev_ret = rev;
-	parse_pathspec(pathspec,
-		       patch_mode ? PATHSPEC_ALL_MAGIC & ~PATHSPEC_FROMTOP : 0,
-		       PATHSPEC_PREFER_FULL,
+	parse_pathspec(pathspec, 0,
+		       PATHSPEC_PREFER_FULL |
+		       patch_mode ? PATHSPEC_PREFIX_ORIGIN : 0,
 		       prefix, argv);
 }
 
@@ -296,7 +296,7 @@ int cmd_reset(int argc, const char **argv, const char *prefix)
 	if (patch_mode) {
 		if (reset_type != NONE)
 			die(_("--patch is incompatible with --{hard,mixed,soft}"));
-		return run_add_interactive(sha1_to_hex(sha1), "--patch=reset", pathspec.raw);
+		return run_add_interactive(sha1_to_hex(sha1), "--patch=reset", &pathspec);
 	}
 
 	/* git reset tree [--] paths... can be used to
diff --git a/commit.h b/commit.h
index 4138bb4..9448fda 100644
--- a/commit.h
+++ b/commit.h
@@ -179,7 +179,7 @@ int in_merge_bases(struct commit *, struct commit *);
 
 extern int interactive_add(int argc, const char **argv, const char *prefix, int patch);
 extern int run_add_interactive(const char *revision, const char *patch_mode,
-			       const char **pathspec);
+			       const struct pathspec *pathspec);
 
 static inline int single_parent(struct commit *commit)
 {
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 28/45] Convert unmerge_cache to take struct pathspec

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/checkout.c | 2 +-
 rerere.c           | 2 +-
 resolve-undo.c     | 4 ++--
 resolve-undo.h     | 2 +-
 4 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 2ddff95..ba5a5c0 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -281,7 +281,7 @@ static int checkout_paths(const struct checkout_opts *opts,
 
 	/* "checkout -m path" to recreate conflicted state */
 	if (opts->merge)
-		unmerge_cache(opts->pathspec.raw);
+		unmerge_cache(&opts->pathspec);
 
 	/* Any unmerged paths? */
 	for (pos = 0; pos < active_nr; pos++) {
diff --git a/rerere.c b/rerere.c
index c52c1f2..ac02eb8 100644
--- a/rerere.c
+++ b/rerere.c
@@ -667,7 +667,7 @@ int rerere_forget(struct pathspec *pathspec)
 
 	fd = setup_rerere(&merge_rr, RERERE_NOAUTOUPDATE);
 
-	unmerge_cache(pathspec->raw);
+	unmerge_cache(pathspec);
 	find_conflict(&conflict);
 	for (i = 0; i < conflict.nr; i++) {
 		struct string_list_item *it = &conflict.items[i];
diff --git a/resolve-undo.c b/resolve-undo.c
index 72b4612..1bfece2 100644
--- a/resolve-undo.c
+++ b/resolve-undo.c
@@ -156,7 +156,7 @@ int unmerge_index_entry_at(struct index_state *istate, int pos)
 	return unmerge_index_entry_at(istate, pos);
 }
 
-void unmerge_index(struct index_state *istate, const char **pathspec)
+void unmerge_index(struct index_state *istate, const struct pathspec *pathspec)
 {
 	int i;
 
@@ -165,7 +165,7 @@ void unmerge_index(struct index_state *istate, const char **pathspec)
 
 	for (i = 0; i < istate->cache_nr; i++) {
 		struct cache_entry *ce = istate->cache[i];
-		if (!match_pathspec(pathspec, ce->name, ce_namelen(ce), 0, NULL))
+		if (!match_pathspec_depth(pathspec, ce->name, ce_namelen(ce), 0, NULL))
 			continue;
 		i = unmerge_index_entry_at(istate, i);
 	}
diff --git a/resolve-undo.h b/resolve-undo.h
index 8458769..81e8803 100644
--- a/resolve-undo.h
+++ b/resolve-undo.h
@@ -11,6 +11,6 @@ extern void resolve_undo_write(struct strbuf *, struct string_list *);
 extern struct string_list *resolve_undo_read(const char *, unsigned long);
 extern void resolve_undo_clear_index(struct index_state *);
 extern int unmerge_index_entry_at(struct index_state *, int);
-extern void unmerge_index(struct index_state *, const char **);
+extern void unmerge_index(struct index_state *, const struct pathspec *);
 
 #endif
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 29/45] checkout: convert read_tree_some to take struct pathspec

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/checkout.c | 9 +++------
 tree.c             | 4 ++--
 tree.h             | 2 +-
 3 files changed, 6 insertions(+), 9 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index ba5a5c0..132bfe6 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -82,12 +82,9 @@ static int update_some(const unsigned char *sha1, const char *base, int baselen,
 	return 0;
 }
 
-static int read_tree_some(struct tree *tree, const char **pathspec)
+static int read_tree_some(struct tree *tree, const struct pathspec *pathspec)
 {
-	struct pathspec ps;
-	init_pathspec(&ps, pathspec);
-	read_tree_recursive(tree, "", 0, 0, &ps, update_some, NULL);
-	free_pathspec(&ps);
+	read_tree_recursive(tree, "", 0, 0, pathspec, update_some, NULL);
 
 	/* update the index with the given tree's info
 	 * for all args, expanding wildcards, and exit
@@ -265,7 +262,7 @@ static int checkout_paths(const struct checkout_opts *opts,
 		return error(_("corrupt index file"));
 
 	if (opts->source_tree)
-		read_tree_some(opts->source_tree, opts->pathspec.raw);
+		read_tree_some(opts->source_tree, &opts->pathspec);
 
 	ps_matched = xcalloc(1, opts->pathspec.nr);
 
diff --git a/tree.c b/tree.c
index 62fed63..ff72f67 100644
--- a/tree.c
+++ b/tree.c
@@ -47,7 +47,7 @@ static int read_one_entry_quick(const unsigned char *sha1, const char *base, int
 }
 
 static int read_tree_1(struct tree *tree, struct strbuf *base,
-		       int stage, struct pathspec *pathspec,
+		       int stage, const struct pathspec *pathspec,
 		       read_tree_fn_t fn, void *context)
 {
 	struct tree_desc desc;
@@ -116,7 +116,7 @@ static int read_tree_1(struct tree *tree, struct strbuf *base,
 
 int read_tree_recursive(struct tree *tree,
 			const char *base, int baselen,
-			int stage, struct pathspec *pathspec,
+			int stage, const struct pathspec *pathspec,
 			read_tree_fn_t fn, void *context)
 {
 	struct strbuf sb = STRBUF_INIT;
diff --git a/tree.h b/tree.h
index 69bcb5e..9dc90ba 100644
--- a/tree.h
+++ b/tree.h
@@ -25,7 +25,7 @@ typedef int (*read_tree_fn_t)(const unsigned char *, const char *, int, const ch
 
 extern int read_tree_recursive(struct tree *tree,
 			       const char *base, int baselen,
-			       int stage, struct pathspec *pathspec,
+			       int stage, const struct pathspec *pathspec,
 			       read_tree_fn_t fn, void *context);
 
 extern int read_tree(struct tree *tree, int stage, struct pathspec *pathspec);
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 30/45] Convert report_path_error to take struct pathspec

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/checkout.c |  2 +-
 builtin/commit.c   | 14 ++++++--------
 builtin/ls-files.c | 19 +++++++++++--------
 cache.h            |  2 +-
 4 files changed, 19 insertions(+), 18 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 132bfe6..cd459e1 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -273,7 +273,7 @@ static int checkout_paths(const struct checkout_opts *opts,
 		match_pathspec_depth(&opts->pathspec, ce->name, ce_namelen(ce), 0, ps_matched);
 	}
 
-	if (report_path_error(ps_matched, opts->pathspec.raw, opts->prefix))
+	if (report_path_error(ps_matched, &opts->pathspec, opts->prefix))
 		return 1;
 
 	/* "checkout -m path" to recreate conflicted state */
diff --git a/builtin/commit.c b/builtin/commit.c
index 564f359..eba53af 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -181,20 +181,18 @@ static int commit_index_files(void)
  * and return the paths that match the given pattern in list.
  */
 static int list_paths(struct string_list *list, const char *with_tree,
-		      const char *prefix, const char **pattern)
+		      const char *prefix, const struct pathspec *pattern)
 {
 	int i;
 	char *m;
 
-	if (!pattern)
+	if (!pattern->nr)
 		return 0;
 
-	for (i = 0; pattern[i]; i++)
-		;
-	m = xcalloc(1, i);
+	m = xcalloc(1, pattern->nr);
 
 	if (with_tree) {
-		char *max_prefix = common_prefix(pattern);
+		char *max_prefix = common_prefix(pattern->raw);
 		overlay_tree_on_cache(with_tree, max_prefix ? max_prefix : prefix);
 		free(max_prefix);
 	}
@@ -205,7 +203,7 @@ static int list_paths(struct string_list *list, const char *with_tree,
 
 		if (ce->ce_flags & CE_UPDATE)
 			continue;
-		if (!match_pathspec(pattern, ce->name, ce_namelen(ce), 0, m))
+		if (!match_pathspec_depth(pattern, ce->name, ce_namelen(ce), 0, m))
 			continue;
 		item = string_list_insert(list, ce->name);
 		if (ce_skip_worktree(ce))
@@ -395,7 +393,7 @@ static char *prepare_index(int argc, const char **argv, const char *prefix,
 
 	memset(&partial, 0, sizeof(partial));
 	partial.strdup_strings = 1;
-	if (list_paths(&partial, !current_head ? NULL : "HEAD", prefix, pathspec.raw))
+	if (list_paths(&partial, !current_head ? NULL : "HEAD", prefix, &pathspec))
 		exit(1);
 
 	discard_cache();
diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index bd64829..a33ea18 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -351,15 +351,16 @@ void overlay_tree_on_cache(const char *tree_name, const char *prefix)
 	}
 }
 
-int report_path_error(const char *ps_matched, const char **pathspec, const char *prefix)
+int report_path_error(const char *ps_matched,
+		      const struct pathspec *pathspec,
+		      const char *prefix)
 {
 	/*
 	 * Make sure all pathspec matched; otherwise it is an error.
 	 */
 	struct strbuf sb = STRBUF_INIT;
-	const char *name;
 	int num, errors = 0;
-	for (num = 0; pathspec[num]; num++) {
+	for (num = 0; num < pathspec->nr; num++) {
 		int other, found_dup;
 
 		if (ps_matched[num])
@@ -367,13 +368,16 @@ int report_path_error(const char *ps_matched, const char **pathspec, const char
 		/*
 		 * The caller might have fed identical pathspec
 		 * twice.  Do not barf on such a mistake.
+		 * FIXME: parse_pathspec should have eliminated
+		 * duplicate pathspec.
 		 */
 		for (found_dup = other = 0;
-		     !found_dup && pathspec[other];
+		     !found_dup && other < pathspec->nr;
 		     other++) {
 			if (other == num || !ps_matched[other])
 				continue;
-			if (!strcmp(pathspec[other], pathspec[num]))
+			if (!strcmp(pathspec->items[other].original,
+				    pathspec->items[num].original))
 				/*
 				 * Ok, we have a match already.
 				 */
@@ -382,9 +386,8 @@ int report_path_error(const char *ps_matched, const char **pathspec, const char
 		if (found_dup)
 			continue;
 
-		name = quote_path_relative(pathspec[num], -1, &sb, prefix);
 		error("pathspec '%s' did not match any file(s) known to git.",
-		      name);
+		      pathspec->items[num].original);
 		errors++;
 	}
 	strbuf_release(&sb);
@@ -580,7 +583,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 
 	if (ps_matched) {
 		int bad;
-		bad = report_path_error(ps_matched, pathspec.raw, prefix);
+		bad = report_path_error(ps_matched, &pathspec, prefix);
 		if (bad)
 			fprintf(stderr, "Did you forget to 'git add'?\n");
 
diff --git a/cache.h b/cache.h
index cc90d7b..e5c3e26 100644
--- a/cache.h
+++ b/cache.h
@@ -1267,7 +1267,7 @@ extern int ws_blank_line(const char *line, int len, unsigned ws_rule);
 #define ws_tab_width(rule)     ((rule) & WS_TAB_WIDTH_MASK)
 
 /* ls-files */
-int report_path_error(const char *ps_matched, const char **pathspec, const char *prefix);
+int report_path_error(const char *ps_matched, const struct pathspec *pathspec, const char *prefix);
 void overlay_tree_on_cache(const char *tree_name, const char *prefix);
 
 char *alias_lookup(const char *alias);
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 31/45] Convert refresh_index to take struct pathspec

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/add.c    | 15 +++++++--------
 builtin/commit.c |  2 +-
 builtin/rm.c     |  2 +-
 cache.h          |  2 +-
 read-cache.c     |  5 +++--
 5 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/builtin/add.c b/builtin/add.c
index 2b20d7d..cae002d 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -121,19 +121,18 @@ static char *prune_directory(struct dir_struct *dir, const char **pathspec, int
 	return seen;
 }
 
-static void refresh(int verbose, const char **pathspec)
+static void refresh(int verbose, const struct pathspec *pathspec)
 {
 	char *seen;
-	int i, specs;
+	int i;
 
-	for (specs = 0; pathspec[specs];  specs++)
-		/* nothing */;
-	seen = xcalloc(specs, 1);
+	seen = xcalloc(pathspec->nr, 1);
 	refresh_index(&the_index, verbose ? REFRESH_IN_PORCELAIN : REFRESH_QUIET,
 		      pathspec, seen, _("Unstaged changes after refreshing the index:"));
-	for (i = 0; i < specs; i++) {
+	for (i = 0; i < pathspec->nr; i++) {
 		if (!seen[i])
-			die(_("pathspec '%s' did not match any files"), pathspec[i]);
+			die(_("pathspec '%s' did not match any files"),
+			    pathspec->items[i].match);
 	}
         free(seen);
 }
@@ -412,7 +411,7 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 	}
 
 	if (refresh_only) {
-		refresh(verbose, pathspec.raw);
+		refresh(verbose, &pathspec);
 		goto finish;
 	}
 
diff --git a/builtin/commit.c b/builtin/commit.c
index eba53af..75ce807 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1198,7 +1198,7 @@ int cmd_status(int argc, const char **argv, const char *prefix)
 		       prefix, argv);
 
 	read_cache_preload(&s.pathspec);
-	refresh_index(&the_index, REFRESH_QUIET|REFRESH_UNMERGED, s.pathspec.raw, NULL, NULL);
+	refresh_index(&the_index, REFRESH_QUIET|REFRESH_UNMERGED, &s.pathspec, NULL, NULL);
 
 	fd = hold_locked_index(&index_lock, 0);
 	if (0 <= fd)
diff --git a/builtin/rm.c b/builtin/rm.c
index eb1b745..b84d6c7 100644
--- a/builtin/rm.c
+++ b/builtin/rm.c
@@ -251,7 +251,7 @@ int cmd_rm(int argc, const char **argv, const char *prefix)
 	}
 
 	parse_pathspec(&pathspec, 0, PATHSPEC_PREFER_CWD, prefix, argv);
-	refresh_index(&the_index, REFRESH_QUIET, pathspec.raw, NULL, NULL);
+	refresh_index(&the_index, REFRESH_QUIET, &pathspec, NULL, NULL);
 
 	seen = NULL;
 	seen = xcalloc(pathspec.nr, 1);
diff --git a/cache.h b/cache.h
index e5c3e26..26d0f98 100644
--- a/cache.h
+++ b/cache.h
@@ -490,7 +490,7 @@ extern void fill_stat_cache_info(struct cache_entry *ce, struct stat *st);
 #define REFRESH_IGNORE_MISSING	0x0008	/* ignore non-existent */
 #define REFRESH_IGNORE_SUBMODULES	0x0010	/* ignore submodules */
 #define REFRESH_IN_PORCELAIN	0x0020	/* user friendly output, not "needs update" */
-extern int refresh_index(struct index_state *, unsigned int flags, const char **pathspec, char *seen, const char *header_msg);
+extern int refresh_index(struct index_state *, unsigned int flags, const struct pathspec *pathspec, char *seen, const char *header_msg);
 
 struct lock_file {
 	struct lock_file *next;
diff --git a/read-cache.c b/read-cache.c
index 827ae55..9715af0 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1097,7 +1097,8 @@ static void show_file(const char * fmt, const char * name, int in_porcelain,
 	printf(fmt, name);
 }
 
-int refresh_index(struct index_state *istate, unsigned int flags, const char **pathspec,
+int refresh_index(struct index_state *istate, unsigned int flags,
+		  const struct pathspec *pathspec,
 		  char *seen, const char *header_msg)
 {
 	int i;
@@ -1132,7 +1133,7 @@ int refresh_index(struct index_state *istate, unsigned int flags, const char **p
 			continue;
 
 		if (pathspec &&
-		    !match_pathspec(pathspec, ce->name, ce_namelen(ce), 0, seen))
+		    !match_pathspec_depth(pathspec, ce->name, ce_namelen(ce), 0, seen))
 			filtered = 1;
 
 		if (ce_stage(ce)) {
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 32/45] Convert {read,fill}_directory to take struct pathspec

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/add.c      |  2 +-
 builtin/clean.c    |  2 +-
 builtin/grep.c     |  2 +-
 builtin/ls-files.c |  2 +-
 dir.c              | 16 +++++++++++-----
 dir.h              |  4 ++--
 wt-status.c        |  4 ++--
 7 files changed, 19 insertions(+), 13 deletions(-)

diff --git a/builtin/add.c b/builtin/add.c
index cae002d..7b9e50c 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -405,7 +405,7 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 		}
 
 		/* This picks up the paths that are not tracked */
-		baselen = fill_directory(&dir, pathspec.raw);
+		baselen = fill_directory(&dir, &pathspec);
 		if (pathspec.nr)
 			seen = prune_directory(&dir, pathspec.raw, baselen);
 	}
diff --git a/builtin/clean.c b/builtin/clean.c
index fdd4980..d540ca4 100644
--- a/builtin/clean.c
+++ b/builtin/clean.c
@@ -214,7 +214,7 @@ int cmd_clean(int argc, const char **argv, const char *prefix)
 		       PATHSPEC_PREFER_CWD,
 		       prefix, argv);
 
-	fill_directory(&dir, pathspec.raw);
+	fill_directory(&dir, &pathspec);
 
 	for (i = 0; i < dir.nr; i++) {
 		struct dir_entry *ent = dir.entries[i];
diff --git a/builtin/grep.c b/builtin/grep.c
index 1c7397a..301d280 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -523,7 +523,7 @@ static int grep_directory(struct grep_opt *opt, const struct pathspec *pathspec,
 	if (exc_std)
 		setup_standard_excludes(&dir);
 
-	fill_directory(&dir, pathspec->raw);
+	fill_directory(&dir, pathspec);
 	for (i = 0; i < dir.nr; i++) {
 		const char *name = dir.entries[i]->name;
 		int namelen = strlen(name);
diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index a33ea18..37f0392 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -218,7 +218,7 @@ static void show_files(struct dir_struct *dir)
 
 	/* For cached/deleted files we don't need to even do the readdir */
 	if (show_others || show_killed) {
-		fill_directory(dir, pathspec.raw);
+		fill_directory(dir, &pathspec);
 		if (show_others)
 			show_other_files(dir);
 		if (show_killed)
diff --git a/dir.c b/dir.c
index 6094ba8..20e6a3e 100644
--- a/dir.c
+++ b/dir.c
@@ -99,7 +99,7 @@ char *common_prefix(const char **pathspec)
 	return len ? xmemdupz(*pathspec, len) : NULL;
 }
 
-int fill_directory(struct dir_struct *dir, const char **pathspec)
+int fill_directory(struct dir_struct *dir, const struct pathspec *pathspec)
 {
 	size_t len;
 
@@ -107,10 +107,10 @@ int fill_directory(struct dir_struct *dir, const char **pathspec)
 	 * Calculate common prefix for the pathspec, and
 	 * use that to optimize the directory walk
 	 */
-	len = common_prefix_len(pathspec);
+	len = common_prefix_len(pathspec->raw);
 
 	/* Read the directory and prune it */
-	read_directory(dir, pathspec ? *pathspec : "", len, pathspec);
+	read_directory(dir, pathspec->nr ? pathspec->raw[0] : "", len, pathspec);
 	return len;
 }
 
@@ -1438,14 +1438,20 @@ static int treat_leading_path(struct dir_struct *dir,
 	return rc;
 }
 
-int read_directory(struct dir_struct *dir, const char *path, int len, const char **pathspec)
+int read_directory(struct dir_struct *dir, const char *path, int len, const struct pathspec *pathspec)
 {
 	struct path_simplify *simplify;
 
+	/*
+	 * Check out create_simplify()
+	 */
+	if (pathspec)
+		GUARD_PATHSPEC(pathspec, PATHSPEC_FROMTOP | PATHSPEC_MAXDEPTH);
+
 	if (has_symlink_leading_path(path, len))
 		return dir->nr;
 
-	simplify = create_simplify(pathspec);
+	simplify = create_simplify(pathspec ? pathspec->raw : NULL);
 	if (!len || treat_leading_path(dir, path, len, simplify))
 		read_directory_recursive(dir, path, len, 0, simplify);
 	free_simplify(simplify);
diff --git a/dir.h b/dir.h
index 89427fd..70fdc98 100644
--- a/dir.h
+++ b/dir.h
@@ -134,8 +134,8 @@ extern int match_pathspec_depth(const struct pathspec *pathspec,
 				int prefix, char *seen);
 extern int within_depth(const char *name, int namelen, int depth, int max_depth);
 
-extern int fill_directory(struct dir_struct *dir, const char **pathspec);
-extern int read_directory(struct dir_struct *, const char *path, int len, const char **pathspec);
+extern int fill_directory(struct dir_struct *dir, const struct pathspec *pathspec);
+extern int read_directory(struct dir_struct *, const char *path, int len, const struct pathspec *pathspec);
 
 extern int is_excluded_from_list(const char *pathname, int pathlen, const char *basename,
 				 int *dtype, struct exclude_list *el);
diff --git a/wt-status.c b/wt-status.c
index 5dec9af..1105fdd 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -503,7 +503,7 @@ static void wt_status_collect_untracked(struct wt_status *s)
 			DIR_SHOW_OTHER_DIRECTORIES | DIR_HIDE_EMPTY_DIRECTORIES;
 	setup_standard_excludes(&dir);
 
-	fill_directory(&dir, s->pathspec.raw);
+	fill_directory(&dir, &s->pathspec);
 	for (i = 0; i < dir.nr; i++) {
 		struct dir_entry *ent = dir.entries[i];
 		if (cache_name_is_other(ent->name, ent->len) &&
@@ -517,7 +517,7 @@ static void wt_status_collect_untracked(struct wt_status *s)
 		dir.flags = DIR_SHOW_IGNORED;
 		if (s->show_untracked_files != SHOW_ALL_UNTRACKED_FILES)
 			dir.flags |= DIR_SHOW_OTHER_DIRECTORIES;
-		fill_directory(&dir, s->pathspec.raw);
+		fill_directory(&dir, &s->pathspec);
 		for (i = 0; i < dir.nr; i++) {
 			struct dir_entry *ent = dir.entries[i];
 			if (cache_name_is_other(ent->name, ent->len) &&
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 33/45] Convert add_files_to_cache to take struct pathspec

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/add.c    | 8 +++++---
 builtin/commit.c | 2 +-
 cache.h          | 2 +-
 3 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/builtin/add.c b/builtin/add.c
index 7b9e50c..70bfc05 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -81,13 +81,15 @@ static void update_callback(struct diff_queue_struct *q,
 	}
 }
 
-int add_files_to_cache(const char *prefix, const char **pathspec, int flags)
+int add_files_to_cache(const char *prefix,
+		       const struct pathspec *pathspec, int flags)
 {
 	struct update_callback_data data;
 	struct rev_info rev;
 	init_revisions(&rev, prefix);
 	setup_revisions(0, NULL, &rev, NULL);
-	init_pathspec(&rev.prune_data, pathspec);
+	if (pathspec)
+		copy_pathspec(&rev.prune_data, pathspec);
 	rev.diffopt.output_format = DIFF_FORMAT_CALLBACK;
 	rev.diffopt.format_callback = update_callback;
 	data.flags = flags;
@@ -446,7 +448,7 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 
 	plug_bulk_checkin();
 
-	exit_status |= add_files_to_cache(prefix, pathspec.raw, flags);
+	exit_status |= add_files_to_cache(prefix, &pathspec, flags);
 
 	if (add_new_files)
 		exit_status |= add_files(&dir, flags);
diff --git a/builtin/commit.c b/builtin/commit.c
index 75ce807..b5a92fa 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -329,7 +329,7 @@ static char *prepare_index(int argc, const char **argv, const char *prefix,
 	 */
 	if (all || (also && pathspec.nr)) {
 		fd = hold_locked_index(&index_lock, 1);
-		add_files_to_cache(also ? prefix : NULL, pathspec.raw, 0);
+		add_files_to_cache(also ? prefix : NULL, &pathspec, 0);
 		refresh_cache_or_die(refresh_flags);
 		update_main_cache_tree(WRITE_TREE_SILENT);
 		if (write_cache(fd, active_cache, active_nr) ||
diff --git a/cache.h b/cache.h
index 26d0f98..6297783 100644
--- a/cache.h
+++ b/cache.h
@@ -1233,7 +1233,7 @@ void packet_trace_identity(const char *prog);
  * return 0 if success, 1 - if addition of a file failed and
  * ADD_FILES_IGNORE_ERRORS was specified in flags
  */
-int add_files_to_cache(const char *prefix, const char **pathspec, int flags);
+int add_files_to_cache(const char *prefix, const struct pathspec *pathspec, int flags);
 
 /* diff.c */
 extern int diff_auto_refresh_index;
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 34/45] Convert common_prefix() to use struct pathspec

[Nguyễn Thái Ngọc Duy]

The code now takes advantage of nowildcard_len field.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/commit.c   |  2 +-
 builtin/ls-files.c |  2 +-
 dir.c              | 31 +++++++++++++++----------------
 dir.h              |  2 +-
 4 files changed, 18 insertions(+), 19 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index b5a92fa..1c4b73a 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -192,7 +192,7 @@ static int list_paths(struct string_list *list, const char *with_tree,
 	m = xcalloc(1, pattern->nr);
 
 	if (with_tree) {
-		char *max_prefix = common_prefix(pattern->raw);
+		char *max_prefix = common_prefix(pattern);
 		overlay_tree_on_cache(with_tree, max_prefix ? max_prefix : prefix);
 		free(max_prefix);
 	}
diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index 37f0392..f211539 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -551,7 +551,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 		       prefix, argv);
 
 	/* Find common prefix for all pathspec's */
-	max_prefix = common_prefix(pathspec.raw);
+	max_prefix = common_prefix(&pathspec);
 	max_prefix_len = max_prefix ? strlen(max_prefix) : 0;
 
 	/* Treat unmatching pathspec elements as errors */
diff --git a/dir.c b/dir.c
index 20e6a3e..a646e55 100644
--- a/dir.c
+++ b/dir.c
@@ -60,26 +60,25 @@ inline int git_fnmatch(const char *pattern, const char *string,
 	return fnmatch(pattern, string, fnm_flags);
 }
 
-static size_t common_prefix_len(const char **pathspec)
+static size_t common_prefix_len(const struct pathspec *pathspec)
 {
-	const char *n, *first;
+	int n;
 	size_t max = 0;
-	int literal = limit_pathspec_to_literal();
 
-	if (!pathspec)
-		return max;
-
-	first = *pathspec;
-	while ((n = *pathspec++)) {
-		size_t i, len = 0;
-		for (i = 0; first == n || i < max; i++) {
-			char c = n[i];
-			if (!c || c != first[i] || (!literal && is_glob_special(c)))
+	GUARD_PATHSPEC(pathspec, PATHSPEC_FROMTOP | PATHSPEC_MAXDEPTH);
+
+	for (n = 0; n < pathspec->nr; n++) {
+		size_t i = 0, len = 0;
+		while (i < pathspec->items[n].nowildcard_len &&
+		       (n == 0 || i < max)) {
+			char c = pathspec->items[n].match[i];
+			if (c != pathspec->items[0].match[i])
 				break;
 			if (c == '/')
 				len = i + 1;
+			i++;
 		}
-		if (first == n || len < max) {
+		if (n == 0 || len < max) {
 			max = len;
 			if (!max)
 				break;
@@ -92,11 +91,11 @@ static size_t common_prefix_len(const char **pathspec)
  * Returns a copy of the longest leading path common among all
  * pathspecs.
  */
-char *common_prefix(const char **pathspec)
+char *common_prefix(const struct pathspec *pathspec)
 {
 	unsigned long len = common_prefix_len(pathspec);
 
-	return len ? xmemdupz(*pathspec, len) : NULL;
+	return len ? xmemdupz(pathspec->items[0].match, len) : NULL;
 }
 
 int fill_directory(struct dir_struct *dir, const struct pathspec *pathspec)
@@ -107,7 +106,7 @@ int fill_directory(struct dir_struct *dir, const struct pathspec *pathspec)
 	 * Calculate common prefix for the pathspec, and
 	 * use that to optimize the directory walk
 	 */
-	len = common_prefix_len(pathspec->raw);
+	len = common_prefix_len(pathspec);
 
 	/* Read the directory and prune it */
 	read_directory(dir, pathspec->nr ? pathspec->raw[0] : "", len, pathspec);
diff --git a/dir.h b/dir.h
index 70fdc98..d3f76df 100644
--- a/dir.h
+++ b/dir.h
@@ -127,7 +127,7 @@ struct dir_struct {
 #define MATCHED_EXACTLY 3
 extern int simple_length(const char *match);
 extern int no_wildcard(const char *string);
-extern char *common_prefix(const char **pathspec);
+extern char *common_prefix(const struct pathspec *pathspec);
 extern int match_pathspec(const char **pathspec, const char *name, int namelen, int prefix, char *seen);
 extern int match_pathspec_depth(const struct pathspec *pathspec,
 				const char *name, int namelen,
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 35/45] Remove diff_tree_{setup,release}_paths

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/blame.c | 12 ++++++------
 builtin/reset.c |  9 +++++----
 diff.h          |  2 --
 notes-merge.c   |  4 ++--
 revision.c      |  5 +++--
 tree-diff.c     | 18 ++++--------------
 6 files changed, 20 insertions(+), 30 deletions(-)

diff --git a/builtin/blame.c b/builtin/blame.c
index 86100e9..da2dc4c 100644
--- a/builtin/blame.c
+++ b/builtin/blame.c
@@ -407,7 +407,7 @@ static struct origin *find_origin(struct scoreboard *sb,
 	paths[0] = origin->path;
 	paths[1] = NULL;
 
-	diff_tree_setup_paths(paths, &diff_opts);
+	init_pathspec(&diff_opts.pathspec, paths);
 	diff_setup_done(&diff_opts);
 
 	if (is_null_sha1(origin->commit->object.sha1))
@@ -457,7 +457,7 @@ static struct origin *find_origin(struct scoreboard *sb,
 		}
 	}
 	diff_flush(&diff_opts);
-	diff_tree_release_paths(&diff_opts);
+	free_pathspec(&diff_opts.pathspec);
 	if (porigin) {
 		/*
 		 * Create a freestanding copy that is not part of
@@ -493,7 +493,7 @@ static struct origin *find_rename(struct scoreboard *sb,
 	diff_opts.output_format = DIFF_FORMAT_NO_OUTPUT;
 	diff_opts.single_follow = origin->path;
 	paths[0] = NULL;
-	diff_tree_setup_paths(paths, &diff_opts);
+	init_pathspec(&diff_opts.pathspec, paths);
 	diff_setup_done(&diff_opts);
 
 	if (is_null_sha1(origin->commit->object.sha1))
@@ -515,7 +515,7 @@ static struct origin *find_rename(struct scoreboard *sb,
 		}
 	}
 	diff_flush(&diff_opts);
-	diff_tree_release_paths(&diff_opts);
+	free_pathspec(&diff_opts.pathspec);
 	return porigin;
 }
 
@@ -1073,7 +1073,7 @@ static int find_copy_in_parent(struct scoreboard *sb,
 	diff_opts.output_format = DIFF_FORMAT_NO_OUTPUT;
 
 	paths[0] = NULL;
-	diff_tree_setup_paths(paths, &diff_opts);
+	init_pathspec(&diff_opts.pathspec, paths);
 	diff_setup_done(&diff_opts);
 
 	/* Try "find copies harder" on new path if requested;
@@ -1156,7 +1156,7 @@ static int find_copy_in_parent(struct scoreboard *sb,
 	}
 	reset_scanned_flag(sb);
 	diff_flush(&diff_opts);
-	diff_tree_release_paths(&diff_opts);
+	free_pathspec(&diff_opts.pathspec);
 	return retval;
 }
 
diff --git a/builtin/reset.c b/builtin/reset.c
index 7c6e8b6..ab3917d 100644
--- a/builtin/reset.c
+++ b/builtin/reset.c
@@ -130,12 +130,13 @@ static void update_index_from_diff(struct diff_queue_struct *q,
 	}
 }
 
-static int read_from_tree(const char **pathspec, unsigned char *tree_sha1)
+static int read_from_tree(const struct pathspec *pathspec,
+			  unsigned char *tree_sha1)
 {
 	struct diff_options opt;
 
 	memset(&opt, 0, sizeof(opt));
-	diff_tree_setup_paths(pathspec, &opt);
+	copy_pathspec(&opt.pathspec, pathspec);
 	opt.output_format = DIFF_FORMAT_CALLBACK;
 	opt.format_callback = update_index_from_diff;
 
@@ -144,7 +145,7 @@ static int read_from_tree(const char **pathspec, unsigned char *tree_sha1)
 		return 1;
 	diffcore_std(&opt);
 	diff_flush(&opt);
-	diff_tree_release_paths(&opt);
+	free_pathspec(&opt.pathspec);
 
 	return 0;
 }
@@ -329,7 +330,7 @@ int cmd_reset(int argc, const char **argv, const char *prefix)
 		struct lock_file *lock = xcalloc(1, sizeof(struct lock_file));
 		int newfd = hold_locked_index(lock, 1);
 		if (reset_type == MIXED) {
-			if (read_from_tree(pathspec.raw, sha1))
+			if (read_from_tree(&pathspec, sha1))
 				return 1;
 		} else {
 			int err = reset_index(sha1, reset_type, quiet);
diff --git a/diff.h b/diff.h
index d1bc914..b8df245 100644
--- a/diff.h
+++ b/diff.h
@@ -180,8 +180,6 @@ const char *diff_line_prefix(struct diff_options *);
 
 extern const char mime_boundary_leader[];
 
-extern void diff_tree_setup_paths(const char **paths, struct diff_options *);
-extern void diff_tree_release_paths(struct diff_options *);
 extern int diff_tree(struct tree_desc *t1, struct tree_desc *t2,
 		     const char *base, struct diff_options *opt);
 extern int diff_tree_sha1(const unsigned char *old, const unsigned char *new,
diff --git a/notes-merge.c b/notes-merge.c
index 0f67bd3..f1a3b39 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -169,7 +169,7 @@ static struct notes_merge_pair *diff_tree_remote(struct notes_merge_options *o,
 		       sha1_to_hex(mp->remote));
 	}
 	diff_flush(&opt);
-	diff_tree_release_paths(&opt);
+	free_pathspec(&opt.pathspec);
 
 	*num_changes = len;
 	return changes;
@@ -255,7 +255,7 @@ static void diff_tree_local(struct notes_merge_options *o,
 		       sha1_to_hex(mp->local));
 	}
 	diff_flush(&opt);
-	diff_tree_release_paths(&opt);
+	free_pathspec(&opt.pathspec);
 }
 
 static void check_notes_merge_worktree(struct notes_merge_options *o)
diff --git a/revision.c b/revision.c
index e257614..c0ced84 100644
--- a/revision.c
+++ b/revision.c
@@ -1886,12 +1886,13 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s
 		revs->limited = 1;
 
 	if (revs->prune_data.nr) {
-		diff_tree_setup_paths(revs->prune_data.raw, &revs->pruning);
+		copy_pathspec(&revs->pruning.pathspec, &revs->prune_data);
 		/* Can't prune commits with rename following: the paths change.. */
 		if (!DIFF_OPT_TST(&revs->diffopt, FOLLOW_RENAMES))
 			revs->prune = 1;
 		if (!revs->full_diff)
-			diff_tree_setup_paths(revs->prune_data.raw, &revs->diffopt);
+			copy_pathspec(&revs->diffopt.pathspec,
+				      &revs->prune_data);
 	}
 	if (revs->combine_merges)
 		revs->ignore_merges = 0;
diff --git a/tree-diff.c b/tree-diff.c
index 5a87614..f4c92f6 100644
--- a/tree-diff.c
+++ b/tree-diff.c
@@ -229,11 +229,11 @@ static void try_to_follow_renames(struct tree_desc *t1, struct tree_desc *t2, co
 	diff_opts.break_opt = opt->break_opt;
 	diff_opts.rename_score = opt->rename_score;
 	paths[0] = NULL;
-	diff_tree_setup_paths(paths, &diff_opts);
+	init_pathspec(&diff_opts.pathspec, paths);
 	diff_setup_done(&diff_opts);
 	diff_tree(t1, t2, base, &diff_opts);
 	diffcore_std(&diff_opts);
-	diff_tree_release_paths(&diff_opts);
+	free_pathspec(&diff_opts.pathspec);
 
 	/* Go through the new set of filepairing, and see if we find a more interesting one */
 	opt->found_follow = 0;
@@ -252,9 +252,9 @@ static void try_to_follow_renames(struct tree_desc *t1, struct tree_desc *t2, co
 			choice = p;
 
 			/* Update the path we use from now on.. */
-			diff_tree_release_paths(opt);
+			free_pathspec(&opt->pathspec);
 			opt->pathspec.raw[0] = xstrdup(p->one->path);
-			diff_tree_setup_paths(opt->pathspec.raw, opt);
+			init_pathspec(&opt->pathspec, opt->pathspec.raw);
 
 			/*
 			 * The caller expects us to return a set of vanilla
@@ -328,13 +328,3 @@ int diff_root_tree_sha1(const unsigned char *new, const char *base, struct diff_
 	free(tree);
 	return retval;
 }
-
-void diff_tree_release_paths(struct diff_options *opt)
-{
-	free_pathspec(&opt->pathspec);
-}
-
-void diff_tree_setup_paths(const char **p, struct diff_options *opt)
-{
-	init_pathspec(&opt->pathspec, p);
-}
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 36/45] Remove init_pathspec() in favor of parse_pathspec()

[Nguyễn Thái Ngọc Duy]

While at there, move free_pathspec() to pathspec.c

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/blame.c    |  8 +-------
 builtin/log.c      |  2 +-
 builtin/ls-files.c | 11 +++++------
 diff-lib.c         |  2 +-
 dir.c              | 58 ------------------------------------------------------
 merge-recursive.c  |  2 +-
 pathspec.c         |  6 ++++++
 pathspec.h         |  1 -
 revision.c         |  2 +-
 tree-diff.c        | 10 +++++-----
 10 files changed, 21 insertions(+), 81 deletions(-)

diff --git a/builtin/blame.c b/builtin/blame.c
index da2dc4c..5b91e94 100644
--- a/builtin/blame.c
+++ b/builtin/blame.c
@@ -407,7 +407,7 @@ static struct origin *find_origin(struct scoreboard *sb,
 	paths[0] = origin->path;
 	paths[1] = NULL;
 
-	init_pathspec(&diff_opts.pathspec, paths);
+	parse_pathspec(&diff_opts.pathspec, PATHSPEC_ALL_MAGIC, 0, "", paths);
 	diff_setup_done(&diff_opts);
 
 	if (is_null_sha1(origin->commit->object.sha1))
@@ -485,15 +485,12 @@ static struct origin *find_rename(struct scoreboard *sb,
 	struct origin *porigin = NULL;
 	struct diff_options diff_opts;
 	int i;
-	const char *paths[2];
 
 	diff_setup(&diff_opts);
 	DIFF_OPT_SET(&diff_opts, RECURSIVE);
 	diff_opts.detect_rename = DIFF_DETECT_RENAME;
 	diff_opts.output_format = DIFF_FORMAT_NO_OUTPUT;
 	diff_opts.single_follow = origin->path;
-	paths[0] = NULL;
-	init_pathspec(&diff_opts.pathspec, paths);
 	diff_setup_done(&diff_opts);
 
 	if (is_null_sha1(origin->commit->object.sha1))
@@ -1058,7 +1055,6 @@ static int find_copy_in_parent(struct scoreboard *sb,
 			       int opt)
 {
 	struct diff_options diff_opts;
-	const char *paths[1];
 	int i, j;
 	int retval;
 	struct blame_list *blame_list;
@@ -1072,8 +1068,6 @@ static int find_copy_in_parent(struct scoreboard *sb,
 	DIFF_OPT_SET(&diff_opts, RECURSIVE);
 	diff_opts.output_format = DIFF_FORMAT_NO_OUTPUT;
 
-	paths[0] = NULL;
-	init_pathspec(&diff_opts.pathspec, paths);
 	diff_setup_done(&diff_opts);
 
 	/* Try "find copies harder" on new path if requested;
diff --git a/builtin/log.c b/builtin/log.c
index 8f0b2e8..812bcb9 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -469,7 +469,7 @@ int cmd_show(int argc, const char **argv, const char *prefix)
 	init_grep_defaults();
 	git_config(git_log_config, NULL);
 
-	init_pathspec(&match_all, NULL);
+	memset(&match_all, 0, sizeof(match_all));
 	init_revisions(&rev, prefix);
 	rev.diff = 1;
 	rev.always_show_header = 1;
diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index f211539..50842ba 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -320,13 +320,12 @@ void overlay_tree_on_cache(const char *tree_name, const char *prefix)
 	}
 
 	if (prefix) {
-		static const char *(matchbuf[2]);
-		matchbuf[0] = prefix;
-		matchbuf[1] = NULL;
-		init_pathspec(&pathspec, matchbuf);
-		pathspec.items[0].nowildcard_len = pathspec.items[0].len;
+		static const char *(matchbuf[1]);
+		matchbuf[0] = NULL;
+		parse_pathspec(&pathspec, PATHSPEC_ALL_MAGIC,
+			       PATHSPEC_PREFER_CWD, prefix, matchbuf);
 	} else
-		init_pathspec(&pathspec, NULL);
+		memset(&pathspec, 0, sizeof(pathspec));
 	if (read_tree(tree, 1, &pathspec))
 		die("unable to read tree entries %s", tree_name);
 
diff --git a/diff-lib.c b/diff-lib.c
index 4729157..68fc991 100644
--- a/diff-lib.c
+++ b/diff-lib.c
@@ -499,7 +499,7 @@ int do_diff_cache(const unsigned char *tree_sha1, struct diff_options *opt)
 	struct rev_info revs;
 
 	init_revisions(&revs, NULL);
-	init_pathspec(&revs.prune_data, opt->pathspec.raw);
+	copy_pathspec(&revs.prune_data, &opt->pathspec);
 	revs.diffopt = *opt;
 
 	if (diff_cache(&revs, tree_sha1, NULL, 1))
diff --git a/dir.c b/dir.c
index a646e55..e354abb 100644
--- a/dir.c
+++ b/dir.c
@@ -1630,64 +1630,6 @@ int remove_path(const char *name)
 	return 0;
 }
 
-static int pathspec_item_cmp(const void *a_, const void *b_)
-{
-	struct pathspec_item *a, *b;
-
-	a = (struct pathspec_item *)a_;
-	b = (struct pathspec_item *)b_;
-	return strcmp(a->match, b->match);
-}
-
-int init_pathspec(struct pathspec *pathspec, const char **paths)
-{
-	const char **p = paths;
-	int i;
-
-	memset(pathspec, 0, sizeof(*pathspec));
-	if (!p)
-		return 0;
-	while (*p)
-		p++;
-	pathspec->raw = paths;
-	pathspec->nr = p - paths;
-	if (!pathspec->nr)
-		return 0;
-
-	pathspec->items = xmalloc(sizeof(struct pathspec_item)*pathspec->nr);
-	for (i = 0; i < pathspec->nr; i++) {
-		struct pathspec_item *item = pathspec->items+i;
-		const char *path = paths[i];
-
-		item->match = path;
-		item->original = path;
-		item->len = strlen(path);
-		item->flags = 0;
-		if (limit_pathspec_to_literal()) {
-			item->nowildcard_len = item->len;
-		} else {
-			item->nowildcard_len = simple_length(path);
-			if (item->nowildcard_len < item->len) {
-				pathspec->has_wildcard = 1;
-				if (path[item->nowildcard_len] == '*' &&
-				    no_wildcard(path + item->nowildcard_len + 1))
-					item->flags |= PATHSPEC_ONESTAR;
-			}
-		}
-	}
-
-	qsort(pathspec->items, pathspec->nr,
-	      sizeof(struct pathspec_item), pathspec_item_cmp);
-
-	return 0;
-}
-
-void free_pathspec(struct pathspec *pathspec)
-{
-	free(pathspec->items);
-	pathspec->items = NULL;
-}
-
 int limit_pathspec_to_literal(void)
 {
 	static int flag = -1;
diff --git a/merge-recursive.c b/merge-recursive.c
index ea9dbd3..8395b9b 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -297,7 +297,7 @@ static int get_files_dirs(struct merge_options *o, struct tree *tree)
 {
 	int n;
 	struct pathspec match_all;
-	init_pathspec(&match_all, NULL);
+	memset(&match_all, 0, sizeof(match_all));
 	if (read_tree_recursive(tree, "", 0, 0, &match_all, save_files_dirs, o))
 		return 0;
 	n = o->current_file_set.nr + o->current_directory_set.nr;
diff --git a/pathspec.c b/pathspec.c
index 0b658cb..767ce65 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -370,3 +370,9 @@ void copy_pathspec(struct pathspec *dst, const struct pathspec *src)
 	memcpy(dst->items, src->items,
 	       sizeof(struct pathspec_item) * dst->nr);
 }
+
+void free_pathspec(struct pathspec *pathspec)
+{
+	free(pathspec->items);
+	pathspec->items = NULL;
+}
diff --git a/pathspec.h b/pathspec.h
index 44253c8..9e9e442 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -44,7 +44,6 @@ struct pathspec {
 #define PATHSPEC_PREFIX_ORIGIN (1<<6)
 #define PATHSPEC_KEEP_ORDER (1<<7)
 
-extern int init_pathspec(struct pathspec *, const char **);
 extern void parse_pathspec(struct pathspec *pathspec,
 			   unsigned magic_mask,
 			   unsigned flags,
diff --git a/revision.c b/revision.c
index c0ced84..56ada54 100644
--- a/revision.c
+++ b/revision.c
@@ -1113,7 +1113,7 @@ static void prepare_show_merge(struct rev_info *revs)
 			i++;
 	}
 	free_pathspec(&revs->prune_data);
-	init_pathspec(&revs->prune_data, prune);
+	parse_pathspec(&revs->prune_data, PATHSPEC_ALL_MAGIC, 0, "", prune);
 	revs->limited = 1;
 }
 
diff --git a/tree-diff.c b/tree-diff.c
index f4c92f6..e1145c6 100644
--- a/tree-diff.c
+++ b/tree-diff.c
@@ -195,7 +195,6 @@ static void try_to_follow_renames(struct tree_desc *t1, struct tree_desc *t2, co
 	struct diff_options diff_opts;
 	struct diff_queue_struct *q = &diff_queued_diff;
 	struct diff_filepair *choice;
-	const char *paths[1];
 	int i;
 
 	/*
@@ -228,8 +227,6 @@ static void try_to_follow_renames(struct tree_desc *t1, struct tree_desc *t2, co
 	diff_opts.single_follow = opt->pathspec.raw[0];
 	diff_opts.break_opt = opt->break_opt;
 	diff_opts.rename_score = opt->rename_score;
-	paths[0] = NULL;
-	init_pathspec(&diff_opts.pathspec, paths);
 	diff_setup_done(&diff_opts);
 	diff_tree(t1, t2, base, &diff_opts);
 	diffcore_std(&diff_opts);
@@ -247,14 +244,17 @@ static void try_to_follow_renames(struct tree_desc *t1, struct tree_desc *t2, co
 		 */
 		if ((p->status == 'R' || p->status == 'C') &&
 		    !strcmp(p->two->path, opt->pathspec.raw[0])) {
+			const char *path[2];
+
 			/* Switch the file-pairs around */
 			q->queue[i] = choice;
 			choice = p;
 
 			/* Update the path we use from now on.. */
+			path[0] = p->one->path;
+			path[1] = NULL;
 			free_pathspec(&opt->pathspec);
-			opt->pathspec.raw[0] = xstrdup(p->one->path);
-			init_pathspec(&opt->pathspec, opt->pathspec.raw);
+			parse_pathspec(&opt->pathspec, PATHSPEC_ALL_MAGIC, 0, "", path);
 
 			/*
 			 * The caller expects us to return a set of vanilla
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 37/45] Remove match_pathspec() in favor of match_pathspec_depth()

[Nguyễn Thái Ngọc Duy]

match_pathspec_depth was created to replace match_pathspec (see
61cf282 (pathspec: add match_pathspec_depth() - 2010-12-15). It took
more than two years, but the replacement finally happens :-)

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/add.c          |  30 +++++++-------
 builtin/check-ignore.c |   4 +-
 dir.c                  | 107 -------------------------------------------------
 dir.h                  |   1 -
 pathspec.c             |  19 ++++-----
 pathspec.h             |   4 +-
 6 files changed, 25 insertions(+), 140 deletions(-)

diff --git a/builtin/add.c b/builtin/add.c
index 70bfc05..aa0709e 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -100,26 +100,24 @@ int add_files_to_cache(const char *prefix,
 	return !!data.add_errors;
 }
 
-static char *prune_directory(struct dir_struct *dir, const char **pathspec, int prefix)
+static char *prune_directory(struct dir_struct *dir, struct pathspec *pathspec, int prefix)
 {
 	char *seen;
-	int i, specs;
+	int i;
 	struct dir_entry **src, **dst;
 
-	for (specs = 0; pathspec[specs];  specs++)
-		/* nothing */;
-	seen = xcalloc(specs, 1);
+	seen = xcalloc(pathspec->nr, 1);
 
 	src = dst = dir->entries;
 	i = dir->nr;
 	while (--i >= 0) {
 		struct dir_entry *entry = *src++;
-		if (match_pathspec(pathspec, entry->name, entry->len,
-				   prefix, seen))
+		if (match_pathspec_depth(pathspec, entry->name, entry->len,
+					 prefix, seen))
 			*dst++ = entry;
 	}
 	dir->nr = dst - dir->entries;
-	add_pathspec_matches_against_index(pathspec, seen, specs);
+	add_pathspec_matches_against_index(pathspec, seen);
 	return seen;
 }
 
@@ -409,7 +407,7 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 		/* This picks up the paths that are not tracked */
 		baselen = fill_directory(&dir, &pathspec);
 		if (pathspec.nr)
-			seen = prune_directory(&dir, pathspec.raw, baselen);
+			seen = prune_directory(&dir, &pathspec, baselen);
 	}
 
 	if (refresh_only) {
@@ -423,23 +421,23 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 
 		path_exclude_check_init(&check, &dir);
 		if (!seen)
-			seen = find_pathspecs_matching_against_index(pathspec.raw);
+			seen = find_pathspecs_matching_against_index(&pathspec);
 
 		/*
 		 * file_exists() assumes exact match
 		 */
 		GUARD_PATHSPEC(&pathspec, PATHSPEC_FROMTOP);
 
-		for (i = 0; pathspec.raw[i]; i++) {
-			if (!seen[i] && pathspec.raw[i][0]
-			    && !file_exists(pathspec.raw[i])) {
+		for (i = 0; i < pathspec.nr; i++) {
+			const char *path = pathspec.items[i].match;
+			if (!seen[i] && !file_exists(path)) {
 				if (ignore_missing) {
 					int dtype = DT_UNKNOWN;
-					if (is_path_excluded(&check, pathspec.raw[i], -1, &dtype))
-						dir_add_ignored(&dir, pathspec.raw[i], strlen(pathspec.raw[i]));
+					if (is_path_excluded(&check, path, -1, &dtype))
+						dir_add_ignored(&dir, path, pathspec.items[i].len);
 				} else
 					die(_("pathspec '%s' did not match any files"),
-					    pathspec.raw[i]);
+					    pathspec.items[i].original);
 			}
 		}
 		free(seen);
diff --git a/builtin/check-ignore.c b/builtin/check-ignore.c
index 6e55f06..80e6bf7 100644
--- a/builtin/check-ignore.c
+++ b/builtin/check-ignore.c
@@ -93,9 +93,9 @@ static int check_ignore(int argc, const char **argv, const char *prefix)
 	 * should not be ignored, in order to be consistent with
 	 * 'git status', 'git add' etc.
 	 */
-	seen = find_pathspecs_matching_against_index(pathspec.raw);
+	seen = find_pathspecs_matching_against_index(&pathspec);
 	for (i = 0; i < pathspec.nr; i++) {
-		const char *full_path = pathspec.raw[i];
+		const char *full_path = pathspec.items[i].match;
 		if (!seen[i]) {
 			exclude = last_exclude_matching_path(&check, full_path,
 							     -1, &dtype);
diff --git a/dir.c b/dir.c
index e354abb..9627d7e 100644
--- a/dir.c
+++ b/dir.c
@@ -140,113 +140,6 @@ int within_depth(const char *name, int namelen,
  *
  * It returns 0 when there is no match.
  */
-static int match_one(const char *match, const char *name, int namelen)
-{
-	int matchlen;
-	int literal = limit_pathspec_to_literal();
-
-	/* If the match was just the prefix, we matched */
-	if (!*match)
-		return MATCHED_RECURSIVELY;
-
-	if (ignore_case) {
-		for (;;) {
-			unsigned char c1 = tolower(*match);
-			unsigned char c2 = tolower(*name);
-			if (c1 == '\0' || (!literal && is_glob_special(c1)))
-				break;
-			if (c1 != c2)
-				return 0;
-			match++;
-			name++;
-			namelen--;
-		}
-	} else {
-		for (;;) {
-			unsigned char c1 = *match;
-			unsigned char c2 = *name;
-			if (c1 == '\0' || (!literal && is_glob_special(c1)))
-				break;
-			if (c1 != c2)
-				return 0;
-			match++;
-			name++;
-			namelen--;
-		}
-	}
-
-	/*
-	 * If we don't match the matchstring exactly,
-	 * we need to match by fnmatch
-	 */
-	matchlen = strlen(match);
-	if (strncmp_icase(match, name, matchlen)) {
-		if (literal)
-			return 0;
-		return !fnmatch_icase(match, name, 0) ? MATCHED_FNMATCH : 0;
-	}
-
-	if (namelen == matchlen)
-		return MATCHED_EXACTLY;
-	if (match[matchlen-1] == '/' || name[matchlen] == '/')
-		return MATCHED_RECURSIVELY;
-	return 0;
-}
-
-/*
- * Given a name and a list of pathspecs, returns the nature of the
- * closest (i.e. most specific) match of the name to any of the
- * pathspecs.
- *
- * The caller typically calls this multiple times with the same
- * pathspec and seen[] array but with different name/namelen
- * (e.g. entries from the index) and is interested in seeing if and
- * how each pathspec matches all the names it calls this function
- * with.  A mark is left in the seen[] array for each pathspec element
- * indicating the closest type of match that element achieved, so if
- * seen[n] remains zero after multiple invocations, that means the nth
- * pathspec did not match any names, which could indicate that the
- * user mistyped the nth pathspec.
- */
-int match_pathspec(const char **pathspec, const char *name, int namelen,
-		int prefix, char *seen)
-{
-	int i, retval = 0;
-
-	if (!pathspec)
-		return 1;
-
-	name += prefix;
-	namelen -= prefix;
-
-	for (i = 0; pathspec[i] != NULL; i++) {
-		int how;
-		const char *match = pathspec[i] + prefix;
-		if (seen && seen[i] == MATCHED_EXACTLY)
-			continue;
-		how = match_one(match, name, namelen);
-		if (how) {
-			if (retval < how)
-				retval = how;
-			if (seen && seen[i] < how)
-				seen[i] = how;
-		}
-	}
-	return retval;
-}
-
-/*
- * Does 'match' match the given name?
- * A match is found if
- *
- * (1) the 'match' string is leading directory of 'name', or
- * (2) the 'match' string is a wildcard and matches 'name', or
- * (3) the 'match' string is exactly the same as 'name'.
- *
- * and the return value tells which case it was.
- *
- * It returns 0 when there is no match.
- */
 static int match_pathspec_item(const struct pathspec_item *item, int prefix,
 			       const char *name, int namelen)
 {
diff --git a/dir.h b/dir.h
index d3f76df..13da62c 100644
--- a/dir.h
+++ b/dir.h
@@ -128,7 +128,6 @@ struct dir_struct {
 extern int simple_length(const char *match);
 extern int no_wildcard(const char *string);
 extern char *common_prefix(const struct pathspec *pathspec);
-extern int match_pathspec(const char **pathspec, const char *name, int namelen, int prefix, char *seen);
 extern int match_pathspec_depth(const struct pathspec *pathspec,
 				const char *name, int namelen,
 				int prefix, char *seen);
diff --git a/pathspec.c b/pathspec.c
index 767ce65..1ea1c4c 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -15,8 +15,8 @@
  * If seen[] has not already been written to, it may make sense
  * to use find_pathspecs_matching_against_index() instead.
  */
-void add_pathspec_matches_against_index(const char **pathspec,
-					char *seen, int specs)
+void add_pathspec_matches_against_index(const struct pathspec *pathspec,
+					char *seen)
 {
 	int num_unmatched = 0, i;
 
@@ -26,14 +26,14 @@ void add_pathspec_matches_against_index(const char **pathspec,
 	 * mistakenly think that the user gave a pathspec that did not match
 	 * anything.
 	 */
-	for (i = 0; i < specs; i++)
+	for (i = 0; i < pathspec->nr; i++)
 		if (!seen[i])
 			num_unmatched++;
 	if (!num_unmatched)
 		return;
 	for (i = 0; i < active_nr; i++) {
 		struct cache_entry *ce = active_cache[i];
-		match_pathspec(pathspec, ce->name, ce_namelen(ce), 0, seen);
+		match_pathspec_depth(pathspec, ce->name, ce_namelen(ce), 0, seen);
 	}
 }
 
@@ -45,15 +45,10 @@ void add_pathspec_matches_against_index(const char **pathspec,
  * nature of the "closest" (i.e. most specific) matches which each of the
  * given pathspecs achieves against all items in the index.
  */
-char *find_pathspecs_matching_against_index(const char **pathspec)
+char *find_pathspecs_matching_against_index(const struct pathspec *pathspec)
 {
-	char *seen;
-	int i;
-
-	for (i = 0; pathspec[i];  i++)
-		; /* just counting */
-	seen = xcalloc(i, 1);
-	add_pathspec_matches_against_index(pathspec, seen, i);
+	char *seen = xcalloc(pathspec->nr, 1);
+	add_pathspec_matches_against_index(pathspec, seen);
 	return seen;
 }
 
diff --git a/pathspec.h b/pathspec.h
index 9e9e442..30aea46 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -54,8 +54,8 @@ extern void free_pathspec(struct pathspec *);
 
 extern int limit_pathspec_to_literal(void);
 
-extern char *find_pathspecs_matching_against_index(const char **pathspec);
-extern void add_pathspec_matches_against_index(const char **pathspec, char *seen, int specs);
+extern char *find_pathspecs_matching_against_index(const struct pathspec *pathspec);
+extern void add_pathspec_matches_against_index(const struct pathspec *pathspec, char *seen);
 extern const char *check_path_for_gitlink(const char *path);
 extern void die_if_path_beyond_symlink(const char *path, const char *prefix);
 
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 38/45] tree-diff: remove the use of pathspec's raw[] in follow-rename codepath

[Nguyễn Thái Ngọc Duy]

Put a checkpoint to guard unsupported pathspec features in future.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 tree-diff.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tree-diff.c b/tree-diff.c
index e1145c6..21a50d8 100644
--- a/tree-diff.c
+++ b/tree-diff.c
@@ -224,7 +224,7 @@ static void try_to_follow_renames(struct tree_desc *t1, struct tree_desc *t2, co
 	DIFF_OPT_SET(&diff_opts, RECURSIVE);
 	DIFF_OPT_SET(&diff_opts, FIND_COPIES_HARDER);
 	diff_opts.output_format = DIFF_FORMAT_NO_OUTPUT;
-	diff_opts.single_follow = opt->pathspec.raw[0];
+	diff_opts.single_follow = opt->pathspec.items[0].match;
 	diff_opts.break_opt = opt->break_opt;
 	diff_opts.rename_score = opt->rename_score;
 	diff_setup_done(&diff_opts);
@@ -243,7 +243,7 @@ static void try_to_follow_renames(struct tree_desc *t1, struct tree_desc *t2, co
 		 * the future!
 		 */
 		if ((p->status == 'R' || p->status == 'C') &&
-		    !strcmp(p->two->path, opt->pathspec.raw[0])) {
+		    !strcmp(p->two->path, opt->pathspec.items[0].match)) {
 			const char *path[2];
 
 			/* Switch the file-pairs around */
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 39/45] parse_pathspec: make sure the prefix part is wildcard-free

[Nguyễn Thái Ngọc Duy]

Prepending prefix to pathspec is a trick to workaround the fact that
commands can be executed in a subdirectory, but all git commands run
at worktree's root. The prefix part should always be treated as
literal string. Make it so.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 cache.h    |  2 ++
 path.c     | 15 ++++++++++++++-
 pathspec.c | 16 ++++++++++++----
 pathspec.h |  2 +-
 setup.c    |  9 +++++----
 5 files changed, 34 insertions(+), 10 deletions(-)

diff --git a/cache.h b/cache.h
index 6297783..17e0b26 100644
--- a/cache.h
+++ b/cache.h
@@ -401,6 +401,7 @@ extern void setup_work_tree(void);
 extern const char *setup_git_directory_gently(int *);
 extern const char *setup_git_directory(void);
 extern char *prefix_path(const char *prefix, int len, const char *path);
+extern char *prefix_path_gently(const char *prefix, int *p_len, const char *path);
 extern const char *prefix_filename(const char *prefix, int len, const char *path);
 extern int check_filename(const char *prefix, const char *name);
 extern void verify_filename(const char *prefix,
@@ -712,6 +713,7 @@ const char *real_path(const char *path);
 const char *real_path_if_valid(const char *path);
 const char *absolute_path(const char *path);
 const char *relative_path(const char *abs, const char *base);
+int normalize_path_copy_len(char *dst, const char *src, int *prefix_len);
 int normalize_path_copy(char *dst, const char *src);
 int longest_ancestor_length(const char *path, struct string_list *prefixes);
 char *strip_path_suffix(const char *path, const char *suffix);
diff --git a/path.c b/path.c
index d3d3f8b..7baf334 100644
--- a/path.c
+++ b/path.c
@@ -487,8 +487,14 @@ const char *relative_path(const char *abs, const char *base)
  *
  * Note that this function is purely textual.  It does not follow symlinks,
  * verify the existence of the path, or make any system calls.
+ *
+ * prefix_len != NULL is for a specific case of prefix_pathspec():
+ * assume that src == dst and src[0..prefix_len-1] is already
+ * normalized, any time "../" eats up to the prefix_len part,
+ * prefix_len is reduced. In the end prefix_len is the remaining
+ * prefix that has not been overridden by user pathspec.
  */
-int normalize_path_copy(char *dst, const char *src)
+int normalize_path_copy_len(char *dst, const char *src, int *prefix_len)
 {
 	char *dst0;
 
@@ -563,11 +569,18 @@ int normalize_path_copy(char *dst, const char *src)
 		/* Windows: dst[-1] cannot be backslash anymore */
 		while (dst0 < dst && dst[-1] != '/')
 			dst--;
+		if (prefix_len && *prefix_len > dst - dst0)
+			*prefix_len = dst - dst0;
 	}
 	*dst = '\0';
 	return 0;
 }
 
+int normalize_path_copy(char *dst, const char *src)
+{
+	return normalize_path_copy_len(dst, src, NULL);
+}
+
 /*
  * path = Canonical absolute path
  * prefixes = string_list containing normalized, absolute paths without
diff --git a/pathspec.c b/pathspec.c
index 1ea1c4c..bc27692 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -149,10 +149,14 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 	magic |= short_magic;
 	*p_short_magic = short_magic;
 
-	if (magic & PATHSPEC_FROMTOP)
+	if (magic & PATHSPEC_FROMTOP) {
 		match = xstrdup(copyfrom);
-	else
-		match = prefix_path(prefix, prefixlen, copyfrom);
+		prefixlen = 0;
+	} else {
+		match = prefix_path_gently(prefix, &prefixlen, copyfrom);
+		if (!match)
+			die("%s: '%s' is outside repository", elt, copyfrom);
+	}
 	*raw = item->match = match;
 	/*
 	 * Prefix the pathspec (keep all magic) and put to
@@ -166,6 +170,7 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 	} else
 		item->original = elt;
 	item->len = strlen(item->match);
+	item->prefix = prefixlen;
 
 	if ((flags & PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP) &&
 	    (item->len >= 1 && item->match[item->len - 1] == '/') &&
@@ -197,8 +202,11 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 
 	if (limit_pathspec_to_literal())
 		item->nowildcard_len = item->len;
-	else
+	else {
 		item->nowildcard_len = simple_length(item->match);
+		if (item->nowildcard_len < prefixlen)
+			item->nowildcard_len = prefixlen;
+	}
 	item->flags = 0;
 	if (item->nowildcard_len < item->len &&
 	    item->match[item->nowildcard_len] == '*' &&
diff --git a/pathspec.h b/pathspec.h
index 30aea46..9f9ecd2 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -21,7 +21,7 @@ struct pathspec {
 		const char *match;
 		const char *original;
 		unsigned magic;
-		int len;
+		int len, prefix;
 		int nowildcard_len;
 		int flags;
 	} *items;
diff --git a/setup.c b/setup.c
index acdc634..e661a71 100644
--- a/setup.c
+++ b/setup.c
@@ -5,10 +5,11 @@
 static int inside_git_dir = -1;
 static int inside_work_tree = -1;
 
-static char *prefix_path_gently(const char *prefix, int len, const char *path)
+char *prefix_path_gently(const char *prefix, int *p_len, const char *path)
 {
 	const char *orig = path;
 	char *sanitized;
+	int len = *p_len;
 	if (is_absolute_path(orig)) {
 		const char *temp = real_path(path);
 		sanitized = xmalloc(len + strlen(temp) + 1);
@@ -19,7 +20,7 @@ static char *prefix_path_gently(const char *prefix, int len, const char *path)
 			memcpy(sanitized, prefix, len);
 		strcpy(sanitized + len, path);
 	}
-	if (normalize_path_copy(sanitized, sanitized))
+	if (normalize_path_copy_len(sanitized, sanitized, p_len))
 		goto error_out;
 	if (is_absolute_path(orig)) {
 		size_t root_len, len, total;
@@ -44,7 +45,7 @@ static char *prefix_path_gently(const char *prefix, int len, const char *path)
 
 char *prefix_path(const char *prefix, int len, const char *path)
 {
-	char *r = prefix_path_gently(prefix, len, path);
+	char *r = prefix_path_gently(prefix, &len, path);
 	if (!r)
 		die("'%s' is outside repository", path);
 	return r;
@@ -53,7 +54,7 @@ char *prefix_path(const char *prefix, int len, const char *path)
 int path_inside_repo(const char *prefix, const char *path)
 {
 	int len = prefix ? strlen(prefix) : 0;
-	char *r = prefix_path_gently(prefix, len, path);
+	char *r = prefix_path_gently(prefix, &len, path);
 	if (r) {
 		free(r);
 		return 1;
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 40/45] parse_pathspec: preserve prefix length via PATHSPEC_PREFIX_ORIGIN

[Nguyễn Thái Ngọc Duy]

The prefix length is passed from one command to another via the new
magic 'prefix'. The magic is for parse_pathspec's internal use only,
not visible to parse_pathspec's callers.

Prefix length is not preserved across commands when --literal-pathspecs
is specified (no magic is allowed, including 'prefix'). That's OK
because we all paths are literal. No magic, no special treatment
regarding prefix. (This may be no longer true if we make :(glob)
default)

Other options to preserve the prefix include saving it to env variable
or quoting. Env var way (at least _one_ env var) is not suitable
because the prefix is not the same for all pathspecs. Pathspecs
starting with "../" will eat into the prefix part.

We could also preserve 'prefix' across commands is quote the prefix
part, then dequote on receiving. But it may not be 100% accurate, we
may dequote longer than the original prefix part, for example. That
may be good or not, but it's not the purpose.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 pathspec.c | 41 ++++++++++++++++++++++++++++++++++++-----
 1 file changed, 36 insertions(+), 5 deletions(-)

diff --git a/pathspec.c b/pathspec.c
index bc27692..3291b58 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -92,9 +92,9 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 				const char *elt)
 {
 	unsigned magic = 0, short_magic = 0;
-	const char *copyfrom = elt;
+	const char *copyfrom = elt, *long_magic_end = NULL;
 	char *match;
-	int i;
+	int i, pathspec_prefix = -1;
 
 	if (elt[0] != ':') {
 		; /* nothing to do */
@@ -111,18 +111,29 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 				nextat = copyfrom + len + 1;
 			if (!len)
 				continue;
-			for (i = 0; i < ARRAY_SIZE(pathspec_magic); i++)
+			for (i = 0; i < ARRAY_SIZE(pathspec_magic); i++) {
 				if (strlen(pathspec_magic[i].name) == len &&
 				    !strncmp(pathspec_magic[i].name, copyfrom, len)) {
 					magic |= pathspec_magic[i].bit;
 					break;
 				}
+				if (!prefixcmp(copyfrom, "prefix:")) {
+					char *endptr;
+					pathspec_prefix = strtol(copyfrom + 7,
+								 &endptr, 10);
+					if (endptr - copyfrom != len)
+						die("invalid parameter for pathspec magic 'prefix'");
+					/* "i" would be wrong, but it does not matter */
+					break;
+				}
+			}
 			if (ARRAY_SIZE(pathspec_magic) <= i)
 				die(_("Invalid pathspec magic '%.*s' in '%s'"),
 				    (int) len, copyfrom, elt);
 		}
 		if (*copyfrom != ')')
 			die(_("Missing ')' at the end of pathspec magic in '%s'"), elt);
+		long_magic_end = copyfrom;
 		copyfrom++;
 	} else {
 		/* shorthand */
@@ -149,7 +160,14 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 	magic |= short_magic;
 	*p_short_magic = short_magic;
 
-	if (magic & PATHSPEC_FROMTOP) {
+	if (pathspec_prefix >= 0 &&
+	    (prefixlen || (prefix && *prefix)))
+		die("BUG: 'prefix' magic is supposed to be used at worktree's root");
+
+	if (pathspec_prefix >= 0) {
+		match = xstrdup(copyfrom);
+		prefixlen = pathspec_prefix;
+	} else if (magic & PATHSPEC_FROMTOP) {
 		match = xstrdup(copyfrom);
 		prefixlen = 0;
 	} else {
@@ -164,7 +182,20 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 	 */
 	if (flags & PATHSPEC_PREFIX_ORIGIN) {
 		struct strbuf sb = STRBUF_INIT;
-		strbuf_add(&sb, elt, copyfrom - elt);
+		const char *start = elt;
+		if (prefixlen && !limit_pathspec_to_literal()) {
+			/* Preserve the actual prefix length of each pattern */
+			if (long_magic_end) {
+				strbuf_add(&sb, start, long_magic_end - start);
+				strbuf_addf(&sb, ",prefix:%d", prefixlen);
+				start = long_magic_end;
+			} else {
+				if (*start == ':')
+					start++;
+				strbuf_addf(&sb, ":(prefix:%d)", prefixlen);
+			}
+		}
+		strbuf_add(&sb, start, copyfrom - start);
 		strbuf_addstr(&sb, match);
 		item->original = strbuf_detach(&sb, NULL);
 	} else
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 41/45] Kill limit_pathspec_to_literal() as it's only used by parse_pathspec()

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 dir.c      | 8 --------
 pathspec.c | 8 ++++++--
 pathspec.h | 2 --
 3 files changed, 6 insertions(+), 12 deletions(-)

diff --git a/dir.c b/dir.c
index 9627d7e..4f5a2c9 100644
--- a/dir.c
+++ b/dir.c
@@ -1523,14 +1523,6 @@ int remove_path(const char *name)
 	return 0;
 }
 
-int limit_pathspec_to_literal(void)
-{
-	static int flag = -1;
-	if (flag < 0)
-		flag = git_env_bool(GIT_LITERAL_PATHSPECS_ENVIRONMENT, 0);
-	return flag;
-}
-
 /*
  * Frees memory within dir which was allocated for exclude lists and
  * the exclude_stack.  Does not free dir itself.
diff --git a/pathspec.c b/pathspec.c
index 3291b58..7b3dfc0 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -91,11 +91,15 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 				const char *prefix, int prefixlen,
 				const char *elt)
 {
+	static int literal_global = -1;
 	unsigned magic = 0, short_magic = 0;
 	const char *copyfrom = elt, *long_magic_end = NULL;
 	char *match;
 	int i, pathspec_prefix = -1;
 
+	if (literal_global < 0)
+		literal_global = git_env_bool(GIT_LITERAL_PATHSPECS_ENVIRONMENT, 0);
+
 	if (elt[0] != ':') {
 		; /* nothing to do */
 	} else if (elt[1] == '(') {
@@ -183,7 +187,7 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 	if (flags & PATHSPEC_PREFIX_ORIGIN) {
 		struct strbuf sb = STRBUF_INIT;
 		const char *start = elt;
-		if (prefixlen && !limit_pathspec_to_literal()) {
+		if (prefixlen && !literal_global) {
 			/* Preserve the actual prefix length of each pattern */
 			if (long_magic_end) {
 				strbuf_add(&sb, start, long_magic_end - start);
@@ -231,7 +235,7 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 				     elt, ce_len, ce->name);
 		}
 
-	if (limit_pathspec_to_literal())
+	if (literal_global)
 		item->nowildcard_len = item->len;
 	else {
 		item->nowildcard_len = simple_length(item->match);
diff --git a/pathspec.h b/pathspec.h
index 9f9ecd2..8f339a5 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -52,8 +52,6 @@ extern void parse_pathspec(struct pathspec *pathspec,
 extern void copy_pathspec(struct pathspec *dst, const struct pathspec *src);
 extern void free_pathspec(struct pathspec *);
 
-extern int limit_pathspec_to_literal(void);
-
 extern char *find_pathspecs_matching_against_index(const struct pathspec *pathspec);
 extern void add_pathspec_matches_against_index(const struct pathspec *pathspec, char *seen);
 extern const char *check_path_for_gitlink(const char *path);
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 42/45] pathspec: support :(literal) syntax for noglob pathspec

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/glossary-content.txt |  4 ++++
 builtin/add.c                      |  2 +-
 builtin/diff.c                     |  2 +-
 dir.c                              | 15 ++++++++++++---
 pathspec.c                         | 11 ++++++++---
 pathspec.h                         |  4 +++-
 t/t6130-pathspec-noglob.sh         | 18 ++++++++++++++++++
 tree-diff.c                        |  2 +-
 tree-walk.c                        |  5 ++++-
 9 files changed, 52 insertions(+), 11 deletions(-)

diff --git a/Documentation/glossary-content.txt b/Documentation/glossary-content.txt
index eb7ba84..8b91aeb 100644
--- a/Documentation/glossary-content.txt
+++ b/Documentation/glossary-content.txt
@@ -323,6 +323,10 @@ top `/`;;
 	The magic word `top` (mnemonic: `/`) makes the pattern match
 	from the root of the working tree, even when you are running
 	the command from inside a subdirectory.
+
+literal;;
+	Wildcards in the pattern such as `*` or `?` are treated
+	as literal characters.
 --
 +
 Currently only the slash `/` is recognized as the "magic signature",
diff --git a/builtin/add.c b/builtin/add.c
index aa0709e..8bc6b45 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -426,7 +426,7 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 		/*
 		 * file_exists() assumes exact match
 		 */
-		GUARD_PATHSPEC(&pathspec, PATHSPEC_FROMTOP);
+		GUARD_PATHSPEC(&pathspec, PATHSPEC_FROMTOP | PATHSPEC_LITERAL);
 
 		for (i = 0; i < pathspec.nr; i++) {
 			const char *path = pathspec.items[i].match;
diff --git a/builtin/diff.c b/builtin/diff.c
index 6b4e3f9..b78435f 100644
--- a/builtin/diff.c
+++ b/builtin/diff.c
@@ -372,7 +372,7 @@ int cmd_diff(int argc, const char **argv, const char *prefix)
 	}
 	if (rev.prune_data.nr) {
 		/* builtin_diff_b_f() */
-		GUARD_PATHSPEC(&rev.prune_data, PATHSPEC_FROMTOP);
+		GUARD_PATHSPEC(&rev.prune_data, PATHSPEC_FROMTOP | PATHSPEC_LITERAL);
 		if (!path)
 			path = rev.prune_data.items[0].match;
 		paths += rev.prune_data.nr;
diff --git a/dir.c b/dir.c
index 4f5a2c9..8d9ed24 100644
--- a/dir.c
+++ b/dir.c
@@ -65,7 +65,10 @@ static size_t common_prefix_len(const struct pathspec *pathspec)
 	int n;
 	size_t max = 0;
 
-	GUARD_PATHSPEC(pathspec, PATHSPEC_FROMTOP | PATHSPEC_MAXDEPTH);
+	GUARD_PATHSPEC(pathspec,
+		       PATHSPEC_FROMTOP |
+		       PATHSPEC_MAXDEPTH |
+		       PATHSPEC_LITERAL);
 
 	for (n = 0; n < pathspec->nr; n++) {
 		size_t i = 0, len = 0;
@@ -189,7 +192,10 @@ int match_pathspec_depth(const struct pathspec *ps,
 {
 	int i, retval = 0;
 
-	GUARD_PATHSPEC(ps, PATHSPEC_FROMTOP | PATHSPEC_MAXDEPTH);
+	GUARD_PATHSPEC(ps,
+		       PATHSPEC_FROMTOP |
+		       PATHSPEC_MAXDEPTH |
+		       PATHSPEC_LITERAL);
 
 	if (!ps->nr) {
 		if (!ps->recursive ||
@@ -1338,7 +1344,10 @@ int read_directory(struct dir_struct *dir, const char *path, int len, const stru
 	 * Check out create_simplify()
 	 */
 	if (pathspec)
-		GUARD_PATHSPEC(pathspec, PATHSPEC_FROMTOP | PATHSPEC_MAXDEPTH);
+		GUARD_PATHSPEC(pathspec,
+			       PATHSPEC_FROMTOP |
+			       PATHSPEC_MAXDEPTH |
+			       PATHSPEC_LITERAL);
 
 	if (has_symlink_leading_path(path, len))
 		return dir->nr;
diff --git a/pathspec.c b/pathspec.c
index 7b3dfc0..e57196d 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -70,6 +70,7 @@ static struct pathspec_magic {
 	const char *name;
 } pathspec_magic[] = {
 	{ PATHSPEC_FROMTOP, '/', "top" },
+	{ PATHSPEC_LITERAL,   0, "literal" },
 };
 
 /*
@@ -92,13 +93,15 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 				const char *elt)
 {
 	static int literal_global = -1;
-	unsigned magic = 0, short_magic = 0;
+	unsigned magic = 0, short_magic = 0, global_magic = 0;
 	const char *copyfrom = elt, *long_magic_end = NULL;
 	char *match;
 	int i, pathspec_prefix = -1;
 
 	if (literal_global < 0)
 		literal_global = git_env_bool(GIT_LITERAL_PATHSPECS_ENVIRONMENT, 0);
+	if (literal_global)
+		global_magic |= PATHSPEC_LITERAL;
 
 	if (elt[0] != ':') {
 		; /* nothing to do */
@@ -163,6 +166,7 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 
 	magic |= short_magic;
 	*p_short_magic = short_magic;
+	magic |= global_magic;
 
 	if (pathspec_prefix >= 0 &&
 	    (prefixlen || (prefix && *prefix)))
@@ -235,7 +239,7 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 				     elt, ce_len, ce->name);
 		}
 
-	if (literal_global)
+	if (magic & PATHSPEC_LITERAL)
 		item->nowildcard_len = item->len;
 	else {
 		item->nowildcard_len = simple_length(item->match);
@@ -395,7 +399,8 @@ const char **get_pathspec(const char *prefix, const char **pathspec)
 {
 	struct pathspec ps;
 	parse_pathspec(&ps,
-		       PATHSPEC_ALL_MAGIC & ~PATHSPEC_FROMTOP,
+		       PATHSPEC_ALL_MAGIC &
+		       ~(PATHSPEC_FROMTOP | PATHSPEC_LITERAL),
 		       PATHSPEC_PREFER_CWD,
 		       prefix, pathspec);
 	return ps.raw;
diff --git a/pathspec.h b/pathspec.h
index 8f339a5..d5db7d5 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -4,9 +4,11 @@
 /* Pathspec magic */
 #define PATHSPEC_FROMTOP	(1<<0)
 #define PATHSPEC_MAXDEPTH	(1<<1)
+#define PATHSPEC_LITERAL	(1<<2)
 #define PATHSPEC_ALL_MAGIC	  \
 	(PATHSPEC_FROMTOP	| \
-	 PATHSPEC_MAXDEPTH)
+	 PATHSPEC_MAXDEPTH	| \
+	 PATHSPEC_LITERAL)
 
 #define PATHSPEC_ONESTAR 1	/* the pathspec pattern sastisfies GFNM_ONESTAR */
 
diff --git a/t/t6130-pathspec-noglob.sh b/t/t6130-pathspec-noglob.sh
index 39ef619..49c148e 100755
--- a/t/t6130-pathspec-noglob.sh
+++ b/t/t6130-pathspec-noglob.sh
@@ -47,18 +47,36 @@ test_expect_success 'no-glob option matches literally (vanilla)' '
 	test_cmp expect actual
 '
 
+test_expect_success 'no-glob option matches literally (vanilla)' '
+	echo vanilla >expect &&
+	git log --format=%s -- ":(literal)foo" >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'no-glob option matches literally (star)' '
 	echo star >expect &&
 	git --literal-pathspecs log --format=%s -- "f*" >actual &&
 	test_cmp expect actual
 '
 
+test_expect_success 'no-glob option matches literally (star)' '
+	echo star >expect &&
+	git log --format=%s -- ":(literal)f*" >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'no-glob option matches literally (bracket)' '
 	echo bracket >expect &&
 	git --literal-pathspecs log --format=%s -- "f[o][o]" >actual &&
 	test_cmp expect actual
 '
 
+test_expect_success 'no-glob option matches literally (bracket)' '
+	echo bracket >expect &&
+	git log --format=%s -- ":(literal)f[o][o]" >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'no-glob environment variable works' '
 	echo star >expect &&
 	GIT_LITERAL_PATHSPECS=1 git log --format=%s -- "f*" >actual &&
diff --git a/tree-diff.c b/tree-diff.c
index 21a50d8..ccf9d7c 100644
--- a/tree-diff.c
+++ b/tree-diff.c
@@ -202,7 +202,7 @@ static void try_to_follow_renames(struct tree_desc *t1, struct tree_desc *t2, co
 	 * path. Magic that matches more than one path is not
 	 * supported.
 	 */
-	GUARD_PATHSPEC(&opt->pathspec, PATHSPEC_FROMTOP);
+	GUARD_PATHSPEC(&opt->pathspec, PATHSPEC_FROMTOP | PATHSPEC_LITERAL);
 #if 0
 	/*
 	 * We should reject wildcards as well. Unfortunately we
diff --git a/tree-walk.c b/tree-walk.c
index 37b157e..676bd7f 100644
--- a/tree-walk.c
+++ b/tree-walk.c
@@ -636,7 +636,10 @@ enum interesting tree_entry_interesting(const struct name_entry *entry,
 	enum interesting never_interesting = ps->has_wildcard ?
 		entry_not_interesting : all_entries_not_interesting;
 
-	GUARD_PATHSPEC(ps, PATHSPEC_FROMTOP | PATHSPEC_MAXDEPTH);
+	GUARD_PATHSPEC(ps,
+		       PATHSPEC_FROMTOP |
+		       PATHSPEC_MAXDEPTH |
+		       PATHSPEC_LITERAL);
 
 	if (!ps->nr) {
 		if (!ps->recursive ||
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 43/45] pathspec: make --literal-pathspecs disable pathspec magic

[Nguyễn Thái Ngọc Duy]

--literal-pathspecs and its equivalent environment variable are
probably used for scripting. In that setting, pathspec magic may be
unwanted. Disabling globbing in individual pathspec can be done via
:(literal) magic.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git.txt | 4 ++--
 pathspec.c            | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/Documentation/git.txt b/Documentation/git.txt
index 7efaa59..3bbbbdc 100644
--- a/Documentation/git.txt
+++ b/Documentation/git.txt
@@ -439,8 +439,8 @@ help ...`.
 	linkgit:git-replace[1] for more information.
 
 --literal-pathspecs::
-	Treat pathspecs literally, rather than as glob patterns. This is
-	equivalent to setting the `GIT_LITERAL_PATHSPECS` environment
+	Treat pathspecs literally (i.e. no globbing, no pathspec magic).
+	This is equivalent to setting the `GIT_LITERAL_PATHSPECS` environment
 	variable to `1`.
 
 
diff --git a/pathspec.c b/pathspec.c
index e57196d..61a47b8 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -103,7 +103,7 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 	if (literal_global)
 		global_magic |= PATHSPEC_LITERAL;
 
-	if (elt[0] != ':') {
+	if (elt[0] != ':' || literal_global) {
 		; /* nothing to do */
 	} else if (elt[1] == '(') {
 		/* longhand */
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 44/45] pathspec: support :(glob) syntax

[Nguyễn Thái Ngọc Duy]

:(glob)path differs from plain pathspec that it uses wildmatch with
WM_PATHNAME while the other uses fnmatch without FNM_PATHNAME. The
difference lies in how '*' (and '**') is processed.

With the introduction of :(glob) and :(literal) and their global
options --[no]glob-pathspecs, the user can:

 - make everything literal by default via --noglob-pathspecs
   --literal-pathspecs cannot be used for this purpose as it
   disables _all_ pathspec magic.

 - individually turn on globbing with :(glob)

 - make everything globbing by default via --glob-pathspecs

 - individually turn off globbing with :(literal)

The implication behind this is, there is no way to gain the default
matching behavior (i.e. fnmatch without FNM_PATHNAME). You either get
new globbing or literal. The old fnmatch behavior is considered
deprecated and discouraged to use.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git.txt              | 19 +++++++++++++++++
 Documentation/glossary-content.txt | 29 +++++++++++++++++++++++++
 builtin/add.c                      |  9 ++++++--
 builtin/ls-tree.c                  |  2 +-
 cache.h                            |  2 ++
 dir.c                              | 28 ++++++++++++++-----------
 dir.h                              |  9 ++++----
 git.c                              |  8 +++++++
 pathspec.c                         | 43 +++++++++++++++++++++++++++++++++-----
 pathspec.h                         |  4 +++-
 tree-walk.c                        |  9 ++++----
 11 files changed, 131 insertions(+), 31 deletions(-)

diff --git a/Documentation/git.txt b/Documentation/git.txt
index 3bbbbdc..d96df8d 100644
--- a/Documentation/git.txt
+++ b/Documentation/git.txt
@@ -443,6 +443,17 @@ help ...`.
 	This is equivalent to setting the `GIT_LITERAL_PATHSPECS` environment
 	variable to `1`.
 
+--glob-pathspecs:
+	Add "glob" magic to all pathspec. This is equivalent to setting
+	the `GIT_GLOB_PATHSPECS` environment variable to `1`. Disabling
+	globbing on individual pathspecs can be done using pathspec
+	magic ":(literal)"
+
+--noglob-pathspecs:
+	Add "literal" magic to all pathspec. This is equivalent to setting
+	the `GIT_NOGLOB_PATHSPECS` environment variable to `1`. Enabling
+	globbing on individual pathspecs can be done using pathspec
+	magic ":(glob)"
 
 GIT COMMANDS
 ------------
@@ -834,6 +845,14 @@ GIT_LITERAL_PATHSPECS::
 	literal paths to Git (e.g., paths previously given to you by
 	`git ls-tree`, `--raw` diff output, etc).
 
+GIT_GLOB_PATHSPECS::
+	Setting this variable to `1` will cause git to treat all
+	pathspecs as glob patterns (aka "glob" magic).
+
+GIT_NOGLOB_PATHSPECS::
+	Setting this variable to `1` will cause git to treat all
+	pathspecs as literal (aka "literal" magic).
+
 
 Discussion[[Discussion]]
 ------------------------
diff --git a/Documentation/glossary-content.txt b/Documentation/glossary-content.txt
index 8b91aeb..40f2d29 100644
--- a/Documentation/glossary-content.txt
+++ b/Documentation/glossary-content.txt
@@ -327,6 +327,35 @@ top `/`;;
 literal;;
 	Wildcards in the pattern such as `*` or `?` are treated
 	as literal characters.
+
+glob;;
+	Git treats the pattern as a shell glob suitable for
+	consumption by fnmatch(3) with the FNM_PATHNAME flag:
+	wildcards in the pattern will not match a / in the pathname.
+	For example, "Documentation/{asterisk}.html" matches
+	"Documentation/git.html" but not "Documentation/ppc/ppc.html"
+	or "tools/perf/Documentation/perf.html".
++
+Two consecutive asterisks ("`**`") in patterns matched against
+full pathname may have special meaning:
+
+ - A leading "`**`" followed by a slash means match in all
+   directories. For example, "`**/foo`" matches file or directory
+   "`foo`" anywhere, the same as pattern "`foo`". "**/foo/bar"
+   matches file or directory "`bar`" anywhere that is directly
+   under directory "`foo`".
+
+ - A trailing "/**" matches everything inside. For example,
+   "abc/**" matches all files inside directory "abc", relative
+   to the location of the `.gitignore` file, with infinite depth.
+
+ - A slash followed by two consecutive asterisks then a slash
+   matches zero or more directories. For example, "`a/**/b`"
+   matches "`a/b`", "`a/x/b`", "`a/x/y/b`" and so on.
+
+ - Other consecutive asterisks are considered invalid.
++
+Glob magic is incompatible with literal magic.
 --
 +
 Currently only the slash `/` is recognized as the "magic signature",
diff --git a/builtin/add.c b/builtin/add.c
index 8bc6b45..d07cd1b 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -426,11 +426,16 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 		/*
 		 * file_exists() assumes exact match
 		 */
-		GUARD_PATHSPEC(&pathspec, PATHSPEC_FROMTOP | PATHSPEC_LITERAL);
+		GUARD_PATHSPEC(&pathspec,
+			       PATHSPEC_FROMTOP |
+			       PATHSPEC_LITERAL |
+			       PATHSPEC_GLOB);
 
 		for (i = 0; i < pathspec.nr; i++) {
 			const char *path = pathspec.items[i].match;
-			if (!seen[i] && !file_exists(path)) {
+			if (!seen[i] &&
+			    ((pathspec.items[i].magic & PATHSPEC_GLOB) ||
+			     !file_exists(path))) {
 				if (ignore_missing) {
 					int dtype = DT_UNKNOWN;
 					if (is_path_excluded(&check, path, -1, &dtype))
diff --git a/builtin/ls-tree.c b/builtin/ls-tree.c
index bdb03f3..1056634 100644
--- a/builtin/ls-tree.c
+++ b/builtin/ls-tree.c
@@ -173,7 +173,7 @@ int cmd_ls_tree(int argc, const char **argv, const char *prefix)
 	 * cannot be lifted until it is converted to use
 	 * match_pathspec_depth() or tree_entry_interesting()
 	 */
-	parse_pathspec(&pathspec, 0,
+	parse_pathspec(&pathspec, PATHSPEC_GLOB,
 		       PATHSPEC_PREFER_CWD,
 		       prefix, argv + 1);
 	for (i = 0; i < pathspec.nr; i++)
diff --git a/cache.h b/cache.h
index 17e0b26..bdc98e2 100644
--- a/cache.h
+++ b/cache.h
@@ -365,6 +365,8 @@ static inline enum object_type object_type(unsigned int mode)
 #define GIT_NOTES_REWRITE_REF_ENVIRONMENT "GIT_NOTES_REWRITE_REF"
 #define GIT_NOTES_REWRITE_MODE_ENVIRONMENT "GIT_NOTES_REWRITE_MODE"
 #define GIT_LITERAL_PATHSPECS_ENVIRONMENT "GIT_LITERAL_PATHSPECS"
+#define GIT_GLOB_PATHSPECS_ENVIRONMENT "GIT_GLOB_PATHSPECS"
+#define GIT_NOGLOB_PATHSPECS_ENVIRONMENT "GIT_NOGLOB_PATHSPECS"
 
 /*
  * Repository-local GIT_* environment variables
diff --git a/dir.c b/dir.c
index 8d9ed24..77cd427 100644
--- a/dir.c
+++ b/dir.c
@@ -38,26 +38,28 @@ int fnmatch_icase(const char *pattern, const char *string, int flags)
 	return fnmatch(pattern, string, flags | (ignore_case ? FNM_CASEFOLD : 0));
 }
 
-inline int git_fnmatch(const char *pattern, const char *string,
-		       int flags, int prefix)
+inline int git_fnmatch(const struct pathspec_item *item,
+		       const char *pattern, const char *string,
+		       int prefix)
 {
-	int fnm_flags = 0;
-	if (flags & GFNM_PATHNAME)
-		fnm_flags |= FNM_PATHNAME;
 	if (prefix > 0) {
 		if (strncmp(pattern, string, prefix))
 			return FNM_NOMATCH;
 		pattern += prefix;
 		string += prefix;
 	}
-	if (flags & GFNM_ONESTAR) {
+	if (item->flags & PATHSPEC_ONESTAR) {
 		int pattern_len = strlen(++pattern);
 		int string_len = strlen(string);
 		return string_len < pattern_len ||
 		       strcmp(pattern,
 			      string + string_len - pattern_len);
 	}
-	return fnmatch(pattern, string, fnm_flags);
+	if (item->magic & PATHSPEC_GLOB)
+		return wildmatch(pattern, string, WM_PATHNAME, NULL);
+	else
+		/* wildmatch has not learned no FNM_PATHNAME mode yet */
+		return fnmatch(pattern, string, 0);
 }
 
 static size_t common_prefix_len(const struct pathspec *pathspec)
@@ -68,7 +70,8 @@ static size_t common_prefix_len(const struct pathspec *pathspec)
 	GUARD_PATHSPEC(pathspec,
 		       PATHSPEC_FROMTOP |
 		       PATHSPEC_MAXDEPTH |
-		       PATHSPEC_LITERAL);
+		       PATHSPEC_LITERAL |
+		       PATHSPEC_GLOB);
 
 	for (n = 0; n < pathspec->nr; n++) {
 		size_t i = 0, len = 0;
@@ -163,8 +166,7 @@ static int match_pathspec_item(const struct pathspec_item *item, int prefix,
 	}
 
 	if (item->nowildcard_len < item->len &&
-	    !git_fnmatch(match, name,
-			 item->flags & PATHSPEC_ONESTAR ? GFNM_ONESTAR : 0,
+	    !git_fnmatch(item, match, name,
 			 item->nowildcard_len - prefix))
 		return MATCHED_FNMATCH;
 
@@ -195,7 +197,8 @@ int match_pathspec_depth(const struct pathspec *ps,
 	GUARD_PATHSPEC(ps,
 		       PATHSPEC_FROMTOP |
 		       PATHSPEC_MAXDEPTH |
-		       PATHSPEC_LITERAL);
+		       PATHSPEC_LITERAL |
+		       PATHSPEC_GLOB);
 
 	if (!ps->nr) {
 		if (!ps->recursive ||
@@ -1347,7 +1350,8 @@ int read_directory(struct dir_struct *dir, const char *path, int len, const stru
 		GUARD_PATHSPEC(pathspec,
 			       PATHSPEC_FROMTOP |
 			       PATHSPEC_MAXDEPTH |
-			       PATHSPEC_LITERAL);
+			       PATHSPEC_LITERAL |
+			       PATHSPEC_GLOB);
 
 	if (has_symlink_leading_path(path, len))
 		return dir->nr;
diff --git a/dir.h b/dir.h
index 13da62c..da67237 100644
--- a/dir.h
+++ b/dir.h
@@ -208,10 +208,9 @@ extern int fnmatch_icase(const char *pattern, const char *string, int flags);
 /*
  * The prefix part of pattern must not contains wildcards.
  */
-#define GFNM_PATHNAME 1		/* similar to FNM_PATHNAME */
-#define GFNM_ONESTAR  2		/* there is only _one_ wildcard, a star */
-
-extern int git_fnmatch(const char *pattern, const char *string,
-		       int flags, int prefix);
+struct pathspec_item;
+extern int git_fnmatch(const struct pathspec_item *item,
+		       const char *pattern, const char *string,
+		       int prefix);
 
 #endif
diff --git a/git.c b/git.c
index 39ba6b1..65c386a 100644
--- a/git.c
+++ b/git.c
@@ -143,6 +143,14 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			setenv(GIT_LITERAL_PATHSPECS_ENVIRONMENT, "0", 1);
 			if (envchanged)
 				*envchanged = 1;
+		} else if (!strcmp(cmd, "--glob-pathspecs")) {
+			setenv(GIT_GLOB_PATHSPECS_ENVIRONMENT, "1", 1);
+			if (envchanged)
+				*envchanged = 1;
+		} else if (!strcmp(cmd, "--noglob-pathspecs")) {
+			setenv(GIT_NOGLOB_PATHSPECS_ENVIRONMENT, "1", 1);
+			if (envchanged)
+				*envchanged = 1;
 		} else {
 			fprintf(stderr, "Unknown option: %s\n", cmd);
 			usage(git_usage_string);
diff --git a/pathspec.c b/pathspec.c
index 61a47b8..7ddcf55 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -57,7 +57,6 @@ char *find_pathspecs_matching_against_index(const struct pathspec *pathspec)
  *
  * Possible future magic semantics include stuff like:
  *
- *	{ PATHSPEC_NOGLOB, '!', "noglob" },
  *	{ PATHSPEC_ICASE, '\0', "icase" },
  *	{ PATHSPEC_RECURSIVE, '*', "recursive" },
  *	{ PATHSPEC_REGEXP, '\0', "regexp" },
@@ -71,6 +70,7 @@ static struct pathspec_magic {
 } pathspec_magic[] = {
 	{ PATHSPEC_FROMTOP, '/', "top" },
 	{ PATHSPEC_LITERAL,   0, "literal" },
+	{ PATHSPEC_GLOB,   '\0', "glob" },
 };
 
 /*
@@ -93,6 +93,8 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 				const char *elt)
 {
 	static int literal_global = -1;
+	static int glob_global = -1;
+	static int noglob_global = -1;
 	unsigned magic = 0, short_magic = 0, global_magic = 0;
 	const char *copyfrom = elt, *long_magic_end = NULL;
 	char *match;
@@ -103,6 +105,22 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 	if (literal_global)
 		global_magic |= PATHSPEC_LITERAL;
 
+	if (glob_global < 0)
+		glob_global = git_env_bool(GIT_GLOB_PATHSPECS_ENVIRONMENT, 0);
+	if (glob_global)
+		global_magic |= PATHSPEC_GLOB;
+
+	if (noglob_global < 0)
+		noglob_global = git_env_bool(GIT_NOGLOB_PATHSPECS_ENVIRONMENT, 0);
+
+	if (glob_global && noglob_global)
+		die(_("global 'glob' and 'noglob' pathspec settings are incompatible"));
+
+	if ((global_magic & PATHSPEC_LITERAL) &&
+	    (global_magic & ~PATHSPEC_LITERAL))
+		die(_("global 'literal' pathspec setting is incompatiable "
+		      "with all other global pathspec settings"));
+
 	if (elt[0] != ':' || literal_global) {
 		; /* nothing to do */
 	} else if (elt[1] == '(') {
@@ -166,12 +184,20 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 
 	magic |= short_magic;
 	*p_short_magic = short_magic;
+
+	/* --noglob-pathspec adds :(literal) _unless_ :(glob) is specifed */
+	if (noglob_global && !(magic & PATHSPEC_GLOB))
+		global_magic |= PATHSPEC_LITERAL;
+
 	magic |= global_magic;
 
 	if (pathspec_prefix >= 0 &&
 	    (prefixlen || (prefix && *prefix)))
 		die("BUG: 'prefix' magic is supposed to be used at worktree's root");
 
+	if ((magic & PATHSPEC_LITERAL) && (magic & PATHSPEC_GLOB))
+		die("%s: 'literal' and 'glob' are incompatible", elt);
+
 	if (pathspec_prefix >= 0) {
 		match = xstrdup(copyfrom);
 		prefixlen = pathspec_prefix;
@@ -247,10 +273,17 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 			item->nowildcard_len = prefixlen;
 	}
 	item->flags = 0;
-	if (item->nowildcard_len < item->len &&
-	    item->match[item->nowildcard_len] == '*' &&
-	    no_wildcard(item->match + item->nowildcard_len + 1))
-		item->flags |= PATHSPEC_ONESTAR;
+	if (magic & PATHSPEC_GLOB) {
+		/*
+		 * FIXME: should we enable ONESTAR in _GLOB for
+		 * pattern "* * / * . c"?
+		 */
+	} else {
+		if (item->nowildcard_len < item->len &&
+		    item->match[item->nowildcard_len] == '*' &&
+		    no_wildcard(item->match + item->nowildcard_len + 1))
+			item->flags |= PATHSPEC_ONESTAR;
+	}
 	return magic;
 }
 
diff --git a/pathspec.h b/pathspec.h
index d5db7d5..9a0c5d5 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -5,10 +5,12 @@
 #define PATHSPEC_FROMTOP	(1<<0)
 #define PATHSPEC_MAXDEPTH	(1<<1)
 #define PATHSPEC_LITERAL	(1<<2)
+#define PATHSPEC_GLOB		(1<<3)
 #define PATHSPEC_ALL_MAGIC	  \
 	(PATHSPEC_FROMTOP	| \
 	 PATHSPEC_MAXDEPTH	| \
-	 PATHSPEC_LITERAL)
+	 PATHSPEC_LITERAL	| \
+	 PATHSPEC_GLOB)
 
 #define PATHSPEC_ONESTAR 1	/* the pathspec pattern sastisfies GFNM_ONESTAR */
 
diff --git a/tree-walk.c b/tree-walk.c
index 676bd7f..a44f528 100644
--- a/tree-walk.c
+++ b/tree-walk.c
@@ -639,7 +639,8 @@ enum interesting tree_entry_interesting(const struct name_entry *entry,
 	GUARD_PATHSPEC(ps,
 		       PATHSPEC_FROMTOP |
 		       PATHSPEC_MAXDEPTH |
-		       PATHSPEC_LITERAL);
+		       PATHSPEC_LITERAL |
+		       PATHSPEC_GLOB);
 
 	if (!ps->nr) {
 		if (!ps->recursive ||
@@ -685,8 +686,7 @@ enum interesting tree_entry_interesting(const struct name_entry *entry,
 				return entry_interesting;
 
 			if (item->nowildcard_len < item->len) {
-				if (!git_fnmatch(match + baselen, entry->path,
-						 item->flags & PATHSPEC_ONESTAR ? GFNM_ONESTAR : 0,
+				if (!git_fnmatch(item, match + baselen, entry->path,
 						 item->nowildcard_len - baselen))
 					return entry_interesting;
 
@@ -727,8 +727,7 @@ match_wildcards:
 
 		strbuf_add(base, entry->path, pathlen);
 
-		if (!git_fnmatch(match, base->buf + base_offset,
-				 item->flags & PATHSPEC_ONESTAR ? GFNM_ONESTAR : 0,
+		if (!git_fnmatch(item, match, base->buf + base_offset,
 				 item->nowildcard_len)) {
 			strbuf_setlen(base, base_offset + baselen);
 			return entry_interesting;
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v1 45/45] Rename field "raw" to "_raw" in struct pathspec

[Nguyễn Thái Ngọc Duy]

This patch is essentially no-op. It helps catching new use of this
field though. This field is introduced as an intermediate step for the
pathspec conversion and will be removed eventually. At this stage no
more access sites should be introduced.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/ls-tree.c | 2 +-
 dir.c             | 4 ++--
 pathspec.c        | 6 +++---
 pathspec.h        | 2 +-
 4 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/builtin/ls-tree.c b/builtin/ls-tree.c
index 1056634..7882352 100644
--- a/builtin/ls-tree.c
+++ b/builtin/ls-tree.c
@@ -36,7 +36,7 @@ static int show_recursive(const char *base, int baselen, const char *pathname)
 	if (ls_options & LS_RECURSIVE)
 		return 1;
 
-	s = pathspec.raw;
+	s = pathspec._raw;
 	if (!s)
 		return 0;
 
diff --git a/dir.c b/dir.c
index 77cd427..e52c899 100644
--- a/dir.c
+++ b/dir.c
@@ -115,7 +115,7 @@ int fill_directory(struct dir_struct *dir, const struct pathspec *pathspec)
 	len = common_prefix_len(pathspec);
 
 	/* Read the directory and prune it */
-	read_directory(dir, pathspec->nr ? pathspec->raw[0] : "", len, pathspec);
+	read_directory(dir, pathspec->nr ? pathspec->_raw[0] : "", len, pathspec);
 	return len;
 }
 
@@ -1356,7 +1356,7 @@ int read_directory(struct dir_struct *dir, const char *path, int len, const stru
 	if (has_symlink_leading_path(path, len))
 		return dir->nr;
 
-	simplify = create_simplify(pathspec ? pathspec->raw : NULL);
+	simplify = create_simplify(pathspec ? pathspec->_raw : NULL);
 	if (!len || treat_leading_path(dir, path, len, simplify))
 		read_directory_recursive(dir, path, len, 0, simplify);
 	free_simplify(simplify);
diff --git a/pathspec.c b/pathspec.c
index 7ddcf55..9640d88 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -365,7 +365,7 @@ void parse_pathspec(struct pathspec *pathspec,
 		raw[0] = prefix;
 		raw[1] = NULL;
 		pathspec->nr = 1;
-		pathspec->raw = raw;
+		pathspec->_raw = raw;
 		return;
 	}
 
@@ -375,7 +375,7 @@ void parse_pathspec(struct pathspec *pathspec,
 
 	pathspec->nr = n;
 	pathspec->items = item = xmalloc(sizeof(*item) * n);
-	pathspec->raw = argv;
+	pathspec->_raw = argv;
 	prefixlen = prefix ? strlen(prefix) : 0;
 
 	for (i = 0; i < n; i++) {
@@ -436,7 +436,7 @@ const char **get_pathspec(const char *prefix, const char **pathspec)
 		       ~(PATHSPEC_FROMTOP | PATHSPEC_LITERAL),
 		       PATHSPEC_PREFER_CWD,
 		       prefix, pathspec);
-	return ps.raw;
+	return ps._raw;
 }
 
 void copy_pathspec(struct pathspec *dst, const struct pathspec *src)
diff --git a/pathspec.h b/pathspec.h
index 9a0c5d5..84f279d 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -15,7 +15,7 @@
 #define PATHSPEC_ONESTAR 1	/* the pathspec pattern sastisfies GFNM_ONESTAR */
 
 struct pathspec {
-	const char **raw; /* get_pathspec() result, not freed by free_pathspec() */
+	const char **_raw; /* get_pathspec() result, not freed by free_pathspec() */
 	int nr;
 	unsigned int has_wildcard:1;
 	unsigned int recursive:1;
-- 
1.8.0.rc0.19.g7bbb31d

Re: [PATCH v1 00/45] nd/parse-pathspec and :(glob) pathspec magic

[Junio C Hamano]

Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:

> Probably not much to say. A big portion of this series is the
> conversion to struct pathspec, which enables more use of pathspec
> magic. :(glob) magic is added to verify that the conversion makes
> sense.

I haven't read any of these patches, but there remains only one user
of the _raw field in dir.c (read_directory) and get_pathspec() is
called by nobody other than builtin/mv.c after this series.

Very encouraging.

> Andrew Wong (1):
>   setup.c: check that the pathspec magic ends with ")"
>
> Nguyễn Thái Ngọc Duy (44):
>   clean: remove unused variable "seen"
>   Move struct pathspec and related functions to pathspec.[ch]
>   pathspec: i18n-ize error strings in pathspec parsing code
>   pathspec: add copy_pathspec
>   Add parse_pathspec() that converts cmdline args to struct pathspec
>   parse_pathspec: save original pathspec for reporting
>   parse_pathspec: add PATHSPEC_PREFER_{CWD,FULL}
>   Convert some get_pathspec() calls to parse_pathspec()
>   parse_pathspec: a special flag for max_depth feature
>   parse_pathspec: support stripping submodule trailing slashes
>   parse_pathspec: support stripping/checking submodule paths
>   parse_pathspec: support prefixing original patterns
>   Guard against new pathspec magic in pathspec matching code
>   clean: convert to use parse_pathspec
>   commit: convert to use parse_pathspec
>   status: convert to use parse_pathspec
>   rerere: convert to use parse_pathspec
>   checkout: convert to use parse_pathspec
>   rm: convert to use parse_pathspec
>   ls-files: convert to use parse_pathspec
>   archive: convert to use parse_pathspec
>   check-ignore: convert to use parse_pathspec
>   add: convert to use parse_pathspec
>   reset: convert to use parse_pathspec
>   Convert read_cache_preload() to take struct pathspec
>   Convert run_add_interactive to use struct pathspec
>   Convert unmerge_cache to take struct pathspec
>   checkout: convert read_tree_some to take struct pathspec
>   Convert report_path_error to take struct pathspec
>   Convert refresh_index to take struct pathspec
>   Convert {read,fill}_directory to take struct pathspec
>   Convert add_files_to_cache to take struct pathspec
>   Convert common_prefix() to use struct pathspec
>   Remove diff_tree_{setup,release}_paths
>   Remove init_pathspec() in favor of parse_pathspec()
>   Remove match_pathspec() in favor of match_pathspec_depth()
>   tree-diff: remove the use of pathspec's raw[] in follow-rename codepath
>   parse_pathspec: make sure the prefix part is wildcard-free
>   parse_pathspec: preserve prefix length via PATHSPEC_PREFIX_ORIGIN
>   Kill limit_pathspec_to_literal() as it's only used by parse_pathspec()
>   pathspec: support :(literal) syntax for noglob pathspec
>   pathspec: make --literal-pathspecs disable pathspec magic
>   pathspec: support :(glob) syntax
>   Rename field "raw" to "_raw" in struct pathspec
>
>  Documentation/git.txt              |  23 +-
>  Documentation/glossary-content.txt |  33 +++
>  archive.c                          |  18 +-
>  archive.h                          |   4 +-
>  builtin/add.c                      | 156 ++++++--------
>  builtin/blame.c                    |  14 +-
>  builtin/check-ignore.c             |  34 +--
>  builtin/checkout.c                 |  46 ++--
>  builtin/clean.c                    |  24 +--
>  builtin/commit.c                   |  37 ++--
>  builtin/diff-files.c               |   2 +-
>  builtin/diff-index.c               |   2 +-
>  builtin/diff.c                     |   6 +-
>  builtin/grep.c                     |  10 +-
>  builtin/log.c                      |   2 +-
>  builtin/ls-files.c                 |  75 +++----
>  builtin/ls-tree.c                  |  13 +-
>  builtin/mv.c                       |  13 +-
>  builtin/rerere.c                   |   8 +-
>  builtin/reset.c                    |  33 +--
>  builtin/rm.c                       |  24 +--
>  builtin/update-index.c             |   6 +-
>  cache.h                            |  34 +--
>  commit.h                           |   2 +-
>  diff-lib.c                         |   3 +-
>  diff.h                             |   3 +-
>  dir.c                              | 261 +++++-----------------
>  dir.h                              |  18 +-
>  git.c                              |   8 +
>  merge-recursive.c                  |   2 +-
>  notes-merge.c                      |   4 +-
>  path.c                             |  15 +-
>  pathspec.c                         | 431 +++++++++++++++++++++++++++++++++----
>  pathspec.h                         |  59 ++++-
>  preload-index.c                    |  21 +-
>  read-cache.c                       |   5 +-
>  rerere.c                           |   7 +-
>  rerere.h                           |   4 +-
>  resolve-undo.c                     |   4 +-
>  resolve-undo.h                     |   2 +-
>  revision.c                         |  11 +-
>  setup.c                            | 157 +-------------
>  t/t0008-ignores.sh                 |   8 +-
>  t/t6130-pathspec-noglob.sh         |  18 ++
>  tree-diff.c                        |  48 +++--
>  tree-walk.c                        |  21 +-
>  tree.c                             |   4 +-
>  tree.h                             |   2 +-
>  wt-status.c                        |  18 +-
>  wt-status.h                        |   2 +-
>  50 files changed, 983 insertions(+), 772 deletions(-)

Re: [PATCH v1 22/45] archive: convert to use parse_pathspec

[Junio C Hamano]

Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:

> @@ -232,11 +228,18 @@ static int path_exists(struct tree *tree, const char *path)
>  static void parse_pathspec_arg(const char **pathspec,
>  		struct archiver_args *ar_args)
>  {
> -	ar_args->pathspec = pathspec = get_pathspec("", pathspec);
> +	/*
> +	 * must be consistent with parse_pathspec in path_exists()
> +	 * Also if pathspec patterns are dependent, we're in big
> +	 * trouble as we test each one separately
> +	 */
> +	parse_pathspec(&ar_args->pathspec, 0,
> +		       PATHSPEC_PREFER_FULL,
> +		       "", pathspec);
>  	if (pathspec) {
>  		while (*pathspec) {
>  			if (!path_exists(ar_args->tree, *pathspec))
> -				die("path not found: %s", *pathspec);
> +				die(_("pathspec '%s' did not match any files"), *pathspec);
>  			pathspec++;
>  		}

You do not use ar_args->pathspec even though you used parse_pathspec()
to grok it?  What's the point of this change?

>  	}
> diff --git a/archive.h b/archive.h
> index 895afcd..4a791e1 100644
> --- a/archive.h
> +++ b/archive.h
> @@ -1,6 +1,8 @@
>  #ifndef ARCHIVE_H
>  #define ARCHIVE_H
>  
> +#include "pathspec.h"
> +
>  struct archiver_args {
>  	const char *base;
>  	size_t baselen;
> @@ -8,7 +10,7 @@ struct archiver_args {
>  	const unsigned char *commit_sha1;
>  	const struct commit *commit;
>  	time_t time;
> -	const char **pathspec;
> +	struct pathspec pathspec;
>  	unsigned int verbose : 1;
>  	unsigned int worktree_attributes : 1;
>  	unsigned int convert : 1;

Re: [PATCH v1 10/45] parse_pathspec: a special flag for max_depth feature

[Eric Sunshine]

On Fri, Mar 15, 2013 at 2:06 AM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:
> match_pathspec_depth() and tree_entry_interesting() check max_depth
> field in order to support "git grep --max-depth". The feature
> activation is tied to "recursive" field, which led to some unwated

s/unwated/unwanted/

> activation, e.g. 5c8eeb8 (diff-index: enable recursive pathspec
> matching in unpack_trees - 2012-01-15).

Re: [PATCH v1 40/45] parse_pathspec: preserve prefix length via PATHSPEC_PREFIX_ORIGIN

[Eric Sunshine]

On Fri, Mar 15, 2013 at 2:06 AM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:
> Prefix length is not preserved across commands when --literal-pathspecs
> is specified (no magic is allowed, including 'prefix'). That's OK
> because we all paths are literal. No magic, no special treatment

s/we all/we know all/
...or...
s/we all/all/

> We could also preserve 'prefix' across commands is quote the prefix
> part, then dequote on receiving. But it may not be 100% accurate, we

s/is quote/by quoting/
s/dequote/dequoting/

> may dequote longer than the original prefix part, for example. That
> may be good or not, but it's not the purpose.

Re: [PATCH v1 44/45] pathspec: support :(glob) syntax

[Eric Sunshine]

On Fri, Mar 15, 2013 at 2:06 AM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:
> +GIT_GLOB_PATHSPECS::
> +       Setting this variable to `1` will cause git to treat all
> +       pathspecs as glob patterns (aka "glob" magic).

Per recent git -> Git normalization, probably: s/git/Git/

> +
> +GIT_NOGLOB_PATHSPECS::
> +       Setting this variable to `1` will cause git to treat all
> +       pathspecs as literal (aka "literal" magic).

Ditto: s/git/Git/

> @@ -103,6 +105,22 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
>         if (literal_global)
>                 global_magic |= PATHSPEC_LITERAL;
>
> +       if (glob_global < 0)
> +               glob_global = git_env_bool(GIT_GLOB_PATHSPECS_ENVIRONMENT, 0);
> +       if (glob_global)
> +               global_magic |= PATHSPEC_GLOB;
> +
> +       if (noglob_global < 0)
> +               noglob_global = git_env_bool(GIT_NOGLOB_PATHSPECS_ENVIRONMENT, 0);
> +
> +       if (glob_global && noglob_global)
> +               die(_("global 'glob' and 'noglob' pathspec settings are incompatible"));
> +
> +       if ((global_magic & PATHSPEC_LITERAL) &&
> +           (global_magic & ~PATHSPEC_LITERAL))
> +               die(_("global 'literal' pathspec setting is incompatiable "
> +                     "with all other global pathspec settings"));

s/incompatiable/incompatible/

-- ES

Re: [PATCH v1 22/45] archive: convert to use parse_pathspec

[Duy Nguyen]

On Sat, Mar 16, 2013 at 12:56 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
>
>> @@ -232,11 +228,18 @@ static int path_exists(struct tree *tree, const char *path)
>>  static void parse_pathspec_arg(const char **pathspec,
>>               struct archiver_args *ar_args)
>>  {
>> -     ar_args->pathspec = pathspec = get_pathspec("", pathspec);
>> +     /*
>> +      * must be consistent with parse_pathspec in path_exists()
>> +      * Also if pathspec patterns are dependent, we're in big
>> +      * trouble as we test each one separately
>> +      */
>> +     parse_pathspec(&ar_args->pathspec, 0,
>> +                    PATHSPEC_PREFER_FULL,
>> +                    "", pathspec);
>>       if (pathspec) {
>>               while (*pathspec) {
>>                       if (!path_exists(ar_args->tree, *pathspec))
>> -                             die("path not found: %s", *pathspec);
>> +                             die(_("pathspec '%s' did not match any files"), *pathspec);
>>                       pathspec++;
>>               }
>
> You do not use ar_args->pathspec even though you used parse_pathspec()
> to grok it?  What's the point of this change?

parse_pathspec() here is needed because write_archive_entries needs it
later. tree_entry_interesting() has not supported "seen" feature like
match_pathspec_depth() to detect unused pathspecs. For simplicity,
just check each pathspec individually. We can revisit this when we add
"seen" feature to tree_entry_interesting.
-- 
Duy

Re: [PATCH v1 22/45] archive: convert to use parse_pathspec

[Junio C Hamano]

Duy Nguyen <pclouds@gmail.com> writes:

> On Sat, Mar 16, 2013 at 12:56 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
>>
>>> @@ -232,11 +228,18 @@ static int path_exists(struct tree *tree, const char *path)
>>>  static void parse_pathspec_arg(const char **pathspec,
>>>               struct archiver_args *ar_args)
>>>  {
>>> -     ar_args->pathspec = pathspec = get_pathspec("", pathspec);
>>> +     /*
>>> +      * must be consistent with parse_pathspec in path_exists()
>>> +      * Also if pathspec patterns are dependent, we're in big
>>> +      * trouble as we test each one separately
>>> +      */
>>> +     parse_pathspec(&ar_args->pathspec, 0,
>>> +                    PATHSPEC_PREFER_FULL,
>>> +                    "", pathspec);
>>>       if (pathspec) {
>>>               while (*pathspec) {
>>>                       if (!path_exists(ar_args->tree, *pathspec))
>>> -                             die("path not found: %s", *pathspec);
>>> +                             die(_("pathspec '%s' did not match any files"), *pathspec);
>>>                       pathspec++;
>>>               }
>>
>> You do not use ar_args->pathspec even though you used parse_pathspec()
>> to grok it?  What's the point of this change?
>
> parse_pathspec() here is needed because write_archive_entries needs it
> later.

That is not the issue I was pointing out.  Even though you parse the
pathspec into args->pathspec, the "if() { while () {} }" here still
uses strings contained in **pathspec, as if they are literal strings
and not ":(glob)Documentation" and such, and will not match the named
directory.

Technically, erroring out saying "':(glob)Documentation' does not exist
as a path in the tree" is correct, but it would be nicer to have the
code inspect parse_pathspec() result and independently barf, saying
"this command does not support magic pathspecs, give me leading paths
and nothing else", until we do support magic pathspecs, no?

Re: [PATCH v1 22/45] archive: convert to use parse_pathspec

[Duy Nguyen]

On Sun, Mar 17, 2013 at 12:00 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Duy Nguyen <pclouds@gmail.com> writes:
>
>> On Sat, Mar 16, 2013 at 12:56 AM, Junio C Hamano <gitster@pobox.com> wrote:
>>> Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
>>>
>>>> @@ -232,11 +228,18 @@ static int path_exists(struct tree *tree, const char *path)
>>>>  static void parse_pathspec_arg(const char **pathspec,
>>>>               struct archiver_args *ar_args)
>>>>  {
>>>> -     ar_args->pathspec = pathspec = get_pathspec("", pathspec);
>>>> +     /*
>>>> +      * must be consistent with parse_pathspec in path_exists()
>>>> +      * Also if pathspec patterns are dependent, we're in big
>>>> +      * trouble as we test each one separately
>>>> +      */
>>>> +     parse_pathspec(&ar_args->pathspec, 0,
>>>> +                    PATHSPEC_PREFER_FULL,
>>>> +                    "", pathspec);
>>>>       if (pathspec) {
>>>>               while (*pathspec) {
>>>>                       if (!path_exists(ar_args->tree, *pathspec))
>>>> -                             die("path not found: %s", *pathspec);
>>>> +                             die(_("pathspec '%s' did not match any files"), *pathspec);
>>>>                       pathspec++;
>>>>               }
>>>
>>> You do not use ar_args->pathspec even though you used parse_pathspec()
>>> to grok it?  What's the point of this change?
>>
>> parse_pathspec() here is needed because write_archive_entries needs it
>> later.
>
> That is not the issue I was pointing out.  Even though you parse the
> pathspec into args->pathspec, the "if() { while () {} }" here still
> uses strings contained in **pathspec, as if they are literal strings
> and not ":(glob)Documentation" and such, and will not match the named
> directory.

No, the literal strings are reparsed in path_exists() before being fed
to read_tree_recursive. So ":(glob)Documentation" should match the
tree "Documentation".

> Technically, erroring out saying "':(glob)Documentation' does not exist
> as a path in the tree" is correct, but it would be nicer to have the
> code inspect parse_pathspec() result and independently barf, saying
> "this command does not support magic pathspecs, give me leading paths
> and nothing else", until we do support magic pathspecs, no?
-- 
Duy

Re: [PATCH v1 22/45] archive: convert to use parse_pathspec

[Junio C Hamano]

Duy Nguyen <pclouds@gmail.com> writes:

> No, the literal strings are reparsed in path_exists() before being fed
> to read_tree_recursive.

Yuck.  OK.  Thanks.

Re: [PATCH v1 11/45] parse_pathspec: support stripping submodule trailing slashes

[Junio C Hamano]

Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:

> This flag is equivalent to builtin/ls-files.c:strip_trailing_slashes()
> and is intended to replace that function when ls-files is converted to
> use parse_pathspec.
>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
>  pathspec.c | 9 +++++++++
>  pathspec.h | 1 +
>  2 files changed, 10 insertions(+)
>
> diff --git a/pathspec.c b/pathspec.c
> index b2446c3..2da8bc9 100644
> --- a/pathspec.c
> +++ b/pathspec.c
> @@ -204,6 +204,15 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
>  	*raw = item->match = match;
>  	item->original = elt;
>  	item->len = strlen(item->match);
> +
> +	if ((flags & PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP) &&

I see that having cheap and expensive variant at these steps is a
way to achieve a bug-for-bug compatible rewrite of the existing
code, but in the longer term should we lose one of them?  After all,
a path that points at the working tree of another repository that is
beyond a symlink cannot be added to us as a submodule, so CHEAP can
be used only when we know that any intermediate component on the
path is not a symlink pointing elsewhere, and the user in the
codepath of ls-files may know it.

To put it differently, shouldn't CHEAP and EXPENSIVE be accompanied
by in-code comment and updates to Documentation/technical/api-*.txt
to help users of the API to decide which one to use?

Re: [PATCH v1 14/45] Guard against new pathspec magic in pathspec matching code

[Junio C Hamano]

Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:

> GUARD_PATHSPEC() marks pathspec-sensitive code (basically anything in
> 'struct pathspec' except fields "nr" and "original"). GUARD_PATHSPEC()
> is not supposed to fail. The steps for a new pathspec magic or
> optimization would be:
>
>  - update parse_pathspec, add extra information to struct pathspec
>
>  - grep GUARD_PATHSPEC() and update all relevant code (or note those
>    that won't work with your new stuff). Update GUARD_PATHSPEC mask
>    accordingly.
>
>  - update parse_pathspec calls to disable new magic early at command
>    parsing level. Make sure parse_pathspec() catches unsupported
>    syntax, not until GUARD_PATHSPEC catches it.
>
>  - add tests to verify supported/unsupported commands both work as
>    expected.

I think Documentation/technical/api-*.txt wants to have all of the
above.  It is not clear from the description what the second bitmask
is supposed to mean, without reading the implementation; I am guessing
that you are supposed to list the kinds of magic that are supported
in the codepath?

> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
>  builtin/diff.c |  2 ++
>  dir.c          |  2 ++
>  pathspec.h     |  7 +++++++
>  tree-diff.c    | 19 +++++++++++++++++++
>  tree-walk.c    |  2 ++
>  5 files changed, 32 insertions(+)
>
> diff --git a/builtin/diff.c b/builtin/diff.c
> index 8c2af6c..d237e0a 100644
> --- a/builtin/diff.c
> +++ b/builtin/diff.c
> @@ -371,6 +371,8 @@ int cmd_diff(int argc, const char **argv, const char *prefix)
>  		die(_("unhandled object '%s' given."), name);
>  	}
>  	if (rev.prune_data.nr) {
> +		/* builtin_diff_b_f() */
> +		GUARD_PATHSPEC(&rev.prune_data, PATHSPEC_FROMTOP);
>  		if (!path)
>  			path = rev.prune_data.items[0].match;
>  		paths += rev.prune_data.nr;
> diff --git a/dir.c b/dir.c
> index 1e9db41..6094ba8 100644
> --- a/dir.c
> +++ b/dir.c
> @@ -297,6 +297,8 @@ int match_pathspec_depth(const struct pathspec *ps,
>  {
>  	int i, retval = 0;
>  
> +	GUARD_PATHSPEC(ps, PATHSPEC_FROMTOP | PATHSPEC_MAXDEPTH);
> +
>  	if (!ps->nr) {
>  		if (!ps->recursive ||
>  		    !(ps->magic & PATHSPEC_MAXDEPTH) ||
> diff --git a/pathspec.h b/pathspec.h
> index 1cef9c6..ed5d3a6 100644
> --- a/pathspec.h
> +++ b/pathspec.h
> @@ -27,6 +27,13 @@ struct pathspec {
>  	} *items;
>  };
>  
> +#define GUARD_PATHSPEC(ps, mask) \
> +	do { \
> +		if ((ps)->magic & ~(mask))	       \
> +			die("BUG:%s:%d: unsupported magic %x",	\
> +			    __FILE__, __LINE__, (ps)->magic & ~(mask)); \
> +	} while (0)
> +
>  /* parse_pathspec flags */
>  #define PATHSPEC_PREFER_CWD (1<<0) /* No args means match cwd */
>  #define PATHSPEC_PREFER_FULL (1<<1) /* No args means match everything */
> diff --git a/tree-diff.c b/tree-diff.c
> index 826512e..5a87614 100644
> --- a/tree-diff.c
> +++ b/tree-diff.c
> @@ -198,6 +198,25 @@ static void try_to_follow_renames(struct tree_desc *t1, struct tree_desc *t2, co
>  	const char *paths[1];
>  	int i;
>  
> +	/*
> +	 * follow-rename code is very specific, we need exactly one
> +	 * path. Magic that matches more than one path is not
> +	 * supported.
> +	 */
> +	GUARD_PATHSPEC(&opt->pathspec, PATHSPEC_FROMTOP);
> +#if 0
> +	/*
> +	 * We should reject wildcards as well. Unfortunately we
> +	 * haven't got a reliable way to detect that 'foo\*bar' in
> +	 * fact has no wildcards. nowildcard_len is merely a hint for
> +	 * optimization. Let it slip for now until wildmatch is taught
> +	 * about dry-run mode and returns wildcard info.
> +	 */
> +	if (opt->pathspec.has_wildcard)
> +		die("BUG:%s:%d: wildcards are not supported",
> +		    __FILE__, __LINE__);
> +#endif
> +
>  	/* Remove the file creation entry from the diff queue, and remember it */
>  	choice = q->queue[0];
>  	q->nr = 0;
> diff --git a/tree-walk.c b/tree-walk.c
> index d399ca9..37b157e 100644
> --- a/tree-walk.c
> +++ b/tree-walk.c
> @@ -636,6 +636,8 @@ enum interesting tree_entry_interesting(const struct name_entry *entry,
>  	enum interesting never_interesting = ps->has_wildcard ?
>  		entry_not_interesting : all_entries_not_interesting;
>  
> +	GUARD_PATHSPEC(ps, PATHSPEC_FROMTOP | PATHSPEC_MAXDEPTH);
> +
>  	if (!ps->nr) {
>  		if (!ps->recursive ||
>  		    !(ps->magic & PATHSPEC_MAXDEPTH) ||

Re: [PATCH v1 11/45] parse_pathspec: support stripping submodule trailing slashes

[Duy Nguyen]

On Mon, Mar 18, 2013 at 4:55 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
>
>> This flag is equivalent to builtin/ls-files.c:strip_trailing_slashes()
>> and is intended to replace that function when ls-files is converted to
>> use parse_pathspec.
>>
>> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
>> ---
>>  pathspec.c | 9 +++++++++
>>  pathspec.h | 1 +
>>  2 files changed, 10 insertions(+)
>>
>> diff --git a/pathspec.c b/pathspec.c
>> index b2446c3..2da8bc9 100644
>> --- a/pathspec.c
>> +++ b/pathspec.c
>> @@ -204,6 +204,15 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
>>       *raw = item->match = match;
>>       item->original = elt;
>>       item->len = strlen(item->match);
>> +
>> +     if ((flags & PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP) &&
>
> I see that having cheap and expensive variant at these steps is a
> way to achieve a bug-for-bug compatible rewrite of the existing
> code, but in the longer term should we lose one of them?  After all,
> a path that points at the working tree of another repository that is
> beyond a symlink cannot be added to us as a submodule, so CHEAP can
> be used only when we know that any intermediate component on the
> path is not a symlink pointing elsewhere, and the user in the
> codepath of ls-files may know it.

I did not concentrate on the future part. But yes only one should
remain in long term. Cheap vs expensive does not say much.

> To put it differently, shouldn't CHEAP and EXPENSIVE be accompanied
> by in-code comment and updates to Documentation/technical/api-*.txt
> to help users of the API to decide which one to use?

Will do (also for the comment on 14/45)
-- 
Duy

Re: [PATCH v1 27/45] Convert run_add_interactive to use struct pathspec

[John Keeping]

On Fri, Mar 15, 2013 at 01:06:42PM +0700, Nguyễn Thái Ngọc Duy wrote:
> This passes the pathspec, more or less unmodified, to
> git-add--interactive. The command itself does not process pathspec. It
> simply passes the pathspec to other builtin commands. So if all those
> commands support pathspec, we're good.

This breaks "git reset --keep" in a subdirectory for me.

I ran "git reset --keep <branch>" in a subdirectory and got:

    fatal: BUG: parse_pathspec cannot take no argument in this case

Bisecting points to this commit.

The simplest test case is:

    ( cd t && ../bin-wrappers/git reset --keep HEAD )

which works on master but not pu.

> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
>  builtin/add.c      | 26 ++++++++++----------------
>  builtin/checkout.c |  9 ++++-----
>  builtin/reset.c    |  8 ++++----
>  commit.h           |  2 +-
>  4 files changed, 19 insertions(+), 26 deletions(-)
> 
> diff --git a/builtin/add.c b/builtin/add.c
> index ec6fbe3..2b20d7d 100644
> --- a/builtin/add.c
> +++ b/builtin/add.c
> @@ -139,16 +139,12 @@ static void refresh(int verbose, const char **pathspec)
>  }
>  
>  int run_add_interactive(const char *revision, const char *patch_mode,
> -			const char **pathspec)
> +			const struct pathspec *pathspec)
>  {
> -	int status, ac, pc = 0;
> +	int status, ac, i;
>  	const char **args;
>  
> -	if (pathspec)
> -		while (pathspec[pc])
> -			pc++;
> -
> -	args = xcalloc(sizeof(const char *), (pc + 5));
> +	args = xcalloc(sizeof(const char *), (pathspec->nr + 6));
>  	ac = 0;
>  	args[ac++] = "add--interactive";
>  	if (patch_mode)
> @@ -156,11 +152,9 @@ int run_add_interactive(const char *revision, const char *patch_mode,
>  	if (revision)
>  		args[ac++] = revision;
>  	args[ac++] = "--";
> -	if (pc) {
> -		memcpy(&(args[ac]), pathspec, sizeof(const char *) * pc);
> -		ac += pc;
> -	}
> -	args[ac] = NULL;
> +	for (i = 0; i < pathspec->nr; i++)
> +		/* pass original pathspec, to be re-parsed */
> +		args[ac++] = pathspec->items[i].original;
>  
>  	status = run_command_v_opt(args, RUN_GIT_CMD);
>  	free(args);
> @@ -175,17 +169,17 @@ int interactive_add(int argc, const char **argv, const char *prefix, int patch)
>  	 * git-add--interactive itself does not parse pathspec. It
>  	 * simply passes the pathspec to other builtin commands. Let's
>  	 * hope all of them support all magic, or we'll need to limit
> -	 * the magic here. There is still a problem with prefix. But
> -	 * that'll be worked on later on.
> +	 * the magic here.
>  	 */
>  	parse_pathspec(&pathspec, PATHSPEC_ALL_MAGIC & ~PATHSPEC_FROMTOP,
>  		       PATHSPEC_PREFER_FULL |
> -		       PATHSPEC_SYMLINK_LEADING_PATH,
> +		       PATHSPEC_SYMLINK_LEADING_PATH |
> +		       PATHSPEC_PREFIX_ORIGIN,
>  		       prefix, argv);
>  
>  	return run_add_interactive(NULL,
>  				   patch ? "--patch" : NULL,
> -				   pathspec.raw);
> +				   &pathspec);
>  }
>  
>  static int edit_patch(int argc, const char **argv, const char *prefix)
> diff --git a/builtin/checkout.c b/builtin/checkout.c
> index 3c19cb4..2ddff95 100644
> --- a/builtin/checkout.c
> +++ b/builtin/checkout.c
> @@ -256,7 +256,7 @@ static int checkout_paths(const struct checkout_opts *opts,
>  
>  	if (opts->patch_mode)
>  		return run_add_interactive(revision, "--patch=checkout",
> -					   opts->pathspec.raw);
> +					   &opts->pathspec);
>  
>  	lock_file = xcalloc(1, sizeof(struct lock_file));
>  
> @@ -1115,10 +1115,9 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
>  		 * cannot handle. Magic mask is pretty safe to be
>  		 * lifted for new magic when opts.patch_mode == 0.
>  		 */
> -		parse_pathspec(&opts.pathspec,
> -			       opts.patch_mode == 0 ? 0 :
> -			       (PATHSPEC_ALL_MAGIC & ~PATHSPEC_FROMTOP),
> -			       0, prefix, argv);
> +		parse_pathspec(&opts.pathspec, 0,
> +			       opts.patch_mode ? PATHSPEC_PREFIX_ORIGIN : 0,
> +			       prefix, argv);
>  
>  		if (!opts.pathspec.nr)
>  			die(_("invalid path specification"));
> diff --git a/builtin/reset.c b/builtin/reset.c
> index da7282e..7c6e8b6 100644
> --- a/builtin/reset.c
> +++ b/builtin/reset.c
> @@ -216,9 +216,9 @@ static void parse_args(struct pathspec *pathspec,
>  		}
>  	}
>  	*rev_ret = rev;
> -	parse_pathspec(pathspec,
> -		       patch_mode ? PATHSPEC_ALL_MAGIC & ~PATHSPEC_FROMTOP : 0,
> -		       PATHSPEC_PREFER_FULL,
> +	parse_pathspec(pathspec, 0,
> +		       PATHSPEC_PREFER_FULL |
> +		       patch_mode ? PATHSPEC_PREFIX_ORIGIN : 0,
>  		       prefix, argv);
>  }
>  
> @@ -296,7 +296,7 @@ int cmd_reset(int argc, const char **argv, const char *prefix)
>  	if (patch_mode) {
>  		if (reset_type != NONE)
>  			die(_("--patch is incompatible with --{hard,mixed,soft}"));
> -		return run_add_interactive(sha1_to_hex(sha1), "--patch=reset", pathspec.raw);
> +		return run_add_interactive(sha1_to_hex(sha1), "--patch=reset", &pathspec);
>  	}
>  
>  	/* git reset tree [--] paths... can be used to
> diff --git a/commit.h b/commit.h
> index 4138bb4..9448fda 100644
> --- a/commit.h
> +++ b/commit.h
> @@ -179,7 +179,7 @@ int in_merge_bases(struct commit *, struct commit *);
>  
>  extern int interactive_add(int argc, const char **argv, const char *prefix, int patch);
>  extern int run_add_interactive(const char *revision, const char *patch_mode,
> -			       const char **pathspec);
> +			       const struct pathspec *pathspec);
>  
>  static inline int single_parent(struct commit *commit)
>  {
> -- 
> 1.8.0.rc0.19.g7bbb31d

Re: [PATCH v1 27/45] Convert run_add_interactive to use struct pathspec

[Duy Nguyen]

On Tue, Mar 19, 2013 at 1:26 AM, John Keeping <john@keeping.me.uk> wrote:
> On Fri, Mar 15, 2013 at 01:06:42PM +0700, Nguyễn Thái Ngọc Duy wrote:
>> This passes the pathspec, more or less unmodified, to
>> git-add--interactive. The command itself does not process pathspec. It
>> simply passes the pathspec to other builtin commands. So if all those
>> commands support pathspec, we're good.
>
> This breaks "git reset --keep" in a subdirectory for me.
>
> I ran "git reset --keep <branch>" in a subdirectory and got:
>
>     fatal: BUG: parse_pathspec cannot take no argument in this case
>
> Bisecting points to this commit.
>
> The simplest test case is:
>
>     ( cd t && ../bin-wrappers/git reset --keep HEAD )
>
> which works on master but not pu.

Beautiful. I got messed up with C operator precedence. This should fix
it. I'll check the rest of parse_pathspec calls later.

diff --git a/builtin/reset.c b/builtin/reset.c
index ab3917d..b665218 100644
--- a/builtin/reset.c
+++ b/builtin/reset.c
@@ -219,7 +219,7 @@ static void parse_args(struct pathspec *pathspec,
        *rev_ret = rev;
        parse_pathspec(pathspec, 0,
                       PATHSPEC_PREFER_FULL |
-                      patch_mode ? PATHSPEC_PREFIX_ORIGIN : 0,
+                      (patch_mode ? PATHSPEC_PREFIX_ORIGIN : 0),
                       prefix, argv);
 }

-- 
Duy

Re: [PATCH v1 27/45] Convert run_add_interactive to use struct pathspec

[John Keeping]

On Tue, Mar 19, 2013 at 08:58:23AM +0700, Duy Nguyen wrote:
> On Tue, Mar 19, 2013 at 1:26 AM, John Keeping <john@keeping.me.uk> wrote:
> > On Fri, Mar 15, 2013 at 01:06:42PM +0700, Nguyễn Thái Ngọc Duy wrote:
> >> This passes the pathspec, more or less unmodified, to
> >> git-add--interactive. The command itself does not process pathspec. It
> >> simply passes the pathspec to other builtin commands. So if all those
> >> commands support pathspec, we're good.
> >
> > This breaks "git reset --keep" in a subdirectory for me.
> >
> > I ran "git reset --keep <branch>" in a subdirectory and got:
> >
> >     fatal: BUG: parse_pathspec cannot take no argument in this case
> >
> > Bisecting points to this commit.
> >
> > The simplest test case is:
> >
> >     ( cd t && ../bin-wrappers/git reset --keep HEAD )
> >
> > which works on master but not pu.
> 
> Beautiful. I got messed up with C operator precedence. This should fix
> it. I'll check the rest of parse_pathspec calls later.

Yes, this fixes it.  Thanks.

> diff --git a/builtin/reset.c b/builtin/reset.c
> index ab3917d..b665218 100644
> --- a/builtin/reset.c
> +++ b/builtin/reset.c
> @@ -219,7 +219,7 @@ static void parse_args(struct pathspec *pathspec,
>         *rev_ret = rev;
>         parse_pathspec(pathspec, 0,
>                        PATHSPEC_PREFER_FULL |
> -                      patch_mode ? PATHSPEC_PREFIX_ORIGIN : 0,
> +                      (patch_mode ? PATHSPEC_PREFIX_ORIGIN : 0),
>                        prefix, argv);
>  }
> 
> -- 
> Duy

Re: [PATCH v1 39/45] parse_pathspec: make sure the prefix part is wildcard-free

[Junio C Hamano]

This seems to break t7300-clean.sh #8

Re: [PATCH v1 39/45] parse_pathspec: make sure the prefix part is wildcard-free

[Duy Nguyen]

On Wed, Mar 20, 2013 at 1:34 AM, Junio C Hamano <gitster@pobox.com> wrote:
> This seems to break t7300-clean.sh #8

Repeatedly? I saw some t7300-clean.sh failures when running tests in
parallel, but never been able to reproduce it alone.
-- 
Duy

Re: [PATCH v1 39/45] parse_pathspec: make sure the prefix part is wildcard-free

[Duy Nguyen]

On Wed, Mar 20, 2013 at 8:32 AM, Duy Nguyen <pclouds@gmail.com> wrote:
> On Wed, Mar 20, 2013 at 1:34 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> This seems to break t7300-clean.sh #8
>
> Repeatedly? I saw some t7300-clean.sh failures when running tests in
> parallel, but never been able to reproduce it alone.

Never mind. There is a codepath that does not initialize the new field
"prefix" properly. Fixing it now..
-- 
Duy

[PATCH v2 00/45] parse_pathspec and :(glob) magic

[Nguyễn Thái Ngọc Duy]

Junio please pull the series from github [1]. I don't want to spam the
list with 45 patches again so I only send patches that are substantially
different from v1:

 - documentation for parse_pathspec
 - fix wrong operator precendence in git-reset
 - fix initialization that may lead to crashes in git-grep (and
   probably the failure in t7300-clean.sh)

Typos found by Eric are also fixed, just not sent here.

[1] https://github.com/pclouds/git/commits/parse-pathspec

Nguyễn Thái Ngọc Duy (6):
  Add parse_pathspec() that converts cmdline args to struct pathspec
  parse_pathspec: support stripping submodule trailing slashes
  parse_pathspec: support stripping/checking submodule paths
  Guard against new pathspec magic in pathspec matching code
  Convert run_add_interactive to use struct pathspec
  parse_pathspec: make sure the prefix part is wildcard-free
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v2 06/45] Add parse_pathspec() that converts cmdline args to struct pathspec

[Nguyễn Thái Ngọc Duy]

Currently to fill a struct pathspec, we do:

   const char **paths;
   paths = get_pathspec(prefix, argv);
   ...
   init_pathspec(&pathspec, paths);

"paths" can only carry bare strings, which loses information from
command line arguments such as pathspec magic or the prefix part's
length for each argument.

parse_pathspec() is introduced to combine the two calls into one. The
plan is gradually replace all get_pathspec() and init_pathspec() with
parse_pathspec(). get_pathspec() now becomes a thin wrapper of
parse_pathspec().

parse_pathspec() allows the caller to reject the pathspec magics that
it does not support. When a new pathspec magic is introduced, we can
enable it per command after making sure that all underlying code has no
problem with the new magic.

"flags" parameter is currently unused. But it would allow callers to
pass certain instructions to parse_pathspec, for example forcing
literal pathspec when no magic is used.

With the introduction of parse_pathspec, there are now two functions
that can initialize struct pathspec: init_pathspec and
parse_pathspec. Any semantic changes in struct pathspec must be
reflected in both functions. init_pathspec() will be phased out in
favor of parse_pathspec().

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/technical/api-setup.txt |  19 +++-
 dir.c                                 |   4 +-
 dir.h                                 |   2 +
 pathspec.c                            | 168 ++++++++++++++++++++++++++--------
 pathspec.h                            |  11 +++
 5 files changed, 163 insertions(+), 41 deletions(-)

diff --git a/Documentation/technical/api-setup.txt b/Documentation/technical/api-setup.txt
index 4f63a04..59a947e 100644
--- a/Documentation/technical/api-setup.txt
+++ b/Documentation/technical/api-setup.txt
@@ -8,6 +8,23 @@ Talk about
 * is_inside_git_dir()
 * is_inside_work_tree()
 * setup_work_tree()
-* get_pathspec()
 
 (Dscho)
+
+Pathspec
+========
+
+See glossary-context.txt for the syntax of pathspec. In memory, a
+pathspec set is represented by "struct pathspec" and is prepared by
+parse_pathspec(). This function takes several arguments:
+
+- magic_mask specifies what features that are NOT supported by the
+  following code. If a user attempts to use such a feature,
+  parse_pathspec() can reject it early.
+
+- flags specifies other things that the caller wants parse_pathspec to
+  perform.
+
+- prefix and args come from cmd_* functions
+
+get_pathspec() is obsolete and should never be used in new code.
diff --git a/dir.c b/dir.c
index 97ad45b..a442467 100644
--- a/dir.c
+++ b/dir.c
@@ -338,7 +338,7 @@ int match_pathspec_depth(const struct pathspec *ps,
 /*
  * Return the length of the "simple" part of a path match limiter.
  */
-static int simple_length(const char *match)
+int simple_length(const char *match)
 {
 	int len = -1;
 
@@ -350,7 +350,7 @@ static int simple_length(const char *match)
 	}
 }
 
-static int no_wildcard(const char *string)
+int no_wildcard(const char *string)
 {
 	return string[simple_length(string)] == '\0';
 }
diff --git a/dir.h b/dir.h
index c3eb4b5..89427fd 100644
--- a/dir.h
+++ b/dir.h
@@ -125,6 +125,8 @@ struct dir_struct {
 #define MATCHED_RECURSIVELY 1
 #define MATCHED_FNMATCH 2
 #define MATCHED_EXACTLY 3
+extern int simple_length(const char *match);
+extern int no_wildcard(const char *string);
 extern char *common_prefix(const char **pathspec);
 extern int match_pathspec(const char **pathspec, const char *name, int namelen, int prefix, char *seen);
 extern int match_pathspec_depth(const struct pathspec *pathspec,
diff --git a/pathspec.c b/pathspec.c
index ab53b8a..ebe9844 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -103,10 +103,6 @@ void die_if_path_beyond_symlink(const char *path, const char *prefix)
 /*
  * Magic pathspec
  *
- * NEEDSWORK: These need to be moved to dir.h or even to a new
- * pathspec.h when we restructure get_pathspec() users to use the
- * "struct pathspec" interface.
- *
  * Possible future magic semantics include stuff like:
  *
  *	{ PATHSPEC_NOGLOB, '!', "noglob" },
@@ -115,7 +111,6 @@ void die_if_path_beyond_symlink(const char *path, const char *prefix)
  *	{ PATHSPEC_REGEXP, '\0', "regexp" },
  *
  */
-#define PATHSPEC_FROMTOP    (1<<0)
 
 static struct pathspec_magic {
 	unsigned bit;
@@ -127,7 +122,7 @@ static struct pathspec_magic {
 
 /*
  * Take an element of a pathspec and check for magic signatures.
- * Append the result to the prefix.
+ * Append the result to the prefix. Return the magic bitmap.
  *
  * For now, we only parse the syntax and throw out anything other than
  * "top" magic.
@@ -138,10 +133,15 @@ static struct pathspec_magic {
  * the prefix part must always match literally, and a single stupid
  * string cannot express such a case.
  */
-static const char *prefix_pathspec(const char *prefix, int prefixlen, const char *elt)
+static unsigned prefix_pathspec(struct pathspec_item *item,
+				unsigned *p_short_magic,
+				const char **raw, unsigned flags,
+				const char *prefix, int prefixlen,
+				const char *elt)
 {
-	unsigned magic = 0;
+	unsigned magic = 0, short_magic = 0;
 	const char *copyfrom = elt;
+	char *match;
 	int i;
 
 	if (elt[0] != ':') {
@@ -183,7 +183,7 @@ static const char *prefix_pathspec(const char *prefix, int prefixlen, const char
 				break;
 			for (i = 0; i < ARRAY_SIZE(pathspec_magic); i++)
 				if (pathspec_magic[i].mnemonic == ch) {
-					magic |= pathspec_magic[i].bit;
+					short_magic |= pathspec_magic[i].bit;
 					break;
 				}
 			if (ARRAY_SIZE(pathspec_magic) <= i)
@@ -194,15 +194,128 @@ static const char *prefix_pathspec(const char *prefix, int prefixlen, const char
 			copyfrom++;
 	}
 
+	magic |= short_magic;
+	*p_short_magic = short_magic;
+
 	if (magic & PATHSPEC_FROMTOP)
-		return xstrdup(copyfrom);
+		match = xstrdup(copyfrom);
 	else
-		return prefix_path(prefix, prefixlen, copyfrom);
+		match = prefix_path(prefix, prefixlen, copyfrom);
+	*raw = item->match = match;
+	item->len = strlen(item->match);
+	if (limit_pathspec_to_literal())
+		item->nowildcard_len = item->len;
+	else
+		item->nowildcard_len = simple_length(item->match);
+	item->flags = 0;
+	if (item->nowildcard_len < item->len &&
+	    item->match[item->nowildcard_len] == '*' &&
+	    no_wildcard(item->match + item->nowildcard_len + 1))
+		item->flags |= PATHSPEC_ONESTAR;
+	return magic;
+}
+
+static int pathspec_item_cmp(const void *a_, const void *b_)
+{
+	struct pathspec_item *a, *b;
+
+	a = (struct pathspec_item *)a_;
+	b = (struct pathspec_item *)b_;
+	return strcmp(a->match, b->match);
+}
+
+static void NORETURN unsupported_magic(const char *pattern,
+				       unsigned magic,
+				       unsigned short_magic)
+{
+	struct strbuf sb = STRBUF_INIT;
+	int i, n;
+	for (n = i = 0; i < ARRAY_SIZE(pathspec_magic); i++) {
+		const struct pathspec_magic *m = pathspec_magic + i;
+		if (!(magic & m->bit))
+			continue;
+		if (sb.len)
+			strbuf_addstr(&sb, " ");
+		if (short_magic & m->bit)
+			strbuf_addf(&sb, "'%c'", m->mnemonic);
+		else
+			strbuf_addf(&sb, "'%s'", m->name);
+		n++;
+	}
+	/*
+	 * We may want to substitue "this command" with a command
+	 * name. E.g. when add--interactive dies when running
+	 * "checkout -p"
+	 */
+	die(_("%s: pathspec magic not supported by this command: %s"),
+	    pattern, sb.buf);
+}
+
+/*
+ * Given command line arguments and a prefix, convert the input to
+ * pathspec. die() any magic in magic_mask is used.
+ */
+void parse_pathspec(struct pathspec *pathspec,
+		    unsigned magic_mask, unsigned flags,
+		    const char *prefix, const char **argv)
+{
+	struct pathspec_item *item;
+	const char *entry = argv ? *argv : NULL;
+	int i, n, prefixlen;
+
+	memset(pathspec, 0, sizeof(*pathspec));
+
+	/* No arguments, no prefix -> no pathspec */
+	if (!entry && !prefix)
+		return;
+
+	/* No arguments with prefix -> prefix pathspec */
+	if (!entry) {
+		static const char *raw[2];
+
+		pathspec->items = item = xmalloc(sizeof(*item));
+		memset(item, 0, sizeof(*item));
+		item->match = prefix;
+		item->nowildcard_len = item->len = strlen(prefix);
+		raw[0] = prefix;
+		raw[1] = NULL;
+		pathspec->nr = 1;
+		pathspec->raw = raw;
+		return;
+	}
+
+	n = 0;
+	while (argv[n])
+		n++;
+
+	pathspec->nr = n;
+	pathspec->items = item = xmalloc(sizeof(*item) * n);
+	pathspec->raw = argv;
+	prefixlen = prefix ? strlen(prefix) : 0;
+
+	for (i = 0; i < n; i++) {
+		unsigned short_magic;
+		entry = argv[i];
+
+		item[i].magic = prefix_pathspec(item + i, &short_magic,
+						argv + i, flags,
+						prefix, prefixlen, entry);
+		if (item[i].magic & magic_mask)
+			unsupported_magic(entry,
+					  item[i].magic & magic_mask,
+					  short_magic);
+		if (item[i].nowildcard_len < item[i].len)
+			pathspec->has_wildcard = 1;
+		pathspec->magic |= item[i].magic;
+	}
+
+	qsort(pathspec->items, pathspec->nr,
+	      sizeof(struct pathspec_item), pathspec_item_cmp);
 }
 
 /*
  * N.B. get_pathspec() is deprecated in favor of the "struct pathspec"
- * based interface - see pathspec_magic above.
+ * based interface - see pathspec.c:parse_pathspec().
  *
  * Arguments:
  *  - prefix - a path relative to the root of the working tree
@@ -221,32 +334,11 @@ static const char *prefix_pathspec(const char *prefix, int prefixlen, const char
  */
 const char **get_pathspec(const char *prefix, const char **pathspec)
 {
-	const char *entry = *pathspec;
-	const char **src, **dst;
-	int prefixlen;
-
-	if (!prefix && !entry)
-		return NULL;
-
-	if (!entry) {
-		static const char *spec[2];
-		spec[0] = prefix;
-		spec[1] = NULL;
-		return spec;
-	}
-
-	/* Otherwise we have to re-write the entries.. */
-	src = pathspec;
-	dst = pathspec;
-	prefixlen = prefix ? strlen(prefix) : 0;
-	while (*src) {
-		*(dst++) = prefix_pathspec(prefix, prefixlen, *src);
-		src++;
-	}
-	*dst = NULL;
-	if (!*pathspec)
-		return NULL;
-	return pathspec;
+	struct pathspec ps;
+	parse_pathspec(&ps,
+		       PATHSPEC_ALL_MAGIC & ~PATHSPEC_FROMTOP,
+		       0, prefix, pathspec);
+	return ps.raw;
 }
 
 void copy_pathspec(struct pathspec *dst, const struct pathspec *src)
diff --git a/pathspec.h b/pathspec.h
index a621676..937ec91 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -1,6 +1,10 @@
 #ifndef PATHSPEC_H
 #define PATHSPEC_H
 
+/* Pathspec magic */
+#define PATHSPEC_FROMTOP	(1<<0)
+#define PATHSPEC_ALL_MAGIC PATHSPEC_FROMTOP
+
 #define PATHSPEC_ONESTAR 1	/* the pathspec pattern sastisfies GFNM_ONESTAR */
 
 struct pathspec {
@@ -8,9 +12,11 @@ struct pathspec {
 	int nr;
 	unsigned int has_wildcard:1;
 	unsigned int recursive:1;
+	unsigned magic;
 	int max_depth;
 	struct pathspec_item {
 		const char *match;
+		unsigned magic;
 		int len;
 		int nowildcard_len;
 		int flags;
@@ -18,6 +24,11 @@ struct pathspec {
 };
 
 extern int init_pathspec(struct pathspec *, const char **);
+extern void parse_pathspec(struct pathspec *pathspec,
+			   unsigned magic_mask,
+			   unsigned flags,
+			   const char *prefix,
+			   const char **args);
 extern void copy_pathspec(struct pathspec *dst, const struct pathspec *src);
 extern void free_pathspec(struct pathspec *);
 
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v2 11/45] parse_pathspec: support stripping submodule trailing slashes

[Nguyễn Thái Ngọc Duy]

This flag is equivalent to builtin/ls-files.c:strip_trailing_slashes()
and is intended to replace that function when ls-files is converted to
use parse_pathspec.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 pathspec.c | 9 +++++++++
 pathspec.h | 2 ++
 2 files changed, 11 insertions(+)

diff --git a/pathspec.c b/pathspec.c
index fb5443c..57ad9ba 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -204,6 +204,15 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 	*raw = item->match = match;
 	item->original = elt;
 	item->len = strlen(item->match);
+
+	if ((flags & PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP) &&
+	    (item->len >= 1 && item->match[item->len - 1] == '/') &&
+	    (i = cache_name_pos(item->match, item->len - 1)) >= 0 &&
+	    S_ISGITLINK(active_cache[i]->ce_mode)) {
+		item->len--;
+		match[item->len] = '\0';
+	}
+
 	if (limit_pathspec_to_literal())
 		item->nowildcard_len = item->len;
 	else
diff --git a/pathspec.h b/pathspec.h
index aa98597..7935b26 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -31,6 +31,8 @@ struct pathspec {
 #define PATHSPEC_PREFER_CWD (1<<0) /* No args means match cwd */
 #define PATHSPEC_PREFER_FULL (1<<1) /* No args means match everything */
 #define PATHSPEC_MAXDEPTH_VALID (1<<2) /* max_depth field is valid */
+/* stripping the trailing slash if the given path is a gitlink */
+#define PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP (1<<3)
 
 extern int init_pathspec(struct pathspec *, const char **);
 extern void parse_pathspec(struct pathspec *pathspec,
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v2 12/45] parse_pathspec: support stripping/checking submodule paths

[Nguyễn Thái Ngọc Duy]

PATHSPEC_SYMLINK_LEADING_PATH and _STRIP_SUBMODULE_SLASH_EXPENSIVE are
respectively the alternate implementation of
pathspec.c:die_if_path_beyond_symlink() and
pathspec.c:check_path_for_gitlink(). They are intended to replace
those functions when builtin/add.c and builtin/check-ignore.c are
converted to use parse_pathspec.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 pathspec.c | 26 ++++++++++++++++++++++++++
 pathspec.h | 10 ++++++++++
 2 files changed, 36 insertions(+)

diff --git a/pathspec.c b/pathspec.c
index 57ad9ba..40bf178 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -213,6 +213,26 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 		match[item->len] = '\0';
 	}
 
+	if (flags & PATHSPEC_STRIP_SUBMODULE_SLASH_EXPENSIVE)
+		for (i = 0; i < active_nr; i++) {
+			struct cache_entry *ce = active_cache[i];
+			int ce_len = ce_namelen(ce);
+
+			if (!S_ISGITLINK(ce->ce_mode))
+				continue;
+
+			if (item->len <= ce_len || match[ce_len] != '/' ||
+			    memcmp(ce->name, match, ce_len))
+				continue;
+			if (item->len == ce_len + 1) {
+				/* strip trailing slash */
+				item->len--;
+				match[item->len] = '\0';
+			} else
+				die (_("Pathspec '%s' is in submodule '%.*s'"),
+				     elt, ce_len, ce->name);
+		}
+
 	if (limit_pathspec_to_literal())
 		item->nowildcard_len = item->len;
 	else
@@ -328,6 +348,12 @@ void parse_pathspec(struct pathspec *pathspec,
 			unsupported_magic(entry,
 					  item[i].magic & magic_mask,
 					  short_magic);
+
+		if ((flags & PATHSPEC_SYMLINK_LEADING_PATH) &&
+		    has_symlink_leading_path(item[i].match, item[i].len)) {
+			die(_("pathspec '%s' is beyond a symbolic link"), entry);
+		}
+
 		if (item[i].nowildcard_len < item[i].len)
 			pathspec->has_wildcard = 1;
 		pathspec->magic |= item[i].magic;
diff --git a/pathspec.h b/pathspec.h
index 7935b26..b631514 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -33,6 +33,16 @@ struct pathspec {
 #define PATHSPEC_MAXDEPTH_VALID (1<<2) /* max_depth field is valid */
 /* stripping the trailing slash if the given path is a gitlink */
 #define PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP (1<<3)
+/* die if a symlink is part of the given path's directory */
+#define PATHSPEC_SYMLINK_LEADING_PATH (1<<4)
+/*
+ * This is like a combination of ..LEADING_PATH and .._SLASH_CHEAP
+ * (but not the same): it strips the trailing slash if the given path
+ * is a gitlink but also checks and dies if gitlink is part of the
+ * leading path (i.e. the given path goes beyond a submodule). It's
+ * safer than _SLASH_CHEAP and also more expensive.
+ */
+#define PATHSPEC_STRIP_SUBMODULE_SLASH_EXPENSIVE (1<<5)
 
 extern int init_pathspec(struct pathspec *, const char **);
 extern void parse_pathspec(struct pathspec *pathspec,
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v2 14/45] Guard against new pathspec magic in pathspec matching code

[Nguyễn Thái Ngọc Duy]

GUARD_PATHSPEC() marks pathspec-sensitive code, basically all those
that touch anything in 'struct pathspec' except fields "nr" and
"original". GUARD_PATHSPEC() is not supposed to fail. It's mainly to
help the designers to catch unsupported codepaths.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/technical/api-setup.txt | 19 +++++++++++++++++++
 builtin/diff.c                        |  2 ++
 dir.c                                 |  2 ++
 pathspec.h                            |  7 +++++++
 tree-diff.c                           | 19 +++++++++++++++++++
 tree-walk.c                           |  2 ++
 6 files changed, 51 insertions(+)

diff --git a/Documentation/technical/api-setup.txt b/Documentation/technical/api-setup.txt
index 59a947e..f62528e 100644
--- a/Documentation/technical/api-setup.txt
+++ b/Documentation/technical/api-setup.txt
@@ -28,3 +28,22 @@ parse_pathspec(). This function takes several arguments:
 - prefix and args come from cmd_* functions
 
 get_pathspec() is obsolete and should never be used in new code.
+
+parse_pathspec() helps catch unsupported features and reject it
+politely. At a lower level, different pathspec-related functions may
+not support the same set of features. Such pathspec-sensitive
+functions are guarded with GUARD_PATHSPEC(), which will die in an
+unfriendly way when an unsupported feature is requested.
+
+The command designers are supposed to make sure that GUARD_PATHSPEC()
+never dies. They have to make sure all unsupported features are caught
+by parse_pathspec(), not by GUARD_PATHSPEC. grepping GUARD_PATHSPEC()
+should give the designers all pathspec-sensitive codepaths and what
+features they support.
+
+A similar process is applied when a new pathspec magic is added. The
+designer lifts the GUARD_PATHSPEC restriction in the functions that
+support the new magic. At the same time (s)he has to make sure this
+new feature will be caught at parse_pathspec() in commands that cannot
+handle the new magic in some cases. grepping parse_pathspec() should
+help.
diff --git a/builtin/diff.c b/builtin/diff.c
index 8c2af6c..d237e0a 100644
--- a/builtin/diff.c
+++ b/builtin/diff.c
@@ -371,6 +371,8 @@ int cmd_diff(int argc, const char **argv, const char *prefix)
 		die(_("unhandled object '%s' given."), name);
 	}
 	if (rev.prune_data.nr) {
+		/* builtin_diff_b_f() */
+		GUARD_PATHSPEC(&rev.prune_data, PATHSPEC_FROMTOP);
 		if (!path)
 			path = rev.prune_data.items[0].match;
 		paths += rev.prune_data.nr;
diff --git a/dir.c b/dir.c
index 1e9db41..6094ba8 100644
--- a/dir.c
+++ b/dir.c
@@ -297,6 +297,8 @@ int match_pathspec_depth(const struct pathspec *ps,
 {
 	int i, retval = 0;
 
+	GUARD_PATHSPEC(ps, PATHSPEC_FROMTOP | PATHSPEC_MAXDEPTH);
+
 	if (!ps->nr) {
 		if (!ps->recursive ||
 		    !(ps->magic & PATHSPEC_MAXDEPTH) ||
diff --git a/pathspec.h b/pathspec.h
index 3ca6636..7068f7d 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -27,6 +27,13 @@ struct pathspec {
 	} *items;
 };
 
+#define GUARD_PATHSPEC(ps, mask) \
+	do { \
+		if ((ps)->magic & ~(mask))	       \
+			die("BUG:%s:%d: unsupported magic %x",	\
+			    __FILE__, __LINE__, (ps)->magic & ~(mask)); \
+	} while (0)
+
 /* parse_pathspec flags */
 #define PATHSPEC_PREFER_CWD (1<<0) /* No args means match cwd */
 #define PATHSPEC_PREFER_FULL (1<<1) /* No args means match everything */
diff --git a/tree-diff.c b/tree-diff.c
index 826512e..5a87614 100644
--- a/tree-diff.c
+++ b/tree-diff.c
@@ -198,6 +198,25 @@ static void try_to_follow_renames(struct tree_desc *t1, struct tree_desc *t2, co
 	const char *paths[1];
 	int i;
 
+	/*
+	 * follow-rename code is very specific, we need exactly one
+	 * path. Magic that matches more than one path is not
+	 * supported.
+	 */
+	GUARD_PATHSPEC(&opt->pathspec, PATHSPEC_FROMTOP);
+#if 0
+	/*
+	 * We should reject wildcards as well. Unfortunately we
+	 * haven't got a reliable way to detect that 'foo\*bar' in
+	 * fact has no wildcards. nowildcard_len is merely a hint for
+	 * optimization. Let it slip for now until wildmatch is taught
+	 * about dry-run mode and returns wildcard info.
+	 */
+	if (opt->pathspec.has_wildcard)
+		die("BUG:%s:%d: wildcards are not supported",
+		    __FILE__, __LINE__);
+#endif
+
 	/* Remove the file creation entry from the diff queue, and remember it */
 	choice = q->queue[0];
 	q->nr = 0;
diff --git a/tree-walk.c b/tree-walk.c
index d399ca9..37b157e 100644
--- a/tree-walk.c
+++ b/tree-walk.c
@@ -636,6 +636,8 @@ enum interesting tree_entry_interesting(const struct name_entry *entry,
 	enum interesting never_interesting = ps->has_wildcard ?
 		entry_not_interesting : all_entries_not_interesting;
 
+	GUARD_PATHSPEC(ps, PATHSPEC_FROMTOP | PATHSPEC_MAXDEPTH);
+
 	if (!ps->nr) {
 		if (!ps->recursive ||
 		    !(ps->magic & PATHSPEC_MAXDEPTH) ||
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v2 27/45] Convert run_add_interactive to use struct pathspec

[Nguyễn Thái Ngọc Duy]

This passes the pathspec, more or less unmodified, to
git-add--interactive. The command itself does not process pathspec. It
simply passes the pathspec to other builtin commands. So if all those
commands support pathspec, we're good.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/add.c      | 26 ++++++++++----------------
 builtin/checkout.c |  9 ++++-----
 builtin/reset.c    |  8 ++++----
 commit.h           |  2 +-
 4 files changed, 19 insertions(+), 26 deletions(-)

diff --git a/builtin/add.c b/builtin/add.c
index ec6fbe3..2b20d7d 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -139,16 +139,12 @@ static void refresh(int verbose, const char **pathspec)
 }
 
 int run_add_interactive(const char *revision, const char *patch_mode,
-			const char **pathspec)
+			const struct pathspec *pathspec)
 {
-	int status, ac, pc = 0;
+	int status, ac, i;
 	const char **args;
 
-	if (pathspec)
-		while (pathspec[pc])
-			pc++;
-
-	args = xcalloc(sizeof(const char *), (pc + 5));
+	args = xcalloc(sizeof(const char *), (pathspec->nr + 6));
 	ac = 0;
 	args[ac++] = "add--interactive";
 	if (patch_mode)
@@ -156,11 +152,9 @@ int run_add_interactive(const char *revision, const char *patch_mode,
 	if (revision)
 		args[ac++] = revision;
 	args[ac++] = "--";
-	if (pc) {
-		memcpy(&(args[ac]), pathspec, sizeof(const char *) * pc);
-		ac += pc;
-	}
-	args[ac] = NULL;
+	for (i = 0; i < pathspec->nr; i++)
+		/* pass original pathspec, to be re-parsed */
+		args[ac++] = pathspec->items[i].original;
 
 	status = run_command_v_opt(args, RUN_GIT_CMD);
 	free(args);
@@ -175,17 +169,17 @@ int interactive_add(int argc, const char **argv, const char *prefix, int patch)
 	 * git-add--interactive itself does not parse pathspec. It
 	 * simply passes the pathspec to other builtin commands. Let's
 	 * hope all of them support all magic, or we'll need to limit
-	 * the magic here. There is still a problem with prefix. But
-	 * that'll be worked on later on.
+	 * the magic here.
 	 */
 	parse_pathspec(&pathspec, PATHSPEC_ALL_MAGIC & ~PATHSPEC_FROMTOP,
 		       PATHSPEC_PREFER_FULL |
-		       PATHSPEC_SYMLINK_LEADING_PATH,
+		       PATHSPEC_SYMLINK_LEADING_PATH |
+		       PATHSPEC_PREFIX_ORIGIN,
 		       prefix, argv);
 
 	return run_add_interactive(NULL,
 				   patch ? "--patch" : NULL,
-				   pathspec.raw);
+				   &pathspec);
 }
 
 static int edit_patch(int argc, const char **argv, const char *prefix)
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 3c19cb4..2ddff95 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -256,7 +256,7 @@ static int checkout_paths(const struct checkout_opts *opts,
 
 	if (opts->patch_mode)
 		return run_add_interactive(revision, "--patch=checkout",
-					   opts->pathspec.raw);
+					   &opts->pathspec);
 
 	lock_file = xcalloc(1, sizeof(struct lock_file));
 
@@ -1115,10 +1115,9 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 		 * cannot handle. Magic mask is pretty safe to be
 		 * lifted for new magic when opts.patch_mode == 0.
 		 */
-		parse_pathspec(&opts.pathspec,
-			       opts.patch_mode == 0 ? 0 :
-			       (PATHSPEC_ALL_MAGIC & ~PATHSPEC_FROMTOP),
-			       0, prefix, argv);
+		parse_pathspec(&opts.pathspec, 0,
+			       opts.patch_mode ? PATHSPEC_PREFIX_ORIGIN : 0,
+			       prefix, argv);
 
 		if (!opts.pathspec.nr)
 			die(_("invalid path specification"));
diff --git a/builtin/reset.c b/builtin/reset.c
index da7282e..1c8e4a5 100644
--- a/builtin/reset.c
+++ b/builtin/reset.c
@@ -216,9 +216,9 @@ static void parse_args(struct pathspec *pathspec,
 		}
 	}
 	*rev_ret = rev;
-	parse_pathspec(pathspec,
-		       patch_mode ? PATHSPEC_ALL_MAGIC & ~PATHSPEC_FROMTOP : 0,
-		       PATHSPEC_PREFER_FULL,
+	parse_pathspec(pathspec, 0,
+		       PATHSPEC_PREFER_FULL |
+		       (patch_mode ? PATHSPEC_PREFIX_ORIGIN : 0),
 		       prefix, argv);
 }
 
@@ -296,7 +296,7 @@ int cmd_reset(int argc, const char **argv, const char *prefix)
 	if (patch_mode) {
 		if (reset_type != NONE)
 			die(_("--patch is incompatible with --{hard,mixed,soft}"));
-		return run_add_interactive(sha1_to_hex(sha1), "--patch=reset", pathspec.raw);
+		return run_add_interactive(sha1_to_hex(sha1), "--patch=reset", &pathspec);
 	}
 
 	/* git reset tree [--] paths... can be used to
diff --git a/commit.h b/commit.h
index 4138bb4..9448fda 100644
--- a/commit.h
+++ b/commit.h
@@ -179,7 +179,7 @@ int in_merge_bases(struct commit *, struct commit *);
 
 extern int interactive_add(int argc, const char **argv, const char *prefix, int patch);
 extern int run_add_interactive(const char *revision, const char *patch_mode,
-			       const char **pathspec);
+			       const struct pathspec *pathspec);
 
 static inline int single_parent(struct commit *commit)
 {
-- 
1.8.0.rc0.19.g7bbb31d

[PATCH v2 39/45] parse_pathspec: make sure the prefix part is wildcard-free

[Nguyễn Thái Ngọc Duy]

Prepending prefix to pathspec is a trick to workaround the fact that
commands can be executed in a subdirectory, but all git commands run
at worktree's root. The prefix part should always be treated as
literal string. Make it so.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 cache.h    |  2 ++
 path.c     | 15 ++++++++++++++-
 pathspec.c | 17 +++++++++++++----
 pathspec.h |  2 +-
 setup.c    |  9 +++++----
 5 files changed, 35 insertions(+), 10 deletions(-)

diff --git a/cache.h b/cache.h
index 6297783..17e0b26 100644
--- a/cache.h
+++ b/cache.h
@@ -401,6 +401,7 @@ extern void setup_work_tree(void);
 extern const char *setup_git_directory_gently(int *);
 extern const char *setup_git_directory(void);
 extern char *prefix_path(const char *prefix, int len, const char *path);
+extern char *prefix_path_gently(const char *prefix, int *p_len, const char *path);
 extern const char *prefix_filename(const char *prefix, int len, const char *path);
 extern int check_filename(const char *prefix, const char *name);
 extern void verify_filename(const char *prefix,
@@ -712,6 +713,7 @@ const char *real_path(const char *path);
 const char *real_path_if_valid(const char *path);
 const char *absolute_path(const char *path);
 const char *relative_path(const char *abs, const char *base);
+int normalize_path_copy_len(char *dst, const char *src, int *prefix_len);
 int normalize_path_copy(char *dst, const char *src);
 int longest_ancestor_length(const char *path, struct string_list *prefixes);
 char *strip_path_suffix(const char *path, const char *suffix);
diff --git a/path.c b/path.c
index d3d3f8b..7baf334 100644
--- a/path.c
+++ b/path.c
@@ -487,8 +487,14 @@ const char *relative_path(const char *abs, const char *base)
  *
  * Note that this function is purely textual.  It does not follow symlinks,
  * verify the existence of the path, or make any system calls.
+ *
+ * prefix_len != NULL is for a specific case of prefix_pathspec():
+ * assume that src == dst and src[0..prefix_len-1] is already
+ * normalized, any time "../" eats up to the prefix_len part,
+ * prefix_len is reduced. In the end prefix_len is the remaining
+ * prefix that has not been overridden by user pathspec.
  */
-int normalize_path_copy(char *dst, const char *src)
+int normalize_path_copy_len(char *dst, const char *src, int *prefix_len)
 {
 	char *dst0;
 
@@ -563,11 +569,18 @@ int normalize_path_copy(char *dst, const char *src)
 		/* Windows: dst[-1] cannot be backslash anymore */
 		while (dst0 < dst && dst[-1] != '/')
 			dst--;
+		if (prefix_len && *prefix_len > dst - dst0)
+			*prefix_len = dst - dst0;
 	}
 	*dst = '\0';
 	return 0;
 }
 
+int normalize_path_copy(char *dst, const char *src)
+{
+	return normalize_path_copy_len(dst, src, NULL);
+}
+
 /*
  * path = Canonical absolute path
  * prefixes = string_list containing normalized, absolute paths without
diff --git a/pathspec.c b/pathspec.c
index 9cd3f2c..01f8844 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -149,10 +149,14 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 	magic |= short_magic;
 	*p_short_magic = short_magic;
 
-	if (magic & PATHSPEC_FROMTOP)
+	if (magic & PATHSPEC_FROMTOP) {
 		match = xstrdup(copyfrom);
-	else
-		match = prefix_path(prefix, prefixlen, copyfrom);
+		prefixlen = 0;
+	} else {
+		match = prefix_path_gently(prefix, &prefixlen, copyfrom);
+		if (!match)
+			die("%s: '%s' is outside repository", elt, copyfrom);
+	}
 	*raw = item->match = match;
 	/*
 	 * Prefix the pathspec (keep all magic) and put to
@@ -166,6 +170,7 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 	} else
 		item->original = elt;
 	item->len = strlen(item->match);
+	item->prefix = prefixlen;
 
 	if ((flags & PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP) &&
 	    (item->len >= 1 && item->match[item->len - 1] == '/') &&
@@ -197,8 +202,11 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 
 	if (limit_pathspec_to_literal())
 		item->nowildcard_len = item->len;
-	else
+	else {
 		item->nowildcard_len = simple_length(item->match);
+		if (item->nowildcard_len < prefixlen)
+			item->nowildcard_len = prefixlen;
+	}
 	item->flags = 0;
 	if (item->nowildcard_len < item->len &&
 	    item->match[item->nowildcard_len] == '*' &&
@@ -283,6 +291,7 @@ void parse_pathspec(struct pathspec *pathspec,
 		item->match = prefix;
 		item->original = prefix;
 		item->nowildcard_len = item->len = strlen(prefix);
+		item->prefix = item->len;
 		raw[0] = prefix;
 		raw[1] = NULL;
 		pathspec->nr = 1;
diff --git a/pathspec.h b/pathspec.h
index 16bbb7b..bbcfa74 100644
--- a/pathspec.h
+++ b/pathspec.h
@@ -21,7 +21,7 @@ struct pathspec {
 		const char *match;
 		const char *original;
 		unsigned magic;
-		int len;
+		int len, prefix;
 		int nowildcard_len;
 		int flags;
 	} *items;
diff --git a/setup.c b/setup.c
index acdc634..e661a71 100644
--- a/setup.c
+++ b/setup.c
@@ -5,10 +5,11 @@
 static int inside_git_dir = -1;
 static int inside_work_tree = -1;
 
-static char *prefix_path_gently(const char *prefix, int len, const char *path)
+char *prefix_path_gently(const char *prefix, int *p_len, const char *path)
 {
 	const char *orig = path;
 	char *sanitized;
+	int len = *p_len;
 	if (is_absolute_path(orig)) {
 		const char *temp = real_path(path);
 		sanitized = xmalloc(len + strlen(temp) + 1);
@@ -19,7 +20,7 @@ static char *prefix_path_gently(const char *prefix, int len, const char *path)
 			memcpy(sanitized, prefix, len);
 		strcpy(sanitized + len, path);
 	}
-	if (normalize_path_copy(sanitized, sanitized))
+	if (normalize_path_copy_len(sanitized, sanitized, p_len))
 		goto error_out;
 	if (is_absolute_path(orig)) {
 		size_t root_len, len, total;
@@ -44,7 +45,7 @@ static char *prefix_path_gently(const char *prefix, int len, const char *path)
 
 char *prefix_path(const char *prefix, int len, const char *path)
 {
-	char *r = prefix_path_gently(prefix, len, path);
+	char *r = prefix_path_gently(prefix, &len, path);
 	if (!r)
 		die("'%s' is outside repository", path);
 	return r;
@@ -53,7 +54,7 @@ char *prefix_path(const char *prefix, int len, const char *path)
 int path_inside_repo(const char *prefix, const char *path)
 {
 	int len = prefix ? strlen(prefix) : 0;
-	char *r = prefix_path_gently(prefix, len, path);
+	char *r = prefix_path_gently(prefix, &len, path);
 	if (r) {
 		free(r);
 		return 1;
-- 
1.8.0.rc0.19.g7bbb31d

Re: [PATCH v2 00/45] parse_pathspec and :(glob) magic

[Junio C Hamano]

Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:

> Junio please pull the series from github [1]....
> [1] https://github.com/pclouds/git/commits/parse-pathspec

Please write it like this:

    https://github.com/pclouds/git parse-pathspec

so that I can just say

	git fetch <<that thing>>
        git diff nd/magic-pathspecs FETCH_HEAD

I am still getting this out of 7300, though.

expecting success: 

        mkdir -p build docs &&
        touch a.out src/part3.c docs/manual.txt obj.o build/lib.so &&
        would_clean=$(
                cd docs &&
                git clean -n "$(pwd)/../src" |
                sed -n -e "s|^Would remove ||p"
        ) &&
        test "$would_clean" = ../src/part3.c || {
                echo "OOps <$would_clean>"
                false
        }

OOps <>
not ok 8 - git clean with absolute path
#
#
#               mkdir -p build docs &&
#               touch a.out src/part3.c docs/manual.txt obj.o build/lib.so &&
#               would_clean=$(
#                       cd docs &&
#                       git clean -n "$(pwd)/../src" |
#                       sed -n -e "s|^Would remove ||p"
#               ) &&
#               test "$would_clean" = ../src/part3.c || {
#                       echo "OOps <$would_clean>"
#                       false
#               }
#

Re: [PATCH v2 06/45] Add parse_pathspec() that converts cmdline args to struct pathspec

[Eric Sunshine]

On Wed, Mar 20, 2013 at 8:16 AM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:
> +static void NORETURN unsupported_magic(const char *pattern,
> +                                      unsigned magic,
> +                                      unsigned short_magic)
> +{
> +       struct strbuf sb = STRBUF_INIT;
> +       int i, n;
> +       for (n = i = 0; i < ARRAY_SIZE(pathspec_magic); i++) {
> +               const struct pathspec_magic *m = pathspec_magic + i;
> +               if (!(magic & m->bit))
> +                       continue;
> +               if (sb.len)
> +                       strbuf_addstr(&sb, " ");
> +               if (short_magic & m->bit)
> +                       strbuf_addf(&sb, "'%c'", m->mnemonic);
> +               else
> +                       strbuf_addf(&sb, "'%s'", m->name);
> +               n++;
> +       }
> +       /*
> +        * We may want to substitue "this command" with a command

s/substitue/substitute/

> +        * name. E.g. when add--interactive dies when running
> +        * "checkout -p"
> +        */
> +       die(_("%s: pathspec magic not supported by this command: %s"),
> +           pattern, sb.buf);
> +}

Re: [PATCH v2 14/45] Guard against new pathspec magic in pathspec matching code

[Eric Sunshine]

On Wed, Mar 20, 2013 at 8:16 AM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:
> diff --git a/Documentation/technical/api-setup.txt b/Documentation/technical/api-setup.txt
> index 59a947e..f62528e 100644
> --- a/Documentation/technical/api-setup.txt
> +++ b/Documentation/technical/api-setup.txt
> @@ -28,3 +28,22 @@ parse_pathspec(). This function takes several arguments:
>  - prefix and args come from cmd_* functions
>
>  get_pathspec() is obsolete and should never be used in new code.
> +
> +parse_pathspec() helps catch unsupported features and reject it

s/reject it/reject them/

> +politely. At a lower level, different pathspec-related functions may
> +not support the same set of features. Such pathspec-sensitive
> +functions are guarded with GUARD_PATHSPEC(), which will die in an
> +unfriendly way when an unsupported feature is requested.

Re: [PATCH v2 00/45] parse_pathspec and :(glob) magic

[Duy Nguyen]

On Wed, Mar 20, 2013 at 11:02:50AM -0700, Junio C Hamano wrote:
> Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
> 
> > Junio please pull the series from github [1]....
> > [1] https://github.com/pclouds/git/commits/parse-pathspec
> 
> Please write it like this:
> 
>     https://github.com/pclouds/git parse-pathspec
> 
> so that I can just say
> 
> 	git fetch <<that thing>>
>         git diff nd/magic-pathspecs FETCH_HEAD

OK

> I am still getting this out of 7300, though.

I still can't reproduce it. But I think I found a bug that
miscalculates prefix length from absolute paths. Does this "fix" your
test?

-- 8< --
diff --git a/setup.c b/setup.c
index 3584f22..6ae147a 100644
--- a/setup.c
+++ b/setup.c
@@ -14,6 +14,7 @@ char *prefix_path_gently(const char *prefix, int *p_len, const char *path)
 		const char *temp = real_path(path);
 		sanitized = xmalloc(len + strlen(temp) + 1);
 		strcpy(sanitized, temp);
+		*p_len = 0;
 	} else {
 		sanitized = xmalloc(len + strlen(path) + 1);
 		if (len)

-- 8< --

Re: [PATCH v2 00/45] parse_pathspec and :(glob) magic

[Duy Nguyen]

On Thu, Mar 21, 2013 at 12:33:26PM +0700, Duy Nguyen wrote:
> > I am still getting this out of 7300, though.
> 
> I still can't reproduce it. But I think I found a bug that
> miscalculates prefix length from absolute paths. Does this "fix" your
> test?
> 
> -- 8< --
> diff --git a/setup.c b/setup.c
> index 3584f22..6ae147a 100644
> --- a/setup.c
> +++ b/setup.c
> @@ -14,6 +14,7 @@ char *prefix_path_gently(const char *prefix, int *p_len, const char *path)
>  		const char *temp = real_path(path);
>  		sanitized = xmalloc(len + strlen(temp) + 1);
>  		strcpy(sanitized, temp);
> +		*p_len = 0;
>  	} else {
>  		sanitized = xmalloc(len + strlen(path) + 1);
>  		if (len)
> 
> -- 8< --

Nope, that one could cause more crashes. Try this

-- 8< --
diff --git a/setup.c b/setup.c
index 3584f22..3d8eb97 100644
--- a/setup.c
+++ b/setup.c
@@ -14,6 +14,8 @@ char *prefix_path_gently(const char *prefix, int *p_len, const char *path)
 		const char *temp = real_path(path);
 		sanitized = xmalloc(len + strlen(temp) + 1);
 		strcpy(sanitized, temp);
+		if (p_len)
+			*p_len = 0;
 	} else {
 		sanitized = xmalloc(len + strlen(path) + 1);
 		if (len)
-- 8< --

Re: [PATCH v2 00/45] parse_pathspec and :(glob) magic

[Junio C Hamano]

Duy Nguyen <pclouds@gmail.com> writes:

>> I still can't reproduce it. But I think I found a bug that
>> miscalculates prefix length from absolute paths. Does this "fix" your
>> test?
>>  ...
> Nope, that one could cause more crashes. Try this
>
> -- 8< --
> diff --git a/setup.c b/setup.c
> index 3584f22..3d8eb97 100644
> --- a/setup.c
> +++ b/setup.c
> @@ -14,6 +14,8 @@ char *prefix_path_gently(const char *prefix, int *p_len, const char *path)
>  		const char *temp = real_path(path);
>  		sanitized = xmalloc(len + strlen(temp) + 1);
>  		strcpy(sanitized, temp);
> +		if (p_len)
> +			*p_len = 0;

Yes, this one seems to. "$(pwd)/../src" was not handled correctly.

The callchain to this locaiton would look like

	parse_pathspec() with prefix="docs/", prefixlen set to 5
        -> prefix_pathspec(), &prefixlen passed down
          -> prefix_path_gently(), p_len points at the above prefixlen
	     your "this should fix" patch sets *p_len to 0,
             original leaves *p_len as 5.
             -> normalize_path_copy_len() with p_len
	        *p_len is used here.

Why could the test pass for you without it?  It doesn't look like a
bug that depended on uninitialized memory or something from the
above observation.

Re: [PATCH v2 00/45] parse_pathspec and :(glob) magic

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Duy Nguyen <pclouds@gmail.com> writes:
>
>>> I still can't reproduce it. But I think I found a bug that
>>> miscalculates prefix length from absolute paths. Does this "fix" your
>>> test?
>>>  ...
>> Nope, that one could cause more crashes. Try this
>>
>> -- 8< --
>> diff --git a/setup.c b/setup.c
>> index 3584f22..3d8eb97 100644
>> --- a/setup.c
>> +++ b/setup.c
>> @@ -14,6 +14,8 @@ char *prefix_path_gently(const char *prefix, int *p_len, const char *path)
>>  		const char *temp = real_path(path);
>>  		sanitized = xmalloc(len + strlen(temp) + 1);
>>  		strcpy(sanitized, temp);
>> +		if (p_len)
>> +			*p_len = 0;
>
> Yes, this one seems to. "$(pwd)/../src" was not handled correctly.
>
> The callchain to this locaiton would look like
>
> 	parse_pathspec() with prefix="docs/", prefixlen set to 5
>         -> prefix_pathspec(), &prefixlen passed down
>           -> prefix_path_gently(), p_len points at the above prefixlen
> 	     your "this should fix" patch sets *p_len to 0,
>              original leaves *p_len as 5.
>              -> normalize_path_copy_len() with p_len
> 	        *p_len is used here.
>
> Why could the test pass for you without it?  It doesn't look like a
> bug that depended on uninitialized memory or something from the
> above observation.

The change made to prefix_path_gently() in this series is beyond
"disgusting", especially with the above fix-up.

Sometimes it uses the original "len", sometimes it uses the fixed-up
*p_len (e.g. passes it down to normalize_path_copy_len()), and lets
normalize_path_copy_len() further update it, and thenit makes the
caller use the updated *p_len.

Does the caller know what the value in *p_len _mean_ after this
function returns?  Can it afford to lose the original length of the
prefix it saved in a variable, without getting confused?

I think any change that turns a value-passed argument in the
existing code into modifiable pointer-to-variable in this series
should add in-code comment to describe what the variable mean upon
entry and after return, just like normalize_path_copy_len() that was
built out of the original normalize_path_copy().  I didn't look if
there are many others, or if this is the only one that is tricky. it
is tricky that even the original author of the patch got it wrong
X-<.

Re: [PATCH v2 06/45] Add parse_pathspec() that converts cmdline args to struct pathspec

[Junio C Hamano]

Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:

> diff --git a/Documentation/technical/api-setup.txt b/Documentation/technical/api-setup.txt
> index 4f63a04..59a947e 100644
> --- a/Documentation/technical/api-setup.txt
> +++ b/Documentation/technical/api-setup.txt
> @@ -8,6 +8,23 @@ Talk about
>  * is_inside_git_dir()
>  * is_inside_work_tree()
>  * setup_work_tree()
> -* get_pathspec()
>  
>  (Dscho)
> +
> +Pathspec
> +========

asciidoc: ERROR: api-setup.txt: line 15: only book doctypes can
contain level 0 sections

Just demoting this to "--------" should be sufficient, I think.

> +
> +See glossary-context.txt for the syntax of pathspec. In memory, a
> +pathspec set is represented by "struct pathspec" and is prepared by
> +parse_pathspec(). This function takes several arguments:
> +
> +- magic_mask specifies what features that are NOT supported by the
> +  following code. If a user attempts to use such a feature,
> +  parse_pathspec() can reject it early.
> +
> +- flags specifies other things that the caller wants parse_pathspec to
> +  perform.
> +
> +- prefix and args come from cmd_* functions
> +
> +get_pathspec() is obsolete and should never be used in new code.
> diff --git a/dir.c b/dir.c
> index 97ad45b..a442467 100644
> --- a/dir.c
> +++ b/dir.c
> @@ -338,7 +338,7 @@ int match_pathspec_depth(const struct pathspec *ps,
>  /*
>   * Return the length of the "simple" part of a path match limiter.
>   */
> -static int simple_length(const char *match)
> +int simple_length(const char *match)
>  {
>  	int len = -1;
>  
> @@ -350,7 +350,7 @@ static int simple_length(const char *match)
>  	}
>  }
>  
> -static int no_wildcard(const char *string)
> +int no_wildcard(const char *string)
>  {
>  	return string[simple_length(string)] == '\0';
>  }
> diff --git a/dir.h b/dir.h
> index c3eb4b5..89427fd 100644
> --- a/dir.h
> +++ b/dir.h
> @@ -125,6 +125,8 @@ struct dir_struct {
>  #define MATCHED_RECURSIVELY 1
>  #define MATCHED_FNMATCH 2
>  #define MATCHED_EXACTLY 3
> +extern int simple_length(const char *match);
> +extern int no_wildcard(const char *string);
>  extern char *common_prefix(const char **pathspec);
>  extern int match_pathspec(const char **pathspec, const char *name, int namelen, int prefix, char *seen);
>  extern int match_pathspec_depth(const struct pathspec *pathspec,
> diff --git a/pathspec.c b/pathspec.c
> index ab53b8a..ebe9844 100644
> --- a/pathspec.c
> +++ b/pathspec.c
> @@ -103,10 +103,6 @@ void die_if_path_beyond_symlink(const char *path, const char *prefix)
>  /*
>   * Magic pathspec
>   *
> - * NEEDSWORK: These need to be moved to dir.h or even to a new
> - * pathspec.h when we restructure get_pathspec() users to use the
> - * "struct pathspec" interface.
> - *
>   * Possible future magic semantics include stuff like:
>   *
>   *	{ PATHSPEC_NOGLOB, '!', "noglob" },
> @@ -115,7 +111,6 @@ void die_if_path_beyond_symlink(const char *path, const char *prefix)
>   *	{ PATHSPEC_REGEXP, '\0', "regexp" },
>   *
>   */
> -#define PATHSPEC_FROMTOP    (1<<0)
>  
>  static struct pathspec_magic {
>  	unsigned bit;
> @@ -127,7 +122,7 @@ static struct pathspec_magic {
>  
>  /*
>   * Take an element of a pathspec and check for magic signatures.
> - * Append the result to the prefix.
> + * Append the result to the prefix. Return the magic bitmap.
>   *
>   * For now, we only parse the syntax and throw out anything other than
>   * "top" magic.
> @@ -138,10 +133,15 @@ static struct pathspec_magic {
>   * the prefix part must always match literally, and a single stupid
>   * string cannot express such a case.
>   */
> -static const char *prefix_pathspec(const char *prefix, int prefixlen, const char *elt)
> +static unsigned prefix_pathspec(struct pathspec_item *item,
> +				unsigned *p_short_magic,
> +				const char **raw, unsigned flags,
> +				const char *prefix, int prefixlen,
> +				const char *elt)
>  {
> -	unsigned magic = 0;
> +	unsigned magic = 0, short_magic = 0;
>  	const char *copyfrom = elt;
> +	char *match;
>  	int i;
>  
>  	if (elt[0] != ':') {
> @@ -183,7 +183,7 @@ static const char *prefix_pathspec(const char *prefix, int prefixlen, const char
>  				break;
>  			for (i = 0; i < ARRAY_SIZE(pathspec_magic); i++)
>  				if (pathspec_magic[i].mnemonic == ch) {
> -					magic |= pathspec_magic[i].bit;
> +					short_magic |= pathspec_magic[i].bit;
>  					break;
>  				}
>  			if (ARRAY_SIZE(pathspec_magic) <= i)
> @@ -194,15 +194,128 @@ static const char *prefix_pathspec(const char *prefix, int prefixlen, const char
>  			copyfrom++;
>  	}
>  
> +	magic |= short_magic;
> +	*p_short_magic = short_magic;
> +
>  	if (magic & PATHSPEC_FROMTOP)
> -		return xstrdup(copyfrom);
> +		match = xstrdup(copyfrom);
>  	else
> -		return prefix_path(prefix, prefixlen, copyfrom);
> +		match = prefix_path(prefix, prefixlen, copyfrom);
> +	*raw = item->match = match;
> +	item->len = strlen(item->match);
> +	if (limit_pathspec_to_literal())
> +		item->nowildcard_len = item->len;
> +	else
> +		item->nowildcard_len = simple_length(item->match);
> +	item->flags = 0;
> +	if (item->nowildcard_len < item->len &&
> +	    item->match[item->nowildcard_len] == '*' &&
> +	    no_wildcard(item->match + item->nowildcard_len + 1))
> +		item->flags |= PATHSPEC_ONESTAR;
> +	return magic;
> +}
> +
> +static int pathspec_item_cmp(const void *a_, const void *b_)
> +{
> +	struct pathspec_item *a, *b;
> +
> +	a = (struct pathspec_item *)a_;
> +	b = (struct pathspec_item *)b_;
> +	return strcmp(a->match, b->match);
> +}
> +
> +static void NORETURN unsupported_magic(const char *pattern,
> +				       unsigned magic,
> +				       unsigned short_magic)
> +{
> +	struct strbuf sb = STRBUF_INIT;
> +	int i, n;
> +	for (n = i = 0; i < ARRAY_SIZE(pathspec_magic); i++) {
> +		const struct pathspec_magic *m = pathspec_magic + i;
> +		if (!(magic & m->bit))
> +			continue;
> +		if (sb.len)
> +			strbuf_addstr(&sb, " ");
> +		if (short_magic & m->bit)
> +			strbuf_addf(&sb, "'%c'", m->mnemonic);
> +		else
> +			strbuf_addf(&sb, "'%s'", m->name);
> +		n++;
> +	}
> +	/*
> +	 * We may want to substitue "this command" with a command
> +	 * name. E.g. when add--interactive dies when running
> +	 * "checkout -p"
> +	 */
> +	die(_("%s: pathspec magic not supported by this command: %s"),
> +	    pattern, sb.buf);
> +}
> +
> +/*
> + * Given command line arguments and a prefix, convert the input to
> + * pathspec. die() any magic in magic_mask is used.
> + */
> +void parse_pathspec(struct pathspec *pathspec,
> +		    unsigned magic_mask, unsigned flags,
> +		    const char *prefix, const char **argv)
> +{
> +	struct pathspec_item *item;
> +	const char *entry = argv ? *argv : NULL;
> +	int i, n, prefixlen;
> +
> +	memset(pathspec, 0, sizeof(*pathspec));
> +
> +	/* No arguments, no prefix -> no pathspec */
> +	if (!entry && !prefix)
> +		return;
> +
> +	/* No arguments with prefix -> prefix pathspec */
> +	if (!entry) {
> +		static const char *raw[2];
> +
> +		pathspec->items = item = xmalloc(sizeof(*item));
> +		memset(item, 0, sizeof(*item));
> +		item->match = prefix;
> +		item->nowildcard_len = item->len = strlen(prefix);
> +		raw[0] = prefix;
> +		raw[1] = NULL;
> +		pathspec->nr = 1;
> +		pathspec->raw = raw;
> +		return;
> +	}
> +
> +	n = 0;
> +	while (argv[n])
> +		n++;
> +
> +	pathspec->nr = n;
> +	pathspec->items = item = xmalloc(sizeof(*item) * n);
> +	pathspec->raw = argv;
> +	prefixlen = prefix ? strlen(prefix) : 0;
> +
> +	for (i = 0; i < n; i++) {
> +		unsigned short_magic;
> +		entry = argv[i];
> +
> +		item[i].magic = prefix_pathspec(item + i, &short_magic,
> +						argv + i, flags,
> +						prefix, prefixlen, entry);
> +		if (item[i].magic & magic_mask)
> +			unsupported_magic(entry,
> +					  item[i].magic & magic_mask,
> +					  short_magic);
> +		if (item[i].nowildcard_len < item[i].len)
> +			pathspec->has_wildcard = 1;
> +		pathspec->magic |= item[i].magic;
> +	}
> +
> +	qsort(pathspec->items, pathspec->nr,
> +	      sizeof(struct pathspec_item), pathspec_item_cmp);
>  }
>  
>  /*
>   * N.B. get_pathspec() is deprecated in favor of the "struct pathspec"
> - * based interface - see pathspec_magic above.
> + * based interface - see pathspec.c:parse_pathspec().
>   *
>   * Arguments:
>   *  - prefix - a path relative to the root of the working tree
> @@ -221,32 +334,11 @@ static const char *prefix_pathspec(const char *prefix, int prefixlen, const char
>   */
>  const char **get_pathspec(const char *prefix, const char **pathspec)
>  {
> -	const char *entry = *pathspec;
> -	const char **src, **dst;
> -	int prefixlen;
> -
> -	if (!prefix && !entry)
> -		return NULL;
> -
> -	if (!entry) {
> -		static const char *spec[2];
> -		spec[0] = prefix;
> -		spec[1] = NULL;
> -		return spec;
> -	}
> -
> -	/* Otherwise we have to re-write the entries.. */
> -	src = pathspec;
> -	dst = pathspec;
> -	prefixlen = prefix ? strlen(prefix) : 0;
> -	while (*src) {
> -		*(dst++) = prefix_pathspec(prefix, prefixlen, *src);
> -		src++;
> -	}
> -	*dst = NULL;
> -	if (!*pathspec)
> -		return NULL;
> -	return pathspec;
> +	struct pathspec ps;
> +	parse_pathspec(&ps,
> +		       PATHSPEC_ALL_MAGIC & ~PATHSPEC_FROMTOP,
> +		       0, prefix, pathspec);
> +	return ps.raw;
>  }
>  
>  void copy_pathspec(struct pathspec *dst, const struct pathspec *src)
> diff --git a/pathspec.h b/pathspec.h
> index a621676..937ec91 100644
> --- a/pathspec.h
> +++ b/pathspec.h
> @@ -1,6 +1,10 @@
>  #ifndef PATHSPEC_H
>  #define PATHSPEC_H
>  
> +/* Pathspec magic */
> +#define PATHSPEC_FROMTOP	(1<<0)
> +#define PATHSPEC_ALL_MAGIC PATHSPEC_FROMTOP
> +
>  #define PATHSPEC_ONESTAR 1	/* the pathspec pattern sastisfies GFNM_ONESTAR */
>  
>  struct pathspec {
> @@ -8,9 +12,11 @@ struct pathspec {
>  	int nr;
>  	unsigned int has_wildcard:1;
>  	unsigned int recursive:1;
> +	unsigned magic;
>  	int max_depth;
>  	struct pathspec_item {
>  		const char *match;
> +		unsigned magic;
>  		int len;
>  		int nowildcard_len;
>  		int flags;
> @@ -18,6 +24,11 @@ struct pathspec {
>  };
>  
>  extern int init_pathspec(struct pathspec *, const char **);
> +extern void parse_pathspec(struct pathspec *pathspec,
> +			   unsigned magic_mask,
> +			   unsigned flags,
> +			   const char *prefix,
> +			   const char **args);
>  extern void copy_pathspec(struct pathspec *dst, const struct pathspec *src);
>  extern void free_pathspec(struct pathspec *);

Re: [PATCH v2 00/45] parse_pathspec and :(glob) magic

[Duy Nguyen]

On Thu, Mar 21, 2013 at 10:50:02AM -0700, Junio C Hamano wrote:
> > Why could the test pass for you without it?  It doesn't look like a
> > bug that depended on uninitialized memory or something from the
> > above observation.

It depends on uninitialized memory. For absolute paths, prefix is
useless and I should have set the useful prefix length to zero, but I
did not. Later in prefix_pathspec, I rely on this value to set
nowildcard_len without checking if it's sane. The actual pathspec
after prefix_pathspec is "src" (length of 3) but nowildcard_len is 5.

In common_prefix_len(), I use nowildcard_len without sanity checks. So
the code examines 's', 'r', 'c', '\0', '<random>'. In my case,
'<random>' has never been '/'. I guess yours is '/' (which leads to
wrong common prefix length).

I've added an assert() to make sure nowildcard_len and prefix have
sane values before exiting prefix_pathspec. This assert() chokes at
t7300.8 for me.

> The change made to prefix_path_gently() in this series is beyond
> "disgusting", especially with the above fix-up.
> 
> Sometimes it uses the original "len", sometimes it uses the fixed-up
> *p_len (e.g. passes it down to normalize_path_copy_len()), and lets
> normalize_path_copy_len() further update it, and thenit makes the
> caller use the updated *p_len.
> 
> Does the caller know what the value in *p_len _mean_ after this
> function returns?  Can it afford to lose the original length of the
> prefix it saved in a variable, without getting confused?
> 
> I think any change that turns a value-passed argument in the
> existing code into modifiable pointer-to-variable in this series
> should add in-code comment to describe what the variable mean upon
> entry and after return, just like normalize_path_copy_len() that was
> built out of the original normalize_path_copy().  I didn't look if
> there are many others, or if this is the only one that is tricky. it
> is tricky that even the original author of the patch got it wrong
> X-<.
> 

The author of the patch totally forgot that prefix has nothing to do
with prefix. How about this? The prefix length is passed as value as
before. A separate pointer is for passing back the actual prefix
length. You can pull the actual patch from

https://github.com/pclouds/git parse-pathspec

which also includes all document bugs reported so far.

-- 8< --
diff --git a/pathspec.c b/pathspec.c
index 0771e48..126771c 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -205,7 +205,7 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 		match = xstrdup(copyfrom);
 		prefixlen = 0;
 	} else {
-		match = prefix_path_gently(prefix, &prefixlen, copyfrom);
+		match = prefix_path_gently(prefix, prefixlen, &prefixlen, copyfrom);
 		if (!match)
 			die("%s: '%s' is outside repository", elt, copyfrom);
 	}
@@ -284,6 +284,10 @@ static unsigned prefix_pathspec(struct pathspec_item *item,
 		    no_wildcard(item->match + item->nowildcard_len + 1))
 			item->flags |= PATHSPEC_ONESTAR;
 	}
+
+	/* sanity checks, pathspec matchers assume these are sane */
+	assert(item->nowildcard_len <= item->len &&
+	       item->prefix         <= item->len);
 	return magic;
 }
 
@@ -315,7 +319,7 @@ static void NORETURN unsupported_magic(const char *pattern,
 		n++;
 	}
 	/*
-	 * We may want to substitue "this command" with a command
+	 * We may want to substitute "this command" with a command
 	 * name. E.g. when add--interactive dies when running
 	 * "checkout -p"
 	 */
diff --git a/setup.c b/setup.c
index e59146b..6cf2bc6 100644
--- a/setup.c
+++ b/setup.c
@@ -5,24 +5,37 @@
 static int inside_git_dir = -1;
 static int inside_work_tree = -1;
 
-char *prefix_path_gently(const char *prefix, int *p_len, const char *path)
+/*
+ * Normalize "path", prepending the "prefix" for relative paths. If
+ * remaining_prefix is not NULL, return the actual prefix still
+ * remains in the path. For example, prefix = sub1/sub2/ and path is
+ *
+ *  foo          -> sub1/sub2/foo  (full prefix)
+ *  ../foo       -> sub1/foo       (remaining prefix is sub1/)
+ *  ../../bar    -> bar            (no remaining prefix)
+ *  ../../sub1/sub2/foo -> sub1/sub2/foo (but no remaining prefix)
+ *  `pwd`/../bar -> sub1/bar       (no remaining prefix)
+ */
+char *prefix_path_gently(const char *prefix, int len,
+			 int *remaining_prefix, const char *path)
 {
 	const char *orig = path;
 	char *sanitized;
-	int len = *p_len;
 	if (is_absolute_path(orig)) {
 		const char *temp = real_path(path);
 		sanitized = xmalloc(len + strlen(temp) + 1);
 		strcpy(sanitized, temp);
-		if (p_len)
-			*p_len = 0;
+		if (remaining_prefix)
+			*remaining_prefix = 0;
 	} else {
 		sanitized = xmalloc(len + strlen(path) + 1);
 		if (len)
 			memcpy(sanitized, prefix, len);
 		strcpy(sanitized + len, path);
+		if (remaining_prefix)
+			*remaining_prefix = len;
 	}
-	if (normalize_path_copy_len(sanitized, sanitized, p_len))
+	if (normalize_path_copy_len(sanitized, sanitized, remaining_prefix))
 		goto error_out;
 	if (is_absolute_path(orig)) {
 		size_t root_len, len, total;
@@ -47,7 +60,7 @@ char *prefix_path_gently(const char *prefix, int *p_len, const char *path)
 
 char *prefix_path(const char *prefix, int len, const char *path)
 {
-	char *r = prefix_path_gently(prefix, &len, path);
+	char *r = prefix_path_gently(prefix, len, NULL, path);
 	if (!r)
 		die("'%s' is outside repository", path);
 	return r;
@@ -56,7 +69,7 @@ char *prefix_path(const char *prefix, int len, const char *path)
 int path_inside_repo(const char *prefix, const char *path)
 {
 	int len = prefix ? strlen(prefix) : 0;
-	char *r = prefix_path_gently(prefix, &len, path);
+	char *r = prefix_path_gently(prefix, len, NULL, path);
 	if (r) {
 		free(r);
 		return 1;
-- 8< --

Re: [PATCH v2 00/45] parse_pathspec and :(glob) magic

[Duy Nguyen]

On Sat, Mar 23, 2013 at 10:13 AM, Duy Nguyen <pclouds@gmail.com> wrote:
> which also includes all document bugs reported so far.

s/all/fixes for all/
-- 
Duy

Re: [PATCH v2 00/45] parse_pathspec and :(glob) magic

[Eric Sunshine]

On Fri, Mar 22, 2013 at 11:13 PM, Duy Nguyen <pclouds@gmail.com> wrote:
> diff --git a/setup.c b/setup.c
> index e59146b..6cf2bc6 100644
> --- a/setup.c
> +++ b/setup.c
> @@ -5,24 +5,37 @@
>  static int inside_git_dir = -1;
>  static int inside_work_tree = -1;
>
> -char *prefix_path_gently(const char *prefix, int *p_len, const char *path)
> +/*
> + * Normalize "path", prepending the "prefix" for relative paths. If
> + * remaining_prefix is not NULL, return the actual prefix still
> + * remains in the path. For example, prefix = sub1/sub2/ and path is

s/still remains/still remaining/
...or...
s/still remains/which still remains/
...or...
s/still remains/which remains/

Re: [PATCH v2 00/45] parse_pathspec and :(glob) magic

[Junio C Hamano]

Duy Nguyen <pclouds@gmail.com> writes:

> On Sat, Mar 23, 2013 at 10:13 AM, Duy Nguyen <pclouds@gmail.com> wrote:
>> which also includes all document bugs reported so far.
>
> s/all/fixes for all/

Slurped from your github repository and didn't find anything
questionable relative to the original series that was posted here.

Thanks.  Will requeue.

Re: [PATCH v1 23/45] check-ignore: convert to use parse_pathspec

[Adam Spiers]

On Fri, Mar 15, 2013 at 01:06:38PM +0700, Nguyễn Thái Ngọc Duy wrote:
> check-ignore (at least the test suite) seems to rely on the pattern
> order. PATHSPEC_KEEP_ORDER is introduced to explictly express this.
> The lack of PATHSPEC_MAXDEPTH_VALID is sufficient because it's the
> only flag that reorders pathspecs, but it's less obvious that way.

Sorry for the slow response - I only just noticed this today.  (It
would be useful if any future patches to check-ignore Cc: me
explicitly, to catch my mail filters.)

> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
>  builtin/check-ignore.c | 34 +++++++++++++++++++++-------------
>  pathspec.c             |  6 +++++-
>  pathspec.h             |  1 +
>  t/t0008-ignores.sh     |  8 ++++----
>  4 files changed, 31 insertions(+), 18 deletions(-)
> 
> diff --git a/builtin/check-ignore.c b/builtin/check-ignore.c
> index 0240f99..6e55f06 100644
> --- a/builtin/check-ignore.c
> +++ b/builtin/check-ignore.c
> @@ -53,14 +53,14 @@ static void output_exclude(const char *path, struct exclude *exclude)
>  	}
>  }
>  
> -static int check_ignore(const char *prefix, const char **pathspec)
> +static int check_ignore(int argc, const char **argv, const char *prefix)
>  {
>  	struct dir_struct dir;
> -	const char *path, *full_path;
>  	char *seen;
>  	int num_ignored = 0, dtype = DT_UNKNOWN, i;
>  	struct path_exclude_check check;
>  	struct exclude *exclude;
> +	struct pathspec pathspec;
>  
>  	/* read_cache() is only necessary so we can watch out for submodules. */
>  	if (read_cache() < 0)
> @@ -70,31 +70,39 @@ static int check_ignore(const char *prefix, const char **pathspec)
>  	dir.flags |= DIR_COLLECT_IGNORED;
>  	setup_standard_excludes(&dir);
>  
> -	if (!pathspec || !*pathspec) {
> +	if (!argc) {

Is there a compelling reason for introducing argc as a new parameter
to check_ignore(), other than simplifying the above line?  And why
rename the pathspec parameter to argv?  Both these changes are
misleading AFAICS, since paths provided to check_ignore() can come
from sources other than CLI arguments (i.e. via --stdin).  

The introduction of argc also makes it possible to invoke
check_ignore() with arguments which are not self-consistent.

I haven't been following your pathspec work, but FWIW the other
changes in this patch look reasonable at a glance.

Thanks,
Adam

Re: [PATCH v1 23/45] check-ignore: convert to use parse_pathspec

[Duy Nguyen]

On Sat, Apr 13, 2013 at 1:03 AM, Adam Spiers <git@adamspiers.org> wrote:
>> -static int check_ignore(const char *prefix, const char **pathspec)
>> +static int check_ignore(int argc, const char **argv, const char *prefix)
>>  {
>>       struct dir_struct dir;
>> -     const char *path, *full_path;
>>       char *seen;
>>       int num_ignored = 0, dtype = DT_UNKNOWN, i;
>>       struct path_exclude_check check;
>>       struct exclude *exclude;
>> +     struct pathspec pathspec;
>>
>>       /* read_cache() is only necessary so we can watch out for submodules. */
>>       if (read_cache() < 0)
>> @@ -70,31 +70,39 @@ static int check_ignore(const char *prefix, const char **pathspec)
>>       dir.flags |= DIR_COLLECT_IGNORED;
>>       setup_standard_excludes(&dir);
>>
>> -     if (!pathspec || !*pathspec) {
>> +     if (!argc) {
>
> Is there a compelling reason for introducing argc as a new parameter
> to check_ignore(), other than simplifying the above line?  And why
> rename the pathspec parameter to argv?  Both these changes are
> misleading AFAICS, since paths provided to check_ignore() can come
> from sources other than CLI arguments (i.e. via --stdin).

Because I introduced "struct pathspec pathspec;" I need to rename the
argument "pathspec" to something else. Maybe we could rename the
argument to "paths"?

> The introduction of argc also makes it possible to invoke
> check_ignore() with arguments which are not self-consistent.

This is the same problem with main() and other places that follow this
convention. But I don't mind dropping argc either.
-- 
Duy

Re: [PATCH v1 23/45] check-ignore: convert to use parse_pathspec

[Adam Spiers]

On Sat, Apr 13, 2013 at 09:09:33AM +1000, Duy Nguyen wrote:
> On Sat, Apr 13, 2013 at 1:03 AM, Adam Spiers <git@adamspiers.org> wrote:
> >> -static int check_ignore(const char *prefix, const char **pathspec)
> >> +static int check_ignore(int argc, const char **argv, const char *prefix)
> >>  {
> >>       struct dir_struct dir;
> >> -     const char *path, *full_path;
> >>       char *seen;
> >>       int num_ignored = 0, dtype = DT_UNKNOWN, i;
> >>       struct path_exclude_check check;
> >>       struct exclude *exclude;
> >> +     struct pathspec pathspec;
> >>
> >>       /* read_cache() is only necessary so we can watch out for submodules. */
> >>       if (read_cache() < 0)
> >> @@ -70,31 +70,39 @@ static int check_ignore(const char *prefix, const char **pathspec)
> >>       dir.flags |= DIR_COLLECT_IGNORED;
> >>       setup_standard_excludes(&dir);
> >>
> >> -     if (!pathspec || !*pathspec) {
> >> +     if (!argc) {
> >
> > Is there a compelling reason for introducing argc as a new parameter
> > to check_ignore(), other than simplifying the above line?  And why
> > rename the pathspec parameter to argv?  Both these changes are
> > misleading AFAICS, since paths provided to check_ignore() can come
> > from sources other than CLI arguments (i.e. via --stdin).
> 
> Because I introduced "struct pathspec pathspec;" I need to rename the
> argument "pathspec" to something else.

Ah, I see - that makes sense :-)

> Maybe we could rename the argument to "paths"?

Sounds fine to me.

> > The introduction of argc also makes it possible to invoke
> > check_ignore() with arguments which are not self-consistent.
> 
> This is the same problem with main()

How could main() be invoked with argc inconsistent with argv?

> and other places that follow this convention.  But I don't mind
> dropping argc either.

What is the reason for that convention?  I'm willing to be persuaded
either way.

Re: [PATCH v1 23/45] check-ignore: convert to use parse_pathspec

[Duy Nguyen]

On Mon, Apr 15, 2013 at 9:25 AM, Adam Spiers <git@adamspiers.org> wrote:
>> > The introduction of argc also makes it possible to invoke
>> > check_ignore() with arguments which are not self-consistent.
>>
>> This is the same problem with main()
>
> How could main() be invoked with argc inconsistent with argv?

The point is main's caller has to maintain the consistency. So do
check_ignore's callers.
-- 
Duy

Re: [PATCH v1 23/45] check-ignore: convert to use parse_pathspec

[Adam Spiers]

On Mon, Apr 15, 2013 at 09:48:22AM +1000, Duy Nguyen wrote:
> On Mon, Apr 15, 2013 at 9:25 AM, Adam Spiers <git@adamspiers.org> wrote:
> >> > The introduction of argc also makes it possible to invoke
> >> > check_ignore() with arguments which are not self-consistent.
> >>
> >> This is the same problem with main()
> >
> > How could main() be invoked with argc inconsistent with argv?
> 
> The point is main's caller has to maintain the consistency. So do
> check_ignore's callers.

But only the system runtime calls main(), right?  So we can probably
rely on it being called in a consistent manner ;-)