[PATCH/POC 0/7] Support multiple worktrees

[PATCH/POC 0/7] Support multiple worktrees

[Nguyễn Thái Ngọc Duy]

This is what I imagine multi worktree support (aka git-new-workdir)
looks like. Basically two variables will control access to the repo:
$GIT_SUPER_DIR for worktree specific stuff and $GIT_DIR for the rest.

I like the idea of using git_path() to hide path relocation caused by
GIT_SUPER_DIR, but I may have made some design mistakes here..
setup_git_directory() changes are hairy. It's not surprise if I broke
something in there. Not important for a PoC though.

Final series may be a few patches longer as I only lay the foundation
in this series.

Nguyễn Thái Ngọc Duy (7):
  Make git_path() beware of file relocation in $GIT_DIR
  Add new environment variable $GIT_SUPER_DIR
  setup.c: add split-repo support to .git files
  setup.c: add split-repo support to is_git_directory()
  setup.c: reduce cleanup sites in setup_explicit_git_dir()
  setup.c: add split-repo support to setup_git_directory*
  init: add --split-repo with the same functionality as git-new-workdir

 builtin/init-db.c     |  42 +++++++++++++++
 cache.h               |   5 ++
 environment.c         |  37 ++++++++++++--
 path.c                |  45 ++++++++++++++--
 setup.c               | 139 ++++++++++++++++++++++++++++++++++----------------
 t/t0060-path-utils.sh | 115 +++++++++++++++++++++++++++++++++++++++++
 test-path-utils.c     |   7 +++
 trace.c               |   1 +
 8 files changed, 339 insertions(+), 52 deletions(-)

-- 
1.8.5.1.77.g42c48fa

[PATCH/POC 1/7] Make git_path() beware of file relocation in $GIT_DIR

[Nguyễn Thái Ngọc Duy]

We allow the user to relocate certain paths out of $GIT_DIR via
environment variables, e.g. GIT_OBJECT_DIRECTORY, GIT_INDEX_FILE and
GIT_GRAFT_FILE. All callers are not supposed to use git_path() or
git_pathdup() to get those paths. Instead they must use
get_object_directory(), get_index_file() and get_graft_file()
respectively. This is inconvenient and could be missed in review
(there's git_path("objects/info/alternates") somewhere in
sha1_file.c).

This patch makes git_path() and git_pathdup() understand those
environment variables. So if you set GIT_OBJECT_DIRECTORY to /foo/bar,
git_path("objects/abc") should return /tmp/bar/abc. The same is done
for the two remaining env variables.

XXX trim consecutive slashes

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 cache.h               |  1 +
 environment.c         |  9 +++++++--
 path.c                | 35 ++++++++++++++++++++++++++++++++---
 t/t0060-path-utils.sh | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
 test-path-utils.c     |  7 +++++++
 5 files changed, 95 insertions(+), 5 deletions(-)

diff --git a/cache.h b/cache.h
index ce377e1..cdafbd7 100644
--- a/cache.h
+++ b/cache.h
@@ -584,6 +584,7 @@ extern int fsync_object_files;
 extern int core_preload_index;
 extern int core_apply_sparse_checkout;
 extern int precomposed_unicode;
+extern int git_db_env, git_index_env, git_graft_env;
 
 /*
  * The character that begins a commented line in user-editable file
diff --git a/environment.c b/environment.c
index 0a15349..1d74dde 100644
--- a/environment.c
+++ b/environment.c
@@ -81,6 +81,7 @@ static size_t namespace_len;
 
 static const char *git_dir;
 static char *git_object_dir, *git_index_file, *git_graft_file;
+int git_db_env, git_index_env, git_graft_env;
 
 /*
  * Repository-local GIT_* environment variables; see cache.h for details.
@@ -134,15 +135,19 @@ static void setup_git_env(void)
 	if (!git_object_dir) {
 		git_object_dir = xmalloc(strlen(git_dir) + 9);
 		sprintf(git_object_dir, "%s/objects", git_dir);
-	}
+	} else
+		git_db_env = 1;
 	git_index_file = getenv(INDEX_ENVIRONMENT);
 	if (!git_index_file) {
 		git_index_file = xmalloc(strlen(git_dir) + 7);
 		sprintf(git_index_file, "%s/index", git_dir);
-	}
+	} else
+		git_index_env = 1;
 	git_graft_file = getenv(GRAFT_ENVIRONMENT);
 	if (!git_graft_file)
 		git_graft_file = git_pathdup("info/grafts");
+	else
+		git_graft_env = 1;
 	if (getenv(NO_REPLACE_OBJECTS_ENVIRONMENT))
 		read_replace_refs = 0;
 	namespace = expand_namespace(getenv(GIT_NAMESPACE_ENVIRONMENT));
diff --git a/path.c b/path.c
index 24594c4..eda9176 100644
--- a/path.c
+++ b/path.c
@@ -49,10 +49,38 @@ char *mksnpath(char *buf, size_t n, const char *fmt, ...)
 	return cleanup_path(buf);
 }
 
+static int dir_prefix(const char *buf, const char *dir)
+{
+	int len = strlen(dir);
+	return !strncmp(buf, dir, len) &&
+		(is_dir_sep(buf[len]) || buf[len] == '\0');
+}
+
+static void replace_dir(char *buf, int len, const char *newdir)
+{
+	int newlen = strlen(newdir);
+	int buflen = strlen(buf);
+	memmove(buf + newlen + 1, buf + len, buflen - len + 1);
+	memcpy(buf, newdir, newlen);
+	buf[newlen] = '/';
+}
+
+static void adjust_git_path(char *buf, int git_dir_len)
+{
+	/* XXX buffer overflow */
+	char *base = buf + git_dir_len;
+	if (git_graft_env && !strcmp(base, "info/grafts"))
+		strcpy(buf, get_graft_file());
+	else if (git_index_env && !strcmp(base, "index"))
+		strcpy(buf, get_index_file());
+	else if (git_db_env && dir_prefix(base, "objects"))
+		replace_dir(buf, git_dir_len + 7, get_object_directory());
+}
+
 static char *vsnpath(char *buf, size_t n, const char *fmt, va_list args)
 {
 	const char *git_dir = get_git_dir();
-	size_t len;
+	size_t len, total_len;
 
 	len = strlen(git_dir);
 	if (n < len + 1)
@@ -60,9 +88,10 @@ static char *vsnpath(char *buf, size_t n, const char *fmt, va_list args)
 	memcpy(buf, git_dir, len);
 	if (len && !is_dir_sep(git_dir[len-1]))
 		buf[len++] = '/';
-	len += vsnprintf(buf + len, n - len, fmt, args);
-	if (len >= n)
+	total_len = len + vsnprintf(buf + len, n - len, fmt, args);
+	if (total_len >= n)
 		goto bad;
+	adjust_git_path(buf, len);
 	return cleanup_path(buf);
 bad:
 	strlcpy(buf, bad_path, n);
diff --git a/t/t0060-path-utils.sh b/t/t0060-path-utils.sh
index 07c10c8..7ad2730 100755
--- a/t/t0060-path-utils.sh
+++ b/t/t0060-path-utils.sh
@@ -223,4 +223,52 @@ relative_path "<null>"		"<empty>"	./
 relative_path "<null>"		"<null>"	./
 relative_path "<null>"		/foo/a/b	./
 
+test_expect_success 'git_path info/grafts without GIT_GRAFT_FILE' '
+	test-path-utils git_path info/grafts >actual1 &&
+	echo .git/info/grafts >expect1 &&
+	test_cmp expect1 actual1
+'
+
+test_expect_success 'git_path info/grafts with GIT_GRAFT_FILE' '
+	GIT_GRAFT_FILE=foo test-path-utils git_path info/grafts >actual2 &&
+	echo foo >expect2 &&
+	test_cmp expect2 actual2
+'
+
+test_expect_success 'git_path index' '
+	GIT_INDEX_FILE=foo test-path-utils git_path index >actual &&
+	echo foo >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path index/foo' '
+	GIT_INDEX_FILE=foo test-path-utils git_path index/foo >actual &&
+	echo .git/index/foo >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path index2' '
+	GIT_INDEX_FILE=foo test-path-utils git_path index2 >actual &&
+	echo .git/index2 >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path objects' '
+	GIT_OBJECT_DIRECTORY=foo test-path-utils git_path objects >actual &&
+	echo foo/ >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path objects/foo' '
+	GIT_OBJECT_DIRECTORY=foo test-path-utils git_path objects/foo >actual &&
+	echo foo//foo >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path objects2' '
+	GIT_OBJECT_DIRECTORY=foo test-path-utils git_path objects2 >actual &&
+	echo .git/objects2 >expect &&
+	test_cmp expect actual
+'
+
 test_done
diff --git a/test-path-utils.c b/test-path-utils.c
index 3dd3744..55e4fe0 100644
--- a/test-path-utils.c
+++ b/test-path-utils.c
@@ -135,6 +135,13 @@ int main(int argc, char **argv)
 		return 0;
 	}
 
+	if (argc == 3 && !strcmp(argv[1], "git_path")) {
+		int nongit_ok = 0;
+		setup_git_directory_gently(&nongit_ok);
+		puts(git_path("%s", argv[2]));
+		return 0;
+	}
+
 	fprintf(stderr, "%s: unknown function name: %s\n", argv[0],
 		argv[1] ? argv[1] : "(there was none)");
 	return 1;
-- 
1.8.5.1.77.g42c48fa

[PATCH/POC 2/7] Add new environment variable $GIT_SUPER_DIR

[Nguyễn Thái Ngọc Duy]

This is the base for git-new-workdir integration. The git-new-workdir
script creates a separate worktree that shares everything except
worktree-related stuff. The sharing is eanbled by this new env
variable.

In the new worktree, both variables are set at the same time, GIT_DIR
and GIT_SUPER_DIR. Shared paths are checked at adjust_git_path() and
rewritten to use $GIT_SUPER_DIR instead of $GIT_DIR. Let's call this
"split-repo" setup to distinguish from $GIT_DIR-only one.

The "ln -s" is avoided because Windows probably does not have the
support, and symlinks don't survive operations that delete the old
file, then create a new one.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 cache.h               |  2 ++
 environment.c         | 11 +++++++--
 path.c                | 10 ++++++++
 t/t0060-path-utils.sh | 67 +++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 88 insertions(+), 2 deletions(-)

diff --git a/cache.h b/cache.h
index cdafbd7..823582f 100644
--- a/cache.h
+++ b/cache.h
@@ -347,6 +347,7 @@ static inline enum object_type object_type(unsigned int mode)
 
 /* Double-check local_repo_env below if you add to this list. */
 #define GIT_DIR_ENVIRONMENT "GIT_DIR"
+#define GIT_SUPER_DIR_ENVIRONMENT "GIT_SUPER_DIR"
 #define GIT_NAMESPACE_ENVIRONMENT "GIT_NAMESPACE"
 #define GIT_WORK_TREE_ENVIRONMENT "GIT_WORK_TREE"
 #define GIT_PREFIX_ENVIRONMENT "GIT_PREFIX"
@@ -399,6 +400,7 @@ extern int is_inside_git_dir(void);
 extern char *git_work_tree_cfg;
 extern int is_inside_work_tree(void);
 extern const char *get_git_dir(void);
+extern const char *get_git_super_dir(void);
 extern int is_git_directory(const char *path);
 extern char *get_object_directory(void);
 extern char *get_index_file(void);
diff --git a/environment.c b/environment.c
index 1d74dde..d5ae7a3 100644
--- a/environment.c
+++ b/environment.c
@@ -79,7 +79,7 @@ static char *work_tree;
 static const char *namespace;
 static size_t namespace_len;
 
-static const char *git_dir;
+static const char *git_dir, *git_super_dir;
 static char *git_object_dir, *git_index_file, *git_graft_file;
 int git_db_env, git_index_env, git_graft_env;
 
@@ -131,10 +131,12 @@ static void setup_git_env(void)
 		git_dir = DEFAULT_GIT_DIR_ENVIRONMENT;
 	gitfile = read_gitfile(git_dir);
 	git_dir = xstrdup(gitfile ? gitfile : git_dir);
+	git_super_dir = getenv(GIT_SUPER_DIR_ENVIRONMENT);
 	git_object_dir = getenv(DB_ENVIRONMENT);
 	if (!git_object_dir) {
 		git_object_dir = xmalloc(strlen(git_dir) + 9);
-		sprintf(git_object_dir, "%s/objects", git_dir);
+		sprintf(git_object_dir, "%s/objects",
+			git_super_dir ? git_super_dir : git_dir);
 	} else
 		git_db_env = 1;
 	git_index_file = getenv(INDEX_ENVIRONMENT);
@@ -167,6 +169,11 @@ const char *get_git_dir(void)
 	return git_dir;
 }
 
+const char *get_git_super_dir(void)
+{
+	return git_super_dir;
+}
+
 const char *get_git_namespace(void)
 {
 	if (!namespace)
diff --git a/path.c b/path.c
index eda9176..86a7c15 100644
--- a/path.c
+++ b/path.c
@@ -75,6 +75,16 @@ static void adjust_git_path(char *buf, int git_dir_len)
 		strcpy(buf, get_index_file());
 	else if (git_db_env && dir_prefix(base, "objects"))
 		replace_dir(buf, git_dir_len + 7, get_object_directory());
+	else if (get_git_super_dir()) {
+		if (dir_prefix(base, "objects") || dir_prefix(base, "info") ||
+		    dir_prefix(base, "refs") ||
+		    (dir_prefix(base, "logs") && strcmp(base, "logs/HEAD")) ||
+		    dir_prefix(base, "rr-cache") || dir_prefix(base, "hooks") ||
+		    dir_prefix(base, "svn") ||
+		    !strcmp(base, "config") || !strcmp(base, "packed-refs") ||
+		    !strcmp(base, "shallow"))
+			replace_dir(buf, git_dir_len, get_git_super_dir());
+	}
 }
 
 static char *vsnpath(char *buf, size_t n, const char *fmt, va_list args)
diff --git a/t/t0060-path-utils.sh b/t/t0060-path-utils.sh
index 7ad2730..45f114c 100755
--- a/t/t0060-path-utils.sh
+++ b/t/t0060-path-utils.sh
@@ -271,4 +271,71 @@ test_expect_success 'git_path objects2' '
 	test_cmp expect actual
 '
 
+test_expect_success 'git_path super index' '
+	GIT_SUPER_DIR=foo test-path-utils git_path index >actual &&
+	echo .git/index >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path super HEAD' '
+	GIT_SUPER_DIR=foo test-path-utils git_path HEAD >actual &&
+	echo .git/HEAD >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path super objects/*' '
+	GIT_SUPER_DIR=foo test-path-utils git_path objects/foo >actual &&
+	echo foo/objects/foo >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path super info/*' '
+	GIT_SUPER_DIR=foo test-path-utils git_path info/exclude >actual &&
+	echo foo/info/exclude >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path super logs/refs/heads/master' '
+	GIT_SUPER_DIR=foo test-path-utils git_path logs/refs/heads/master >actual &&
+	echo foo/logs/refs/heads/master >expect &&
+	test_cmp expect actual
+'
+
+
+test_expect_success 'git_path super refs/heads/master' '
+	GIT_SUPER_DIR=foo test-path-utils git_path refs/heads/master >actual &&
+	echo foo/refs/heads/master >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path super logs/HEAD' '
+	GIT_SUPER_DIR=foo test-path-utils git_path logs/HEAD >actual &&
+	echo .git/logs/HEAD >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path super hooks/me' '
+	GIT_SUPER_DIR=foo test-path-utils git_path hooks/me >actual &&
+	echo foo/hooks/me >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path super config' '
+	GIT_SUPER_DIR=foo test-path-utils git_path config >actual &&
+	echo foo/config >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path super packed-refs' '
+	GIT_SUPER_DIR=foo test-path-utils git_path packed-refs >actual &&
+	echo foo/packed-refs >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path super shallow' '
+	GIT_SUPER_DIR=foo test-path-utils git_path shallow >actual &&
+	echo foo/shallow >expect &&
+	test_cmp expect actual
+'
+
 test_done
-- 
1.8.5.1.77.g42c48fa

[PATCH/POC 3/7] setup.c: add split-repo support to .git files

[Nguyễn Thái Ngọc Duy]

If a .git file contains

gitsuper: <path>
gitdir: <id>

then we set GIT_SUPER_DIR to <path> and GIT_DIR to
$GIT_SUPER_DIR/repos/<id>.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 cache.h |  1 +
 setup.c | 40 +++++++++++++++++++++++++++++++++-------
 2 files changed, 34 insertions(+), 7 deletions(-)

diff --git a/cache.h b/cache.h
index 823582f..f85ee70 100644
--- a/cache.h
+++ b/cache.h
@@ -410,6 +410,7 @@ extern const char *get_git_namespace(void);
 extern const char *strip_namespace(const char *namespaced_ref);
 extern const char *get_git_work_tree(void);
 extern const char *read_gitfile(const char *path);
+extern const char *read_gitfile_super(const char *path, char **super);
 extern const char *resolve_gitdir(const char *suspect);
 extern void set_git_work_tree(const char *tree);
 
diff --git a/setup.c b/setup.c
index 5432a31..84362a6 100644
--- a/setup.c
+++ b/setup.c
@@ -281,16 +281,23 @@ static int check_repository_format_gently(const char *gitdir, int *nongit_ok)
 /*
  * Try to read the location of the git directory from the .git file,
  * return path to git directory if found.
+ *
+ * If "gitsuper: " line is found and super is not NULL, *super points
+ * to the absolute path of the given path. The next line contains the
+ * repo id.
  */
-const char *read_gitfile(const char *path)
+const char *read_gitfile_super(const char *path, char **super)
 {
-	char *buf;
+	struct strbuf sb = STRBUF_INIT;
+	char *buf, *to_free;
 	char *dir;
 	const char *slash;
 	struct stat st;
 	int fd;
 	ssize_t len;
 
+	if (super)
+		*super = NULL;
 	if (stat(path, &st))
 		return NULL;
 	if (!S_ISREG(st.st_mode))
@@ -298,12 +305,19 @@ const char *read_gitfile(const char *path)
 	fd = open(path, O_RDONLY);
 	if (fd < 0)
 		die_errno("Error opening '%s'", path);
-	buf = xmalloc(st.st_size + 1);
+	to_free = buf = xmalloc(st.st_size + 1);
 	len = read_in_full(fd, buf, st.st_size);
 	close(fd);
 	if (len != st.st_size)
 		die("Error reading %s", path);
 	buf[len] = '\0';
+	if (super &&!prefixcmp(buf, "gitsuper: ")) {
+		char *p = strchr(buf, '\n');
+		*super = buf + strlen("gitsuper: ");
+		*p = '\0';
+		len -= (p + 1) - buf;
+		buf = p + 1;
+	}
 	if (prefixcmp(buf, "gitdir: "))
 		die("Invalid gitfile format: %s", path);
 	while (buf[len - 1] == '\n' || buf[len - 1] == '\r')
@@ -312,6 +326,11 @@ const char *read_gitfile(const char *path)
 		die("No path in gitfile: %s", path);
 	buf[len] = '\0';
 	dir = buf + 8;
+	if (super && *super) {
+		strbuf_addf(&sb, "%s/repos/%s", *super, dir);
+		dir = sb.buf;
+		*super = xstrdup(real_path(*super));
+	}
 
 	if (!is_absolute_path(dir) && (slash = strrchr(path, '/'))) {
 		size_t pathlen = slash+1 - path;
@@ -320,18 +339,25 @@ const char *read_gitfile(const char *path)
 		strncpy(dir, path, pathlen);
 		strncpy(dir + pathlen, buf + 8, len - 8);
 		dir[dirlen] = '\0';
-		free(buf);
-		buf = dir;
+		free(to_free);
+		to_free = buf = dir;
 	}
 
-	if (!is_git_directory(dir))
+	if (!is_git_directory_super(dir, super ? *super : dir))
 		die("Not a git repository: %s", dir);
 	path = real_path(dir);
 
-	free(buf);
+	free(to_free);
+	strbuf_release(&sb);
 	return path;
 }
 
+const char *read_gitfile(const char *path)
+{
+	return read_gitfile_super(path, NULL);
+}
+
+
 static const char *setup_explicit_git_dir(const char *gitdirenv,
 					  char *cwd, int len,
 					  int *nongit_ok)
-- 
1.8.5.1.77.g42c48fa

[PATCH/POC 4/7] setup.c: add split-repo support to is_git_directory()

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 setup.c | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/setup.c b/setup.c
index 84362a6..43f56fb 100644
--- a/setup.c
+++ b/setup.c
@@ -182,28 +182,31 @@ void verify_non_filename(const char *prefix, const char *arg)
  *    a proper "ref:", or a regular file HEAD that has a properly
  *    formatted sha1 object name.
  */
-int is_git_directory(const char *suspect)
+static int is_git_directory_super(const char *suspect, const char *super)
 {
 	char path[PATH_MAX];
 	size_t len = strlen(suspect);
+	size_t super_len = strlen(super);
 
-	if (PATH_MAX <= len + strlen("/objects"))
+	if (PATH_MAX <= super_len + strlen("/objects"))
 		die("Too long path: %.*s", 60, suspect);
-	strcpy(path, suspect);
+	strcpy(path, super);
 	if (getenv(DB_ENVIRONMENT)) {
 		if (access(getenv(DB_ENVIRONMENT), X_OK))
 			return 0;
 	}
 	else {
-		strcpy(path + len, "/objects");
+		strcpy(path + super_len, "/objects");
 		if (access(path, X_OK))
 			return 0;
 	}
 
-	strcpy(path + len, "/refs");
+	strcpy(path + super_len, "/refs");
 	if (access(path, X_OK))
 		return 0;
 
+	if (super != suspect)
+		strcpy(path, suspect);
 	strcpy(path + len, "/HEAD");
 	if (validate_headref(path))
 		return 0;
@@ -211,6 +214,12 @@ int is_git_directory(const char *suspect)
 	return 1;
 }
 
+int is_git_directory(const char *suspect)
+{
+	return is_git_directory_super(suspect, suspect);
+}
+
+
 int is_inside_git_dir(void)
 {
 	if (inside_git_dir < 0)
-- 
1.8.5.1.77.g42c48fa

[PATCH/POC 5/7] setup.c: reduce cleanup sites in setup_explicit_git_dir()

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 setup.c | 20 ++++++++------------
 1 file changed, 8 insertions(+), 12 deletions(-)

diff --git a/setup.c b/setup.c
index 43f56fb..dfe9d08 100644
--- a/setup.c
+++ b/setup.c
@@ -388,16 +388,13 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
 	if (!is_git_directory(gitdirenv)) {
 		if (nongit_ok) {
 			*nongit_ok = 1;
-			free(gitfile);
-			return NULL;
+			goto done_null;
 		}
 		die("Not a git repository: '%s'", gitdirenv);
 	}
 
-	if (check_repository_format_gently(gitdirenv, nongit_ok)) {
-		free(gitfile);
-		return NULL;
-	}
+	if (check_repository_format_gently(gitdirenv, nongit_ok))
+		goto done_null;
 
 	/* #3, #7, #11, #15, #19, #23, #27, #31 (see t1510) */
 	if (work_tree_env)
@@ -408,8 +405,7 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
 
 		/* #18, #26 */
 		set_git_dir(gitdirenv);
-		free(gitfile);
-		return NULL;
+		goto done_null;
 	}
 	else if (git_work_tree_cfg) { /* #6, #14 */
 		if (is_absolute_path(git_work_tree_cfg))
@@ -430,8 +426,7 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
 	else if (!git_env_bool(GIT_IMPLICIT_WORK_TREE_ENVIRONMENT, 1)) {
 		/* #16d */
 		set_git_dir(gitdirenv);
-		free(gitfile);
-		return NULL;
+		goto done_null;
 	}
 	else /* #2, #10 */
 		set_git_work_tree(".");
@@ -442,8 +437,7 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
 	/* both get_git_work_tree() and cwd are already normalized */
 	if (!strcmp(cwd, worktree)) { /* cwd == worktree */
 		set_git_dir(gitdirenv);
-		free(gitfile);
-		return NULL;
+		goto done_null;
 	}
 
 	offset = dir_inside_of(cwd, worktree);
@@ -459,6 +453,8 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
 
 	/* cwd outside worktree */
 	set_git_dir(gitdirenv);
+
+done_null:
 	free(gitfile);
 	return NULL;
 }
-- 
1.8.5.1.77.g42c48fa

[PATCH/POC 6/7] setup.c: add split-repo support to setup_git_directory*

[Nguyễn Thái Ngọc Duy]

XXX bare repos probably not worth supporting..

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 cache.h       |  1 +
 environment.c | 19 ++++++++++++++++--
 setup.c       | 62 +++++++++++++++++++++++++++++++++++++++--------------------
 trace.c       |  1 +
 4 files changed, 60 insertions(+), 23 deletions(-)

diff --git a/cache.h b/cache.h
index f85ee70..4c09223 100644
--- a/cache.h
+++ b/cache.h
@@ -406,6 +406,7 @@ extern char *get_object_directory(void);
 extern char *get_index_file(void);
 extern char *get_graft_file(void);
 extern int set_git_dir(const char *path);
+extern int set_git_dir_super(const char *path, const char *super);
 extern const char *get_git_namespace(void);
 extern const char *strip_namespace(const char *namespaced_ref);
 extern const char *get_git_work_tree(void);
diff --git a/environment.c b/environment.c
index d5ae7a3..8152c7e 100644
--- a/environment.c
+++ b/environment.c
@@ -125,13 +125,17 @@ static char *expand_namespace(const char *raw_namespace)
 static void setup_git_env(void)
 {
 	const char *gitfile;
+	char *super;
 
 	git_dir = getenv(GIT_DIR_ENVIRONMENT);
 	if (!git_dir)
 		git_dir = DEFAULT_GIT_DIR_ENVIRONMENT;
-	gitfile = read_gitfile(git_dir);
+	gitfile = read_gitfile_super(git_dir, &super);
 	git_dir = xstrdup(gitfile ? gitfile : git_dir);
-	git_super_dir = getenv(GIT_SUPER_DIR_ENVIRONMENT);
+	if (super)
+		git_super_dir = xstrdup(super);
+	else
+		git_super_dir = getenv(GIT_SUPER_DIR_ENVIRONMENT);
 	git_object_dir = getenv(DB_ENVIRONMENT);
 	if (!git_object_dir) {
 		git_object_dir = xmalloc(strlen(git_dir) + 9);
@@ -280,6 +284,17 @@ int set_git_dir(const char *path)
 	return 0;
 }
 
+int set_git_dir_super(const char *path, const char *super)
+{
+	if (super != path &&
+	    setenv(GIT_SUPER_DIR_ENVIRONMENT, super, 1))
+		return error("Could not set GIT_SUPER_DIR to '%s'", super);
+	if (setenv(GIT_DIR_ENVIRONMENT, path, 1))
+		return error("Could not set GIT_DIR to '%s'", path);
+	setup_git_env();
+	return 0;
+}
+
 const char *get_log_output_encoding(void)
 {
 	return git_log_output_encoding ? git_log_output_encoding
diff --git a/setup.c b/setup.c
index dfe9d08..f22381e 100644
--- a/setup.c
+++ b/setup.c
@@ -255,7 +255,8 @@ void setup_work_tree(void)
 	if (getenv(GIT_WORK_TREE_ENVIRONMENT))
 		setenv(GIT_WORK_TREE_ENVIRONMENT, ".", 1);
 
-	set_git_dir(remove_leading_path(git_dir, work_tree));
+	set_git_dir_super(remove_leading_path(git_dir, work_tree),
+			  get_git_super_dir());
 	initialized = 1;
 }
 
@@ -368,24 +369,28 @@ const char *read_gitfile(const char *path)
 
 
 static const char *setup_explicit_git_dir(const char *gitdirenv,
+					  const char *super_,
 					  char *cwd, int len,
 					  int *nongit_ok)
 {
 	const char *work_tree_env = getenv(GIT_WORK_TREE_ENVIRONMENT);
 	const char *worktree;
-	char *gitfile;
+	char *gitfile, *super = (char *)super_;
 	int offset;
 
 	if (PATH_MAX - 40 < strlen(gitdirenv))
 		die("'$%s' too big", GIT_DIR_ENVIRONMENT);
 
-	gitfile = (char*)read_gitfile(gitdirenv);
+	gitfile = (char*)read_gitfile_super(gitdirenv, !super ? &super : NULL);
 	if (gitfile) {
 		gitfile = xstrdup(gitfile);
 		gitdirenv = gitfile;
-	}
+		if (!super)
+			super = gitfile;
+	} else if (!super)
+		super = (char *)gitdirenv;
 
-	if (!is_git_directory(gitdirenv)) {
+	if (!is_git_directory_super(gitdirenv, super)) {
 		if (nongit_ok) {
 			*nongit_ok = 1;
 			goto done_null;
@@ -393,7 +398,7 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
 		die("Not a git repository: '%s'", gitdirenv);
 	}
 
-	if (check_repository_format_gently(gitdirenv, nongit_ok))
+	if (check_repository_format_gently(super, nongit_ok))
 		goto done_null;
 
 	/* #3, #7, #11, #15, #19, #23, #27, #31 (see t1510) */
@@ -404,7 +409,7 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
 			die("core.bare and core.worktree do not make sense");
 
 		/* #18, #26 */
-		set_git_dir(gitdirenv);
+		set_git_dir_super(gitdirenv, super);
 		goto done_null;
 	}
 	else if (git_work_tree_cfg) { /* #6, #14 */
@@ -425,7 +430,7 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
 	}
 	else if (!git_env_bool(GIT_IMPLICIT_WORK_TREE_ENVIRONMENT, 1)) {
 		/* #16d */
-		set_git_dir(gitdirenv);
+		set_git_dir_super(gitdirenv, super);
 		goto done_null;
 	}
 	else /* #2, #10 */
@@ -436,34 +441,39 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
 
 	/* both get_git_work_tree() and cwd are already normalized */
 	if (!strcmp(cwd, worktree)) { /* cwd == worktree */
-		set_git_dir(gitdirenv);
+		set_git_dir_super(gitdirenv, super);
 		goto done_null;
 	}
 
 	offset = dir_inside_of(cwd, worktree);
 	if (offset >= 0) {	/* cwd inside worktree? */
-		set_git_dir(real_path(gitdirenv));
+		set_git_dir_super(real_path(gitdirenv), super);
 		if (chdir(worktree))
 			die_errno("Could not chdir to '%s'", worktree);
 		cwd[len++] = '/';
 		cwd[len] = '\0';
 		free(gitfile);
+		if (super != gitdirenv && super != super_)
+			free(super);
 		return cwd + offset;
 	}
 
 	/* cwd outside worktree */
-	set_git_dir(gitdirenv);
+	set_git_dir_super(gitdirenv, super);
 
 done_null:
 	free(gitfile);
+	if (super != gitdirenv && super != super_)
+		free(super);
 	return NULL;
 }
 
 static const char *setup_discovered_git_dir(const char *gitdir,
+					    const char *super,
 					    char *cwd, int offset, int len,
 					    int *nongit_ok)
 {
-	if (check_repository_format_gently(gitdir, nongit_ok))
+	if (check_repository_format_gently(super, nongit_ok))
 		return NULL;
 
 	/* --work-tree is set without --git-dir; use discovered one */
@@ -472,7 +482,7 @@ static const char *setup_discovered_git_dir(const char *gitdir,
 			gitdir = xstrdup(real_path(gitdir));
 		if (chdir(cwd))
 			die_errno("Could not come back to cwd");
-		return setup_explicit_git_dir(gitdir, cwd, len, nongit_ok);
+		return setup_explicit_git_dir(gitdir, super, cwd, len, nongit_ok);
 	}
 
 	/* #16.2, #17.2, #20.2, #21.2, #24, #25, #28, #29 (see t1510) */
@@ -486,7 +496,7 @@ static const char *setup_discovered_git_dir(const char *gitdir,
 	/* #0, #1, #5, #8, #9, #12, #13 */
 	set_git_work_tree(".");
 	if (strcmp(gitdir, DEFAULT_GIT_DIR_ENVIRONMENT))
-		set_git_dir(gitdir);
+		set_git_dir_super(gitdir, super);
 	inside_git_dir = 0;
 	inside_work_tree = 1;
 	if (offset == len)
@@ -516,7 +526,7 @@ static const char *setup_bare_git_dir(char *cwd, int offset, int len, int *nongi
 		gitdir = offset == len ? "." : xmemdupz(cwd, offset);
 		if (chdir(cwd))
 			die_errno("Could not come back to cwd");
-		return setup_explicit_git_dir(gitdir, cwd, len, nongit_ok);
+		return setup_explicit_git_dir(gitdir, NULL, cwd, len, nongit_ok);
 	}
 
 	inside_git_dir = 1;
@@ -596,7 +606,7 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 	struct string_list ceiling_dirs = STRING_LIST_INIT_DUP;
 	static char cwd[PATH_MAX + 1];
 	const char *gitdirenv, *ret;
-	char *gitfile;
+	char *gitfile, *super;
 	int len, offset, offset_parent, ceil_offset = -1;
 	dev_t current_device = 0;
 	int one_filesystem = 1;
@@ -620,7 +630,9 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 	 */
 	gitdirenv = getenv(GIT_DIR_ENVIRONMENT);
 	if (gitdirenv)
-		return setup_explicit_git_dir(gitdirenv, cwd, len, nongit_ok);
+		return setup_explicit_git_dir(gitdirenv,
+					      getenv(GIT_SUPER_DIR_ENVIRONMENT),
+					      cwd, len, nongit_ok);
 
 	if (env_ceiling_dirs) {
 		int empty_entry_found = 0;
@@ -650,21 +662,29 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 	if (one_filesystem)
 		current_device = get_device_or_die(".", NULL, 0);
 	for (;;) {
-		gitfile = (char*)read_gitfile(DEFAULT_GIT_DIR_ENVIRONMENT);
-		if (gitfile)
+		gitfile = (char*)read_gitfile_super(DEFAULT_GIT_DIR_ENVIRONMENT,
+						    &super);
+		if (gitfile) {
 			gitdirenv = gitfile = xstrdup(gitfile);
-		else {
+			if (!super)
+				super = gitfile;
+		} else {
 			if (is_git_directory(DEFAULT_GIT_DIR_ENVIRONMENT))
 				gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
+			super = (char *)gitdirenv;
 		}
 
 		if (gitdirenv) {
-			ret = setup_discovered_git_dir(gitdirenv,
+			ret = setup_discovered_git_dir(gitdirenv, super,
 						       cwd, offset, len,
 						       nongit_ok);
+			if (super != gitdirenv)
+				free(super);
 			free(gitfile);
 			return ret;
 		}
+		if (super != gitdirenv)
+			free(super);
 		free(gitfile);
 
 		if (is_git_directory("."))
diff --git a/trace.c b/trace.c
index 3d744d1..53d800b 100644
--- a/trace.c
+++ b/trace.c
@@ -173,6 +173,7 @@ void trace_repo_setup(const char *prefix)
 		prefix = "(null)";
 
 	trace_printf_key(key, "setup: git_dir: %s\n", quote_crnl(get_git_dir()));
+	trace_printf_key(key, "setup: git_super_dir: %s\n", quote_crnl(get_git_super_dir()));
 	trace_printf_key(key, "setup: worktree: %s\n", quote_crnl(git_work_tree));
 	trace_printf_key(key, "setup: cwd: %s\n", quote_crnl(cwd));
 	trace_printf_key(key, "setup: prefix: %s\n", quote_crnl(prefix));
-- 
1.8.5.1.77.g42c48fa

[PATCH/POC 7/7] init: add --split-repo with the same functionality as git-new-workdir

[Nguyễn Thái Ngọc Duy]

"git init --split-repo abc" will create abc/.git file with the content

gitsuper: `git rev-parse --git-dir`
gitdir: abc

and a new directory in current .git/repos/abc with a valid HEAD.

This is enough to start checking out and do stuff. We should probably
take a branch name and check that branch out though. If that's the
case, this feature may better be parked in "git checkout" instead of
here..

So far it's not any better than git-new-workdir, except that info from
all worktrees created this way is centralized in the super repo's
.git/repos, which makes it possible to ensure what worktree does not
step on one another. In particular:

 - If a worktree updates a ref, we could check if any other worktrees
   are also checking out that ref. Detach those worktrees.

 - prune/gc should be taught about the extra HEADs in .git/repos. fsck
   on the super repo should be taught about extra indexes in
   .git/repos

 - ...

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/init-db.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/builtin/init-db.c b/builtin/init-db.c
index 78aa387..9c7139a 100644
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@ -426,6 +426,38 @@ int init_db(const char *template_dir, unsigned int flags)
 	return 0;
 }
 
+static int create_new_worktree(const char *path)
+{
+	struct strbuf sb = STRBUF_INIT;
+	const char *name;
+	unsigned char sha1[20];
+	FILE *fp;
+
+	for (name = path + strlen(path) - 1; name > path; name--)
+		if (is_dir_sep(*name)) {
+			name++;
+			break;
+		}
+
+	strbuf_addf(&sb, "%s/.git", path);
+	safe_create_leading_directories_const(sb.buf);
+	fp = fopen(sb.buf, "w");
+	fprintf(fp, "gitsuper: %s\ngitdir: %s\n",
+		real_path(get_git_dir()), name);
+	fclose(fp);
+	safe_create_leading_directories_const(git_path("repos/%s/HEAD",
+						       name));
+	fp = fopen(git_path("repos/%s/HEAD", name), "w");
+	get_sha1("HEAD", sha1);
+	fprintf(fp, "%s\n", sha1_to_hex(sha1));
+	fclose(fp);
+#if 0
+	chdir(path);
+	system("git reset --hard");
+#endif
+	return 0;
+}
+
 static int guess_repository_type(const char *git_dir)
 {
 	char cwd[PATH_MAX];
@@ -481,6 +513,7 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)
 	const char *work_tree;
 	const char *template_dir = NULL;
 	unsigned int flags = 0;
+	int split_repo = 0;
 	const struct option init_db_options[] = {
 		OPT_STRING(0, "template", &template_dir, N_("template-directory"),
 				N_("directory from which templates will be used")),
@@ -491,6 +524,7 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)
 			N_("specify that the git repository is to be shared amongst several users"),
 			PARSE_OPT_OPTARG | PARSE_OPT_NONEG, shared_callback, 0},
 		OPT_BIT('q', "quiet", &flags, N_("be quiet"), INIT_DB_QUIET),
+		OPT_BOOL(0, "split-repo", &split_repo, N_("git-new-workdir")),
 		OPT_STRING(0, "separate-git-dir", &real_git_dir, N_("gitdir"),
 			   N_("separate git dir from working tree")),
 		OPT_END()
@@ -498,6 +532,14 @@ int cmd_init_db(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, init_db_options, init_db_usage, 0);
 
+	if (split_repo) {
+		if (real_git_dir)
+			die(_("--split-repo and --separate-git-dir are incompatible"));
+		if (!argv[0])
+			die(_("--split-repo requires a path"));
+		return create_new_worktree(argv[0]);
+	}
+
 	if (real_git_dir && !is_absolute_path(real_git_dir))
 		real_git_dir = xstrdup(real_path(real_git_dir));
 
-- 
1.8.5.1.77.g42c48fa

Re: [PATCH/POC 1/7] Make git_path() beware of file relocation in $GIT_DIR

[Junio C Hamano]

Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:

> +static void adjust_git_path(char *buf, int git_dir_len)
> +{
> +	/* XXX buffer overflow */
> +	char *base = buf + git_dir_len;
> +	if (git_graft_env && !strcmp(base, "info/grafts"))
> +		strcpy(buf, get_graft_file());
> +	else if (git_index_env && !strcmp(base, "index"))
> +		strcpy(buf, get_index_file());
> +	else if (git_db_env && dir_prefix(base, "objects"))
> +		replace_dir(buf, git_dir_len + 7, get_object_directory());
> +}
> +
>  static char *vsnpath(char *buf, size_t n, const char *fmt, va_list args)
>  {
>  	const char *git_dir = get_git_dir();
> -	size_t len;
> +	size_t len, total_len;
>  
>  	len = strlen(git_dir);
>  	if (n < len + 1)
> @@ -60,9 +88,10 @@ static char *vsnpath(char *buf, size_t n, const char *fmt, va_list args)
>  	memcpy(buf, git_dir, len);
>  	if (len && !is_dir_sep(git_dir[len-1]))
>  		buf[len++] = '/';
> -	len += vsnprintf(buf + len, n - len, fmt, args);
> -	if (len >= n)
> +	total_len = len + vsnprintf(buf + len, n - len, fmt, args);
> +	if (total_len >= n)
>  		goto bad;
> +	adjust_git_path(buf, len);

This is a minor tangent, but this part of the patch made me raise my
eyebrow, wondering what Git-specific path mangler is doing in a
function vsnpath that is named as if it is a lot more generic, until
I read the change in context.

The vnspath() is already Git-specific---it is a helper that is used
to create a path inside the $GIT_DIR directory.

We probably should do two things to clear things up:

 - Right now, path.c has definitions of functions in this order:

    char *mksnpath(char *buf, size_t n, const char *fmt, ...)
    static char *vsnpath(char *buf, size_t n, const char *fmt, va_list args)
    char *git_snpath(char *buf, size_t n, const char *fmt, ...)
    char *git_pathdup(const char *fmt, ...)
    char *mkpathdup(const char *fmt, ...)
    char *mkpath(const char *fmt, ...)
    char *git_path(const char *fmt, ...)

   The two functions mkpathdup() and mkpath() are not Git specific
   at all.  They should be moved up to be grouped together with
   mksnpath() that is also not Git-specific.

 - Rename the static vsnpath() to further clarify that it is Git
   specific.

Re: [PATCH/POC 2/7] Add new environment variable $GIT_SUPER_DIR

[Junio C Hamano]

Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:

> This is the base for git-new-workdir integration. The git-new-workdir
> script creates a separate worktree that shares everything except
> worktree-related stuff. The sharing is eanbled by this new env
> variable.
>
> In the new worktree, both variables are set at the same time, GIT_DIR
> and GIT_SUPER_DIR. Shared paths are checked at adjust_git_path() and
> rewritten to use $GIT_SUPER_DIR instead of $GIT_DIR. Let's call this
> "split-repo" setup to distinguish from $GIT_DIR-only one.
>
> The "ln -s" is avoided because Windows probably does not have the
> support, and symlinks don't survive operations that delete the old
> file, then create a new one.
>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
>  cache.h               |  2 ++
>  environment.c         | 11 +++++++--
>  path.c                | 10 ++++++++
>  t/t0060-path-utils.sh | 67 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  4 files changed, 88 insertions(+), 2 deletions(-)
>
> diff --git a/cache.h b/cache.h
> index cdafbd7..823582f 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -347,6 +347,7 @@ static inline enum object_type object_type(unsigned int mode)
>  
>  /* Double-check local_repo_env below if you add to this list. */
>  #define GIT_DIR_ENVIRONMENT "GIT_DIR"
> +#define GIT_SUPER_DIR_ENVIRONMENT "GIT_SUPER_DIR"
>  #define GIT_NAMESPACE_ENVIRONMENT "GIT_NAMESPACE"
>  #define GIT_WORK_TREE_ENVIRONMENT "GIT_WORK_TREE"
>  #define GIT_PREFIX_ENVIRONMENT "GIT_PREFIX"
> @@ -399,6 +400,7 @@ extern int is_inside_git_dir(void);
>  extern char *git_work_tree_cfg;
>  extern int is_inside_work_tree(void);
>  extern const char *get_git_dir(void);
> +extern const char *get_git_super_dir(void);
>  extern int is_git_directory(const char *path);
>  extern char *get_object_directory(void);
>  extern char *get_index_file(void);
> diff --git a/environment.c b/environment.c
> index 1d74dde..d5ae7a3 100644
> --- a/environment.c
> +++ b/environment.c
> @@ -79,7 +79,7 @@ static char *work_tree;
>  static const char *namespace;
>  static size_t namespace_len;
>  
> -static const char *git_dir;
> +static const char *git_dir, *git_super_dir;
>  static char *git_object_dir, *git_index_file, *git_graft_file;
>  int git_db_env, git_index_env, git_graft_env;
>  
> @@ -131,10 +131,12 @@ static void setup_git_env(void)
>  		git_dir = DEFAULT_GIT_DIR_ENVIRONMENT;
>  	gitfile = read_gitfile(git_dir);
>  	git_dir = xstrdup(gitfile ? gitfile : git_dir);
> +	git_super_dir = getenv(GIT_SUPER_DIR_ENVIRONMENT);
>  	git_object_dir = getenv(DB_ENVIRONMENT);
>  	if (!git_object_dir) {
>  		git_object_dir = xmalloc(strlen(git_dir) + 9);
> -		sprintf(git_object_dir, "%s/objects", git_dir);
> +		sprintf(git_object_dir, "%s/objects",
> +			git_super_dir ? git_super_dir : git_dir);
>  	} else
>  		git_db_env = 1;
>  	git_index_file = getenv(INDEX_ENVIRONMENT);
> @@ -167,6 +169,11 @@ const char *get_git_dir(void)
>  	return git_dir;
>  }
>  
> +const char *get_git_super_dir(void)
> +{
> +	return git_super_dir;
> +}
> +
>  const char *get_git_namespace(void)
>  {
>  	if (!namespace)
> diff --git a/path.c b/path.c
> index eda9176..86a7c15 100644
> --- a/path.c
> +++ b/path.c
> @@ -75,6 +75,16 @@ static void adjust_git_path(char *buf, int git_dir_len)
>  		strcpy(buf, get_index_file());
>  	else if (git_db_env && dir_prefix(base, "objects"))
>  		replace_dir(buf, git_dir_len + 7, get_object_directory());
> +	else if (get_git_super_dir()) {
> +		if (dir_prefix(base, "objects") || dir_prefix(base, "info") ||
> +		    dir_prefix(base, "refs") ||
> +		    (dir_prefix(base, "logs") && strcmp(base, "logs/HEAD")) ||
> +		    dir_prefix(base, "rr-cache") || dir_prefix(base, "hooks") ||
> +		    dir_prefix(base, "svn") ||
> +		    !strcmp(base, "config") || !strcmp(base, "packed-refs") ||
> +		    !strcmp(base, "shallow"))
> +			replace_dir(buf, git_dir_len, get_git_super_dir());
> +	}

This is essentially the list of what are shared across workdirs,
right?  I wonder if it is a bad idea to make everything under .git
of the borrowed repository shared by default, with selected
exceptions.  Granted, not sharing by default is definitely safer
than blindly sharing by default, so that alone may be a good
argument to use a set of known-to-be-safe-to-share paths, like this
code does.

Don't we want .git/branches and .git/remotes shared?  After all,
their moral equivalents from .git/config are shared with the code.

The name "super" might need to be rethought, but getting the
mechanism and the semantics right is the more important first step,
and I think this is going in a good direction.

Re: [PATCH/POC 3/7] setup.c: add split-repo support to .git files

[Junio C Hamano]

Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:

> If a .git file contains
>
> gitsuper: <path>
> gitdir: <id>
>
> then we set GIT_SUPER_DIR to <path> and GIT_DIR to
> $GIT_SUPER_DIR/repos/<id>.

I initially thought: "what is with that complexity? isn't it just
the matter of replacing 'gitdir: <path>' with 'gitsuper: <path>'
stored in the file .git???"

Until I realized that there is nowhere to keep per-workdir data if
we only had .git as a pointer, and that is why you have that <id>
thing.  It would have helped me avoid that confusion if the above
description was followed by:

    The latter, $GIT_SUPER_DIR/repos/<id>, is a directory,
    underneath which per-work-dir items like index, HEAD, logs/HEAD
    (what else?) reside.

or something like that.  And $GIT_SUPER_DIR/repos/*/HEAD, especially
when they are detached, plus $GIT_SUPER_DIR/repos/*/index, will work
as the starting point of object reachability scanning when running
repack, fsck, etc.

A few more random thoughts...

 - Reusing "gitdir:" for this purpose is not advisable; use a
   different name.  This <id> is used to identify a workdir, so
   perhaps "gitworkdir: <id>" might be a better name;

 - Do we want to record where the working tree directory is in
   $GIT_SUPER_DIR/repos/<id> somewhere?  Would it help to have such
   a record?

 - How would this interact with core.worktree in .git/config of that
   "super" repository?

Re: [PATCH/POC 3/7] setup.c: add split-repo support to .git files

[Jonathan Nieder]

Junio C Hamano wrote:

>  - Do we want to record where the working tree directory is in
>    $GIT_SUPER_DIR/repos/<id> somewhere?  Would it help to have such
>    a record?

That could be nice for the purpose of garbage collecting them.  I fear
that for users it is too tempting to remove a worktree with "rm -rf"
without considering the relationship from the parent repo that might
be making walking through all reflogs slower or holding on to objects
no one cares about any more.

I imagine it would work like this:

 1. At worktree creation time, full path to the working tree directory
    is stored in $GIT_SUPER_DIR/repos/<id>.

 2. "git gc" notices that the worktree is missing and writes a file
    under $GIT_SUPER_DIR/repos/<id> with a timestamp, saying so.

 3. If the worktree still hasn't existed for a month, "git gc" deletes
    the corresponding $GIT_SUPER_DIR/repos/<id> directory.

Problems:

 * What if I move my worktree with "mv"?  Then I still need the
   corresponding $GIT_SUPER_DIR/repos/<id> directory, and nobody told
   the GIT_SUPER_DIR about it.

 * What if my worktree is on removable media (think "network
   filesystem") and has just been temporarily unmounted instead of
   deleted?

So maybe it would be nicer to:

  i. When the worktree is on the same filesystem, keep a *hard link* to
     some file in the worktree (e.g., the .git file).  If the link count
     goes down, it is safe to remove the $GIT_SUPER_DIR/repos/<id>
     directory.

 ii. When the worktree is on another filesystem, always keep
     $GIT_SUPER_DIR/repos/<id> unless the user decides to manually
     remove it.  Provide documentation or a command to list basic
     information about $GIT_SUPER_DIR/repos directories (path to
     worktree, what branch they're on, etc).

(i) doesn't require any futureproofing.  As soon as someone wants it,
they can implement the check and fall back to (ii) for worktrees
without the magic hard link.

(ii) would benefit from recording the working tree directory as a
possibly unreliable, human-friendly reminder.

>  - How would this interact with core.worktree in .git/config of that
>    "super" repository?

Eek.

Thanks,
Jonathan

Re: [PATCH/POC 2/7] Add new environment variable $GIT_SUPER_DIR

[Duy Nguyen]

On Sat, Dec 14, 2013 at 1:12 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> diff --git a/path.c b/path.c
>> index eda9176..86a7c15 100644
>> --- a/path.c
>> +++ b/path.c
>> @@ -75,6 +75,16 @@ static void adjust_git_path(char *buf, int git_dir_len)
>>               strcpy(buf, get_index_file());
>>       else if (git_db_env && dir_prefix(base, "objects"))
>>               replace_dir(buf, git_dir_len + 7, get_object_directory());
>> +     else if (get_git_super_dir()) {
>> +             if (dir_prefix(base, "objects") || dir_prefix(base, "info") ||
>> +                 dir_prefix(base, "refs") ||
>> +                 (dir_prefix(base, "logs") && strcmp(base, "logs/HEAD")) ||
>> +                 dir_prefix(base, "rr-cache") || dir_prefix(base, "hooks") ||
>> +                 dir_prefix(base, "svn") ||
>> +                 !strcmp(base, "config") || !strcmp(base, "packed-refs") ||
>> +                 !strcmp(base, "shallow"))
>> +                     replace_dir(buf, git_dir_len, get_git_super_dir());
>> +     }
>
> This is essentially the list of what are shared across workdirs,
> right?  I wonder if it is a bad idea to make everything under .git
> of the borrowed repository shared by default, with selected
> exceptions.  Granted, not sharing by default is definitely safer
> than blindly sharing by default, so that alone may be a good
> argument to use a set of known-to-be-safe-to-share paths, like this
> code does.

The exception list could be equally long (most of them are *_HEAD).
.git is also used for temporary files with mkstemp, but I think that's
safe for sharing. What could break is when people add a new local
*_HEAD and forget to update the exception list.

> Don't we want .git/branches and .git/remotes shared?  After all,
> their moral equivalents from .git/config are shared with the code.

Yes. I missed them.
-- 
Duy

Re: [PATCH/POC 3/7] setup.c: add split-repo support to .git files

[Duy Nguyen]

On Sat, Dec 14, 2013 at 3:43 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Junio C Hamano wrote:
>
>>  - Do we want to record where the working tree directory is in
>>    $GIT_SUPER_DIR/repos/<id> somewhere?  Would it help to have such
>>    a record?
>
> That could be nice for the purpose of garbage collecting them.  I fear
> that for users it is too tempting to remove a worktree with "rm -rf"
> without considering the relationship from the parent repo that might
> be making walking through all reflogs slower or holding on to objects
> no one cares about any more.
>
> I imagine it would work like this:
>
>  1. At worktree creation time, full path to the working tree directory
>     is stored in $GIT_SUPER_DIR/repos/<id>.
>
>  2. "git gc" notices that the worktree is missing and writes a file
>     under $GIT_SUPER_DIR/repos/<id> with a timestamp, saying so.
>
>  3. If the worktree still hasn't existed for a month, "git gc" deletes
>     the corresponding $GIT_SUPER_DIR/repos/<id> directory.

I was thinking about doing something like this in "git prune" but
manually. Your idea sounds nicer.

> Problems:
>
>  * What if I move my worktree with "mv"?  Then I still need the
>    corresponding $GIT_SUPER_DIR/repos/<id> directory, and nobody told
>    the GIT_SUPER_DIR about it.

We could store $GIT_SUPER_DIR as relative path. That way if you move
it, you break it. When you fix it, hopefully you remember to fix the
link in repos/<id> too

Alternatively, the setup up code could be taught to verify that
$GIT_SUPER_DIR/repos/id/<backlink> actually points to the current
worktree. If not warn the user or something

>  * What if my worktree is on removable media (think "network
>    filesystem") and has just been temporarily unmounted instead of
>    deleted?

Or we keep update a timestamp in repos/<id> to note the last used time
of this worktree. "gc" or "prune" would warn about unused repos after
a certain amount of time, do not remove them automatically. This could
be iii. to your list below.

> So maybe it would be nicer to:
>
>   i. When the worktree is on the same filesystem, keep a *hard link* to
>      some file in the worktree (e.g., the .git file).  If the link count
>      goes down, it is safe to remove the $GIT_SUPER_DIR/repos/<id>
>      directory.

This can still break with updating by creating a new version, then
renaming it. Although I can't think why anybody (or anything) would
want to do that on .git file. This does not work on Windows though.

>  ii. When the worktree is on another filesystem, always keep
>      $GIT_SUPER_DIR/repos/<id> unless the user decides to manually
>      remove it.  Provide documentation or a command to list basic
>      information about $GIT_SUPER_DIR/repos directories (path to
>      worktree, what branch they're on, etc).

And on Windows, a new partition means a new drive, so it works there too.

>
> (i) doesn't require any futureproofing.  As soon as someone wants it,
> they can implement the check and fall back to (ii) for worktrees
> without the magic hard link.
>
> (ii) would benefit from recording the working tree directory as a
> possibly unreliable, human-friendly reminder.
>
>>  - How would this interact with core.worktree in .git/config of that
>>    "super" repository?
>
> Eek.

I'll see if I can ignore core.worktree when $GIT_SUPER_DIR is set. If
not, ban this use case :)
-- 
Duy

[PATCH v2 00/21] Support multiple worktrees

[Nguyễn Thái Ngọc Duy]

The UI and behavior are taking shape now. On the UI side, you do

  git checkout --to /somewhere -b newbranch origin/master

which will create worktree-only repo at /somewhere. "git prune --repos"
could be used to remove cruft in .git/repos.

On the behavior side, you should be able to do everything in
/somewhere just like in a normal repository. If a ref is updated (from
any repository) that also happens to be your HEAD, it will be
detached. "git rev-list --all" is also taught to take repos/.../HEAD
into account.

The structure of repos/XXX is documented in 17/21. Known issues

 - naming ($GIT_SUPER_DIR, the name of the shared repo and the
   dependent one, the reuse of "gitdir" in .git files)

 - gc --auto support, support for manually pruning .git/repos

 - should probably support the new .git format in enter_repo() so that
   we can push to it

 - not sure if we need UI for deleting repositories created with
   checkout --to, or just "rm -r" and let "gc --auto" clean
   things up. The thing about "rm -r(f)" is that if .git happens to be
   a real repo, the user is screwed so I don't really like to
   encourage doing it.

 - more tests


Nguyễn Thái Ngọc Duy (21):
  path.c: avoid PATH_MAX as buffer size from get_pathname()
  path.c: rename vsnpath() to git_vsnpath()
  path.c: move git_path() closer to similar functions git_pathdup()
  Make git_path() aware of file relocation in $GIT_DIR
  reflog: use avoid constructing .lock path with git_path
  fast-import: use git_path() for accessing .git dir instead of get_git_dir()
  Add new environment variable $GIT_SUPER_DIR
  setup.c: refactor path manipulation out of read_gitfile()
  setup.c: add split-repo support to .git files
  setup.c: add split-repo support to is_git_directory()
  setup.c: reduce cleanup sites in setup_explicit_git_dir()
  environment.c: support super .git file specified by $GIT_DIR
  setup: support $GIT_SUPER_DIR as well as super .git files
  checkout: support checking out into a new working directory
  checkout: clean up half-prepared directories in --to mode
  setup.c: keep track of the .git file location if read
  prune: strategies for split repositories
  refs: adjust reflog path for repos/<id>/HEAD
  refs: detach split repos' HEAD when the linked ref is updated/deleted
  refs.c: refactor do_head_ref(... to do_one_ref("HEAD", ...
  revision: include repos/../HEAD in --all

 Documentation/config.txt               |   3 +-
 Documentation/git-checkout.txt         |   6 +
 Documentation/git-prune.txt            |   4 +
 Documentation/git.txt                  |   8 ++
 Documentation/gitrepository-layout.txt |  30 ++++
 builtin/checkout.c                     | 173 ++++++++++++++++++++++
 builtin/prune.c                        |  65 +++++++++
 builtin/reflog.c                       |   2 +-
 builtin/rev-parse.c                    |   6 +
 cache.h                                |   5 +
 environment.c                          |  37 ++++-
 fast-import.c                          |   5 +-
 path.c                                 | 140 ++++++++++++++----
 refs.c                                 |  88 ++++++++++--
 refs.h                                 |   1 +
 revision.c                             |   1 +
 setup.c                                | 253 ++++++++++++++++++++++++---------
 t/t0060-path-utils.sh                  | 133 +++++++++++++++++
 t/t1501-worktree.sh                    |  52 +++++++
 t/t1510-repo-setup.sh                  |   1 +
 test-path-utils.c                      |   7 +
 trace.c                                |   1 +
 22 files changed, 904 insertions(+), 117 deletions(-)

-- 
1.8.5.1.77.g42c48fa

[PATCH v2 01/21] path.c: avoid PATH_MAX as buffer size from get_pathname()

[Nguyễn Thái Ngọc Duy]

We've been avoiding PATH_MAX whenever possible. This patch avoids
PATH_MAX in get_pathname() and gives it enough room not to worry about
really long paths.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 path.c | 28 ++++++++++++++++------------
 1 file changed, 16 insertions(+), 12 deletions(-)

diff --git a/path.c b/path.c
index 24594c4..4c1c144 100644
--- a/path.c
+++ b/path.c
@@ -16,10 +16,11 @@ static int get_st_mode_bits(const char *path, int *mode)
 
 static char bad_path[] = "/bad-path/";
 
-static char *get_pathname(void)
+static char *get_pathname(size_t *len)
 {
-	static char pathname_array[4][PATH_MAX];
+	static char pathname_array[4][4096];
 	static int index;
+	*len = sizeof(pathname_array[0]);
 	return pathname_array[3 & ++index];
 }
 
@@ -108,24 +109,26 @@ char *mkpath(const char *fmt, ...)
 {
 	va_list args;
 	unsigned len;
-	char *pathname = get_pathname();
+	size_t n;
+	char *pathname = get_pathname(&n);
 
 	va_start(args, fmt);
-	len = vsnprintf(pathname, PATH_MAX, fmt, args);
+	len = vsnprintf(pathname, n, fmt, args);
 	va_end(args);
-	if (len >= PATH_MAX)
+	if (len >= n)
 		return bad_path;
 	return cleanup_path(pathname);
 }
 
 char *git_path(const char *fmt, ...)
 {
-	char *pathname = get_pathname();
+	size_t len;
+	char *pathname = get_pathname(&len);
 	va_list args;
 	char *ret;
 
 	va_start(args, fmt);
-	ret = vsnpath(pathname, PATH_MAX, fmt, args);
+	ret = vsnpath(pathname, len, fmt, args);
 	va_end(args);
 	return ret;
 }
@@ -158,14 +161,15 @@ void home_config_paths(char **global, char **xdg, char *file)
 
 char *git_path_submodule(const char *path, const char *fmt, ...)
 {
-	char *pathname = get_pathname();
+	size_t n;
+	char *pathname = get_pathname(&n);
 	struct strbuf buf = STRBUF_INIT;
 	const char *git_dir;
 	va_list args;
 	unsigned len;
 
 	len = strlen(path);
-	if (len > PATH_MAX-100)
+	if (len > n-100)
 		return bad_path;
 
 	strbuf_addstr(&buf, path);
@@ -180,7 +184,7 @@ char *git_path_submodule(const char *path, const char *fmt, ...)
 	}
 	strbuf_addch(&buf, '/');
 
-	if (buf.len >= PATH_MAX)
+	if (buf.len >= n)
 		return bad_path;
 	memcpy(pathname, buf.buf, buf.len + 1);
 
@@ -188,9 +192,9 @@ char *git_path_submodule(const char *path, const char *fmt, ...)
 	len = strlen(pathname);
 
 	va_start(args, fmt);
-	len += vsnprintf(pathname + len, PATH_MAX - len, fmt, args);
+	len += vsnprintf(pathname + len, n - len, fmt, args);
 	va_end(args);
-	if (len >= PATH_MAX)
+	if (len >= n)
 		return bad_path;
 	return cleanup_path(pathname);
 }
-- 
1.8.5.1.77.g42c48fa

[PATCH v2 02/21] path.c: rename vsnpath() to git_vsnpath()

[Nguyễn Thái Ngọc Duy]

This is the underlying implementation of git_path(), git_pathdup() and
git_snpath() which will prefix $GIT_DIR in the result string. Put git_
prefix in front of it to avoid the confusion that this is a generic
path handling function.#

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 path.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/path.c b/path.c
index 4c1c144..06863b7 100644
--- a/path.c
+++ b/path.c
@@ -50,7 +50,7 @@ char *mksnpath(char *buf, size_t n, const char *fmt, ...)
 	return cleanup_path(buf);
 }
 
-static char *vsnpath(char *buf, size_t n, const char *fmt, va_list args)
+static char *git_vsnpath(char *buf, size_t n, const char *fmt, va_list args)
 {
 	const char *git_dir = get_git_dir();
 	size_t len;
@@ -75,7 +75,7 @@ char *git_snpath(char *buf, size_t n, const char *fmt, ...)
 	char *ret;
 	va_list args;
 	va_start(args, fmt);
-	ret = vsnpath(buf, n, fmt, args);
+	ret = git_vsnpath(buf, n, fmt, args);
 	va_end(args);
 	return ret;
 }
@@ -85,7 +85,7 @@ char *git_pathdup(const char *fmt, ...)
 	char path[PATH_MAX], *ret;
 	va_list args;
 	va_start(args, fmt);
-	ret = vsnpath(path, sizeof(path), fmt, args);
+	ret = git_vsnpath(path, sizeof(path), fmt, args);
 	va_end(args);
 	return xstrdup(ret);
 }
@@ -128,7 +128,7 @@ char *git_path(const char *fmt, ...)
 	char *ret;
 
 	va_start(args, fmt);
-	ret = vsnpath(pathname, len, fmt, args);
+	ret = git_vsnpath(pathname, len, fmt, args);
 	va_end(args);
 	return ret;
 }
-- 
1.8.5.1.77.g42c48fa

[PATCH v2 03/21] path.c: move git_path() closer to similar functions git_pathdup()

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 path.c | 26 +++++++++++++-------------
 1 file changed, 13 insertions(+), 13 deletions(-)

diff --git a/path.c b/path.c
index 06863b7..08aa213 100644
--- a/path.c
+++ b/path.c
@@ -80,6 +80,19 @@ char *git_snpath(char *buf, size_t n, const char *fmt, ...)
 	return ret;
 }
 
+char *git_path(const char *fmt, ...)
+{
+	size_t len;
+	char *pathname = get_pathname(&len);
+	va_list args;
+	char *ret;
+
+	va_start(args, fmt);
+	ret = git_vsnpath(pathname, len, fmt, args);
+	va_end(args);
+	return ret;
+}
+
 char *git_pathdup(const char *fmt, ...)
 {
 	char path[PATH_MAX], *ret;
@@ -120,19 +133,6 @@ char *mkpath(const char *fmt, ...)
 	return cleanup_path(pathname);
 }
 
-char *git_path(const char *fmt, ...)
-{
-	size_t len;
-	char *pathname = get_pathname(&len);
-	va_list args;
-	char *ret;
-
-	va_start(args, fmt);
-	ret = git_vsnpath(pathname, len, fmt, args);
-	va_end(args);
-	return ret;
-}
-
 void home_config_paths(char **global, char **xdg, char *file)
 {
 	char *xdg_home = getenv("XDG_CONFIG_HOME");
-- 
1.8.5.1.77.g42c48fa

[PATCH v2 04/21] Make git_path() aware of file relocation in $GIT_DIR

[Nguyễn Thái Ngọc Duy]

We allow the user to relocate certain paths out of $GIT_DIR via
environment variables, e.g. GIT_OBJECT_DIRECTORY, GIT_INDEX_FILE and
GIT_GRAFT_FILE. All callers are not supposed to use git_path() or
git_pathdup() to get those paths. Instead they must use
get_object_directory(), get_index_file() and get_graft_file()
respectively. This is inconvenient and could be missed in review
(there's git_path("objects/info/alternates") somewhere in
sha1_file.c).

This patch makes git_path() and git_pathdup() understand those
environment variables. So if you set GIT_OBJECT_DIRECTORY to /foo/bar,
git_path("objects/abc") should return /tmp/bar/abc. The same is done
for the two remaining env variables.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 cache.h               |  1 +
 environment.c         |  9 ++++++--
 path.c                | 57 ++++++++++++++++++++++++++++++++++++++++++++++++---
 t/t0060-path-utils.sh | 54 ++++++++++++++++++++++++++++++++++++++++++++++++
 test-path-utils.c     |  7 +++++++
 5 files changed, 123 insertions(+), 5 deletions(-)

diff --git a/cache.h b/cache.h
index ce377e1..cdafbd7 100644
--- a/cache.h
+++ b/cache.h
@@ -584,6 +584,7 @@ extern int fsync_object_files;
 extern int core_preload_index;
 extern int core_apply_sparse_checkout;
 extern int precomposed_unicode;
+extern int git_db_env, git_index_env, git_graft_env;
 
 /*
  * The character that begins a commented line in user-editable file
diff --git a/environment.c b/environment.c
index 0a15349..1d74dde 100644
--- a/environment.c
+++ b/environment.c
@@ -81,6 +81,7 @@ static size_t namespace_len;
 
 static const char *git_dir;
 static char *git_object_dir, *git_index_file, *git_graft_file;
+int git_db_env, git_index_env, git_graft_env;
 
 /*
  * Repository-local GIT_* environment variables; see cache.h for details.
@@ -134,15 +135,19 @@ static void setup_git_env(void)
 	if (!git_object_dir) {
 		git_object_dir = xmalloc(strlen(git_dir) + 9);
 		sprintf(git_object_dir, "%s/objects", git_dir);
-	}
+	} else
+		git_db_env = 1;
 	git_index_file = getenv(INDEX_ENVIRONMENT);
 	if (!git_index_file) {
 		git_index_file = xmalloc(strlen(git_dir) + 7);
 		sprintf(git_index_file, "%s/index", git_dir);
-	}
+	} else
+		git_index_env = 1;
 	git_graft_file = getenv(GRAFT_ENVIRONMENT);
 	if (!git_graft_file)
 		git_graft_file = git_pathdup("info/grafts");
+	else
+		git_graft_env = 1;
 	if (getenv(NO_REPLACE_OBJECTS_ENVIRONMENT))
 		read_replace_refs = 0;
 	namespace = expand_namespace(getenv(GIT_NAMESPACE_ENVIRONMENT));
diff --git a/path.c b/path.c
index 08aa213..93e3ecc 100644
--- a/path.c
+++ b/path.c
@@ -50,10 +50,60 @@ char *mksnpath(char *buf, size_t n, const char *fmt, ...)
 	return cleanup_path(buf);
 }
 
+static void copy_path(char *dst, size_t n, const char *src)
+{
+	if (strlen(src) < n)
+		strcpy(dst, src);
+	else
+		strlcpy(dst, bad_path, n);
+}
+
+static int dir_prefix(const char *buf, const char *dir)
+{
+	int len = strlen(dir);
+	return !strncmp(buf, dir, len) &&
+		(is_dir_sep(buf[len]) || buf[len] == '\0');
+}
+
+/* $buf =~ m|$dir/+$file| but without regex */
+static int is_dir_file(const char *buf, const char *dir, const char *file)
+{
+	int len = strlen(dir);
+	if (strncmp(buf, dir, len) || !is_dir_sep(buf[len]))
+		return 0;
+	while (is_dir_sep(buf[len]))
+		len++;
+	return !strcmp(buf + len, file);
+}
+
+static void replace_dir(char *buf, size_t n, int len, const char *newdir)
+{
+	int newlen = strlen(newdir);
+	int buflen = strlen(buf);
+	if (buflen - len + newlen >= n) {
+		strlcpy(buf, bad_path, n);
+		return;
+	}
+	memmove(buf + newlen + 1, buf + len, buflen - len + 1);
+	memcpy(buf, newdir, newlen);
+	buf[newlen] = '/';
+}
+
+static void adjust_git_path(char *buf, size_t n, int git_dir_len)
+{
+	char *base = buf + git_dir_len;
+	if (git_graft_env && is_dir_file(base, "info", "grafts"))
+		copy_path(buf, n, get_graft_file());
+	else if (git_index_env && !strcmp(base, "index"))
+		copy_path(buf, n, get_index_file());
+	else if (git_db_env && dir_prefix(base, "objects"))
+		replace_dir(buf, n, git_dir_len + 7, get_object_directory());
+}
+
 static char *git_vsnpath(char *buf, size_t n, const char *fmt, va_list args)
 {
 	const char *git_dir = get_git_dir();
-	size_t len;
+	size_t len, total_len;
 
 	len = strlen(git_dir);
 	if (n < len + 1)
@@ -61,9 +111,10 @@ static char *git_vsnpath(char *buf, size_t n, const char *fmt, va_list args)
 	memcpy(buf, git_dir, len);
 	if (len && !is_dir_sep(git_dir[len-1]))
 		buf[len++] = '/';
-	len += vsnprintf(buf + len, n - len, fmt, args);
-	if (len >= n)
+	total_len = len + vsnprintf(buf + len, n - len, fmt, args);
+	if (total_len >= n)
 		goto bad;
+	adjust_git_path(buf, n, len);
 	return cleanup_path(buf);
 bad:
 	strlcpy(buf, bad_path, n);
diff --git a/t/t0060-path-utils.sh b/t/t0060-path-utils.sh
index 07c10c8..bce16f5 100755
--- a/t/t0060-path-utils.sh
+++ b/t/t0060-path-utils.sh
@@ -223,4 +223,58 @@ relative_path "<null>"		"<empty>"	./
 relative_path "<null>"		"<null>"	./
 relative_path "<null>"		/foo/a/b	./
 
+test_expect_success 'git_path info/grafts without GIT_GRAFT_FILE' '
+	test-path-utils git_path info/grafts >actual1 &&
+	echo .git/info/grafts >expect1 &&
+	test_cmp expect1 actual1
+'
+
+test_expect_success 'git_path info/grafts with GIT_GRAFT_FILE' '
+	GIT_GRAFT_FILE=foo test-path-utils git_path info/grafts >actual2 &&
+	echo foo >expect2 &&
+	test_cmp expect2 actual2
+'
+
+test_expect_success 'git_path info/////grafts with GIT_GRAFT_FILE' '
+	GIT_GRAFT_FILE=foo test-path-utils git_path info/////grafts >actual2 &&
+	echo foo >expect2 &&
+	test_cmp expect2 actual2
+'
+
+test_expect_success 'git_path index' '
+	GIT_INDEX_FILE=foo test-path-utils git_path index >actual &&
+	echo foo >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path index/foo' '
+	GIT_INDEX_FILE=foo test-path-utils git_path index/foo >actual &&
+	echo .git/index/foo >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path index2' '
+	GIT_INDEX_FILE=foo test-path-utils git_path index2 >actual &&
+	echo .git/index2 >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path objects' '
+	GIT_OBJECT_DIRECTORY=foo test-path-utils git_path objects >actual &&
+	echo foo/ >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path objects/foo' '
+	GIT_OBJECT_DIRECTORY=foo test-path-utils git_path objects/foo >actual &&
+	echo foo//foo >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path objects2' '
+	GIT_OBJECT_DIRECTORY=foo test-path-utils git_path objects2 >actual &&
+	echo .git/objects2 >expect &&
+	test_cmp expect actual
+'
+
 test_done
diff --git a/test-path-utils.c b/test-path-utils.c
index 3dd3744..55e4fe0 100644
--- a/test-path-utils.c
+++ b/test-path-utils.c
@@ -135,6 +135,13 @@ int main(int argc, char **argv)
 		return 0;
 	}
 
+	if (argc == 3 && !strcmp(argv[1], "git_path")) {
+		int nongit_ok = 0;
+		setup_git_directory_gently(&nongit_ok);
+		puts(git_path("%s", argv[2]));
+		return 0;
+	}
+
 	fprintf(stderr, "%s: unknown function name: %s\n", argv[0],
 		argv[1] ? argv[1] : "(there was none)");
 	return 1;
-- 
1.8.5.1.77.g42c48fa

[PATCH v2 05/21] reflog: use avoid constructing .lock path with git_path

[Nguyễn Thái Ngọc Duy]

git_path() understands the path given to it. Unrecognized paths by
default go to $GIT_SUPER_DIR/repos/<id>/... in split-repo mode. We
want the %s.lock file to be at the same place %s file is. Avoid
git_path() in this case and use log_file as %s, which should contain
the final path of %s file.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/reflog.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/builtin/reflog.c b/builtin/reflog.c
index 6eb24c8..dc1accf 100644
--- a/builtin/reflog.c
+++ b/builtin/reflog.c
@@ -372,7 +372,7 @@ static int expire_reflog(const char *ref, const unsigned char *sha1, int unused,
 	if (!file_exists(log_file))
 		goto finish;
 	if (!cmd->dry_run) {
-		newlog_path = git_pathdup("logs/%s.lock", ref);
+		newlog_path = mkpathdup("%s.lock", log_file);
 		cb.newlog = fopen(newlog_path, "w");
 	}
 
-- 
1.8.5.1.77.g42c48fa

[PATCH v2 06/21] fast-import: use git_path() for accessing .git dir instead of get_git_dir()

[Nguyễn Thái Ngọc Duy]

This allows git_path() to redirect info/fast-import to another place
if needed

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 fast-import.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/fast-import.c b/fast-import.c
index f4d9969..04bba3d 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -3125,12 +3125,9 @@ static void parse_progress(void)
 
 static char* make_fast_import_path(const char *path)
 {
-	struct strbuf abs_path = STRBUF_INIT;
-
 	if (!relative_marks_paths || is_absolute_path(path))
 		return xstrdup(path);
-	strbuf_addf(&abs_path, "%s/info/fast-import/%s", get_git_dir(), path);
-	return strbuf_detach(&abs_path, NULL);
+	return xstrdup(git_path("info/fast-import/%s", path));
 }
 
 static void option_import_marks(const char *marks,
-- 
1.8.5.1.77.g42c48fa

[PATCH v2 07/21] Add new environment variable $GIT_SUPER_DIR

[Nguyễn Thái Ngọc Duy]

If $GIT_SUPER_DIR is defined, the repository is splitted into two
places:

 - HEAD, logs/HEAD, index, other top-level refs and unrecognized files
   are from $GIT_DIR

 - the rest like objects, refs, info, hooks, packed-refs, shallow...
   are from $GIT_SUPER_DIR

The redirection is done by git_path(), git_pathdup() and git_snpath()

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git.txt |  8 ++++++
 cache.h               |  2 ++
 environment.c         | 11 +++++--
 path.c                | 28 ++++++++++++++++++
 t/t0060-path-utils.sh | 79 +++++++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 126 insertions(+), 2 deletions(-)

diff --git a/Documentation/git.txt b/Documentation/git.txt
index 4448ce2..9f4e9f8 100644
--- a/Documentation/git.txt
+++ b/Documentation/git.txt
@@ -764,6 +764,14 @@ Git so take care if using Cogito etc.
 	an explicit repository directory set via 'GIT_DIR' or on the
 	command line.
 
+'GIT_SUPER_DIR'::
+	If this variable is set to a path, non-worktree files that are
+	normally in $GIT_DIR will be taken from this path
+	instead. Worktree-specific files such as HEAD or index are
+	taken from $GIT_DIR. This variable has lower precedence than
+	other path variables such as GIT_INDEX_FILE,
+	GIT_OBJECT_DIRECTORY...
+
 Git Commits
 ~~~~~~~~~~~
 'GIT_AUTHOR_NAME'::
diff --git a/cache.h b/cache.h
index cdafbd7..823582f 100644
--- a/cache.h
+++ b/cache.h
@@ -347,6 +347,7 @@ static inline enum object_type object_type(unsigned int mode)
 
 /* Double-check local_repo_env below if you add to this list. */
 #define GIT_DIR_ENVIRONMENT "GIT_DIR"
+#define GIT_SUPER_DIR_ENVIRONMENT "GIT_SUPER_DIR"
 #define GIT_NAMESPACE_ENVIRONMENT "GIT_NAMESPACE"
 #define GIT_WORK_TREE_ENVIRONMENT "GIT_WORK_TREE"
 #define GIT_PREFIX_ENVIRONMENT "GIT_PREFIX"
@@ -399,6 +400,7 @@ extern int is_inside_git_dir(void);
 extern char *git_work_tree_cfg;
 extern int is_inside_work_tree(void);
 extern const char *get_git_dir(void);
+extern const char *get_git_super_dir(void);
 extern int is_git_directory(const char *path);
 extern char *get_object_directory(void);
 extern char *get_index_file(void);
diff --git a/environment.c b/environment.c
index 1d74dde..d5ae7a3 100644
--- a/environment.c
+++ b/environment.c
@@ -79,7 +79,7 @@ static char *work_tree;
 static const char *namespace;
 static size_t namespace_len;
 
-static const char *git_dir;
+static const char *git_dir, *git_super_dir;
 static char *git_object_dir, *git_index_file, *git_graft_file;
 int git_db_env, git_index_env, git_graft_env;
 
@@ -131,10 +131,12 @@ static void setup_git_env(void)
 		git_dir = DEFAULT_GIT_DIR_ENVIRONMENT;
 	gitfile = read_gitfile(git_dir);
 	git_dir = xstrdup(gitfile ? gitfile : git_dir);
+	git_super_dir = getenv(GIT_SUPER_DIR_ENVIRONMENT);
 	git_object_dir = getenv(DB_ENVIRONMENT);
 	if (!git_object_dir) {
 		git_object_dir = xmalloc(strlen(git_dir) + 9);
-		sprintf(git_object_dir, "%s/objects", git_dir);
+		sprintf(git_object_dir, "%s/objects",
+			git_super_dir ? git_super_dir : git_dir);
 	} else
 		git_db_env = 1;
 	git_index_file = getenv(INDEX_ENVIRONMENT);
@@ -167,6 +169,11 @@ const char *get_git_dir(void)
 	return git_dir;
 }
 
+const char *get_git_super_dir(void)
+{
+	return git_super_dir;
+}
+
 const char *get_git_namespace(void)
 {
 	if (!namespace)
diff --git a/path.c b/path.c
index 93e3ecc..e51fc35 100644
--- a/path.c
+++ b/path.c
@@ -89,8 +89,34 @@ static void replace_dir(char *buf, size_t n, int len, const char *newdir)
 	buf[newlen] = '/';
 }
 
+static void update_super_dir(char *buf, size_t n, int git_dir_len)
+{
+	const char *super_dir_list[] = {
+		"branches", "hooks", "info", "logs", "modules",
+		"objects", "refs", "remotes", "rr-cache", "svn", NULL
+	};
+	const char *super_top_file_list[] = {
+		"config", "packed-refs", "shallow", NULL
+	};
+	char *base = buf + git_dir_len;
+	const char **p;
+	if (is_dir_file(base, "logs", "HEAD"))
+		return;	/* keep this in $GIT_DIR */
+	for (p = super_dir_list; *p; p++)
+		if (dir_prefix(base, *p)) {
+			replace_dir(buf, n, git_dir_len, get_git_super_dir());
+			return;
+		}
+	for (p = super_top_file_list; *p; p++)
+		if (!strcmp(base, *p)) {
+			replace_dir(buf, n, git_dir_len, get_git_super_dir());
+			return;
+		}
+}
+
 static void adjust_git_path(char *buf, size_t n, int git_dir_len)
 {
+
 	char *base = buf + git_dir_len;
 	if (git_graft_env && is_dir_file(base, "info", "grafts"))
 		copy_path(buf, n, get_graft_file());
@@ -98,6 +124,8 @@ static void adjust_git_path(char *buf, size_t n, int git_dir_len)
 		copy_path(buf, n, get_index_file());
 	else if (git_db_env && dir_prefix(base, "objects"))
 		replace_dir(buf, n, git_dir_len + 7, get_object_directory());
+	else if (get_git_super_dir())
+		update_super_dir(buf, n, git_dir_len);
 }
 
 static char *git_vsnpath(char *buf, size_t n, const char *fmt, va_list args)
diff --git a/t/t0060-path-utils.sh b/t/t0060-path-utils.sh
index bce16f5..5e798c4 100755
--- a/t/t0060-path-utils.sh
+++ b/t/t0060-path-utils.sh
@@ -277,4 +277,83 @@ test_expect_success 'git_path objects2' '
 	test_cmp expect actual
 '
 
+test_expect_success 'git_path super index' '
+	GIT_SUPER_DIR=foo test-path-utils git_path index >actual &&
+	echo .git/index >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path super HEAD' '
+	GIT_SUPER_DIR=foo test-path-utils git_path HEAD >actual &&
+	echo .git/HEAD >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path super objects/*' '
+	GIT_SUPER_DIR=foo test-path-utils git_path objects/foo >actual &&
+	echo foo/objects/foo >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path super info/*' '
+	GIT_SUPER_DIR=foo test-path-utils git_path info/exclude >actual &&
+	echo foo/info/exclude >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path super remotes/*' '
+	GIT_SUPER_DIR=foo test-path-utils git_path remotes/foo >actual &&
+	echo foo/remotes/foo >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path super branches/*' '
+	GIT_SUPER_DIR=foo test-path-utils git_path branches/foo >actual &&
+	echo foo/branches/foo >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path super logs/refs/heads/master' '
+	GIT_SUPER_DIR=foo test-path-utils git_path logs/refs/heads/master >actual &&
+	echo foo/logs/refs/heads/master >expect &&
+	test_cmp expect actual
+'
+
+
+test_expect_success 'git_path super refs/heads/master' '
+	GIT_SUPER_DIR=foo test-path-utils git_path refs/heads/master >actual &&
+	echo foo/refs/heads/master >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path super logs/HEAD' '
+	GIT_SUPER_DIR=foo test-path-utils git_path logs/HEAD >actual &&
+	echo .git/logs/HEAD >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path super hooks/me' '
+	GIT_SUPER_DIR=foo test-path-utils git_path hooks/me >actual &&
+	echo foo/hooks/me >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path super config' '
+	GIT_SUPER_DIR=foo test-path-utils git_path config >actual &&
+	echo foo/config >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path super packed-refs' '
+	GIT_SUPER_DIR=foo test-path-utils git_path packed-refs >actual &&
+	echo foo/packed-refs >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'git_path super shallow' '
+	GIT_SUPER_DIR=foo test-path-utils git_path shallow >actual &&
+	echo foo/shallow >expect &&
+	test_cmp expect actual
+'
+
 test_done
-- 
1.8.5.1.77.g42c48fa

[PATCH v2 08/21] setup.c: refactor path manipulation out of read_gitfile()

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 setup.c | 44 ++++++++++++++++++++++++--------------------
 1 file changed, 24 insertions(+), 20 deletions(-)

diff --git a/setup.c b/setup.c
index 5432a31..c040981 100644
--- a/setup.c
+++ b/setup.c
@@ -278,15 +278,31 @@ static int check_repository_format_gently(const char *gitdir, int *nongit_ok)
 	return 0;
 }
 
+static char *path_from_gitfile(const char *path, const char *buf, int len)
+{
+	const char *slash;
+	if (len < 1)
+		die("No path in gitfile: %s", path);
+
+	if (!is_absolute_path(buf) && (slash = strrchr(path, '/'))) {
+		size_t pathlen = slash+1 - path;
+		size_t dirlen = pathlen + len;
+		char *p = xmalloc(dirlen + 1);
+		strncpy(p, path, pathlen);
+		strncpy(p + pathlen, buf, len);
+		p[dirlen] = '\0';
+		return p;
+	} else
+		return xmemdupz(buf, len);
+}
+
 /*
  * Try to read the location of the git directory from the .git file,
  * return path to git directory if found.
  */
 const char *read_gitfile(const char *path)
 {
-	char *buf;
-	char *dir;
-	const char *slash;
+	char *buf, *dir;
 	struct stat st;
 	int fd;
 	ssize_t len;
@@ -303,31 +319,19 @@ const char *read_gitfile(const char *path)
 	close(fd);
 	if (len != st.st_size)
 		die("Error reading %s", path);
-	buf[len] = '\0';
-	if (prefixcmp(buf, "gitdir: "))
-		die("Invalid gitfile format: %s", path);
-	while (buf[len - 1] == '\n' || buf[len - 1] == '\r')
+	while (len > 0 && (buf[len - 1] == '\n' || buf[len - 1] == '\r'))
 		len--;
-	if (len < 9)
-		die("No path in gitfile: %s", path);
 	buf[len] = '\0';
-	dir = buf + 8;
 
-	if (!is_absolute_path(dir) && (slash = strrchr(path, '/'))) {
-		size_t pathlen = slash+1 - path;
-		size_t dirlen = pathlen + len - 8;
-		dir = xmalloc(dirlen + 1);
-		strncpy(dir, path, pathlen);
-		strncpy(dir + pathlen, buf + 8, len - 8);
-		dir[dirlen] = '\0';
-		free(buf);
-		buf = dir;
-	}
+	if (prefixcmp(buf, "gitdir: "))
+		die("Invalid gitfile format: %s", path);
+	dir = path_from_gitfile(path, buf + 8, len - 8);
 
 	if (!is_git_directory(dir))
 		die("Not a git repository: %s", dir);
 	path = real_path(dir);
 
+	free(dir);
 	free(buf);
 	return path;
 }
-- 
1.8.5.1.77.g42c48fa

[PATCH v2 09/21] setup.c: add split-repo support to .git files

[Nguyễn Thái Ngọc Duy]

If a .git file contains

gitsuper: <path>
gitdir: <id>

then we set GIT_SUPER_DIR to <path> and GIT_DIR to
$GIT_SUPER_DIR/repos/<id>.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 cache.h |  1 +
 setup.c | 41 +++++++++++++++++++++++++++++++++++++----
 2 files changed, 38 insertions(+), 4 deletions(-)

diff --git a/cache.h b/cache.h
index 823582f..f85ee70 100644
--- a/cache.h
+++ b/cache.h
@@ -410,6 +410,7 @@ extern const char *get_git_namespace(void);
 extern const char *strip_namespace(const char *namespaced_ref);
 extern const char *get_git_work_tree(void);
 extern const char *read_gitfile(const char *path);
+extern const char *read_gitfile_super(const char *path, char **super);
 extern const char *resolve_gitdir(const char *suspect);
 extern void set_git_work_tree(const char *tree);
 
diff --git a/setup.c b/setup.c
index c040981..e9f0ef6 100644
--- a/setup.c
+++ b/setup.c
@@ -299,14 +299,20 @@ static char *path_from_gitfile(const char *path, const char *buf, int len)
 /*
  * Try to read the location of the git directory from the .git file,
  * return path to git directory if found.
+ *
+ * If "gitsuper: " line is found and super is not NULL, *super points
+ * to the absolute path of the given path. The next line contains the
+ * repo id.
  */
-const char *read_gitfile(const char *path)
+const char *read_gitfile_super(const char *path, char **super)
 {
 	char *buf, *dir;
 	struct stat st;
 	int fd;
 	ssize_t len;
 
+	if (super)
+		*super = NULL;
 	if (stat(path, &st))
 		return NULL;
 	if (!S_ISREG(st.st_mode))
@@ -323,9 +329,30 @@ const char *read_gitfile(const char *path)
 		len--;
 	buf[len] = '\0';
 
-	if (prefixcmp(buf, "gitdir: "))
-		die("Invalid gitfile format: %s", path);
-	dir = path_from_gitfile(path, buf + 8, len - 8);
+	if (super && !prefixcmp(buf, "gitsuper: ")) {
+		char *p = buf, *end = buf + len;
+		while (p < end && *p != '\n' && *p != '\r')
+			p++;
+		if (p == end)
+			die("Invalid gitfile format: %s", path);
+		*super = buf + strlen("gitsuper: ");
+		*p = '\0';
+		*super = path_from_gitfile(path, *super, p - *super);
+		p++;
+		while (p < end && (*p == '\n' || *p == '\r'))
+			p++;
+		if (prefixcmp(p, "gitdir: "))
+			die("Invalid gitfile format: %s", path);
+		p += 8;
+		if (p == end)
+			die("Invalid gitfile format: %s", path);
+		dir = xmalloc(strlen(*super) + strlen("/repos/") + (end - p) + 1);
+		sprintf(dir, "%s/repos/%s", *super, p);
+	} else {
+		if (prefixcmp(buf, "gitdir: "))
+			die("Invalid gitfile format: %s", path);
+		dir = path_from_gitfile(path, buf + 8, len - 8);
+	}
 
 	if (!is_git_directory(dir))
 		die("Not a git repository: %s", dir);
@@ -336,6 +363,12 @@ const char *read_gitfile(const char *path)
 	return path;
 }
 
+const char *read_gitfile(const char *path)
+{
+	return read_gitfile_super(path, NULL);
+}
+
+
 static const char *setup_explicit_git_dir(const char *gitdirenv,
 					  char *cwd, int len,
 					  int *nongit_ok)
-- 
1.8.5.1.77.g42c48fa

[PATCH v2 10/21] setup.c: add split-repo support to is_git_directory()

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 setup.c | 28 ++++++++++++++++++++++------
 1 file changed, 22 insertions(+), 6 deletions(-)

diff --git a/setup.c b/setup.c
index e9f0ef6..01fe89a 100644
--- a/setup.c
+++ b/setup.c
@@ -182,28 +182,38 @@ void verify_non_filename(const char *prefix, const char *arg)
  *    a proper "ref:", or a regular file HEAD that has a properly
  *    formatted sha1 object name.
  */
-int is_git_directory(const char *suspect)
+static int is_git_directory_super(const char *suspect, const char *super)
 {
 	char path[PATH_MAX];
 	size_t len = strlen(suspect);
+	size_t super_len;
+
+	if (!super) {
+		super = suspect;
+		super_len = len;
+	} else
+		super_len = strlen(super);
 
-	if (PATH_MAX <= len + strlen("/objects"))
+	if (PATH_MAX <= super_len + strlen("/objects") ||
+	    PATH_MAX <= len + strlen("/HEAD"))
 		die("Too long path: %.*s", 60, suspect);
-	strcpy(path, suspect);
+	strcpy(path, super);
 	if (getenv(DB_ENVIRONMENT)) {
 		if (access(getenv(DB_ENVIRONMENT), X_OK))
 			return 0;
 	}
 	else {
-		strcpy(path + len, "/objects");
+		strcpy(path + super_len, "/objects");
 		if (access(path, X_OK))
 			return 0;
 	}
 
-	strcpy(path + len, "/refs");
+	strcpy(path + super_len, "/refs");
 	if (access(path, X_OK))
 		return 0;
 
+	if (super != suspect)
+		strcpy(path, suspect);
 	strcpy(path + len, "/HEAD");
 	if (validate_headref(path))
 		return 0;
@@ -211,6 +221,12 @@ int is_git_directory(const char *suspect)
 	return 1;
 }
 
+int is_git_directory(const char *suspect)
+{
+	return is_git_directory_super(suspect, NULL);
+}
+
+
 int is_inside_git_dir(void)
 {
 	if (inside_git_dir < 0)
@@ -354,7 +370,7 @@ const char *read_gitfile_super(const char *path, char **super)
 		dir = path_from_gitfile(path, buf + 8, len - 8);
 	}
 
-	if (!is_git_directory(dir))
+	if (!is_git_directory_super(dir, super ? *super : NULL))
 		die("Not a git repository: %s", dir);
 	path = real_path(dir);
 
-- 
1.8.5.1.77.g42c48fa

[PATCH v2 11/21] setup.c: reduce cleanup sites in setup_explicit_git_dir()

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 setup.c | 20 ++++++++------------
 1 file changed, 8 insertions(+), 12 deletions(-)

diff --git a/setup.c b/setup.c
index 01fe89a..397eecc 100644
--- a/setup.c
+++ b/setup.c
@@ -406,16 +406,13 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
 	if (!is_git_directory(gitdirenv)) {
 		if (nongit_ok) {
 			*nongit_ok = 1;
-			free(gitfile);
-			return NULL;
+			goto done_null;
 		}
 		die("Not a git repository: '%s'", gitdirenv);
 	}
 
-	if (check_repository_format_gently(gitdirenv, nongit_ok)) {
-		free(gitfile);
-		return NULL;
-	}
+	if (check_repository_format_gently(gitdirenv, nongit_ok))
+		goto done_null;
 
 	/* #3, #7, #11, #15, #19, #23, #27, #31 (see t1510) */
 	if (work_tree_env)
@@ -426,8 +423,7 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
 
 		/* #18, #26 */
 		set_git_dir(gitdirenv);
-		free(gitfile);
-		return NULL;
+		goto done_null;
 	}
 	else if (git_work_tree_cfg) { /* #6, #14 */
 		if (is_absolute_path(git_work_tree_cfg))
@@ -448,8 +444,7 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
 	else if (!git_env_bool(GIT_IMPLICIT_WORK_TREE_ENVIRONMENT, 1)) {
 		/* #16d */
 		set_git_dir(gitdirenv);
-		free(gitfile);
-		return NULL;
+		goto done_null;
 	}
 	else /* #2, #10 */
 		set_git_work_tree(".");
@@ -460,8 +455,7 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
 	/* both get_git_work_tree() and cwd are already normalized */
 	if (!strcmp(cwd, worktree)) { /* cwd == worktree */
 		set_git_dir(gitdirenv);
-		free(gitfile);
-		return NULL;
+		goto done_null;
 	}
 
 	offset = dir_inside_of(cwd, worktree);
@@ -477,6 +471,8 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
 
 	/* cwd outside worktree */
 	set_git_dir(gitdirenv);
+
+done_null:
 	free(gitfile);
 	return NULL;
 }
-- 
1.8.5.1.77.g42c48fa

[PATCH v2 12/21] environment.c: support super .git file specified by $GIT_DIR

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 environment.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/environment.c b/environment.c
index d5ae7a3..cbfa879 100644
--- a/environment.c
+++ b/environment.c
@@ -125,13 +125,17 @@ static char *expand_namespace(const char *raw_namespace)
 static void setup_git_env(void)
 {
 	const char *gitfile;
+	char *super;
 
 	git_dir = getenv(GIT_DIR_ENVIRONMENT);
 	if (!git_dir)
 		git_dir = DEFAULT_GIT_DIR_ENVIRONMENT;
-	gitfile = read_gitfile(git_dir);
+	gitfile = read_gitfile_super(git_dir, &super);
 	git_dir = xstrdup(gitfile ? gitfile : git_dir);
-	git_super_dir = getenv(GIT_SUPER_DIR_ENVIRONMENT);
+	if (super)
+		git_super_dir = xstrdup(super);
+	else
+		git_super_dir = getenv(GIT_SUPER_DIR_ENVIRONMENT);
 	git_object_dir = getenv(DB_ENVIRONMENT);
 	if (!git_object_dir) {
 		git_object_dir = xmalloc(strlen(git_dir) + 9);
-- 
1.8.5.1.77.g42c48fa

[PATCH v2 13/21] setup: support $GIT_SUPER_DIR as well as super .git files

[Nguyễn Thái Ngọc Duy]

The same rules for git repo setup apply except:

- detect super .git files, set $GIT_SUPER_DIR accordingly

- if $GIT_SUPER_DIR is set (or specified by super .git files), look
  for non-worktree stuff in this directory instead of $GIT_DIR. This
  mostly affects is_git_directory() and check_repository_format()

- the worktree setting precedence goes from lower to higher:
  core.worktree, GIT_SUPER_DIR then GIT_WORK_TREE

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/config.txt               |  3 +-
 Documentation/gitrepository-layout.txt | 10 ++++
 builtin/rev-parse.c                    |  6 +++
 cache.h                                |  1 +
 environment.c                          | 11 +++++
 setup.c                                | 89 ++++++++++++++++++++++++----------
 t/t1501-worktree.sh                    | 52 ++++++++++++++++++++
 t/t1510-repo-setup.sh                  |  1 +
 trace.c                                |  1 +
 9 files changed, 147 insertions(+), 27 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index a405806..df19aa8 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -381,7 +381,8 @@ false), while all other repositories are assumed to be bare (bare
 
 core.worktree::
 	Set the path to the root of the working tree.
-	This can be overridden by the GIT_WORK_TREE environment
+	This can be overridden by the GIT_WORK_TREE
+	or GIT_SUPER_DIR environment
 	variable and the '--work-tree' command line option.
 	The value can be an absolute path or relative to the path to
 	the .git directory, which is either specified by --git-dir
diff --git a/Documentation/gitrepository-layout.txt b/Documentation/gitrepository-layout.txt
index aa03882..7ce31d4 100644
--- a/Documentation/gitrepository-layout.txt
+++ b/Documentation/gitrepository-layout.txt
@@ -29,6 +29,10 @@ containing superproject to `git checkout` a branch that does not
 have the submodule.  The `checkout` has to remove the entire
 submodule working tree, without losing the submodule repository.
 
+If the text file contains `gitsuper: <path>`, this is a "working
+directory only" repository, attached to another repository and share
+everything with the attached repository except HEAD and the index.
+
 These things may exist in a Git repository.
 
 objects::
@@ -214,6 +218,12 @@ shallow::
 modules::
 	Contains the git-repositories of the submodules.
 
+repos/<id>::
+	If a repository's .git is a file contains two lines `gitsuper:
+	<path>` and `repo: <id>`. The directory `<path>/repos/<id>`
+	contains the real non-shared part of .git directory of the
+	repository in question (e.g. HEAD or index).
+
 SEE ALSO
 --------
 linkgit:git-init[1],
diff --git a/builtin/rev-parse.c b/builtin/rev-parse.c
index 1d9ecaf..f5f766a 100644
--- a/builtin/rev-parse.c
+++ b/builtin/rev-parse.c
@@ -725,6 +725,12 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix)
 				printf("%s%s.git\n", cwd, len && cwd[len-1] != '/' ? "/" : "");
 				continue;
 			}
+			if (!strcmp(arg, "--git-super-dir")) {
+				const char *gitdir = getenv(GIT_SUPER_DIR_ENVIRONMENT);
+				if (gitdir)
+					puts(gitdir);
+				continue;
+			}
 			if (!strcmp(arg, "--resolve-git-dir")) {
 				const char *gitdir = resolve_gitdir(argv[i+1]);
 				if (!gitdir)
diff --git a/cache.h b/cache.h
index f85ee70..4c09223 100644
--- a/cache.h
+++ b/cache.h
@@ -406,6 +406,7 @@ extern char *get_object_directory(void);
 extern char *get_index_file(void);
 extern char *get_graft_file(void);
 extern int set_git_dir(const char *path);
+extern int set_git_dir_super(const char *path, const char *super);
 extern const char *get_git_namespace(void);
 extern const char *strip_namespace(const char *namespaced_ref);
 extern const char *get_git_work_tree(void);
diff --git a/environment.c b/environment.c
index cbfa879..5cbbe11 100644
--- a/environment.c
+++ b/environment.c
@@ -284,6 +284,17 @@ int set_git_dir(const char *path)
 	return 0;
 }
 
+int set_git_dir_super(const char *path, const char *super)
+{
+	if (super && super != path &&
+	    setenv(GIT_SUPER_DIR_ENVIRONMENT, super, 1))
+		return error("Could not set GIT_SUPER_DIR to '%s'", super);
+	if (setenv(GIT_DIR_ENVIRONMENT, path, 1))
+		return error("Could not set GIT_DIR to '%s'", path);
+	setup_git_env();
+	return 0;
+}
+
 const char *get_log_output_encoding(void)
 {
 	return git_log_output_encoding ? git_log_output_encoding
diff --git a/setup.c b/setup.c
index 397eecc..7ccb1f8 100644
--- a/setup.c
+++ b/setup.c
@@ -243,7 +243,8 @@ int is_inside_work_tree(void)
 
 void setup_work_tree(void)
 {
-	const char *work_tree, *git_dir;
+	const char *work_tree, *git_dir, *git_super_dir;
+	char *super_new = NULL, *super_relative = NULL;
 	static int initialized = 0;
 
 	if (initialized)
@@ -252,6 +253,9 @@ void setup_work_tree(void)
 	git_dir = get_git_dir();
 	if (!is_absolute_path(git_dir))
 		git_dir = real_path(get_git_dir());
+	git_super_dir = get_git_super_dir();
+	if (git_super_dir && !is_absolute_path(git_super_dir))
+		git_super_dir = super_new = xstrdup(real_path(get_git_super_dir()));
 	if (!work_tree || chdir(work_tree))
 		die("This operation must be run in a work tree");
 
@@ -262,8 +266,16 @@ void setup_work_tree(void)
 	if (getenv(GIT_WORK_TREE_ENVIRONMENT))
 		setenv(GIT_WORK_TREE_ENVIRONMENT, ".", 1);
 
-	set_git_dir(remove_leading_path(git_dir, work_tree));
+	if (git_super_dir) {
+		super_relative = xstrdup(remove_leading_path(git_super_dir, work_tree));
+		git_super_dir = super_relative;
+	}
+
+	set_git_dir_super(remove_leading_path(git_dir, work_tree),
+			  git_super_dir);
 	initialized = 1;
+	free(super_new);
+	free(super_relative);
 }
 
 static int check_repository_format_gently(const char *gitdir, int *nongit_ok)
@@ -386,24 +398,30 @@ const char *read_gitfile(const char *path)
 
 
 static const char *setup_explicit_git_dir(const char *gitdirenv,
+					  const char *super,
 					  char *cwd, int len,
 					  int *nongit_ok)
 {
 	const char *work_tree_env = getenv(GIT_WORK_TREE_ENVIRONMENT);
 	const char *worktree;
-	char *gitfile;
+	char *gitfile, *super_new = NULL;
 	int offset;
 
 	if (PATH_MAX - 40 < strlen(gitdirenv))
 		die("'$%s' too big", GIT_DIR_ENVIRONMENT);
 
-	gitfile = (char*)read_gitfile(gitdirenv);
+	if (super)
+		gitfile = (char*)read_gitfile(gitdirenv);
+	else {
+		gitfile = (char*)read_gitfile_super(gitdirenv, &super_new);
+		super = super_new;
+	}
 	if (gitfile) {
 		gitfile = xstrdup(gitfile);
 		gitdirenv = gitfile;
 	}
 
-	if (!is_git_directory(gitdirenv)) {
+	if (!is_git_directory_super(gitdirenv, super)) {
 		if (nongit_ok) {
 			*nongit_ok = 1;
 			goto done_null;
@@ -411,7 +429,7 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
 		die("Not a git repository: '%s'", gitdirenv);
 	}
 
-	if (check_repository_format_gently(gitdirenv, nongit_ok))
+	if (check_repository_format_gently(super ? super : gitdirenv, nongit_ok))
 		goto done_null;
 
 	/* #3, #7, #11, #15, #19, #23, #27, #31 (see t1510) */
@@ -422,10 +440,10 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
 			die("core.bare and core.worktree do not make sense");
 
 		/* #18, #26 */
-		set_git_dir(gitdirenv);
+		set_git_dir_super(gitdirenv, super);
 		goto done_null;
 	}
-	else if (git_work_tree_cfg) { /* #6, #14 */
+	else if (git_work_tree_cfg && !super) { /* #6, #14 */
 		if (is_absolute_path(git_work_tree_cfg))
 			set_git_work_tree(git_work_tree_cfg);
 		else {
@@ -443,7 +461,7 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
 	}
 	else if (!git_env_bool(GIT_IMPLICIT_WORK_TREE_ENVIRONMENT, 1)) {
 		/* #16d */
-		set_git_dir(gitdirenv);
+		set_git_dir_super(gitdirenv, super);
 		goto done_null;
 	}
 	else /* #2, #10 */
@@ -454,43 +472,47 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
 
 	/* both get_git_work_tree() and cwd are already normalized */
 	if (!strcmp(cwd, worktree)) { /* cwd == worktree */
-		set_git_dir(gitdirenv);
+		set_git_dir_super(gitdirenv, super);
 		goto done_null;
 	}
 
 	offset = dir_inside_of(cwd, worktree);
 	if (offset >= 0) {	/* cwd inside worktree? */
-		set_git_dir(real_path(gitdirenv));
+		set_git_dir_super(real_path(gitdirenv), super);
 		if (chdir(worktree))
 			die_errno("Could not chdir to '%s'", worktree);
 		cwd[len++] = '/';
 		cwd[len] = '\0';
 		free(gitfile);
+		free(super_new);
 		return cwd + offset;
 	}
 
 	/* cwd outside worktree */
-	set_git_dir(gitdirenv);
+	set_git_dir_super(gitdirenv, super);
 
 done_null:
 	free(gitfile);
+	free(super_new);
 	return NULL;
 }
 
 static const char *setup_discovered_git_dir(const char *gitdir,
+					    const char *super,
 					    char *cwd, int offset, int len,
 					    int *nongit_ok)
 {
-	if (check_repository_format_gently(gitdir, nongit_ok))
+	if (check_repository_format_gently(super ? super : gitdir, nongit_ok))
 		return NULL;
 
 	/* --work-tree is set without --git-dir; use discovered one */
-	if (getenv(GIT_WORK_TREE_ENVIRONMENT) || git_work_tree_cfg) {
+	if (getenv(GIT_WORK_TREE_ENVIRONMENT) ||
+	    (git_work_tree_cfg && !super)) {
 		if (offset != len && !is_absolute_path(gitdir))
 			gitdir = xstrdup(real_path(gitdir));
 		if (chdir(cwd))
 			die_errno("Could not come back to cwd");
-		return setup_explicit_git_dir(gitdir, cwd, len, nongit_ok);
+		return setup_explicit_git_dir(gitdir, super, cwd, len, nongit_ok);
 	}
 
 	/* #16.2, #17.2, #20.2, #21.2, #24, #25, #28, #29 (see t1510) */
@@ -503,8 +525,8 @@ static const char *setup_discovered_git_dir(const char *gitdir,
 
 	/* #0, #1, #5, #8, #9, #12, #13 */
 	set_git_work_tree(".");
-	if (strcmp(gitdir, DEFAULT_GIT_DIR_ENVIRONMENT))
-		set_git_dir(gitdir);
+	if (strcmp(gitdir, DEFAULT_GIT_DIR_ENVIRONMENT) || super)
+		set_git_dir_super(gitdir, super);
 	inside_git_dir = 0;
 	inside_work_tree = 1;
 	if (offset == len)
@@ -534,7 +556,7 @@ static const char *setup_bare_git_dir(char *cwd, int offset, int len, int *nongi
 		gitdir = offset == len ? "." : xmemdupz(cwd, offset);
 		if (chdir(cwd))
 			die_errno("Could not come back to cwd");
-		return setup_explicit_git_dir(gitdir, cwd, len, nongit_ok);
+		return setup_explicit_git_dir(gitdir, NULL, cwd, len, nongit_ok);
 	}
 
 	inside_git_dir = 1;
@@ -613,8 +635,8 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 	const char *env_ceiling_dirs = getenv(CEILING_DIRECTORIES_ENVIRONMENT);
 	struct string_list ceiling_dirs = STRING_LIST_INIT_DUP;
 	static char cwd[PATH_MAX + 1];
-	const char *gitdirenv, *ret;
-	char *gitfile;
+	const char *gitdirenv, *ret, *super;
+	char *gitfile, *super_new = NULL;
 	int len, offset, offset_parent, ceil_offset = -1;
 	dev_t current_device = 0;
 	int one_filesystem = 1;
@@ -637,8 +659,10 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 	 * validation.
 	 */
 	gitdirenv = getenv(GIT_DIR_ENVIRONMENT);
+	super = getenv(GIT_SUPER_DIR_ENVIRONMENT);
 	if (gitdirenv)
-		return setup_explicit_git_dir(gitdirenv, cwd, len, nongit_ok);
+		return setup_explicit_git_dir(gitdirenv, super,
+					      cwd, len, nongit_ok);
 
 	if (env_ceiling_dirs) {
 		int empty_entry_found = 0;
@@ -668,25 +692,38 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 	if (one_filesystem)
 		current_device = get_device_or_die(".", NULL, 0);
 	for (;;) {
-		gitfile = (char*)read_gitfile(DEFAULT_GIT_DIR_ENVIRONMENT);
+		if (super)
+			gitfile = (char*)read_gitfile(DEFAULT_GIT_DIR_ENVIRONMENT);
+		else {
+			gitfile = (char*)read_gitfile_super(DEFAULT_GIT_DIR_ENVIRONMENT,
+							    &super_new);
+			super = super_new;
+		}
 		if (gitfile)
 			gitdirenv = gitfile = xstrdup(gitfile);
 		else {
-			if (is_git_directory(DEFAULT_GIT_DIR_ENVIRONMENT))
+			if (is_git_directory_super(DEFAULT_GIT_DIR_ENVIRONMENT, super))
 				gitdirenv = DEFAULT_GIT_DIR_ENVIRONMENT;
 		}
 
 		if (gitdirenv) {
-			ret = setup_discovered_git_dir(gitdirenv,
+			ret = setup_discovered_git_dir(gitdirenv, super,
 						       cwd, offset, len,
 						       nongit_ok);
 			free(gitfile);
+			free(super_new);
 			return ret;
 		}
 		free(gitfile);
 
-		if (is_git_directory("."))
-			return setup_bare_git_dir(cwd, offset, len, nongit_ok);
+		if (is_git_directory(".")) {
+			ret = setup_bare_git_dir(cwd, offset, len, nongit_ok);
+			free(super_new);
+			return ret;
+		}
+
+		free(super_new);
+		super_new = NULL;
 
 		offset_parent = offset;
 		while (--offset_parent > ceil_offset && cwd[offset_parent] != '/');
diff --git a/t/t1501-worktree.sh b/t/t1501-worktree.sh
index 8f36aa9..dcd8d16 100755
--- a/t/t1501-worktree.sh
+++ b/t/t1501-worktree.sh
@@ -346,4 +346,56 @@ test_expect_success 'relative $GIT_WORK_TREE and git subprocesses' '
 	test_cmp expected actual
 '
 
+test_expect_success 'Split repo setup' '
+	mkdir work &&
+	mkdir -p repo.git/repos/foo &&
+	cp repo.git/HEAD repo.git/index repo.git/repos/foo
+'
+
+test_expect_success 'GIT_SUPER_DIR set' '
+	unset GIT_WORK_TREE GIT_DIR GIT_CONFIG &&
+	GIT_SUPER_DIR=repo.git GIT_DIR=repo.git/repos/foo git rev-parse --git-super-dir >actual &&
+	echo repo.git >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'GIT_DIR set (1)' '
+	(
+		cat <<EOF >gitfile &&
+gitsuper: repo.git
+gitdir: foo
+EOF
+		cd work &&
+		GIT_DIR=../gitfile git rev-parse --git-super-dir >actual &&
+		echo ../repo.git >expect &&
+		test_cmp expect actual
+	)
+'
+
+test_expect_success 'GIT_DIR set (2)' '
+	(
+		cat <<EOF >gitfile &&
+gitsuper: $TRASH_DIRECTORY/repo.git
+gitdir: foo
+EOF
+		cd work &&
+		GIT_DIR=../gitfile git rev-parse --git-super-dir >actual &&
+		echo "$TRASH_DIRECTORY"/repo.git >expect &&
+		test_cmp expect actual
+	)
+'
+
+test_expect_success 'Auto discovery' '
+	(
+		cat <<EOF >.git &&
+gitsuper: repo.git
+gitdir: foo
+EOF
+		cd work &&
+		git rev-parse --git-super-dir >actual &&
+		echo repo.git >expect &&
+		test_cmp expect actual
+	)
+'
+
 test_done
diff --git a/t/t1510-repo-setup.sh b/t/t1510-repo-setup.sh
index cf2ee78..713f4d7 100755
--- a/t/t1510-repo-setup.sh
+++ b/t/t1510-repo-setup.sh
@@ -106,6 +106,7 @@ setup_env () {
 expect () {
 	cat >"$1/expected" <<-EOF
 	setup: git_dir: $2
+	setup: git_super_dir: (null)
 	setup: worktree: $3
 	setup: cwd: $4
 	setup: prefix: $5
diff --git a/trace.c b/trace.c
index 3d744d1..53d800b 100644
--- a/trace.c
+++ b/trace.c
@@ -173,6 +173,7 @@ void trace_repo_setup(const char *prefix)
 		prefix = "(null)";
 
 	trace_printf_key(key, "setup: git_dir: %s\n", quote_crnl(get_git_dir()));
+	trace_printf_key(key, "setup: git_super_dir: %s\n", quote_crnl(get_git_super_dir()));
 	trace_printf_key(key, "setup: worktree: %s\n", quote_crnl(git_work_tree));
 	trace_printf_key(key, "setup: cwd: %s\n", quote_crnl(cwd));
 	trace_printf_key(key, "setup: prefix: %s\n", quote_crnl(prefix));
-- 
1.8.5.1.77.g42c48fa

[PATCH v2 14/21] checkout: support checking out into a new working directory

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git-checkout.txt         |  6 +++
 Documentation/gitrepository-layout.txt |  3 +-
 builtin/checkout.c                     | 94 ++++++++++++++++++++++++++++++++++
 path.c                                 |  3 +-
 4 files changed, 104 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
index 91294f8..06076c8 100644
--- a/Documentation/git-checkout.txt
+++ b/Documentation/git-checkout.txt
@@ -225,6 +225,12 @@ This means that you can use `git checkout -p` to selectively discard
 edits from your current working tree. See the ``Interactive Mode''
 section of linkgit:git-add[1] to learn how to operate the `--patch` mode.
 
+--to=<path>::
+	Check out a new branch in a separate working directory at
+	`<path>`. A new repository is initialized at `<path>` that
+	shares everything with the current repository except working
+	directory specific files.
+
 <branch>::
 	Branch to checkout; if it refers to a branch (i.e., a name that,
 	when prepended with "refs/heads/", is a valid ref), then that
diff --git a/Documentation/gitrepository-layout.txt b/Documentation/gitrepository-layout.txt
index 7ce31d4..3c6149e 100644
--- a/Documentation/gitrepository-layout.txt
+++ b/Documentation/gitrepository-layout.txt
@@ -222,7 +222,8 @@ repos/<id>::
 	If a repository's .git is a file contains two lines `gitsuper:
 	<path>` and `repo: <id>`. The directory `<path>/repos/<id>`
 	contains the real non-shared part of .git directory of the
-	repository in question (e.g. HEAD or index).
+	repository in question (e.g. HEAD or index). Such .git files
+	are created by `git checkout --to`.
 
 SEE ALSO
 --------
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 904fd71..95a1a61 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -48,6 +48,9 @@ struct checkout_opts {
 	const char *prefix;
 	struct pathspec pathspec;
 	struct tree *source_tree;
+
+	const char *new_worktree;
+	const char **saved_argv;
 };
 
 static int post_checkout_hook(struct commit *old, struct commit *new,
@@ -250,6 +253,9 @@ static int checkout_paths(const struct checkout_opts *opts,
 		die(_("Cannot update paths and switch to branch '%s' at the same time."),
 		    opts->new_branch);
 
+	if (opts->new_worktree)
+		die(_("'%s' cannot be used with updating paths"), "--to");
+
 	if (opts->patch_mode)
 		return run_add_interactive(revision, "--patch=checkout",
 					   &opts->pathspec);
@@ -808,6 +814,82 @@ static int switch_branches(const struct checkout_opts *opts,
 	return ret || writeout_error;
 }
 
+static void write_to_file(const char *path, const char *fmt, ...)
+{
+	va_list params;
+	FILE *fp = fopen(path, "w");
+	if (!fp)
+		die_errno(_("could not create file '%s'"), path);
+	va_start(params, fmt);
+	vfprintf(fp, fmt, params);
+	va_end(params);
+	fclose(fp);
+}
+
+static int checkout_new_worktree(const struct checkout_opts *opts,
+				 struct branch_info *new)
+{
+	struct strbuf sb_git = STRBUF_INIT, sb_repo = STRBUF_INIT;
+	struct strbuf sb = STRBUF_INIT;
+	const char *path = opts->new_worktree;
+	struct stat st;
+	const char *name;
+	struct child_process cp;
+	int counter = 0, len;
+
+	if (!new->commit)
+		die(_("no branch specified"));
+
+	len = strlen(path);
+	if (!len || is_dir_sep(path[len - 1]))
+		die(_("'--to' argument '%s' cannot end with a slash"), path);
+
+	for (name = path + len - 1; name > path; name--)
+		if (is_dir_sep(*name)) {
+			name++;
+			break;
+		}
+	strbuf_addstr(&sb_repo, git_path("repos/%s", name));
+	len = sb_repo.len;
+	if (safe_create_leading_directories_const(sb_repo.buf))
+		die_errno(_("could not create leading directories of '%s'"),
+			  sb_repo.buf);
+	while (!stat(sb_repo.buf, &st)) {
+		counter++;
+		strbuf_setlen(&sb_repo, len);
+		strbuf_addf(&sb_repo, "%d", counter);
+	}
+	name = sb_repo.buf + len - strlen(name);
+	if (mkdir(sb_repo.buf, 0777))
+		die_errno(_("could not create directory of '%s'"), sb_repo.buf);
+
+	strbuf_addf(&sb_git, "%s/.git", path);
+	if (safe_create_leading_directories_const(sb_git.buf))
+		die_errno(_("could not create leading directories of '%s'"),
+			  sb_git.buf);
+
+	write_to_file(sb_git.buf, "gitsuper: %s\ngitdir: %s\n",
+		      real_path(get_git_dir()), name);
+	strbuf_addf(&sb, "%s/HEAD", sb_repo.buf);
+	write_to_file(sb.buf, "%s\n", sha1_to_hex(new->commit->object.sha1));
+
+	if (!opts->quiet)
+		fprintf_ln(stderr, _("Enter %s (identifier %s)"), path, name);
+
+	/*
+	 * Rerun checkout in the new worktree. This way is safer than
+	 * set_git_dir() because a path relative to cwd could have
+	 * been cached somewhere undetected.
+	 */
+	setenv("GIT_CHECKOUT_NEW_WORKTREE", "1", 1);
+	setenv(GIT_DIR_ENVIRONMENT, sb_git.buf, 1);
+	setenv(GIT_WORK_TREE_ENVIRONMENT, path, 1);
+	memset(&cp, 0, sizeof(cp));
+	cp.git_cmd = 1;
+	cp.argv = opts->saved_argv;
+	return run_command(&cp);
+}
+
 static int git_checkout_config(const char *var, const char *value, void *cb)
 {
 	if (!strcmp(var, "diff.ignoresubmodules")) {
@@ -1069,6 +1151,9 @@ static int checkout_branch(struct checkout_opts *opts,
 		die(_("Cannot switch branch to a non-commit '%s'"),
 		    new->name);
 
+	if (opts->new_worktree)
+		return checkout_new_worktree(opts, new);
+
 	if (!new->commit && opts->new_branch) {
 		unsigned char rev[20];
 		int flag;
@@ -1111,6 +1196,8 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 			 N_("do not limit pathspecs to sparse entries only")),
 		OPT_HIDDEN_BOOL(0, "guess", &dwim_new_local_branch,
 				N_("second guess 'git checkout no-such-branch'")),
+		OPT_STRING(0, "to", &opts.new_worktree, N_("path"),
+			   N_("check a branch out in a separate working directory")),
 		OPT_END(),
 	};
 
@@ -1119,6 +1206,9 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 	opts.overwrite_ignore = 1;
 	opts.prefix = prefix;
 
+	opts.saved_argv = xmalloc(sizeof(const char *) * (argc + 2));
+	memcpy(opts.saved_argv, argv, sizeof(const char *) * (argc + 1));
+
 	gitmodules_config();
 	git_config(git_checkout_config, &opts);
 
@@ -1127,6 +1217,10 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 	argc = parse_options(argc, argv, prefix, options, checkout_usage,
 			     PARSE_OPT_KEEP_DASHDASH);
 
+	if (getenv("GIT_CHECKOUT_NEW_WORKTREE"))
+		/* recursive execution from checkout_new_worktree() */
+		opts.new_worktree = NULL;
+
 	if (conflict_style) {
 		opts.merge = 1; /* implied */
 		git_xmerge_config("merge.conflictstyle", conflict_style, NULL);
diff --git a/path.c b/path.c
index e51fc35..1a2478a 100644
--- a/path.c
+++ b/path.c
@@ -93,7 +93,8 @@ static void update_super_dir(char *buf, size_t n, int git_dir_len)
 {
 	const char *super_dir_list[] = {
 		"branches", "hooks", "info", "logs", "modules",
-		"objects", "refs", "remotes", "rr-cache", "svn", NULL
+		"objects", "refs", "remotes", "repos", "rr-cache",
+		"svn", NULL
 	};
 	const char *super_top_file_list[] = {
 		"config", "packed-refs", "shallow", NULL
-- 
1.8.5.1.77.g42c48fa

[PATCH v2 15/21] checkout: clean up half-prepared directories in --to mode

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/checkout.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 47 insertions(+), 2 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 95a1a61..6353557 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -20,6 +20,7 @@
 #include "resolve-undo.h"
 #include "submodule.h"
 #include "argv-array.h"
+#include "sigchain.h"
 
 static const char * const checkout_usage[] = {
 	N_("git checkout [options] <branch>"),
@@ -826,6 +827,35 @@ static void write_to_file(const char *path, const char *fmt, ...)
 	fclose(fp);
 }
 
+static const char *junk_work_tree;
+static const char *junk_git_dir;
+static int is_junk;
+static pid_t junk_pid;
+
+static void remove_junk(void)
+{
+	struct strbuf sb = STRBUF_INIT;
+	if (!is_junk || getpid() != junk_pid)
+		return;
+	if (junk_git_dir) {
+		strbuf_addstr(&sb, junk_git_dir);
+		remove_dir_recursively(&sb, 0);
+		strbuf_reset(&sb);
+	}
+	if (junk_work_tree) {
+		strbuf_addstr(&sb, junk_work_tree);
+		remove_dir_recursively(&sb, 0);
+		strbuf_reset(&sb);
+	}
+}
+
+static void remove_junk_on_signal(int signo)
+{
+	remove_junk();
+	sigchain_pop(signo);
+	raise(signo);
+}
+
 static int checkout_new_worktree(const struct checkout_opts *opts,
 				 struct branch_info *new)
 {
@@ -835,7 +865,7 @@ static int checkout_new_worktree(const struct checkout_opts *opts,
 	struct stat st;
 	const char *name;
 	struct child_process cp;
-	int counter = 0, len;
+	int counter = 0, len, ret;
 
 	if (!new->commit)
 		die(_("no branch specified"));
@@ -860,13 +890,21 @@ static int checkout_new_worktree(const struct checkout_opts *opts,
 		strbuf_addf(&sb_repo, "%d", counter);
 	}
 	name = sb_repo.buf + len - strlen(name);
+
+	junk_pid = getpid();
+	atexit(remove_junk);
+	sigchain_push_common(remove_junk_on_signal);
+
 	if (mkdir(sb_repo.buf, 0777))
 		die_errno(_("could not create directory of '%s'"), sb_repo.buf);
+	junk_git_dir = sb_repo.buf;
+	is_junk = 1;
 
 	strbuf_addf(&sb_git, "%s/.git", path);
 	if (safe_create_leading_directories_const(sb_git.buf))
 		die_errno(_("could not create leading directories of '%s'"),
 			  sb_git.buf);
+	junk_work_tree = path;
 
 	write_to_file(sb_git.buf, "gitsuper: %s\ngitdir: %s\n",
 		      real_path(get_git_dir()), name);
@@ -887,7 +925,14 @@ static int checkout_new_worktree(const struct checkout_opts *opts,
 	memset(&cp, 0, sizeof(cp));
 	cp.git_cmd = 1;
 	cp.argv = opts->saved_argv;
-	return run_command(&cp);
+	ret = run_command(&cp);
+	if (!ret)
+		is_junk = 0;
+	strbuf_release(&sb);
+	strbuf_release(&sb_repo);
+	strbuf_release(&sb_git);
+	return ret;
+
 }
 
 static int git_checkout_config(const char *var, const char *value, void *cb)
-- 
1.8.5.1.77.g42c48fa

[PATCH v2 16/21] setup.c: keep track of the .git file location if read

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 cache.h       |  2 +-
 environment.c |  2 +-
 setup.c       | 44 ++++++++++++++++++++++++++++++--------------
 3 files changed, 32 insertions(+), 16 deletions(-)

diff --git a/cache.h b/cache.h
index 4c09223..e036a7d 100644
--- a/cache.h
+++ b/cache.h
@@ -411,7 +411,7 @@ extern const char *get_git_namespace(void);
 extern const char *strip_namespace(const char *namespaced_ref);
 extern const char *get_git_work_tree(void);
 extern const char *read_gitfile(const char *path);
-extern const char *read_gitfile_super(const char *path, char **super);
+extern const char *read_gitfile_super(const char *path, char **super, char **gitfile);
 extern const char *resolve_gitdir(const char *suspect);
 extern void set_git_work_tree(const char *tree);
 
diff --git a/environment.c b/environment.c
index 5cbbe11..61aff23 100644
--- a/environment.c
+++ b/environment.c
@@ -130,7 +130,7 @@ static void setup_git_env(void)
 	git_dir = getenv(GIT_DIR_ENVIRONMENT);
 	if (!git_dir)
 		git_dir = DEFAULT_GIT_DIR_ENVIRONMENT;
-	gitfile = read_gitfile_super(git_dir, &super);
+	gitfile = read_gitfile_super(git_dir, &super, NULL);
 	git_dir = xstrdup(gitfile ? gitfile : git_dir);
 	if (super)
 		git_super_dir = xstrdup(super);
diff --git a/setup.c b/setup.c
index 7ccb1f8..ac7d90c 100644
--- a/setup.c
+++ b/setup.c
@@ -332,7 +332,7 @@ static char *path_from_gitfile(const char *path, const char *buf, int len)
  * to the absolute path of the given path. The next line contains the
  * repo id.
  */
-const char *read_gitfile_super(const char *path, char **super)
+const char *read_gitfile_super(const char *path, char **super, char **gitfile)
 {
 	char *buf, *dir;
 	struct stat st;
@@ -384,6 +384,10 @@ const char *read_gitfile_super(const char *path, char **super)
 
 	if (!is_git_directory_super(dir, super ? *super : NULL))
 		die("Not a git repository: %s", dir);
+
+	if (gitfile)
+		*gitfile = xstrdup(real_path(path));
+
 	path = real_path(dir);
 
 	free(dir);
@@ -393,14 +397,15 @@ const char *read_gitfile_super(const char *path, char **super)
 
 const char *read_gitfile(const char *path)
 {
-	return read_gitfile_super(path, NULL);
+	return read_gitfile_super(path, NULL, NULL);
 }
 
 
 static const char *setup_explicit_git_dir(const char *gitdirenv,
 					  const char *super,
 					  char *cwd, int len,
-					  int *nongit_ok)
+					  int *nongit_ok,
+					  char **gitfile_path)
 {
 	const char *work_tree_env = getenv(GIT_WORK_TREE_ENVIRONMENT);
 	const char *worktree;
@@ -413,7 +418,8 @@ static const char *setup_explicit_git_dir(const char *gitdirenv,
 	if (super)
 		gitfile = (char*)read_gitfile(gitdirenv);
 	else {
-		gitfile = (char*)read_gitfile_super(gitdirenv, &super_new);
+		gitfile = (char*)read_gitfile_super(gitdirenv, &super_new,
+						    gitfile_path);
 		super = super_new;
 	}
 	if (gitfile) {
@@ -500,7 +506,8 @@ done_null:
 static const char *setup_discovered_git_dir(const char *gitdir,
 					    const char *super,
 					    char *cwd, int offset, int len,
-					    int *nongit_ok)
+					    int *nongit_ok,
+					    char **gitfile_path)
 {
 	if (check_repository_format_gently(super ? super : gitdir, nongit_ok))
 		return NULL;
@@ -512,7 +519,8 @@ static const char *setup_discovered_git_dir(const char *gitdir,
 			gitdir = xstrdup(real_path(gitdir));
 		if (chdir(cwd))
 			die_errno("Could not come back to cwd");
-		return setup_explicit_git_dir(gitdir, super, cwd, len, nongit_ok);
+		return setup_explicit_git_dir(gitdir, super, cwd, len,
+					      nongit_ok, gitfile_path);
 	}
 
 	/* #16.2, #17.2, #20.2, #21.2, #24, #25, #28, #29 (see t1510) */
@@ -540,7 +548,8 @@ static const char *setup_discovered_git_dir(const char *gitdir,
 }
 
 /* #16.1, #17.1, #20.1, #21.1, #22.1 (see t1510) */
-static const char *setup_bare_git_dir(char *cwd, int offset, int len, int *nongit_ok)
+static const char *setup_bare_git_dir(char *cwd, int offset, int len,
+				      int *nongit_ok, char **gitfile_path)
 {
 	int root_len;
 
@@ -556,7 +565,8 @@ static const char *setup_bare_git_dir(char *cwd, int offset, int len, int *nongi
 		gitdir = offset == len ? "." : xmemdupz(cwd, offset);
 		if (chdir(cwd))
 			die_errno("Could not come back to cwd");
-		return setup_explicit_git_dir(gitdir, NULL, cwd, len, nongit_ok);
+		return setup_explicit_git_dir(gitdir, NULL, cwd, len,
+					      nongit_ok, gitfile_path);
 	}
 
 	inside_git_dir = 1;
@@ -630,7 +640,8 @@ static int canonicalize_ceiling_entry(struct string_list_item *item,
  * We cannot decide in this function whether we are in the work tree or
  * not, since the config can only be read _after_ this function was called.
  */
-static const char *setup_git_directory_gently_1(int *nongit_ok)
+static const char *setup_git_directory_gently_1(int *nongit_ok,
+						char **gitfile_path)
 {
 	const char *env_ceiling_dirs = getenv(CEILING_DIRECTORIES_ENVIRONMENT);
 	struct string_list ceiling_dirs = STRING_LIST_INIT_DUP;
@@ -662,7 +673,8 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 	super = getenv(GIT_SUPER_DIR_ENVIRONMENT);
 	if (gitdirenv)
 		return setup_explicit_git_dir(gitdirenv, super,
-					      cwd, len, nongit_ok);
+					      cwd, len, nongit_ok,
+					      gitfile_path);
 
 	if (env_ceiling_dirs) {
 		int empty_entry_found = 0;
@@ -696,7 +708,7 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 			gitfile = (char*)read_gitfile(DEFAULT_GIT_DIR_ENVIRONMENT);
 		else {
 			gitfile = (char*)read_gitfile_super(DEFAULT_GIT_DIR_ENVIRONMENT,
-							    &super_new);
+							    &super_new, gitfile_path);
 			super = super_new;
 		}
 		if (gitfile)
@@ -709,7 +721,8 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 		if (gitdirenv) {
 			ret = setup_discovered_git_dir(gitdirenv, super,
 						       cwd, offset, len,
-						       nongit_ok);
+						       nongit_ok,
+						       gitfile_path);
 			free(gitfile);
 			free(super_new);
 			return ret;
@@ -717,7 +730,8 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 		free(gitfile);
 
 		if (is_git_directory(".")) {
-			ret = setup_bare_git_dir(cwd, offset, len, nongit_ok);
+			ret = setup_bare_git_dir(cwd, offset, len,
+						 nongit_ok, gitfile_path);
 			free(super_new);
 			return ret;
 		}
@@ -754,8 +768,9 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
 const char *setup_git_directory_gently(int *nongit_ok)
 {
 	const char *prefix;
+	char *gitfile = NULL;
 
-	prefix = setup_git_directory_gently_1(nongit_ok);
+	prefix = setup_git_directory_gently_1(nongit_ok, &gitfile);
 	if (prefix)
 		setenv(GIT_PREFIX_ENVIRONMENT, prefix, 1);
 	else
@@ -765,6 +780,7 @@ const char *setup_git_directory_gently(int *nongit_ok)
 		startup_info->have_repository = !nongit_ok || !*nongit_ok;
 		startup_info->prefix = prefix;
 	}
+	free(gitfile);
 	return prefix;
 }
 
-- 
1.8.5.1.77.g42c48fa

[PATCH v2 17/21] prune: strategies for split repositories

[Nguyễn Thái Ngọc Duy]

alias REPO=$GIT_SUPER_DIR/repos/<id>

 - split repos are supposed to update mtime of $REPO/gitdir

 - split repos are supposed to keep its location in $REPO/gitdir up to
   date

 - "git checkout --to" is supposed to create $REPO/locked if the new
   repo is on a different partition than the shared one.

 - "git checkout --to" is supposed to make hardlink named $REPO/link
   pointing to the .git file

The pruning rules are:

 - if $REPO/locked exists, repos/<id> is not supposed to be pruned.

 - if $REPO/locked exists and $REPO/gitdir's mtimer is older than a
   really long limit, warn about old unused repo.

 - if $REPO/link exists and its link count is 1, repos/<id> is to be
   pruned.

 - if $REPO/gitdir's mtime is older than a limit, and it points to
   nowhere, repos/<id> is to be pruned. Warn about it some time before
   the prune time.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git-prune.txt            |  4 +++
 Documentation/gitrepository-layout.txt | 19 ++++++++++
 builtin/checkout.c                     | 36 ++++++++++++++++++-
 builtin/prune.c                        | 65 ++++++++++++++++++++++++++++++++++
 setup.c                                | 13 +++++++
 5 files changed, 136 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-prune.txt b/Documentation/git-prune.txt
index bf82410..55a9341 100644
--- a/Documentation/git-prune.txt
+++ b/Documentation/git-prune.txt
@@ -46,6 +46,10 @@ OPTIONS
 --expire <time>::
 	Only expire loose objects older than <time>.
 
+--repos::
+	Prune directories in $GIT_DIR/repos other than `<time>` given
+	by `--expire` (or default 6 months)
+
 <head>...::
 	In addition to objects
 	reachable from any of our references, keep objects
diff --git a/Documentation/gitrepository-layout.txt b/Documentation/gitrepository-layout.txt
index 3c6149e..a8a0426 100644
--- a/Documentation/gitrepository-layout.txt
+++ b/Documentation/gitrepository-layout.txt
@@ -225,6 +225,25 @@ repos/<id>::
 	repository in question (e.g. HEAD or index). Such .git files
 	are created by `git checkout --to`.
 
+repos/<id>/gitdir::
+	A text file containing the absolute path back to the .git file
+	that points to here. This is used to check if the linked
+	repository has been manually removed and there is no need to
+	keep this directory any more. mtime of this file should be
+	updated every time the linked repository is accessed.
+
+repos/<id>/locked::
+	If this file exists, the linked repository may be on a
+	portable device and not available. It does not mean that the
+	linked repository is gone and `repos/<id>` could be
+	removed. The file's content contains a reason string on why
+	the repository is locked.
+
+repos/<id>/link::
+	If this file exists, it is a hard link to the linked .git
+	file. It is used to detect if the linked repository is
+	manually removed.
+
 SEE ALSO
 --------
 linkgit:git-init[1],
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 6353557..aece20d 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -856,6 +856,17 @@ static void remove_junk_on_signal(int signo)
 	raise(signo);
 }
 
+static dev_t get_device_or_die(const char *path)
+{
+	struct stat buf;
+	if (stat(path, &buf))
+		die_errno("failed to stat '%s'", path);
+	/* Ah Windows! Make different drives different "partitions" */
+	if (buf.st_dev == 0 && has_dos_drive_prefix("c:\\"))
+		buf.st_dev = toupper(real_path(path)[0]);
+	return buf.st_dev;
+}
+
 static int checkout_new_worktree(const struct checkout_opts *opts,
 				 struct branch_info *new)
 {
@@ -865,7 +876,7 @@ static int checkout_new_worktree(const struct checkout_opts *opts,
 	struct stat st;
 	const char *name;
 	struct child_process cp;
-	int counter = 0, len, ret;
+	int counter = 0, len, keep_locked = 0, ret;
 
 	if (!new->commit)
 		die(_("no branch specified"));
@@ -900,17 +911,35 @@ static int checkout_new_worktree(const struct checkout_opts *opts,
 	junk_git_dir = sb_repo.buf;
 	is_junk = 1;
 
+	strbuf_addf(&sb, "%s/locked", sb_repo.buf);
+	write_to_file(sb.buf, "initializing\n");
+
 	strbuf_addf(&sb_git, "%s/.git", path);
 	if (safe_create_leading_directories_const(sb_git.buf))
 		die_errno(_("could not create leading directories of '%s'"),
 			  sb_git.buf);
 	junk_work_tree = path;
 
+	strbuf_reset(&sb);
+	strbuf_addf(&sb, "%s/gitdir", sb_repo.buf);
+	write_to_file(sb.buf, "%s\n", real_path(sb_git.buf));
 	write_to_file(sb_git.buf, "gitsuper: %s\ngitdir: %s\n",
 		      real_path(get_git_dir()), name);
+	strbuf_reset(&sb);
 	strbuf_addf(&sb, "%s/HEAD", sb_repo.buf);
 	write_to_file(sb.buf, "%s\n", sha1_to_hex(new->commit->object.sha1));
 
+	if (get_device_or_die(path) != get_device_or_die(get_git_dir())) {
+		strbuf_reset(&sb);
+		strbuf_addf(&sb, "%s/locked", sb_repo.buf);
+		write_to_file(sb.buf, "located on a different file system\n");
+		keep_locked = 1;
+	} else {
+		strbuf_reset(&sb);
+		strbuf_addf(&sb, "%s/link", sb_repo.buf);
+		link(sb_git.buf, sb.buf); /* it's ok to fail */
+	}
+
 	if (!opts->quiet)
 		fprintf_ln(stderr, _("Enter %s (identifier %s)"), path, name);
 
@@ -928,6 +957,11 @@ static int checkout_new_worktree(const struct checkout_opts *opts,
 	ret = run_command(&cp);
 	if (!ret)
 		is_junk = 0;
+	if (!keep_locked) {
+		strbuf_reset(&sb);
+		strbuf_addf(&sb, "%s/locked", sb_repo.buf);
+		unlink_or_warn(sb.buf);
+	}
 	strbuf_release(&sb);
 	strbuf_release(&sb_repo);
 	strbuf_release(&sb_git);
diff --git a/builtin/prune.c b/builtin/prune.c
index 6366917..16eb99c 100644
--- a/builtin/prune.c
+++ b/builtin/prune.c
@@ -102,6 +102,62 @@ static void prune_object_dir(const char *path)
 	}
 }
 
+static const char *prune_repo_dir(const char *id)
+{
+	struct stat st;
+	char *path;
+	int fd, len;
+	if (file_exists(git_path("repos/%s/locked", id)))
+		return NULL;
+	if (!stat(git_path("repos/%s/link", id), &st) && st.st_nlink == 1)
+		return _("hard link count is 1");
+	if (stat(git_path("repos/%s/gitdir", id), &st))
+		return "gitdir does not exist";
+	if (st.st_mtime > expire)
+		return NULL;
+	fd = open(git_path("repos/%s/gitdir", id), O_RDONLY);
+	len = st.st_size;
+	path = xmalloc(len + 1);
+	read_in_full(fd, path, len);
+	close(fd);
+	while (path[len - 1] == '\n' || path[len - 1] == '\r')
+		len--;
+	path[len] = '\0';
+	if (!file_exists(path)) {
+		free(path);
+		return "gitdir points to non-existing file";
+	}
+	free(path);
+	return NULL;
+}
+
+static void prune_repos_dir(void)
+{
+	const char *reason;
+	DIR *dir = opendir(git_path("repos"));
+	struct dirent *d;
+	int removed = 0;
+	if (!dir)
+		return;
+	while ((d = readdir(dir)) != NULL) {
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+		if ((reason = prune_repo_dir(d->d_name)) != NULL) {
+			struct strbuf sb = STRBUF_INIT;
+			printf(_("Removing repos/%s: %s\n"), d->d_name, reason);
+			if (show_only)
+				continue;
+			strbuf_addstr(&sb, git_path("repos/%s", d->d_name));
+			remove_dir_recursively(&sb, 0);
+			strbuf_release(&sb);
+			removed = 1;
+		}
+	}
+	closedir(dir);
+	if (removed)
+		rmdir(git_path("repos"));
+}
+
 /*
  * Write errors (particularly out of space) can result in
  * failed temporary packs (and more rarely indexes and other
@@ -128,10 +184,12 @@ int cmd_prune(int argc, const char **argv, const char *prefix)
 {
 	struct rev_info revs;
 	struct progress *progress = NULL;
+	int prune_repos = 0;
 	const struct option options[] = {
 		OPT__DRY_RUN(&show_only, N_("do not remove, show only")),
 		OPT__VERBOSE(&verbose, N_("report pruned objects")),
 		OPT_BOOL(0, "progress", &show_progress, N_("show progress")),
+		OPT_BOOL(0, "repos", &prune_repos, N_("prune .git/repos/")),
 		OPT_EXPIRY_DATE(0, "expire", &expire,
 				N_("expire objects older than <time>")),
 		OPT_END()
@@ -156,6 +214,13 @@ int cmd_prune(int argc, const char **argv, const char *prefix)
 			die("unrecognized argument: %s", name);
 	}
 
+	if (prune_repos) {
+		if (expire == ULONG_MAX)
+			expire = time(NULL) - 6 * 30 * 24 * 3600; /* 6 months */
+		prune_repos_dir();
+		return 0;
+	}
+
 	if (show_progress == -1)
 		show_progress = isatty(2);
 	if (show_progress)
diff --git a/setup.c b/setup.c
index ac7d90c..95a3f40 100644
--- a/setup.c
+++ b/setup.c
@@ -765,6 +765,17 @@ static const char *setup_git_directory_gently_1(int *nongit_ok,
 	}
 }
 
+static int update_super_gitdir(char *path)
+{
+	FILE *fp = fopen(git_path("gitdir"), "w");
+	if (!fp)
+		return error(_("unable to update %s: %s"),
+			     git_path("gitdir"), strerror(errno));
+	fprintf(stderr, "%s\n", path);
+	fclose(fp);
+	return 0;
+}
+
 const char *setup_git_directory_gently(int *nongit_ok)
 {
 	const char *prefix;
@@ -780,6 +791,8 @@ const char *setup_git_directory_gently(int *nongit_ok)
 		startup_info->have_repository = !nongit_ok || !*nongit_ok;
 		startup_info->prefix = prefix;
 	}
+	if (get_git_super_dir() && gitfile)
+		update_super_gitdir(gitfile);
 	free(gitfile);
 	return prefix;
 }
-- 
1.8.5.1.77.g42c48fa

[PATCH v2 18/21] refs: adjust reflog path for repos/<id>/HEAD

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 refs.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/refs.c b/refs.c
index 5e5a382..3691ef5 100644
--- a/refs.c
+++ b/refs.c
@@ -2676,13 +2676,24 @@ static int copy_msg(char *buf, const char *msg)
 int log_ref_setup(const char *refname, char *logfile, int bufsize)
 {
 	int logfd, oflags = O_APPEND | O_WRONLY;
+	const char *p = NULL;
 
-	git_snpath(logfile, bufsize, "logs/%s", refname);
+	/*
+	 * reflog of repos/XXX/HEAD is repos/XXX/logs/HEAD, not
+	 * refs/repos/XXX/HEAD
+	 */
+	if (!strncmp(refname, "repos/", 6) &&
+	    (p = strchr(refname + 6, '/')) != 0)
+		git_snpath(logfile, bufsize, "%.*s/logs/%s",
+			   (int)(p - refname), refname, p + 1);
+	else
+		git_snpath(logfile, bufsize, "logs/%s", refname);
 	if (log_all_ref_updates &&
 	    (!prefixcmp(refname, "refs/heads/") ||
 	     !prefixcmp(refname, "refs/remotes/") ||
 	     !prefixcmp(refname, "refs/notes/") ||
-	     !strcmp(refname, "HEAD"))) {
+	     !strcmp(refname, "HEAD") ||
+	     (p && !strcmp(p + 1, "HEAD")))) {
 		if (safe_create_leading_directories(logfile) < 0)
 			return error("unable to create directory for %s",
 				     logfile);
-- 
1.8.5.1.77.g42c48fa

[PATCH v2 19/21] refs: detach split repos' HEAD when the linked ref is updated/deleted

[Nguyễn Thái Ngọc Duy]

This is effective when a ref is updated from the super repository as
well as all linked repositories because "repos" directory is shared
between all repos.

We could even forbid a ref update if it's some repo's HEAD. But I'd
rather see a generic, permanent ref locking mechanism in place first
and make use of it.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 refs.c | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)

diff --git a/refs.c b/refs.c
index 3691ef5..d856b1a 100644
--- a/refs.c
+++ b/refs.c
@@ -2462,6 +2462,41 @@ static int repack_without_ref(const char *refname)
 	return repack_without_refs(&refname, 1);
 }
 
+static void detach_repos_refs(const char *refname, const unsigned char *sha1)
+{
+	struct dirent *d;
+	DIR *dir;
+
+	if (!strncmp(refname, "repos/", 6) ||
+	    !strchr(refname, '/') ||
+	    (dir = opendir(git_path("repos"))) == NULL)
+		return;
+
+	while ((d = readdir(dir)) != NULL) {
+		struct strbuf sb_ref = STRBUF_INIT;
+		unsigned char ref_sha1[20];
+		const char *ref;
+		int flags;
+
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+		strbuf_addf(&sb_ref, "repos/%s/HEAD", d->d_name);
+		ref = resolve_ref_unsafe(sb_ref.buf, ref_sha1, 1, &flags);
+		if (flags & REF_ISSYMREF && ref && !strcmp(ref, refname)) {
+			struct strbuf msg = STRBUF_INIT;
+			strbuf_addf(&msg, "repos: detach from %s by %s",
+				    ref, real_path(get_git_dir()));
+			update_ref(msg.buf, sb_ref.buf, sha1, NULL,
+				   REF_NODEREF, DIE_ON_ERR);
+			strbuf_release(&msg);
+			warning(_("detach HEAD of linked repository %s from %s"),
+				d->d_name, refname);
+		}
+		strbuf_release(&sb_ref);
+	}
+	closedir(dir);
+}
+
 static int delete_ref_loose(struct ref_lock *lock, int flag)
 {
 	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
@@ -2485,6 +2520,7 @@ int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
 	lock = lock_ref_sha1_basic(refname, sha1, delopt, &flag);
 	if (!lock)
 		return 1;
+	detach_repos_refs(lock->ref_name, lock->old_sha1);
 	ret |= delete_ref_loose(lock, flag);
 
 	/* removing the loose one could have resurrected an earlier
@@ -2790,6 +2826,7 @@ int write_ref_sha1(struct ref_lock *lock,
 		unlock_ref(lock);
 		return -1;
 	}
+	detach_repos_refs(lock->ref_name, lock->old_sha1);
 	if (write_in_full(lock->lock_fd, sha1_to_hex(sha1), 40) != 40 ||
 	    write_in_full(lock->lock_fd, &term, 1) != 1
 		|| close_ref(lock) < 0) {
-- 
1.8.5.1.77.g42c48fa

[PATCH v2 20/21] refs.c: refactor do_head_ref(... to do_one_head_ref("HEAD", ...

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 refs.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/refs.c b/refs.c
index d856b1a..8ef1cb0 100644
--- a/refs.c
+++ b/refs.c
@@ -1721,32 +1721,33 @@ static int do_for_each_ref(struct ref_cache *refs, const char *base,
 	return do_for_each_entry(refs, base, do_one_ref, &data);
 }
 
-static int do_head_ref(const char *submodule, each_ref_fn fn, void *cb_data)
+static int do_one_head_ref(const char *ref, const char *submodule,
+		      each_ref_fn fn, void *cb_data)
 {
 	unsigned char sha1[20];
 	int flag;
 
 	if (submodule) {
-		if (resolve_gitlink_ref(submodule, "HEAD", sha1) == 0)
-			return fn("HEAD", sha1, 0, cb_data);
+		if (resolve_gitlink_ref(submodule, ref, sha1) == 0)
+			return fn(ref, sha1, 0, cb_data);
 
 		return 0;
 	}
 
-	if (!read_ref_full("HEAD", sha1, 1, &flag))
-		return fn("HEAD", sha1, flag, cb_data);
+	if (!read_ref_full(ref, sha1, 1, &flag))
+		return fn(ref, sha1, flag, cb_data);
 
 	return 0;
 }
 
 int head_ref(each_ref_fn fn, void *cb_data)
 {
-	return do_head_ref(NULL, fn, cb_data);
+	return do_one_head_ref("HEAD", NULL, fn, cb_data);
 }
 
 int head_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
 {
-	return do_head_ref(submodule, fn, cb_data);
+	return do_one_head_ref("HEAD", submodule, fn, cb_data);
 }
 
 int for_each_ref(each_ref_fn fn, void *cb_data)
-- 
1.8.5.1.77.g42c48fa

[PATCH v2 21/21] revision: include repos/../HEAD in --all

[Nguyễn Thái Ngọc Duy]

This makes sure repack won't drop detached HEADS from $GIT_DIR/repos/

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 refs.c     | 21 +++++++++++++++++++++
 refs.h     |  1 +
 revision.c |  1 +
 3 files changed, 23 insertions(+)

diff --git a/refs.c b/refs.c
index 8ef1cb0..a9c8651 100644
--- a/refs.c
+++ b/refs.c
@@ -1740,6 +1740,27 @@ static int do_one_head_ref(const char *ref, const char *submodule,
 	return 0;
 }
 
+int repos_head_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data)
+{
+	DIR *dir = opendir(git_path("repos"));
+	int ret = 0;
+	struct dirent *d;
+
+	if (!dir)
+		return ret;
+
+	while (!ret && (d = readdir(dir)) != NULL) {
+		struct strbuf sb_ref = STRBUF_INIT;
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+		strbuf_addf(&sb_ref, "repos/%s/HEAD", d->d_name);
+		ret = do_one_head_ref(sb_ref.buf, submodule, fn, cb_data);
+		strbuf_release(&sb_ref);
+	}
+	closedir(dir);
+	return ret;
+}
+
 int head_ref(each_ref_fn fn, void *cb_data)
 {
 	return do_one_head_ref("HEAD", NULL, fn, cb_data);
diff --git a/refs.h b/refs.h
index 87a1a79..05686e4 100644
--- a/refs.h
+++ b/refs.h
@@ -70,6 +70,7 @@ extern int for_each_glob_ref(each_ref_fn, const char *pattern, void *);
 extern int for_each_glob_ref_in(each_ref_fn, const char *pattern, const char* prefix, void *);
 
 extern int head_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data);
+extern int repos_head_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data);
 extern int for_each_ref_submodule(const char *submodule, each_ref_fn fn, void *cb_data);
 extern int for_each_ref_in_submodule(const char *submodule, const char *prefix,
 		each_ref_fn fn, void *cb_data);
diff --git a/revision.c b/revision.c
index 05d2d77..c4150ed 100644
--- a/revision.c
+++ b/revision.c
@@ -2004,6 +2004,7 @@ static int handle_revision_pseudo_opt(const char *submodule,
 	if (!strcmp(arg, "--all")) {
 		handle_refs(submodule, revs, *flags, for_each_ref_submodule);
 		handle_refs(submodule, revs, *flags, head_ref_submodule);
+		handle_refs(submodule, revs, *flags, repos_head_ref_submodule);
 		clear_ref_exclusion(&revs->ref_excludes);
 	} else if (!strcmp(arg, "--branches")) {
 		handle_refs(submodule, revs, *flags, for_each_branch_ref_submodule);
-- 
1.8.5.1.77.g42c48fa

Re: [PATCH/POC 2/7] Add new environment variable $GIT_SUPER_DIR

[Junio C Hamano]

Duy Nguyen <pclouds@gmail.com> writes:

> The exception list could be equally long (most of them are *_HEAD).
> .git is also used for temporary files with mkstemp, but I think that's
> safe for sharing. What could break is when people add a new local
> *_HEAD and forget to update the exception list.

Sensible; it may be worth having the above to describe why we choose
to go that "share only the selected ones" route in one of the log
message and/or in-code comment.

Thanks.

Re: [PATCH v2 02/21] path.c: rename vsnpath() to git_vsnpath()

[Ramsay Jones]

On 14/12/13 10:54, Nguyễn Thái Ngọc Duy wrote:
> This is the underlying implementation of git_path(), git_pathdup() and
> git_snpath() which will prefix $GIT_DIR in the result string. Put git_
> prefix in front of it to avoid the confusion that this is a generic
> path handling function.#
> 
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
>  path.c | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/path.c b/path.c
> index 4c1c144..06863b7 100644
> --- a/path.c
> +++ b/path.c
> @@ -50,7 +50,7 @@ char *mksnpath(char *buf, size_t n, const char *fmt, ...)
>  	return cleanup_path(buf);
>  }
>  
> -static char *vsnpath(char *buf, size_t n, const char *fmt, va_list args)
> +static char *git_vsnpath(char *buf, size_t n, const char *fmt, va_list args)

:-D I renamed this _from_ git_vsnpath() in commit 5b3b8fa2 ("path.c: Remove the
'git_' prefix from a file scope function", 04-09-2012), because ... well it's a
file scope function! (i.e. the git_ prefix implies greater than file scope).
I'm not very good at naming things, so ...

ATB,
Ramsay Jones

Re: [PATCH v2 02/21] path.c: rename vsnpath() to git_vsnpath()

[Duy Nguyen]

On Sun, Dec 15, 2013 at 3:23 AM, Ramsay Jones
<ramsay@ramsay1.demon.co.uk> wrote:
> On 14/12/13 10:54, Nguyễn Thái Ngọc Duy wrote:
>> This is the underlying implementation of git_path(), git_pathdup() and
>> git_snpath() which will prefix $GIT_DIR in the result string. Put git_
>> prefix in front of it to avoid the confusion that this is a generic
>> path handling function.#
>>
>> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
>> ---
>>  path.c | 8 ++++----
>>  1 file changed, 4 insertions(+), 4 deletions(-)
>>
>> diff --git a/path.c b/path.c
>> index 4c1c144..06863b7 100644
>> --- a/path.c
>> +++ b/path.c
>> @@ -50,7 +50,7 @@ char *mksnpath(char *buf, size_t n, const char *fmt, ...)
>>       return cleanup_path(buf);
>>  }
>>
>> -static char *vsnpath(char *buf, size_t n, const char *fmt, va_list args)
>> +static char *git_vsnpath(char *buf, size_t n, const char *fmt, va_list args)
>
> :-D I renamed this _from_ git_vsnpath() in commit 5b3b8fa2 ("path.c: Remove the
> 'git_' prefix from a file scope function", 04-09-2012), because ... well it's a
> file scope function! (i.e. the git_ prefix implies greater than file scope).
> I'm not very good at naming things, so ...

maybe gitdir_vsnpath() then to avoid the global scope prefix git_?
-- 
Duy

Re: [PATCH v2 00/21] Support multiple worktrees

[Duy Nguyen]

On Sat, Dec 14, 2013 at 5:54 PM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:
> Known issues

Scripts that expand "$GIT_DIR/objects" and are not aware about the new
env variable. I introduced "test-path-utils --git-path" to test
git_path(). I could move it to "git rev-parse --git-path" for use in
scripts, but there'll be more changes. git-new-workdir's symlink
approach shines here.
-- 
Duy

Re: [PATCH v2 01/21] path.c: avoid PATH_MAX as buffer size from get_pathname()

[Torsten Bögershausen]

On 2013-12-14 11.54, Nguyễn Thái Ngọc Duy wrote:
> We've been avoiding PATH_MAX whenever possible. This patch avoids
> PATH_MAX in get_pathname() and gives it enough room not to worry about
> really long paths.
> 
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
>  path.c | 28 ++++++++++++++++------------
>  1 file changed, 16 insertions(+), 12 deletions(-)
> 
> diff --git a/path.c b/path.c
> index 24594c4..4c1c144 100644
> --- a/path.c
> +++ b/path.c
> @@ -16,10 +16,11 @@ static int get_st_mode_bits(const char *path, int *mode)
>  
>  static char bad_path[] = "/bad-path/";
>  
> -static char *get_pathname(void)
> +static char *get_pathname(size_t *len)
>  {
> -	static char pathname_array[4][PATH_MAX];
> +	static char pathname_array[4][4096];
The PATH_MAX doesn't seem to be that bad:
http://pubs.opengroup.org/onlinepubs/009695399/basedefs/limits.h.html
Or do we have a an OS where PATH_MAX does not work ?

Windows uses MAX_PATH=260 PATH_MAX=259
<http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247%28v=vs.85%29.aspx>

Which means that the current implementation of git can not use path names longer than
259 (260 including the trailing \0)
(Please correct me if this is wrong)

Which means that the code working with the buffers must make sure to stay within range,
and not to write beyond the buffers.

If we really want to go away from PATH_MAX, is a hard-coded value of 4096 so attractive ?
Because we can either

a) Re-define PATH_MAX in git-compat-util.h
b) Use an own  #define in git-compat-util.h, like e.g. GIT_PATH_MAX
c) Change the code to use a "strbuf" which can grow on demand.

I would prefer c) over b) over a)




>  	static int index;
> +	*len = sizeof(pathname_array[0]);
>  	return pathname_array[3 & ++index];
>  }
>  
> @@ -108,24 +109,26 @@ char *mkpath(const char *fmt, ...)
>  {
>  	va_list args;
>  	unsigned len;
> -	char *pathname = get_pathname();
> +	size_t n;
> +	char *pathname = get_pathname(&n);
>  
>  	va_start(args, fmt);
> -	len = vsnprintf(pathname, PATH_MAX, fmt, args);
> +	len = vsnprintf(pathname, n, fmt, args);
Renaming "n" into something like "max" or "max_len" could
make this line 5% easier to read.
(And similar at some places below)
/Torsten

Re: [PATCH v2 01/21] path.c: avoid PATH_MAX as buffer size from get_pathname()

[Duy Nguyen]

On Sun, Dec 15, 2013 at 3:35 PM, Torsten Bögershausen <tboegi@web.de> wrote:
> If we really want to go away from PATH_MAX, is a hard-coded value of 4096 so attractive ?
> Because we can either
>
> a) Re-define PATH_MAX in git-compat-util.h
> b) Use an own  #define in git-compat-util.h, like e.g. GIT_PATH_MAX
> c) Change the code to use a "strbuf" which can grow on demand.
>
> I would prefer c) over b) over a)

Looking at the code again, c) looks feasible after updating
git_snpath() to accept a strbuf instead of buffer/size pair.
-- 
Duy

Re: [PATCH v2 01/21] path.c: avoid PATH_MAX as buffer size from get_pathname()

[Antoine Pelisse]

On Sun, Dec 15, 2013 at 10:02 AM, Duy Nguyen <pclouds@gmail.com> wrote:
> On Sun, Dec 15, 2013 at 3:35 PM, Torsten Bögershausen <tboegi@web.de> wrote:
>> If we really want to go away from PATH_MAX, is a hard-coded value of 4096 so attractive ?
>> Because we can either
>>
>> a) Re-define PATH_MAX in git-compat-util.h
>> b) Use an own  #define in git-compat-util.h, like e.g. GIT_PATH_MAX
>> c) Change the code to use a "strbuf" which can grow on demand.
>>
>> I would prefer c) over b) over a)
>
> Looking at the code again, c) looks feasible after updating
> git_snpath() to accept a strbuf instead of buffer/size pair.

I agree that this is probably the way to go.
We are not trying to avoid PATH_MAX, but static-size array that can overflow.

Antoine,

Re: [PATCH v2 02/21] path.c: rename vsnpath() to git_vsnpath()

[Ramsay Jones]

On 15/12/13 02:25, Duy Nguyen wrote:
> On Sun, Dec 15, 2013 at 3:23 AM, Ramsay Jones
> <ramsay@ramsay1.demon.co.uk> wrote:
>> On 14/12/13 10:54, Nguyễn Thái Ngọc Duy wrote:
>>> This is the underlying implementation of git_path(), git_pathdup() and
>>> git_snpath() which will prefix $GIT_DIR in the result string. Put git_
>>> prefix in front of it to avoid the confusion that this is a generic
>>> path handling function.#
>>>
>>> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
>>> ---
>>>  path.c | 8 ++++----
>>>  1 file changed, 4 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/path.c b/path.c
>>> index 4c1c144..06863b7 100644
>>> --- a/path.c
>>> +++ b/path.c
>>> @@ -50,7 +50,7 @@ char *mksnpath(char *buf, size_t n, const char *fmt, ...)
>>>       return cleanup_path(buf);
>>>  }
>>>
>>> -static char *vsnpath(char *buf, size_t n, const char *fmt, va_list args)
>>> +static char *git_vsnpath(char *buf, size_t n, const char *fmt, va_list args)
>>
>> :-D I renamed this _from_ git_vsnpath() in commit 5b3b8fa2 ("path.c: Remove the
>> 'git_' prefix from a file scope function", 04-09-2012), because ... well it's a
>> file scope function! (i.e. the git_ prefix implies greater than file scope).
>> I'm not very good at naming things, so ...
> 
> maybe gitdir_vsnpath() then to avoid the global scope prefix git_?

Sounds fine to me (but then so does vsnpath ;-) ).

ATB,
Ramsay Jones

Re: [PATCH v2 02/21] path.c: rename vsnpath() to git_vsnpath()

[Duy Nguyen]

On Mon, Dec 16, 2013 at 4:13 AM, Ramsay Jones
<ramsay@ramsay1.demon.co.uk> wrote:
>>>> -static char *vsnpath(char *buf, size_t n, const char *fmt, va_list args)
>>>> +static char *git_vsnpath(char *buf, size_t n, const char *fmt, va_list args)
>>>
>>> :-D I renamed this _from_ git_vsnpath() in commit 5b3b8fa2 ("path.c: Remove the
>>> 'git_' prefix from a file scope function", 04-09-2012), because ... well it's a
>>> file scope function! (i.e. the git_ prefix implies greater than file scope).
>>> I'm not very good at naming things, so ...
>>
>> maybe gitdir_vsnpath() then to avoid the global scope prefix git_?
>
> Sounds fine to me (but then so does vsnpath ;-) ).

OK I go with this. I think it makes sense

vsnpath -> do_git_path

its three callers are

git_vsnpath -> strbuf_git_path (it's updated to take strbuf)
git_path
git_pathdup

In the end all these functions have "git_path" in them :)
-- 
Duy

Re: [PATCH v2 02/21] path.c: rename vsnpath() to git_vsnpath()

[Jonathan Nieder]

Duy Nguyen wrote:
>>> Ramsay Jones wrote:

>>>> :-D I renamed this _from_ git_vsnpath() in commit 5b3b8fa2 ("path.c: Remove the
>>>> 'git_' prefix from a file scope function", 04-09-2012), because ... well it's a
>>>> file scope function! (i.e. the git_ prefix implies greater than file scope).
>>>> I'm not very good at naming things, so ...
[...]
> OK I go with this. I think it makes sense
>
> vsnpath -> do_git_path

I think this renaming would be still losing clarity --- it loses the
information that this is the vsnprintf-like variant of git_path.

Do we actually have a convention that functions with git_ prefix
should be global?  If git_path were not global, would it have to be
renamed?  If git_vsnpath should be renamed to avoid mistaking it for
git's replacement of a hypothetical standard library vsnpath function,
shouldn't git_path, git_pathdup, etc be renamed for the same reason as
well?

> its three callers are
>
> git_vsnpath -> strbuf_git_path (it's updated to take strbuf)
> git_path
> git_pathdup

Yeah, independently of everything else, a strbuf variant sounds nice.

Thanks,
Jonathan

Re: [PATCH v2 02/21] path.c: rename vsnpath() to git_vsnpath()

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

> Duy Nguyen wrote:
>>>> Ramsay Jones wrote:
>
>>>>> :-D I renamed this _from_ git_vsnpath() in commit 5b3b8fa2 ("path.c: Remove the
>>>>> 'git_' prefix from a file scope function", 04-09-2012), because ... well it's a
>>>>> file scope function! (i.e. the git_ prefix implies greater than file scope).
>>>>> I'm not very good at naming things, so ...
> [...]
>> OK I go with this. I think it makes sense
>>
>> vsnpath -> do_git_path
>
> I think this renaming would be still losing clarity --- it loses the
> information that this is the vsnprintf-like variant of git_path.
>
> Do we actually have a convention that functions with git_ prefix
> should be global?  If git_path were not global, would it have to be
> renamed?  If git_vsnpath should be renamed to avoid mistaking it for
> git's replacement of a hypothetical standard library vsnpath function,
> shouldn't git_path, git_pathdup, etc be renamed for the same reason as
> well?
>
>> its three callers are
>>
>> git_vsnpath -> strbuf_git_path (it's updated to take strbuf)
>> git_path
>> git_pathdup
>
> Yeah, independently of everything else, a strbuf variant sounds nice.

OK, then strbuf_vsnpath_in_gitdir() perhaps?

Re: [PATCH v2 02/21] path.c: rename vsnpath() to git_vsnpath()

[Ramsay Jones]

On 16/12/13 17:11, Jonathan Nieder wrote:
> Duy Nguyen wrote:
>>>> Ramsay Jones wrote:
> 
>>>>> :-D I renamed this _from_ git_vsnpath() in commit 5b3b8fa2 ("path.c: Remove the
>>>>> 'git_' prefix from a file scope function", 04-09-2012), because ... well it's a
>>>>> file scope function! (i.e. the git_ prefix implies greater than file scope).
>>>>> I'm not very good at naming things, so ...
> [...]
>> OK I go with this. I think it makes sense
>>
>> vsnpath -> do_git_path
> 
> I think this renaming would be still losing clarity --- it loses the
> information that this is the vsnprintf-like variant of git_path.
> 
> Do we actually have a convention that functions with git_ prefix
> should be global?

Hmm, probably not no. However, any symbol that starts with git_ positively
screams global symbol to me. Maybe, I just need to close my ears. ;-)

I didn't intend to provoke so much discussion. I was simply pointing out
that this symbol had already been renamed, in the exact opposite direction,
once before. Please just ignore me.

ATB,
Ramsay Jones

Re: [PATCH/POC 3/7] setup.c: add split-repo support to .git files

[Duy Nguyen]

On Sat, Dec 14, 2013 at 3:43 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Problems:
>
>  * What if I move my worktree with "mv"?  Then I still need the
>    corresponding $GIT_SUPER_DIR/repos/<id> directory, and nobody told
>    the GIT_SUPER_DIR about it.
>
>  * What if my worktree is on removable media (think "network
>    filesystem") and has just been temporarily unmounted instead of
>    deleted?
>
> So maybe it would be nicer to:
>
>   i. When the worktree is on the same filesystem, keep a *hard link* to
>      some file in the worktree (e.g., the .git file).  If the link count
>      goes down, it is safe to remove the $GIT_SUPER_DIR/repos/<id>
>      directory.

Link count goes down to 1 if I move the worktree to a different
filesystem and it's not safe to remove $GIT_SUPER_DIR/repos/<id> in
this case, I think.

>  ii. When the worktree is on another filesystem, always keep
>      $GIT_SUPER_DIR/repos/<id> unless the user decides to manually
>      remove it.  Provide documentation or a command to list basic
>      information about $GIT_SUPER_DIR/repos directories (path to
>      worktree, what branch they're on, etc).
>
> (i) doesn't require any futureproofing.  As soon as someone wants it,
> they can implement the check and fall back to (ii) for worktrees
> without the magic hard link.
>
> (ii) would benefit from recording the working tree directory as a
> possibly unreliable, human-friendly reminder.
-- 
Duy