From: "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: gitster@pobox.com, newren@gmail.com, peff@peff.net,
jrnieder@gmail.com, sunshine@sunshineco.com, pclouds@gmail.com,
Derrick Stolee <derrickstolee@github.com>,
Derrick Stolee <dstolee@microsoft.com>
Subject: [PATCH 16/27] unpack-trees: make sparse aware
Date: Mon, 25 Jan 2021 17:42:02 +0000 [thread overview]
Message-ID: <45cf57c9c40bebb7383b8aab19c82fc4e41d2cd3.1611596534.git.gitgitgadget@gmail.com> (raw)
In-Reply-To: <pull.847.git.1611596533.gitgitgadget@gmail.com>
From: Derrick Stolee <dstolee@microsoft.com>
As a first step to integrate 'git status' and 'git add' with the sparse
index, we must start integrating unpack_trees() with sparse directory
entries. These changes are currently impossible to trigger because
unpack_trees() calls ensure_full_index() if command_requires_full_index
is true. This is the case for all commands at the moment. As we expand
more commands to be sparse-aware, we might find that more changes are
required to unpack_trees(). The current changes will suffice for
'status' and 'add'.
unpack_trees() calls the traverse_trees() API using unpack_callback()
to decide if we should recurse into a subtree. We must add new abilities
to skip a subtree if it corresponds to a sparse directory entry.
It is important to be careful about the trailing directory separator
that exists in the sparse directory entries but not in the subtree
paths.
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
dir.h | 2 +-
preload-index.c | 2 ++
read-cache.c | 3 +++
unpack-trees.c | 24 ++++++++++++++++++++++--
4 files changed, 28 insertions(+), 3 deletions(-)
diff --git a/dir.h b/dir.h
index facfae47402..300305ec335 100644
--- a/dir.h
+++ b/dir.h
@@ -503,7 +503,7 @@ static inline int ce_path_match(const struct index_state *istate,
char *seen)
{
return match_pathspec(istate, pathspec, ce->name, ce_namelen(ce), 0, seen,
- S_ISDIR(ce->ce_mode) || S_ISGITLINK(ce->ce_mode));
+ S_ISSPARSEDIR(ce) || S_ISDIR(ce->ce_mode) || S_ISGITLINK(ce->ce_mode));
}
static inline int dir_path_match(const struct index_state *istate,
diff --git a/preload-index.c b/preload-index.c
index ed6eaa47388..323fc8c5100 100644
--- a/preload-index.c
+++ b/preload-index.c
@@ -54,6 +54,8 @@ static void *preload_thread(void *_data)
continue;
if (S_ISGITLINK(ce->ce_mode))
continue;
+ if (S_ISSPARSEDIR(ce))
+ continue;
if (ce_uptodate(ce))
continue;
if (ce_skip_worktree(ce))
diff --git a/read-cache.c b/read-cache.c
index 65679d70d7c..ab0c2b86ec0 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1572,6 +1572,9 @@ int refresh_index(struct index_state *istate, unsigned int flags,
if (ignore_submodules && S_ISGITLINK(ce->ce_mode))
continue;
+ if (istate->sparse_index && S_ISSPARSEDIR(ce))
+ continue;
+
if (pathspec && !ce_path_match(istate, ce, pathspec, seen))
filtered = 1;
diff --git a/unpack-trees.c b/unpack-trees.c
index b324eec2a5d..90644856a80 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -583,6 +583,13 @@ static void mark_ce_used(struct cache_entry *ce, struct unpack_trees_options *o)
{
ce->ce_flags |= CE_UNPACKED;
+ /*
+ * If this is a sparse directory, don't advance cache_bottom.
+ * That will be advanced later using the cache-tree data.
+ */
+ if (S_ISSPARSEDIR(ce))
+ return;
+
if (o->cache_bottom < o->src_index->cache_nr &&
o->src_index->cache[o->cache_bottom] == ce) {
int bottom = o->cache_bottom;
@@ -980,6 +987,9 @@ static int do_compare_entry(const struct cache_entry *ce,
ce_len -= pathlen;
ce_name = ce->name + pathlen;
+ /* remove directory separator if a sparse directory entry */
+ if (S_ISSPARSEDIR(ce))
+ ce_len--;
return df_name_compare(ce_name, ce_len, S_IFREG, name, namelen, mode);
}
@@ -989,6 +999,10 @@ static int compare_entry(const struct cache_entry *ce, const struct traverse_inf
if (cmp)
return cmp;
+ /* If ce is a sparse directory, then allow equality here. */
+ if (S_ISSPARSEDIR(ce))
+ return 0;
+
/*
* Even if the beginning compared identically, the ce should
* compare as bigger than a directory leading up to it!
@@ -1239,6 +1253,7 @@ static int unpack_callback(int n, unsigned long mask, unsigned long dirmask, str
struct cache_entry *src[MAX_UNPACK_TREES + 1] = { NULL, };
struct unpack_trees_options *o = info->data;
const struct name_entry *p = names;
+ unsigned recurse = 1;
/* Find first entry with a real name (we could use "mask" too) */
while (!p->mode)
@@ -1280,12 +1295,16 @@ static int unpack_callback(int n, unsigned long mask, unsigned long dirmask, str
}
}
src[0] = ce;
+
+ if (S_ISSPARSEDIR(ce))
+ recurse = 0;
}
break;
}
}
- if (unpack_nondirectories(n, mask, dirmask, src, names, info) < 0)
+ if (recurse &&
+ unpack_nondirectories(n, mask, dirmask, src, names, info) < 0)
return -1;
if (o->merge && src[0]) {
@@ -1315,7 +1334,8 @@ static int unpack_callback(int n, unsigned long mask, unsigned long dirmask, str
}
}
- if (traverse_trees_recursive(n, dirmask, mask & ~dirmask,
+ if (recurse &&
+ traverse_trees_recursive(n, dirmask, mask & ~dirmask,
names, info) < 0)
return -1;
return mask;
--
gitgitgadget
next prev parent reply other threads:[~2021-01-25 17:57 UTC|newest]
Thread overview: 61+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-25 17:41 [PATCH 00/27] [RFC] Sparse Index Derrick Stolee via GitGitGadget
2021-01-25 17:41 ` [PATCH 01/27] sparse-index: add guard to ensure full index Derrick Stolee via GitGitGadget
2021-01-25 17:41 ` [PATCH 02/27] sparse-index: implement ensure_full_index() Derrick Stolee via GitGitGadget
2021-01-27 3:05 ` Elijah Newren
2021-01-27 13:43 ` Derrick Stolee
2021-01-27 16:38 ` Elijah Newren
2021-01-28 5:25 ` Junio C Hamano
2021-01-25 17:41 ` [PATCH 03/27] t1092: compare sparse-checkout to sparse-index Derrick Stolee via GitGitGadget
2021-01-27 3:08 ` Elijah Newren
2021-01-27 13:30 ` Derrick Stolee
2021-01-27 16:54 ` Elijah Newren
2021-01-25 17:41 ` [PATCH 04/27] test-read-cache: print cache entries with --table Derrick Stolee via GitGitGadget
2021-01-27 3:25 ` Elijah Newren
2021-01-25 17:41 ` [PATCH 05/27] test-tool: read-cache --table --no-stat Derrick Stolee via GitGitGadget
2021-01-25 17:41 ` [PATCH 06/27] test-tool: don't force full index Derrick Stolee via GitGitGadget
2021-01-25 17:41 ` [PATCH 07/27] unpack-trees: ensure " Derrick Stolee via GitGitGadget
2021-01-27 4:43 ` Elijah Newren
2021-01-25 17:41 ` [PATCH 08/27] sparse-checkout: hold pattern list in index Derrick Stolee via GitGitGadget
2021-01-27 17:00 ` Elijah Newren
2021-01-28 13:12 ` Derrick Stolee
2021-01-25 17:41 ` [PATCH 09/27] sparse-index: convert from full to sparse Derrick Stolee via GitGitGadget
2021-01-27 17:30 ` Elijah Newren
2021-01-25 17:41 ` [PATCH 10/27] submodule: sparse-index should not collapse links Derrick Stolee via GitGitGadget
2021-01-25 17:41 ` [PATCH 11/27] unpack-trees: allow sparse directories Derrick Stolee via GitGitGadget
2021-01-27 17:36 ` Elijah Newren
2021-01-25 17:41 ` [PATCH 12/27] sparse-index: check index conversion happens Derrick Stolee via GitGitGadget
2021-01-27 17:46 ` Elijah Newren
2021-01-25 17:41 ` [PATCH 13/27] sparse-index: create extension for compatibility Derrick Stolee via GitGitGadget
2021-01-27 18:03 ` Elijah Newren
2021-01-25 17:42 ` [PATCH 14/27] sparse-checkout: toggle sparse index from builtin Derrick Stolee via GitGitGadget
2021-01-27 18:18 ` Elijah Newren
2021-01-28 15:26 ` Derrick Stolee
2021-01-25 17:42 ` [PATCH 15/27] [RFC-VERSION] *: ensure full index Derrick Stolee via GitGitGadget
2021-02-01 20:22 ` Elijah Newren
2021-02-01 21:10 ` Derrick Stolee
2021-01-25 17:42 ` Derrick Stolee via GitGitGadget [this message]
2021-02-01 20:50 ` [PATCH 16/27] unpack-trees: make sparse aware Elijah Newren
2021-02-09 17:23 ` Derrick Stolee
2021-01-25 17:42 ` [PATCH 17/27] dir.c: accept a directory as part of cone-mode patterns Derrick Stolee via GitGitGadget
2021-02-01 22:12 ` Elijah Newren
2021-01-25 17:42 ` [PATCH 18/27] status: use sparse-index throughout Derrick Stolee via GitGitGadget
2021-01-25 17:42 ` [PATCH 19/27] status: skip sparse-checkout percentage with sparse-index Derrick Stolee via GitGitGadget
2021-01-25 17:42 ` [PATCH 20/27] sparse-index: expand_to_path() trivial implementation Derrick Stolee via GitGitGadget
2021-01-25 17:42 ` [PATCH 21/27] sparse-index: expand_to_path no-op if path exists Derrick Stolee via GitGitGadget
2021-02-01 22:34 ` Elijah Newren
2021-01-25 17:42 ` [PATCH 22/27] add: allow operating on a sparse-only index Derrick Stolee via GitGitGadget
2021-02-01 23:08 ` Elijah Newren
2021-01-25 17:42 ` [PATCH 23/27] submodule: die_path_inside_submodule is sparse aware Derrick Stolee via GitGitGadget
2021-01-25 17:42 ` [PATCH 24/27] dir: use expand_to_path in add_patterns() Derrick Stolee via GitGitGadget
2021-02-01 23:21 ` Elijah Newren
2021-01-25 17:42 ` [PATCH 25/27] fsmonitor: disable if index is sparse Derrick Stolee via GitGitGadget
2021-01-25 17:42 ` [PATCH 26/27] pathspec: stop calling ensure_full_index Derrick Stolee via GitGitGadget
2021-02-01 23:24 ` Elijah Newren
2021-02-02 2:39 ` Derrick Stolee
2021-01-25 17:42 ` [PATCH 27/27] cache-tree: integrate with sparse directory entries Derrick Stolee via GitGitGadget
2021-02-01 23:54 ` Elijah Newren
2021-02-02 2:41 ` Derrick Stolee
2021-02-02 3:05 ` Elijah Newren
2021-01-25 20:10 ` [PATCH 00/27] [RFC] Sparse Index Junio C Hamano
2021-01-25 21:18 ` Derrick Stolee
2021-02-02 3:11 ` Elijah Newren
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=45cf57c9c40bebb7383b8aab19c82fc4e41d2cd3.1611596534.git.gitgitgadget@gmail.com \
--to=gitgitgadget@gmail.com \
--cc=derrickstolee@github.com \
--cc=dstolee@microsoft.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=jrnieder@gmail.com \
--cc=newren@gmail.com \
--cc=pclouds@gmail.com \
--cc=peff@peff.net \
--cc=sunshine@sunshineco.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).