Git Mailing List Archive on lore.kernel.org
 help / color / Atom feed
From: "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: gitster@pobox.com, newren@gmail.com,
	Derrick Stolee <derrickstolee@github.com>,
	Derrick Stolee <dstolee@microsoft.com>
Subject: [PATCH 08/10] pathspec: stop calling ensure_full_index
Date: Tue, 13 Apr 2021 14:01:35 +0000
Message-ID: <6d7f30f2b90ac2151645a29c6a12639074c8f2d3.1618322498.git.gitgitgadget@gmail.com> (raw)
In-Reply-To: <pull.932.git.1618322497.gitgitgadget@gmail.com>

From: Derrick Stolee <dstolee@microsoft.com>

The add_pathspec_matches_against_index() focuses on matching a pathspec
to file entries in the index. This already works correctly for its only
use: checking if untracked files exist in the index.

The compatibility checks in t1092 already test that 'git add <dir>'
works for a directory outside of the sparse cone. That provides coverage
for removing this guard.

This finalizes our ability to run 'git add .' without expanding a sparse
index to a full one. This is evidenced by an update to t1092 and by
these performance numbers for p2000-sparse-operations.sh:

Test                                    HEAD~1            HEAD
--------------------------------------------------------------------------------
2000.10: git add . (full-index-v3)      1.37(1.02+0.18)   1.38(1.01+0.20) +0.7%
2000.11: git add . (full-index-v4)      1.26(1.00+0.15)   1.27(0.99+0.17) +0.8%
2000.12: git add . (sparse-index-v3)    2.39(2.29+0.14)   0.06(0.05+0.07) -97.5%
2000.13: git add . (sparse-index-v4)    2.42(2.32+0.14)   0.06(0.05+0.06) -97.5%

While the 97% improvement is shown by the test results, it is worth
noting that expanding the sparse index was adding overhead in previous
commits. Comparing to the full index case, we see the performance go
from 1.27s to 0.06s, a 95% improvement.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 pathspec.c                               | 2 --
 t/t1092-sparse-checkout-compatibility.sh | 6 ++++++
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/pathspec.c b/pathspec.c
index 54813c0c4e8e..b51b48471fe6 100644
--- a/pathspec.c
+++ b/pathspec.c
@@ -37,8 +37,6 @@ void add_pathspec_matches_against_index(const struct pathspec *pathspec,
 			num_unmatched++;
 	if (!num_unmatched)
 		return;
-	/* TODO: audit for interaction with sparse-index. */
-	ensure_full_index(istate);
 	for (i = 0; i < istate->cache_nr; i++) {
 		const struct cache_entry *ce = istate->cache[i];
 		if (sw_action == PS_IGNORE_SKIP_WORKTREE && ce_skip_worktree(ce))
diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh
index c210dba78067..738013b00191 100755
--- a/t/t1092-sparse-checkout-compatibility.sh
+++ b/t/t1092-sparse-checkout-compatibility.sh
@@ -471,6 +471,12 @@ test_expect_success 'sparse-index is not expanded' '
 	echo >>sparse-index/extra.txt &&
 	GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \
 		git -C sparse-index add extra.txt &&
+	test_region ! index ensure_full_index trace2.txt &&
+
+	rm trace2.txt &&
+	echo >>sparse-index/untracked.txt &&
+	GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \
+		git -C sparse-index add . &&
 	test_region ! index ensure_full_index trace2.txt
 '
 
-- 
gitgitgadget


  parent reply index

Thread overview: 127+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-13 14:01 [PATCH 00/10] Sparse-index: integrate with status and add Derrick Stolee via GitGitGadget
2021-04-13 14:01 ` [PATCH 01/10] t1092: add tests for status/add and sparse files Derrick Stolee via GitGitGadget
2021-04-20 21:52   ` Elijah Newren
2021-04-21 13:21     ` Derrick Stolee
2021-04-21 15:14   ` Matheus Tavares Bernardino
2021-04-23 20:12     ` Derrick Stolee
2021-04-13 14:01 ` [PATCH 02/10] unpack-trees: make sparse aware Derrick Stolee via GitGitGadget
2021-04-20 23:00   ` Elijah Newren
2021-04-21 13:41     ` Derrick Stolee
2021-04-21 16:11       ` Elijah Newren
2021-04-22  2:24         ` Matheus Tavares Bernardino
2021-04-21 17:27     ` Derrick Stolee
2021-04-21 18:55       ` Matheus Tavares Bernardino
2021-04-21 19:10         ` Elijah Newren
2021-04-21 19:51           ` Matheus Tavares Bernardino
2021-04-21 18:56       ` Elijah Newren
2021-04-23 20:16         ` Derrick Stolee
2021-04-13 14:01 ` [PATCH 03/10] dir.c: accept a directory as part of cone-mode patterns Derrick Stolee via GitGitGadget
2021-04-20 23:21   ` Elijah Newren
2021-04-21 13:47     ` Derrick Stolee
2021-04-13 14:01 ` [PATCH 04/10] status: skip sparse-checkout percentage with sparse-index Derrick Stolee via GitGitGadget
2021-04-20 23:26   ` Elijah Newren
2021-04-21 13:51     ` Derrick Stolee
2021-04-13 14:01 ` [PATCH 05/10] status: use sparse-index throughout Derrick Stolee via GitGitGadget
2021-04-21  0:44   ` Elijah Newren
2021-04-21 13:55     ` Derrick Stolee
2021-04-13 14:01 ` [PATCH 06/10] dir: use expand_to_path() for sparse directories Derrick Stolee via GitGitGadget
2021-04-21  0:52   ` Elijah Newren
2021-04-21  0:53     ` Elijah Newren
2021-04-21 14:03       ` Derrick Stolee
2021-04-13 14:01 ` [PATCH 07/10] add: allow operating on a sparse-only index Derrick Stolee via GitGitGadget
2021-04-13 14:01 ` Derrick Stolee via GitGitGadget [this message]
2021-04-21  0:57   ` [PATCH 08/10] pathspec: stop calling ensure_full_index Elijah Newren
2021-04-13 14:01 ` [PATCH 09/10] t7519: add sparse directories to FS monitor tests Derrick Stolee via GitGitGadget
2021-04-13 14:01 ` [PATCH 10/10] fsmonitor: test with sparse index Derrick Stolee via GitGitGadget
2021-04-21  7:00   ` Elijah Newren
2021-04-13 20:45 ` [PATCH 00/10] Sparse-index: integrate with status and add Matheus Tavares Bernardino
2021-04-14 16:31   ` Derrick Stolee
2021-04-23 21:34 ` [PATCH v2 0/8] Sparse-index: integrate with status Derrick Stolee via GitGitGadget
2021-04-23 21:34   ` [PATCH v2 1/8] t1092: add tests for status/add and sparse files Derrick Stolee via GitGitGadget
2021-05-13 12:40     ` Matheus Tavares Bernardino
2021-05-14 12:27       ` Derrick Stolee
2021-04-23 21:34   ` [PATCH v2 2/8] unpack-trees: preserve cache_bottom Derrick Stolee via GitGitGadget
2021-04-23 21:34   ` [PATCH v2 3/8] unpack-trees: compare sparse directories correctly Derrick Stolee via GitGitGadget
2021-05-13  3:26     ` Elijah Newren
2021-04-23 21:34   ` [PATCH v2 4/8] unpack-trees: stop recursing into sparse directories Derrick Stolee via GitGitGadget
2021-05-13  3:31     ` Elijah Newren
2021-04-23 21:34   ` [PATCH v2 5/8] dir.c: accept a directory as part of cone-mode patterns Derrick Stolee via GitGitGadget
2021-04-23 21:34   ` [PATCH v2 6/8] status: skip sparse-checkout percentage with sparse-index Derrick Stolee via GitGitGadget
2021-04-23 21:34   ` [PATCH v2 7/8] status: use sparse-index throughout Derrick Stolee via GitGitGadget
2021-04-23 21:34   ` [PATCH v2 8/8] fsmonitor: test with sparse index Derrick Stolee via GitGitGadget
2021-05-13  4:12   ` [PATCH v2 0/8] Sparse-index: integrate with status Elijah Newren
2021-05-14 18:28     ` Derrick Stolee
2021-05-14 18:30   ` [PATCH v3 00/12] " Derrick Stolee via GitGitGadget
2021-05-14 18:31     ` [PATCH v3 01/12] sparse-index: skip indexes with unmerged entries Derrick Stolee via GitGitGadget
2021-05-14 18:31     ` [PATCH v3 02/12] sparse-index: include EXTENDED flag when expanding Derrick Stolee via GitGitGadget
2021-05-18  1:33       ` Elijah Newren
2021-05-18 14:57         ` Derrick Stolee
2021-05-18 17:48           ` Elijah Newren
2021-05-18 18:16             ` Derrick Stolee
2021-05-14 18:31     ` [PATCH v3 03/12] t1092: expand repository data shape Derrick Stolee via GitGitGadget
2021-05-18  1:49       ` Elijah Newren
2021-05-18 14:59         ` Derrick Stolee
2021-05-14 18:31     ` [PATCH v3 04/12] t1092: add tests for status/add and sparse files Derrick Stolee via GitGitGadget
2021-05-14 18:31     ` [PATCH v3 05/12] unpack-trees: preserve cache_bottom Derrick Stolee via GitGitGadget
2021-05-14 18:31     ` [PATCH v3 06/12] unpack-trees: compare sparse directories correctly Derrick Stolee via GitGitGadget
2021-05-14 18:31     ` [PATCH v3 07/12] unpack-trees: stop recursing into sparse directories Derrick Stolee via GitGitGadget
2021-05-18  2:03       ` Elijah Newren
2021-05-18  2:06         ` Elijah Newren
2021-05-18 19:20           ` Derrick Stolee
2021-05-14 18:31     ` [PATCH v3 08/12] dir.c: accept a directory as part of cone-mode patterns Derrick Stolee via GitGitGadget
2021-05-14 18:31     ` [PATCH v3 09/12] status: skip sparse-checkout percentage with sparse-index Derrick Stolee via GitGitGadget
2021-05-14 18:31     ` [PATCH v3 10/12] status: use sparse-index throughout Derrick Stolee via GitGitGadget
2021-05-14 18:31     ` [PATCH v3 11/12] wt-status: expand added sparse directory entries Derrick Stolee via GitGitGadget
2021-05-18  2:27       ` Elijah Newren
2021-05-18 18:26         ` Derrick Stolee
2021-05-18 19:04           ` Derrick Stolee
2021-05-19  8:38             ` Elijah Newren
2021-05-14 18:31     ` [PATCH v3 12/12] fsmonitor: integrate with sparse index Derrick Stolee via GitGitGadget
2021-05-21 11:59     ` [PATCH v4 00/12] Sparse-index: integrate with status Derrick Stolee via GitGitGadget
2021-05-21 11:59       ` [PATCH v4 01/12] sparse-index: skip indexes with unmerged entries Derrick Stolee via GitGitGadget
2021-05-21 11:59       ` [PATCH v4 02/12] sparse-index: include EXTENDED flag when expanding Derrick Stolee via GitGitGadget
2021-05-21 11:59       ` [PATCH v4 03/12] t1092: expand repository data shape Derrick Stolee via GitGitGadget
2021-05-21 11:59       ` [PATCH v4 04/12] t1092: add tests for status/add and sparse files Derrick Stolee via GitGitGadget
2021-05-21 11:59       ` [PATCH v4 05/12] unpack-trees: preserve cache_bottom Derrick Stolee via GitGitGadget
2021-05-21 11:59       ` [PATCH v4 06/12] unpack-trees: compare sparse directories correctly Derrick Stolee via GitGitGadget
2021-05-21 11:59       ` [PATCH v4 07/12] unpack-trees: be careful around sparse directory entries Derrick Stolee via GitGitGadget
2021-05-28 11:36         ` Derrick Stolee
2021-05-21 11:59       ` [PATCH v4 08/12] dir.c: accept a directory as part of cone-mode patterns Derrick Stolee via GitGitGadget
2021-05-21 11:59       ` [PATCH v4 09/12] status: skip sparse-checkout percentage with sparse-index Derrick Stolee via GitGitGadget
2021-05-21 11:59       ` [PATCH v4 10/12] status: use sparse-index throughout Derrick Stolee via GitGitGadget
2021-05-21 11:59       ` [PATCH v4 11/12] wt-status: expand added sparse directory entries Derrick Stolee via GitGitGadget
2021-05-21 11:59       ` [PATCH v4 12/12] fsmonitor: integrate with sparse index Derrick Stolee via GitGitGadget
2021-06-07 12:33       ` [PATCH v5 00/14] Sparse-index: integrate with status Derrick Stolee via GitGitGadget
2021-06-07 12:33         ` [PATCH v5 01/14] sparse-index: skip indexes with unmerged entries Derrick Stolee via GitGitGadget
2021-06-07 12:34         ` [PATCH v5 02/14] sparse-index: include EXTENDED flag when expanding Derrick Stolee via GitGitGadget
2021-06-08 18:56           ` Elijah Newren
2021-06-09 17:39             ` Derrick Stolee
2021-06-09 18:11               ` Elijah Newren
2021-06-07 12:34         ` [PATCH v5 03/14] t1092: replace incorrect 'echo' with 'cat' Derrick Stolee via GitGitGadget
2021-06-08 19:18           ` Elijah Newren
2021-06-07 12:34         ` [PATCH v5 04/14] t1092: expand repository data shape Derrick Stolee via GitGitGadget
2021-06-07 12:34         ` [PATCH v5 05/14] t1092: add tests for status/add and sparse files Derrick Stolee via GitGitGadget
2021-06-07 12:34         ` [PATCH v5 06/14] unpack-trees: preserve cache_bottom Derrick Stolee via GitGitGadget
2021-06-07 12:34         ` [PATCH v5 07/14] unpack-trees: compare sparse directories correctly Derrick Stolee via GitGitGadget
2021-06-07 12:34         ` [PATCH v5 08/14] unpack-trees: unpack sparse directory entries Derrick Stolee via GitGitGadget
2021-06-09  3:48           ` Elijah Newren
2021-06-09 20:21             ` Derrick Stolee
2021-06-07 12:34         ` [PATCH v5 09/14] dir.c: accept a directory as part of cone-mode patterns Derrick Stolee via GitGitGadget
2021-06-07 12:34         ` [PATCH v5 10/14] diff-lib: handle index diffs with sparse dirs Derrick Stolee via GitGitGadget
2021-06-07 15:26           ` Derrick Stolee
2021-06-08  1:05             ` Junio C Hamano
2021-06-08 13:00               ` Derrick Stolee
2021-06-09  5:47           ` Elijah Newren
2021-06-09  6:32             ` Junio C Hamano
2021-06-09  8:11               ` Elijah Newren
2021-06-09 20:33                 ` Derrick Stolee
2021-06-10 17:45                   ` Derrick Stolee
2021-06-10 21:31                     ` Elijah Newren
2021-06-11 12:57                       ` Derrick Stolee
2021-06-11 17:27                         ` Derrick Stolee
2021-06-07 12:34         ` [PATCH v5 11/14] status: skip sparse-checkout percentage with sparse-index Derrick Stolee via GitGitGadget
2021-06-07 12:34         ` [PATCH v5 12/14] status: use sparse-index throughout Derrick Stolee via GitGitGadget
2021-06-07 12:34         ` [PATCH v5 13/14] wt-status: expand added sparse directory entries Derrick Stolee via GitGitGadget
2021-06-09  5:27           ` Elijah Newren
2021-06-09 20:49             ` Derrick Stolee
2021-06-07 12:34         ` [PATCH v5 14/14] fsmonitor: integrate with sparse index Derrick Stolee via GitGitGadget

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=6d7f30f2b90ac2151645a29c6a12639074c8f2d3.1618322498.git.gitgitgadget@gmail.com \
    --to=gitgitgadget@gmail.com \
    --cc=derrickstolee@github.com \
    --cc=dstolee@microsoft.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=newren@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Git Mailing List Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/git/0 git/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 git git/ https://lore.kernel.org/git \
		git@vger.kernel.org
	public-inbox-index git

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.git


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git