All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jeff King <peff@peff.net>
To: git@vger.kernel.org
Cc: "René Scharfe" <l.s.r@web.de>, "SZEDER Gábor" <szeder.dev@gmail.com>
Subject: [PATCH v2 06/13] packfile: fix pack basename computation
Date: Fri, 5 Apr 2019 14:06:22 -0400	[thread overview]
Message-ID: <20190405180622.GF32243@sigill.intra.peff.net> (raw)
In-Reply-To: <20190405180306.GA21113@sigill.intra.peff.net>

When we have a multi-pack-index that covers many packfiles, we try to
avoid opening the .idx for those packfiles. To do that we feed the pack
name to midx_contains_pack(). But that function wants to see only the
basename, which we compute using strrchr() to find the final slash. But
that leaves an extra "/" at the start of our string.

We can fix this by incrementing the pointer. That also raises the
question of what to do when the name does not have a '/' at all. This
should generally not happen (we always find files in "pack/"), but it
doesn't hurt to be defensive here.

Let's wrap all of that up in a helper function and make it publicly
available, since a later patch will need to use it, too.

The tests don't notice because there's nothing about opening those .idx
files that would cause us to give incorrect output. It's just a little
slower. The new test checks this case by corrupting the covered .idx,
and then making sure we don't complain about it.

We also have to tweak t5570, which intentionally corrupts a .idx file
and expects us to notice it. When run with GIT_TEST_MULTI_PACK_INDEX,
this will fail since we now will (correctly) not bother opening the .idx
at all. We can fix that by unconditionally dropping any midx that's
there, which ensures we'll have to read the .idx.

Signed-off-by: Jeff King <peff@peff.net>
---
 packfile.c                  | 12 +++++++++++-
 packfile.h                  |  6 ++++++
 t/t5319-multi-pack-index.sh | 14 ++++++++++++++
 t/t5570-git-daemon.sh       |  1 +
 4 files changed, 32 insertions(+), 1 deletion(-)

diff --git a/packfile.c b/packfile.c
index 6e40bd89c7..7a2dd2fdbe 100644
--- a/packfile.c
+++ b/packfile.c
@@ -466,6 +466,16 @@ static unsigned int get_max_fd_limit(void)
 #endif
 }
 
+const char *pack_basename(struct packed_git *p)
+{
+	const char *ret = strrchr(p->pack_name, '/');
+	if (ret)
+		ret = ret + 1; /* skip past slash */
+	else
+		ret = p->pack_name; /* we only have a base */
+	return ret;
+}
+
 /*
  * Do not call this directly as this leaks p->pack_fd on error return;
  * call open_packed_git() instead.
@@ -482,7 +492,7 @@ static int open_packed_git_1(struct packed_git *p)
 
 	if (!p->index_data) {
 		struct multi_pack_index *m;
-		const char *pack_name = strrchr(p->pack_name, '/');
+		const char *pack_name = pack_basename(p);
 
 		for (m = the_repository->objects->multi_pack_index;
 		     m; m = m->next) {
diff --git a/packfile.h b/packfile.h
index b40fc34fb2..45bf792d79 100644
--- a/packfile.h
+++ b/packfile.h
@@ -31,6 +31,12 @@ char *sha1_pack_name(const unsigned char *sha1);
  */
 char *sha1_pack_index_name(const unsigned char *sha1);
 
+/*
+ * Return the basename of the packfile, omitting any containing directory
+ * (e.g., "pack-1234abcd[...].pack").
+ */
+const char *pack_basename(struct packed_git *p);
+
 struct packed_git *parse_pack_index(unsigned char *sha1, const char *idx_path);
 
 typedef void each_file_in_pack_dir_fn(const char *full_path, size_t full_path_len,
diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh
index 8c4d2bd849..1ebf19ec3c 100755
--- a/t/t5319-multi-pack-index.sh
+++ b/t/t5319-multi-pack-index.sh
@@ -117,6 +117,20 @@ test_expect_success 'write midx with one v2 pack' '
 
 compare_results_with_midx "one v2 pack"
 
+test_expect_success 'corrupt idx not opened' '
+	idx=$(test-tool read-midx $objdir | grep "\.idx\$") &&
+	mv $objdir/pack/$idx backup-$idx &&
+	test_when_finished "mv backup-\$idx \$objdir/pack/\$idx" &&
+
+	# This is the minimum size for a sha-1 based .idx; this lets
+	# us pass perfunctory tests, but anything that actually opens and reads
+	# the idx file will complain.
+	test_copy_bytes 1064 <backup-$idx >$objdir/pack/$idx &&
+
+	git -c core.multiPackIndex=true rev-list --objects --all 2>err &&
+	test_must_be_empty err
+'
+
 test_expect_success 'add more objects' '
 	for i in $(test_seq 6 10)
 	do
diff --git a/t/t5570-git-daemon.sh b/t/t5570-git-daemon.sh
index 58ee787685..19e271bda6 100755
--- a/t/t5570-git-daemon.sh
+++ b/t/t5570-git-daemon.sh
@@ -90,6 +90,7 @@ test_expect_success 'fetch notices corrupt pack' '
 test_expect_success 'fetch notices corrupt idx' '
 	cp -R "$GIT_DAEMON_DOCUMENT_ROOT_PATH"/repo_pack.git "$GIT_DAEMON_DOCUMENT_ROOT_PATH"/repo_bad2.git &&
 	(cd "$GIT_DAEMON_DOCUMENT_ROOT_PATH"/repo_bad2.git &&
+	 rm -f objects/pack/multi-pack-index &&
 	 p=$(ls objects/pack/pack-*.idx) &&
 	 chmod u+w $p &&
 	 printf %0256d 0 | dd of=$p bs=256 count=1 seek=1 conv=notrunc
-- 
2.21.0.729.g7d31bf3764


  parent reply	other threads:[~2019-04-05 18:06 UTC|newest]

Thread overview: 48+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-04-04 23:21 [PATCH 0/12] a rabbit hole of update-server-info fixes Jeff King
2019-04-04 23:22 ` [PATCH 01/12] t5319: fix bogus cat-file argument Jeff King
2019-04-05  0:44   ` Ramsay Jones
2019-04-05  1:41     ` Jeff King
2019-04-05  1:46       ` Jeff King
2019-04-04 23:22 ` [PATCH 02/12] t5319: drop useless --buffer from cat-file Jeff King
2019-04-04 23:22 ` [PATCH 03/12] packfile: factor out .pack to .idx name conversion Jeff King
2019-04-04 23:25 ` [PATCH 04/12] packfile: check midx coverage with .idx rather than .pack Jeff King
2019-04-05  8:05   ` René Scharfe
2019-04-05 13:21     ` Jeff King
2019-04-05 13:30       ` Jeff King
2019-04-05 12:01   ` SZEDER Gábor
2019-04-05 13:40     ` Jeff King
2019-04-04 23:27 ` [PATCH 05/12] http: simplify parsing of remote objects/info/packs Jeff King
2019-04-05 10:41   ` René Scharfe
2019-04-05 18:11     ` Jeff King
2019-04-05 20:17       ` René Scharfe
2019-04-04 23:27 ` [PATCH 06/12] server-info: fix blind pointer arithmetic Jeff King
2019-04-04 23:28 ` [PATCH 07/12] server-info: simplify cleanup in parse_pack_def() Jeff King
2019-04-04 23:28 ` [PATCH 08/12] server-info: use strbuf to read old info/packs file Jeff King
2019-04-04 23:29 ` [PATCH 09/12] server-info: drop nr_alloc struct member Jeff King
2019-04-04 23:30 ` [PATCH 10/12] packfile.h: drop extern from function declarations Jeff King
2019-04-04 23:30 ` [PATCH 11/12] server-info: drop objdirlen pointer arithmetic Jeff King
2019-04-04 23:31 ` [PATCH 12/12] update_info_refs(): drop unused force parameter Jeff King
2019-04-05  5:46 ` [PATCH 0/12] a rabbit hole of update-server-info fixes Junio C Hamano
2019-04-05 18:03 ` [PATCH v2 0/13] a rabbit hole of update-server-info (and midx!) fixes Jeff King
2019-04-05 18:03   ` [PATCH v2 01/13] packfile.h: drop extern from function declarations Jeff King
2019-04-05 19:19     ` Ramsay Jones
2019-04-05 19:25       ` Jeff King
2019-04-08  5:13         ` Junio C Hamano
2019-04-08 20:32           ` Jeff King
2019-04-09 15:08         ` Junio C Hamano
2019-04-09 15:15           ` Jeff King
2019-04-05 18:04   ` [PATCH v2 02/13] pack-revindex: open index if necessary Jeff King
2019-04-05 18:04   ` [PATCH v2 03/13] t5319: fix bogus cat-file argument Jeff King
2019-04-05 18:05   ` [PATCH v2 04/13] t5319: drop useless --buffer from cat-file Jeff King
2019-04-05 18:06   ` [PATCH v2 05/13] midx: check both pack and index names for containment Jeff King
2019-04-05 20:18     ` René Scharfe
2019-04-05 18:06   ` Jeff King [this message]
2019-04-05 18:12   ` [PATCH v2 07/13] http: simplify parsing of remote objects/info/packs Jeff King
2019-04-05 18:13   ` [PATCH v2 08/13] server-info: fix blind pointer arithmetic Jeff King
2019-04-05 18:13   ` [PATCH v2 09/13] server-info: simplify cleanup in parse_pack_def() Jeff King
2019-04-05 18:16     ` Jeff King
2019-04-05 18:13   ` [PATCH v2 10/13] server-info: use strbuf to read old info/packs file Jeff King
2019-04-05 18:14   ` [PATCH v2 11/13] server-info: drop nr_alloc struct member Jeff King
2019-04-05 18:14   ` [PATCH v2 12/13] server-info: drop objdirlen pointer arithmetic Jeff King
2019-04-05 18:14   ` [PATCH v2 13/13] update_info_refs(): drop unused force parameter Jeff King
2019-04-05 18:19   ` [PATCH v2 0/13] a rabbit hole of update-server-info (and midx!) fixes Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190405180622.GF32243@sigill.intra.peff.net \
    --to=peff@peff.net \
    --cc=git@vger.kernel.org \
    --cc=l.s.r@web.de \
    --cc=szeder.dev@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.