All of lore.kernel.org
 help / color / mirror / Atom feed
From: Patrick Steinhardt <ps@pks.im>
To: git@vger.kernel.org
Subject: [PATCH] receive-pack: fix stale packfile locks when dying
Date: Thu, 9 Mar 2023 14:09:23 +0100	[thread overview]
Message-ID: <e16bd81bf9e251aa6959fbe10a3fbc215a4a1c12.1678367338.git.ps@pks.im> (raw)

[-- Attachment #1: Type: text/plain, Size: 2102 bytes --]

When accepting a packfile in git-receive-pack(1), we feed that packfile
into git-index-pack(1) to generate the packfile index. As the packfile
would often only contain unreachable objects until the references have
been updated, concurrently running garbage collection might be tempted
to delete the packfile right away and thus cause corruption. To fix
this, we ask git-index-pack(1) to create a `.keep` file before moving
the packfile into place, which is getting deleted again once all of the
reference updates have been processed.

Now in production systems we have observed that those `.keep` files are
sometimes not getting deleted as expected, where the result is that
repositories tend to grow packfiles that are never deleted over time.
This seems to be caused by a race when git-receive-pack(1) is killed
after we have migrated the kept packfile from the quarantine directory
into the main object database. While this race window is typically small
it can be extended for example by installing a `proc-receive` hook.

Fix this race by installing an atexit(3P) handler that unlinks the keep
file.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/receive-pack.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index cd5c7a28ef..0a6030d775 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -2186,6 +2186,12 @@ static const char *parse_pack_header(struct pack_header *hdr)
 
 static const char *pack_lockfile;
 
+static void unlink_pack_lockfile(void)
+{
+	if (pack_lockfile)
+		unlink(pack_lockfile);
+}
+
 static void push_header_arg(struct strvec *args, struct pack_header *hdr)
 {
 	strvec_pushf(args, "--pack_header=%"PRIu32",%"PRIu32,
@@ -2281,6 +2287,7 @@ static const char *unpack(int err_fd, struct shallow_info *si)
 		if (status)
 			return "index-pack fork failed";
 		pack_lockfile = index_pack_lockfile(child.out, NULL);
+		atexit(unlink_pack_lockfile);
 		close(child.out);
 		status = finish_command(&child);
 		if (status)
-- 
2.40.0.rc0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

             reply	other threads:[~2023-03-09 13:09 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-03-09 13:09 Patrick Steinhardt [this message]
2023-03-09 15:59 ` [PATCH] receive-pack: fix stale packfile locks when dying Jeff King
2023-03-10  6:24   ` Patrick Steinhardt
2023-03-10  8:37     ` Jeff King
2023-03-09 18:26 ` Taylor Blau
2023-03-09 18:48 ` Junio C Hamano
2023-03-09 18:49   ` Junio C Hamano
2023-03-10  6:52 ` [PATCH v2] " Patrick Steinhardt
2023-03-10  8:51   ` Jeff King
2023-03-10 11:49     ` Patrick Steinhardt
2023-03-10 13:20       ` Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=e16bd81bf9e251aa6959fbe10a3fbc215a4a1c12.1678367338.git.ps@pks.im \
    --to=ps@pks.im \
    --cc=git@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.