From: Jeff King <peff@peff.net>
To: Taylor Blau <me@ttaylorr.com>
Cc: git@vger.kernel.org
Subject: Re: [PATCH 0/2] pack-write,repack: prevent opening packs too early
Date: Tue, 7 Sep 2021 12:07:01 -0400 [thread overview]
Message-ID: <YTeOJTlGpXBD8mns@coredump.intra.peff.net> (raw)
In-Reply-To: <YS/zRlL40SkanszX@nand.local>
On Wed, Sep 01, 2021 at 05:40:22PM -0400, Taylor Blau wrote:
> Hmm. As I was wondering about about, this is more complicated than meets
> the eye. Consider t5616.36, which tests that repacking does not loosen
> promisor objects.
>
> In builtin/repack.c:repack_promisor_object(), the repack builtin tells
> pack-objects about the pack that it just wrote with `--keep-pack` (and
> we rely on that working in order to not loosen all of the objects that
> we just wrote).
>
> Except when we iterate through `get_all_packs()`, we don't see the keep
> pack yet, because it is still prefixed with .tmp.
>
> So, this does get kind of tricky. There are some internal callers that
> do want to know about .tmp packs and a whole host of other callers that
> don't or shouldn't. Maybe that should point us towards "we should be
> more careful about the order we write packs in, even temporary ones".
So I happened to be looking at some packing stuff again today, and I
realized the situation is much less dire than I made it out to be.
On the reading side, we _won't_ look at tmp_pack_*, nor tmp_idx_*,
because they don't end in '.pack' or '.idx'. So really, the only
confusing case is the ".tmp-$$-etc.idx" that is generated by
pack-objects (because it thinks of that as the final name to use
itself), and then later rearranged by git-repack.
So I think we _are_ OK as long as the correct order is observed during
the rename-into-place steps.
I.e., your original patch (plus the extra post in index-pack) would mean
that we're fully covered. The .tmp-*.idx ones would appear in the
correct order due to pack-objects, and the final pack-*.idx ones due to
git-repack.
-Peff
next prev parent reply other threads:[~2021-09-07 16:07 UTC|newest]
Thread overview: 81+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-01 2:05 [PATCH 0/2] pack-write,repack: prevent opening packs too early Taylor Blau
2021-09-01 2:06 ` [PATCH 1/2] pack-write.c: rename `.idx` file into place last Taylor Blau
2021-09-01 2:06 ` [PATCH 2/2] builtin/repack.c: move `.idx` files " Taylor Blau
2021-09-01 3:53 ` [PATCH 0/2] pack-write,repack: prevent opening packs too early Jeff King
2021-09-01 4:29 ` Taylor Blau
2021-09-01 4:59 ` Jeff King
2021-09-01 5:12 ` Taylor Blau
2021-09-01 6:08 ` Jeff King
2021-09-01 21:40 ` Taylor Blau
2021-09-07 16:07 ` Jeff King [this message]
2021-09-07 19:42 ` [PATCH 0/3] rename *.idx file into place last (also after *.bitmap) Ævar Arnfjörð Bjarmason
2021-09-07 19:42 ` [PATCH 1/3] pack-write: use more idiomatic strbuf usage for packname construction Ævar Arnfjörð Bjarmason
2021-09-07 22:21 ` Taylor Blau
2021-09-07 23:22 ` Ævar Arnfjörð Bjarmason
2021-09-07 19:42 ` [PATCH 2/3] pack-write: split up finish_tmp_packfile() function Ævar Arnfjörð Bjarmason
2021-09-07 22:28 ` Taylor Blau
2021-09-07 19:42 ` [PATCH 3/3] pack-write: rename *.idx file into place last (really!) Ævar Arnfjörð Bjarmason
2021-09-07 22:31 ` Taylor Blau
2021-09-07 22:36 ` [PATCH 0/3] rename *.idx file into place last (also after *.bitmap) Taylor Blau
2021-09-07 19:48 ` [PATCH 0/2] pack-write,repack: prevent opening packs too early Ævar Arnfjörð Bjarmason
2021-09-08 0:38 ` [PATCH v2 0/4] rename *.idx file into place last (also after *.bitmap) Ævar Arnfjörð Bjarmason
2021-09-08 0:38 ` [PATCH v2 1/4] pack.h: line-wrap the definition of finish_tmp_packfile() Ævar Arnfjörð Bjarmason
2021-09-08 0:38 ` [PATCH v2 2/4] pack-write: refactor renaming in finish_tmp_packfile() Ævar Arnfjörð Bjarmason
2021-09-08 4:22 ` Taylor Blau
2021-09-08 0:38 ` [PATCH v2 3/4] pack-write: split up finish_tmp_packfile() function Ævar Arnfjörð Bjarmason
2021-09-08 0:38 ` [PATCH v2 4/4] pack-write: rename *.idx file into place last (really!) Ævar Arnfjörð Bjarmason
2021-09-08 1:14 ` Ævar Arnfjörð Bjarmason
2021-09-08 9:18 ` Ævar Arnfjörð Bjarmason
2021-09-08 4:24 ` Taylor Blau
2021-09-08 22:17 ` [PATCH v2 0/3] prevent opening packs too early Taylor Blau
2021-09-08 22:17 ` [PATCH v2 1/3] pack-write.c: rename `.idx` files into place last Taylor Blau
2021-09-08 22:17 ` [PATCH v2 2/3] builtin/repack.c: move " Taylor Blau
2021-09-08 22:17 ` [PATCH v2 3/3] builtin/index-pack.c: " Taylor Blau
2021-09-08 23:52 ` [PATCH v2 0/3] prevent opening packs too early Ævar Arnfjörð Bjarmason
2021-09-09 0:50 ` Ævar Arnfjörð Bjarmason
2021-09-09 1:13 ` Taylor Blau
2021-09-09 1:33 ` Ævar Arnfjörð Bjarmason
2021-09-09 2:36 ` Ævar Arnfjörð Bjarmason
2021-09-09 2:49 ` Taylor Blau
2021-09-09 3:24 ` [PATCH 0/9] packfile: avoid .idx rename races Taylor Blau
2021-09-09 3:24 ` [PATCH 1/9] pack.h: line-wrap the definition of finish_tmp_packfile() Taylor Blau
2021-09-09 3:24 ` [PATCH 2/9] bulk-checkin.c: store checksum directly Taylor Blau
2021-09-09 7:38 ` Ævar Arnfjörð Bjarmason
2021-09-09 3:24 ` [PATCH 3/9] pack-write: refactor renaming in finish_tmp_packfile() Taylor Blau
2021-09-09 19:29 ` Junio C Hamano
2021-09-09 21:07 ` Taylor Blau
2021-09-09 23:30 ` Junio C Hamano
2021-09-09 23:31 ` Taylor Blau
2021-09-10 1:29 ` Junio C Hamano
2021-09-09 3:24 ` [PATCH 4/9] pack-write.c: rename `.idx` files after `*.rev` Taylor Blau
2021-09-09 7:46 ` Ævar Arnfjörð Bjarmason
2021-09-09 14:37 ` Taylor Blau
2021-09-09 19:32 ` Junio C Hamano
2021-09-09 3:25 ` [PATCH 5/9] builtin/repack.c: move `.idx` files into place last Taylor Blau
2021-09-09 19:38 ` Junio C Hamano
2021-09-09 21:08 ` Taylor Blau
2021-09-09 3:25 ` [PATCH 6/9] index-pack: refactor renaming in final() Taylor Blau
2021-09-09 19:45 ` Junio C Hamano
2021-09-09 21:11 ` Taylor Blau
2021-09-09 3:25 ` [PATCH 7/9] builtin/index-pack.c: move `.idx` files into place last Taylor Blau
2021-09-09 7:52 ` Ævar Arnfjörð Bjarmason
2021-09-09 19:45 ` Junio C Hamano
2021-09-09 3:25 ` [PATCH 8/9] pack-write: split up finish_tmp_packfile() function Taylor Blau
2021-09-09 3:25 ` [PATCH 9/9] pack-objects: rename .idx files into place after .bitmap files Taylor Blau
2021-09-09 7:54 ` Ævar Arnfjörð Bjarmason
2021-09-09 19:52 ` Junio C Hamano
2021-09-09 21:13 ` Taylor Blau
2021-09-09 8:06 ` [PATCH 0/9] packfile: avoid .idx rename races Ævar Arnfjörð Bjarmason
2021-09-09 14:40 ` Taylor Blau
2021-09-09 19:52 ` Junio C Hamano
2021-09-09 23:24 ` [PATCH v2 " Taylor Blau
2021-09-09 23:24 ` [PATCH v2 1/9] pack.h: line-wrap the definition of finish_tmp_packfile() Taylor Blau
2021-09-09 23:24 ` [PATCH v2 2/9] bulk-checkin.c: store checksum directly Taylor Blau
2021-09-09 23:24 ` [PATCH v2 3/9] pack-write: refactor renaming in finish_tmp_packfile() Taylor Blau
2021-09-09 23:24 ` [PATCH v2 4/9] pack-write.c: rename `.idx` files after `*.rev` Taylor Blau
2021-09-09 23:24 ` [PATCH v2 5/9] builtin/repack.c: move `.idx` files into place last Taylor Blau
2021-09-09 23:24 ` [PATCH v2 6/9] index-pack: refactor renaming in final() Taylor Blau
2021-09-09 23:24 ` [PATCH v2 7/9] builtin/index-pack.c: move `.idx` files into place last Taylor Blau
2021-09-09 23:24 ` [PATCH v2 8/9] pack-write: split up finish_tmp_packfile() function Taylor Blau
2021-09-09 23:25 ` [PATCH v2 9/9] pack-objects: rename .idx files into place after .bitmap files Taylor Blau
2021-09-10 1:35 ` [PATCH v2 0/9] packfile: avoid .idx rename races Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YTeOJTlGpXBD8mns@coredump.intra.peff.net \
--to=peff@peff.net \
--cc=git@vger.kernel.org \
--cc=me@ttaylorr.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.