From: Taylor Blau <me@ttaylorr.com>
To: Jonathan Tan <jonathantanmy@google.com>
Cc: me@ttaylorr.com, git@vger.kernel.org, peff@peff.net,
gitster@pobox.com, avarab@gmail.com
Subject: Re: [PATCH v2 3/8] midx: preliminary support for `--refs-snapshot`
Date: Wed, 22 Sep 2021 22:18:07 -0400 [thread overview]
Message-ID: <YUvj33QX5fvjNoiJ@nand.local> (raw)
In-Reply-To: <20210922230012.2608849-1-jonathantanmy@google.com>
On Wed, Sep 22, 2021 at 04:00:12PM -0700, Jonathan Tan wrote:
> > This approach will cause a problem when multi-pack bitmaps are able to
> > be generated from `git repack`, since the reference tips can change
> > during the repack. Even though we ignore commits that don't exist in
> > the MIDX (when doing a scan of the ref tips), it's possible that a
> > commit in the MIDX reaches something that isn't.
> >
> > This can happen when a multi-pack index contains some pack which refers
> > to loose objects (which by definition aren't included in the multi-pack
> > index).
> >
> > By taking a snapshot of the references before we start repacking, we can
> > close that race window.
>
> I can understand why we want the refs to remain the same both for the
> MIDX generation and the MIDX bitmap generation (one reason that comes to
> mind is how we select the commits for which to generate bitmaps), but I
> don't understand what referring to loose objects has to do with it. I
> think that using the same set of refs for MIDX generation and bitmap
> generation is a good enough reason to do this, and we don't need to
> mention loose objects.
The point there is that a pack which contains objects that are ancestors
of loose objects can show up at any time, including just before we
select which packs to go into a MIDX.
This is particularly common at GitHub, where all of our test-merges and
objects created via the web interface are written loose. So we could
compute a test merge, store the result loose, and have somebody push a
pack up on top of it.
If that pack shows up after the repack, but before we write a MIDX, then
without the refs-snapshot code, we would include that pack, select its
commits as candidates for bitmap selection, and then ultimately discover
that the bitmap isn't closed, since the loose object won't be included.
But that may be a little into the weeds for the patch message. Anyway,
I did have to think about it for a minute, but I'm pretty sure that's
what I was thinking when I wrote this.
Thanks,
Taylor
next prev parent reply other threads:[~2021-09-23 2:18 UTC|newest]
Thread overview: 76+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-11 3:32 [PATCH 0/8] repack: introduce `--write-midx` Taylor Blau
2021-09-11 3:32 ` [PATCH 1/8] midx: expose 'write_midx_file_only()' publicly Taylor Blau
2021-09-11 5:00 ` Junio C Hamano
2021-09-11 16:17 ` Taylor Blau
2021-09-11 10:07 ` Ævar Arnfjörð Bjarmason
2021-09-11 16:21 ` Taylor Blau
2021-09-11 3:32 ` [PATCH 2/8] builtin/multi-pack-index.c: support --stdin-packs mode Taylor Blau
2021-09-11 10:05 ` Ævar Arnfjörð Bjarmason
2021-09-11 16:25 ` Taylor Blau
2021-09-11 16:28 ` Taylor Blau
2021-09-12 2:08 ` Eric Sunshine
2021-09-12 2:21 ` Taylor Blau
2021-09-12 15:15 ` Ævar Arnfjörð Bjarmason
2021-09-12 22:30 ` Junio C Hamano
2021-09-12 22:32 ` Ævar Arnfjörð Bjarmason
2021-09-14 19:02 ` Jeff King
2021-09-14 23:48 ` Taylor Blau
2021-09-15 1:55 ` Eric Sunshine
2021-09-11 3:32 ` [PATCH 3/8] midx: preliminary support for `--refs-snapshot` Taylor Blau
2021-09-11 3:32 ` [PATCH 4/8] builtin/repack.c: keep track of existing packs unconditionally Taylor Blau
2021-09-11 3:32 ` [PATCH 5/8] builtin/repack.c: extract showing progress to a variable Taylor Blau
2021-09-11 3:32 ` [PATCH 6/8] builtin/repack.c: support writing a MIDX while repacking Taylor Blau
2021-09-11 3:32 ` [PATCH 7/8] builtin/repack.c: make largest pack preferred Taylor Blau
2021-09-11 10:17 ` Ævar Arnfjörð Bjarmason
2021-09-11 16:35 ` Taylor Blau
2021-09-11 3:32 ` [PATCH 8/8] builtin/repack.c: pass `--refs-snapshot` when writing bitmaps Taylor Blau
2021-09-11 10:27 ` Ævar Arnfjörð Bjarmason
2021-09-11 11:19 ` Ævar Arnfjörð Bjarmason
2021-09-11 16:51 ` Taylor Blau
2021-09-14 18:55 ` Jeff King
2021-09-14 23:34 ` Taylor Blau
2021-09-14 23:56 ` Ævar Arnfjörð Bjarmason
2021-09-15 4:31 ` Taylor Blau
2021-09-11 16:49 ` Taylor Blau
2021-09-15 18:24 ` [PATCH v2 0/8] repack: introduce `--write-midx` Taylor Blau
2021-09-15 18:24 ` [PATCH v2 1/8] midx: expose `write_midx_file_only()` publicly Taylor Blau
2021-09-22 23:14 ` Jonathan Tan
2021-09-23 3:09 ` Taylor Blau
2021-09-15 18:24 ` [PATCH v2 2/8] builtin/multi-pack-index.c: support `--stdin-packs` mode Taylor Blau
2021-09-22 22:29 ` Josh Steadmon
2021-09-23 2:03 ` Taylor Blau
2021-09-22 23:11 ` Jonathan Tan
2021-09-23 2:06 ` Taylor Blau
2021-09-15 18:24 ` [PATCH v2 3/8] midx: preliminary support for `--refs-snapshot` Taylor Blau
2021-09-22 22:34 ` Josh Steadmon
2021-09-23 2:08 ` Taylor Blau
2021-09-22 23:00 ` Jonathan Tan
2021-09-23 2:18 ` Taylor Blau [this message]
2021-09-15 18:24 ` [PATCH v2 4/8] builtin/repack.c: keep track of existing packs unconditionally Taylor Blau
2021-09-22 22:56 ` Jonathan Tan
2021-09-23 2:59 ` Taylor Blau
2021-09-15 18:24 ` [PATCH v2 5/8] builtin/repack.c: extract showing progress to a variable Taylor Blau
2021-09-15 18:24 ` [PATCH v2 6/8] builtin/repack.c: support writing a MIDX while repacking Taylor Blau
2021-09-22 22:39 ` Jonathan Tan
2021-09-23 2:40 ` Taylor Blau
2021-09-15 18:24 ` [PATCH v2 7/8] builtin/repack.c: make largest pack preferred Taylor Blau
2021-09-15 18:24 ` [PATCH v2 8/8] builtin/repack.c: pass `--refs-snapshot` when writing bitmaps Taylor Blau
2021-09-24 18:22 ` Jonathan Tan
2021-10-01 22:38 ` Taylor Blau
2021-09-15 19:22 ` [PATCH v2 0/8] repack: introduce `--write-midx` Junio C Hamano
2021-09-15 19:29 ` Junio C Hamano
2021-09-15 21:19 ` Taylor Blau
2021-09-16 22:16 ` Junio C Hamano
2021-09-29 1:54 ` [PATCH v3 0/9] " Taylor Blau
2021-09-29 1:55 ` [PATCH v3 1/9] midx: expose `write_midx_file_only()` publicly Taylor Blau
2021-09-29 1:55 ` [PATCH v3 2/9] builtin/multi-pack-index.c: support `--stdin-packs` mode Taylor Blau
2021-09-29 1:55 ` [PATCH v3 3/9] midx: preliminary support for `--refs-snapshot` Taylor Blau
2021-09-29 1:55 ` [PATCH v3 4/9] builtin/repack.c: keep track of existing packs unconditionally Taylor Blau
2021-09-29 1:55 ` [PATCH v3 5/9] builtin/repack.c: rename variables that deal with non-kept packs Taylor Blau
2021-09-29 1:55 ` [PATCH v3 6/9] builtin/repack.c: extract showing progress to a variable Taylor Blau
2021-09-29 1:55 ` [PATCH v3 7/9] builtin/repack.c: support writing a MIDX while repacking Taylor Blau
2021-09-29 1:55 ` [PATCH v3 8/9] builtin/repack.c: make largest pack preferred Taylor Blau
2021-09-29 1:55 ` [PATCH v3 9/9] builtin/repack.c: pass `--refs-snapshot` when writing bitmaps Taylor Blau
2021-09-29 4:24 ` [PATCH v3 0/9] repack: introduce `--write-midx` Junio C Hamano
2021-10-01 20:01 ` Jonathan Tan
2021-10-01 22:40 ` Taylor Blau
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YUvj33QX5fvjNoiJ@nand.local \
--to=me@ttaylorr.com \
--cc=avarab@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=jonathantanmy@google.com \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).