From: "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: johannes.schindelin@gmx.de, peff@peff.net, szeder.dev@gmail.com,
Derrick Stolee <dstolee@microsoft.com>,
Junio C Hamano <gitster@pobox.com>
Subject: [PATCH v2 0/2] [v2.24.0-rc0 BUG] fetch.writeCommitGraph fails on first fetch
Date: Wed, 23 Oct 2019 13:01:33 +0000 [thread overview]
Message-ID: <pull.415.v2.git.1571835695.gitgitgadget@gmail.com> (raw)
In-Reply-To: <pull.415.git.1571765335.gitgitgadget@gmail.com>
UPDATE for V2: We now know the full repro, and a test is added. Thanks
Szeder and Peff for your insights!
While dogfooding, Johannes found a bug in the fetch.writeCommitGraph config
behavior. While his example initially happened during a clone with
--recurse-submodules, (UPDATE) and the submodule is important, but
--recurse-submodules is not:
$ git clone <url> test
$ cd test
$ git -c fetch.writeCommitGraph=true fetch origin
Computing commit graph generation numbers: 100% (12/12), done.
BUG: commit-graph.c:886: missing parent <hash1> for commit <hash2>
Aborted (core dumped)
In the repo I had cloned, there were really 60 commits to scan, but only 12
were in the list to write when calling compute_generation_numbers(). A
commit in the list expects to see a parent, but that parent is not in the
list. The simple example I used for my testing was
https://github.com/derrickstolee/numbers. Thie repo HAS A SUBMODULE, I just
forgot. Sorry for derailing the investigation somewhat.
The details above are the start of the commit message for [PATCH 1/2],
including a test that fails when fetching after cloning a repo with a
submodule.
In [PATCH 2/2], I actually have the fix. I tried to include as much detail
as I could for how I investigated the problem and why I think this is the
right solution. I added details that have come from the on-list discussion,
including what the submodule code is doing and why REACHABLE is no longer
used in commit-reach.c.
Thanks, -Stolee
Derrick Stolee (2):
t5510-fetch.sh: demonstrate fetch.writeCommitGraph bug
commit-graph: fix writing first commit-graph during fetch
commit-graph.c | 11 +++++++----
commit-reach.c | 1 -
object.h | 3 ++-
t/t5510-fetch.sh | 17 +++++++++++++++++
4 files changed, 26 insertions(+), 6 deletions(-)
base-commit: d966095db01190a2196e31195ea6fa0c722aa732
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-415%2Fderrickstolee%2Ffetch-first-write-fail-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-415/derrickstolee/fetch-first-write-fail-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/415
Range-diff vs v1:
-: ---------- > 1: 6ac0a05746 t5510-fetch.sh: demonstrate fetch.writeCommitGraph bug
1: a1e5280d4b ! 2: ca59b118f1 commit-graph: fix writing first commit-graph during fetch
@@ -2,10 +2,13 @@
commit-graph: fix writing first commit-graph during fetch
- While dogfooding, Johannes found a bug in the fetch.writeCommitGraph
- config behavior. While his example initially happened during a clone
- with --recurse-submodules, we found that it is actually a problem with
- the first fetch after a new clone and no existing commit-graph file:
+ The previous commit includes a failing test for an issue around
+ fetch.writeCommitGraph and fetching in a repo with a submodule. Here, we
+ fix that bug and set the test to "test_expect_success".
+
+ The prolem arises with this set of commands when the remote repo at
+ <url> has a submodule. Note that --recurse-submodules is not needed to
+ demonstrate the bug.
$ git clone <url> test
$ cd test
@@ -14,12 +17,7 @@
BUG: commit-graph.c:886: missing parent <hash1> for commit <hash2>
Aborted (core dumped)
- In the repo I had cloned, there were really 60 commits to scan, but
- only 12 were in the list to write when calling
- compute_generation_numbers(). A commit in the list expects to see a
- parent, but that parent is not in the list.
-
- As an initial test, I converted the code in builtin/fetch.c that calls
+ As an initial fix, I converted the code in builtin/fetch.c that calls
write_commit_graph_reachable() to instead launch a "git commit-graph
write --reachable --split" process. That code worked, but is not how we
want the feature to work long-term.
@@ -60,20 +58,28 @@
flag inside close_reachable(), but the tips did not have the flag, so
that did nothing.
+ It turns out that the calculate_changed_submodule_paths() method is at
+ fault. Thanks, Peff, for pointing out this detail! More specifically,
+ for each submodule, the collect_changed_submodules() runs a revision
+ walk to essentially do file-history on the list of submodules. That
+ revision walk marks commits UNININTERESTING if they are simiplified away
+ by not changing the submodule.
+
Instead, I finally arrived on the conclusion that I should use a flag
that is not used in any other part of the code. In commit-reach.c, a
number of flags were defined for commit walk algorithms. The REACHABLE
flag seemed like it made the most sense, and it seems it was not
- actually used in the file.
+ actually used in the file. The REACHABLE flag was used in early versions
+ of commit-reach.c, but was removed by 4fbcca4 (commit-reach: make
+ can_all_from_reach... linear, 2018-07-20).
Add the REACHABLE flag to commit-graph.c and use it instead of
UNINTERESTING in close_reachable(). This fixes the bug in manual
testing.
- I have failed to produce a test using the file:// protocol that
- demonstrates this bug.
-
Reported-by: Johannes Schindelin <johannes.schindelin@gmx.de>
+ Helped-by: Jeff King <peff@peff.net>
+ Helped-by: Szeder Gábor <szeder.dev@gmail.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
diff --git a/commit-graph.c b/commit-graph.c
@@ -147,3 +153,16 @@
* sha1-name.c: 20
* list-objects-filter.c: 21
* builtin/fsck.c: 0--3
+
+ diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
+ --- a/t/t5510-fetch.sh
+ +++ b/t/t5510-fetch.sh
+@@
+ )
+ '
+
+-test_expect_failure 'fetch.writeCommitGraph with submodules' '
++test_expect_success 'fetch.writeCommitGraph with submodules' '
+ pwd="$(pwd)" &&
+ git clone dups super &&
+ (
--
gitgitgadget
next prev parent reply other threads:[~2019-10-23 13:01 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-22 17:28 [PATCH 0/1] [v2.24.0-rc0 BUG] fetch.writeCommitGraph fails on first fetch Derrick Stolee via GitGitGadget
2019-10-22 17:28 ` [PATCH 1/1] commit-graph: fix writing first commit-graph during fetch Derrick Stolee via GitGitGadget
2019-10-22 20:33 ` Jeff King
2019-10-22 21:45 ` Jeff King
2019-10-22 23:35 ` SZEDER Gábor
2019-10-23 0:35 ` Derrick Stolee
2019-10-23 0:48 ` Jeff King
2019-10-23 1:22 ` Jeff King
2019-10-23 13:01 ` Derrick Stolee via GitGitGadget [this message]
2019-10-23 13:01 ` [PATCH v2 1/2] t5510-fetch.sh: demonstrate fetch.writeCommitGraph bug Derrick Stolee via GitGitGadget
2019-10-23 14:18 ` SZEDER Gábor
2019-10-23 20:46 ` Derrick Stolee
2019-10-24 12:18 ` SZEDER Gábor
2019-10-23 13:01 ` [PATCH v2 2/2] commit-graph: fix writing first commit-graph during fetch Derrick Stolee via GitGitGadget
2019-10-23 15:04 ` SZEDER Gábor
2019-10-24 10:39 ` Derrick Stolee
2019-10-30 14:31 ` SZEDER Gábor
2019-10-24 12:18 ` [PATCH v3 0/2] [v2.24.0-rc0 BUG] fetch.writeCommitGraph fails on first fetch Derrick Stolee via GitGitGadget
2019-10-24 12:18 ` [PATCH v3 1/2] t5510-fetch.sh: demonstrate fetch.writeCommitGraph bug Derrick Stolee via GitGitGadget
2019-10-24 12:18 ` [PATCH v3 2/2] commit-graph: fix writing first commit-graph during fetch Derrick Stolee via GitGitGadget
2019-10-24 13:40 ` [PATCH v4 0/2] [v2.24.0-rc0 BUG] fetch.writeCommitGraph fails on first fetch Derrick Stolee via GitGitGadget
2019-10-24 13:40 ` [PATCH v4 1/2] t5510-fetch.sh: demonstrate fetch.writeCommitGraph bug Derrick Stolee via GitGitGadget
2019-10-24 13:40 ` [PATCH v4 2/2] commit-graph: fix writing first commit-graph during fetch Derrick Stolee via GitGitGadget
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=pull.415.v2.git.1571835695.gitgitgadget@gmail.com \
--to=gitgitgadget@gmail.com \
--cc=dstolee@microsoft.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=johannes.schindelin@gmx.de \
--cc=peff@peff.net \
--cc=szeder.dev@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).