All of lore.kernel.org
 help / color / mirror / Atom feed
From: Glen Choo <chooglen@google.com>
To: git@vger.kernel.org
Cc: Glen Choo <chooglen@google.com>,
	Jonathan Tan <jonathantanmy@google.com>,
	Junio C Hamano <gitster@pobox.com>
Subject: [PATCH v3 0/3] builtin/fetch: skip unnecessary tasks when using --negotiate-only
Date: Tue, 21 Dec 2021 16:11:31 -0800	[thread overview]
Message-ID: <20211222001134.28933-1-chooglen@google.com> (raw)
In-Reply-To: <20211217000235.68996-1-chooglen@google.com>

cmd_fetch() performs certain tasks with the assumption that objects are
fetched, but `git fetch --negotiate-only` does not fetch objects, as its
name implies. This is results in behavior that is unnecessary at best,
and incorrect at worst:

* Submodules are updated if enabled by recurse.submodules=true, but
  negotiation fetch doesn't actually update the repo, so this doesn't
  make sense (introduced in [1]).
* Commit graphs will be written if enabled by
  fetch.writeCommitGraph=true. But according to
  Documentation/config/fetch.txt [2], this should only be done if a
  pack-file is downloaded.
* gc is run, but according to [3], we only do this because we expect
  `git fetch` to introduce objects.

Make `git fetch --negotiate-only` handle these tasks more rigorously by
doing the following:

* Make cmd_fetch() return early if we know for certain that objects will
  not be fetched
* Disable submodule recursion and die() if a user explicitly asks for it

[1] 7dce19d374 (fetch/pull: Add the --recurse-submodules option, 2010-11-12)
[2] 50f26bd035 (fetch: add fetch.writeCommitGraph config setting, 2019-09-02)
[3] 131b8fcbfb (fetch: run gc --auto after fetching, 2013-01-26)

Changes since v2:
* added a prepatory patch that introduces a "goto cleanup"
* drop an unnecessary line move (as suggested by Junio)
* check for user-given --recurse-submodules and die() (as suggested by
  Jonathan and Junio)
* update --negotiate-only's documentation 

Changes since v1:
* added more context to commit message
* added a NEEDSWORK comment 

Glen Choo (3):
  builtin/fetch: use goto cleanup in cmd_fetch()
  builtin/fetch: skip unnecessary tasks when using --negotiate-only
  builtin/fetch: die on --negotiate-only and --recurse-submodules

 Documentation/fetch-options.txt |  1 +
 builtin/fetch.c                 | 41 +++++++++++++++++++++++++++++----
 t/t5516-fetch-push.sh           | 12 ++++++++++
 t/t5702-protocol-v2.sh          | 17 ++++++++++++++
 4 files changed, 67 insertions(+), 4 deletions(-)

Range-diff against v2:
-:  ---------- > 1:  97e2cba633 builtin/fetch: use goto cleanup in cmd_fetch()
1:  9d18270a41 ! 2:  3a3a04b649 builtin/fetch: skip unnecessary tasks when using --negotiate-only
    @@ Commit message
         return early whenever objects are not fetched, but for now this only
         considers --negotiate-only.
     
    +    A similar optimization would be to skip irrelevant tasks in `git fetch
    +    --dry-run`. This optimization was not done in this commit because a dry
    +    run will actually fetch objects; we'd presumably still want to recurse
    +    into submodules and run gc.
    +
         [1] 7dce19d374 (fetch/pull: Add the --recurse-submodules option, 2010-11-12)
         [2] 50f26bd035 (fetch: add fetch.writeCommitGraph config setting, 2019-09-02)
         [3] 131b8fcbfb (fetch: run gc --auto after fetching, 2013-01-26)
    @@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
     +		 * submodules, so there is no need to read .gitmodules.
     +		 */
     +		recurse_submodules = RECURSE_SUBMODULES_OFF;
    -+		if (!negotiation_tip.nr)
    -+			die(_("--negotiate-only needs one or more --negotiate-tip=*"));
     +	}
     +
      	if (recurse_submodules != RECURSE_SUBMODULES_OFF) {
      		int *sfjc = submodule_fetch_jobs_config == -1
      			    ? &submodule_fetch_jobs_config : NULL;
    -@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
    - 		fetch_config_from_gitmodules(sfjc, rs);
    - 	}
    - 
    --	if (negotiate_only && !negotiation_tip.nr)
    --		die(_("--negotiate-only needs one or more --negotiate-tip=*"));
    --
    - 	if (deepen_relative) {
    - 		if (deepen_relative < 0)
    - 			die(_("Negative depth in --deepen is not supported"));
     @@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
      		result = fetch_multiple(&list, max_children);
      	}
      
    -+	string_list_clear(&list, 0);
    -+
     +	/*
     +	 * Skip irrelevant tasks because we know objects were not
     +	 * fetched.
    @@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
     +	 * of them.
     +	 */
     +	if (negotiate_only)
    -+		return result;
    ++		goto cleanup;
     +
      	if (!result && (recurse_submodules != RECURSE_SUBMODULES_OFF)) {
      		struct strvec options = STRVEC_INIT;
      		int max_children = max_jobs;
    -@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
    - 		strvec_clear(&options);
    - 	}
    - 
    --	string_list_clear(&list, 0);
    --
    - 	prepare_repo_settings(the_repository);
    - 	if (fetch_write_commit_graph > 0 ||
    - 	    (fetch_write_commit_graph < 0 &&
     
      ## t/t5516-fetch-push.sh ##
     @@ t/t5516-fetch-push.sh: test_expect_success 'push with negotiation proceeds anyway even if negotiation f
-:  ---------- > 3:  97792b5214 builtin/fetch: die on --negotiate-only and --recurse-submodules

base-commit: abe6bb3905392d5eb6b01fa6e54d7e784e0522aa
-- 
2.33.GIT


  parent reply	other threads:[~2021-12-22  0:11 UTC|newest]

Thread overview: 50+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-12-07 19:29 [PATCH] builtin/fetch: skip unnecessary tasks when using --negotiate-only Glen Choo
2021-12-09 22:12 ` Jonathan Tan
2021-12-09 22:36   ` Glen Choo
2021-12-13 22:58     ` Jonathan Tan
2021-12-16 18:11       ` Glen Choo
2021-12-17  0:02 ` [PATCH v2] " Glen Choo
2021-12-17 23:35   ` Junio C Hamano
2021-12-20 19:37     ` Glen Choo
2021-12-20 19:56       ` Junio C Hamano
2021-12-20 20:54         ` Glen Choo
2021-12-20 22:12           ` Junio C Hamano
2021-12-21  0:18             ` Glen Choo
2021-12-21 23:07       ` Glen Choo
2021-12-22  0:11   ` Glen Choo [this message]
2021-12-22  0:11     ` [PATCH v3 1/3] builtin/fetch: use goto cleanup in cmd_fetch() Glen Choo
2021-12-22  0:11     ` [PATCH v3 2/3] builtin/fetch: skip unnecessary tasks when using --negotiate-only Glen Choo
2021-12-22  6:42       ` Junio C Hamano
2021-12-22 17:28         ` Glen Choo
2021-12-22 19:29           ` Junio C Hamano
2021-12-22 20:27             ` Glen Choo
2021-12-22  0:11     ` [PATCH v3 3/3] builtin/fetch: die on --negotiate-only and --recurse-submodules Glen Choo
2021-12-22  6:46       ` Junio C Hamano
2021-12-23 19:08       ` Jonathan Tan
2022-01-13  0:44     ` [PATCH v4 0/3] fetch: skip unnecessary tasks when using --negotiate-only Glen Choo
2022-01-13  0:44       ` [PATCH v4 1/3] fetch: use goto cleanup in cmd_fetch() Glen Choo
2022-01-13  0:45       ` [PATCH v4 2/3] fetch: skip tasks related to fetching objects Glen Choo
2022-01-13  0:45       ` [PATCH v4 3/3] fetch --negotiate-only: do not update submodules Glen Choo
2022-01-13  1:16         ` Junio C Hamano
2022-01-18 18:54       ` [PATCH v5 0/3] fetch: skip unnecessary tasks when using --negotiate-only Glen Choo
2022-01-18 18:54         ` [PATCH v5 1/3] fetch: use goto cleanup in cmd_fetch() Glen Choo
2022-01-18 18:54         ` [PATCH v5 2/3] fetch: skip tasks related to fetching objects Glen Choo
2022-01-18 18:54         ` [PATCH v5 3/3] fetch --negotiate-only: do not update submodules Glen Choo
2022-01-18 22:05           ` Junio C Hamano
2022-01-18 23:41             ` Glen Choo
2022-01-19  0:26               ` Junio C Hamano
2022-01-19  0:00         ` [PATCH v6 0/3] fetch: skip unnecessary tasks when using --negotiate-only Glen Choo
2022-01-19  0:00           ` [PATCH v6 1/3] fetch: use goto cleanup in cmd_fetch() Glen Choo
2022-01-19  0:00           ` [PATCH v6 2/3] fetch: skip tasks related to fetching objects Glen Choo
2022-01-19  0:00           ` [PATCH v6 3/3] fetch --negotiate-only: do not update submodules Glen Choo
2022-01-20  2:38             ` Jiang Xin
2022-01-20 17:40               ` Glen Choo
2022-01-20 17:49           ` [PATCH v7 0/3] fetch: skip unnecessary tasks when using --negotiate-only Glen Choo
2022-01-20 17:49             ` [PATCH v7 1/3] fetch: use goto cleanup in cmd_fetch() Glen Choo
2022-01-20 17:49             ` [PATCH v7 2/3] fetch: skip tasks related to fetching objects Glen Choo
2022-01-20 17:49             ` [PATCH v7 3/3] fetch --negotiate-only: do not update submodules Glen Choo
2022-01-20 23:08               ` Junio C Hamano
2022-01-20 23:16                 ` Glen Choo
2022-01-20 21:58             ` Re* [PATCH v7 0/3] fetch: skip unnecessary tasks when using --negotiate-only Junio C Hamano
2022-01-20 23:15               ` Glen Choo
2022-01-21  2:17               ` Jiang Xin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20211222001134.28933-1-chooglen@google.com \
    --to=chooglen@google.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=jonathantanmy@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.