[PATCH v4 0/4] Add --base option to git-format-patch to record base tree info

[PATCH v4 0/4] Add --base option to git-format-patch to record base tree info

[Xiaolong Ye]

V4 mainly addresses Junio's comments on V3, Changes include:

 - Polish up the documentation to make output files git-format-patch.1 and
   git-format-patch.html more sensible.

 - Add error handling when base commit is not ancestor of revision list
   specified in cmdline.

 - Specify topo order to do the traverse work, and show the base tree info
   block in a more natural sequence.
 
 - If --base=auto is set and there is more than one best merge base, instead
   of picking up a random one, it will error out for they may be complicated
   situation such as criss-cross merges.

 - Add tests for the --base option and format.base configuration.

 - Fix a segfault error due to bases structure hasn't been initialized when
   --base option is not set, Thanks for Ramsay's report.

Xiaolong Ye (4):
  patch-ids: make commit_patch_id() a public helper function
  format-patch: add '--base' option to record base tree info
  format-patch: introduce --base=auto option
  format-patch: introduce format.base configuration

 Documentation/git-format-patch.txt |  64 +++++++++++++++++++
 builtin/log.c                      | 123 +++++++++++++++++++++++++++++++++++++
 patch-ids.c                        |   2 +-
 patch-ids.h                        |   2 +
 t/t4014-format-patch.sh            |  48 +++++++++++++++
 5 files changed, 238 insertions(+), 1 deletion(-)

-- 
2.8.1.120.g24d6b3f

base-commit: 7b0d47b3b6b5b64e02a5aa06b0452cadcdb18355

[PATCH v4 1/4] patch-ids: make commit_patch_id() a public helper function

[Xiaolong Ye]

Make commit_patch_id() available to other builtins.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com>
---
 patch-ids.c | 2 +-
 patch-ids.h | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/patch-ids.c b/patch-ids.c
index b7b3e5a..a4d0016 100644
--- a/patch-ids.c
+++ b/patch-ids.c
@@ -4,7 +4,7 @@
 #include "sha1-lookup.h"
 #include "patch-ids.h"
 
-static int commit_patch_id(struct commit *commit, struct diff_options *options,
+int commit_patch_id(struct commit *commit, struct diff_options *options,
 		    unsigned char *sha1)
 {
 	if (commit->parents)
diff --git a/patch-ids.h b/patch-ids.h
index c8c7ca1..eeb56b3 100644
--- a/patch-ids.h
+++ b/patch-ids.h
@@ -13,6 +13,8 @@ struct patch_ids {
 	struct patch_id_bucket *patches;
 };
 
+int commit_patch_id(struct commit *commit, struct diff_options *options,
+		    unsigned char *sha1);
 int init_patch_ids(struct patch_ids *);
 int free_patch_ids(struct patch_ids *);
 struct patch_id *add_commit_patch_id(struct commit *, struct patch_ids *);
-- 
2.8.1.120.g24d6b3f

[PATCH v4 2/4] format-patch: add '--base' option to record base tree info

[Xiaolong Ye]

Maintainers or third party testers may want to know the exact base tree
the patch series applies to. Teach git format-patch a '--base' option
to record the base tree info and append it at the end of the_first_
message(either the cover letter or the first patch in the series).

The base tree info consists of the "base commit", which is a well-known
commit that is part of the stable part of the project history everybody
else works off of, and zero or more "prerequisite patches", which are
well-known patches in flight that is not yet part of the "base commit"
that need to be applied on top of "base commit" in topological order
before the patches can be applied.

The "base commit" is shown as "base-commit: " followed by the 40-hex of
the commit object name.  A "prerequisite patch" is shown as
"prerequisite-patch-id: " followed by the 40-hex "patch id", which is a
sum of SHA-1 of the file diffs associated with a patch, with whitespace
and line numbers ignored, it's reasonably stable and unique.

For example, we have history where base is Z, with three prerequisites
X-Y-Z, before the patch series A-B-C, i.e.

	P---X---Y---Z---A---B---C

We could say "git format-patch --base=P -3 C"(or variants thereof, e.g.
with "--cover-letter" of using "Z..C" instead of "-3 C" to specify the
range), then we could get base tree information block showing at the
end of _first_ message as below:

	base-commit: P
	prerequisite-patch-id: X
	prerequisite-patch-id: Y
	prerequisite-patch-id: Z

Helped-by: Junio C Hamano <gitster@pobox.com>
Helped-by: Wu Fengguang <fengguang.wu@intel.com>
Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com>
---
 Documentation/git-format-patch.txt | 56 +++++++++++++++++++++++
 builtin/log.c                      | 92 ++++++++++++++++++++++++++++++++++++++
 t/t4014-format-patch.sh            | 15 +++++++
 3 files changed, 163 insertions(+)

diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt
index 6821441..2a4c293 100644
--- a/Documentation/git-format-patch.txt
+++ b/Documentation/git-format-patch.txt
@@ -265,6 +265,11 @@ you can use `--suffix=-patch` to get `0001-description-of-my-change-patch`.
   Output an all-zero hash in each patch's From header instead
   of the hash of the commit.
 
+--base=<commit>::
+	Record the base tree information to identify the state the
+	patch series applies to.  See the BASE TREE INFORMATION section
+	below for details.
+
 --root::
 	Treat the revision argument as a <revision range>, even if it
 	is just a single commit (that would normally be treated as a
@@ -520,6 +525,57 @@ This should help you to submit patches inline using KMail.
 5. Back in the compose window: add whatever other text you wish to the
    message, complete the addressing and subject fields, and press send.
 
+BASE TREE INFORMATION
+---------------------
+
+The base tree information block is used for maintainers or third party
+testers to know the exact state the patch series applies to. It consists
+of the 'base commit', which is a well-known commit that is part of the
+stable part of the project history everybody else works off of, and zero
+or more 'prerequisite patches', which are well-known patches in flight
+that is not yet part of the 'base commit' that need to be applied on top
+of 'base commit' in topological order before the patches can be applied.
+
+The 'base commit' is shown as "base-commit: " followed by the 40-hex of
+the commit object name.  A 'prerequisite patch' is shown as
+"prerequisite-patch-id: " followed by the 40-hex 'patch id', which is a
+sum of SHA-1 of the file diffs associated with a patch, with whitespace
+and line numbers ignored, it's reasonably stable and unique.
+
+For example, the patch submitter has a commit history of this shape:
+
+................................................
+---P---X---Y---Z---A---B---C
+................................................
+
+where 'P' is the well-known public commit (e.g. one in Linus's tree),
+'X', 'Y', 'Z' are prerequisite patches in flight, and 'A', 'B', 'C'
+are the work being sent out, the submitter could say `git format-patch
+--base=P -3 C` (or variants thereof, e.g. with `--cover` or using
+`Z..C` instead of `-3 C` to specify the range), and the identifiers
+for P, X, Y, Z are appended at the end of the _first_ message (either
+the cover letter or the first patch in the series). Then we could get
+base tree information block showing at the end of _first_ message as
+below:
+
+------------
+base-commit: P
+prerequisite-patch-id: X
+prerequisite-patch-id: Y
+prerequisite-patch-id: Z
+------------
+
+For non-linear topology, such as
+
+................................................
+---P---X---A---M---C
+    \         /
+     Y---Z---B
+................................................
+
+The submitter could also use `git format-patch --base=P -3 C` to generate
+patches for A, B and C, and the identifiers for P, X, Y, Z are appended
+at the end of the _first_ message.
 
 EXAMPLES
 --------
diff --git a/builtin/log.c b/builtin/log.c
index 9430b80..73bc36d 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -1191,6 +1191,85 @@ static int from_callback(const struct option *opt, const char *arg, int unset)
 	return 0;
 }
 
+struct base_tree_info {
+	struct object_id base_commit;
+	int nr_patch_id, alloc_patch_id;
+	struct object_id *patch_id;
+};
+
+static void prepare_bases(struct base_tree_info *bases,
+			  const char *base_commit,
+			  struct commit **list,
+			  int total)
+{
+	struct commit *base = NULL, *commit;
+	struct rev_info revs;
+	struct diff_options diffopt;
+	unsigned char sha1[20];
+	int i;
+
+	diff_setup(&diffopt);
+	DIFF_OPT_SET(&diffopt, RECURSIVE);
+	diff_setup_done(&diffopt);
+
+	base = lookup_commit_reference_by_name(base_commit);
+	if (!base)
+		die(_("Unknown commit %s"), base_commit);
+	oidcpy(&bases->base_commit, &base->object.oid);
+
+	init_revisions(&revs, NULL);
+	revs.max_parents = 1;
+	revs.topo_order = 1;
+	for (i = 0; i < total; i++) {
+		if (!in_merge_bases(base, list[i]) || base == list[i])
+			die(_("base commit should be the ancestor of revision list"));
+		list[i]->object.flags &= ~UNINTERESTING;
+		add_pending_object(&revs, &list[i]->object, "rev_list");
+		list[i]->util = (void *)1;
+	}
+	base->object.flags |= UNINTERESTING;
+	add_pending_object(&revs, &base->object, "base");
+
+	if (prepare_revision_walk(&revs))
+		die(_("revision walk setup failed"));
+	/*
+	 * Traverse the prerequisite commits list,
+	 * get the patch ids and stuff them in bases structure.
+	 */
+	while ((commit = get_revision(&revs)) != NULL) {
+		struct object_id *patch_id;
+		if (commit->util)
+			continue;
+		if (commit_patch_id(commit, &diffopt, sha1))
+			die(_("cannot get patch id"));
+		ALLOC_GROW(bases->patch_id, bases->nr_patch_id + 1, bases->alloc_patch_id);
+		patch_id = bases->patch_id + bases->nr_patch_id;
+		hashcpy(patch_id->hash, sha1);
+		bases->nr_patch_id++;
+	}
+}
+
+static void print_bases(struct base_tree_info *bases)
+{
+	int i;
+
+	/* Only do this once, either for the cover or for the first one */
+	if (is_null_oid(&bases->base_commit))
+		return;
+
+	/* Show the base commit */
+	printf("base-commit: %s\n", oid_to_hex(&bases->base_commit));
+
+	/* Show the prerequisite patches */
+	for (i = bases->nr_patch_id - 1; i >= 0; i--)
+		printf("prerequisite-patch-id: %s\n", oid_to_hex(&bases->patch_id[i]));
+
+	free(bases->patch_id);
+	bases->nr_patch_id = 0;
+	bases->alloc_patch_id = 0;
+	oidclr(&bases->base_commit);
+}
+
 int cmd_format_patch(int argc, const char **argv, const char *prefix)
 {
 	struct commit *commit;
@@ -1215,6 +1294,9 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	int reroll_count = -1;
 	char *branch_name = NULL;
 	char *from = NULL;
+	char *base_commit = NULL;
+	struct base_tree_info bases;
+
 	const struct option builtin_format_patch_options[] = {
 		{ OPTION_CALLBACK, 'n', "numbered", &numbered, NULL,
 			    N_("use [PATCH n/m] even with a single patch"),
@@ -1277,6 +1359,8 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 			    PARSE_OPT_OPTARG, thread_callback },
 		OPT_STRING(0, "signature", &signature, N_("signature"),
 			    N_("add a signature")),
+		OPT_STRING(0, "base", &base_commit, N_("base-commit"),
+			   N_("add prerequisite tree info to the patch series")),
 		OPT_FILENAME(0, "signature-file", &signature_file,
 				N_("add a signature from a file")),
 		OPT__QUIET(&quiet, N_("don't print the patch filenames")),
@@ -1513,6 +1597,12 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 		signature = strbuf_detach(&buf, NULL);
 	}
 
+	memset(&bases, 0, sizeof(bases));
+	if (base_commit) {
+		reset_revision_walk();
+		prepare_bases(&bases, base_commit, list, nr);
+	}
+
 	if (in_reply_to || thread || cover_letter)
 		rev.ref_message_ids = xcalloc(1, sizeof(struct string_list));
 	if (in_reply_to) {
@@ -1526,6 +1616,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 			gen_message_id(&rev, "cover");
 		make_cover_letter(&rev, use_stdout,
 				  origin, nr, list, branch_name, quiet);
+		print_bases(&bases);
 		total++;
 		start_number--;
 	}
@@ -1591,6 +1682,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 				       rev.mime_boundary);
 			else
 				print_signature();
+			print_bases(&bases);
 		}
 		if (!use_stdout)
 			fclose(stdout);
diff --git a/t/t4014-format-patch.sh b/t/t4014-format-patch.sh
index eed2981..a6ce727 100755
--- a/t/t4014-format-patch.sh
+++ b/t/t4014-format-patch.sh
@@ -1460,4 +1460,19 @@ test_expect_success 'format-patch -o overrides format.outputDirectory' '
 	test_path_is_dir patchset
 '
 
+test_expect_success 'format-patch --base' '
+	git checkout side &&
+	git format-patch --stdout --base=HEAD~~~ -1 >patch &&
+	grep -e "^base-commit:" -A3 patch >actual &&
+	echo "base-commit: $(git rev-parse HEAD~~~)" >expected &&
+	echo "prerequisite-patch-id: $(git show --patch HEAD~~ | git patch-id --stable | awk "{print \$1}")" >>expected &&
+	echo "prerequisite-patch-id: $(git show --patch HEAD~ | git patch-id --stable | awk "{print \$1}")" >>expected &&
+	test_cmp expected actual
+'
+
+test_expect_success 'format-patch --base error handling' '
+	! git format-patch --base=HEAD~ -2 &&
+	! git format-patch --base=HEAD~ -3
+'
+
 test_done
-- 
2.8.1.120.g24d6b3f

Re: [PATCH v4 2/4] format-patch: add '--base' option to record base tree info

[Junio C Hamano]

Xiaolong Ye <xiaolong.ye@intel.com> writes:

> Maintainers or third party testers may want to know the exact base tree
> the patch series applies to. Teach git format-patch a '--base' option
> to record the base tree info and append it at the end of the_first_

It probably was a good idea to add stress during the discussion to
compare various possibilities, but there no longer is a need to
italicise "first" like this, I think.

> message(either the cover letter or the first patch in the series).

Please have space before "(" (also found elsewhere in this message)
to make this readable.

>
> The base tree info consists of the "base commit", which is a well-known
> commit that is part of the stable part of the project history everybody
> else works off of, and zero or more "prerequisite patches", which are
> well-known patches in flight that is not yet part of the "base commit"
> that need to be applied on top of "base commit" in topological order
> before the patches can be applied.
>
> The "base commit" is shown as "base-commit: " followed by the 40-hex of
> the commit object name.  A "prerequisite patch" is shown as
> "prerequisite-patch-id: " followed by the 40-hex "patch id", which is a
> sum of SHA-1 of the file diffs associated with a patch, with whitespace
> and line numbers ignored, it's reasonably stable and unique.

Let's be more helpful to end users.  They do not need to know the
exact formula, especially when there is a command to generate or
check the id for themselves:

    "patch id", which can be obtained by passing the patch through the
    "git patch-id --stable" command

or something?  

> For example, we have history where base is Z, with three prerequisites
> X-Y-Z, before the patch series A-B-C, i.e.

base is Z???

	Imagine that on top of the public commit P, you applied
	well-known patches X, Y and Z from somebody else, and then
	built your three-patch series A, B, C.

perhaps?

>
> 	P---X---Y---Z---A---B---C
>
> We could say "git format-patch --base=P -3 C"(or variants thereof, e.g.
> with "--cover-letter" of using "Z..C" instead of "-3 C" to specify the
> range), then we could get base tree information block showing at the
> end of _first_ message as below:

Again, if "first" is _so_ important to stress, it probably is worth
saying that by "first" you mean either patch 1/n or patch 0/n when
the cover letter exists.

Also "could" may have made sense while we were having discussion on
possible design of the hypothetical feature, but with the patch
applied, the feature becomes a reality, so you can and should stop
living in the hypothetical world and do s/could/can/ the above.

	With "git format-patch --base=P -3 C" (or variants...), the
	base tree information block is shown at the end of the first
	message the command outputs (either the first patch, or the
	cover letter), like this:

perhaps?

I assume that the patch to the documentation has the same text I
commented on the above, so I won't repeat my comments to them.

> 	base-commit: P
> 	prerequisite-patch-id: X
> 	prerequisite-patch-id: Y
> 	prerequisite-patch-id: Z
>
> Helped-by: Junio C Hamano <gitster@pobox.com>
> Helped-by: Wu Fengguang <fengguang.wu@intel.com>
> Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com>
> ---
>  Documentation/git-format-patch.txt | 56 +++++++++++++++++++++++
>  builtin/log.c                      | 92 ++++++++++++++++++++++++++++++++++++++
>  t/t4014-format-patch.sh            | 15 +++++++
>  3 files changed, 163 insertions(+)
> ...
> +static void prepare_bases(struct base_tree_info *bases,
> +			  const char *base_commit,
> +			  struct commit **list,
> +			  int total)
> +{
> +	struct commit *base = NULL, *commit;
> +	struct rev_info revs;
> +	struct diff_options diffopt;
> +	unsigned char sha1[20];
> +	int i;
> +
> +	diff_setup(&diffopt);
> +	DIFF_OPT_SET(&diffopt, RECURSIVE);
> +	diff_setup_done(&diffopt);
> +
> +	base = lookup_commit_reference_by_name(base_commit);
> +	if (!base)
> +		die(_("Unknown commit %s"), base_commit);
> +	oidcpy(&bases->base_commit, &base->object.oid);
> +
> +	init_revisions(&revs, NULL);
> +	revs.max_parents = 1;
> +	revs.topo_order = 1;
> +	for (i = 0; i < total; i++) {
> +		if (!in_merge_bases(base, list[i]) || base == list[i])
> +			die(_("base commit should be the ancestor of revision list"));

This check looks overly expensive, but I do not think of a more
efficient way to do this, given that "All the commits from our
series must reach the specified base" is what you seem to want.

My understanding is that if base=P is given and you are doing
"format-patch Z..C" in this picture:

    Q---P---Z---B---*---C
     \             /
      .-----------A

your list would become A, B and C, and you want to detect that P is
not an ancestor of A.  merge_bases_many() computes a wrong thing for
this use case, and you'd need to go one-by-one.

Unless there is some clever trick to take advantage of the previous
traversal you made in order to find out A, B and C are the commits
that are part of your series somehow.

Anybody with clever ideas?

Re: [PATCH v4 2/4] format-patch: add '--base' option to record base tree info

[Ye Xiaolong]

On Tue, Apr 12, 2016 at 12:08:33PM -0700, Junio C Hamano wrote:
>Xiaolong Ye <xiaolong.ye@intel.com> writes:
>
>> Maintainers or third party testers may want to know the exact base tree
>> the patch series applies to. Teach git format-patch a '--base' option
>> to record the base tree info and append it at the end of the_first_
>
>It probably was a good idea to add stress during the discussion to
>compare various possibilities, but there no longer is a need to
>italicise "first" like this, I think.
>
>> message(either the cover letter or the first patch in the series).
>
>Please have space before "(" (also found elsewhere in this message)
>to make this readable.
>
>>
>> The base tree info consists of the "base commit", which is a well-known
>> commit that is part of the stable part of the project history everybody
>> else works off of, and zero or more "prerequisite patches", which are
>> well-known patches in flight that is not yet part of the "base commit"
>> that need to be applied on top of "base commit" in topological order
>> before the patches can be applied.
>>
>> The "base commit" is shown as "base-commit: " followed by the 40-hex of
>> the commit object name.  A "prerequisite patch" is shown as
>> "prerequisite-patch-id: " followed by the 40-hex "patch id", which is a
>> sum of SHA-1 of the file diffs associated with a patch, with whitespace
>> and line numbers ignored, it's reasonably stable and unique.
>
>Let's be more helpful to end users.  They do not need to know the
>exact formula, especially when there is a command to generate or
>check the id for themselves:
>
>    "patch id", which can be obtained by passing the patch through the
>    "git patch-id --stable" command
>
>or something?  
>
>> For example, we have history where base is Z, with three prerequisites
>> X-Y-Z, before the patch series A-B-C, i.e.
>
>base is Z???
>
>	Imagine that on top of the public commit P, you applied
>	well-known patches X, Y and Z from somebody else, and then
>	built your three-patch series A, B, C.
>
>perhaps?
>
>>
>> 	P---X---Y---Z---A---B---C
>>
>> We could say "git format-patch --base=P -3 C"(or variants thereof, e.g.
>> with "--cover-letter" of using "Z..C" instead of "-3 C" to specify the
>> range), then we could get base tree information block showing at the
>> end of _first_ message as below:
>
>Again, if "first" is _so_ important to stress, it probably is worth
>saying that by "first" you mean either patch 1/n or patch 0/n when
>the cover letter exists.
>
>Also "could" may have made sense while we were having discussion on
>possible design of the hypothetical feature, but with the patch
>applied, the feature becomes a reality, so you can and should stop
>living in the hypothetical world and do s/could/can/ the above.
>
>	With "git format-patch --base=P -3 C" (or variants...), the
>	base tree information block is shown at the end of the first
>	message the command outputs (either the first patch, or the
>	cover letter), like this:
>
>perhaps?
>
>I assume that the patch to the documentation has the same text I
>commented on the above, so I won't repeat my comments to them.
>

Thanks for the review,  I'll follow all the comments above and
make changes to commit log as well as documentation.
 
>> 	base-commit: P
>> 	prerequisite-patch-id: X
>> 	prerequisite-patch-id: Y
>> 	prerequisite-patch-id: Z
>>
>> Helped-by: Junio C Hamano <gitster@pobox.com>
>> Helped-by: Wu Fengguang <fengguang.wu@intel.com>
>> Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com>
>> ---
>>  Documentation/git-format-patch.txt | 56 +++++++++++++++++++++++
>>  builtin/log.c                      | 92 ++++++++++++++++++++++++++++++++++++++
>>  t/t4014-format-patch.sh            | 15 +++++++
>>  3 files changed, 163 insertions(+)
>> ...
>> +static void prepare_bases(struct base_tree_info *bases,
>> +			  const char *base_commit,
>> +			  struct commit **list,
>> +			  int total)
>> +{
>> +	struct commit *base = NULL, *commit;
>> +	struct rev_info revs;
>> +	struct diff_options diffopt;
>> +	unsigned char sha1[20];
>> +	int i;
>> +
>> +	diff_setup(&diffopt);
>> +	DIFF_OPT_SET(&diffopt, RECURSIVE);
>> +	diff_setup_done(&diffopt);
>> +
>> +	base = lookup_commit_reference_by_name(base_commit);
>> +	if (!base)
>> +		die(_("Unknown commit %s"), base_commit);
>> +	oidcpy(&bases->base_commit, &base->object.oid);
>> +
>> +	init_revisions(&revs, NULL);
>> +	revs.max_parents = 1;
>> +	revs.topo_order = 1;
>> +	for (i = 0; i < total; i++) {
>> +		if (!in_merge_bases(base, list[i]) || base == list[i])
>> +			die(_("base commit should be the ancestor of revision list"));
>
>This check looks overly expensive, but I do not think of a more
>efficient way to do this, given that "All the commits from our
>series must reach the specified base" is what you seem to want.

Yes, that's what I want to make sure, for normal case, if patch
submitter has history as below:

	P---Z---A---B---C---D

and she may unintentionally specify wrong base by doing
"format-patch --base=B -4" while P or Z is the actual base,
the recevier such as robot would get confused or fooled if we
just provide B as the base commit in this case.

>
>My understanding is that if base=P is given and you are doing
>"format-patch Z..C" in this picture:
>
>    Q---P---Z---B---*---C
>     \             /
>      .-----------A
>
>your list would become A, B and C, and you want to detect that P is
>not an ancestor of A.  merge_bases_many() computes a wrong thing for
>this use case, and you'd need to go one-by-one.
>
>Unless there is some clever trick to take advantage of the previous
>traversal you made in order to find out A, B and C are the commits
>that are part of your series somehow.
>
>Anybody with clever ideas?
>

Re: [PATCH v4 2/4] format-patch: add '--base' option to record base tree info

[Ye Xiaolong]

On Tue, Apr 12, 2016 at 12:08:33PM -0700, Junio C Hamano wrote:
>> +static void prepare_bases(struct base_tree_info *bases,
>> +			  const char *base_commit,
>> +			  struct commit **list,
>> +			  int total)
>> +{
>> +	struct commit *base = NULL, *commit;
>> +	struct rev_info revs;
>> +	struct diff_options diffopt;
>> +	unsigned char sha1[20];
>> +	int i;
>> +
>> +	diff_setup(&diffopt);
>> +	DIFF_OPT_SET(&diffopt, RECURSIVE);
>> +	diff_setup_done(&diffopt);
>> +
>> +	base = lookup_commit_reference_by_name(base_commit);
>> +	if (!base)
>> +		die(_("Unknown commit %s"), base_commit);
>> +	oidcpy(&bases->base_commit, &base->object.oid);
>> +
>> +	init_revisions(&revs, NULL);
>> +	revs.max_parents = 1;
>> +	revs.topo_order = 1;
>> +	for (i = 0; i < total; i++) {
>> +		if (!in_merge_bases(base, list[i]) || base == list[i])
>> +			die(_("base commit should be the ancestor of revision list"));
>
>This check looks overly expensive, but I do not think of a more
>efficient way to do this, given that "All the commits from our
>series must reach the specified base" is what you seem to want.
>
>My understanding is that if base=P is given and you are doing
>"format-patch Z..C" in this picture:
>
>    Q---P---Z---B---*---C
>     \             /
>      .-----------A
>

How about we compute the merge base of the specified rev list in
cmdline (it should be Q in above case), then check whether specified
base (P in this case) could be reachable from it, if it couldn't, we
just error out.

>your list would become A, B and C, and you want to detect that P is
>not an ancestor of A.  merge_bases_many() computes a wrong thing for
>this use case, and you'd need to go one-by-one.
>
>Unless there is some clever trick to take advantage of the previous
>traversal you made in order to find out A, B and C are the commits
>that are part of your series somehow.
>
>Anybody with clever ideas?
>

Re: [PATCH v4 2/4] format-patch: add '--base' option to record base tree info

[Junio C Hamano]

Ye Xiaolong <xiaolong.ye@intel.com> writes:

>>> +	for (i = 0; i < total; i++) {
>>> +		if (!in_merge_bases(base, list[i]) || base == list[i])
>>> +			die(_("base commit should be the ancestor of revision list"));
>>
>>This check looks overly expensive, but I do not think of a more
>>efficient way to do this, given that "All the commits from our
>>series must reach the specified base" is what you seem to want.
>>
>>My understanding is that if base=P is given and you are doing
>>"format-patch Z..C" in this picture:
>>
>>    Q---P---Z---B---*---C
>>     \             /
>>      .-----------A
>>
>
> How about we compute the merge base of the specified rev list in
> cmdline (it should be Q in above case), then check whether specified
> base (P in this case) could be reachable from it, if it couldn't, we
> just error out.

What commits are you considering "the specified rev list in cmdline"
in the example?  Do you mean "commits in the list[], i.e. those to
be shown as patches?"

That is, you are proposing to find the topologically-youngest common
ancestors of A, B and C, which is Q?

There is no canned way to compute that (merge_bases_many() is not
that function).

You however can do repeated pair-wise merge base computations to
reduce the complexity from your O(n) loop to O(log n), I guess.  Do
a pair-wise merge base between A and B (which is Q), and do a merge
base between C (which is the remaining one) and Q to arrive at Q.

[PATCH v4 3/4] format-patch: introduce --base=auto option

[Xiaolong Ye]

Introduce --base=auto to record the base commit info automatically, the
base_commit will be the merge base of tip commit of the upstream branch
and revision-range specified in cmdline.

Helped-by: Junio C Hamano <gitster@pobox.com>
Helped-by: Wu Fengguang <fengguang.wu@intel.com>
Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com>
---
 Documentation/git-format-patch.txt |  4 ++++
 builtin/log.c                      | 32 ++++++++++++++++++++++++++++----
 t/t4014-format-patch.sh            | 14 ++++++++++++++
 3 files changed, 46 insertions(+), 4 deletions(-)

diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt
index 2a4c293..8283eea 100644
--- a/Documentation/git-format-patch.txt
+++ b/Documentation/git-format-patch.txt
@@ -577,6 +577,10 @@ The submitter could also use `git format-patch --base=P -3 C` to generate
 patches for A, B and C, and the identifiers for P, X, Y, Z are appended
 at the end of the _first_ message.
 
+If set `--base=auto` in cmdline, it will track base commit automatically,
+the base commit will be the merge base of tip commit of the remote-tracking
+branch and revision-range specified in cmdline.
+
 EXAMPLES
 --------
 
diff --git a/builtin/log.c b/builtin/log.c
index 73bc36d..510a427 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -1205,6 +1205,9 @@ static void prepare_bases(struct base_tree_info *bases,
 	struct commit *base = NULL, *commit;
 	struct rev_info revs;
 	struct diff_options diffopt;
+	struct branch *curr_branch;
+	struct commit_list *base_list;
+	const char *upstream;
 	unsigned char sha1[20];
 	int i;
 
@@ -1212,10 +1215,31 @@ static void prepare_bases(struct base_tree_info *bases,
 	DIFF_OPT_SET(&diffopt, RECURSIVE);
 	diff_setup_done(&diffopt);
 
-	base = lookup_commit_reference_by_name(base_commit);
-	if (!base)
-		die(_("Unknown commit %s"), base_commit);
-	oidcpy(&bases->base_commit, &base->object.oid);
+	if (!strcmp(base_commit, "auto")) {
+		curr_branch = branch_get(NULL);
+		upstream = branch_get_upstream(curr_branch, NULL);
+		if (upstream) {
+			if (get_sha1(upstream, sha1))
+				die(_("Failed to resolve '%s' as a valid ref."), upstream);
+			commit = lookup_commit_or_die(sha1, "upstream base");
+			base_list = get_merge_bases_many(commit, total, list);
+			/* There should be one and only one merge base. */
+			if (!base_list || base_list->next)
+				die(_("Could not find exact merge base."));
+			base = base_list->item;
+			free_commit_list(base_list);
+			oidcpy(&bases->base_commit, &base->object.oid);
+		} else {
+			die(_("Failed to get upstream, if you want to record base commit automatically,\n"
+			      "please use git branch --set-upstream-to to track a remote branch.\n"
+			      "Or you could specify base commit by --base=<base-commit-id> manually."));
+		}
+	} else {
+		base = lookup_commit_reference_by_name(base_commit);
+		if (!base)
+			die(_("Unknown commit %s"), base_commit);
+		oidcpy(&bases->base_commit, &base->object.oid);
+	}
 
 	init_revisions(&revs, NULL);
 	revs.max_parents = 1;
diff --git a/t/t4014-format-patch.sh b/t/t4014-format-patch.sh
index a6ce727..4603915 100755
--- a/t/t4014-format-patch.sh
+++ b/t/t4014-format-patch.sh
@@ -1475,4 +1475,18 @@ test_expect_success 'format-patch --base error handling' '
 	! git format-patch --base=HEAD~ -3
 '
 
+test_expect_success 'format-patch --base=auto' '
+	git checkout -b new master &&
+	git branch --set-upstream-to=master &&
+	echo "A" >>file &&
+	git add file &&
+	git commit -m "New change #A" &&
+	echo "B" >>file &&
+	git add file &&
+	git commit -m "New change #B" &&
+	git format-patch --stdout --base=auto -2 >patch &&
+	grep -e "^base-commit:" patch >actual &&
+	echo "base-commit: $(git rev-parse master)" >expected &&
+	test_cmp expected actual
+'
 test_done
-- 
2.8.1.120.g24d6b3f

[PATCH v4 4/4] format-patch: introduce format.base configuration

[Xiaolong Ye]

We can set format.base=auto to record the base commit info automatically,
it is equivalent to set --base=auto in cmdline.

The format.base has lower priority than command line option, so if user
set format.base=auto and pass the command line option in the meantime,
base_commit will be the one passed to command line option.

Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com>
---
 Documentation/git-format-patch.txt |  4 ++++
 builtin/log.c                      | 21 ++++++++++++++-------
 t/t4014-format-patch.sh            | 19 +++++++++++++++++++
 3 files changed, 37 insertions(+), 7 deletions(-)

diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt
index 8283eea..738121d 100644
--- a/Documentation/git-format-patch.txt
+++ b/Documentation/git-format-patch.txt
@@ -581,6 +581,10 @@ If set `--base=auto` in cmdline, it will track base commit automatically,
 the base commit will be the merge base of tip commit of the remote-tracking
 branch and revision-range specified in cmdline.
 
+If 'format.base=auto' is set in configuration file, it is equivalent
+to set '--base=auto' in cmdline.
+
+
 EXAMPLES
 --------
 
diff --git a/builtin/log.c b/builtin/log.c
index 510a427..489434a 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -705,6 +705,7 @@ static int do_signoff;
 static const char *signature = git_version_string;
 static const char *signature_file;
 static int config_cover_letter;
+static int config_base_commit;
 static const char *config_output_directory;
 
 enum {
@@ -786,6 +787,12 @@ static int git_format_config(const char *var, const char *value, void *cb)
 	}
 	if (!strcmp(var, "format.outputdirectory"))
 		return git_config_string(&config_output_directory, var, value);
+	if (!strcmp(var, "format.base")){
+		if (value && !strcasecmp(value, "auto")) {
+			config_base_commit = 1;
+			return 0;
+		}
+	}
 
 	return git_log_config(var, value, cb);
 }
@@ -1215,7 +1222,12 @@ static void prepare_bases(struct base_tree_info *bases,
 	DIFF_OPT_SET(&diffopt, RECURSIVE);
 	diff_setup_done(&diffopt);
 
-	if (!strcmp(base_commit, "auto")) {
+	if (base_commit && strcmp(base_commit, "auto")) {
+		base = lookup_commit_reference_by_name(base_commit);
+		if (!base)
+			die(_("Unknown commit %s"), base_commit);
+		oidcpy(&bases->base_commit, &base->object.oid);
+	} else if ((base_commit && !strcmp(base_commit, "auto")) || config_base_commit) {
 		curr_branch = branch_get(NULL);
 		upstream = branch_get_upstream(curr_branch, NULL);
 		if (upstream) {
@@ -1234,11 +1246,6 @@ static void prepare_bases(struct base_tree_info *bases,
 			      "please use git branch --set-upstream-to to track a remote branch.\n"
 			      "Or you could specify base commit by --base=<base-commit-id> manually."));
 		}
-	} else {
-		base = lookup_commit_reference_by_name(base_commit);
-		if (!base)
-			die(_("Unknown commit %s"), base_commit);
-		oidcpy(&bases->base_commit, &base->object.oid);
 	}
 
 	init_revisions(&revs, NULL);
@@ -1622,7 +1629,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	}
 
 	memset(&bases, 0, sizeof(bases));
-	if (base_commit) {
+	if (base_commit || config_base_commit) {
 		reset_revision_walk();
 		prepare_bases(&bases, base_commit, list, nr);
 	}
diff --git a/t/t4014-format-patch.sh b/t/t4014-format-patch.sh
index 4603915..6005b7c 100755
--- a/t/t4014-format-patch.sh
+++ b/t/t4014-format-patch.sh
@@ -1489,4 +1489,23 @@ test_expect_success 'format-patch --base=auto' '
 	echo "base-commit: $(git rev-parse master)" >expected &&
 	test_cmp expected actual
 '
+
+test_expect_success 'format-patch format.base option' '
+	test_when_finished "git config --unset format.base" &&
+	git config format.base auto &&
+	git format-patch --stdout -1 >patch &&
+	grep -e "^base-commit:" patch >actual &&
+	echo "base-commit: $(git rev-parse master)" >expected &&
+	test_cmp expected actual
+'
+
+test_expect_success 'format-patch --base overrides format.base' '
+	test_when_finished "git config --unset format.base" &&
+	git config format.base auto &&
+	git format-patch --stdout --base=HEAD~ -1 >patch &&
+	grep -e "^base-commit:" patch >actual &&
+	echo "base-commit: $(git rev-parse HEAD~)" >expected &&
+	test_cmp expected actual
+'
+
 test_done
-- 
2.8.1.120.g24d6b3f

Re: [PATCH v4 4/4] format-patch: introduce format.base configuration

[Junio C Hamano]

Xiaolong Ye <xiaolong.ye@intel.com> writes:

> +static int config_base_commit;

This variable is used as a simple boolean whose name is overly broad
(if it were named "config_base_auto" this complaint would not
apply).  If you envision possible future enhancements for this
configuration variable, "int config_base_commit" might make sense
but I don't think of anything offhand that would be happy with
"int".

> @@ -786,6 +787,12 @@ static int git_format_config(const char *var, const char *value, void *cb)
>  	}
>  	if (!strcmp(var, "format.outputdirectory"))
>  		return git_config_string(&config_output_directory, var, value);
> +	if (!strcmp(var, "format.base")){

Style. s/)){/)) {/

> +		if (value && !strcasecmp(value, "auto")) {

Does it make sense to allow "Auto" here?  Given that the command
line parsing uses strcmp() to require "auto", I do not think so.

> +			config_base_commit = 1;
> +			return 0;
> +		}

When a value other than "auto" is given, is it sane to ignore them
without even warning?

I am wondering if this wants to be a format.useAutoBase boolean
variable.

> @@ -1215,7 +1222,12 @@ static void prepare_bases(struct base_tree_info *bases,
>  	DIFF_OPT_SET(&diffopt, RECURSIVE);
>  	diff_setup_done(&diffopt);
>  
> -	if (!strcmp(base_commit, "auto")) {
> +	if (base_commit && strcmp(base_commit, "auto")) {
> +		base = lookup_commit_reference_by_name(base_commit);
> +		if (!base)
> +			die(_("Unknown commit %s"), base_commit);
> +		oidcpy(&bases->base_commit, &base->object.oid);
> +	} else if ((base_commit && !strcmp(base_commit, "auto")) || config_base_commit) {

It may be a poor design to teach prepare_bases() about "auto" thing.
Doesn't it belong to the caller?  The caller used to say "If a base
is given, then call that function, by the way, the base must be a
concrete one", and with the new "auto" feature, the caller loosens
the last part of the statement and says "If a base is given, call
that function, but if it is specified as "auto", I'd have to compute
it for the user before doing so".

Re: [PATCH v4 4/4] format-patch: introduce format.base configuration

[Ye Xiaolong]

On Tue, Apr 12, 2016 at 12:47:23PM -0700, Junio C Hamano wrote:
>Xiaolong Ye <xiaolong.ye@intel.com> writes:
>
>> +static int config_base_commit;
>
>This variable is used as a simple boolean whose name is overly broad
>(if it were named "config_base_auto" this complaint would not
>apply).  If you envision possible future enhancements for this
>configuration variable, "int config_base_commit" might make sense
>but I don't think of anything offhand that would be happy with
>"int".
>
>> @@ -786,6 +787,12 @@ static int git_format_config(const char *var, const char *value, void *cb)
>>  	}
>>  	if (!strcmp(var, "format.outputdirectory"))
>>  		return git_config_string(&config_output_directory, var, value);
>> +	if (!strcmp(var, "format.base")){
>
>Style. s/)){/)) {/
>
>> +		if (value && !strcasecmp(value, "auto")) {
>
>Does it make sense to allow "Auto" here?  Given that the command
>line parsing uses strcmp() to require "auto", I do not think so.
>
>> +			config_base_commit = 1;
>> +			return 0;
>> +		}
>
>When a value other than "auto" is given, is it sane to ignore them
>without even warning?
>
>I am wondering if this wants to be a format.useAutoBase boolean
>variable.
>

Thanks for the reminder, seems boolean variable is more suitable for
this case, I'll adopt to it in next iteration.
>> @@ -1215,7 +1222,12 @@ static void prepare_bases(struct base_tree_info *bases,
>>  	DIFF_OPT_SET(&diffopt, RECURSIVE);
>>  	diff_setup_done(&diffopt);
>>  
>> -	if (!strcmp(base_commit, "auto")) {
>> +	if (base_commit && strcmp(base_commit, "auto")) {
>> +		base = lookup_commit_reference_by_name(base_commit);
>> +		if (!base)
>> +			die(_("Unknown commit %s"), base_commit);
>> +		oidcpy(&bases->base_commit, &base->object.oid);
>> +	} else if ((base_commit && !strcmp(base_commit, "auto")) || config_base_commit) {
>
>It may be a poor design to teach prepare_bases() about "auto" thing.
>Doesn't it belong to the caller?  The caller used to say "If a base

Good point, as I understand your comments, we need to extract the "auto"
thing from prepare_bases() and call it early, then we could have a
concrete base before calling prepare_bases().

Thanks,
Xiaolong.
>is given, then call that function, by the way, the base must be a
>concrete one", and with the new "auto" feature, the caller loosens
>the last part of the statement and says "If a base is given, call
>that function, but if it is specified as "auto", I'd have to compute
>it for the user before doing so".