All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>
To: git@vger.kernel.org
Cc: "Junio C Hamano" <gitster@pobox.com>,
	"Eric Sunshine" <sunshine@sunshineco.com>,
	"Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>
Subject: [PATCH v2 15/27] shallow.c: implement a generic shallow boundary finder based on rev-list
Date: Sun, 12 Jun 2016 17:53:57 +0700	[thread overview]
Message-ID: <20160612105409.22156-16-pclouds@gmail.com> (raw)
In-Reply-To: <20160612105409.22156-1-pclouds@gmail.com>

Instead of a custom commit walker like get_shallow_commits(), this new
function uses rev-list to mark NOT_SHALLOW to all reachable commits,
except borders. The definition of reachable is to be defined by the
protocol later. This makes it more flexible to define shallow boundary.

The way we find border is paint all reachable commits NOT_SHALLOW.  Any
of them that "touches" commits without NOT_SHALLOW flag are considered
shallow (e.g. zero parents via grafting mechanism). Shallow commits and
their true parents are all marked SHALLOW. Then NOT_SHALLOW is removed
from shallow commits at the end.

There is an interesting observation. With a generic walker, we can
produce all kinds of shallow cutting. In the following graph, every
commit but "x" is reachable. "b" is a parent of "a".

           x -- a -- o
          /    /
    x -- c -- b -- o

After this function is run, "a" and "c" are both considered shallow
commits. After grafting occurs at the client side, what we see is

                a -- o
                    /
         c -- b -- o

Notice that because of grafting, "a" has zero parents, so "b" is no
longer a parent of "a".

This is unfortunate and may be solved in two ways. The first is change
the way shallow grafting works and keep "a -- b" connection if "b"
exists and always ends at shallow commits (iow, no loose ends). This is
hard to detect, or at least not cheap to do.

The second way is mark one "x" as shallow commit instead of "a" and
produce this graph at client side:

           x -- a -- o
               /    /
         c -- b -- o

More commits, but simpler grafting rules.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 commit.h  |  2 ++
 shallow.c | 78 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 80 insertions(+)

diff --git a/commit.h b/commit.h
index 5d58be0..b717be1 100644
--- a/commit.h
+++ b/commit.h
@@ -258,6 +258,8 @@ extern int for_each_commit_graft(each_commit_graft_fn, void *);
 extern int is_repository_shallow(void);
 extern struct commit_list *get_shallow_commits(struct object_array *heads,
 		int depth, int shallow_flag, int not_shallow_flag);
+extern struct commit_list *get_shallow_commits_by_rev_list(
+		int ac, const char **av, int shallow_flag, int not_shallow_flag);
 extern void set_alternate_shallow_file(const char *path, int override);
 extern int write_shallow_commits(struct strbuf *out, int use_pack_protocol,
 				 const struct sha1_array *extra);
diff --git a/shallow.c b/shallow.c
index 60f1505..40c2485 100644
--- a/shallow.c
+++ b/shallow.c
@@ -10,6 +10,8 @@
 #include "diff.h"
 #include "revision.h"
 #include "commit-slab.h"
+#include "revision.h"
+#include "list-objects.h"
 
 static int is_shallow = -1;
 static struct stat_validity shallow_stat;
@@ -137,6 +139,82 @@ struct commit_list *get_shallow_commits(struct object_array *heads, int depth,
 	return result;
 }
 
+static void show_commit(struct commit *commit, void *data)
+{
+	commit_list_insert(commit, data);
+}
+
+/*
+ * Given rev-list arguments, run rev-list. All reachable commits
+ * except border ones are marked with not_shallow_flag. Border commits
+ * are marked with shallow_flag. The list of border/shallow commits
+ * are also returned.
+ */
+struct commit_list *get_shallow_commits_by_rev_list(int ac, const char **av,
+						    int shallow_flag,
+						    int not_shallow_flag)
+{
+	struct commit_list *result = NULL, *p;
+	struct commit_list *not_shallow_list = NULL;
+	struct rev_info revs;
+	int both_flags = shallow_flag | not_shallow_flag;
+
+	/*
+	 * SHALLOW (excluded) and NOT_SHALLOW (included) should not be
+	 * set at this point. But better be safe than sorry.
+	 */
+	clear_object_flags(both_flags);
+
+	is_repository_shallow(); /* make sure shallows are read */
+
+	init_revisions(&revs, NULL);
+	save_commit_buffer = 0;
+	setup_revisions(ac, av, &revs, NULL);
+
+	if (prepare_revision_walk(&revs))
+		die("revision walk setup failed");
+	traverse_commit_list(&revs, show_commit, NULL, &not_shallow_list);
+
+	/* Mark all reachable commits as NOT_SHALLOW */
+	for (p = not_shallow_list; p; p = p->next)
+		p->item->object.flags |= not_shallow_flag;
+
+	/*
+	 * mark border commits SHALLOW + NOT_SHALLOW.
+	 * We cannot clear NOT_SHALLOW right now. Imagine border
+	 * commit A is processed first, then commit B, whose parent is
+	 * A, later. If NOT_SHALLOW on A is cleared at step 1, B
+	 * itself is considered border at step 2, which is incorrect.
+	 */
+	for (p = not_shallow_list; p; p = p->next) {
+		struct commit *c = p->item;
+		struct commit_list *parent;
+
+		if (parse_commit(c))
+			die("unable to parse commit %s",
+			    oid_to_hex(&c->object.oid));
+
+		for (parent = c->parents; parent; parent = parent->next)
+			if (!(parent->item->object.flags & not_shallow_flag)) {
+				c->object.flags |= shallow_flag;
+				commit_list_insert(c, &result);
+				break;
+			}
+	}
+	free_commit_list(not_shallow_list);
+
+	/*
+	 * Now we can clean up NOT_SHALLOW on border commits. Having
+	 * both flags set can confuse the caller.
+	 */
+	for (p = result; p; p = p->next) {
+		struct object *o = &p->item->object;
+		if ((o->flags & both_flags) == both_flags)
+			o->flags &= ~not_shallow_flag;
+	}
+	return result;
+}
+
 static void check_shallow_file_for_update(void)
 {
 	if (is_shallow == -1)
-- 
2.8.2.524.g6ff3d78

  parent reply	other threads:[~2016-06-12 10:56 UTC|newest]

Thread overview: 64+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-06-10 12:26 [PATCH 00/27] nd/shallow-deepen updates Nguyễn Thái Ngọc Duy
2016-06-10 12:26 ` [PATCH 01/27] remote-curl.c: convert fetch_git() to use argv_array Nguyễn Thái Ngọc Duy
2016-06-10 12:26 ` [PATCH 02/27] transport-helper.c: refactor set_helper_option() Nguyễn Thái Ngọc Duy
2016-06-10 12:26 ` [PATCH 03/27] upload-pack: move shallow deepen code out of receive_needs() Nguyễn Thái Ngọc Duy
2016-06-10 12:26 ` [PATCH 04/27] upload-pack: move "shallow" sending code out of deepen() Nguyễn Thái Ngọc Duy
2016-06-10 20:05   ` Junio C Hamano
2016-06-10 12:26 ` [PATCH 05/27] upload-pack: remove unused variable "backup" Nguyễn Thái Ngọc Duy
2016-06-10 20:06   ` Junio C Hamano
2016-06-10 12:26 ` [PATCH 06/27] upload-pack: move "unshallow" sending code out of deepen() Nguyễn Thái Ngọc Duy
2016-06-10 20:09   ` Junio C Hamano
2016-06-10 12:26 ` [PATCH 07/27] upload-pack: use skip_prefix() instead of starts_with() Nguyễn Thái Ngọc Duy
2016-06-10 12:26 ` [PATCH 08/27] upload-pack: tighten number parsing at "deepen" lines Nguyễn Thái Ngọc Duy
2016-06-10 12:26 ` [PATCH 09/27] upload-pack: make check_non_tip() clean things up error Nguyễn Thái Ngọc Duy
2016-06-10 20:25   ` Junio C Hamano
2016-06-10 12:26 ` [PATCH 10/27] upload-pack: move rev-list code out of check_non_tip() Nguyễn Thái Ngọc Duy
2016-06-10 20:36   ` Junio C Hamano
2016-06-10 12:26 ` [PATCH 11/27] fetch-pack: use skip_prefix() instead of starts_with() Nguyễn Thái Ngọc Duy
2016-06-10 12:26 ` [PATCH 12/27] fetch-pack: use a common function for verbose printing Nguyễn Thái Ngọc Duy
2016-06-10 12:27 ` [PATCH 13/27] fetch-pack.c: mark strings for translating Nguyễn Thái Ngọc Duy
2016-06-10 12:27 ` [PATCH 14/27] fetch-pack: use a separate flag for fetch in deepening mode Nguyễn Thái Ngọc Duy
2016-06-10 12:27 ` [PATCH 15/27] shallow.c: implement a generic shallow boundary finder based on rev-list Nguyễn Thái Ngọc Duy
2016-06-10 12:27 ` [PATCH 16/27] upload-pack: add deepen-since to cut shallow repos based on time Nguyễn Thái Ngọc Duy
2016-06-10 12:27 ` [PATCH 17/27] fetch: define shallow boundary with --shallow-since Nguyễn Thái Ngọc Duy
2016-06-10 12:27 ` [PATCH 18/27] clone: define shallow clone boundary based on time " Nguyễn Thái Ngọc Duy
2016-06-10 12:27 ` [PATCH 19/27] t5500, t5539: tests for shallow depth since a specific date Nguyễn Thái Ngọc Duy
2016-06-10 12:27 ` [PATCH 20/27] refs: add expand_ref() Nguyễn Thái Ngọc Duy
2016-06-10 12:27 ` [PATCH 21/27] upload-pack: support define shallow boundary by excluding revisions Nguyễn Thái Ngọc Duy
2016-06-10 12:27 ` [PATCH 22/27] fetch: define shallow boundary with --shallow-exclude Nguyễn Thái Ngọc Duy
2016-06-10 12:27 ` [PATCH 23/27] clone: define shallow clone " Nguyễn Thái Ngọc Duy
2016-06-10 12:27 ` [PATCH 24/27] t5500, t5539: tests for shallow depth excluding a ref Nguyễn Thái Ngọc Duy
2016-06-10 12:27 ` [PATCH 25/27] upload-pack: split check_unreachable() in two, prep for get_reachable_list() Nguyễn Thái Ngọc Duy
2016-06-10 12:27 ` [PATCH 26/27] upload-pack: add get_reachable_list() Nguyễn Thái Ngọc Duy
2016-06-10 12:27 ` [PATCH 27/27] fetch, upload-pack: --deepen=N extends shallow boundary by N commits Nguyễn Thái Ngọc Duy
2016-06-10 23:42 ` [PATCH 00/27] nd/shallow-deepen updates Eric Sunshine
2016-06-13 17:10   ` Junio C Hamano
2016-06-14  9:21     ` Duy Nguyen
2016-06-12 10:53 ` [PATCH v2 " Nguyễn Thái Ngọc Duy
2016-06-12 10:53   ` [PATCH v2 01/27] remote-curl.c: convert fetch_git() to use argv_array Nguyễn Thái Ngọc Duy
2016-06-12 10:53   ` [PATCH v2 02/27] transport-helper.c: refactor set_helper_option() Nguyễn Thái Ngọc Duy
2016-06-12 10:53   ` [PATCH v2 03/27] upload-pack: move shallow deepen code out of receive_needs() Nguyễn Thái Ngọc Duy
2016-06-12 10:53   ` [PATCH v2 04/27] upload-pack: move "shallow" sending code out of deepen() Nguyễn Thái Ngọc Duy
2016-06-12 10:53   ` [PATCH v2 05/27] upload-pack: remove unused variable "backup" Nguyễn Thái Ngọc Duy
2016-06-12 10:53   ` [PATCH v2 06/27] upload-pack: move "unshallow" sending code out of deepen() Nguyễn Thái Ngọc Duy
2016-06-12 10:53   ` [PATCH v2 07/27] upload-pack: use skip_prefix() instead of starts_with() Nguyễn Thái Ngọc Duy
2016-06-12 10:53   ` [PATCH v2 08/27] upload-pack: tighten number parsing at "deepen" lines Nguyễn Thái Ngọc Duy
2016-06-12 10:53   ` [PATCH v2 09/27] upload-pack: make check_non_tip() clean things up on error Nguyễn Thái Ngọc Duy
2016-06-12 10:53   ` [PATCH v2 10/27] upload-pack: move rev-list code out of check_non_tip() Nguyễn Thái Ngọc Duy
2016-06-12 10:53   ` [PATCH v2 11/27] fetch-pack: use skip_prefix() instead of starts_with() Nguyễn Thái Ngọc Duy
2016-06-12 10:53   ` [PATCH v2 12/27] fetch-pack: use a common function for verbose printing Nguyễn Thái Ngọc Duy
2016-06-12 10:53   ` [PATCH v2 13/27] fetch-pack.c: mark strings for translating Nguyễn Thái Ngọc Duy
2016-06-12 10:53   ` [PATCH v2 14/27] fetch-pack: use a separate flag for fetch in deepening mode Nguyễn Thái Ngọc Duy
2016-06-12 10:53   ` Nguyễn Thái Ngọc Duy [this message]
2016-06-12 10:53   ` [PATCH v2 16/27] upload-pack: add deepen-since to cut shallow repos based on time Nguyễn Thái Ngọc Duy
2016-06-12 10:53   ` [PATCH v2 17/27] fetch: define shallow boundary with --shallow-since Nguyễn Thái Ngọc Duy
2016-06-12 10:54   ` [PATCH v2 18/27] clone: define shallow clone boundary based on time " Nguyễn Thái Ngọc Duy
2016-06-12 10:54   ` [PATCH v2 19/27] t5500, t5539: tests for shallow depth since a specific date Nguyễn Thái Ngọc Duy
2016-06-12 10:54   ` [PATCH v2 20/27] refs: add expand_ref() Nguyễn Thái Ngọc Duy
2016-06-12 10:54   ` [PATCH v2 21/27] upload-pack: support define shallow boundary by excluding revisions Nguyễn Thái Ngọc Duy
2016-06-12 10:54   ` [PATCH v2 22/27] fetch: define shallow boundary with --shallow-exclude Nguyễn Thái Ngọc Duy
2016-06-12 10:54   ` [PATCH v2 23/27] clone: define shallow clone " Nguyễn Thái Ngọc Duy
2016-06-12 10:54   ` [PATCH v2 24/27] t5500, t5539: tests for shallow depth excluding a ref Nguyễn Thái Ngọc Duy
2016-06-12 10:54   ` [PATCH v2 25/27] upload-pack: split check_unreachable() in two, prep for get_reachable_list() Nguyễn Thái Ngọc Duy
2016-06-12 10:54   ` [PATCH v2 26/27] upload-pack: add get_reachable_list() Nguyễn Thái Ngọc Duy
2016-06-12 10:54   ` [PATCH v2 27/27] fetch, upload-pack: --deepen=N extends shallow boundary by N commits Nguyễn Thái Ngọc Duy

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20160612105409.22156-16-pclouds@gmail.com \
    --to=pclouds@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=sunshine@sunshineco.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.