From: "Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>
To: git@vger.kernel.org
Cc: "Junio C Hamano" <gitster@pobox.com>,
"Shawn O. Pearce" <spearce@spearce.org>,
"Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>
Subject: [PATCH v3 3/3] index-pack: eliminate unlimited recursion in get_base_data()
Date: Sat, 14 Jan 2012 19:19:55 +0700 [thread overview]
Message-ID: <1326543595-28300-4-git-send-email-pclouds@gmail.com> (raw)
In-Reply-To: <1326543595-28300-1-git-send-email-pclouds@gmail.com>
Revese the order of delta applying so that by the time a delta is
applied, its base is either non-delta or already inflated.
get_base_data() is still recursive, but because base's data is always
ready, the inner get_base_data() call never has any chance to call
itself again.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
builtin/index-pack.c | 53 +++++++++++++++++++++++++++++++++++++++++--------
1 files changed, 44 insertions(+), 9 deletions(-)
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 38ff03a..dc6a584 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -515,14 +515,52 @@ static int is_delta_type(enum object_type type)
return (type == OBJ_REF_DELTA || type == OBJ_OFS_DELTA);
}
+/*
+ * This function is part of find_unresolved_deltas(). There are two
+ * walkers going in the opposite ways.
+ *
+ * The first one in find_unresolved_deltas() traverses down from
+ * parent node to children, deflating nodes along the way. However,
+ * memory for deflated nodes is limited by delta_base_cache_limit, so
+ * at some point parent node's deflated content may be freed.
+ *
+ * The second walker is this function, which goes from current node up
+ * to top parent if necessary to deflate the node. In normal
+ * situation, its parent node would be already deflated, so it just
+ * needs to apply delta.
+ *
+ * In worst case scenario, parent node is no longer deflated because
+ * we're running out of delta_base_cache_limit, then we need to
+ * re-deflate parents, possibly up to the top base.
+ *
+ * All deflated objecsts here are subject to be freed if we exceed
+ * delta_base_cache_limit, just like in find_unresolved_deltas(), we
+ * just need to make sure the last node is not freed.
+ */
static void *get_base_data(struct base_data *c)
{
if (!c->data) {
struct object_entry *obj = c->obj;
+ struct base_data **delta = NULL;
+ int delta_nr = 0, delta_alloc = 0;
- if (is_delta_type(obj->type)) {
- void *base = get_base_data(c->base);
- void *raw = get_data_from_pack(obj);
+ while (is_delta_type(c->obj->type) && !c->data) {
+ ALLOC_GROW(delta, delta_nr + 1, delta_alloc);
+ delta[delta_nr++] = c;
+ c = c->base;
+ }
+ if (!delta_nr) {
+ c->data = get_data_from_pack(obj);
+ c->size = obj->size;
+ base_cache_used += c->size;
+ prune_base_data(c);
+ }
+ for (; delta_nr > 0; delta_nr--) {
+ void *base, *raw;
+ c = delta[delta_nr - 1];
+ obj = c->obj;
+ base = get_base_data(c->base);
+ raw = get_data_from_pack(obj);
c->data = patch_delta(
base, c->base->size,
raw, obj->size,
@@ -530,13 +568,10 @@ static void *get_base_data(struct base_data *c)
free(raw);
if (!c->data)
bad_object(obj->idx.offset, "failed to apply delta");
- } else {
- c->data = get_data_from_pack(obj);
- c->size = obj->size;
+ base_cache_used += c->size;
+ prune_base_data(c);
}
-
- base_cache_used += c->size;
- prune_base_data(c);
+ free(delta);
}
return c->data;
}
--
1.7.8.36.g69ee2
next prev parent reply other threads:[~2012-01-14 12:21 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-12-26 12:04 [PATCH 1/3] Eliminate recursion in setting/clearing marks in commit list Nguyễn Thái Ngọc Duy
2011-12-26 12:04 ` [PATCH 2/3] index-pack: eliminate recursion in find_unresolved_deltas Nguyễn Thái Ngọc Duy
2011-12-26 12:04 ` [PATCH 3/3] index-pack: eliminate unlimited recursion in get_delta_base() Nguyễn Thái Ngọc Duy
2012-01-09 3:59 ` [PATCH v2 0/3] nd/index-pack-no-recurse Nguyễn Thái Ngọc Duy
2012-01-09 19:30 ` Junio C Hamano
2012-01-14 12:19 ` [PATCH v3 " Nguyễn Thái Ngọc Duy
2012-01-14 12:19 ` [PATCH v3 1/3] Eliminate recursion in setting/clearing marks in commit list Nguyễn Thái Ngọc Duy
2012-01-14 15:23 ` Peter Baumann
2012-01-15 9:25 ` Nguyen Thai Ngoc Duy
2012-01-14 12:19 ` [PATCH v3 2/3] index-pack: eliminate recursion in find_unresolved_deltas Nguyễn Thái Ngọc Duy
2012-01-14 12:19 ` Nguyễn Thái Ngọc Duy [this message]
2012-01-09 3:59 ` [PATCH v2 1/3] Eliminate recursion in setting/clearing marks in commit list Nguyễn Thái Ngọc Duy
2012-01-09 22:09 ` Junio C Hamano
2012-01-09 3:59 ` [PATCH v2 2/3] index-pack: eliminate recursion in find_unresolved_deltas Nguyễn Thái Ngọc Duy
2012-01-09 22:10 ` Junio C Hamano
2012-01-10 12:23 ` Nguyen Thai Ngoc Duy
2012-01-12 20:32 ` Junio C Hamano
2012-01-09 3:59 ` [PATCH v2 3/3] index-pack: eliminate unlimited recursion in get_delta_base() Nguyễn Thái Ngọc Duy
2012-01-09 22:51 ` Junio C Hamano
2012-01-10 13:03 ` Nguyen Thai Ngoc Duy
2012-01-10 13:16 ` Nguyen Thai Ngoc Duy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1326543595-28300-4-git-send-email-pclouds@gmail.com \
--to=pclouds@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=spearce@spearce.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).