From: Christian Couder <christian.couder@gmail.com>
To: git@vger.kernel.org
Cc: Junio C Hamano <gitster@pobox.com>, Jeff King <peff@peff.net>,
Duy Nguyen <pclouds@gmail.com>,
Johannes Schindelin <Johannes.Schindelin@gmx.de>,
Stefan Beller <sbeller@google.com>,
Christian Couder <chriscool@tuxfamily.org>
Subject: [PATCH v3 0/8] Add delta islands support
Date: Thu, 9 Aug 2018 17:55:24 +0200 [thread overview]
Message-ID: <20180809155532.26151-1-chriscool@tuxfamily.org> (raw)
This patch series is upstreaming work made by GitHub and available in:
https://github.com/peff/git/commits/jk/delta-islands
The above work has been already described in the following article:
https://githubengineering.com/counting-objects/
The above branch contains only one patch. In this patch series the
patch has been split into 5 patches (1/8, 2/8, 4/8, 5/8 and 6/8) with
their own commit message, and on top of that 3 new patches (3/8, 7/8
and 8/8) have been added. The new patches implement things that were
requested after the previous iterations.
I kept Peff as the author of the original 5 patches and took the
liberty to add his Signed-off-by to them.
As explained in details in the Counting Object article referenced
above, the goal of the delta island mechanism is for a hosting
provider to make it possible to have the "forks" of a repository share
as much storage as possible while preventing object packs to contain
deltas between different forks.
If deltas between different forks are not prevented, when users clone
or fetch a fork, preparing the pack that should be sent to them can be
very costly CPU wise, as objects from a different fork should not be
sent, which means that a lot of deltas might need to be computed
again (instead of reusing existing deltas).
The following changes have been made since the previous iteration:
* suggested by Duy: move the code computing the write order for a
layer to a new separate function called compute_layer_order() in
builtin/pack-objects.c in new patch 3/8
I think that indeed this makes the following patch (4/8) shorter and
easier to understand as well as the overall result nicer.
* suggested by Duy and Peff: rework the way the 'tree_depth' field is
moved from 'struct object_entry' to 'struct packing_data' in
pack-object.h in patch 7/8
* suggested by Duy and Peff: move field 'layer' from
'struct object_entry' to 'struct packing_data' in pack-object.h in
new patch 8/8
This patch series is also available on GitHub in:
https://github.com/chriscool/git/commits/delta-islands
The previous versions are available there:
V2: https://github.com/chriscool/git/commits/delta-islands19
V1: https://github.com/chriscool/git/commits/delta-islands6
V2: https://public-inbox.org/git/20180805172525.15278-1-chriscool@tuxfamily.org/
V1: https://public-inbox.org/git/20180722054836.28935-1-chriscool@tuxfamily.org/
Christian Couder (3):
pack-objects: refactor code into compute_layer_order()
pack-objects: move tree_depth into 'struct packing_data'
pack-objects: move 'layer' into 'struct packing_data'
Jeff King (5):
packfile: make get_delta_base() non static
Add delta-islands.{c,h}
pack-objects: add delta-islands support
repack: add delta-islands support
t: add t5319-delta-islands.sh
Documentation/config.txt | 19 ++
Documentation/git-pack-objects.txt | 97 ++++++
Documentation/git-repack.txt | 5 +
Makefile | 1 +
builtin/pack-objects.c | 137 +++++---
builtin/repack.c | 9 +
delta-islands.c | 506 +++++++++++++++++++++++++++++
delta-islands.h | 11 +
pack-objects.c | 12 +
pack-objects.h | 39 +++
packfile.c | 10 +-
packfile.h | 7 +
t/t5319-delta-islands.sh | 143 ++++++++
13 files changed, 948 insertions(+), 48 deletions(-)
create mode 100644 delta-islands.c
create mode 100644 delta-islands.h
create mode 100755 t/t5319-delta-islands.sh
--
2.18.0.555.g17f9c4abba
next reply other threads:[~2018-08-09 15:56 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-08-09 15:55 Christian Couder [this message]
2018-08-09 15:55 ` [PATCH v3 1/8] packfile: make get_delta_base() non static Christian Couder
2018-08-09 15:55 ` [PATCH v3 2/8] Add delta-islands.{c,h} Christian Couder
2018-08-11 9:04 ` SZEDER Gábor
2018-08-11 10:32 ` Christian Couder
2018-08-11 14:12 ` Jeff King
2018-08-12 4:32 ` Christian Couder
2018-08-09 15:55 ` [PATCH v3 3/8] pack-objects: refactor code into compute_layer_order() Christian Couder
2018-08-09 15:55 ` [PATCH v3 4/8] pack-objects: add delta-islands support Christian Couder
2018-08-09 15:55 ` [PATCH v3 5/8] repack: " Christian Couder
2018-08-09 15:55 ` [PATCH v3 6/8] t: add t5319-delta-islands.sh Christian Couder
2018-08-09 15:55 ` [PATCH v3 7/8] pack-objects: move tree_depth into 'struct packing_data' Christian Couder
2018-08-09 15:55 ` [PATCH v3 8/8] pack-objects: move 'layer' " Christian Couder
2018-08-09 19:34 ` [PATCH v3 0/8] Add delta islands support Christian Couder
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180809155532.26151-1-chriscool@tuxfamily.org \
--to=christian.couder@gmail.com \
--cc=Johannes.Schindelin@gmx.de \
--cc=chriscool@tuxfamily.org \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=pclouds@gmail.com \
--cc=peff@peff.net \
--cc=sbeller@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).