Re: [PATCH v4 12/14] commit-graph: create options for split files

From: Ramsay Jones <ramsay@ramsayjones.plus.com>
To: Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>,
	git@vger.kernel.org
Cc: peff@peff.net, avarab@gmail.com, git@jeffhostetler.com,
	jrnieder@google.com, steadmon@google.com,
	johannes.schindelin@gmx.de, Junio C Hamano <gitster@pobox.com>,
	Derrick Stolee <dstolee@microsoft.com>
Subject: Re: [PATCH v4 12/14] commit-graph: create options for split files
Date: Thu, 6 Jun 2019 19:41:46 +0100	[thread overview]
Message-ID: <02c9a0e2-4505-14a4-fa68-958a7f0c7935@ramsayjones.plus.com> (raw)
In-Reply-To: <ac5586a20f7ed600f8912b1aed07dea5098ea046.1559830527.git.gitgitgadget@gmail.com>

On 06/06/2019 15:15, Derrick Stolee via GitGitGadget wrote:
> From: Derrick Stolee <dstolee@microsoft.com>
> 
> The split commit-graph feature is now fully implemented, but needs
> some more run-time configurability. Allow direct callers to 'git
> commit-graph write --split' to specify the values used in the
> merge strategy and the expire time.
> 
> Update the documentation to specify these values.
> 
> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
>  Documentation/git-commit-graph.txt       | 21 +++++++++++++-
>  Documentation/technical/commit-graph.txt |  7 +++--
>  builtin/commit-graph.c                   | 20 +++++++++++---
>  builtin/commit.c                         |  2 +-
>  builtin/gc.c                             |  3 +-
>  commit-graph.c                           | 35 ++++++++++++++++--------
>  commit-graph.h                           | 12 ++++++--
>  t/t5323-split-commit-graph.sh            | 35 ++++++++++++++++++++++++
>  8 files changed, 112 insertions(+), 23 deletions(-)
> 
> diff --git a/Documentation/git-commit-graph.txt b/Documentation/git-commit-graph.txt
> index 624470e198..365e145e82 100644
> --- a/Documentation/git-commit-graph.txt
> +++ b/Documentation/git-commit-graph.txt
> @@ -26,7 +26,7 @@ OPTIONS
>  	Use given directory for the location of packfiles and commit-graph
>  	file. This parameter exists to specify the location of an alternate
>  	that only has the objects directory, not a full `.git` directory. The
> -	commit-graph file is expected to be at `<dir>/info/commit-graph` and
> +	commit-graph file is expected to be in the `<dir>/info` directory and
>  	the packfiles are expected to be in `<dir>/pack`.
>  
>  
> @@ -51,6 +51,25 @@ or `--stdin-packs`.)
>  +
>  With the `--append` option, include all commits that are present in the
>  existing commit-graph file.
> ++
> +With the `--split` option, write the commit-graph as a chain of multiple
> +commit-graph files stored in `<dir>/info/commit-graphs`. The new commits
> +not already in the commit-graph are added in a new "tip" file. This file
> +is merged with the existing file if the following merge conditions are
> +met:
> ++
> +* If `--size-multiple=<X>` is not specified, let `X` equal 2. If the new
> +tip file would have `N` commits and the previous tip has `M` commits and
> +`X` times `N` is greater than  `M`, instead merge the two files into a
> +single file.
> ++
> +* If `--max-commits=<M>` is specified with `M` a positive integer, and the
> +new tip file would have more than `M` commits, then instead merge the new
> +tip with the previous tip.
> ++
> +Finally, if `--expire-time=<datetime>` is not specified, let `datetime`
> +be the current time. After writing the split commit-graph, delete all
> +unused commit-graph whose modified times are older than `datetime`.
>  
>  'read'::
>  
> diff --git a/Documentation/technical/commit-graph.txt b/Documentation/technical/commit-graph.txt
> index aed4350a59..729fbcb32f 100644
> --- a/Documentation/technical/commit-graph.txt
> +++ b/Documentation/technical/commit-graph.txt
> @@ -248,10 +248,11 @@ When writing a set of commits that do not exist in the commit-graph stack of
>  height N, we default to creating a new file at level N + 1. We then decide to
>  merge with the Nth level if one of two conditions hold:
>  
> -  1. The expected file size for level N + 1 is at least half the file size for
> -     level N.
> +  1. `--size-multiple=<X>` is specified or X = 2, and the number of commits in
> +     level N is less than X times the number of commits in level N + 1.
>  
> -  2. Level N + 1 contains more than 64,0000 commits.
> +  2. `--max-commits=<C>` is specified with non-zero C and the number of commits
> +     in level N + 1 is more than C commits.
>  
>  This decision cascades down the levels: when we merge a level we create a new
>  set of commits that then compares to the next level.
> diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c
> index c2c07d3917..18e3b61fb6 100644
> --- a/builtin/commit-graph.c
> +++ b/builtin/commit-graph.c
> @@ -10,7 +10,7 @@ static char const * const builtin_commit_graph_usage[] = {
>  	N_("git commit-graph [--object-dir <objdir>]"),
>  	N_("git commit-graph read [--object-dir <objdir>]"),
>  	N_("git commit-graph verify [--object-dir <objdir>]"),
> -	N_("git commit-graph write [--object-dir <objdir>] [--append|--split] [--reachable|--stdin-packs|--stdin-commits]"),
> +	N_("git commit-graph write [--object-dir <objdir>] [--append|--split] [--reachable|--stdin-packs|--stdin-commits] <split options>"),
>  	NULL
>  };
>  
> @@ -25,7 +25,7 @@ static const char * const builtin_commit_graph_read_usage[] = {
>  };
>  
>  static const char * const builtin_commit_graph_write_usage[] = {
> -	N_("git commit-graph write [--object-dir <objdir>] [--append|--split] [--reachable|--stdin-packs|--stdin-commits]"),
> +	N_("git commit-graph write [--object-dir <objdir>] [--append|--split] [--reachable|--stdin-packs|--stdin-commits] <split options>"),
>  	NULL
>  };
>  
> @@ -135,6 +135,7 @@ static int graph_read(int argc, const char **argv)
>  }
>  
>  extern int read_replace_refs;
> +struct split_commit_graph_opts split_opts;

This 'split_opts' variable needs to be marked 'static'.

Thanks.

ATB,
Ramsay Jones