* [PATCH v5 01/22] config: mark an error message up for translation
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
@ 2017-03-06 9:41 ` Christian Couder
2017-03-06 9:41 ` [PATCH v5 02/22] t1700: change here document style Christian Couder
` (20 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:41 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
config.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/config.c b/config.c
index c6b874a7bf..2ac1aa19b0 100644
--- a/config.c
+++ b/config.c
@@ -1728,8 +1728,8 @@ int git_config_get_untracked_cache(void)
if (!strcasecmp(v, "keep"))
return -1;
- error("unknown core.untrackedCache value '%s'; "
- "using 'keep' default value", v);
+ error(_("unknown core.untrackedCache value '%s'; "
+ "using 'keep' default value"), v);
return -1;
}
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 02/22] t1700: change here document style
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
2017-03-06 9:41 ` [PATCH v5 01/22] config: mark an error message up for translation Christian Couder
@ 2017-03-06 9:41 ` Christian Couder
2017-03-06 9:41 ` [PATCH v5 03/22] config: add git_config_get_split_index() Christian Couder
` (19 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:41 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
This improves test indentation by getting rid of the outdated
here document style.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
t/t1700-split-index.sh | 170 ++++++++++++++++++++++++-------------------------
1 file changed, 85 insertions(+), 85 deletions(-)
diff --git a/t/t1700-split-index.sh b/t/t1700-split-index.sh
index 6096f2c630..5ea227e6a1 100755
--- a/t/t1700-split-index.sh
+++ b/t/t1700-split-index.sh
@@ -19,12 +19,12 @@ test_expect_success 'enable split index' '
own=8299b0bcd1ac364e5f1d7768efb62fa2da79a339
base=39d890139ee5356c7ef572216cebcd27aa41f9df
fi &&
- cat >expect <<EOF &&
-own $own
-base $base
-replacements:
-deletions:
-EOF
+ cat >expect <<-EOF &&
+ own $own
+ base $base
+ replacements:
+ deletions:
+ EOF
test_cmp expect actual
'
@@ -32,51 +32,51 @@ test_expect_success 'add one file' '
: >one &&
git update-index --add one &&
git ls-files --stage >ls-files.actual &&
- cat >ls-files.expect <<EOF &&
-100644 $EMPTY_BLOB 0 one
-EOF
+ cat >ls-files.expect <<-EOF &&
+ 100644 $EMPTY_BLOB 0 one
+ EOF
test_cmp ls-files.expect ls-files.actual &&
test-dump-split-index .git/index | sed "/^own/d" >actual &&
- cat >expect <<EOF &&
-base $base
-100644 $EMPTY_BLOB 0 one
-replacements:
-deletions:
-EOF
+ cat >expect <<-EOF &&
+ base $base
+ 100644 $EMPTY_BLOB 0 one
+ replacements:
+ deletions:
+ EOF
test_cmp expect actual
'
test_expect_success 'disable split index' '
git update-index --no-split-index &&
git ls-files --stage >ls-files.actual &&
- cat >ls-files.expect <<EOF &&
-100644 $EMPTY_BLOB 0 one
-EOF
+ cat >ls-files.expect <<-EOF &&
+ 100644 $EMPTY_BLOB 0 one
+ EOF
test_cmp ls-files.expect ls-files.actual &&
BASE=$(test-dump-split-index .git/index | grep "^own" | sed "s/own/base/") &&
test-dump-split-index .git/index | sed "/^own/d" >actual &&
- cat >expect <<EOF &&
-not a split index
-EOF
+ cat >expect <<-EOF &&
+ not a split index
+ EOF
test_cmp expect actual
'
test_expect_success 'enable split index again, "one" now belongs to base index"' '
git update-index --split-index &&
git ls-files --stage >ls-files.actual &&
- cat >ls-files.expect <<EOF &&
-100644 $EMPTY_BLOB 0 one
-EOF
+ cat >ls-files.expect <<-EOF &&
+ 100644 $EMPTY_BLOB 0 one
+ EOF
test_cmp ls-files.expect ls-files.actual &&
test-dump-split-index .git/index | sed "/^own/d" >actual &&
- cat >expect <<EOF &&
-$BASE
-replacements:
-deletions:
-EOF
+ cat >expect <<-EOF &&
+ $BASE
+ replacements:
+ deletions:
+ EOF
test_cmp expect actual
'
@@ -84,18 +84,18 @@ test_expect_success 'modify original file, base index untouched' '
echo modified >one &&
git update-index one &&
git ls-files --stage >ls-files.actual &&
- cat >ls-files.expect <<EOF &&
-100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0 one
-EOF
+ cat >ls-files.expect <<-EOF &&
+ 100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0 one
+ EOF
test_cmp ls-files.expect ls-files.actual &&
test-dump-split-index .git/index | sed "/^own/d" >actual &&
- q_to_tab >expect <<EOF &&
-$BASE
-100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0Q
-replacements: 0
-deletions:
-EOF
+ q_to_tab >expect <<-EOF &&
+ $BASE
+ 100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0Q
+ replacements: 0
+ deletions:
+ EOF
test_cmp expect actual
'
@@ -103,54 +103,54 @@ test_expect_success 'add another file, which stays index' '
: >two &&
git update-index --add two &&
git ls-files --stage >ls-files.actual &&
- cat >ls-files.expect <<EOF &&
-100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0 one
-100644 $EMPTY_BLOB 0 two
-EOF
+ cat >ls-files.expect <<-EOF &&
+ 100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0 one
+ 100644 $EMPTY_BLOB 0 two
+ EOF
test_cmp ls-files.expect ls-files.actual &&
test-dump-split-index .git/index | sed "/^own/d" >actual &&
- q_to_tab >expect <<EOF &&
-$BASE
-100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0Q
-100644 $EMPTY_BLOB 0 two
-replacements: 0
-deletions:
-EOF
+ q_to_tab >expect <<-EOF &&
+ $BASE
+ 100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0Q
+ 100644 $EMPTY_BLOB 0 two
+ replacements: 0
+ deletions:
+ EOF
test_cmp expect actual
'
test_expect_success 'remove file not in base index' '
git update-index --force-remove two &&
git ls-files --stage >ls-files.actual &&
- cat >ls-files.expect <<EOF &&
-100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0 one
-EOF
+ cat >ls-files.expect <<-EOF &&
+ 100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0 one
+ EOF
test_cmp ls-files.expect ls-files.actual &&
test-dump-split-index .git/index | sed "/^own/d" >actual &&
- q_to_tab >expect <<EOF &&
-$BASE
-100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0Q
-replacements: 0
-deletions:
-EOF
+ q_to_tab >expect <<-EOF &&
+ $BASE
+ 100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0Q
+ replacements: 0
+ deletions:
+ EOF
test_cmp expect actual
'
test_expect_success 'remove file in base index' '
git update-index --force-remove one &&
git ls-files --stage >ls-files.actual &&
- cat >ls-files.expect <<EOF &&
-EOF
+ cat >ls-files.expect <<-EOF &&
+ EOF
test_cmp ls-files.expect ls-files.actual &&
test-dump-split-index .git/index | sed "/^own/d" >actual &&
- cat >expect <<EOF &&
-$BASE
-replacements:
-deletions: 0
-EOF
+ cat >expect <<-EOF &&
+ $BASE
+ replacements:
+ deletions: 0
+ EOF
test_cmp expect actual
'
@@ -158,18 +158,18 @@ test_expect_success 'add original file back' '
: >one &&
git update-index --add one &&
git ls-files --stage >ls-files.actual &&
- cat >ls-files.expect <<EOF &&
-100644 $EMPTY_BLOB 0 one
-EOF
+ cat >ls-files.expect <<-EOF &&
+ 100644 $EMPTY_BLOB 0 one
+ EOF
test_cmp ls-files.expect ls-files.actual &&
test-dump-split-index .git/index | sed "/^own/d" >actual &&
- cat >expect <<EOF &&
-$BASE
-100644 $EMPTY_BLOB 0 one
-replacements:
-deletions: 0
-EOF
+ cat >expect <<-EOF &&
+ $BASE
+ 100644 $EMPTY_BLOB 0 one
+ replacements:
+ deletions: 0
+ EOF
test_cmp expect actual
'
@@ -177,26 +177,26 @@ test_expect_success 'add new file' '
: >two &&
git update-index --add two &&
git ls-files --stage >actual &&
- cat >expect <<EOF &&
-100644 $EMPTY_BLOB 0 one
-100644 $EMPTY_BLOB 0 two
-EOF
+ cat >expect <<-EOF &&
+ 100644 $EMPTY_BLOB 0 one
+ 100644 $EMPTY_BLOB 0 two
+ EOF
test_cmp expect actual
'
test_expect_success 'unify index, two files remain' '
git update-index --no-split-index &&
git ls-files --stage >ls-files.actual &&
- cat >ls-files.expect <<EOF &&
-100644 $EMPTY_BLOB 0 one
-100644 $EMPTY_BLOB 0 two
-EOF
+ cat >ls-files.expect <<-EOF &&
+ 100644 $EMPTY_BLOB 0 one
+ 100644 $EMPTY_BLOB 0 two
+ EOF
test_cmp ls-files.expect ls-files.actual &&
test-dump-split-index .git/index | sed "/^own/d" >actual &&
- cat >expect <<EOF &&
-not a split index
-EOF
+ cat >expect <<-EOF &&
+ not a split index
+ EOF
test_cmp expect actual
'
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 03/22] config: add git_config_get_split_index()
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
2017-03-06 9:41 ` [PATCH v5 01/22] config: mark an error message up for translation Christian Couder
2017-03-06 9:41 ` [PATCH v5 02/22] t1700: change here document style Christian Couder
@ 2017-03-06 9:41 ` Christian Couder
2017-03-06 9:41 ` [PATCH v5 04/22] split-index: add {add,remove}_split_index() functions Christian Couder
` (18 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:41 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
This new function will be used in a following commit to know
if we want to use the split index feature or not.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
cache.h | 1 +
config.c | 10 ++++++++++
2 files changed, 11 insertions(+)
diff --git a/cache.h b/cache.h
index 80b6372cf7..03de80daae 100644
--- a/cache.h
+++ b/cache.h
@@ -1926,6 +1926,7 @@ extern int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest);
extern int git_config_get_maybe_bool(const char *key, int *dest);
extern int git_config_get_pathname(const char *key, const char **dest);
extern int git_config_get_untracked_cache(void);
+extern int git_config_get_split_index(void);
/*
* This is a hack for test programs like test-dump-untracked-cache to
diff --git a/config.c b/config.c
index 2ac1aa19b0..2a97696be7 100644
--- a/config.c
+++ b/config.c
@@ -1736,6 +1736,16 @@ int git_config_get_untracked_cache(void)
return -1; /* default value */
}
+int git_config_get_split_index(void)
+{
+ int val;
+
+ if (!git_config_get_maybe_bool("core.splitindex", &val))
+ return val;
+
+ return -1; /* default value */
+}
+
NORETURN
void git_die_config_linenr(const char *key, const char *filename, int linenr)
{
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 04/22] split-index: add {add,remove}_split_index() functions
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
` (2 preceding siblings ...)
2017-03-06 9:41 ` [PATCH v5 03/22] config: add git_config_get_split_index() Christian Couder
@ 2017-03-06 9:41 ` Christian Couder
2017-03-06 9:41 ` [PATCH v5 05/22] read-cache: add and then use tweak_split_index() Christian Couder
` (17 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:41 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
Also use the functions in cmd_update_index() in
builtin/update-index.c.
These functions will be used in a following commit to tweak
our use of the split-index feature depending on the setting
of a configuration variable.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
builtin/update-index.c | 18 ++++++------------
split-index.c | 22 ++++++++++++++++++++++
split-index.h | 2 ++
3 files changed, 30 insertions(+), 12 deletions(-)
diff --git a/builtin/update-index.c b/builtin/update-index.c
index d530e89368..24fdadfa4b 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -1099,18 +1099,12 @@ int cmd_update_index(int argc, const char **argv, const char *prefix)
}
if (split_index > 0) {
- init_split_index(&the_index);
- the_index.cache_changed |= SPLIT_INDEX_ORDERED;
- } else if (!split_index && the_index.split_index) {
- /*
- * can't discard_split_index(&the_index); because that
- * will destroy split_index->base->cache[], which may
- * be shared with the_index.cache[]. So yeah we're
- * leaking a bit here.
- */
- the_index.split_index = NULL;
- the_index.cache_changed |= SOMETHING_CHANGED;
- }
+ if (the_index.split_index)
+ the_index.cache_changed |= SPLIT_INDEX_ORDERED;
+ else
+ add_split_index(&the_index);
+ } else if (!split_index)
+ remove_split_index(&the_index);
switch (untracked_cache) {
case UC_UNSPECIFIED:
diff --git a/split-index.c b/split-index.c
index 615f4cac05..f519e60f87 100644
--- a/split-index.c
+++ b/split-index.c
@@ -317,3 +317,25 @@ void replace_index_entry_in_base(struct index_state *istate,
istate->split_index->base->cache[new->index - 1] = new;
}
}
+
+void add_split_index(struct index_state *istate)
+{
+ if (!istate->split_index) {
+ init_split_index(istate);
+ istate->cache_changed |= SPLIT_INDEX_ORDERED;
+ }
+}
+
+void remove_split_index(struct index_state *istate)
+{
+ if (istate->split_index) {
+ /*
+ * can't discard_split_index(&the_index); because that
+ * will destroy split_index->base->cache[], which may
+ * be shared with the_index.cache[]. So yeah we're
+ * leaking a bit here.
+ */
+ istate->split_index = NULL;
+ istate->cache_changed |= SOMETHING_CHANGED;
+ }
+}
diff --git a/split-index.h b/split-index.h
index c1324f521a..df91c1bda8 100644
--- a/split-index.h
+++ b/split-index.h
@@ -31,5 +31,7 @@ void merge_base_index(struct index_state *istate);
void prepare_to_write_split_index(struct index_state *istate);
void finish_writing_split_index(struct index_state *istate);
void discard_split_index(struct index_state *istate);
+void add_split_index(struct index_state *istate);
+void remove_split_index(struct index_state *istate);
#endif
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 05/22] read-cache: add and then use tweak_split_index()
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
` (3 preceding siblings ...)
2017-03-06 9:41 ` [PATCH v5 04/22] split-index: add {add,remove}_split_index() functions Christian Couder
@ 2017-03-06 9:41 ` Christian Couder
2017-03-06 9:41 ` [PATCH v5 06/22] update-index: warn in case of split-index incoherency Christian Couder
` (16 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:41 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
This will make us use the split-index feature or not depending
on the value of the "core.splitIndex" config variable.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
read-cache.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/read-cache.c b/read-cache.c
index 9054369dd0..99bc274b8d 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1558,10 +1558,27 @@ static void tweak_untracked_cache(struct index_state *istate)
}
}
+static void tweak_split_index(struct index_state *istate)
+{
+ switch (git_config_get_split_index()) {
+ case -1: /* unset: do nothing */
+ break;
+ case 0: /* false */
+ remove_split_index(istate);
+ break;
+ case 1: /* true */
+ add_split_index(istate);
+ break;
+ default: /* unknown value: do nothing */
+ break;
+ }
+}
+
static void post_read_index_from(struct index_state *istate)
{
check_ce_order(istate);
tweak_untracked_cache(istate);
+ tweak_split_index(istate);
}
/* remember to discard_cache() before reading a different cache! */
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 06/22] update-index: warn in case of split-index incoherency
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
` (4 preceding siblings ...)
2017-03-06 9:41 ` [PATCH v5 05/22] read-cache: add and then use tweak_split_index() Christian Couder
@ 2017-03-06 9:41 ` Christian Couder
2017-03-06 9:41 ` [PATCH v5 07/22] t1700: add tests for core.splitIndex Christian Couder
` (15 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:41 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
When users are using `git update-index --(no-)split-index`, they
may expect the split-index feature to be used or not according to
the option they just used, but this might not be the case if the
new "core.splitIndex" config variable has been set. In this case
let's warn about what will happen and why.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
builtin/update-index.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/builtin/update-index.c b/builtin/update-index.c
index 24fdadfa4b..d74d72cc7f 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -1099,12 +1099,21 @@ int cmd_update_index(int argc, const char **argv, const char *prefix)
}
if (split_index > 0) {
+ if (git_config_get_split_index() == 0)
+ warning(_("core.splitIndex is set to false; "
+ "remove or change it, if you really want to "
+ "enable split index"));
if (the_index.split_index)
the_index.cache_changed |= SPLIT_INDEX_ORDERED;
else
add_split_index(&the_index);
- } else if (!split_index)
+ } else if (!split_index) {
+ if (git_config_get_split_index() == 1)
+ warning(_("core.splitIndex is set to true; "
+ "remove or change it, if you really want to "
+ "disable split index"));
remove_split_index(&the_index);
+ }
switch (untracked_cache) {
case UC_UNSPECIFIED:
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 07/22] t1700: add tests for core.splitIndex
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
` (5 preceding siblings ...)
2017-03-06 9:41 ` [PATCH v5 06/22] update-index: warn in case of split-index incoherency Christian Couder
@ 2017-03-06 9:41 ` Christian Couder
2017-03-06 9:41 ` [PATCH v5 08/22] Documentation/config: add information " Christian Couder
` (14 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:41 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
t/t1700-split-index.sh | 37 +++++++++++++++++++++++++++++++++++++
1 file changed, 37 insertions(+)
diff --git a/t/t1700-split-index.sh b/t/t1700-split-index.sh
index 5ea227e6a1..aa2aff1778 100755
--- a/t/t1700-split-index.sh
+++ b/t/t1700-split-index.sh
@@ -216,4 +216,41 @@ test_expect_success 'rev-parse --shared-index-path' '
)
'
+test_expect_success 'set core.splitIndex config variable to true' '
+ git config core.splitIndex true &&
+ : >three &&
+ git update-index --add three &&
+ git ls-files --stage >ls-files.actual &&
+ cat >ls-files.expect <<-EOF &&
+ 100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0 one
+ 100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0 three
+ 100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0 two
+ EOF
+ test_cmp ls-files.expect ls-files.actual &&
+ BASE=$(test-dump-split-index .git/index | grep "^base") &&
+ test-dump-split-index .git/index | sed "/^own/d" >actual &&
+ cat >expect <<-EOF &&
+ $BASE
+ replacements:
+ deletions:
+ EOF
+ test_cmp expect actual
+'
+
+test_expect_success 'set core.splitIndex config variable to false' '
+ git config core.splitIndex false &&
+ git update-index --force-remove three &&
+ git ls-files --stage >ls-files.actual &&
+ cat >ls-files.expect <<-EOF &&
+ 100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0 one
+ 100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0 two
+ EOF
+ test_cmp ls-files.expect ls-files.actual &&
+ test-dump-split-index .git/index | sed "/^own/d" >actual &&
+ cat >expect <<-EOF &&
+ not a split index
+ EOF
+ test_cmp expect actual
+'
+
test_done
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 08/22] Documentation/config: add information for core.splitIndex
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
` (6 preceding siblings ...)
2017-03-06 9:41 ` [PATCH v5 07/22] t1700: add tests for core.splitIndex Christian Couder
@ 2017-03-06 9:41 ` Christian Couder
2017-03-06 9:41 ` [PATCH v5 09/22] Documentation/git-update-index: talk about core.splitIndex config var Christian Couder
` (13 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:41 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
Documentation/config.txt | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index 47603f5484..f102879261 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -334,6 +334,10 @@ core.trustctime::
crawlers and some backup systems).
See linkgit:git-update-index[1]. True by default.
+core.splitIndex::
+ If true, the split-index feature of the index will be used.
+ See linkgit:git-update-index[1]. False by default.
+
core.untrackedCache::
Determines what to do about the untracked cache feature of the
index. It will be kept, if this variable is unset or set to
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 09/22] Documentation/git-update-index: talk about core.splitIndex config var
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
` (7 preceding siblings ...)
2017-03-06 9:41 ` [PATCH v5 08/22] Documentation/config: add information " Christian Couder
@ 2017-03-06 9:41 ` Christian Couder
2017-03-06 9:41 ` [PATCH v5 10/22] config: add git_config_get_max_percent_split_change() Christian Couder
` (12 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:41 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
Documentation/git-update-index.txt | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/Documentation/git-update-index.txt b/Documentation/git-update-index.txt
index 7386c93162..e091b2a409 100644
--- a/Documentation/git-update-index.txt
+++ b/Documentation/git-update-index.txt
@@ -171,6 +171,12 @@ may not support it yet.
given again, all changes in $GIT_DIR/index are pushed back to
the shared index file. This mode is designed for very large
indexes that take a significant amount of time to read or write.
++
+These options take effect whatever the value of the `core.splitIndex`
+configuration variable (see linkgit:git-config[1]). But a warning is
+emitted when the change goes against the configured value, as the
+configured value will take effect next time the index is read and this
+will remove the intended effect of the option.
--untracked-cache::
--no-untracked-cache::
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 10/22] config: add git_config_get_max_percent_split_change()
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
` (8 preceding siblings ...)
2017-03-06 9:41 ` [PATCH v5 09/22] Documentation/git-update-index: talk about core.splitIndex config var Christian Couder
@ 2017-03-06 9:41 ` Christian Couder
2017-03-06 9:41 ` [PATCH v5 11/22] read-cache: regenerate shared index if necessary Christian Couder
` (11 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:41 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
This new function will be used in a following commit to get the
value of the "splitIndex.maxPercentChange" config variable.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
cache.h | 1 +
config.c | 15 +++++++++++++++
2 files changed, 16 insertions(+)
diff --git a/cache.h b/cache.h
index 03de80daae..0bb9adcd31 100644
--- a/cache.h
+++ b/cache.h
@@ -1927,6 +1927,7 @@ extern int git_config_get_maybe_bool(const char *key, int *dest);
extern int git_config_get_pathname(const char *key, const char **dest);
extern int git_config_get_untracked_cache(void);
extern int git_config_get_split_index(void);
+extern int git_config_get_max_percent_split_change(void);
/*
* This is a hack for test programs like test-dump-untracked-cache to
diff --git a/config.c b/config.c
index 2a97696be7..35b6f02960 100644
--- a/config.c
+++ b/config.c
@@ -1746,6 +1746,21 @@ int git_config_get_split_index(void)
return -1; /* default value */
}
+int git_config_get_max_percent_split_change(void)
+{
+ int val = -1;
+
+ if (!git_config_get_int("splitindex.maxpercentchange", &val)) {
+ if (0 <= val && val <= 100)
+ return val;
+
+ return error(_("splitIndex.maxPercentChange value '%d' "
+ "should be between 0 and 100"), val);
+ }
+
+ return -1; /* default value */
+}
+
NORETURN
void git_die_config_linenr(const char *key, const char *filename, int linenr)
{
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 11/22] read-cache: regenerate shared index if necessary
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
` (9 preceding siblings ...)
2017-03-06 9:41 ` [PATCH v5 10/22] config: add git_config_get_max_percent_split_change() Christian Couder
@ 2017-03-06 9:41 ` Christian Couder
2017-03-06 9:41 ` [PATCH v5 12/22] t1700: add tests for splitIndex.maxPercentChange Christian Couder
` (10 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:41 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
When writing a new split-index and there is a big number of cache
entries in the split-index compared to the shared index, it is a
good idea to regenerate the shared index.
By default when the ratio reaches 20%, we will push back all
the entries from the split-index into a new shared index file
instead of just creating a new split-index file.
The threshold can be configured using the
"splitIndex.maxPercentChange" config variable.
We need to adjust the existing tests in t1700 by setting
"splitIndex.maxPercentChange" to 100 at the beginning of t1700,
as the existing tests are assuming that the shared index is
regenerated only when `git update-index --split-index` is used.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
read-cache.c | 32 ++++++++++++++++++++++++++++++++
t/t1700-split-index.sh | 1 +
2 files changed, 33 insertions(+)
diff --git a/read-cache.c b/read-cache.c
index 99bc274b8d..aeb413a508 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -2212,6 +2212,36 @@ static int write_shared_index(struct index_state *istate,
return ret;
}
+static const int default_max_percent_split_change = 20;
+
+static int too_many_not_shared_entries(struct index_state *istate)
+{
+ int i, not_shared = 0;
+ int max_split = git_config_get_max_percent_split_change();
+
+ switch (max_split) {
+ case -1:
+ /* not or badly configured: use the default value */
+ max_split = default_max_percent_split_change;
+ break;
+ case 0:
+ return 1; /* 0% means always write a new shared index */
+ case 100:
+ return 0; /* 100% means never write a new shared index */
+ default:
+ break; /* just use the configured value */
+ }
+
+ /* Count not shared entries */
+ for (i = 0; i < istate->cache_nr; i++) {
+ struct cache_entry *ce = istate->cache[i];
+ if (!ce->index)
+ not_shared++;
+ }
+
+ return (int64_t)istate->cache_nr * max_split < (int64_t)not_shared * 100;
+}
+
int write_locked_index(struct index_state *istate, struct lock_file *lock,
unsigned flags)
{
@@ -2229,6 +2259,8 @@ int write_locked_index(struct index_state *istate, struct lock_file *lock,
if ((v & 15) < 6)
istate->cache_changed |= SPLIT_INDEX_ORDERED;
}
+ if (too_many_not_shared_entries(istate))
+ istate->cache_changed |= SPLIT_INDEX_ORDERED;
if (istate->cache_changed & SPLIT_INDEX_ORDERED) {
int ret = write_shared_index(istate, lock, flags);
if (ret)
diff --git a/t/t1700-split-index.sh b/t/t1700-split-index.sh
index aa2aff1778..9d7c01c3e1 100755
--- a/t/t1700-split-index.sh
+++ b/t/t1700-split-index.sh
@@ -8,6 +8,7 @@ test_description='split index mode tests'
sane_unset GIT_TEST_SPLIT_INDEX
test_expect_success 'enable split index' '
+ git config splitIndex.maxPercentChange 100 &&
git update-index --split-index &&
test-dump-split-index .git/index >actual &&
indexversion=$(test-index-version <.git/index) &&
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 12/22] t1700: add tests for splitIndex.maxPercentChange
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
` (10 preceding siblings ...)
2017-03-06 9:41 ` [PATCH v5 11/22] read-cache: regenerate shared index if necessary Christian Couder
@ 2017-03-06 9:41 ` Christian Couder
2017-03-06 9:41 ` [PATCH v5 13/22] Documentation/config: add splitIndex.maxPercentChange Christian Couder
` (9 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:41 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
t/t1700-split-index.sh | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 72 insertions(+)
diff --git a/t/t1700-split-index.sh b/t/t1700-split-index.sh
index 9d7c01c3e1..00a64bed97 100755
--- a/t/t1700-split-index.sh
+++ b/t/t1700-split-index.sh
@@ -254,4 +254,76 @@ test_expect_success 'set core.splitIndex config variable to false' '
test_cmp expect actual
'
+test_expect_success 'set core.splitIndex config variable to true' '
+ git config core.splitIndex true &&
+ : >three &&
+ git update-index --add three &&
+ BASE=$(test-dump-split-index .git/index | grep "^base") &&
+ test-dump-split-index .git/index | sed "/^own/d" >actual &&
+ cat >expect <<-EOF &&
+ $BASE
+ replacements:
+ deletions:
+ EOF
+ test_cmp expect actual &&
+ : >four &&
+ git update-index --add four &&
+ test-dump-split-index .git/index | sed "/^own/d" >actual &&
+ cat >expect <<-EOF &&
+ $BASE
+ 100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0 four
+ replacements:
+ deletions:
+ EOF
+ test_cmp expect actual
+'
+
+test_expect_success 'check behavior with splitIndex.maxPercentChange unset' '
+ git config --unset splitIndex.maxPercentChange &&
+ : >five &&
+ git update-index --add five &&
+ BASE=$(test-dump-split-index .git/index | grep "^base") &&
+ test-dump-split-index .git/index | sed "/^own/d" >actual &&
+ cat >expect <<-EOF &&
+ $BASE
+ replacements:
+ deletions:
+ EOF
+ test_cmp expect actual &&
+ : >six &&
+ git update-index --add six &&
+ test-dump-split-index .git/index | sed "/^own/d" >actual &&
+ cat >expect <<-EOF &&
+ $BASE
+ 100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0 six
+ replacements:
+ deletions:
+ EOF
+ test_cmp expect actual
+'
+
+test_expect_success 'check splitIndex.maxPercentChange set to 0' '
+ git config splitIndex.maxPercentChange 0 &&
+ : >seven &&
+ git update-index --add seven &&
+ BASE=$(test-dump-split-index .git/index | grep "^base") &&
+ test-dump-split-index .git/index | sed "/^own/d" >actual &&
+ cat >expect <<-EOF &&
+ $BASE
+ replacements:
+ deletions:
+ EOF
+ test_cmp expect actual &&
+ : >eight &&
+ git update-index --add eight &&
+ BASE=$(test-dump-split-index .git/index | grep "^base") &&
+ test-dump-split-index .git/index | sed "/^own/d" >actual &&
+ cat >expect <<-EOF &&
+ $BASE
+ replacements:
+ deletions:
+ EOF
+ test_cmp expect actual
+'
+
test_done
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 13/22] Documentation/config: add splitIndex.maxPercentChange
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
` (11 preceding siblings ...)
2017-03-06 9:41 ` [PATCH v5 12/22] t1700: add tests for splitIndex.maxPercentChange Christian Couder
@ 2017-03-06 9:41 ` Christian Couder
2017-03-06 9:41 ` [PATCH v5 14/22] sha1_file: make check_and_freshen_file() non static Christian Couder
` (8 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:41 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
Documentation/config.txt | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index f102879261..b64aa7db2d 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -2851,6 +2851,19 @@ showbranch.default::
The default set of branches for linkgit:git-show-branch[1].
See linkgit:git-show-branch[1].
+splitIndex.maxPercentChange::
+ When the split index feature is used, this specifies the
+ percent of entries the split index can contain compared to the
+ total number of entries in both the split index and the shared
+ index before a new shared index is written.
+ The value should be between 0 and 100. If the value is 0 then
+ a new shared index is always written, if it is 100 a new
+ shared index is never written.
+ By default the value is 20, so a new shared index is written
+ if the number of entries in the split index would be greater
+ than 20 percent of the total number of entries.
+ See linkgit:git-update-index[1].
+
status.relativePaths::
By default, linkgit:git-status[1] shows paths relative to the
current directory. Setting this variable to `false` shows paths
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 14/22] sha1_file: make check_and_freshen_file() non static
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
` (12 preceding siblings ...)
2017-03-06 9:41 ` [PATCH v5 13/22] Documentation/config: add splitIndex.maxPercentChange Christian Couder
@ 2017-03-06 9:41 ` Christian Couder
2017-03-06 9:41 ` [PATCH v5 15/22] read-cache: touch shared index files when used Christian Couder
` (7 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:41 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
This function will be used in a commit soon, so let's make
it available globally.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
cache.h | 3 +++
sha1_file.c | 2 +-
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/cache.h b/cache.h
index 0bb9adcd31..a35e9d5187 100644
--- a/cache.h
+++ b/cache.h
@@ -1273,6 +1273,9 @@ extern int has_pack_index(const unsigned char *sha1);
extern void assert_sha1_type(const unsigned char *sha1, enum object_type expect);
+/* Helper to check and "touch" a file */
+extern int check_and_freshen_file(const char *fn, int freshen);
+
extern const signed char hexval_table[256];
static inline unsigned int hexval(unsigned char c)
{
diff --git a/sha1_file.c b/sha1_file.c
index 6628f06da3..f02f10189f 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -667,7 +667,7 @@ static int freshen_file(const char *fn)
* either does not exist on disk, or has a stale mtime and may be subject to
* pruning).
*/
-static int check_and_freshen_file(const char *fn, int freshen)
+int check_and_freshen_file(const char *fn, int freshen)
{
if (access(fn, F_OK))
return 0;
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 15/22] read-cache: touch shared index files when used
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
` (13 preceding siblings ...)
2017-03-06 9:41 ` [PATCH v5 14/22] sha1_file: make check_and_freshen_file() non static Christian Couder
@ 2017-03-06 9:41 ` Christian Couder
2017-03-06 9:41 ` [PATCH v5 16/22] config: add git_config_get_expiry() from gc.c Christian Couder
` (6 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:41 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
When a split-index file is created, let's update the mtime of the
shared index file that the split-index file is referencing.
In a following commit we will make shared index file expire
depending on their mtime, so updating the mtime makes sure that
the shared index file will not be deleted soon.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
read-cache.c | 29 ++++++++++++++++++++++++++---
1 file changed, 26 insertions(+), 3 deletions(-)
diff --git a/read-cache.c b/read-cache.c
index aeb413a508..13375fa0ff 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1674,6 +1674,19 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist)
die("index file corrupt");
}
+/*
+ * Signal that the shared index is used by updating its mtime.
+ *
+ * This way, shared index can be removed if they have not been used
+ * for some time.
+ */
+static void freshen_shared_index(char *base_sha1_hex, int warn)
+{
+ const char *shared_index = git_path("sharedindex.%s", base_sha1_hex);
+ if (!check_and_freshen_file(shared_index, 1) && warn)
+ warning("could not freshen shared index '%s'", shared_index);
+}
+
int read_index_from(struct index_state *istate, const char *path)
{
struct split_index *split_index;
@@ -2245,6 +2258,7 @@ static int too_many_not_shared_entries(struct index_state *istate)
int write_locked_index(struct index_state *istate, struct lock_file *lock,
unsigned flags)
{
+ int new_shared_index, ret;
struct split_index *si = istate->split_index;
if (!si || alternate_index_output ||
@@ -2261,13 +2275,22 @@ int write_locked_index(struct index_state *istate, struct lock_file *lock,
}
if (too_many_not_shared_entries(istate))
istate->cache_changed |= SPLIT_INDEX_ORDERED;
- if (istate->cache_changed & SPLIT_INDEX_ORDERED) {
- int ret = write_shared_index(istate, lock, flags);
+
+ new_shared_index = istate->cache_changed & SPLIT_INDEX_ORDERED;
+
+ if (new_shared_index) {
+ ret = write_shared_index(istate, lock, flags);
if (ret)
return ret;
}
- return write_split_index(istate, lock, flags);
+ ret = write_split_index(istate, lock, flags);
+
+ /* Freshen the shared index only if the split-index was written */
+ if (!ret && !new_shared_index)
+ freshen_shared_index(sha1_to_hex(si->base_sha1), 1);
+
+ return ret;
}
/*
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 16/22] config: add git_config_get_expiry() from gc.c
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
` (14 preceding siblings ...)
2017-03-06 9:41 ` [PATCH v5 15/22] read-cache: touch shared index files when used Christian Couder
@ 2017-03-06 9:41 ` Christian Couder
2017-03-06 9:41 ` [PATCH v5 17/22] read-cache: unlink old sharedindex files Christian Couder
` (5 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:41 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
This function will be used in a following commit to get the expiration
time of the shared index files from the config, and it is generic
enough to be put in "config.c".
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
builtin/gc.c | 18 +++---------------
cache.h | 3 +++
config.c | 13 +++++++++++++
3 files changed, 19 insertions(+), 15 deletions(-)
diff --git a/builtin/gc.c b/builtin/gc.c
index a2b9e8924e..56ab74f6ba 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -64,17 +64,6 @@ static void report_pack_garbage(unsigned seen_bits, const char *path)
string_list_append(&pack_garbage, path);
}
-static void git_config_date_string(const char *key, const char **output)
-{
- if (git_config_get_string_const(key, output))
- return;
- if (strcmp(*output, "now")) {
- unsigned long now = approxidate("now");
- if (approxidate(*output) >= now)
- git_die_config(key, _("Invalid %s: '%s'"), key, *output);
- }
-}
-
static void process_log_file(void)
{
struct stat st;
@@ -131,10 +120,9 @@ static void gc_config(void)
git_config_get_int("gc.auto", &gc_auto_threshold);
git_config_get_int("gc.autopacklimit", &gc_auto_pack_limit);
git_config_get_bool("gc.autodetach", &detach_auto);
- git_config_date_string("gc.pruneexpire", &prune_expire);
- git_config_date_string("gc.worktreepruneexpire", &prune_worktrees_expire);
- git_config_date_string("gc.logexpiry", &gc_log_expire);
-
+ git_config_get_expiry("gc.pruneexpire", &prune_expire);
+ git_config_get_expiry("gc.worktreepruneexpire", &prune_worktrees_expire);
+ git_config_get_expiry("gc.logexpiry", &gc_log_expire);
git_config(git_default_config, NULL);
}
diff --git a/cache.h b/cache.h
index a35e9d5187..65ab507a76 100644
--- a/cache.h
+++ b/cache.h
@@ -1932,6 +1932,9 @@ extern int git_config_get_untracked_cache(void);
extern int git_config_get_split_index(void);
extern int git_config_get_max_percent_split_change(void);
+/* This dies if the configured or default date is in the future */
+extern int git_config_get_expiry(const char *key, const char **output);
+
/*
* This is a hack for test programs like test-dump-untracked-cache to
* ensure that they do not modify the untracked cache when reading it.
diff --git a/config.c b/config.c
index 35b6f02960..f20d7d88f7 100644
--- a/config.c
+++ b/config.c
@@ -1712,6 +1712,19 @@ int git_config_get_pathname(const char *key, const char **dest)
return ret;
}
+int git_config_get_expiry(const char *key, const char **output)
+{
+ int ret = git_config_get_string_const(key, output);
+ if (ret)
+ return ret;
+ if (strcmp(*output, "now")) {
+ unsigned long now = approxidate("now");
+ if (approxidate(*output) >= now)
+ git_die_config(key, _("Invalid %s: '%s'"), key, *output);
+ }
+ return ret;
+}
+
int git_config_get_untracked_cache(void)
{
int val = -1;
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 17/22] read-cache: unlink old sharedindex files
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
` (15 preceding siblings ...)
2017-03-06 9:41 ` [PATCH v5 16/22] config: add git_config_get_expiry() from gc.c Christian Couder
@ 2017-03-06 9:41 ` Christian Couder
2017-03-06 9:41 ` [PATCH v5 18/22] t1700: test shared index file expiration Christian Couder
` (4 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:41 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
Everytime split index is turned on, it creates a "sharedindex.XXXX"
file in the git directory. This change makes sure that shared index
files that haven't been used for a long time are removed when a new
shared index file is created.
The new "splitIndex.sharedIndexExpire" config variable is created
to tell the delay after which an unused shared index file can be
deleted. It defaults to "2.weeks.ago".
A previous commit made sure that each time a split index file is
created the mtime of the shared index file it references is updated.
This makes sure that recently used shared index file will not be
deleted.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
read-cache.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 63 insertions(+), 1 deletion(-)
diff --git a/read-cache.c b/read-cache.c
index 13375fa0ff..16c05f359b 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -2199,6 +2199,65 @@ static int write_split_index(struct index_state *istate,
return ret;
}
+static const char *shared_index_expire = "2.weeks.ago";
+
+static unsigned long get_shared_index_expire_date(void)
+{
+ static unsigned long shared_index_expire_date;
+ static int shared_index_expire_date_prepared;
+
+ if (!shared_index_expire_date_prepared) {
+ git_config_get_expiry("splitindex.sharedindexexpire",
+ &shared_index_expire);
+ shared_index_expire_date = approxidate(shared_index_expire);
+ shared_index_expire_date_prepared = 1;
+ }
+
+ return shared_index_expire_date;
+}
+
+static int should_delete_shared_index(const char *shared_index_path)
+{
+ struct stat st;
+ unsigned long expiration;
+
+ /* Check timestamp */
+ expiration = get_shared_index_expire_date();
+ if (!expiration)
+ return 0;
+ if (stat(shared_index_path, &st))
+ return error_errno(_("could not stat '%s"), shared_index_path);
+ if (st.st_mtime > expiration)
+ return 0;
+
+ return 1;
+}
+
+static int clean_shared_index_files(const char *current_hex)
+{
+ struct dirent *de;
+ DIR *dir = opendir(get_git_dir());
+
+ if (!dir)
+ return error_errno(_("unable to open git dir: %s"), get_git_dir());
+
+ while ((de = readdir(dir)) != NULL) {
+ const char *sha1_hex;
+ const char *shared_index_path;
+ if (!skip_prefix(de->d_name, "sharedindex.", &sha1_hex))
+ continue;
+ if (!strcmp(sha1_hex, current_hex))
+ continue;
+ shared_index_path = git_path("%s", de->d_name);
+ if (should_delete_shared_index(shared_index_path) > 0 &&
+ unlink(shared_index_path))
+ warning_errno(_("unable to unlink: %s"), shared_index_path);
+ }
+ closedir(dir);
+
+ return 0;
+}
+
static struct tempfile temporary_sharedindex;
static int write_shared_index(struct index_state *istate,
@@ -2220,8 +2279,11 @@ static int write_shared_index(struct index_state *istate,
}
ret = rename_tempfile(&temporary_sharedindex,
git_path("sharedindex.%s", sha1_to_hex(si->base->sha1)));
- if (!ret)
+ if (!ret) {
hashcpy(si->base_sha1, si->base->sha1);
+ clean_shared_index_files(sha1_to_hex(si->base->sha1));
+ }
+
return ret;
}
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 18/22] t1700: test shared index file expiration
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
` (16 preceding siblings ...)
2017-03-06 9:41 ` [PATCH v5 17/22] read-cache: unlink old sharedindex files Christian Couder
@ 2017-03-06 9:41 ` Christian Couder
2017-03-06 9:42 ` [PATCH v5 19/22] read-cache: refactor read_index_from() Christian Couder
` (3 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:41 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
t/t1700-split-index.sh | 44 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 44 insertions(+)
diff --git a/t/t1700-split-index.sh b/t/t1700-split-index.sh
index 00a64bed97..f5a95a6c28 100755
--- a/t/t1700-split-index.sh
+++ b/t/t1700-split-index.sh
@@ -326,4 +326,48 @@ test_expect_success 'check splitIndex.maxPercentChange set to 0' '
test_cmp expect actual
'
+test_expect_success 'shared index files expire after 2 weeks by default' '
+ : >ten &&
+ git update-index --add ten &&
+ test $(ls .git/sharedindex.* | wc -l) -gt 1 &&
+ just_under_2_weeks_ago=$((5-14*86400)) &&
+ test-chmtime =$just_under_2_weeks_ago .git/sharedindex.* &&
+ : >eleven &&
+ git update-index --add eleven &&
+ test $(ls .git/sharedindex.* | wc -l) -gt 1 &&
+ just_over_2_weeks_ago=$((-1-14*86400)) &&
+ test-chmtime =$just_over_2_weeks_ago .git/sharedindex.* &&
+ : >twelve &&
+ git update-index --add twelve &&
+ test $(ls .git/sharedindex.* | wc -l) = 1
+'
+
+test_expect_success 'check splitIndex.sharedIndexExpire set to 16 days' '
+ git config splitIndex.sharedIndexExpire "16.days.ago" &&
+ test-chmtime =$just_over_2_weeks_ago .git/sharedindex.* &&
+ : >thirteen &&
+ git update-index --add thirteen &&
+ test $(ls .git/sharedindex.* | wc -l) -gt 1 &&
+ just_over_16_days_ago=$((-1-16*86400)) &&
+ test-chmtime =$just_over_16_days_ago .git/sharedindex.* &&
+ : >fourteen &&
+ git update-index --add fourteen &&
+ test $(ls .git/sharedindex.* | wc -l) = 1
+'
+
+test_expect_success 'check splitIndex.sharedIndexExpire set to "never" and "now"' '
+ git config splitIndex.sharedIndexExpire never &&
+ just_10_years_ago=$((-365*10*86400)) &&
+ test-chmtime =$just_10_years_ago .git/sharedindex.* &&
+ : >fifteen &&
+ git update-index --add fifteen &&
+ test $(ls .git/sharedindex.* | wc -l) -gt 1 &&
+ git config splitIndex.sharedIndexExpire now &&
+ just_1_second_ago=-1 &&
+ test-chmtime =$just_1_second_ago .git/sharedindex.* &&
+ : >sixteen &&
+ git update-index --add sixteen &&
+ test $(ls .git/sharedindex.* | wc -l) = 1
+'
+
test_done
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 19/22] read-cache: refactor read_index_from()
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
` (17 preceding siblings ...)
2017-03-06 9:41 ` [PATCH v5 18/22] t1700: test shared index file expiration Christian Couder
@ 2017-03-06 9:42 ` Christian Couder
2017-03-06 9:42 ` [PATCH v5 20/22] read-cache: use freshen_shared_index() in read_index_from() Christian Couder
` (2 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:42 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
It looks better and is simpler to review when we don't compute
the same things many times in the function.
It will also help make the following commit simpler.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
read-cache.c | 14 ++++++++------
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git a/read-cache.c b/read-cache.c
index 16c05f359b..89c95d59b3 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1691,6 +1691,8 @@ int read_index_from(struct index_state *istate, const char *path)
{
struct split_index *split_index;
int ret;
+ char *base_sha1_hex;
+ const char *base_path;
/* istate->initialized covers both .git/index and .git/sharedindex.xxx */
if (istate->initialized)
@@ -1708,15 +1710,15 @@ int read_index_from(struct index_state *istate, const char *path)
discard_index(split_index->base);
else
split_index->base = xcalloc(1, sizeof(*split_index->base));
- ret = do_read_index(split_index->base,
- git_path("sharedindex.%s",
- sha1_to_hex(split_index->base_sha1)), 1);
+
+ base_sha1_hex = sha1_to_hex(split_index->base_sha1);
+ base_path = git_path("sharedindex.%s", base_sha1_hex);
+ ret = do_read_index(split_index->base, base_path, 1);
if (hashcmp(split_index->base_sha1, split_index->base->sha1))
die("broken index, expect %s in %s, got %s",
- sha1_to_hex(split_index->base_sha1),
- git_path("sharedindex.%s",
- sha1_to_hex(split_index->base_sha1)),
+ base_sha1_hex, base_path,
sha1_to_hex(split_index->base->sha1));
+
merge_base_index(istate);
post_read_index_from(istate);
return ret;
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 20/22] read-cache: use freshen_shared_index() in read_index_from()
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
` (18 preceding siblings ...)
2017-03-06 9:42 ` [PATCH v5 19/22] read-cache: refactor read_index_from() Christian Couder
@ 2017-03-06 9:42 ` Christian Couder
2017-03-06 9:42 ` [PATCH v5 21/22] Documentation/config: add splitIndex.sharedIndexExpire Christian Couder
2017-03-06 9:42 ` [PATCH v5 22/22] Documentation/git-update-index: explain splitIndex.* Christian Couder
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:42 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
This way a share index file will not be garbage collected if
we still read from an index it is based from.
As we need to read the current index before creating a new
one, the tests have to be adjusted, so that we don't expect
an old shared index file to be deleted right away when we
create a new one.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
read-cache.c | 1 +
t/t1700-split-index.sh | 14 +++++++-------
2 files changed, 8 insertions(+), 7 deletions(-)
diff --git a/read-cache.c b/read-cache.c
index 89c95d59b3..e447751823 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1719,6 +1719,7 @@ int read_index_from(struct index_state *istate, const char *path)
base_sha1_hex, base_path,
sha1_to_hex(split_index->base->sha1));
+ freshen_shared_index(base_sha1_hex, 0);
merge_base_index(istate);
post_read_index_from(istate);
return ret;
diff --git a/t/t1700-split-index.sh b/t/t1700-split-index.sh
index f5a95a6c28..af3ec0da5a 100755
--- a/t/t1700-split-index.sh
+++ b/t/t1700-split-index.sh
@@ -329,17 +329,17 @@ test_expect_success 'check splitIndex.maxPercentChange set to 0' '
test_expect_success 'shared index files expire after 2 weeks by default' '
: >ten &&
git update-index --add ten &&
- test $(ls .git/sharedindex.* | wc -l) -gt 1 &&
+ test $(ls .git/sharedindex.* | wc -l) -gt 2 &&
just_under_2_weeks_ago=$((5-14*86400)) &&
test-chmtime =$just_under_2_weeks_ago .git/sharedindex.* &&
: >eleven &&
git update-index --add eleven &&
- test $(ls .git/sharedindex.* | wc -l) -gt 1 &&
+ test $(ls .git/sharedindex.* | wc -l) -gt 2 &&
just_over_2_weeks_ago=$((-1-14*86400)) &&
test-chmtime =$just_over_2_weeks_ago .git/sharedindex.* &&
: >twelve &&
git update-index --add twelve &&
- test $(ls .git/sharedindex.* | wc -l) = 1
+ test $(ls .git/sharedindex.* | wc -l) -le 2
'
test_expect_success 'check splitIndex.sharedIndexExpire set to 16 days' '
@@ -347,12 +347,12 @@ test_expect_success 'check splitIndex.sharedIndexExpire set to 16 days' '
test-chmtime =$just_over_2_weeks_ago .git/sharedindex.* &&
: >thirteen &&
git update-index --add thirteen &&
- test $(ls .git/sharedindex.* | wc -l) -gt 1 &&
+ test $(ls .git/sharedindex.* | wc -l) -gt 2 &&
just_over_16_days_ago=$((-1-16*86400)) &&
test-chmtime =$just_over_16_days_ago .git/sharedindex.* &&
: >fourteen &&
git update-index --add fourteen &&
- test $(ls .git/sharedindex.* | wc -l) = 1
+ test $(ls .git/sharedindex.* | wc -l) -le 2
'
test_expect_success 'check splitIndex.sharedIndexExpire set to "never" and "now"' '
@@ -361,13 +361,13 @@ test_expect_success 'check splitIndex.sharedIndexExpire set to "never" and "now"
test-chmtime =$just_10_years_ago .git/sharedindex.* &&
: >fifteen &&
git update-index --add fifteen &&
- test $(ls .git/sharedindex.* | wc -l) -gt 1 &&
+ test $(ls .git/sharedindex.* | wc -l) -gt 2 &&
git config splitIndex.sharedIndexExpire now &&
just_1_second_ago=-1 &&
test-chmtime =$just_1_second_ago .git/sharedindex.* &&
: >sixteen &&
git update-index --add sixteen &&
- test $(ls .git/sharedindex.* | wc -l) = 1
+ test $(ls .git/sharedindex.* | wc -l) -le 2
'
test_done
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 21/22] Documentation/config: add splitIndex.sharedIndexExpire
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
` (19 preceding siblings ...)
2017-03-06 9:42 ` [PATCH v5 20/22] read-cache: use freshen_shared_index() in read_index_from() Christian Couder
@ 2017-03-06 9:42 ` Christian Couder
2017-03-06 9:42 ` [PATCH v5 22/22] Documentation/git-update-index: explain splitIndex.* Christian Couder
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:42 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
Documentation/config.txt | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index b64aa7db2d..2afd5d982b 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -2864,6 +2864,18 @@ splitIndex.maxPercentChange::
than 20 percent of the total number of entries.
See linkgit:git-update-index[1].
+splitIndex.sharedIndexExpire::
+ When the split index feature is used, shared index files that
+ were not modified since the time this variable specifies will
+ be removed when a new shared index file is created. The value
+ "now" expires all entries immediately, and "never" suppresses
+ expiration altogether.
+ The default value is "2.weeks.ago".
+ Note that a shared index file is considered modified (for the
+ purpose of expiration) each time a new split-index file is
+ created based on it.
+ See linkgit:git-update-index[1].
+
status.relativePaths::
By default, linkgit:git-status[1] shows paths relative to the
current directory. Setting this variable to `false` shows paths
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 22/22] Documentation/git-update-index: explain splitIndex.*
2017-03-06 9:41 [PATCH v5 00/22] Add configuration options for split-index Christian Couder
` (20 preceding siblings ...)
2017-03-06 9:42 ` [PATCH v5 21/22] Documentation/config: add splitIndex.sharedIndexExpire Christian Couder
@ 2017-03-06 9:42 ` Christian Couder
21 siblings, 0 replies; 23+ messages in thread
From: Christian Couder @ 2017-03-06 9:42 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Nguyen Thai Ngoc Duy,
Ævar Arnfjörð Bjarmason, Ramsay Jones, Jeff King,
Christian Couder
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
Documentation/config.txt | 2 +-
Documentation/git-update-index.txt | 37 +++++++++++++++++++++++++++++--------
2 files changed, 30 insertions(+), 9 deletions(-)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index 2afd5d982b..24ed9c476d 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -2873,7 +2873,7 @@ splitIndex.sharedIndexExpire::
The default value is "2.weeks.ago".
Note that a shared index file is considered modified (for the
purpose of expiration) each time a new split-index file is
- created based on it.
+ either created based on it or read from it.
See linkgit:git-update-index[1].
status.relativePaths::
diff --git a/Documentation/git-update-index.txt b/Documentation/git-update-index.txt
index e091b2a409..1579abf3c3 100644
--- a/Documentation/git-update-index.txt
+++ b/Documentation/git-update-index.txt
@@ -163,14 +163,10 @@ may not support it yet.
--split-index::
--no-split-index::
- Enable or disable split index mode. If enabled, the index is
- split into two files, $GIT_DIR/index and $GIT_DIR/sharedindex.<SHA-1>.
- Changes are accumulated in $GIT_DIR/index while the shared
- index file contains all index entries stays unchanged. If
- split-index mode is already enabled and `--split-index` is
- given again, all changes in $GIT_DIR/index are pushed back to
- the shared index file. This mode is designed for very large
- indexes that take a significant amount of time to read or write.
+ Enable or disable split index mode. If split-index mode is
+ already enabled and `--split-index` is given again, all
+ changes in $GIT_DIR/index are pushed back to the shared index
+ file.
+
These options take effect whatever the value of the `core.splitIndex`
configuration variable (see linkgit:git-config[1]). But a warning is
@@ -394,6 +390,31 @@ Although this bit looks similar to assume-unchanged bit, its goal is
different from assume-unchanged bit's. Skip-worktree also takes
precedence over assume-unchanged bit when both are set.
+Split index
+-----------
+
+This mode is designed for repositories with very large indexes, and
+aims at reducing the time it takes to repeatedly write these indexes.
+
+In this mode, the index is split into two files, $GIT_DIR/index and
+$GIT_DIR/sharedindex.<SHA-1>. Changes are accumulated in
+$GIT_DIR/index, the split index, while the shared index file contains
+all index entries and stays unchanged.
+
+All changes in the split index are pushed back to the shared index
+file when the number of entries in the split index reaches a level
+specified by the splitIndex.maxPercentChange config variable (see
+linkgit:git-config[1]).
+
+Each time a new shared index file is created, the old shared index
+files are deleted if their modification time is older than what is
+specified by the splitIndex.sharedIndexExpire config variable (see
+linkgit:git-config[1]).
+
+To avoid deleting a shared index file that is still used, its
+modification time is updated to the current time everytime a new split
+index based on the shared index file is either created or read from.
+
Untracked cache
---------------
--
2.12.0.206.g74921e51d6.dirty
^ permalink raw reply related [flat|nested] 23+ messages in thread