Git Mailing List Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH 0/3] commit-graph: introduce 'core.useBloomFilters'
@ 2020-06-30 17:17 Taylor Blau
  2020-06-30 17:17 ` [PATCH 1/3] commit-graph: pass a 'struct repository *' in more places Taylor Blau
                   ` (3 more replies)
  0 siblings, 4 replies; 18+ messages in thread
From: Taylor Blau @ 2020-06-30 17:17 UTC (permalink / raw)
  To: git; +Cc: peff, dstolee

Hi,

Here are some patches that we have been using at GitHub to control
whether or not Bloom filters stored in commit-graphs are read during
normal operation.

We're planning on using these patches as part of a two-phase roll-out of
changed-path Bloom filters, where the first phase conditions whether or
not repositories *write* Bloom filters, and the second phase (controlled
via the new 'core.useBloomFilters') controls whether repositories *read*
their Bloom filters.

This can also be handy for debugging purposes, say, for e.g., if Bloom
filters are suspected to be corrupt, they can be softly disabled without
dropping the rest of the data in the commit-graph.

Thanks in advance for your review.

-Taylor

Taylor Blau (3):
  commit-graph: pass a 'struct repository *' in more places
  t4216: fix broken '&&'-chain
  commit-graph: respect 'core.useBloomFilters'

 Documentation/config/core.txt |  5 +++++
 builtin/commit-graph.c        |  2 +-
 commit-graph.c                | 17 ++++++++++-------
 commit-graph.h                |  4 +++-
 fuzz-commit-graph.c           |  5 +++--
 repo-settings.c               |  3 +++
 repository.h                  |  1 +
 t/helper/test-read-graph.c    |  3 ++-
 t/t4216-log-bloom.sh          |  6 ++++--
 9 files changed, 32 insertions(+), 14 deletions(-)

--
2.27.0.224.g4cfa086e50

^ permalink raw reply	[flat|nested] 18+ messages in thread
* Re: [PATCH 3/3] commit-graph: respect 'core.useBloomFilters'
@ 2020-07-01  9:58 Son Luong Ngoc
  2020-07-13 19:22 ` Taylor Blau
  0 siblings, 1 reply; 18+ messages in thread
From: Son Luong Ngoc @ 2020-07-01  9:58 UTC (permalink / raw)
  To: peff; +Cc: dstolee, git, me

Hi folks,

On Tue, 30 Jun 2020 15:33:40 -0400, Jeff King wrote:

> > > It might even be worth considering whether "changed paths" needs more
> > > context (or would if we add new features in the future). On a "git
> > > commit-graph write" command-line it is perfectly clear, but would
> > > core.commitGraphChangedPaths be worth it? It's definitely more specific,
> > > but it's also way more ugly. ;)
> >
> > Here's a third option what about 'graph.readChangedPaths'. I think that
> > Stolee and I discussed a new top-level 'graph' section, since we now
> > have a few commit-graph-related configuration variables in 'core'.
>
> Yes, I like that even better. Probably "graph" is sufficiently specific
> within Git's context, though I guess it _could_ bring to mind "git log
> --graph". So many overloaded terms. :)

I would suggest using 'commitgraph.readChangedPaths' as I was planning on
implementing the same config in [1] but never got around to it.

From an end-user perspective, not server admin, 'graph' is very much
correlated to 'git log --graph'.

Using 'commitgraph' instead of core could also help us enabling more config
down the line that equate to the current options in 'git commit-graph write'.

I.e. something like 'commitgraph.writeSplit' might be desirable to tune the
behavior of 'gc.writeCommitGraph' to use '--split=replace' strategy.

---

@Taylor: Thanks a lot for implementing this.

On Tue, 30 Jun 2020 13:17:36 -0400, Taylor Blau wrote:

> We're planning on using these patches as part of a two-phase roll-out of
> changed-path Bloom filters, where the first phase conditions whether or
> not repositories *write* Bloom filters, and the second phase (controlled
> via the new 'core.useBloomFilters') controls whether repositories *read*
> their Bloom filters.

Could you elaborate a bit more on the 'two-phase roll-out' mentioned here?

I was looking for a way to verify whether a commit-graph chain has been
written with Bloom filter (and force it to rewrite if not) but there seems
to be no straightforward way?

Do we need to implement a flag in 'git commit-graph verify' to check
for Bloom filter existence?

[1]: https://github.com/gitgitgadget/git/pull/633

Regards,
Son Luong.

^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, back to index

Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-30 17:17 [PATCH 0/3] commit-graph: introduce 'core.useBloomFilters' Taylor Blau
2020-06-30 17:17 ` [PATCH 1/3] commit-graph: pass a 'struct repository *' in more places Taylor Blau
2020-06-30 20:52   ` Derrick Stolee
2020-06-30 17:17 ` [PATCH 2/3] t4216: fix broken '&&'-chain Taylor Blau
2020-06-30 17:50   ` Eric Sunshine
2020-06-30 18:39     ` Taylor Blau
2020-06-30 19:03       ` Jeff King
2020-06-30 19:12         ` Taylor Blau
2020-06-30 19:19           ` Jeff King
2020-06-30 19:48         ` Eric Sunshine
2020-06-30 18:55     ` Jeff King
2020-06-30 17:17 ` [PATCH 3/3] commit-graph: respect 'core.useBloomFilters' Taylor Blau
2020-06-30 19:18   ` Jeff King
2020-06-30 19:27     ` Taylor Blau
2020-06-30 19:33       ` Jeff King
2020-08-03 19:02 ` [PATCH 0/3] commit-graph: introduce 'core.useBloomFilters' Taylor Blau
2020-07-01  9:58 [PATCH 3/3] commit-graph: respect 'core.useBloomFilters' Son Luong Ngoc
2020-07-13 19:22 ` Taylor Blau

Git Mailing List Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/git/0 git/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 git git/ https://lore.kernel.org/git \
		git@vger.kernel.org
	public-inbox-index git

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.git


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git