All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/7] In grep, no adding submodule ODB as alternates
@ 2021-08-10 18:28 Jonathan Tan
  2021-08-10 18:28 ` [PATCH 1/7] submodule: lazily add submodule ODBs " Jonathan Tan
                   ` (10 more replies)
  0 siblings, 11 replies; 78+ messages in thread
From: Jonathan Tan @ 2021-08-10 18:28 UTC (permalink / raw)
  To: git; +Cc: Jonathan Tan

This patch series removes the need to add submodule ODBs as alternates
in all codepaths covered by t7814. I believe that that is also all the
codepaths covered by "git grep", but if it isn't, the uncovered
codepaths will still work - they will just not benefit from the
performance improvements.

In doing this work of migrating away from adding submodule ODBs as
alternates, I'm mainly motivated by the possibility of adding partial
clone submodule support, but this has benefits even for those who do not
use partial clones, as described in the documentation in patch 1.

To reviewers: you can cherry-pick the last patch onto one of the earlier
ones to observe what happens when the code still accesses a submodule
object as if it were in the_repository.

Jonathan Tan (7):
  submodule: lazily add submodule ODBs as alternates
  grep: use submodule-ODB-as-alternate lazy-addition
  grep: typesafe versions of grep_source_init
  grep: read submodule entry with explicit repo
  grep: allocate subrepos on heap
  grep: add repository to OID grep sources
  t7814: show lack of alternate ODB-adding

 builtin/grep.c                     | 49 +++++++++++++++++++++---------
 grep.c                             | 48 ++++++++++++++++++-----------
 grep.h                             | 10 ++++--
 object-file.c                      |  5 +++
 submodule.c                        | 25 ++++++++++++++-
 submodule.h                        |  8 +++++
 t/README                           | 10 ++++++
 t/t7814-grep-recurse-submodules.sh |  3 ++
 8 files changed, 122 insertions(+), 36 deletions(-)

-- 
2.33.0.rc1.237.g0d66db33f3-goog


^ permalink raw reply	[flat|nested] 78+ messages in thread
* [RFC PATCH 0/3] grep: don't add subrepos to in-memory alternates
@ 2019-09-18  1:56 Matheus Tavares
  2019-09-18 19:55 ` Junio C Hamano
  0 siblings, 1 reply; 78+ messages in thread
From: Matheus Tavares @ 2019-09-18  1:56 UTC (permalink / raw)
  To: git; +Cc: Jonathan Nieder

Make git-grep --recurse-submodules stop adding subrepos to the in-memory
alternates list and, instead, pass a reference to the subrepo struct
down to the threads.

This series is based on ew/hashmap (it uses the added container_of
macro).

There're some possibly needed changes I'm still unsure about. All
feedback will be highly appreciated:

- textconv cache is written to the_repository's object database even for
  submodules. Should it perhaps be written to submodules' odb instead?

- Considering the following call chain: grep_source_load_driver() >
  userdiff_find_by_path() > git_check_attr() > collect_some_attrs() >
  prepare_attr_stack() > bootstrap_attr_stack():

  * The last function tries to read the attributes from the
    .gitattributes and .git/info/attributes files of the_repository.
    However, for paths inside the submodule, shouldn't it try to read
    these files from the submodule?

  * This function will also call: read_attr() > read_attr_from_index() >
    read_blob_data_from_index() which might, in turn, call
    read_object_file(). Shouldn't we pass the subrepo to it so that it
    can call repo_read_object_file()? (Again, for paths inside the
    submodule, read_object_file() won't be able to find the object as
    we won't be adding to alternates anymore.)

Matheus Tavares (3):
  diff: use the given repo at diff_populate_filespec()
  object: allow parse_object_or_die() to handle any repo
  grep: don't add submodules to the alternates list

 builtin/grep.c  | 82 +++++++++++++++++++++++++++++++++++--------------
 builtin/prune.c |  4 +--
 bundle.c        | 13 +++++---
 diff.c          |  2 +-
 grep.c          | 26 ++++++++--------
 object.c        |  5 +--
 object.h        |  4 ++-
 pack-bitmap.c   |  5 +--
 reachable.c     |  4 +--
 upload-pack.c   |  2 +-
 10 files changed, 94 insertions(+), 53 deletions(-)

-- 
2.23.0


^ permalink raw reply	[flat|nested] 78+ messages in thread

end of thread, other threads:[~2021-09-27 17:46 UTC | newest]

Thread overview: 78+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-10 18:28 [PATCH 0/7] In grep, no adding submodule ODB as alternates Jonathan Tan
2021-08-10 18:28 ` [PATCH 1/7] submodule: lazily add submodule ODBs " Jonathan Tan
2021-08-10 21:13   ` Junio C Hamano
2021-08-13 16:53     ` Jonathan Tan
2021-08-11 21:33   ` Emily Shaffer
2021-08-13 16:23     ` Jonathan Tan
2021-08-10 18:28 ` [PATCH 2/7] grep: use submodule-ODB-as-alternate lazy-addition Jonathan Tan
2021-08-11 21:36   ` Emily Shaffer
2021-08-13 16:31     ` Jonathan Tan
2021-08-10 18:28 ` [PATCH 3/7] grep: typesafe versions of grep_source_init Jonathan Tan
2021-08-10 21:38   ` Junio C Hamano
2021-08-11 21:42   ` Emily Shaffer
2021-08-11 23:07     ` Ramsay Jones
2021-08-13 16:32       ` Jonathan Tan
2021-08-11 22:45   ` Matheus Tavares Bernardino
2021-08-12 16:49     ` Junio C Hamano
2021-08-13 16:33       ` Jonathan Tan
2021-08-10 18:28 ` [PATCH 4/7] grep: read submodule entry with explicit repo Jonathan Tan
2021-08-11 21:44   ` Emily Shaffer
2021-08-13 16:39     ` Jonathan Tan
2021-08-10 18:28 ` [PATCH 5/7] grep: allocate subrepos on heap Jonathan Tan
2021-08-11 21:50   ` Emily Shaffer
2021-08-13 16:42     ` Jonathan Tan
2021-08-10 18:28 ` [PATCH 6/7] grep: add repository to OID grep sources Jonathan Tan
2021-08-11 21:52   ` Emily Shaffer
2021-08-13 16:44     ` Jonathan Tan
2021-08-11 23:28   ` Matheus Tavares Bernardino
2021-08-13 16:47     ` Jonathan Tan
2021-08-10 18:28 ` [PATCH 7/7] t7814: show lack of alternate ODB-adding Jonathan Tan
2021-08-11 21:55   ` Emily Shaffer
2021-08-11 22:22     ` Matheus Tavares Bernardino
2021-08-13 16:50       ` Jonathan Tan
2021-08-11 21:29 ` [PATCH 0/7] In grep, no adding submodule ODB as alternates Emily Shaffer
2021-08-11 22:49 ` Josh Steadmon
2021-08-13 21:05 ` [PATCH v2 0/8] " Jonathan Tan
2021-08-13 21:05   ` [PATCH v2 1/8] submodule: lazily add submodule ODBs " Jonathan Tan
2021-08-13 21:05   ` [PATCH v2 2/8] grep: use submodule-ODB-as-alternate lazy-addition Jonathan Tan
2021-08-13 21:05   ` [PATCH v2 3/8] grep: typesafe versions of grep_source_init Jonathan Tan
2021-08-16 15:06     ` Matheus Tavares Bernardino
2021-08-13 21:05   ` [PATCH v2 4/8] grep: read submodule entry with explicit repo Jonathan Tan
2021-08-13 21:05   ` [PATCH v2 5/8] grep: allocate subrepos on heap Jonathan Tan
2021-08-13 21:44     ` Junio C Hamano
2021-08-16 19:42       ` Jonathan Tan
2021-08-13 21:05   ` [PATCH v2 6/8] grep: add repository to OID grep sources Jonathan Tan
2021-08-16 14:48     ` Matheus Tavares Bernardino
2021-08-16 19:44       ` Jonathan Tan
2021-08-13 21:05   ` [PATCH v2 7/8] submodule-config: pass repo upon blob config read Jonathan Tan
2021-08-16 14:32     ` Matheus Tavares Bernardino
2021-08-16 19:57       ` Matheus Tavares Bernardino
2021-08-16 20:02       ` Jonathan Tan
2021-08-16 15:48     ` Matheus Tavares Bernardino
2021-08-16 20:09       ` Jonathan Tan
2021-08-16 20:57         ` Jonathan Tan
2021-08-13 21:05   ` [PATCH v2 8/8] t7814: show lack of alternate ODB-adding Jonathan Tan
2021-08-16 15:14   ` [PATCH v2 0/8] In grep, no adding submodule ODB as alternates Matheus Tavares Bernardino
2021-08-16 21:09 ` [PATCH v3 " Jonathan Tan
2021-08-16 21:09   ` [PATCH v3 1/8] submodule: lazily add submodule ODBs " Jonathan Tan
2021-08-16 21:09   ` [PATCH v3 2/8] grep: use submodule-ODB-as-alternate lazy-addition Jonathan Tan
2021-08-16 21:09   ` [PATCH v3 3/8] grep: typesafe versions of grep_source_init Jonathan Tan
2021-08-16 21:09   ` [PATCH v3 4/8] grep: read submodule entry with explicit repo Jonathan Tan
2021-08-16 21:09   ` [PATCH v3 5/8] grep: allocate subrepos on heap Jonathan Tan
2021-08-16 21:09   ` [PATCH v3 6/8] grep: add repository to OID grep sources Jonathan Tan
2021-09-27 12:08     ` Ævar Arnfjörð Bjarmason
2021-09-27 16:45       ` [RFC PATCH 0/3] grep: don'\''t add subrepos to in-memory alternates Matheus Tavares
2021-09-27 17:30         ` Ævar Arnfjörð Bjarmason
2021-08-16 21:09   ` [PATCH v3 7/8] submodule-config: pass repo upon blob config read Jonathan Tan
2021-08-16 21:09   ` [PATCH v3 8/8] t7814: show lack of alternate ODB-adding Jonathan Tan
2021-08-17 19:29   ` [PATCH v3 0/8] In grep, no adding submodule ODB as alternates Matheus Tavares Bernardino
2021-09-08  0:26   ` Junio C Hamano
2021-09-08 15:31     ` Matheus Tavares Bernardino
2021-09-08 18:45       ` Junio C Hamano
  -- strict thread matches above, loose matches on Subject: below --
2019-09-18  1:56 [RFC PATCH 0/3] grep: don't add subrepos to in-memory alternates Matheus Tavares
2019-09-18 19:55 ` Junio C Hamano
2019-09-19  5:18   ` Matheus Tavares Bernardino
2019-09-20 16:26     ` Junio C Hamano
2019-09-21 20:34       ` Matheus Tavares Bernardino
2019-09-28  3:24         ` Junio C Hamano
2019-09-28  4:20           ` Matheus Tavares Bernardino

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.