Re: [PATCHv2] builtin/clone: support submodule groups

[Michael J Gruber <git@drmicha.warpmail.net>]

Stefan Beller venit, vidit, dixit 30.11.2015 20:31:
> + cc Duy, Michael, who discussed the sparse checkout recently
> 
> On Wed, Nov 25, 2015 at 9:00 PM, Trevor Saunders <tbsaunde@tbsaunde.org> wrote:
>> Seeing the recent sparse checkout discussion I realized it might be
>> useful to have a similar sort of feature for sparse checkouts.  So say I
>> had a mobile and desktop client in the same repo and wanted to be able to
>> checkout the commoncode and one client without having to explicitly list
>> all the paths I care about. It seems like UI wise you might want to use
>> --group there too, or at least explaining the difference to users might
>> be interesting, but maybe that's worrying way too much abouta possible
>> future feature.
> 
> For reference what I was proposing (coverletter [RFC PATCH 0/5]
> Submodule Groups):
> -->8--
> This is also available at
> https://github.com/stefanbeller/git/tree/submodule-groups
> It applies on top of the submodule-parallel-patch series I sent a few
> minutes ago.
> 
> Consider having a real large software project in Git with each component
> in a submodule (such as an operating system, Android, Debian, Fedora,
> no toy OS such as https://github.com/gittup/gittup as that doesn't quite
> demonstrate the scale of the problem).
> 
> If you have lots of submodules, you probably don't need all of them at once,
> but you have functional units. Some submodules are absolutely required,
> some are optional and only for very specific purposes.
> 
> This patch series adds meaning to a "groups" field in the .gitmodules file.
> 
> So you could have a .gitmodules file such as:
> 
> [submodule "gcc"]
>         path = gcc
>         url = git://...
>         groups = default,devel
> [submodule "linux"]
>         path = linux
>         url = git://...
>         groups = default
> [submodule "nethack"]
>         path = nethack
>         url = git://...
>         groups = optional,games
> 
> and by this series you can work on an arbitrary subgroup of these
> submodules such
> using these commands:
> 
>     git clone --group default --group devel git://...
>     # will clone the superproject and recursively
>     # checkout any submodule being in at least one of the groups.
> 
>     git submodule add --group default --group devel git://... ..
>     # will add a submodule, adding 2 submodule
>     # groups to its entry in .gitmodule
> 
>     # as support for clone we want to have:
>     git config submodule.groups default
>     git submodule init --groups
>     # will init all submodules from the default group
> 
>     # as support for clone we want to have:
>     git config submodule.groups default
>     git submodule update --groups
>     # will update all submodules from the default group
> 
> Any feedback welcome, specially on the design level!
> (Do we want to have it stored in the .gitmodules file? Do we want to have
> the groups configured in .git/config as "submodule.groups", any other way
> to make it future proof and extend the groups syntax?)
> -->8--
> 
> I think the biggest advantage with the groups is to have it not depending on the
> path. Consider your one repository containing both mobile and desktop code,
> where you have a sparse checkout for mobile.
> 
> Now what happens if files are renamed, i.e. the leading directory?
> As this change comes in from your dear coworker, who has no idea how
> your .git/info/sparse-checkout looks like, it may happen that more files appear
> in your worktree as your patterns did not cover the renamed case.
> Or some file contents go missing as they are in one of the ignored paths.
> 
> This groups feature would solve that as the groups are not dependent on
> the paths or the data itself. However the groups in this proposal are only
> meant to be applied on a submodule level, not in a repository itself.
> 
> How would you do that?
> I could imagine a file like .gitgroups (It should be part of the repository,
> such that everybody talks about the same groups), with a similar syntax like
> .gitattributes where some file patterns are assigned one or more groups.
> And in either .git/config (as it is for the submodules here) or in a file
> .git/info/groups (as the sparse checkouts do it in that directory)
> you'd configure the groups you are interested in.
> 
> It would be cool to have the same mechanism for both sparse-group-checkout
> and submodules, so I'd propose to use a config option in .git/config,
> such as checkout-group which covers both submodules as well as the
> pattern groups as specified by .gitgroups.
> 
> ---
> Beware, this is just a first shot spinning around some ideas.
> 
> Thanks,
> Stefan

I think we have to solve more basic issues for sparse checkouts first.
I'm using them with extra worktrees now and everything seems to be
working fine. But we need to get the UI right for the simple case (no
submodules, maybe not even extra worktrees) first: setting up patterns
before checkout etc. Having submodules in mind doesn't hurt, tough.

I still consider sparse checkouts a local "cludge" (not technically
cludgy) in the sense that it helps you cater to some specific local
needs; not something whose config you'd want to transport as part of the
object store.

Minor implementation detail: Do we have any precedence of comma
separated values for config values? I'd say we rather use multiple
entries, don't we?

Michael