[PATCH] clone: allow initial sparse checkouts

[PATCH] clone: allow initial sparse checkouts

[Robin H. Johnson]

Presently if you want to perform a sparse checkout, you must either do a
full clone and then recheckout, or do a git init, manually set up
sparse, and then fetch and checkout.

This patch implements easily accessible sparse checkouts during clone,
in the --sparse-checkout option.

$ git clone REPO --sparse-checkout PATH

Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
---
 Documentation/git-clone.txt |  5 +++++
 builtin/clone.c             | 24 ++++++++++++++++++++++++
 2 files changed, 29 insertions(+)

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 0363d00..1c21207 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -14,6 +14,7 @@ SYNOPSIS
 	  [-o <name>] [-b <name>] [-u <upload-pack>] [--reference <repository>]
 	  [--separate-git-dir <git dir>]
 	  [--depth <depth>] [--[no-]single-branch]
+	  [--sparse-checkout <path>]
 	  [--recursive | --recurse-submodules] [--] <repository>
 	  [<directory>]
 
@@ -209,6 +210,10 @@ objects from the source repository into a pack in the cloned repository.
 	The result is Git repository can be separated from working
 	tree.
 
+--sparse-checkout <path>::
+	Perform the initial checkout as a sparse checkout, checking out only the
+	paths specified by this option. This option may occur multiple times, with
+	one path per instance.
 
 <repository>::
 	The (possibly remote) repository to clone from.  See the
diff --git a/builtin/clone.c b/builtin/clone.c
index 43e772c..1137371 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -48,6 +48,7 @@ static int option_verbosity;
 static int option_progress = -1;
 static struct string_list option_config;
 static struct string_list option_reference;
+static struct string_list option_sparse_checkout_path;
 
 static int opt_parse_reference(const struct option *opt, const char *arg, int unset)
 {
@@ -97,6 +98,8 @@ static struct option builtin_clone_options[] = {
 		   N_("separate git dir from working tree")),
 	OPT_STRING_LIST('c', "config", &option_config, N_("key=value"),
 			N_("set config inside the new repository")),
+	OPT_STRING_LIST(0, "sparse-checkout", &option_sparse_checkout_path, N_("path"),
+			N_("set path for sparse checkout")),
 	OPT_END()
 };
 
@@ -270,6 +273,24 @@ static void setup_reference(void)
 	for_each_string_list(&option_reference, add_one_reference, NULL);
 }
 
+static void setup_sparse_checkout(void)
+{
+	FILE *info_sparse_checkout_fp;
+	struct string_list_item *item;
+	char *path = git_pathdup("info/sparse-checkout");
+	git_config_set("core.sparsecheckout", "true");
+	safe_create_leading_directories(path);
+	info_sparse_checkout_fp = fopen(path, "w");
+	if (!info_sparse_checkout_fp)
+		die(_("unable to create %s"), path);
+	for_each_string_list_item(item, &option_sparse_checkout_path) {
+		fprintf(info_sparse_checkout_fp, "%s\n", item->string);
+	}
+	fclose(info_sparse_checkout_fp);
+	adjust_shared_perm(path);
+	free(path);
+}
+
 static void copy_alternates(struct strbuf *src, struct strbuf *dst,
 			    const char *src_repo)
 {
@@ -873,6 +894,9 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 	init_db(option_template, INIT_DB_QUIET);
 	write_config(&option_config);
 
+	if(option_sparse_checkout_path.nr)
+		setup_sparse_checkout();
+
 	git_config(git_default_config, NULL);
 
 	if (option_bare) {
-- 
1.9.0.291.g027825b.dirty

Re: [PATCH] clone: allow initial sparse checkouts

[Duy Nguyen]

On Sun, Feb 23, 2014 at 9:31 AM, Robin H. Johnson <robbat2@gentoo.org> wrote:
> Presently if you want to perform a sparse checkout, you must either do a
> full clone and then recheckout, or do a git init, manually set up
> sparse, and then fetch and checkout.

I think you could do "clone -n" (no checkout), set up sparse, then checkout.

> This patch implements easily accessible sparse checkouts during clone,
> in the --sparse-checkout option.
>
> $ git clone REPO --sparse-checkout PATH

Or take a file as input if there are lots of paths/rules.
-- 
Duy

Re: [PATCH] clone: allow initial sparse checkouts

[Robin H. Johnson]

On Sun, Feb 23, 2014 at 09:52:16AM +0700,  Duy Nguyen wrote:
> On Sun, Feb 23, 2014 at 9:31 AM, Robin H. Johnson <robbat2@gentoo.org> wrote:
> > Presently if you want to perform a sparse checkout, you must either do a
> > full clone and then recheckout, or do a git init, manually set up
> > sparse, and then fetch and checkout.
> I think you could do "clone -n" (no checkout), set up sparse, then checkout.
Yes, I think there are a few more ways, but still would be nice to 

> 
> > This patch implements easily accessible sparse checkouts during clone,
> > in the --sparse-checkout option.
> >
> > $ git clone REPO --sparse-checkout PATH
> Or take a file as input if there are lots of paths/rules.
How much demand for taking a file of rules, and opinions of syntax to do
that vs specify on the commandline?

--sparse-checkout-from FILE
vs.
--sparse-checkout '<PATH'
or something other prefix character.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

Re: [PATCH] clone: allow initial sparse checkouts

[Duy Nguyen]

On Sun, Feb 23, 2014 at 2:32 PM, Robin H. Johnson <robbat2@gentoo.org> wrote:
>> > This patch implements easily accessible sparse checkouts during clone,
>> > in the --sparse-checkout option.
>> >
>> > $ git clone REPO --sparse-checkout PATH
>> Or take a file as input if there are lots of paths/rules.
> How much demand for taking a file of rules,

I don't know. I guess it depends on each repo's layout. If the layout
is simple, usually you would need one or two rules so it's ok to type
again and again. If it's more complicated and starts using '!' rules,
it's probably best to save in a file.

> and opinions of syntax to do
> that vs specify on the commandline?
>
> --sparse-checkout-from FILE

I think this one is better. But if you don't see a need for it, we can
always delay implementing it until an actual use case comes up.

> vs.
> --sparse-checkout '<PATH'
> or something other prefix character.
-- 
Duy

Re: [PATCH] clone: allow initial sparse checkouts

[Robin H. Johnson]

On Sun, Feb 23, 2014 at 03:43:47PM +0700,  Duy Nguyen wrote:
> On Sun, Feb 23, 2014 at 2:32 PM, Robin H. Johnson <robbat2@gentoo.org> wrote:
> >> > This patch implements easily accessible sparse checkouts during clone,
> >> > in the --sparse-checkout option.
> >> >
> >> > $ git clone REPO --sparse-checkout PATH
> >> Or take a file as input if there are lots of paths/rules.
> > How much demand for taking a file of rules,
> I don't know. I guess it depends on each repo's layout. If the layout
> is simple, usually you would need one or two rules so it's ok to type
> again and again. If it's more complicated and starts using '!' rules,
> it's probably best to save in a file.
> 
> > and opinions of syntax to do
> > that vs specify on the commandline?
> >
> > --sparse-checkout-from FILE
> 
> I think this one is better. But if you don't see a need for it, we can
> always delay implementing it until an actual use case comes up.
I think I'd prefer to delay that part then.
What I'm concerned about if we do have it, is what ordering semantics
there should be, eg for something like:
--sparse-checkout '!X' --sparse-checkout-from F --sparse-checkout Y

Should that be [!X, *F, Y], or [*F, !X, Y], or something else?
Would the option parser need to be modified to handle this?
Or do we just make them mutually exclusive?

The only other clean alternative would be implementing ONLY
--sparse-checkout-from, and letting uses use fds creatively:
--sparse-checkout-from <(echo X; echo Y)
But the msysgit crowd would probably mumble complaints under their
breath at me.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

Re: [PATCH] clone: allow initial sparse checkouts

[Junio C Hamano]

"Robin H. Johnson" <robbat2@gentoo.org> writes:

> The only other clean alternative would be implementing ONLY
> --sparse-checkout-from, and letting uses use fds creatively:
> --sparse-checkout-from <(echo X; echo Y)

Not all POSIX shells have such an abomination that is process
substitution.  You can easily work it around by adopting the usual
convention to use "-" to read from the standasrd input, though.

	(echo X; echo Y) | cmd --sparse-checkout-from -

Re: [PATCH] clone: allow initial sparse checkouts

[Robin H. Johnson]

On Mon, Feb 24, 2014 at 09:47:16AM -0800,  Junio C Hamano wrote:
> "Robin H. Johnson" <robbat2@gentoo.org> writes:
> > The only other clean alternative would be implementing ONLY
> > --sparse-checkout-from, and letting uses use fds creatively:
> > --sparse-checkout-from <(echo X; echo Y)
> Not all POSIX shells have such an abomination that is process
> substitution.  You can easily work it around by adopting the usual
> convention to use "-" to read from the standasrd input, though.
> 
> 	(echo X; echo Y) | cmd --sparse-checkout-from -
Is that a vote that you'd like to see a --sparse-checkout-from variant
of my patch?

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

Re: [PATCH] clone: allow initial sparse checkouts

[Junio C Hamano]

"Robin H. Johnson" <robbat2@gentoo.org> writes:

> On Mon, Feb 24, 2014 at 09:47:16AM -0800,  Junio C Hamano wrote:
>> "Robin H. Johnson" <robbat2@gentoo.org> writes:
>> > The only other clean alternative would be implementing ONLY
>> > --sparse-checkout-from, and letting uses use fds creatively:
>> > --sparse-checkout-from <(echo X; echo Y)
>> Not all POSIX shells have such an abomination that is process
>> substitution.  You can easily work it around by adopting the usual
>> convention to use "-" to read from the standasrd input, though.
>> 
>> 	(echo X; echo Y) | cmd --sparse-checkout-from -
> Is that a vote that you'd like to see a --sparse-checkout-from variant
> of my patch?

Honestly, I do not particularly care too much about this feature,
regardless of the interface [*1*].

It is just a vote that says "if --something-from form is going to be
implemented, it should be able to read from the standard input with
'-', unless there is a compelling reason not to do so".


[Footnote]

*1* In the longer term, I think sparse checkout is broken as a
concept and I view this "use sparse checkout from the get-go" merely
a stop-gap measure to make the band-aid a bit less painful to use.
What you really want is a narrow clone, which is conceptually cleaner
but a lot harder implentation-wise.

Not that keeping a band-aid usable longer is necessarily bad in the
real world, though---so even I said *I*'m not interested, that is
different from saying I'm not taking a patch on this topic.