[PATCH 1/2] Make die_if_checked_out() ignore missing worktree checkouts.

[PATCH 1/2] Make die_if_checked_out() ignore missing worktree checkouts.

[Peter Jones]

Currently if you do, for example:

$ git worktree add path foo

And "foo" has already been checked out at some other path, but the user
has removed it without pruning, you'll get an error that the branch is
already checked out.  It isn't meaningfully checked out, the repo's
data is just stale and no longer reflects reality.

This makes it so that if nothing is present where a worktree is
supposedly checked out, we ignore that the worktree exists, and let it
get cleaned up the next time worktrees are pruned.

(I would prune it instead, but prune isn't available from libgit
currently.)

Signed-off-by: Peter Jones <pjones@redhat.com>
---
 branch.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/branch.c b/branch.c
index 579494738a7..60322ded953 100644
--- a/branch.c
+++ b/branch.c
@@ -360,6 +360,9 @@ void die_if_checked_out(const char *branch, int ignore_current_worktree)
 	wt = find_shared_symref("HEAD", branch);
 	if (!wt || (ignore_current_worktree && wt->is_current))
 		return;
+	if (access(wt->path, F_OK) < 0 &&
+	    (errno == ENOENT || errno == ENOTDIR))
+		return;
 	skip_prefix(branch, "refs/heads/", &branch);
 	die(_("'%s' is already checked out at '%s'"),
 	    branch, wt->path);
-- 
2.23.0

[PATCH 2/2] Make "git branch -d" prune missing worktrees automatically.

[Peter Jones]

Currently, if you do:

$ git branch zonk origin/master
$ git worktree add zonk zonk
$ rm -rf zonk
$ git branch -d zonk

You get the following error:

$ git branch -d zonk
error: Cannot delete branch 'zonk' checked out at '/home/pjones/devel/kernel.org/git/zonk'

It isn't meaningfully checked out, the repo's data is just stale and no
longer reflects reality.

This makes it so that if nothing is present where a worktree is
supposedly checked out, deleting the branch will automatically prune it.

Signed-off-by: Peter Jones <pjones@redhat.com>
---
 builtin/branch.c   |  2 +-
 builtin/worktree.c | 14 ++++++++++++++
 worktree.h         |  6 ++++++
 3 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/builtin/branch.c b/builtin/branch.c
index 2ef214632f0..d611f8183b4 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -236,7 +236,7 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 		if (kinds == FILTER_REFS_BRANCHES) {
 			const struct worktree *wt =
 				find_shared_symref("HEAD", name);
-			if (wt) {
+			if (wt && prune_worktree_if_missing(wt) < 0) {
 				error(_("Cannot delete branch '%s' "
 					"checked out at '%s'"),
 				      bname.buf, wt->path);
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 4de44f579af..b3ad915c3c3 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -133,6 +133,20 @@ static int prune_worktree(const char *id, struct strbuf *reason)
 	return 0;
 }
 
+int prune_worktree_if_missing(const struct worktree *wt)
+{
+	struct strbuf reason = STRBUF_INIT;
+
+	if (access(wt->path, F_OK) >= 0 ||
+	    (errno != ENOENT && errno == ENOTDIR)) {
+		errno = EEXIST;
+		return -1;
+	}
+
+	strbuf_addf(&reason, _("Removing worktrees/%s: worktree directory is not present"), wt->id);
+	return prune_worktree(wt->id, &reason);
+}
+
 static void prune_worktrees(void)
 {
 	struct strbuf reason = STRBUF_INIT;
diff --git a/worktree.h b/worktree.h
index caecc7a281c..75762c25752 100644
--- a/worktree.h
+++ b/worktree.h
@@ -132,4 +132,10 @@ void strbuf_worktree_ref(const struct worktree *wt,
 const char *worktree_ref(const struct worktree *wt,
 			 const char *refname);
 
+/*
+ * Prune a worktree if it is no longer present at the checked out location.
+ * Returns < 0 if the checkout is there or if pruning fails.
+ */
+int prune_worktree_if_missing(const struct worktree *wt);
+
 #endif
-- 
2.23.0

Re: [PATCH 2/2] Make "git branch -d" prune missing worktrees automatically.

[Eric Sunshine]

On Thu, Oct 17, 2019 at 12:28 PM Peter Jones <pjones@redhat.com> wrote:
> Currently, if you do:
>
> $ git branch zonk origin/master
> $ git worktree add zonk zonk
> $ rm -rf zonk
> $ git branch -d zonk
>
> You get the following error:
>
> $ git branch -d zonk
> error: Cannot delete branch 'zonk' checked out at '/home/pjones/devel/kernel.org/git/zonk'
>
> It isn't meaningfully checked out, the repo's data is just stale and no
> longer reflects reality.

Echoing SEZDER's comment on patch 1/2, this behavior is an intentional
design choice and safety feature of the worktree implementation since
worktrees may exist on removable media or remote filesystems which
might not always be mounted; hence, the presence of commands "git
worktree prune" and "git worktree remove".

A couple comment regarding this patch...

> Signed-off-by: Peter Jones <pjones@redhat.com>
> ---
> diff --git a/builtin/worktree.c b/builtin/worktree.c
> @@ -133,6 +133,20 @@ static int prune_worktree(const char *id, struct strbuf *reason)
> +int prune_worktree_if_missing(const struct worktree *wt)
> +{
> +       struct strbuf reason = STRBUF_INIT;
> +
> +       if (access(wt->path, F_OK) >= 0 ||
> +           (errno != ENOENT && errno == ENOTDIR)) {
> +               errno = EEXIST;
> +               return -1;
> +       }
> +
> +       strbuf_addf(&reason, _("Removing worktrees/%s: worktree directory is not present"), wt->id);
> +       return prune_worktree(wt->id, &reason);
> +}

"git worktree" tries to clean up after itself as much as possible. For
instance, it is careful to remove the .git/worktrees directory when
the last worktree itself is removed (or pruned). So, the caller of
this function would also want to call delete_worktrees_dir_if_empty()
to follow suit.

> diff --git a/worktree.h b/worktree.h
> @@ -132,4 +132,10 @@ void strbuf_worktree_ref(const struct worktree *wt,
> +/*
> + * Prune a worktree if it is no longer present at the checked out location.
> + * Returns < 0 if the checkout is there or if pruning fails.
> + */
> +int prune_worktree_if_missing(const struct worktree *wt);

It's rather ugly that this function is declared in top-level
worktree.h whereas the actual implementation is in builtin/worktree.c.
I'd expect to see a preparatory patch which moves prune_worktree()
(and probably delete_worktrees_dir_if_empty()) to top-level
worktree.c.

These minor implementation comments aside, before considering this
patch series, it would be nice to see a compelling argument as to why
this change of behavior, which undercuts a deliberate design decision,
is really desirable.

Re: [PATCH 2/2] Make "git branch -d" prune missing worktrees automatically.

[Peter Jones]

On Thu, Oct 17, 2019 at 06:44:26PM +0200, SZEDER Gábor wrote:
> >  	if (!wt || (ignore_current_worktree && wt->is_current))
> >  		return;
> > +	if (access(wt->path, F_OK) < 0 &&
> > +	    (errno == ENOENT || errno == ENOTDIR))
> > +		return;
> 
> I think this check is insuffient: even if the directory of the working
> tree is not present, the working tree might still exist, and should
> not be ignored (or deleted/pruned in the second patch).
> 
> See the description of 'git worktree lock' for details.

Ah, thanks for that, I had not realized "lock" was relevant here as I
have never used it.  That explains some of what seemed to me like a very
strange usage model.

On Thu, Oct 17, 2019 at 01:28:09PM -0400, Eric Sunshine wrote:
> Echoing SEZDER's comment on patch 1/2, this behavior is an intentional
> design choice and safety feature of the worktree implementation since
> worktrees may exist on removable media or remote filesystems which
> might not always be mounted; hence, the presence of commands "git
> worktree prune" and "git worktree remove".

Okay, I see that use case now - I hadn't realized there was an
intentional design decision here, and honestly that's anything but clear
from the *code*.  It's surprising, for example, that my patches didn't
break a single test case.

> A couple comment regarding this patch...

Sure...

> > diff --git a/builtin/worktree.c b/builtin/worktree.c
> > @@ -133,6 +133,20 @@ static int prune_worktree(const char *id, struct strbuf *reason)
> > +int prune_worktree_if_missing(const struct worktree *wt)
> > +{
> > +       struct strbuf reason = STRBUF_INIT;
> > +
> > +       if (access(wt->path, F_OK) >= 0 ||
> > +           (errno != ENOENT && errno == ENOTDIR)) {
> > +               errno = EEXIST;
> > +               return -1;
> > +       }
> > +
> > +       strbuf_addf(&reason, _("Removing worktrees/%s: worktree directory is not present"), wt->id);
> > +       return prune_worktree(wt->id, &reason);
> > +}
> 
> "git worktree" tries to clean up after itself as much as possible. For
> instance, it is careful to remove the .git/worktrees directory when
> the last worktree itself is removed (or pruned). So, the caller of
> this function would also want to call delete_worktrees_dir_if_empty()
> to follow suit.

Okay, will fix.

> > diff --git a/worktree.h b/worktree.h
> > @@ -132,4 +132,10 @@ void strbuf_worktree_ref(const struct worktree *wt,
> > +/*
> > + * Prune a worktree if it is no longer present at the checked out location.
> > + * Returns < 0 if the checkout is there or if pruning fails.
> > + */
> > +int prune_worktree_if_missing(const struct worktree *wt);
> 
> It's rather ugly that this function is declared in top-level
> worktree.h whereas the actual implementation is in builtin/worktree.c.

I don't disagree, but I didn't want to move stuff into an exposed API if
I didn't have to, and that seemed like an appropriate enough header.  I
can do it the other way though, no problem.

> I'd expect to see a preparatory patch which moves prune_worktree()
> (and probably delete_worktrees_dir_if_empty()) to top-level
> worktree.c.

Sure thing.

> These minor implementation comments aside, before considering this
> patch series, it would be nice to see a compelling argument as to why
> this change of behavior, which undercuts a deliberate design decision,
> is really desirable.

Okay, so just for clarity, when you say there's a deliberate design
decision, which behavior here are you talking about?  If you mean making
"lock" work, I don't have any issue with that.  If you mean not cleaning
up when we do other commands, then I don't see why that's a concern -
after all, that's exactly what "lock" is for.

Assuming it is the "lock" behavior we're talking about, I don't think I
actually have any intention of breaking this design decision, just
making my workflow (without "lock") nag at me less for what seem like
pretty trivial issues.

I can easily accommodate "git worktree lock".  What bugs me though, is
that using worktrees basically means I have to replace fairly regular
filesystem activities with worktree commands, and it doesn't seem to be
*necessary* in any way.  And I'm going to forget.  A lot.

To me, there doesn't seem to be any reason these need to behave any different:

$ git worktree add foo foo
$ rm -rf foo
vs
$ git worktree add foo foo
$ git worktree remove foo

And in fact the only difference right now, aside from some very
minuscule storage requirements that haven't gotten cleaned up, is the
first one leaves an artifact that tells it to give me errors later until
I run "git worktree prune" myself.  

I'll send another revision of this patchset as a reply to this mail,
which should clear up some of our differences.

-- 
  Peter

[PATCH v2 1/4] libgit: Add a read-only helper to test the worktree lock

[Peter Jones]

Add the function is_worktree_locked(), which is a helper to tell if a
worktree is locked without having to be able to modify it.

Signed-off-by: Peter Jones <pjones@redhat.com>
---
 builtin/worktree.c |  2 +-
 worktree.c         | 16 ++++++++++++++++
 worktree.h         |  5 +++++
 3 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/builtin/worktree.c b/builtin/worktree.c
index 4de44f579af..86305cc1fe1 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -245,7 +245,7 @@ static void validate_worktree_add(const char *path, const struct add_opts *opts)
 	if (!wt)
 		goto done;
 
-	locked = !!worktree_lock_reason(wt);
+	locked = is_worktree_locked(wt);
 	if ((!locked && opts->force) || (locked && opts->force > 1)) {
 		if (delete_git_dir(wt->id))
 		    die(_("unable to re-add worktree '%s'"), path);
diff --git a/worktree.c b/worktree.c
index 5b4793caa34..4924805c389 100644
--- a/worktree.c
+++ b/worktree.c
@@ -244,6 +244,22 @@ int is_main_worktree(const struct worktree *wt)
 	return !wt->id;
 }
 
+int is_worktree_locked(const struct worktree *wt)
+{
+	struct strbuf path = STRBUF_INIT;
+	int locked = 0;
+
+	if (wt->lock_reason_valid && wt->lock_reason)
+		return 1;
+
+	strbuf_addstr(&path, worktree_git_path(wt, "locked"));
+	if (file_exists(path.buf))
+		locked = 1;
+
+	strbuf_release(&path);
+	return locked;
+}
+
 const char *worktree_lock_reason(struct worktree *wt)
 {
 	assert(!is_main_worktree(wt));
diff --git a/worktree.h b/worktree.h
index caecc7a281c..5ff16c414b5 100644
--- a/worktree.h
+++ b/worktree.h
@@ -56,6 +56,11 @@ struct worktree *find_worktree(struct worktree **list,
  */
 int is_main_worktree(const struct worktree *wt);
 
+/*
+ * Return true if the given worktree is locked
+ */
+int is_worktree_locked(const struct worktree *wt);
+
 /*
  * Return the reason string if the given worktree is locked or NULL
  * otherwise.
-- 
2.23.0

[PATCH v2 2/4] libgit: Expose more worktree functionality.

[Peter Jones]

Add delete_worktrees_dir_if_empty() and prune_worktree() to the public
API, so they can be used from more places.  Also add a new function,
prune_worktree_if_missing(), which prunes unlocked worktrees if they
aren't present on the filesystem.

Signed-off-by: Peter Jones <pjones@redhat.com>
---
 builtin/worktree.c | 73 +-------------------------------------
 worktree.c         | 88 ++++++++++++++++++++++++++++++++++++++++++++++
 worktree.h         | 19 ++++++++++
 3 files changed, 108 insertions(+), 72 deletions(-)

diff --git a/builtin/worktree.c b/builtin/worktree.c
index 86305cc1fe1..8ff37309be9 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -62,77 +62,6 @@ static int delete_git_dir(const char *id)
 	return ret;
 }
 
-static void delete_worktrees_dir_if_empty(void)
-{
-	rmdir(git_path("worktrees")); /* ignore failed removal */
-}
-
-static int prune_worktree(const char *id, struct strbuf *reason)
-{
-	struct stat st;
-	char *path;
-	int fd;
-	size_t len;
-	ssize_t read_result;
-
-	if (!is_directory(git_path("worktrees/%s", id))) {
-		strbuf_addf(reason, _("Removing worktrees/%s: not a valid directory"), id);
-		return 1;
-	}
-	if (file_exists(git_path("worktrees/%s/locked", id)))
-		return 0;
-	if (stat(git_path("worktrees/%s/gitdir", id), &st)) {
-		strbuf_addf(reason, _("Removing worktrees/%s: gitdir file does not exist"), id);
-		return 1;
-	}
-	fd = open(git_path("worktrees/%s/gitdir", id), O_RDONLY);
-	if (fd < 0) {
-		strbuf_addf(reason, _("Removing worktrees/%s: unable to read gitdir file (%s)"),
-			    id, strerror(errno));
-		return 1;
-	}
-	len = xsize_t(st.st_size);
-	path = xmallocz(len);
-
-	read_result = read_in_full(fd, path, len);
-	if (read_result < 0) {
-		strbuf_addf(reason, _("Removing worktrees/%s: unable to read gitdir file (%s)"),
-			    id, strerror(errno));
-		close(fd);
-		free(path);
-		return 1;
-	}
-	close(fd);
-
-	if (read_result != len) {
-		strbuf_addf(reason,
-			    _("Removing worktrees/%s: short read (expected %"PRIuMAX" bytes, read %"PRIuMAX")"),
-			    id, (uintmax_t)len, (uintmax_t)read_result);
-		free(path);
-		return 1;
-	}
-	while (len && (path[len - 1] == '\n' || path[len - 1] == '\r'))
-		len--;
-	if (!len) {
-		strbuf_addf(reason, _("Removing worktrees/%s: invalid gitdir file"), id);
-		free(path);
-		return 1;
-	}
-	path[len] = '\0';
-	if (!file_exists(path)) {
-		free(path);
-		if (stat(git_path("worktrees/%s/index", id), &st) ||
-		    st.st_mtime <= expire) {
-			strbuf_addf(reason, _("Removing worktrees/%s: gitdir file points to non-existent location"), id);
-			return 1;
-		} else {
-			return 0;
-		}
-	}
-	free(path);
-	return 0;
-}
-
 static void prune_worktrees(void)
 {
 	struct strbuf reason = STRBUF_INIT;
@@ -144,7 +73,7 @@ static void prune_worktrees(void)
 		if (is_dot_or_dotdot(d->d_name))
 			continue;
 		strbuf_reset(&reason);
-		if (!prune_worktree(d->d_name, &reason))
+		if (!prune_worktree(d->d_name, &reason, expire))
 			continue;
 		if (show_only || verbose)
 			printf("%s\n", reason.buf);
diff --git a/worktree.c b/worktree.c
index 4924805c389..08454a4e65d 100644
--- a/worktree.c
+++ b/worktree.c
@@ -608,3 +608,91 @@ int other_head_refs(each_ref_fn fn, void *cb_data)
 	free_worktrees(worktrees);
 	return ret;
 }
+
+void delete_worktrees_dir_if_empty(void)
+{
+	rmdir(git_path("worktrees")); /* ignore failed removal */
+}
+
+int prune_worktree(const char *id, struct strbuf *reason, timestamp_t expire)
+{
+	struct stat st;
+	char *path;
+	int fd;
+	size_t len;
+	ssize_t read_result;
+
+	if (!is_directory(git_path("worktrees/%s", id))) {
+		strbuf_addf(reason, _("Removing worktrees/%s: not a valid directory"), id);
+		return 1;
+	}
+	if (file_exists(git_path("worktrees/%s/locked", id)))
+		return 0;
+	if (stat(git_path("worktrees/%s/gitdir", id), &st)) {
+		strbuf_addf(reason, _("Removing worktrees/%s: gitdir file does not exist"), id);
+		return 1;
+	}
+	fd = open(git_path("worktrees/%s/gitdir", id), O_RDONLY);
+	if (fd < 0) {
+		strbuf_addf(reason, _("Removing worktrees/%s: unable to read gitdir file (%s)"),
+			    id, strerror(errno));
+		return 1;
+	}
+	len = xsize_t(st.st_size);
+	path = xmallocz(len);
+
+	read_result = read_in_full(fd, path, len);
+	if (read_result < 0) {
+		strbuf_addf(reason, _("Removing worktrees/%s: unable to read gitdir file (%s)"),
+			    id, strerror(errno));
+		close(fd);
+		free(path);
+		return 1;
+	}
+	close(fd);
+
+	if (read_result != len) {
+		strbuf_addf(reason,
+			    _("Removing worktrees/%s: short read (expected %"PRIuMAX" bytes, read %"PRIuMAX")"),
+			    id, (uintmax_t)len, (uintmax_t)read_result);
+		free(path);
+		return 1;
+	}
+	while (len && (path[len - 1] == '\n' || path[len - 1] == '\r'))
+		len--;
+	if (!len) {
+		strbuf_addf(reason, _("Removing worktrees/%s: invalid gitdir file"), id);
+		free(path);
+		return 1;
+	}
+	path[len] = '\0';
+	if (!file_exists(path)) {
+		free(path);
+		if (stat(git_path("worktrees/%s/index", id), &st) ||
+		    st.st_mtime <= expire) {
+			strbuf_addf(reason, _("Removing worktrees/%s: gitdir file points to non-existent location"), id);
+			return 1;
+		} else {
+			return 0;
+		}
+	}
+	free(path);
+	return 0;
+}
+
+int prune_worktree_if_missing(const struct worktree *wt)
+{
+	struct strbuf reason = STRBUF_INIT;
+	int ret;
+
+	if (is_worktree_locked(wt) ||
+	    access(wt->path, F_OK) >= 0 ||
+	    (errno != ENOENT && errno == ENOTDIR)) {
+		errno = EEXIST;
+		return -1;
+	}
+
+	strbuf_addf(&reason, _("Removing worktrees/%s: worktree directory is not present"), wt->id);
+	ret = prune_worktree(wt->id, &reason, TIME_MAX);
+	return ret;
+}
diff --git a/worktree.h b/worktree.h
index 5ff16c414b5..636bbb1c449 100644
--- a/worktree.h
+++ b/worktree.h
@@ -137,4 +137,23 @@ void strbuf_worktree_ref(const struct worktree *wt,
 const char *worktree_ref(const struct worktree *wt,
 			 const char *refname);
 
+/*
+ * Clean up the 'worktrees' directory, if necessary.
+ */
+void delete_worktrees_dir_if_empty(void);
+
+/*
+ * Prune a worktree if it's older than expire.
+ * Returns 0 on success, < 0 on failure.
+ */
+int prune_worktree(const char *id, struct strbuf *reason, timestamp_t expire);
+
+/*
+ * Prune a worktree if it is not locked and is no longer present at the
+ * checked out location.
+ * Returns < 0 if the checkout is there, if the worktree is locked, or if
+ * pruning fails.
+ */
+int prune_worktree_if_missing(const struct worktree *wt);
+
 #endif
-- 
2.23.0

Re: [PATCH v2 2/4] libgit: Expose more worktree functionality.

[Junio C Hamano]

Peter Jones <pjones@redhat.com> writes:

Same comment on the commit title as 1/4; also, we tend not to upcase
the first word after the <area>: word and omit the full-stop on the
title (see "git shortlog -32 --no-merges" on our project for
examples).

> Add delete_worktrees_dir_if_empty() and prune_worktree() to the public
> API, so they can be used from more places.  Also add a new function,
> prune_worktree_if_missing(), which prunes unlocked worktrees if they
> aren't present on the filesystem.

It probably is cleaner to do the "also" part as a separate step, as
that allows readers to skip this step without reading it deeply, but
let's see how it is done.

> @@ -144,7 +73,7 @@ static void prune_worktrees(void)
>  		if (is_dot_or_dotdot(d->d_name))
>  			continue;
>  		strbuf_reset(&reason);
> -		if (!prune_worktree(d->d_name, &reason))
> +		if (!prune_worktree(d->d_name, &reason, expire))
>  			continue;
>  		if (show_only || verbose)
>  			printf("%s\n", reason.buf);
> diff --git a/worktree.c b/worktree.c
> index 4924805c389..08454a4e65d 100644
> --- a/worktree.c
> +++ b/worktree.c
> @@ -608,3 +608,91 @@ int other_head_refs(each_ref_fn fn, void *cb_data)
> +int prune_worktree(const char *id, struct strbuf *reason, timestamp_t expire)

This is not a mere code movement, because the original relied on the
file-scope static "expire", and the public version wants to give
callers control over the expiration value.  That is a good change
that deserves to be advertised and explained in the proposed log
message.

> +int prune_worktree_if_missing(const struct worktree *wt)
> +{
> +	struct strbuf reason = STRBUF_INIT;
> +	int ret;
> +
> +	if (is_worktree_locked(wt) ||
> +	    access(wt->path, F_OK) >= 0 ||
> +	    (errno != ENOENT && errno == ENOTDIR)) {
> +		errno = EEXIST;
> +		return -1;
> +	}

When access() failed but not because the named path did not exist
(i.e. the directory may still exist---it is just this invocation of
the process happened to fail to see it---or it may not exist but we
cannot see far enough to notice that it does not exist) then we play
safe, assume it does exist, and refrain from calling prune_worktree()
on it.  Which makes sense, but do we need to set errno to EEXIST
here?  Does prune_worktree() ensure the value left in errno when it
returns failure in a similar way to allow the caller of this new
helper make effective and reliable use of errno?

> +	strbuf_addf(&reason, _("Removing worktrees/%s: worktree directory is not present"), wt->id);
> +	ret = prune_worktree(wt->id, &reason, TIME_MAX);
> +	return ret;
> +}

[PATCH v2 3/4] Make die_if_checked_out() prune missing checkouts of unlocked worktrees.

[Peter Jones]

Currently if you do, for example:

$ git worktree add path foo

And "foo" has already been checked out at some other path, but the user
has removed it without pruning, and the worktree is not locked, you'll
get an error that the branch is already checked out.  It isn't
meaningfully checked out, the repo's data is just stale and no longer
reflects reality.

This makes it so that if nothing is present where a worktree is
supposedly checked out, and it is not locked, we ignore that the
worktree exists, then it should be cleaned up.

Signed-off-by: Peter Jones <pjones@redhat.com>
---
 branch.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/branch.c b/branch.c
index 579494738a7..760ef387144 100644
--- a/branch.c
+++ b/branch.c
@@ -360,6 +360,12 @@ void die_if_checked_out(const char *branch, int ignore_current_worktree)
 	wt = find_shared_symref("HEAD", branch);
 	if (!wt || (ignore_current_worktree && wt->is_current))
 		return;
+
+	if (prune_worktree_if_missing(wt) >= 0) {
+		delete_worktrees_dir_if_empty();
+		return;
+	}
+
 	skip_prefix(branch, "refs/heads/", &branch);
 	die(_("'%s' is already checked out at '%s'"),
 	    branch, wt->path);
-- 
2.23.0

Re: [PATCH v2 3/4] Make die_if_checked_out() prune missing checkouts of unlocked worktrees.

[Junio C Hamano]

Peter Jones <pjones@redhat.com> writes:

[jc: won't repeat comments on the title]

> @@ -360,6 +360,12 @@ void die_if_checked_out(const char *branch, int ignore_current_worktree)
>  	wt = find_shared_symref("HEAD", branch);
>  	if (!wt || (ignore_current_worktree && wt->is_current))
>  		return;

die-if-checked-out is called from callers that expect to be stopped
before they do any harm, so it feels dirty to make a side effect
like this.

If the user tries to check out a branch that used to be checked out
in an already removed worktree, doesn't that indicate that an
earlier worktree removal was done incorrectly, which is something
worth reporting to the user and give the user a chance to think and
choose what corrective action(s) need to be taken?

For that, instead of automatically losing information like this
patch does, it may make more sense to fail the checkout and stop at
giving diagnosis (e.g. "our record shows that the branch is checked
out in that worktree, but you seem to have lost it.  if you forgot
to prune it, then here is the command you can give to do so.")
without actually touching the filesystem.

Thanks.


> +
> +	if (prune_worktree_if_missing(wt) >= 0) {
> +		delete_worktrees_dir_if_empty();
> +		return;
> +	}
> +
>  	skip_prefix(branch, "refs/heads/", &branch);
>  	die(_("'%s' is already checked out at '%s'"),
>  	    branch, wt->path);

[PATCH v2 4/4] Make "git branch -d" prune missing worktrees automatically.

[Peter Jones]

Currently, if you do:

$ git branch zonk origin/master
$ git worktree add zonk zonk
$ rm -rf zonk
$ git branch -d zonk

You get the following error:

$ git branch -d zonk
error: Cannot delete branch 'zonk' checked out at '/home/pjones/devel/kernel.org/git/zonk'

It isn't meaningfully checked out, the repo's data is just stale and no
longer reflects reality.

This makes it so that if nothing is present where a worktree is
supposedly checked out, deleting the branch will automatically prune it.

Signed-off-by: Peter Jones <pjones@redhat.com>
---
 builtin/branch.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/builtin/branch.c b/builtin/branch.c
index 2ef214632f0..a2a1e89c66b 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -236,13 +236,17 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 		if (kinds == FILTER_REFS_BRANCHES) {
 			const struct worktree *wt =
 				find_shared_symref("HEAD", name);
-			if (wt) {
+			int rc = -1;
+
+			if (wt && (rc = prune_worktree_if_missing(wt)) < 0) {
 				error(_("Cannot delete branch '%s' "
 					"checked out at '%s'"),
 				      bname.buf, wt->path);
 				ret = 1;
 				continue;
 			}
+			if (rc >= 0)
+				delete_worktrees_dir_if_empty();
 		}
 
 		target = resolve_refdup(name,
-- 
2.23.0

Re: [PATCH v2 1/4] libgit: Add a read-only helper to test the worktree lock

[Junio C Hamano]

Peter Jones <pjones@redhat.com> writes:

> Subject: Re: [PATCH v2 1/4] libgit: Add a read-only helper to test the worktree lock

Having a word "worktree" somewhere on the title is good, but have it
as the "I am changing this area"; "libgit" does not give readers the
hint that this is a step about the worktree subsystem.

    Subject: [PATCH v2 1/4] worktree: add is_worktree_locked() helper

When the new helper function is properly named, like yours, there is
not much need to explain what it does (i.e. "to test the worktree
lock"), so just "worktree: add is_worktree_locked()" is sufficient.

> Add the function is_worktree_locked(), which is a helper to tell if a
> worktree is locked without having to be able to modify it.

I do not see the reason why your proposed title and log message
stress the fact that this helper can be used even by callers that
are not permitted to modify the worktree (i.e. the emphasis on
"read-only").  Asking for worktree_lock_reason() can be done by
anybody, but I do not think we particularly advertise it as
read-only.

Perhaps drop "without having to..."?

> -	locked = !!worktree_lock_reason(wt);
> +	locked = is_worktree_locked(wt);
>  	if ((!locked && opts->force) || (locked && opts->force > 1)) {
>  		if (delete_git_dir(wt->id))
>  		    die(_("unable to re-add worktree '%s'"), path);
> diff --git a/worktree.c b/worktree.c
> index 5b4793caa34..4924805c389 100644
> --- a/worktree.c
> +++ b/worktree.c
> @@ -244,6 +244,22 @@ int is_main_worktree(const struct worktree *wt)
>  	return !wt->id;
>  }
>  
> +int is_worktree_locked(const struct worktree *wt)
> +{
> +	struct strbuf path = STRBUF_INIT;
> +	int locked = 0;
> +
> +	if (wt->lock_reason_valid && wt->lock_reason)
> +		return 1;
> +
> +	strbuf_addstr(&path, worktree_git_path(wt, "locked"));
> +	if (file_exists(path.buf))
> +		locked = 1;

If you write

	locked = file_exists(path.buf);

here, then readers do not have to scan backwards and find that the
variable is initialized to zero, and that no other statement since
its initialization touches its value, in order to see what value is
returned when file does not exist.  Writing the RHS !!file_exists()
concisely allows readers to tell that this function returns only 0
or 1 without having to check what file_exists() returns, but that
may probably be overkill.

> +	strbuf_release(&path);
> +	return locked;
> +}

I wondered why this is not just

	#define is_worktree_locked(wt) (!!worktree_lock_reason(wt))

There are a few differences compared to worktree_lock_reason():

 - this can be called on the main worktree by mistake and would
   probably yield "not locked" (but the existing guard is a mere
   assert() which probably is stripped away in production builds)

 - this can be used by a process that cannot even read the contents
   of the locked file for the reason;

 - because reason is not read, reason or reason_valid fields are not
   updated, and repeated calls on the same worktree structure would
   result in repeated lstat() calls.

Shouldn't we be advising the callers that the last one as a
potential downside?  The fact that the new helper is usable even by
read-only callers hints that any caching of earlier results is
disabled, but it is somewhat a round-about way to say so.

As I do not see why being able to take "const struct worktree *", as
opposed to non-const version is a huge advantage, for this helper, I
wonder if it would make even more sense to introduce one more level
to "lock-reason-valid" and allow caching of is_worktree_locked().

Currently, "lock-reason-valid" only tells us "lock-reason may be
NULL, but that does not necessarily mean it is not locked---you have
to check it" boolean, but it could be instead a tristate:

    A: lock-reason may be NULL but that is only because we haven't
       even tried to see if the lock file exists

    B: NULL-ness of lock-reason reliably tells if the worktree is
       locked or not because we have tried file_exists(), but if the
       field has non-NULL value, that is *not* the string we read;
       if you want to know the reason, you must read the file.

    C: NULL in lock-reason means it is not locked; non-NULL in
       lock-reason is what we read form the file.

Also, it may make sense to correct the first difference and in a
more meaningful way than assert(), given that the reason why this
helper is introduced is eventually to perform an destructive action
later in the series.  Perhaps

	if (is_main_worktree(wt))
		BUG("is-worktree-locked called for the main worktree");

at the front.

Thanks.

>  const char *worktree_lock_reason(struct worktree *wt)
>  {
>  	assert(!is_main_worktree(wt));
> diff --git a/worktree.h b/worktree.h
> index caecc7a281c..5ff16c414b5 100644
> --- a/worktree.h
> +++ b/worktree.h
> @@ -56,6 +56,11 @@ struct worktree *find_worktree(struct worktree **list,
>   */
>  int is_main_worktree(const struct worktree *wt);
>  
> +/*
> + * Return true if the given worktree is locked
> + */
> +int is_worktree_locked(const struct worktree *wt);
> +
>  /*
>   * Return the reason string if the given worktree is locked or NULL
>   * otherwise.

Re: [PATCH 2/2] Make "git branch -d" prune missing worktrees automatically.

[Eric Sunshine]

[cc:+duy]

On Fri, Oct 18, 2019 at 3:43 PM Peter Jones <pjones@redhat.com> wrote:
> On Thu, Oct 17, 2019 at 01:28:09PM -0400, Eric Sunshine wrote:
> > Echoing SEZDER's comment on patch 1/2, this behavior is an intentional
> > design choice and safety feature of the worktree implementation since
> > worktrees may exist on removable media or remote filesystems which
> > might not always be mounted; hence, the presence of commands "git
> > worktree prune" and "git worktree remove".
>
> Okay, I see that use case now - I hadn't realized there was an
> intentional design decision here, and honestly that's anything but clear
> from the *code*.

It can indeed sometimes be difficult to get a high-level functional
overview by examining code in isolation. In this case, at least,
git-worktree documentation tries to be clear about the "why" and "how"
of the pruning behavior (which is not to say that the documentation --
or the code -- can't be improved to communicate this better).

> It's surprising, for example, that my patches didn't break a single
> test case.

Tests suites are never perfect, and an attempt to prune a dangling
worktree by deleting a branch likely never occurred to the
git-worktree implementer(s).

> > These minor implementation comments aside, before considering this
> > patch series, it would be nice to see a compelling argument as to why
> > this change of behavior, which undercuts a deliberate design decision,
> > is really desirable.
>
> Okay, so just for clarity, when you say there's a deliberate design
> decision, which behavior here are you talking about? If you mean making
> "lock" work, I don't have any issue with that. If you mean not cleaning
> up when we do other commands, then I don't see why that's a concern -
> after all, that's exactly what "lock" is for.

To clarify, I'm talking about Duy's deliberate design decision to
model git-worktree auto-pruning after Git's own garbage-collection
behavior. That model includes, not only explicit locking, but a grace
period before dangling worktree administrative files can be pruned
automatically (see the gc.worktreePruneExpire configuration).

The point of git-worktree's grace period (just like git-gc's grace
period) is to avoid deleting potentially precious information
permanently. For instance, the worktree-local "index" file might have
some changes staged but not yet committed. Under the existing model,
those staged changes are immune from being accidentally deleted
permanently until after the grace period expires or until they are
thrown away deliberately (say, via "git worktree prune --expire=now").

> Assuming it is the "lock" behavior we're talking about, I don't think I
> actually have any intention of breaking this design decision, just
> making my workflow (without "lock") nag at me less for what seem like
> pretty trivial issues.

The ability to lock a worktree is an extra safety measure built atop
the grace period mechanism to provide a way to completely override
auto-pruning; it is not meant as an alternate or replacement safety
mechanism to the grace period, but instead augments it. So, a behavior
change which respects only one of those safety mechanisms but not the
other is likely flawed.

And, importantly, people may already be relying upon this behavior of
having an automatic grace period -- without having to place a worktree
lock manually -- so changing behavior arbitrarily could break existing
workflows and result in data loss.

> I can easily accommodate "git worktree lock". What bugs me though, is
> that using worktrees basically means I have to replace fairly regular
> filesystem activities with worktree commands, and it doesn't seem to be
> *necessary* in any way. And I'm going to forget. A lot.
>
> To me, there doesn't seem to be any reason these need to behave any different:
>
> $ git worktree add foo foo
> $ rm -rf foo
> vs
> $ git worktree add foo foo
> $ git worktree remove foo
>
> And in fact the only difference right now, aside from some very
> minuscule storage requirements that haven't gotten cleaned up, is the
> first one leaves an artifact that tells it to give me errors later until
> I run "git worktree prune" myself.

I understand the pain point, but I also understand Duy's motivation
for being very careful about pruning worktree administrative files
automatically (so as to avoid data loss, such as changes already
staged to a worktree-local "index" file). While the proposed change
may address the pain point, it nevertheless creates the possibility of
accidental loss which Duy was careful to avoid when designing worktree
mechanics. Although annoying, the current behavior gives you the
opportunity to avoid that accidental loss by forcing you to take
deliberate action to remove the worktree administrative files.

Perhaps there is some way to address the pain point without breaking
the fundamental promise made by git-worktree about being careful with
worktree metadata[*], but the changes proposed by this patch series
seem insufficient (even if the patch is reworked to respect worktree
locking). I've cc:'d Duy in case he wants to chime in.

[*] For instance, perhaps before auto-pruning, it could check whether
the index is recording staged changes or conflict information, and
only allow auto-pruning if the index is clean. *But* there may be
other ways for information to be lost permanently (beyond a dirty
"index") which don't occur to me at present, so this has to be
considered carefully.

Re: [PATCH 2/2] Make "git branch -d" prune missing worktrees automatically.

[Phillip Wood]

On 08/11/2019 10:14, Eric Sunshine wrote:
> [cc:+duy]
> 
> On Fri, Oct 18, 2019 at 3:43 PM Peter Jones <pjones@redhat.com> wrote:
>> On Thu, Oct 17, 2019 at 01:28:09PM -0400, Eric Sunshine wrote:
>>> Echoing SEZDER's comment on patch 1/2, this behavior is an intentional
>>> design choice and safety feature of the worktree implementation since
>>> worktrees may exist on removable media or remote filesystems which
>>> might not always be mounted; hence, the presence of commands "git
>>> worktree prune" and "git worktree remove".
>>
>> Okay, I see that use case now - I hadn't realized there was an
>> intentional design decision here, and honestly that's anything but clear
>> from the *code*.
> 
> It can indeed sometimes be difficult to get a high-level functional
> overview by examining code in isolation. In this case, at least,
> git-worktree documentation tries to be clear about the "why" and "how"
> of the pruning behavior (which is not to say that the documentation --
> or the code -- can't be improved to communicate this better).
> 
>> It's surprising, for example, that my patches didn't break a single
>> test case.
> 
> Tests suites are never perfect, and an attempt to prune a dangling
> worktree by deleting a branch likely never occurred to the
> git-worktree implementer(s).
> 
>>> These minor implementation comments aside, before considering this
>>> patch series, it would be nice to see a compelling argument as to why
>>> this change of behavior, which undercuts a deliberate design decision,
>>> is really desirable.
>>
>> Okay, so just for clarity, when you say there's a deliberate design
>> decision, which behavior here are you talking about? If you mean making
>> "lock" work, I don't have any issue with that. If you mean not cleaning
>> up when we do other commands, then I don't see why that's a concern -
>> after all, that's exactly what "lock" is for.
> 
> To clarify, I'm talking about Duy's deliberate design decision to
> model git-worktree auto-pruning after Git's own garbage-collection
> behavior. That model includes, not only explicit locking, but a grace
> period before dangling worktree administrative files can be pruned
> automatically (see the gc.worktreePruneExpire configuration).
> 
> The point of git-worktree's grace period (just like git-gc's grace
> period) is to avoid deleting potentially precious information
> permanently. For instance, the worktree-local "index" file might have
> some changes staged but not yet committed. Under the existing model,
> those staged changes are immune from being accidentally deleted
> permanently until after the grace period expires or until they are
> thrown away deliberately (say, via "git worktree prune --expire=now").
> 
>> Assuming it is the "lock" behavior we're talking about, I don't think I
>> actually have any intention of breaking this design decision, just
>> making my workflow (without "lock") nag at me less for what seem like
>> pretty trivial issues.
> 
> The ability to lock a worktree is an extra safety measure built atop
> the grace period mechanism to provide a way to completely override
> auto-pruning; it is not meant as an alternate or replacement safety
> mechanism to the grace period, but instead augments it. So, a behavior
> change which respects only one of those safety mechanisms but not the
> other is likely flawed.
> 
> And, importantly, people may already be relying upon this behavior of
> having an automatic grace period -- without having to place a worktree
> lock manually -- so changing behavior arbitrarily could break existing
> workflows and result in data loss.
> 
>> I can easily accommodate "git worktree lock". What bugs me though, is
>> that using worktrees basically means I have to replace fairly regular
>> filesystem activities with worktree commands, and it doesn't seem to be
>> *necessary* in any way. And I'm going to forget. A lot.
>>
>> To me, there doesn't seem to be any reason these need to behave any different:
>>
>> $ git worktree add foo foo
>> $ rm -rf foo
>> vs
>> $ git worktree add foo foo
>> $ git worktree remove foo
>>
>> And in fact the only difference right now, aside from some very
>> minuscule storage requirements that haven't gotten cleaned up, is the
>> first one leaves an artifact that tells it to give me errors later until
>> I run "git worktree prune" myself.
> 
> I understand the pain point, but I also understand Duy's motivation
> for being very careful about pruning worktree administrative files
> automatically (so as to avoid data loss, such as changes already
> staged to a worktree-local "index" file). While the proposed change
> may address the pain point, it nevertheless creates the possibility of
> accidental loss which Duy was careful to avoid when designing worktree
> mechanics. Although annoying, the current behavior gives you the
> opportunity to avoid that accidental loss by forcing you to take
> deliberate action to remove the worktree administrative files.
> 
> Perhaps there is some way to address the pain point without breaking
> the fundamental promise made by git-worktree about being careful with
> worktree metadata[*], but the changes proposed by this patch series
> seem insufficient (even if the patch is reworked to respect worktree
> locking). I've cc:'d Duy in case he wants to chime in.

I agree that we want to preserve the safe guards in the worktree design. 
I wonder if detaching the HEAD of the missing worktree would solve the 
problem without losing data. In the case where something wants to 
checkout the same branch as the missing worktree then I think that is a 
good solution. I think it should be OK for branch deletion as well.

Best Wishes

Phillip

> [*] For instance, perhaps before auto-pruning, it could check whether
> the index is recording staged changes or conflict information, and
> only allow auto-pruning if the index is clean. *But* there may be
> other ways for information to be lost permanently (beyond a dirty
> "index") which don't occur to me at present, so this has to be
> considered carefully.
> 

Re: [PATCH 2/2] Make "git branch -d" prune missing worktrees automatically.

[Eric Sunshine]

On Fri, Nov 8, 2019 at 9:56 AM Phillip Wood <phillip.wood123@gmail.com> wrote:
> On 08/11/2019 10:14, Eric Sunshine wrote:
> > Perhaps there is some way to address the pain point without breaking
> > the fundamental promise made by git-worktree about being careful with
> > worktree metadata[*], but the changes proposed by this patch series
> > seem insufficient (even if the patch is reworked to respect worktree
> > locking). I've cc:'d Duy in case he wants to chime in.
>
> I agree that we want to preserve the safe guards in the worktree design.
> I wonder if detaching the HEAD of the missing worktree would solve the
> problem without losing data. In the case where something wants to
> checkout the same branch as the missing worktree then I think that is a
> good solution. I think it should be OK for branch deletion as well.

I would feel very uncomfortable making "automatic HEAD detachment"
(decapitation?) the default behavior. Although doing so may (in some
fashion) safeguard precious information in .git/worktrees/<id>, it
potentially brings its own difficulties. For instance, if someone
takes an action which automatically detaches HEAD of a missing
worktree which had some branch checked out (and possibly some changes
staged in the worktree-specific "index"), and then builds more commits
on that branch, then that worktree gets into a state akin to rebased
upstream (for which git-rebase documentation devotes an entire
section[1], "Recovering From Upstream Rebase"). While a power-user may
be able to recover from such a state, allowing the general Git user to
get into such a situation by default seems contraindicated.

I'm not even convinced that hiding the suggested "auto-detach"
behavior behind a configuration variable so power-users can enable it
is entirely a good idea either since, while it may eliminate some
pain, it also potentially allows abandoned worktree entries to
accumulate.

[1]: https://git-scm.com/docs/git-rebase#_recovering_from_upstream_rebase

Re: [PATCH 1/2] Make die_if_checked_out() ignore missing worktree checkouts.

[SZEDER Gábor]

On Thu, Oct 17, 2019 at 12:28:25PM -0400, Peter Jones wrote:
> Currently if you do, for example:
> 
> $ git worktree add path foo
> 
> And "foo" has already been checked out at some other path, but the user
> has removed it without pruning, you'll get an error that the branch is
> already checked out.  It isn't meaningfully checked out, the repo's
> data is just stale and no longer reflects reality.
> 
> This makes it so that if nothing is present where a worktree is
> supposedly checked out, we ignore that the worktree exists, and let it
> get cleaned up the next time worktrees are pruned.
> 
> (I would prune it instead, but prune isn't available from libgit
> currently.)
> 
> Signed-off-by: Peter Jones <pjones@redhat.com>
> ---
>  branch.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/branch.c b/branch.c
> index 579494738a7..60322ded953 100644
> --- a/branch.c
> +++ b/branch.c
> @@ -360,6 +360,9 @@ void die_if_checked_out(const char *branch, int ignore_current_worktree)
>  	wt = find_shared_symref("HEAD", branch);
>  	if (!wt || (ignore_current_worktree && wt->is_current))
>  		return;
> +	if (access(wt->path, F_OK) < 0 &&
> +	    (errno == ENOENT || errno == ENOTDIR))
> +		return;

I think this check is insuffient: even if the directory of the working
tree is not present, the working tree might still exist, and should
not be ignored (or deleted/pruned in the second patch).

See the description of 'git worktree lock' for details.

>  	skip_prefix(branch, "refs/heads/", &branch);
>  	die(_("'%s' is already checked out at '%s'"),
>  	    branch, wt->path);
> -- 
> 2.23.0
>