[PATCH v2 00/25] Lockfile correctness and refactoring

[PATCH v2 00/25] Lockfile correctness and refactoring

[Michael Haggerty]

This is a second attempt at renovating the lock file code.  Thanks to
Peff, Junio, Torsten, and Eric for their helpful reviews of v1.

v1 of this patch series [1] did some refactoring and then added a new
feature to the lock_file API: the ability to activate a new version of
a locked file while retaining the lock.

But the review of v1 turned up even more correctness issues in the
existing implementation of lock files.  So this v2 dials back the
scope of the changes (it omits the new feature) but does more work to
fix problems with the current lock file implementation.

The main theme of this patch series is to better define the state
diagram for lock_file objects and to fix code that left them in
incorrect, indeterminate, or unexpected states.  There are also a few
patches that convert several functions to use strbufs instead of
limiting pathnames to a maximum length.

I hope that submitting these patches separately will make it easier
for them to be accepted without first having to decide wither the
activate-file-while-retaining-lock feature is a good one.

[1] http://thread.gmane.org/gmane.comp.version-control.git/245609

Michael Haggerty (25):
  api-lockfile: expand the documentation
  unable_to_lock_die(): rename function from unable_to_lock_index_die()
  rollback_lock_file(): do not clear filename redundantly
  rollback_lock_file(): set fd to -1
  lockfile: unlock file if lockfile permissions cannot be adjusted
  hold_lock_file_for_append(): release lock on errors
  lock_file(): always add lock_file object to lock_file_list
  struct lock_file: replace on_list field with flags field
  lockfile.c: document the various states of lock_file objects
  lockfile: define a constant LOCK_SUFFIX_LEN
  delete_ref_loose(): don't muck around in the lock_file's filename
  prepare_index(): declare return value to be (const char *)
  write_packed_entry_fn(): convert cb_data into a (const int *)
  lock_file(): exit early if lockfile cannot be opened
  remove_lock_file(): call rollback_lock_file()
  commit_lock_file(): inline temporary variable
  commit_lock_file(): make committing an unlocked lockfile a NOP
  lockfile: avoid transitory invalid states
  try_merge_strategy(): remove redundant lock_file allocation
  try_merge_strategy(): use a statically-allocated lock_file object
  commit_lock_file(): use a strbuf to manage temporary space
  Change lock_file::filename into a strbuf
  resolve_symlink(): use a strbuf for internal scratch space
  resolve_symlink(): take a strbuf parameter
  trim_last_path_elm(): replace last_path_elm()

 Documentation/technical/api-lockfile.txt |  40 ++++-
 builtin/commit.c                         |  16 +-
 builtin/merge.c                          |  15 +-
 builtin/reflog.c                         |   2 +-
 builtin/update-index.c                   |   2 +-
 cache.h                                  |   6 +-
 config.c                                 |   6 +-
 lockfile.c                               | 282 +++++++++++++++++++------------
 refs.c                                   |  20 ++-
 shallow.c                                |   6 +-
 10 files changed, 243 insertions(+), 152 deletions(-)

-- 
1.9.1

[PATCH v2 01/25] api-lockfile: expand the documentation

[Michael Haggerty]

Document a couple more functions and the flags argument as used by
hold_lock_file_for_update() and hold_lock_file_for_append().

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 Documentation/technical/api-lockfile.txt | 36 +++++++++++++++++++++++++++++---
 1 file changed, 33 insertions(+), 3 deletions(-)

diff --git a/Documentation/technical/api-lockfile.txt b/Documentation/technical/api-lockfile.txt
index dd89404..b53e300 100644
--- a/Documentation/technical/api-lockfile.txt
+++ b/Documentation/technical/api-lockfile.txt
@@ -28,9 +28,39 @@ hold_lock_file_for_update::
 	the final destination (e.g. `$GIT_DIR/index`) and a flag
 	`die_on_error`.  Attempt to create a lockfile for the
 	destination and return the file descriptor for writing
-	to the file.  If `die_on_error` flag is true, it dies if
-	a lock is already taken for the file; otherwise it
-	returns a negative integer to the caller on failure.
+	to the file.  The flags parameter is a combination of
++
+--
+LOCK_NODEREF::
+
+	Usually symbolic links in path are resolved in path and the
+	lockfile is created by adding ".lock" to the resolved path;
+	however, if `LOCK_NODEREF` is set, then the lockfile is
+	created by adding ".lock" to the path argument itself.
+
+LOCK_DIE_ON_ERROR::
+
+	If a lock is already taken for the file, `die()` with an error
+	message.  If this option is not specified, return a negative
+	integer to the caller on failure.
+--
+
+hold_lock_file_for_append::
+
+	Like `hold_lock_file_for_update()`, except that additionally
+	the existing contents of the file (if any) are copied to the
+	lockfile and its write pointer is positioned at the end of the
+	file before returning.
+
+unable_to_lock_error::
+
+	Emit an error describing that there was an error locking the
+	specified path.  The err parameter should be the errno of the
+	problem that caused the failure.
+
+unable_to_lock_die::
+
+	Like `unable_to_lock_error()`, but also `die()`.
 
 commit_lock_file::
 
-- 
1.9.1

Re: [PATCH v2 01/25] api-lockfile: expand the documentation

[Jeff King]

On Mon, Apr 07, 2014 at 01:33:43AM +0200, Michael Haggerty wrote:

> +unable_to_lock_error::
> +
> +	Emit an error describing that there was an error locking the
> +	specified path.  The err parameter should be the errno of the
> +	problem that caused the failure.
> +
> +unable_to_lock_die::
> +
> +	Like `unable_to_lock_error()`, but also `die()`.

The die() function is still called unable_to_lock_index_die() at this
point in the series.  Presumably you change it later. I don't know if it
is worth caring about the order or not; it's a doc change, so it's not
like it breaks bisectability.

-Peff

[PATCH v2 02/25] unable_to_lock_die(): rename function from unable_to_lock_index_die()

[Michael Haggerty]

This function is used for other things besides the index, so rename it
accordingly.

Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 builtin/update-index.c | 2 +-
 cache.h                | 2 +-
 lockfile.c             | 6 +++---
 refs.c                 | 2 +-
 4 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/builtin/update-index.c b/builtin/update-index.c
index d12ad95..3b1721b 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -891,7 +891,7 @@ int cmd_update_index(int argc, const char **argv, const char *prefix)
 		if (newfd < 0) {
 			if (refresh_args.flags & REFRESH_QUIET)
 				exit(128);
-			unable_to_lock_index_die(get_index_file(), lock_error);
+			unable_to_lock_die(get_index_file(), lock_error);
 		}
 		if (write_cache(newfd, active_cache, active_nr) ||
 		    commit_locked_index(lock_file))
diff --git a/cache.h b/cache.h
index 107ac61..3a873a4 100644
--- a/cache.h
+++ b/cache.h
@@ -543,7 +543,7 @@ struct lock_file {
 #define LOCK_DIE_ON_ERROR 1
 #define LOCK_NODEREF 2
 extern int unable_to_lock_error(const char *path, int err);
-extern NORETURN void unable_to_lock_index_die(const char *path, int err);
+extern NORETURN void unable_to_lock_die(const char *path, int err);
 extern int hold_lock_file_for_update(struct lock_file *, const char *path, int);
 extern int hold_lock_file_for_append(struct lock_file *, const char *path, int);
 extern int commit_lock_file(struct lock_file *);
diff --git a/lockfile.c b/lockfile.c
index 8fbcb6a..d4bfb3f 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -181,7 +181,7 @@ int unable_to_lock_error(const char *path, int err)
 	return -1;
 }
 
-NORETURN void unable_to_lock_index_die(const char *path, int err)
+NORETURN void unable_to_lock_die(const char *path, int err)
 {
 	die("%s", unable_to_lock_message(path, err));
 }
@@ -190,7 +190,7 @@ int hold_lock_file_for_update(struct lock_file *lk, const char *path, int flags)
 {
 	int fd = lock_file(lk, path, flags);
 	if (fd < 0 && (flags & LOCK_DIE_ON_ERROR))
-		unable_to_lock_index_die(path, errno);
+		unable_to_lock_die(path, errno);
 	return fd;
 }
 
@@ -201,7 +201,7 @@ int hold_lock_file_for_append(struct lock_file *lk, const char *path, int flags)
 	fd = lock_file(lk, path, flags);
 	if (fd < 0) {
 		if (flags & LOCK_DIE_ON_ERROR)
-			unable_to_lock_index_die(path, errno);
+			unable_to_lock_die(path, errno);
 		return fd;
 	}
 
diff --git a/refs.c b/refs.c
index 28d5eca..67fe8b7 100644
--- a/refs.c
+++ b/refs.c
@@ -2118,7 +2118,7 @@ static struct ref_lock *lock_ref_sha1_basic(const char *refname,
 			 */
 			goto retry;
 		else
-			unable_to_lock_index_die(ref_file, errno);
+			unable_to_lock_die(ref_file, errno);
 	}
 	return old_sha1 ? verify_lock(lock, old_sha1, mustexist) : lock;
 
-- 
1.9.1

Re: [PATCH v2 02/25] unable_to_lock_die(): rename function from unable_to_lock_index_die()

[Jeff King]

On Mon, Apr 07, 2014 at 01:33:44AM +0200, Michael Haggerty wrote:

> This function is used for other things besides the index, so rename it
> accordingly.

Oh, and here it is. I should really have just read ahead before
responding to patch 1.

We can either re-order these first two, or just not worry about it.

-Peff

Re: [PATCH v2 02/25] unable_to_lock_die(): rename function from unable_to_lock_index_die()

[Michael Haggerty]

On 04/07/2014 08:47 PM, Jeff King wrote:
> On Mon, Apr 07, 2014 at 01:33:44AM +0200, Michael Haggerty wrote:
> 
>> This function is used for other things besides the index, so rename it
>> accordingly.
> 
> Oh, and here it is. I should really have just read ahead before
> responding to patch 1.
> 
> We can either re-order these first two, or just not worry about it.

Thanks for catching this.  Yes, swapping the first two patches solves
the problem and doesn't conflict.

Michael

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

[PATCH v2 03/25] rollback_lock_file(): do not clear filename redundantly

[Michael Haggerty]

It is only necessary to clear the lock_file's filename field if it was
not already clear.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 lockfile.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lockfile.c b/lockfile.c
index d4bfb3f..7701267 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -277,6 +277,6 @@ void rollback_lock_file(struct lock_file *lk)
 		if (lk->fd >= 0)
 			close(lk->fd);
 		unlink_or_warn(lk->filename);
+		lk->filename[0] = 0;
 	}
-	lk->filename[0] = 0;
 }
-- 
1.9.1

[PATCH v2 04/25] rollback_lock_file(): set fd to -1

[Michael Haggerty]

When rolling back the lockfile, call close_lock_file() so that the
lock_file's fd field gets set back to -1.  This keeps the lock_file
object in a valid state, which is important because these objects are
allowed to be reused.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 lockfile.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lockfile.c b/lockfile.c
index 7701267..1122542 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -275,7 +275,7 @@ void rollback_lock_file(struct lock_file *lk)
 {
 	if (lk->filename[0]) {
 		if (lk->fd >= 0)
-			close(lk->fd);
+			close_lock_file(lk);
 		unlink_or_warn(lk->filename);
 		lk->filename[0] = 0;
 	}
-- 
1.9.1

[PATCH v2 05/25] lockfile: unlock file if lockfile permissions cannot be adjusted

[Michael Haggerty]

If the call to adjust_shared_perm() fails, lock_file returns -1, which
to the caller looks like any other failure to lock the file.  So in
this case, roll back the lockfile before returning so that the lock
file is deleted immediately and the lockfile object is left in a
predictable state (namely, unlocked).  Previously, the lockfile was
retained until process cleanup in this situation.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 lockfile.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/lockfile.c b/lockfile.c
index 1122542..b101f77 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -148,9 +148,11 @@ static int lock_file(struct lock_file *lk, const char *path, int flags)
 			lock_file_list = lk;
 			lk->on_list = 1;
 		}
-		if (adjust_shared_perm(lk->filename))
-			return error("cannot fix permission bits on %s",
-				     lk->filename);
+		if (adjust_shared_perm(lk->filename)) {
+			error("cannot fix permission bits on %s", lk->filename);
+			rollback_lock_file(lk);
+			return -1;
+		}
 	}
 	else
 		lk->filename[0] = 0;
-- 
1.9.1

[PATCH v2 06/25] hold_lock_file_for_append(): release lock on errors

[Michael Haggerty]

If there is an error copying the old contents to the lockfile, roll
back the lockfile before exiting so that the lockfile is not held
until process cleanup.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 lockfile.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lockfile.c b/lockfile.c
index b101f77..e78a35f 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -212,13 +212,13 @@ int hold_lock_file_for_append(struct lock_file *lk, const char *path, int flags)
 		if (errno != ENOENT) {
 			if (flags & LOCK_DIE_ON_ERROR)
 				die("cannot open '%s' for copying", path);
-			close(fd);
+			rollback_lock_file(lk);
 			return error("cannot open '%s' for copying", path);
 		}
 	} else if (copy_fd(orig_fd, fd)) {
 		if (flags & LOCK_DIE_ON_ERROR)
 			exit(128);
-		close(fd);
+		rollback_lock_file(lk);
 		return -1;
 	}
 	return fd;
-- 
1.9.1

[PATCH v2 07/25] lock_file(): always add lock_file object to lock_file_list

[Michael Haggerty]

It used to be that if locking failed, lock_file() usually did not
register the lock_file object in lock_file_list but sometimes it did.
This confusion was compounded if lock_file() was called via
hold_lock_file_for_append(), which has its own failure modes.

The ambiguity didn't have any ill effects, because lock_file objects
cannot be removed from the lock_file_list anyway.  But it is
unnecessary to leave this behavior inconsistent.

So change lock_file() to *always* ensure that the lock_file object is
registered in lock_file_list regardless of whether an error occurs.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 lockfile.c | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)

diff --git a/lockfile.c b/lockfile.c
index e78a35f..120a6d6 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -130,6 +130,22 @@ static int lock_file(struct lock_file *lk, const char *path, int flags)
 	 */
 	static const size_t max_path_len = sizeof(lk->filename) - 5;
 
+	if (!lock_file_list) {
+		/* One-time initialization */
+		sigchain_push_common(remove_lock_file_on_signal);
+		atexit(remove_lock_file);
+	}
+
+	if (!lk->on_list) {
+		/* Initialize *lk and add it to lock_file_list: */
+		lk->fd = -1;
+		lk->owner = 0;
+		lk->on_list = 1;
+		lk->filename[0] = 0;
+		lk->next = lock_file_list;
+		lock_file_list = lk;
+	}
+
 	if (strlen(path) >= max_path_len)
 		return -1;
 	strcpy(lk->filename, path);
@@ -138,16 +154,7 @@ static int lock_file(struct lock_file *lk, const char *path, int flags)
 	strcat(lk->filename, ".lock");
 	lk->fd = open(lk->filename, O_RDWR | O_CREAT | O_EXCL, 0666);
 	if (0 <= lk->fd) {
-		if (!lock_file_list) {
-			sigchain_push_common(remove_lock_file_on_signal);
-			atexit(remove_lock_file);
-		}
 		lk->owner = getpid();
-		if (!lk->on_list) {
-			lk->next = lock_file_list;
-			lock_file_list = lk;
-			lk->on_list = 1;
-		}
 		if (adjust_shared_perm(lk->filename)) {
 			error("cannot fix permission bits on %s", lk->filename);
 			rollback_lock_file(lk);
-- 
1.9.1

[PATCH v2 08/25] struct lock_file: replace on_list field with flags field

[Michael Haggerty]

This makes space for other bits, which will arrive soon.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 cache.h    | 2 +-
 lockfile.c | 9 +++++++--
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/cache.h b/cache.h
index 3a873a4..59ce5cd 100644
--- a/cache.h
+++ b/cache.h
@@ -537,7 +537,7 @@ struct lock_file {
 	struct lock_file *next;
 	int fd;
 	pid_t owner;
-	char on_list;
+	unsigned char flags;
 	char filename[PATH_MAX];
 };
 #define LOCK_DIE_ON_ERROR 1
diff --git a/lockfile.c b/lockfile.c
index 120a6d6..0486c58 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -4,6 +4,11 @@
 #include "cache.h"
 #include "sigchain.h"
 
+/* Values for lock_file::flags: */
+
+/* This lock_file instance is in the lock_file_list */
+#define LOCK_FLAGS_ON_LIST 0x01
+
 static struct lock_file *lock_file_list;
 static const char *alternate_index_output;
 
@@ -136,11 +141,11 @@ static int lock_file(struct lock_file *lk, const char *path, int flags)
 		atexit(remove_lock_file);
 	}
 
-	if (!lk->on_list) {
+	if (!(lk->flags & LOCK_FLAGS_ON_LIST)) {
 		/* Initialize *lk and add it to lock_file_list: */
 		lk->fd = -1;
 		lk->owner = 0;
-		lk->on_list = 1;
+		lk->flags |= LOCK_FLAGS_ON_LIST;
 		lk->filename[0] = 0;
 		lk->next = lock_file_list;
 		lock_file_list = lk;
-- 
1.9.1

[PATCH v2 09/25] lockfile.c: document the various states of lock_file objects

[Michael Haggerty]

Document the valid states of lock_file objects, how they get into each
state, and how the state is encoded in the object's fields.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 lockfile.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)

diff --git a/lockfile.c b/lockfile.c
index 0486c58..2438430 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -4,6 +4,58 @@
 #include "cache.h"
 #include "sigchain.h"
 
+/*
+ * File write-locks as used by Git.
+ *
+ * When a file at $FILENAME needs to be written, it is done as
+ * follows:
+ *
+ * 1. Obtain a lock on the file by creating a lockfile at path
+ *    $FILENAME.lock.  The file is opened for read/write using O_CREAT
+ *    and O_EXCL mode to ensure that it doesn't already exist.  Such a
+ *    lock file is respected by writers *but not by readers*.
+ *
+ *    Usually, if $FILENAME is a symlink, then it is resolved, and the
+ *    file ultimately pointed to is the one that is locked and later
+ *    replaced.  However, if LOCK_NODEREF is used, then $FILENAME
+ *    itself is locked and later replaced, even if it is a symlink.
+ *
+ * 2. Write the new file contents to the lockfile.
+ *
+ * 3. Move the lockfile to its final destination using rename(2).
+ *
+ * Instead of (3), the change can be rolled back by deleting lockfile.
+ *
+ * This module keeps track of all locked files in lock_file_list.
+ * When the first file is locked, it registers an atexit(3) handler;
+ * when the program exits, the handler rolls back any files that have
+ * been locked but were never committed or rolled back.
+ *
+ * A lock_file object can be in several states:
+ *
+ * - Uninitialized.  In this state the object's flags field must be
+ *   zero but the rest of its contents need not be initialized.  As
+ *   soon as the object is used in any way, it is irrevocably
+ *   registered in the lock_file_list, and flags & LOCK_FLAGS_ON_LIST
+ *   is set.
+ *
+ * - Locked, lockfile open (after hold_lock_file_for_update() or
+ *   hold_lock_file_for_append()).  In this state, the lockfile
+ *   exists, filename holds the filename of the lockfile, fd holds a
+ *   file descriptor open for writing to the lockfile, and owner holds
+ *   the PID of the process that locked the file.
+ *
+ * - Locked, lockfile closed (after close_lock_file()).  Same as the
+ *   previous state, except that the lockfile is closed and fd is -1.
+ *
+ * - Unlocked (after commit_lock_file(), rollback_lock_file(), or a
+ *   failed attempt to lock).  In this state, filename[0] == '\0' and
+ *   fd is -1.  The object is left registered in the lock_file_list,
+ *   and flags & LOCK_FLAGS_ON_LIST is set.
+ *
+ * See Documentation/api-lockfile.txt for more information.
+ */
+
 /* Values for lock_file::flags: */
 
 /* This lock_file instance is in the lock_file_list */
-- 
1.9.1

[PATCH v2 10/25] lockfile: define a constant LOCK_SUFFIX_LEN

[Michael Haggerty]

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 lockfile.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/lockfile.c b/lockfile.c
index 2438430..1bd0ae1 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -178,14 +178,17 @@ static char *resolve_symlink(char *p, size_t s)
 	return p;
 }
 
+/* We append ".lock" to the filename to derive the lockfile name: */
+#define LOCK_SUFFIX_LEN 5
 
 static int lock_file(struct lock_file *lk, const char *path, int flags)
 {
 	/*
-	 * subtract 5 from size to make sure there's room for adding
-	 * ".lock" for the lock file name
+	 * subtract LOCK_SUFFIX_LEN from size to make sure there's
+	 * room for adding ".lock" for the lock file name:
 	 */
-	static const size_t max_path_len = sizeof(lk->filename) - 5;
+	static const size_t max_path_len = sizeof(lk->filename) -
+					   LOCK_SUFFIX_LEN;
 
 	if (!lock_file_list) {
 		/* One-time initialization */
@@ -302,7 +305,7 @@ int commit_lock_file(struct lock_file *lk)
 	if (lk->fd >= 0 && close_lock_file(lk))
 		return -1;
 	strcpy(result_file, lk->filename);
-	i = strlen(result_file) - 5; /* .lock */
+	i = strlen(result_file) - LOCK_SUFFIX_LEN; /* .lock */
 	result_file[i] = 0;
 	if (rename(lk->filename, result_file))
 		return -1;
-- 
1.9.1

[PATCH v2 11/25] delete_ref_loose(): don't muck around in the lock_file's filename

[Michael Haggerty]

It's bad manners.  Especially since, if unlink_or_warn() failed, the
memory wasn't restored to its original contents.

So make our own copy to work with.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 refs.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/refs.c b/refs.c
index 67fe8b7..64ea2f0 100644
--- a/refs.c
+++ b/refs.c
@@ -2485,12 +2485,14 @@ static int repack_without_ref(const char *refname)
 static int delete_ref_loose(struct ref_lock *lock, int flag)
 {
 	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
-		/* loose */
-		int err, i = strlen(lock->lk->filename) - 5; /* .lock */
-
-		lock->lk->filename[i] = 0;
-		err = unlink_or_warn(lock->lk->filename);
-		lock->lk->filename[i] = '.';
+		/*
+		 * loose.  The loose file name is the same as the
+		 * lockfile name, minus ".lock":
+		 */
+		char *loose_filename = xmemdupz(lock->lk->filename,
+						strlen(lock->lk->filename) - 5);
+		int err = unlink_or_warn(loose_filename);
+		free(loose_filename);
 		if (err && errno != ENOENT)
 			return 1;
 	}
-- 
1.9.1

[PATCH v2 12/25] prepare_index(): declare return value to be (const char *)

[Michael Haggerty]

Declare the return value to be const to make it clear that we aren't
giving callers permission to write over the string that it points at.
(The return value is the filename field of a struct lock_file, which
can be used by a signal handler at any time and therefore shouldn't be
tampered with.)

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 builtin/commit.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index d9550c5..2e2d8bb 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -302,8 +302,8 @@ static void refresh_cache_or_die(int refresh_flags)
 		die_resolve_conflict("commit");
 }
 
-static char *prepare_index(int argc, const char **argv, const char *prefix,
-			   const struct commit *current_head, int is_status)
+static const char *prepare_index(int argc, const char **argv, const char *prefix,
+				 const struct commit *current_head, int is_status)
 {
 	int fd;
 	struct string_list partial;
-- 
1.9.1

[PATCH v2 13/25] write_packed_entry_fn(): convert cb_data into a (const int *)

[Michael Haggerty]

This makes it obvious that we have no plans to change the integer
pointed to, which is actually the fd field from a struct lock_file.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 refs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/refs.c b/refs.c
index 64ea2f0..cd471ab 100644
--- a/refs.c
+++ b/refs.c
@@ -2176,7 +2176,7 @@ static void write_packed_entry(int fd, char *refname, unsigned char *sha1,
  */
 static int write_packed_entry_fn(struct ref_entry *entry, void *cb_data)
 {
-	int *fd = cb_data;
+	const int *fd = cb_data;
 	enum peel_status peel_status = peel_entry(entry, 0);
 
 	if (peel_status != PEEL_PEELED && peel_status != PEEL_NON_TAG)
-- 
1.9.1

[PATCH v2 14/25] lock_file(): exit early if lockfile cannot be opened

[Michael Haggerty]

This is a bit easier to read than the old version, which nested part
of the non-error code in an "if" block.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 lockfile.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/lockfile.c b/lockfile.c
index 1bd0ae1..0879214 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -213,16 +213,16 @@ static int lock_file(struct lock_file *lk, const char *path, int flags)
 		resolve_symlink(lk->filename, max_path_len);
 	strcat(lk->filename, ".lock");
 	lk->fd = open(lk->filename, O_RDWR | O_CREAT | O_EXCL, 0666);
-	if (0 <= lk->fd) {
-		lk->owner = getpid();
-		if (adjust_shared_perm(lk->filename)) {
-			error("cannot fix permission bits on %s", lk->filename);
-			rollback_lock_file(lk);
-			return -1;
-		}
-	}
-	else
+	if (lk->fd < 0) {
 		lk->filename[0] = 0;
+		return -1;
+	}
+	lk->owner = getpid();
+	if (adjust_shared_perm(lk->filename)) {
+		error("cannot fix permission bits on %s", lk->filename);
+		rollback_lock_file(lk);
+		return -1;
+	}
 	return lk->fd;
 }
 
-- 
1.9.1

[PATCH v2 15/25] remove_lock_file(): call rollback_lock_file()

[Michael Haggerty]

It does just what we need.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 lockfile.c | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/lockfile.c b/lockfile.c
index 0879214..f6bee35 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -69,12 +69,8 @@ static void remove_lock_file(void)
 	pid_t me = getpid();
 
 	while (lock_file_list) {
-		if (lock_file_list->owner == me &&
-		    lock_file_list->filename[0]) {
-			if (lock_file_list->fd >= 0)
-				close(lock_file_list->fd);
-			unlink_or_warn(lock_file_list->filename);
-		}
+		if (lock_file_list->owner == me)
+			rollback_lock_file(lock_file_list);
 		lock_file_list = lock_file_list->next;
 	}
 }
-- 
1.9.1

[PATCH v2 16/25] commit_lock_file(): inline temporary variable

[Michael Haggerty]

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 lockfile.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/lockfile.c b/lockfile.c
index f6bee35..47762c6 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -297,12 +297,14 @@ int close_lock_file(struct lock_file *lk)
 int commit_lock_file(struct lock_file *lk)
 {
 	char result_file[PATH_MAX];
-	size_t i;
+
 	if (lk->fd >= 0 && close_lock_file(lk))
 		return -1;
+
 	strcpy(result_file, lk->filename);
-	i = strlen(result_file) - LOCK_SUFFIX_LEN; /* .lock */
-	result_file[i] = 0;
+	/* remove ".lock": */
+	result_file[strlen(result_file) - LOCK_SUFFIX_LEN] = 0;
+
 	if (rename(lk->filename, result_file))
 		return -1;
 	lk->filename[0] = 0;
-- 
1.9.1

[PATCH v2 17/25] commit_lock_file(): make committing an unlocked lockfile a NOP

[Michael Haggerty]

It was previously a bug to call commit_lock_file() with a lock_file
object that was not active (an illegal access would happen within the
function).  It was presumably never done, but this would be an easy
programming error to overlook.  So guard the file-renaming code with
an if statement to change committing an unlocked file into a NOP.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
Alternatively, we could make it a fatal error (e.g., an assert() or
if...die()) to call commit_lock_file() on an unlocked file, or omit a
warning in this case.  But since it is so hard to test code related to
locking failures, I have the feeling that such an error is most likely
to occur in some error-handling path, maybe when some other lockfile
acquisition has failed, and it would be better to let the code
continue its attempted cleanup instead of dying.  But it would be easy
to persuade me to change my opinion.

 Documentation/technical/api-lockfile.txt | 4 +++-
 lockfile.c                               | 3 +++
 2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/Documentation/technical/api-lockfile.txt b/Documentation/technical/api-lockfile.txt
index b53e300..bfb2676 100644
--- a/Documentation/technical/api-lockfile.txt
+++ b/Documentation/technical/api-lockfile.txt
@@ -68,7 +68,9 @@ commit_lock_file::
 	with an earlier call to `hold_lock_file_for_update()`,
 	close the file descriptor and rename the lockfile to its
 	final destination.  Returns 0 upon success, a negative
-	value on failure to close(2) or rename(2).
+	value on failure to close(2) or rename(2).  It is a NOP to
+	call `commit_lock_file()` for a `lock_file` object that is not
+	currently locked.
 
 rollback_lock_file::
 
diff --git a/lockfile.c b/lockfile.c
index 47762c6..24aaa53 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -301,6 +301,9 @@ int commit_lock_file(struct lock_file *lk)
 	if (lk->fd >= 0 && close_lock_file(lk))
 		return -1;
 
+	if (!lk->filename[0])
+		return 0;
+
 	strcpy(result_file, lk->filename);
 	/* remove ".lock": */
 	result_file[strlen(result_file) - LOCK_SUFFIX_LEN] = 0;
-- 
1.9.1

Re: [PATCH v2 17/25] commit_lock_file(): make committing an unlocked lockfile a NOP

[Jeff King]

On Mon, Apr 07, 2014 at 01:33:59AM +0200, Michael Haggerty wrote:

> It was previously a bug to call commit_lock_file() with a lock_file
> object that was not active (an illegal access would happen within the
> function).  It was presumably never done, but this would be an easy
> programming error to overlook.  So guard the file-renaming code with
> an if statement to change committing an unlocked file into a NOP.
> 
> Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
> ---
> Alternatively, we could make it a fatal error (e.g., an assert() or
> if...die()) to call commit_lock_file() on an unlocked file, or omit a
> warning in this case.  But since it is so hard to test code related to
> locking failures, I have the feeling that such an error is most likely
> to occur in some error-handling path, maybe when some other lockfile
> acquisition has failed, and it would be better to let the code
> continue its attempted cleanup instead of dying.  But it would be easy
> to persuade me to change my opinion.

Yeah, I would have expected a die("BUG") here.

I think it is worth making it a fatal mistake and catching it. Rolling
back an uninitialized lockfile is probably OK; we are canceling an
operation that never started. But committing a lockfile that we didn't
actually fill out could be a sign of a serious error, and we may be
propagating a bogus success code. E.g., imagine that receive-pack claims
to have written your ref, but actually commit_lock_file was a silent
NOP. I'd much rather have it die loudly so we can track down the case.

-Peff

Re: [PATCH v2 17/25] commit_lock_file(): make committing an unlocked lockfile a NOP

[Michael Haggerty]

On 04/07/2014 09:31 PM, Jeff King wrote:
> On Mon, Apr 07, 2014 at 01:33:59AM +0200, Michael Haggerty wrote:
> 
>> It was previously a bug to call commit_lock_file() with a lock_file
>> object that was not active (an illegal access would happen within the
>> function).  It was presumably never done, but this would be an easy
>> programming error to overlook.  So guard the file-renaming code with
>> an if statement to change committing an unlocked file into a NOP.
>> [...]
> 
> Yeah, I would have expected a die("BUG") here.
> 
> I think it is worth making it a fatal mistake and catching it. Rolling
> back an uninitialized lockfile is probably OK; we are canceling an
> operation that never started. But committing a lockfile that we didn't
> actually fill out could be a sign of a serious error, and we may be
> propagating a bogus success code. E.g., imagine that receive-pack claims
> to have written your ref, but actually commit_lock_file was a silent
> NOP. I'd much rather have it die loudly so we can track down the case.

OK, I will change this to a fatal error in v3.

Michael

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

[PATCH v2 18/25] lockfile: avoid transitory invalid states

[Michael Haggerty]

Because remove_lock_file() can be called any time by the signal
handler, it is important that any lock_file objects that are in the
lock_file_list are always in a valid state.  And since lock_file
objects are often reused (but are never removed from lock_file_list),
that means we have to be careful whenever mutating a lock_file object
to always keep it in a well-defined state.

This was formerly not the case, because part of the state was recorded
by whether lk->filename was the empty string or a valid filename.  It
is wrong to assume that this string can be updated atomically; for
example, even

    strcpy(lk->filename, value)

is unsafe.  But the old code was even more reckless; for example,

    strcpy(lk->filename, path);
    if (!(flags & LOCK_NODEREF))
            resolve_symlink(lk->filename, max_path_len);
    strcat(lk->filename, ".lock");

During the call to resolve_symlink(), lk->filename contained the name
of the file that was being locked, not the name of the lockfile.  If a
signal would have been raised during that interval, then the signal
handler would have deleted the valuable file!

We could probably continue to use the filename field to encode the
state by being careful to write characters 1..N-1 of the filename
first, and then overwrite the NUL at filename[0] with the first
character of the filename, but that would be awkward and error-prone.

So, instead of encoding part of the lock_file state in the filename
field, add a new bit "LOCK_FLAGS_LOCKFILE_ACTIVE" to flags, and use
this bit to distinguish between a lock_file object that is active
vs. one that is inactive.  Be careful to set this bit only when
filename really contains the name of a file that should be deleted on
cleanup.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 lockfile.c | 55 +++++++++++++++++++++++++++++++++++++++----------------
 1 file changed, 39 insertions(+), 16 deletions(-)

diff --git a/lockfile.c b/lockfile.c
index 24aaa53..b955cca 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -27,11 +27,14 @@
  * Instead of (3), the change can be rolled back by deleting lockfile.
  *
  * This module keeps track of all locked files in lock_file_list.
- * When the first file is locked, it registers an atexit(3) handler;
- * when the program exits, the handler rolls back any files that have
- * been locked but were never committed or rolled back.
+ * When the first file is locked, it registers an atexit(3) handler
+ * and a signal handler; when the program exits, the handler rolls
+ * back any files that have been locked but were never committed or
+ * rolled back.
  *
- * A lock_file object can be in several states:
+ * Because the signal handler can be called at any time, a lock_file
+ * object must always be in a well-defined state.  The possible states
+ * are as follows:
  *
  * - Uninitialized.  In this state the object's flags field must be
  *   zero but the rest of its contents need not be initialized.  As
@@ -40,18 +43,25 @@
  *   is set.
  *
  * - Locked, lockfile open (after hold_lock_file_for_update() or
- *   hold_lock_file_for_append()).  In this state, the lockfile
- *   exists, filename holds the filename of the lockfile, fd holds a
- *   file descriptor open for writing to the lockfile, and owner holds
- *   the PID of the process that locked the file.
+ *   hold_lock_file_for_append()).  In this state:
+ *   - the lockfile exists
+ *   - flags & LOCK_FLAGS_LOCKFILE_ACTIVE is set
+ *   - filename holds the filename of the lockfile
+ *   - fd holds a file descriptor open for writing to the lockfile
+ *   - owner holds the PID of the process that locked the file
  *
- * - Locked, lockfile closed (after close_lock_file()).  Same as the
- *   previous state, except that the lockfile is closed and fd is -1.
+ * - Locked, lockfile closed (after close_lock_file() or an
+ *   unsuccessful commit_lock_file()).  Same as the previous state,
+ *   except that the lockfile is closed and fd is -1.
  *
- * - Unlocked (after commit_lock_file(), rollback_lock_file(), or a
- *   failed attempt to lock).  In this state, filename[0] == '\0' and
- *   fd is -1.  The object is left registered in the lock_file_list,
- *   and flags & LOCK_FLAGS_ON_LIST is set.
+ * - Unlocked (after rollback_lock_file(), a successful
+ *   commit_lock_file(), or a failed attempt to lock).  In this state:
+ *   - flags & LOCK_FLAGS_LOCKFILE_ACTIVE is unset
+ *   - filename[0] == '\0' (usually, though there are transitory states
+ *     in which this condition doesn't hold)
+ *   - fd is -1
+ *   - the object is left registered in the lock_file_list,
+ *     and flags & LOCK_FLAGS_ON_LIST is set.
  *
  * See Documentation/api-lockfile.txt for more information.
  */
@@ -61,6 +71,13 @@
 /* This lock_file instance is in the lock_file_list */
 #define LOCK_FLAGS_ON_LIST 0x01
 
+/*
+ * The filename field points at an active lockfile (i.e., a file that
+ * needs to be deleted if the process dies)
+ */
+#define LOCK_FLAGS_LOCKFILE_ACTIVE 0x02
+
+
 static struct lock_file *lock_file_list;
 static const char *alternate_index_output;
 
@@ -192,6 +209,8 @@ static int lock_file(struct lock_file *lk, const char *path, int flags)
 		atexit(remove_lock_file);
 	}
 
+	assert(!(lk->flags & LOCK_FLAGS_LOCKFILE_ACTIVE));
+
 	if (!(lk->flags & LOCK_FLAGS_ON_LIST)) {
 		/* Initialize *lk and add it to lock_file_list: */
 		lk->fd = -1;
@@ -214,6 +233,7 @@ static int lock_file(struct lock_file *lk, const char *path, int flags)
 		return -1;
 	}
 	lk->owner = getpid();
+	lk->flags |= LOCK_FLAGS_LOCKFILE_ACTIVE;
 	if (adjust_shared_perm(lk->filename)) {
 		error("cannot fix permission bits on %s", lk->filename);
 		rollback_lock_file(lk);
@@ -301,7 +321,7 @@ int commit_lock_file(struct lock_file *lk)
 	if (lk->fd >= 0 && close_lock_file(lk))
 		return -1;
 
-	if (!lk->filename[0])
+	if (!(lk->flags & LOCK_FLAGS_LOCKFILE_ACTIVE))
 		return 0;
 
 	strcpy(result_file, lk->filename);
@@ -310,6 +330,7 @@ int commit_lock_file(struct lock_file *lk)
 
 	if (rename(lk->filename, result_file))
 		return -1;
+	lk->flags &= ~LOCK_FLAGS_LOCKFILE_ACTIVE;
 	lk->filename[0] = 0;
 	return 0;
 }
@@ -334,6 +355,7 @@ int commit_locked_index(struct lock_file *lk)
 			return -1;
 		if (rename(lk->filename, alternate_index_output))
 			return -1;
+		lk->flags &= ~LOCK_FLAGS_LOCKFILE_ACTIVE;
 		lk->filename[0] = 0;
 		return 0;
 	}
@@ -343,10 +365,11 @@ int commit_locked_index(struct lock_file *lk)
 
 void rollback_lock_file(struct lock_file *lk)
 {
-	if (lk->filename[0]) {
+	if (lk->flags & LOCK_FLAGS_LOCKFILE_ACTIVE) {
 		if (lk->fd >= 0)
 			close_lock_file(lk);
 		unlink_or_warn(lk->filename);
+		lk->flags &= ~LOCK_FLAGS_LOCKFILE_ACTIVE;
 		lk->filename[0] = 0;
 	}
 }
-- 
1.9.1

Re: [PATCH v2 18/25] lockfile: avoid transitory invalid states

[Johannes Sixt]

Am 4/7/2014 1:34, schrieb Michael Haggerty:
> Because remove_lock_file() can be called any time by the signal
> handler, it is important that any lock_file objects that are in the
> lock_file_list are always in a valid state.  And since lock_file
> objects are often reused (but are never removed from lock_file_list),
> that means we have to be careful whenever mutating a lock_file object
> to always keep it in a well-defined state.
> ...
> So, instead of encoding part of the lock_file state in the filename
> field, add a new bit "LOCK_FLAGS_LOCKFILE_ACTIVE" to flags, and use
> this bit to distinguish between a lock_file object that is active
> vs. one that is inactive.  Be careful to set this bit only when
> filename really contains the name of a file that should be deleted on
> cleanup.

Since this flag is primarily for communication between the main code and a
signal handler, the only safe way is to define the flag as volatile
sig_atomic_t, not to make it a bit of a larger type!

-- Hannes

Re: [PATCH v2 18/25] lockfile: avoid transitory invalid states

[Michael Haggerty]

On 04/07/2014 08:16 AM, Johannes Sixt wrote:
> Am 4/7/2014 1:34, schrieb Michael Haggerty:
>> Because remove_lock_file() can be called any time by the signal
>> handler, it is important that any lock_file objects that are in the
>> lock_file_list are always in a valid state.  And since lock_file
>> objects are often reused (but are never removed from lock_file_list),
>> that means we have to be careful whenever mutating a lock_file object
>> to always keep it in a well-defined state.
>> ...
>> So, instead of encoding part of the lock_file state in the filename
>> field, add a new bit "LOCK_FLAGS_LOCKFILE_ACTIVE" to flags, and use
>> this bit to distinguish between a lock_file object that is active
>> vs. one that is inactive.  Be careful to set this bit only when
>> filename really contains the name of a file that should be deleted on
>> cleanup.
> 
> Since this flag is primarily for communication between the main code and a
> signal handler, the only safe way is to define the flag as volatile
> sig_atomic_t, not to make it a bit of a larger type!

Thanks for the feedback.  You are obviously right, and I will fix it.

But I have a feeling that this line of thought is going to lead to the
signal handler's not being able to do anything.  How far can we afford
to pursue strict correctness?  We have

	struct lock_file {
		struct lock_file *next;
		int fd;
		pid_t owner;
		char on_list;
		char filename[PATH_MAX];
	};

	static struct lock_file *lock_file_list;

The signal handler currently reads

    lock_file_list
    lock_file::next
    lock_file::fd
    lock_file::owner
    lock_file::filename
    *lock_file::filename

and writes lock_file_list.  Among other things it calls close(),
unlink(), vsnprintf(), and fprintf() (the last two via warning()).

But most of these actions are undefined under the C99 standard:

> If the signal occurs other than as the result of calling the abort
> or raise function, the behavior is undefined if the signal handler
> refers to any object with static storage duration other than by
> assigning a value to an object declared as volatile sig_atomic_t, or
> the signal handler calls any function in the standard library other
> than the abort function, the _Exit function, or the signal function
> with the first argument equal to the signal number corresponding to
> the signal that caused the invocation of the handler.

I don't have time to rewrite *all* of Git right now, so how can we get
reasonable safety and portability within a feasible amount of work?

Michael

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

Re: [PATCH v2 18/25] lockfile: avoid transitory invalid states

[Johannes Sixt]

Am 4/7/2014 13:13, schrieb Michael Haggerty:
> On 04/07/2014 08:16 AM, Johannes Sixt wrote:
>> Am 4/7/2014 1:34, schrieb Michael Haggerty:
>>> So, instead of encoding part of the lock_file state in the filename
>>> field, add a new bit "LOCK_FLAGS_LOCKFILE_ACTIVE" to flags, and use
>>> this bit to distinguish between a lock_file object that is active
>>> vs. one that is inactive.  Be careful to set this bit only when
>>> filename really contains the name of a file that should be deleted on
>>> cleanup.
>>
>> Since this flag is primarily for communication between the main code and a
>> signal handler, the only safe way is to define the flag as volatile
>> sig_atomic_t, not to make it a bit of a larger type!
> 
> Thanks for the feedback.  You are obviously right, and I will fix it.
> 
> But I have a feeling that this line of thought is going to lead to the
> signal handler's not being able to do anything.  How far can we afford
> to pursue strict correctness?  ...
> 
> The signal handler currently reads
> 
>     lock_file_list
>     lock_file::next
>     lock_file::fd
>     lock_file::owner
>     lock_file::filename
>     *lock_file::filename
> 
> and writes lock_file_list.  Among other things it calls close(),
> unlink(), vsnprintf(), and fprintf() (the last two via warning()).
> 
> But most of these actions are undefined under the C99 standard:

Good point. But not all is lost because some of the functions are
well-defined under POSIX, particularly close() and unlink(). (*printf are
not, though.)

> I don't have time to rewrite *all* of Git right now, so how can we get
> reasonable safety and portability within a feasible amount of work?

It shouldn't be *that* bad. We can make all members volatile, except
filename (because we wouldn't be able to strcpy(lk->filename, ...) without
a type cast).

How far *do* you want to go? I'm certainly not opposed to field-test your
current changeset (plus and adjustment to use sig_atomic_t) -- overall it
is an improvement. And then we will see how it works.

Just as food for thought: A compiler barrier should be sufficient to
inhibit that the compiler reorders code across accesses of the volatile
flag. Like in the main code:

	strcpy(lk->filename, ...);
	BARRIER();
	lk->is_active = 1;	/* volatile sig_atomic_t */

and in the signal handler:

	if (!lk->is_active)
		return;
	BARRIER();
	unlink(lk->filename);

with some suitable definition of BARRIER(). I don't think that we need an
explicit memory barrier (in practice) because that should be implied by
the context switch leading to the signal handler.

-- Hannes

Re: [PATCH v2 18/25] lockfile: avoid transitory invalid states

[Michael Haggerty]

On 04/07/2014 02:12 PM, Johannes Sixt wrote:
> Am 4/7/2014 13:13, schrieb Michael Haggerty:
>> On 04/07/2014 08:16 AM, Johannes Sixt wrote:
>>> Am 4/7/2014 1:34, schrieb Michael Haggerty:
>>>> So, instead of encoding part of the lock_file state in the filename
>>>> field, add a new bit "LOCK_FLAGS_LOCKFILE_ACTIVE" to flags, and use
>>>> this bit to distinguish between a lock_file object that is active
>>>> vs. one that is inactive.  Be careful to set this bit only when
>>>> filename really contains the name of a file that should be deleted on
>>>> cleanup.
>>>
>>> Since this flag is primarily for communication between the main code and a
>>> signal handler, the only safe way is to define the flag as volatile
>>> sig_atomic_t, not to make it a bit of a larger type!
>>
>> Thanks for the feedback.  You are obviously right, and I will fix it.
>>
>> But I have a feeling that this line of thought is going to lead to the
>> signal handler's not being able to do anything.  How far can we afford
>> to pursue strict correctness?  ...
>>
>> The signal handler currently reads
>>
>>     lock_file_list
>>     lock_file::next
>>     lock_file::fd
>>     lock_file::owner
>>     lock_file::filename
>>     *lock_file::filename
>>
>> and writes lock_file_list.  Among other things it calls close(),
>> unlink(), vsnprintf(), and fprintf() (the last two via warning()).
>>
>> But most of these actions are undefined under the C99 standard:
> 
> Good point. But not all is lost because some of the functions are
> well-defined under POSIX, particularly close() and unlink(). (*printf are
> not, though.)
> 
>> I don't have time to rewrite *all* of Git right now, so how can we get
>> reasonable safety and portability within a feasible amount of work?
> 
> It shouldn't be *that* bad. We can make all members volatile, except
> filename (because we wouldn't be able to strcpy(lk->filename, ...) without
> a type cast).
> 
> How far *do* you want to go? I'm certainly not opposed to field-test your
> current changeset (plus and adjustment to use sig_atomic_t) -- overall it
> is an improvement. And then we will see how it works.

For now I think I'd just like to get the biggest problems fixed without
making anything worse.  Given that there might be a GSoC student working
in this neighborhood, he/she might be able to take up the baton.

I changed the patch series to use a new "volatile sig_atomic_t active"
field rather than a bit in a "flags" field.  I'll wait a short time to
see if there is more feedback before pushing it to the list; meanwhile
you can find it here if you have time to look at it and/or test it:

    http://github.com/mhagger/git, branch "lock-correctness"

Michael


-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

Re: [PATCH v2 18/25] lockfile: avoid transitory invalid states

[Jeff King]

On Mon, Apr 07, 2014 at 03:12:49PM +0200, Michael Haggerty wrote:

> > How far *do* you want to go? I'm certainly not opposed to field-test your
> > current changeset (plus and adjustment to use sig_atomic_t) -- overall it
> > is an improvement. And then we will see how it works.
> 
> For now I think I'd just like to get the biggest problems fixed without
> making anything worse.  Given that there might be a GSoC student working
> in this neighborhood, he/she might be able to take up the baton.
> 
> I changed the patch series to use a new "volatile sig_atomic_t active"
> field rather than a bit in a "flags" field.

That seems like a good place to stop for now.

If any code touches the fields, it can unset the "active" flag (even
temporarily), and restore it when the structure is in a known state.

I'm not sure if we can ever reach full safety. If you have an event
loop, the sane thing to do is set an atomic flag in your signal handler
and then handle it on the next iteration of the loop. But all of our
signal handlers are jumped to from arbitrary code, and are just going to
die(). There's nothing to return to, so any useful work we do has to be
done from the handler.

-Peff

[PATCH v2 19/25] try_merge_strategy(): remove redundant lock_file allocation

[Michael Haggerty]

By the time the "if" block is entered, the lock_file instance from the
main function block is no longer in use, so re-use that one instead of
allocating a second one.

Note that the "lock" variable in the "if" block shadowed the "lock"
variable at function scope, so the only change needed is to remove the
inner definition.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 builtin/merge.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/builtin/merge.c b/builtin/merge.c
index e15d0e1..f714961 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -671,7 +671,6 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,
 	if (!strcmp(strategy, "recursive") || !strcmp(strategy, "subtree")) {
 		int clean, x;
 		struct commit *result;
-		struct lock_file *lock = xcalloc(1, sizeof(struct lock_file));
 		int index_fd;
 		struct commit_list *reversed = NULL;
 		struct merge_options o;
-- 
1.9.1

[PATCH v2 20/25] try_merge_strategy(): use a statically-allocated lock_file object

[Michael Haggerty]

Even the one lockfile object needn't be allocated each time the
function is called.  Instead, define one statically-allocated
lock_file object and reuse it for every call.

Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 builtin/merge.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/builtin/merge.c b/builtin/merge.c
index f714961..0cde893 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -657,16 +657,16 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,
 			      struct commit_list *remoteheads,
 			      struct commit *head, const char *head_arg)
 {
+	static struct lock_file lock;
 	int index_fd;
-	struct lock_file *lock = xcalloc(1, sizeof(struct lock_file));
 
-	index_fd = hold_locked_index(lock, 1);
+	index_fd = hold_locked_index(&lock, 1);
 	refresh_cache(REFRESH_QUIET);
 	if (active_cache_changed &&
 			(write_cache(index_fd, active_cache, active_nr) ||
-			 commit_locked_index(lock)))
+			 commit_locked_index(&lock)))
 		return error(_("Unable to write index."));
-	rollback_lock_file(lock);
+	rollback_lock_file(&lock);
 
 	if (!strcmp(strategy, "recursive") || !strcmp(strategy, "subtree")) {
 		int clean, x;
@@ -699,14 +699,14 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common,
 		for (j = common; j; j = j->next)
 			commit_list_insert(j->item, &reversed);
 
-		index_fd = hold_locked_index(lock, 1);
+		index_fd = hold_locked_index(&lock, 1);
 		clean = merge_recursive(&o, head,
 				remoteheads->item, reversed, &result);
 		if (active_cache_changed &&
 				(write_cache(index_fd, active_cache, active_nr) ||
-				 commit_locked_index(lock)))
+				 commit_locked_index(&lock)))
 			die (_("unable to write %s"), get_index_file());
-		rollback_lock_file(lock);
+		rollback_lock_file(&lock);
 		return clean ? 0 : 1;
 	} else {
 		return try_merge_command(strategy, xopts_nr, xopts,
-- 
1.9.1

[PATCH v2 21/25] commit_lock_file(): use a strbuf to manage temporary space

[Michael Haggerty]

Avoid relying on the filename length restrictions that are currently
checked by lock_file().

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 lockfile.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/lockfile.c b/lockfile.c
index b955cca..bf018b5 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -316,7 +316,8 @@ int close_lock_file(struct lock_file *lk)
 
 int commit_lock_file(struct lock_file *lk)
 {
-	char result_file[PATH_MAX];
+	static struct strbuf result_file = STRBUF_INIT;
+	int err;
 
 	if (lk->fd >= 0 && close_lock_file(lk))
 		return -1;
@@ -324,11 +325,12 @@ int commit_lock_file(struct lock_file *lk)
 	if (!(lk->flags & LOCK_FLAGS_LOCKFILE_ACTIVE))
 		return 0;
 
-	strcpy(result_file, lk->filename);
 	/* remove ".lock": */
-	result_file[strlen(result_file) - LOCK_SUFFIX_LEN] = 0;
-
-	if (rename(lk->filename, result_file))
+	strbuf_add(&result_file, lk->filename,
+		   strlen(lk->filename) - LOCK_SUFFIX_LEN);
+	err = rename(lk->filename, result_file.buf);
+	strbuf_reset(&result_file);
+	if (err)
 		return -1;
 	lk->flags &= ~LOCK_FLAGS_LOCKFILE_ACTIVE;
 	lk->filename[0] = 0;
-- 
1.9.1

[PATCH v2 22/25] Change lock_file::filename into a strbuf

[Michael Haggerty]

For now, we still make sure to allocate at least PATH_MAX characters
for the strbuf because resolve_symlink() doesn't know how to expand
the space for its return value.  (That will be fixed in a moment.)

Another alternative would be to just use a strbuf as scratch space in
lock_file() but then store a pointer to the naked string in struct
lock_file.  But lock_file objects are often reused.  By reusing the
same strbuf, we can avoid having to reallocate the string most times
when a lock_file object is reused.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 builtin/commit.c | 12 ++++++------
 builtin/reflog.c |  2 +-
 cache.h          |  2 +-
 config.c         |  6 +++---
 lockfile.c       | 45 +++++++++++++++++++--------------------------
 refs.c           |  6 +++---
 shallow.c        |  6 +++---
 7 files changed, 36 insertions(+), 43 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index 2e2d8bb..1833595 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -330,7 +330,7 @@ static const char *prepare_index(int argc, const char **argv, const char *prefix
 			die(_("unable to create temporary index"));
 
 		old_index_env = getenv(INDEX_ENVIRONMENT);
-		setenv(INDEX_ENVIRONMENT, index_lock.filename, 1);
+		setenv(INDEX_ENVIRONMENT, index_lock.filename.buf, 1);
 
 		if (interactive_add(argc, argv, prefix, patch_interactive) != 0)
 			die(_("interactive add failed"));
@@ -341,10 +341,10 @@ static const char *prepare_index(int argc, const char **argv, const char *prefix
 			unsetenv(INDEX_ENVIRONMENT);
 
 		discard_cache();
-		read_cache_from(index_lock.filename);
+		read_cache_from(index_lock.filename.buf);
 
 		commit_style = COMMIT_NORMAL;
-		return index_lock.filename;
+		return index_lock.filename.buf;
 	}
 
 	/*
@@ -368,7 +368,7 @@ static const char *prepare_index(int argc, const char **argv, const char *prefix
 		    close_lock_file(&index_lock))
 			die(_("unable to write new_index file"));
 		commit_style = COMMIT_NORMAL;
-		return index_lock.filename;
+		return index_lock.filename.buf;
 	}
 
 	/*
@@ -453,9 +453,9 @@ static const char *prepare_index(int argc, const char **argv, const char *prefix
 		die(_("unable to write temporary index file"));
 
 	discard_cache();
-	read_cache_from(false_lock.filename);
+	read_cache_from(false_lock.filename.buf);
 
-	return false_lock.filename;
+	return false_lock.filename.buf;
 }
 
 static int run_status(FILE *fp, const char *index_file, const char *prefix, int nowarn,
diff --git a/builtin/reflog.c b/builtin/reflog.c
index c12a9784..d7df34e 100644
--- a/builtin/reflog.c
+++ b/builtin/reflog.c
@@ -431,7 +431,7 @@ static int expire_reflog(const char *ref, const unsigned char *sha1, int unused,
 			 write_str_in_full(lock->lock_fd, "\n") != 1 ||
 			 close_ref(lock) < 0)) {
 			status |= error("Couldn't write %s",
-				lock->lk->filename);
+					lock->lk->filename.buf);
 			unlink(newlog_path);
 		} else if (rename(newlog_path, log_file)) {
 			status |= error("cannot rename %s to %s",
diff --git a/cache.h b/cache.h
index 59ce5cd..6ab2483 100644
--- a/cache.h
+++ b/cache.h
@@ -538,7 +538,7 @@ struct lock_file {
 	int fd;
 	pid_t owner;
 	unsigned char flags;
-	char filename[PATH_MAX];
+	struct strbuf filename;
 };
 #define LOCK_DIE_ON_ERROR 1
 #define LOCK_NODEREF 2
diff --git a/config.c b/config.c
index 6821cef..78c3dad 100644
--- a/config.c
+++ b/config.c
@@ -1706,7 +1706,7 @@ out_free:
 	return ret;
 
 write_err_out:
-	ret = write_error(lock->filename);
+	ret = write_error(lock->filename.buf);
 	goto out_free;
 
 }
@@ -1821,7 +1821,7 @@ int git_config_rename_section_in_file(const char *config_filename,
 				}
 				store.baselen = strlen(new_name);
 				if (!store_write_section(out_fd, new_name)) {
-					ret = write_error(lock->filename);
+					ret = write_error(lock->filename.buf);
 					goto out;
 				}
 				/*
@@ -1847,7 +1847,7 @@ int git_config_rename_section_in_file(const char *config_filename,
 			continue;
 		length = strlen(output);
 		if (write_in_full(out_fd, output, length) != length) {
-			ret = write_error(lock->filename);
+			ret = write_error(lock->filename.buf);
 			goto out;
 		}
 	}
diff --git a/lockfile.c b/lockfile.c
index bf018b5..3befd7e 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -196,13 +196,6 @@ static char *resolve_symlink(char *p, size_t s)
 
 static int lock_file(struct lock_file *lk, const char *path, int flags)
 {
-	/*
-	 * subtract LOCK_SUFFIX_LEN from size to make sure there's
-	 * room for adding ".lock" for the lock file name:
-	 */
-	static const size_t max_path_len = sizeof(lk->filename) -
-					   LOCK_SUFFIX_LEN;
-
 	if (!lock_file_list) {
 		/* One-time initialization */
 		sigchain_push_common(remove_lock_file_on_signal);
@@ -216,26 +209,26 @@ static int lock_file(struct lock_file *lk, const char *path, int flags)
 		lk->fd = -1;
 		lk->owner = 0;
 		lk->flags |= LOCK_FLAGS_ON_LIST;
-		lk->filename[0] = 0;
+		strbuf_init(&lk->filename, PATH_MAX);
 		lk->next = lock_file_list;
 		lock_file_list = lk;
 	}
 
-	if (strlen(path) >= max_path_len)
-		return -1;
-	strcpy(lk->filename, path);
-	if (!(flags & LOCK_NODEREF))
-		resolve_symlink(lk->filename, max_path_len);
-	strcat(lk->filename, ".lock");
-	lk->fd = open(lk->filename, O_RDWR | O_CREAT | O_EXCL, 0666);
+	strbuf_addstr(&lk->filename, path);
+	if (!(flags & LOCK_NODEREF)) {
+		resolve_symlink(lk->filename.buf, lk->filename.alloc);
+		strbuf_setlen(&lk->filename, strlen(lk->filename.buf));
+	}
+	strbuf_addstr(&lk->filename, ".lock");
+	lk->fd = open(lk->filename.buf, O_RDWR | O_CREAT | O_EXCL, 0666);
 	if (lk->fd < 0) {
-		lk->filename[0] = 0;
+		strbuf_reset(&lk->filename);
 		return -1;
 	}
 	lk->owner = getpid();
 	lk->flags |= LOCK_FLAGS_LOCKFILE_ACTIVE;
-	if (adjust_shared_perm(lk->filename)) {
-		error("cannot fix permission bits on %s", lk->filename);
+	if (adjust_shared_perm(lk->filename.buf)) {
+		error("cannot fix permission bits on %s", lk->filename.buf);
 		rollback_lock_file(lk);
 		return -1;
 	}
@@ -326,14 +319,14 @@ int commit_lock_file(struct lock_file *lk)
 		return 0;
 
 	/* remove ".lock": */
-	strbuf_add(&result_file, lk->filename,
-		   strlen(lk->filename) - LOCK_SUFFIX_LEN);
-	err = rename(lk->filename, result_file.buf);
+	strbuf_add(&result_file, lk->filename.buf,
+		   lk->filename.len - LOCK_SUFFIX_LEN);
+	err = rename(lk->filename.buf, result_file.buf);
 	strbuf_reset(&result_file);
 	if (err)
 		return -1;
 	lk->flags &= ~LOCK_FLAGS_LOCKFILE_ACTIVE;
-	lk->filename[0] = 0;
+	strbuf_reset(&lk->filename);
 	return 0;
 }
 
@@ -355,10 +348,10 @@ int commit_locked_index(struct lock_file *lk)
 	if (alternate_index_output) {
 		if (lk->fd >= 0 && close_lock_file(lk))
 			return -1;
-		if (rename(lk->filename, alternate_index_output))
+		if (rename(lk->filename.buf, alternate_index_output))
 			return -1;
 		lk->flags &= ~LOCK_FLAGS_LOCKFILE_ACTIVE;
-		lk->filename[0] = 0;
+		strbuf_reset(&lk->filename);
 		return 0;
 	}
 	else
@@ -370,8 +363,8 @@ void rollback_lock_file(struct lock_file *lk)
 	if (lk->flags & LOCK_FLAGS_LOCKFILE_ACTIVE) {
 		if (lk->fd >= 0)
 			close_lock_file(lk);
-		unlink_or_warn(lk->filename);
+		unlink_or_warn(lk->filename.buf);
 		lk->flags &= ~LOCK_FLAGS_LOCKFILE_ACTIVE;
-		lk->filename[0] = 0;
+		strbuf_reset(&lk->filename);
 	}
 }
diff --git a/refs.c b/refs.c
index cd471ab..15d9e04 100644
--- a/refs.c
+++ b/refs.c
@@ -2489,8 +2489,8 @@ static int delete_ref_loose(struct ref_lock *lock, int flag)
 		 * loose.  The loose file name is the same as the
 		 * lockfile name, minus ".lock":
 		 */
-		char *loose_filename = xmemdupz(lock->lk->filename,
-						strlen(lock->lk->filename) - 5);
+		char *loose_filename = xmemdupz(lock->lk->filename.buf,
+						lock->lk->filename.len - 5);
 		int err = unlink_or_warn(loose_filename);
 		free(loose_filename);
 		if (err && errno != ENOENT)
@@ -2828,7 +2828,7 @@ int write_ref_sha1(struct ref_lock *lock,
 	if (write_in_full(lock->lock_fd, sha1_to_hex(sha1), 40) != 40 ||
 	    write_in_full(lock->lock_fd, &term, 1) != 1
 		|| close_ref(lock) < 0) {
-		error("Couldn't write %s", lock->lk->filename);
+		error("Couldn't write %s", lock->lk->filename.buf);
 		unlock_ref(lock);
 		return -1;
 	}
diff --git a/shallow.c b/shallow.c
index 0b267b6..cb29a66 100644
--- a/shallow.c
+++ b/shallow.c
@@ -269,8 +269,8 @@ void setup_alternate_shallow(struct lock_file *shallow_lock,
 	if (write_shallow_commits(&sb, 0, extra)) {
 		if (write_in_full(fd, sb.buf, sb.len) != sb.len)
 			die_errno("failed to write to %s",
-				  shallow_lock->filename);
-		*alternate_shallow_file = shallow_lock->filename;
+				  shallow_lock->filename.buf);
+		*alternate_shallow_file = shallow_lock->filename.buf;
 	} else
 		/*
 		 * is_repository_shallow() sees empty string as "no
@@ -316,7 +316,7 @@ void prune_shallow(int show_only)
 	if (write_shallow_commits_1(&sb, 0, NULL, SEEN_ONLY)) {
 		if (write_in_full(fd, sb.buf, sb.len) != sb.len)
 			die_errno("failed to write to %s",
-				  shallow_lock.filename);
+				  shallow_lock.filename.buf);
 		commit_lock_file(&shallow_lock);
 	} else {
 		unlink(git_path("shallow"));
-- 
1.9.1

[PATCH v2 23/25] resolve_symlink(): use a strbuf for internal scratch space

[Michael Haggerty]

Aside from shortening and simplifying the code, this removes another
place where the path name length is arbitrarily limited.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 lockfile.c | 33 ++++++++++++---------------------
 1 file changed, 12 insertions(+), 21 deletions(-)

diff --git a/lockfile.c b/lockfile.c
index 3befd7e..c4e32a9 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -150,44 +150,35 @@ static char *last_path_elm(char *p)
 static char *resolve_symlink(char *p, size_t s)
 {
 	int depth = MAXDEPTH;
+	static struct strbuf link = STRBUF_INIT;
 
 	while (depth--) {
-		char link[PATH_MAX];
-		int link_len = readlink(p, link, sizeof(link));
-		if (link_len < 0) {
-			/* not a symlink anymore */
-			return p;
-		}
-		else if (link_len < sizeof(link))
-			/* readlink() never null-terminates */
-			link[link_len] = '\0';
-		else {
-			warning("%s: symlink too long", p);
-			return p;
-		}
+		if (strbuf_readlink(&link, p, strlen(p)) < 0)
+			break;
 
-		if (is_absolute_path(link)) {
+		if (is_absolute_path(link.buf)) {
 			/* absolute path simply replaces p */
-			if (link_len < s)
-				strcpy(p, link);
+			if (link.len < s)
+				strcpy(p, link.buf);
 			else {
 				warning("%s: symlink too long", p);
-				return p;
+				break;
 			}
 		} else {
 			/*
-			 * link is a relative path, so I must replace the
+			 * link is a relative path, so replace the
 			 * last element of p with it.
 			 */
 			char *r = (char *)last_path_elm(p);
-			if (r - p + link_len < s)
-				strcpy(r, link);
+			if (r - p + link.len < s)
+				strcpy(r, link.buf);
 			else {
 				warning("%s: symlink too long", p);
-				return p;
+				break;
 			}
 		}
 	}
+	strbuf_reset(&link);
 	return p;
 }
 
-- 
1.9.1

[PATCH v2 24/25] resolve_symlink(): take a strbuf parameter

[Michael Haggerty]

Change resolve_symlink() to take a strbuf rather than a string as
parameter.  This simplifies the code and removes an arbitrary pathname
length restriction.  It also means that lock_file's filename field no
longer needs to be initialized to a large size.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 lockfile.c | 53 +++++++++++++++++++----------------------------------
 1 file changed, 19 insertions(+), 34 deletions(-)

diff --git a/lockfile.c b/lockfile.c
index c4e32a9..8e7bcda 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -133,53 +133,40 @@ static char *last_path_elm(char *p)
 #define MAXDEPTH 5
 
 /*
- * p = path that may be a symlink
- * s = full size of p
+ * path contains a path that might be a symlink.
  *
- * If p is a symlink, attempt to overwrite p with a path to the real
- * file or directory (which may or may not exist), following a chain of
- * symlinks if necessary.  Otherwise, leave p unmodified.
+ * If path is a symlink, attempt to overwrite it with a path to the
+ * real file or directory (which may or may not exist), following a
+ * chain of symlinks if necessary.  Otherwise, leave path unmodified.
  *
- * This is a best-effort routine.  If an error occurs, p will either be
- * left unmodified or will name a different symlink in a symlink chain
- * that started with p's initial contents.
- *
- * Always returns p.
+ * This is a best-effort routine.  If an error occurs, path will
+ * either be left unmodified or will name a different symlink in a
+ * symlink chain that started with the original path.
  */
-
-static char *resolve_symlink(char *p, size_t s)
+static void resolve_symlink(struct strbuf *path)
 {
 	int depth = MAXDEPTH;
 	static struct strbuf link = STRBUF_INIT;
 
 	while (depth--) {
-		if (strbuf_readlink(&link, p, strlen(p)) < 0)
+		if (strbuf_readlink(&link, path->buf, path->len) < 0)
 			break;
 
-		if (is_absolute_path(link.buf)) {
+		if (is_absolute_path(link.buf))
 			/* absolute path simply replaces p */
-			if (link.len < s)
-				strcpy(p, link.buf);
-			else {
-				warning("%s: symlink too long", p);
-				break;
-			}
-		} else {
+			strbuf_reset(path);
+		else {
 			/*
 			 * link is a relative path, so replace the
 			 * last element of p with it.
 			 */
-			char *r = (char *)last_path_elm(p);
-			if (r - p + link.len < s)
-				strcpy(r, link.buf);
-			else {
-				warning("%s: symlink too long", p);
-				break;
-			}
+			char *r = last_path_elm(path->buf);
+			strbuf_setlen(path, r - path->buf);
 		}
+
+		strbuf_addbuf(path, &link);
 	}
 	strbuf_reset(&link);
-	return p;
 }
 
 /* We append ".lock" to the filename to derive the lockfile name: */
@@ -200,16 +187,14 @@ static int lock_file(struct lock_file *lk, const char *path, int flags)
 		lk->fd = -1;
 		lk->owner = 0;
 		lk->flags |= LOCK_FLAGS_ON_LIST;
-		strbuf_init(&lk->filename, PATH_MAX);
+		strbuf_init(&lk->filename, 0);
 		lk->next = lock_file_list;
 		lock_file_list = lk;
 	}
 
 	strbuf_addstr(&lk->filename, path);
-	if (!(flags & LOCK_NODEREF)) {
-		resolve_symlink(lk->filename.buf, lk->filename.alloc);
-		strbuf_setlen(&lk->filename, strlen(lk->filename.buf));
-	}
+	if (!(flags & LOCK_NODEREF))
+		resolve_symlink(&lk->filename);
 	strbuf_addstr(&lk->filename, ".lock");
 	lk->fd = open(lk->filename.buf, O_RDWR | O_CREAT | O_EXCL, 0666);
 	if (lk->fd < 0) {
-- 
1.9.1

[PATCH v2 25/25] trim_last_path_elm(): replace last_path_elm()

[Michael Haggerty]

Rewrite last_path_elm() to take a strbuf parameter and to trim off the
last path name element in place rather than returning a pointer to the
beginning of the last path name element.  This simplifies the function
a bit and makes it integrate better with its caller, which is also
strbuf-based.  Rename the function accordingly.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
 lockfile.c | 38 ++++++++++++++++----------------------
 1 file changed, 16 insertions(+), 22 deletions(-)

diff --git a/lockfile.c b/lockfile.c
index 8e7bcda..f9c8515 100644
--- a/lockfile.c
+++ b/lockfile.c
@@ -100,32 +100,28 @@ static void remove_lock_file_on_signal(int signo)
 }
 
 /*
- * p = absolute or relative path name
+ * path = absolute or relative path name
  *
- * Return a pointer into p showing the beginning of the last path name
- * element.  If p is empty or the root directory ("/"), just return p.
+ * Remove the last path name element from path (leaving the preceding
+ * "/", if any).  If path is empty or the root directory ("/"), set
+ * path to the empty string.
  */
-static char *last_path_elm(char *p)
+static void trim_last_path_elm(struct strbuf *path)
 {
-	/* r starts pointing to null at the end of the string */
-	char *r = strchr(p, '\0');
-
-	if (r == p)
-		return p; /* just return empty string */
-
-	r--; /* back up to last non-null character */
+	int i = path->len;
 
 	/* back up past trailing slashes, if any */
-	while (r > p && *r == '/')
-		r--;
+	while (i && path->buf[i - 1] == '/')
+		i--;
 
 	/*
-	 * then go backwards until I hit a slash, or the beginning of
-	 * the string
+	 * then go backwards until a slash, or the beginning of the
+	 * string
 	 */
-	while (r > p && *(r-1) != '/')
-		r--;
-	return r;
+	while (i && path->buf[i - 1] != '/')
+		i--;
+
+	strbuf_setlen(path, i);
 }
 
 
@@ -155,14 +151,12 @@ static void resolve_symlink(struct strbuf *path)
 		if (is_absolute_path(link.buf))
 			/* absolute path simply replaces p */
 			strbuf_reset(path);
-		else {
+		else
 			/*
 			 * link is a relative path, so replace the
 			 * last element of p with it.
 			 */
-			char *r = last_path_elm(path->buf);
-			strbuf_setlen(path, r - path->buf);
-		}
+			trim_last_path_elm(path);
 
 		strbuf_addbuf(path, &link);
 	}
-- 
1.9.1

Re: [PATCH v2 00/25] Lockfile correctness and refactoring

[Jeff King]

On Mon, Apr 07, 2014 at 01:33:42AM +0200, Michael Haggerty wrote:

> This is a second attempt at renovating the lock file code.  Thanks to
> Peff, Junio, Torsten, and Eric for their helpful reviews of v1.
> 
> v1 of this patch series [1] did some refactoring and then added a new
> feature to the lock_file API: the ability to activate a new version of
> a locked file while retaining the lock.
> 
> But the review of v1 turned up even more correctness issues in the
> existing implementation of lock files.  So this v2 dials back the
> scope of the changes (it omits the new feature) but does more work to
> fix problems with the current lock file implementation.
> 
> The main theme of this patch series is to better define the state
> diagram for lock_file objects and to fix code that left them in
> incorrect, indeterminate, or unexpected states.  There are also a few
> patches that convert several functions to use strbufs instead of
> limiting pathnames to a maximum length.

Looks OK to me, modulo the few comments I sent.

I still think resolve_symref should probably not be "best-effort", but
that can come later on top.

-Peff