[PATCH 03/20] raceproof_create_file(): new function

[Michael Haggerty <mhagger@alum.mit.edu>]

Add a function that tries to create a file and any containing
directories in a way that is robust against races with other processes
that might be cleaning up empty directories at the same time.

The actual file creation is done by a callback function, which, if it
fails, should set errno to EISDIR or ENOENT according to the convention
of open(). raceproof_create_file() detects such failures, and
respectively either tries to delete empty directories that might be in
the way of the file or tries to create the containing directories. Then
it retries the callback function.

This function is not yet used.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
The actual file creation is done by a callback function, so I think
this function is flexible enough to be applicable in other
circumstances where similar races might occur. Perhaps when creating
loose object files in the ODB?

I was thinking about moving this function, along with
safe_create_leading_directories() and
safe_create_leading_directories_const(), to a more general module like
path.c. But it didn't seem worth the code churn.

 cache.h     | 16 ++++++++++++++
 sha1_file.c | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 85 insertions(+)

diff --git a/cache.h b/cache.h
index 7d3f80c..6e53cc8 100644
--- a/cache.h
+++ b/cache.h
@@ -976,6 +976,22 @@ enum scld_error {
 enum scld_error safe_create_leading_directories(char *path);
 enum scld_error safe_create_leading_directories_const(const char *path);
 
+typedef int create_file_fn(const char *path, void *cb);
+
+/*
+ * Create a file at path using fn, creating leading directories if
+ * necessary. If fn fails with errno==ENOENT, then try to create the
+ * containing directory and call fn again. If fn fails with
+ * errno==EISDIR, then delete the directory that is in the way if it
+ * is empty and call fn again. Retry a few times in case we are racing
+ * with another process that is trying to clean up the directory
+ * that contains path.
+ *
+ * In any case, the return value of this function and the errno that
+ * it sets are those resulting from the last call of fn.
+ */
+int raceproof_create_file(const char *path, create_file_fn fn, void *cb);
+
 int mkdir_in_gitdir(const char *path);
 extern char *expand_user_path(const char *path);
 const char *enter_repo(const char *path, int strict);
diff --git a/sha1_file.c b/sha1_file.c
index a1ac646..31dcfe8 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -177,6 +177,75 @@ enum scld_error safe_create_leading_directories_const(const char *path)
 	return result;
 }
 
+int raceproof_create_file(const char *path, create_file_fn fn, void *cb)
+{
+	/*
+	 * The number of times we will try to remove empty directories
+	 * in the way of path. This is only 1 because if another
+	 * process is racily creating directories that conflict with
+	 * us, we don't want to fight against them.
+	 */
+	int remove_directories_remaining = 1;
+
+	/*
+	 * The number of times that we will try to create the
+	 * directories containing path. We are willing to attempt this
+	 * more than once, because another process could be trying to
+	 * clean up empty directories at the same time as we are
+	 * trying to create them.
+	 */
+	int create_directories_remaining = 3;
+
+	/* A scratch copy of path, filled lazily if we need it: */
+	struct strbuf path_copy = STRBUF_INIT;
+
+	int save_errno;
+	int ret;
+
+retry_fn:
+	ret = fn(path, cb);
+	save_errno = errno;
+	if (!ret)
+		goto out;
+
+	if (errno == EISDIR && remove_directories_remaining > 0) {
+		/*
+		 * A directory is in the way. Maybe it is empty; try
+		 * to remove it:
+		 */
+		if (!path_copy.len)
+			strbuf_addstr(&path_copy, path);
+
+		if (!remove_dir_recursively(&path_copy, REMOVE_DIR_EMPTY_ONLY)) {
+			remove_directories_remaining--;
+			goto retry_fn;
+		}
+	} else if (errno == ENOENT && create_directories_remaining > 0) {
+		/*
+		 * Maybe the containing directory didn't exist, or
+		 * maybe it was just deleted by a process that is
+		 * racing with us to clean up empty directories. Try
+		 * to create it:
+		 */
+		enum scld_error scld_result;
+
+		if (!path_copy.len)
+			strbuf_addstr(&path_copy, path);
+
+		do {
+			create_directories_remaining--;
+			scld_result = safe_create_leading_directories(path_copy.buf);
+			if (scld_result == SCLD_OK)
+				goto retry_fn;
+		} while (scld_result == SCLD_VANISHED && create_directories_remaining > 0);
+	}
+
+out:
+	strbuf_release(&path_copy);
+	errno = save_errno;
+	return ret;
+}
+
 static void fill_sha1_path(char *pathbuf, const unsigned char *sha1)
 {
 	int i;
-- 
2.7.0